Posts Tagged ‘ETL’

As we move into the modern cloud data architecture era, enterprises are deploying 2 primary classes of data integration tools to handle the traditional ETL and ELT use cases. The first type of Data integration tool is GUI-Based Data Integration solutions. Talend, Infosphere Datastage, Informatica, and Matillion are good examples. These tools leverage a UI […]

MicroStrategy World 2020 is about a month away, happening February 4-6 in Orlando, FL. Sunny, warm weather and the latest MicroStrategy releases will make for an awesome and exciting, education-packed week. Perficient is proud to be a Silver sponsor of the event this year.  Our experts look forward to meeting you in the expo hall […]

Oracle Data Integration Platform Cloud (DIPC) DIPC is a unified, powerful, data-driven data integration platform on cloud which can accept data in any format from any source system either on premise or on cloud and process that data as per organization needs. With DIPC, you get all the capabilities of most popular E-LT (Extract – Load […]

Following my previous blog post on how to add a new Dimension to a Data Sync task, this post looks at how to add a Fact and perform a lookup on dimensions while loading the target fact table in a data warehouse using Data Sync. To refer to the blog post on adding a Dimension […]

In this and the following post, I will cover the steps entailed in adding dimension and fact tasks in Oracle Data Sync. The latest releases of Data Sync included a few important features such as performing look-ups during an ETL job. So I intend to cover these best practices when adding new dimension and fact […]

Introduction On a daily basis, data growth is expanding at a pace greater than the expansion of the universe itself. It makes our lives better, but it also has the capability of reflecting the vulnerabilities of a person or an organization. Data is like Infinity Gauntlet. If you know how to use it, like Thanos […]

Everything is data and everyone is data! Everybody says we live in the technology age, the age of the internet, the age of space and cosmos, the age of the digital world etc. But all of these developments and advancements have been made possible by the Holy Grail called Data. We learned how to store […]

My focus at Perficient lately is a fast-moving project with 15+ individuals, complex requirements, and a tight deadline. The experience is extremely valuable. The project is an ETL platform on AWS that uses Lambda for event-driven processing, Elastic MapReduce (EMR) for managed Hadoop clusters, RDS and S3 for persistence, and a handful of other services. […]

Introduction:   In general, the ETL (Extraction, Transformation and Loading) process is being implemented through ETL tools such as Datastage, Informatica, AbInitio, SSIS, and Talend to load data into the data warehouse. The same process can also be accomplished through programming such as Apache Spark to load the data into the database. Let’s see how it […]

The need exists for the ability to read and extract data from BICS, especially when the ETL strategy involves a Staging approach, as written about in Best Practice to ETL with Data Sync (BICS). But Data Sync does not support the direct data read from BICS. The built-in Oracle (BICS) connection in Data Sync only supports a data write to BICS. […]

Integrating data from non-Oracle source systems with an Oracle prepackage solution is a common practice that requires source to Target integration. While implementing BI Apps/Oracle EHA (Enterprise Healthcare Analytics) with a new source, I used several best practices, which can be leveraged while integrating a new source in an Oracle Warehouse using ODI. As a […]

In my last blog, I presented a brief overview on Informatica Data Quality (IDQ) tools, the significance of Data Profiling and how to use the Analyst tool to profile data. In this second blog, I will introduce a few commonly used Informatica Developer tool Data Transformations. But, Why Data Transformations? Long story short, Data Quality […]

Many of us have used the looping logic in programming languages or in others tool that we use. Looping logic is when you process each record continuously until it reaches its maximum limit. So, what’s new in looping logic? Let’s see here. In Informatica, it is quite problematic to loop a single record and there […]

 “Small Change & Big Difference” Today in almost all the sectors, say, Banking, Healthcare, Insurance, Telecom..we handle billions and trillions of data in a batch mode processing.  In enterprise data warehouse, capturing the changes in everyday transactions and loading into datawarehouse will be a time-consuming process if the data volume is high. Though ETL tools […]

Introduction In today’s fast paced world, Organizations have started using different types of software systems to compete and to be up to the speed of the world. This eventually increases the need for communication between different software systems and the same is growing by leaps and bounds. And with the Current data warehousing and data […]

In this blog, I will finish the Best Practice to ETL with Data Sync (BICS) post by showing you how to configure the Data Sync to load data in staging database. First, set up a new database connection in Data Sync for SQL Server or Oracle database. For SQL Server > Set Connection Type to MSSQL > Fill in the rest […]

As Oracle BI Cloud Service (BICS) starts to become popular, I am going to write a series blogs about best use of the Data Sync to upload data to BICS schema service.  In this post, I am going to share with you a best practice to use Data Sync for ETL. As an ETL tool, Data […]

What is Unstructured Data? Unstructured data is an information that does not have a predefined data model or does not fit well into relational tables. It is broadly classified into two types Non-Textual unstructured data is a multimedia data like still images, videos, and MP3 audio files Textual unstructured data are like email messages, instant […]

What is Hybrid SCD? Slowly Changing Dimension (SCD) Type 6 is also called as “Hybrid SCD” that combines three fundamental SCD techniques. Type 6 can be used when you want to maintain complete history and would also like to have an easy way to manage current version. The point of “type 6” or “Hybrid” processing […]

It has been decades since people started to develop data warehousing (DW) systems. In fact, most of the delivery strategy and approach on DW is to follow the traditional waterfall cycle – discovery, requirement, design, development, test, training, and transition, etc. With this pattern, only the previous step is completed; the team will start on […]