Posts Tagged ‘ETL’

In a few days Oracle will release Data Sync, a utility that facilitates the process of loading data from various on-premise sources into BI Cloud Services (BICS). The Data Sync utility comes as an addition to several other already available means of loading data into BICS: data load wizard to upload data from files, SQL […]

Deciding “if, when or how” to move from one safety application to another can be a daunting task full of angst, pressure and fear of the unknown. Arguably, most of this distress is experienced in the initial analysis of the migration options and sifting through the solutions available to an organization. This blog entry will attempt […]

  The speed in which we receive information from multiple devices and the ever-changing customer interactions providing new ways of customer experience, creates DATA! Any company that knows how to harness the data and produce actionable information is going to make a big difference to their bottom line. So Why Virtualization? The simple answer is […]

As an avid user of IBM DataStage for over 8 years I was recently on a client engagement where I had to use Oracle’s Oracle Data Integrator (ODI).  At first I was disappointed as I was out of my confront zone.  But after a week or two I was very surprised how quickly I was […]

  Well I know I’m a little late on this but I figured if you haven’t heard already this would be a great blog post to inform you of the latest update to IBM’s InfoSphere Information Server Version 11.3. For those that don’t know IBM’s InfoSphere Information Server Version 11.3, the following platform is made […]

On a recent project, I was faced with a requirement to scan the contents of a folder and load all the files into their respective staging tables. There were multiple file types – Customer file, Store file, Products file, Sales file, etc. Every day, we received zero to many files for each type of file. […]

Data loaded into a TM1 or SPSS model will, in most cases, include files consisting of thousands (or hundreds of thousands) of records. It is not reasonable, given the number of fields and records in files of this size, for you to visually inspect all fields in every record (of every file) for missing or […]

In the last post in this series, we looked at why Azure Machines are the most likely candidate for workable SQL Server-based cloud BI implementations.   Today, we talk turkey about actual limitations and things you need to know as you look at a potential solution in this environment. The first thing to know is that Azure […]

In one of my earlier posts I recommended an approach for loading larger amounts of data into TM1 applications, and provided a high-level explanation of this approach, which I call the “File Route” solution. Some More Thoughts                  I like this load-strategy because, in following best practice guidelines, it leverages TM1’s proven ETL tool, TurboIntegrator to […]

Several weeks ago, I was watching a cable network news channel and the news anchor was discussing what it will take to win the next presidential election. As he discussed it in more detail, he commented about how there will need to be much better analysis of data to pinpoint the right segment of the […]

Logical deletes are a common practice when loading dimensions in a data warehouse. If a dimensional record is deleted in the source database, it may need to continue to exist in your data warehouse. In my recent project, this was one of the requirements set during the design phase. Since ODI was being used as […]

I often use the SQL Server (MSDB) Database to store and execute SSIS packages.  However, I found it cumbersome to migrate multiple packages back and forth from network locations to the MSDB or from the various environments (DEV, QA, PROD).  So, I created a simple script to generate the dtutil commands necessary to quickly migrate […]