First Experience with Informatica Cloud

I remember that I heard the cloud concept about ten years ago while now the real cloud age is here. The cloud deployment and practice in CRM area might be the pioneer comparing with others. It is glad that we are seeing more and more BI vendors are boosting their cloud strategy and product development to be upon cloud. Recently I have played the development over Informatica cloud for an integration effort. In the proof of concept, the major goal is to integrate the business records from Salesforce and Oracle CRM On Demand into a local oracle database to serve for further analysis and reporting.

Overview and Architecture

The Future of Big Data

With some guidance, you can craft a data platform that is right for your organization’s needs and gets the most return from your data capital.

Get the Guide

First of all, the cloud user would need to register a free-trial account which can be available for multiple core components such as Data Synchronization, Contact Validation, Data Masking, Data Replication, Data Assessment, etc. The following architectural charts depict the main components and how the data can be exchanged between cloud and your company. The secure agent plays key role not only to secure the information but also to manage the session related parameters.

Integration Steps

There are some available tutorial videos on Cloud  for the beginners, which are pretty straightforward and presented screenshot by screenshot. If you have previous experience on On-Prem  Informatica development, it would cost less time to get the work done through the cloud. The typical steps are as followings:

1. Install secure agent in your local computer where is better to enable the agent and register with your existing accounts.
2. Find out the connectors which will assist you to read or write specific source. In my case I installed Salesforce and Oracle CRM OD connectors. The trial account are licensed with about one month work with these connectors.
3. Build connections to read and write to Salesforce, Oracle CRM OD, enterprise Oracle database 11g. Test it out.
4. Create data synchronization job which enable the options to move data between cloud and cloud, cloud and local, etc. It is required to define job name, source, and target type. User can add data filter based on the source fields, and accomplish field mapping by dragging and dropping the field labels. Test it to make sure it works, and then schedule it to run in regular fashion.
5. If there are multiple jobs to run, the user can create a task flow to accommodate all available jobs (not limited to DS job) and make them sequential or parallel due to requirement.

Recommendations

1. In general, the development over Informatica cloud are pretty straightforward, but it would require additional design on each ETL requirement such as extract strategy, complex business logic, upsert load against target, etc. In my case I used data filter to extract incremental records, each time the job only looked at recent delta rows which was modified since last extract date.
2. Most cloud based application systems like Salesforce, Oracle CRM OD are available to access data via REST soap protocol which is either WSDL v1.0 or v2.0. It will be good to understand better on the source objects WSDL structure because you would only see encapsulated objects in Informatica source definition. The SOAP UI is a great tool to perform necessary testing against source Web Services to help verify the data correctness and performance.
3. The performance of loading from Salesforce is quite good, but it is with low speed from Oracle CRM On Demand. It has been challenge to perform the load task even though I made efforts to determine and improve it: Used integration template to parallelize the job; Increased the memory number and other related parameters. To further identify the bottleneck, we performed load test against CRM OD production app first via tool like SOAP UI. The testing consequence indicates that the slow loading speed from Oracle CRM Demand is caused by the bottleneck in Oracle API, not in the Informatica cloud.
4. Informatica community is a helpful place where you can find useful tutorials, solutions and best practices. Try to visit it often to look for your answer as there are many experts.