How to Find the Right Partner for Data Integration

Choices can make us all feel like Goldilocks. One option is too big, another option is too small. Data integration can feel the same way. It’s hard to know what’s “just right” for your organization.

The Future of Big Data

With some guidance, you can craft a data platform that is right for your organization’s needs and gets the most return from your data capital.

Get the Guide

Wait – what exactly is data integration? It’s the combination of technical and business processes used to combine data from different sources into meaningful, valuable information. And it starts by defining where your data is coming from. Is the data in an existing database, spreadsheets or your CRM system? Are you picking the right data that is valuable enough to spend money moving it between systems? Which processes will get the job done?

There are a lot of questions involved. But what you’re really trying to do boils down to two things:

1. Who is going to do the work? When choosing a partner, you usually have two options – internal or external. Do you have somebody at your company with the skills for the given data integration? Does a business analyst have experience with an off-the-shelf ETL tool? Do you have an internal developer with the bandwidth to develop a custom integration? If not, hire somebody externally with the experience to do the job right. With this route, you can also train internal resources for the next time the same type of project comes up. Your external options include:
   • +

   Independent Consultants

    – The price may typically be less, but there is also more risk that an individual consultant may move onto other projects – or may not be able to work with you in the future and re-use the knowledge gained about your business.
   • +

   Niche Consulting Companies

    – Can you find a smaller, maybe regional, consulting company with multiple people who specialize in what you are trying to do? This can give you a very personalized experience with a company that has multiple people to help complete your project. This type of company will also be around to be a long-term partner who will intimately know your business. At Sundog, we fit into this category.
   • +

   Large Consulting Companies

    – These companies have thousands of consultants all across the globe. They will have a wide variety of people with varied experiences for a lot of large and complex jobs. The downside is you might not get the same team each time to develop partnerships with people who really know you and your business.
2. What tools will get the job done?
   Now this question comes with a bit of “the chicken or the egg” debate. What comes first – the tool or the people using it? Ideally it’s better to pick the right tool, and then find the right people with the experience to use it. But sometimes you may have team members with experience using certain data migration tools, so you may need to start there.Now onto tools – from the simplest and least expensive to the most complex and expensive. Here are some of the tools used most often in conjunction with Salesforce integrations:
   • +

   Apex Data Loader or Online Data Loading Wizards

   •  These free tools are the first tools used by most admins to populate data into Salesforce. They do the typical query, add, edit, delete, upsert (add/edit) options that you need a tool to do. You can even look at automating the Apex Data Loader once you get your process started:

   https://www.sundoginteractive.com/sunblog/posts/automating-the-apex-data-loader

   • +

   Jitterbit /Informatica/MuleSoft/Cast Iron

    These third-party ETL tools can solve most of your data integration needs. Once you pick one, you will probably stay with it so you do not have licenses for two products that mostly do the same thing. Free introductory versions give you some great point-and-click options for setting up your first simple integrations. Upgrade to the paid version to get powerful ETL options that allow for scheduled integrations and free your internal people from these data migration jobs.
   • +

   WebMethods/SAP PI

    These middleware tools provide powerful methods for larger organizations to move and transform critical data. But with this power comes a much higher cost. Message queuing functionality allows for scalability and mitigates downtime between integration points.
   • +

   Custom Integrations

    For simple point-to-point integration, don’t rule out a custom integration either. It’s not very difficult to create a .NET exe that will bring data, for example, from a SQL server database, transform it and then push it to a Salesforce database. This allows you to not pay for ongoing license costs, and it could leverage the knowledge of internal developers at your company, too.

Back to Goldilocks, there are a lot of choices in front of you. In the end, find somebody whom you can trust – somebody with the experience to give you good options on the right tools to solve your business challenges. Business processes will be built around your tools, and then you’ll have a re-usable set of partners, pieces and processes for your future data integration projects. And that’s no fairy tale.