Data & Intelligence

BI Tools – Continuous Integration

Automate everything!  The mantra of the real developer – I’d rather spend 2 hours automating a task than 30 minutes of repetitive typing!  Continuous integration finally legitimizes that innate desire to automate during the development process.

Continuous Integration (CI) tools automate the application build process and automated tests on a regular interval (or on demand as code are checked in).  This allows for frequent feedback on some of the basic aspects of quality and can improve confidence in the overall platform’s state moving towards a release.  In BI, continuous integration is especially helpful in detecting synchronization issues between data models, integration (ETL) code, semantic code (report models/universes) and reports.

CI tools generally have two parts, the controller and the scripting interface.

The controller manages builds and reports on successes and issues.  Integration with the source control, issues management, and even the documentation management system are all useful features of the CI controller.

Data Intelligence - The Future of Big Data
The Future of Big Data

With some guidance, you can craft a data platform that is right for your organization’s needs and gets the most return from your data capital.

Get the Guide

The scripting interface provides developers a mechanism by which to act upon each of the platform’s components.  Usability in scripting means the use of a well-known and supported scripting language such as Perl, JavaScript, vbscript,  PowerShell, etc.  Alternately, some CI tools rely on a dedicated build tool such as Make, Maven, or Ant to handle scripting giving more flexibility.

Effective CI tools:

  • Can control all aspects of the build process through scripts, plug-ins, or black magic.  If you have to sign in and pull a lever in the middle of a CI build, you’ve defeated the purpose.
  • Report build breaks and test failures directly through the issues management/ticketing system with meaningful, useful messages.
  • Are triggered by a variety of conditions, especially the commit of code into the source control system.
  • Allow no-touch production builds that maintain the security envelope of the production system.

There are many open source and commercial CI tools and their features vary widely.  None, as far as I know, are specific to BI, nor do they need to be.  However, a nice user interface that helps out with the task of operating the ETL, BI delivery, DBMS, etc. tools is a nice way to go.

Tools I’ve run across, on the open source side:

  • Hudson/Jenkins
  • Apache Continuum
  • CruiseControl

And in the commercial space:

  • Atlassian Bamboo
  • IBM Rational
  • Microsoft Team Foundation Server (TFS)

Once again, I favor the Atlassian tool Bamboo, here simply because of its integration with the other Atlassian tools.  So, if you don’t have JIRA or Confluence, ignore this.  Bamboo is a very capable CI platform (although I think that Hudson and others are more configurable) with a beautiful user interface.  Is supports remote job execution via remote agents and is competitively priced.  And, getting it to create JIRA issues is a no-brainer.

That said, most CI tools are up to the task of automated builds.  What you should be looking for is a tool that fits well within your overall environment and makes your team more productive through continuous feedback.

About the Author

More from this Author

Thoughts on “BI Tools – Continuous Integration”

  1. Hi Chris,

    Do we have any automated tool available for Coding Guidelines enforcement for BI? (Somehting similar to Resharper or FxCop that is used for .NET).

    Also is there any tool available to integrate into Build environment to enforce coding guidelines?

    Thanks,
    Ajeet

  2. Ajeet –

    Great question, but unfortunately no good answer on my end. We’ve got great guidelines enforcement in the model via the built-in “check” functionality in PowerDesigner. But nothing on the ETL code end of things.

    Chris

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Subscribe to the Weekly Blog Digest:

Sign Up