BI Tools – Test Management Overview

“Our BI system needs better testing.  How?” always piques my interest.  The breadth of the BI ecosystem means that testing the system end-to-end involves crossing many technologies, areas of responsibility (functional competencies), and physical systems.  This covers most (all) the tool classes in this series, at very least:

1. Data extracts/data acquisition – wide variety of tool/methods, possibly ETL, messaging/queuing, etc.
2. Data ingestion (transform, load) – ETL tool.
3. Data quality manipulation (matching, cleansing, stewardship activities) – ETL tool, cleansing tool, master data tool, stewardship interface(s).
4. Data restructuring (3NF/relational to dimensional/star to OLAP cubes to in-memory analytics) – technology specific.
5. BI delivery (reporting, visualization) – BI tool, visualization tool, notification tool
6. Metadata – metadata platform, BI tool
7. Operations (reporting issues, performance characteristics) – ETL tool, BI tool, issues management tool

Testing of BI systems varies as widely as the teams that delivery those systems, from formal and rigorous to informal and anecdotal relying largely on reporting of issues by users.  Normally, I’d call this an issue of maturity, but sometimes it’s simply an issue of style or business alignment – sometimes the cost of testing isn’t justified.  Here though, I’ll assume you’re interested in increasing your test effectiveness.  Since this is a discussion of tools, I’ll leave the testing process to QA experts, which I am not.  My interest here is primarily in the overall BI tools environment and how testing can be applied across the board.

Revolutionize Your Business With Generative AI

From product design and software development to virtual agents, content creation, and reporting, GenAI is transforming business. Our AI experts help you unlock GenAI’s full potential and drive growth.

Let’s Get Started

As I was writing on this topic, I realized that it was going to be too lengthy for a single post, so I’ll break it up as:

1. Test Data – Creating test data and test data sets including tooling that may be employed.
2. Test Fixtures – How to get your system to a predefined state before running a test series.
3. Test Cases – The actual tests, how to capture them, execute them, and regress.
4. Test Management – Putting it all together and integrating testing into the development process (from a tools standpoint).

This subject is one where my opinion are constantly changing as I run across innovative testing environments.  I’d love to read about anything you’ve found helpful in configuring your testing environment.  Comment away!