Single source test data

Most people might not think test data preparation is important in software engineering. However it becomes an improvement area as more and more companies start to apply TDD (test driven development) now. Managing test data differently may impact your project more than you think.

First, let’s look at a question many developers might have:  How to prepare test data for integration tests?

Here “integration tests” means writing reusable tests to test server side integrated functionalities, not to test user interface. The difference between unit tests and integrated tests could be summarized as: Unit tests ensure we do things right, Integration tests ensure we’ve done right things.

To explain the problem of test data preparation, let me describe a little bit.

Traditionally, test data preparation is like:

– Tester/BA writes test case documents. Test data disperses in test case steps.

– For developer who needs to write integration test code to ensure user story functionality, he has to prepare test data by self. The test data is in format of SQL or XML aiming to be used by code.

Problems are:

– It is hard and painful for developers to prepare test data because of lack of business knowledge and real life data. To translate each test scenario into test data correctly is not an easy job. In result, the reality is in most time integration tests only cover main test scenarios or even no integration test at all. The testing work is left to test team when UI testing.

Choosing a Global Software Development Partner to Accelerate Your Digital Strategy

To be successful and outpace the competition, you need a software development partner that excels in exactly the type of digital projects you are now faced with accelerating, and in the most cost effective and optimized way possible.

Get the Guide

– Neither test case document creation nor developer’s integration test data preparation involves customer. The test data is created by imagination.

Questions for improvement:

– Instead of having test data multiple places, can we reuse single source of test data which provides multiple views for both customer and developer’s usage.  (More about single source, see here)

– How to involve customer into test data preparation and even test cases effectively?
Solution 1 – Eclipse TPTP:
The first thought came into mind was using Excel file. The idea is to store test data with test scenarios in Excel file in certain format. Use the Excel file to confirm with client. Then use the same file as input for developer’s integration testing. We just need to build a simple utility to pull Excel test data into test code, execute tests, then update testing result back to the Excel file. And this way we can integrate with XUnit well.

Eclipse Test and Performance Tools Platform (TPTP) has already implemented this thought about using Excel for single source test data.  Check it out.

Solutions 2 – FIT / FITNESS:
The other choice is FIT.  You will find that FIT actually solves the above problems perfectly!
And it has the additional benefits:
– Test data and executable test cases are defined early which is perfect for test driven development (TDD).
– Test case is determined.  Client can “feel” the requirements. Developers have clear goal to achieve.

FIT can help greatly in applying TDD successfully.

Comparison – Eclipse TPTP Vs. FIT
Now we have two options for “single source test data”:

Although Eclipse TPTP has some limitations and is not perfect for TDD, in some cases we might still consider using it. Because:

–  We can still do some interactions with customer using the Excel data file.

– It is a lightweight tool, and is easier to be applied in projects.

Finally, will you start considering improving your test data management and try to make test data single source?

Thanks!