Technical

Running Automated Tests in Parallel with TestNG

tiethnic colleagues sitting at desk looking at laptop computer in office. stock photo

Running automated tests in parallel can be a time saver.  There are 2 aspects I will be discussing: the first one refers to using data providers, while the second one implies that you are grouping and running tests using XML suite files.

 

Using data providers

Whenever a test method receives parameter values through a data provider method, we annotate the provider method with the ‘@DataProvider’ annotation. By default, we don’t specify any attribute to this annotation. The test method is invoked once, with one of the parameter values at a time. 

If in turn, we would like for the values to be passed to the test method in parallel, so that the same method but with different values runs in parallel, we can update the ‘@DataProvider’ annotation. For that, we only need to add the ‘parallel’ attribute, and assign it the ‘true’ value, as a String:

@DataProvider(parallel = true)

By default, in TestNG, the maximum number of tests that will run in parallel that are using this data provider is 10.

 

Suite-level parallelism configuration

Whenever we have XML configuration suites files, whether or not any test in the suite is using a @DataProvider, we can apply parallelism to the tests in the suite. Looking at a simple XML configuration suite is made up of: one ‘suite’ tag; one or more ‘test’ tags inside the ‘suite’ tag; one or more ‘package’ or ‘class’ tags inside each ‘test’ tag. A simple example of a suite file using packages would be:

<suite name="Parallel Suite">

<test name="First test tag">

<packages>

<package name="firstPackage"/>

</packages>

</test>

<test name="Second test tag">

<packages>

<package name="secondPackage"/>

</packages>

</test>

</suite>

 

Now, in order to assign parallelism to the tests, we can either do that on a ‘suite’ tag level or a ‘test’ tag level:

  • on a suite tag level, we can have parallelism so that:

We run each ‘test’ tag in parallel – that means that everything inside the test tag will run sequentially, but the test tags themselves will run in parallel. To achieve this, at the ‘suite’ tag level, we need to add the attribute ‘parallel’ with value ‘tests’:

 

<suite name="Parallel Suite" parallel="tests">
    • we run classes in parallel – this means the ‘test’ tags will run sequentially, but whenever the current ‘test’ tag runs, the classes inside the ‘test’ tag will run in parallel. To achieve this, at the ‘suite’ tag level, we need to add the attribute ‘parallel’ with value ‘classes’:
<suite name="Parallel Suite" parallel="classes">
    • we run methods in parallel – this means the ‘test’ tags will run sequentially, but whenever the current ‘test’ tag runs, the methods inside the ‘test’ tag will run in parallel. To achieve this, at the ‘suite’ tag level, we need to add the attribute ‘parallel’ with value ‘methods’:
<suite name="Parallel Suite" parallel="methods">

 

  • on a test tag level: We can add, for each test tag, parallelism for either classes or methods. This is achieved using the same attribute and values as before. Either:

<test name=“First test tag” parallel=“classes”>

or

<test name=“First test tag” parallel=“methods”>

However, too much parallelism can lead to unexpected results and unpredictability. Especially if you have dependencies between tests, possibly data used by each test, where each test updates this data. So, use this mechanism carefully.

Conclusion

As we have seen in this article, our tests can easily be configured to run in parallel with TestNG. We can try different settings to find the best way to configure them so that they run efficiently and predictably.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Mangesh Sonwane

Mangesh Sonwane is an Associate Technical Consultant at Perficient in Nagpur GDC. He has an experience of 1+ years in Drupal and AEM. Apart from this passionate about learning new changes and expanding his knowledge in Automation. He is very committed to his work and is always ready to face any challenging projects.

More from this Author

Follow Us
TwitterLinkedinFacebookYoutubeInstagram