In our automation testing project practice, we may need to change the attributes of @Test (e.g. priority and group) for the existing scripts due to the requirements change. Or we may need to collect the regression result data to generate a customized report. But for the large number of existing scripts, it could take a lot of time to modify the scripts code one by one. TestNG listeners can resolve these problems in a simpler way.
TestNG provides us multiple listener interfaces to customize the actions upon various testing events or status (such as test fail, test finish, test skip, before test start), or define different TestNG behaviors.
Below is a brief introduction for some listeners which can be found in TestNG api.
IAnnotationTransformer: dynamically defines @Test attributes, such as group, always run, priority …
IAnnotationTransformer2: defines the attributes of the annotations like @Configuration, @DataProvider and @Factory. But it is NOT applicable to @Test
IAnnotationTransformer3: defines the annotations in @Listener.
IClassListener: The interface where actions can be defined after the class is invoked.
IConfigurationListener: The interface where actions can be defined when the configuration method is failed/skipped/succeeded.
IConfigurationListener2: The interface where actions can be defined before a configuration method is invoked.
IReporter: can be used to generate customized report.
ISuiteListener: The interface where actions can be defined when the suite is started or finished.
ITestListener : The interface where actions can be defined for test running.
Below is the list of listeners which may not be used frequently.
I hereby would discuss two of the mentioned listeners as they may be implemented properly in our automated testing projects.
- IAnnotationTransformer: This listener provides the interfaces where we can batch process the test cases dynamically. There is one method to do this.
This method will be invoked by TestNG to give testers a chance to modify a TestNG annotation read from your test classes. Therefore, we can implement this listener to set the case priority, group, time, etc.
For example: we would like all test cases to run in order based on the test case IDs, assuming the test case ID is numerical ONLY. We can then set the priority for every test case based on its ID. TestNG will run the test cases one-by-one in the ascending order of the IDs.
We can also set the attributes of @Test (such as group, time, enable….) in the same way.
It’s a good way to set the attributes dynamically. We can then batch process the test cases, for example, running all the test cases whose priority is lower than 185 as indicated in the Green Rectangle in the above picture.
2.ITestListener : To monitor the status of a test case during runtime, this listener would be the best choice.
In our project practice, we use this listener to improve the efficiency of our test results analysis. If the case faila, this failure is recorded and the throwable exception information is caught. Then we can compare the content of the caught information to the one that was caught in the regression run of last time, assuming the information is stored in an excel file. If they are the same, it means the same test case failed at the same place in both regression runs. We may determine that the cause of the failure for this test case is the same in both runs. Thus, if the cause of failure has been determined from the analysis process of the last regression run, then we may determine that the cause of failure remains same this time. Such a comparison will save much time for the automation testers in the analysis process. The following is the partial code.
