1.Challenges of test results display
A number of report generating tools are often used on automation test, such as Junit, TestNG, ReportNG. The reports generated by these tools need some extra work to get a relative intuitive and concise report. ExtentReport and Extentx is such a platform which can provide a real time and elegant dashboard easily.
2.Integrating Extent into automated testing
ExtentReport is a HTML reporting library for .NET and Java which is extremely easy to use and create beautiful execution reports.
Extentx is another service in Extent, this is a web program that can make a perfect collection report.
Using ExtentReport to generate test reports, a separate test report is generated each time the test is completed. But each project has a report that will be redundant when there are more than one project. Therefore, you can use Extentx when you want to manage multiple projects.
Figure 1 Integrating extent into selenium
The following is an introduction to the framework:
In our project, extent is integrated into the selenium. The whole frame can be divided into four layers, from bottom to top respectively DB layer, Utility, Test and Report.
When a test case starts execution, test data is get from the database, then use TestBase to aggregates PageManager, CustomAssertion, ComplexReportFactory, and set the name of the report and execute the browser used, for example chrome. CustomAssertion passes the results of the assertion to the report and put test results into the database. ComplexReportFactory controls whether to produce a local report or a service report. The local reports are stored in the OUT-PUT folder and can be managed using server reports when there are multiple local reports.
The detail of Utility layer:
PageManager class is used to initialize base pages, which is the father of all the Page objects and invokes the selenium Page factory initialization Page in the constructor. And you can also define some common methods for operating page elements, like getText, sendKeys, click and so on.
The main purpose of the CustomAssertion class is to pass the results to the test report, as well as take a screenshot on error page and record the error messages.
In ComplexReportFactory class, here are two modes of presentation, namely ExtentHtmlReporter and ExtentxReporter. ExtentHtmlReporter configures the local report, sets the path, name of the report. ExtentxReporter configures the server report, sets the project, report name, display port, address, etc. In this class, there are the reporter object controls whether to produce a local report or a service report.
3.Analyze the test report generated by Extent
All these test reports are intuitive and elegant that it can be easy to get information, such as the name of the browser used, the name of test case, the time and status of the test case execution.
Looking at Figure 2, from the two parts of “Feature Trends by Status” and “Scenario Trends By Status”, it can be seen that total of 5 rounds of test and the time of the start of each execution. In each execution, 2 test cases were executed, one is passed which is marked as Green successfully executed, and one failed. In “Report Performance” part, it can be seen the time of each test case is executed.
Figure 2 Test reports
Figure 3 is the report list, project name, test case name, start time, end time, and duration can be found. For example, in the fifth run, a total of 6 test cases were executed, with 5 passed and 1 failed.
There are 30 steps in the 6 test cases. 29 steps were successful and one step was failed.
Figure 3 Test reports
When click one of the build in “report list”, such as “ExtentReport_20175819_065835”. The detail of the report is displayed. As shown in Figure 4 below, it also can be seen test case name, start time, end time, how long did the test run, every step of each test case. For example, as shown in Figure 4, we can see the browser used is Firefox, the error step is the fifth, the error message and the screenshot of the error page.
Figure 4 Test reports
Figure 5 is a screenshot of the fifth step above.
Figure 5 error screenshot
Figure 6 is the “top failed” list, which shows the list of the most failures during the execution of these test cases. We can see the test case name and count of failures.
Figure 6 Test reports
When click one of the build in “top failed”, such as” Firefox-TC001_US001_LocationList-verifyLocationListAppear”. The detail of the build is displayed. As shown in Figure 7 below, we can see the history of the failure and the detailed failure information for each record, such as failure steps and screenshots.
Figure 7 Test reports
4.Conclusion
After using ExtentReport and Extentx, the automation test generated the perfect collection report. This report shows both the summary and step details of the test (status, error, etc.) and also provides analysis dashboard, these pages use Tab tag links to facilitate separation, quick switching view, and so on. These advantages make it easier for users to read test reports and manage it. The advantages of the A report are summarized below:
- Simple and intuitive
- Settings can be set as needed, such as setting the title, displaying the browser used, screenshot and error information.
- View more detailed information, such as specific steps, statistics of error messages, test execution times (including start times, end times, total usage times)
- There are two report forms, local reports and server reports. When there are multiple local reports in same projects, we can use server reports to manage them.
- Use Extentx to manage the report of multiple projects.
