Myth 1: Automation replaces Manual Testers.
Reality: After the scripts are developed, the common misconception is that we can replace the manual testers with Automation, but this is not true. Automation is a program which will test the flow of the system. Even a small defect can be missed if we don’t write scripts to look for it, but this can always be found during manual test execution. Also, if the application being tested undergoes changes, this often results in the failure of scripts and the manual tester must test the changes until the scripts are updated.
Even though automation reduces the amount of time taken by manual testing, one needs to spend time on test results analysis to make sure that automation has not missed out any critical defects.
Solution:
- We always need a human brain to analyze the test results. Robots cannot replace humans in automation testing.
- For applications, which are changing frequently, automation testing can only be used as a reference and not as a replacement for a manual tester.
- Automation should be used for static applications which are independent of other modules and which need to be regression tested.
Myth 2: Automation executes fast whereas manual testing executes slow.
Reality: Automation runs faster than manual testing but sometimes when the properties involved for the object are not straight forward, then automation testing would take time in identifying those objects.
For example: If a page has multiple headers and has similar properties, then automation might take some time in identifying those headers whereas manual tester does an easy check for its existence.
Preparation of Test data also involves time because it involves writing a logic for Automation testing while manual tester does it faster because they know the test data to choose for the application which in turn increases the execution speed.
While testing, applications communicating with multiple systems both manual and automation takes the same time to execute because of the dependency on external system.
Solution:
One must take an account of the time needed for execution, the complexity of the application and number of external system involved. All these must be done during requirement analysis. A feasibility study must be done on the time taken for execution and effort spent in automating application.
- All the stakeholders should be made aware of the automation effort involved.
- The estimation effort should be calculated by taking the following factors into consideration: Objects complexity, flow of the system, time needed to build the scripts and lastly the execution time of automation verses manual testing.
Myth 3: Users just need to click a button to execute the automated test cases.
Reality: When automation is completed and delivered to end users, they assume that scripts can be executed without any change in test data but this happens only in a rare situation. End users should also spend their time in execution. They should be aware of the flows of the application, keywords used in the application and the test data needs to be given.
Solution:
We should have a periodic discussion about the test case flow and test data to the end users which will avoid wrong expectations.
- We should also understand the test data used which can be constant and which can be used as an input. This will minimize the number of data inputs needed for each test case.
Myth 4: Expecting 100% automation test execution without any failures.
Reality: Automation is also like application development with lot of limitations and hence executing all scripts without errors is impossible. There can be many reasons like slowness of system, Environment outage, data issues, UI changes etc.
Solution:
End users should understand the reason of the failure before passing it to the automation team. This helps in understanding the flows of the system and the inputs needed for it rather than depending on automation team.
- Automation is ideal for stable systems or for the systems for which the development is complete.
- We must make sure all test cases are updated with latest data and there is no factor affecting the system execution.
Myth 5: When automating a new application, one can study the existing framework and this doesn’t need time to invest on.
Reality: Practically, Automation is not easy. It needs a lot of time, money and mainly patience. If used in a right way, automation can help organization in a big way. Testers need to understand the domain, the test cases to be automated, choosing the right framework to build a strong foundation in automation.
Automation is also like an application development which needs a lot of testing. The automation scripts must be tested thoroughly with all set of test data both positive and negative. Improper testing or partly tested tool results in failure of scripts and leads to the lack of confidence in the tool.
Solution:
- Understanding the application thoroughly which needs to be automated would give a better clarity to the scripts.
- While setting the timeline for automating the scripts, the time needed for requirement gathering, design, coding and testing should also be considered.
- Discuss with Manual testers and developers on the key areas before deciding the suitable framework and tool.
Conclusion:
Automation testing can deliver long-term benefits but not suitable for immediate results as the scripts should be constantly updated and maintained. We should also understand that automation is not only a magic tool to meet the deadlines of execution but also has its own limitations hence automation testing is highly effective and can yield best results when it is combined with manual testing.
