What is test automation
Test automation is an activity that aims to test the most frequently used and least frequently changed parts of an application automatically. This can be done by test automation tools by the help of programming languages supported. Therefore, related test cases are written as a piece of code. Thought test automation activity is criticized under testing activity, it is a software development activity. Every change in the application should be reflected to the test automation scripts so, even the cleverest test scripts are produced, and maintenance activity comes out as problem of test automation since the application is a live object.
|Test Automation Decision Table|
Test automation has an important role in software quality assurance. Because the needs of the sector and trend towards to test automation is getting high, there are a large number of test automation tools exist in the market, but the most famous tools can be listed as follows and more can be found in the list below :
- QTP: Supported by HP, originally develop by Mercury and imported by HP. It has many features, easily record and replay, fast and easily development, short learning curve and powerful support by HP, however for only Windows and costly license price.
- IBM Rational Robot: Developed by IBM, short learning curve for testing team and gives options for more sophisticated scripts for test automation engineer, supports most of the UI platforms, however for only Windows and costly license price.
- Selenium: Open source UI test automation framework, best of the selenium is that you write test cases with your favorite language for webpage testing. Although Selenium is an Firefox plug-in, it can be used for any web browsers. There are lots of blogs, websites and forum to gather help about new features.
Important points for test automation
Test automation is a live object this means that every change made for the item in the scope of test automation should affect the test automation scripts. Keeping updated the scripts is crucial. Therefore, test automation should be started after the test cases are mature and should be apply to suitable test cases for test automation. This can reduce reworks and unnecessary scripting efforts.
Automation scripts should not cause a fail in the system since the aim of the test is to find defects in the software in such a case finding the root of a failure is not feasible. Also in such condition which the automation scripts find a bug or error, it should recover itself then reports the issue and never stops until the full scripts runs. This can be explain by “transition handling” which means that test automation is composed of many separate test cases and these test cases run in a desired/fixed order one by one. Every test case starts then performs its job and then ends then the following test case does its job but the starting and the end points for all the test cases are the same. When a failure is catch the test case doesn’t do its job but goes to the end point which is the starting point of the following test case. This can be illustrated by flower leaf model as shown below:
Another import point is that spending effort for the automation is beneficial or not. The idea behind the automation is to perform many things on terminals without using human efforts instead the human can perform test analysts or other test activities. However maintenance effort should be eliminated because as we said automation is a live object. There should take a statistical study on the aim of the automation and the gains after a successful automation is applied to the project. This is named as automation’s return of investment (ROI), a ration of benefits to cost. A mathematical formula can be written as and more about the calculation of ROI can be found here
ROI should be greater than 100% otherwise test automation is not better than manual test execution. ROI is calculated for only one execution of the test set but the idea on the automation is reusing it for every iteration so if you have a frequently release candidate, you have to test your application before release this means that you should run the automation more than one. If you run the automation for 3 times effective ROI can be written as:
Graph shows the ROI with execution times:
What are the test automation methodologies
Since an aim to automate everything for every condition is an utopia, focusing on an applicable automation target should be more realistic. It can be described by SMART which stands for test automation should be Specific, Measurable, Achievable, Realistic, and Timely. For example, 40% of master test cases with previously defined 10 users on 3 popular web browsers for an e-commerce firm could be applicable test automation strategy. After you have decided a strategy, you can apply a test methodology to practice your test automation. Test methodologies can be described as follows:
- · Record and Replay:
o Records every steps to create a test case
o Easily replay the existing test case
o It doesn’t have loop or if condition or other complicated features
- · Functional Test Automation:
o Every test case is automated by a different test scripts
o Test scripts include test data
o No import/export data option
- · Data-Driven Test Automation:
o In addition to functional test automation
o Test scripts can run by different data
o Data can be stored/read from an external storage/file
o Only data-sets are updated not the scripts
- · Keyword-Driven Test Automation:
o Every case named by a keyword
o The important keywords are sequentially run as a separate test cases
o New test cases can be easily created not need a programming knowledge
o Data and scripts are separately located
o Error handling and Transition handling are difficult
o Creating complex test cases is difficult job
- · Hybrid Test Automation:
o Combination of keyword-driven and data-driven automation techniques
o Exception handling and transition handling mechanism can be used
o Desired data can be used for different test cases
o Data and scripts are separately located