Skip to main content

Test Automation

Image
By

What is test automation


Test automation is an activity that aims to test the most frequently used and least frequently changed parts of an application automatically. This can be done by test automation tools by the help of programming languages supported. Therefore, related test cases are written as a piece of code. Thought test automation activity is criticized under testing activity, it is a software development activity. Every change in the application should be reflected to the test automation scripts so, even the cleverest test scripts are produced, and maintenance activity comes out as problem of test automation since the application is a live object.
 
Test Automation Decision Table

Test automation has an important role in software quality assurance. Because the needs of the sector and trend towards to test automation is getting high, there are a large number of test automation tools exist in the market, but the most famous tools can be listed as follows and more can be found in the list below : 
  •  QTP: Supported by HP, originally develop by Mercury and imported by HP. It has many features, easily record and replay, fast and easily development, short learning curve and powerful support by HP, however for only Windows and costly license price. 
  • IBM Rational Robot: Developed by IBM, short learning curve for testing team and gives options for more sophisticated scripts for test automation engineer, supports most of the UI platforms, however for only Windows and costly license price.   
  • Selenium: Open source UI test automation framework, best of the selenium is that you write test cases with your favorite language for webpage testing. Although Selenium is an Firefox plug-in, it can be used for any web browsers. There are lots of blogs, websites and forum to gather help about new features.

 

Important points for test automation


Test automation is a live object this means that every change made for the item in the scope of test automation should affect the test automation scripts. Keeping updated the scripts is crucial. Therefore, test automation should be started after the test cases are mature and should be apply to suitable test cases for test automation. This can reduce reworks and unnecessary scripting efforts.

Automation scripts should not cause a fail in the system since the aim of the test is to find defects in the software in such a case finding the root of a failure is not feasible. Also in such condition which the automation scripts find a bug or error, it should recover itself then reports the issue and never stops until the full scripts runs. This can be explain by “transition handling” which means that test automation is composed of many separate test cases and these test cases run in a desired/fixed order one by one. Every test case starts then performs its job and then ends then the following test case does its job but the starting and the end points for all the test cases are the same. When a failure is catch the test case doesn’t do its job but goes to the end point which is the starting point of the following test case. This can be illustrated by flower leaf model as shown below:



Another import point is that spending effort for the automation is beneficial or not. The idea behind the automation is to perform many things on terminals without using human efforts instead the human can perform test analysts or other test activities. However maintenance effort should be eliminated because as we said automation is a live object. There should take a statistical study on the aim of the automation and the gains after a successful automation is applied to the project. This is named as automation’s return of investment (ROI), a ration of benefits to cost. A mathematical formula can be written as and more about the calculation of ROI can be found here




ROI should be greater than 100% otherwise test automation is not better than manual test execution. ROI is calculated for only one execution of the test set but the idea on the automation is reusing it for every iteration so if you have a frequently release candidate, you have to test your application before release this means that you should run the automation more than one. If you run the automation for 3 times effective ROI can be written as:

Graph shows the ROI with execution times:

 

What are the test automation methodologies


Since an aim to automate everything for every condition is an utopia, focusing on an applicable automation target should be more realistic. It can be described by SMART which stands for test automation should be Specific, Measurable, Achievable, Realistic, and Timely. For example, 40% of master test cases with previously defined 10 users on 3 popular web browsers for an e-commerce firm could be applicable test automation strategy. After you have decided a strategy, you can apply a test methodology to practice your test automation. Test methodologies can be described as follows:
  • ·         Record and Replay:
o  Records every steps to create a test case
o   Easily replay the existing test case
o   It doesn’t have loop or if condition or other complicated features
  • ·         Functional Test Automation:
o   Every test case is automated by a different test scripts
o   Test scripts include test data
o   No import/export data option
  • ·         Data-Driven Test Automation:
o   In addition to functional test automation
o   Test scripts can run by different data
o   Data can be stored/read from an external storage/file
o   Only data-sets are updated not the scripts
  • ·         Keyword-Driven Test Automation:
o   Every case named by a keyword
o   The important keywords are sequentially run as a separate test cases
o   New test cases can be easily created not need a programming knowledge
o   Data and scripts are separately located
o   Error handling and Transition handling are difficult
o   Creating complex test cases is difficult job
  • ·         Hybrid Test Automation:
o   Combination of keyword-driven and data-driven automation techniques
o   Exception handling and transition handling mechanism can be used
o   Desired data can be used for different test cases
o   Data and scripts are separately located
Labels:Software testing

Popular posts for software testing and automation

Selenium Error "Element is not currently interactable and may not be manipulated"

By
Selenium webdriver can drive different browsers like as Firefox, Chrome or Internet Explorer. These browsers actually cover the majority of internet users, so testing these browsers possibly covers the 90% of the internet users. However, there is no guaranty that the same automation scripts can work without a failure on these three browsers. For this reason, automation code should be error-prone for the browsers you want to cover. The following error is caught when the test script run for Chrome and Internet Explorer, but surprisingly there is no error for the Firefox. Selenium gives an error like below: Traceback (most recent call last):   File "D:\workspace\sample_project\sample_run.py", line 10, in <module>     m.login()   File "D:\workspace\ sample_project \test_case_imps.py", line 335, in login     driver.find_element_by_id("id_username").clear()   File "C:\Python27\lib\site-packages\selenium-2.35.0-py2.7.egg\selenium\webdriver\r
Read more

Change Default Timeout and Wait Time of Capybara

By
One of the biggest challenge for automation is handling timeout problem. Most of the time, timeout is 60 seconds but it may sometimes not enough if you have badly designed asynchronous calls or the third party ajax calls. This makes handling timeout more complex. set large enough to tolerate network related problems. For Selenium based automation frameworks, like Capybara, default Webdriver timeout is set to Net::ReadTimeout (Net::ReadTimeout) Changing ReadTimeout If you have timeout problem for Capybara, it gives an error like above. This means that the page is not fully loaded in given timeout period. Even you can see that page is loaded correctly but webdriver wait until the Ajax calls finish. class BufferedIO #:nodoc: internal use only def initialize (io) @io = io @read_timeout = 60 @continue_timeout = nil @debug_output = nil @rbuf = '' end . . . . . def rbuf_fill beg
Read more

Create an Alias for Interactive Console Work: Selenium and Capybara

By
If you are working on shell most of the time Aliases are very helpfull and time saving. For testing purposes you can use Alias for getting ready your test suites. In this post, I want to explain both running Selenium and Capybara on console and creating aliases for each.  This post is for Windows machines, if you are using Unix-like see   this post . Creating Scripts for Selenium and Capybara First of all, it is assumed that you have installed Selenium and Capybara correctly and they work on your machines. If you haven't installed, you can see my previous posts. I am using the Selenium with Python and the Capybara with Ruby. You can use several different language for Selenium but Capybara works only with Ruby.  Create scripts in a directory called scripts (in your home folder, like as  ~/scripts ) for your automation tool as following, save them as capybara.rb, sel.py :  Creating Aliases Depends on your favourite shell, you need to add the alias to .bashrc bash
Read more

Page-Object Pattern for Selenium Test Automation with Python

By
Page-object model is a pattern that you can apply it to develop efficient automation framework. With the page-model, it is possible to minimize maintenance cost. Basically page-object means that your every page is inherited from a base class which includes basic functionalities for every page. If you have some new functionalities that every page should have, you can simple add it to the base class. Base class is like the following: In this part we are creating pages which are inherited from base page. Every page has its own functionalities written as python functions. Some functions return to a new page, it means that these functions leave the current page and produce a new page. You should write as much as functions you need in the assertion part because this is the only part you can use the webdriver functions to interact with web pages . This part can be evaluate as providing data to assertion part.   The last part is related to asserting your test cases against to the
Read more

Performance Testing on CI: Locust is running on Jenkins

By
For a successful Continuous Integration pipeline, there should be jobs for testing the performance of the application. It is necessary if the application is still performing well. Generally performance testing is thought as kinds of activities performed one step before going to live. In general approach it is true but don't forget to test your application's performance as soon as there is an testable software, such as an api end point, functions, and etc. For CI it is a good approach to testing performance after functional testing and just before the deployment of next stage. In this post, I want to share some info about Jenkins and Locust. In my previous post you can find some information about Locust and Jenkins. Jenkins operates the CI environment and Locust is a tool for performance testing. To run the Locust on Jenkins you need command line arguments which control the number of clients ,   hatch rate,  running locust without web interface and there should be so
Read more