Skip to main content

Well Defined Load Testing Structure

Image
By
Testing functional requirements of your system in the way how it behaves under different types of load is a basic explanation for performance testing. However we test the functions of the system, performance testing is classified under "non-functional testing" because we don't aim to test the functions we want to test how the system is functioning under different amount of loads. Therefore  performance testing should be performed after the functional testing has been completed. Performance testing is crucial if you are testing an application which has an un-predictable load of user or if you have specified the quality of service in your Service-Level Agreement (SLA). Performance can not be defined by one of testing criteria since there are number of parameters in the system. I mean by this, there are subsets for performance testing which can be listed as
  • Performance Testing
  • Load Testing
  • Endurance Testing
  • Stress Testing
  • Spike
We can explain the differences between the performance testing activities as: performance testing is an general approach for performing testing to investigate CPU usage, memory, response time, speed, stability and scalability of the system under test (SUT), however the load testing aims to show the behaviour of the SUT for optimal and peak level of load and the endurance testing is to find how long can be the SUT available without error under specific loads and to find how long the time between two error point as mean time between failure (MTBF) and Spike testing is to find how the SUT behave when the load is suddenly  increased to peak level. Stress testing finds how much load can be eligible for  the SUT and to find the break point in terms of load for the SUT. 


Well Defined Load Testing

Since the performance testing is a detailed testing activity, in this post, I want to give some basic information about structure of load testing. Using realistic data for load testing is important to define user profile and distribution of data efficiently and correctly. Statistical variables can be used for this study; for example, any web application can be tracked by Google Analytics or Yandex Metrica in a very detail form of data. These application can show you how many users have been used the link and how many of them came as a first time and how many of them came as returning user. With this info number of users for each function of the SUT can be easily estimated.

Google Analytics
For this study, first you need to introduce the functions of the SUT, and then define the steps of each function. After that you can subtract the repeated steps then regenerate the combination of the function. Let's design an example for an e-commerce web application, first we should define the main functions or functional requirements which should be a subset of the regression test cases have already been defined. If the number of regression test cases are to many then  you can reduce the number by applying the risk analysis, it depends on the sector but we can cover 75% of the test cases take the risk of 25% of the usage rate of the function can be taken into the account. The number can be reduced 25% by the assumption of Pareto Principle. Assume that the followings are the main functions of the web application and the number of the users are given for each:
  1. log in: 80
  2. product detail view: 40
  3. search: 30
  4. basket showing: 20
  5. sign-up: 10
  6. buying: 5
These numbers show how many users use the SUT concurrently. Load testing is performed for the load between nominal and the peak value so if we know that these numbers are the average or nominal load then the percentage can be used directly otherwise we should not choose them under the average of the load. If we assume that the one of the the web server in the SUT can service 100 request concurrently then we should have 100 virtual users to perform load testing. Then the following table shows that the percentage of each function and the assigined number of virtual users.
 
Number of Virtual Users for Each Function
Load testing should be performed at least 3 times and taken the average values so that the SUT is behave consistent for the load testing scenarios. The important point for load testing, the result should be compared to the threshold value which has been defined by result of healthy system. So if the result is better than the threshold, the newly added feature/release can go live!
Labels:Software testing

Popular posts for software testing and automation

Selenium Error "Element is not currently interactable and may not be manipulated"

By
Selenium webdriver can drive different browsers like as Firefox, Chrome or Internet Explorer. These browsers actually cover the majority of internet users, so testing these browsers possibly covers the 90% of the internet users. However, there is no guaranty that the same automation scripts can work without a failure on these three browsers. For this reason, automation code should be error-prone for the browsers you want to cover. The following error is caught when the test script run for Chrome and Internet Explorer, but surprisingly there is no error for the Firefox. Selenium gives an error like below: Traceback (most recent call last):   File "D:\workspace\sample_project\sample_run.py", line 10, in <module>     m.login()   File "D:\workspace\ sample_project \test_case_imps.py", line 335, in login     driver.find_element_by_id("id_username").clear()   File "C:\Python27\lib\site-packages\selenium-2.35.0-py2.7.egg\selenium\webdriver\r
Read more

Change Default Timeout and Wait Time of Capybara

By
One of the biggest challenge for automation is handling timeout problem. Most of the time, timeout is 60 seconds but it may sometimes not enough if you have badly designed asynchronous calls or the third party ajax calls. This makes handling timeout more complex. set large enough to tolerate network related problems. For Selenium based automation frameworks, like Capybara, default Webdriver timeout is set to Net::ReadTimeout (Net::ReadTimeout) Changing ReadTimeout If you have timeout problem for Capybara, it gives an error like above. This means that the page is not fully loaded in given timeout period. Even you can see that page is loaded correctly but webdriver wait until the Ajax calls finish. class BufferedIO #:nodoc: internal use only def initialize (io) @io = io @read_timeout = 60 @continue_timeout = nil @debug_output = nil @rbuf = '' end . . . . . def rbuf_fill beg
Read more

Create an Alias for Interactive Console Work: Selenium and Capybara

By
If you are working on shell most of the time Aliases are very helpfull and time saving. For testing purposes you can use Alias for getting ready your test suites. In this post, I want to explain both running Selenium and Capybara on console and creating aliases for each.  This post is for Windows machines, if you are using Unix-like see   this post . Creating Scripts for Selenium and Capybara First of all, it is assumed that you have installed Selenium and Capybara correctly and they work on your machines. If you haven't installed, you can see my previous posts. I am using the Selenium with Python and the Capybara with Ruby. You can use several different language for Selenium but Capybara works only with Ruby.  Create scripts in a directory called scripts (in your home folder, like as  ~/scripts ) for your automation tool as following, save them as capybara.rb, sel.py :  Creating Aliases Depends on your favourite shell, you need to add the alias to .bashrc bash
Read more

Page-Object Pattern for Selenium Test Automation with Python

By
Page-object model is a pattern that you can apply it to develop efficient automation framework. With the page-model, it is possible to minimize maintenance cost. Basically page-object means that your every page is inherited from a base class which includes basic functionalities for every page. If you have some new functionalities that every page should have, you can simple add it to the base class. Base class is like the following: In this part we are creating pages which are inherited from base page. Every page has its own functionalities written as python functions. Some functions return to a new page, it means that these functions leave the current page and produce a new page. You should write as much as functions you need in the assertion part because this is the only part you can use the webdriver functions to interact with web pages . This part can be evaluate as providing data to assertion part.   The last part is related to asserting your test cases against to the
Read more

Performance Testing on CI: Locust is running on Jenkins

By
For a successful Continuous Integration pipeline, there should be jobs for testing the performance of the application. It is necessary if the application is still performing well. Generally performance testing is thought as kinds of activities performed one step before going to live. In general approach it is true but don't forget to test your application's performance as soon as there is an testable software, such as an api end point, functions, and etc. For CI it is a good approach to testing performance after functional testing and just before the deployment of next stage. In this post, I want to share some info about Jenkins and Locust. In my previous post you can find some information about Locust and Jenkins. Jenkins operates the CI environment and Locust is a tool for performance testing. To run the Locust on Jenkins you need command line arguments which control the number of clients ,   hatch rate,  running locust without web interface and there should be so
Read more