Well Defined Load Testing Structure

Testing functional requirements of your system in the way how it behaves under different types of load is a basic explanation for performance testing. However we test the functions of the system, performance testing is classified under "non-functional testing" because we don't aim to test the functions we want to test how the system is functioning under different amount of loads. Therefore  performance testing should be performed after the functional testing has been completed. Performance testing is crucial if you are testing an application which has an un-predictable load of user or if you have specified the quality of service in your Service-Level Agreement (SLA). Performance can not be defined by one of testing criteria since there are number of parameters in the system. I mean by this, there are subsets for performance testing which can be listed as
  • Performance Testing
  • Load Testing
  • Endurance Testing
  • Stress Testing
  • Spike
We can explain the differences between the performance testing activities as: performance testing is an general approach for performing testing to investigate CPU usage, memory, response time, speed, stability and scalability of the system under test (SUT), however the load testing aims to show the behaviour of the SUT for optimal and peak level of load and the endurance testing is to find how long can be the SUT available without error under specific loads and to find how long the time between two error point as mean time between failure (MTBF) and Spike testing is to find how the SUT behave when the load is suddenly  increased to peak level. Stress testing finds how much load can be eligible for  the SUT and to find the break point in terms of load for the SUT. 


Well Defined Load Testing

Since the performance testing is a detailed testing activity, in this post, I want to give some basic information about structure of load testing. Using realistic data for load testing is important to define user profile and distribution of data efficiently and correctly. Statistical variables can be used for this study; for example, any web application can be tracked by Google Analytics or Yandex Metrica in a very detail form of data. These application can show you how many users have been used the link and how many of them came as a first time and how many of them came as returning user. With this info number of users for each function of the SUT can be easily estimated.

Google Analytics
For this study, first you need to introduce the functions of the SUT, and then define the steps of each function. After that you can subtract the repeated steps then regenerate the combination of the function. Let's design an example for an e-commerce web application, first we should define the main functions or functional requirements which should be a subset of the regression test cases have already been defined. If the number of regression test cases are to many then  you can reduce the number by applying the risk analysis, it depends on the sector but we can cover 75% of the test cases take the risk of 25% of the usage rate of the function can be taken into the account. The number can be reduced 25% by the assumption of Pareto Principle. Assume that the followings are the main functions of the web application and the number of the users are given for each:
  1. log in: 80
  2. product detail view: 40
  3. search: 30
  4. basket showing: 20
  5. sign-up: 10
  6. buying: 5
These numbers show how many users use the SUT concurrently. Load testing is performed for the load between nominal and the peak value so if we know that these numbers are the average or nominal load then the percentage can be used directly otherwise we should not choose them under the average of the load. If we assume that the one of the the web server in the SUT can service 100 request concurrently then we should have 100 virtual users to perform load testing. Then the following table shows that the percentage of each function and the assigined number of virtual users.
 
Number of Virtual Users for Each Function
Load testing should be performed at least 3 times and taken the average values so that the SUT is behave consistent for the load testing scenarios. The important point for load testing, the result should be compared to the threshold value which has been defined by result of healthy system. So if the result is better than the threshold, the newly added feature/release can go live!

Comments

Popular posts from this blog

Selenium Error "Element is not currently interactable and may not be manipulated"

Change Default Timeout and Wait Time of Capybara

Performance Testing on CI: Locust is running on Jenkins

Selenium Chromedriver Error Message: u'unknown error: unable to discover open pages

Getting the text of elements in Espresso