Skip to main content

Posts

Showing posts with the label Performance

TestOps: What is it and Why we need it

Years of experiences in the software industry shows that there are many more thing that has not been discovered yet. Every approach found in the software industry aims to accelerate the development speed with more customer satisfaction.  Hence the new term "TestOps" is also creating different perspectives on the relationship between test activities to the operational activities within the DevOps culture. In the post, I want to explain my experience with the TestOps.  What is TestOps It shortly stands for tests and operations. Within the DevOps culture, it is a sub-discipline DevOps. There are two approaches to TestOps, these are related to the test approach that you are applying the overall test approach. Mainly if you are applying ` shift-left testing ` then the TestOps should be adopted to have more collaboration with the development team, ` TestOps shift-right `. On the other hand, if you are applying ` shift-right testing ` then the TestOps should be adopted to have more

Automated Performance Testing with K6

Automated testing is running the tests and reporting results by tools with scripts. Automated performance testing is designing the performance test scenarios to run them spontaneously by tools and evaluating the result by tools to decide to go or not to go further. Automated Performance Testing is A subset of the performance test scenarios Designed to run spontaneously by tools Designed to evaluate the result by tools Designed not to break the system Having decision metrics, ex: max response time, max-average resp time, ... Designed to create reports for the concurrent and historical run What Types of Performance Testing Should be Automated Automated performance testing can be applied to all types of performance testing. However, each type needs a different level of maturity and sanity. Automated Performance Testing is Easily applied to load testing Hard to apply stress test, spike test, but have benefits Very hard to apply Soak-Endura

Basics about Performance Testing Tools

I had an opportunity to give a speech about how we can automate performance testing in CI/CD pipeline in a conference held in Istanbul for the first time " Test automation and Digital QA Submit Istanbul, #TAS19 ". My subject was "Automated Performance Testing", the first part of the speech is to explain what is performance tests and the second part is to explain performance tools, and the last section is to explain what is automated performance testing and how to implement it to CI/CD.  In this post, I want to explain my thoughts about the basics of performance testing tools. Next posts will focus on automated performance testing and using K6 as a performance testing tool. How to Evaluate Performance Testing Tools Before diving into the performance testing tools and technologies we have in the industry, we should define the criteria for evaluation. Each tool has its own advantages or disadvantages and similarity between ancestors or others. When we star

Performance Testing

I had an opportunity to give a speech about how we can automate performance testing in CI/CD pipeline in a conference held in Istanbul for the first time " Test automation and Digital QA Submit Istanbul, #TAS19 ". My subject was "Automated Performance Testing", the first part of the speech is to explain what is performance tests.  In this post, I want to explain my thoughts about performance testing. Next posts will focus on automated performance testing and using K6 as a performance testing tool. What is Performance Testing By Definition Defining the performance of the system by testing the functionalities of the system in terms of the non-functionalities   Performance Testing is A non-functional testing Testing the functionalities of the system Both black-box and white-box testing Defining how the system works under any loads Applied under different load levels Applied under different load increments Applied under different dur

Isolated - Scalable Performance Testing: Locust in Dockers

I have shared some posts how to run Locust in Local or in Cloud, as slave or master. At this time I want to share how you can run it in a Docker. To fully get the benefits of Locust I am using it with Python 3 so I created a Docker file and images upload to the Docker Hub , and the project is on the GitHub .  The Dockerfile has the minimum available requirements but we also have a new file called `requirements.txt` which you can add required python libraries to install inside the container by pip install lib . The Dockerfile is: When you got the docker-locust image, you can now run your script. At the end of the Dockerfile you can see that ENTRYPOINT [ "/usr/local/bin/locust" ] this enable us to use the image as service  which means that you can directly call it same as using Locust installed locally. See the run command below: Running Locust in a Docker Running the Locust with my image is pretty simple if you used without docker, just try other options w

Confusion of Using Selenium for Performance Testing

Time to time, I see some questions about integration test automation script to performance testing tool on Stackoverflow . There is a confusion about performance testing and test automation for whom newly started performance testing after doing some test automation. It is demanded that they have some automation code and want to use these scripts for performance testing to simulate user behaviour. Theoretically it seems a very good idea that tons of users are performing some cases and you are capturing the response time and finding bottle neck in terms of client side. However this is not applicable in practice because there are lots of obstacle to handle. The followings outline the obstacles and misunderstandings you will face, for the idea. I can open some browsers and simulate real user behaviour Automation scripts run over a browser, for web project. Most of the application are becoming as web app. For GUI based application what the hack test automation will have a role on pe

Performance Testing on CI: Integration of Locust and Jenkins

It is a raising question around performance testing people that how to integrate, super hero tool, Locust to Jenkins as a step to continues integration (CI) pipeline. There have been some solutions for this but they are not simple, plus you need to write own log writer to handle this problem.  Actually there was a commit to save the result of the test but it has not been merged to Locust. With these commit we can integrate Locust to Jenkins. Update Locust This  update  is still open so you can not see it in latest Locust version yet. However you can  use this by updating your local repository, or just change two files  main.py  and  stats.py  with the files in your local. For Window change the files in the following directory:  C:\Python27\Lib\site-packages\locustio-0.7.3-py2.7.egga\locust\stats.py and for Unix-base OS change the files in the following directory:  /Library/Python/2.7/site-packages/locust/stats.py.  Alternatively, you can clone this commit from github and instal

Why Performance Test on Cloud

When a software comes to production the weakest points are mostly related to performance of the application. The reason is that most of the organization test the application functionally by testers and on acceptance testing period again the applications is under testing to check functional requirements. The missing point is non-functional testing like performance of the application.  Another failure point is testing the performance as the last level of development process. This means that any misleading failures in the service level can cause a very big problem on production stage so it requires extra source if possible and extra budget. As the terminology says "finding the big issue earlier stage of the development can reduce the fixing effort exponentially " . Therefore test the performance of the software output whenever it is ready, this could be an api end point, a function servers data to system, a web servers, and etc.  Another failure is related to technolo

New Modern Open Source Performance Testing Tool : Locust

If you are bored with using old fashioned Java GUI to test the performance of your applications, then you should try Locust. Jmeter , which is old fashioned as I mentioned, is very  mature and used by a big community but you have create your test case with an ugly GUI as much features as allows you. For me the best part of Locust is that you can write your test case with Python which is my favourite language and it is very simple to learn. The installation is very simple, you just need to type: > pip install locustio Some feature / deficiency  of Locust:  You can write your test case in Python You can define the user time consumption on a page with min_wait and max_wait  as milliseconds in locust class so that get a real user simulation.  You can give weight for each test cases so you can simulate real user behaviours like during a specific period of time: 1 person signs up, 10 persons login, 60 persons visit pages, ...  You can make http request as you