Performance Testing on CI: Integration of Locust and Jenkins
It is a raising question around performance testing people that how to integrate, super hero tool,Locust to Jenkins as a step to continues integration (CI) pipeline. There have been some solutions for this but they are not simple, plus you need to write own log writer to handle this problem. Actually there was a commit to save the result of the test but it has not been merged to Locust. With these commit we can integrate Locust to Jenkins.
Update LocustThis update is still open so you can not see it in latest Locust version yet. However you can use this by updating your local repository, or just change two files main.py and stats.py with the files in your local. For Window change the files in the following directory: C:\Python27\Lib\site-packages\locustio-0.7.3-py2.7.egga\locust\stats.py and for Unix-base OS change the files in the following directory: /Library/Python/2.7/site-packages/locust/stats.py. Alternatively, you can clone this commit from github and install by python setup.py install.
Fix the Bug of The Locust UpdateIt was committed on Jul 12, 2015 but it is still in review status and it has a bug. If they haven't solved it I have added just the following lines of code to fix it. You should find the store_stats(filename, stats)function in stats.py at line 499 and change it with the following function:
Add Build Parameters
Basically, with the performance, we should check the response time with pre-defined treshhold, if any failures and if any redirections. Therefore we need to add some steps to check the log file if these criteria are passed. Instead of defining performance criterias into code directly, I added there build parameters to check min_respons_time, max_response_time and avg_response_time, because these may changes over the time.
Run LocustIf you update your code, you can run your tests with adding new parameters --statsfile=result.log by that you will save the result into result.log file. This file will be present in workspace. The whole command should look like, for Windows I am adding a Window batch command:
C:\Python27\Scripts\locust.exe -f C:\Users\Administrator\Documents\performance\amazon.py --clients=20 --hatch-rate=1 --num-request=500 --no-web --only-summary --statsfile=result.log
Evaluate Test Result
To check the log file create a build step to with execute python script, to add this you need to add python code executer plugin. Then add the following code into this step:
Run the Job
Save the job and click "Build with Parameters", set the values for each then click the "Build" button.
Check the Job Result
Click build number and then click console log to the result of the job.