Headless Miracles: Chromedriver Headless VS Chromdriver

You may have heard that we are running the cases in the headless mode so that we could accelerate the execution of the test cases.  So is this true all the time? In this post, I have a little test to compare the headless mode in Chromedriver with version 2.33 and Chromedriver. The tests were run in Windows.

I am using Capybara, I have around 200 test cases written in Cucumber. Tests are running parallel with 15 execution lines. This execution is controlled by tags so we can get the execution time when a tag finished. With this way, we can compare the tag specific time differences and the total time difference. I am using the following Chromedriver instances written in env.rb file in the project.

TAGS Chrome Headless DIFF
signup 90.0009999275 70.003000021 22.22%
login 100.000999928 80.003000021 20.00%
basket_a 120.000999928 120.003000021 0.00%
order_d 160.001999855 150.003999949 6.25%
search 180.003000021 180.003999949 0.00%
filter_ps 180.003000021 190.003999949 -5.56%
order 290.004999876 280.006000042 3.45%
view 310.004999876 290.006000042 6.45%
address 330.006000042 310.006000042 6.06%
guest_checkout 380.006000042 350.006999969 7.89%
order_3d 380.006000042 360.006999969 5.26%
basket 390.006999969 370.006999969 5.13%
filter 410.006999969 430.007999897 -4.88%
order_credit 450.007999897 450.009000063 0.00%
member 520.009000063 500.013999939 3.85%
TOTAL 530.009000063 510.015000105 3.77%

You can see the result of the test in the table below. In total, we have got 3.77% gain in time with running the chromedriver in headless mode. What about the required resources for running the test. The following images show that the performance indicators of the machine that the tests run. Check the memory usage, processor usage, and network usage over time.

Chromedriver in Headless Mode Windows Perfmon 
Chromedriver Windows Perfmon 

To sum up, the headless miracle is not true for chromedriver since getting 3,77% gain in time and almost the same resource requiring is not that good enough reason for calling it a miracle. If you say You are using Phantomjs, so it is better than Chromedriver, read this post. I am planning to test this for my case, when I do it I will add the result of the test.


Popular posts from this blog

Testing WEB Sevices and Automating SOAP Services

Performance Testing on CI: Locust is running on Jenkins

Change Default Timeout and Wait Time of Capybara

Performance Testing on CI: Integration of Locust and Jenkins

Create an Alias for Interactive Console Work: Selenium and Capybara