Skip to main content

Posts

How to Add Screenshot to Jenkins Cucumber Reports for Capybara

If you are using Capybara and Cucumber for test automation and Jenkins for CI/CD process you can run your test over Jenkins and see the result by Cucumber Result Plugin , for more information check the plugin page . By this plugin you can also see the screenshots taken when test cases fail.  To produce the result you need to save the result of the test as .json file, you can use the following command:      cucumber features -f json -o cucumber_result.json To enable to take screenshot if a case fails you need to set the application hook  of after case. The following configuration makes it automatically. See it and change screenshot path with your path. The key point is embedding the image after taking it then reports plugin make it visible on the report. Cucumber Result Plugin Screenshot for the failed test case

Appium vs Calabash

If you want to test your mobile application and looking for an open source testing tools then most probably you will face Calabash and Appium . These are the top two most popular tools for mobile automation. At first glance, they seem to be learnt easily but in detail you would have to deal with many challenges and must spend little bit longer time than you expected. Since these tools are open-sourced and still not mature level of quality there would be some problems when you want to use them for your special needs.  In this post I want to write down my experience about Calabash and Appium. This post may help you to choose which one is suitable for your needs. I used following versions: Calabash-Android: 0.7.3 Calabash-iOS: 0.19.1 Appium: 1.5.2 Appium Support Many Languages But Calabash is RUBY! In the background Appium uses Selenium which was created 14 year ago by Thougthwork, check thoughts about Selenium  for mobile testing by Thougthwork. Since Selenium supports Ja

QA in Production

For a while, I have been thinking about the responsibilities of QA Engineers by broadening them to the production environment. In this post, I want to write the reasoning behind the needs of QA Engineers in production. Integration Testing: Tools and Technology Every day we are facing new integration points with third-party tools or technologies for a win-win strategy. Most of the time these technologies are in the term of the trial period or just a cheap alternative to the present one. These technologies or tools simply don't have adequate documentation so it makes the development and testing of integration riskier. This means that the development, test, uat or preprod test environment is not enough for catching all possible bugs, as a result, the production environment is the final destination for checking logs and monitoring the integration results. Validation of Non-Functional Requirements There are also unclear points behind the non-functional requirements like performance a

Change Default Timeout and Wait Time of Capybara

One of the biggest challenge for automation is handling timeout problem. Most of the time, timeout is 60 seconds but it may sometimes not enough if you have badly designed asynchronous calls or the third party ajax calls. This makes handling timeout more complex. set large enough to tolerate network related problems. For Selenium based automation frameworks, like Capybara, default Webdriver timeout is set to Net::ReadTimeout (Net::ReadTimeout) Changing ReadTimeout If you have timeout problem for Capybara, it gives an error like above. This means that the page is not fully loaded in given timeout period. Even you can see that page is loaded correctly but webdriver wait until the Ajax calls finish. class BufferedIO #:nodoc: internal use only def initialize (io) @io = io @read_timeout = 60 @continue_timeout = nil @debug_output = nil @rbuf = '' end . . . . . def rbuf_fill beg

What Reminds From Testistanbul 2016

First of all, I am appreciated to attend to an international conference on software testing. It is 7th international Testistanbul conference. There was good number of attendants from newly starters to over 30 years of experienced professionals. Most of the attendants were from Turkey thought there were some guys from abroad and some international companies to present their products. Testistanbul conference is important because it is definitely the most valuable conference on software testing in Turkey. It gives us opportunity to meet largest professional group and to share the knowledges in domestic market. The conference topics were as follows:  09:00 - 09:30 OPENING CEREMONY SPEECH: FORMULA 1, CONTINUOUS INTEGRATION, CONTINUOUS DELIVERY AND TEST DATA MANAGEMENT PROCESSES - TURKEY SOFTWARE QUALITY REPORT (TSQR) 2016 / 17 (In Turkish) Koray Yitmen 09:30 - 10:15 IBM SPONSOR SPEECH: SHIFT LEFT FOR HIGHER QUALITY AT GREATER SPEED Mehmet

Confusion of Using Selenium for Performance Testing

Time to time, I see some questions about integration test automation script to performance testing tool on Stackoverflow . There is a confusion about performance testing and test automation for whom newly started performance testing after doing some test automation. It is demanded that they have some automation code and want to use these scripts for performance testing to simulate user behaviour. Theoretically it seems a very good idea that tons of users are performing some cases and you are capturing the response time and finding bottle neck in terms of client side. However this is not applicable in practice because there are lots of obstacle to handle. The followings outline the obstacles and misunderstandings you will face, for the idea. I can open some browsers and simulate real user behaviour Automation scripts run over a browser, for web project. Most of the application are becoming as web app. For GUI based application what the hack test automation will have a role on pe

Mobile Application Testing Change Host to Redirect Test Environment

When you need to test the server side development via Mobile application in isolation, you can redirect the host of the API over the hosts file in the emulator. By this way, with the same application that users have, you can test the new development before passing it to live servers. Android Application I have using GenyMotion for emulating android devices. You can edit the hosts file in GenyMotion by the following commands, at the third step you need to change `hosts` file then push it emulator. adb root adb pull /etc/hosts hosts adb remount adb push hosts /system/etc See the example below, the host of the application is www.morhipo.com which is now hitting the test server. The third step is important, if you don't remount new host file can not be push to device . ~ adb root adbd is already running as root ~ adb pull /etc/hosts hosts ~ cat hosts ~ echo "10.1.6.37 www.morhipo.com" > hosts ~ cat hosts 10.1.6.37 www.morhipo.com ~ adb remount remount

Performance Testing on CI: Integration of Locust and Jenkins

It is a raising question around performance testing people that how to integrate, super hero tool, Locust to Jenkins as a step to continues integration (CI) pipeline. There have been some solutions for this but they are not simple, plus you need to write own log writer to handle this problem.  Actually there was a commit to save the result of the test but it has not been merged to Locust. With these commit we can integrate Locust to Jenkins. Update Locust This  update  is still open so you can not see it in latest Locust version yet. However you can  use this by updating your local repository, or just change two files  main.py  and  stats.py  with the files in your local. For Window change the files in the following directory:  C:\Python27\Lib\site-packages\locustio-0.7.3-py2.7.egga\locust\stats.py and for Unix-base OS change the files in the following directory:  /Library/Python/2.7/site-packages/locust/stats.py.  Alternatively, you can clone this commit from github and instal

Why Performance Test on Cloud

When a software comes to production the weakest points are mostly related to performance of the application. The reason is that most of the organization test the application functionally by testers and on acceptance testing period again the applications is under testing to check functional requirements. The missing point is non-functional testing like performance of the application.  Another failure point is testing the performance as the last level of development process. This means that any misleading failures in the service level can cause a very big problem on production stage so it requires extra source if possible and extra budget. As the terminology says "finding the big issue earlier stage of the development can reduce the fixing effort exponentially " . Therefore test the performance of the software output whenever it is ready, this could be an api end point, a function servers data to system, a web servers, and etc.  Another failure is related to technolo