Skip to main content

Posts

What Reminds From Testistanbul 2016

First of all, I am appreciated to attend to an international conference on software testing. It is 7th international Testistanbul conference. There was good number of attendants from newly starters to over 30 years of experienced professionals. Most of the attendants were from Turkey thought there were some guys from abroad and some international companies to present their products. Testistanbul conference is important because it is definitely the most valuable conference on software testing in Turkey. It gives us opportunity to meet largest professional group and to share the knowledges in domestic market. The conference topics were as follows:  09:00 - 09:30 OPENING CEREMONY SPEECH: FORMULA 1, CONTINUOUS INTEGRATION, CONTINUOUS DELIVERY AND TEST DATA MANAGEMENT PROCESSES - TURKEY SOFTWARE QUALITY REPORT (TSQR) 2016 / 17 (In Turkish) Koray Yitmen 09:30 - 10:15 IBM SPONSOR SPEECH: SHIFT LEFT FOR HIGHER QUALITY AT GREATER SPEED Mehmet

Confusion of Using Selenium for Performance Testing

Time to time, I see some questions about integration test automation script to performance testing tool on Stackoverflow . There is a confusion about performance testing and test automation for whom newly started performance testing after doing some test automation. It is demanded that they have some automation code and want to use these scripts for performance testing to simulate user behaviour. Theoretically it seems a very good idea that tons of users are performing some cases and you are capturing the response time and finding bottle neck in terms of client side. However this is not applicable in practice because there are lots of obstacle to handle. The followings outline the obstacles and misunderstandings you will face, for the idea. I can open some browsers and simulate real user behaviour Automation scripts run over a browser, for web project. Most of the application are becoming as web app. For GUI based application what the hack test automation will have a role on pe

Mobile Application Testing Change Host to Redirect Test Environment

When you need to test the server side development via Mobile application in isolation, you can redirect the host of the API over the hosts file in the emulator. By this way, with the same application that users have, you can test the new development before passing it to live servers. Android Application I have using GenyMotion for emulating android devices. You can edit the hosts file in GenyMotion by the following commands, at the third step you need to change `hosts` file then push it emulator. adb root adb pull /etc/hosts hosts adb remount adb push hosts /system/etc See the example below, the host of the application is www.morhipo.com which is now hitting the test server. The third step is important, if you don't remount new host file can not be push to device . ~ adb root adbd is already running as root ~ adb pull /etc/hosts hosts ~ cat hosts ~ echo "10.1.6.37 www.morhipo.com" > hosts ~ cat hosts 10.1.6.37 www.morhipo.com ~ adb remount remount

Performance Testing on CI: Integration of Locust and Jenkins

It is a raising question around performance testing people that how to integrate, super hero tool, Locust to Jenkins as a step to continues integration (CI) pipeline. There have been some solutions for this but they are not simple, plus you need to write own log writer to handle this problem.  Actually there was a commit to save the result of the test but it has not been merged to Locust. With these commit we can integrate Locust to Jenkins. Update Locust This  update  is still open so you can not see it in latest Locust version yet. However you can  use this by updating your local repository, or just change two files  main.py  and  stats.py  with the files in your local. For Window change the files in the following directory:  C:\Python27\Lib\site-packages\locustio-0.7.3-py2.7.egga\locust\stats.py and for Unix-base OS change the files in the following directory:  /Library/Python/2.7/site-packages/locust/stats.py.  Alternatively, you can clone this commit from github and instal

Why Performance Test on Cloud

When a software comes to production the weakest points are mostly related to performance of the application. The reason is that most of the organization test the application functionally by testers and on acceptance testing period again the applications is under testing to check functional requirements. The missing point is non-functional testing like performance of the application.  Another failure point is testing the performance as the last level of development process. This means that any misleading failures in the service level can cause a very big problem on production stage so it requires extra source if possible and extra budget. As the terminology says "finding the big issue earlier stage of the development can reduce the fixing effort exponentially " . Therefore test the performance of the software output whenever it is ready, this could be an api end point, a function servers data to system, a web servers, and etc.  Another failure is related to technolo

Create an Alias for Interactive Console Work: Selenium and Capybara

If you are working on shell most of the time Aliases are very helpfull and time saving. For testing purposes you can use Alias for getting ready your test suites. In this post, I want to explain both running Selenium and Capybara on console and creating aliases for each.  This post is for Windows machines, if you are using Unix-like see   this post . Creating Scripts for Selenium and Capybara First of all, it is assumed that you have installed Selenium and Capybara correctly and they work on your machines. If you haven't installed, you can see my previous posts. I am using the Selenium with Python and the Capybara with Ruby. You can use several different language for Selenium but Capybara works only with Ruby.  Create scripts in a directory called scripts (in your home folder, like as  ~/scripts ) for your automation tool as following, save them as capybara.rb, sel.py :  Creating Aliases Depends on your favourite shell, you need to add the alias to .bashrc bash

Performance Testing on CI: Locust is running on Jenkins

For a successful Continuous Integration pipeline, there should be jobs for testing the performance of the application. It is necessary if the application is still performing well. Generally performance testing is thought as kinds of activities performed one step before going to live. In general approach it is true but don't forget to test your application's performance as soon as there is an testable software, such as an api end point, functions, and etc. For CI it is a good approach to testing performance after functional testing and just before the deployment of next stage. In this post, I want to share some info about Jenkins and Locust. In my previous post you can find some information about Locust and Jenkins. Jenkins operates the CI environment and Locust is a tool for performance testing. To run the Locust on Jenkins you need command line arguments which control the number of clients ,   hatch rate,  running locust without web interface and there should be so

Continous Integration Definition of DONE

Continuous Integration is easy if you are ready for the definition of DONE (DoD). It is not only accepting a story as successfull after developped by a developer but it also includes a process which ends after deploying to live and ensuring that it works and does not cause a defect in the live system. Handling DONE actually means of CI. With Scrum there is always one shippable product at the end of each sprint. This shippable product has to include some new features. Definition of Done helps to Scrum to clear the definition of every stages of development, such providing checklist for discussion, estimationa and design, so it makes the user stories more understandable and applicable.  What Definition of Done includes: Identifying the list of acceptance criteria Development Unit testing Funtional testing Regression testing Acceptance testing Performance testing Deployment Documentation Tracking  DONE Every stage can also have sub-stages, DoD should be prepea

Test Case Desing Approach: imperative vs declarative

Test case design is the most important part of test automation since it is the starting point of the automation code. Writing effective and efficient test case can make your code more stable and more successful. For BDD approach, the feature files hold the test cases or scenarios and these scenarios are derived by depending on the acceptance criterias written in the stories. Therefore the feature files should cover all the acceptance criteria plus some more test cases produced from exploratory testing.  The approach for the writing your scenarios can be imperative or declarative or mix of both. In this post want to explain my understanding from the design approaches and their prons and cons.  For dictionary meaning of imperative is giving authoritative command,  and declarative is taking the form of a simple statement according to the Oxford dictionary . My understanding from this explanation is that you can say "do this ..." as an imperative way or you can say

Jenkins: How to Sync Local Repository with GitLab / Trigger from GitLab

If you want to update your code in the local repository from git (gitlab, github, gitorious), you need to run git pull  into your repository. If you want to do this by Jenkins, you can create a job that can perform this action. There may be easier way to solve it but you can follow the instruction below: Create a Job to Pull the repo Jenkins > New Item > Freestyle Project > "OK" Build > Build Steps > Execute Windows Batch Command> Enter your command and save it Install GitLab Hook Plugin Jenkins > Manage Jenkins > Manage Plugins > Find Correct Plugin > Install it Create Deploy Keys on GitLab GitLab > Your Project Page > Settings > Deploy Keys > Click 'New Deploy Key' Be sure that you are changing your project settings! Create Deploy Keys and Services on GitLab GitLab > Your Project Page > Settings > Services > Click 'GitLab CI' Enter token, run the fo