Skip to main content

Posts

Create an Alias for Interactive Console Work: Selenium and Capybara

If you are working on shell most of the time Aliases are very helpfull and time saving. For testing purposes you can use Alias for getting ready your test suites. In this post, I want to explain both running Selenium and Capybara on console and creating aliases for each.  This post is for Windows machines, if you are using Unix-like see   this post . Creating Scripts for Selenium and Capybara First of all, it is assumed that you have installed Selenium and Capybara correctly and they work on your machines. If you haven't installed, you can see my previous posts. I am using the Selenium with Python and the Capybara with Ruby. You can use several different language for Selenium but Capybara works only with Ruby.  Create scripts in a directory called scripts (in your home folder, like as  ~/scripts ) for your automation tool as following, save them as capybara.rb, sel.py :  Creating Aliases Depends on your favourite shell, you need to add the alias to .bashrc bash

Performance Testing on CI: Locust is running on Jenkins

For a successful Continuous Integration pipeline, there should be jobs for testing the performance of the application. It is necessary if the application is still performing well. Generally performance testing is thought as kinds of activities performed one step before going to live. In general approach it is true but don't forget to test your application's performance as soon as there is an testable software, such as an api end point, functions, and etc. For CI it is a good approach to testing performance after functional testing and just before the deployment of next stage. In this post, I want to share some info about Jenkins and Locust. In my previous post you can find some information about Locust and Jenkins. Jenkins operates the CI environment and Locust is a tool for performance testing. To run the Locust on Jenkins you need command line arguments which control the number of clients ,   hatch rate,  running locust without web interface and there should be so

Continous Integration Definition of DONE

Continuous Integration is easy if you are ready for the definition of DONE (DoD). It is not only accepting a story as successfull after developped by a developer but it also includes a process which ends after deploying to live and ensuring that it works and does not cause a defect in the live system. Handling DONE actually means of CI. With Scrum there is always one shippable product at the end of each sprint. This shippable product has to include some new features. Definition of Done helps to Scrum to clear the definition of every stages of development, such providing checklist for discussion, estimationa and design, so it makes the user stories more understandable and applicable.  What Definition of Done includes: Identifying the list of acceptance criteria Development Unit testing Funtional testing Regression testing Acceptance testing Performance testing Deployment Documentation Tracking  DONE Every stage can also have sub-stages, DoD should be prepea

Test Case Desing Approach: imperative vs declarative

Test case design is the most important part of test automation since it is the starting point of the automation code. Writing effective and efficient test case can make your code more stable and more successful. For BDD approach, the feature files hold the test cases or scenarios and these scenarios are derived by depending on the acceptance criterias written in the stories. Therefore the feature files should cover all the acceptance criteria plus some more test cases produced from exploratory testing.  The approach for the writing your scenarios can be imperative or declarative or mix of both. In this post want to explain my understanding from the design approaches and their prons and cons.  For dictionary meaning of imperative is giving authoritative command,  and declarative is taking the form of a simple statement according to the Oxford dictionary . My understanding from this explanation is that you can say "do this ..." as an imperative way or you can say

Jenkins: How to Sync Local Repository with GitLab / Trigger from GitLab

If you want to update your code in the local repository from git (gitlab, github, gitorious), you need to run git pull  into your repository. If you want to do this by Jenkins, you can create a job that can perform this action. There may be easier way to solve it but you can follow the instruction below: Create a Job to Pull the repo Jenkins > New Item > Freestyle Project > "OK" Build > Build Steps > Execute Windows Batch Command> Enter your command and save it Install GitLab Hook Plugin Jenkins > Manage Jenkins > Manage Plugins > Find Correct Plugin > Install it Create Deploy Keys on GitLab GitLab > Your Project Page > Settings > Deploy Keys > Click 'New Deploy Key' Be sure that you are changing your project settings! Create Deploy Keys and Services on GitLab GitLab > Your Project Page > Settings > Services > Click 'GitLab CI' Enter token, run the fo

Configuration of Jenkins on Windows Server for Capybara and Calabash Projects

I think the most challenging part of test automation is nowadays having Continuous Integration and one step more is having Continuous Deployment. These two terms have been a buzzwords around the software community, and everyone is talking about them but there are very few companies that can apply these. The reasons behind this can be listed as: Mis-understanding of agile and scrum Lack of "real" test engineers and Lack of Dev-ops culture  Lack of stable test environments  Lack of vision in the software industry, which saving the day is more critical than saving the future Out-of-budget, not investing on good people but investing in tools On the other hand, there are lots of open-source tools that you can just build and configure your own CI, and maybe next step goes to continuous deployment. In this post, I want to explain how you can set up CI on a Windows server. I will use the following tools/technology and language: Ruby, Gem (many Gems), Capybara, Cucumbe

Basics of Continuous Integration(CI)

Continuos Integration shortly as CI is a development practice which developers push their code frequently to the development environment and then automated build system build the code and then test it and gives feedback about the newly added code. By this way, codes are added to the main repository in a small piece which reduces the complexity and deltas between new code and the code in development environment. Developers also can reach the latest build code easily. Think that every developer pushes one time in a day, the delta is just a day but if they push one time for week then the delta is 5 times bigger, correspondingly complexity is also 5 times bigger. CI Pipeline One of the biggest problem for delivering a feature late is spending longer time for testing product and fixing defects. A classical approach tells us to test at the end of the development but it is usual that planned time is consumed during analysis and development periods. Therefore, after the develo

An Error for Selenium - Chrome vs Firefox: stale element reference

I faced an selenium error like " stale element reference " when I changed Capybara's driver to Chrome which was not occurred in webkit and firefox. This error occurs when referencing an element which is not in DOM. Chrome gives this but it doesn't appear when I use Firefox because the dynamic object is appear after the Javascript execution finishes which means that Chrome doesn't wait until javascript execution ends .  You can escape this error by checking your scenario if there is an element requiring in the DOM which is destructed by an action in the scenario or if you have loop and the DOM is updating for each iteration the object you look for changes. However in  my case, scenario is passing with FF but not Chrome which because the Chrome is passing the next step as soon as Javascript execution starts.  I just call waiting function and then assign the variable. Finally scenario is passing for both browsers.        stale element reference: element is

Small Customisation for Capybara and Calabash

One of the biggest challenge in test automation is updating existing codes. Because of the update in the development, it is very prone to have some changes for the attributes of ui elements and less likely to have some change in behaviour in actions. The second option is most of the time called as bug, otherwise if the business is changed, it could be called as a new feature. For any reasons that cause the automation code gives failures you need to update something. Sometimes I need to refresh my mind :) For easy maintenance, one of the good approach keep your codes modular. It means that each module should responsible for one job and a update problem should be solved by applying just one change in your system. Using Capybara plus Cucumber gives you this features but if you are using them professionally. For this framework, business rules are kept on Cucumber so you can update business rules from your .feature files but related update should reflect to the code. In thi

Testing WEB Sevices and Automating SOAP Services

Web Services are server and client applications to interact with rest of the world over HTTP, or sometimes over SMTP. It is written for accepting requests and giving appropriate answers for each requests. It is assumed that it returns the same response if there is no changing in the request and the condition. Therefore testing and automating Web Services are very easy. You just need to know sending and receiving data and condition, then the rest of the jobs is playing with data and assertions. Information about the  Web Services  should be present in  Web Services  documentations and it should be understandable to the people from third party system.  Web applications take to the next step with the  Web Services  because you can open your system to rest of world without requiring language specific restrictions. This makes integrity of applications very easy. To be able to integrate two or more system, you just know that you are sending some data and receiving data.  I had a t

Page-Object Pattern for Selenium Test Automation with Python

Page-object model is a pattern that you can apply it to develop efficient automation framework. With the page-model, it is possible to minimize maintenance cost. Basically page-object means that your every page is inherited from a base class which includes basic functionalities for every page. If you have some new functionalities that every page should have, you can simple add it to the base class. Base class is like the following: In this part we are creating pages which are inherited from base page. Every page has its own functionalities written as python functions. Some functions return to a new page, it means that these functions leave the current page and produce a new page. You should write as much as functions you need in the assertion part because this is the only part you can use the webdriver functions to interact with web pages . This part can be evaluate as providing data to assertion part.   The last part is related to asserting your test cases against to the

Mobile Test Automation: Appium with Python Client

Appium is an open source tool for mobile test automation, you can automate functional test cases for Android and iOS application. As in my some previous posts, I explained Calabash and I just used Ruby for writing test cases because Calabash only supports Ruby and as I know they have plan to support Java. However Appium doesn't have this language constriction. Since it uses Webdriver, it supports the languages which are supported by Webdriver. It means that Java, Objective-C, PHP, Python, C#, Clojure, Perl, JavaScript with Node.js and plus Ruby are supported. You just install the language client you want to test and then write your test cases. Check for the client for your favourite languages from Appium page . Install Appium: npm install -g appium Install Appium Client: npm install wd Install Python Appium Client: pip install Appium-Python-Client Ensure that you have Python installed, recommend to have 2.7 or above: python --version Ensure that you have

Effective Testing: Using LOGCAT for Android Testing

If you tend to use tools effectively, testing is easy and enjoyable. For mobile testing there are lots of challenges and to be honest if you don't push yourself to improve your ability as every developer does you can not test the mobile application good enough. Instead of testing mobile application manually, we can automate the stories whenever the stories are prepared by business analyst with writing cucumber files and then we can start writing the code of automation and finally we can run it when the stories are developed by developer for agile development environment. This is iteration based approach, means you can repeat this steps for each iteration. At the end of the milestone you can run your whole automation code and see the result. For the remaining time you can still test application by using Exploratory Testing techniques to find-out any bugs.  The important point is to make your job effective and efficient. For this you should automate or scripted some repetiti

Fix for Calabash-android Run INSTRUMENTATION_FAILED exception

When you set-up a new android devices or update your .apk file, you may possible get INSTRUMENTATION_FAILED exception: android.util.AndroidException: INSTRUMENTATION_FAILED: com.package.name/sh.calaba.instrumentationbackend.CalabashInstrumentationTestRunner at com.android.commands.am.Am.runInstrument(Am.java:1093) at com.android.commands.am.Am.onRun(Am.java:371) at com.android.internal.os.BaseCommand.run(BaseCommand.java:47) at com.android.commands.am.Am.main(Am.java:100) at com.android.internal.os.RuntimeInit.nativeFinishInit(Native Method) at com.android.internal.os.RuntimeInit.main(RuntimeInit.java:251) To fix this issue you can follow the instruction below: Create new folder Copy your .apk file into the newly created folder go to your newly created folder and run this command to create calabash-android project calabash-android gen run calabash-android via your new devices calabash-android run latest.apk features ADB_DEVICE_ARG=192.168.56.101:5555 resign .apk

Tools for Screenshot Comparison Testing: Huxley, Wraith, Selenium-Wraith

Testing the user interface (UI) testing may be the most difficult part of test automation. When we think from the perspective of users there are not hundreds but there may be thousands of different users' environment but it is clear that simulating and testing UI related cases for all possible users' environment is impossible! However we can cover easily the %80 of them by using statistical data taken from analytics tools as Pareto Analysis says 20% of reasons cause the %80 of all problem. If you are doing some kind of internet business, there are two important factors for user environments which are operating system and browsers for normal internet users.  Therefore analytic tools should give us feedback about operating system and versions of operating system; browsers and versions of browsers. To get a general trend we can look at the w3shcool.com statistic but sure it is more convenient to take your own statistic depends on your user profile. In general approach