Skip to main content

Posts

Test Case Desing Approach: imperative vs declarative

Test case design is the most important part of test automation since it is the starting point of the automation code. Writing effective and efficient test case can make your code more stable and more successful. For BDD approach, the feature files hold the test cases or scenarios and these scenarios are derived by depending on the acceptance criterias written in the stories. Therefore the feature files should cover all the acceptance criteria plus some more test cases produced from exploratory testing.  The approach for the writing your scenarios can be imperative or declarative or mix of both. In this post want to explain my understanding from the design approaches and their prons and cons.  For dictionary meaning of imperative is giving authoritative command,  and declarative is taking the form of a simple statement according to the Oxford dictionary . My understanding from this explanation is that you can say "do this ..." as an imperative way or you can say

Jenkins: How to Sync Local Repository with GitLab / Trigger from GitLab

If you want to update your code in the local repository from git (gitlab, github, gitorious), you need to run git pull  into your repository. If you want to do this by Jenkins, you can create a job that can perform this action. There may be easier way to solve it but you can follow the instruction below: Create a Job to Pull the repo Jenkins > New Item > Freestyle Project > "OK" Build > Build Steps > Execute Windows Batch Command> Enter your command and save it Install GitLab Hook Plugin Jenkins > Manage Jenkins > Manage Plugins > Find Correct Plugin > Install it Create Deploy Keys on GitLab GitLab > Your Project Page > Settings > Deploy Keys > Click 'New Deploy Key' Be sure that you are changing your project settings! Create Deploy Keys and Services on GitLab GitLab > Your Project Page > Settings > Services > Click 'GitLab CI' Enter token, run the fo

Configuration of Jenkins on Windows Server for Capybara and Calabash Projects

I think the most challenging part of test automation is nowadays having Continuous Integration and one step more is having Continuous Deployment. These two terms have been a buzzwords around the software community, and everyone is talking about them but there are very few companies that can apply these. The reasons behind this can be listed as: Mis-understanding of agile and scrum Lack of "real" test engineers and Lack of Dev-ops culture  Lack of stable test environments  Lack of vision in the software industry, which saving the day is more critical than saving the future Out-of-budget, not investing on good people but investing in tools On the other hand, there are lots of open-source tools that you can just build and configure your own CI, and maybe next step goes to continuous deployment. In this post, I want to explain how you can set up CI on a Windows server. I will use the following tools/technology and language: Ruby, Gem (many Gems), Capybara, Cucumbe

Basics of Continuous Integration(CI)

Continuos Integration shortly as CI is a development practice which developers push their code frequently to the development environment and then automated build system build the code and then test it and gives feedback about the newly added code. By this way, codes are added to the main repository in a small piece which reduces the complexity and deltas between new code and the code in development environment. Developers also can reach the latest build code easily. Think that every developer pushes one time in a day, the delta is just a day but if they push one time for week then the delta is 5 times bigger, correspondingly complexity is also 5 times bigger. CI Pipeline One of the biggest problem for delivering a feature late is spending longer time for testing product and fixing defects. A classical approach tells us to test at the end of the development but it is usual that planned time is consumed during analysis and development periods. Therefore, after the develo

An Error for Selenium - Chrome vs Firefox: stale element reference

I faced an selenium error like " stale element reference " when I changed Capybara's driver to Chrome which was not occurred in webkit and firefox. This error occurs when referencing an element which is not in DOM. Chrome gives this but it doesn't appear when I use Firefox because the dynamic object is appear after the Javascript execution finishes which means that Chrome doesn't wait until javascript execution ends .  You can escape this error by checking your scenario if there is an element requiring in the DOM which is destructed by an action in the scenario or if you have loop and the DOM is updating for each iteration the object you look for changes. However in  my case, scenario is passing with FF but not Chrome which because the Chrome is passing the next step as soon as Javascript execution starts.  I just call waiting function and then assign the variable. Finally scenario is passing for both browsers.        stale element reference: element is

Small Customisation for Capybara and Calabash

One of the biggest challenge in test automation is updating existing codes. Because of the update in the development, it is very prone to have some changes for the attributes of ui elements and less likely to have some change in behaviour in actions. The second option is most of the time called as bug, otherwise if the business is changed, it could be called as a new feature. For any reasons that cause the automation code gives failures you need to update something. Sometimes I need to refresh my mind :) For easy maintenance, one of the good approach keep your codes modular. It means that each module should responsible for one job and a update problem should be solved by applying just one change in your system. Using Capybara plus Cucumber gives you this features but if you are using them professionally. For this framework, business rules are kept on Cucumber so you can update business rules from your .feature files but related update should reflect to the code. In thi

Testing WEB Sevices and Automating SOAP Services

Web Services are server and client applications to interact with rest of the world over HTTP, or sometimes over SMTP. It is written for accepting requests and giving appropriate answers for each requests. It is assumed that it returns the same response if there is no changing in the request and the condition. Therefore testing and automating Web Services are very easy. You just need to know sending and receiving data and condition, then the rest of the jobs is playing with data and assertions. Information about the  Web Services  should be present in  Web Services  documentations and it should be understandable to the people from third party system.  Web applications take to the next step with the  Web Services  because you can open your system to rest of world without requiring language specific restrictions. This makes integrity of applications very easy. To be able to integrate two or more system, you just know that you are sending some data and receiving data.  I had a t

Page-Object Pattern for Selenium Test Automation with Python

Page-object model is a pattern that you can apply it to develop efficient automation framework. With the page-model, it is possible to minimize maintenance cost. Basically page-object means that your every page is inherited from a base class which includes basic functionalities for every page. If you have some new functionalities that every page should have, you can simple add it to the base class. Base class is like the following: In this part we are creating pages which are inherited from base page. Every page has its own functionalities written as python functions. Some functions return to a new page, it means that these functions leave the current page and produce a new page. You should write as much as functions you need in the assertion part because this is the only part you can use the webdriver functions to interact with web pages . This part can be evaluate as providing data to assertion part.   The last part is related to asserting your test cases against to the

Mobile Test Automation: Appium with Python Client

Appium is an open source tool for mobile test automation, you can automate functional test cases for Android and iOS application. As in my some previous posts, I explained Calabash and I just used Ruby for writing test cases because Calabash only supports Ruby and as I know they have plan to support Java. However Appium doesn't have this language constriction. Since it uses Webdriver, it supports the languages which are supported by Webdriver. It means that Java, Objective-C, PHP, Python, C#, Clojure, Perl, JavaScript with Node.js and plus Ruby are supported. You just install the language client you want to test and then write your test cases. Check for the client for your favourite languages from Appium page . Install Appium: npm install -g appium Install Appium Client: npm install wd Install Python Appium Client: pip install Appium-Python-Client Ensure that you have Python installed, recommend to have 2.7 or above: python --version Ensure that you have

Effective Testing: Using LOGCAT for Android Testing

If you tend to use tools effectively, testing is easy and enjoyable. For mobile testing there are lots of challenges and to be honest if you don't push yourself to improve your ability as every developer does you can not test the mobile application good enough. Instead of testing mobile application manually, we can automate the stories whenever the stories are prepared by business analyst with writing cucumber files and then we can start writing the code of automation and finally we can run it when the stories are developed by developer for agile development environment. This is iteration based approach, means you can repeat this steps for each iteration. At the end of the milestone you can run your whole automation code and see the result. For the remaining time you can still test application by using Exploratory Testing techniques to find-out any bugs.  The important point is to make your job effective and efficient. For this you should automate or scripted some repetiti

Fix for Calabash-android Run INSTRUMENTATION_FAILED exception

When you set-up a new android devices or update your .apk file, you may possible get INSTRUMENTATION_FAILED exception: android.util.AndroidException: INSTRUMENTATION_FAILED: com.package.name/sh.calaba.instrumentationbackend.CalabashInstrumentationTestRunner at com.android.commands.am.Am.runInstrument(Am.java:1093) at com.android.commands.am.Am.onRun(Am.java:371) at com.android.internal.os.BaseCommand.run(BaseCommand.java:47) at com.android.commands.am.Am.main(Am.java:100) at com.android.internal.os.RuntimeInit.nativeFinishInit(Native Method) at com.android.internal.os.RuntimeInit.main(RuntimeInit.java:251) To fix this issue you can follow the instruction below: Create new folder Copy your .apk file into the newly created folder go to your newly created folder and run this command to create calabash-android project calabash-android gen run calabash-android via your new devices calabash-android run latest.apk features ADB_DEVICE_ARG=192.168.56.101:5555 resign .apk

Tools for Screenshot Comparison Testing: Huxley, Wraith, Selenium-Wraith

Testing the user interface (UI) testing may be the most difficult part of test automation. When we think from the perspective of users there are not hundreds but there may be thousands of different users' environment but it is clear that simulating and testing UI related cases for all possible users' environment is impossible! However we can cover easily the %80 of them by using statistical data taken from analytics tools as Pareto Analysis says 20% of reasons cause the %80 of all problem. If you are doing some kind of internet business, there are two important factors for user environments which are operating system and browsers for normal internet users.  Therefore analytic tools should give us feedback about operating system and versions of operating system; browsers and versions of browsers. To get a general trend we can look at the w3shcool.com statistic but sure it is more convenient to take your own statistic depends on your user profile. In general approach

Running Capybara Under Chrome, Poltergeist or Firefox

As a default Capybara runs under Firefox but if you want to change your driver you have to change your javascript driver to your new driver. But you also must define your driver as registered Capybara driver. For example if you want to run with Chrome and change your driver to chrome but not define it, it gives some errors like following irb(main):007:0> visit "http://www.amazon.com" Capybara::DriverNotFoundError: no driver called :chrome was found, available drivers: :rack_test, :selenium, :poltergeist from /Library/Ruby/Gems/2.0.0/gems/capybara-2.4.1/lib/capybara/session.rb:77:in `driver' from /Library/Ruby/Gems/2.0.0/gems/capybara-2.4.1/lib/capybara/session.rb:226:in `visit' from /Library/Ruby/Gems/2.0.0/gems/capybara-2.4.1/lib/capybara/dsl.rb:51:in `block (2 levels) in ' if you add the following line into your env.rb module, you will successfully run under Chrome driver. Capybara.register_driver :chrome do |app| Capybara::Selenium::Driver

Effective Use of Cucumber with Calabash

BDD, which is stand for behaviour driven development, is created by Dan North . BDD is developed on the top of TDD, which is stand for test driven development to remove the gap of unit testing and acceptance testing. In TDD, every unit is started to be written right after defining the expectations from the units, by the unit testing. However by BDD, regarding the agile story, every unit is specified and tested in terms of desired behaviour. This 'desired behaviour' is what exactly business requires in agile stories.  BDD can be applied by the help of some tools to interpret the business 'desired behaviour' to test code. There are some tools that help this aim such as Behat  for PHP, JBhave  for Java world, and the most famous one Cucumber  for the Ruby world. In this post I want to give some points on how to effectively we can use Cucumber for a Calabash project to automate some agile stories for sample android application. Cucumber takes some keywords lik

Android Test Automation with Cucumber and Calabash

In my previous post I explained the Cucumber Capybara and Calabash and behaviour driven development. In this post I want to explain how we can implement and run cucumber and calabash for automating test cases for android applications. If you have not set-up your environment yet, you can read this post . You should also have an android virtual devices or a real devices that is connected to your computer in debugging mode. Your android version should be higher than 2.2.  If your system is ready for testing, you can clone the repository that I created for mobile test automation purposes. Sure, it may be more sophisticated if a real android developer develop something but from this application you can have a fresh starting. git clone https://github.com/gunesmes/calabash_android_automation.git You can check the images on the github so that get information what the application is looking like. Just open the project file and  look at the two important files in feature directory: cala