Posts

Showing posts from 2014

Fix for Calabash-android Run INSTRUMENTATION_FAILED exception

When you set-up a new android devices or update your .apk file, you may possible get INSTRUMENTATION_FAILED exception:


android.util.AndroidException: INSTRUMENTATION_FAILED: com.package.name/sh.calaba.instrumentationbackend.CalabashInstrumentationTestRunner at com.android.commands.am.Am.runInstrument(Am.java:1093) at com.android.commands.am.Am.onRun(Am.java:371) at com.android.internal.os.BaseCommand.run(BaseCommand.java:47) at com.android.commands.am.Am.main(Am.java:100) at com.android.internal.os.RuntimeInit.nativeFinishInit(Native Method) at com.android.internal.os.RuntimeInit.main(RuntimeInit.java:251)
To fix this issue you can follow the instruction below:

Create new folderCopy your .apk file into the newly created foldergo to your newly created folder and run this command to create calabash-android project calabash-android genrun calabash-android via your new devices calabash-android run latest.apk features ADB_DEVICE_ARG=192.168.56.101:5555resign .apk if you need calabash-a…

Tools for Screenshot Comparison Testing: Huxley, Wraith, Selenium-Wraith

Image
Testing the user interface (UI) testing may be the most difficult part of test automation. When we think from the perspective of users there are not hundreds but there may be thousands of different users' environment but it is clear that simulating and testing UI related cases for all possible users' environment is impossible! However we can cover easily the %80 of them by using statistical data taken from analytics tools as Pareto Analysis says 20% of reasons cause the %80 of all problem. If you are doing some kind of internet business, there are two important factors for user environments which are operating system and browsers for normal internet users.  Therefore analytic tools should give us feedback about operating system and versions of operating system; browsers and versions of browsers. To get a general trend we can look at the w3shcool.com statistic but sure it is more convenient to take your own statistic depends on your user profile.

In general approach  testing Chr…

Running Capybara Under Chrome, Poltergeist or Firefox

Image
As a default Capybara runs under Firefox but if you want to change your driver you have to change your javascript driver to your new driver. But you also must define your driver as registered Capybara driver. For example if you want to run with Chrome and change your driver to chrome but not define it, it gives some errors like following irb(main):007:0> visit "http://www.amazon.com" Capybara::DriverNotFoundError: no driver called :chrome was found, available drivers: :rack_test, :selenium, :poltergeist from /Library/Ruby/Gems/2.0.0/gems/capybara-2.4.1/lib/capybara/session.rb:77:in `driver' from /Library/Ruby/Gems/2.0.0/gems/capybara-2.4.1/lib/capybara/session.rb:226:in `visit' from /Library/Ruby/Gems/2.0.0/gems/capybara-2.4.1/lib/capybara/dsl.rb:51:in `block (2 levels) in 'if you add the following line into your env.rb module, you will successfully run under Chrome driver. Capybara.register_driver :chrome do |app| Capybara::Selenium::Driver.new(app,…

Effective Use of Cucumber with Calabash

Image
BDD, which is stand for behaviour driven development, is created by Dan North. BDD is developed on the top of TDD, which is stand for test driven development to remove the gap of unit testing and acceptance testing. In TDD, every unit is started to be written right after defining the expectations from the units, by the unit testing. However by BDD, regarding the agile story, every unit is specified and tested in terms of desired behaviour. This 'desired behaviour' is what exactly business requires in agile stories. 
BDD can be applied by the help of some tools to interpret the business 'desired behaviour' to test code. There are some tools that help this aim such as Behat for PHP, JBhave for Java world, and the most famous one Cucumber for the Ruby world. In this post I want to give some points on how to effectively we can use Cucumber for a Calabash project to automate some agile stories for sample android application.


Cucumber takes some keywords like Given, When, And,…

Android Test Automation with Cucumber and Calabash

Image
In my previous post I explained the Cucumber Capybara and Calabash and behaviour driven development. In this post I want to explain how we can implement and run cucumber and calabash for automating test cases for android applications. If you have not set-up your environment yet, you can read this post. You should also have an android virtual devices or a real devices that is connected to your computer in debugging mode. Your android version should be higher than 2.2.  If your system is ready for testing, you can clone the repository that I created for mobile test automation purposes. Sure, it may be more sophisticated if a real android developer develop something but from this application you can have a fresh starting.

git clone https://github.com/gunesmes/calabash_android_automation.git
You can check the images on the github so that get information what the application is looking like. Just open the project file and  look at the two important files in feature directory: calabash_step.r…

What Reminds from Continous Devilery and Design Conference by Thoughtworks

Image
The conference was hold on September 11, 2014 in İstanbul, Turkey. I want to briefly explain the importance of the conference and what was learnt from it. Thoughtworks, as a well-known consultancy firm in the world, has entered the Turkey's software market. This means that there will be something changes in software industry since there are lots of consultancy firm in Turkey and they have got one more competitor. However this is not an ordinary competitor, they may possibly bring lots of experiences and different kind of consulting culture from the other consultancy firms. 
What is the situation in Turkey?As I mention, there are lots of software / test consultancy firm but majority of them work as providing developers / tester to companies to work as outsourced. Therefore the guys generally fell alone when doing their jobs because there is generally no technical supports for them to improve their abilities. Also the major problem in Turkey is software development process. Even they…

Happy Tester's Day

Image
Be happy guys! We have a day for celebration. Yes, it is today as September 9 which is celebrating as tester's day. The origin of the story goes to the finding first bug in the system. Actually this bug is not a bug we find in the software, it was a real bug which was stuck and dead in the cables. And then it led some flaws on the computer Mark II Aiken Relay Calculator at Harvard University on September 9, 1945. You can see very famous and first bug report in the picture below. 




However, the first time the term "bug" was used Thomas Alva Edison for describing a trouble in a system in the letter written to Theodora Puskas in 1878. And later on, he used the term "bug" for explanation about a problem in his phonograph when he gave an interview for the Pall Mall Gazette in 1889 according to the the Yale Book of Quotations.


Mobile Test Automation: Calabash, Cucumber

Image
Mobile test automation is just another era for test automation since the trends for mobile devices are increasing. For more sophisticated testing approach, we should apply the technics what we have learnt from the test automation for desktop/web applications. As test automation is a part of continuos development, it should be run after each commit to test environment. For mobile application we can use the same method to be more responsible. 
Cucumber is used describe the test cases and it very easy to understand for anyone so we can use it for user behaviour as behaviour driven development, BDD. Capybara is a library to simulate user behaviour for just web application. However we can use Calabash for mobile test automation.  It means that we can use Calabash and Cucumber together for mobile test automations. As in Capybara, in Calabash there are not much functions to learn. It has basically, finding element, inserting text into textEdit fields, clicking and assertion functions. The imp…

How to Run Capybara with Poltergeist Mode by Terminal

Image
In Poltergeist mode or headless mode, it is different from Selenium since there is no browsers which you can track failure visually. The advantages of Poltergeist are: it is faster, it demands less source when we compare to Selenium, and eventually it is more suitable for Continues Integration (CI). Installation and further information about Poltergeist you can visit git page.

Running Capybara with terminal may help you to get quick result. By this way you can check if button, field, or any object can be reachable easily and this method may help you to reduce failure issue on your CI and get you more realistic results.

You can just paste the following commands into your terminal, it will go to home page of Amazon.



New Modern Open Source Performance Testing Tool : Locust

Image
If you are bored with using old fashioned Java GUI to test the performance of your applications, then you should try Locust. Jmeter, which is old fashioned as I mentioned, is very mature and used by a big community but you have create your test case with an ugly GUI as much features as allows you. For me the best part of Locust is that you can write your test case with Python which is my favourite language and it is very simple to learn. The installation is very simple, you just need to type:

> pip install locustio
Some feature / deficiency of Locust: 

You can write your test case in PythonYou can define the user time consumption on a page with min_wait and max_wait as milliseconds in locust class so that get a real user simulation. You can give weight for each test cases so you can simulate real user behaviours like during a specific period of time: 1 person signs up, 10 persons login, 60 persons visit pages, ... You can make http request as you do in your code such as get, post, put…

Capybara - Cucumber: Handling iframe, Pop-ups and Facebook Login

Image
For any automation frameworks, handling pop-ups is somehow difficult. By opening a pop-up window, the window changes so you can not control the new window with existing webdriver since it is already assigned to main windows when it was initiated. Normally automation framework can not work if you don't switch to pop-up windows. For Capybara you can easily handle it by assigning the webdriver to new pop-up window. Then you can perform your test case as same way what you have done and when you are done with the pop-up, close the pop-up and then switch back the webdriver to the main window. 
    main = page.driver.browser.window_handles.first
    popup = page.driver.browser.window_handles.last
    page.driver.browser.switch_to.window(popup)

Since iframe is a part of another source which is used to display another webpage in a webpage, you must handle it exactly same  as pop-up windows. 

page.driver.browser.switch_to.frame(iframe_name)

However, for iframe you must take the name of iframe to…

How to Check the CSS of an Object by Capybara with jQuery

Image
Testing the CSS of an object is sometimes needed. If you want to check the background-color, size, clip, ... of the objects, you must control the CSS code of the element. Capybara let you check it very easily with executing javascript from console by sending command to console of the browser. There are two function that can be used for sending command to console as follows:
page.evaluate_script() page.execute_script()
You can send your code as below. It tries to creates members object and then adds new profiles to members. With this way we can check if metrics of is shown as described format. # from javascript add 10 more user to the community and totally there should be 11 members # check the boundary, more button should be still not exit page.execute_script("window.members = App.Communities.models[0].attributes._members") for i in 1..number.to_i page.execute_script("members.add(new App.Models.Profile({_res:'profiles.#{i}'}))") end
In this example we used…

Deterministic Way of Test Automation

Image
Test automation scripts should be simple. This doesn't mean that software product code should be complex. Actually,when everything is simple then the life will be simpler. However sometimes we need to explain something in a complex way. Most of the time, simple is difficult. There may be different reasons behind the complexity but it is clear that if we knew it clearly, we would explain it simply. Let's look at what Albert Einstein says: 
"If you can't explain it simply, you don't understand it well enough."
As the subject is how simplicity can be applied to test automation, I will focus on simple but efficient test scripts. To enable to write simple test scripts, we must clarify undefined, non-clear, dependent factors on every case/stories. This kind of approach is called as determinism in philosophy. According to the Princeton University dictionary, determinism means as: 
"determinism ((philosophy) a philosophical theory holding that all events are inevit…

Capybara: How to Control from Terminal

Image
When you write test scripts, you may want to get quick result if the entered command is working as what it is expected. The easiest way of controlling the command is to try it on simultaneously from your terminal. Using Capybara via terminal is very easy and fast. 
Just copy and paste what is written in your env.rb file and see the result. However basically you can type the following lines after typing irb for Ruby execution window. 
require 'capybara/dsl' include Capybara::DSL Capybara.default_driver = :selenium
And then you can type your commands like as below, actually you can do what you are doing in your Ruby file: 
irb(main):004:0> visit "http://www.amazon.com"
irb(main):005:0> fill_in "twotabsearchtextbox", :with => "nexus 5"
irb(main):006:0> find(:xpath, "//*[@class='nav-submit-input']").click()





Creating Test Data: How to call Python Function from Java

Image
For my testing purposes, I need to create an e-mail before running automated test cases so that it should used it for new registration and a new one for sending invitation to unregistered e-mail and there are some other test cases which use new e-mail for testing. I have create Python function which produce a new e-mail address in this format: gunesmess+1@gmail.com and next run it will create gunesmes+2@gmail.com and for every run, it creates a new one just increasing the last used e-mail number. You can use this file for your testing too. Just create numberRun.txt file in you directory.

For my recent project, I need to write my automated test cases in Java so I have learnt some Java but not as professional. For this project again I need to create new unregistered e-mail address for using almost the same scenarios. Therefore I needed to call Python function in my Java code, instead of writing it in Java. The following code is used for calling:

What you need is that you must install J…

Using Examples Table in Cucumber

Image
Using Examples in Cucumber is a great thing for testing world! Examples is a keyword and you can define a set of variable to repeat your case with the whole set of data in the examples block. 
We can re-write the examples which checks the login page of amazon.com with Examples feature of Cucumber. Check the following code:

In the amazon_login.feature file we are reading <username>, <password> and <message> variables from the Examples. This means that the scenario which has the steps used these variables will run 2 times, which is the number of row in the examples.

There should some changing in the Capybara, Ruby file. "(.*?)" this means the function takes a string parameter but we are sending table variable so we need to change (.*) for string variable in table and (\d+) for integer variable in table.    

As you can see from the result there are 3 scenarios (3 passed) and9 steps (9 passed). This means that "Scenario: User goes to login page" run for 1…

Test Automation with Cucumber - Capybara - Selenium in Ruby

Image
For a company which uses agile development methodology, test automation is very critical. However meaning of test automation in agile is little bit different from the automation activities in waterfall or V-model development methodology. In agile a user story should be automated before the feature is developed and when it is ready then automated test cases can be run to test user stories.

For agile test automation, using only selenium is not enough or is not the best approach because of the written test cases. There should be another tool used for writing test cases which automation script can understand and perform what test cases/user stories explain. Cucumber is the tool what I want to explain. With Cucumber we can write test cases and with language called Gherkin which is very easy and understandable for everyone. So we just need to learn the keywords like "Given, When, Then, Examples, Scenario, Scenario Outline, ..." they are not too many, for more information about Gher…

Good QA Assignment: You may Want to Ask in Your Interviews

Image
QA position is not like development or analysis position so finding a good QA engineer is not as easy as development engineer. You may ask some technical questions to your QA candidate but half of the QA responsible is related to social ability, and the rest part is related to knowledge about testing domain and practices. I want to share a good review question and a related answers below. Question:Before you start- some directions: You need a computer, it can be a Windows XP/Vista/Windows 7, or a Mac OSX/Linux machine, of course Download the KeePass installer and install it on your machine you intend to test with, install the Classic Edition Version 1.17 from: http://keepass.info/download.html The problem:KeePass Password Safe is a free, open source, light-weight and easy-to-use password manager for Windows. You can store your passwords in a highly-encrypted database, which is locked with one master password or key file. Part 1:Use your skills as a tester to produce a report (in the format…