TEST THE RISK!

One of the biggest challenge for automation is handling timeout problem. Most of the time, timeout is 60 seconds but it may sometimes not enough if you have badly designed asynchronous calls or the third party ajax calls. This makes handling timeout more complex. set large enough to tolerate network related problems. For Selenium based automation frameworks, like Capybara, default Webdriver timeout is set to Net::ReadTimeout (Net::ReadTimeout) Changing ReadTimeout If you have timeout problem for Capybara, it gives an error like above. This means that the page is not fully loaded in given timeout period. Even you can see that page is loaded correctly but webdriver wait until the Ajax calls finish. class BufferedIO #:nodoc: internal use only def initialize (io) @io = io @read_timeout = 60 @continue_timeout = nil @debug_output = nil @rbuf = '' end . . . . . def rbuf_fill beg

Time to time, I see some questions about integration test automation script to performance testing tool on Stackoverflow . There is a confusion about performance testing and test automation for whom newly started performance testing after doing some test automation. It is demanded that they have some automation code and want to use these scripts for performance testing to simulate user behaviour. Theoretically it seems a very good idea that tons of users are performing some cases and you are capturing the response time and finding bottle neck in terms of client side. However this is not applicable in practice because there are lots of obstacle to handle. The followings outline the obstacles and misunderstandings you will face, for the idea. I can open some browsers and simulate real user behaviour Automation scripts run over a browser, for web project. Most of the application are becoming as web app. For GUI based application what the hack test automation will have a role on pe

If you are working on shell most of the time Aliases are very helpfull and time saving. For testing purposes you can use Alias for getting ready your test suites. In this post, I want to explain both running Selenium and Capybara on console and creating aliases for each.  This post is for Windows machines, if you are using Unix-like see   this post . Creating Scripts for Selenium and Capybara First of all, it is assumed that you have installed Selenium and Capybara correctly and they work on your machines. If you haven't installed, you can see my previous posts. I am using the Selenium with Python and the Capybara with Ruby. You can use several different language for Selenium but Capybara works only with Ruby.  Create scripts in a directory called scripts (in your home folder, like as  ~/scripts ) for your automation tool as following, save them as capybara.rb, sel.py :  Creating Aliases Depends on your favourite shell, you need to add the alias to .bashrc bash

I faced an selenium error like " stale element reference " when I changed Capybara's driver to Chrome which was not occurred in webkit and firefox. This error occurs when referencing an element which is not in DOM. Chrome gives this but it doesn't appear when I use Firefox because the dynamic object is appear after the Javascript execution finishes which means that Chrome doesn't wait until javascript execution ends .  You can escape this error by checking your scenario if there is an element requiring in the DOM which is destructed by an action in the scenario or if you have loop and the DOM is updating for each iteration the object you look for changes. However in  my case, scenario is passing with FF but not Chrome which because the Chrome is passing the next step as soon as Javascript execution starts.  I just call waiting function and then assign the variable. Finally scenario is passing for both browsers.        stale element reference: element is

One of the biggest challenge in test automation is updating existing codes. Because of the update in the development, it is very prone to have some changes for the attributes of ui elements and less likely to have some change in behaviour in actions. The second option is most of the time called as bug, otherwise if the business is changed, it could be called as a new feature. For any reasons that cause the automation code gives failures you need to update something. Sometimes I need to refresh my mind :) For easy maintenance, one of the good approach keep your codes modular. It means that each module should responsible for one job and a update problem should be solved by applying just one change in your system. Using Capybara plus Cucumber gives you this features but if you are using them professionally. For this framework, business rules are kept on Cucumber so you can update business rules from your .feature files but related update should reflect to the code. In thi

Page-object model is a pattern that you can apply it to develop efficient automation framework. With the page-model, it is possible to minimize maintenance cost. Basically page-object means that your every page is inherited from a base class which includes basic functionalities for every page. If you have some new functionalities that every page should have, you can simple add it to the base class. Base class is like the following: In this part we are creating pages which are inherited from base page. Every page has its own functionalities written as python functions. Some functions return to a new page, it means that these functions leave the current page and produce a new page. You should write as much as functions you need in the assertion part because this is the only part you can use the webdriver functions to interact with web pages . This part can be evaluate as providing data to assertion part.   The last part is related to asserting your test cases against to the

Testing the user interface (UI) testing may be the most difficult part of test automation. When we think from the perspective of users there are not hundreds but there may be thousands of different users' environment but it is clear that simulating and testing UI related cases for all possible users' environment is impossible! However we can cover easily the %80 of them by using statistical data taken from analytics tools as Pareto Analysis says 20% of reasons cause the %80 of all problem. If you are doing some kind of internet business, there are two important factors for user environments which are operating system and browsers for normal internet users.  Therefore analytic tools should give us feedback about operating system and versions of operating system; browsers and versions of browsers. To get a general trend we can look at the w3shcool.com statistic but sure it is more convenient to take your own statistic depends on your user profile. In general approach

As a default Capybara runs under Firefox but if you want to change your driver you have to change your javascript driver to your new driver. But you also must define your driver as registered Capybara driver. For example if you want to run with Chrome and change your driver to chrome but not define it, it gives some errors like following irb(main):007:0> visit "http://www.amazon.com" Capybara::DriverNotFoundError: no driver called :chrome was found, available drivers: :rack_test, :selenium, :poltergeist from /Library/Ruby/Gems/2.0.0/gems/capybara-2.4.1/lib/capybara/session.rb:77:in `driver' from /Library/Ruby/Gems/2.0.0/gems/capybara-2.4.1/lib/capybara/session.rb:226:in `visit' from /Library/Ruby/Gems/2.0.0/gems/capybara-2.4.1/lib/capybara/dsl.rb:51:in `block (2 levels) in ' if you add the following line into your env.rb module, you will successfully run under Chrome driver. Capybara.register_driver :chrome do |app| Capybara::Selenium::Driver

In Poltergeist mode or headless mode, it is different from Selenium since there is no browsers which you can track failure visually. The advantages of Poltergeist are: it is faster, it demands less source when we compare to Selenium, and eventually it is more suitable for Continues Integration (CI). Installation and further information about Poltergeist you can visit git page . Running Capybara with terminal may help you to get quick result. By this way you can check if button, field, or any object can be reachable easily and this method may help you to reduce failure issue on your CI and get you more realistic results. You can just paste the following commands into your terminal, it will go to home page of Amazon.

When you write test scripts, you may want to get quick result if the entered command is working as what it is expected. The easiest way of controlling the command is to try it on simultaneously from your terminal. Using Capybara via terminal is very easy and fast.  Just copy and paste what is written in your env.rb file and see the result. However basically you can type the following lines after typing irb for Ruby execution window.  require 'capybara/dsl' include Capybara::DSL Capybara.default_driver = :selenium And then you can type your commands like as below, actually you can do what you are doing in your Ruby file:  irb(main):004:0> visit "http://www.amazon.com" irb(main):005:0> fill_in "twotabsearchtextbox", :with => "nexus 5" irb(main):006:0> find(:xpath, "//*[@class='nav-submit-input']").click()

For a company which uses agile development methodology, test automation is very critical. However meaning of test automation in agile is little bit different from the automation activities in waterfall or V-model development methodology. In agile a user story should be automated before the feature is developed and when it is ready then automated test cases can be run to test user stories. For agile test automation, using only selenium is not enough or is not the best approach because of the written test cases. There should be another tool used for writing test cases which automation script can understand and perform what test cases/user stories explain. Cucumber is the tool what I want to explain. With Cucumber we can write test cases and with language called Gherkin which is very easy and understandable for everyone. So we just need to learn the keywords like " Given, When, Then, Examples, Scenario, Scenario Outline , ..." they are not too many, for more information a

If you use selenium webdriver to automate regression test cases, you may have got the following error. Because a bug in chromedriver version 2.3 causes this error. According to Google Dev ., this problem had fixed in version 2.6.  Assigning the chrome driver to driver leads to this error by the line 38, which is self.driver = webdriver.Chrome() . To solve the problem, you need to change the chromedriver version 2.3 in your execution path, like C:\Windows\System32 under Windows, with newer version. You can download chrome driver from googleapis .   Error message may like this: Traceback (most recent call last):   File "D:\workspace\automation_project\run.py", line 32, in <module>     m = Project(browser, server)   File "D:\workspace\automation_project\test_case_imps.py", line 38, in __init__     self.driver = webdriver.Chrome()   File "C:\Python27\lib\site-packages\selenium-2.35.0-py2.7.egg\selenium\webdriver\chrome\webdriver.py", line

Selenium webdriver can drive different browsers like as Firefox, Chrome or Internet Explorer. These browsers actually cover the majority of internet users, so testing these browsers possibly covers the 90% of the internet users. However, there is no guaranty that the same automation scripts can work without a failure on these three browsers. For this reason, automation code should be error-prone for the browsers you want to cover. The following error is caught when the test script run for Chrome and Internet Explorer, but surprisingly there is no error for the Firefox. Selenium gives an error like below: Traceback (most recent call last):   File "D:\workspace\sample_project\sample_run.py", line 10, in <module>     m.login()   File "D:\workspace\ sample_project \test_case_imps.py", line 335, in login     driver.find_element_by_id("id_username").clear()   File "C:\Python27\lib\site-packages\selenium-2.35.0-py2.7.egg\selenium\webdriver\r

As I tried to explain in my previous posts Selenium is an automation framework for web application and Webdriver is the newer version of Selenium RC. I have used Selenium Webdriver for automatizing web application with version of selenium-2.25.0-py2.7.egg which prepared for the Python. However there are newer version this Webdriver package.In this post I explain how to update Webdriver version on Windows. If you check the site-packages folder where the Webdriver installation folders are present, you can see the related folder. In this folder there is easy_install.py which is used upgrading. If you can't find this file first you need you install setuptool . To update the Selenium version, you can follow the instruction below: Check easy_install file in the directory like as C:\Python27\Lib\site-packages if there is easy_install go to next step, if you don't have it install setuptools from here. Download Selenium newer version from here . Select the link what language

Because of an upgrade, I needed to move to new computer and new operation system so I moved my automation code to new system which is from Window 7 to Windows 8. When run the code I got the following error indicating that the Selenium can't load the profile which means that the version of the browser is not supported by the Selenium yet. If you get an error like the following one, you need to find the supported version of the browsers. Traceback (most recent call last):   File "D:\workspace\ automation_project \run_test_cases.py", line 12 4, in <module>     m = Markafoni(browser, server)   File "D:\workspace\ automation_project \test_case.py", line 8 64, in __init__     self.driver = webdriver.Firefox()   File "C:\Python27\lib\site-packages\selenium-2.25.0-py2.7.egg\selenium\webdriver\firefox\webdriver.py", line 51, in __init__     self.binary, timeout),   File "C:\Python27\lib\site-packages\selenium-2.25.0-py2.7.egg\sele

As selenium user know, Selenium script can be run via FireFox, Chrome and Internet Explorer if you set up the environments correctly, you may read this post if you are un-sure. If you face an error while launching Internet explorer in selenium test automation, you need to set all the security zone in the same mode (enabled, or disabled mode). If you change security zone, selenium gives the following errors, and the last one says " Protected Mode settings are not the same for all zones " so you can set the protected mode by Tool > Internet Options > Security Tab then set Internet, Local Internet, Trusted Sites, and Restricted Sites as same mode, enabled or disabled. Then click "Apply" and "OK" and repeat your tests. File "C:\Python27\lib\site-packages\selenium-2.25.0-py2.7.egg\selenium\webdriver\ie\webdriver.py", line 53, in __init__     desired_capabilities=DesiredCapabilities.INTERNETEXPLORER)   File "C:\Python27\lib\site-packa