Skip to main content

Posts

Mobile Test Automation: Calabash on Xamarin Test-Cloud

If you are testing your application against emulator / simulator, you will still have risks that the expected features may possible not work on several real devices. To handle the real device testing you should have many devices to run your tests. In the current market, we have lots of possibilities for device and operating system versions combinations. Best approach to find most used devices, you should use some statistical data. Most of the cases Pareto analysis can help for selecting high coverage devices. However if you want to find more bug before release, you can focus on the newest device with latest OS version and the oldest devices with the oldest OS version. Any case you should have at least 5-10 android devices and 3-5 iOS devices for a good level of coverage in the beginning. Therefore managing the devices is another problem if you want to have your own local test suites. As a good alternative you can use the cloud services. In this post I want to share some information a...

Why "Test Risk"

Recently I have had lots of interviews for QA Analyst roles in the company. We called engineers from the level of beginner to senior test engineers for these roles. One of the most important questions in our interview is for understanding why we are testing. Actually this question is more about the philosophy of the testing, as we think the philosophy is generally not liked by people, but everyone who does something about a subject everyday from 9 to 6 should think about why he does this job. I wonder whether the testing job is just only to feed himself or he has some other passion about his profession. That's why this question is very important for me. One of the training given by Ståle Amland which is " Exploratory Testing – Risk-Based Agile Testing ", he explains the philosophy of testing as epistemology of testing and he is saying that " all good testers should  practice Epistemology ", see these slides in his training: What is epistemology is acco...

Create Alias for Interactive Console for Capybara on Windows

If you are working on shell most of the time Aliases are very help full and time saving. For testing purposes you can use Alias for getting ready your test suites. In this post, I want to explain both running Selenium and Capybara on console and creating aliases for each. This post is for Windows machines, if you are using Unix-like see this post . Creating Scripts for Selenium and Capybara First of all, it is assumed that you have installed  Selenium  and  Capybara  correctly and they work on your machines. If you haven't installed, you can see my previous posts. I am using Capybara with Ruby. You can use several different language for Selenium but Capybara works only with Ruby.  Create scripts in a directory called scripts (better to use this all user so the path should look like  c:/scripts ) for your automation tool as following, save them as capybara.rb  and sel.py respectively: Creating Aliases For Windows Creating Profile File F...

First Checks for Mobile Application Testing

Mobile application testing world is differentiated from web application testing in terms of the power of the control. The definition control , how much things you can do when users install the application to their mobile devices. Think, if you store warnings in code and deliver them inside the application code, you can not manage it unless the users update the application. For sustainable mobile application development, you should have the control of the applications as much as you can do. Therefore the architectural design of the application should be ready for this flexibility. In this post I want to share the most important and primary checks for mobile application testing. Security First Whatever you develop, you can not disregard the security of the users. If you have an open deep-link for critical information, it will one day be distinguished by some one. Critical actions, such as login, sign-up, credit card submit, should not be sent over get requests , they...

SQLCMD: Alternative Way of Working with Microsoft SQL Database for Ruby

If you have Microsoft SQL server in your test environment and want to populate test data in Capybara, Calabash scripts you will most probably have difficult time to installing appropriate Ruby gems. One of most popular ruby gem for Mssql connection is tiny_tds but I was not able to install it for both Windows and Unix like environment just because of dependencies. I got the following errors: Errno::EACCES: Permission denied @ rb_sysopen - /Users/mesutgunes/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/tiny_tds-1.1.0/.codeclimate.yml An error occurred while installing tiny_tds (1.1.0), and Bundler cannot continue. Make sure that `gem install tiny_tds -v '1.1.0'` succeeds before bundling. Another popular ruby gem is sequel . It does not give an error for installation but when it comes to usage, it gives the following error for not found adapter. Main problem with the Mssql in unix is finding compatible adapter. See the log: irb#1(main):002:0> db = Sequel.odbc(...

Adding Slave Jenkins to Master Jenkins

Jenkins is an orchestration tool that you can use it for general automated jobs. Using one Jenkins for every kinds of job create some inefficiencies so the better idea is grouping jobs for their functionalities and having them done with slave Jenkins. Each slaves can be attached to master as a node and waiting for command. With this way you can run Windows related jobs in Windows; unix or mac related jobs by unix system and so on. Moreover you can add multiple and exactly same slaves so that to run the jobs more quickly. In this post I want to share an idea for adding slaves to master Jenkins.  1. Go to master Jenkins > Nodes > New Nodes >    - Enter "Web Automation Slave 1"    - Select "Permanent Agent"    - Click "OK"      - Enter Labels > " regression "      - Click "OK"      - Click "Web Automation Slave 1"      - Copy the URL of this page   ...

Importance of Testing: Case of Samsung Note 7

Test it or let the customers do it in wild! Testing is not a state it is a process, any failures in the process, make the quality ruined. Moreover, any failures detected in any part of a product make the product fail in terms of customers. Therefore we need to deploy the quality as a process of products and services which are given to customers along the product life cycle. In this post, I want to explain the importance of testing with the case of the successful Korean technology company 'Samsung'. The real problem with the product is that design of the battery has some defective points because they want to push boundaries too much for having more power small battery cell. For more detail, you can this post . Let's look at product life cycle management (PLM) first. It explains a product from an idea to delivering a product and terminating support. It divides the product life cycle management into four main stages as follows: Conceive  : Imagine, specify, plan, innovate Des...

Repeatable Test Case Design: Escaping Flaky Behavior

Automation means repeating some steps to have some tasks done with test code. However when it comes to test automation fulfilling this meaning is not that easy because of unpredictable reasons. You may face that a case has run successfully for a period of time but it fails for without an unknown reason. Actually there is always reasons which can make test failure but it may not be possible to handle it when writing the test. This kind of test cases are called as  flaky   test cases by Google . Flaky tests are the last thing you may want in CI environment because test are writing for fast feedback not for checking accidentally failed test cases. In this post I want to explain what I have done to overcome, actually minimise the effect of flakiness. Nature Of Flakiness Most of the time flakiness cause by our own mistakes. This could be not understanding the automation frameworks deeply or could be the result of non-adequate analysing the case. The other reasons like...

Using ChromeOptions for Capybara: Disable Images and More

Chromedriver lets us set preferences via capabilities when it is initiated by Webdriver. These preferences can be used for specific purposes for example disabling images can reduce network needs and fasten the test runs; and disabling popups can make your cases more stable by reducing non-deterministic cases. In this post I want to explain how to use these feature of Chromedriver for Capybara with Ruby, you can also apply the same method for other languages with their bindings. Sometimes bandwidth is crucial if you have parallel pipelines for test runs. Think that you have 10 parallel pipes and very pipeline consume bandwidth to complete its tasks, the most heavy part of website generally images. If we disable the images we can fasten the overall test runs. To disable the images you should set the images option to 2 in preferences. For Capybara you can disable the images by following Chrome instance: Sometimes you need to disable popups which intermittently occurs and make...