BugsFollow.Me

What is Selenium WebDriver

noreply@blogger.com (Artsiomi S.) — Thu, 20 Apr 2017 10:07:00 +0000

If you are just starting looking into Test Automation you must have heard something about Selenium, but not yet sure what it is. Let’s add some context and details to Selenium.

How Test Automation is working

Test Automation is pretty similar to manual testing and is dead simple. You need to make computer do something automatically. It can be done writing new small programs (scripts) which are called Automated Tests.

Automated Test is actually a small program — piece of code in some programming language which simulates actions of real user.

If we talk about Web — Automated Test would be mainly interacting with Browser to do something like opening pages, clicking somewhere, etc. And this is where Selenium comes into play.

What is Selenium

Selenium is a set of programming libraries which provides your scripts a way to interact with browser.

Selenium gives you an ability to:

Open pages
Finding some elements on a page
Performing various actions over elements like clicking or entering text into a text field
Take a screenshot of what’s displayed in browser

These actions are basically everything what user can do, so it is sufficient to write good Automated Tests.

Better part of it is that’s all this comes at no cost. Selenium is free and open source.

If you want to know more about Selenium and it’s history you can refer to official website — seleniumhq.org. A good place to start is Introduction to Selenium.

What is “WebDriver” part about

WebDriver brings a bit of confusion when you start learning about Selenium.

Why Selenium is sometimes called Selenium WebDriver? Because it’s using WebDriver standard underneath to interact with different Browsers.

But what is WebDriver standard? It’s a standard developed by creators of Selenium. Now it’s a W3C Standard available here — https://www.w3.org/TR/webdriver/.

This standard describes all the actions which user can perform over a browser and all the ways in which browser can respond.

Given this WebDriver standard is an ideal intermediate level between Selenium libraries and actual browser.

Selenium WebDriver is actually trying to work with something based on WebDriver standard. And here is where some wonderful things come into place — Browser developers (Google, Mozilla, etc.) were willing to support WebDriver standard and provide special versions of their browsers which can be controlled automatically (via WebDriver interface). These special versions of browsers are usually called Drivers.

Why this concept of WebDriver is good? Because when new browser arises like Edge — we will be able to automate it once it supports WebDriver standard. And no new work will be required from Selenium side.

This was a great effort by Selenium team to make it all work. Thanks to them and browser developers to let this happen.

But why selenium is called Selenium WebDriver, can’t is be just Selenium? This is a bit legacy thing. When Selenium was first developed — there were no such thing as WebDriver, it was working very differently. When new version was ready and WebDriver was in place — it was called “Selenium 2”, and sometimes to highlight how version two is different from the first one it’s called “Selenium WebDriver”. There is also a third version of Selenium which is still using WebDriver underneath.

Why do we need a testing framework (Selenium)

noreply@blogger.com (Artsiomi S.) — Tue, 18 Apr 2017 08:15:00 +0000

When you start learning automation testing in Selenium in any programming language — you will always get to a point when you need some testing framework (at least you will feel this way).

But why do we really need a testing framework? Is selenium bad without frameworks?

What is a testing framework

“Framework” is kinda buzzword in testing world. Everyone are looking for one, or looking to create one.

Actually what most people are calling a framework is just a set of libraries wrapping Selenium API.

It’s not such a big thing as frameworks in a typical software development sense. They are just not so mature, contains just some couple of features, doesn’t imply any rules on how to structure the code.

Typical features of testing frameworks

Features of testing frameworks doesn’t vary greatly, usually they just address different areas of software testing and try to ease test development or provide a solution to some routine tasks.

Features of testing frameworks (in no particular order):

Reporting — many frameworks include beautiful business-ready reports out of the box. It’s good when you have a report which suits your needs, but can be troublesome to modify it later if you will need some extra information in it.
Logging — most frameworks include detailed logging of actions performed by Selenium API and other things. It’s good in any way if coverage is good enough. It can ease debugging and support of tests. Sometimes logs also include screenshots or video of test execution.
Validations — some frameworks address how you validate things in tests. Proper validations may ease the Reporting and Logging mentioned above.
Page and Elements handling — some frameworks dictate how pages and elements should be organized. This is not a technical decision, but rather a try to structure and organize the tests. Sometimes it makes sense, but some decisions are very subjective and represents a specific opinion of a framework’s author.
Waits and AJAX handling — some frameworks try to give you all-in-one solution to all the AJAX and async moments in tests. There is no silver bullet, so if your framework tells it has one — be careful with this. More mature frameworks doesn’t give you all-in-one solution, but rather give you good opportunities to make Waiting easier.
Browsers handling — most frameworks try to ease working with different browsers. And many of them did a great job with this. But still sometimes you will be in situations when you will need to add something custom to make it work everywhere. Nothing bad about frameworks, but just remember — that you need to do something on your own and not depend on others.
Locating elements — some frameworks question the way you locate the elements and try to give you their own approach on this. Again this kind of features is pretty subjective and won’t suit everyone’s needs.
Parallel runs — some frameworks state that you can run tests in parallel and provide you some things to do this. Parallelization is a tricky area. Usually it’s more about Continuous integration and the way you run the tests rather than Testing frameworks.

The need in testing framework

There is no vital need in testing frameworks.

All the mentioned features are not a vital things to include in your tests, they just cover something which you may need to do yourself.

So the frameworks won’t make your tests better, they just speed up your work and provide some specific solutions to everyone’s problems.

Selenium is not bad in any way and doesn’t require you to use frameworks. Actually all the interaction between the browser and the pages is performed by Selenium and not the frameworks. It means that there are no any advantages on using one framework over another in terms of interaction with pages — they all use the same Selenium API underneath.

So what to do if you start learning automation and are thinking about which framework to choose? My bet is — get anything popular and wide-spread and start with it. Once you will meet some problems you can start looking into changing the framework or writing your own.

Approaches to handling AJAX in Web Automation (Selenium)

noreply@blogger.com (Artsiomi S.) — Fri, 03 Apr 2015 15:23:00 +0000

Unfortunately pretty much places in web applications automation are based on "Magic". What do I mean by saying magic?

Example of Magic:

Test is failing (or some operation is failing) and you don't know why
You are trying something - and is works
BUT you can't explain why the thing you done helps to fix the situation, and is it reliable or not

So, talking about JS/AJAX handling - it's mostly based on Magic for now. Nobody knows how to handle it properly, neither I do.

In this article I'll try to explain how to get rid of Magic in JS/AJAX handling as much as possible. Just theory, without code examples.

Why do we need to handle JS / AJAX in specific way?

Page loading can be divided into two parts: synchronous ans asynchronous.

Synchronous:

Receiving most of the page content from server
All the elements are added"at once"
Basically it is what happens before "loading" indicator is stopped in browser (usually at the top of the browser UI)
How to handle:

In most cases we don't need to handle this specifically, because WebDriver waits for page to load before any action on elements. However some exceptions apply. I don't know whether the list of exceptions exist - so you probably will find it out the hard way.
Since all the elements are drawn (rendered) at once - to handle synchronous loading - you need to determine unique element on the page you want to work with.

By waiting for this unique element - you will be sure that all synchronous elements are loaded as well.
Element needs to be unique because otherwise your test may find this element on previous page and think that loading is already completed.

Asynchronous:

Any client-side processing - JS, AJAX
This may happen at any time, but usually is tied to some user interactions
Each element is added separately - after the processing is completed
How to handle:

Each case of asynchronous loading need to be handled separately.
More details on approaches below.

As you may see the main difference here is adding of elements (adding to DOM).

How JS is related here? - JS just is a way to add element on client-side (in browser).

How AJAX is related here? - AJAX gets some additional info from server, to be added on page via JS

How Asynchronous loading is handled?

I know following approach, which will be described in more details below:

Use Sleep to pause execution
Use ClickWithMouseOver or some other Magic method
Use JQuery or JS Event-based waiting
Use specific waiting for each case

Use Sleep to pause execution

There is some element which will be added now? Ok, I'll wait 5 sec for it, just like human do. Seems reasonable, but in fact it's not.

This is the most wide-spread approach, and in fact the most straight-forward.

Human will stop waiting after he sees the element appeared. And if element is not appearing withing 5 seconds - he can wait more.

Sleep approach - instead determines some time to wait, This timeout time may be fixed (hopefully) or vary by each place.

Bad:

It's slow. Every problem place will be just waiting.
It's not reliable. If test will still fail - the only thing you can do - increase Sleep time.
It's not effective. Your element may appear in 2 seconds, but you will still wait for full 5 seconds before continuing.

Good:

It's dead simple.
It's cheap to implement. (not counting test support time here)

Fortunately many people understand drawbacks of this approach, and try to eliminate it. But in some complex cases may fallback to this.

Use ClickWithMouseOver or some other Magic method

The second most used approach is pure Magic.

The most simple example:

You have a test case which was working, but now it fails on clicking a button.
You run it locally - and it passes only sometimes (40% of executions).
You have hard time thinking how may this happen.
You try everything and occasionally try Clicking with Mouse over the button
Hurray! it helps - test passes 100% of execution.

Do you do everything right here? Seems legit. But it may be not.

In place of Clicking with Mouse over can be some other method which helps, like:

Waiting for some element on page before Click
Checking of button attributes before Click
Even some kind of Waiting for AJAX (based on JQuery.active), more details in next section
Checking for some info in DB before click

What is common for all this functions? - They all are not doing anything helpful or needed for the test case.

How the hell then they helps up to pass the test? - They introduce some kind of delay. It may be even 0.5s to move the mouse pointer over the button, or to process some info from DB.

It appears that this approach is just a kind of unintentional Sleep calling. Instead of waiting for some specific time - you slow down your test just enough to make it passed.

Bad:

People usually don't understand when they do this
Extra unneeded actions performed
It's highly not stable, if test will fail later - you will just look for other Magic method which helps
You have confidence that you're not using sleeps, but in fact - you are

Good:

Tests are a bit faster then with using Sleeps

I won't recommend this approach in any cases. Even using Sleeps is better that this one - because you are using Sleep intentionally, without any Magic.

Use JQuery or JS Event-based waiting

Most of the sites use JQuery for different reasons, including performing AJAX calls. But this is not the panacea. Developers can implement AJAX without using JQuery.

If your site is using JQuery for AJAX - you can use JQuery to see whether AJAX call is ended or not. Seems ideal. But again it's not.

There are two approaches for handling JQuery AJAX calls ending, let's stop on them briefly:

.jQuery.active == 0

This counter shows how many requests are "in progress" now
This is not working, because there may be gaps between different requests.

ajaxStars and ajaxStop events

These events are triggered on Start and Stop of each request accordingly.
These events are commonly used to display "loading" icon
This is not working for the same reasons - there may be gaps between requests. Getting back to "loading" icon - you may see how it's flickering sometimes - this is a visualization of gaps between the requests.

Bad:

Tied to site implementation. May be not suitable for everyone
Gaps between requests can be handled correctly. Maybe only with some fallbacks to timeouts and sleeps. Like assuming that gap won't be longer than 3 seconds, and wait up to 3 seconds after active=0 before proceeding.
Involves low-level JS communication with site
A bit more complex to implement than previous approaches

Good:

It's slightly more reliable than previous approaches
It may work very well for sites with small amount of AJAX (when there is only 1 consequent request and may not be gaps)

Use specific waiting for each case

The most complex and the most suitable approach in my opinion.

I believe that you can't handle all the JS/AJAX cases in one way.

Instead of trying to achieve it - dig deeper in each specific AJAX call.

AJAX calls usually are tend to get some info from server, that means that as a result - something new will be displayed on page. You need to find out which new element will be displayed the last - and wait for this specific element.

Bad:

Digging into details of AJAX calls may be complex
Code base is significantly bigger - since you write separate waiting for each place
Implementation may change, and you will need to support waiting
It touches pretty low-level interactions of elements on page
Implementations of this is pretty time consuming

Good:

You perfectly know how it works in every case. Finally no Magic here
No sleeps or timeouts involved, you wait for exact condition
It's stable to application slow downs (due to some performance issues or high-load)

In summary

I hope this article gives you more detailed view on AJAX in Web Applications and how to handle it in Automated testing.

Please post any other approaches in comments.

Random Data in Test Automation

noreply@blogger.com (Artsiomi S.) — Thu, 02 Apr 2015 09:30:00 +0000

Using random data can be pretty tricky. As always there are two types of people - who uses random a lot, and who don't.

Let's compare these two ways.

Random	No Random
Every run you get a new set of data to use Coverage is increasing with each test run You don't need to think about duplicates in data, assuming random is working well	Every run is on same (or identical) set of data Pesticide Paradox in place You need to handle duplicates - delete data from previous runs, or in some other way

From the table above using Random seems to be the best choice, but there are some pitfalls.

Tests are not repeatable

One of the main ideas about automated testing is - tests should be repeatable. This means that no matter how many times you run the same test against the same system - it should provide the same outcome.

With using random values - this is not the case. You always run slightly different test case, and if some value appears to be buggy - you will see this only in logs, and won't be able to run the automated test on the same values again.

Tips:

Even though your tests are using random data - leave an option to provide specific data for test run.
Write separate test cases with specific data for found bugs. This will help you to find regressions.

Random generated value needs to be in the same Equivalence class

If you don't know about partitions - please read about Equivalence Partitioning. There is a good article about it on Software Testing Class - Boundary Value Analysis and Equivalence Class Partitioning With Simple Example

Let's assume that we need to test checkout in online shop - we need to select a product, and enter quantity of items we want to buy. And the requirement is - nobody can order more than 10 pcs.

So valid values will be 1-10 items, and invalid values - below or equal to 0, above 10.

As you see we can't just put random number in here - we need to limit randomness range.

Your testing solution need to be able to generate two kind of random quantity values - valid and invalid.

And for such a simple case it sounds easy and obvious. But when it come to something more complex (for example Full name for a person, Address or Filename) - preparing random data can be troublesome.

Tips:

Randomly generate only valid values for positive cases, you probably have some rules for valid values and you can stick to them
Use non-random values for negative cases - range of invalid values is usually far bigger than valid ones, while outcome of test coverage is pretty small
Separate generation of random data from other code. If you will have some separate Class/Generator which will contain all the rules - it will be much easier for you to modify the rules

DB overload with randomized test data

If you don't care about clearing you data - you will soon be overloaded.

So no matter that random values give you an opportunity to bypass duplicates validation - you need to keep in mind this validation, and clear the created data.

Timestamp is not effectively random/unique

Let's say you need a Last name - just add an UNIX timestamp to the end and it will be random each time. Easy and seems unique. But in fact it's not.

UNIX timestamp is unique only across one machine for executing tests, without any changes of time and timezone (including DST). But in automation we usually run the tests across different machines - so timezones can overlap and provide duplicates for our "unique" values.

Tips:

Use GUID or some similar mechanism to maintain uniqueness
If your random value contains some illegal characters (for example dashes in GUID) - just replace them with some specific character. The value will still be unique.
If you are still using Timestamps - don't try to truncate or process them in any way, because any processing will limit it uniqueness even more.

In summary

If you are not concerned about Pesticide Paradox - I recommend you to not deal with Random data in your tests, or use it as least as possible. This way you won't need to handle all the things mentioned above, but you will need to implement some mechanism of clearing created data.

Why developers are so paranoid about failing Integration tests and Why this is not a problem

noreply@blogger.com (Artsiomi S.) — Tue, 31 Mar 2015 17:54:00 +0000

When it comes to developers talking about Web Integration tests - you may hear something like "Test should not fail if I move this button" or "If the text on the button changes slightly - test should still pass"

But from my point of view it's ok to fail test because of that. And I would even believe failing test more than passing one.

For a long time I cannot understand the reason of this misunderstanding. And today while reading a book on unit testing - I finally got it.

tl;dr changes in Web Page are more sensitive than changes in API

When you talk about unit testing you really don't want your tests to break as a result of refactoring. It's a well-known fact, and I won't argue with that.
It's so because unit tests are working with pretty specific interface. And changes in interface will be caught by compiler - tests wont compile.

Back to Web Integration tests - we don't have such a detailed interface to interact with. Our accessible interface is pretty wide - whole page with all the positions, images, animations and so on.
Also we don't have a compiler to check all the bindings to an interface (Locators for elements) are suitable for us to interact with.

	Communication interface	Entity	Properties (non-functional)	Who is affected	Price of unexpected interface changes
Unit Test	API (closed or open)	Method	Method name, Parent service, Method parameters	Developers, Tests	Decide yourself
Integration Test	Web Page	Element on page (field, label, image, etc.)	Position on page (in DOM), any style related info (size, color, etc), text content, value. etc	End users, Tests	Decide yourself

So moving back to the button - it is in fact a change in communication interface between a Web Page and Web Integration test.

We have two options how to work with it:

Ignore the change
we can do this by making very narrow Locator for an element - which won't touch anything except the button itself (most probably by some unique button id)
Fail the test
we can write a bit broader Locator - which will know some more info about the button (text, siblings, etc.)

Let's take look at advantages:

Ignore the change

Advantages

We can change anything about the button (refactor) except of it's functionality and unique id
Tests will fail only on functional defects

Disadvantages

Developers need to provide unique id's for all the elements, and maintain it through the development process
Button may change it's place (change interface) and we actually won't know about this

Fail the test

Advantages or Disadvantages (depending on point of view)

Tests will fail on interface changes
Tests will need to be fixed by human in case of changes

In summary

It's totally up to you to determine the price of unexpected interface changes.

Since there is no automatic way to check that these changes are valid - I think it's ok to fail the test, and let the human decide whether that change was expected. With a good approach Web Automation you will only need to make changes in one place (update one Locator) which is pretty quick, but in the same time - you become a little more confident in your test results.

Testing tip: Google Chrome clearing cache/cookies with a new Chrome User

noreply@blogger.com (Artsiomi S.) — Mon, 15 Apr 2013 06:36:00 +0000

While testing Web Applications we can frequently face problems related to caching or some old cookies.

The easiest way to clean them in Google Chrome is to use Ctrl+Alt+Delete key combination (or menu Tools -> Clear browsing data...): it will clean all the cookies entirely. And it works in most cases.

But what if you don't want to lose all of your cookies (especially if you use Chrome for your personal needs)? You have two options here:

Use incognito mode.

For this press Ctrl+Shift+N key combination. In this mode you won't have any cache or cookies in place and they won't be saved at the end of session as well.
For mode details you can refer official article - Incognito mode.

Create new Google Chrome user and clear everything (or just cookies) for him.

In this way cookies won't be cleared at the end of session, so you may continue your work the next day.
For details on creating new user you can refer official article - Manage multiple users on Chrome.
User needs to be created only one time, later you can just switch to it and clear.

Personally I prefer 2nd option with new user.

Good luck guys, hopefully this simple tip will be helpful to someone.