Mogoblog

Mogotest: Automated Cross Browser Render Issue Detection

2010-08-18T00:00:00-04:00

We’ve been rather busy since we launched paid plans for Mogotest just three weeks ago. We’re very pleased to announce our new automated issue detection feature for discovering cross browser rendering inconsistencies. Whereas we’ve always spidered your site and presented you with a set of screenshots for each URL, this feature will compare the rendering output from two browsers and detect any major rendering inconsistencies, saving you the time hassle of having to manually compare screenshots by inspection. The results for your entire site are available in a concise report and the comparison views have been enhanced to highlight the rendering differences.

Some screenshots should help illustrate things:

Figure 1: Overview of browser render results.

Fig. 1 shows the new overview report when a test job is completed. You have a matrix of results indicating your site’s browser compatibility at the render level relative to a reference browser. The reference browser is the one used during primary development and should represent the canonical rendering of the site. Here we’re using Firefox 3.5 on Linux as our reference browser. In this case, when testing the techstars.org site, we see that the /iphone-app/ path is particularly problematic across most browsers and that Internet Explorer 6 has render issues across most paths.

If you click on a cell, you’re taken to a diff view of that path showing differences in that browser relative to the reference browser. Fig. 2 illustrates a rendering issue with Chrome 5 on Windows on the /iphone-app/ path.

Figure 2: Sample error diff between Firefox 3.5 (Linux) and Chrome 5 (Windows).

Below the screenshots you can see the list of render issues detected. When you select an error it highlights the element in both the reference and comparison browser views, adding an overlay with the upper-left-hand and lower-right-hand coordinates to aide in debugging. If you decide that the render issue is acceptable, you can simply choose to ignore the issue and you won’t be bothered with it again.

The improved reporting has cascaded down to our notification emails, too. Whereas previously you would have to click through to our site to see if there were any rendering issues, we now report problems by browser in the notification email (see Fig. 3). Additionally, we indicate the change in errors from the previous test run to help you better determine if it’s something you need to look into immediately or if it’s a problem that can be deferred.

Figure 3: Enhanced email reporting.

The new automated issue detection feature is now available for all paying accounts at no additional cost. If you’d like to see it in action, try running a free page test. And since automated issue detection is something that could always be improved, please report any issues you run into and we’ll do our best to handle them better.

Mogotest Public Beta

2010-07-19T00:00:00-04:00

Mogotest is now in public beta. This means anyone can now try out a limited version of the service for free. It’s as simple as entering a website address into the free page test box on our homepage. While these tests won’t showcase absolutely everything Mogotest has to offer, it’s a great way to run a quick test and try out the service without having to go through the hassle of creating a user account. We hope you like it!

Now that we’re in beta, we’ll be doing a bit more spit & polish work and then will begin charging. We’d like to thank all of you that have contributed to the success of our alpha and now beta programs. It’s been a great five months and Mogotest has evolved considerably with your feedback. Most recently, we’ve added new features like Chrome browser support and a new “Problems” tab for site-level test run notices, while also improving spidering, authentication handling, and general service responsiveness. We simply wouldn’t have been able to do this without your help.

So how much will it cost? Our individual user plans will be $45 / month. We’ll start charging on July 28, 2010. If you don’t wish to upgrade to a paid plan at this time, don’t worry; you’ll still have read-only access to all your existing test run data. If at any point you decide you do want to upgrade you’ll instantly regain the ability to perform test runs, add sites, and modify your test groups.

As always, please let us know if you have any questions or concerns about our switch to paid plans.

How to Perform Basic Authentication in Selenium

2010-06-23T00:00:00-04:00

Performing basic authentication in Selenium 1.x may seem impossible at first glance. Selenium 1.x drives the browser via JavaScript, but is unable to do anything when the basic authentication dialog pops up. One way to get around this is to embed the user credentials in the URL. Traditionally this has worked well, but newer browsers have disabled that functionality due to the security risks inherently involved. Fortunately, there is another way: send over the authentication credentials in the request header.

Selenium as a Proxy Server

In order to customize the request headers in Selenium you must be using it as a proxy server. Unless you use the -avoidProxy argument when you start up the Selenium server, it will serve as a proxy server bound to localhost (i.e., it is not an open proxy). You also must use a browser launcher that configures the browser to use the Selenium server as an HTTP proxy. I discuss this in more depth in my article on how to accept self-signed SSL certificates in Selenium.

It is a best practice to only perform basic authentication over HTTPS if you’re accessing publicly accessible pages. If all your work is on an internal network and it’s a trusted environment, basic authentication over HTTP may be acceptable. Just realize that basic authentication is plain text and easily sniffable. If you’re connecting to a server with a self-signed certificate you may want to read the aforementioned article on how to do so with Selenium.

Basic Authentication with Selenium

It probably seems obvious, but you really need to make sure your credentials work. If they don’t, the browser is going to pop up a basic authentication dialog and your Selenium browser session is going to be effectively locked-up. An example of how to do so in Ruby can be seen in Example 1.

require 'rubygems'
require 'net/http'

# Check credentials via SSL.
http = Net::HTTP.new('www.example.com', 443)
http.use_ssl = true

# Make the request with the basic authentication credentials.
req = Net::HTTP::Get.new('/')
req.basic_auth('my_user', 'my_password')
response = http.request(req)

# 401 is the HTTP Unauthorized status code.
if response.code == 401
  puts "Basic authentication failed"
  exit(-1)
else
  log_in_with_selenium()
end

Example 1: Ensuring your basic authentication credentials work (in Ruby).

Once you’ve verified that your basic authentication credentials do in fact work, you can proceed to add the credentials to your Selenium request. You do so by adding the “Authorization” header with a value of “Basic ” followed by the base 64 encoded “username:password” pair. The browser will cache this value for all requests within the same session, so you only need to authenticate on the first request. Example 2 shows how this looks in Ruby using the selenium-client gem.

require 'rubygems'
require 'base64'
require 'selenium/client'

# Note that the start URL is not on the domain that we need to authorize for.
# If you set the start URL there, you may see the authorization dialog before
# you're able to modify the request.
Selenium::Client::Driver.new \
        :host => 'localhost',
        :port => 4444,
        :browser => '*googlechrome',
        :url => 'http://localhost:4444',
        :timeout_in_seconds => 180,
        :highlight_located_element => false

# Add in the basic authentication header.
encoded = Base64.encode64("#{my_username}:#{my_password}").strip
browser.remote_control_command('addCustomRequestHeader',
                              ['Authorization', "Basic #{encoded}"])

# Request the page, fully authorized.
browser.open('http://www.example.com/')

Example 2: Adding basic authentication credentials to Selenium request (in Ruby).

Conclusion

Since basic authentication credentials can be supplied as an HTTP request header, you can perform basic authentication with Selenium with a little bit of work. You need to use the Selenium server as a proxy for your browser requests, configure acceptance of self-signed SSL certificates if appropriate, and verify that the credentials work before issuing your request. Once you do so, you’ll be able to test sites without worrying about the browser locking up on the authentication dialog. Note that this article only addresses basic authentication (the most popular of the HTTP authentication methods). Digest authentication is a completely different beast and the method laid out in this article will not work for it.

Google Analytics Integration with OAuth

2010-05-27T00:00:00-04:00

One of the new features we’ve been working on behind the scenes at Mogotest is a Google Analytics (GA) integration. By integrating with the Web’s most popular analytics service, we can help users see how specific browser rendering problems are affecting their bounce rates, conversions, and traffic funnels. The sort of stuff you’d like to know about as a site owner, right? Unsurprisingly, we also use GA ourselves to monitor our own traffic metrics and have some automated reports that help us measure our day to day progress.

In late 2008, Google began moving away from their own AuthSub authorization system and started offering access to all their Data APIs via OAuth. That’s A Good Thing™. But since there doesn’t seem to be a wealth of information on the ‘net regarding the GA integration process, I figured it might be worthwhile to document how we integrated it with a Rails-based application and share it with the community. The OAuth Ruby Gem makes most of this process very simple, but there are a few tricky bits that might trip you up if you haven’t worked with OAuth and the Google APIs before.

The first thing you need to do whenever you’re tying into a system using OAuth is obtain the consumer token and secret from the provider. In this case, the provider is Google and we can retrieve this information by visiting Google's Domain Management Page. Unfortunately, this URL isn’t the easiest to find. I probably never would have found it if it wasn’t for Dan Sinclair's earlier GA + OAuth tutorial. Here’s what it looks like:

Anyway, once you’re on the Domain Manager page, add the domain for your application (the address from which authorization requests will originate). You’ll need to verify ownership by either adding a meta tag to your website, adding a new page, or modifying a DNS record. Once you’ve done that Google will give you the OAuth consumer key and consumer secret you need for the next steps.

NOTE: this verification requirement can be painful if you don’t have anything in production yet or need a separate set of credentials for testing or development. I’d suggest pointing DNS for some URL like test.yourdomain.com to a staging server to get it up and running and then changing it post-verification so you have at least two sets of OAuth credentials for the separate environments.

To authorize access to a specific GA account (like one belonging to one of our users), you need an access token. To get an access token, you first need to create a request token. The workflow for getting a request token using the OAuth gem involves creating a new Consumer object, passing in the consumer key and consumer secret, along with a few other options, and then calling the get_request_token method. For convenience, I’ve wrapped up the process of requesting a new Consumer into a library method, since we’ll need to do it a couple times. Make sure that the token paths specified are exactly what is shown below in order for this to work with Google:

class GoogleAuth
  def self.consumer
    OAuth::Consumer.new(oauth_consumer_key, oauth_consumer_secret, {
                        :site               => 'https://www.google.com',
                        :request_token_path => '/accounts/OAuthGetRequestToken',
                        :access_token_path  => '/accounts/OAuthGetAccessToken',
                        :authorize_path     => '/accounts/OAuthAuthorizeToken' })
  end
end

Our call to get_request_token is going to get made inside the new action of our AuthorizationsController. We’ll retrieve the request token, specifying a callback and a scope. The callback is the URL that the user will be directed to after authorizing with Google. The scope is used to specify the Google service that the user is authorizing us to access. Google provides many different API services in addition to analytics – Calendar, Documents, YouTube, etc – but we want to scope to the Analytics API here. We set the token and secret in the user session because we’ll need them later when the user returns to the site. Finally, we get the authorization URL from the request token and redirect the user to it. Note that we also encode instructions to return to our callback URL after successful authorization.

def new
  @request_token = GoogleAuth.consumer.get_request_token({ :oauth_callback => callback_account_authorizations_url },
                                                         { :scope => 'https://www.google.com/analytics/feeds/' })
  session[:request_token] = @request_token.token
  session[:request_secret] = @request_token.secret

  redirect_to("#{@request_token.authorize_url}&oauth_callback=#{CGI.escape(callback_account_authorizations_url)}")
end

From the users’ point of view, they’ll get a link to click to “enable Google Analytics”, which will take them to the new_authorizations_url corresponding to the action above. This will then auto-redirect them to the Google authorization page, which will resemble the screenshot below:

After they authorize us to access their data, they’ll be redirected back to our callback URL. When Google redirects to this URL they’ll include an important query parameter that we need to create our precious access token: the OAuth verifier.

Here’s what the callback action should look like in our controller:

def callback
  unless session[:request_token] && session[:request_secret] 
    flash[:error] = "No authentication information was found in the session. Please try again."
    redirect_to(root_url)
    return
  end

  unless params[:oauth_token].blank? || session[:request_token] ==  params[:oauth_token]
    flash[:error] = "Authentication information does not match session information. Please try again."
    redirect_to(root_url)
    return
  end

  @request_token = OAuth::RequestToken.new(GoogleAuth.consumer, session[:request_token], session[:request_secret])
  @access_token = @request_token.get_access_token(:oauth_verifier => params[:oauth_verifier])
    
  session[:request_token] = nil
  session[:request_secret] = nil

  current_user.update_attributes(:access_token => @access_token.token, :access_secret => @access_token.secret)

  flash[:notice] = "You have successfully linked your Google Analytics account."
  redirect_to(account_path)

rescue Net::HTTPServerException => e
  case e.message
  when '401 "Unauthorized"'
    flash[:error] = "This authentication request is no longer valid. Please try again."
  else
    flash[:error] = "There was a problem trying to authenticate you. Please try again."
  end 

  redirect_to(:action => 'new')
end

The first two conditional blocks just check to make sure that the token and secret are found in the user session and that the token provided by Google in the parameters matches the one that the user previously established. The next step is creating an access token from the verifier that Google sent back to us. This is the token we’ll use to access the GA resources for the user going forward.

We can then clear out the session variables and update the user model with the access token and secret that we’ve obtained. By storing these values in our database, and associating them with the user account that authorized us, we can rebuild the tokens we need to obtain analytics data from Google at any time, without having to go through this process again.

Here’s the controller code for our AuthorizationsController in its entirety (including a method to de-authorize us):

class AuthorizationsController < ApplicationController
  before_filter :require_user

  def new
    @request_token = GoogleAuth.consumer.get_request_token({ :oauth_callback => callback_account_authorizations_url },
                                                           { :scope => 'https://www.google.com/analytics/feeds/' })
    session[:request_token] = @request_token.token
    session[:request_secret] = @request_token.secret

    redirect_to("#{@request_token.authorize_url}&oauth_callback=#{CGI.escape(callback_account_authorizations_url)}")
  end

  def create
    redirect_to(:action => :callback)
  end

  # oauth callback
  def callback
    unless session[:request_token] && session[:request_secret] 
      flash[:error] = "No authentication information was found in the session. Please try again."
      redirect_to(root_url)
      return
    end

   unless params[:oauth_token].blank? || session[:request_token] ==  params[:oauth_token]
     flash[:error] = "Authentication information does not match session information. Please try again."
     redirect_to(root_url)
     return
   end

    @request_token = OAuth::RequestToken.new(GoogleAuth.consumer, session[:request_token], session[:request_secret])
    @access_token = @request_token.get_access_token(:oauth_verifier => params[:oauth_verifier])
    
    session[:request_token] = nil
    session[:request_secret] = nil

    current_user.update_attributes(:access_token => @access_token.token, :access_secret => @access_token.secret)

    flash[:notice] = "You have successfully linked your Google Analytics account."
    redirect_to(account_path)

  rescue Net::HTTPServerException => e
    case e.message
    when '401 "Unauthorized"'
      flash[:error] = "This authentication request is no longer valid. Please try again."
    else
      flash[:error] = "There was a problem trying to authenticate you. Please try again."
    end 

    redirect_to(:action => 'new')
  end
  
  def destroy
    current_user.update_attributes(:access_token => nil, :access_secret => nil)
    flash[:notice] = "You have successfully removed your Google Analytics account link."
    redirect_to(account_url)
  end
end

And example routes:

ActionController::Routing::Routes.draw do |map|
  map.resource :authorizations, :collection => { :callback => :any }
  …
end

At this point we’re all set up to use Google Analytics data in our application. So we should probably do something interesting with it. Fortunately, Tony Pitale has made interfacing with GA itself – that is, once OAuth connection has been established – insanely easy with his Garb gem. To use Garb we can simply create a new Garb session object using a previously established OAuth access token (I personally like to wrap this up into a method on the User model):

session = Garb::Session.new
session.access_token = OAuth::AccessToken.new(GoogleAuth.consumer, user.access_token, user.access_secret)

And then we can retrieve account and profile data from Garb…

accounts = Garb::Account.all(session)
profile_id = accounts.first.profiles.first.id

In addition to accessing raw account info, stat values, etc, you can create some pretty elaborate “reports” in Garb. A sample report might look like:

class PageViewReport
  extend Garb::Resource

  metrics :unique_pageviews
  dimensions :page_path

  filters { eql(:page_path, '/') }
  filters { eql(:page_path, '/blog') }
end

report = PageViewReport.results(profile_id, :session => session,:start_date => 10.days.ago, :end_date => Time.now)
report[0].page_path
# => '/blog'
report[0].unique_pageviews
# => 500

Tony does a much better job explaining how reports work, and what you can do with them. I’d suggest checking out his Wiki, GitHub repository, or the series of posts over at Viget Extend for more information on Garb.

To present this data to a user, we can now toss together some pretty graphs and charts using my favorite JavaScript graphing library, raphael.js. This is a good way to visualize browser marketshare data, page access trends, and other analytical data that they might want to drill down on.

How to Accept Self-Signed SSL Certificates in Selenium

2010-04-13T00:00:00-04:00

Accessing pages with self-signed SSL certificates in Selenium can be a bit tricky. The core of the issue is every major browser raises a security issue when accessing a page using a self-signed SSL certificate. Since this security check takes effect before the Selenium Javascript can execute, there’s no way to instruct the browser to accept the certificate.

Selenium as a Proxy Server

As it turns out, there is a solution built into Selenium, but there’s some work involved to get it going properly. Unless you use the -avoidProxy argument when you start up the Selenium server, it will serve as a proxy for connections originating on localhost. If you start the Selenium server with the -trustAllSSLCertificates argument the proxy will be able to handle any type of SSL certificate issue for any site. The corollary to using Selenium server as a proxy is that your browser sessions need to be configured to use the Selenium server as a proxy. It should be noted that this is not the same as proxy injection mode; Selenium is not injecting itself into your page, it’s simply proxying the content for you.

Configuring the Selenium Browser Launchers

You configure the browser’s proxy settings through the browser launcher. Some of the provided launchers do this already. For Internet Explorer you can use the *iexploreproxy launcher and for Firefox you can use the *firefoxproxy launcher. Another option is to use a custom browser launcher. We do this with both Firefox and Internet Explorer and the code can be found on our GitHub repository.

We found the default chrome launcher for Firefox is very close to what we want and didn’t want to have to manage our own custom profiles. So we use that launcher as a base and modified it such that if Selenium server is started with -trustAllSSLCertificates option then then generated Firefox profile will set up the Selenium server as the proxy. The patch actually modifies the *firefox launcher, but a custom launcher is likely a safer option for you.

The reason we use a custom launcher with Internet Explorer is that we’ve experienced reliability issues with the *iexploreproxy launcher. We’ve found *iexplore to be more reliable so our custom launcher simply configures Internet Explorer to use Selenium as its proxy server and then uses HTA to perform the browser automation functions, similar to how *iexplore operates. The code for this launcher is also available in our GitHub repository.

An extra step is required for proxying SSL content with Internet Explorer. Selenium bundles a special certification authority (CA) certificate that is used to trust all the other SSL certificates. Windows will not trust this CA certificate until it is installed in the trusted root store.

Installing the CyberVillains Certificate on Windows

The following figures show the steps necessary to install the CyberVillains certificate on Windows. This is only necessary if you’re using Internet Explorer. Selenium is able to manipulate Firefox to trust the certificate with its custom-built profile.

The CyberVillains certificate is bundled in the most recent Selenium RC releases. If you download the distribution and extract it, you should be able to get going starting with Figure 1. In the explorer address bar in Figure 1 you can see where to find the certificate. You must substitute the base path (C:\, here) with wherever you extracted the files.

Figure 1: Double-click the CyberVillains certificate in the selenium server distribution.

Figure 2: Install the CyberVillains certificate.

Figure 3: Click through the SSL certificate import wizard.

Figure 4: Choose the Trusted Root Certification Authorities certificate store.

Figure 5: Complete the import.

Figure 6: Accept the security warning.

Figure 7: Wrap everything up.

At this stage everything is set up and you should be able to run your Selenium server with the -trustAllSSLCertificates argument without any problems.

Conclusion

To recap, accepting self-signed certificates in Selenium is a two-staged process. You need to instruct the Selenium server to trust all SSL certificates and then tell the browser to use the Selenium server as a proxy. You may find that creating a custom browser launcher is the best way to configure your browser’s proxy settings. After doing this, you’ll be able to test sites running on domains with self-signed and invalid SSL certificates.

Selenium Grid News

2010-03-31T00:00:00-04:00

It’s no secret that we’re big fans of the Selenium project. Selenium’s screen capture facilities provide a key bit of the infrastructure that Mogotest runs on. And, since we’re distributing those jobs to our cloud-based render agents, we need a way to manage that, too. That’s where Selenium Grid comes into play.

Selenium Grid is basically a framework for distributing Selenium tasks across many hosts (in our case, via our EC2 render cluster). This allows us to run our tests in parallel and deliver results to users as quickly and efficiently as possible.

Grid was originally developed and maintained by Phillippe Hanrigou of ThoughtWorks and has received contributions from numerous people, including Mogo co-founder Kevin Menard. Earlier this month, Phillippe announced that he would be stepping down from the project and was looking for a new maintainer. Kevin was one of the folks who was nominated to fill his shoes, and it now looks like he’s officially got the gig.

What does this announcement mean for Selenium Grid? It means that the project will have an active maintainer who (despite his obvious personality flaws) has a serious commitment to pushing the project codebase forward, fixing bugs, and fostering community involvement – things that every mature open source project needs. Good stuff! And it’s good for us, too. For exactly the same reasons.

So congratulations on the new responsibility, Kevin. I’m definitely looking forward to seeing how Se Grid evolves under the new project leadership.

Web Development is Hard Work

2010-03-10T00:00:00-05:00

Web development can really suck. If you’re building websites or applications that run on the Web, you’re juggling a lot of different technologies: HTML, CSS, JavaScript, one or more of a dozen different server-side languages, database administration, blah blah blah.

And, because you’re a good developer – the kind who wants their work to look and behave consistently for your users – you want to test as much of this stuff as possible. Unit tests make a huge difference in the way we write and refactor code, but testing the actual end user experience… that’s harder to do.

As web developers ourselves, we’re acutely aware of some common pain points. One of the biggest ones is cross-browser testing. Doing it by hand is downright painful, and requires a veritable QA lab of different platforms and browsers. And different versions of browsers. And service patches. And updates. That can be a lot of work, especially if you’re a small shop, overworked, or simply have better things to deal with.

We’re building Mogotest to make testing your websites and Web-based applications easier. So you can spend less time checking your site in different browsers and more time concentrating on the features that make your project truly unique.

What we do right now:

Cross-browser render testing
View comparison tools
Validation and verification

Where we want to get to (the cool stuff):

Full-on view-level regression testing
Providing tools that make it easier for you to detect and fix common rendering problems

The service is in private alpha right now. But if you’re interested, please sign up and we’ll get you an invitation as soon as more are available.

Thanks for your interest in Mogo!

*back to work*