We had the following objectives in mind:

• Since, we run the tests in firefox, the test process should be capable of creating an appropriate profile with required locale settings
• The tests should be generic enough so that they don’t have to be written and maintained separately for every locale

Setup

The first step involves creating a new browser profile and switching the locale (italian in this case) and passing it on to the Capybara driver.

1
2
3
4
5
6
7
8
9
10
11

Capybara.default_driver = :selenium

it_profile = Selenium::WebDriver::Firefox::Profile.new
it_profile['intl.accept_languages'] = 'it' # Italian Locale

Capybara.register_driver :selenium do |app|
  Capybara::Selenium::Driver.new(app, :browser => :firefox,
                                      :profile => it_profile)
end

Capybara.current_session.driver.browser.capabilities.firefox_profile = it_profile

The advantage of creating a profile on-the-fly is that the tests do not require every machine to have a pre-configured firefox profile that has the necessary settings.

Using translation keys in step definitions

Inside step definition, instead of hard-coding text, use the translation keys directly. This is a good practice even when writing tests for a single locale as your tests are not affected by textual changes.

1
2
3

When /^I click on the stock alert button$/ do |arg|
  find(button, text: t('stock_tab.alert_caption')).click
end

When it’s a step with text as parameter, use the key directly in the feature. Yes, this is not so readable as using the actual text but using sensible translation keys will take it one step closer to using actual text.

1
2

Given I am logged in as administrator
And I click on t(settings)

The key passed as parameter can be used with I18n directly.

1
2
3

And /I click on t\(:?([^\)]*)\)/ do |key|
  find(button, text: t(key)).click
end

• What is Unix Copy-on-write (COW)
• Why is current version of ruby (1.x.x) not COW friendly
• How does the GC packaged with Ruby 2.0 fix that

Holy COW

The fork functionality in Unix systems uses an optimization strategy where memory is shared between the parent and the child processes. The shared memory is maintained till either the parent or one of the children modify their copy of the resource. At that point, a true private copy is created to prevent the changes from being visible to other processes. The primary advantage is that if any of the processes do not make any modifications, no private copy needs to be ever created. This is called Copy-on-write (COW) technique.

1
2
3
4
5
6
7
8
9

shared_array = [1,2,3,4,5]

if fork
  #will be executed by parent process
  parent_array = [6,7,8]
else
  #will be executed by child process
  child_array = [9,10,11]
end

If you are not familiar with Ruby fork, a call to fork creates a new process. In the above example, the code inside the if block will be executed by the parent process and else block will be executed by the child process (In the child process fork returns nil). A snapshot of how resources will be shared in memory is shown below.

As can be seen above, the shared_array is maintained as a common resource between parent and child processes. When shared_array is modified by the child process, as copy-on-write implies, a private copy is created so that it is not visible to the other process. This is illustrated below.

1
2
3
4
5
6
7
8
9
10

shared_array = [1,2,3,4,5]

if fork
  #will be executed by parent process
  parent_array = [6,7,8]
else
  #will be executed by child process
  shared_array << 5
  child_array = [9,10,11]
end

Mark and Sweep objects

The advantages of Copy-on-write cannot be leveraged between multiple ruby processes due to the inherent nature of the way the garbage collector works in Ruby. The garbage collector uses a simple Mark and Sweep algorithm to identify unused objects. In simplest terms

Each object in memory has a flag (typically a single bit) reserved for the garbage collector. Starting from the root-set, all objects that can be accessed are recursively traversed and marked as being ‘in-use’. At the end of the cycle, the GC sweeps all objects that have not been marked and restores free space for future objects.

The way ruby creates objects, the GC flag or reserved bit is stored in the object itself. So, as you would have guessed by now, when the GC runs in one of the processes, the GC flag would be modified in all the objects even if they are present in the shared pool. Now, the OS would sense this and trigger a copy-on-write making private copies of the objects in each child’s memory space.

This is the reason why Ruby 1.8 or 1.9 is not COW friendly :(

GC Bit and objects should keep distance

The most sensible thing to do would be to pull out the GC flag (it’s actually called FL_MARK bit) from objects and maintain them separately. And this is exactly what Ruby 2.0 claims to do.

For each heap allocated by ruby, there is a corresponding bitmap which is linked to the header of the heap. The bitmap can store 0 or 1 values effectively replacing the GC bit which was previously stored inside the objects present in the heap. This technique is called bitmap marking. So in effect, the Mark step will not modify any live objects in the heap. Only the bitmap will be changed. Hence, the shared objects can remain that way until one of the processes actually modified them.

References:

1. Narihiro Nakamura’s patch for Bitmap Marking GC
2. Pat Shaughnessy’s excellent blog post

1. Job Control

Job control allows you to interact with background jobs, suspend foreground jobs and manage multiple jobs in a single session. Also, when you later realize that the currently running command must be pushed to the background, job control makes it easy to do that.

• Start running the process using the command ruby data_cruncher.rb
• Press ctrl+z to suspend the process
• type bg to resume it in the background

That’s it. The job will run in the background. Note that using bg command later is the same as running the parent process with & at the end like this: ruby data_cruncher.rb &.

You can verify active jobs by using the jobs command. The -l option will include the process ID.

1
2

$ jobs -l
1]  + 13701 running    ruby data_cruncher.rb

Note that the number 1 at the beginning is the jobspec which is a number allocated to each job. 13701 is the process ID. The jobspec is useful when there are multiple active jobs. The significance of jobspec is illustrated below.

$ jobs -l 1] + 13701 running ruby data_cruncher.rb 2] + 14312 running tail -f data_cruncher.log $ fg %1 [1] + 13701 continued ruby data_cruncher.rb

fg is used to pop out a process to the foreground. fg and bg commands can identify specific jobs with %<jobspec>. If you are using bash shell, before you logout, you will have to fire another command disown -h to make sure the process is not killed when the terminal session ends. -h is used to ignore HUP signal. For more information on HUP, see next section.

1
2
3
4
5
6

$ jobs -l
1]  + 13701 running    ruby data_cruncher.rb
2]  + 14312 running    tail -f data_cruncher.log
$ disown %1
$ jobs -l
1]  + 14312 running    tail -f data_cruncher.log

What disown actually does is mark the job so that SIGHUP signal is not sent to the job when the parent receives it. To explain SIGHUP, let’s dive into the next technique.

2. nohup to the rescue

To give some context, inter-process communication happens in linux using signals. When a signal is sent, the OS interrupts the normal flow of execution of the process. The most important signals relevant to this blog post are described below. For an exhaustive list, see here

SIGINT (CTRL+C) – Sent to a process from controlling terminal when user wishes to interrupt the process

SIGTSTP (CTRL+Z) – Sent to a process from controlling terminal to request it to stop temporarily

SIGHUP – Sent to a process when controlling terminal is closed

When a user logs out of a terminal session, the HUP signal is used to warn all dependent processes of the logout action. nohup command is used to ignore the HUP signal for a specific process.

1

$ nohup ruby data_cruncher.rb &

This will cause the script to run in the background and will pay no heed to HUP signal, if received. By default, STDOUT and STDERR will be redirected to nohup.out. To explicitly mention the output log file,

1

$ nohup ruby data_cruncher.rb > data_cruncher.log 2>&1 &

What does 2>&1 mean? 1 and 2 are the standard file descriptors for STDOUT and STDERR. We redirect STDERR to STDOUT. Why the ampersand? 1 instead of &1 would indicate a regular file by the name 1 and not the STDOUT file descriptor.

3. Screen your commands

Screen is the most sophisticated way to run background jobs when you want to continously monitor their activity. According to wikipedia,

GNU Screen is a software application that can be used to multiplex several virtual consoles, allowing a user to access multiple separate terminal sessions inside a single terminal window or remote terminal session.

What follows is a smash-course of Screen:

• Start screen by typing screen. You will be greeted with a welcome message.
• Every program under screen runs in a virtual window. The windows are numbered from 0.
• You are currently in window 0. To start another window, press ctrl+A+C
• Now you have two windows 0 and 1. There are several ways to switch between them. We will use ctrl+A+“
• You should see all the windows that are active. Choose the one you want.

Now, this is where things get interesting. Now press ctrl+A+D You will be detached from current screen session and back to the bash shell. Note that the session has not been killed and all the programs that you started running with screen will be still active.

To reattach, type screen -r Press ctrl+A+“ and you see that nothing has been lost. One obvious perk of screen that job control or nohup does not offer is that you always have easy and immediate access to the process that you were running in the background.

Screen is an awesome utility and this page has more valuable info.