Crowd Interactive Tech Blog

Android Builds on Travis CI with Maven

2013-05-09T16:11:55Z

The Android SDK includes a flavour of JUnit that supports unit tests and instrumentation tests. This is sufficient if you are coding alone, but working in teams is a different matter. If you want to maintain a healthy master branch of your project whilst working in a fast paced and agile development process, you need tools that automate your application builds and test each change that is introduced in the project.

We rely on Travis CI to build and test web applications. Hence, we wanted to harness our current CI service for Android applications as well.

After a quick web search, we found this great blog post that helped us to make some configuration experiments with Maven. This was a nice starting point, but we felt some details were left apart, specially if you have no experience with Maven.

Therefore, we prepared this post as a step-by-step guide to get you started quickly with Maven and explain some changes to the Android application structure.

Requirements

Basically, you need to do 3 things:

Change the structure of your Android Project.
The building tools (Maven, Android Maven Plugin, Android SDK).
Configure a 'travis.yml' file to make it work on Travis CI.

Step 1: Application structure

You have to change the directory structure of your Android application to comply with the structure shown below.

 Project-parent-folder:
   -Android-Application-folder
      -pom.xml
   -Android-Tests-folder
      -pom.xml
   -Android-library-project-folder
      -pom.xml
   -pom.xml

NOTES:

We assume you are using the Eclipse IDE.
If you already have the project in your Eclipse Workspace, you will need to delete it from your workspace and re-import it again (after you change the structure).

Step 2: MAVEN

Download Maven and extract the distribution archive to the directory in which you want to install Maven (e.g. apache-maven-3.0.5-bin.tar.gz).

Supossing you extracted the archive on your $HOME directory, then run on your terminal:

export M2_HOME=$HOME/apache-maven-3.0.5
export M2=$M2_HOME/bin
export PATH=$PATH:$M2
source ~/.bashrc

To verify that it is correctly installed:

mvn --version

Step 3: Eclipse Maven Android Plugin

You have to install a plugin to make the Android project work with Maven within Eclipse.

In Eclipse, go to:

"Help" > "Eclipse Marketplace"

If you are using Eclipse ADT or do not have the Eclipse Marketplace, install it from:

"Help" > "Install new Software" > "Switch to the Juno Repository" >
"General Purpose Tools" > "Marketplace Client"

Once in Eclipse MarketPlace:

Search for: android m2e
Install Android Configurator for M2E

Step 4: Eclipse Maven Android Project

To make Eclipse detect the project dependencies, you have to convert the Android and dependent Android Library projects to Maven:

Right click on each project:
- "Maven" > "Convert to Maven project"
Right click on your Android project,
- "Maven" > "Update Project"
- Select the related projects and accept the changes.

This should get rid of any Eclipse error about some dependencies not found.

Also, you have to configure the 'pom.xml' files for each project. You can adapt these samples according to your project:

Step 5: Install Maven Android SDK Deployer

If you want to avoid the pain of configuring the Android SDKs in your local Maven repositories, this tool is a must.

Once you run this on your terminal, you will have all the Android SDK available as Maven dependencies.

git clone https://github.com/mosabua/maven-android-sdk-deployer.git
cd android-sdk-deployer
mvn install

Step 6: Install External Android Library Projects

If you are using any Android library project, you have to perform some additional steps:

Compress the "library" folder of each library project and change the extension of the file to '.apklib'
The .apklib files should be put in your Application 'libs' folder (e.g. AndroidProject/libs)
Install each library on your local maven repository, as follows:

Run on your terminal, from your Android Application Project's root folder:

mvn install:install-file \
-Dfile=libs/your-library-file-1.apklib \
-DgroupId=com.your-library-package \
-DartifactId=your-library-artifact-id \
-Dversion=1 \
-Dpackaging=apklib

Step 7: Installing and deploying the application

These commands are for building the application and deploying to a device.

mvn clean install
mvn android:deploy

Step 8: Running Tests

On the Emulator:

mvn clean install -Pintegration-tests -Dandroid.device=your_emulator_name

on the Device:

mvn clean install -Pintegration-tests -Dandroid.device=usb

Step 9: Travis CI configuration

If everything works as it should in your local environment, then, all you need is to add a '.travis.yml' file to your application.

This is an example of the configuration we are using:

language: java
jdk: oraclejdk7
env:
 matrix:
   # android-16 is always included
   - ANDROID_SDKS=sysimg-16           ANDROID_TARGET=android-16  ANDROID_ABI=armeabi-v7a
   - ANDROID_SDKS=android-17,sysimg-17 ANDROID_TARGET=android-17  ANDROID_ABI=armeabi-v7a
before_install:
  # Install base Android SDK
  - sudo apt-get update -qq
  - if [ `uname -m` = x86_64 ]; then sudo apt-get install -qq --force-yes libgd2-xpm ia32-libs ia32-libs-multiarch > /dev/null; fi
  - wget http://dl.google.com/android/android-sdk_r21.1-linux.tgz
  - tar xzf android-sdk_r21.1-linux.tgz
  - export ANDROID_HOME=$PWD/android-sdk-linux
  - export PATH=${PATH}:${ANDROID_HOME}/tools:${ANDROID_HOME}/platform-tools

  # Install required components.
  # For a full list, run `android list sdk -a --extended`
  # Note that sysimg-16 downloads the ARM, x86 and MIPS images (we should optimize this).
  # Other relevant API's:
  #  addon-google_apis-google-16
  - android update sdk --filter platform-tools,android-16,extra-android-support,66,$ANDROID_SDKS --no-ui --force > /dev/null

  # Create and start emulator
  - echo no | android create avd --force -n test -t $ANDROID_TARGET --abi $ANDROID_ABI
  - emulator -avd test -no-skin -no-audio -no-window &

before_script:
  - chmod +x wait_for_emulator
  - ./wait_for_emulator

script: mvn install -Pintegration-tests -Dandroid.device=test

Also, note that we use the 'wait-for-emulator' script:

#!/bin/bash

bootanim=""
failcounter=0
until [[ "$bootanim" =~ "stopped" ]]; do
   bootanim=`adb -e shell getprop init.svc.bootanim 2>&1`
   echo "$bootanim"
   if [[ "$bootanim" =~ "not found" ]]; then
      let "failcounter += 1"
      if [[ $failcounter -gt 3 ]]; then
        echo "Failed to start emulator"
        exit 1
      fi
   fi
   sleep 1
done
echo "Done"

Both the Travis configuration file and Wait-for-emulator script were adapted from this Maven Android Example.

Finally

Push some changes to your Github repository (you need to have the Travis hook activated) and you should see that Travis builds and tests your Android application. Nice!

References

Check out MagmaConf, our very own international conference for Ruby specialists, coming soon in Mexico!

Bye Google Reader

2013-03-19T05:23:50Z

A couple of days ago I got up in the morning and, as usual, my laptop was on, I sat comfortably in front of my computer and headed to the one I visit first, Google Reader, but:

That box appeared over my precious feeds, my dismay was great when I received the bitter news from the hands of Google, the company I trust my personal information, my digital life. That menacing box marked "Google Reader will not be available after July 1, 2013" and a simple blue button with no other choice but "OK", I clicked the button and the box disappeared, but that feeling of discomfort and uncertainty did not.

Back in 2005

Doing some research, it didn't take long to learn from Google's official blog, that as part of a Second Spring Cleaning, Google Reader (among other products) will be gone by the first of July this year.

It is a rare feeling, like being kicked out of your own house, where you spend a lot of time, and you have grown accustomed to, I had started using Google Reader shortly after its launch in 2005, before that I used another service called Rojo, which offered functionality comparable to Google Reader. At that time Rojo was the best (in my humble opinion) way to read feeds in 2006. But when the Google app hit the market , of course I had to try it. I loved it and I stayed with it for more than six years, watching it evolve into the product it became to be. But in recent months, it seemed lacking in features... maybe that was a sign of what laid ahead, I did not see it coming.

Alternatives

So I started reviewing my options to continue reading my feeds. I thought of finding a desktop application, but none of the ones I've tried ever convinced me. The idea of having all my feeds in a single place, available and synced to use in all of my devices, was something I was never ready to give up. Many sites have listed alternative to Google Reader, but the one from lifehacker stands over the rest.

Abandon Ship

So, I started packing up and getting ready to move one. The first step before leaving Google Reader is taking with you your highly valued feed list, with that would be enough, but if you are worried about your followers, people you follow, favorites, among other things , Google offers a service that seems effective, this is called Google Takeout, an gets your "luggage" ready for you. It can be found withing your Reader settings, in the import tab / export.

Google Takeout generates a zip file with the selected services, in this scenario with the Google Reader files, among the files you can find in the package we have:

Followers
Followed
Liked
Notes
Shared
Shared by Followers
Starred
Subscriptions

All in standard formats such as JSON and XML. And well, with suitcases in hand, it's time to choose a new home.

Feedly

Many authors agree, and rightly so, that Feedly is one of the best, or the best alternative to Reader in the current app market, but in my opinion it is a little overloaded with eye candy, but it works well enough. The navigation is through shortcuts like Reader, and it has different layouts for different content you may have in your feeds.

Currently, Feedly lets you import all your feeds from Google Reader, you just have to grant permission to a third-party application, it is not necessary to do the import of your feeds, but it's always a good idea to have your feeds backed up in an XML file.

Conclusion

I will continue using Google Reader to the last day, although I'm already using Feedly as a new reader. Sounds like a bad move and I resent Google's decision to terminate Reader, but this will give room for new products and new ideas from development enthusiasts to come to the scene.

Thanks for reading, looking forward to see your comments!

And, see you at MagmaConf!

RailsConf 2013 Schedule App, unofficial

2013-04-29T23:37:43Z

Last Friday(04/27) a few co-workers and I were reviewing the RailsConf 2013 program and I found that it was very hard to see the schedule on my phone's small screen. When I tried to create a calendar of the talks I wanted to attend, I noticed that it wasn't possible.

The RailsConf site has a fixed width, so it's not easy to read on the small screen, so I thought - "there has to be an app for this". First I thought that I should contact the RailsConf organizers and make my request, then just sit and wait for the answer. That probably would have been easier, but then I thought to myself: "Wait a minute!, I'm a web developer, I build applications everyday, I write code, I know how to do it, I know how to write responsive CSS". So I decided to build this app.

Building the app sounds like a straightforward process, but it has a couple of complications, starting with the fact that I didn't have the program data, I tried inspecting the RailsConf Site's code looking for a JSON call or something similar, but I found nothing. I started to worry about it, because I really wanted to do this, but without the data it would be useless. Then one of my friends said joking: "you could perform a screen scrapping". That's a crazy idea; but it might work, I thought.

Then I remembered a post about screen scrapping so I went to Eduardo and asked him to do the scrapping and to provide me of a JSON with the data, which he did in a few minutes.

So now I have the JSON, and I just need to write the app to show the data.

I was there, creating my rails new and I really wanted to make something useful for me and everybody, and also to make it look good. I'm tired of Twitter's Bootstrap, 'cause now the internet looks all the same; so I found this new front-end framework which came very handy Gumby 2. It's really simple to use and it's written in Sass, which I love.

I wrote this is app this past Saturday and between Airports and Airplanes, fixed a couple of bugs during my trip to Portland, OR.

Stop talking and show me the app!

just go to http://railsconfschedule.com/

Features?

It is a really simple app, it loads the data from of couple of JSONs and displays them nicely on the page.
It uses Omnioauth Authentication with Github as strategy.
You can build your day-by-day schedule. Add and Remove talks.
it is responsive, it looks beautiful in your phone, it actually looks better than the bigger screen, I think. ;)

TODO:

Show Talk description
Import to Google Calendar.

RailsConf now has an app, is it worth it?

It's totally worth it, I've build and app in just a day, I'm using it for managing my own schedule, and I have some pull request from a couple of friends who would like to help to make this better. And It was really fun.

Please Contribute

No matter if you think that the app is good or not, contribute to make it better! Fork the repo, hack a little bit and make it better!. Submit your pull request and help us with this project.

If you have never performed a pull request, I highly encourage you to go for this one, you could continue helping other projects after this, and the community will be thankful. It doesn't matter if you only fix a typo, or a security loophole... just go ahead and contribute.

Yo can find the code at https://github.com/hecbuma/railsconf/

Final words

If you identify a problem, and you have the tools and knowledge to solve it, go ahead and fix it!. You will be helping everyone. You may think that the problem is too simple to merit your contribution, but your work could be a lifesaver to the rest of us.

I hope you enjoy this app!

Fun with ActiveRecord::Observer

2013-04-09T18:27:45Z

Disclaimer

ActiveRecord::Observer will be extracted to a separated gem when Rails 4 is released, so if you are planning to upgrade to the latest Rails version and want to use ActiveRecord::Observer, you'll need to add the rails-observer gem to your Gemfile.

Using ActiveRecord::Observer is a good practice, instead of bloating your model with a lot of callbacks, moving this behavior to another class with only this responsability sounds great, but ActiveRecord::Observer has some "limitations", I quoted that because they seem like limitations, but the problem is that ActiveRecord::Observer's capabilities aren't fully documented or aren't visible, you have to dig in the source code and figure out how it works.

By default, ActiveRecord::Observer watched a limited set of callbacks defined by the ActiveRecord::Callbacks::CALLBACKS constant, which are:

From the rails source

CALLBACKS = [
  :after_initialize, :after_find, :after_touch, :before_validation, :after_validation,
  :before_save, :around_save, :after_save, :before_create, :around_create,
  :after_create, :before_update, :around_update, :after_update,
  :before_destroy, :around_destroy, :after_destroy, :after_commit, :after_rollback
]

But what if you want to add you own callbacks?, is there a way to do that? There surely is one.

Adding your own callbacks

ActiveRecord::Observer functionality is fairly easy to understand, there is a notify_observers method in the ActiveModel::Observing module that we can use to define our own callbacks with something like this:

module Notifications
  def notify_completed
    notify_observers :after_complete
  end
end

class Action < ActiveRecord::Base
  include Notifications

  def complete!
    ...
    notify_completed
  end
end

class ActionObserver < ActiveRecord::Observer
  def after_complete(action)
    # your callback logic here
    ...
  end
end

And that's it, you'll have the after_complete callback available in your observer class, this method will receive the model instance where the callback is triggered as an argument.

But, what if I want to know more about the event that triggered the callback, not just the instance where it ocurred? Maybe I want to find out more about the user that triggered the action, is there a way to do that?, well, try this.

Sending more arguments in the callback

If you look in the notify_observers source, you'll see that it only receives one argument at a time, but you can see that this method uses a class method with the same name but this one accepts many arguments.

In the original notify_observers they always use self as the second argument, we are going to change that with another object, I'll use a hash, but you can wrap your arguments into a class, the important thing here is that the second argument needs to be an object.

module Notifications
  def notify_completed(user)
    self.class.notify_observers :after_complete, { action: self, user: user }
  end
end

class Action < ActiveRecord::Base
  include Notifications

  def complete!(user)
    ...
    notify_completed(user)
  end
end

class YourModelObserver < ActiveRecord::Observer
  def after_complete(event)
    # your callback logic here
        # The event argument will be a Hash with action and user keys.
    ...
  end
end

And that's it.

There are many things that Rails can do for you, but you'll need to go beyond the docs, diving deep in the source is the best way to find those things.

Hope you find this useful, thanks for reading!

Introducing Triggerino, an Arduino based action trigger

2013-04-09T18:33:04Z

First of all, I need to say that I was fully inspired by this blogpost, I liked so much that project, that I decided to fork this project and use Ruby instead of Python.

A little bit of history

One of the things I'm passionated about is the DIY philosophy, when I was young I used to spend my time doing experiments with electronics, but then I grew up and I replaced that passion with programming.

In the past RubyConf there was a talk from Austin Vance that involved Ruby and Arduino, I must say that was very interesting, that talk woke up my passion with electronics, and the best of all, it mixes both passions.

That day at the end of the conferences, I went to the closest Radioshack® and I bought my Arduino starter kit, and since then I've been experimenting with my Arduino and Ruby, with the time those experiments became Triggerino.

What is Triggerino?

The main idea of this is to have a box with a big red button, and have the ability of run tasks from my ruby project pushing that button. Cool, isn't it?

What can I do with Triggerino?

Run your test suite
Deploy your project
Run rake tasks
...
Run whatever you want

Installation

Add these lines to your application's Gemfile:

gem 'triggerino'
gem 'dino',
  git://github.com/supherman/dino.git',
  branch: 'lcdv3'

And then execute:

bundle

Usage

Create a "triggerino" dir
```
mkdir triggerino
```

Define your action classes

touch triggerino/my_action.rb

class MyAction
  include Triggerino::Action

  def perform
    # puts your action logic here
  end

  def to_s
    'A descriptive name for your action'
  end
end

Wire your arduino like this:

Also add 2 push buttons: one will be the action selector and it needs to be connected to the pin 10, and a the other one will be the trigger and it needs to be connected to the pin 9.

Run the listener
```
bundle exec triggerino
```

Need to know more about Arduino and Ruby?

Keep and eye on the dino gem, people started to contribute on the project and they are doing amazing things that will make work with Arduino and Ruby easier.

And If you like robots and electronic stuff, you'll want to take a look at Artoo, a framework that will help you build robots with Ruby, it has support for Arduino, Parrot AR Drone 2.0, Sphero and Roomba.

Also, if you are a dev with an itch to learn more about this kind of topics, I think you should go to MagmaConf, there will be 2 talks about electronics and Ruby; so it will be a good chance to learn something new.

And that's it for now, hope you find this useful and interesting. ;)

Thanks for reading.

Templating with Handlebars

2013-03-14T16:55:17Z

Templating with Handlebars

Remember that time you started creating your own app for no other reason than just learning, experimenting and having fun?

It started out as a super simple single-task application. At the beginning you were pretty happy with it, but not much later you realized it was useless, so you decided to extend it a little bit.

If you are anything like me you probably used a single view (because you're cool or maybe you were using Backbone) and started putting tons of ajax calls everywhere to load dynamic content. It didn't take long until your code turned into a complete mess (with tons of html within your Javascript.. yuck! ). You probably thought that there must be a better way to do it.. and you were right. Templates are exactly what you need.

If you do a little research about templating engines for javascript, you will find that there are a lot of them. So which one should you choose?

Well honestly you could choose any and it will do the work. But if you're looking for a simple, powerful and fast engine that can work in both simple and complex apps, I will recommend you to try Handlebars.

Why Handlebars?

It uses simple syntax
It is Easy to use
It's fast!

Handlebars' 'logic-less' syntax is incredible. It is really simple and easy to read, and it even looks cool. Being easy to read makes it really easy to use too and since the templates can be pre-compiled, Handlebars can go from fast to super-fast.

Take a look at this minimalistic example:

We have a very basic document.

You can notice that the document is completely empty. There are just a few lines in the script section (inside the head). In the first two lines we're requiring Jquery and Handlebars, and in the third one we're defining our Handlebars template.

If you save this file and open it on your browser you'd see, well.. nothing! We did define our template but we're not using it anywhere.

Opening the browser's console and running this lines would give us a different result though.

var source   = $("#person-template").html();
var template = Handlebars.compile(source);
var data = {name: "ivan", email: "iwano@hotmail.com", city:"Colima"};
var html    = template(data);
$('body').append(html);

Congrats! you just created your first Handlebars template :D

Pretty much all we did was getting the template we defined in the head of our document, compiling it, feeding it with an object to be evaluated, and then appending the output HTML to the body of the document.

It is no coincidence that the attributes of the object we send to the template match the ones defined in it. If our object would have had only a name attribute:

var context = {name: "ivan"};

Handlebars would have evaluated only the name attribute and returned nothing for the missing email and city ones:

<div class="person">
  <h1>ivan</h1>
  <h2></h2>
  <h3></h3>
</div>

In Handlebars you put whatever you want to be evaluated inside 2 sets of keys {{ expression }}.

Blocks

You can also use blocks in your Handlebars templates. Let's say we have the following object:

var data = { 
  persons:
    [{
       name: 'ivan',
       email: 'iwano@hotmail.com',
       city: 'Colima'
    },
    {
       name: 'Michael',
       email: 'mscott@dunder.com',
       city: 'Scranton'
    }]
}

And you define your template like this:

{{#persons}}
   <div class="person">
      <h1>{{name}}</h1>
      <h2>{{email}}</h2>
      <h3>{{city}}</h3>
   </div>
{{/persons}}

We would get an output like this:

<div class="person">
  <h1>ivan</h1>
  <h2>iwano@hotmail.com</h2>
  <h3>Colima</h3>
</div>
<div class="person">
  <h1>Michael</h1>
  <h2>mscott@dunder.com</h2>
  <h3>Scranton</h3>
</div>

With this we can iterate over each of the elements in our collection changing the context towards each of them.

If our object would happened not to be a collection, this would still work printing the only one item.

We could have used this as well:

{{#each persons}}
   .
   .
{{/each}}

Some other blocks you can use:

if

{{#if email}}
   .
   .
{{/if}}

unless

{{#unless email}}
   .
   .
{{/unless}}

This blocks do exactly what you'd expect.

Notice that all blocks have an opening and a closing tag.

Blocks

You can even create your own helpers and use them whenever you want:

Handlebars.registerHelper('instroduceYourself', function(person) {
  return "Hi my name is " + person.name + " and i'm from " + person.city + "!";
});

To call your brand new helper from inside your template you would do something like this:

<div class='introduction'>
  {{instroduceYourself person}}
</div>

So you would get something like:

<div class='introduction'>
  Hi my name is ivan and i'm from Colima!
</div>

You could define the helper whenever you want or even in a separate file (just make sure to include it after including Handlebars!).

Handlebars HTML-escapes all outputs by default, but you can always change that by using {{{ expression }}} (3 keys), in case (for some reason) you have something like:

{
    name: '<h1>ivan</h1>',
    email: '<a href='mailto:iwano@hotmail.com >ivan</a>',
    city: '<h2>Colima</h2>'
 }

Handlebars in a Rails/Backbone app

If you'd like to use handlebars in your rails/backbone app follow these simple steps:

in your Gemfile

gem 'handlebars_assets'

From your console run:

bundle install

In your application.js

//= require handlebars.runtime

And that's it, you're ready to create your templates with Handlebars. Just make sure to let your views know that you'll be using Handlebars by putting 'HandlebarsAssets' in your template attribute:

template : HandlebarsTemplates['backbone/templates/your_template']

Wrapping up

Templates: +1
Handlebars: +2
html-escaped: {{ expression }}
html-unescaped: {{{ expression }}}
helpers: {{ helper_name parameters}}
blocks: {{#blockname}} . {{/blockname}}

If you'd like to read more about Handlebars what better place than the official Handlebars' site.

And if you haven't got your ticket for MagmaConf 2013 (if you have it you rock!) what are you waiting for!

Thanks for reading!

Using Kaminari

2013-02-20T18:43:37Z

Hello there,

Today I’d like to talk about pagination in Rails. When I started to learn RoR I did a simple application that included some products "to sell", with description, stock, price, an image of the product, etc. You could add, delete, edit and all this kind of things that we always have in this type of applications.

One of the problems that I found doing that was that if I added several different products I could get a very large list of items and see them in a scroll down list, but that was a bit tedious >.<!

After a while I was able to find a solution to the problem, and that’s what I want to show you now, how to use "kaminari".

Kaminari gives us an easy way of pagination; first you have to add it into the Gemfile.

gem 'kaminari'

Then run bundle to install it.

$ bundle

Kaminari uses a scope page for an ActiveRecord model. For example, if you have 50 items and then write this:

Product.page(5)

… you are getting nothing. Using page you can decide in what page you are going to be located in your application. Default per_page is 25 so, in this case we are trying to be in page 5, but since we only have 50 items we need to add 75 more because 125 items is the minimum amount for this. You can change this value using the per scope. Look at this example:

Product.page(4).per(2)

Which means that if you have a list with 10 items you are going to have as a result 5 pages with 2 items per page, and you are going to be located in the 4th page, but you are not going to be able to navigate to other pages because we are selecting page 4 by default. You can see what I mean here:

Fixing that, you have to modify your controller, in my case that is "products_controller.rb". I used the index action and the :page parameter to navigate in all the pages.

def index
   @products = Product.order("description").page(params[:page]).per(2)
end

We need to modify our template to use the helper method called paginate and we have to transfer the list of products that we want to paginate. So, we have:

products/index.html.erb

<H1><% title "Products" %></H1>
<table>
  <tr>
    <th>Image url</th>
    <th>Title</th>
    <th>Description</th>
    <th>Price</th>
    <th>Buy</th>
  </tr>

<%= paginate @products %>
<div id="products">
    <%= render @products %>
</div>
</table>
<p>
  Item Count: <%= @order.order_items.count %>
</p>
<p><%= link_to "Create a New Product", new_product_path, :id => "new_product" %></p>

products/_product.html.erb

<tr>
  <td><%= image_tag "/images/products/#{product.image_url}" %><td>
  <td><%= product.title %><%= product.description %></td>
  <td><%= print_price(product.price) %></td>
  <td><%= raw(print_stock(product.stock)) %></td>
  <td><%= button_to "Add to Cart", order_items_path(:product_id => product.id) %></td>
  <td><%= link_to 'Show', product %></td>
  <td><%= link_to 'Edit', edit_product_path(product) %></td>
  <td><%= link_to 'Destroy', product, :confirm => 'Are you sure?', :method => :delete %></td>
</tr>

To navigate between the pages we have "Prev" and "Next" by default, as you can see in the first image. You can change these words adding the next code in config/locales/en.yml

en:
  hello: "Hello world"
  views:
    pagination:
      previous: "&lt; Previous Previous"
      next: "Next Next &gt;"
      truncate: "..."

Now you will see the changes:

We can use kaminari for any ActiveRecord queries and it's very easy. You need to use the scope per after page and it will work like a charm.

This is a simple example to understand how to use kaminari. I hope it is useful to you.

You can find more information here.

Sources:

Stay tuned,

BuildNotify, the CCMenu for Linux

2013-02-16T19:31:15Z

Sometimes you're working in a team where everyone is modifying files, adding features or refactorizing, which makes it complicated to follow changes. That's when the TDD comes to place, ensuring the changes only modify the behaviour you want to modify, and nothing else. But it can be hard to know for sure if those changes will affect production environments due to the specific setup features that might exist in the developer's machine. That's when services like CruiseControl come in handy, to make sure that every build is successful on production.

Mac users have a cool app called CCMenu, that lets you connect the build server to a tray system icon, where everytime someone sends a build you can easily tell if something went wrong. Linux users have a very nice clone called BuildNotify.

BuildNotify, as described in their website, is a CCMenu equivalent for Linux. It resides in your system tray and notifies you of the build status for different projects on your continuous integration servers. BuildNotify is largely inspired on the awesome CCMenu available for Mac.

Features

Monitor projects on multiple CruiseControl continuous integration servers.

Access to overall continuous integration status from the system tray.

Access individual project pages through the tray menu.

Receive notifications for fixed/broken/still failing builds.

Easy access to the last build time for each project

Customize build notifications.

Ubuntu installation (see special instructions for ver. 12.10 below)

The installation is pretty straightforward

Run the following commands in a terminal

sudo add-apt-repository ppa:anay/ppa
sudo apt-get update
sudo apt-get install python-buildnotify

And done, you have installed the CCMenu for Linux

Ubuntu 12.10 installation

Sadly, there is no repository for Quantal yet. So don't follow the previous steps because they will fail. To achieve this installation in "Quantal" follow the instructions for the fix.

First add the repository:

sudo add-apt-repository ppa:anay/ppa

Then go to Software Sources

Look for the "Other Software" tab and search for the two "anay" repositories and click on Edit.

Change the Distribution from "quantal" to "precise".

When you finish with main and Source Code try the next commands. Now the system will find the repository and finish the process successfully.

sudo apt-get update
sudo apt-get install python-buildnotify

Extra Information

CCMenu is a great tool, and now thanks to Anay we have it on Linux with BuildNotify.

Remember to check if you have installed the dependencies: pyqt4, pytz and python-dateutil.

This is the official project on bitbucket, and the PPA is currently available at launchpad.

Hope you enjoy working with this as much as I do. Remember you can contact me via email, or follow me on twitter

Thanks for reading!

Upgrading a Rails application to 4.0

2013-02-20T18:51:45Z

Now that Ruby 2.0 has been released as well as the first Rails 4.0 beta its a good time to try and migrate your apps from Rails 3.2 to Rails 4.0 and get all the goodies. If you are coming from Rails 3.1, I strongly recommend that you first upgrade to Rails 3.2 before attempting the migration to 4.0.

If you still want to use Rails 4.0 edge from the Github repo, a few weeks ago I wrote a blog post on how to install Rails 4 beta which can still help you.

Your mileage will vary depending on the Gems that you application rely on. So far I have found that most of the gems that I use require a very small changes to have them work nicely with Rails 4.0, in most of the cases just modifying the gemspec file to require rails-4.0.0.beta has been enough.

As usual having a test suite can give you some peace of mind, if you don't have one, then, good luck!

Upgrading Gemfile

The first step to update your app is to modify your Gemfile to use the '4.0.0.beta1' version of Rails:

ruby '2.0.0'

gem 'rails', '4.0.0.beta1'

# Gems used only for assets and not required
# in production environments by default.
group :assets do
  gem 'sass-rails',   '~> 4.0.0.beta1'
  gem 'coffee-rails', '~> 4.0.0.beta1'

  gem 'uglifier', '>= 1.0.3'
end

gem 'activerecord-deprecated_finders'

One thing to notice, is that we are including a new gem activerecord-deprecated_finders, which is here to keep compatibility if your application uses deprecated finders like:

User.find(:all, conditions: { email: 'some@email.com' }, limit: 1)

Right now, it is not a requirement that you change your application to remove deprecated ActiveRecord finders, but, from now on you should only use the new ActiveRecord API.

We should give it a try and update our Gemfile.lock with bundler:

$ bundle update 
$ bundle --binstubs

If we are lucky enough, all other gems which are being used by our application should just work with Rails 4.0, if not, we should look around failing gem's repository, there is a good chance that there is a branch being worked for compatibility with Rails 4.0.

Application.rb

One of the biggest changes in Rails 4, is that application.rb is no longer holding general application configuration, we can leave it as it is right now or we can follow the Rails 4 pattern and extract all of our global application configuration into initializers.

All new Rails 4 applications now have the following initializers:

backtrace_silencers.rb       mime_types.rb    session_store.rb
filter_parameter_logging.rb  inflections.rb   secret_token.rb   wrap_parameters.rb

This change made application.rb become almost empty.

Another important change in Rails 4, is that we can have encrypted session cookies just by changing the session_store configuration from :cookie_store to :encryptedsessionstore, but this will invalid all previously set cookies. There is a way to keep signed cookies as valid and have new session cookies being encrypted with the use of :upgradesignaturetoencryptioncookie_store, for more details on how to use it here is nice post on the EnvyLabs blog.

Also, make sure that you remove the whiny_nils configuration from development.rb and test.rb, since it has been deprecated and does nothing on Rails 4.

Eager loading

On Rails 4 there is a new setting for our application that tells Rails to eager load all of our application classes, instead of relying on Rails autoloading. Eager loading our classes gives us a chance to have our application running in Thread Safe mode, but this does not mean in any way that our application is Thread Safe, this is true if any of our classes or a gem does changes to shared memory without a lock.

By default, on a new Rails 4 application, a setting in our production.rb will tell Rails that we want to eager load and remove Rack::Lock middleware by default.

Recomendations for this setting are as follows:

# development
config.cache_classes = false
config.eager_load = false

# test
config.cache_classes = true
config.eager_load = false

# production
config.cache_classes = true
config.eager_load = true

Commit comments for this feature gives more information about eager loading.

I strongly suggest that you test your application as much as you can, just to be sure that there is no a side effect on enabling eager load on production.

Strong parameters

Rails 4 changes the way we pass form parameters to create or update a model within a controller. This is done with the introduction of Strong Parameters, which is a way to whitelist the parameters that can be passed to a model. It differs from protected attributes, this whitelist is defined on the controller and not the model, also it's possible to define multiple whitelists for different use cases.

# Methods defined in a controller
def create
  @user = User.new create_user_params
  …
end

def update
  @user = User.find params[:id]
  @user.update_attributes update_user_params
  …
end

def create_user_params
  params.require(:user).permit(:name, :age, :admin)
end

def update_user_params
  params.require(:user).permit(:name, :age)
end

This is a big change and a deal breaker for apps being migrated over Rails 4, since this will require a lot of effort on what was already working code.

So, we have 2 options here:

Use the protected_attributes gem to keep compatibility and avoid big changes, in this case we need to create an initializer with a setting to use it:

Rails.application.config.active_record.whitelist_attributes = true

The second option is to bring the strong_parameters gem to your application before upgrading to Rails 4, on every model were you want to migrate to strong parameters you must include the ActiveModel::ForbiddenAttributesProtection module.

class User < ActiveRecord::Base
  include ActiveModel::ForbiddenAttributesProtection
end

Once you are done migrating all your models, then you can remove the inclusion of that module on your models and add an initializer to have the module included into ActiveRecord::Base class:

ActiveRecord::Base.send(:include, ActiveModel::ForbiddenAttributesProtection)

And finally also disable the white list configuration in your application.rb

config.active_record.whitelist_attributes = false

Extraction and deprecation

Rails 4 was an opportunity for the core team to extract code that was deprecated, either because its not needed or because a better pattern came up or just to simplify Rails dependencies.

So, if your application depends on Observers, Active Record Session Store, Action Caching, Page Caching then you should include in your migrated application the following gems, depending on your case:

rails-observers, in this case you will need an initializer to specify the observers configuration ( see below this list )
activerecord-session_store
actionpack-action_caching
actionpack-page_caching

Configure your observers:

 Rails.application.config.active_record.observers = :my_observer

Final steps

At this point you should be able to test your application with your test suite and, just to be safe, with the help of a Q&A team. Hopefully there will not be any big issue that prevent you from moving to Rails 4.

Even when Rails 4 is still a moving target, it could be a good time to evaluate what will be the effort of moving to this version of Rails.

If you run in any kind of issues while testing your application, please report them to the Rails issue tracker, so they can be addressed before Rails 4 see the light.

So far I only know of 2 applications that run Rails 4 in production, one of them is 37Signals' Basecamp Next and Github's gists service.

More features that shipped with Ruby 2.0

2013-03-05T01:16:48Z

I had the honor to attend Heroku Waza last week. The conference in general was amazing, however, there was a talk that (to me) was more interesting than any other.

Ruby 2.0 was released last week, and at Waza, Matz tried to give us a glimpse of all the new features that shipped with it.

Unfortunately, time was not enough and Matz only had a chance to talk in detail about the features that Akira Matsuda already described at RubyConf last year.

However, Matz did mention a few more, so, I decided to investigate a and write a little bit more about it, so, here they are.

Keyword arguments

If you come from a language like C#, this is very familiar, and its probably one of the things that you miss the most.

To do something similar in 1.9.2, you have to pass an options hash to the function where you want named parameters.

It works, however, things get ugly when you need defaults for those parameters.

It still works, but code begins to look complicated. You are doing things like merging a hash into another hash just to get the parameters of the options.

Enter Named Arguments.

As you can see, you get the same functionality with less code.

Also, you can combine them with regular parameters.

If you ask me, methods and their options are more descriptive this way. It will make Ruby code even more legible than it already is.

I can see Rails taking advantage of this all over the place.

To_h

We can now convert objects to hashes using the, very simple, .to_h that comes with some classes.

A simple example, I couldn't think of a better one right now. However, I am sure that we will see plenty of this feature being used in the wild very soon.

Symbol array literals

This one blew my mind, I loved it.

The example speaks for itself, get a list of symbols from a list of words. Beautiful.

DTrace support

I have never actually used DTrace, but, I've heard of it.

From it's Wikipedia page:

DTrace can be used to get a global overview of a running system, such as the amount of memory, CPU time, filesystem and network resources used by the active processes. It can also provide much more fine-grained information, such as a log of the arguments with which a specific function is being called, or a list of the processes accessing a specific file.

That sounds like a lot of debugging power. DTrace will become a powerful ally in the quest for memory leaks and any number of weird runtime errors.

Conclusion

Go upgrade your code to use Ruby 2.0 right now. It's not 100% backwards compatible with 1.9.x, but, it's pretty close.

So far I haven't found anything challenging when migrating apps from 1.9.

Also, I didn't mention performance. Ruby 2.0 + Rails 4 is faster. Noticeable faster. But, I'll write more about that in another post.

Cheers.

Respect the tomato

2013-02-12T22:08:34Z

In the past few months, I've been using the Pomodoro Technique® at work, specially when pair programming.

This technique is a really simple time-management tool created by Francesco Cirillo, and it takes its name from kitchen timers, those you can easily identify because they are shaped like tomatoes.

In this post, I'd like to mention the advantages we get when applying the Pomodoro Technique® while working in pairs.

First of all, I'd like to explain briefly how this technique works: the main idea is to focus your attention on a single task for 25-minute long spans, avoiding distractions in your work environment. You have to take a 5-minute break.

During the break, you are allowed to do anything you want: check your email, make phone calls, attend to other people, get some coffee, etc. The point is to change the focus on anything but the task you are working for. The 25-minute session is known as a Pomodoro, the italian word for tomato. After the break, a new Pomodoro begins, in order to continue the task or start a new one.

Once you complete four pomodori (plural for pomodoro), a longer break begins, that could last 20 or 25 minutes.

The main goal of this way of doing things is to focus fully on a single task, without forgetting that is important to take regular breaks. The outcome of using this method is a fresh and focused mind.

Pomodoro and pair programming

The Pomodoro Technique® works great with pair programming because distractions are minimized, specially when the rest of the team and coworkers know and understand this way of accomplishing goals.

Another good thing is the possibility of switching drivers allowing programmers to take different roles in each pomodoro.

Both users focus on finishing their tasks as soon as they can, because the countdown urges them to accomplish the activities planned for the sprint.

Advantages

These are the main advantages from my point of view:

Initial planning and decomposition of tasks into smaller entities.
Less anxiety about getting the work done.
Short iterations keep motivation.
Less distractions help you to stay focused.
You learn to prioritize your activities.

Conclusion

From my experience, using this technique has helped me to discipline my mind during working hours, forcing me to break activities down to smaller (easier to handle) tasks, focusing my attention and avoiding distractions, specially when pair programming, because we have to respect the time for work and the time for breaks of everyone involved in the activity. We have to respect the tomato.

I quickly adapted to the technique, even more after noticing the benefits reflected in my productivity and concentration. Now I can hardly work without it.

Distractions will never disappear completely, but they will be significantly minimized. And in case they keep stealing your time, this technique will help you get rid of them.

It is worth trying this tool if you are looking for a way to stay focused, finish your work as planned, and maximize your time.

Resources and interesting reading

How to limit two decimal digits in a jQuery input field

2012-11-27T16:09:43Z

Hello there, this time I'd like to explain how to limit two decimal digits in a jQuery input field. This is how to do it:

HTML code

  <input type="text" name="one" class="two-digits"><br>
  <input type="text" name="two" class="two-digits">

Javascript code

 // apply the two-digits behaviour to elements with 'two-digits' as their class
 $( function() {
 $('.two-digits').keyup(function(){
   if($(this).val().indexOf('.')!=-1){         
       if($(this).val().split(".")[1].length > 2){                
           if( isNaN( parseFloat( this.value ) ) ) return;
           this.value = parseFloat(this.value).toFixed(2);
       }  
    }            
    return this; //for chaining
 });
});

A tool for creating sample code

Here's an useful tool to create sample code suitable for our needs.

(http://jsfiddle.net/c4Wqn/)

In Haml and CoffeeScript

Haml code

 %input#other_amount.numeric{type: 'text', name: 'other_amount'}

Coffescript code:

//Use this function to validate an input to limit with 2 decilmals after decimal point
$.fn.twoDigits = ->
   unless $(this).val().indexOf(".") is -1
   if $(this).val().split(".")[1].length > 2
     return if isNaN(parseFloat(this.val()))
     result = parseFloat(this.val()).toFixed(2)
     this.val result
     result

How to summon from our view

 $('#other_amount').twoDigits()

Hope you found this guide useful. Shoot me up any comments you may have. Thank you for visiting our blog!

How to manage file uploads with Backbone.js, paperclip, jquery file upload and...

2013-02-11T18:38:47Z

Hello there,

A few days ago I was looking for a step-by-step tutorial on how to manage file uploads with the tools I mention in the title, and I couldn't find one, so I decided to write one myself!

In the first part of the tutorial, I show you a gallery made in Backbone plus Coffeescript, Haml, Eco templates, the jQuery file upload tool, Rails, Paperclip, and Amazon AWS.

First of all, we need to create a new Rails project:

$ rails new jqueryfileuploadbackbone
$ cd jqueryfileuploadbackbone

The Gemfile configuration should be the following:

 source 'https://rubygems.org'
# Bundle edge Rails instead:     
# gem 'rails', :git =>; 'git://github.com/rails/rails.git'

gem 'haml'
gem 'jquery-rails'
gem 'rails', '3.2.11'
gem 'paperclip', '~>; 3.4.0'
gem 'aws-sdk', '~>; 1.6.0'
gem 'aws-s3'
#gem 'mysql2'
gem 'pg'
gem 'devise'
gem 'jquery-fileupload-rails'
gem 'rb-readline'
gem 'debugger'
# Gems used only for assets and not required
# in production environments by default.
group :assets do
  gem 'backbone-on-rails'
  gem 'backbone-support'
  gem 'coffee-rails', '~>; 3.2.1'
  gem 'eco'
  gem 'haml_assets'
  gem 'sass-rails', '~>; 3.2.3'
  gem 'sass'

   # See https://github.com/sstephenson/execjs#readme for more    supported runtimes
  # gem 'therubyracer', :platforms =>; :ruby

 gem 'uglifier', '>= 1.0.3'
end

group :test, :development do
  gem 'fabrication'
  gem 'guard-rspec'
  gem 'rspec-rails'
end

#  To use ActiveModel has_secure_password
# gem 'bcrypt-ruby', '~>; 3.0.0'

# To use Jbuilder templates for JSON
# gem 'jbuilder'

# Use unicorn as the app server
# gem 'unicorn'

# Deploy with Capistrano
# gem 'capistrano'

# To use debugger
# gem 'debugger'

Now we need to install all the Gems using the next command

  $ bundle install

Now we need to create a model to save our images and additional data, I'll create one called 'qrcodes', where 'image' will be where I save the image name, which is a requirement of the Paperclip Gem.

 $ rails generate model Qrcode description:string
 $ rake db:setup

Now we need to prepare our model to upload images, for this effect just run the next command generator for Paperclip, this is where we specify our model and the attribute where we'll save our images.

   $ rails generate paperclip Qrcode image

Now we need to run all of our migrations

  $ rake db:migrate

Now comes the moment to generate our Backbone structure (models, views, routes, etc), we can use Scaffold like the following:

  $ rails g backbone:install

Now we continue with the creation of our controller and two methods to upload and show all of the images that were transferred.

  $ rails g controller qrcodes index create --skip javascript

After using the previous command, we are going to create our routes to use it, for this we're going to edit the file routes.rb located in

 config/routes.rb

We are going to declare the next two lines if they don't already exist

get 'qrcodes/index'
root to: 'qrcodes#index'

Now we need to delete the public index file, to show the root page index in the qrcodes controller, used in the previous step.

public/index.html.erb

If we have done everything correctly, we can now run our server using the rails server command so we can receive pop alerts in the address:

http://localhost:3000

Now let's modify the following file to enable our routes:

app/assets/javascripts/jqueryfileuploadbackbone.js.coffee

Adding the following code:

  window.Jqueryfileuploadbackbone =
      Models: {}
      Collections: {}
      Views: {}
      Routers: {}
      ManageImages: ->
      new Jqueryfileuploadbackbone.Routers.Qrcodes()
         alert 'in backbone'
         Backbone.history.start()

   $(document).ready ->
      Jqueryfileuploadbackbone.ManageImages()

At this point we need to verify that all works fine. If we had any issues with Javascript we need to check the application.js file. For example, I had to change the original content because I had problems with the Backbone templates and I fixed it like this:

//= require jquery
//= require jquery_ujs
//= require_self
//= require underscore
//= require backbone
//= require jqueryfileuploadbackbone
//= require_tree ../templates/
//= require_tree .//models
//= require_tree .//collections
//= require_tree .//views
//= require_tree .//routers

Now we're going to edit our routers file to show our images list, the file is:

  app/assets/javascripts/routers/qrcodes_router.js.coffee

And add the following content:

 class Jqueryfileuploadbackbone.Routers.Qrcodes extends Backbone.Router
    routes:
        '' : 'showManageImages'
    showManageImages: ->
      @collection = new Jqueryfileuploadbackbone.Collections.Qrcodes()
      @collection.fetch()
      view = new Jqueryfileuploadbackbone.Views.QrcodesIndex collection: @collection
      $('#container-app').html view.render().el

In the previous code we can see that we need to use our collection to show all the images, but it is necesary to declare first the right routes in the config/routes.rb file like the following:

 scope api do
  resources :qrcodes, format: false
end

Now we are going to use our model to fetch all the data from the qrcode table, for that we use the file:

 app/assets/javascripts/models/qrcode.js.coffee

And we are going to specify the routes that we declared in the routes file, this is what I have:

 class Jqueryfileuploadbackbone.Models.Qrcode extends Backbone.Model
   urlRoot: 'api/qrcodes'

In the same way, we are going to edit the collection file

  app/assets/javacripts/collections/qrcodes.js.coffee

Its content is:

 class Jqueryfileuploadbackbone.Collections.Qrcodes extends Backbone.Collection
   url: 'api/qrcodes'
   model: Jqueryfileuploadbackbone.Models.Qrcode

Important note: In this case I'm using Coffescript and it's of great importance to be careful with the indentation spaces at all times.

If you remember, in the Backbone router file we declared an div "container-app", this will be used to attaching our Backbone app in ROR, the view in charge of doing that is:

 app/views/qrcodes/index.html.erb

We are using haml (included in the Gemfile), and now we're going to substitute the file extension .haml for .erb

app/views/qrcodes/index.html.haml

The contents of the file are:

%h1 List of Images
#container-app

Now we are going to edit our qrcodes_index.js.coffee view

 app/assets/javascripts/views/qrcodes_index.js.coffee

As an example, we could use:

class Jqueryfileuploadbackbone.Views.QrcodesIndex extends Backbone.View
  template: JST['qrcodes/index']
  render:->
   @$el.html @template
   @

Continuing working with haml, we need to add the .haml extension to use it in our Eco template. The original file was:

 app/assets/templates/qrcodes/index.jst.eco

Now it should look like this:

 app/assets/templates/qrcodes/index.jst.eco.haml

And we need to a link:

 =link_to 'Add new Image', '#qrcode/add'

And now get to the interesting part, where we deal with file uploads. To do this we will create a new view:

 app/assets/javascripts/views/qrcodes/add_view.js.coffee

The content of the file to use the jQuery File Upload plugin in Backbone is:

 class Jqueryfileuploadbackbone.Views.QrcodesAddView extends Backbone.View
   template: JST['qrcodes/add']
   render: ->
    @$el.html @template
    @uploadQrcode()
   @

  uploadQrcode:=>
    @$el.fileupload
    add: (e, data)->
       $('#qrcode_image').hide()
       $("#fileupload-loading").html 'Cargando...'
       data.submit()


    formData: [
      name: 'authenticity_token'
     value: $("meta[name=\'csrf-token\']").attr('content')
   ]

   done: (e, data) ->
     window.location = '/'

And add template with the following content:

The template file name is:

app/assets/templates/qrcodes/add.jst.eco.haml

The content of the form to upload images is:

= form_tag 'api/qrcodes', :multipart => true, :id => 'fileupload' do
  .fields
    = file_field(:qrcode, :image)
    #fileupload-loading{style: 'height: 50px; width: 200px;'}
    %br

Now let's add our route in qrcodes_router.js.coffee to use our new view to add images

The updated content should be:

class Jqueryfileuploadbackbone.Routers.Qrcodes extends Backbone.Router
   routes:
     '' : 'showManageImages'
     'qrcode/add' : 'addNewImage'
   showManageImages: -> 
      @collection = new Jqueryfileuploadbackbone.Collections.Qrcodes()
      @collection.fetch() view = new Jqueryfileuploadbackbone.Views.QrcodesIndex collection: @collection
      $('#container-app').html view.render().el

   addNewImage : ->
     view = new Jqueryfileuploadbackbone.Views.QrcodesAddView
     $('#container-app').html view.render().el

Now we need to declare our file upload plugin:

   app/assets/javascripts/application.js

And we need to add the jquery file upload Gem (included in the Gemfile)

  //=require jquery-fileupload/basic

This the moment to configure the Paperclip gem to manage uploads in our server, first edit the file

 app/models/qrcode.rb

The last content you need to add is:

class Qrcode < ActiveRecord::Base
 attr_accessible :description, :image
 has_attached_file :image,
    :url => '/system/qrcodes/:filename'

end

Afterwards, we edit the qrcodes_controller.rb file with the following code:

 class QrcodesController < ApplicationController
   respond_to :json

   def index
  end

  def create
    respond_with Qrcode.create(params[:qrcode])
  end
end

Additionally, run all the migrations:

  $ rake db:migrate

And now we can run our server

 $ rails server

Let's test what we did. First, click in the add new image link. Then, browse for an image with the right format in the form. If the file uploads, we have a working application!

Locally, the file should be located in the public/qrcodes directory.

In another tutorial I will teach you how to list and delete images.

Hope you found this tutorial useful. You can find the repo in github at:

 https://github.com/heridev/jqueryfileuploadbackbone

See you next time!

How to get what you need with Nokogiri

2013-01-15T19:31:36Z

Recently, I was asked to automate the retrieval of information from a website that is not online anymore? Maybe you are thinking that finding such a site was the difficult part, but actually, it was the easy one.

So, how to come up with an older, no-longer-running, version of a website? Well, that's easy, you only need to pay a visit to the Internet Archive Wayback Machine at http://web.archive.org/ and find the website you are looking for in the database.

Now, getting to the interesting part: automating the data collection. To achieve this goal we can use several techniques and technologies. In this post I'm going to explain the technique known as Screen Scraping. In a nutshell, with this technique you extract data from sources that are not typically designed for this purpose. Namely, an HTML document.

Let's picture an scenario. You want to get certain data from a website, but the site doesn't have any web services to accomplish this purpose. Screen Scraping is useful for situations exactly like this.

In my case, I had to collect data from a copy of a website stored on the Internet Archive Wayback Machine database, this means there was only HTML documents as sources of data. There weren't any web services or other sources of data available. Specifically, I needed to generate an XML file with the data from all the Magma Rails 2012 speakers (http://web.archive.org/web/20120608203123/http://www.magmarails.com/). So, HTML is designed to present data, the question is: can I use the HTML source to automatically generate another piece of data? With Nokogiri (http://nokogiri.org/), we can use any website, look for the specific information we need, process the data and make any use we want of it.

Let's start with the Screen Scraping using Nokogiri. To begin, we need to install nokogiri

gem install nokogiri

The next step is to identify the HTML elements we need to access and get its information. Nokogiri allows us to match the HTML elements using the CSS selectors. I have described to you the problem and the source of data. The CSS selector that matches all the speakers from the HTML document is:

.speaker-item

Tip: You can use Selector Gadget (http://www.selectorgadget.com/) to make the identification of the needed elements easier. Selector Gadget is a very nice open-source bookmarklet.

Following with the example, now we have to create our Ruby script file. At the top of the file we need to require two gems:

require 'nokogiri'
require 'open-uri'

Obviously, Nokogiri must be included. Now we have to get the HTML from the website and assign it to an object.

html = open("http://web.archive.org/web/20120502173130/http://magmarails.com/")
document = Nokogiri::HTML(html.read)
document.encoding = 'utf-8'

As you can see, first, we opened the HTML document, after we created a Nokogiri HTML document with that, and finally we encoded the document to UTF-8. There is another way to do this:

document = Nokogiri::HTML(open("http://web.archive.org/web/20120502173130/http://magmarails.com/"))

But, if the website has utf-8 encoding and you need to get the accented characters, you should send the HTML to Nokogiri as a raw string, because there is an issue between nokogiri and open-uri and you might have trouble if you don't do this.

The next step is to generate the XML file. But, before doing that I would like to explain you how Nokogiri access the elements of the HTML document. As I already mentioned, we can access the elements using CSS selectors, this is the main reason why I feel very comfortable with Nokogiri. How can we select the elements? Easy, Nokogiri brings us two methods to do this.

css(selector)

and

at_css(selector)

Where 'selector' is a string with any CSS selector, for instance:

'.container ul li'

The difference between css and at_css is that the first one returns a collection of all the coincidences for the selector, while at_css returns only the first instance.

Okay, we just need to get the data from all the '.speaker-item' elements from the website. The following does the work:

speakers = document.css('.speaker-item')

What that line of code does is to store all the elements with the class .speaker-item on my speakers variable.

Now we must declare a variable for auto-incrementing the speaker's id.

speaker_id = 1

The final step is to create the XML file and put data on it, we now can access all of the speaker's elements and get the data. We can get the data from a selected element with Nokogiri using the text property. For instance:

puts document.at_css('h1').text

The following code creates the XML file, and writes on it the entire structure with the information obtained from our speakers collection. Now, we need to use an iteration to access each .speaker-item element on the collection and get the data.

File.open('speakers.xml', 'w+') do |f|
  # For each speaker on speakers list do
  speakers.each do |speaker|
    f.puts("<dict>\n")
    f.puts("\t<key>name</key>\n")
    # The speaker name is at the h1 element
    f.puts("\t<string>#{speaker.at_css('h1').text}</string>\n")
    f.puts("\t<key>speakerId</key>\n")
    # The speaker_id is generated automatically with the iteration
    f.puts("\t<integer>#{speaker_id}</integer>\n")
    f.puts("\t<key>twitter</key>\n")
    # The user's twitter is at '.data p' element
    f.puts("\t<string>#{speaker.at_css('.data p').text}</string>\n")
    f.puts("\t<key>bio</key>\n")
    # The user's bio is at '.bio p' element
    f.puts("\t<string>#{speaker.at_css('.bio p').text}</string>\n")
    f.puts("</dict>")
    speaker_id += 1
  end
end

And that's it. The product of our code is an XML file with 216 automatically generated lines. This is a very basic example, but the potential of Nokogiri is only limited by our creativity.

As I already told you, this is only a small example. To give a larger example to better appreciate the potential of Nokogiri and the Screen Scraping technique, imagine you want to retrieve data from a website that includes a large quantity of data shown on an HTML table. With Nokogiri, you can access all of the rows in that table, and instantiate objects to do something with them or to store those rows in a database accessible from our application. If there were 10 to 20 rows of data you would probably do it manually, but if you had over a 1,000 rows you would need the help of Nokogiri.

To conclude, Screen Scraping with Nokogiri is a great tool when we need to automatically extract data from a website, we don't have any web services available, and our sources are HTML documents exclusively.

Hope you enjoyed this short tutorial.

*Nokogiri can parse XML, SAX and Reader documents too.

Sources:

Coding hardware with Ruby + Phidgets

2013-01-15T03:15:29Z

Ever wondered if you can take Ruby to the next level? Maybe program some hardware with it? Well, me too! And Phidgets is a great tool for accomplishing this goal.

Phidgets (Physical Widgets), as the website stands are '... a set of "plug and play" building blocks for low cost USB sensing and control from your PC.' This means that we have a wide variety of sensors and controllers for us to choose to play with. Phidgets works with C/C++, C#, Cocoa, Delphi, Flash AS3, Flex AS3, Java, LabVIEW, MATLAB, Max/MSP, MRS, Python, REALBasic, Visual Basic.NET, Visual Basic 6.0, Visual Basic for Applications, Visual Basic Script, and Visual C/C++/Borland.NET... and now Ruby!

In order to start working with Phidgets + Ruby + OSX there's this really good wiki, but if you want to start right away then I suggest to begin by installing the OSX driver and the phidgets-ffi ruby gem.

After doing that, we can go to 'System Preferences' and under the 'Other' group we will find the Phidgets pane.

Enter the Phidgets pane and see the Phidgets devices connected to the computer. In my case I have a RFID reader and the Phidget InterfaceKit 8/8/8, with 8 digital inputs, 8 digital outputs and 8 analog inputs with configurable data acquisition rates on the board.

One of the coolest thigs about this pane is that if you double click on the interface kit you get a window where you can test it, see its information and read its inputs, here in the image you can see that I have two analogue sensors, a rotation sensor and a slider sensor (both potentiometers) inputs 0 and 5. If you move those sensors you can see the numbers change in real time.

Now that we can confirm that everything is working as it should, let's get down to writing some code.

I'm going to show you how to code a small example of the PhidgetRFID reader. The first thing that you need to do is install the Phidgets gem.

gem install phidgets-ffi

As the documentation of the gem states, phidgets-ffi is dependent on the ffi(>=1.0.9) gem. So, if the above command doesn't automatically install the gem for you, then enter:

gem install ffi

The example code is too small so I'm going to paste it in full and then explain:

Line 2 will include the libraries included in the Phidgets gem, Line 6 is interesting because we are creating a Phidgets RFID object which we can configure in Lines 12 and 13 in the on_attach hook from the RFID object we just created. Line 12 is critical because we're activating the antenna and without it we wouldn't be able to read tags, Line 13 is simply to turn on the LED in the reader. We need to have the sleep call in that configuration in order to give time for the reader to connect.

Events on Line 17 and Line 21, 'on_tag' and 'on_tag_lost' get called when we add or remove a RFID tag in the reader. And that's pretty much it!

Much of the coding for Phidgets follows the line of configuring the board and hooking up to events. Those events are called when the sensors activate by either a change of the brightness of the light, temperature variations, the flip of a switch, touching something, etcetera. And it's then when we can do stuff with our code, either logging those events or taking action using actuators, including servos, DC and stepper motors.

If you want to know more, you can find more examples of how to program Phidgets with Ruby by looking at the Phidget's gem source code, have fun!