Runtime Revolution Blog

PS2Format gem, batch transfers made easy

Wed, 15 Jul 2015 16:59:00 +0000

Usually whenever in your project you get involved with payments, you want an easy way to make multiple bank related transfers. This was the reason behind the creation of the PS2Format gem.

What is a PS2 file?

PS2 is a file format used to make batch transfer payments and supported by portuguese banks. It provides you with a wide variety of account transfers, allowing you to send them in a batch style and be executed by your bank.

And what's on it?

The file is composed by three section: header, operations and footer.

The header provides the information about who is sending the file and general information about the operations. You need to provide a valid NIB and a valid operation type. Regarding the operation type, there is a wide range of operations that can be performed for both incomes and expenses, ranging from payroll to telecomunication payments.
You can provide multiple operations to be performed within the same file. Like on the header, you provide a NIB, in this case for the "client" bank account, and an operation type. You also need to provide the operation amount and the company's reference, if the operation is an income.
The final section of the file is the footer which acts like a summary for the whole file, containing the total number of operations and its total amount.

So, how can I create an PS2 file?

Let's try to create a file for handling a company's payroll.

We first create a new batch transfer object which will implicitly create the file's header:

ps2_file = PS2Format::BatchTransfer.new(ordering_nib: "123456789012345678901", operation_type: PS2::OperationType.expense[:payroll])

Now we need to add the actual operations, one for each company's employee:

1
2

ps2_file.add_operation(amount: 5000, nib: "003503730000539151280" , operation_type: PS2::OperationType.expense[:payroll], processing_date: Date.today)
ps2_file.add_operation(amount: 3000, nib: "003503730000539151280" , operation_type: PS2::OperationType.expense[:payroll], processing_date: Date.today)

You can always check if your batch is valid...

ps2_file.valid?

and check for the errors, if used in a Rails context...

ps2_file.errors

After all operations are correctly added you can save the batch in the form of a file:

ps2_file.save "payroll.ps2"

or send it to a stream:

ps2_file.marshall stream

This simple API gives you the ability to generate files of financial operations for portuguese banks. After this you only need to send it to your bank. Different banks have different ways to do it, so, you must follow their instructions.

More Information

If you need more information you are welcome to check the README file in our Github Repository.

What's Next

Currently, there is a new format for financial transfers, SEPA. With SEPA is now easier to make tranfers to foregner banks within the eurozone. This is the evolution of the PS2 format and can also be a new challenge to the Runtime Revolution team, a SEPA gem. You just have to wait and see.

PS2Format gem was built by the Runtime Revolution team. Find us on Twitter if you have questions, ideas, grab the source code, fork and contribute on GitHub right now! Keep an eye on our page and if you have new ideas let us know.

If you find this interesting, or at least useful, please check out other Runtime Revolution's Open-Source Projects.

Citizenship - simple yet useful validation of Portuguese cards and more

Mon, 01 Jun 2015 00:00:00 +0000

Citizenship is a simple gem that allows validation of Portuguese documents or common identifiers, such as citizen card numbers, bank account numbers (NIB), fiscal numbers (NIF) and more.

As most of these IDs have some kind of checksum, Citizenship calculates and verifies them accordingly. We also provide some convenient methods for validating local phone numbers and email addresses.

What is this useful for?

Many applications, Rails based or not, have user related fields (e.g. portuguese related card ID's). So you'll need for sure to validate those before doing real stuff with them.

Currently, it validates:

NIB (i.e. banking account number)
NIF (i.e. fiscal number)
local phone numbers
identification card ("bilhete de identidade")
citizen card ("cartão do cidadão")
zip code
e-mail

How does validation work?

Sometimes, when your application cannot check these identifiers (e.g. ID, phone, ...) against real databases, you have to fallback to other strategies. A very simple validation may be restricted to a "syntax validation": "if it's a phone number it is supposed to only have digits, and possibly a leading plus sign".

Citizen related ID's (e.g. fiscal number, citizen/identification card ID) and bank related identifiers implement checksum/control digits in order to reduce the probability of error whenever using them and provide a basic and minimum validation algorithm.

The checksum/control digits may be part of the ID itself (e.g. NIF, NIB) or may be complementary, as happens with old Portuguese identification cards.

How can you use it?

It's dead simple.

For convenience, we provide bang methods "!", which either return the value or raise an exception, and question mark methods "?" which either return true or false, depending on validation.

1
2

result = Citizenship.valid_nif?("123456789") # returns true
id = Citizenship.valid_identification_card!('156 944 8', '8') # returns '156 944 8'

Integration with Rails

Although you can use this gem outside of Rails context, of course it supports Rails by adding some useful ActiveModel validators.
It's quite easy. To validate an attribute "fiscal_number" which is supposed to be a valid NIF (fiscal number), you would do something like this:

class User < ActiveRecord::Base
  validates :fiscal_number, nif: true
  ...
end

More Information

For more information about how to use the Citizenship Gem please look at the README file on the Github Repository.

What's Next

Currently, as of v1.0.0, the zip codes validation is based only on a regex pattern match. That may fit some simple purposes but is not very accurate indeed.

We will be integrating a full "database" of valid zip codes (easily obtained at CTT's web site - Portuguese postals).

We may add IBAN validation and extend currently features beyond Portuguese borders, allowing validation for assets used in other european countries.

Citizenship was built by the Runtime Revolution team. Find us on Twitter if you have questions, ideas, grab the source code, fork and contribute on GitHub right now! Keep an eye on our page and if you have new ideas let us know.

If you find this interesting, or at least useful, please check out other Runtime Revolution's Open-Source Projects.

Capybara: emulating mobile browsers with chromedriver

Fri, 21 Nov 2014 20:38:08 +0000

In a recent project update where we have done some changes to our responsive site, we broke the native datepicker feature that we use when the browser is a smartphone.

To help avoid this kind of situation, we decided to research for a solution that would allow us to create tests for mobile browsers specific features that we have on the site.

Since we use capybara to do these tests, our first trial was to check if we could do this with the driver we were currently using, poltergeist.
After some investigation we found out that although we could resize the browser and change the User Agent, it didn't supported native datepickers.

Then we checked capybara-webkit, but rapidly came to the same conclusion.

It was after it that we came across chromedriver, Google Chrome driver for selenium. In a recent update to the driver, they added support to use the mobile emulator present in Google Chrome dev tools that supports among other things, native datepickers.

This allowed us to create the tests for the specific features that we needed without having to create more complex setups, like using selenium ios-driver or selendroid.

To run it in headless mode in an ubuntu machine, first you need to be sure you have the following dependencies installed on your system:

Xvfb:
```
1
```
```
sudo apt-get install xvfb
```

Google Chrome (source):

Gets the repository key

wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -

Set the repository:

sudo sh -c 'echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'

Update
```
1
```
```
sudo apt-get update
```

Install the stable version

sudo apt-get install google-chrome-stable

To install the chromedriver

Download the latest version of chrome driver from here

E.g:

wget http://chromedriver.storage.googleapis.com/2.12/chromedriver_linux64.zip

Unzip it to /usr/local/bin

E.g:

unzip chromedriver_linux64.zip -d /usr/local/bin

Update the chromedriver file permissions:
- E.g:
```
1
```
```
chmod 0755 /usr/local/bin/chromedriver
```

Ensure you have the headless and the selenium-webdriver gems on your gemfile:

1
2

gem 'selenium-webdriver'
gem 'headless'

Add the following to your /spec/rails_helper.rb file:

require 'headless'

headless = Headless.new
headless.start

at_exit do
 headless.destroy
end
...

Capybara.register_driver :apple_iphone_5 do |app|
  mobile_emulation = { "deviceName" => "Apple iPhone 5" }
  capabilities = Selenium::WebDriver::Remote::Capabilities.chrome('chromeOptions' => {'mobileEmulation' => mobile_emulation})
  Capybara::Selenium::Driver.new(app, 
    :browser => :chrome, 
    :desired_capabilities => capabilities)
end

then, on the tests where you want to run with the mobile browser:

  before :each do
    Capybara.current_driver = :apple_iphone_5
  end

  after :each do
    Capybara.use_default_driver
  end

Hope this helps you start using selenium with chromedriver.

More details and settings in Capybara Github, Headeless Github and Chromedriver mobile emulation page.

Metaprogramming in Ruby - Part I

Tue, 16 Jul 2013 00:00:00 +0000

Here at Runtime Revolution we love Ruby. We love it for several reasons but one of the traits we most cherish is it's metaprogramming capabilities.

In this series of posts, I'll introduce and explain the concept, make a tour of Ruby's metaprogramming infrastructure and explore some common metaprogramming techniques.

What is metaprogramming?

Metaprogramming is the process by which a program manipulates other programs or itself.

Although the concept may be somehow strange to some developers, metaprogramming is something that has presence on the daily life of a software maker. Maybe some examples of every day typical metaprogramming products will shed some light on it's ubiquity:

Lexical analyzer generators

Lexical analysis consists on translating an expression into a list of it's representative tokens. This is the first stage of interpreting or compiling your favorite programming language source code files.

Lexical analyzer generators are applications that given a set of scanning rules, produce a lexer. Hence they are software applications that write other software applications given that a lexical analyzer generator output is a lexer (a.k.a. Scanner or Tokenizer).

Interpreters/Compilers

Interpreters or compilers are canonical examples of the use of metaprogramming. Both receive a representation of a set of instructions and produce some other representation of the same set of instruction but as another software program. In the case of MRI 1.8.7 or lesser, your Ruby source code will turn out to be interpreted and executed by C code (for 1.9 and beyond, things are slightly different). In the case of JRuby, a Ruby implementation for the Java Virtual Machine, at a given point, your code will run as JVM's byte code and possibly just-in-time compiled to machine code. In both cases we have a program that writes (manipulates) another program.

C++ Templates

C++ templates are probably the most typical example of a subset of metaprogramming unsurprisingly called Template metaprogramming. With C++ templates you write code which in turn will be pre-processed to standard C/C++ source code that will be merged with the rest of the code base and subject to the regular compilation step.

Most software written in "dynamic" languages

Most software written in Ruby, Lisp, Python, Smalltalk, and other languages able to alter the program at runtime. This is usually referred as dynamic metaprogramming.

This is true even for the most unaware of developers, since in these environments, even if the developers don't actively use metaprogramming features, most libraries and even core language constructs use them. For instance, in the case of Ruby, metaprogramming is present at various levels of the Ruby ecosystem. It's is massively employed in Ruby on Rails but also in Ruby itself. e.g. the attr_acessor method.

What is dynamic metaprogramming?

Dynamic metaprogramming is the process by which a program manipulates itself at runtime.

The first three examples above illustrate a metaprogramming pattern where the resulting program lives on a separated runtime of it's producer. A compiled program, for instance, not only runs on a different runtime but typically is a loose representation of your original implementation, where many of your origin language concepts and constructs (e.g.: scopes, classes, etc.) don't even exist.

This is not the case with dynamic metaprogramming. With dynamic metaprogramming you manipulate your program, on the fly, within the same context. We may think of languages with support for dynamic metaprogramming as languages that have a built-in API to themselves.

Why dynamic metaprogramming?

Why dynamic metaprogramming? It's not that we don't already have our share of concepts, methodologies, paradigms and techniques that we need to throw one more for software developers to wrap their heads around. What's the gain?

Provide an elegant and powerful way to handle cross application concerns

Since using metaprogramming features allows you to write code that deeply interacts and changes code yet to be written by another developer, it's a very convenient way to address cross-application concerns and develop in a very generic fashion. This in turn is one of the features needed to be able to create high level libraries that correctly interact with application code. At this point you're able to write libraries that handle functional (instead of technical) issues, resulting in effective code reuse at a very high level.

In the end, this allows for some very efficient transfer of complexity from application developer to framework/library developer, benefiting an entire language ecosystem.

Dynamic code generation is superior to static code generation

UI Generation libraries provide a good showcase for the above statement. With more static languages (e.g. C/C++) every take on UI generation was made by generating code. This would often result in marginal gains of productivity simply because that customization of generated code would invalidate recreation of the UI code. Regenerating would mean crushing any customization made before hand. This is not an issue with dynamic code generation.

Easy internal DSL development

Dynamic metaprogramming features are very useful for creating internal DSLs. That means you can create a very high level of abstraction for a particular problem, high enough for someone with a functional knowledge of a given domain to be able to use your (mini) language without general programming knowledge. Even if creating a DSL is not desirable, metaprogramming features, when used properly, are useful in creating very high level APIs.

Ruby's metaprogramming infrastructure

Ruby has a very complete metaprogramming infrastructure, some of the most relevant components are:

Strong introspective abilities

Type introspection is the ability that a program has to examine it's own types.
it enables the creation of further metaprogramming infrastructure. Ruby's introspection is deeply embedded in it's object model , thus creating a very intuitive API.

Runtime creation of core language constructs

In Ruby, you can create most of the core language constructs at runtime. This means you can create new classes and methods (or even remove existing methods). Changing existing classes is also possible (this is usually refered to as Open Classes), even core classes.

Other features useful for metaprogramming

Ruby has several other features useful for metaprogramming, such as Metaclasses, Hook Methods (method_missing, included, inherited, etc. ), Closures and methods as first level citizens, and many others.

We will delve into this topics on the next series of this post which will have a much more hands-on approach. I promise :).

Meanwhile, if you want to learn a bit more about metaprogramming in Ruby, check out the excellent Metaprogramming Ruby.

Our new Rails Gem - Survey

Tue, 04 Jun 2013 00:00:00 +0000

Survey is a Rails Engine that brings quizzes, surveys and contests into your Rails application. Survey was initially extracted from a real application that includes contests and quizzes.
While we were building two of our latest client projects we found the need to have a reusable and generic way to integrate questionnaires, surveys and competitions into different application use cases.

Ever had the feeling you were creating something that already exists over and over again? In this case we did, so we extracted Survey from those projects. Like most good development shops we are both producers and consumers of open source, and we also love to write it in order to deliver nicer software quickly.

We wrote Survey because none of the existing gems did what we needed. Some of them are developer oriented, and had to be configured with YML files or in code, but we needed something backed by models that our end users could directly manage in the application's back-office.

How does it work?

Survey is organized as a typical questionnaire domain. On the top of the hierarchy we have the Survey class itself, with associated questions. Answers are related to surveys via Attempts, to model that more than one attempt is possible in some scenarios. The correct answer to a question is modeled via an Option class that relates Questions and Answers, allowing for different relations other than correctness in some custom scenarios. Depending on what you are dealing with you can assign a correct answer to the question and decide if the answers are valued with weights or not. In this model, Attempt objects have a collection of answers made by the participants. We wanted the model to be flexible enough that it can be linked with an existing object model and interact with it without imposing too much constraints.

The following UML diagram will tell you the infrastructure:

How can you use it?

So what kind of stuff can you make with survey? How can it make creating surveys easier ? Let’s begin:
Dealing with models - The easy way to create a relation between your target model (let’s say for example, model User) and Survey is to use the has_surveys ActiveRecord::Extension. Just that! Now the User will have a connection to Survey model and every user has a maximum number of attempts and a history log from the Survey Perspective. Survey will also track every new attempt to submit surveys, or deny it if the user already raise the maximum number of attempts to answer it.

1
2
3

class User < ActiveRecord::Base
  has_surveys
end

Ok, now our User class has a valid relationship with Survey. We can keep moving forward and think how we would create a sample survey:

# Let's build our own survey
my_survey = Survey::Survey.new do |survey|
  survey.name = "Star Wars Quiz" 
  survey.description = "The following questions are related with first movie of Star Wars"
  survey.attempts_number = 3
  survey.active = true
end

# Let's add some questions and options
question_1 = Survey::Question.new do |question|
  question.text = 'How many droids help Luke in the first movie?'
  # by default when we don't specify the weight of a option
  # its value is equal to one
  question.options = [
    Survey::Option.new(:text => '1 droid',  :correct => false),
    Survey::Option.new(:text => '2 droids', :correct => true),
    Survey::Option.new(:text => '3 droids', :correct => false)
  ]
end

question_2 = Survey::Question.new do |question|
  question.text = 'Who is the evil guy?'
  question.options = [
    Survey::Option.new(:text => 'Darth Vader', :weight => 100),
    Survey::Option.new(:text => 'Obi Wan',     :weight => 0),
    Survey::Option.new(:text => 'Jabba',       :weight => 50)
  ]
end

my_survey.questions = [question_1, question_2]
my_survey.save!

# Let's answer it with User 1
attempt = Attempt.new(:survey => my_survey, :participant => user_1)
answer_1 = Survey::Answer.new(:option => question_1.options.first )
answer_2 = Survey::Answer.new(:option => question_2.options.first )
attempt.answers = [answer_1, answer_2]
attempt.save
# Let's answer it with User 2
attempt = Attempt.new(:survey => my_survey, :participant => user_2)
answer_1 = Survey::Answer.new(:option => question_1.options.first )
answer_2 = Survey::Answer.new(:option => question_2.options.last )
attempt.answers = [answer_1, answer_2]
attempt.save

# Let's pull some metrics

survey = Survey::Survey.active.first

# select all the attempts from this survey
survey_answers = survey.attempts

# check the highest score for user 2
user_2_highest_score  = survey_answers.for_participant(user_2).high_score
puts user_2_highest_score
=> 51

#check the highest score made for this survey
global_highest_score = survey_answers.high_score
puts global_highest_score
=> 101

One great thing to do is to use the data collected by the survey and create metrics with it. For example you can see how many users already spent their valid number of attempts or for example, see how many correct answers the user has, make a sum of answer weights, or something else entirely. You can define your own strategies to collect metrics around the user's answers. Also, any model, not only the User model, can respond to surveys.

Scaffold framework integration

Survey offers you a basic structure of different controllers and views that can be generated using a Rails generator. If you are using RailsAdmin or ActiveAdmin templates are included to make building backoffice pages for Survey easy. Depending on your needs, you can use one of the following rake tasks.

For ActiveAdmin:

rails generate survey active_admin

If you want a traditional rails controller you can execute the following rake task with a namespace contests (this variable is optional):

Rails generate survey plain namespace:contests

These generators will make building your Survey backoffice easier.

What is it good for?

Survey is a great choice when you have the concept of questionnaire or contest in your domain model. It is also good when you are using RailsAdmin or ActiveAdmin because it has default templates that you can customize according to your needs, in other words you don’t need to make everything from beginning, because Survey takes care of the basics for you.
Survey has the ability to assign different weights to the questionnaire’s questions; this is quite good when you are dealing with tests or feedback surveys.
Survey controls the number of attempts made by each participant. You can define a maximum number of attempts per questionnaire and restrict the submission to those who have a valid number of attempts. Here's one simple and straightforward way of doing it:

Survey::Survey.new(:name => "Star Wars Quiz", :attempts_number => 4)

Survey in the general case can be valuable if you want to collect metrics from surveys afterwards. Although, because many times surveys have their particular form and style, also relevant metrics diverge from project to project, so Survey provides the basic data, and you have to use that data to extract your own goals and metrics.

More Information

For more information about how to use the Survey Gem please visit the README file on the Github Repository.

What's Next

We wanted to enable end users to create surveys and other questionnaires without constantly depending on developers. Maybe some day Survey will grow, and someone will use it to build other cool open source projects :)

Survey was built on Rails by the Runtime Revolution team. Find us on Twitter if you have questions, ideas, and grab the source code on GitHub right now! Keep an eye on our Survey Roadmap and if you have new ideas let us know.

If you find Survey interesting, also check out other Runtime Revolution's Open-Source Projects.

Welcome to Runtime's new Blog

Mon, 03 Jun 2013 00:00:00 +0000

It is with great pleasure that we welcome you to our new Runtime Revolution Blog.

Here we will be posting about Ruby, Rails, the web and open-source projects. In a way to celebrate our blog birth we will publish a series of posts around our open-source projects and related themes like meta-programming and parallelism.

Who are we ?

We don't just create web applications, we also solve problems. At Runtime we write flexible and well designed software to help business develop strategic solutions and launch their concepts in the right way.

You can follow us @twitter or fork us @github!