Bogdan Gusiev's blog

"Naming Things" Solution

2017-11-23T00:00:00+02:00

Naming things in software is really a big deal. It gets harder and harder over time when a code base gets larger. I’ve never encountered an article that would try to address the issue. So here, I will try to define good name criteria as well as naming strategies that appeared to be useful in my experience.

Name like your users do

The most basic idea of giving a thing like class, method or function a good name is to name by how your users suppose to call it. Like if there is a tweet in twitter I would assume that there should be a thing named “Tweet” inside twitter source code. It is as simple as it is, but when you model things right your internal infrastructure just reflects the actual interaction between software users. This principle applies well to the structure we call OOD: the right OOD has maximum number of classes that your users can see through software interface and minimum number of classes that your users can not see. And those things that users can see should have their name from how your users call it or would call it.

Good example when things can get a different name:

Lets say we have Article and Email and both of them are having a header. But in the article it is usually called “title” but in the email it is usually called “subject”. If you don’t have any shared code between the two, it is better to stick with different names rather than the same: it will make you closer to how your users call it and will avoid unnecessary naming collision that is described later.

It is worse noting that this is not the only one principle to follow. There can be a number of reasons you may go with a different name rather than the name that comes naturally. They are described below.

Avoid simplifications through naming

If there is some completely new thing that needs to be named for both end users and software, ensure that there is no other thing that has the same name in your software. Human brain tend to simplify the world around us and use the same name for different things.

Here is a good example: Suppose your software allows people to subscribe to newsletter and to unsubscribe from email notifications. In this case, human brain interprets this two possibilities as logically opposite while they are not: there is actually three states the person can have: subscribed to newsletter, unsubscribed from email notifications and undefined (or default); or even 2 independent options with “Yes/No” states. Even though all software developers in your team understand the difference, you can not expect all people in the world to understand that. In order to avoid the automatic simplification, you’d better call those states: opted in for newsletter (or simply “opted_in” internally) and unsubscribed from email notifications (or simply “unsubscribed” internally).

The idea that “Opted In” should stuck as a term for newsletter subscription and “Unsubscribed” should stuck for email notifications. When people speak fast, they will always shorten those terms. It means that through the daily conversation certain verbs will be used instead of describing the entire action. Example: “How many subscribed people do we have?” will have some uncertainty in case “to subscribe” will be used for both: describing newsletter subscription and notification emails subscription instead of just one.

Naming Uniqueness

Here is a good example from my practice on how naming uniqueness can be broken: English has two words “Company” and “Campaign”. In my native tongue they actually translated into a single word: “Компания” /compania/. It is better to just avoid using these both terms for naming in the international team. While this is a rare case, it gives the idea on how naming problem appears: Russian conversation don’t make a difference between 2 things that are pretty different. Same case appears naturally within a single language all the time.

The other source of naming collision can still be a fast speech between team members. Suppose your software works with “WebPageSnapshot” as well as “ServerSnapshot” When people speak fast, they will tend to use “Snapshot” for both terms. It can lead to misunderstanding in daily conversation. The phrase: “I made a patch that touches a snapshot source code, please take a look” doesn’t define the actual place to look at. It will almost certainly lead to time waste from people assumptions they understood each other. Either the speaker or the listener could not be aware that both types of snapshot exist and they may feel natural with the assumption that “Snapshot” can only mean one of the types. Please only use adjectives to distinguish between terms if they are actually same thing at some level of abstraction like: “VideoFile” and “ImageFile” are just “File” at some levels in your software (in other words: share code and behaviour).

Using same adjectives to ensure things are connected by their names is a good idea. Consider the business rule: “Affiliate Member” can participate in “Affiliate Campaign”. We will not make any confusion by using same adjective for both terms. Avoiding adjectives in fast conversation will not be that misleading: “The member with email ‘bogdan@example.com’ has joined the campaign 2 days ago” A business rule like: ‘“Member Affiliate” can participate in “Campaign Affiliate”’ is worse than a weapon of mass destruction for communication inside your team. The general rule: ensure that shorten versions of names are easy to use in daily conversations.

Naming internal objects

What I mean by internal here is a thing that is pretty known inside your software but completely unseen by the end users. Examples from Rails: “UsersController” or “UsersHelper”

Note that there is no problem in calling every controller like “WhateverController” (with the same suffix) because all of those controllers actually share code and behaviour, so they are the same thing at some basic level while “WebPageSnapshot” and “ServerSnapshot” were not in the previous example. Some objects are just internal - end users don’t interact with them directly. There are some principles that would allow to give them a good name too.

At first, ensure that internal thing doesn’t take a name of the thing that is actually available to the end users. This is very important to do: leave good names for things your users care most and can interact with.

At second, it is good to give them a longer names to encode more information on what they do. There could be understanding illusion connected to a name. Internals of your software tend to do some specific work. When you give them a short name, software developers tend to assume they understand what they do. It can be a complete illusion.

At third, sometimes it is even a good idea to give a confusing name to something. You may avoid that illusion by giving a name that would cause software developer to never assume what this thing does before looking at it.
Choose wisely if you want your developers to assume what a method does based on name or not. These assumptions can be misleading for complex cases. In my project, we have a method called def fuck_short_url(text). It has more comments than lines of code: it makes a very specific replacement of links inside a text fragment and we don’t want any developer to assume they know what it does based on name. Here is the method source code for the most curious readers: https://gist.github.com/7a823c15ac350aaf948668f0fc3a3238

Summary

There could be 3 types of names:

Direct name - like your users call it
Descriptive name - long and describing the function of an object or method
Strange Name - eliminates the name based assumption

Good principles to follow:

Follow naming that was already given by software users or community/business overall
Name things uniquely to avoid confusion
Avoid naming collision through simplification of terms and shortening of actions
Give longer names to more complex things
Give shorter names to things that are frequently used

Guiding Software Development Principles

2017-07-07T00:00:00+03:00

It is becoming popular to talk about software development principles from time to time. The new major concepts appear and die due to the fact that most of them are designed to be perfect but they are not. If you would stay in IT long enough, you encounter articles and talks where people discovered an amazing concept “X” that solved all their development problems and brought their team to inevitable success. Unfortunately that “permanent success” is usually temporary or even a self-deception. The truth is: there is no silver bullets in software development just like in any other engineering process. Engineering of world class products requires precise resources control, trade offs and balance between easy and complex, fast and slow, and beautiful and ugly. Here, I will try to explain my guiding software development principles as well as the why-s.

Priority Matters

We often forget that most principles don’t match not only with the real solutions but also each other: you can not swim and guard clothing at the same time. So first thing you need to consider is priority of your principles. Some of them should be more important than others. In this way you can solve contradiction between them: if there are two principles and they can not be followed at the same time, pick the higher priority one.

Less is More

There should be very limited number of principles to follow. More principles under the same concept just creates more contradiction and leaves less room to be situational and follow additional guidelines that would fit best for given problem. I think the ideal number in your “guiding principles” that apply almost to everything in your software should be 3. Higher numbers are possible.

Christian God was so kind that he left us with 12 that still have very small contradiction like: should I steal a weapon from a person that wants to kill me tomorrow? That’s an amazing result for human moral that is at least 1000x times older than software development. Based on our level of understanding of software engineering, we can effort only 3 at the same time.

Build for Humans

Software is always build by humans and for humans. Software should do what it should do in a convenient, fast and robust way. If any principle violates the usefulness of software, it can be only partially followed. And that is fine because principles are imperfect. They should not be taken as dogmas because they are defined imperfectly by imperfect people.

Getting to the core

So here are principles that guide me through software development sprints as well as through software development marathons:

Software should do what it should do.
Model the real world
Build an Architecture
Track Dependencies

Lets go through them one by one:

1. Model the real world

This is the most important and genius principles I ever encounter that is even older than computers comes for analogue computers: model the real world. Try to be as close as possible to the piece of world you are modelling in the software: if there are users of your software, there should be a User in your code. If you simulate an iPhone, there should be an essence of iPhone in your software that performs all of its functions no matter how many of them there are. Also this principle requires you to name things right making them closer to processes they simulate. There will be always a gap between how the process is simulated in software and the real process, but this gap needs to be as close as possible to ensure software does things efficiently.

We tend to think that a software is complex. But the complexity comes from several factors. First is the processes being modeled are complex - here you would need to rethink them to make them simpler. But the second reason is software developer didn’t model them right. If you model things as they are, you eliminate second factor. If you model things even better than they are, you can even eliminate the first factor. It is worth mentioning that is only possible if you have a power to alter the process or the process you are modeling doesn’t exist yet. This is mostly the case for the modern software. If you are about to make a “select your seat” functionality for the plane ticket, there is no way you can simplify plane seat locations. If you make a new e-commerce web site, you are an author of most processes and can make product finding functionality as simple as possible until you reach a fact that it needs to fit not just your needs but also all your users needs which are quite different.

2. Build an Architecture

Architecture is often seen as a luxurius term that can only be understoon and build by an elite part of development team. Sometimes in outsource companies this is given to the team by someone like company CTO that is not even part of the team. But architecture is something more trivial and someone everyone in a team is building. Lets try to define architecture in the most basic way: Architecture is the code that is used by your software the most number of times. As you model more and more processes, the code base grows. It needs to be generalized and reused. We all know which problems come from unreused code: bug fixes and changes need to be made more than once that inevitably leads to the lack of consistency in software where buttons that would suppose to do the same thing do it in a slightly or not so slightly different way. But code reuse is more than that: architecture is just a code reuse metric. If you have a code that is easy to reuse, the architecture arises on its own because people tend to reuse existing code instead of writing new. From the other hand, you can measure the “architectureness” of any code piece including third party(aka gems) by counting how many times it is used. That definition is well aligned with the popular idea that architecture is hard to change: if you reuse a particular piece more, it gets harder and harder to change it because the change affects more places. Following this principle also requires a good test coverage. The most reused code is usually the most covered by tests. You can not test everything, but you can define your priority based on number of usages of particular module.

3. Track Dependencies

This is the most advanced and the most complex principle of all three. I have heard so many times: “Hey, why couldn’t we just break up our code into reusable modules that are independent?” or “Can we just write many independent micro-services?”. Yes, this is all about real breaking of software into independent pieces. Many theoretical-software-developers think it is a silver bullet, but it is not. Firstly because you violate Principle 1: model the real world. If things are interconnected and mutually dependent in the real world, you can not model them differently. It is a big talent to be able to find those possibilities with the minimum sacrifices of principles 1 and 2 which are more important to follow, but still sometimes violated due to the need of breaking software into independent modules. It is worse clarifying that there are 3 level of dependencies:

Mutually Dependent
One-way Dependent
Independent

Here is more formal definition: if you have code blocks A and B, they are totally independent if each block can perform its functions (aka pass all tests written) without other one. They are one-way dependent if A can perform its work without B or vice versa. And, finally, they are mutually dependent if neither can perform its work without another. In most cases, when Rails developers talk about several modules in their app, that is just a bullshit because there is no way one model can perform its work without others. They all are mutually dependent. And this is a very effective design because this is how the real world works: no modules - everything is mutually dependent as much as flowers and bees. You may start modeling pollination process from Bee class and Flower class and even consider them “independent modules” but in a few hours you would end up referencing one from another all the time.

The name of the principle as “Track Dependencies” comes from the fact that there are 3 level of interdependencies and you should at least stop thinking too abstract about them: figure out how much they are dependent and control that. Usually making things completely independent is unnecessary within one application. One-way dependent is good enough. You in-app dependency tree should be based on your vendor libraries dependency tree that is very well defined. You may consider your Gemfile dependency tree as the ideal dependency tree. Maybe you will come along the same structure in your app but it will be 10 times harder due to reasons that are hard to explain in a small article.

There is far more to say about it, but I’ll probably make a dedicated article about “Track Dependencies” later.

0. Software should do what it should do.

As I said earlier. You should always sacrifice principles if you can not build software that meets all the requirements with them. It is worth admitting that this is due to the fact that you are not a perfect software developer. Maybe the task wants you to be 2x better at software development than you are. And you can not become 2x better in a blink of an eye while your project has deadlines. Most likely you would be able to complete the task following all your principles with infinite resources, but they will not happen any time soon. You need sacrifices due lack of time, brain power, people and restrictions from security, performance, usability and other sources. Those are main reasons that causes the code to look ugly and complex, but never give up the beauty completely.

Summary

There is more to say about each principle above. It lacks examples but it is hard to find good ones that don’t require a lot of context to understand. I’ll write a more detailed article on each principle. Also, even thought the priority of principles is pretty important to follow, there might be exceptions. Exceptions are hard to formalize. You should judge them based on your practical experience and still make an internal note why the priority was changes for a particular case, but not the rest.

Something is wrong with Single Reponsibility Principle

2016-01-12T00:00:00+02:00

I’ve never been a fan of Single Responsibility Principle aka SRP

If any physical thing I used has multiple responsibilities, why would software be built from SRP things?

The role of information technology software is to build a model of the outer world. But is there many things in real world that follow SRP?

SRP feels so elegant from the first glance: why isn’t it wonderful that every tool you have does only one specific job. And all your tools are always sharp and you know exactly which tool is designed for what kind of job.

Well, in fact even common instruments are usually multifunctional.

It feels like hammer should be a perfect example of SRP, but it is not. Hint: Pay attention to what is on opposite sides of the handle: these are two different ends designed to do completely different job. It is a good example how solution is done based on the process but not on SRP. SRP is too idealistic even for a hammer. Convenience requires to merge two instruments into one to make it easy to switch from riving in to pulling out. It always requires in depth integration of functions into processes.

The pliers show even deeper integration in their design, allowing many processes associated with a wire to be done with using a single tool:

And this multi-functionality factor expands as we move to more complex and famous devices. Like:

Moving into another direction: none of two things on the image below follows SRP. Both of them are carrying not just an electrical charge, but some mass and spin as well. It tells that their behaviour is described by their electromagnetic and gravitational responsibilities. See that two is still not one.

Same thing happen in software world: utilities are better when designed to support a specific process and classes are made to simulate real world object that have tons of responsibilities.

So next time you blame ActiveRecord::Base for having 200+ methods, try to think how well it is designed to support a CRUD process of a database entity.

Summary

Principles discusses in software development always were very hot mainly because all engineering principles are pretty controversial. But I can definitely say that SRP is one of the most controversial among them.

Choosing a Templating Language

2015-02-06T00:00:00+02:00

Whenever you are serious about choosing the frontend template engine to generate HTML, here is some advice how you can make your choice easier and more effective. There are too many template languages right now, but there are only a few main criteria how you can limit your selection to 2-3 of them before getting into details.

Here they are:

Logic-less OR with Programming language

The most common way to generate a template that would seem obvious to any developer would be using programming language. Example: ERB for Ruby, JSP for Java etc. That is a pretty straight way to implement loops, conditions and more rare view structures.

On the other hand, logic-less templates like Mustache restrict you from using complicated structures in the template forcing you to extract and put them to a different layer.

Example of the same condition in logic-less or programming language template engines:

<% if @user.posts.empty? && @user.comments.empty? %>
  <a href="#">Delete</a>
<% end %>

In logic-less template language, it would look like:

<% if possible_to_delete_user? %>
  <a href="#">Delete</a>
<% end %>

def possible_to_delete_user?
  @user.posts.empty? && @user.comments.empty?
end

That looks more clean but less flexible as the logic-less template forces you to make quality even if you would like to save time instead.

In general I would recommend Logic-Free languages for a huge team with lots of beginner developers that need a strong force — like language restriction — to be controlled.

Frontend Rendering support

Frontend rendering can be the ultimate reason to switch to this type of template language. The decision here usually comes from more strong reasons of the application design rather than template language properties. I will assume that you already know if you want to use frontend rendering or not. And in case of yes, you would need to think if it is ok to have a single language for both — front and back end — or a different one for backend rendering. Two template languages overhead can be not so large overhead sometimes. For example, when the backend templates are only used for static layout like html head, or footer and header of the page, or admin pages.

It is pretty obvious that such a template language needs to use JavaScript in some way even though it is CoffeeScript.

Minimalistic OR HTML Compatible

HTML is being blamed for too ambiguous structures: you should always repeat yourself with closing tags, often used html attributes don’t have shortcuts etc. Minimalistic template languages are not trying to look like HTML and provide a convenient way to write an XML-like structure. When I write HAML instead of ERB, my fingers run a significantly smaller distance.

On the other hand, minimalistic templates can be hard to understand for old school software developers that parse HTML on autopilot. If your team has too many developers like that, then a Minimalistic template engine isn’t worth it.

Sandbox Mode support

The Sandbox Mode support requirement is pretty exotic. It is only required if you would like to allow users to change the appearance of site pages significantly by allowing them to edit the template themselves without modifying the site source code. Maybe it is not obvious to everyone, but in this case, template language would need to support many limitations to the template source code that will prevent your server from blowing up. The most trivial example of how a template could blow up a server would be loops nested to each other one hundred times that will consume too many resources. Sandbox is a super mode of the logic-less template. When not just full power programming language is not allowed, but additional limitations exist to make any possible template safe. Most popular template languages like ERB, HAML or EJS (just like ERB but Javascript instead of Ruby) don’t support Sandbox. But here are two that do: liquid and scribble

Summary

After a clear answer to what kind of template you need, the choice becomes more obvious as there will be only a few template engines matching given requirements. I’ve even spent some time letting to build this nice table for you:

Logic-less	FrontEnd	Minimalistic	Variants
yes	yes	yes
yes	yes	no	mustache, handlebars
yes	no	yes	hamstache
yes	no	no	mustache, curly
no	yes	yes	Haml-coffee, Jade
no	yes	no	EJS, ECO
no	no	yes	HAML, Slim
no	no	no	ERB +alternative compilers
sandbox	yes	yes
sandbox	yes	no
sandbox	no	yes
sandbox	no	no	Liquid, Scribble

This table doesn’t include all variants, but I’ve tried to fill in as many cells in the last column giving at least 2 variants if possible.

The current software world of Ruby & JavaScript doesn’t give full spectrum of choices when you would like to have Sandbox mode support. As I said before: Sandbox Mode is very rare requirement.

But in other cases you have enough choices. It is strange that there is no logic-less, frontend and minimalistic template engine now. If there is one - please suggest me in a comment.

My Talk from RubyC conference

2014-07-25T00:00:00+03:00

Here is a video from the excellent Rubyc conference. Hope you will find this video funny as well: All about parsers in ruby

Took part in a podcast with Alexey Vasiliev

2014-05-27T00:00:00+03:00

Last weekend I took part in the a podcast with Alexey Vasiliev. We talked about my path in Open Source, Rails and IT industry in ukraine.

It is all in Russian. You can listen it here:
http://www.rwpod.com/posts/2014/05/24/cafe-002.html

Minimalistic State Machine: Less is More

2014-04-01T00:00:00+03:00

I have tried a bunch of ActiveRecord’s state machine gems. Their functionality is very close to each other. It allows to validate a state transition and reinvent ruby programming language to define transition methods. The fact that all of them produces custom API to define a ruby method doesn’t look right…

Samples using common state machine gems

# State Machine gem example
state_machine :state, :initial => :parked do
  event :approve! do
    transition :pending => :approved
  end
  after_transition :pending => :approved, :do => :send_approval_email
end

# Workflow gem example
workflow do
  state :pending do
    event :approve!, :transitions_to => :approved, :after_transition => :send_approval_email
  end
end

# And finally plain ruby/rails example
def approve!
  update_attributes!(:state => "approved")
  send_approval_email
end

Maybe I don’t know all modern trends but ruby version looks more human readable to me.

So why do we still tend to use state machine gems

The main reason is state change validation. It won’t be easy to always check if it is possible to transition object from state A to state B in plain Ruby. This part of state machine is something that really matter.

In my head it appears as some kind of allowed changes map:

order_state_changes_map = { 
  nil => [:pending], # Initial state is always pending
  :pending => [:approved, :rejected], # Pending can be changed to to paid and delivered
  :approved => :paid # Delivered can only be changed to paid
}

If the state machine’s main goal is to validate changes than let’s implement it as a validation:

class Order < AR::Base
  validates :state, :changes => order_state_changes_map
end

With this pattern you can define state change methods in ruby:

validates :state, :changes => { 
  nil => [:pending], # Initial state is always pending
  :pending => [:approved, :rejected], # Pending can be changed to to paid and delivered
  :approved => :paid # Delivered can only be changed to paid
}

def upproved!
  # ....
end

def upproved?
  # ....
end

Now it looks really readable and a person who never saw this kind of API before can figure out what is happening.

Give a try to: ChangesValidator in your app

Datagrid slides from Ruby Meditation 3

2013-11-04T00:00:00+02:00

Here are my slides about Datagrid gem from Ruby Meditation Conference last weekend. Datagrid is a gem I’ve released almost 3 years ago. It is focused on table data representation and now allowes a full power flexibility to be called a Framework.

http://github.com/bogdan/datagrid

Support validation through ajax in Rails 4.x

2013-05-24T00:00:00+03:00

I have a constant feeling that in a modern web every web form should be submitted with AJAX not the old retro way with POST request. This is better for numerous reasons.

At first, it gives better user experience At second, AJAX is faster than regular request. At the end, it doesn’t require a work to redraw the page with it’s previous state if validation fails.

Currently I am trying to make rails app support this out of the box. And here is first small step to this goal:

This patch allows a complete validation though ajax using active model validators. This is a live demo of what it allows to do. If you feel especially interested in this feature, say your +1 here or on github.

Attach git commits to pivotal stories

2013-04-08T00:00:00+03:00

I heard about 10 different ways to organize work flow around pivotaltracker and git and about 5 gems to support that process. All of them seems abusing for me: the outcome of pivotal/git integration is a git commit attached to pivotal story and the rest of conventions like naming branches or merge policy doesn’t make any sense in long time perspective and is just a waste of time in fact.

That’s why I’ve decided to make the most trivial integration that covers this idea and don’t provide additional complexity: Meet git-storyid gem - the only one git/pivotaltracker integration that saves your time.

Installation

gem install git-storyid

Usage

git storyid -m "Initial implementation of campaign tags"
# Api token (https://www.pivotaltracker.com/profile): a56f0e9a4fbXXXXXXXXXXXXXX
# Use SSL (y/n): y
# Your pivotal initials (e.g. BG): BG
# Project ID: 494XXX

Now interactive menu is displayed to select one of Started stories owned by you:

[1] Removing Billing Page
[2] Welcome Email
[3] Email Shares -  Capture
[4] Speed up activities by dates aggregation
[5] Mass Email to Customer List - thurs AM
[6] Investigate production error
[7] Tag campaign insertion points

Indexes(csv): 7
[campaign-tags 3020407]  [#44116647] Initial implementation of campaign tags
 1 file changed, 1 insertion(+), 2 deletions(-)

Result commit:

commit 3020407e92cb125083cf50ad494ff15169a7f2e6
Author: Bogdan Gusiev <youremail@gmail.com>
Date:   Fri Mar 15 12:42:32 2013 +0200

[#44116647] Initial implementation of campaign tags

Feature: Tag campaign insertion points and 
campaigns with an identifier, 
so only campaigns with matching identifier will get shown

How should I name branches?

Branch names is not something that you should focus on. Unlike commit messages, branches are removed from history after merge. That is why you should name them in a way that takes less of your time. I personally prefer to have only one branch for myself called bogdandev when I work on a project. When the feature is not done - there is no sense to work on another feature.

Background Jobs Frameworks: Resque vs DelayedJob

2013-04-08T00:00:00+03:00

Here are my slides from Ruby Meditation meet-up in Kiev last weekend. Covers some internals and some features of DelayedJob and Resque. http://x.gusiev.com/background-jobs-frameworks

Where to work as a rubyist in Ukraine

2013-02-15T00:00:00+02:00

Last year I had a lot of changes in my career: I had to switch jobs twice while planning to do it only once. Most of the projects I was working on before was about building time wasting features to raise more money for more time wasting features. I wanted to find the right project for myself: A project that brings something good to people. Ruby is extremely good for startups, but easy investments is a decease for most of them in Ukraine. In September I met Allan Grant, CEO of Curebit. Curebit has a great team of sales and marketing people but was lacking great engineers. Allan was the first business guy in my life that was more focused on building product than raising money — build features endless cycle. So we started to work together.

Curebit is a San Francisco-based startup that recently opened an office in Kiev, and I’d like to invite you to interview with us.

Allan is also a Rails developer (http://github.com/allangrant) of Ukrainian origin. Our company, Curebit, raised $1.2 million from great investors like YCombinator (the creators of Hacker News) and 500Startups. The company is small (10 people) but profitable, and growing. We are a product company working on interesting architectural challenges and we are making the best product in our market (referral systems for e-commerce companies). We are now moving fast from being a good product to being a good platform. This fact give us really advanced challenges to fight with.

The team in Ukraine is only 6 people right now, but everyone is AMAZING and we want to get the best developers on-board. I am leading the engineering team, and Sergii Iurevych (OOCSS expert) leads design & front-end development. We pay everyone generously a San Francisco level salary because we know that our country has engineers that are worth it.

Most importantly (this is hard to explain in blog post), we have unique values that make Curebit a great place to work. We prefer to cut features/move deadlines instead of giving up code quality/polish. We don’t keep track of vacation days, because everyone we work with loves their work. We’re building a company to create value in the world, not just for profit. We have a culture, and we don’t write it down or talk about — we just live it.

We would love to meet you, even if you are not looking for a job right now. Please shoot us email to bogdan@curebit.com

If you want to see our product, check out the site at http://curebit.com (feel free to create fake account), or play with this demo account: http://curebit.com/demo (it’s an old demo that doesn’t show much).

Rails Contribution slides from ruby meetup in Kiev

2012-12-11T00:00:00+02:00

Here are slides I’ve shown on ruby meetup in Kiev. It’s all about my expirience with power hacking rails internals. Give some inside on how ActiveSupport Callbacks and Rails Router are working.

Smart Javascript Initialization Strategy

2012-10-08T00:00:00+03:00

The JavaScript initialization process is a pain in the ass since dynamic HTML and AJAX became popular. We need to take care all the time about what we need to initialize after every change in DOM. Thanks to jQuery#live and jQuery#on: We have a not-so-right but still very effective solution. It allows us to fix the problem with initialization of typical events. But initialization is not always about binding DOM events. In more advanced cases, it could be wrapping widget class over an element or a custom jQuery method call like a date-picker.

Here is an example:

// This line will be required not only on `$(docuemnt).ready` 
// but every time an update in DOM occurs that adds some new date input on a page.
$(".date-picker").datePicker();

Or advanced example:

var ImageGallery = function(){
  /* Class with 100+ lines of code here */
}; 
$(".image-gallery").each(function(element){
  new ImageGallery(element);
});

In the example above, we see two sections: definition and initialization. This fact makes code hard to follow as moving from class to DOM element in HTML template requires at least two transitions in your editor.

And when a large page update comes after an AJAX query, this problem is getting even worse. All initializers should called again:

$.get("/profile/edit", data, function(response) {
  showSomeProfileEditingPopup(response);
  $(".profile-birth-date").datePicker();
  new ImageGallery($('.profile-image-gallery'));
  // More initialization here
});

Tired of handling things like this?

Change the way of thinking about initialization

Initialization code is a bridge between actual JS class and DOM element. In 99% of cases, it has one-to-one or one-to-many relation. Then why do we always need that intermediate initialization code?

The DOM element could have the class itself:

<div class="profile-image-gallery" data-class="ImageGallery">

Now your way from DOM element to JS class requires only one transition. Moreover, all DOM elements could be initialized with one call. It needs to run over DOM and find all elements with data-class attributes that are not initialized and wrap them with a specified class:

var Initializer = {
  perform: function() {
    $('[data-class]:not(.initialized)').each(function(i, element) {
      var element = $(element);
      element.addClass('initialized');
      var klass = window[element.data(element)];
      new klass(element);
    });
  }
};

This code bit will probably require some additional functionality to support options, but the general idea is strong enough to handle all possible complexity in the future. Now, whenever DOM changes, everything we need to do is call:

Initializer.perform();

In order to attach third-party libraries to this initialization process, we need to create a simple wrapping class around them:

var DatePicker = function (element) {
  $(element).datePicker()
}

Now we are able to convert inputs to date pickers with a unified initialization process:

<input type="text" name="profile[birth_date]" data-class="DatePicker">

Conclusion

JavaScript coding is annoying for most people who came from other languages because it keeps things fragmented that should be bound together. In order to avoid this problem, we need to invent conventions to follow, so that every person in a team understands that. But the cost of convention described here is not that much and makes code more clean and easier to follow.

Strict validation is a main stream

2012-09-17T00:00:00+03:00

It’s nice to see that strict validation feature that I’ve done almost a year ago is finally getting popular. Spend a moment and understand what it does by Rails documentation.

Full text search - Solr(Sunspot) vs PostgreSQL

2012-08-16T00:00:00+03:00

This is short notes about my experience with full text search based on PostgreSQL and Solr. Solr was primary used with a help of Sunspot gem that was definitely a good idea, but unfortunately caused some drawbacks. PostgreSQL provides full text search out of the box and to my opinion doesn’t need any additional ruby specific tools other than ORM.

TL;DR

PostgreSQL approach is simple and fit well to agile project. Solr is pretty powerful, but you need to pay for that power. And this is significant cost. Think twice if you really need this power from the first day of full text search in your project.

Search index

Solr search index is something you create almost manually by defining a new data schema. Sunspot sets after_save hooks to your model to update it’s solr index.

Here is an example of schema definition based on Sunspot gem:

class Product < ActiveRecord::Base
  searchable do
    text :title
    text :material
    text :category do
      category.title
    end
    text :brand do
      brand.title
    end
  end # do
end

PostgreSQL search index might be yet another column in the database, set by pre save hook:

class Product < ActiveRecord::Base

  before_validation :set_searchable_content
  protected
  def set_searchable_content
    self.searchable_content = [title, material, category.title, brand.title].join(" ")
  end
end

And don’t forget about database index:

CREATE INDEX source_idx ON products USING gin(to_tsvector('english', searchable_content));

While this is very basic use case, it shows up the direction where your code will go in each variant.

Advanced high level DSL with a lot of “unknown unknowns” in Solr case
Just ruby code that everyone would understand from the first look in case of Postgres

As an example of how you would solve more advanced problem, I want to show facet search implementation:

# Suppose to receive a list of ids to count between different categories
def self.search_facets(ids)
  return [] if ids.blank?
  ProductCategory.select("product_categories.id, count(product_categories.id) as product_count").
     joins(:products).
     where("products.id" => ids).
     group("product_categories.id").
end

While Solr based facet would be easier:

Product.search do |s|
  s.keywords params[:query]
  s.facet :category_id
  s.paginate :page => params[:page], :per_page => 10
end

At first look you might say that Solr way is more clear, but you still don’t know what it does in deep details. Sunspot gives DSL is yet another language to query data. It includes all types of operations (e.g. comparison operators, while postgres lets use SQL - language you should know already.

There is also easy to spot that Solr and Sunspot tool chain brings more features to you out of the box. But this is where Solr advantages ends. Note stat Solr config file is not as readable as Sunspot DSL. Here is a short example that tells Solr to do some string transformation before create a search index:

<fieldType name="text" class="solr.TextField" omitNorms="false">
  <analyzer>
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <!-- remove punctualtion -->
    <filter class="solr.StandardFilterFactory"/>
    <!-- lower case letters -->
    <filter class="solr.LowerCaseFilterFactory"/>
    <!-- language semantic support -->
    <filter class="solr.SnowballPorterFilterFactory" language="English"/>
  </analyzer>
</fieldType>

When you go deeper into solr you might need to deal with configs like example above. While in case of PostgreSQL you can do any string transformation using ruby.

Deployment and Testing

Solr is a standalone application and you need to take care about Solr configuration. It is a part of a code since some search business logic depends on it. This might require:

different solr config files across different branches
test suite that covers some config options
different solr instances launched for development and test environments
a need to rebuild search index when you switch branches

Postgres built in search doesn’t have these problems at all. This will give a significant performance boost in agile process. Deployment, testing and switching branches doesn’t require any full text search specific action.

Reindex process

Reindex complexity comes from several factors.

First of them is complicated search index that cover more than one model (Ex: Product.belongs_to :brand and Brand#title should be a part of Product index). It makes both approaches more complex as business logic becomes more complex.

But in case of Solr this is not the only one thing to care. With database size growth you gain more problems. Solr index do not have that nice migration mechanism and dump transfer tools as Relational databases. No matter which gem you will use to connect Solr and Relational database: you need to take care when search index schema changes.

Conclusion

Conceptually I would like my database to respond for every data search operation e.g. full text search. This is where PostgreSQL is going. But this feature is not that strong now e.g. it still lacks spell checking and result highlight features. Solr is old school tool having all full text search features you ever imagine, but it’s heavy configuration file and technology stack should be easier.

How many clicks do you need to debug ajax request?

2012-06-16T00:00:00+03:00

Here I am gonna use Chrome browser as an example:

Open context menu
Click inspect element.
Click “Network” tab
Spawn again an ajax request (at least one click. Sometimes more)
Click on failed request at Network tab
Click “Preview” or “Response” subtab

THIS SHOULD CHANGE

Why we can not automate this daily routine work?

With this tiny script you don’t need any: Error response will be displayed in Popup at once after it occur.

Assumption Driven Approach

2012-06-06T00:00:00+03:00

This post shows some examples of how things can go wrong way because of people actions based on assumptions but not on facts. It is more conceptual than practical, so be patient and don’t blame my grammar mistakes too much.

Evil of assumptions

This is my favorite example of how wrong assumption can be:

Very popular ruby questions for newbies:

What is the difference between `private` and `protected` methods?

Question seems good and reasonable. But most people don’t know the right answer even after years with ruby. Most people assume that private methods can not be called from inherited classes, but protected can be.

But that’s only an assumption. Have you ever tried to ensure that it’s right? The answer is: This is not true. Private method can be called from inherited class in the same way as protected.

class A
  private
  def a
    puts 'hello'
  end
end

class B < A
  def b
    a
  end
end

B.new.b # No Exception here

More about private and public methods here.

This is good example on how myths can spread the world: Almost every person is surprised by the output of program above.

This makes me feel that many interviewers ask questions while don’t know what they are asking about. From this point I can state that a lot of ruby interviews goes in a wrong direction by providing an assumption as a fact.

Class variables assumption

Yet another assumption that can be made based on ruby documentation is connected with ruby class variables:

class A
  @@x = 2
  def self.pr
    @@x
  end
end

class B < A
  @@x = 5
end

puts B.pr
puts A.pr

You can assume that last two lines produce 5 and 2, but they both produce 5. Because class variables are shared across all classes.

Assumption example in the performance field

Most popular assumptions field is performance. And here we come up with the most recent bad assumption example:

Ruby 2.0 offers lazy enumeration feature.

(1...100000).to_a.lazy.select{|a| a.odd? }.map{|a| a**2}.to_a

In short: there will be only one iteration over an array with no intermediate array between select and map. If you are an assumption-driven person, you already assumed that #lazy not only saves your memory, but also improve performance.

And problem is that it is not true. #lazy is very slow.

How to use assumption in a right way?

Why we are still using assumptions a lot? Because they are cheap: they allow us to move faster without proofing that we are moving the right direction.

Here are some MUSTs that we need to follow:

Always make sure that you are 100% right when you are interviewing other person.
Never make assumptions in performance field.

And here are some more conceptual advices:

Make assumptions depend on facts, but not other assumptions.
Prove any assumption with fact before going with another assumption
Do not tell people about assumptions as if they are facts.
When we can’t make a good assumption from the beginning, maybe it’s time to gather more facts

Benchmark your performance patches

2012-05-01T00:00:00+03:00

After a dozen performance patches to many gems want to share some practical experience I gain. Tools I’ve picked up was perftools.rb and ruby built-in benchmark library. They are fit well for cases when optimization stays at Ruby level and doesn’t require to fix something in native extensions or dig into IO operations.

General flow on how to tackle performance is clear:

perftools.rb shows slow method calls in a nice call graph format.
Benchmark proves that we made an improvement.

Perftools does all the hard work for you.

require 'perftools'
PerfTools::CpuProfiler.start("/tmp/profile_result") do
  # code to profile
end

Run test and open result:

gem install perftools.rb
ruby test.rb
pprof.rb --gif /tmp/profile_result > /tmp/profile_result.gif
$IMAGE_VIEWER /tmp/profile_result.gif

Example output: Rails route generator call graph

Here is short instruction how to read call graph:

Each block represents a method that was called. Percentage in a block shows how much time was spend in current method with and without all it’s nested calls comparing to overall time. Arrow with number shows how many times parent method called it’s children method.

That’s it. All you need to do now is find blocks with most percentage and try reduce them.

This flow works fine till the moment when benchmark results before and after patch should be compared. Ruby Benchmark can not merge them together and represent in a human readable form like this:

Running benchmark with current working tree
Checkout HEAD^
Running benchmark with HEAD^
Checkout to previous HEAD again

                    user     system      total        real
----------------------------------headers parsing when long
After patch:    0.100000   0.000000   0.100000 (  0.089926)
Before patch:   0.700000   0.000000   0.700000 (  0.697444)

----------------------------------headers parsing when tiny
After patch:    0.000000   0.000000   0.000000 (  0.009930)
Before patch:   0.020000   0.000000   0.020000 (  0.024283)

---------------------------------headers parsing when empty
After patch:    0.010000   0.000000   0.010000 (  0.002160)
Before patch:   0.000000   0.000000   0.000000 (  0.002354)

So, decided to create a library that fixes this problem. Try out DiffBench if you ever come up with performance patch.

DiffBench in action examples:

Rails
- ActiveRecord - #5467
- ActiveModel - #5431
- ActiveSupport - #4493
- ActionPack - #5957
Mail - #396, #366

Web Forms 2.0: Submit with Ajax and Client Side Validation

2012-03-01T00:00:00+02:00

Web 2.0 world force us to build Forms in a new level of quality. AJAX and dynamic HTML changes a way how forms should work internally. Let’s call it Forms 2.0. And of course this facts bring a lot of complexity every time we deal with forms. Sometimes such form makes a real problem comparing to regular Web 1.0 form. Here will be a talk how to make Forms 2.0 as simple as Forms 1.0.

Sumbit with Ajax

Submit a form with AJAX have never been a problem:

$.ajax({
  url: $(form).attr('action'),
  data: $(form).serialize(),
  type: $(form).attr('method'),
  success: function () {
    //do
  }
});

More over submit with AJAX doesn’t require to rerender a page if it was not successfull: less server load and no need to support ‘new’ and ‘edit’ mode in HTML template. Problem that comes up with ajax submit is Validation. If you ever find yourself stuck with syncing client and server side validation or bug that form doesn’t have some data that was there before submit then you know what I am talking about too.

Solving validation problem

There are number of visions in the Ruby community on how to solve the problem: some people prefer stay with Forms 1.0. Some people generate JavaScript validators based on ActiveModel validators. Other guys write their own validation in JavaScript and perform it on both client and server side. The complexity of all these solution goes beyond believe. And here I am going to describe much simpler way of doing it.

What if we pass the validation result as JSON from backend in a simple key:value format like:

{email: "is not valid", password: "is too short"}

It’s fast as wind and easy to implement:

No SQL queries during validation request
- Except uniqueness, that is more likely a feature than a problem
No Rendering just small JSON object as response
No additional controller action - just modify existing one

From Assumptions to Coding

As we agreed - controller should return JSON instead of HTML:

 class UsersController < ApplicationController
   def create
     @user = User.new(params[:user])
     if @user.save
-      redirect_to root_path
+      render :json => {:redirect => root_path}
     else
-      render :action => :new
+      render :json => {:errors => user.errors}
     end
   end
 end

All we need to do after that is just assign errors to the form to the prepared spots identified with HTML5 custom attribute:

 <form id="new_user" action="/users" method="post">
-  <div class="field">
+  <div class="field" validate="email">
     <label for="user_email">Email</label><br />
     <input id="user_email" name="user[email]" type="text" />
   </div>
-  <div class="field">
+  <div class="field" validate="password">
     <label for="user_password">Password</label><br />
     <input id="user_password" name="user[password]" type="password" />
   </div>
   <div class="actions">
     <input name="commit" type="submit" value="Create User" />
   </div>
 </form>

All the dirty work can be done by AjaxSubmit library that implements all the ideas described above:

$('#new_user').ajaxForm();

The main goal reached: Make super modern AJAX form is as easy as regular form. If you still can’t believe that it is so easy - check out LIVE demo.

AjaxSubmit: Is it flexible?

AjaxSubmit is a library that has 2 years of experience behind. About 7 different engineers made their contribution during this time. Take a look at the Read Me for more information about API, callbacks and configuration options.

You can also checkout Mailtrap that is a perfect example of how your site could look like with this library in every web form.

New Rails Release with a few features from me

2012-01-23T00:00:00+02:00

Upcoming Rails 3.2 release has many useful features.

Some of them are coming from me:

ActiveRecord #pluck method
Strict validation concept
Customizable mass assignment sanitizer behavior
Gotcha for select[multiple] tag

Here is a more detailed description than in the Changelog.

Pluck method

People around made a lot of PR on it. So, You probably already heard about it.

pluck can be used to query a single column from the underlying table of a model. It accepts a column name as argument and returns an array of values of the specified column with the corresponding data type.

Client.where(:active => true).pluck(:id)
# SELECT id FROM clients WHERE active = 1

Client.uniq.pluck(:role)
# SELECT DISTINCT role FROM clients

Pluck in Rails guides

Strict validation concept

Some validation are not connected with end user and breaking it is more likely a bug than problem with user input. A few constraints can be embed to DB(like foreign keys), but most of them can not. And it’s a good idea to use AM validators to do such checks as well.

Use case:

class Article < AR::Base
  validates :author, :presence => true
end

u = Article.new
u.save # => false

#author is usually set in controller from current_user. If somebody forgets to do it - user see error: Author cann't be blank. And it is not right to show error message to user that he is not able to fix. More over DEV team is not notified about the problem.

In order to fix that You can use validate! method that generates validator that always raises exception when fails:

class Article < AR::Base
  validates! :author, :presence => true
end

u = User.new
u.save # raises ActiveModel::StrictValidationFailed

Customizing Mass Assignment Sanitizer

Added an ability to specify your own behavior on mass assignment protection, controlled by option:

ActiveModel::MassAssignmentSecurity.mass_assignment_sanitizer

The main idea behind this change is to make Mass Assignment Sanitizer to raise exceptions in development and test environments while leaving logging behavior in production. This will be the default Rails application template in version 3.2. It will allow to detect mass assignment protection issues caused by bug in code more effectively before they will be deployed to production.

Power hacking select[multiple]

Quote from Rails documentation that comes along with the patch:

The HTML specification says:

When multiple parameter passed to select and all options got deselected web browsers do not send any value to server. Unfortunately this introduces a gotcha: if an User model has many roles and have role_ids accessor, and in the form that edits roles of the user the user deselects all roles from role_ids multiple select box, no role_ids parameter is sent. So, any mass-assignment idiom like

  @user.update_attributes(params[:user])

wouldn’t update roles.

To prevent this the helper generates an auxiliary hidden field before every multiple select. The hidden field has the same name as multiple select and blank value.

<input type="hidden" name="user[role_ids]" value=""/>
<select id="user_role_ids" name="user[role_ids]" multiple="multiple">
  ....
</select>

This way, the client either sends only the hidden field (representing the deselected multiple select box), or both fields. Since the HTML specification says key/value pairs have to be sent in the same order they appear in the form, and parameters extraction gets the last occurrence of any repeated key in the query string, that works for ordinary forms.

Unfortunately this trick can introduce a side effects described well in this stackoverflow question. If you know an effective way to fix that - let me know.

Have fun with upgrading

New release of Datagrid: Mongoid, Rails Engines and HTML columns support

2012-01-10T00:00:00+02:00

Datagrid version 0.5.0 has been released.

Datagrid is a Ruby on Rails plugin that helps you to build and represent table-like data with:

Customizable filtering
Columns
Sort order
Localization
Export to CSV

Here are some major features from new release.

Mongoid support

Previously Datagrid supported only ActiveRecord ORM. Now Mongoid is also available: see the demo.

New datagrid driver architecture allows to add yet another ORM in less then 40 lines of code.

HTML columns

In order to use view helpers to render columns You can use :html option. This will call column block in the view context:

column(:completed, :html => true) { |asset| asset.completed? ? image_tag("green.gif") : image_tag("red.gif") }
# or do it in partial
column(:actions, :html => true) { |asset| render :partial => "admin/assets/actions", :object => asset }

Table customization with Rails Engines

Rails Engines allows you to embed views into Rails plugins and make them customizable. If You need serious customization of datagrid <table> and HTML Columns doesn’t help, you can customize datagrid internal views by running:

rake datagrid:copy_partials

This will create the following files in your Rails root directory:


|~app/
| ~views/
|   ~datagrid/
|     |-_head.html.erb
|     |-_row.html.erb
|     |_table.html.erb

Now You are able to customize whatever You want.

Improve your Email delivery code with Mailtrap

2011-11-18T00:00:00+02:00

Today, I want introduce the Mailtrap web service aimed to help developers build and improve their email delivery functionality.

Usually people use same SMTP settings for all environments, but this actually creates many problems in Staging and Development environments. Mailtrap is a special SMTP service for these environments aimed to solve specific problems.

Problems

Testing user registration is hell

In order to register yet another user You need to have new unique email address that should be hosted somewhere.

Perform mass email sending is a problem too

Suppose you have a daily email to all customers and need check if your delivery logic works fine. In this case you need to know what emails were send, which customers received them. And again you need a batch of email addresses to test with.

Discuss email layout and functional problems with product team

In order to share email You need to forward it to some inbox or attach to bug ticket as file. This is still not as easy as it could be.

Send emails to real customers by mistake

In some cases copy production database to staging is a good idea. But You have a risk that real emails appear on development servers. You need to take care about that.

My personal score is about 2500 emails sent by mistake at a time. Some people I know told me that their personal record is more than 2 millions emails sent by mistake. (Share your personal fuckup story in comments.)

Existing solutions

You can go in many ways to fix the problem e.g.:

setup the script on your smtp server that changes recipient email
use services like mailinator.com (only in some cases)
use custom Ruby specific gems like mail_view or mailcatcher.

All of them solves only some of the problems described above.

Our Idea

Development SMTP service should work in a different way then production one: It should not deliver emails but just gather them in one place and provide a shared access for team to debug it. Mailtrap is the first SMTP service built specifically for this purpose.

Unlike many alternative solutions, Mailtrap is platform independent, doesn’t require any daemon to run and is just an SMTP server from application stand point. All you need to do is setup host, username and password.

Features

This is not about high load, not about delivery statistics but about:

Ability to access ANY sent email without registration
Share Email is as easy as copypasting the URL
Developer tools for debugging & improving email template:
- Syntax highlight
- Easy view source
- Apply CSS reset
Access to the history

Try Mailtrap right now

Slides: Fighting with fat models

2011-11-11T00:00:00+02:00

Here are my slides from RubyC conference.

This presentation covers a hot topic for projects that are more than a year old - it’s fat models. The problem appear naturally as models are the best place to allocate business logic code. There are number of existing techniques to manage large amount of code in Rails application. But they are suitable only in a half of use cases. For the rest of them we can use Traits pattern that is described well in this presentation.

Fighting with fat models

Resque-way

2011-10-17T00:00:00+03:00

My article about Resque was posted on Engine Yard last week. It has very good notes from EY engineers and doesn’t have my regular grammar mistakes. Enjoy your reading.

Advanced RSpec Techniques presentation

2011-07-08T00:00:00+03:00

I’ve always pay attention to Rspec and posted some articles about Advanced Rspec features. Right now about 2k tests written by me personally in 3 years of development with Rails.

Here is the slides from my talk about all invented patterns and best practices:
Do rspec: fogotten features of RSpec

Reporting made easy by Datagrid gem for Rails

2011-06-26T00:00:00+03:00

4 years ago I was working on some enterprise projects with a lot of reports. From that time I was thinking about perfect report gem that would provide easy DSL for making filters and sortable columns to build reports and make it all reusable with standard OOP techniques. Since that time this idea never left my head and now finally I have enough knowledge and opportunities to build such tool.

Defining a grid

The idea of Datagrid DSL is to define a scope of ActiveRecord models and define different independent criterias to filter this scope. Than convert filtered data to table view with defined columns. So, typical datagrid report consists of:

A scope of records to query data
Filters with parameters to make a subsets of this data
Columns to display this data

And we can easily split their definition with the following ruby DSL

class SimpleReport

  include Datagrid

  scope do
    User.includes(:group)
  end

  filter(:category, :enum, :select => ["first", "second"])
  filter(:group_id, :multiple => true)
  filter(:group_name, :header => "Group") do |value|
    self.joins(:group).where(:groups => {:name => value})
  end


  column(:name)
  column(:group, :order => "groups.name") do
    self.group.name
  end
  column(:active, :header => "Activated", :order => false)
    !self.disabled
  end
end

Grid instance API

And now we can create and manipulate reports:

report = SimpleReport.new(
        :group_id          => [1,2],
        :category          => "first",
        :order             => :group,
        :descending        => true
)

report.assets # => Array of User instances: 
              # SELECT * FROM users WHERE users.group_id in (1,2) AND users.logins_count >= 1 AND users.category = 'first' ORDER BY groups.name DESC

report.assets.paginate(:page => params[:page]) # => Yes, it is

report.header # => ["Group", "Name", "Activated"]
report.rows   # => [
              #      ["Steve", "Spammers", true],
              #      [ "John", "Spoilers", true],
              #      ["Berry", "Good people", false]
              #    ]
report.data   # => [ header, *rows]

report.to_csv # => Yes, it is

Feel the Power and Flexibility by examples

I love flexibility. That is why datagrid has a lot of things for non trivial use cases. In order to proof that I’ll show you some examples of what you can do:

Documentation

Datagrid WIKI is main documentation source of the gem. Also feel free to ask questions right after this post.

In my TODO list

Current version of datagrid gem have fully document back-end staff. It’s production ready - we are running three projects that use datagrid. Now I am working on front-end that is already part of the distribution, but not yet documented.

Patching dbext vim plugin to support Rails naming conventions

2011-05-30T00:00:00+03:00

dbext is an excellent vim plugin that allows you to execute SQL queries directly from vim. For example: select * from <word under cursor>. When dbext is used with Rails there is a problem that you can work with some variation of table name like:

UserRole
user_role_id
user_role_ids
user_role

But still wants to use it as table name in your database queries so that you won’t type it manually.

So, I made it to automatically convert word under cursor into a table name with the following configuration options:

let g:dbext_table_names_number = 2 
    # 0 - disabled, 1 - singular, 2 - plural
let g:dbext_table_names_case = 2 
    # 0 - disabled, 1 - camelcase, 2 - underscore

Also there is an option to automatically strip _id suffix from word to convert it to table name:

let g:dbext_table_names_strip_id = 1

All these options match Rails table name convention by default. Staff in my dbext fork.

Once you install it in your vim runtime put your cursor on any table name variation like example above and run :DBSelectFromTable to get:

select * from user_roles

Other commands are also patched. Please let me know if any bugs found.

Don’t know if non-rails developers would love this patch but pretty sure that it won’t annoy them. Please support my pull request if you like it.

Logging HTTP api calls just like database queries

2011-05-24T00:00:00+03:00

Http api calls are pretty similar to SQL queries - they query data from external source to process them in ruby space. So I think it's a good idea to log them just like ActiveRecord does with SQL queries.

  Location Load (25.1ms)   SELECT * FROM "locations" WHERE ("locations"."id" = 2548) ORDER BY title

This simple peace of code could be very helpful in case of debugging HTTP api in ruby. Screenshot with the result Installation is simple:

gem install http_logger

And:

require 'http_logger'

Net::HTTP.logger = Logger.new(...) 
    # Default: Rails.logger if defined?(Rails)

Net::HTTP.colorize = true 
    # Default: true

Finally get rid of WordPress and PHP

2011-05-20T00:00:00+03:00

After a few tries migrate to blog engine with programming language I know well, I finally found jekyll - lightweight static site generator. My main concerns in this choice was:

Edit posts in my favorite text editor with favorite markup
NO PHP
Syntax highlight out of the box

Migration from WordPress

Three main migration points was:

Posts that was migrated with several lines of ruby code
Comments that imported to disqus as jekyll is static site engine so the comments should be hosted somewhere else
Generate tag cloud and posts by tag pages that also have a lot of workarounds in the web

One problem I am currently have is “post preview” feature that is currently unsupported in jekyll. My current approach for this looks like this:

post.content | replace_first:'<!--more-->','<!--' | append:'-->'

Problems

Jekyll looks not very extensible. Some methods are not available in the API. Example: integrate SASS require more pain the ass than it should be. I am planning to fix that in a few days. Another problem I ran into is that github native support don’t allow you any extension. My suggestion is to generate static content locally with your own code on top of jekyll and upload static content to git repository.

Currently I build site locally from source code and push static content into separated repository.

Result

About 12 hours of work to transfer 30 posts and their comments. Reworked site design to be more clean.

JavaScript injection in JSON inserted directly into script tag

2011-04-06T00:00:00+03:00

Pass data from backend to javascript is done in many different ways. One of the most simple is to inject(<%= %>) value as function argument inside of script tag. Unfortunately this pattern has well known XSS vulnerability but in a little different form than same injection in html template.

<script type="text/javascript">
    App.initizalizeSomething(<%= data.to_json %>)
</script>

The reason is that browser treat </script> as close script tag no matter where is it inserted into script. So, the script tag can be closed unexpectedly and opened again with any code if the data argument will contain correctly formed sequence, like:

</script><script>alert('hello')</script>

Use #html_escape helper is wrong here because it has different type of escaping. For example you don't need to escape double quote in this case. Rails core team is aware of that problem and implemented special helper:

# A utility method for escaping HTML entities in JSON strings
# using \uXXXX JavaScript escape sequences for string literals:
#
#   json_escape('is a > 0 & a < 10?')
#   # => is a \u003E 0 \u0026 a \u003C 10?
#
# Note that after this operation is performed the output is not
# valid JSON. In particular double quotes are removed:
#
#   json_escape('{"name":"john","created_at":"2010-04-28T01:39:31Z","id":1}')
#   # => {name:john,created_at:2010-04-28T01:39:31Z,id:1}
#
# This method is also aliased as +j+, and available as a helper
# in Rails templates:
#
#   <%=j @person.to_json %>
#
def json_escape(s)

Implementation can be found in Rails source code.

Do git commits associated to hoptoad errors

2011-03-28T00:00:00+03:00

We are trying to build a better development workflow by connecting all our tools together. So, this time we tried to connect HopToad Application with git commits because sometimes it's better to just paste the link to hoptoad error rather than describe all steps to reproduce. This idea was resulted in the command line interface for hoptoad api with a little git integration - Shelltoad.

Currently, in order to resolve hoptoad error we do the following.
Check the list of errors:

$ shelltoad
[#3374331] production ActionController::MethodNotAllowed: Only put requests are allowed
[#4023713] production ActiveRecord::StatementInvalid: PGError: ERROR:
       duplicate key value violates unique constraint "index_companies_on_slug" :
       INSERT INTO "companies" .....

View the information on the required error. '713' is automatically guess for 4023713


$ shelltoad 713
ActiveRecord::StatementInvalid: PGError: ERROR: duplicate key value violates unique constraint
 "index_companies_on_slug" : INSERT INTO "companies"
("slug", "created_at", "title", "updated_at", "external_url", "logo_id", "custom") VALUES('abbott-associates
/var/data/www/apps/startwire/shared/bundle/ruby/1.8/gems/activerecord-
2.3.8/lib/active_record/connection_adapters/abstract_adapter.rb:221
/var/data/www/apps/startwire/releases/20110218113457/app/models/company.rb:50
/var/data/www/apps/startwire/releases/20110218113457/app/utils/network_map/populate.rb:9
/var/data/www/apps/startwire/releases/20110218113457/app/utils/network_map/populate.rb:74
/var/data/www/apps/startwire/releases/20110218113457/app/utils/network_map/populate.rb:70
....

Fix the error in the code ... Prepare git commit:


$ git add .

Commit the fix to git with connection to hoptoad issue id


$ shelltoad commit 713
[dev 47f09ec]     http://xxx.hoptoadapp.com//errors/4023713
 1 files changed, 1 insertions(+), 1 deletions(-)

Commit message will look like this:

http://startdatelabs.hoptoadapp.com//errors/4023713

ActiveRecord::StatementInvalid: PGError: ERROR: duplicate key value violates unique constraint
 "index_companies_on_slug" : INSERT INTO "companies"
("slug", "created_at", "title", "updated_at", "external_url", "logo_id", "custom") VALUES('abbott-associates

This help us maintain the links between git and hoptoad automatically.

Debug ajax requests in more easy way

2011-03-18T00:00:00+02:00

When I have to debug ajax requests, I always feel pain in the ass accessing the exception message and backtrace. It require at least 3 clicks to get them and only in the plain html format, that web framework use to generate. Is it better to see it at once in the browser popup?

This can be done with Ajax callbacks offered by javascript framework(if yours doesn't - get rid of it). We can catch errors there and display popup. JQuery example with #ajaxError callback that fires on every unsuccessful ajax call:

(function() {
      var popup = null;
      $(document).ajaxError(function(event, xhr, settings, exception) {
        if (popup) {
          popup.close();
          popup = null;
        }
        popup = window.open(null, "error", "width=800,height=600");
        $(popup.document.documentElement).html(xhr.responseText);
        return true;
      });
    }());

Checked in FF and Chrome(IE is not a browser). Screenshot with the result. Be aware of browser popup blocker.
You always have a control when popup the error. I like it to be "in any case". But You can customize it with xhr.status and if statement on the server side.

In some cases it might be useful not only in the development but in production as well. But it should be an iframe inside of pretty html layout, so that designers won't blame programmers too much.

Dynamic context in Rspec - don't repeat yourself

2010-11-03T00:00:00+02:00

We used to have less problems with subject and behavior when writing our unit tests. And context is what testing is all about. Web apps manage data and the same method can return different values based on current database state. Advanced rspec features make it possible to use very effective technique to organize tests. In order to reflect your non-linear business process in unit tests you can not only nest contexts in each other but also make your context more dynamic.

Context callback

Rspec #let method is used to lazy load things into context. In spite of lazy initialization, let blocks are defined before the before each hook. This allow us to create context callback.

Understand by example. In this example orders should be delivered to confirmed customer account just after creation and should not be delivered if the account is not confirmed yet.

describe Order do
  context "after create" do #defining a partial context
    before(:each) do
      subject.customer.confirmed = _confirmed
      subject.save!
    end

    context "when customer is confirmed" do 
      let(:_confirmed) { true }
      it { should be_delivered }
    end
    
    context "when customer is not confirmed" do 
      let(:_confirmed) { false }
      it { should_not be_delivered }
    end
  end
end

Here you can see partial context definition and custom behavior in two nested contexts. I used underscore prefix in order to identify the callback method, that is defined differently in different contexts.

This happened because let as method is defined at once but before block is called only when it is reached.

Testing utility function

Another example that is a kind of pattern matching(Erlang term), designed to test utility functions.
Suppose we have a boolean expression evaluation function:

describe Expression
  describe ".run" do

    subject { Expression.run(arg) }

    context "with '&' where both true" do
      let(:arg) { "true & true" }
      it {should be_true}
    end

    context "with '&' where one false" do
      let(:arg) { "false & true" }
      it {should be_false}
    end
    ........
  end
end

Very good strategy to run same function with different arguments.

Summary

Rspec is far ahead of all unit testing frameworks. Unlike most of Rspec clones (e.g. for other programming languages), Rspec authors got in deep to the testing problems and invent flexible and elegant syntax.

Good passwords theory

2010-09-30T00:00:00+03:00

All of us are using passwords and most of us are typing them several times a day. And everyone knows the criteria to generate a good password. Random passwords are very strong from protection point of view but not from usability point of view. No body think about password unforgettable-ness and less-painful-to-type-ness. I saw many passwords that are very painful because they are just random. And I think it's nice to lose some randomness and make it easier to type and remember. There is still a lot of enough strong passwords that match this criteria as well.
Try to type and remember something like:

sudosuamigo1;
s3host-large
you&me2gether

Instead of:

1aWhuyZ
!nzu?Us0
n]1m!uaA

And you feel much better.

Populate database pattern - why is it so cool?

2010-08-26T00:00:00+03:00

Once my co-worker propose an idea to fill database with some fake data because we didn't have a gui forms yet and we have to show something that works to customer. That was a start for great pattern - populator.

Advantages

Later we didn't drop that functionality because it was helpful for everyone to always see what's new appear in application. And we find more and more useful aspects of having populator. It was so easy to attach new people to project because we deliver a short set of data within a source code that demonstrates everything we have in our app. Now whole team has a full data set and see if any page doesn't work because of a side effect. Frontend people feel more comfortable with populator because they don't care of how to create many assets in order to see pagination and things like that . Also UI issues became easier to detect because every element present on a screen for every person letting him detect browser specific bugs.

Implementation

I won't recommend any specific library to populate the db except the one to generate fake(Lorem ipsum) content. We prefer Faker but that is up to personal choice. Important note is that inserting data directly into db is bad idea because all application logic is ignored in this case: validation will be ignored and counters cache might be broken. That is why Populate gem is not recommended.

One nice improvement we made is pass _SCALE_ parameter to populator in order to control how many objects should be generated. Use minimal amount if you need to test that populator works and regular amount to get working system.
Another good idea is soft save(ignore if not valid) and output the number of created objects after at the end. This letting you if something went wrong during populate process.

Here is an example of how it may look like:

User.transaction do
  # apply the SCALE parameter to the argument. 
  # Default SCALE is 10 so 10 * 1 = 10. 
  # Generate 10 users by default
  scale(1).times do 
    user = User.new
    user.email = Faker::Internet.email
    user.first_name = Faker::Name.first_name
    user.last_name = Faker::Name.last_name
    user.password = user.password_confirmation = 'monkey'
    user.save # this may be false in case of uniqueness restriction or other problems
  end
end


users = User.all
puts "Users: #{users.count}" # make sure we actually made some users

At the end

I beleive that Populator should be a part of the build of every project. Yes, it needs some support but in fact it saves lots and lots of time.

Ultimate rspec matcher to test named_scope or scoped

2010-07-11T00:00:00+03:00

After having a good practice on using Ultimate rspec matcher to test validation I think it's time to implement one for testing named scopes - custom finders. Testing these finders is daily task. Here is how it can be done with minimum amount of code and maximum readability.

Discovery (Animal Planet)

What do we expect from the custom finder? We expect that it should find assets A, B, C and should not find assets D, E, F. And sometimes the order is important: it should find A, B C with exact order. With respect to let rspec feature let's take an example: Product has and belongs to many categories. We need to have a scope to filter products within the specified category:

describe "#by_category_id" do
      let(:given_category) do 
        Factory.create(:given_category)
      end
 

      let(:product_in_given_category) do
        Factory.create(
          :product,
          :categories => [category]
        )
      end

      let(:product_not_in_given_category) do
        Factory.create(
          :product,
          :categories => [Factory.create(:category)]
        )
      end

      # This might be tricky to redefine subject as the finder result
      # but in this way we can delegate the matcher to subject and 
      # avoid writing test descriptions.
      subject { Product.by_category_id(given_category.id) }

      it { should discover(product_in_given_category) }
      it { should_not discover(product_not_in_given_category) }

    end

Factory girl was used in this example because factories kickass when we test finders. As you can see the example has a perfect readability with no one line of plain English text. I didn't include the description in my examples but you can easily make them if they make sense for you.
Note: Be aware of the lazy loading of your finder. let is initialized lazy too. You should make sure it is called before the actual query to the database. If you don't want to care about lazy loading their is let! method that could be easily copy-pasted from Rspec 2.0. Unlike let it doesn't have lazy initialization:

def let!(name, &block)
  let(name, &block)
  before { __send__(name) } 
end

Testing sort order

If the ordering is done in non-trivial way let's discover.with_exact_order.

describe "#most_commented named scope" do
  let(:uncommented_post) { Factory.create(:post)}
  let!(:less_commented_post) { Factory.create(:post, :comments => [Factory.build(:comment)])}
  let!(:more_commented_post) { 
    Factory.create(:post, :comments => [Factory.build(:comment), Factory.build(:comment)])}
  }

  subject { described_class.most_commented }
 it {should discover(more_commented_post, less_commented_post).with_exact_order }
 it {should_not discover(uncommented_post) }
end

Be careful with default order. MySQL and Postgres sort objects as they were created by default. That is why generate objects in reverse order e.g. less_commented_post before more_commented_post is important to make sure that ordering is your code behavior rather than default db behavior.

Summary

I 've add this matcher to the previous one. Both matchers are available here accept_values_for. Let's start thinking of what else we can do.

Ultimate rspec matcher to test validation

2010-06-30T00:00:00+03:00

After a first thousand of tests using Rspec I fount it very annoying to repeat my self testing the standard code as spec code is usually twice longer than code it test. I've started to look for a way to simplify the patterns and make it reusable. Among other nice rspec tricks there is possibility to write custom Rspec matchers. Spec the validation is just two lines of code for each attribute now.

Existing solutions

After doing some search around I found Shoulda and remarkable gems that provide a way of doing this. Here is one of remarkable variant (others are just ugly):

it { should validate_numericality_of(:age, :greater_than => 18, :only_integer => true) }

But that is huge contradiction with BDD methodology because it doesn't make you think about behavior. People use to copy-paste code from model to spec and all possible bugs within. As the result nothing is tested... I don't even say about "test first".

Easy DSL and Implementation

When I implement validation everything I care about is: what values should be accepted and what values should not. An easy DSL that realize this pattern would be:

describe User do
  it { should accept_values_for(:email, "john@example.com", "lambda@gusiev.com") }
  it { should_not accept_values_for(:email, "invalid", nil, "a@b", "john@.com") }
end

That's it! Two lines of code per attribute. And that is perfectly fine for "test first" principle.
Rspec authors take care about custom Rspec matcher. So the implementation is very easy. Fork me on github: accept_values_for rpec matcher. The accept_values_for pattern is a true BDD way and unlike Remarkable it really do testing.

validates_uniqueness_of

This is very special validation because you always need more then one record to test it. So, the pattern for uniqueness validation is:

describe User do
  context "if user with email 'aa@bb.com' exists" do
    before do
      User.create!(@valid_attributes.merge(:email => 'aa@bb.com')
    end
    it { should_not accept_values_for(:email, "aa@bb.com") }
  end
end

Summary

Custom matcher is just 50 lines of code that make your life much easier. Do not scare to write your own. Custom matcher to test ActiveRecord#named_scope coming soon.

Advanced non-flat controllers hierarchy with rails

2010-06-08T00:00:00+03:00

REST became the best practice for organizing CRUD server side interface. But what to do when we have something more than hello-world application that goes forward from data CRUD and provides many presentations of the same data with different filters and different layouts?

How the flat controller architecture goes to hell

Starting from REST-full controller people use to stick to them and add more and more actions to the same class. And that is how the controller code goes to hell:

class UsersController < ApplicationController
#
# Filters
#
skip_before_filter :check_user_is_valid, :only => [:edit, :update]
before_filter :require_user, :only => [
:update, :edit, :settings, :disconnect_twitter, :connect_facebook_account, :google_contacts
]

before_filter :load_user, :except => [
:index, :new, :create, :remote_validate_email, :autocomplete_for_user_full_name,
:remote_validate_facebook_uid, :google_contacts
]

before_filter :check_permissions, :only => [ :edit, :update, :settings, :update_photo, :connect_facebook_account ]

before_filter :load_invitation, :only => [:new, :create]
before_filter :ensure_authenticated_to_facebook, :only => :connect_facebook_account
before_filter :initialize_form, :only => [:edit]

#
# And 20 almost independent actions goes here
# And the twice longer tests file for this controller
#

end

Namespaces and nesting

The root of the problem is in ignoring the following simple rule: The default choice for implementing new feature is do it in other controller It's always better to have 20 controllers with one action then one controller with 20 actions. Usually there is no compromise variant. And 20 controllers with one action ain't as bad as you imagine.

In order to handle the large number of controller we are actively using the namespaces and nested resources features. Let me give an example: User has many projects, Project belongs to category. We need to list projects per user and per category. In the flat hierarchy you would be confused but namespaces and nesting solves the problem.

map.resources :categories do |c|
  c.namespace :categories do |categories|
    categories.resouces :projects
  end
end
map.resources :users do |u|
  c.namespace :users do |users|
    users.resouces :projects
  end
end

class Categories::ProjectsController

  def index
    @projects = ....
  end
end

class Users::ProjectsController

  def index
    @projects = ....
  end
end

Everyone heard about restful authentication but not so many people applied this idea to other not so restful from the first sight things. Like TwitterConnectionController:

class Users::TwitterConnectionController < ApplicationController

  def create
  end

  def destroy
  end
end

Many people will be pushing all such staff to UsersController until their editors would run out of memory.

Out of scope

State control actions are not in REST but should be one day. Placing them in the same controller with CRUD is generally a good idea:

class ArticlesController 

  def {new, create, update, edit, destroy}
  end

  def publish
    @article.publish!
    redirect_to articles_path
  end

end

Some sugar from generators

Rails generate script supports namespaces very well:

$ ./script/generate rspec_controller users/categories
      create  app/controllers/users
      create  app/helpers/users
      create  app/views/users/categories
      create  spec/controllers/users
      create  spec/helpers/users
      create  spec/views/users/categories
      create  spec/controllers/users/categories_controller_spec.rb
      create  spec/helpers/users/categories_helper_spec.rb
      create  app/controllers/users/categories_controller.rb
      create  app/helpers/users/categories_helper.rb

And classes created in appropriate namespace and folder as well as specs for them. The only one thing you should do manually is add routes.

Drawbacks

Every engeneering solution has it's drawbacks. While you have different controllers that operates on the same classes you might need to reuse functionality among them. I preffer to solve it by mixing in a module:

class Users::ProjectController

 include UserNestedResource

end

Some people use inheritance instead. Both ways are almost the same. Nothing hard here as well.

Summary

At the end I 'll just repeat it again:
Default policy for placing two actions that has some kind of connection is SPLIT.

DOM element class - css reference or javascript reference?

2010-04-20T00:00:00+03:00

Designer have used element class attribute for CSS styling for ages. Nowadays classes are also used in javascript selectors. This overlapping cause additional problems: Changes in js may cause broken design and changes in design may cause broken js. The last one may be hard to detect especially when design and css are handled by different person.

Element class seems the best way to query the element from DOM in js code. Other variants like it's relative possition to other elements are very fragile. Let's say that we have the following DOM:

<a> Delete</a>
<a> Edit</a>

And both links are handled with javascript. How would you separate them?

By order(first, second) will cause problems if designer would like to change the order.
By id will fail if you would have a list of elements where each has these links
By label is just a fuck up and won't work for multi language application

The only one variant that provide the best stability is reference By class. But as stated above class also serves for another purpose - styling. The thing that might help is a special prefix(for example "js") for all 'JavaScript classes'. Like:

<a class="js-new-comment big-blue-button"> New Comment </a>

And the convention that classes with prefix should only be used in javascript selectors and classes without prefix in CSS.

Yes, sometimes it may look like:

<input type='button' class="js-follow-button follow-button"> Follow this person </input>

But that makes sense to eliminate maintenance problem described above.

Encapsulation in Ruby: 'private' doesn't give expected level of privacy

2010-04-05T00:00:00+03:00

The classic OOP pattern usually called encapsulation implemented in ruby with private and protected ~~keywords~~ methods. The distinction between API and implementation works great with both. The problem is that private doesn't hide method for inherited classes.

See the simple example below:


class A
  private
  def implementation
    puts 'private A'
  end
end

class B < A
  def api
    implementation
  end
end

B.new.api

And we won't get exception at the last line.
So, in fact private do not do what you expect it to do. Use cases that distinguish private and protected are more specific. At first: implementation method can not be called like self.implementation even from class A.

class A
  def api
    self.implementation #exception here
  end
  private
  def implementation
    puts 'private A'
  end
end

A.new.api

Seamless pure gap. You can do such call if the method would be protected. Can't imagine the case when this limitation is useful.

The second difference is a bit esoteric. Two instances of the same class can access protected methods of each other but not private methods.

class A
  def api(another) #suppose to receive an instance of a
    implementation == another.implementation
  end
  protected
  def implementation
    puts 'private A'
  end
end
A.new.api(A.new)

Fine for protected but fail for private. Well, this kind of protection do not make a real because of it's rareness.

My conclusion: don't use private at all

You might do any decision on the points above.
My conclusion is to not use private at all, because it doesn't give the level of privacy that I expect. Difference form protected is so pure that it do not have any real benefits.

A dream to develop application that I will use

2010-03-29T00:00:00+03:00

Developing sites for communities you never join, enterprise applications for company you never work in, services you never find useful... a serious problem of IT industry. All misunderstanding between customers and developers come from that reason.

It would be so great if we could always feel ourselves as a user of our applications and make something useful, but not spend time on improving features that won't be familiar. Being a user of the developed application makes it really possible to develop ergonomic and pretty design.

Unfortunately most of the companies are ignoring that simple rule and unfortunately in some cases it is hard to follow. But anyway that is the only one way to build the best product.

Advanced SQL and scopes stack with ActiveRecord

2010-01-24T00:00:00+02:00

If you ever work with rails application that is a little bit more complex than a simple CRUD you would know that some of the ActiveRecord magic doesn’t work for complex SQL queries. I am primary talking about scopes stack feature.

Let’s review the following scope that suppose to be usable in different combinations with others:

scope :network_of, lambda {|user|
{
:select => "u.*"
:from => "users u, followings f1, followings f2",
:conditions => "f1.follower_id = #{user.id} AND " + 
"f1.followed_id = f2.follower_id AND " +
"f2.followed_id = u.id"
}

It suppose to returns all people that are followed by people that are followed by the given person(second circle). From the SQL point of view that is the simpliest and fastest way to do that with a plain SQL. But this solution will have some issues with Active Record scopes stack magic. See that User.network_for(current_user).all(:limit =>5) will result in SQL exception primary because :limit => 5 doesn’t know about the table alias users u. We can not use it in fact.
The second problem comes to the foreground when we will try to use ActiveRecord features like User.network_for(current_user).all(:include => :orders). ActiveRecord handles :include in the very different ways and in some cases you will see the SQL exception here as well.
The problem is that :include sometimes appends some joins to the query that is concatenated to the last table in the :from parameter. To solve that we should make “users” table to be the last one declared in :from parameter.

:select => "users.*"
:from => "followings f1, followings f2, users",
:conditions => "f1.follower_id = #{user.id} AND " + 
"f1.followed_id = f2.follower_id AND " +
"f2.followed_id = users.id"

Summary I would say that using :joins instead of :from/:conditions would give more flexibility and stackability to your scopes but sometimes :from is more clear and here you got the tip how to use it.

Vim as IDE - new development era for me

2009-12-01T00:00:00+02:00

I was using Vim to do the minor editing in config files for ages. Vim is advanced on basic navigation and editing operations. However I was always prefer IDE for programming because of it's specific to language navigation('go to definition' feature is awesome!), integrated debugger. But while gaining the experience I noticed that all advanced features began to be less important then basic ones. And you should try Vim if you feel the same.

The basic concept of vim is that: Vim think in the same way as you do. For example: I want to delete 'd' everything inside 'i' brackets ')' and vim understands my command: di)
You shouldn't type a dozen of button to move cursor around and finally delete the selected fragment. It's hard to adapt because people are use to think in the way that there text editor do. Vim does vice versa. Some people are saying that they don't like vim because it make them think a lot. That's wrong! Vim is trying to make you free from thinking about editing and concentrate on the content.
General editing in Vim considered up to every small details. Ask yourself if there is some usual task that I am doing inefficiently and search the Vim feature to optimize that in 3-5 keys. And in ninety nine percent of cases you will find it as built in vim command or a plugin.

Objects behaviour inheritance with RSpec

2009-10-27T00:00:00+02:00

About half of a year ago I was writing about object interface and Liskov Substitution Princeple. In short: Any class instance that extends the base class should pass all unit tests behaviour tests written for base class instance. It was a surprise for me that this concept has already been implemented in RSpec.

My previous article was primary inspired by Java programming language and it's interface concept. Unlike Java, Ruby does not have interfaces, but behaviour inheritance is still actual for both languages. RSpec seems the first testing framework that provide the ability to validate LSP and behavior inheritance with it_should_behave_like.
With Ruby modules(mixins) feature we can build reusable code and include it in different classes(read more). With RSpec we can bundle the tests as well.

Let's review the following module that uses one of the Rails callback and adds some logging:


module LoggedModel
  def after_save
    super
    handle_logging
  end
end

and the some tests group for this module:


describe LoggableModel
  it "should be loggable" do
    LoggableModel.should ...
  end
end

Now, we have a tested code that is going to be used in many cases like this:

class MyModel
  include LoggableModel
  def after_save
    do_some_other_thing
  end
end

OK, let's see what we have: after_save in MyModel overwrites after_save in LoggableModel and breaks the logging. This is simplest example when the behavior inheritance may be broken. Rspec shared examples groups allows you to ensure that the code in LoggableModel is used correctly from any inherited class. Let's change the definithin of LoggableModel tests.


shared_examples_for "logged model" do
  it "should be loggable" do
    subject.should...
  end
end

'Subject' is the ultimate RSpec magic that let us make a simple abstraction with the tested class and reuse these shared examples in MyModel spec:

describe MyModel do
  it_should_behave_like 'loggable model'
end

In this way we will rerun the LoggableModel examples for MyModel and make sure that it's behavior wasn't broken.

Computer science ain't as good as it should be

2009-07-30T00:00:00+03:00

When I start studying computer science in the University class I truly believe that it is all I need to get started writing good programs. Algorithms > Functional programming > OOP > Databases > MVC is a step-by-step path of all people that learn programming of the web application. But...

Programming basics like conditions, variables, cycles and functions are mandatory for everyone just like writing and speaking for a baby.
But more advanced knowledge are less useful and optional. All of these:

Object hierarchy
Normalized database model

are turned into dust by more important

Do it as fast as possible
Customer wants!

Each framework declare it's own way to organize the application Classes. If you are working on CRUD(create-read-update-delete) data application you don't need to think about object hierarchy yourself anymore. All you need to do is understand where you need place the function that handle data processing. And the principles a well declined in the framework manual. Normalized database is only a myth in a real world application. In 80% of cases the database design is evident to the developer. In other 20% the database couldn't be normalized at all.

When you start learning programming try to let your knowledge be "moved to production" as fast as possible. In other words: Have a programming practice as fast as possible and you will understand what kind of programming theory you really need. Concentrate your education on all cases you met and avoid learning things that you could possible come across.

When BDD user stories is productive

2009-06-29T00:00:00+03:00

When you are skilled Unit tests writer it is not very easy to estimate the potential of BDD. Moreover in most cases BDD is used as a human-readable wrapper for unit tests. Do you really think that programmers need such abstract layer? Of course not... Users needs them.

Saying Users I mean every non technical person who will treat to application quality some how. Featured innovation of BDD is User Stories - a human-readable bit of specification with technical test implementation behind it.
Let's review the following example from the blog specification:


  Given I signed up as Author
    When I write article "TDD and BDD"
    And text of article is "BDD is ...."
    And I post article
    Then I should see "Article is created"
    And I should see article title "TDD and BDD"

As you can see unlike TDD human readability is the term that was brought to the high priority in BDD. But that is not yet a BDD story.
We need the programming implementation for each phrase in the story, like this:


"I signed up as Author":
     signInAsAuthor();

Programmer's task is to chose the implementation that will validate described behavior. User doesn't know about any programming back end of the story. When the test failed he just knows that some behavior of the application is not working properly any more.
From this point of view the key principle of user story implementation is that it should guarantee with the most high probability that the user can complete described actions.
Therefore user stories are the most productive when the implementation is done using GUI Robots like SWT bot, Selenium.

Pragmatic review of Google Wave technology

2009-06-08T00:00:00+03:00

Have a good time watching the Google Wave presentation video. Looks like an excellent open technology that will make our life easier. I like the overall idea. It is really awesome. But I personally admit some things that seems useless to me...

Starting from the very beginning I need to say: Wow! Now we can organize all communication threads to the widespread object called Wave that will be readable and editable by any other application. No more browsing dozen of sites and tracking all changes around - all interested waves can be reorganized in the way you want.
Let's review the features now:

Act as Wiki. Wiki can be handled via Waves too, but not sure about parallel editing. That looks sexy on presentation but has no sense in action.
Translation to another languages. I would like to have that on my desktop. Don't think it would be useful then doing the real time communication. Machine translation never was good enough.
Spell checker based on language model. Many people around worked on language model and no one has built it strong enough to handle automatic spell check. Let's wait for release and see if Google guys reach the success here.

Double select from a single table in complex SQL query.

2009-06-02T00:00:00+03:00

When I start learning SQL I have to write a lot of nested queries when working on complex select statements from a single table. Now, I feel more comfortable with it and show how to avoid nesting. There are cases when you have to compare one row of the table with all others to get the result:

Select next element to current by the value of some field
Select records that are unique by some complex expression
Select all records that have the same parameter as given record.

You can get rid of nesting in all above cases. Let's review the example.

Let's say we have the list of elements in `elements` table and we have the id of some element in this table. Now we have to get the next element from the list ordered by `sort_field`. That could be easily done with the nested select statement:

select * from elements e 
where e.sort_field >= 
            (select * from elements where element_id = #value#) and
         e.element_id != #value#
order by e.sort_field limit 1;

Looks not very cool. Let's do the select statement from `element` table twice to avoid nesting:

-- The e1 is the result row and e2 is a helper row 
--that was picked up using nested select previously.
select e1.* from elements e1, elements e2
--conditional statement will look like
where e1.sort_field >= e2.sort_field and 
     e1.element_id != e2.element_id and 
     e2.element_id = #value#
-- and the end of the query remains almost the same
order by e1.sort_field limit 1

Do not scare of double select from single table. That is the same technique as if they were different tables.

How to choose a sexy title for your article

2009-05-13T00:00:00+03:00

Most of the people will not spend a minute reading your article if they wouldn't get interested from the very beginning. What resides at the beginning of any article? It is the title of course. That is why choose a good title is very important for every content posted in the net.

Title should not reflect the overall idea of your article, let it just be cool. Attracting users attention is more important. It should be the key point of what are you righting about

Here is my idea: Title looks cool for the reader if it is close to his thoughts.
Reader should feel himself close to the topic you are going to speak about. Let's find out and see what do our mind filled in and describe some title choosing principles:

Emotions

Emotions acts like a catalyst. Expressing strong emotion in the title will give you the most effective result. People are getting interested not only when your emotion correspond to their but also vice versa.
Example: JBoss Seam usage makes the application code crazy!

Questions

In most cases the reader is somebody who want to know more about the world. Put a question in the title and it will engage everybody who wants to know the answer.
Like this one: What do you expect from the interface?

Famous topics

Humor is top theme on every forum, every blog, every site all the time. Joking title is a good way to awake the interest.
Sex. You noticed the title of this article, right?
Weather
Politics
Sports
etc.

Making a slight reference in the title to one of above mentioned topics will attract much more attention to your article
I hope this information was helpful. Thanks for reading.

What do you expect from the interface?

2009-05-05T00:00:00+03:00

Read a great article today - Liskov Substitution Principle (LSP). It let me understand that Interface is not just a list of methods.

Let's review the following utility function:

public static void processList(List list) {
          list.clear();
          list.add(new Object());
}

It accepts the object that implements java.utils.List interface as the parameter. What do you expect to receive in result? The list with one element, right?
But does the result really meet your expectation? Who knows, the object that implements list interface do not guarantee you 'Act as container' behavior. It just implements all the methods of the interface and nothing else.
Actually all methods of the interface ~~should have~~ have a notation of their usage. If you do not follow these notations - you should not inherit your object from the interface even if the object has all the methods of the interface.

Idea: Maybe one day programming languages will have unit tests as part of interface and validate each instance that implements this interface with them.

JBoss Seam usage makes the application code crazy!

2009-05-04T00:00:00+03:00

Seam Application has a number of good innovations but let's find out if we lose some advantages that we had before. I started application development with JBoss Seam one year ago. With the code base growth I have had more and more problems in workflow development .

I noticed that there are many problems which we have not had before, for example:

Utility methods calls are not straight and clear now.
Find all usages IDE has feature became useless because of mass EL calls.
There is much of the meta programming in such a severe OOP language as Java.
It is impossible to debug JSF templates
Seam Exceptions look so unclear.

Programming with Seam has all disadvantages of script languages like PHP or Ruby. I definitely admit that some features like Conversations and Native AJAX support made a big step forward. There is a lot of information in the net about How cool is seam but I tried to reveal the underside.

Think twice before switching to Seam.

Update and create timestamps with MySQL

2009-04-30T00:00:00+03:00

A lot of relational tables need created and update timestamps columns. I prefer having them for all tables with no exception. However, most of applications I am working on are running MySQL. MySQL has minor limitation on timestamps. Unfortunately you can create only one time stamp column that has DEFAULT NOW() value. Read more to see how to avoid this limitation.

The simplest way to do this is create the following columns in the table:

stamp_created timestamp default now(), 
stamp_updated timestamp default now() on update now())

But MySQL will return the following error:
ERROR 1293 (HY000): Incorrect table definition; there can be only one TIMESTAMP column with CURRENT_ in DEFAULT or ON UPDATE clause
You can use only one of the definitions in one table. However here is the way how to create both timestamps columns:

 create table test_table( 
id integer not null auto_increment primary key, 
stamp_created timestamp default '0000-00-00 00:00:00', 
stamp_updated timestamp default now() on update now() 
);

Note that it is necessary to enter nulls into both columns during "insert":

 mysql> insert into test_table(stamp_created, stamp_updated) values(null, null); 
Query OK, 1 row affected (0.06 sec)
mysql> select * from t5; 
+----+---------------------+---------------------+ 
| id | stamp_created       | stamp_updated       |
+----+---------------------+---------------------+
|  2 | 2009-04-30 09:44:35 | 2009-04-30 09:44:35 |
+----+---------------------+---------------------+
2 rows in set (0.00 sec)  
mysql> update test_table set id = 3 where id = 2; 
Query OK, 1 row affected (0.05 sec) Rows matched: 1  Changed: 1  Warnings: 0  
mysql> select * from test_table;
+----+---------------------+---------------------+
| id | stamp_created       | stamp_updated       | 
+----+---------------------+---------------------+ 
|  3 | 2009-04-30 09:44:35 | 2009-04-30 09:46:59 | 
+----+---------------------+---------------------+ 
2 rows in set (0.00 sec)  
mysql>

Aphorisms about programming

2009-04-29T00:00:00+03:00

I am not clever. I am just yet another way to use google.
Good programmer should be lazy.
The hole in your pocket is not critical unless you have a hole in your other pocket.

Software is like sex - better when it is free. Linus Torvalds.
Sometimes the theory meets practice. Practice wins. Always.
10% of your zeals brings you 90% of success. Other 90% just 10% of success.
About haste: The perfection of a clock is not in the speed of the movement, but in its correctness.
Don't be satisfied with an apparent fix; full understanding is the standard.
If you only know one language, no matter how well you know it, you're not a great programmer.
The only "best practice" you should be using all the time is "Use Your Brain".

Ant task to install tomcat service26px

2009-04-28T00:00:00+03:00

If you are familiar to run Apache Tomcat as Windows service, you can install it with the ant task. build.xml file sample under the cut line.

First of all let's do some properties setup:

catalina.home=d:/java/tomcat-6.0.18
tomcat.executable=${catalina.home}/bin/tomcat6.exe
tomcat.service.executable=\
${catalina.home}/bin/service.bat
tomcat.service.name=myproject
# parameters that will be used in the jvm
tomcat.service.jvm.ms=128
tomcat.service.jvm.mx=256

Service startup task consists of 5 exec commands:

Stop service if currently running
Delete service if installed
Install service with bat script
Update service with some parameters
Start service

Here is the implementation:


    <target name="tomcat.service.install" description="Install tomcat service">
        
        <exec command="net stop ${tomcat.service.name}"
              failifexecutionfails="false"
              failonerror="false"/>

        <exec command="${tomcat.executable} //DS//${tomcat.service.name}"
              failifexecutionfails="false"
              failonerror="false"/>
        <exec executable="${tomcat.service.executable}" 
                  failifexecutionfails="true" failonerror="true">
            <env key="CATALINA_HOME" value="${catalina.home}"/>
            <arg value="install"/>
            <arg value="${tomcat.service.name}"/>
        </exec>
       <exec executable="${tomcat.executable}">
            <arg value="//US//${tomcat.service.name}"/>
            <arg value="--JvmMs=${jvm.ms}"/>
            <arg value="--JvmMx=${jvm.mx}"/>
            <arg value="--Startup=auto"/>
        </exec>
        <exec command="net start ${tomcat.service.name}" 
                  failifexecutionfails="true" failonerror="true"/>
    </target>

Clear upload file input field

2009-04-23T00:00:00+03:00

Many web developers came to the problem that they are not able to change value of file input field from Java Script in the web application. There is no access to value field of file input tag because of security restriction. However there is a trick how to erase this field if you need it in your application.

You can simply redraw the html block where your input tag is located. In this case all data will remain the same except the selected file value.


<div id="uploadFile_div">
<input type="file" class="fieldMoz" id="uploadFile" 
            onkeydown="return false;" size="40" name="uploadFile"/>
</div>
<a onclick="clearFileInputField('uploadFile_div')" 
                         href="javascript:noAction();">Clear</a>

Java Script function below looks strange but acts exactly in the way we want:

 
<script>
function clearFileInputField(tagId) {
    document.getElementById(tagId).innerHTML = 
                    document.getElementById(tagId).innerHTML;
}
</script>

Components of success

2009-04-10T00:00:00+03:00

People from different countries are reaching success in completely different way. And that is good because no matter who you are you can be successful. I heard a lot of success stories and now can summarize - what components are required to build your own success story...

Here are the components:

Hard work

That's it. This is may be the most important one. Don't think you can be successful while sitting on the chair and do nothing.

Intellect

Many people work hard whole life and stay where they are without moving ahead. That is a pity that sometimes hard work is done in the wrong direction. Also try to look several steps ahead and see the perspective of the work you do.

Risk

Risk is a very helpful component. You can go the most reliable way. But in fact add some risk can seriously improve the result. Do something without risk is impossible because there is always a risk that your work is useless.

Luck

Yes, that's it. This component is very important. All above components are nothing without luck and there is also some good and bad tricks in our way that we can not forecast.

Brain async

2009-04-09T00:00:00+03:00

Many people came to the question: Why do our brain is having so many gaps: forget important info, quickly change opinion, stuck when doing usual actions, etc? Well, try to look from the different side.
Our brain is about 4 million of neurons('processors') that works completely async: neural impulses follow though brain without any rules and permissions, every neuron process the information in there own way, no body is able to re implement such a large information processing system. From this point of gap are normal and true solutions and that is being solved by our brain every day is a miracle.

Technical documentation

2009-04-08T00:00:00+03:00

There is no one technical book I read more than a half. Is everybody doing this way? I recommend book authors to explain themselves in more short form.

Upload files with Selenium IDE

2009-04-03T00:00:00+03:00

I started using selenium about 2 weeks ago. Find it as a very good QA tool. But after a few successfully written tests I met the problem: Selenium is not able to use file input field. JavaScript permission restriction doesn't allow it.

Here is my solution written with JInvoke library: Jinvoke provides the classes to simulate the input to file chooser form.

The problem I meat is selenium is stuck when simulate the click on upload file input. I have to launch the concurent thread that do the file name input. Thread code(Note file name should be given in Java format like 'c:/boot.ini):

import javax.swing.*;
import java.awt.event.KeyEvent;
import java.awt.*;

/**
 * @author Bogdan Gusiev
 *         Date 29.03.2009
 */
public class FileChooserThread extends Thread {

    public FileChooserThread(String file) {
        super(new FileRunner(file));
    }
}

class FileRunner implements Runnable {

    private String fullName;

    public FileRunner(String fileName) {
        this.fullName = fileName;
    }

    public void run() {
        try {
            Thread.sleep(1000);
            Robot robot = new Robot(); //input simulation class
            for (char c : fullName.toCharArray()) {
                if (c == ':') {
                    robot.keyPress(KeyEvent.VK_SHIFT);
                    robot.keyPress(KeyEvent.VK_SEMICOLON);
                    robot.keyRelease(KeyEvent.VK_SHIFT);
                } else if (c == '/') {
                    robot.keyPress(KeyEvent.VK_BACK_SLASH);
                } else {
                    robot.keyPress(KeyStroke.getKeyStroke(
                                   Character.toUpperCase(c), 0).getKeyCode());
                }
            }
            robot.keyPress(KeyEvent.VK_ENTER);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

}

Here is selenium call method:

protected void chooseFile(String element, String fileName) {
           Number positionLeft = selenium.getElementPositionLeft(element);
           Number positionTop = selenium.getElementPositionTop(element);
           new FileChooserThread(fileName).start(); //launch input thread.
           //this method will held current thread while FileChooser gives the file name
           selenium.clickAt("file", positionLeft + "," + positionTop); 
}