Emphasized Insanity

Bag O' Links - 26/2/2011

2011-02-26T00:00:00+02:00

Yeah, since i’ve been working for fiverr.com it’s been really hard to find the time and compile a decent bag’o’links.. hopefully i’ll get back to that soon along with posting some cool Rails experiences from Fiverr.

Well, back to business:

Treasures

Rack-modpagespeed – a rack app that does all kinds of cool things to make your pages load faster.
Terminator – ninja command line.
Rails cells – not really sure what it’s actually good for.

Rails I18n - default interpolation values

2011-02-26T00:00:00+02:00

In the past few weeks i’ve been busy I18n’ing on of the applications i’m working on. As everyone already know and i assume also agree,
I18n’ing adult applications is a real PITA.

one of the problems i was facing is the need to find and replace a certain string that appeared in 30+ translation values. One option
was to go over one by one and replace that string with an interpolation key in the translations – after that working hard to also add that interpolation value to
every place one of the translation keys was called… shiish.

Lazy as i’m i thought it would have been nice if Rails knew how to grab default interpolation values from somewhere so i decided to give it a try

this is a first draft and a VERY ugly mixin i wrote that loads default values from SettingsLogic (in this case, edit your own):

i hope it helps.

Convention over Manipulation

2010-11-10T00:00:00+02:00

In the past 5 years or so I came across countless Rails applications, some were written by several of the best coders i know around (Ryan Bigg and rBoard, Eric Davis and Redmine.. the list obviously carries on) and some were written by less “experienced” developers. Some write awesome codes and have great insights on almost everything you ask them, while others just try to “get things done” and not necessarily find the right way out.

People love Rails. Every person i forced into using it (@kensodev – you for example) almost immediately fell in love with it and started calling out those famous “i can’t believe it does that” statements rails noobs so often yell in excitement.

From that point on I see people going into 2 different directions:

Those that learn how to use Ruby and Rails’s strengths when needed, and when not to.
Those that will bend, twist, fold and squeeze Ruby and Rails into something it is not.

Those who turn to the second path aren’t stupid, bad developers or people that enjoy software sodomy (maybe), they are just people that haven’t invested enough time and effort in finding the right solution for their problem – an act that will probably end like the famous Twitter “Rails can’t scale” saga.

By manipulations i mean…

Twitter did something like that, although i am not in the heart of things i do remember reading some of the discussions made by Obie Fernandez and Alex from Twitter and it seems like they were using Rails wrong, and where it shouldn’t have been.
Keeping aside the fact that Rails and Ruby (just like any other language and framework) aren’t a magical solution to every single software pickle around, when you do decide to use a tool you better do it right.
Twitter overused a method named #kind_of? that identifies an object’s class (according to Obie’s response about scala at twitter) as a result of an architecture failure or just plain “if it works don’t touch it” mantra, and probably there was a better solution even before Rails was switched out.

Many developers abuse their tools, Rails developers aren’t unique. iv’e seen .Net pages that held a VIEWSTATE across the entire page but with Ruby and Rails i think it’s easier to skip that line between “Being creative” and “Being a smartass” which may lead to undesired side-affects on your application.

Another good example of a pointless manipulation is here on The Israeli Ruby on Rails group – session variable and model code discussion.
Long story short, Michael wanted a way to pass a session variable into a model. why you ask? because he needed that value to be rendered as part of to_xml.

Argh.

This is a perfect example of someone that bends conventions (MVC in this case) just to write “less code”. Obviously, the right answer to his question is “RTFM dude, use builder” and a happy little something.erb.xml view (which would have also given them the benefits of ActiveSupport and friends). further more the boys at Michal’s did this WTF plugin to make it easier for them to manipulate rails next time. sigh.

It’s easy to manipulate a framework like Rails. Ruby is durable, dynamic and fun – you don’t really want to use anything else when you are already using it. But sometimes like Michael showed – it’s easy to misuse those wonderful tools.
The thing that you as a developer should keep in mind is that you can probably solve every problem you have without bending things too much. yes, we all bend and manipulate Rails code in one way or another but the thing we need to keep in mind is to stay focus on the primary goal: finding the best solution, not using the first one we think is working.
Rails always supplies you with a tool to perform a general task (to_xml for example) and a way to customize it on your own (xml views) – personally i always check if the tool/method/object i think of using is the simple or the customizable one and if i really need more than the simple need in a task.

i wouldn’t use #to_xml on an instance if i had to include/exclude too many fields from it or it’s associations, i’d use builder. but if all i need is the basic object in xml format, there is no reason i should re-build the entire thing.

Conclusion

Put an effort in finding the right solution for your problem. don’t think that because rails is easy and simple you can make do what ever you want and it won’t cry :)
There are people always willing to help on IRC (#rubyonrails, and #railsbridge on irc.freenode.new) and on twitter if you just drop by and ask.. it’s easier to learn that way.

Bag O' Links - 9/11/2010

2010-11-09T00:00:00+02:00

Treasures

HTML5 Boilerplate – TML5 Boilerplate is the professional badass’s base HTML/CSS/JS template for a fast, robust and future-proof site. really awesome.
Developing your Resume – your resume, on Github.
Google API console – test drive your Google API usage.
Hola – async’ed JS chat application.
.gitignore template for Rails – looks good, only `database.yml` is missing (i copy it from a server-resident shared folder after capistrano’s code-update).
RubyRacer – Run V8 JS interpreter through Ruby.
Ruby XMPP bot – running on GMail’s chat, nice (reminds me of Barking Iguana’s XMPP4R-simple makes XMPP simple test).
ZeroMQ – another message queuing engine.
The Devchix wiki – really good stuff here, i actually converted their installation process into a muppet/babushka recipe for new server installations.

Bag O' Links - 3/11/2010

2010-11-03T00:00:00+02:00

Treasures

Shapado – Rails + MongoDB based stackoverflow clone.
SproutsProject – Sprouts is an open-source, cross-platform project generation and configuration tool for ActionScript 2, ActionScript 3, Adobe AIR and Flex projects.
Whereuat – Adds a slide out panel to your rails app that directs clients to test stories that have been marked as ‘delivered’ in Pivotal Tracker. brilliant.
RubyRegex – Ruby regular expressions library.
OpenStack – open source cloud control and management tools.
Jo – Awesome JS UI framework.
SocialStream – a Devise dependent social features plugin for rails.
ruote – a workflow engine, not a state machine.

Scaling a 500 million rows table - planning

2010-10-26T00:00:00+02:00

438 million, 218 thousand and 363 rows.

Current count of indexes on the table on the other hand, is 0.

I imagine you all ask how long does it take to perform a `select (*)` on it, well, I stopped waiting after about 4 minutes.

This peculiar situation happens in one of our client’s projects, the table itself fills up from a daemon that listens to some kind of a stream with the current daily amount that goes somewhere around 4 million rows per one single day. all we are storing is a simple integer and a foreign key ( “sample” ).

Crazy, i know.

This table (“samples table”) should allow the app to access any subset of query, but mostly based on a `WHERE user_id = xxx` clause, so i can’t offload “old” rows away into oblivion (or an archive).

After a little research, i decided on the following options:

NoSQL indexed storage (Redis, Mongo or CouchDb)

The amount of data is huge, so i was initially looking for some information regarding data size limitations on those NoSQLs:

Redis 1.x had some trouble with large datasets, but Redis 2.x now supports virtual memory storage which basically gives me some room to maneuver.
MongoDB is limited as far as the collection count (585 max) but that doesn’t bother my case, don’t need that much. what i do care about is the limit on the list size which is basically as far as your memory goes (2GB on a 32bit installation, 4GB on 64bit) which is still kind of a trouble.
With CouchDB it’s a litter different, it depends basically on your `_id` column size (number of bits you define for usage).

What i am planning on doing is to create some kind of sampling and to keep to most recent data in a NoSQL storage engine.

Use internal MySQL partitioning

Partitioning seems like a reasonable RDBMS level solution, but on mysql it’s limited to 1000 partitions only and they are also not very dynamic (i can’t create an automatic partitioning engine that will.

Current direction

We decided on trying the following flow:

Having the HA data in a NoSQL implementation, in our case it means we keep about 6 to 10 million rows in a NoSQL instance.
The most important data (insertions in the last 48 hours) needs to stay at the top resolution, but older data can lose resolution so we came up with this idea:

We will create a cron task that will run every hour processing all the samples from the last hour and will avg it up, later storing it in a statistics table with only the hourly avg as the sample value.
another task will do the same scoping out from hours to days, and from days to weeks which will be our lowest resolution.

This method drops our row counts in places we can afford data resolution decrease in 10s of millions of rows.
This process is still under development so if anyone has a better idea and care to enlighten us, please do so.

Bag O' Links - 24/10/2010 (back!)

2010-10-24T00:00:00+02:00

Yeah, i know i said bag’o’links will go out of service a few months ago and that i’m going to move it the Nautilus6 website, but overall there were few other things i had to do with Nautilus6 prior to that.. i’ll get to it when i can but it doesn’t mean i can’t re-post links does it?

Links and fun

25 points of usability checklist – Some points to follow when your UXing.
Learning Javascript with object graphs – Javascript is hard, graphs are easy.
Asynchronous javascript queue – Elad Ossadon (the less-adorable Elad) is showing some JS techniques.
Rails views in Javascript – an old post yeah, but might be a good baseline for a solution i heard about to the “maintain templates on both client side and server side” problem. sounds weird but i’ll explain everything in a future post.
W3’s guide to RTL supported web pages – yes, we’re a minority but we need support too :)
Cross browser CSS guidelines – from smashing.
Scaling with Hadoop – from sitepoint.
Callbacks in Rails3 – slideshow.
Useful CSS3 tools – timesaving.
Cuking it right – writing maintainable acceptance tests.
Beginning Erlang for ruby developers – slideshow.

Favorite RailsRumble Apps

Last weekend this year’s “RailsRumble” took place, of course i missed it but i did had a chance to peek in and choose some favs:

RailsWizard – generate your own Rails template by a simple click/choose interface. i like it so far and anxious to see how far it goes.
Caviar – charge your clients by results.
Empower – build a development environment around your app, nice.
FontStacks – generate CSS that contains awesome fonts (font-face).
GitWrite – blogging for nerds, looks like a web supported Jekyll.. nice, i like it and waiting for some improvements and additions.

Treasures

webmock – Library for stubbing HTTP requests and setting expectations on HTTP requests in Ruby
RailsERD – tool that extracts an ERD schema out of your models.
Test Driven Javascript – that’s a good book folks, go get it.
jsKata.timezone – a javascript library that works with timezones and makes it easy to handle for us noobs.
Cool set of CSS3 buttons – nice.
Backbone.js – MVC to your JS. wicked.
50 free CSS templates – some are really good looking.
Formalize – CSS framework for forms.
Updated list of Cheatsheets for developers – web developers that is.
MVC in Mootools – an MVC framework that uses Mootools.

Last sad note

About 10 days ago we lost a friend. Fares Yussuf Donaldson (a.k.a invalidrecord on IRC) died.
We will miss you buddy.

No more bag of links

2010-06-18T00:00:00+03:00

For the past 18~ months or so i’ve been trying to keep a reasonable posting rate of my “Bag o Links” collection. Recently it has become too hard to spare the time needed to bounce my favorite links up, especially due to the fact we are investing most of our ‘extra’ time in Nautilus6 to sketching out a new company website.

Bag o links will be back on the new company site pretty soon i think, but for now it’s on hold.

Bag O' Links - 24/5/2010

2010-05-24T00:00:00+03:00

Phew! it was a long time since i last posted Bago’. Israel keeps me busy.

Dive into HTML5 – HTML5 guide. looks ok.
Very simple parallelization with Ruby – not the best implementation i’ve seen, but will work on small scale solutions.
Ruby’s #inject- i have been using ruby and rails for more than 4 years, and i got how to use #inject only yesterday.
Ruby’s TempFile explained – Avdi’s share on Tempfile.
OrientDB – Another NoSQL implementation. supposed to be really fast.
Rails 2.3.6 changeset – probably my last version.
Ruby VM shootout – interesting.
Upgrading Rails 2 to Rails 3 – wuhoo.
Ruby Proxy Pattern and Dynamic Delegation with ActiveSupport BasicObject – implementation and an excellent walkthrough.
Ruby’s equality operands – a good read.
Write-Through Cacheing is an Essential Part of a Healthy Scaling Strategy – great scaling read.
List of generators for Designers – some are nice.
Testing Javascript with cucumber – ha! nice.
MongoDB master-slave installation – howto.
Feature flipper for Rails – deploy specific features, awesome.

Treasures

jsChat – a nice chat engine using javascript and MongoDB.
Google Fonts API – nice, font-face will get a boost from this one. (here are the fonts)
fancy-buttons – Easily style buttons with beautiful CSS3 features like gradients, rounded corners, etc. Don’t worry the buttons also degrade nicely for browsers with no support. requires Compass though, which is annoying.
ZeroMQ – java based MQ manager. looks great.
ShowOff – presentations made simple. wicked.
Google’s S3? – Storage for developers.
to_xls – convert objects to excel spreadsheets.
KingSOA – KingSoa orchestrates a SOA landscape, by knowing where services live and how to call them.
Linkscape – a library to access SEOmoz’s API.
ActiveCart – a basic shopping cart implementation.
MongoHub – native MacOS MongoDB GUI.

Bag O' Links - 4/5/2010

2010-05-04T00:00:00+03:00

Treasures

A quick example of Facebook’s REST+OAuth in Ruby – simple really.
Guilded -a framework for building web based components centered around current web standards and best practices. i wonder how it will turn out.
CloudyScripts – Scripts to facilitate programming for infrastructure clouds.
Ogit – Facebook Open Graph terminal.
Barista – Transparent coffeescript support for rails 3.
CSS3 for IE – Another attempt.
SuperModel – In-Memory models for Rails.
a collection of MongoDB utilities – is right here.
TwiBot – a Twitter bot in Ruby.
jStackMenu – a really nice stack menu implementation in jQuery.
Sonia – a Team/Projects monitor panel.
Fedena – open source, school/campus management ERP.
Cinch – IRC framework in ruby.