See the session abstract for further details before digging into the slides below.

I’d also like to take a moment and state what a great experience the different tracks, venue, organization, catering, Internet access and entertainment bits was.Well on par, if not beyond, the vast majority of international conferences out there.

At no cost (free, grátis, gratis, gratuit) to attendees.

Bean bags, hammocks, work spaces, huge ass presentation areas, dedicated sound / camera crews, quizzes, projects, soldering workstations and a constant pizza and red bull supply.

A great reflection of the innovation at work inside SAPO and a peek into emerging technologies being rolled out.

I’ll dedicate a few blog posts to cover some Ruby VM sections in detail during the next few weeks if there’s sufficient demand – please leave a comment if that appeals to you.

Follow me @ github or twitter if you enjoyed this presentation.

Ruby hashes and the symbol table

MRI ships with a generic hash table for storage of variables, methods, class inheritance chains and the like.Please head off to an excellent overview on this subject by Jake Douglas and please do come back.

Hashes is tightly coupled with the symbol table implementation and as such requires a comparison (2 values as arguments) function

as well as a hashing function for each value (single argument)

A new object hash structure wraps the hash callback functions and is assigned as the hash type to the symbol table that represent the hash object.

Hashing symbols and strings

Symbols and strings are somewhat interchangeable at a lower level and conversion between either is relatively snappy.We extract a strhash function ( leeched from st.c ) and define


String#strhash
Symbol#strhash # thanks raggi!
'key'.strhash == :key.strhash #=> true


We also now need to define new hash and comparison functions to take advantage of this feature.Apologies for the fugly 1.8 / 1.9 conditionals.

and let the symbol table that powers our string like hash call them

Conversion and coercion

Special case handling for certain value assignments is required for the most prolific use case, that of representing request parameters, in a possibly nested fashion.

• Hash converts to StrHash
• Array converts Hash elements to StrHash

Installation


sudo gem install methodmissing-hwia
'require "hwia_rails"' # from within an initializer


This extension is compatible with MRI > 1.8.6 && 1.9.2 and Rails > 2.3.x

Examples

The Rails compatibility layer …


methodmissing:~ lourens$ irb
>> require 'rubygems'
=> true
>> require 'hwia_rails'
LoadError: Rails environment required!
from /opt/local/lib/ruby/gems/1.8/gems/hwia-1.0.0/lib/hwia_rails.rb:2
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
from (irb):2
>> require 'activesupport'
=> true
>> require 'hwia_rails'
=> []
>> HashWithIndifferentAccess
=> StrHash


… basic usage


methodmissing:~ lourens$ irb
>> require 'rubygems'
=> true
>> require 'hwia'
=> true
>> h = StrHash[:a => 1, 'b' => 2]
=> {:a=>1, "b"=>2}
>> h[:a]
=> 1
>> h['a']
=> 1
>> h[:b]
=> 2
>> h.class
=> StrHash
>> h = { 'a' => 1, 'b' => 2 }
=> {"a"=>1, "b"=>2}
>> h = h.strhash
=> {"a"=>1, "b"=>2}
>> h[:a]
=> 1
>> h[:b]
=> 2


This implementation is compatible with the current (30/08/2009) Rails master branch’s test suite.Please refer to the test cases of either project for further examples.

Performance

Benchmarks available from the README.

You’d notice very little, if any, performance improvements using this extension in the request / response cycle for the vast majority of apps.HashWithIndifferentAccess is limited to query params and ActiveRecord nested attributes assignment.

Consider this a step towards an extraction of a query param object, reusable across many different frameworks and projects.

Follow me @ github or twitter if you enjoyed this article.Thanks for reading!

Here’s a representation of the pure ruby version :

The game plan

The following inefficiencies would have to be addressed :

• Instance variables Instance variable lookups in general require some symbol table overhead.
• Block procedure arguments Block proc arguments are up to 6x slower than passing explicit objects.
• Method dispatch The splat operator and __send__, although efficient, does have some noticeable overheads.
• Kernel convenience method

Additional method dispatch in the critical path through this global function.

Ivar lookup overhead is mostly negated for Ruby 1.9 as the RObject struct stores the first three ivars for each object.

Implementation

The callback structure

We introduce a new callback struct with 2 members :

• object Ruby object to invoke the callback on
• method Method to call.Defaults to #call if no method argument given

Wrapping the C struct as T_DATA has the least overheads in accessing either the object and the method when our callback fires.

GC integration’s a piece of cake as well.

Allocation

Wrapping C structures requires an allocator function (Callback.allocate) to initialize our callback struct and register our mark and free functions with the GC.This is usually invoked prior to #initialize and is transparent for our use cases.

Definition

Three definition (setup) styles supported :

• Object and method arguments
• Block argument
• Farmed off an existing object

We handle each of the argument or block definition styles in an initialize method and handle edge cases such as not assigning a callback method that’s not defined for the object.It’s also important to ensure that our callback is frozen at this point ( I’m looking at you lifo, this one’s legit ) for consistency.

Our global function, Callback(), which is defined much like the popular Array() and Float() coercion helpers, and the farm out API reuse the alloc and init covered above.

Results

An intuitive callback API that’s slower than method dispatch, but faster than lambda callbacks.

Have a look at the source or install the gem to play around.

sudo gem install methodmissing-callback

Tested on MRI versions > 1.8.6 and 1.9.2 – please have a look through the test suite for further examples.

Follow me @ github or twitter if you enjoyed this article.Thanks for reading and special thanks to Aman Gupta for review !

Micro benchmarks and rubyspec compliance are frequently referenced to compare implementations and quite often cited in random rants as well.What happens at that intermediate, libc, level ?

In this post we’ll explore the ISO C memory allocation / deallocation functions as invoked by the following interpreters :

* ruby-1.8.7-p72 (./configure)
* ruby-1.9.1-p129 (./configure)
* jruby-1.3.1 (binary distribution)

Requirements

* A DTrace enabled OS (OS X Leopard, Solaris etc.)
* One or more interpreters installed
* A local checkout of extlib

I’ve opted to NOT test with the latest 1.9.2 preview – see why

The DTrace PID provider

A set of C level probes is spawned for the lifetime of a given command or existing process.When the process dies, the probe’s is unset.You don’t have to recompile or relink – it just works.See the Sun wiki for further information.

This is very different from static probes that ships with MySQL 6 or a previously released patched tree by Joyent.

Memory Allocation

Regardless of the underlying GC implementation, there’s three ISO C functions that handle memory allocation.

* malloc allocates X bytes of memory
* calloc allocates memory for X objects of Y size each
* realloc shrinks or expands the size of a previously allocated memory region

Each allocation function returns a pointer to the allocated (or reallocated) memory region which can be released with the free function call.

Of note is that most implementations never release memory from the heap back to the underlying OS – it’s assigned to a pool for reallocation to reduce system call (sbrk) overhead of the current process.

Toolbox

Ruby 1.8.7

I happen to use this as my stock 1.8 series interpreter – 1.8.6 should yield similar output.

Ruby 1.9.1

JRuby 1.3.1

Again, I’ve noticed a 1.4 version flying around, but happened to have 1.3.1 installed.

Passing Thoughts

Note the liberal memory allocation overheads on the 1.8 series, but an otherwise overall similar distribution pattern to 1.9.I can’t comment much on the JVM case due to limited Java experience.Anyone care to chime in ?

This is an exercise to test the waters for a potential audience for these experiments.I have a few contexts lined up, with a specific MRI focus, but can get into the bowels of most things.Suggestions and thoughts welcome.

Here follows a quick rundown of each talk as well as embedded slides.Source code and slides have also been made available on my Github account.

Embracing Events

An in depth overview of Event-driven Programming and how the Operating System can help with that.Topics touched upon include :

• Context Switches ( Threads vs Fibers )
• Scheduling ( Cooperative vs Preemptive )
• Programming Models ( Evented vs Batched )
• Frameworks ( Eventmachine + Neverblock )
• Operating System nofication ( Signals + POSIX realtime )
• Practices from Online Gaming ( Project Darkstar )
• Testing Event-driven Code

Perhaps touching on gaming at a Rails conf was inappropriate, but I foresee significant production use of small mountable apps around core Rails infrastructure in the near future.Lightweight processes are no different than tasklets.

I’m looking to spend some time in the coming weeks on the JRuby wrap of the Project Darkstar Server module and hope that’ll at the very least inspire similar projects and advocate production use of JRuby.

Special thanks to Muhammed Ali for the last minute diagrams.

Inside Matz Ruby

Very much challenged by a lack of time ( 20 minutes !!! ) one had to touch on a handful of interesting concepts, but it sort of came together as a “patched” ( pun intended ) talk jumping from class representations to threading to gc.

As per suggestion from wycats, watch out for a series of “play along at home” posts that focus on the MRI interpreter.To raise the game, I’ll try my best to have them VM agnostic ( 1.8 and 1.9 ).

Thank You’s

It’s been a great week – no fuzz, no drama.Lotta coffee, little sleep, lotta code.I’d like to take a moment and reflect on the hospitality of the following individuals / companies, in addition to the RailsWayCon team.

Mathias Meyer and Jonathan Weiss

Recommendations for meals, coffee, transport, being loony and making time available from busy schedules, and families.The definite deployment and scaling experts through Peritor, do watch out for a unique EC2 product to be launched soon.

Alex, Thilo && Jan at Upstream Agile

Great office space for co-working and an ample supply of Club-Mate.Alex and Thilo recently built DoingText and Jan’s working on a stealth Couch DB project.

Sven Fuchs && Clemens Kofler

Localization and i18n experts.When not co-working with Clemens and the 90 dB his macbook fan emits, Sven specializes in dancing after 02:30am.