rkh.im

Ruby 2.1

2013-09-24T15:00:00+00:00

Yesterday the first preview for Ruby 2.1 was announced. The release notes gave a first idea of what's new, but didn't go too much into detail. To find out more, you are pretty much left with diving through the Ruby issue tracker. Since I was already aware of most of these changes, I thought I'd write a quick overview post.

Side note: Obviously Ruby 2.1 includes all the juicy Ruby 2.0 features that I'm not going to repeat in this post.

Refinements

Refinements were added in Ruby 2.0, but turned out to be highly controversial, so their functionality was reduced and the feature was marked experimental.

Refinements allowed you to apply certain monkey patches only to a single Ruby file:

module Foo
  refine String do
    def foo
      self + "foo"
    end
  end
end

using Foo
puts "bar".foo

Outside of the above file, Strings will not respond to the foo method.

In Ruby 2.1, refinements are no longer experimental and can also be applied within a module, without affecting the top level scope of a file.

module Foo
  refine String do
    def foo
      self + "foo"
    end
  end
end

module Bar
  using Foo
  puts "bar".foo
end

Please note that using refinements extensively can lead to very confusing code and that developers behind other Ruby implementations have already indicated they might not implement this feature at all.

Decimal Literals

You might be aware that floats are far from ideal when doing calculations with fixed decimal points:

irb(main):001:0> 0.1 * 3
=> 0.30000000000000004

A lot of Ruby developers fall back to using integers and only fixing it when displaying the result. However, this only works well if you have indeed have fixed decimal points. If not, you have to use rationals. Which isn't too bad, except there was no nice syntax for them (short of changing the return value of Integer#/).

Ruby 2.1 introduces the r suffix for decimal/rational literals to fix this:

irb(main):001:0> 0.1r
=> (1/10)
irb(main):002:0> 0.1r * 3
=> (3/10)

Frozen String Literals

When you have a string literal in your code, Ruby will create a new string literal every time that line of code is executed. This has to happen, as strings in Ruby are mutable. This is also why symbols are more efficient in many cases. However, symbols are not strings. For instance, if you want to compare some user input to a symbol, you'll either have to convert the symbol to a string or the string to a symbol. This means you either open yourself up to a denial of service attack, as symbols are not garbage collected, or you again end up with an additional string creation.

Using symbols for string interpolation will also result in a new string creation, same is true when you want to write a symbol to a socket, the list goes on.

One way to combat this is to store a string in a constant and then use this constant instead:

class Foo
  BAR = "bar"

  def bar?(input)
    input == BAR
  end
end

Now, to deal with the issue of mutability, often times you also freeze the string (which prevents changes from Ruby land, but unfortunately does not give you a performance advantage):

class Foo
  BAR = "bar".freeze

  def bar?(input)
    input == BAR
  end
end

This can get tedious. Fortunately, Ruby 2.1 introduces a syntax doing something equivalent under the hood:

class Foo
  def bar?(input)
    input == "bar"f
  end
end

This will create a frozen string object once and then reuse it whenever the code is executed.

If your taste is somewhat like mine, you probably think it looks very strange. The reasoning is that it works like the decimal suffix explained above. Here's a trick I found to make it look more bearable:

class Foo
  def bar?(input)
    input == %q{bar}f
  end
end

There is an open issue suggesting to add this suffix to arrays and hashes, too.

Hat tip to Charlie Somerville for this patch and suggesting a syntax I would have much preferred.

Required Keyword Arguments

For some reason this wasn't even mentioned in the announcement.

Ruby 2.0 introduced keyword arguments:

def foo(a: 10)
  puts a
end

foo(a: 20) # 20
foo        # 10

However, this way keyword arguments always needed a default value. In Ruby 2.1 you can now have required keyword arguments:

def foo(a:)
  puts a
end

foo(a: 20) # 20
foo        # ArgumentError: missing keyword: a

Method Definition returns Method Name

In previous Ruby versions, defining a method via def returned nil:

def foo() end # => nil

In Ruby 2.1, it now returns the method name as a symbol:

def foo() end # => :foo

This is useful for meta programming and other neat things. For instance, did you know that the private method also takes arguments?

# only foo will be private
class Foo
  def foo
  end

  private :foo

  # bar is not affected
  def bar
  end
end

I use this style a lot in mustermann.

Now that def returns the name, this can be used to make a single method private:

# only foo and bar will be private
class Foo
  private def foo
  end

  private \
  def bar
  end

  def baz
  end
end

I have created an issue to expand this behavior to other ways of defining methods as well.

Removing Garbage Bytes from Strings

Ruby now comes with a handy method to remove garbage bytes from strings:

some_string.scrub("")

This has perviously such a pain to get working across Ruby implementations that I even wrote a library for this.

StringScanner supports Named Captures

StringScanner in the standard library is awesome. In fact, Rails is using it for parsing route patterns, and so will Sinatra 2.0. Check out Aaron's tutorial on practicingruby.com.

Ruby 1.9 introduced named captures for regular expressions, however, StringScanner didn't support it:

require 'strscan'
s = StringScanner.new("foo")
s.scan(/(?<bar>.*)/)
puts s[:bar]

On Ruby 2.0 this will result in:

TypeError: no implicit conversion of Symbol into Integer

And on Ruby 2.1:

foo

This is actually a patch I added (and I used goto, oh my god).

Accessing Network Interfaces

You can now access the network interfaces via Socket.getifaddrs:

require 'socket'

Socket.getifaddrs.each do |i|
  puts "#{i.name}: #{i.addr.ip_address}" if i.addr.ip?
end

For me, the above prints:

lo0: fe80::1%lo0
lo0: 127.0.0.1
lo0: ::1
en0: fe80::1240:f3ff:fe7e:594e%en0
en0: 192.168.178.30
en2: fe80::3e07:54ff:fe6f:147a%en2

Faster Numbers for Serious Math

Ruby 2.1 is now faster when it comes to large numbers, as it uses 128 bit integers for representing bignums internally now if available. They also got an additional speed boost by using the GNU Multiple Precision Arithmetic Library.

VM changes

The underlying virtual machine now doesn't solely rely on a global method cache, instead it also does call site caching. I gave a talk about this topic if you're interested.

RGenGC

Ruby now has a "partially" generational GC. This will speed up GC, which up until now was a conservative, stop the world, mark and sweep GC.

It basically still is. For some objects. This is hard to change due to Ruby's internal/external C API.

However, in Ruby 2.1, the VM will classify objects as sunny or as shady, depending on whether they need to have the normal treatment or not. There are different operations that will turn a sunny object into a shady one. For instance, handing them to a C extension. Other objects, like open files, are classified shady to begin with.

The sunny objects can be handled by a generational GC. Koichi Sasada gave a talk about this at RubyKaigi and at Euruko.

Updated RubyGems

RubyGems has been updated to 2.2.0, which comes with a few minor enhancements.

Just a Preview

Please be aware that yesterdays release is just a preview and all of the above is subject to change.

The long way to Mexico

2013-03-11T10:00:00+00:00

Both @bmizerany and @konstantinhaase will be at @magmarails next week. Get your ticket now: magmarails.eventbrite.com
— Sinatra (@sinatra) October 5, 2011

In 2011 I was supposed to speak at MagmaRails.

At the time, I was working on Rubinius and related projects from Portland, Oregon. The whole thing was sponsored by Engine Yard. In case you didn't know, they rock.

Anyhow, they would have covered my trip to Manzanillo. Would have. But then Hurricane Jova decided to cancel the event.

Three times since have I been invited to Mexico. And so far, I couldn't make it.

A few things changed since then: First, I'm back in Germany, which is a bit farther away from Mexico than the US is. And second, I no longer have Engine Yard sponsoring me. But I still work on Open Source full time. Silly me.

Doing a lot of Open Source means I get invited to speak at a few conferences each year. Since Travis CI is by far not making enough money (yet) to cover my travel expenses or pay me enough so I can do it myself, I always insist on the conference covering the costs.

I did the same for the three invitations I've got from MagmaRails, by now called MagmaConf. However, it's not easy to find sponsors for a community organized conference in Mexico. Every time I was told that they can't cover my costs, maybe next year.

But this time we won't declare defeat that easily. Instead, we're trying to crowdfund my plane tickets with the Bring Konstantin to MagmaConf campaign. You would make the conference organizers, the attendees and of course myself extremely happy by shipping in, even if it's just a little.

We created this fundraiser because we believe that the Ruby community is awesome and there is a strong chance that we can reach the goal. Plus, Konstantin is pitching in with some pretty nice perks, so it's totally worth the contribution!

— MagmaConf

Check out the conference's blog post on why they think I should be speaking there.

What's new in Sinatra 1.4?

2013-03-09T21:10:00+00:00

I'm about to release Sinatra 1.4.0.

A few house-keeping jobs are still outstanding, like bringing the website up to speed and running it on a few more apps in production (though we've been running it in production for Travis CI for quite a while without issues) and then we're good to go.

If you want to give it a try right now, you can install the prerelease via gem install sinatra --pre. For more infos, check out The Bleeding Edge from the official Sinatra documentation.

Sinatra follows Semantic Versioning, meaning that this is the first release bringing new features since 1.3.0, released in 2011. Let's take a look at these new features.

New HTTP Methods

With 1.3.0, we added support for PATCH. 1.4.0 will include LINK and UNLINK.

Like PATCH, they were originally included in RFC 2068 and have now been proposed again.

link '/resource/:id' do |id|
  resource = Resource.find(id)
  resource.links << env['X_LINK']
  resource.save

  "link established"
end

unlink '/resource/:id' do |id|
  resource = Resource.find(id)
  resource.links.delete env['X_LINK']
  resource.save

  "link removed"
end

See RFC 5988 for more infos on resource linking.

Template Updates

This release adds support for Yajl, Rabl, Wlang and Stylus templates.

get '/style.css' do
  stylus :style
end

get '/list.json' do
  yajl :list
end

Moreover, ERb, Haml, Slim, Liquid and Wlang templates can now be nested using blocks:

get '/' do
  slim(:outer) { slim :inner }
end

__END__

@@ outer
html
  body
    == yield

@@ inner
p Hello World

You can now configure the default layout on a per engine basis, and passing in nil as layout is now treated the same as passing in false rather than using the default layout. Also, we've solved a caching issue when using multiple view directories.

Play Nice in Classic Mode

Up until this release, using Sinatra in classic mode would add certain private methods to Object:

require 'sinatra'

["some_object"].instance_eval do
  get '/example' do
    "this works in Sinatra 1.3"
  end
end

The above example will now raise a NameError.

This is important for objects implementing method_missing without implementing a corresponding respond_to?.

In 1.4, instead of including Sinatra::Delegator in Object, it will only extend the main object.

Better Routes Parsing

1.4 has a more robust route parsing. For instance, given the following code:

get '/:name.?:format?' do |name|
  name
end

Running the above with Sinatra 1.3 and requesting /foo.bar will return foo.bar, on 1.4 it will now return foo.

Plus signs in the URL (not in query params) will now once again be matched as plus signs, not as spaces.

Mime-Type Parameters

The request object now exposes Mime-Type parameters when parsing the Accept header:

get '/', provides: :jpeg do
  compression = request.preferred_type('image/jpeg').
                  params['compress']
  generate_image(compression)
end

The above example will take the compress parameter into account, including multiple entries with weight:

$ curl -H 'Accept: image/jpeg; compress=0.25; q=0.1,
    image/jpeg; compress=0.5; q=0.8' http://localhost:4567

New Servers

In addition to WebRick, Thin or Mongrel, Sinatra will now automatically pick up Puma, Trinidad, ControlTower or Net HTTP Server when installed. The logic for picking the server has been improved and now depends on the Ruby implementation used.

Other Changes

Exception handling has been improved a lot.
The register method (for registering extensions) is now also delegated in classic mode.
You can now redirect to URI and Addressable::URI instances.
Generated Content-Disposition values now include the file name even for inline, not just attachment.
You can now pass a status option to send_file for setting the status code.
The provides condition now honors an already set Content-Type (for instance, in a before filter).
Status, headers and body will be set before running after filters. This is very useful if your after filter is modifying the body.
When calling new on your modular application (or Sinatra::Application), it will now return a wrapper object that exposes a few methods, like settings, instead of the middleware stack directly.
Sinatra will now respect the $PORT environment variable, which is for instance set by Heroku.
Improved compatibility with Rack 1.5, RDoc 4.0 and RubyGems 2.0.
By default, Sinatra will now only serve localhost in development mode. You should not be running your production system in development mode.
The documentation has been largely improved and converted from RDoc to Markdown.
The chat example has been fixed to work with the latest jQuery.

New Configuration Options

The protection setting now takes a session option to explicitly enable/disable session based protection modes. This is useful if you set up a custom session implementation, which Sinatra will otherwise miss.
You can now disable :x_cascade to avoid sending the X-Cascade: pass header if no route matches.

What's next?

Currently there are three fully maintained branches in the Sinatra repository: master is where the main development is happening, but both 1.2.x and 1.3.x currently receive bug fixes and improvements. With the release of 1.4.0, this will change.

From that point on, 1.3.x will only receive major bug fixes and security fixes, no more general improvements, unless they can easily be backported from 1.4.

The 1.2.x branch (which is the branch for Ruby 1.8.6 compatibility) will be discontinued. This includes security fixes. You are very much urged to upgrade if you are still running on Sinatra 1.2.

Once the first new feature makes it into master, it will move on to 1.5 and I will create a 1.4.x branch. However, there is currently no roadmap whatsoever for a 1.5 release.

I do have some plans for Sinatra 2.0, though. I have had the urge to rewrite large parts of Sinatra for a while and I have a few prototypes lying around. Goal is to end up with a simpler, more flexible and performant code base. It will fully leverage the new Rack stream hijacking or the Rack successor (Rack 2.0, Ponies or whatever it will be called) should we have one by then.

It will probably be Ruby 2.0 only. Before you freak out: It's still a long way until then, there won't be a release any time soon and even once there is a release, there will be a maintained 1.x branch at least until Sinatra 2.1 has been released.

Almost Sinatra

2013-03-08T08:35:27+00:00

A while back I implemented a functional Sinatra clone in six lines of Ruby code.

I recently gave a talk about this and other useless but amusing things at a conference in Melbourne and they've just released the video:

And here are the slides:

I hope you enjoy it as much as I did!

Reloading Ruby Code

2010-08-30T19:06:04+00:00

As the core of my Ruby Summer Of Code project, I have partly rewritten ActiveSupport::Dependencies. I will give an introduction to common reloading strategies and their implementation and discuss my changes to Dependencies. Even though this part of ActiveSupport is hidden away and not well known, all Rails developers rely on its proper functioning, as it is responsible for autoloading and reloading Ruby code. It is also responsible for producing error messages like "A copy of Something has been removed from the module tree but is still active!" or "Object is not missing constant Something!". But before focusing on Dependencies, let's talk about the topic in general.

In this article I will focus on reloading code, since autoloading code is rather simple: You define a const_missing hook, which is triggered whenever an undefined constant is used. Map the constant name to a path (with Dependencies, Foo::BarBlah will be mapped to foo/bar_blah) and search for that file inside a list of directories, in that case Dependencies.autoload_paths and, for files that should be autoloaded but not reloaded, Dependencies.autoload_once_paths. Some other implementations use Ruby's $LOAD_PATHS, which has the advantage of simply trying to require 'foo/bar_blah' instead of having to search for a matching file by hand. That way autoloading constants from gems or other formats (i.e. foo.so instead of foo.rb) just works. On the other hand this may easily lead to loading other files than intended and reloading files that should not be reloaded.

You might have already noticed that such an autoloading approach is working on two different levels: Constants and files. The artificial mapping from constants to files is present in most Ruby project, but it is not a low-level Ruby feature. Neither is code reloading. However, especially in Rails-land, reloading is assumed to just work. That is impossible by design. But there are a couple of different approaches trying to get you as close as possible. The key design decisions are when to reload what code and how to do that. Of these decisions, when to reload is the easiest one, as it does not bring any implications for the code that is reloaded. The reloader can be triggered on any file changes or on specified points/events in your program flow, or a combination of the two. In a web app, you probably only want to reload code on a new request. What code you should reload heavily depends on how you are reloading code.

In the following I will mainly focus on reloading Rack applications. I will avoid going into how Rack works. I don't think you have to understand every juicy detail, but if you have no clue what Rack does, now would be a good time to check it out. A lot of the strategies and libraries described here should also be usable for other applications. However, if you're level of interest in Ruby and Rails has brought you as far as this blog post, you really should get to know Rack.

Abusing Open Classes

The concept of Ruby's open classes is demonstrated in the above example. You might already know this technique, especially either from monkey-patching other modules/classes or from using modules as namespaces. It can also be used for reloading. About anything that can be defined in Ruby, can also be redefined at runtime.

Imagine you have a Rack application called Foo in foo.rb. If you load foo.rb twice, then the definition of Foo monkey-patches itself. If one method was changed in-between the first and the second loading of foo.rb, the second version will override the first. If a method did not change, it will be overridden with an unchanged version, which is about the same as not overriding it. Well, except for the fact that it will invalidate any method caches and revert any inlining which might have occurred thus far. But the method's implementation will remain the same. This is not only extremely simple, but as it turns out also rather fast. If you want to reload foo.rb on every request you could set up a simple middleware for that:

However, this is not require-friendly. Even if foo.rb requires bar.rb, only foo.rb will be reloaded. One solution would be to use load instead of require. However, we want to keep the reloader as invisible as possible. Moreover, this would make reloading bar only if it changed impossible, as it will be reloaded whenever foo is reloaded. One option we could use for now is to remove bar from $LOADED_FEATURES. That way require 'bar' would load it again. But now bar will not be reloaded unless foo is reloaded. A better way would be to actively reload bar. The example below reloads any files that where loaded and have changed. We can use mtime for checking if I file changed

Note however, that this really includes any files that have been loaded, even if those are in a gem or the standard library. However, those files usually do not change. Still, this approach might not be useful for bigger apps, as there are files you might not want to reload even if they changed (think initializers). In case this approach is exactly what you want, there is a Rack middleware doing exactly this: Rack::Reloader. Not only does it also include error handling and has some nifty features like the ability to set a cool down phase, you probably have it already installed, as it ships with Rack. Combine it with the autoloader code from above and you got Dependencies Lite™.

For smaller or well written applications this will work perfectly fine, but you will not be able to use it for a Rails or even a Sinatra app. First, instance and class variables don't get invalidated:

The body gets reevaluated:

Inheritance cannot be changed:

Methods and constants cannot be removed (at least not by removing them):

While the last one is usually not a big issue, the other two make it unusable for Rails. Sinatra will keep the old routes and append the modified ones on a reload. To solve this, you can call Sinatra::Application.reset!. This will only work if you always reload all files. If you want to have partial reloading, try Sinatra::Reloader. But still, for your own code you should be aware of these issues. A rule of thumb: Using open classes should work just fine if you are willing to manually restart your application in case one of the above issues should occur and if you favor inheritance and mixins over alias_method_chain, write your files so that executing them twice does not do any harm and you avoid black magic. You probably don't if you are developing a Rails app. You probably do if you are developing a Sinatra or Rack app.

Actually restarting your application

OK, so apparently relying on open classes is not usable without constrains. One solution that should work without restrictions would be to actually restart the application instead of reloading just some files. Think about it: It's what you would probably do manually if you're application has no reloading baked in. The implementations usually rely on fork. The strategy is simple: Load everything that should not be reloaded, fork and load everything that should be reloaded. Answer requests from that fork. For reloading: Kill the fork, fork away and again, load everything that should be reloaded. The main issue is watching for changes.

As reloading is triggered from the outside, you have no clue what files are loaded. One strategy would be to again simply reload on every request. Shotgun does this. Another option would be to simply watch the current directory for any changes. A general purpose implementation would be rerun. Since rerun is intended to be usable with any application, not just Rack apps, it does lack the preloading feature. Therefor reloading probably occurs a lot less than with shotgun, but takes longer.

My favorite implementation of this strategy is "Magical Reloading Sparkles" by Jonathan D. Stott (aka namelessjon). Unicorn has a signal-based redeployment mechanism. Jonathan's code will trigger it on any code change and cause unicorn to kill and refork all workers. In contrast to shotgun it is also easily possible to specify exactly which code should be preloaded.

So, why not use this approach? As it does a lot more work under the hood (besides reloading more code than the other approaches, it also fires a new system process) and eats more resources, it is generally slower. I would still recommend it for complex applications, if it only reloads on changes or for really small apps that fire up instantly. Under no circumstances would I recommend using shotgun for a Rails app, but for small Sinatra apps it should be acceptably fast. The main issue, however, is portability: Unix forking is not available on Windows nor JRuby, neither is Unicorn. Offering it as default strategy for Rails would limit official support to Unix and MRI. Note that it would probably be possible to implement something similar for Windows.

Replacing Constants

We can do something similar to restarting the app: Remove the old constants before reloading. That way, the old code does not survive: No old instance variables or methods lying around, aliasing works as the body will only be reloaded once per constant incarnation, and since the constant will be redefined instead of reopened on every reload, we can even change its superclass:

Let's try to use that:

The above gist also points to one of the main issues of this approach: Invalidating references. When using open classes, Foo always referenced the same object, it just evolved. Now Foo is a different object before and after each reload.

If you pass Foo to run, Rack::Builder will use the object passed, which would remain unchanged on the next reload. By wrapping it a new constant lookup will be triggered on every request and thus changes will be picked up. It gets even worse as soon as you realize that the same goes for instances:

This cannot be fixed entirely. However, it is possible to alleviate these problems.

The Rails Way

Imagine you where able to remove everything and load all from scratch. That way no old references would survive and no constants could be kept alive. In a nutshell, this is what Rails is trying to do, except, not really. As mentioned above, reloading everything might not be what you want. You might lose state or you even might execute files twice that should not be run twice otherwise. Therefore ActiveSupports divides all constants into two groups: The part that should be reloaded and the part that should not be reloaded. As a rule of thumb the reloadable constants are your app code (models, controllers, extra, etc) and the rest is kept (lib, initializers, external dependencies, etc).

Everything works fine if you only reference reloadable constants from other reloadable constants (at least if the references would otherwise survive to the next request and would be reused). In order to achieve this, it might be necessary to add constants explicitly to the reloadable pool. This is possible by calling Module#unloadable on the constant. Note however that marking a constant as reloadable will essentially cause the constants' source file to be reloadable, so if it has any side-effects besides defining the constants, those will reoccur on any request.

The hard thing is to track which constants belong to what pool. ActiveSupport creates a watch stack for that. It tracks which constants are already known and is able to return a list of new constants for a block of code. It then hooks this mechanism into the autoloading code, require and load. Autoloaded constants will automatically be added to the reloadable pool. However, if the file defining the constant requires other files, ActiveSupport has to be careful to only add the autoloaded constant to the pool, not necessarily its dependencies. Another threat is accidentally loading a file twice before removing its constant, as in that case you would have the issues of the open class approach you were trying to get rid off. ActiveSupport therefore juggles with multiple lists besides the watch stack to track all files that have ever been autoloaded, all files that have been loaded since the last reload ("ActiveSupport::Dependencies.clear", btw), all automatically loaded constants since the last reload and all constants explicitly marked for reload. Also, combinations of those lists are used extensively.

ActiveSupport removes all reloadable constants after each request and relies on the autoloading hook to reload any required constants. That way on reach request only the constants needed to serve the page should be loaded. In production mode, constants will simply not be removed and autoloading will happen in an eager manner (before the app starts serving), since autoloading is not thread-safe. If loading a file defining a constant raises an error, ActiveSupport also takes care of removing that constant again, thus avoiding "broken" constants.

The Rails Way, reloaded

ActiveSupport 3.0, even though a lot less than 2.3.5 did, either blows up in your face or does not even mind if anything went wrong. This was the initial motivation for my project proposal. My goal was to reduce error messages. In order to do that, I changed the inner architecture of ActiveSupport::Dependencies away from a procedural, list juggling to an object oriented approach where every constant has a wrapper knowing whether it is already activated, it is reloadable, and how and when to reload it. At the end of each request still all constants are removed, but the wrappers are kept. Now, the wrapper can decide whether to restore the wrapped constant or to reload it from source.

In order to do so the developer can choose a reloading strategy. This is probably only relevant for plugin or middleware developers (due to extensive reference keeping, the constant removing approach is rather annoying when writing Rack middleware). The default strategy is the world reloader. The world reload will all cause all constants to be reloaded if a single file referencing a constant with this strategy was changed. This is essentially the same as the current Rails strategy plus checking for changes. ActiveSupport can also pretend a change occurred on every request and skip the mtime checking. This is an advantage if you have a setup where loading files is rather cheap and checking mtimes is expensive. Another strategy best set on per constant level, is the open class strategy. It is activated via unloadable :monkey_patching. If you do so, changing that file will only reload the constant defined in it and restore the constant before reloading it, thus using the open class and keeping all references to that class intact. A third strategy, that is rather experimental, tries to divide the reloadable pool into smaller sub-pools, reloading only the pools affected by file changes. As Yehuda Katz pointed out, this almost always works in demo scenarios but probably fails when tried for real world cases. It is of course also possible to change the default strategy. Such sub-pools are created by associations, require, require_dependency and explicitly by Module#associate_with. I therefore named it sloppy reloading. Since every strategy is just a mixin, plugin developers doing a lot of rails core work, or developers of other frameworks using ActiveSupport (think Padrino) could easily apply their own strategy (unloadable MyLib::MyStrategy).

But how does this help with reducing errors? The wrapper approach allows easier tracking of what is happening to a constant. The wrapper has hooks to whenever a constant is added or updated, which reduces the likelihood of a "is already activated" or "is not missing constant" error. Moreover, it shifts errors from the autoloading point to the moment old references are actually used. A old reference no one is touching anymore does no harm. My branch therefore modifies the old constant to be a proxy for the new one as soon as it is replaced by another version. In most cases this will not cause an error but redirect the methods and result in the behavior expected if you would not be aware of the reloading that's going on under the hood. This even works when including modules, as Ruby relies on append_features, which will also be delegated to the new constant.

Another issue dealt with is using require on reloadable files. Usually, developers are told to simply avoid require and use require_dependency instead. If you require a file containing an explicitly reloadable constant without referencing that constant, it will only be loaded the first time. At that point Ruby adds the file to $LOADED_FEATURES and will not load it again on a require. After the request finished, ActiveSupport will remove the constant. On a reload you might expect require to load the file or the constant to be defined and then try to access the constant implicitly. In that case you will not be able to reach the constant. The patched version of ActiveSupport therefore takes care of removing files from the $LOADED_FEATURES if the constant is removed. Tracking such events based on the list architecture would have been a nightmare.

One last, technically minor issue, is offering an alias for unloadable. A lot of developers I know initially seem rather confused by unloadable, sometimes even assuming it does the exact opposite (as the constant it "not loadable"). I therefore added an alias I rather prefer: reloadable, especially when used with a strategy: reloadable :sloppy.

Numbers, please

Lies, damned lies, and statistics, so here we go. In a simple Sinatra application, I get the following numbers:

	no changes	file changed
No Reloader	0.0044671058654	-
Rack::Reloader	0.0231933116912	0.0236261129379
Sinatra::Reloader	0.0088394482930	0.0090538899103
ActiveSupport	0.0207427183787	0.0207780686060
ActiveSupport Reloaded	0.0147878090540	0.0208899911244
Shotgun	0.9925425291061	1.0038710657755
Magical Reloading Sparkles	0.0044739915830	0.9384773521974

In a simple Rails 3.0 app serving a page took 0.00793675 seconds, while with my RSoC branch it took 0.00379425 seconds. Note that you might want to disable mtime checking on JRuby, as it is about 10 times slower than on MRI and 5 times slower than on Rubinius. If it is worth checking the mtime on JRuby really depends on how long your files take to load. If you do a lot of instance variable caching and such you might still prefer checking for changes.

Strengths and Weaknesses of Seaside

2010-08-25T19:14:00+00:00

This is an excerpt from my bachelor thesis.

Core principles

Seaside's approach to web development differs from most other web frameworks, such as Django or Ruby On Rails, by explicitly breaking with common patterns and principals the web is built upon, such as being stateless, having meaningful, maybe even restful URLs or using template systems.

When doing web development you often have to work with the construct of a session, that allows one to keep state in between requests made from the same browser session. Usually the session is serialized and stored in a cookie or a database. Seaside takes advantage of Smalltalk's image based nature by simply keeping the session as an object in the currently active process.

This eliminates the overhead of serializing every object in the session and the need to keep the number of objects kept by the session small. Seaside goes even further by storing closures and continuations in the session.

While most other frameworks use the session as little as possible, in Seaside it is used for nearly everything. It solely depends on the state of your session what page is displayed. Usually every Seaside application features a single entrance point, a so called root component.

User navigation is realized by storing continuations and callbacks (closures) for the currently displayed page. User interaction will then trigger any associated callbacks, thereby changing the session's state, and then re-render the possibly changed content.

Advantages over other web frameworks

This gives some advantages over classic web development. Applications, though written for the web, are developed in a way rather similar to desktop applications. Interactive websites, also known as Rich Internet Applications, can rapidly be developed.

In the prototyping phase or when developing small sites, using the image for storing data eliminates any need for implementing a persistence layer, like an object-database-mapping mechanism.

Working on such a high abstraction level compared to normal web development encourages the developer to use object orientation and decoupling on other levels than when working with other web frameworks. Rather than thinking of constructs and working with object representing requests, streams, cookies or URLs, websites are defined by components, callbacks and canvases. This can also encourage a creativity, since it requires developers to think outside of their common patterns.

Downsides of developing with Seaside

There are some downsides one has to be aware of when choosing Seaside. Due to the session based nature and the close coupling of "rendering", it is rather complicated to implement public APIs. In many cases Smalltalk developers fall back to other libraries and protocols to accomplish this.

When developing software with Seaside, the commonly used path is to write in Smalltalk whatever can be written in Smalltalk. This is especially true as Seaside is able to generade some JavaScript from Smalltalk, and tools like Glorp do the same for SQL. This is rather cosy for the developer, who probably favors Smalltalk over the other languages anyway.

However, this poses the threat of doing computation in the Seaside image, that could usually be done in the web browser or at database level. Moreover, as this results in changes to the HTML output more often than would be necessary when filling in content via JavaScript, this also conflicts with established caching techniques. The latter is not much of an issue, since HTTP level caching is not feasible with Seaside, as will be discussed in a moment.

Seaside applications usually also conflict with established usage patterns. Tabbed browsing is imposible, so is storing bookmarks for any page besides the entrance page. If a user shares a link with another user, not only might that link not work when the current session expires. This usually happens 10 minutes after the last HTTP request.

If the session is not yet expired, the other user opening the link will end up in the same session as the one sharing the link with him or her. This will not only lead to unexpected behavior, but represents a major security risk.

Seaside and scalability - deployment issues to be aware of

There are also implications on the infrastructure level, especially when using reverse proxies. At the server side two types of proxies are commonly used: caches and load balancers.

HTTP caches are simply not feasible with Seaside. Techniques like the ETag, Last-Modified and Cache-Control HTTP headers are unusable, since they rely on the same content being available under the same URI for multiple requests. Only application layer caching is an option, but it lacks general Seaside solutions and does not keep requests from reaching the image in the first place.

Although it would be possible to cache a response, the same HTTP request is extremely unlikely to reoccur and the cache would only allocate additional system resources, increasing deployment costs even more.

Load balancers are rather important when scaling web applications. When running a web application you usually do not want to start an application process for every incoming request. Therefore, in a simple setup, one process, an application server, is started and all incoming requests are handled by this process.

The Seaside equivalent is starting a Smalltalk image with Seaside. However, increasing load might have a major performance impact on the image. Therefore you often want to launch multiple images on the same computer or even multiple computers each running at least one instance of your application and distribute load equally among them.

This is where the load balancer comes in. The load balancer listens for any incoming requests and forwards them as equally distributed as possible to the different images. To reduce overhead the load balancer has to be as fast as possible. Larger setups therefore use hardware load balancers working on ethernet level.

For distribution, the proxy often relies on round robin, and URI based sharding, as these algorithmes are simple, efficient and can be implemented at a very low level. As for the same reasons tabs and bookmarks are not usable, you cannot rely on URI based sharding.

Load balancing is a major concern when deploying Seaside applications, since most Smalltalk implementations rely on green threads and most Seaside application are not thread-safe. A small number of users accessing the application simultaneously will already result in a noticeable increase of response time.

When having identical application servers, round robin is the simples way one can imagine to distribute load among them: The first request is forwarded to the first application server, the second to the second, and so on. Until you reach the last app server and start with the first server again. Without additional tools, this approach is not usable with Seaside, as requests from a single browser session always have to be forwarded to the same application server.

The common approach to address this issue is to use an object database, in most cases the proprietary Gemstone/S system, to persist the current session and thus share it over the network among all application servers. While this works remarkably well, it comes with the large overhead of synchronizing the session and all referenced objects (including the closures and continuations) for each and every request.

Advanced load balancers do not strictly rely on round robin, but track how many outstanding requests the individual application servers have or how low their response time is, and distribute incoming requests accordingly.

An alternative approach is to always forward requests from the same client to the same application server. While this approach works even without sharing sessions among images, it needs additional overhead and does not distribute load as equally as round robin or any similar approach.

Imagine one user would be all a single image could handle. In that case all users would have to be equally active to distribute load perfectly among the servers. Moreover, if you have six users on five application servers, the load balancer would have to forward two users to the same server.

To solve the performance issues of those two users, a system administrator could decide to start yet another application server and hook it into the load balancer. However, unless a new user appears, that application server will never be used, as users are always forwarded to the same application servers. If one application server crashes, the load balancer will no longer be able to serve to users previously using that server.

In a real world example you would therefore configure the load balancer to only forward to the same image in the most cases, with the ability of switching images. Unless you are willing to risk losing the session, that would mean sharing sessions again, but lower the synchronization overhead.

By design, this overhead would still be larger than for any web framework embracing web patterns like REST, and makes scaling Seaside a hard task to accomplish.

Why Seaside?

Despite the scalability issues just discussed, we decided to use Seaside. As a powerful web framework it allowed us to easily implement the infrastructure described earlier.

Another important factor was that we should favor technologies our external partners were using themselves. This largely influenced our decision towards VisualWorks and Glorp.

The only remaining competitor in for this setup would have been the Iliad web framework, which is commonly used with Gnu Smalltalk rather than VisualWorks. We therefore assumed better support for Seaside. Moreover, our customer was also developing applications with Seaside, which made it a perfect candidate for the project.

Fortunately, scalability was not a concern, as the system will probably only have one or two users accessing it simultaneously.