You can hear my dulcet tones at Ruby5 eposide 106 hosted with Jon “Lark” Larkowski.

It was great fun – he made me do the accent at the beginning and we had a lot of laughs over the “it’s gonna rain” part.

Hope you find it of use and if you have some Ruby news, please do submit stories to Lark.

Came across this twice in the past two weeks.

How NOT to do it:

$('form :submit').click(function() {
    $(this).attr("disabled", true);
    $(this).val("wait...");
    $(this).addClass("disabled");
});

Doing it the above way, the click event will indeed fire, but on some browsers the submit event will not.

The event to bind to is submission of a form, not clicking of a submit button.

How to do it:

$('form').submit(function() {
    $(":submit").attr("disabled", "disabled");
    $(":submit").val("wait...");
    $(":submit").addClass("disabled");
});

Also, the :submit selector is a jQuery cross-browser selector for input and button elements that could submit the form, used in preference to input[type="submit"]

A little “remember me later” for conditionally deploying a Rails 3 app running on Ruby 1.9 on Heroku.

$ruby_version = `ruby --version`.split[1].to_f
$heroku = ENV['USER'].match(/^repo\d+/)

source :rubygems
source 'http://gems.github.com'

gem 'rails', '3.0.0.beta4'
# gem 'rails', :git => 'git://github.com/rails/rails.git'

gem 'bson_ext', '~> 1.0.1'
gem 'decent_exposure'
gem 'haml'
gem 'mongoid', '>= 2.0.0.beta.9'

unless $heroku
  if $ruby_version < 1.9
    gem 'ruby-debug'
  else
    gem 'ruby-debug19'
  end

  group :development, :test do
    gem 'rspec-rails', '>= 2.0.0.beta.17'
    gem 'rails3-generators' #for HAML
  end
end

The --without=test option isn’t (yet) being used on Heroku deploys, thus the :test group is in the unheroku block. Update: check if .bundle/config WITHOUT="" works.

Update: Mongoid changed gem versioning for lexicographical sorting (a “dot” between “beta” and the number
Update: Latest mongoid (beta.16 and something after beta.9) requires Rails 3.0.0.rc which doesn’t support Ruby 1.9.1 which isn’t yet on Heroku
Update: Heroku user doesn’t have “repo” in it anymore and USER env var isn’t available – find new mechanism if bundler config WITHOUT option doesn’t work

Also:

• Only the --colo(u)r and --format options work in the .rspec file
• use_transactional_fixtures=false in spec_helper.rb (talk to Chelimsky about transactional fixtures in non-ActiveRecord environments)
• In application.rb:
  • require 'mongoid/railtie'
  • require 'decent_exposure/railtie'
  • g.orm :mongoid
  • g.template-engine :haml
  • g.text_framework :rspec, :fixture => false, :views => false

WindyCityDB – great conference!

I did a lightning talk on squealer, and here is the PDF of the slides (1.7MB)

squealer Lightning Talk slides

One evening in May, 2006 at a ThoughtWorks Sydney company meeting, Richard McLaren (then a Senior VP of ThoughtWorks) and I made a bet with Jon Tirsen (now Google überwunderkind) and Roy Singham (still Chairman of ThoughtWorks) about Ruby in the Enterprise.

The claim from Roy and Jon was that within five years, 10% of applications running in the Enterprise would be written in Ruby. Richard and I bet against that. The prize is a case each of the winners’ favourite potable.

I’ve since been steadily working to try to lose that bet, and now code daily in Ruby at Hashrocket.

Alas, with just one year to go, I think Richard and I are going to win. Although there’s been a decent growth in Ruby’s adoption in the Enterprise, my esteemed colleagues underestimated the lack of capability, talent, and intellect of the majority of corporate IT decision makers.

Object/Relational Mappings (ORM’s) are in the wrong place in the architecture.

An application should have minimal impedance mismatch with the persistence of its own data. External or ancillary systems should bear the cost of mapping between paradigms. If you are writing an application with the concepts modelled in an object-oriented fashion, then the state of those objects should be stored in an object-oriented fashion. Mapping from the object-oriented structure to a relational structure should be done where and when it’s needed. If you want to access application data in a relational way for reports, do the mapping for the report.

There has been a low uptake of OODBMS’s. Why is outside the scope of this post. However, there are other, older (albeit feeling newer) data storage techniques available that more readily persist an object graph than a relational database. A flurry of open-source implementations have come along. You may have heard of MongoDB, neo4j, Redis, Hadoop, CouchDB, Memcached, and Raven DB (a native .NET document database). These are all under active development, and almost all have been proven on many large commercial projects (Raven was being developed by Oren while he was at speakerconf 2010!).

For an introductory look at evaluating the landscape, check out Sarah Mei’s RailsConf presentation slides.

One of the projects I’ve worked on at Hashrocket involved a large MongoDB database. The reasons for the choice for MongoDB were clear:

• the way the users handled most of the data more naturally fitted an embedded document model;
• the scale and speed required were far greater than relational databases could achieve; and
• the need for ACID transactions was almost nil

Additionally, the data was mostly read, rather than written (although MongoDB writes blazingly fast too). I’m hoping to address the equivocations in those reasons in a subsequent post.

MongoDB uses a binary form of JSON. As JSON is a serialization format for object state, it’s a good candidate for persisting an object graph. JavaScript, remember, uses composition rather than inheritance, so there is also a tendency to “embed” document instances within other document instances. So, for MongoDB, most of the object graph is stored a tree structure. Cyclic graphs and associated objects (those that aren’t aggregated with composition) are handled with references to identifiers.

As a consequence of using Rails and MongoDB, the team looked to various ways of manipulating the JSON structures from Ruby: directly through the driver, MongoMapper, MongoDoc, and Mongoid. The rationales behind these libraries and their different and interesting approaches are the subject of additional blogs (hopefully jointly with Nunemaker, Hill, and Jordan respectively). The needs and makeup of the project we were on fell into the lap of mongoid to solve.

DBAs want to interact directly with the database. Because of this “new-fangled” approach to data storage that has been around since before SQL, some are unwilling or unable to learn the query language to interact directly with the database. So, to make their world a less scary place and let them retire without the stress of intellectual endeavour, I built a simple, declarative language where the mapping from the tree structure of MongoDB to the tuple space in an RDBMS can be scripted.

Durran Jordan (author of Mongoid) and I paired on “squealer” (and on emptying bottles) in our evenings while he was stationed in the Chicago office of Hashrocket. The name came from “SQL” (which some of us still pronounce “squeal” rather than “sequel”) and the facetious notion that it turned the data into a pig.

We decided to focus on mySQL as the target and MongoDB as the source as these are the technologies we were using. Upcoming targets will include PostgreSQL, SQL Server, Oracle. Upcoming sources will include Redis and CouchDB.

UPDATE: squealer 2.2 now supports PostgreSQL!

Matt Yoho provided multiple insights into implementing the Ruby DSL, and Bernerd Schaefer helped me reduce the DSL syntax and provide sensible defaults (as well as the righteous progress bar). Bernerd then struck on the idea of reflecting on the Mongoid declarations in the Rails domain model classes to generate an initial squealer script and SQL DDL to build the target SQL database. He then set about writing it overnight and we tweaked it over lunch. We called this tool “skewer”, because that’s what you do to something to make it squeal . It’s part of the squealer RubyGem.

Next time: how to use skewer and squealer.

After 4.5 years at ThoughtWorks, I’ve put down the purple elephant and donned my spacesuit to join Hashrocket and open the Chicago office.

Hashrocket is the world’s best Ruby on Rails shop — a world-class team of spectacular developers, lead by my former and now current colleague Obie Fernandez.

Our Chicago office has been set up to help draw in more talent to meet our growing client needs. Some devs would prefer to live in a big city and Chicago has a strong Ruby community.

If you’re a Rubyist in Chicago, you can look forward to Hashrocket soon hosting some of the user groups and other events. We will be open at 661 W Lake St on Feb 1, 2010.

Announcing PreparedEval, prepared statements for generating JavaScript (and the like) from Java. It prepares Java strings for JavaScript evaluation, similar to SQL prepared statements. Saves you from quote escape nightmares! Great for Selenium tests and Rhino JavaScript.

As a follow-up to my post on escaped, quoted strings, Taking the idea of using a different encoding scheme further (and not being super happy about the need to pump the string through unescape()) I thought about interpolation and wondered about prepared statements in SQL. Some quick research turned nothing up, but while chatting with Ola Bini he also mentioned prepared statements, so this convinced me to knock something together that might be useful.

More information is at http://wiki.github.com/delitescere/PreparedEval.

There’s no runtime dependencies so it’s nice and tiny if you just want the JAR.

Let me know if you use if to help with CSS / XPath selectors for Selenium or if you have any suggestions on improving the API.

When you have more backslashes than words in the strings you are constructing, perhaps it’s time to look at another way of handling special characters like quotes.

The problem

You’re generating JavaScript code from another language with C-like quoting and escaping syntax (e.g. Java).

As part of that code generation, you’re substituting strings that have both double quotes (“) and single quotes (‘). My team’s actual problem stems from using XPath locators which include quoted text and sending that to Selenium so we can obtain the particular element that contains the text^*.

Let’s say we want to pop up an alert within which are some double quotes:

String original = "Hello \"stranger\", how are you?";
String quoted = original.replaceAll("\"", "\\\"");
String script = "alert(\"" + quoted + "\")";
eval(script);

We then expect the browser to have this JavaScript code to process:

alert("Hello \"stranger\", how are you?")

and when then expect the alert to display the string:

Hello “stranger”, how are you?

But why doesn’t it work?

Well, as you can already see, there’s lots of escaping of quotes going on so let’s analyze each line of the code:

1. Needs to escape the quotes around the word stranger in a string literal for the Java compiler.
2. Replaces all double quotes with a backslash and double quotes (or does it??). The intent is to inject the same escaped quotes for the benefit of the JavaScript interpreter, when it’s a string literal in JavaScript code. To do that we inject the backslash character to escape the quote. But the backslash character also needs to be escaped for the Java compiler so we escape it too.
3. Then constructs another string which is the JavaScript function to execute in the browser, containing our escaped-quotes string. Again, we’ve had to escape quotes in the string literals for the Java compiler.
4. Sends the command to the JavaScript interpreter (e.g. a browser) for evaluation.

What does the JavaScript interpreter do with the command we’ve asked it to evaluate? Something like this:

ERROR: Threw an exception: missing ) after argument list

All that trouble we went to to escape the quotes and the escape character has disappeared! In fact, the result is identical to not having tried to inject the escape characters at all.

What’s going on? Let’s try escaping the backslash:

String original = "Hello \"stranger\", how are you?";
String quoted = original.replaceAll("\"", "\\\\\"");
String script = "alert(\"" + quoted + "\")";
eval(script);

This works. Why? We’re using String replaceAll(). It seems that not only is backslash an escape character for the Java compiler, it is also an escape character for the regex Matcher. From the JavaDoc of Matcher replaceAll():

Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.

Hmm, that vital piece of information might have been useful in the JavaDoc for String replaceAll() which delegates to Matcher replaceAll(). Obviously we can’t rely on the Java standard libraries for names being obvious (Principle Of Least Surprise does not hold with these APIs). Let’s try another method, String replace(). After looking at the JavaDoc we see:

Replaces each substring of this string that matches the literal target sequence with the specified literal replacement sequence…

The word “literal” is used a couple of times here, so we might interpret that to mean the special backslash and dollar sign handling doesn’t apply here. Indeed, looking at the JavaDoc of the functionality String replace() delegates to, Matcher.quoteReplacement():

Returns a literal replacement String for the specified String. This method produces a String that will work use as a literal replacement s in the appendReplacement method of the Matcher class. The String produced will match the sequence of characters in s treated as a literal sequence. Slashes (‘\’) and dollar signs (‘$’) will be given no special meaning.

Apart from the fact that the author meant “backslashes” not “slashes”, this makes it obvious. Again, the JavaDoc for String replace() could have been more explicit and actually useful.

String original = "Hello \"stranger\", how are you?";
String quoted = original.replace("\"", "\\\"");
String script = "alert(\"" + quoted + "\")";
eval(script);

The strategic lesson is to not rely on the name of a library method to mean what you think it means, nor to rely on the API documentation to tell you what you really need to know about using the method. Although this is exactly what you should be able to do, you can’t.

So we’ve found out that at least three parts of our infrastructure (Java compiler, JavaScript interpreter, and the standard Java regular expression matcher) treat backslash specially. Are there more? Probably, and typically they’ll change as time goes by.

Perhaps the following approach goes from the frying pan into the fire (in which case you might not want to use something like it), but it uses a blend of the C-style escaping with encoding^†…

Different encodings

Instead, let’s escape the literal prison and use a different mechanism to embed quotes in the string.

String original = "Hello \"stranger\", how are you?";
String quoted = original.replace("\"", "%22");
quoted = "unescape(\"" + quoted + "\")";
String script = "alert(\"" + quoted + "\")";
eval(script);

Now line 2 uses the character encoded version of double quote (the ASCII/UTF-8 hexadecimal value).
Also, line 4 includes the instruction for JavaScript to decode the string which would revert the %22 back to a double quote.

Here, we are generating JavaScript and using a naive implementation of escape() so we can use the inbuilt JavaScript unescape() to decode the quotes at runtime.

Given we’re dealing with JavaScript, we can also use single quotes for string literals so we’d include the encoded version of them in our substitution too:

String quoted = original.replace("\"", "%22").replace("'", "%27");

Footnotes

*Why are you trying to find an element based on the text within it?

That is a long story but let’s imagine a website where the HTML structure is very poor, you want to run some functional tests against it, you don’t have access to the source of the application that generates the HTML, and you have only marginal influence on the third-party who writes that application.

In some unfortunate circumstances, there is nothing unique about an element except the text within it, or even worse, the text within a nearby element.

†Is it encode or escape?

Probably a naming accident during the rushed implementation that JavaScript originally was, the escape() and unescape() functions are probably better called encode() and decode().

“Escaping” refers to using a special character to have compiler temporarily “escape” from processing other special characters and rather treat them as normal, literal characters.

You use the same characters, but just prefix them with the escape character. In C-like languages, the escape character in string literals is backslash.

Encoding refers to using a different form to represent the data. So JavaScript encodes a double-quote character as the string %22. The URI standard also encodes them as %22 and the HTML standard encodes them as the HTML entity " or " (as 34 is the decimal encoding of double-quote), and in C we could use \%x22

In these encoding schemes, the %, & and \ characters are used as escape characters to indicate to the relevant parser that it needs to deviate from its normal processing and go into “encoding” mode.

I’m visiting our India operation for a few months, mostly at our Pune office. I’m trying my hand at various coaching activities on a number of fronts, including working with some of our technical leads, aspiring architects, business development team, and recruitment.

I’ve always been a fan of our teams in India and China. In the early days of the ThoughtWorks Sydney office, to help fill the rapidly growing demand pipeline, we were able to bring people in from TW offices around the world, including some from India (Nitesh rocks!). We also had to distribute some delivery work to India (and later, China too) on some large projects. This was an eye-opening time for us in learning how to make Agile software development work across teams in different places and timezones. (People like Ben Hogan, Naresh Jain, Shyam Mohan, and Jane Robarts have been able to share some of that experience at conferences.)

It wasn’t the reduced rates that these projects were able to offer to clients that appealed to them – it was the large capacity of ThoughtWorks talent available that we simply couldn’t fill quickly enough locally. ThoughtWorks and clients alike insisted on top grade delivery team members, no matter where they happened to be located during any given iteration (we rotated people through the locations quite a bit to engender better collaboration, shared understanding, and teamwork).

Being a senior member of the company for four years, I had always felt a little incomplete in not having visited any offshore teams – based on the stories from others who had visited, I was missing out. I’m pleased now I’m here and I can experience for myself the atmosphere, attitude and aptitude of the Pune office and I’ll get to meet everyone from Chennai and Bangalore at the ThoughtWorks India Away Day in a few weeks.

It’s also pleasing that we have evolved the offshore capability to the point it is a compelling set of offerings that really appeal to companies looking for top quality, technical innovation, agile software delivery, and commitment to results within leaner budgets. http://offshore.thoughtworks.com/

I’m very excited to be here right now. Amanda is knocking out her “F# In Action” book from Manning as fast as an autorickshaw darts between pedestrians, I’m working with some awesome new and old faces, and the dal makhani is delicious!

However, I’m not having much luck getting people to call me “RoganJoshG”…