Bamboo Blog

Experiments with the HTML5 Audio Data API

oliver.n@new-bamboo.co.uk (olivern) — Wed, 26 Oct 2011 00:00:00 +0000

About a month ago we had a hack day at New Bamboo, a day where all the bambinos were free to work on any kind of project they wanted to. I decided to experiment with the HTML5 Audio Data Api, and canvas.fm, (code) is the outcome of that experiment.

Canvas.fm is a small app that allows you to listen to music available on SoundCloud. Whilst the audio is playing it draws a visualisation of the current song using HTML5 canvas. It uses a server component, powered by Node.js, to convert SoundCloud's streaming MP3 audio into the OGG format - which Firefox supports.

Audio Data API

Browser specifics

The audio data API was originally introduced in Firefox 4 in the second half of 2010. Using this API it is possible to read & write audio data from both audio and video elements.

It is worth noting that more recently a webkit audio data API has also emerged. The Mozilla implementation is, at the time of writing, significantly simpler and comes with much more comprehensive documentation and examples.

Because I originally only had a day to put something together I decided to focus on the Mozilla implementation of the audio data API. I hope in the future to be able to make canvas.fm work on both webkit and firefox.

Implementation Details

There are a number of events that the Mozilla audio API gives us which we can hook into to manipulate the audio data. In canvas.fm we are only reading audio data so we are primarily interested in the loadedMetaData event and the MozAudioAvailable event.

Loaded Meta Data

The loaded meta data event is fired, as the name suggests, when the browser has loaded the meta data for this audio element. This event is not specific to the audio data API, but it signals that we can now gain access to some meta information about the audio that we will later need to process the data.

Specifically these are:

the number of channels available in the audio source — 1 for mono, 2 for stereo etc.
sampleRate — the number of samples per second in the audio which is typically around 44.1KHz.
frameBufferLength — this will be discussed in the next section.

MozAudioAvailable

This event is where the bulk of the processing happens. The event handler is passed the frame buffer and the time in the audio track that the first element of this frame buffer represents.

The frame buffer is a typed array of floats that represent the decoded audio data. Each element of the array represents a tiny sample of the audio that is being played, with values normalised between 1 and -1. The values actually represent the sum of all the amplitudes of frequencies for a particular channel at a particular point in time.

The length of this array can be obtained from the audio elements frameBufferLength that we grabbed in the loadedMetaData event handler. The full length will be this value multiplied by the number of channels because the channel data is actually interleaved with each other, e.g. [channel1, channel2, channeln, channel1, channel2, channeln...].

Fast Fourier Transform

Ultimately we want the amplitudes of the various frequencies that make up the audio being played, rather than the sum of all the frequencies for a channel, which is what we've currently got.

This transformation is possible using Fourier Analysis, the detail of which is in the realm of physics and way beyond the scope of this article. Essentially we are transforming the original data, which is in the time domain, into what we need to graph, which is data represented in the frequency domain.

A Fast Fourier Transform algorithm, implemented in JavaScript, does all the hard work for us. What we end up with is an array of frequency amplitudes, which is precisely what we need to draw our visualisation.

Visualisation

An obvious choice for visualising our audio data would be a very familiar graphic equaliser. There are already several of these around, and although they are a good first step I wanted to do more. These kind of visualisations only allow you to see what is happening now, without giving an impression of the song as a whole.

I decided to use a variation on the traditional graphic equaliser. I wanted to draw a disc, showing the various changes in frequency strength throughout a certain piece of music. In the end you would end up with a circle, showing the frequency shifts and variations of the whole song, hopefully it would also produce a visually interesting image.

To make this possible without having to keep state of the whole frequency spectrum for a whole song, we can take advantage of canvas transformations. The canvas code that is doing the drawing doesn't change its position as the audio is being played. Instead we rotate the canvas to prevent the canvas overwriting existing lines, and to give us the circular image that we want.

var draw = function (samples) {
  samples.forEach(function (sample) {
    intensity = sample * 10
    ctx.fillStyle = 'rgba(0,0,0, ' + intensity + ')'
    ctx.fillRect(0, 512 - i, 1, 1)
  })
}

The above snippet shows the draw function that takes care of drawing on the canvas drawing context, ctx. On each sample of the song we rotate the canvas by an amount that means the canvas is rotated a full 360°, or 2π radians, by the time the audio is complete.

var rotate = function (rad) {
  ctx.rotate(rad)
}

Streaming Audio

The SoundCloud API is very simple and, most importantly for canvas.fm, allows us to get access to a streaming MP3 of any track available on the SoundCloud service. Unfortunately Firefox doesn't support the MP3 audio format due to licensing issues.

Thankfully it is relatively trivial to convert MP3s to Firefox's preferred audio format, OGG. Since canvas.fm allows users to search and choose from the entire SoundCloud music catalogue, we cannot pre-convert all of these audio streams.

Instead, Canvas.fm uses Node.js to handle SoundCloud's incoming MP3 stream, and then pipes this into ffmpeg, giving us an OGG formatted audio stream which we can then pipe into an HTTP response. Using Node.js makes this kind of stream handling incredibly simple and efficient.

var track = Track.create({id: request.params['track_id']}),
    converter = Converter.create();

track.stream(function (err, trackStream) {
  if (err) throw(err)

  trackStream.pipe(converter.process.stdin)
  converter.process.stdout.pipe(response)
})

Above is an excerpt from the HTTP request handler in the canvas.fm Node.js app. track is a simple wrapper around the SoundCloud API. Calling the stream method makes a few HTTP requests to SoundCloud's API and then yields the MP3 stream to the callback.

Inside the callback we use the pipe method built into Node.js to take the MP3 stream and pipe it into the converter object. This is just a wrapper around an ffmpeg child process. We can then pipe the output of this ffmpeg process back into the response.

The pipe method takes care of any complexity in handling these streams, making sure that they are always kept in-sync by pausing, draining and resuming streams as required. All this means that, regardless of the size of the MP3 file from SoundCloud, the memory footprint of the Node.js process doesn't grow in size.

Conclusion

Combining these technologies made it fairly simple to put together a fairly sophisticated looking visualisation. Below are a few samples of the images that canvas.fm can produce.

Translation of Matz Q&A article after joining Heroku

makoto@new-bamboo.co.uk (makoto) — Tue, 12 Jul 2011 00:00:00 +0000

NOTE from translator:

This is a translation of an article from "PublicKey" - Enterprise IT x Cloud Computing x Web Technology / Blog.

Mr Junichi Niino kindly allowed me to translate his original article from Japanese to English.

Also, special thanks to @knowtheory for editing my translation.

Title: "I am looking forward to accelerating Ruby's progress" - Q&A with Matz regarding joining Heroku as a chief architect.

Date: 13th July 2011

Introduction

As mentioned in my previous article, It has been announced that Yukihiro Matsumoto (Matz) is joining Heroku as Chief Architect of Ruby. I asked Matz about the story behind his future role via email.

My mission is to make Ruby core more functional and higher quality.

PublicKey (Q going forward): Can you explain to me how this came about?

Mr Matsumoto (Matz going forward): When I last met Mr Marc Benioff (CEO of Salesforce.com) he asked me how he could support the development of Ruby.

So I told him that I want to improve the situation which the majority of Ruby Core developers face, that they are either doing their work in their own spare time or they worry about their job security.

He said he could provide us some support and that's why I decided to join Salesforce.com through Heroku (Note: Several members of Ruby Core are currently under negotiation to join Heroku).

So, the core of our work will not change. Our mission is to develop Ruby Core, to make Ruby more functional and higher quality. Having said that, I am expecting that the job security we are being provided and Ruby usage feedback we get from heavy Ruby users such as Heroku will accelerate the progress of Ruby development. Otherwise, there would be no point to accepting the offer.

However, joining Heroku as Chief Architect does not mean that I swear loyalty to Heroku and Salesforce.com. This does not change relationship with NaCl and Rakuten which continue to support me and for whom I also remain chairman of the Ruby association.

As for the reason for the title? I picked the name "Chief Architect" out of multiple options as it sounded most business neutral. I am not going to be involved with any business decision about Heroku in future.

Q: Do you have any expectation about your role at Heroku?

Matz: As I already mentioned, it basically remains the same, but our development progress will be faster. Besides that, I would like to have closer communication with Heroku, so that we can work out removing constraints (if any) about using Ruby in cloud environments.

Also, I would like to preserve my good relationship with other companies such as Engine Yard and VMWare and their trust that I am "Ruby's Matz" even after joining Heroku.

Q: Do you have any vision of Ruby and Cloud at this moment?

Matz: To be frank, Many of Ruby Core developers (including myself) do not have much interest in the Web, but it is clear that cloud computing has become more and more important. So I would like to receive more input from these cloud environment and reflect that into the future direction of Ruby.

Q: Are you going to work in Silicon Valley, or join from Matsue remotely?

Matz: My lifestyle won't change (my working style may change though), so it is most likely that I will join the development from Matsue remotely.

As far as I am concerned, I don't feel that moving to Silicon Valley would necessarily make me happier than i am now.

However, I would like to have more direct communication with people in San Francisco/Silicon Valley (including Heroku) a few times a year.

(NOTE: To be precise, the headquarters of Heroku are in San Francisco, not Silicon Valley, but it says "Silicon Valley company" at the press release.)

Q: Any other comment?

Matz: I have heard that many people (outside of Japan) did not know that Ruby core team hasn't had a sustainable development model until now. I hope this movement becomes a role model for any Japanese software developer who publishes information from Japan.

Q: Thank you.

Degradable JavaScript Applications Using HTML5 pushState

oliver.n@new-bamboo.co.uk (olivern) — Thu, 03 Feb 2011 00:00:00 +0000

What is the problem?

Using the location hash to keep track of current page state and enable back button navigation is more and more common with large, full featured, client side JavaScript apps. Whilst the behaviour this gives is definitely an improvement to the user experience, implementing this with the location hash has some shortcomings.

Thankfully, as with everything else on the web, HTML5 is here to solve all your problems, with two methods and an event, pushState, replaceState & onpopstate.

What is pushState?

pushState & replaceState

The history object gains two new methods, pushState & replaceState, that allow us to change the current location of the browser without making an HTTP request. These new locations are stored in the browsers history so that both the back and forward button work as expected.

history.pushState({foo: "bar"}, "foo page", "/foo")

If you have a modern browser you can try out the above code in the console. You should see the url bar change to http://blog.new-bamboo.co.uk/foo. The browser doesn't check whether that url exists on the server, it does't even make an HTTP request. You'll notice that if you press the back button the url will change back, and pressing forward will set it to http://blog.new-bamboo.co.uk/foo again.

The other method, replaceState, is very similar but instead of pushing a new state into the browsers history it replaces the current state (as the name suggests).

history.replaceState({foo: "baz"}, "bar page", "/bar")

Both methods take three params, a state object (more about this later), a title and a url. The title is currently ignored by browsers, my guess is that at some point it will set the window title. The url param is what is used to set the current location in the browsers url bar. It is an optional parameter and if omitted the current location will be used. You can read more about these methods at MDC.

onpopstate

The onpopstate event fires whenever the browser location changes from the use of the back or forward button being used, regardless of whether pushState or replaceState were used to set the current location.

window.onpopstate = function (event) {
  // see what is available in the event object
  console.log(event)
}

Used in combination with the pushState & replaceState it can be used to handle changes in the browsers location. If the popstate event is fired as a result of navigating to a point in history that was added with either pushState or replaceState then the state object passed when calling those methods will be available as event.state.

Unfortunately there is no native onpushstate event, so to be notified of when something is pushed onto the history you will have to roll your own. Again to get more details about this event head over to MDC.

pushState vs location.hash

pushState has some considerable advantages over location.hash for the purpose of not breaking the back button. Firstly, although it is now part of the HTML5 spec, the onhashchange event is not supported everywhere, and until recently checking for changes to the hash required polling.

Also, as the name suggests, the onhashchange event only fires when the hash changes, it is not possible to detect when the same location is navigated to consecutively. With push state you can push the same location onto the browser history as many times as you wish and onpopstate will fire for each of these locations.

The biggest short comings of using the location hash are that the url anchor is a purely client side piece of information. When navigating to a location with an anchor the server is never sent the hash proportion of the location. This can cause issues when JavaScript is not available to provide the routing based on the hash location.

Since the whole url can be changed when using pushState, any time that url is navigated to without JavaScript available, the server gets the full location. This allows a server to respond with the full request.

Davis.js vs Sammy.js

Here at New Bamboo we have made full use of Sammy.js for over a year now for routing in JavaScript applications and it has served us well. Sammy.js uses location.hash for its routing so that you can use links with an anchor to control page state.

<a href="#/hello/bob">Say Hello to Bob</a>
<a href="#/hello/dave">Say Hello to Dave</a>

var app = Sammy(function () {
  this.get('#/hello/:name', function (ctx) {
    alert('Hello ' + ctx.params['name'])
  })
})

The above example shows a very simple Sammy app. When clicking on the links the location hash changes and the route is run, displaying a friendly greeting. This works brilliantly when JavaScript is available, however if the links are clicked and there is no JavaScript then nothing will happen. The server wouldn't know anything about our intention to greet either Bob or Dave.

I created a small library called Davis.js, which is heavily influenced by Sammy, and that uses pushState rather than location hash for its routing, the above example using Davis looks like this:

<a href="/hello/bob">Say hello Bob</a>
<a href="/hello/dave">Say hello Dave</a>

var app = Davis(function () {
  this.get('/hello/:name', function (req) {
    alert('Hello ' + req.params['name'])
  })
})

As you can see the two are almost identical, the only difference is the href of the links. If JavaScript is unavailable with the Davis example, then clicking on the links will make a regular request to the server. As long as the server can also respond to the /hello/:name route then the user will still get their friendly greeting.

Davis really is just a simplified version of Sammy using HTML5 pushState - be sure to check out a slightly more complex example and the docs.

Reuseable templates for dryer code

Where Davis really shines is in combination with a templating system that can be shared between both the client and the server. A good choice for this would be Mustache templates. A simple greeting template might look like this.

<h1>Hello {{name}}</h1>

If both the server and the client use this exact same template to respond to the /hello/:name route then it is relatively simple to make our simple app degrade gracefully when JavaScript is unavailable. So our JavaScript route might look like this:

this.get('/hello/:name', function (req) {
  var html = Mustache.to_html("<h1>Hello {{name}}</h1>", {name: req.params['name']});
})

And the server, using Sinatra for example, could use the same template and do the same thing.

get '/hello/:name' do
  Mustache.render("<h1>Hello {{name}}</h1>", {:name => params[:name]})
end

Mustache is currently very easy to integrate into Sinatra apps, however it isn't as straightforward with a Rails app. With the help of Mark Evans I have put together a simple gem called Poirot that enables use of Mustache partials within a Rails 3 app.

# in greetings controller

def show
  @name = params[:name]
end

# in app/views/greetings/show.html.erb

<!-- to include the template for JavaScript to share -->
<%= template_include_tag "greeting" %>

<%= render :partial => "greeting" %>

# in app/views/greetings/_greeting.html.mustache
<h1>Hello {{name}}</h1>

The gem gives the templates access to all the instance variables defined in the controller and the normal Rails view helpers. A default view class is created but if you need more control over what variables are available in the Mustache template you can extend it.

# in app/views/greetings/greeting.rb

module Greetings
  class Greeting < Poirot::View
    def time
      Time.now
    end
  end
end

The greeting mustache template would now also have access to the current time.

Conclusion

Using HTML5 pushState in combination with re-useable templates can provide a good starting point for more degradable and accessible JavaScript applications. Support for pushState is good in all good modern browsers, it is already being used by GitHub and Flickr, so the next time you need some kind of back button support in JavaScript apps why not try it out.

Learning Japanese the Rubyist way

makoto@new-bamboo.co.uk (makoto) — Fri, 17 Dec 2010 00:00:00 +0000

Introduction
Step 1: How to read Japanese characters
Step 2: Japanese and OO
Step 3: Japanese and functional
Step 4: Writing Japanese programming language in Ruby
Summary
Ruby Advent Calendar

Introduction

Have you ever thought about learning Japanese, but it looks too difficult to learn? Surprisingly, Japanese and Ruby share some common features and concepts. This is a shortcut version of my presentation called "Japanese and Ruby" which I presented at LRUG.

When you finish reading this post, hopefully you find Japanese language less magical, and may even add "Learn Japanese" to one of your 2011 new year resolutions.

Learning a new language always has a bit of steep learning curve. Go and get some coffee before you start!!

Step 1: How to read Japanese characters

One of the first big hurdle when learning a language is to remember all characters. This is not a issue if you are learning a language based on alphabet, but many non western languages have their own character sets.

To make the matters worse, Japanese uses three different character set, Kanji (Chinese character), Hiragana, and Katakana. Hiragana and Katakana each has 46 characters and there are a lots of Kanji (possibly 50,000, though we use ONLY 2000 ~ 3000 in daily use).

Here is the mapping of Hiragana, Katakana, and Alphabet.

(The diagram is from Wikipedia)

The point here is not to overwhelm you with the amount of information, but to let you think "WHY" Japanese uses 3 character sets.

Originally, Japanese did not have its own character set, so we used to borrow characters from China (Kanji). Since Chinese grammar and Japanese grammar are completely different, it was not easy to map all these Kanji into Japanese sentence. That's when Hiragana and Katakana were born to supplement Kanji. Hiragana is often used as a glue to combine words into sentence where Kanji alone is not good enough.

For example "行" is a Kanji character which means "to go". Japanese has many different ways to change the ending of verb (eg: goes) end we use Hiragana to supplement. Here are the examples.

Japanese	Alphabet	Meaning
行く	Iku	I go
行かない	Ikanai	I do not go
行こう	Ikou	Let's go, the more casual way
行きましょう	Ikimasho	Let's go, the more polite way
行け	Ike	Go, very non polite way

NOTE: If the above examples do not look like Japanese, you have an encoding issue. Make sure that your browser encoding is set to UTF-8

Katakana, on the other hand, was often used alongside with Kanji so that people can understand how to pronounce the Kanji. Nowadays, Katakana is often used to represent new words which came from foreign countries.

eg: 漢字（カンジ）, ルビー

(Trivia. The above example is expressed with html5 ruby tag )

(Another Trivia. Before multibyte became common, Japanese computers were only able to handle Alphabet and single-byte Katakana (eg: ﾙﾋﾞｰ), instead of multibyte (eg: ルビー). Some banks' ATM slips still use this single byte Katakana)

Even though they are the conventions, you can use Kanji, Katakana, and Hiragana interchangeably.

The following 3 all means "Cherry blossom bloom" and pronounces the same "Sakura Saku"

桜咲く　
サクラさく　
さくらさく

(Trivia. The world "Karaoke" is the combination of Kanji "Kara"(空 , means "Empty") and English "Oke" Orchestra.)

Here is the quick recap of what you learnt so far.

Kanji came first to import Chinese words
Hiragana was created to suit for domestic use
Katakana is used to adopt new words

Doesn't this "There are many ways to achieve one thing" concept familiar with Ruby's philosophy?

Ruby came first to bring concept of OO & Functional
Ruby was created to suit for every day scripting use
Ruby keeps evolving by adopting new concepts (Fiber, Multinationalization/M17N, Refinements etc)

Step 2: Japanese and OO

Satoshi Nakashima is a well known Japanese blogger who used to work at Microsoft as one of the development team members who shipped Windows 95 and Internet Explorer.

He was once asked "Are there anything it helped to create Windows 95 as a Japanese?" He initially did not come up with anything, but then thought that Japanese grammar structure is more suited to Object Oriented programming. To explain his thought, I will explain you some basic Japanese grammar.

English and Japanese has very different grammatical order.

English grammar structure is called "SVO"(Subject - Verb - Object), while Japanese one is called "SOV"(Subject - Object - Verb)

If I put "I eat bacon" in Japanese order, it is going to be "I bacon eat"(Watashi ha bacon wo tabemasu "私はベーコンを食べます")

At first glance, English order is clearer as "what you do"(verb) comes next to "who does it"(subject). It's almost like command line options (eg: git clone url).

The problem of command line options is that there are so many choices that it's hard to figure out which command you are supposed to use.

On the other hand, Japanese grammatical order is more similar to GUI. You often (right-mouse) click an object you are interested, then it suggests the possible actions. This is much more user friendly because you do not have to know all the possible actions and its argument options.

As you already know, Ruby is one of the best scripting languages to express OO (though you can write in procedural, or "command oriented way" if you wish)

# Procedural 
open("box") 
open("car")
open("file", "foo.txt")  

# OO 
Box.new.open 
Car.new.open 
File.open("foo.txt")

In the above example, they both do exactly the same thing, but the implementation will be quite different. For procedural example, I imagine that you have to keep adding nested "if" statement as logic becomes more complicated. On the other hand, the logic of OO way is kept isolated within each class.

Step 3: Japanese and functional

I often says Japanese is a politician's language. What does this mean? My definitions of politicians's are

they do not commit to anything unless necessary
they mean different things depending on context

In Japanese grammar, there is a term called "Postpositional" ("Pre-positional" is often used in English, such as for you, after dinner, and so on). Postpositional is used to decide the role of noun which it supports. This enables you to change the order of structure very flexibly, chain as many sentence as you like, and also let you omit subject.

Here are some examples of what I just said.

Japanese	English	How to pronounce	Structure	How it is ordered if written in English
私はベーコンを食べます	I eat bacon	Watashi ha bacon wo tabemasu	SOV	I bacon eat
ベーコンを私は食べます	I eat bacon	Bacon wo watashi ha tabemasu	OSV	Bacon I eat
ベーコンを食べます	I eat bacon	Bacon wo tabemasu	OV	Bacon eat

And this is the example of chaining too much sentence together.

One of the common mistakes Japanese people make when writing a sentence is chaining too much, because it is very hard to digest the whole sequence (One of my friends explained this as "Don't write a sentence which could cause stack overflow"). What makes English very logical and concise (in my opinion) is because the subject and verb comes at the beginning. Even though you can still write verbose sentence in English, this strict ordering forces you to write things relatively concise.

On the other hand, you can write a lot of sentence in Japanese meaning nothing because it omits subject, and also the verb you used at the very end have very loose relationship to the sentence you started at the beginning.

Now let's move back to how this (loosely) relates to some of the concepts in Ruby.

they do not commit to anything unless necessary => Lazy evaluation
they mean different things depending on context => Block

eg:

% 10000.times => #<Enumerator: 10000:times>

User.order('users.id DESC').limit(20).includes(:items)

File.open("/tmp.txt").each do |line|     
  puts line 
end

Functional features of Ruby lets you do crazy metaprogramming. Though they are powerful, abusing may confuse people to understand the code and may cause unexpected bug ;-P

Step 4: Writing Japanese programming language in Ruby

So, how are you doing so far? Easy peasy Japanesey?

(Trivia: the above expression is apparently common phrase in UK, derived from some TV commercial saying "easy peasy lemon squeezy")

When you learn a new language, reading books/articles are not enough. You always need to practice. Having said that, speaking to real Japanese people from day one may be a bit too difficult (or you just do not have a Japanese friend ;-( ), so here is a toy for you to play around.

Japanize

Some of my colleague once asked me "Are there any Japanese programming languages? What I mean is not just to be able to write Japanese text as string, but all programming syntax (such as "if", "loop") are actually in Japanese". Yes, there are some. Nadeshiko and Mind are the ones. However, I decided to write it myself using Ruby, and here is the result.

Looks amazing, isn't it?

Here are few more Japanese examples to understand what I just showed.

'に' and 'を' are postpositional which means that the words in front of them (1 and 2) are objects.
'たす'(hiragana) and '足す'(kanji + hiragana) are both verbs and mean "to add"
'て' is also postpositional which says this is end of one sentence and next sentence will start (equivalent to "and")

In my programme, I simply used postpositional as delimiters to split a Japanese phrase into words.

(Trivia: Japanese words are not separated by space ,so tokeniziing Japanese are very important part of natural language processing)

１に２をたして４を掛ける

Becomes

[1, 2, :+, 4, :*]

Japanese grammar is a bit like reverse polish notation, or a stack machine which is often used by a compiler to process a programming language. So the above array is equivalent to the following mathematical calculation.

describe Evaluator do   
  it "must calculate all operands" do     
    Evaluator.new([1, 2, :+, 3 , :* , 1, :-, 2, :/]).       
      evaluate.must_equal ((((1+2) * 3) - 1 ) / 2)   
  end 
end 
# NOTE: This is minitest which comes by default in Ruby 1.9

There are a few more secrets

ruby-1.9.2-p0 > 1
 => 1 
ruby-1.9.2-p0 > １
NameError: undefined local variable or method `１' for main:Object
    from (irb):8
    from /Users/makoto/.rvm/rubies/ruby-1.9.2-p0/bin/irb:17:in `<main>'
ruby-1.9.2-p0 >

If you see the video closely, you can notice that the number I typed is slightly different from normal ascii number. It's unicode number, so it raises "undefined local variable or method" error.

Since Japanese does not have any space between words, you can catch an entire sentence as a method

ruby-1.9.2-p0 >  １に２をたす
NameError: undefined local variable or method `１に２をたす' for main:Object
    from (irb):9
    from /Users/makoto/.rvm/rubies/ruby-1.9.2-p0/bin/irb:17:in `<main>'
ruby-1.9.2-p0 >

So I just passes the entire expression as one method and catches at method_missing

# -*- encoding: utf-8 -*-
def method_missing (*obj) 
  if 数字[obj[0][0]]
    parsed = Parser.new(obj[0].to_s).parse
    Evaluator.new(parsed).evaluate 
  else
    super 
  end
end

This is how japanize works.

When I was researching how to implement very simple compiler/interpreter, I learnt a lot from an article written by Koichi Sasada (the creator of Ruby 1.9 Virtual Machine). The article is written in Japanese, but there is one sample code which implements some basic VM functionalities in Ruby.

RubiMaVM

The code handles not just maths, but also loop and if statement. If you are curious enough, you could implement something similar on top of Japanize. I will accept pull request as long as it looks like Japanese !!

Summary

Here are the list of things you learnt through this post.

Japanese uses 3 characters, Kanji, Hiragana, and Katakana.
Japanese grammar structure is Subject - Object - Verb(SOV)
Japanese order can be flexible thanks to postpositional

Even though Matz did not intend to reflect Japanese language into the design of Ruby, I think there are certain influence, since anyone's thought is influenced by the language they use.

(NOTE: @yukihiro_matz says "Japanese and Ruby? I try not to think too much about Japanese culture. The method chain looks like Japanese, but it's just a coincident. Having said that, the support of M17N is heavily influenced by the use case of Japanese people. Otherwise, I wouldn't spend too much time on such a hard problem". You can compare how @matz_translated bot actually translated the sentence).

If you are interested more, the full slide of my talk at LRUG is here.

Ruby and japanese

View more presentations from inouemak.

The talk was videotaped, so it may be uploaded here later.

Ruby Advent Calendar

This is Day 17 of Ruby Advent Calendar(Each day leading up to December 25th, one person posts an article to their blog and adds a link to their blog on the Advent Calendar). The previous entry was written by matschaffer or gautamrege (order seems a bit screwed up), and the next will be written by elight.

Ruby divide at RubyKaigi 2010 and what can you do as a Rubyist? (Part 2)

makoto@new-bamboo.co.uk (makoto) — Tue, 21 Sep 2010 00:00:00 +0000

From Matz On Pusher

At previous post
"RubyKaigi must end (for now)"
So, what can we do?
Summary

At previous post.

At previous post, I went through the following Ruby divides I noticed during RubyKaigi 2010.

Japanese / English = Ruby core ML and many variable documentations are written in Japanese and non Japanese developers are hard to access such information.
Committers / Users = There seems divides between C dvelopers (C Ruby committers ), Java developers (JRuby community), and the rest (Ruby users).
Before / After Rails = Japanese Ruby scene is very unique from the rest of the world(aka Galapagos). Some part are very advanced, but others are missing important aspect of Rails culture (eg: github, rubygems).
Hobby / Enterprise = The ratio of Japanese Rubyists who uses Ruby at day job seems low.

These problems probably exist on any Ruby community(except English/Japanese bit) at certain extent, but the divide is especially deep in Japan, partly caused by the language gap, but some of them are inherent problems since Ruby was born in this country.

"RubyKaigi must end (at least once)"

Some of these problems have been pointed by @shyouhei" a year ago at his blog called "Ruby Must end(at least once)"(The English translation is here.

In his blog,

RubyKaigi is maintained by the hard work of the enthusiastic volunteers, but it is coming to the stage where the growth of Ruby community is going beyond the capacity of these volunteers.
The majority of the attendees are simply happy to attend the Kaigi and mingle with other attendees, but these networking efforts do not result back to Ruby community to keep up the work (such as documentation, more ruby committers and maintainers for bug fixes).
As a solution, RubyKaigi should end for now and think about new ways to have a sustainable and scalable organisation/community.

This idea became the foundation of the theme for RubyKaigi2010.

Since the end of RubyKaigi, there have been a lot of conversation on Twittersphere suggesting various solutions, and some people actually made some actions.

@pwim started Tokyo Ruby User group
asakusa.rb and sfruby.rb became syster meetups (by @a_matsuda, @sarahmei, and @ultrasaurus).
Mengurume Matsue Ramen contribute some of their sales to Ruby Association (NOTE: This was in action before/during the Kaigi, too).

From "Ruby on 松江ラーメン・キュービックシリーズ" at amazon

So, what can we do?

I also spent a bit of time thinking about these problems, and here are my list of suggestions to both Japanese and non Japanese Rubyists. I would love to know other people's suggestions,too.

Read "What can you do as a Rubyist?"

Unlike list of Rails contributors, Ruby has less contributors (RoR has 1600 contributors, while Ruby has about 700 for last 5 years each).

About a year ago, @fdiary wrote an article called "What can you do as a Rubyist" at Rubyist Magazine (The full translation is here). In this article, he talked about the following

Use Ruby Software, so that you can hack and extend the system you actually make use of.
Update to the latest (especially Ruby 1.9), take profiling, check memory usage, and report if you find any bugs.
Do the periodic code review via commit logs, so that you will get better understanding about the software (great way to learn the code)
When you get stack at bug fix, report and ask for help.
Patrol blogs/twitter and help other people.

This is a great way to contribute to Ruby even if you are not the hard core C developers, and looks great way to improve your Ruby skills, debugging skills, and performance profiling/improvement skills.

Let's make Christmas Ruby

Hacking into Ruby source code seems hard. Why don't you start an easy and fun way first?

The funniest Rubyist during the Kaigi was @technohippy. He is known as the "first and last author of Google Wave book". Apparently his book was printed on the day Google announced not to continue the Google Wave development. During his lightning talk, he mentioned 3 of his funny Ruby clones(objective ruby, xruby, and fruby). The best one was "Fuzzy Ruby" which has "if maybe", "if perhaps", and "if probably" syntax to return true in different probability.

During the talk, he mentioned that we have passed the time to keep receiving christmas present from Matz, and now it is time to give something back to Matz.

count = 0
1000.times do
  if true probably
    count += 1 
  end
end
puts count #=> 839

fuzzy ruby example

This looks like just a joke (honestly, I don't think Matz is going to accept "if perhaps" as a new syntax), but I think this is a great way to get to know the Ruby internal, and it could potentially increase the number of Ruby committers in future.

Here are the slides/videos about hacking Ruby internals.

Jruby Hacking Guide
Hacking Parse.y = Slide, Video

More sister user groups

It's great that asakusa.rb and sfruby.rb are sister groups after many foreign Rubyist visited Japan, but we can have more partnership among other Japanese ruby user groups and non Japanese ruby user group. Apparently there are more than 20 ruby user groups in Japan. This is especially important for groups outside of Tokyo. Most foreigners tend to stay either in Tokyo or Kyoto, but you don't see many foreigners at other cities. Hence other regional Rubyists do not have chance to interact with non Japanese Rubyists. It would be fun if we can have Ruby User GoKon to match up Japanese user groups and non Japanese user groups at next Ruby Kaigi!!

Let's go to RubyKaigi 2011

Last, not least. Let's go to next Ruby Kaigi and make the final Ruby Kaigi memorable event and also great start of something new.

Summary

It's impossible to do everything I suggested here, but would be nice if you find something you could start from today, or you come up something else by reading this blog post. So what am I going to do? As you might have noticed, I started writing some of translations to Japanese Fine Software Writings wiki. If you would like to add more Japanese software writing (doesn't necessarily have to be "fine" nor "Ruby"), or would like to contribute as editors, please feel free to edit the wiki.

Ruby divide at RubyKaigi 2010 and what can you do as a Rubyist? (Part 1)

makoto@new-bamboo.co.uk (makoto) — Tue, 14 Sep 2010 00:00:00 +0000

Ruby divide at RubyKaigi 2010 and what can you do as a Rubyist? (Part 1)

Introduction

It is almost 2 weeks since RubyKaigi finished. For full details of the conference, check out Jake Scruggs' near real time report.

Ruby Kaigi was one of the most memorable conferences I have ever attended. There were countless efforts by volunteer staff to make the conference as enjoyable as possible. I met lots of interesting people and enjoyed every conversation in the conference hall, hallways, and Izakaya . All the topics discussed there were very thought provoking.

The theme of this year's RubyKaigi ("Kaigi" means "Meeting") was "Conflicts and Resolutions" and many talks covered various conflicts relating to the (especially Japanese) Ruby community like below.

Japanese / English
Committers / Users
Before / After Rails
Hobby / Enterprise

The resolution to these conflicts was rather interesting. @takahashim, the chairman of the Japan Ruby Society, announced next year's conference theme as the "Final RubyKaigi". He proposed a big assignment to all of the attendees. "What would you do if you know that next RubyKaigi is the last?"

As Jake mentioned, I suspect that many non Japanese attendees did not understand the whole context due to some language barriers. During this post, I will try my best to explain the background of each conflict so that we can think together what we can do to the community and RubyKaigi 2011

Who am I ?

Before I start, here is a bit of introduction about myself which tells you why my experience made me a rare highbred between Japanese and non Japanese Rubyists.

I was born and brought up in Japan. I worked in Tokyo as a DBA (DataBase Administrator) for 5 years, then moved to London 6 years ago. My encounter with Ruby is similar to most of you (non-Japanese Rubyists). I got to know Ruby through Ruby On Rails around 2006. As I learnt more about Rails, I learnt that the real power comes from its underlying language, Ruby.

Even though I did not experience the pre-Rails era, I can access all the Japanese resources which were written before Rails stormed the Ruby community. I also have been reading Rubyist Magazine(ja) and kept dreaming about meeting these "legendary Japanese Ruby Hackers" one day. It was exciting not only to meet Matz, but also other Ruby committers and core community members. It was also a great honour to present as a speaker about "Building Real Time Web" which was about doing WebSocket with (Javascript and ) Ruby, and a story about how we came up with Pusher.

My speaker pass and Japanese fan with Matz cartoon

Japanese / English

Before I headed to Japan, many people told me "wow, a ruby conference in Japan sounds interesting, but I don't speak Japanese, so wouldn't understand anything". In short answer, that's fine. There were always at least 2 streams, one of which was in English (most English talks were done by some well known rubyists like @wycats).

In long answer, you were able to enjoy even Japanese talks, thanks to a crowd sourcing real time translation effort led by @lchin and @fdiary. There were 2 big IRC & Twitter streaming boards next to the presentation slides, which were created by @june29 and used our own Pusher for the backend (the source code is here)

The most surprising moment was when @yukihiro_matz appeared on stage as a English - Japanese interpreter during Rails 3 / Ruby 1.9.2 panel discussions. (this was the last minute replacement of Jeremy Kemper's keynote).

I did not know that @tenderlove (the author of Nokogiri) was the only person who has commit rights in both Ruby and Rails. During the panel discussion, @wycats mentioned that there are communication problems between Japanese and non Japanese Ruby developers. The mailing lists are divided into two (English/Japanese), and some of the most important discussions are done on the Japanese one. They mentioned the past trial of translating all Japanese mailing lists into English, but it halted (due to lack of manpower). @yukihiro_matz later tweeted his worry about merging the two into one English one, because he fully understands how hard it is for some Japanese committers to communicate difficult topics in English.

Here is the panel discussion video.

It was hilarious to see @yukihiro_matz translating @wycats 's complaint to a Japanese audience.

Committers / Users

One of the things you don't get at any other Ruby conferences is having the chance to meet Ruby core team members. I only knew a few names, so I was surprised to see that there were 26 of them on stage during Ruby 1.9.2 Q&A session. According to @yugui, the release manager of Ruby 1.9.2, the significant point of the release is spent (hence some delay) to pass all of the RubySpec (created by Rubinius team), so that other Ruby implementations can use the spec as a reference point to guarantee compatibility.

From rubykaigi2010

Besides the Q&A session, there were lots of other talks related to Ruby internals.

Keynote by Matz = About Ruby 2.0 features
Best Imitation of Your Class = Talk about Classbox which Matz mentioned at his keynote
We can make the GC X times slower than the original
How Did Yarv2llvm Fail
AOT Compiler for Ruby

Unfortunately, they were all in Japanese (and the IRC translation logs don't seem to be available), but you can read their short summaries in English.

One thing I realised during the conference is that RubyKaigi is actually a C developers' conference in disguise. I probably saw as much C code in the slides as Ruby code during the conference. Apparently, there is a book called "Ruby Hacking Guide" and there are even "RHG book reading group"s in Japan. There is an English version, but looks it is obsolete and not even complete.

One of my personal concerns is the gap between these "Ruby committers" and normal Ruby users. I have seen a few of these committers assuming that most Rubyists are converts from C/Java, but that's not entirely true.

I am a big fan of being a polyglot programmer, but learning C/Java as a Rubyist is a bit harder than going the opposite way, and I haven't seen lots of material targeting these. I hope someone knowledgeable in these languages gives a talk for "Introduction to C/C++/Java for Rubyists". Rubinius could fit into these gaps, but there were no related talks during the Kaigi.

Before / After Rails

For the majority of non Japanese Rubyists, it will be difficult to find someone around you who has never used Rails.

However, this does not apply to many Japanese Rubyists because there were already other influential applications, such as tDiary, Hiki, qwik or Rabbit, most of which I've never heard before.

"How to survive in a post rails world" was presented by @hsbt, one of the tDiary committers who recently started working in new company with a colleague who lives and breathes Rails and Github. Half of his slides are in Japanese, but you can still get some idea of what he talks about. He basically says that many of the "Before Rails" people still use sourceforge and raa, so suggests to put their work into Github.

How to survive in post Rails' world.

View more presentations from Hiroshi SHIBATA.

I was a bit surprised that some of the well known Japanese Rubyists don't actually use Github. Since nowadays we assume that all the active projects are in Github, some of the "Before Rails" projects may be neglected just because they are not on Github. @hsbt stressed that it is now inevitable for Rubyists to live without such a system and suggested to move lots of good "Before Rails" heritage into some sort of social coding platform (NOTE: tDiary is still in active development and now they are hosted at github).

Another related talk was done by @a_matsuda a week after the RubyKaigi at a conference called "Ruby World Conf" , organised in Matsue (Matz's home town).

The title is "Ruby in the World, and Rails in Japan".

Ruby in the World, and Rails in Japan

View more presentations from Akira Matsuda.

In his slides, he even says Japanese Ruby is a "Galapagos". The word "Galapagos" is often referred to Japanese mobile phone technologies. It evolved in a isolated place , so it is very unique (and often advanced), but has problems blending into the global standard.

I recently saw some Japanese person tweeting that "Using Rails or Gems are very inconvenient because you can not install them on a shared hosting environment". This kind of conversation sounds so 2006 ~ 7 for us, because all Ruby/Rails people abroad just use a cheap VPS or Heroku for casual use.

Hobby / Enterprise

In the same slide, he also mentions that "Ruby super rockstars in japan tends not to work with Ruby but something LEGACY" . Apparently @tendorlove asked him what these talented programmers do without Rails or Rubygems, and his answer was this.

Ruby Ruby - Esoteric, Obfuscated Ruby Programming

View more presentations from mametter.

Playing Esoteric programming or code golf are actually a great way to brash up your programming skills, so doing it in itself is nothing wrong. However, it is sad if programmers with such high skills can not find Ruby related jobs when he/she wishes (NOTE: the above slide is just an example of a super-skilled Japanese Rubyist, and I don't know what he does for his day job).

During one of the Kaigi events called "I am a Rubyist working abroad, any questions", I had a chance to ask all attendees how many people are actually using Ruby in their day job in Japan, and was surprised to see that less than half the people in the room raised hands. This is very similar to the situation during 2007 ~ 2008 in Europe (and @pat also said it was similar in Australia). Nowadays most Rubyists outside of Japan already work as full time Ruby (or Rails) programmers. This is rather unfortunate considering that Japan is where Ruby was born. I guess this is because Ruby has been seen as "hobby" language for such a long time.

There seemed to be strong interest to bring business to the ruby community during RubyKaigi. There were 3 sub events mostly focused on the business side.

I only attended the third event, organized by CookPad, and it was kind of interesting. "Cookpad" is the most popular cooking recipe site and 6th biggest Rails site in the world.

The first half of the talk was about how to scale such a huge site, but the second half was about their working environment, and they stressed how much effort they were putting in to provide best working environment for developers (btw, they were recruiting both Japanese and non Japanese developers massively during the Kaigi). This is normal in America or Europe to attract talented developers, but it is very unusual in Japan, because the Japanese IT industry is often called 3K , "Kitsui"(Hard), "Kaerenai"(Can't go home), and "Kyuryo Yasui"(Low salary).

Summary

Even though the RubyKaigi2010 was a big success, thanks to the hard work of all the volunteer staff and a combination of interesting talks covering a wide range of the Ruby divide, it revealed that the Japanese Ruby community is facing some growing pains.

So, why is finishing RubyKaigi next year the solution?

I will explain in more detail in the next post.

Adding awesome mini views to your web apps

max@new-bamboo.co.uk (max) — Mon, 02 Aug 2010 00:00:00 +0000

In a couple of our internal apps, we use a UI design paradigm that we affectionately call an "awesome bar". I really like these, and I think they make many types of applications much more useful and intuitive. While they have generally been hard to create in web apps, the introduction of canvas and inline SVG has made them a synch to put together. I see a lot of great examples of canvas and SVG that are absolutely mind-bending to look at, but very few examples which enhance the traditional applications and websites we use on a day-to-day basis.

I won't claim to have invented these, but I was reminded of how cool they are recently when I read this post: http://blog.asmartbear.com/creativity-over-optimization.html. I will give a brief explanation of what these are with some examples, why I think they rock, and some sample code.

What are awesome bars?

To give a quick introduction of what these are I have made a screencast of the awesome bar from our nifty scheduling application called Mandays. You will see that the horizontal bar at the bottom is a miniature representation of the grid at the top, and when you draw in it, the awesome bar at the bottom updates itself. You can then use the awesome bar to navigate your timeline.

What is so great about them?

Awesome bars allow you to view a piece of information at both a macro level and a micro level simultaneously. With traditional scrollbars, there is always a percentage of a given "document" that is off the screen.

However, as humans we seem to have the ability to put together a mental image of what a whole document looks like even though we view only a portion at a time. A commmenter on the blog post I linked to earlier says that:

"It’s appealing because it *resembles* what it’s representing, whereas [a scrollbar] *symbolizes* that. It’s the difference between ‘bang’ and ‘the sound a gun makes when it’s shot’."

Awesome bars put the UI element in context. Searching for a specific spot in a document doesn't require scanning blindly through the whole thing via a small portal.

Types of interaction

In Mandays, the awesome bar allows interaction, and (controversially) replaces the native scrollbar. It simply provides you with a bit more context on where you are navigating too.

In our backlog management app called Truestory, the awesome bar can actually be interacted with directly. Though it has drag and drop of the main list, it was always a pain to take a story from the bottom to the top. With the awesome bar involved, we can drag stories to the approximate place in the backlog directly via the mini view.

However, some minibars don't have any interactive elements, and are purely informational.

Building an awesome bar for a blog

As an example, I have built a very simple HTML application that demonstrates the minibar in use on a blog post with comments. The rationale for this is that it is often hard to tell from a scrollbar how long a given post is, and what percentage of the page is filled with comments. To cut to the chase, have a look at the demo here, or poke about with the source here.

This is a read-only awesome bar, and just shows which bit of the page you are viewing. The nice thing about it is that the concept is really simple, and therefore easy to tweak. I have used JQuery here to make it simpler, but you may want to replace those parts with your library of choice (things like offset and position of DOM elements are made easy by it).

Some canvas basics

I decided to use a canvas element for this example but SVG plus raphael.js, would have been just as easy. There is a library to make canvas work on IE, but I haven't bothered with it here.

If you haven't used canvas before, all you really need to know for this example is that you can draw rectangles like so:


    ctx.fillRect(left_offset, top_offset, width, height);

The zero coordinate is the top left of the canvas by default, hence the left_offset and top_offset.

Layout

First I created a simple CSS layout with the blog post and comments on the left, and a position:fixed minibar on the right.

Encapsulating the minibar code

Next comes a small MiniBar function that we can create with the necessary configuration. When it is initialised, it sets the #minibar div and the canvas inside it to the height of the window (this would need to be bound to window resize in a proper application).


    var resize_canvas = function(){
      canvas.height =  $(window).height();
    };

Small potential gotcha is that canvas elements behave like images, and if you set the dimensions with CSS, the content of the canvas is scaled as well. What we are doing here is more like the following:

<canvas height="PIXELS" ></canvas>

Adopting a simpl[istic] scale factor

Once the canvas is set to the right height, we set about drawing the elements onto it. This is really easy, and just involves multiplying the height of the elements in the DOM with a scale factor derived from the ratio between the entire document and the viewable area.

The scale factor looks like this:


    var scale_factor = function(){
      return $(window).height() / holder_elem.height();
    }

using the height of the body is probably more reliable if you have a header and footer etc.

Drawing some stuff

The first thing we draw is the yellow block of the main article. We get the height with JQuery, and apply the scale factor.


    var draw_content = function(){
      var   content_height = $('#content').height();
      ctx.fillStyle = "rgb(255,255,0)";  
      ctx.fillRect (0, 0, 100, content_height * scale_factor() );
    }

We set the colour manually, but could derive that from the element itself.

Drawing the comments is a bit more involved because they have top offset, and I decided to highlight comments the author has made:


    var draw_comments = function(){
      $(".comment").each(function(){
        if (this.className.indexOf('author') != -1){
          ctx.fillStyle = "rgb(255, 100, 100)"; 
        } else {
          ctx.fillStyle = "rgb(200,200,200)";  
        }
        ctx.fillRect(0, $(this).offset().top * scale_factor(), 100, $(this).height() * scale_factor() );
      })
    }

Adding a viewer portal

We can use the same scale factor to place a box on the canvas to represent the part of the page we are viewing. The height of the box is the height of the window multiplied by the scale factor, and the distance from the top of the canvas is the scroll offset of the window.


    var draw_viewer = function(){
      ctx.fillStyle = "rgba(255,255,255,0.5)";  
      ctx.strokeRect(0, $(window).scrollTop() * scale_factor(), 100, $(window).height() * scale_factor() );
    }

We need the viewer to move when we scroll, so we bind to the window scroll method:


    $(window).scroll(function(){ draw() })

Because we have the view on the canvas itself, we have to clear the canvas first, and then redraw everything. This isn't very expensive though.

Taking the example forward

To make this more interactive, you could turn the viewer into a DOM element which has position:absolute. This allows it to be dragged, which can in turn affect the scroll position of the entire page. However, this is relatively easy to do, and can be an exercise for anyone interested.

HTML5-powered Ajax file uploads

pablo@new-bamboo.co.uk (pablo) — Fri, 30 Jul 2010 00:00:00 +0000

Introduction

File uploads have traditionally had very bad usability on the web. The standard solution was uploading files as part of a form, leaving the user to just wait until the process was done. We could offer barely any feedback of what was going on.

Several options appeared to make the process more bearable for the user. Some alternatives were client-based, such as using some Flash-powered element like SWFUpload. Other alternatives laid more on the side of the server, like leveraging Nginx's mod_uploadprogress with a pinch of Ajax. However, there was still the question of why there was no solution that avoided proprietary technology, required minimal hassle OR was free of far-fetched hacks.

Until now.

An important part of Panda (new blog here btw!), our video encoding service, is file uploads. Our users need to upload large videos and we offer an HTTP interface and a Javascript widget to do exactly that. However, our widget is based in Flash, and we would always rather offer solutions based on webstandards. Therefore, we set out to see how the upcoming HTML5 specification could help us. Specifically the features contained in two specs: the File API and XMLHttpRequest Level 2. I'll explain here how to make effective use of them. If you're interested in seeing the end result checkout the panda_uploader plugin on Github

Compatibility and detection

At the time of writing, this technique only works in the latest Webkit and Gecko browsers. Therefore you will still need to fall back to other methods if you wish to support Internet Explorer, Opera or others.

The best way to know if the client browser supports this feature (or any other feature for that matter) is object detection. In short, use Javascript to check whether elements of the API exist or not. In this case, the code is the following:

function supportAjaxUploadWithProgress() {
    return supportFileAPI() && supportAjaxUploadProgressEvents();

    function supportFileAPI() {
        var fi = document.createElement('INPUT');
        fi.type = 'file';
        return 'files' in fi;
    };

    function supportAjaxUploadProgressEvents() {
        var xhr = new XMLHttpRequest();
        return !! (xhr && ('upload' in xhr) && ('onprogress' in xhr.upload));
    };
}

If the function supportAjaxUploadWithProgress() returns true, you are good to go. If not, you'll have to revert to a different upload technique.

Simplest Ajax upload

The frontend code is pretty simple initially. The spec establishes that file input fields have a files property that gives us access to some attributes of the file, like so:

<input id="the-file" name="file" type="file" />

var fileInput = document.getElementById('the-file');
console.log(fileInput.files); // A FileList with all selected files

Note that fileInput.files is a FileList because the HTML5 spec allows for file inputs to select multiple files. Let's keep it simple though, assuming only one file will be selected. The general case is easy to infer from there.

Once the user selects a file, the list will be filled up with actual file objects:

var file = fileInput.files[0];
console.log(file.fileName); // "my-holiday-photo.jpg"
console.log(file.size); // 1282632
console.log(file.type); // image/jpeg

You can use this file object as an argument to the XMLHttpRequest.send() call, to send the file asynchronously over to the server:

var xhr = new XMLHttpRequest();
xhr.open('POST', '/upload/uri', true);
xhr.send(file); // Simple!

Feedback events

But we still don't have any feedback on how the upload process is going. Let's make the example complete with a simple progress indicator. This will show the progress on the debug console:

var fileInput = document.getElementById('the-file');
var file = fileInput.files[0];

var xhr = new XMLHttpRequest();
xhr.upload.addEventListener('progress', onprogressHandler, false);
xhr.open('POST', '/upload/uri', true);
xhr.send(file); // Simple!

function onprogressHandler(evt) {
    var percent = event.loaded/event.total*100;
    console.log('Upload progress: ' + percent + '%');
}

Please note that the event is not set on the xhr object itself, but on xhr.upload. If you get this wrong, you'll be notified of the progress of the response from the server after the request is complete, rather than the upload preceding it.

Additionally, there are some other events that can be useful:

xhr.upload.onloadstart: the upload begins
xhr.upload.onload: the upload ends successfully
xhr.upload.onerror: the upload ends in error
xhr.upload.onabort: the upload has been aborted by the user

And of course we'll have to use our old friend xhr.onreadystaterequest to read the response from the server.

Reading the raw data

There's still one problem though, which is not evident from just reading the code above. When a file is uploaded using this method, the browser will send a request whose content will be just the file contents as raw data. This means that your server-side framework won't be able to make it available to you as per normal.

For example, in PHP normally you would use the $_FILES global variable; in Rails the params method; in Django request.POST dictionary. None of these would work with this technique. It also means you won't be given the name of the file, which is normally available using the mentioned accessors.

But don't worry, both things can be solved easily. First, the file name: you can pass it to the server on the very same upload request by putting it in a request header. For example:

xhr.setRequestHeader("X-File-Name", file.name);

If you do this before calling send(), you will be able to read the file name form the headers on the server side. Popular methods include:

PHP: $_SERVER['HTTP_X_FILE_NAME']
Rails: request.env['HTTP_X_FILE_NAME']
Django: request.META['HTTP_X_FILE_NAME']

Now there's the matter of how to read the file itself. Again, this will depend on your server side technology:

PHP: file_get_contents("php://input")
Rails: request.env['rack.input']
Django: request.raw_post_data

Being a good netizen

Another small detail is that, by default, the request is sent without a mimetype. Depending on your setup, this may not be a problem for you, but it's recommended that you set it. This is so third party applications can interoperate better with your code. And makes you a good netizen!.

For this, there's only one line you need to add to the code above:

xhr.setRequestHeader("Content-Type", "application/octet-stream");

Firefox!

This section was added as an update on 21/Oct/2010

Hang on, there's one last thing! Firefox implements the above from version 3.5, but there are a couple of pitfalls you have to be aware of.

Set events BEFORE opening the XmlHttp connection

Specially the "progress" event; otherwise Firefox won't fire it. This means that you have to do as follows:

xhr.upload.addEventListener('progress', onprogressHandler, false);
xhr.open('POST', '/upload/uri', true);

Instead of the following, "wrong" code (for Firefox anyway!):

xhr.open('POST', '/upload/uri', true);
xhr.upload.addEventListener('progress', onprogressHandler, false);

On Firefox 3.5, two names change

On Firefox 3.5, instead of xhr.send() you have to do xhr.sendAsBinary(). Also, instead of file.name, the name of the file is stored in file.fileName. Fortunately, Firefox 3.6 adopts the W3C convention and these changes don't apply.

You can still support both conventions in you code. For the file name do the following:

filename = file.name || file.fileName;

And for the send() call, do is:

if ('getAsBinary' in file) {
  // Firefox 3.5
  xhr.sendAsBinary(file.getAsBinary());
}
else {
  // W3C-blessed interface
  xhr.send(file);
}

A complete example

We have published a complete example of how this works. It's on GitHub and you are invited to play with it:

http://github.com/newbamboo/example-html5-upload

This uses Sinatra with a piece of Rack middleware that we have also made available separately, called Rack::RawUpload. This middleware will convert a raw upload into a normal request. This way, you won't need to fiddle around with the file name and its contents.

Custom event emitters in Javascript

ismael@new-bamboo.co.uk (ismael) — Wed, 14 Jul 2010 00:00:00 +0000

Websockets-based activity dashboard app. Read on to know more about the ideas behind it

In this post I'll attempt to summarise some patterns for designing event-based Javascript applications extracted from our projects and previous blog posts on the subject. I'll end with an overview of the techniques described and how they play together in a real-world application. This post is a follow-up from a presentation at the London Javascript User Group.

Background

We've blogged about Javascript patterns for complex applications before. If you're short on time, that post describes the way we lay out our client-side apps here at New Bamboo. It's an MVC-like setup where we have Model objects doing the persistence and data binding, and View objects listening to state change in those models via the observer patterns and custom events. If you haven't, I strongly recommend you read that post before this one.

The key component of this approach is using custom events to decouple the communication between layers. This is a simple View example:


var UserPane = function(event_source, $container){
  
  function update(event, user){
    $container.append('<li>:'+user.name+'</li>:')
  }
  
  $(event_source).bind('user_added', update)
}

And you would use it like:


new UserPane(document, $('ul#user_list'));

// Wrap DOM events and re-trigger with useful data using jQuery
$('#users input:radio').change(function(evt){
  $(document).trigger('user_added', {name: $(this).val()});
});

The example defines a View class that knows in which DOM element to render content ($container) and binds itself to the 'user_added' event triggered on the document object by other layers in the application.

We're passing the document element as the source of events, but you're free to swap it by anything that you can bind to and trigger events from. It doesn't even need to be a DOM element, and this is where OO and evented programming starts to look less counter-intuitive than one might have thought: you can build your own Event Emitters and model your domain around them.

Event Emitters In the wild

That is exactly what libraries such as Node.js are doing with an EventEmitter class you can extend from. The following is an object that emits a "tick" event with a random number every 5 seconds.


var EventEmitter = require('events').EventEmitter,
    puts = require('sys').puts;

var Ticker = function( interval ){
  var self = this, 
      nextTick = function(){
    self.emit('tick', Math.random(1000));
    setTimeout(nextTick, interval);
  }

  nextTick();
};

// Extend from EventEmitter 'addListener' and 'emit' methods
Ticker.prototype = new EventEmitter;

// A ticker instance with an interval of 5 seconds
var ticktock = new Ticker( 5000 );

// Bind an event handler to the 'tick' event
ticktock.addListener('tick', function( number ) {
  puts('number emitted: '+ number);
});

The nice thing about this is that objects interacting with the Ticker instance only need to bind themselves to events emitted by it. There's no need for public methods apart from addListener (or $.bind in our previous jQuery example).

How we use Event Emitters

Back in the browser side of things, you can build event emitters that represent objects in your domain and trigger events when their status changes. Then other parts of the application can observe those events and react accordingly. A nice abstraction of this pattern is js-model, an ORM-like Javascript library we developed to model domain objects with extra server persistence baked-in.


// Js-model example
var Post = Model("post")

Post.bind("add", function(new_post) {
  addObjectIntoUI(new_post)
})

// This will trigger the 'add' event on the Post object, 
// executing any handlers observing it.
var post = new Post({ foo: "bar" })

More info on js-model on its documentation page.

This is also the approach that we've taken with our Pusher websockets service. With it you can bind your local objects to events coming from the server in a completely transparent way.


var server = new Pusher('you app key', 'some channel');

server.bind('user_added', function(user){
  $('ul#user_list').append('<li>:'+user.name+'</li>:')
})

And of course you would build your own objects to do the binding and keep your main app clean and expressive.


new UserPane( server, $('ul#user_list') );

I talk about how this is done in this post.

Do it yourself

Because Pusher.js is framework-agnostic, we don't use jQuery's event-binding API. Instead, Pusher implements its own bind method. But once you're using this pattern in one part of your app there's no reason you couldn't use it for the rest of your domain objects. The only thing they need to do is implement binding and triggering of events. This is a simple abstract prototype you can use in your own objects. This is the Ticker example again using that snippet:


/* Periodically send out dummy events 
--------------------------------------------*/
var Ticker = function( interval ){
  var self = this, 
      nextTick = function(){
    self.trigger('tick', Math.random(1000));
    setTimeout(nextTick, interval);
  }

  nextTick();
};

// Extend from AbstractEventsDispatcher 'bind' and 'trigger' methods
Ticker.prototype = new AbstractEventsDispatcher;

var ticktock = new Ticker( 5000 );

ticktock.bind('tick', function( number ) {
  puts('number emitted: '+ number);
});

The only reason I've changed 'addListener' to 'bind' and 'emit' to 'trigger' is to keep it familiar for jQuery users.

What's important here is that once you model your objects around this minimal event-centric interface ('bind' and 'trigger') it makes little difference what your event emitter objects actually do -and how they do it. You app can be fully evented while retaining the best bits of object orientation such as polymorphism and inheritance.

A working application

I've recently built an activity dashboard app that uses Pusher when deployed to production but a mocked, in-browser event emitter object in development to simulate a stream of events coming from the server. My View objects are oblivious to the origin of those events and just focus on binding to them and rendering them onscreen as they come. You can see it working on the video above.

It starts by defining a series of self-contained widget objects. I then loop through them and instantiate each with the same instance of an event emitter -in this case Pusher or a mock server implementing bind for custom events.

This is the basic code:


var server = new MockServer() // or new Pusher('app_key', 'channel_name'), or your own event emitter

// Instantiate widgets
for(widget in Widgets.available){
  new widget( server );
}

And this is a simplified widget that listens to 'closed_order' events and plays a sound using HTML5's audio tag:


Widgets.available.soundAlert = function( server ){
  
  // add audio files to document
  var $audio = $('<audio src="/audio/alert.mp3" preload="true" />').appendTo('body');

  function play(data){
    $audio.load().play();
  }
  
  // bind to events you want to alert
  server.bind( 'order_closed', play )
}

I can add as many widgets as I want (and even remove them, pause them and inspect them in run-time), with no dependencies between them. Because they bind themselves to events they're interested in, each one is completely orthogonal to the system.

Finally, this is an example of the MockServer object I use to replace Pusher for development purposes:


/* Periodically send out dummy events 
--------------------------------------------*/
var MockServer = function(){

  var self = this, interval, event_names = ['order_closed', 'order_cancelled', 'order_shipped'];

  function randomEvent(){
    var event_name = event_names[Math.round(Math.random(2)*2)];
    
    self.trigger(event_name, {
      info: 'Mock order, 3 products',
      total: Math.random(1000)
    });
  }

  interval = setInterval(randomEvent, 2000);
};
// Extend event-binding interface
MockServer.prototype = new AbstractEventsDispatcher;

Wrap up

This illustrates the general architecture that permeates all recent Javascript projects that we've developed. In future articles we'll talk about techniques and specific problems in more detail.

Even though I've included an abstract class that I used for the examples on this page, the emphasis is on the patterns more than on particular implementations and APIs.

Living on the edge of the WebSocket protocol

makoto@new-bamboo.co.uk (makoto) — Tue, 15 Jun 2010 00:00:00 +0000

Introduction
Getting Info
So What’s Changed?
What about version incompatibility?
Summary

Introduction

Hello. It's been a while since I last blogged about WebSocket. It's been exciting that many new frameworks and services are out including our own Pusher.

Having said that, we have to be aware that the protocol is still actively being changed.

Just a few weeks ago, there was an important announcement at Chromium blog.

Developers should be aware that starting from WebKit nightly build r59903 and Chrome 6.0.414.0 (r47952), the client will talk to a server using -76 version of the protocol, so it will fail to open WebSocket connections with a WebSocket server based on draft-hixie-thewebsocketprotocol-75. Since -75 version of the protocol is obsoleted and no longer supported in future version of browsers, to support new clients you need to update the server implementation. (Note that Chrome 5 uses -75 version of protocol).

And I find this is a very bold statement.

The WebSocket protocol is still actively being changed. Until there is more consensus, we will continue to update our implementation to follow the latest draft of specification, rather than worrying about breaking changes.

This means that you (or the framework maintainer) must be aware of these changes and must act quickly to keep up to date.

Luckily, there are already people who analysed these new changes and blogged about it.

Martyn and I spent last Friday applying these changes to em-websocket (which we use for Pusher) and they're now available as of version 0.1.1. We're already seeing ~ 2% of clients connecting to pusherapp using the new protocol.

I'd like to share our findings and hope that other framework/library authors will do the same. The last thing I want to hear is "WebSocket keeps changing and most frameworks/libraries/services are broken". That will not increase the general adoption of WebSocket.

I also hope that this is technically interesting enough for people who want to know under the hood of WebSocket implementation, but not necessarily write a framework. (NOTE: If you are interested in just using WebSocket in various languages, then you can use Pusher via various language bindings)

Getting Info

First things first, you need to know about the fact that the change did happen, so here are a little bit of tips.

Follow Twitter

If you really want to stay ahead of the game, you should either follow @chromiumbuild or search "websocket" twitter keyword. There are lots of noises, but you know about the change very soon. Twitter was buzzing a week before the update was announced on the chromium blog.

Use Chrome dev channel

Instead of using normal chrome, you can use the dev channel version.

There is Stable/Beta/Dev channel and they get updated according to the schedule.

Dev channel is updated every week and you are automatically updated as soon as you restart your browser, so you need to be aware of the consequences.

You can also use Webkit Nightly build, but not sure where I can find the schedule of Safari update. Does anyone have an idea?

Reading the draft

Reading the entire draft (over 50 pages) is a pain. However, you can read just section 1 and 5 (18 pages together) to understand the change on the server side.

Referring to other implementations.

WebSocket implementation in Google Go and Pywebsocket are implemented by a Google engineer (Fumitoshi Ukai).

I highly recommend reading pywwebsocket source codes. It is an apache module, but standalone.py can run on its own without configuring apache.

When we were writing draft76 patch for Ruby server (em-websocket), we actually used "echo_client.py" to hit the Ruby server for testing. This is much easier than checking via 2 browsers. pywebsocket is actually used to test Chromium browser implementation, so it will be highly likely kept up to date.

[example]$ ./echo_client.py -p 8080 --draft75
Send: Hello
Recv: Hello
Send: 日本
Recv: 日本

[example]$ ./echo_client.py -p 8080
Send: Hello
Recv: Hello
Send: 日本
Recv: 日本

The detailed tutorial of how to install pywebsocket is here.

Please note that you actually need to build and install it even when you use standalone.py. If you don't do install, you will get the following error message.

[src]$ echo $PYTHONPATH 
/Users/makoto/src/pywebsocket-read-only/src 
[mod_pywebsocket]$ sudo python ./standalone.py -p 9999 
Traceback (most recent call last): 
  File "./standalone.py", line 474, in <module> 
    _main() 
  File "./standalone.py", line 361, in _main 
    default=handshake.DEFAULT_WEB_SOCKET_PORT, 
AttributeError: 'module' object has no attribute 
'DEFAULT_WEB_SOCKET_PORT'

If you understand Japanese

html5-developers-jp google group is very active and looks Google engineers are also following. I posted some questions prior to start implementing the upgrade support, and got lots of useful feedbacks (Yes, you can also show off your google translation skills by posting in Japanese, but don't forget that they do some funny translations).

So What's Changed?

This section goes into much more technical detail so may wish to skip it.

The initial upgrade request was changed to include some extra handshake protocol, to make sure that it will fail fast if WebSocket request is received by non WebSocket server (especially normal HTTP server, unless the HTTP server opt in to handle this handshake protocol explicitly).

It also added better way to handle closing phase which I will explain later.

Opening Handshake

From Client to Server

Old

GET /demo HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: example.com
Origin: http://example.com
WebSocket-Protocol: sample

New

GET /demo HTTP/1.1
Host: example.com
Connection: Upgrade
Sec-WebSocket-Key2: 12998 5 Y3 1  .P00
Sec-WebSocket-Protocol: sample
Upgrade: WebSocket
Sec-WebSocket-Key1: 4 @1  46546xW%0l 1 5
Origin: http://example.com

^n:ds[4U

Here are the list of changes.

Name("Web Socket" => "WebSocket")
"Sec-" prefix
New fields (Sec-WebSocket-Key1/2)
Some random characters at the body of request/response (^n:ds[4U and 8jKS'y:G*Co,Wxa-)

#2 Sec- prefix, the explanation is described at Security model

It is similarly intended to fail to establish a connection when data from other protocols, especially HTTP, is sent to a WebSocket server, for example as might happen if an HTML |form| were submitted to a WebSocket server. This is primarily achieved by requiring that the server prove that it read the handshake, which it can only do if the handshake contains the appropriate parts which themselves can only be sent by a WebSocket handshake; in particular, fields starting with |Sec-| cannot be set by an attacker from a Web browser, even when using |XMLHttpRequest|.

NOTE: I am not sure why Sec- cannot be set from a Web browser (unless Sec- is a reserved word for HTTP protocol). If you know any info, please comment.

#3 and #4 are used for opening handshake and I will explain in next section.

From Server to Client

Old

HTTP/1.1 101 Web Socket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
WebSocket-Origin: http://example.com
WebSocket-Location: ws://example.com/demo
WebSocket-Protocol: sample

New

HTTP/1.1 101 WebSocket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
Sec-WebSocket-Origin: http://example.com
Sec-WebSocket-Location: ws://example.com/demo
Sec-WebSocket-Protocol: sample

8jKS'y:G*Co,Wxa-

The opening handshake protocol at server side is described in 13 steps (The client side handshake protocol has 45 steps!!, and this is another reason I use pywebsocket for client testing rather than creating my own client).

The description steps are understandable, but quite hard to read, so it's better to read the source code of other implementations. I refer to the pywebsocket version and php version described by @WebReflection.

Here are the steps.

Extract numbers at Key1(eg: 4 @1 46546xW%0l 1 5) and concatenate them
Count number of spaces at Key1
Devide #1 by #2
Change the format of #3 into "big-endian 32 bit integer"
Repeat #1 by #4 for Key2(eg: 12998 5 Y3 1 .P00)
Concatenate #4, #5, and the body(eg: ^n:ds[4U) of the request
Digest the result in MD5 format

Did you get it? Probably better to read the code. Here is the Ruby implementation.

def solve_challange(first, second, third)
  # Refer to 5.2 4-9 of the draft 76
  sum = (extract_nums(first) / count_spaces(first)).to_a.pack("N*") +
        (extract_nums(second) / count_spaces(second)).to_a.pack("N*") +
        third
  Digest::MD5.digest(sum)
end

def extract_nums(string)
  string.scan(/[0-9]/).join.to_i
end

def count_spaces(string)
  string.scan(/ /).size        
end

Closing handshake

In earlier drafts, you could terminate connection, but there was no proper protocol to acknowledge the closing phase between client and server. This lack of "Orderly close" was criticised here.

To solve this problem, Closing handshake is specified so that both client and server can send a 0xFF frame with length 0x00 to begin the closing handshake.

What about version incompatibility?

The biggest disruption of the new draft is incompatibility with the prior version. Unfortunately, the draft version number is not specified at the HTTP request header (and probably won't be included until the draft get stabilised). For now it's easy enough to get by. You either

A: Try with draft76, catch exception, and fallback with draft75
B: Identify the version by checking newly added fields.

Pywebsocket uses A

class Handshaker(object):
    """This class performs Web Socket handshake."""

    def __init__(self, request, dispatcher, allowDraft75=False, strict=False):
        """Construct an instance.

        Handshaker will add attributes such as ws_resource in performing
        handshake.
        """

        if allowDraft75:
            self._fallbackHandshaker = draft75.Handshaker(
                request, dispatcher, strict)

    def do_handshake(self):
        try:
            self._handshaker.do_handshake()
        except HandshakeError, e:
            if self._fallbackHandshaker:
                self._logger.warning('fallback to old protocol')
                self._fallbackHandshaker.do_handshake()
                return
            raise e

We (em-websocket) use B

  version = request['Sec-WebSocket-Key1'] ? 76 : 75

  case version
  when 75
    Handler75.new(request, response, debug)
  when 76
    Handler76.new(request, response, debug)
  else
    raise "Must not happen"
  end

Summary

The updated WebSocket protocol changes the opening and closing handshakes. For now we can support draft 75 and 76, though the implementation is a bit hacky. Our changes are tested against the pywebsocket script (which is used by both Chromium developers to test their own implementations), and against Chrome and Webkit development builds, but please let us know if we are missing anything.

Bamboo Blog

Experiments with the HTML5 Audio Data API

Audio Data API

Browser specifics

Implementation Details

Loaded Meta Data

MozAudioAvailable

Fast Fourier Transform

Visualisation

Streaming Audio

Conclusion

Translation of Matz Q&A article after joining Heroku

Title: "I am looking forward to accelerating Ruby's progress" - Q&A with Matz regarding joining Heroku as a chief architect.

Introduction

My mission is to make Ruby core more functional and higher quality.

PublicKey (Q going forward): Can you explain to me how this came about?

Q: Do you have any expectation about your role at Heroku?

Q: Do you have any vision of Ruby and Cloud at this moment?

Q: Are you going to work in Silicon Valley, or join from Matsue remotely?

Q: Any other comment?

Degradable JavaScript Applications Using HTML5 pushState

What is the problem?

What is pushState?

pushState & replaceState

onpopstate

pushState vs location.hash

Davis.js vs Sammy.js

Reuseable templates for dryer code

Conclusion

Learning Japanese the Rubyist way

Introduction

Step 1: How to read Japanese characters

Step 2: Japanese and OO

Step 3: Japanese and functional

Step 4: Writing Japanese programming language in Ruby

Summary

Ruby Advent Calendar

Ruby divide at RubyKaigi 2010 and what can you do as a Rubyist? (Part 2)

Contents

At previous post.

"RubyKaigi must end (at least once)"

So, what can we do?

Read "What can you do as a Rubyist?"

Let's make Christmas Ruby

More sister user groups

Let's go to RubyKaigi 2011

Summary

Ruby divide at RubyKaigi 2010 and what can you do as a Rubyist? (Part 1)

Ruby divide at RubyKaigi 2010 and what can you do as a Rubyist? (Part 1)

Introduction

Who am I ?

Japanese / English

Committers / Users

Before / After Rails

Hobby / Enterprise

Summary

Adding awesome mini views to your web apps

What are awesome bars?

What is so great about them?

Types of interaction

Building an awesome bar for a blog

Some canvas basics

Layout

Encapsulating the minibar code

Adopting a simpl[istic] scale factor

Drawing some stuff

Adding a viewer portal

Taking the example forward

HTML5-powered Ajax file uploads

Introduction

Compatibility and detection

Simplest Ajax upload

Feedback events

Reading the raw data

Being a good netizen

Firefox!

Set events BEFORE opening the XmlHttp connection

On Firefox 3.5, two names change

A complete example

Custom event emitters in Javascript