Features

Code warrior provide some basic algorithm questions like quicksort, tree traversal, with some more advence questions too. The questions is all open source, so it can accept new algorithm questions on community.

Code-Warrior questions

Code warrior also provide a web interface for people to check their status, get score by solving questions and share the code by Github:gist.

How to use?

First, you can install the code-warrior cli by npm:

npm install -g code-warrior

After that, you can create a directory for practice, init project on that directory:

war init

Code warrior will require your github username and password for authenticate.
Your local directory should be like this after init:

arena/
history/
node_modules/
package.json
.config.json

You can check the questions by command:

war list

And download question to the arena folder by

war -l [level] -s [id]

Or ignore the id, Code-Warrior will return first unanswered question.

Each question include a readme file, a test case and file to implement.

Implement the question in index.js, and pass the test cases.
you can test question by

war

or just use mocha

mocha arena

Then, commit the question on Code-Warrior:

war commit

If the test cases passed on both local and server, you can gain score according to the level of question. Check your status on site by:

war status

Or login with Github on Code-Warrior site.

More

Right now it only provide javascript for writing the question. But I will try to add ruby and python as language options.

Also, another secret command war legend, which can download custom question for interviewing people, is working in progress.

You can contact me by daizenga [at] gmail.com if you have any suggestion on it.

Event stream

First, what is event stream? Actually it is nothing special, it is just an event listener that listen to specific event.

For example,

$("#clickme").on('click', function(event){ alert(event.target)})

Can transfer to event stream by Bacon.js’s asEventStream helper:

clicked = $("#clickme").asEventStream('click')

And add handler to event stream, listen to click event:

clicked.onValue(function(event){ alert(event.target) })

So what’s the different?

Remember what I said in the beginning, event stream is functional. Which means it provide functional interface to manipulate events:

clicked
  .map(function(event) { return event.target })
  .onValue(function(element) { alert(element) })
// will map the event to event.target

clicked.skip(1).take(4).onValue(function(event) { alert(event.target) })
// will only take the 2-5 click event.

clicked
  .filter(function(event) { return event.type == 'click' })
  .onValue(function(event) { alert(event.target)})
// will only take 'click' event on event stream

Merge

A powerful feature of event stream is it can merge with multiple stream. For example, if we want to listen 2 click event with enable and disable state, you can merge the stream:

enable = $('#enable').asEventStream('click').map(true)
disable = $('#disable').asEventStream('click').map(false)
enable.merge(disable).onValue(function(state) { alert(state) })

Property

Moreover, it provide property: an event stream with state. What property different with event stream is it will remember the state of stream, which is the event object or mapped value.

buttonState = enable.merge(disable).toProperty(false)
// with initial state false
buttonState.onValue(function(state) {
  $('#button').toggleClass('enable', state)
})

Also, it provide scan and assign helper to provide advance usage.

Message Queue

We can also use the event stream as message queue.

messageQueue = new Bacon.Bus()
messageQueue.plug(enable.map({type: 'enable'}))
messageQueue.plug(disable.map({type: 'disable'}))
//plug event stream to queue

messageQueue.onValue(function(event){ alert(event.type)})
// listen and alert event state

messageQueue.push({ type: 'disable'})
// push event manually, alert event

The project worzone provide a more detailed implementation example for messageQueue

Ajax

You can use the .ajax() helper to pass stream params to .ajax(params) and listen the promise object as event stream

response = enable.map({url: '/enable', method: 'post' }).ajax()
response.onValue(function(data) {
  alert(data)
})

So what is it trying to solve?

FRP, functional reactive programming on events. Which reduce the duplicated part of event handling, and make code more looks like pure logic and functions.

I recommand to read the example of Bacon.js tutorial, and todomvc example with Bacon.js. It shows how the functional declarative can simplify codes.

Bigpipe is an unique frontend technique used by Facebook to increase their page rendering speed.

When I read the article talk about bigpipe on facebook, I was pretty shocked about how facebook implements those unique ideas to increase their page rendering speed.

Recently I am using node.js for web development, I think the async structure of node is a perfect environment to use this technique, so I wrote a small experiments app with express:

Streaming

The technique behind Bigpipe is actually pretty simple, what Bigpipe do is using http streaming to load the page seperatly. When the page load, Facebook will return basic layout, css and assets manager(bootLoader) to user first. Then other slower content like news feeds, notification will returned later on the same request as Pagelet. A pagelet contain it’s own css, javascripts and contents, after pagelet loaded, it will render itself to page, and resources dependencies is managed by bootLoader so it won’t load duplicated resources.

The adventage of this approach is that the slower part of the request won’t block the whole page rendering. User can get response for the completed part first, than receive the slower part.

experiments on express

In this example, I use express as web framework, async for async rendering and jQuery for render content to layout.

Demo

you can clone the gist and run it on local:

git clone https://gist.github.com/4589002.git
cd 4589002
npm install
npm start

The Obstacle of IO

The reason is on this is because the I/O is expensive:

• L1/cache: 3 cycles
• L2/cache: 14 cycles
• RAM: 250 cycles
• Disk: 41,000,000 cycles
• Network: 240,000,000 cycles

In the case like web application, every request is not computing heavy but require lots of access on database and disk. So the most time-costing task in a request is waiting database or disk to response.

To solve this problems, the application servers running multiprocess or multithread to make it able to handle more request at same time. however, the new process solution will cost large among of memory because the each fork will copy the memory data to new process. Thread solution is more kindly on memory but still cost more memory.

What is event loops?

So the event loops become the solution of this.

Asynchronous I/O aka Evented I/O

The problem of I/O obstacle, is because I/O can run concurrently, but your code is not, So while the code is accessing the I/O, the process can only idle and wait for the response:

Sync I/O in node.js:

fs = require('fs')

var data = fs.readFileSync('file.txt')
// wait for IO to return content
console.log(data)

But with event loop, the program don’t need to wait for the I/O but can handle next task directly.

fs = require('fs')

fs.readFile('file.txt', function(err, data){
  // Called when data is ready
  console.log(data)
});
// returns

The node.js event loop is using libuv to handle the I/O multiplex in the core function of libuv:

//src/unix/core.c
int uv_run2(uv_loop_t* loop, uv_run_mode mode) {
  int r;

  if (!uv__loop_alive(loop))
    return 0;

  do {
    uv__update_time(loop);
    uv__run_timers(loop);
    uv__run_idle(loop);
    uv__run_prepare(loop);
    uv__run_pending(loop);
    uv__io_poll(loop, (mode & UV_RUN_NOWAIT ? 0 : uv_backend_timeout(loop)));
    uv__run_check(loop);
    uv__run_closing_handles(loop);
    r = uv__loop_alive(loop);
  } while (r && !(mode & (UV_RUN_ONCE | UV_RUN_NOWAIT)));

  return r;
}

We can know the event loop is just a while loop, what it do is keep polling I/O for avalible fd(file descriptor) and trigger event callback while the fd is ready.

in unix, there’s multiple way to polling fd:

• select
• poll
• epoll (linux)
• kqueue (BSD, unix, osx)

node.js is using kqueue on mac os/unix.

Reactor pattern

In javascript, function can be passed as first class object, so the code on node.js is heavily using callback and event to make efficient async I/O.

example a db query:

db.posts.find(function(err, posts){
  console.log(posts);
});

could pass a function as callback to event loop, trigger it when the data is available. and process can handle other tasks during response.

Also, the process.nextTick can also pass function to event loop, execute the function when the process is avaliable for task.

//Blocked:

// some time consuming task
for(var i=0;i < 10000000000; i++){}
console.log('done');

console.log('return')
//>> done
//>> return

-

//Async:

process.nextTick(function(){
  // some time consuming task
  for(var i=0;i < 10000000000; i++){}
  console.log('done');
});

console.log('return')
//>> return
//>> done

reference

• The truth about event loops online masterclass
• Understanding process.nextTick()
• Understanding the node.js event loop
• Integration of GLib Main Event Loop and Node.js (chinese)

In the last day of 2012, I think it is a good time to review what I have done in this year, what I can improve and my expectation for the incoming year.

Work

Ruby, python and coffeescript

In 2012, I mainly work on three different languages: I was working on Rails from last year’s project, and join spling.com afterward. Start to accomondate with django and celery. And write some side project with coffeescript in my offtime. Derive from those experience of side projects. Now I am mainly working on node.js. The build-in event loop of node.js is really powerful, not only make your code can work in real time application but also make it more efficient to handle requests. And the community is very active too. Lots of packages and new thoughts are invented (or reinvented) in node. Although javascript(and coffeescript) is not as clean and pleasure to work with as Ruby on Rails. But so far I feel pretty comfortable on this platform, I will continue to write code on node.js in 2013.

Source code reading

Another big improvement this year is I started to read more source code in open source projects. Start with Sinatra, I took the class of reading source code of Rails, and read through express, railsway.js and Tower.js, some client side framework like ember.js and angular.js. Other than web frameworks, I randomly read the source of some interesting projects too. Although I don’t feel my coding speed is faster, but I learned something from design and code details in those projects. Also it largely improved my code reading speed and ability to work on existing project.

Messaging queue

Working on celery and node.js makes me learned more how to work async, in spling.com, we process the link we a sequence of small tasks, like web parse and api calls. How to run those tasks in parallel and aggregate results make me easier to work on node.js.

Javascript frameworks

In the frontend part, I worked on Backbone.js in spling. But I also survey and read sources of other front-end frameworks and examples of Todos mvc. like ember.js, angular.js and components. Ember.js is a powerful framework, but too many magics behind and learning curve is pretty steep. For the need of javascript MVC, I still prefer Backbone and I would like to try more things on components. I think it is a powerful concept, and I am going to build a new project base on it.

Projects

In last year, I was planning to more involved in open source development. But in the end didn’t contribute any valuable code to big projects :( However, I still fork and patch some projects for my own requirement and create some projects on github. included:

• Papercut : node.js carriar wave clone.
• n37.co : express url shortener.
• Resume : build my resume to html, pdf and host on github pages.
• Bands-in-town-watcher : Running cron job to check changes on bands schedule.
• mail-parser : ruby dsl parser for parsing email.
• Chatty(incomplete) : socket.io chat room lib.

Also, I initiate a few private projects too:

• tilio : node pinterest clone.
• masxaro : Rails application that parse receipts email and display what user have bought.
• getflow : My attempt for Pomodoro, but I am still fighting to produce more quality concentration time.
• playdiary.tw : A wordpress blog that aggregate my friend’s travel and food articles, the purpose of this blog is to learn facebook fan page and ad. Right now I stop posting new article on it, Because posting other people’s article is not that fun. However I am still planing to rebuild this blog with static content generator and my own articles.

Other than those, thanks @amonter5 teach me the basic of iOS development.

Overall, although I created numbers of projects, those project’s quality is not good enough. Need to improve completeness, have more documents for projects in 2013. Also trying to maintain an open source package may be a good way to learn how to create popular projects.

Life

Other than code, I enjoy a lot of the New York life. Thanks my bro, Alan, who take me to all those great trips and concerts.

Travels

• Taiwan
  Back to the home after 2 years. Meet friends and reunion with family.

• Gwan
  For my brother’s wedding, it’s an unforgetable one. Well arranged, the church there is really beautiful.

• Japan
  For snowboard and meet with my friend. Great trip, hot spring after snowboard is the best. Stay with one of my best friend in highschool in sendai and watch K-ON movie is very…Otakuness.

• Vegas and Grand Canyon
  With my parents, treval with parents is one of my most meaningful memory. Vegas and Grand Canyon is great, hope we can have another trip soon.

• Miami
  Jet Pack rules, the coolest water activity I ever have. another great trip include nude swiming in keywest beach.

• Peru
  Treveled around Cusco, walked through Inca trail in a 4 days 3 nights hike and visited Maccu Picchu. Than went to Puno and lake Titicaca, Back to Lima and watch Nasca line and sandboarding in dessert. Fly to Amazon Jungle and live in Jungle launge for Safari. Try the best 2 restaurants in Peru. To much memory there. It’s the best trip ever.

Experence

• Gun shooting
  
• Jet Pack
• Wakeboard
  
• Rafting
• Kayaking to Status of Liberty
• Bartender bootcamp

Concerts and games

• JayZ
• Orion festivel, Metalica

• Slipknot
• Linkin Park
• Off Spring
• RHCP
• Knicks vs Rockets
• L’arc En Ciel
• Live-In Theater Interactive Murder Mystery
• Sleep No More
• Spiderman - Turn off the dark

Plan for 2013

Code

More coding!
Seriously, I need to concentrate and push myself more on coding. One of my problem is I can’t keep my productivity in high gears, and spend too much time on reading fourm and online comics. Also need to code more efficiently with tools and learn from great programmers.

Project

Currently trying to build a components based static html engine: Meteora, with components, theme and widget can be managed by dependencies and can be reused easily. Main target is to complete this and use it to rebuild my blog, playdiary.tw and my brother’s blog.

One blog every week

Also, improve my blogging and English writing. Have to keep the blog speed to 1~2 hour and keep writing. However, a high quality blog like Source code odyssey series will take about 3~4 days to finish. So how to keep quality and speed might be a problem but, I have to keep the tempo of writing blog.

some blog ideas for 2013:

• Event loop
• Hadoop
• Build your Iaas : AppHouse
• Meteor
• Jquery deffered
• Advence Git
• Chef
• Source code odyssey : Railsway.js
• Source code odyssey : Pow
• Source code odyssey : Hubot
• Source code odyssey : Redis

Reading

Read more non-fiction and technical books:

• Machine Learning for Hackers
• Enterprise application patterns
• Continuously delivery
• Javascript Ninja
• Big data
• Rails essential patterns
• Create your own programming language

Novel

Finish a novel is one of my dreams, and should be a goal in 2013. Some drafts in my dropbox need to be completed.

Travel

Probally not as much as 2012, but I want to visit Europe and Japan again if I have chance in 2013. and meet my friend in San Francisco and developers in Silicon Valley. Also a weekend trip to Chicago will be great too.

And…

I don’t know this project will works or not, but I will try my best on it. Wait and see in 2013…

You can check this project on Github

During the catastrophe of Sandy storm, my place have no electricity and water, so I stay in my friends place for whole week. And I need to find something to do except eating, surfing internet and boardgame. So I just wrote another test framework.

Inspired by Zach Holoman’s gist I try to host the framework in the gist form. Because every gist is also a git repository.

gist source

Spec

The README.md file is also a runnable spec in coffeescript. run

npm test

in project can execute the spec

Global delegation

One of the problem is the format of syntax. Because I want to write syntax without extra this keyword, so instead of calling the test block with ‘call’, I have to declare test syntax globally.

#Instead this
describe = (title, block)->
  @it = (desc, fn)->
    #add test to test suite
  block.call(@)

describe "syntax", ->
  @it "have this keyword", ->

#Got to do it Globally
global.it = (desc, fn)->
  #add test to test suite
describe = (title, block)->
  block()

describe "syntax", ->
  it "have no this keyword", ->

But there’s 2 problem of declaring global, one is the confrontation of global keywords, which is a tradeoff for simpler syntax. Another problem is when implementing nested block, test need to be delegated into right suite. Using a stack to track the current suite can solve this problem.

suites = []
global.it = (desc, fn)-> 
  suites[0].tests.push title: desc, fn: fn

describe = (title, block)->
  suite = new Suite(title)
  suites.unshift(suite)
  block()
  suites.shift()

So the global it will always add test to the top of stack, which is the current test suite.

Hook and run

For the before/after function, I use the EventEmitter to trigger the hooks, and delegate event to the parent suite:

class Suite extends EventEmitter
  constructor: (@title, @parent)->
    ...
    @delegate ['before', 'after']
    @on 'result', @reportResult
    @on 'end', @reportSummary

  delegate: (events)->
    events.forEach (event)=>
      @on event, => @parent?.emit event

  run: ->
    @emit 'start'
    @tests.forEach (test)->
      try
        @emit 'before'
        test.fn()
        @emit 'result', test
        @emit 'after'
      catch err
        @emit 'result', test, err
    @emit 'end'

Async

For testing the async callback, it also provide a done callback for the async function. And I use async module to handle the test execution sequence. It will detect the function param, if there is a specific callback param in function, execute in async mode.

Source

Summary

Credit to VisionMedia’s mocha framework, lots of solution is comming from there.

But still this is an intersting small project to do in the storm days.

And I am so happy to back to the normal life.

A startup is a human institution designed to deliver a new product or service under conditions of extreme uncertainty.

Recently I finally finish reading the lean startup book.

The author Eric Ries is writing blog for sharing his startup experience for several years. This book is include his experience of his own startup and tutoring experience. He points out lots of the critical problems that a startup will face. Include finding market fit, testing hypothesis and validate growth.

As an employee of a startup, It is a really interesting book.

Problems

The whole book is contentrate on a problem that lots of startup will face:

Imagine your team work on the product really hard, but in the end nobody is using that product. Or, When the team finish up a feature. But you have no idea if it will make any change to your low customer conversion rate. That is the most depressing thing a startup can have.

Those waste of productivity will draining team’s energy, depress the passion and most important, burning money.

Eric Ries, as an entrepreneur and consultant of startup. Knowing this problem and lean startup is the book focusing on how to use scientific management skills and matric to valuate decision to make team be able to learning from those waste.

Testing

In the startup, usually the whole business idea is an hypothesis without prove. So we have to use all kinds of test to validate our hypothesis is correct or not. Include talking to customer, film the conceptual film, preview with minimal viable product, etc. After the product is out, we need to know the growth engine of product and collect the data to validate the direction is correct or not.

In the story of great company, usually there is a leader with great vision who will lead the team to create outstanding product. But Eric point out the truth that, most of the team fails on those vision too. For a well-grounded team, we need to validate the data, continous fixing direction and provide features that customer really want. That require not intuitive, but the decision based on data. Continously testing every business assumption to increase the matrix.

Pivot

Another concept that Eric avoke is Pivot, which is when the testing failed to prove the business hypothesis is correct, or the growth rate is not as expected. The company need to commit the failure and make change of direction.

But when to pivot is a critical problem, in book Eric said they will have a Pivot or preserve meeting everytime they have this issue. Basically I think Pivot is a flexible way to find the market fit. But not nessassary can help company to find market. In my opinion that Pivot should be pulled out from a failed hypothesis, and should follow the passion of whole team.

Engineering

In engineering side, a team should focus on create product with minimal feature, and continous deployment with small batch and testing continously too. Other than that this book doesn’t focus on technical too much. But follow the best practice, doing testing and continous integration. Code review can increase the maintainability of product. Also, try not to reuse all codes when Pivot, The old code will only drag the productivity but not fit the new requirement.

Another problem in startup is the cross functional team. In a big company, there will be developer, product manager, designer and system operator. But in startup, every person need to work cross functional to improve the development process. A developer need to be able to open new machine and deploy, fixing user interface, adding new features and new tests. Same as designer, operator and product manager. Team need to lower the obstacle stop the cross functional work.

Team

How to build the team for lean startup? In lean startup it focus on educate the team to fit the requirement. Its true that it is hard to hire the ‘Ninja’ or ‘Rockstar’ in startup. Find people smart and get things done. Teach them to fit the team is much easier.

Culture

Lean startup doesn’t mention this too much, but I think the culture is an important part of a company. A culture is defined by how the founder and team work, It can be a small work habit or development methology. So a startup need to caucious what culture they want to build, and work toward that way. Like the hacker culture in Facebook, Startup culture in Apple, 20% time culture in Google, etc.

Conclusion

This book is based on author’s experience and is really informative. Although all the method aren’t nessassary fit into all startup. But some of the concept like the data based decision, small batch and all the concept can boarden the entrepreneur’s view The startup isn’t nessassary to be a chaos of intuition and vision. It is a managable and repeatable process of new company.

When I first saw this declaration, I just feel that this might be extract from rails and herokus best practices, but actually this declaration is more then practices but a methodology to improve the maintainbility and scalability of web application. so I like to share Heroku engineer’s slide about 12 factors app:

The Twelve-Factor App

I. Codebase

One codebase tracked in revision control, many deploys

II. Dependencies

Explicitly declare and isolate dependencies

III. Config

Store config in the environment

IV. Backing Services

Treat backing services as attacked resources

V. Build, release, run

Strictly separate build and run stages

VI. Processes

Execute the app as one or more stateless processes

VII. Port binding

Export services via port binding

Concurrency

Scale out via the process model

IX. Disposability

Maximize robustness with fast startup and graceful shutdown

X. Dev/prod parity

Keep development, staging, and production as similar as possible

XI. Logs

Treat logs as event streams

XII. Admin processes

Run admin/management tasks as one-off processes

Binding

One common problem of javascript application is how to manipulate DOM element and insert data into DOM, for example, a jQuery app retrive user data and show on screen will look like this:

1
2
3

$.get('/user', function(data){
  $('#username').text(data['name']);
});

When the application growth, the code of updating data onto dom will become huge repeating logic, for solving this problem, Ember.js use the handlebar template and binding, for example, a handlebar template will look like this:

1
2
3

<script type="text/x-handlebars">
  Hello, { { App.user.name }}
</script>

And binding with a Ember.js Object:

1
2
3
4
5
6
7
8
9
10
11
12

App.user = Ember.Object.create({
  name: null

  getName: function(){
    var self = this;
    $.get('/user', function(data){
      self.set('name', data['name']);
    });
  }
});

App.user.getName(1);

Behind the scene, handlebar will compile template with script tags:

Hello <script id="metamorph-0-start" type="text/x-placeholder"></script>
Jimmy
<script id="metamorph-0-end" type="text/x-placeholder"></script>

And handlebar will auto update the value between tags. So the data and dom will always be synced. This save lots of work for updating data.

Also, Ember can calculate the compounded values and update it automatically, for example, we can change the name attribute that combine the firstName and LastName value together:

1
2
3
4
5
6
7

App.user = Ember.Object.create({
  firstName: "Jimmy",
  lastName: "Chao",
  name: function(){
    return this.get('firstName') + this.get('lastName');
  }.property('firstName', 'lastName')
});

The template will automatically updated when firstName and lastName value changed. We can also bind other object with property binding:

1
2
3
4
5

App.profile = Ember.Object.create({
  usernameBinding: 'App.user.name'
});

App.profile.get('username'); // "Jimmy Chao"

The attribute and template will also automatic updated when value changed

View

In Ember.js, view is in charge of handling events and present data. We know the data is autobinding on template, but what about events? It turns out Ember.js will populate event all the way to parent view to handle, and event can be bound on template for specific element:

1
2
3
4
5
6
7
8
9
10
11
12

App.userView = Ember.View.create({
  templateName: 'user-view',
  name: "Bob",
  click: function(){
    alert('clicked');
  },
  sayHello: function(){
    alert('hello!');
  }
});

App.userView.appendTo(App.rootElement); //append view to the application

And define the event in action helper:

1
2
3
4

<script type="text/x-handlebars" data-template-name='user-view'>
  Hello,  <br />
  <a href="#" >Say Hello</a>
</script>

Then when user click link, it will trigger both ‘click’ and ‘sayHello’ event. The view template and view is connected together and will always be sync, so we don’t have to rerender or trigger change event when value changed. this is the different of Ember.js view and other view templates. But the cons is you can only use handlebar.js to have autoupdate template.

Combounded View

Another problem for javascript application is the view structure. When application grow, it will generate main view and sub views under the main. For solving this problem, Ember.js provide ContainerView that can define childView in it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

App.mainView = Ember.ContainerView.create({
  childViews: ['navigationView', 'mainView','footerView'],
  navigationView: App.View.create({
    templateName: "nav-view"
  }),
  outletView: App.View.create({
    template: Ember.Handlebars.compile( '{ {outlet}}' )
  }),
  footerView: App.View.create({
    templateName: "footer-view"
  })
});

App.MainView.appendTo(App.rootElement);

With this ContainerView, we can define any numbers of sub view and append them to application.

And for displaying multiple elements, Ember.js also provide CollectionView, which can render view for each items in content:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

var HelloView = Ember.View.extend({
  template: Ember.Handlebars.compile('Hello { {view.content}}')
});

App.usersView = Ember.CollectionView.create({
  content: ["Jimmy", "Bob", "Jack"],
  itemViewClass: HelloView
});

App.usersView.appendTo(App.rootElement);

//Append:
//<div class='ember-view'> Hello Jimmy</div>
//<div class='ember-view'> Hello Bob</div>
//<div class='ember-view'> Hello Jack</div>

This will display 3 hello view and append to rootElement, the content will also automatic binding with views.

Objects

Ember.js provide a series of extend methods for Object, all Ember.js Object is inherited those methods, include Event, Observer, toString ,getter, setter and more, most common use case is Ember.Object.create() and Ember.Object.extend(). extend is defining new Object Class and create is create a instance of class. Also, we have some convenience method. Like reopen and reopenClass can do the monkey patch, which is directlly modify the defined object class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

var User = Ember.Object.extend({
  firstName: null,
  lastName: null
});

User.reopen({
  name: function(){
    return this.firstName + " " + this.lastName;
  }.property('firstName','lastName')
});

var user = User.create({ firstName: "John", lastName: "Doe"});

console.log(user.get('name')); //John Doe

Another is Mixin, which is like module in ruby, abstract class and interface in java. Can extend parameter and function to class, which make the object cleaner.

1
2
3
4
5
6
7
8
9
10

var Admin = Ember.Mixin.create({
  isAdmin : true
});

var User = Ember.Object.extend(Admin,{
  name: null
});

var user = User.create();
console.log(user.isAdmin);  // true

Application

Here is what magic happened, and is the most confusing part of Ember.js when I started learning. Because the documentation and Guide doesn’t mention lots of things on controller and router. But the code document is great. Strongly recommend to read the source code of Ember.js.

Application, inherit Ember.Namespace, is the starting point of every Ember.js application.

Each application start with a creation of namespace:

window.App = Ember.Application.create();

All classes should be attached to the application namespace and application should be the only global varible.

App.UserView = Ember.View.extend();
App.UserController = Ember.Controller.extend()

After classes loaded. We can call App.initilize() to start the application. What initilize do here is, first, it bind the application with rootElement(default to body). Also enable the event deligation, application object will also be the event hub of all other classes. And the important one is, it will load the ApplicationController, App.ApplicationView and App.Router. Create and attach all controller instance to router. For example, App.UserController will have an instance App.router.userController after application initialized. Also, router will trigger state and start the corresponding action after intialized.

Controller

In Ember.js, controller is the place to put model manipulation codes and binding model with views. We can call the ‘connectOutlet’ function to application controller and pass the name and content to invoke corresponding controllers and views to { {outlet}} in ApplicationView. Like Rails, Ember.js have naming convention on Controllers and Views, the UsersController and UsersView will automatically bind together when calling connectOutlet. Controller will be avaliable in view by property ‘controller’.

1
2
3
4

users = ['Jimmy', 'Bob', 'John'];
App.router.applicationController.connectOutlet('users', users);
//invote the UsersController and UsersView instance with content users.
//Render UsersView to the { {outlet}} position of ApplicationView template.

Router

Router is inherited from Ember.StateMachine that represent each route as a state of Ember application. Each route will have route url and connectOutlets function. connectOutlets will be called with router instance, Which we can access all controller instance form it, and call the connectOutlet method to set the view and controller to mainView.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

App.Router = Ember.Router({
  root: Ember.Route.extend({
    index: Ember.Route.extend({
      route: '/',
      connectOutlets: function(router){
        //get the application controller
        var controller = router.get('applicationController');
        //get the model, using fixed data here
        var users = ['Jimmy', 'Bob', 'John'];
        //connect the UsersController, UsersView, users to ApplicationView
        controller.connectOutlet('users', users);
      }
    })
  });
});

Each route is also a state of application, so the route can have specific transition in certain state For example, when viewing a single post, user can edit the post. When can represent this use case as:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

App.Router = Ember.Router.extend({
  root: Ember.Route.extend({
    posts: Ember.Route.extend({
      route: '/posts/:id',
      connectOutlets: loadPost,

      showEdit: Ember.Route.transitionTo('edit'),
      edit: Ember.Route.extend({
        route: '/edit'
        connectOutlets: loadPostForEdit
      });
    })
  });
});


//with action helper in template:
//<script type="text/x-handlerbars" data-template-name='post-view'>
// ...
// <a href="#" {{action showEdit }}> Edit</a>
//</script>
//will transit url to /posts/:id/edit

So that only in posts state, user can transit into the edit state.

Todos App example:

For understand the Ember.js Application More, I am starting with the TodoMvc example of Ember.js. Lets go through a view classes to of the todo exmaple to understand more how Ember.js works:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96

//app.js
window.Todos = Ember.Application.create({
  VERSION: '1.0',
  // binding root element with id='todoapp'
  rootElement: '#todoapp',
  // used as localstroage namespace
  storeNamespace: 'todos-emberjs',
  // Extend to inherit outlet support
  ApplicationController: Ember.Controller.extend(),
  ready: function() {
    //initialize router, controller and views
    this.initialize();
  }
});

//router.js
Todos.Router = Ember.Router.extend({
  root: Ember.Route.extend({
    showAll: Ember.Route.transitionTo('index'),

    index: Ember.Route.extend({
      route: '/',
      connectOutlets: function(router){
        var controller = router.get('applicationController');

        // Get the models, in this example is entriesController
        // Which is an instance of ArrayProxy
        var context = Todos.entriesController;
        context.set( 'filterBy', '');

        // Connect todosController and todosView with context to applicationView.
        controller.connectOutlet('todos', context);
      }
    });
  })
});

//controllers/todos.js
Todos.TodosController = Ember.ArrayProxy.extend({
  entries: function(){
    var filter = this.getPath('content.filterBy');

    if(Ember.empty(filter)){
      //content will be injected by router
      return this.get('content');
    }
    ...
  }.property( 'content.remaining', 'content.filterBy')
});

//views/todos.js
Todos.TodosView = Ember.CollectionView.extend({
  //binding content to todosController.entries
  contentBinding: 'controller.entries'
  ...
  //view class for each todos item
  itemViewClass: Ember.View.extend({
    ...
    doubleClick: function() {
      this.get('content').set('editing', true);
    },
    ...
    //innerview for editing item
    ItemEditorView: Ember.TextField.extend({
      valueBinding: 'content.title',
      ...
      change: function(){
        if(Ember.empty(this.getPath('content.title'))){
          //if no value, remove object from todosController.content
          this.getPath('controller.content').removeObject(
            this.get('content');
          );
        } else {
          //if not, update the content
          this.get('content').set('title',
            this.getPath('content.title').trim());
        }
      },
      whenDone: function(){
        this.get('content').set('editing', flase);
      },
      ...
    });
  })
});


//todosView:
//<script id="todosTemplate" type="text/x-handlebars">
//{{#unless view.content.editing}}
//  {{view Ember.Checkbox checkedBinding="view.content.completed" class="toggle"}}
//  <label>{{view.content.title}}</label>
//  <button {{action removeItem target="this"}} class="destroy" ></button>
//{{else}}
//  {{view view.ItemEditorView contentBinding="view.content"}}
//{{/unless}}

You can see http://todomvc.com/architecture-examples/emberjs/ for demostration.

Conclusion

After reading this article, I hope you can have a common understanding of Ember.js and what problem it is trying to solve. Ember.js, unlike Backbone concentrate on simple and minimal code, it is a very opinion framework. Therefore it has steeper learning curve than Backbone. Also the document can’t cover all topics of Ember.js. So read the comment in codes is actually a very useful way to understand how Ember.js work.

Comparing these 2 frameworks. Backbone.js is simple, easy to use and Integrate RESTful Api by model. But when codes grow. You have to be careful about the repeating code and logics. Also might need some plugin like layoutManager to maitain better structure.

On the other hand, Ember is harder to use. But it have better build-in structure for large scale application. Some automatic binding for controllers and views, state machine router, and solve some common problem like syncing and updating. But you can only use handlebar for the template. Also have lesser plugins and resources than Backbone. Such as the module for server data: Ember-data is still in development stage.

Overall, those 2 framework is both ready for production use. At the source code level, I like the structure of Ember.js, It is very moduler and have some suger syntax like mixin and reopen on object level. Also with very good code documentation. I will like to use Ember in my next project to compare with backbone.

Reference:

• Ember.js
• TodoMvc example in ember.js
• What is Ember?
• Ember routing
• parisjs-app
• webapp-codelab

A Url Shortener

A url shortener, as its name, is a application that provide user a shorter version of url. Like the most famous one: tinyurl.com, which map the url address into a short sequence of code, like tinyurl.com/ccxngqo. Make user easier to share the link on social media like twitter or forum.

What the application do is actually pretty simple, they store the url in the table and set a id for it. When user request the id, redirect user to the url. The problem here is: how to make the id shorter? The obvious solution is to use the character as id but not integer id. So most url shortener will use the code like ‘aSgY2’, which combine the upper, lower case letters and numbers as id. With this method, one character can present 26+26+10=62 choice. A 5-letter code can present 62^{5=916,132,832} of urls. far better then just use 5 numbers.

There are two methods to map the long url into short code, one is to convert the increment interger id into code. Which is easier and can use up all of the space. However, the space (of 5-6 digits code) will be running out eventially because there are billions of urls on internet. So it have to increase code length to increase the storage of url. Or it have to expire the unused url and reuse these code. But when it comes to expire, The incremental id will become more complex since it need to track the smaller free id first. Also, the application have to maintain a index of url -> id to increase the search speed, and it will have to maintain index often if the url will expire.

Another solution is to hash the url address, so the url will match to a code generated by url string that same url will always map to same code. This method works well with the expire method because application doesn’t need to track which code is free. However, this method won’t be able to use up all of the available codes and the hash will collision(which is, two different url generate the same code) when numbers of url increase.

Hash

For the project, I choose the hash method to generate code. Because this method works well with redis hash map data type. Lets bring some code in the project:

The hash here use the node.js crypto module to hash the url, and reduce the code by mod the hash result into the range of 5 digits (916,132,832). And map the number into usable symbols as return code.

But, what happen when the code collide? and how often will it happen when the url increase?

Collision

(Warning, this section include some math!)

The hash collision problem, can be seen as a generalize birthday problem. Which is: what’s the probability of two people have same birthday for N people?

Lets solve the birthday problem first:

First, the number of days in a year is 365 (exclude the leap year). So the probability of at lease 2 people have same birthday equal to 1 - probability of all people have unique birthday.

Probability_of_same_birthday(N) = 1 - Probability_of_unique_birthday(N)

Probability_of_unique_birthday(N) = 1 * 364/365 * 363/365 * … * (365 - N + 1)/ 365
                                  = ( 365 * 364 * … (365 - N + 1)) / 365^N
                                  = 365!/ ((365 - N)! * 365^N)
                                  = N! * Combination(365, N) / 365^N

In the hash context, the range of code is 62 ^ 5 = 916,132,832, so the probability of having all unique hash in N url is:

Probability_of_unique_hash(N) = N! * Combination(916132832, N) / 916132832^N

The approximation of Probability is:

P(N) = 1 - e^(-n^2/916132832*2)

So the probability of collision in 916132832 space with n hash P(n) is:

P(10) = 5.458 * 10^-8
P(1000) = 5.456 * 10^-4
P(100000) = 0.995
P(200000) = 1

P(N) = 0.001, N ~= 1355 
P(N) = 0.01, N ~= 4292
P(N) = 0.10, N ~= 13895 
P(N) = 0.50, N ~= 35638

As the birthday problem, the collision chance is actually higher than we thought, the 916132832 space can only hold around 35638 hash without too much collision.

So to solve the collision problem, I choose to use the other hash method when collision happen. The node.js crypto module have md5, sha1, sha256, sha512 can generate different code. So if collision happened, it will switch to another hash method. If all four hash collide, the code will increase one digit to hold the new hash. And start with md5 to generate hash again.

Here’s the code that rehash url:

With this method, I set the expire time as one month, and the application don’t need to track the expiration of url and will automatically reuse the empty spot. There might have duplication of url, but the probabillity is small, since the url used often will always occupy the first or second hash spot.

Conclusion

This project introduce some pro and cons on using increase integer or hash code as id. So we know why the url shortener is implmented in a certain way. I would like to know how the bit.ly and tinyurl.com do the url shortening under the hood. Because the fall back hash method still need some prove to be usable under large amounts of urls.

Express.js is developed by TJ Holowaychuk, who rebuild the web development on Node.js with express, jade, mocha, stylus and more. Express is inspired by the simple of Sinatra provide a simple and elegant interface for http request, also with connect middle support that let user can easily extend the framework.

Usage

Here’s the hello world sample of express. It assign a callback for get request on root ‘/’, and start the server to listen port 3000. More example can be found on express/examples and Documentation of express.

1
2
3
4
5
6
7
8
9
10

var express = require('express');

var app = express.createServer();

app.get('/', function(req, res){
  res.send('Hello World');
});

app.listen(3000);
console.log('Express started on port 3000');

Connect

Express is using the Connect Middleware. But What is connect? From the configuation function of Express:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

app.defaultConfiguration = function(){
  var self = this;

  // default settings
  this.set('env', process.env.NODE_ENV || 'development');
  debug('booting in %s mode', this.get('env'));

  // implicit middleware
  this.use(connect.query());
  this.use(middleware.init(this));

  ...
  // router
  this._router = new Router(this);
  this.routes = this._router.map;
  ...
};

We can see that express call the use method with connect.query(), the query() parse the query string and attach to the request object req.query.

In the source of connect.query():

1
2
3
4
5
6
7
8

module.exports = function query(options){
  return function query(req, res, next){
    req.query = ~req.url.indexOf('?')
      ? qs.parse(parse(req.url).query, options)
      : {};
    next();
  };
};

The next() method invoke the next method on the process chain. We can see that it return a method that parse the req.url with qs.parse function and return to req.query. So we can get the query string from req.query in our application.

So we can see the middleware as a function that execute before the request and add functionality to (req, res) object. So we can easily use a function as middleware. In example, we can refactor the hello world as middleware:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

var express = require('express');

var app = express.createServer();

app.use(function(req, res, next){
  req.message = 'Hello world';
  next();
});

app.get('/', function(req, res){
  res.send(req.message);
});

app.listen(3000);
console.log('Express started on port 3000');

In more advance usage, we can build a flash message using middleware:

1
2
3
4
5

app.use(function(req, res, next){
  res.locals.flash = req.session.flash || "";
  req.session.flash = "";
  next();
});

Also, there is another kind middleware called param callback. which will only be executed when the route have the param key:

1
2
3
4
5
6
7
8
9
10
11
12

app.param('user_id', function(req, res, next, id){
  User.find(id, function(err, user){
    if (err) {
      next(err);
    } else if (user) {
      req.user = user;
      next();
    } else {
      next(new Error('failed to load user'));
    }
  });
});

Will be invoked when route params have ‘user_id’:

1
2
3

app.get('/:user_id', function(req, res){
  send('User name is' + req.user.name );
});

Application

The main application is on lib/express.js. It produce express app by createApplication function. First, the application merge with connect, set request and response object, than call the initialize function of application defined in lib/application.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

var connect = require('connect')
  , proto = require('./application')
  , req = require('./request')
  , res = require('./response')

…
…

function createApplication() {
  var app = connect();
  utils.merge(app, proto);
  app.request = { __proto__: req };
  app.response = { __proto__: res };
  app.init();
  return app;
}

In application.js, the init function initialize the settings and default configuration of express, and also load the Router to route the Http requests.

Also an important part, some of the settings is depend on process.env, like cache. Which can be enabled by exports NODE_ENV=production.

this.set('env', process.env.NODE_ENV || 'development');

Router

Lets step into the router, basically, the router provide route method to assign function to route, and matchRequest to return route.

In route

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

/**
 * Route `method`, `path`, and one or more callbacks.
 *
 * @param {String} method
 * @param {String} path
 * @param {Function} callback...
 * @return {Router} for chaining
 * @api private
 */

Router.prototype.route = function(method, path, callbacks){
  var method = method.toLowerCase()
    , callbacks = utils.flatten([].slice.call(arguments, 2));

  // ensure path was given
  if (!path) throw new Error('Router#' + method + '() requires a path');

  // create the route
  debug('defined %s %s', method, path);
  var route = new Route(method, path, callbacks, {
      sensitive: this.caseSensitive
    , strict: this.strict
  });

  // add it
  (this.map[method] = this.map[method] || []).push(route);
  return this;
};

The route function take the Http method (like ‘get’), path and callback. normalize them and create Route object. add to the map[method] array in Router.

Also, it use the meta programming skill to generate public helper for user, so we can call the app.get or app.post method to assign route.

In application.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

var methods = require('methods')

…

/**
 * Delegate `.VERB(...)` calls to `.route(VERB, ...)`.
 */

methods.forEach(function(method){
  app[method] = function(path){
    if ('get' == method && 1 == arguments.length) return this.set(path);
    var args = [method].concat([].slice.call(arguments));
    if (!this._usedRoutnner) this.use(this.router);
    return this._router.route.apply(this._router, args);
  }
});

But after the route is set, how do express invoke the handle function? We can see the usage from the test case:

1
2
3
4
5
6
7
8
9
10
11
12
13

  describe('.middleware', function(){
    it('should dispatch', function(done){
      router.route('get', '/foo', function(req, res){
        res.send('foo');
      });

      app.use(router.middleware);

      request(app)
      .get('/foo')
      .expect('foo', done);
    })
  })

In the test case we can see the app.use(router.middleware) mount the routes to the application. In router.middleware, it call the _dispatch(req, res, next) function, it call the route.matchRequest method to get the matched route, handle param callback and execute the callback chain:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

Router.prototype._dispatch = function(req, res, next){
  var params = this.params
    , self = this;

  debug('dispatching %s %s (%s)', req.method, req.url, req.originalUrl);

  // route dispatch
  (function pass(i, err){

    ...

    // match next route
    function nextRoute(err) {
      pass(req._route_index + 1, err);
    }

    // match route
    req.route = route = self.matchRequest(req, i);

  …
  
    // we have a route
    // start at param 0
    req.params = route.params;
    keys = route.keys;
    i = 0;

    // param callbacks
    function param(err) {
    … // execute param callbacks and execute route callbacks(err) in the end.

  }

    param(err);

    // invoke route callbacks
    function callbacks(err) {
      var fn = route.callbacks[i++];
      try {
        if ('route' == err) {
          nextRoute();
        } else if (err && fn) {
          if (fn.length < 4) return callbacks(err);
          fn(err, req, res, callbacks);
        } else if (fn) {
          fn(req, res, callbacks);
        } else {
          nextRoute(err);
        }
      } catch (err) {
        callbacks(err);
      }
    }
  })(0);
};

Request / Response

Request and Response object in express is inheritate http.IncomingMessage and http.ServerResponse from node. Request provide helpers for http header, content-type, URL and param parsing.

Response is providing helpers for status and results too. Also provide the send function from ‘response-send’ module. Which detect response type and set corresponding content-type.

View

In the lib/view.js, the main functions is render and lookup, which find the template and call the render function on engine. The render function on app initialize new View, the view set the lookup(name) to path, and call the view.render method to return result:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

View.prototype.lookup = function(path){
  var ext = this.ext;

  // <path>.<engine>
  if (!utils.isAbsolute(path)) path = join(this.root, path);
  if (exists(path)) return path;

  // <path>/index.<engine>
  path = join(dirname(path), basename(path, ext), 'index' + ext);
  if (exists(path)) return path;
};
…

View.prototype.render = function(options, fn){
  this.engine(this.path, options, fn);
};

in the app.render:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

app.render = function(name, options, fn){
  var self = this
    , opts = {}
    , cache = this.cache
    , engines = this.engines
    , view;

  …

  // primed cache
  if (opts.cache) view = cache[name];

  // view
  if (!view) {
    view = new View(name, {
        defaultEngine: this.get('view engine')
      , root: this.get('views') || process.cwd() + '/views'
      , engines: engines
    });

    if (!view.path) {
      var err = new Error('Failed to lookup view "' + name + '"');
      err.view = view;
      return fn(err);
    }

    // prime the cache
    if (opts.cache) cache[name] = view;
  }

  // render
  try {
    view.render(opts, fn);
  } catch (err) {
    fn(err);
  }
};

MVC example

In the examples/mvc directory, it shows you how to structure a mvc style application in express.

1
2
3
4
5
6
7
8
9
10
11

controllers/
  main/
  pet/
  user-pet/
  user/
views/
public/
lib/
  boot.js
db.js
index.js

The followed structure is a sample of mvc in express, the index.js load the boot.js, which mount all the routes under controllers/ in a RESTful style. And set the views path and static path.

There are no restriction on how to build the structure of express application. The Railway is one of the example that it extend express to be a Rails like Mvc application.

Conclusion

Express is a lightweight web framework and it is a good entry point to know how to build a web framework in Node.js. Hope this walkthrough can help you understand how express work. And can be more comfortable with new framework like meteor.

Also it is a well styled and documented project, so when you find some problem in developing express application, don’t be hesitate to use the source code (luke!) and the Guide of Express

Recently, the Go language release it’s version 1.0, which have a more constant api that aiming on attracting enterprise developers to use go as part of their solution.

I read the tutorial when it’s first release. At that time, Go was positioned as a new system language, Like C or C++. But right now it become more powerful that can be use on any general programming topic that require efficiency and concurrency.

As a language, Go has pretty weird syntax such as postfix type decleration and interface oriented. But other then that, Go is more like a compiled version scripting language. Which is pretty easy to write, and focusing on making threading easier through It’s Go routine.

Install go

On OSX you can use homebrew:

brew install go

Concurrent

One main feature of Go is go routine:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

package main

import (
  "fmt"
  "runtime"
)

func HelloGo() {
  fmt.Println("Hello Go!")
}

func main(){
  for i := 0; i < 10; i++ {
      go HelloGo()
  }
}

It’s equivalent to Java:

1
2
3
4
5
6
7
8
9
10
11
12

public class HelloRunnable implements Runnable {

    public void run() {
        System.out.println("Hello Go!");
    }

    public static void main(String args[]) {
      for(int i = 0; i < 10; i++){
          (new Thread(new HelloRunnable())).start();
      }
    }
}

While using go routine, the function will fork to run on different thread. Which won’t block the main process and can be efficient on multicore.

Although there are other framework that can make threading easy too, but Go provide language level support and another useful feature: Channel

In Java, if we want to implement a multithread program, a big problem is how to passing data between Threads. One solution is using shared memory. For example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

public class Counter{
  private int counter = 0;
  //synchronized access for multithreads
  public synchronized void increment() {
        counter++;
    }
    public int getCounter(){
      return counter;
    }

    public static void main(){
      final Counter counter = new Counter();
      for (int i = 0; i < 2; i++){
          new Thread(new Runnable() {
              public void run() {
                      for(int i = 0; i < 100000000; i++)
                          counter.increment();
              }
          }).start();
      }
      
        try{
          //wait for thread finish
          Thread.sleep(10000);
        }catch (InterruptedException e){
          System.out.println(e.getMessage());
        }
        System.out.println(counter.getCounter());
        // 200000000
    }
}

In this example, There are 2 thread both increasing the counter, if without the synchronzed keyword, the counter will not be 200000000 in the end because when thread 1 and 2 accessing counter at same time, result will be overwritten by other thread and only increase by 1. The synronized keyword enable deadlock to prevent this happened. But even with lock protection, the communication between threads is still a problem that reduce performence and make the parallel computation harder. Therefore the Go language make a new approach: channel, to solve this problem:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

package main

import "fmt"

func counter(id int, channel chan int, closer bool) {
  for i := 0; i < 10000000; i++ {
    // send count data to channel
    fmt.Println("process", id," send", i)
    channel <- 1
  }
  if closer { close(channel ) }
}

func main() {
  channel := make(chan int)
  go counter(1, channel, false)
  go counter(2, channel, true)

  x := 0
  // receiving data from channel
  for i := range channel {
    fmt.Println("receiving")
    x += i
  }

  fmt.Println(x)
}

This version of counter send the data through go channel. There are 2 counter routine running in thread, invoke by go counter(channel, closer). In the main process, the for loop wait to read the channel until it is closed.

What happened here is the cpu core will switch between threads, and main process can be run when there’s data in channel, if not, switch to other threads. Until the channel is closed.

Example in 5 count:

Thread 1 send 0
Thread 2 send 0
Thread 1 send 1
receiving
receiving
Thread 2 send 1
receiving
Thread 1 send 2
receiving
Thread 2 send 2
receiving
Thread 1 send 3
receiving
Thread 2 send 3
receiving
Thread 1 send 4
receiving
Thread 2 send 4
receiving
receiving

Moreover, we can increase the channel buffer to optimize threads, for example:

channel := make(chan int, 10)

Increase buffer to 10 will make the execute sequence different:

Thread 1 send 0
Thread 2 send 0
Thread 1 send 1
Thread 2 send 1
receiving
Thread 1 send 2
Thread 2 send 2
receiving
Thread 1 send 3
Thread 2 send 3
receiving
Thread 1 send 4
Thread 2 send 4
receiving
receiving
receiving
receiving
receiving
receiving
receiving

By the channel model, thread can easily wait and receive data but not exchange data by shared memory. Making parallel programming more easy and natural.

However, accessing channel is still a heavy job. If we tweak the counter program above to only send back the result:

func counter(id int, channel chan int, closer bool) {
  x := 0
  for i := 0; i < 10000000; i++ {
    x++;
  }
  channel <- x
}

The execution time can be pretty different:

//count to 200000000
//channel count:
real    0m32.650s
//result only:
real    0m0.359s

Interface Oriented

In Go, there is no Class. Go take a lightweight approach that the data and function are seperate. And we can attach function to data as receiver.

1
2
3
4
5
6
7
8
9
10
11
12

//define datatype Name is string
type Name string

//define name.print() that type Name can receive function call "print()"
func (name Name) print() {
  fmt.Println(name)
}

func main() {
  name := Name("John")
  name.print() // John
}

On the other hand, interface is the collection of methods. That describe the behavior of a type. Go is using ducktype approach, as long as the type have all method defined in interface, it can be treat as the interface type:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

package main

import (
  "fmt"
)

//define interface Runner which can Run
type Runner interface{
  Run()
}

type Person string

//A person can Run => p.Run()
func (p Person) Run() {
  fmt.Println(p, "is running")
}

type Dog struct{
  Name string
  Type string
}

//A dog can Run => d.Run()
func (d Dog) Run() {
  fmt.Println("A" ,d.Type, "dog is running")
}

//A runner can start running
func Start(r Runner) {
  r.Run()
}

func main() {
  p := Person("John")
  Start(p) // "John is running"
  d := Dog{"Johnny","Husky"}
  Start(d) // "A Husky dog is running"
}

Map Reduce

Map Reduce is a programming model that distribute the calculation and collect the results. The most famous example is Google. When Google build the index of web pages. Because the calculation is too big. They need to distribute the task to different server, and run as parallel as possible. The result is that they use the map reduce structure, each server received some amount of web page. Parse and count the words of that page, and the result index will be collected and added to the main index.

Here is the structure of Map Reduce by Go, from Dan Bravender

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

package main

import (
  "fmt"
  "math/rand"
)

type Object interface {}

func MapReduce(input chan Object,
               mapper func(Object, chan Object),
               reducer func(chan Object) Object,
               pool_size int) Object {

    worker_outputs := make(chan chan Object, pool_size)
    go func() {
      //read input, assign input to mapper(item, output)
      for item := range input {
          worker_output := make(chan Object)
          worker_outputs <- worker_output
          go mapper(item, worker_output)
      }
      close(worker_outputs)
    }()

    reduce_input := make(chan Object)
    go func() {
      //listening worker_outputs as reduce_input
      for worker := range worker_outputs {
          reduce_input <- (<-worker)
      }
      close(reduce_input)
    }()

    return reducer(reduce_input)
}

I use the type “Object” to reference an empty interface. Because the Go don’t have generic type.

The first section send input to mapper, and receive worker_outputs as the collection of each worker’s result channel. Afterward, second section of the code send worker_output to reducer. Reducer return the result of calculation.

For example, we can try to calculate PI by some amounts of random points. Assume there’s a circle with radius 1. we get a random point on the square of 1. The PI will be the number of points in circle / total points. That the more points we collect, the more accurate the result will be.

For example, we can distribute the calculate of 2000 points to 2000 worker, and the PI will be equal to the result / total points

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

func CalculatePI(n Object, c chan Object) {
  sum := 0
  for i := 0; i < n.(int); i++ {
    x := rand.Float64()
    y := rand.Float64()
    if (x * x + y * y ) <= 1.0 {
      sum += 1
    }
  }
  c <- sum
  close(c)
}

func CollectPI(in chan Object) Object {
  sum := 0
  for i := range in {
    sum += i.(int)
  }
  return sum
}

func main() {
  input := make(chan Object)
  go func(){
    for i := 0; i < 2000; i++ {
      input <- 2000
    }
    close(input)
  }()
  points_in_circle := MapReduce(input, CalculatePI, CollectPI, 10).(int)
  result := float64(points_in_circle) * 4 / (2000 * 2000)
  fmt.Println(result)
}
//output: 3.141711
//output on 20000 * 20000 points: 3.14156549

Unittest

Go provide a unittest package and a buildin command for testing:

go test

Will execute all *_test.go files as test cases.

Let’s write some test for the MapReduce mapper and reducer we have:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

package main

import (
  "testing"
)

func TestCalculatePI(t *testing.T ) {
  result := make(chan Object)
  go CalculatePI(2000, result)
  sum := <-result
  PI := float64(sum.(int)) * 4 / 2000
  if PI > 3.5 {
    t.Error("Calculation Error!!")
  }
}
func buildChannel(in chan Object, a int, b int) {
  for i := 0; i < b; i++ {
    in <- a
  }
  close(in)
}

func TestCollectPI(t *testing.T){
  result := make(chan Object)
  go buildChannel(result, 10, 10)
  sum := CollectPI(result).(int)
  if sum != 100 {
    t.Error("Calculation Error!!")
  }
}

The go testing framework passing a pointer for test case. There is no extra syntax like assert to learn. Just use simple ==, != to check the results is correct or not. If Error happened, call testcase.Error for report error. Else the case will be mark as passed.

Conclusion

Go is a very interesting language. On the first hand, it might be awkward because of it’s syntax and no object oriented support. But once you know the simplicity and concurrency of Go. It would be a handy tool for the need fast calculation. However, we’re still waiting for a more complete eco system for developer and successful usecase for Go language.

Backbone is a framework for front-end JavaScript, unlike jQuery focus on easier DOM manipulation and event binding, backbone provide a structure for separating data model and DOM, just like a MVC framework separate model, view and controller. Make heavy JavaScript application easier to develop and maintain.

Why Backbone?

In jQuery, we will write a sequence of chain selector for assign data model to DOM element like:

1
2
3
4
5
6
7
8

var article = {
  author:"Joe",
  content: "testing"
};

$('#article').click(function(event){
  $(this).find('.content').text(article.content);
});

For backbone, it provide the Model class and View class, which is the main two element of backbone.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

var Article = Backbone.Model;


var ArticleView = Backbone.View.extend({

  el: $('#article'),

  initialize: function(){
    this.model.bind('change',this.updateContent, this);
  },

  events: {
    "click .content" : "updateContent"
  },

  updateContent: function(){
    this.$('.content').text(this.model.get("content"));
  }

});

var article = new Article({
  author:"Joe",
  content: "testing"
});

var articleView = new ArticleView({model:article});

OK, it’s seen like a completely overwork, that’s my first thought when I saw a clientside MVC framework too, but we have to dig further to understand why people create it.

Event Binding

First, it separate the data model and view - which include event handle and DOM element, we can easily bind data model in view like:

this.model.bind('change', this.updateContent, this);

It means when the data model change, the view will be automatically updated to DOM element.

Also, backbone provide literal event binding:

events:{
  "click .content": "updateContent"
}

which bind the “click” event to “.content” class, trigger “updateContent” function in current view. It’s useful when binding multiple events.

Restful synchronization

Furthermore, the backbone provide a RESTful format API for synchronize data model with server. every model class in backbone have a corresponding url for synchronize, and when the “model.save” or “model.fetch” method is called, the model will make an Ajax request to update data with server. For example:

article.save({author:"John"});

will first find article.sync method, if undefined, call Backbone.sync. The sync method will make an ajax request to server, and update model, trigger event and update to view.

the url of model is defined by RESTful API:

• create -> POST /model
• read -> GET /model[/id]
• update -> PUT /model/id
• delete -> DELETE /model/id

and we can also change it by overwrite url parameter

Dependency

backbone.js depend on underscore.js and jQuery/Zepto library, since it is only a 5k framework focus on the structure of JavaScript application.

Collection and Router

Backbone provide Collection, which is the collection of Backbone.Model, and provide manipulation on underscore.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

var Article = Backbone.Model;

var Articles = Backbone.Collection.extend({
  model: Article
});

var articles = new Articles();

articles.add([{
  author:"Joe",
  content: "testing"
},{
  author:"John",
  content: "developing"
}]);

articles.each(function(article){
  console.log(article.get("author"));
});

the Router in backbone is like the route map in MVC framework, it provide clientside fragment url routing, eg. route “/#article/12” to match method “article(id)”. So the url can be recorded and bookmarked by browser.

Template

In Backbone, we can create template in view. In that way we can add and remove multiple view and render with template. The underscore.js provide template engine or we can use other engine too.

In view:

1
2
3
4
5
6
7
8
9
10

render:function(){

    var template = "<%=content %>;

    $el.html(
        this.template(this.model.toJSON())
    );

    return this;
}

Conclusion

JavaScript MVC framework is good for rich client application and also can keep your javascript cleaner. But whether to use it depend on how much script in your application, for small page, the MVC is total overhead, but it is good for large scale page like Mail or Desktop like application.

More

Backbone
Sample todo application
Annotated source

Invitation

After handing out my resume on school’s career fair, I got an email from Amazon campus recruiter that they have an interview event for campus hire hold on Amazon campus in Seattle, and they would like to pay me the trip to the interview. That really surprise me because what I heard is they will usually have one or two phone interview before you get to the on-site interview. However, this is a great oppurtunity, and also a free trip visiting Seattle. So I just go for it.

Preparation

My sister in law live in Seattle, and she introduced me a friend who work in Amazon. I got lots of great advices about how to prepare the interview with Amazon. The position I applied for is general software engineer so I will not inteview with specific team members. Therefore I just prepare for general questions.

First, for technical interview. There’re two books that really explain the process and questions they will ask:

• Cracking the code interviews

• Programming interviews exposed

  Those two book is really helpful. And is the main source of preparation. I started a github project and push answers on it for preparation, Also I wrote the code on notebook for practicing writing code on whiteboard.

  Also, TopCoder is your good friend, attend some Single Round Matches or do some questions in practice room to really get the tempo of solving questions.

  For algorithms, the recruiter recommanded some books and resources:

• Algorithm in Nutshells is a digest book for the algorithms and data structures.

• TopCoder algorithms tutorials teach you the skills to solve problems on Topcoder using algorithms and data structures you learned.
• Introduction to Algorithms is the masterbook of algorithms.
• Programming Pearl is another book about sophiscated pragramming questions. The friend in Amazon told me it is a good book for solving algorithm questions.

• Also, http://caterpillar.onlyfun.net/ is a great resouce for basic algorithms if you can read chinese :)

  For coding questions, I mainly prepared in Java so I reviewed the language basic through some internet resources, also remember to check regular expression and basic file and folder read, write Api in the language.

  Also I read other resouces from internet:

• My job interview at google is a great interview article that contains a lots of resources worth for reading. Also I translated it into Chinese before.
• Five essential phone screen question is also helpful on the basic problems.

• Euler project is a resource for mathmatical computer programming questions.

  Before the departure, I asked my brother for some mock up interview and try to solve the problems by writing codes and speak my thought in the same time. Also I got a mock up interview on phone with the friend in Amazon, that is helpful because he can point out my errors and ask questions on my answer like real interviewer. (actually he is, all Amazon Engineer need to be interviewer).

Interview

After 8 hours flight and transfer, I arrived in Seattle at night and took Taxi to Hilton Bellevue. The main traffic tool Amazon suggest is Taxi, they will reimburse all traffic expenses during the interview.

Next day, I took taxi to Amazon campus near Bellevue, the texi driver said Amazon just move their campus to south lake union recently. I arrived around 10:00, the interview hold on 11:00. So I visited the South Lake Union park before interview.

On 11:00, the interview began with a lunch with Amazon engineers, who will also be the interviewer afterward. It is not part of the interview, but a chance to let you understand more about Amazon. I sit on the table with an engineer who work in customer service team. I asked some questions about software development and life in Amazon. The interesting thing is, Amazon is composed by lots of small and medium teams running Agile, each team using the languages and services by their requirements and scale, the stack is differ from team to team. The culture looks pretty Engineer driven. And he said the life and workload balance is reasonable in Amazon. After lunch, we was taken to different rooms with whiteboard, which is the place we will stay in the rest of the interview. There’s four sessions, each is 45 minutes and no rest netween sessions. It seens like a really hard schedule for both side.

First session was from an enginner in Kindle app store. I signed the NDA so better not to talk about the question. But it is a simple programming question, not too hard but need to handle some details. I wrote the question in a python-like pseudo code first, and rewrote in java. It takes more time but make you able to write code with fewer errors. I am pretty comfortable in this way, but need to control the time of writing pseudo code and real code.

Second session was from web team (probally), He asked about some data structure. how do you implement it, pros and cons, and some optimize questions. As the same I write the data structure in Pseudo code first and rewrite with Java.

Third session was from an engineer in wireless team. He asked a tricky question about data structure. After answer that, he asked about the pro and cons of my implmentation and the design decision. I made a small mistake when answering a design question. I said I put the method in class just to keep code concise for interview, but I should rather talked more about the pro and cons about how I made the decision.

Last session was from kindle team. He asked a practical question that he solved in Amazon website before. In the beginning I wrote the pseudo code in python, and he suggest me just wrote in python. But I got confused with Python api of folder. And didn’t answer well on the optimize question afterward. So the lesson is, just make sure you are familier with API of the language you use.

Afterward, they escorted me to the lobby and I took taxi back to meet my sister in law. Having some authentic Taiwanese dinner and met some friends who works in Amazon and Microsoft. I took to flight back to New York in the next morning.

Conclusion

In the end I didn’t get the job, and they encouraging me to try again in half year. But I learned a lot through the preparation of the interview. The interview process is pretty fun - solving different problems with engineers on whiteboard - is like playing a big quiz game. Not every programmer like these pop questions, but at least I think it is pretty interesting and challenging to solve. I will definatlly have more technical interview in future. And this experience will help a lot.

Thanks for the friends who help me on the interview. And for this oppurtunity to visit Seattle - it is a nice city to live. Wish I can visit there again in near future.

Bootstrap, a css framework that developed by Twitter, created by Mark Otto and Jacob Thronton. Which original was the internal framework in twitter for their products and quick prototyping. On the github page, Bootstrap is described as:

Simple and flexible HTML, CSS, and Javascript for popular user interface components
and interactions.

That is, Bootstrap is not only just a css framework, but actually an UI framework base on HTML, CSS and Javascripts. It includes the basic css framework that handle details like css reset, typography and grid system. Moreover, it includes the base UI elements like navigation bar, button, form and paginition… and the interaction UI elements like dropdown, popover and tooltips. Evenmore, all of the color and elements is customizable because the css is generated by less, all setting is in varibles.less file and allow user to change the variables or reuse the elements class in less.

As twitter engineers describe:

In the earlier days of Twitter, engineers used almost any library they were familiar with to meet front-end requirements. Bootstrap began as an answer to the challenges that presented and development quickly accelerated during Twitter’s first Hackweek. In twitter, this framework is widly use among different products as the base UI framework

Bootstrap solves the problem of how to reuse css and UI components. Also it improves the productivity of designer and programmer because it is very easy to use on prototyping new products.

Project Structure

The project is hosted on Github

It includes 3 main parts: less css files, jQuery plugins, and documents.

docs/    #example and github-page
img/     #css sprite for icons
js/      #jQuery plugins for UI interaction
less/    #main css files
  |- ...
  |-bootstrap.less   #main entry point, includes all other .less file
  |-grid.less        #setting grid system
  |-layouts.less     #setting container width
  |-mixins.less      #mixin functions for clearfix, grid, border, box-shadow and more.
  |-reset.less       #css reset
  |-responsive.less  #responsive css, will compile to bootstrap-responsive.css
  |-scaffolding.less #basic structural layout
  |-type.less        #Typography for elements
  |-utilities.less   #Quick css functions like float:left and display:hide
  |-variables.less   #All varibales like font, background, grid-width 
  |-...
Makefile             #build script on bootstrap, use lessc to compile all .less file to bootstrap.css

Reset

CSS reset is an important topic to display unify layout among all browsers and devices. It reset the layout by overwriting the different default style on browser. There are also a lot of reset framework that we can choose beside bootstrap, like YUI reset or Eric Meyer’s Reset Twitter’s reset is adapted from normalize.css

Grid System

Grid system is a way to simplify the positioning and floating of css, which defines the grid with fixed width, so user can use the grid without setting float and calulating persentages. There are differents grid system like 960gs, YUI Grid or blueprint css. Each have different implementation.

On Bootstrap 2.0, it use the 960px-12column Grid system:

Each grid is 60px width with 20px left-margin, we can easily use the grid in our html file by adding the span* class to elements:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

<div class="container">
  <div class="row">
    <div class="span4">
      <h1>Heading</h1>
      <p>
      Donec id elit non mi porta gravida at eget metus. Fusce dapibus, tellus ac cursus 
      commodo, tortor mauris condimentum nibh, ut fermentum massa justo sit amet risus. 
      Etiam porta sem malesuada magna mollis euismod. Donec sed odio dui.
      </p>
    </div>
    <div class="span4">...</div>
    <div class="span4">...</div>
  </div>
</div>

Output:

We can also use the fluid layout to make the layout and grid width become fluid with screen width:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

<div class="container-fluid">
  <div class="row-fluid">
    <div class="span4">
      <h1>Heading</h1>
      <p>
      Donec id elit non mi porta gravida at eget metus. Fusce dapibus, tellus ac cursus 
      commodo, tortor mauris condimentum nibh, ut fermentum massa justo sit amet risus. 
      Etiam porta sem malesuada magna mollis euismod. Donec sed odio dui.
      </p>
    </div>
    <div class="span4">...</div>
    <div class="span4">...</div>
  </div>
</div>

UI Elements

The Bootstrap provide decent style for html elements like button, form and table. Also, It provide some popular UI elements such as navigation bar, tooltip, popover, modal…, Here we will just pick some popular UI elements.

Button

1
2
3

<div class="btn btn-primary">Active Action</div>
<div class="btn">Action</div>
<div class="btn btn-large btn-danger">Don't press this!</div>

Here’s part of the buttons source:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

//less/buttons.less
// Core
.btn {
  display: inline-block;
  padding: 4px 10px 4px;
  margin-bottom: 0; // For input.btn
  font-size: @baseFontSize;
  line-height: @baseLineHeight;
  color: @grayDark;
  text-align: center;
  text-shadow: 0 1px 1px rgba(255,255,255,.75);
  vertical-align: middle;
  .buttonBackground(@white, darken(@white, 10%));
  border: 1px solid #ccc;
  border-bottom-color: #bbb;
  .border-radius(4px);
  @shadow: inset 0 1px 0 rgba(255,255,255,.2), 0 1px 2px rgba(0,0,0,.05);
  .box-shadow(@shadow);
  cursor: pointer;

  // Give IE7 some love
  .reset-filter();
  .ie7-restore-left-whitespace();
}

Navigation

Navigation bar is the top menu bar of twitter, we can set it as fixed or non-fixed:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

<div class="navbar navbar-fixed-top">
  <div class="navbar-inner">
    <div class="container">
      <a class="brand" href="#">
        Neethack
      </a>
      <ul class="nav">
        <li class="active">
          <a href="#">Home</a>
        </li>
        <li class="divider-vertical"></li>
        <li><a href="#">Link</a></li>
        <li><a href="#">Link</a></li>
      </ul>
    </div>
  </div>
</div>

By setting the .nav-collapse class and include bootstrap-collapse.js, bootstrap-responsive.css, the nav-bar can automatic correspond with mobile device:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

<div class="navbar navbar-fixed-top">
  <div class="navbar-inner">
    <div class="container-fluid">
      <!-- placeholder when navbar collapse -->
      <a class="btn btn-navbar" data-toggle="collapse" data-target=".nav-collapse">
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
      </a>
      <a class="brand" href="#">Neethack</a>
      <!-- set .nav-collapse for responsive -->
      <div class="nav-collapse">
        <ul class="nav">
          <li class="active"><a href="#">Home</a></li>
          <li class="divider-vertical"></li>
          <li><a href="#about">About</a></li>
          <li><a href="#contact">Contact</a></li>
        </ul>
      </div>
    </div>
  </div>
</div>

Form

Bootstrap enhance the html form components into more decent elements, we can just use original form elements and append the grid class .span* to control the width.

1
2
3
4
5
6
7
8
9
10
11

<form>
  <fieldset>
    <label for="input01">Label name</label>
    <input id="input01" type="text" class="span3" placeholder="Type something…">
    <span class="help-inline">Associated help text!</span>
    <label class="checkbox">
      <input type="checkbox"> Check me out
    </label>
    <button type="submit" class="btn">Submit</button> 
  </fieldset>
</form>

Here’s only a few examples of UI elements, check more UI elements on the components page.

Responsive

Responsive design is a big topic recently, since more and more mobile and tablet devices have become the main consumer of web pages. Bootstrap provide the responsive design by using the media query and grids. Take a look at the bootstrap-responsive.less file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

// LANDSCAPE PHONE TO SMALL DESKTOP & PORTRAIT TABLET
// --------------------------------------------------

@media (max-width: 767px) {
  // GRID & CONTAINERS
  // -----------------
  // Remove width from containers
  .container {
    width: auto;
    padding: 0 20px;
  }
  // Fluid rows
  .row-fluid {
    width: 100%;
  }
  // Undo negative margin on rows
  .row {
    margin-left: 0;
  }
  // Make all columns even
  .row > [class*="span"],
  .row-fluid > [class*="span"] {
    float: none;
    display: block;
    width: auto;
    margin: 0;
  }
}

This is a small part in responsive.less. As we can see, it use the media query to apply the propriate style for device. On the device that max-width is under 767px, like LANDSCAPE Phone and PORTRAIT TABLET, container width will be set to auto, and the span width will be set to full width to display propally on device. For example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

<div class="container">
  <div class="row">
    <div class="span4">
      <h1>Heading</h1>
      <p>
      Donec id elit non mi porta gravida at eget metus. Fusce dapibus, tellus ac cursus 
      commodo, tortor mauris condimentum nibh, ut fermentum massa justo sit amet risus. 
      Etiam porta sem malesuada magna mollis euismod. Donec sed odio dui.
      </p>
    </div>
    <div class="span4">...</div>
    <div class="span4">...</div>
  </div>
</div>

display in desktop device as:

but when the device width is lower than 767px, the span width will become full, the layout will become vertical:

Bootstrap provide that with media query, all UI elements include navigation, form, tab and others is responsive. The user only need to include the bootstrap-responsive.css to apply those feature without designing mobile version of pages.

Compile your own bootstrap

Bootstrap use less to compile the .less file into css. And all the params is in variables.less file. Therefore we can easily change the color and layout setting and recompile our own version of bootstrap.

First, we need to install the node.js with npm(node package manager) Using npm to install less.js(you can also install by rubygem, bug less.rb have some bugs on finding .less file on reletive folder)

npm install -g less 

Next, install javascript compressor uglify-js by npm

npm install -g uglify-js 

After that, we can start to tune on the color and varibles, and pack it with

make bootstrap 

We can start to use our own version of bootstrap.css. With bootstrap-responsive.css, bootstrap.min.js

Here’s some Sample Page that build with different color scheme.

Moreover, you can also start to build your own new css framework from preboot.less includes basic grids and mixins.

Conclusion

Bootstrap is one of the easiest framework to use and integrate. Because it have great project documents and rich examples. Also it is open sourced that we can easily fork and customize on the base of twitter. For the nearly future, apperently there will be more and more css UI framework that make web development much easier.

Sinatra

Sinatra is a rack-base , lightweight web framework implemented in ruby. Written and desinged by Blake Mizerany. Famous for it’s dsl syntax and simpliness.

Source structure

examples/
lib/
  sinatra/
    base.rb   #all codes are in here
    main.rb   #Application class, extends Base class in base.rb
    showException.rb #output exception and trace message as Html error page
  sinatra.rb  
test/
Rakefile
Gemfile.gem

base.rb

Main Sinatra application, includes:

1.Rack Module :
Implement Rack:Request and Rack:Response

2.Helper Module : Helper methods that available in routes, filters and views , handle tasks like redirect, status code, url, html header, session, mime type, http stearming… etc

3.Template Module : Handle multiple template engines using tilt

4.Base class:
The main class that include all modules above. Handling routes and invoke correspond code blocks and filters.

5.Application class: Inherit Base class, the run instance of Sinatra application.

6.Delegator module:
Delegate DSL methods in Top-level file to Sinatra Application.

main.rb

Patch Sinatra::Application class, set the hooks to run application at exit and Parse option. Also it includes the delegator to send all methods on Top-level to application.

Dependencies

Sinatra source seperate the declaration of external ,stdlib and project depedencies. Which is pretty clean and easy to understand:

1
2
3
4
5
6
7
8
9
10
11
12
13

# external dependencies
require 'rack'
require 'tilt'
require "rack/protection"

# stdlib dependencies
require 'thread'
require 'time'
require 'uri'

# other files we need
require 'sinatra/showexceptions'
require 'sinatra/version'

Delegator

Delegator is an interesting part in Sinatra, since it creates a really simple API that user can just write method with HTTP verb in Top level file, without creating any class.
For example:

1
2
3
4
5
6

#myapp.rb
require 'sinatra'

get '/' do
  'Hello world!'
end

Execute this file will run Sinatra application handle route “/” with GET request. But how do Sinatra do this?

Take a look at the source of Delegator:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

# Sinatra delegation mixin. Mixing this module into an object causes all
# methods to be delegated to the Sinatra::Application class. Used primarily
# at the top-level.
module Delegator #:nodoc:
  def self.delegate(*methods)
    methods.each do |method_name|
      define_method(method_name) do |*args, &block|
        return super(*args, &block) if respond_to? method_name
        Delegator.target.send(method_name, *args, &block)
      end
      private method_name
    end
  end

  delegate :get, :patch, :put, :post, :delete, :head, :options, :template, :layout,
           :before, :after, :error, :not_found, :configure, :set, :mime_type,
           :enable, :disable, :use, :development?, :test?, :production?,
           :helpers, :settings

  class << self
    attr_accessor :target
  end

  self.target = Application
end

First, if we want to delegate method to another class, we can include the methods in files :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

module Delegator
  self.target = Application

  def get(*args,&block)
    target.get(*args, &block)
  end
end

include Delegator

#then we can call 'get' method in file and delegate to target 
get '/' do
  "Hello delegator!!"
end

But how would we do if we have lots of method to delegate? In Sinatra, it has lots of methods and Http verbs to be delegated. The code will be pretty ugly if we have to implement all these repeated methods. The answer here is metaprogramming: We can use ruby’s ability of metaprogramming to create repeated methods in a few lines of code:

1
2
3
4
5
6
7
8
9
10
11
12

module Delegator
  def self.delegate(method_name)
    define_method(method_name) do |*args, &block|
      Delegator.target.send(method_name, *args, &block)
    end
    private method_name
  end

  delegate :get

  self.target = Application
end

In ruby, we can use “define_method” to create method programmically, and use “send(method_name, *args, &block)” To call the target method by the method_name. This makes the code a lot cleaner in Sinatra

Routes

In sinatra, after user call the dsl methods(like get, post) in file, the HTTP verbs ,path and code block will be registered in application, And will be executed when receiving matched request.

When the dsl method get called, the Application will generate a Proc with the name of HttpVerb and path (like “get /”) save the Proc, url path (include the keys, pattern and conditions on paths, like “/:id” ) in @routes.

Here’s the simplify version of routes :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

class App

  class << self
    attr_reader :routes

    def get(path, options={}, &block)
      route("GET", path, options, &block)
    end

    def route(verb, path, options, &block)
      @routes ||= {}
      signature = compile!(verb, path, block, options)
      @routes[verb] ||= []
      @routes[verb] << signature
    end

    def compile!(verb, path, block, options)
      unbound_method = generate_method("#{verb} #{path}",&block)

      [path, proc {|base| unbinded_method.bind(base).call() } ]
    end

    def generate_method(method_name, &block)
      define_method(method_name, &block)
      method = instance_method method_name
      remove_method method_name
      method
    end
  end
end

App.get "/" do
  "Hello world"
end

base = App.new

App.routes["GET"][0][1].call(base)
#print:: "Hello world"

In here, sinatra generates the code block as an unbound_method, it is a kind of instance method that you can bind it to any other instance dynamically before call. Sinatra use this to bind Application instance with Proc on runtime.

Route call

After register the code block, sinatra wait for request and invoke correspond routes to handle request. The entry point of all request is the rack call interface. All rack application must implement the interface.

Overall, the request execution stack is: call => call! => invoke => dispatch! => route! => route_eval

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

def call(env)
  dup.call!(env)
end

def call!(env)
  @env = env
  @request = Request.new(env)
  @response = Response.new
  @params = indifferent_params(@request.params)
  template_cache.clear if settings.reload_templates
  force_encoding(@params)

  @response['Content-Type'] = nil
  invoke { dispatch!}
  invoke { error_block!(response.status) }

  unless @response['Content-Type']
    if Array === body and body[0].respond_to? :content_type
      content_type body[0].content_type
    else
      content_type :html
    end

    @response.finish
  end
end

The code above is the first part of how Sinatra handle incoming requests. First, as a Rack application, all request will invoke the call(env) function Sinatra application will duplicate an instance, invoke the call!(env) on new instance (because HTTP is stateless) in the call! function, sinatra will new the Rack::Request and Rack::Response object by env, than set the params.

After all object is set, it will start to invoke the routes by “invoke{ dispatch! }”, the result will be store on @response, and return to user by call the @response.finish

1
2
3
4
5
6
7
8
9
10
11
12
13

  # Run the block with 'throw :halt' support and apply result to the response.
  def invoke
    res = catch(:halt) { yield }
    res = [res] if Fixnum === res or String === res
    if Array === res and Fixnum === res.first
      status(res.shift)
      body(res.pop)
      headers(*res)
    elsif res.respond_to? :each
      body res
    end
    nil # avoid double setting the same response tuple twice
  end

The invoke function wrap and execute the handler codeblock ,catch the :halt (which throw by route! as interrupt signal), and than set status, header and result to @response.

for example, when you execute the code wrapped by invoke, you can set the @response by throw :halt and Array response:

invoke do 
  #do something...
  throw :halt ,[200,"Hello world!"] #this will go to @response
end

With the structure like this, error_block or other function can also throw :halt with result and return to user.

1
2
3
4
5
6
7
8
9
10
11
12

# Dispatch a request with error handling.
def dispatch!
  invoke do
    static! if settings.static? && (request.get? || request.head?)
    filter! :before
    route!
  end
rescue ::Exception => boom
  invoke { handle_exception!(boom) }
ensure
  filter! :after unless env['sinatra.static_file']
end

In dispatch function, it check the static file first, than execute before filter, then execute the route! function follow by the after filter.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

def route!(base = settings, pass_block=nil)
  if routes = base.routes[@request.request_method]
    routes.each do |pattern, keys, conditions, block|
      pass_block = process_route(pattern, keys, conditions) do |*args|
        route_eval { block[*args] }
      end
    end
  end

  # Run routes defined in superclass.
  if base.superclass.respond_to?(:routes)
    return route!(base.superclass, pass_block)
  end

  route_eval(&pass_block) if pass_block
  route_missing
end

# Run a route block and throw :halt with the result.
def route_eval
  throw :halt, yield
end

At the bottom of execution stack, the route! function check the registered routes with request path and params. If it find correct route, execute the codeblock and throw :halt with result. the invoke function will catch the :halt, than set the result to @response.

If no route is executed, route_missing will be called and return not_found page.

Template

Sinatra is compatible with a lots of different templates, from erb, haml, markdown to sass, less… almost any kind of templates that you can find, but how do Sinatra handle all of these different format? It turns out using Tilt gem that includes all kinds of template engines.

For example, with Tilt, we can compile an erb template like this:

1
2
3
4
5
6
7
8
9
10
11

require "tilt"

template = Tilt[:erb]
# => Tilt::ErubisTemplate 

# pass the file path or pass content body
compiled_template = template.new("path/to/file") { "hello world" }
# => <Tilt::ErubisTemplate: ... @path="path/to/file" @data = "hello world">

compiled_template.render()
# => "hello world"

In application, we can call “erb” method to render erb template:

get "/" do
  #render erb template in views/index.html.erb
  erb :index
end

under the hood in Template module:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

def erb(template, options={}, locals={})
  render :erb, template, options, locals
end

...

def render(engine, data, options={}, locals={}, &block)
  # merge app-level options
  ...

  # compile and render template
  begin
    layout_was      = @default_layout
    @default_layout = false
    template        = compile_template(engine, data, options, views)
    output          = template.render(scope, locals, &block)
  ensure
    @default_layout = layout_was
  end

  # render layout
  ...

  output
end

def compile_template(engine, data, options, views)
  eat_errors = options.delete :eat_errors
  template_cache.fetch engine, data, options do
    template = Tilt[engine]
    raise "Template engine not found: #{engine}" if template.nil?

    case data
    when Symbol
      body, path, line = settings.templates[data]
      if body
        body = body.call if body.respond_to?(:call)
        template.new(path, line.to_i, options) { body }
      else
        found = false
        @preferred_extension = engine.to_s
        find_template(views, data, template) do |file|
          path ||= file # keep the initial path rather than the last one
          if found = File.exists?(file)
            path = file
            break
          end
        end
        throw :layout_missing if eat_errors and not found
        template.new(path, 1, options)
      end
    when Proc, String
      body = data.is_a?(String) ? Proc.new { data } : data
      path, line = settings.caller_locations.first
      template.new(path, line.to_i, options, &body)
    else
      raise ArgumentError, "Sorry, don't know how to render #{data.inspect}."
    end
  end
end