Just In Time

HAProxy: Routing by domain name

2009-11-02T17:58:00.000-08:00

Without too much effort I was able to find multiple search results that said HAProxy could direct traffic according to domain name. Unfortunately I was hard pressed to find any example configurations demonstrating that functionality. Turns out it is in section 7.7 of the documentation.

If we wanted to direct traffic for foo.com and bar.com to different servers we could do so as follows:


frontend http_proxy
  bind 192.168.0.1:80
  acl is_foo hdr_dom(host) -i foo
  acl is_bar hdr_dom(host) -i bar
  use_backend cluster1 if is_foo
  use backend cluster2 if is_bar

backend cluster1
  server server1 192.168.0.2:80
  server server2 192.168.0.3:80

backend cluster2
  server server3 192.168.0.4:80

So first we create a frontend called http_proxy that will listen on port 80 of 192.168.0.1. The next lines are the key parts.

We create an acl called is_foo which checks the Host portion of the HTTP header for the provided value, in this case 'foo' in one and 'bar' in the second acl. The '-i' flag in both examples sets the match to be case insensitive.

The last part of the frontend configuration is to tell it to use a backend named cluster1 or cluster2 depending on if the is_foo or is_bar acl matches.

Finally we define the backends cluster1 and cluster2. The backend cluster1 has 2 servers, backend cluster2 only has 1.

So in this case if we sent a request for http://foo.com to 192.168.0.1:80 the acl that checks for the value 'foo' in the host should pass, and our request should be proxied to port 80 of either 192.168.0.2 or 192.168.0.3.

The practical example shown in the HAProxy documention is that you could use this to direct traffic for assets.foo.com to a server for static content, and www.foo.com over to your application server. However, if you found this article through a search, I'm sure you already have in mind how you want to use this.

In addition to the hdr_dom shown in this example, there are many other hdr_* matchers, as well as many other configuration options with HAProxy. For full details and options see the official HAProxy documentation.

Tiny email sending service with Amazon SQS

2009-10-26T19:50:00.000-07:00

I found the "Dynamo Paper" by Amazon to be interesting. In particular I liked the idea of small services. Nothing new, but I liked the concept and wanted to play with it.

The result was the 71 line SQS Mailer, available on Github.

For this experiment the code polls an Amazon SQS queue. If there are messages in that queue, it retrieves them, parses them, and sends them out as emails via SMTP. If there are no messages waiting, it just sleeps for 5 seconds and tries again. So if you ran multiple applications that all needed to send email, they could send it out through this single service by sending a message to the queue. Which SMTP server account the message would go out through is controlled by a combination of a potentially application-specific identifier in the message you send to the queue, and a service-side lookup in accounts.yml.

Will this be useful? Not sure. But it was fun to put together!

Basically I sketched this out onto some notepaper one night, and then later filled in the functionality for each method when I had time:

At the current Amazon SQS pricing of $0.01 per 10,000 requests, leaving this idling would incur a cost of about $6.50 per year from AWS.

As the disclaimer in the README file states, this doesn't handle failure scenarios with any grace. If I decide it is useful for anything, I will give that some attention. Or you can fork the project and let me know what you come up with.

Abort: couldn't find mercurial libraries

2009-09-06T18:58:00.000-07:00

I've been setting up my fresh Snow Leopard installation today.

I followed the guide from Hivelogic: Compiling Mercurial on Snow Leopard. The installation completed without issue and 'which hg' showed everything was in place.

However, when I tried to issue an 'hg' command the following error popped up:

"abort: couldn't find mercurial libraries"

Google came to the rescue, and revealed these tweets:

https://wincent.com/twitter/155

https://wincent.com/twitter/156

For Snow Leopard, I just needed:

export PYTHONPATH=/usr/local/lib/python2.6/site-packages

If you hit the same, hope this works for you.

Typing? I'm a programmer not a secretary.

2008-11-27T06:00:00.000-08:00

This last week has seen a few posts on the necessity, or lack thereof, for good touch typing skills.

We Are Typists First, Programmers Second - Jeff Atwood

Is Typing A Necessity For Programming? - Abhijit Nadgouda

I originally was going to say something along the lines of "I think it depends on how you view programming." I was then going to differentiate between whether you just view programming as a job (where you are looking to scrape by with minimum investment), or a skill/craft/art to be cultivated.

But in the process of assembling my ideas, I thought back to how I learned to touch type. I wanted a job that at the time seemed to pay pretty decent. However, the job required a minimum typing speed of 40WPM. I was a 2 finger typist up till that point, which wasn't going to cut it. Since I wanted the job I put forth some effort. In about 1 month, using some free typing tutor programs, I learned to touch type sufficiently to pass the test and get the job. Once on the job further increases in speed and accuracy came.

The glorious job that prompted this investment of effort was data entry. It was boring as could be. Really. If you have never done data entry yourself it would be hard to imagine the level of boredom involved. There was no craft or art to it, just glossy eyes and sore wrists.

Now programming is not "just typing". It's not just entering characters into a buffer. Believe me, I speak from experience, it is not just data entry. So a 10x improvement in your typing speed will not equate to a 10x increase in your productivity. Hopefully between all that typing, programming requires you to stop and think.

Nevertheless, whether you use some auto-complete super-fancy intellisensical snippet-fu brain scanning IDE or a bare bones text editor, you most likely use a keyboard as your central physical tool to transfer your thoughts from your brain into the computer. If that is the case, even if programming is just your "job", you owe it to yourself to put forth a little bit of effort and learn to touch type.

It really is worth the effort, and the initial awkwardness. If you don't know how to touch type you won't understand until you do. There is something, well, just "cool" about being able to see words appear on the screen as you think them without giving any conscious thought to directing the individual fingers and keystrokes.

If you are going to use a tool on a daily basis, I can't imagine any reasonable argument against taking the time to learn to use it well.

Knowing Your Languages

2008-11-08T17:03:00.000-08:00

You may only speak one language, but how well do you know your tools? There are likely many similarities between learning a language and learning the tools you use daily.

Today I was thinking about the difference between someone who is just learning a spoken language and someone who knows it well, and how this may at times be seen by their knowledge of it's vocabulary. For example, imagine someone explaining "I need a tool that you use to hit those little metal things that connect two pieces of wood together. It has a wooden handle about 12" long and has a metal part on top. You know? The tool that people who build houses use."

Do you know what tool they are talking about? Probably. When I was learning Spanish, this is the sort of description I would find myself having to use at times. (To all the patient people who put up with me, Thank you.)

Of course, a person who knew the language well could have conveyed the same thought more concisely and clearly: "I need a hammer."

This same idea goes beyond spoken language.

I use VIM on a daily basis. Until recently my VIM vocabulary consisted of :wq, dd, i, /, :%s and esc. Not a vary wide vocabulary at all for such a powerful tool. I could get most anything done, but it was more like that first long-winded description of a hammer. I am continuing to work on expanding my VIM vocabulary, and in doing so finding that what previously took many keystrokes can now be done nearly instantly with a single command.

However, the real value of a rich vocabulary is much more than simply the ability to be concise. A rich vocabulary allows you multiple options to express your thoughts depending on the intended goal.

Instead of simply stating that it was "humid", a story-teller may choose to describe the air as "thick with moisture, soaking one's clothes and sapping one's strength." Not as concise, but perhaps adding important details.

The same is true with our programming language vocabulary. The goal of a rich vocabulary in Ruby, Erlang, or (insert your favorite language here) is not always to write the least lines of code. Our goal may be to increase maintainability, expose intentions, improve performance, etc. By improving our vocabulary and grasp of the language, we are able to express our thoughts both efficiently and in the way that best fits the specific situation at hand.

How is your vocabulary with the tools that you use? What are you doing to improve?

Remove Gems By Prefix

2008-09-27T07:49:00.001-07:00

If you are working with Merb or DM you know that approximately a gazillion gems are involved. When, for whatever reason, I want to remove them it is a pain to do by hand.

Here is a simple script to help:

#!/bin/bash

prefix=$1

gem list --local | grep $prefix | awk '{ print $1 }' | xargs sudo gem uninstall

I saved mine as remove_gem_by_prefix, gave it executable permissions and placed it in my path.

Now I can issue the commands:

$ remove_gem_by_prefix merb

$ remove_gem_by_prefix dm

And they all get blasted. You do need to be careful to make sure the prefix you provide is unique to the gems you wish to remove.

Git-O-Mator

2008-09-15T21:12:00.000-07:00

Inspired by Rein Henrichs Hack && Ship, I have created Git-O-Mator.

Git-O-Mator is more name than these 2 simple scripts deserve, but what fun is naming small?

Git-O-Mator can be found on GitHub, and contains the scripts new_repo and hack.

New_repo

New_repo was originally mentioned in the last blog post "Quick Remote Git Repository Creation Script", and has the syntax of:

new_repo foo

Which will create a Git repository locally and on a remote host account that you configure.

Hack

Hack is very similar to Hack && Ship. Syntax is:

hack on

This will automatically switch to the default branch which you can easily configure. Alternatively you can use:

hack on foo

Where "foo" is the name of the specific branch you wish to use. If the branch does not exist, it is created.

To rebase with the master, you have:

hack sync

To merge and push to your remote repository:

hack push

It also supports Rein's ssp (Simple Software Process) with:

hack ssp

Which is equivalent to hack sync && hack push. If you prefer hack sync && rake && hack push, then you may use the -t flag for testing:

hack ssp -t

Silly little experiment. If you see room for improvement, feel free to fork and improve!

Quick Remote Git Repository Creation Script

2008-09-13T14:04:00.001-07:00

If you are using Git for your source code management, GitHub is an awesome tool. It especially shines for public projects where you freely allow others to fork your code and possibly pull patches back in.

Sometimes I'm just working on a project that I would prefer to keep in a private repository. GitHub provides paying accounts with such an option. However, I already have hosting accounts that are terribly underused. Here is a little script I use to create a remote git repository on one of my VPS accounts that I can then pull from and push to.

Prerequisites
A hosting account which you can ssh / scp into.

The Script

$ vim new_repo


#! /bin/sh
PWD=`pwd`

# You must adjust these variables for your specifc hosting account.
# Remote user you will connect as.
REMOTE_USER="admin"
# The IP address you will SSH / SCP to.
REMOTE_HOST="123.456.123.456"
# The remote path you wish to store your .git repositories in.
REMOTE_REPO_PATH="/home/admin/repos/"

if [ -d $1 ]; then
  echo "EXITING: Local directory '$1' already exists."
  exit 0
else
  mkdir $1
  cd $1
  git init-db
  touch README
  git add .
  git commit -m "Initial Repository Creation"
  cd ..
  git clone --bare $1/.git $1.git
  echo "** Copying new repository $1.git to $REMOTE_HOST:$REMOTE_REPO_PATH"
  scp -r $1.git $REMOTE_USER@$REMOTE_HOST:$REMOTE_REPO_PATH
  rm -rf $1.git
  rm -rf $1
  echo "** Cloning locally at $PWD/$1"
  git clone $REMOTE_USER@$REMOTE_HOST:$REMOTE_REPO_PATH$1.git
fi

exit 0

Or get it from GitHub

http://github.com/jpease/git-o-mator

Configuration

Of course, you will need to REMOTE_USER="admin" with an actual user on your hosting account, adjust REMOTE_HOST="123.456.123.456" to point to your accounts IP address, and edit REMOTE_REPO_PATH="/home/admin/repos/" with whatever path you wish to contain your Git repositories.

Once that is done, provide executable permissions.

$ chmod +x new_repo

That's it. Now if you execute:

$ ./new_repo testing

You will end up with /home/user/repos/testing.git on your remote host, and ./testing locally. From ./testing you can git push to send commits to the remote repository, and git pull to retrieve from the remote repository.

If you see room for improvement, I'm sure there is some, please leave a comment with your revision!

Compiling Nginx on OS X Leopard in 5 minutes

2008-08-30T19:43:00.000-07:00

I went to compile Nginx on my laptop this evening, and couldn't find a how-to for Leopard. Well at least not on the first page or 2 of search results.

Danger Weary Traveller!

I happen to be running OS X version 10.5.4 build 9E17.

If you are running something different, these directions may fail. In fact, for all I know your computer may actually blow-up. Yes, as in burst into flames and melt into a smoldering heap of plastic mush. Compiling stuff really is that scary. Hide the children. Don eye protection.

False Advertising Disclaimer

Whether it will really take you 5 minutes or not probably depends on your processor and whether or not your copy & paste skills are up to par. But seriously, it took me longer to write this post than to do the actual install.

Related Sites

Official Nginx Site
Nginx Wiki
Official PCRE Site

Satisfy PCRE Dependency
Nginx requires PCRE to be installed. PCRE stands for Prank Calling Really is Evil, or something like that. But whatever it is, Nginx thinks it just can't live without it.

$ sudo mkdir -p /usr/local/src
$ cd /usr/local/src
$ curl -O ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-7.7.tar.gz
$ tar xvfz pcre-7.7.tar.gz
$ cd pcre-7.7
$ ./configure --prefix=/usr/local
$ make
$ sudo make install
$ cd ..

Install Nginx
$ curl -O http://sysoev.ru/nginx/nginx-0.6.32.tar.gz
$ tar xvfz nginx-0.6.32.tar.gz
$ cd nginx-0.6.32
$ ./configure --prefix=/usr/local --with-http_ssl_module
$ make
$ sudo make install

Now if you ask:

$ which nginx

You should receive the answer /usr/local/sbin/nginx

If not, you probably need to add /usr/local to your PATH.

Nginx will setup a default index.html file in /usr/local/html. The default nginx.conf file will be in /usr/local/conf and will be set to serve this index.html file. To start Nginx:

$ sudo nginx

And finally, to see the fruits of your labors open your favorite browser and navigate to http://localhost

Next you get to configure Nginx to serve up whatever it is you want to serve. But that search query will already provide you with many, many results...

The Real Goal of Education

2008-07-16T08:45:00.000-07:00

Is education about memorizing facts? No, it's about learning to learn. It's about learning how to think.

"...developers today code in something called Python, but when I was in school C was all the rage. The need for reasoning, though, remains constant..." - Our Googley advice to students: Major in learning

DataMapper: Many-to-many

2008-06-23T18:55:00.001-07:00

You can define many-to-many associations using this syntax:

#item.rb
has n, :things, :through => Resource

#thing.rb
has n, :items, :through => Resource

In this case it will create a table items_things to manage the assignments. Easy enough. Depending on the relationship you are mapping out, it may make more sense to use the :through => :model syntax.

Now if you create an instance of an Item, you will get a method "things=". At first I thought this would be how you would assign a new Thing to your Item. Such as:

@item = Item.get(1)
@thing = Thing.get(1)
@item.things = @thing

FAIL! If you try adding a single Thing to your Item with this method you will receive a failure that the class you sent did not have a map method defined. Ah, ok. That makes sense, things= is obviously plural and will want an array of things. So how do you assign just one thing?

If you are unsure, a handy tool is to remember the "methods" method. So in this case you can try out:

@item.methods

You will see that you have not only a "things=" method, but also a plain old "things" method. If you try:

@item.things.methods

You will see that you have a << method, as should be expected. So to add our single Thing to our Item we do:

@item.things << @thing

That's it.

DataMapper: Parent / Child Relationship

2008-06-06T20:08:00.000-07:00

I couldn't find an example at datamapper.org /docs for a parent child relationship.

After a little searching, I found the answer in the integration tests. Here is the example:


 class Node
   include DataMapper::Resource

   def self.default_repository_name
     ADAPTER
   end

   property :id, Integer, :serial => true
   property :name, String

     has n, :children, :class_name => 'Node', :child_key => [ :parent_id ]
   belongs_to :parent, :class_name => 'Node', :child_key => [ :parent_id ]
 end

Hope that helps someone else.

Marshal data too short

2008-03-13T15:33:00.000-07:00

If you marshal data and then store it in a DB, make sure to either base64 encode it or store it in a binary data type (such as BLOB), otherwise it will probably cause errors.

Merb Monday: Vacation Edition

2008-03-03T10:00:00.000-08:00

I'm on vacation visiting family (with limited Internet access). Back to semi-normal next week.

Merb Monday: The Leap Year Edition

2008-02-25T13:27:00.000-08:00

News
Official documentation has been updated to include 0.9. Still a little light, but coming along. If you would like to help, create your own git branch and have at it!

Ivey had a baby. Congratulations!

Expect 0.9.1 (developer release) today (or soon).

You get an "extra" day this week. Use it wisely.

Teaser
Code name: Merblets.

Q&A

Q: What!? No RJS?

A: You can do anything rjs can do with *.js.erb templates (Answer by ezmobius)

Q: I read that Merb is template agnostic. What templates languages do I have available?

A: erb out of the box (erubis), haml (merb_haml as a dependency in init.rb), markaby (Answer by sqred)

Q: What does a named route in Merb look like?

A: r.match("/login").to(:controller => "Sessions", :action => "new").name(:login)
(Answer by jodo)

Q: Does Merb have a debugger tool?

A: You can use ruby-debug. Start Merb with the -D option to enable ruby-debug support.
Enter "debugger" somewhere in your code as a breakpoint. (Answer by jdempsey)

Q: How do I find out what version of Merb I'm using?

A: merb -v (Answer by ezmobius)

Q: How can I see the current routes?

A: merb -i
merb.show_routes
(Answer by ezmobius)

Q: Can the cookie session be used for large production apps?

A: As long as you don't store really sensitive info in the session then cookie sessions are fine (Answer by ezmobius)

Merb Monday: State of flux

2008-02-18T14:47:00.000-08:00

This week's edition is a little short. Certainly not for lack of activity in #merb, but due to the continued disturbance in the force caused by the 0.9 transition.

0.5 to 0.9 is a big jump (no wonder they skipped 0.6, 0.7 and 0.8!) Most of the discussion this week continued to be related to the transition to 0.9 and trouble ticket type "stuff".

News
0.9 was released (kinda), however this is a "developer" release and still has some wrinkles to be ironed out before a general public release will be pushed to RubyForge. If you are anxious to give it a spin, check Merbivore.

It looks like there will likely be a 0.9.1 "developer" release in the near future. Once we get to a 0.9.x general public release things should settle down and this series should get back to normal.

merbful_authentication should soon be updated to be compatible with 0.9.

Required Reading
0.9 introduces some really cool changes. One of those is Rack integration. You should read Ezra's post "So merb-core is built on rack you say? Why should I care?"

Q: If I use ActiveRecord as the ORM, do I then use ActiveRecord's migrations (and its syntax) by default?

A: Yes. You can use ActiveRecord with Merb as you would with Rails. (Answer by amoeba)

Q: How do you generate an app in Merb 0.9?

A: ./script/generate is gone. In it's place now lives merb-gen.

merb-gen myapp # a normal merb app merb-gen myapp --flat # a flattened app merb-gen myapp --very-flat # a single-file app

(Answer by ivey)

Answer Of The Week

And the answer to many of the questions this week which followed the format of:

Q: _____________________ in Merb 0.9?

A: "It's not in -core anymore" (Answer by ivey)

Audience Participation

Q: Assuming you are using ActiveRecord with Merb, how can you share models with a Rails application?

If you leave your answer in the comments, I'll post it next week. (hint) (Question & link by seebg)

Merb Monday: "That's the spirit"

2008-02-11T06:37:00.000-08:00

Tidbits from #merb for the week of Feb 4 - Feb 10.

A lot of the conversation this week revolved around the pre 0.9 release. Many of the questions involved bugs or other issues that one can expect in a pre-release. In general I didn't include them as they will quickly be outdated and likely be of little on going value.

Q: How do I install (freeze) gems within my application?

A: Navigate to the root of your Merb application.

gem install foo -i gems

In your merb.yml file include a line:

dependency "foo"

(Answer by wycats)

Q: How do I support a new MIME type? Say for an Atom feed.

A:

Merb.add_mime_type(:atom,:to_atom,%w[application/x-atom])

Replace application/x-atom with whatever the correct header for atom is then make a template: foo.atom.erb

In your controller, use:

provides :atom

(Answer by ezmobius)

Q: Where could I define a constant which would then be accessible in different views, but should have a different value depending on the environment merb is running in?

A: config/environment/*.rb where * represents the desired environment, for example development.rb.

Q: How do I find the current controller and action in the view?

A: In a helper you could create the two following methods.

def controller_name controller.request.controller_name.split('_').first end def action_name controller.request.action end

(Answer by slurry)

Q: What happened to MERB_ENV?

A: Merb's application constants have been re-factored as follows:

Old constant : New method MERB_FRAMEWORK_ROOT : Merb.framework_root MERB_ROOT : Merb.root MERB_VIEW_ROOT : Merb.view_path MERB_SKELETON_DIR : Merb.skeleton_path MERB_LOGGER : Merb.logger MERB_PATHS : Merb.load_paths MERB_ENV : Merb.environment

(Answer by slurry)

Q: Is there a way to list all the generators available?

A: ruby ./script/generate

Q: Where is the source now?

A: The Merb project has switched from using SVN to Git for source code management. With that change the source is now at Github. Specifically you will want to use wycats repo, which you can find at: git://github.com/wycats/

The source code is broken down into merb-core, merb-more, and merb-plugins.

For more information on how to work with Git, see "So you want to contribute to Merb-core".

Once you have cloned the Git repos to your local system, you will then move into that directory and issue a "rake install" command to build the gem.
(Answer by ivey)

Q: Where are bugs / tickets being tracked now?

A: Along with the move to Github, bugs and ticket tracking has been moved from DevJaVu over to Lighthouse, specifically at merb.lighthouseapp.com.

Answer of the week

Here is one little snippet of conversation that may provide the answer regardless of your specific problem:

Q: Can I do _____________?

A:
hassox: I guess so
hassox: but I can't really help you on it...
grrt: :) I'll see how far I get
hassox: that's the spirit :)
grrt: it's great to just explore

Sometimes it's best to just try something out.

Audience Participation

Q: Merb-core has methods "render" and "display". What's the difference?

Leave your answer in the comments. (Hint)

Working Remotely: The "Get Real" List.

2008-02-04T11:00:00.000-08:00

Every once in a while a post shows up on Digg or elsewhere about how to work remotely. Often it will be phrased like "How to work from the beach".

After reading these lists, I can't help put wonder... "do any of these people have any idea what they are talking about?"

Most of these articles are nothing more than recommendations for various online applications from flickr to mozy to zoho - which in my opinion have nothing to do with your ability to work remotely or not.

My Credentials
I lived in Cerro Punta, Chiriqui, Panama for 14 months. That's up in the mountains... in Central America... where most people don't have telephone service... or refrigerators.

How To Really Work Remotely
(from the beach or wherever you want)
The ability to do this has nothing to do with online applications. Here are the seven requirements that I needed:

1) Laptop
A laptop is the tool of choice for a number of reasons. It is easy to transport to your destination. If the power goes out, you still can work (and hopefully it will come back on before your battery dies).

2) Battery Back-Up / UPS / Surge Protector
With a laptop, the battery back-up portion may not be as critical - but you definitely will want to have all of your hardware plugged in to quality surge protector. Surges happen.

3) Data Back-Up
Your business likely depends upon your data. Get an external drive and make sure you are regularly backing up your data. As a second step, an internet based back-up can't hurt. But with a slower Internet connection (or even a fast one) this may be a slow way to recover your data in case of a catastrophe, and thus I wouldn't recommend it as your only back up.

4) Internet
The Internet itself, not the online applications, is your connection to the outside world. Depending on your specific usage, the speed and reliability required will vary. You may also wish to secure two means of connection, as the reliability may be less than you are used to. I was able to secure a cellular based service (advertised at 256K, but more like 56-128k down/15k up), and also a wireless service (1MB up/down). The cell service was slow, but cool that it basically worked country-wide (yes, even at the beach). Between the two I was able to have very reliable service and even good enough quality for VoIP phone calls.

5) Phone number
You (likely) need a phone number that can follow you wherever you happen to be. I used Packet8's VoIP service for my business line. This meant I could simply plug my little VoIP box into my router in Panama, and all my incoming / outgoing calls went through my TX number. Skype also worked pretty well.

6) A local relative / friend
Mail still needed to be picked up from my PO Box. Checks still needed to be deposited in my Bank account. Having a trustworthy person that you are able to turn these simple, but important, tasks over to will be a huge help. There are mail processing services available, but not in every city (or even state). Having clients send mail to an address in a different state might make them uncomfortable.

7) A business / job that does not require your physical presence
In the end this is the one that didn't work out. My line of work lends itself to remote work extremely well. Even when I am local, in person contact is very limited during the course of a project. The problem that arose was the initial phase of acquiring new projects. I would get leads, perform the initial interviews & evaluations, compile a proposal and even get the proposal accepted. But then many clients would want to meet face to face to sign the contract. Obviously this won't be true for every business, or geographical location, but for my principally Texas based clients this became a sticking point.

So that's what worked for me. For 14 months. In a cloud forest in Central America. And amazingly I didn't need any online applications to do it!

Merb Monday: Environment variables, partials, etc.

2008-02-04T09:00:00.000-08:00

I see a lot questions asked and answered in the #merb IRC room.

Many of them are questions that later I wish I remembered the answer to! So I'm going to try to keep track of questions I see asked and answered and record them here for the benefit of myself and others, and hopefully publish them here every Monday.

I might re-phrase the questions and answers slightly, as the writing tends to be quite terse on IRC.

Q: How do you check the current environment in Merb? I'm looking for the Merb equivalent to RAILS_ENV.

A: Merb.environment (Answered by: Booster)

Q: How do you render partials?

A: You use the partial method.

Scenario 1: You wish to render a partial template found in the same directory as your action's view template.

partial :foo

This will look in the current directory for a file named _foo.html.erb

Scenario 2: You wish to render a partial template found in a different directory than the action's view template.

partial 'common/foo'

This will look for the file views/common/_foo.html.erb
(Answered by: jp_n9)

Q: I'm trying to run my db:migrate and I'm getting a "no such file to load -- spec/rake/spectask" error.

A: Make sure you have the rspec gem installed.

sudo gem install rspec

(Answered by: octopod)

Q: How do I start merb in production mode?

A: Use the -e option. For example,

merb -e production

For a complete list of parameters type:

merb --help

(Answered by: Phoop)

Q: How do I get Merb to log data to the log file?

A: Make sure that you start Merb as a daemon with the -d directive, this will cause output to be logged to the log file located in the log/ directory (instead of being output to the console window).

merb -d

(Answered by: ezmobius)

Q: How can I use HAML for my templates?

A: Name your files with the .html.haml extension instead of the default .html.erb extension.

Example: name.html.haml

Second, you need to edit dependencies.rb to include the line:

dependency "haml"

You will also need to have the haml gem installed on your system. Restarting Merb will also likely be required. (Answered by: QaDeS, bradly)

Q: Is there an easy way to get the current URL in a merb view?

A: I think it's request.uri (Answered by: hassox)

In my testing the request variable does not appear to be directly available from a view template. However if you need that information you can easily pass the entire request variable, or just request.uri, into your view from the controller.

Monitoring Swiftiply & Merb with Monit

2008-01-31T10:09:00.000-08:00

Swiftiply combined with Merb is a pretty nice way to serve up your application. And while we hope that both are stable, it would be nice to know that just in case anything went wrong they would restart. It would also be nice for the application to automatically start upon reboot.

That is what we are going to attempt to setup using Monit.

Prerequisites:
We assume you already have Swiftiply and Merb setup and functioning properly. We are also assuming that you have Monit installed and functioning properly.

Versions:

Ubuntu 7.10 (Gutsy)
Ruby: 1.8.6
Merb: 0.5.2
Swiftiply: 0.6.1.1
Monit 4.8.1
Assumes your Merb app is "frozen" (rake merb:freeze)

Those are the version I was using at the time of the article. Different versions may lead to varying results.

Setup Monit To Auto-Start

> sudo vim /etc/default/monit

This is a pre-existing file. Edit as follows:

startup=1

Create init script for Swiftiply
The big problem with trying to monitor Swiftiply was that it does not write a pidfile for Monit to track. Fortunately someone solved the problem, and Google found it. Big credit for the script: http://pastie.textmate.org/107044.txt

> sudo vim /etc/init.d/swiftiply

This will be a new blank file. Edit as follows:

#!/bin/bash function generatepid(){ current_pid=`pgrep -f 'swiftiply -c /etc/swiftiply.conf'` pidfile_pid=`cat /var/run/swiftiply.pid` if [[ $current_pid = '' ]]; then rm /var/run/swiftiply.pid elif [[ $pidfile_pid != $current_pid ]]; then echo $current_pid > /var/run/swiftiply.pid fi } case "$1" in start) echo -n "Starting swiftiply:" swiftiply -c /etc/swiftiply.conf generatepid echo ;; stop) echo -n "Stopping swiftiply:" killall -9 swiftiply generatepid echo ;; restart) $0 stop $0 start ;; pid) generatepid ;; *) echo "usage: $0 [start|stop|restart|pid]" esac exit 0

This does assume that you Swiftiply configuration file is location at /etc/swiftiply.conf, if not change to the correct location.

Create init Script For Your Merb App
I will use "app1" but you can replace that with whatever name you would like as an identifier for this specific Merb application.

> sudo vim /etc/init.d/merb_app1

This will be a new blank file. Edit as follows:

#!/bin/bash case "$1" in start) echo -n "Starting Merb for App1:" export SWIFT=1 export HOME="/home/admin" #Assuming you have a user admin cd /full/path/to/your/app/current/ sudo ./script/merb -d -h 127.0.0.1 -p 30000 -e production echo ;; stop) echo -n "Stopping Merb for App1:" cd /full/path/to/your/app/current/ sudo ./script/stop_merb echo ;; restart) $0 stop $0 start ;; *) echo "usage: $0 [start|stop|restart]" esac exit 0

The -p directive should point to the same port that you have configured Swiftiply to listen to for this specific app. Also, this is assuming that you have Merb frozen. If not, then you will need to change "./script/merb" and "./script/stop_merb" to their unfrozen counterparts.

Set Permissions
Without these permissions set to execute the init scripts will not work.

> sudo chmod 755 /etc/init.d/swiftiply > sudo chmod 755 /etc/init.d/merb_app1

Configure Your Main Monit File

> sudo vim /etc/monit/monitrc

Add the following in to the pre-existing configuration:

check process swiftiply with pidfile /var/run/swiftiply.pid group root start program = "/etc/init.d/swiftiply start" stop program = "/etc/init.d/swiftiply stop" if 5 restarts within 5 cycles then timeout include /etc/monit/app1

Configure App1 Monit File
You could have just added the following in the main file, but I prefer to use the include statement and keep them in separate files.

> sudo vim /etc/monit/app1

This will be a new blank file. Edit as follows:

check process merb_app1 with pidfile /full/path/to/your/app/current/log/merb.30000.pid start program = "/etc/init.d/merb_app1 start" stop program = "/etc/init.d/merb_app1 stop" if 5 restarts within 5 cycles then timeout depends on swiftiply

30000 should be replaced with the specific port that you have Swiftiply listening on for this application. The "depends on swiftiply" will make sure that swiftiply is started first.

Does it work?

It'd take a miracle...

> sudo monit -t > sudo /etc/init.d/monit force-reload > sudo monit validate > sudo monit summary

Caveats
This setup only checks to make sure the processes are up and running.

Right now Merb overwrites log/merb.port#.pid with the last mongrel process started.

Unfortunately that means if you start 5 mongrels, you will only have the pid of the last one, and thus will only be able to monitor that one process with Monit.

On the up side, it still means you can guarantee at least one process to be up and responding to requests - which is better than none.

Update

Merb 0.9 is supposed to add the ability to specify a pidfile which will allow us to monitor multiple Merb processes even on the same port.

Useful?

If so, why not recommend me on Working With Rails!

Book Review: Programming Erlang

2008-01-30T08:15:00.000-08:00

Author: Joe Armstrong

Erlang really appears to be an interesting language, and the author's enthusiasm for the subject shows - which is good. It helped me to keep going, when at times the code was a little hard to follow.

The book does a good job of introducing the language. In particular later chapters give emphasis to the topics of concurrency through multiple processes, multi-cores and distributed programs. All very timely subjects for a world that is connected to the Internet, and where even laptops have multi-core processors

Unfortunately, I feel the author somewhat forgot his audience.

On page 5 he starts out with a description that fit me almost perfectly, and probably many other readers: "Once upon a time a programmer came across a book describing a funny programming language. It had an unfamiliar syntax [...] it wasn't even object-oriented. The programs were, well, different....Not only were the programs different, but the whole approach to programming was different."

After reading the book I don't feel like the "Erlang Master" that the road map described (pg 9). While the syntax is now familiar, I still look at the Erlang code and it feels foreign. I still have to "decipher" the code instead of read it.

I would have liked the book to more fully address the items from the "Once upon a time" paragraphs. Being that "the whole approach to programming" is different than the OO that many readers are used to, I would have liked to have seen a chapter (or three) on how to best get into that mode of thinking.

I do think the book is a good jumping off point. It gives you more than enough to get started.

However, if you are unfamiliar with languages where functions accept functions which also accept functions as parameters and in turn returns another function as a result, you may end up feeling (as I did) that you only have half the puzzle.

Serving PHP5 with Nginx on Ubuntu 7.10 (Gutsy)

2008-01-27T11:35:00.000-08:00

There are lots of tutorials out there to setup PHP with Nginx. I'm probably just retarded, but none of them got me to a working system. A detail left out of one, a configuration file left out of another, etc.

What follows is exactly what I did to get Nginx to serve PHP5 on a fresh Slicehost VPS slice running Ubuntu 7.10. Hopefully it will help someone else.

NOTE: The following commands assume that you have already setup a user account with sudo privileges and are executing the commands as that user, NOT as root.

Update your system and install basic tools required

> sudo aptitude update -y > sudo locale-gen en_GB.UTF-8 > sudo /usr/sbin/update-locale LANG=en_GB.UTF-8 > sudo aptitude safe-upgrade -y > sudo aptitude full-upgrade -y > sudo aptitude install build-essential -y

Install MySQL

> sudo aptitude install mysql-server mysql-client libmysqlclient15-dev -y

Install PHP5

> sudo aptitude install php5-cli php5-cgi php5-mysql php5-xcache -y

Note: Xcache is installed at this point and available for you to setup, but by default is not turned on.

Install Nginx

> sudo aptitude install nginx -y

Go to your IP address and you should now recieve the message "Welcome to nginx!"

FastCGI Parameter Configuration

We will place all of our fastcgi parameters in a single file which we can include in as necessary.

> sudo vim /etc/nginx/fastcgi_params

This will be a new empty file, add the following and save:

fastcgi_param QUERY_STRING $query_string; fastcgi_param REQUEST_METHOD $request_method; fastcgi_param CONTENT_TYPE $content_type; fastcgi_param CONTENT_LENGTH $content_length; fastcgi_param SCRIPT_NAME $fastcgi_script_name; fastcgi_param REQUEST_URI $request_uri; fastcgi_param DOCUMENT_URI $document_uri; fastcgi_param DOCUMENT_ROOT $document_root; fastcgi_param SERVER_PROTOCOL $server_protocol; fastcgi_param GATEWAY_INTERFACE CGI/1.1; fastcgi_param SERVER_SOFTWARE nginx; fastcgi_param REMOTE_ADDR $remote_addr; fastcgi_param REMOTE_PORT $remote_port; fastcgi_param SERVER_ADDR $server_addr; fastcgi_param SERVER_PORT $server_port; fastcgi_param SERVER_NAME $server_name; # PHP only, required if PHP was built with --enable-force-cgi-redirect # fastcgi_param REDIRECT_STATUS 200;

Nginx Configuration

> sudo vim /etc/nginx/sites-available/default

This is a pre-existing file. Find the part that looks similar to the following and edit it as so and save:

location ~ \.php$ { fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME /var/www/nginx-default$fastcgi_script_name; include /etc/nginx/fastcgi_params; }

We need to remember to restart Nginx so that it picks up our configuration changes.

> sudo /etc/init.d/nginx stop > sudo /etc/init.d/nginx start

Spawn-fcgi

We still need a script to start our fast cgi processes. We will extract one from Lighttpd.

> mkdir ~/sources > cd ~/sources > wget http://www.lighttpd.net/download/lighttpd-1.4.18.tar.bz2 > tar -xvjf lighttpd-1.4.18.tar.bz2 > cd lighttpd-1.4.18 > ./configure > make > sudo cp src/spawn-fcgi /usr/bin/spawn-fcgi

Let's get automated!

> sudo touch /usr/bin/php-fastcgi > sudo vim /usr/bin/php-fastcgi

This is a new empty file, add the following and save:

#!/bin/sh /usr/bin/spawn-fcgi -a 127.0.0.1 -p 9000 -u www-data -f /usr/bin/php5-cgi

Next...

> sudo touch /etc/init.d/init-fastcgi > sudo vim /etc/init.d/init-fastcgi

This is also a new empty file, add the following and save:

#!/bin/bash PHP_SCRIPT=/usr/bin/php-fastcgi RETVAL=0 case "$1" in start) $PHP_SCRIPT RETVAL=$? ;; stop) killall -9 php RETVAL=$? ;; restart) killall -9 php $PHP_SCRIPT RETVAL=$? ;; *) echo "Usage: php-fastcgi {start|stop|restart}" exit 1 ;; esac exit $RETVAL

We need to change some permissions to make this all work.

> sudo chmod 755 /usr/bin/php-fastcgi > sudo chmod 755 /etc/init.d/init-fastcgi

Create a Test File

> sudo vim /var/www/nginx-default/index.php

Let's just print out the information page for our PHP installation.

<?php echo phpinfo(); ?>

Start It Up

> /etc/init.d/init-fastcgi start

Now go to your IP address/index.php and you should see the PHP info page displayed.

Set To Startup Automatically Upon Reboot

> sudo update-rc.d init-fastcgi defaults

You might want to test and make sure that it actually starts up upon reboot...

> sudo reboot

Final Words
If you think I have done anything stupid in the way this is setup, you may very well be right! Please leave a comment with a suggestion on how to improve the setup.

You probably will want to look into configuring Nginx for VirtualHosts, which is not considered in this post.

Credits
Much of this information was taken from the excellent post: How To Install A Complete LEMP (Linux - EnginxX (Nginx HTTP SERVER) - Mysql - PHP) Server (Not LAMP...) On Ubuntu/Debian, which got me about 90% there.

Merb: Multiple CSS files, one server request.

2008-01-18T12:01:00.000-08:00

In my previous post Continued Education: Performance Yahoo I referred to Steve Souder' book "High Performance Web Sites".

Rule #1 is "Make fewer HTTP requests."

Now we already know that Merb is "built for speed", but generally that is considered within the context of the server side of things.

I was happy to see that Merb also allows us to follow Souder's #1 Rule for making the client side snappy too. So how does Merb allow us to make fewer HTTP requests? By dynamically combining multiple stylesheet or script files into one.

Let's see some code!
It's this simple:

# app/views/layout/application.html.erb <% require_css :reset %> <% require_css :master %> <%= include_required_css :bundle => true %>

We define the various CSS files we wish to be included using require_css (in this example reset.css and master.css), then by setting the :bundle => true paramenter everything will be served up in a single HTTP request in a file called all.css! Now that's pretty cool.

But wait there's more!
The same technique works for Javascripts too.
You can have the same functionality for your javascripts using this syntax:

<% require_js :foo $> <%= include_required_js :bundle => true %>

But wait there's even more!
Now what is really cool is that you don't have to place your require_css or require_js statements only in the layout. You can place those in your views or even partials. ~~That means that for any given request, the bundled file will only include the files that are needed for that specific view.~~ Edit: It looks like you may need to manually define and uniquely name each possible combination of files. It also appears that a CSS file required in a view will appear first in the generated combined CSS file and thus be over ridden by the (likely) more general file defined in the layout - which probably isn't the desired behavior.

Getting it to work
By default the bundling action should only occur when in production mode. At the moment (Merb 0.5.2) it appears there is a bug and the bundling action doesn't realize when it is in production mode - so nothing happens.

What you need to do is edit your config/merb.yml file and uncomment the line that reads:

:bundle_assets: true

That's it! Should work. Enjoy.

One Gottcha
By forcing :bundle_assets to true in the merb.yml file, we are enabling this functionality for all environments instead of just for production. This isn't really ideal

Credits
Thanks to dudleyf in #merb for helping me figure this out. Thanks to codahale for writing the code for :bundle.

Continued Education: Performance Yahoo

2008-01-18T10:08:00.000-08:00

In this edition of "Continued Education" I'm linking up Steve Souders' (Chief Performance Yahoo!) Google Talk "High Performance Web Sites: 14 rules for faster pages".

I've recently been reading his book that covers the same topic, but sometimes it's nice to be able to just veg-out and watch.

Good practical information that is fairly easy to implement and can greatly improve our users' experiences.

To Git, or not to Git.

2007-12-14T07:35:00.000-08:00

I just read Joe Moore's (Pivotal Labs) thoughts on the Linus Torvalds' Git talk that was presented at Google.

I had also watched Linus' Google Talk video about a month ago, and had been hearing about Git in the Merb IRC room.

Git: A cruel piece of software?
The chairman introducing Linus describes Git as "a revision control system which is expressly designed to make you feel less intelligent than you thought you were" and as "a software tool which only he [Linus] is smart enough to know how to use." Was he joking or partly serious? I wasn't entirely sure.

With that preface, I had the impression that using Git was going to have a cliff like learning curve. Nevertheless, I decided to give it a try so I could at least know what others were talking about.

A collaboration of 1
Unlike Pivotal Labs, I'm just a solitary developer using SCM to track changes on my own code. So my needs are fairly limited. I don't really have any "collaboration" needs.

For someone with my limited SCM needs, probably any SCM would work just fine.

Initial thoughts
After using Git a bit, here are a my basic thoughts.

It's actually not too difficult at all (at least at a basic user level).
It is a command line tool. That's fine for me, as I always have a terminal open anyways.
Basic commands are easy to remember: "git status", "git add", "git commit", "git push"
It's pretty easy to setup custom ignore files and setup a remote repo.
Committing locally is fast & independent of your internet connection.
Deployment via Capistrano (v2.1) supports Git.

It appears that branching and merging are very easy, but honestly I don't really do too much (ok, any) of either. I will experiment with that in the future to see if it adds any value to my specific work flow.

You won't find the same level of application support for Git as SVN at the moment. For example, the other day I read about the release of SVNmate, a bundle for Textmate which adds some SVN status icons to your files. There currently is not a bundle offering the same for Git (although there is a bundle "work in progress"). Not necessarily deal-breakers, but things to be aware of.

Personally, I'm going to continue using Git and become more familiar with it's capabilities.

NOTE: Git does have the gitk command which opens a GUI. I haven't used it, other than a cursory glance, and thus am not familiar with it's features.

Related Resources:
Git Cheat Sheet: http://zrusin.blogspot.com/2007/09/git-cheat-sheet.html
Git Tutorial: http://www.kernel.org/pub/software/scm/git/docs/tutorial.html