schwarz - Home

OpenURI and MemCached sitting in a tree

2008-01-14T14:39:00Z

The ultimate frankenstein-like experiment of harnessing OpenURI and MemCached has been completed!

For a number of projects that used openuri, I needed a way to reduce hits on remote services or rest apis, or even speed up the experience on the client end.

Getting it



sudo gem install openuri_memcached

The beast is alive! (Usage, for those not on late hours right now)

Use exactly the same as you would openuri, only.. enable it.



  require 'openuri_memcached'
  OpenURI::Cache.enable!
  open("http://germanforblack.com").read # Slow as a wet week

Quit your app (leave memcached running) and run the same example, it should now happen in less then … some time that is really fast.

Small print options

To get started run your memcached server

$ memcached -d

The default address that this gem will terminate against is localhost:11211, you can change this using:

OpenURI::Cache.host = ['serverone.com:11211', 'servertwo:11211']

The cache defaults to 15 minutes, however this can be changed using:

OpenURI::Cache.expiry = 60 * 10 # Ten long minutes

Let me know if you have any issues with it, or if you have any use for it!

A ruby Kuler parser

2007-12-06T22:24:00Z

I wrote this little diddy a couple of weeks ago; Since then it has sat on my desktop as a constant reminder of the little time I have to do anything with the snippets of crap that I write on a regular basis.

Due to this very reason, its probably better on the web for anyone who was interested in grabbing colours from Kuler, Adobe labs' user submitted colour... thing.

Usage


require 'kuler'
include Kuler
Kuler::recent
=> ["0E2F32", "C1E6B7", "006A69", "AAC593", "236555"]

This will give you an Array of recently posted colours' in hex values, other options are Kuler::rating and Kuler::popular

Having a poke around the Kuler site should show you that no decent API exits, however you can send some query strings to their awfully formatted RSS feed.

Javascript: Sleep, keypress delays and bashing bad articles

2007-10-20T22:41:00Z

Today I read the high scalability article aimed to reduce the amount of requests to your server by not sending XMLHttpRequests for every keystroke.

It was suggested that a onblur event was used to measure when the user had stopped typing.

Will your users expect to have to tab out of a field before something else happens? Will it confuse them if they've started typing in another text area? Probably.

At work, I'd whipped up something elegant to handle this very same paradigm for validating or searching user input.

Example

(using jquery to make those events really easy)


$('input.search').keyup(function () {
	// Lets have a clean version of the search string, you could really do more there though
	var search_term = new RegExp(this.value.replace(/\\/g, ''), "i");
	var search_execution = function () {
		// Expensive server queries here - buy some real servers guys! (what on earth are you requesting?)
	}.sleep(175);
});

175 milliseconds delay before the method is executed is a good typing rate. Maybe you want to change this to something that you're more comfortable with though.

Hold on, last time I checked, sleep() wasn't a part of the standard JavaScript language. Rather than using a simple setTimeout(), I decided that the ajax'd method shouldn't be executed again until it had finished the last time it ran.

The sleep method


Function.prototype.sleep = function (millisecond_delay) {
	if(window.sleep_delay != undefined) clearTimeout(window.sleep_delay);
	var function_object = this;
	window.sleep_delay = setTimeout(function_object, millisecond_delay);
};

er, yeah.. thats about it. I just really wanted to post the sleep method. I'd written it over a year ago now, but it's still used on a pretty regular basis. I use it for searching conducted in the DOM by hiding elements that don't match a simple regex.

Grabbing mysql production databases to your local system with rake

2007-10-05T17:06:00Z

After reading the comments on Nate Clarke's article I decided it would be a good idea to republish the <strike>stolen</strike> modified script that I found elsewhere (Honestly, I have no idea. If anyone can cite the original author I'll give credit where due)


namespace :db do
desc "Sync your local database with a remote one REMOTE=name_of_database LOCAL=your_local_db"
  task :sync do
    `ssh domain.com "mysqldump --skip-extended-insert -u db_username -p #{ENV['REMOTE']} | bzip2 " | bzcat | mysql -u root #{ENV['LOCAL'] || "app_development"}`
  end
end

The rundown

We use this almost daily
Use RSA keys and you'll only need to auth once (for the database)
I'm using --skip-extended-insert because it will write each query on a new line (for larger databases mysql can be known to crack the shits on a default setup when everything is inserted in a massive chunk)
Its using bzip to compress it over the wire (sadly, this was the original authors smarts)

Usage


rake db:sync REMOTE=my_db_with_lots_of_data

This is great for testing your app with different versions of data (for ridiculous migrations and such

Extreme expires headers for nginx and mongrel

2007-09-19T10:45:00Z

Like everyone else, I read ezra's nginx config and put it straight to use.

After recently becoming addicted to yslow, tweaking configs day and night (well, it happened once. yslow is great though) I decided to add expires headers to anything that isn't served by rails, your js, css, images etc.

Without further ado, my config:


worker_processes  2;
pid /var/run/nginx.pid;
events {
  worker_connections 1024;
}

http {
  include conf/mime.types;
  default_type  application/octet-stream;
  log_format main '$remote_addr - $remote_user [$time_local] '
                  '"$request" $status  $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';

  access_log  /var/log/nginx_access.log  main;
  error_log  /var/log/nginx_error.log debug;
  sendfile on;

  tcp_nopush on;
  tcp_nodelay off;
  gzip on;
  gzip_http_version 1.0;
# More is heavier on the CPU
  gzip_comp_level 5;
  gzip_proxied any;
  gzip_types text/plain text/html text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;

  upstream schwarz {
    server 127.0.0.1:4000;
    server 127.0.0.1:4001;
    server 127.0.0.1:4002;
  }
  
  server {
    listen 80;
    client_max_body_size 50M;
    server_name .germanforblack.com;
    root /var/www/public;

    access_log  /var/log/nginx.vhost.access.log  main;
    
    location / {
      proxy_set_header  X-Real-IP  $remote_addr;

      # needed for HTTPS
      proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_redirect false;
      proxy_max_temp_file_size 0;
      
      if (-f $request_filename) { 
	expires max;
        break;
      }

      if (-f $request_filename/index.html) {
       rewrite (.*) $1/index.html break;
      }
      # Tasty caching
      if (-f $request_filename.html) {
        rewrite (.*) $1.html break;
      }

      if (!-f $request_filename) {
        proxy_pass http://schwarz;
        break;
      }
    }

    error_page   500 502 503 504  /500.html;
    location = /500.html {
      root   /var/www/public;
    }
  }
}

Note the expires method. You just sped up your app dramatically

How to use lighttpd to serve a static site for development

2007-09-18T13:50:00Z

Write the following into your /etc/lighttpd/lighttpd.conf


server.bind = "0.0.0.0"
server.port = 3000
server.document-root = CWD
server.dir-listing = "enable"

mimetype.assign = (  
  ".css"        =>  "text/css",
  ".gif"        =>  "image/gif",
  ".htm"        =>  "text/html",
  ".html"       =>  "text/html",
  ".jpeg"       =>  "image/jpeg",
  ".jpg"        =>  "image/jpeg",
  ".js"         =>  "text/javascript",
  ".png"        =>  "image/png",
  ".swf"        =>  "application/x-shockwave-flash",
  ".txt"        =>  "text/plain"
)

Not bad, now, a really nice way to execute this, would be from the directory of where your sites' html & assets are at.

Place the folowing in to your ~/.bash_login or ~/.profile file.

alias li="lighttpd -D -f /etc/lighttpd/lighttpd.conf"

Proof is in the pudding:


bens-pb:~/sites ben$ li
2007-09-18 23:44:55: (log.c.75) server started

Ctrl+c will quit the server


2007-09-18 23:45:30: (server.c.1216) [note] graceful shutdown started 
2007-09-18 23:45:30: (log.c.135) server stopped

Shazam! Easy.

Modern javascript slides

2007-08-30T11:52:00Z

My slides from the Melbourne Ruby User Group, “Modern Javascript” are available in PDF.

Unfortunately no audio was recorded and the slides will make little sense without speech over the top. Hopefully a reference of the talk for those who wanted it.

Introducing GoogleQuery

2007-08-19T09:24:00Z

So, I love Hpricot because it feels like an extension of my hand.

Google (and Yahoo) have awesome human searchable queries like “people in china”.

The top most search result will show you the amount of people in china.

I quickly decided that this would be useless; however incredible to show my mad-ill flow with css selectors.

First, grab the gem from Rubyforge using:

sudo gem install google_query

Leap your ass into irb, require the gem and start screwing around

See how you get spanked by the pound

GoogleQuery::Currency.get 'AUD to GBP'

=> 1 U.S. dollar = 0.490484599 British pounds

Population in Melbourne

GoogleQuery::Population.get 'melbourne'

=> Population: 3,850,000 (Est.) (2nd)

Current time in London

GoogleQuery::Time.get 'london'

=> 2:26 PM on Monday, July 30

On the command line

bens-pb:~ ben$ gpop melbourne

=> Population: 3,850,000 (Est.) (2nd)

bens-pb:~ ben$ gtime london

=> 2:26 PM on Monday, July 30

I couldn’t resist naming it gmoney

bens-pb:~ ben$ gmoney AUD GBP

=> 1 Australian dollar = 0.424269178 British pounds

There you have it, my first gem ever.

Some queries may be broken, this could be due to google changing their search results screen or just no damn results.