Online Aspect by Josh Fraser

Happy Birthday Cohen

2009-07-12T20:12:02Z

We’re going to take a short break from our regular programming in honor of David Cohen. David runs the TechStars program and honestly our company wouldn’t be where it is now if it weren’t for him. I know most of the other TechStars teams feel the same way. Today is David’s birthday, and a few of us wanted to take a minute to say thanks and wish him a great day. Enjoy.

Happy Birthday David Cohen from Josh Fraser on Vimeo.

Queue events that occur before JavaScript is loaded

2009-06-25T10:16:10Z

One of the common recommendations for speeding up your website is to put your JavaScript at the bottom of your page instead of including it inside the head tag. The difference this simple placement can have is impressive, especially if you are dealing with sizable JavaScript libraries that are usually 50k at best.

One downside with putting your JavaScript at the bottom is that your fast clicking visitors may click on links that won’t work. The reason this happens is because the JavaScript that those links trigger hasn’t been downloaded yet. Usually those links will work on the second or third try, but it makes for a bad user experience and a poor first impression.

I decided to fix it by queuing up those user-triggered actions and replaying them as soon as the document is ready. I wrote a wrapper that I can use anytime I have code that depends on my JavaScript being downloaded and available.

The concept is simple. Instead of calling functions directly when a user triggers an action, I add the function call to a queue. When the document is ready, I loop through the queue and execute each of the actions in the order that they occurred. I put this code inline inside my head tag so it is available as soon possible. The rest of my JavaScript can then be included right before the closing body tag without worrying about this race condition between the browser and website visitor.

<script type="text/javascript">

var loaded = false;

var action_queue = new Array();

function when_ready(callback) {

    // skip the queue if the document has already loaded

    if (loaded == true)

        eval(callback);

    else {

        action_queue.push(callback);

    }

}

function dequeue_actions() {

    for (i in action_queue) {

        eval(action_queue[i]);

        delete(action_queue[i]); // cleanup after ourselves

    }

    loaded = true;

}

</script>

I then trigger dequeue_actions() as soon as the document is ready:

// using jQuery

$(document).ready(function(){

    dequeue_actions();

});

// this works too

onload = dequeue_actions;

You can then safely make function calls using when_ready(). For example:

<a onclick="select('foo')">foo</a>

becomes

<a onclick="when_ready('select(\'foo\')')">foo</a>

In my testing, the results have been very smooth with delays being almost unnoticeable. Of course, your experience will vary depending on the size of your document and how long it takes for your document to be ready.

This code is pure JavaScript and should work in every modern browser. I’ve tested it in IE6+, FF2+ and Safari 3+.

Reading GET variables with JavaScript

2009-06-10T08:42:45Z

One of the things that isn’t immediately obvious in JavaScript is how to access GET variables. I’ve seen lots of different implementations for this around the web, but the majority of them are bulkier than they need to be. Here’s my favorite way to do it:

<script type='text/javascript'>

function $_GET(q,s) {

    s = (s) ? s : window.location.search;

    var re = new RegExp('&amp;'+q+'=([^&amp;]*)','i');

    return (s=s.replace(/^\?/,'&amp;').match(re)) ?s=s[1] :s='';

}

</script>

What this gives you is a JavaScript implementation of PHP’s $_GET functionality. I use a regular expression to keep the code to a minimum. Here is a simple example of how to use it:

// this code would print "hello world" if it was at http://localhost/index.php?var1=hello&var2=world

var var1 = $_GET('var1');

var var2 = $_GET('var2');

document.write(var1 + " " + var2);

Another thing I like about this implementation is that it makes it easy to parse GET variables from arbitrary search strings (ex “?var1=hello&var2=world”). This is handy if you need to access GET variables from an HTML src parameter such as an image or script tag.

// get the src parameter and split it down to the search query string

var src = document.getElementById('example').src;

params = src.split('?');

var var1 = $_GET('var1','?'+params[1]);

Looking for a job? Don’t be this guy.

2009-06-05T20:40:40Z

I just received this email:

Hi,

Please see my attached resume.
I’m very intelligent and creative. I have a very eclectic arsenals of skills for the solution of problems.
I’ve worked in numerous startups, including several of my own.

Reed

The sad thing is, I get an email like that just about every day. I thought I would share my response in hopes that it will help someone from making the same mistakes.

We’re not hiring right now, but here are a few free tips:

“I’m very intelligent and creative.” doesn’t come off as confident, it comes off as cocky

If you had spent 2 minutes looking at our site you would have known that my email address is josh@eventvue.com not careers, not jobs… just josh.

No mention about what excites you about EventVue? Keep in mind I get several resumes in my inbox EVERY DAY. It’s not hard to get my attention. Comment on my blog. Send me an engaging question. @me on twitter. I’ll respond. Just don’t send me something that has been copied and pasted to a dozen different companies.

“FW: about me” is your subject line? I’d work on that one a bit.

We’re a startup trying to build cutting edge stuff. The fact that you sent me an email from a Hotmail account communicates that you aren’t much of an early adopter. That’s too bad, because I bet you’re a smart guy.

I understand that startups are different. Your career center probably didn’t tell you this stuff. That’s why I am.

Update. Reed responded:

Don’t worry I’m very creative and intelligent. It’s not a boast. It’s who I am.

If you read my resume then you know that I’m also an internationally known composer.
I can write top level music in any style you can think of, including the most modern remix and such.
I attached a song from one of my Cds. All my Cds have been in or near the 10 ten the country on jazz radio.

I have not only hotmail but gmail and facebook and twitter and others.

Anyway, I’ll keep your advice in mind.

I’ll check out your blog.

Not sure that helped, but at least now I know he’s a good composer. The music was pretty.

How to use variable variables in PHP

2009-05-31T22:19:46Z

One of the biggest time-savers in PHP is the ability to use variable variables. While often intimidating for newcomers to PHP, variable variables are extremely powerful once you get the hang of them.

Variable variables are just variables whose names can be programatically set and accessed. For example, the code below creates a variable called $hello and outputs the string “world”. The double dollar sign declares that the value of $a should be used as the name of newly defined variable.

<?php

$a = 'hello';

$$a = 'world'

echo $hello;

?>

When I started with PHP about 10 years ago, everyone was still using global variables. That meant that anything you passed as a GET variable could be used as a local variable. It was very convenient, but unfortunately not very secure. For me, typing $HTTP_GET_VARS['count'] just wasn’t as fun as being able to use $count. I found myself adding long declaration lists to the top of my files that did nothing but convert my GET/POST variables to local variables. My code started to look like this:

<?php

$salutation = $HTTP_GET_VARS['salutation'];

$fname = $HTTP_GET_VARS['fname'];

$lname = $HTTP_GET_VARS['lname'];

$email = $HTTP_GET_VARS['email'];

...

?>

Do that for a couple dozen variables and you’ll start telling yourself there has to be a better way. Nowadays you can use $_GET instead of $HTTP_GET_VARS, but the better solution is to use variable variables. Now my code looks more like this:

<?php

// create an array of all the GET/POST variables you want to use

$fields = array('salutation','fname','lname','email','company','job_title','addr1','addr2','city','state',

                'zip','country','phone','work_phone');

// convert each REQUEST variable (GET, POST or COOKIE) to a local variable

foreach($fields as $field)

    ${$field} = sanitize($_REQUEST[$field]);

?>

This has several benefits. I reduced 14 lines of code down to 3. I now have one place to sanitize all my external input. And if I ever decide to change a variable name, I have one less place in my code to fix.

This benefit of this technique increases as you use the $fields array throughout your code. I now utilize the $fields array when saving my form data to the database. I use it for loading existing user values from the database. I use it for passing my form fields back to smarty:

<?php

$form = array();

foreach($fields as $field) 

    $form[] = $_REQUEST[$field];

$smarty->assign('form',$form);

?>

Variable variables have become one of my favorite features of PHP. They’ve allowed me to tighten up a lot of my code and made it a lot more maintainable.

Have you done anything cool with variable variables? What other PHP tricks have revolutionized the way you write code?

The protocols powering the real-time web

2009-05-26T05:59:40Z

In the past few weeks there has been a lot of discussion around the rise of the real-time web, including posts from TechCrunch, GigaOm, ReadWriteWeb and Scoble. A lot of the talk has been around Twitter, Facebook, Friendfeed, OneRiot and of course Google. You don’t have to be a genius to figure out that real-time is the future of the web. I believe there is a huge need for the tech community to develop new protocols that will power this fundamental shift in how web apps work.

The problem is our existing protocols are request driven instead of event driven. The web we know and love wasn’t built with real-time in mind.

Tim O’Reilly sent a tweet from OSCON08 that really captures the essence of the polling problem:

On monday friendfeed polled flickr nearly 3 million times for 45000 users, only 6K of whom were logged in. Architectural mismatch. #oscon08

At EventVue we have a dedicated server that does little more than poll for new blog posts from attendees. We have a few tricks to reduce the pain, but we’re still polling thousands of blogs every hour even though 99% of them haven’t added any fresh content since the last time we checked. With blog posts, people are used to having a small delay before they show up in Google Reader or other services. We’re not so forgiving when it takes 30 minutes for a tweet to show up in a client application, even though getting real-time data from twitter using polling is virtually impossible.

So what is the solution?

Some people have said that XMPP holds the answer, but how many developers do you know who have set up an XMPP server before? Right. Me too. XMPP may be a viable transport method but I think we’d be better off using something that is simpler and more familiar to developers.

Another prominent response to the polling problem is the Simple Update Protocol (SUP) that was proposed by Paul Buchheit from Friendfeed. SUP is certainly an improvement over our current protocols, but what frustrates me is that it only reduces polling instead of eliminating it altogether. It may make sense for FriendFeed, but it’s not something I would add to my blog.

My favorite approach is PubSubHubbub that was proposed by Brad Fitzpatrick and Brett Slatkin from Google. PubSubHubbub might have a horrible name, but the protocol is exactly what we need to fix our polling problems. It’s lightweight, simple to understand and built on top of basic HTTP.

PubSubHubbub is a simple extension to ATOM that uses webhook callbacks to deliver practically instant notifications between servers when a feed is updated. The protocol is decentralized and free. Anyone can run a hub. Anyone can be a publisher or a subscriber. I like that it eliminates polling altogether and is incredibly simple to implement. I took a stab at writing the PHP client library and was able to take it from protocol spec to code in less than 2 hours.

If you’re interested, you can check out my PubSubHubbub PHP library and download and install the PubSubHubbub Wordpress plugin I wrote as well.

It’s worth mentioning the role that Gnip plays in all of this. Gnip has been leading the charge against the evils of polling. I’ve been a big fan of their service and have written before how they helped EventVue. But at the end of the day, the winning technology shouldn’t be in the hands of one company — it should be open and distributed. Open protocols don’t eliminate the need for Gnip. Trusted hubs like Gnip will play an important role in handling the flow of data between publishers and subscribers. Companies will pay good money to off-load that work, and Gnip is already at the center of that opportunity. I’d love to see Gnip embrace the open protocols that are being developed and lead the drive for adoption of PubSubHubbub in particular.

I’m excited about PubSubHubbub for a few reasons. First, it opens the door for a whole new range of real-time applications that simply aren’t possible today. It’s also a chance for me to contribute to solving a really big problem and an opportunity for me to get in on the ground level of something I believe is going to be huge. I wasn’t able to contribute to the design of HTTP or sit in on the conversations that led to the development of the RSS protocol. But one day I’m going to be able to brag that Online Aspect was the very first blog on the web to support PubSubHubbub. And for a geek like me, that’s pretty cool.

How to speed up your website

2009-06-07T23:10:25Z

There are few things as frustrating as having to wait for a website to load. Not only do slow websites make for a poor user experience, they can also have a big impact on your bottom line:

Google discovered that adding 500ms to their load time resulted in a 20% loss in page views.
Amazon discovered that every 100ms they added resulted in a 1% loss of conversions.

Steve Souders is the main thought leader on how to make websites splitting fast. Steve works at Google, but before that he worked at Yahoo on an extremely useful project called YSlow. I have used a lot of his research to speed up my own projects and he’s taught me a lot of simple things that can make a big difference in web performance.

Steve recently taught a class at Stanford on high performance websites and the videos are available online. The full set costs $600, but you can watch the first 3 for free. I would recommend anyone building stuff on the web to take the time to watch and learn.

Update 06/07/09: Google recently announced their own version of YSlow called PageSpeed. It’s got some additional features that YSlow doesn’t have — like giving you optizimed images that can be saved directly from the plugin. Check it out.

Verifying domain name ownership

2009-05-15T06:36:19Z

I got a nice shout-out on TechCrunch today for discovering an issue with the new Kindle Publisher program. The vulnerability allowed anyone to claim a blog as their own and take advantage of the 30% rev-share that Amazon offers on their $1.99 subscription fee. Erick Schonfeld did a nice job covering the issue and explaining the implications of the hack. You can read about it on the TechCrunch article.

The interesting thing about this vulnerability is that there are already accepted methods in place for verifying that someone owns a domain name. I understand that Amazon may have wanted to remove the friction from getting people started, but this stuff matters too much to get wrong — especially when there is a large audience and money to be gained.

For those who are interested in the best way to do domain name ownership (ahem, Amazon) Google would be a great role model for you to follow. There is a nice explanation on how Google’s domain verification process works on their help pages:

To verify that you own a site, you can either add a meta tag to your home page (proving that you have access to the source files), or upload an HTML file with the name you specify to your server (proving that you have access to the server).

Each verification method has its advantages. Verifying using a meta tag is ideal if you aren’t able to upload a file to your server. If you have direct access to your server, you may find it easier and faster to upload an HTML file.

Amazon would do well to follow Google’s lead.

Managing code releases

2009-04-08T08:13:39Z

Recently I decided to streamline my code release process. I use subversion for my source control which means I push code live by running svn up on each of our production servers. I’m lazy, so I wanted an easier way to do this all at once. The end result is a simple shell script that lets me run svn update commands on multiple servers at once. It shows me the status of svn on each server and gives me chance to confirm that everything is okay before going ahead with the launch.

This example assumes you have two servers (app1 and app2) that are using public key authentication. Obviously, you’ll need to modify this script to work in your own environment. Make sure you replace “/var/www/” with your own document root and change appX.yourdomain.com to the IP address of each production server.

#!/bin/sh

# connect to each server and echo their current status

echo "Connecting to app1...\n"

ssh app1.yourdomain.com 'cd /var/www/; svn status --show-updates; exit'

echo "\nConnecting to app2...\n"

ssh app2.yourdomain.com 'cd /var/www/; svn status --show-updates; exit'

# add additional servers here as needed

tput smso

# confirm the release before publishing

echo "\nDo you want to publish these changes to production? (y/n)\n"

tput rmso

read answer

if [ $answer == "y" ]; then

    # if "y", proceed with the release

    echo "\nPublishing to production..."

    echo "\nPublishing to app1..."

    ssh app1.yourdomain.com 'cd /var/www/; svn up; exit'

    echo "\nPublishing to app2..."

    ssh app2.yourdomain.com 'cd /var/www/; svn up; exit'

    # add additional servers here as needed

    echo "\nDone"

else

    # if "n", cancel the release.

    echo "\nCanceled"

    exit;

fi

Too many DNS lookups in an SPF record

2009-03-29T08:39:53Z

I recently noticed I was having new email deliverability issues. It surprised me since things had been going well since switching to AuthSMTP for our outgoing mail. The first thing I checked was my SPF record. It looked like this:

v=spf1 a mx include:aspmx.googlemail.com include:authsmtp.com include:salesforce.com -all

At first glance everything seems okay. Basically it says to include all A records, MX records, and to include the SPF records provided by Google Apps, AuthSMTP and Salesforce. Since that covers every legitimate sender, I finish it off with the -all which indicates a hard fail. Ok, so the syntax is good. You can’t tell that anything is wrong without digging a little deeper. When you actually try to evaluate it you’ll get this error message:

Results - PermError SPF Permanent Error: Too many DNS lookups

After a little research I found out that you are only allowed 10 DNS lookups and fetching the TXT and SPF records count toward that total. That means after you add in the A and MX lookups, we’re at 7 before we even look inside the includes. Let’s pull up the SPF record for Google Apps:

v=spf1 redirect=_spf.google.com

That redirect counts as another DNS lookup. That puts me up to 8 DNS lookups. Thankfully the Salesforce SPF record is nice and clean:

v=spf1 ip4:204.14.232.0/25 ip4:204.14.234.0/25 ip4:63.150.46.16 ip4:207.126.144.0/20 ip4:64.18.0.0/20 mx ~all

That leaves AuthSMTP:

v=spf1 include:spf-a.authsmtp.com include:spf-b.authsmtp.com include:spf-c.authsmtp.com include:spf-d.authsmtp.com ~all

Ouch! That’s 4 more lookups and the worst part of it is that spf-d.authsmtp.com doesn’t even do anything!

The first thing I did was take out the MX lookup since it’s redundant. I also replaced aspmx.googlemail.com with _spf.google.com which is what it redirects to anyway. Technically, this isn’t a good idea since Google could change it on me — but remember I don’t have a lot of options here. I’m just happy to see my revised record pass the test:

v=spf1 a include:_spf.google.com include:authsmtp.com include:salesforce.com -all

I also sent an email to the AuthSMTP team. They responded within 30 minutes saying that they would remove the extra DNS record and look at how they can clean things up.

I learned something tonight. Remember to count the DNS lookups in your SPF record. It turns out they can add up faster than points on a teenagers drivers license. And if you’re using a lot of includes like I am, remember to do periodic checks to make sure nothing has changed.

Resources:

I wrote about Sending email through Gmail over a year ago. While I absolutely don’t recommend you try this anymore, it has some useful information on SPF records and email deliverability in general.
Kitterman have a great tool to help validate your SPF records.