Steven Benner's Blog

Will the really real programmers please stand up?

Steven Benner — Thu, 12 Aug 2010 18:00:56 +0000

How do you define a real programmer? There are many facets that you can use to judge someone’s skill as a programmer: ability, experience, enthusiasm, dedication, etc. But I recently read an article by RethinkDB blogger Salva, and he boils all of these points down to one item, memorization. Some people agree with this idea, but I do not, at all.

I have heard several companies bragging about having interview scripts like this, saying that this kind of testing is the most effective way to filter out unqualified applicants. However I think that the post by Salva perfectly demonstrates just how flawed this concept is. If there is one thing you should have learned in school, it’s that standardized tests are a terrible metric for gauging skill or intelligence.

But more to the point, I think that this is the wrong way to define a real programmer. Allow me to present you with my definition of a real programmer.

What is a real programmer?

This comes from one of my favorite quote in all of history:

“Real knowledge is to know the extent of ones ignorance” – Confucius

Having an encyclopedic knowledge of a college curriculum is not required to be a great programmer. Real world software development isn’t about memorizing low-level programming techniques, or math formulas. These are great tools to have at your disposal, but are certainly not the end-all-be-all of development. It’s all about problem solving, in the most efficient and elegant way that circumstances allow.

As a programmer you will be supplied with problems every day, and it’s your job to figure out how to solve these problems. To do this you need to know what you language of choice is capable of, this comes with experience. You will also need to be able to look at the possible solutions and pick the best one for the situation at hand.

Now here’s the fact that breaks the machine, you do not know every possible solution for every problem, and you never will. I don’t care how experienced you are, how much education you have had, or how many millions of lines of code you have written, because there are so many possible ways to approach the practical problems you find in software development that it is literally impossible to know everything. So you need to have a mix of creativity and resources that let you learn how to solve new problems.

Real programmers are the ones who can learn fast, and learn by doing. These are the people who constantly strive to keep up with the technology they love. They have ample real world experience building, learning and growing their craft, but know that they still don’t know everything.

In short the defining characteristic of real programmers is that they never stop learning.

My experience with real programmers

I’ve worked as a peon coder, link-in-the-chain guy, I’ve worked as a manager who did the hiring and firing, I’ve worked as a one-man-shop serving milti-billion dollar corporations, and I’ve worked as a solo freelancer. This experience has taught me two things:

I am not the greatest programmer in the world, despite what my website says, and
That you can rarely tell a good programmer from a bad one by the resume, or the interview.

I’ve seen guys with terrible resumes turn out to be so far above my level that I still have them on my guru list, and I’ve seen guys with amazing resumes who couldn’t grasp even the most basic concepts.

How to find a real programmer

It is true that there is often a gap between what candidates say they can do and what they’re really capable of. That is why testing candidates is still a very good idea, but you need to test for higher level skills and abilities. Testing a candidate’s knowledge on obscure techniques and patterns means passing off great people for not remembering stuff that they simply do not need to be top notch coders.

In my experience, it is very easy to separate the wheat from the chaff: go through the normal interview process to see if this is a person you can work with (and let’s face it, that’s all that the interview is really for). Then give them a small project, something realistic that resembles what their real work would be like, and send them home. Not a knowledge test, but a real “build something that does this” task.

When you get the finished result of their labor you will know if you’ve got the real deal. Is it done the way you would have expected (or better)? Is it high quality, showing some love some thought? Did they get it back to you reasonably quickly? Did they do anything cool to show off? If so, congratulations! You’ve just found a great programmer!

code_by_google != bad_programmer

Yes, they could have just coded by Google. But you know what? That shows they can learn and adapt. I have plenty of respect for a person who doesn’t immediately know how to accomplish a complex task, but can quickly find out how and do it on their own without bothering the other programmers on your team.

Remember the key point of my real programmers definition, they never stop learning. Google is the greatest programming resource that has ever existed. You cannot look down on someone for using the most powerful learning tool computer science has ever known. I would go so far as to say knowing how to code by Google is the single most important skill in a programmers tool box, because if you want to grow as a programmer this is going to be the tool to use.

Someone who knows how to search for code examples and how to learn from the work of others will be more or less self-sufficient. They can learn and grow their skills on their own without needing someone else to do it for them. The ability to learn and grow your knowledge is the single most important skill for any developer. Without the ability to grow you will find yourself quickly deprecated.

I do expect people to know how to use the language and/or framework they were hired to work in, but I judge them primarily based on the work they submit. A guy who can figure out how to do things that he doesn’t know how to do, on his own, on the fly, is a real programmer.

The catch-22 of impossibly high standards

Of course it’s pretty absurd to require knowledge that 99% of programmers have absolutely no need for in real life. But perhaps you don’t entirely know why. If you have very strict hiring standards then you are only going to find people who match your specific model, and your code will suffer because of this.

Computer science grows at a fantastical rate, every day there is another small innovation that someone came up with. This small innovation will slowly propegate out to the rest of the programming community and become part of everyone’s toolbox. This happens because some creative programmer tried some alternative idea and found a new way to solve a problem.

If your team consists entirely of people with the same background, skills and knowledge then your creativity bucket will be quite small. This means you will not see as much innovation as a varied team with people asking questions that wouldn’t normally be asked, and people offering solutions that wouldn’t normally be offered.

Most alternative idea’s will, of course, be immediately shot down. But that one in a hundred that actually sticks will give your team an advantage. This alternative idea has given a level of innovation to your project that would not have existed if you didn’t have a radical element thinking in a unique way.

Get real

This was a rather long-winded rant, but let me sum it up in a few nice bullet points.

People who are more interested in the buzz words and CS theory than actual experience and a history of practical application of skills are a perpetual thorn in the side of the programming industry.
Don’t look down on other programmers because they don’t fit your model definition of a programmer, they just might be better than you.
If 99 out of 100 candidates fail your interview then you are looking for something that doesn’t exist.
Without a varied group of developers in your team you will suffer from a lack of creativity.
Don’t test a CS curriculum, test the ability to create.
If you ever finish learning, then your career as a programmer is over. Go study law.

In short

Look for someone who truly understand the concepts, who can offer creative and alternative ideas, and who shows the ability to grow as a programmer. Then you will finally find a real programmer.

The 5 types of programmers

Steven Benner — Sun, 18 Jul 2010 22:11:45 +0000

In my code journeys and programming adventures I’ve encountered many strange foes, and even stranger allies. I’ve identified at least five different kinds of code warriors, some make for wonderful comrades in arms, while others seem to foil my every plan.

However they all have their place in the pantheon of software development. Without a healthy mix of these different programming styles you’ll probably find your projects either take too long to complete, are not stable enough or are too perfect for humans to look upon.

The duct tape programmer

The code may not be pretty, but damnit, it works!

This guy is the foundation of your company. When something goes wrong he will fix it fast and in a way that won’t break again. Of course he doesn’t care about how it looks, ease of use, or any of those other trivial concerns, but he will make it happen, without a bunch of talk or time-wasting nonsense. The best way to use this person is to point at a problem and walk away.

The OCD perfectionist programmer

You want to do what to my code?

This guy doesn’t care about your deadlines or budgets, those are insignificant when compared to the art form that is programming. When you do finally receive the finished product you will have no option but submit to the stunning glory and radiant beauty of perfectly formatted, no, perfectly beautiful code, that is so efficient that anything you would want to do to it would do nothing but defame a masterpiece. He is the only one qualified to work on his code.

The anti-programming programmer

I’m a programmer, damnit. I don’t write code.

His world has one simple truth; writing code is bad. If you have to write something then you’re doing it wrong. Someone else has already done the work so just use their code. He will tell you how much faster this development practice is, even though he takes as long or longer than the other programmers. But when you get the project it will only be 20 lines of actual code and will be very easy to read. It may not be very fast, efficient, or forward-compatible, but it will be done with the least effort required.

The half-assed programmer

What do you want? It works doesn’t it?

The guy who couldn’t care less about quality, that’s someone elses job. He accomplishes the tasks that he’s asked to do, quickly. You may not like his work, the other programmers hate it, but management and the clients love it. As much pain as he will cause you in the future, he is single-handedly keeping your deadlines so you can’t scoff at it (no matter how much you want to).

The theoretical programmer

Well, that’s a possibility, but in practice this might be a better alternative.

This guy is more interested the options than what should be done. He will spend 80% of his time staring blankly at his computer thinking up ways to accomplish a task, 15% of his time complaining about unreasonable deadlines, 4% of his time refining the options, and 1% of his time writing code. When you receive the final work it will always be accompanied by the phrase “if I had more time I could have done this the right way”.

Where do you fit?

Personally, I’d have to classify myself as the perfectionist. So, which type of programmer are you? Or perhaps you know another programming archetype that is missing from my list? Post a comment below and I’ll add it to a new updated list.

Will the really real programmers please stand up?

Setting up an Ubuntu LAMP server. Part 1: Initial setup

Steven Benner — Sun, 27 Jun 2010 23:23:43 +0000

This will be the first part in a new series of articles about setting up your own Ubuntu LAMP (Linux, Apache, MySQL, PHP) server. If you are already proficient with Linux and/or Ubuntu then you can skip this first part because it is just about the basic post-install set up and configuration. For those of us who haven’t set up an Ubuntu system a hundred times over, here is the quick and dirty base Ubuntu set up guide.

Once you have your Ubuntu operating system installed there are still a few things you need to do to get it ready for prime time. A fresh install of Ubuntu will have a root account (you chose the password during installation) and the SSH server running. At the moment my recommendation for version is 10.04 LTS.

Without further ado, lets get this show on the road. Start by connecting to your system via SSH and logging in to the root account. (For Windows users I recommend PuTTY as an SSH client)

Create your account

The first thing to do is to create your account. Logging in as root is bad and you should never do it after you create a personal account. Create a new account with the adduser command.

adduser steve

Of course use whatever name you like for your account. It will ask you for various pieces of account information, all of which is optional except for the password. Be sure to use a strong password, this is the single most important piece of security for the entire system. If you choose a dictionary word for your password then it is almost inevitable that sooner or later, someone will find it and destroy your server.

Give your new account sudo access

The reason you never have to log in as root is because your account will have super-user access via the sudo command. Sudo will let you execute other commands at nearly the same level as the root account. Open the sudo configuration file with the visudo command.

visudo

This will open a text file, at the bottom of that file add the following line (replacing “steve” with your account name).

steve ALL=(ALL) ALL

Now save the file and exit the text editor. Chances are this file was opened in nano, so you exit by hitting CTRL+X on your keyboard. It will ask you if you want to save the changes, say yes by typing “y” and hitting the enter key.

Configure the SSH server

We need to add your new account to the SSH server so you can log in under that name. We will also do a couple minor tweaks to make the server much more secure. Nothing too advanced, just enough to get your SSH of the easy attack list. Open the SSH server configuration file in nano.

nano /etc/ssh/sshd_config

Here are the thing we need to do in this file:

Change the port to something other than 22. If you leave SSH running on port 22 it will constantly be under attack by botnets and hackers that try to find SSH logins for every IP on the internet. This little trick will save you from millions of automated hack attempts.
Disable root login. We do not want anyone (even you) to be able to SSH in as root. It is a huge security risk because root can do anything to the system.
Disable X11Forwarding. X11Forwarding is another security risk for numerous reasons. You simply don’t need it so disable it.
Add your account to the list of users that can SSH in. The SSH server will only allow the log in of accounts that are added to the configuration file.

First and foremost, lets make sure your new account can log in via SSH. Add the following line to the file (I usually just add this at the bottom).

AllowUser steve

Again, replace “steve” with your account name.

Now for the security stuff. Find the following config statements and update them accordingly:

Port 12345
PermitRootLogin no
X11Forwarding no

Remember your port number! You will need it to SSH in to your server again.

Use any port number you like, I would recommend 4 or 5 digits because port sniffers start at 1 and scan up from there. So a large number will take much longer to find. This will thwart most lazy port sniffers. Do not use any of the standard service ports (e.g. 23, 24, 25, 80, etc). For a list of standard port numbers check out this Wikipedia list. Basically any port number greater than 1024 should be okay to use. For better security use something 10000 or above.

Now reload the SSH server by entering the following command:

/etc/init.d/ssh reload

Now you can logout and log back in using your new account.

logout

SSH back in to your server and log in with your new account. You should now be logged in as yourself in your home directory. For now on when you need to execute something that requires super-user permissions you will do so via the sudo command.

Update all of the installed software

The next step is to get all of the software on your system up to date. This is one of the best things about running Ubuntu, updates are incredibly easy using the aptitude tool. Frankly, this is even easier than Windows Update, and far more reliable.

Update the packages list with the following command:

sudo aptitude update

This grabs all of the latest package information from the Ubuntu servers for everything you have installed.

Note that when you use sudo it will ask for your password. This is just the password for your account login. It will save your sudo session for a few minutes, so if you need to execute several sudo commands it will only ask for your password the first time you run it.

Now you can upgrade everything that needs updates with the safe-upgrade command.

sudo aptitude safe-upgrade

It will list all of the packages (programs) that it wants to update and ask if you want to proceed. Enter “y” for yes and hit enter. The aptitude program will automatically download and upgrade everything that needs to be updated.

You should do this as often as possible, once or twice a week is more than enough if you’re anal about keeping everything updated with the latest fixes and security updates.

Install the base software

There are a couple software packages that you will almost certainly need at some point, no matter what you do with your server. So let’s go ahead and install those now.

Lets start with wget, the tool you will use to download files from the web to your server. It’s just a simple file download tool that you will be using.

sudo aptitude install wget

The next item on the list is the build-essential package. It includes all of the software you need to build Debian/Ubuntu software packages on your machine. Most of the software you will be using will be downloaded pre-built and auto installed by the aptitude software, but if you need to build Apache modules or any other non-standard pieces of software you will require all of the suff in the build-essential package.

sudo aptitude install build-essential

This will download and automatically install everything in this package.

Useful commands you should know

Here are some of the most useful commands you should know for managing your system.

sudo !!: This is one of the best little tricks to know. When you try to run a command but get a permissions error because you needed to sudo it, then you can run “sudo !!” which will execute the last command under sudo. This will save you lots of typing.
free -m: The free command will display current memory usage information. This will be an invaluable tool for optimizing as well as diagnosing problems. The -m flag tells it to display the numbers in megabytes, instead of just bytes.
df -h and du -h: The df and du commands report disk usage information. The df (disk free) command reports how much disk space is used and available on each partition. The du (disk usage) command reports how much space all of the files and folders in the current directory are taking. Both commands should be run with the -h flag which tells it we want human-readable (xxxKB, MB, GB) numbers.
top: Use the top command to find out information about the processes that are currently running. This app shows a constantly refreshed list of the processes with memory utilization, CPU usage, time running and such. It’s a very useful tool for finding out which processes are taking the most CPU and memory. There are a lot of feature in this one so read the man page. Press “q” to exit the app.
man: Of course real men don’t read instructions, so I don’t know why I’m listing this here. Perhaps it’s all of the times other real men have told me to RTFM. Anyway, man is the manual tool. For example entering “man top” will display the manual for the top application.

Conclusion

That’s it. You now have an Ubuntu server running the latest software with the basic best practices for accounts and SSH security. You can go anywhere from here, set up a LAMP server, a mail server, a game server, whatever. Look forward to the next part of my Ubuntu server series where I’ll discuss installing and configuring the web server.

Ultimate web site optimization trick: data URIs

Steven Benner — Tue, 18 May 2010 04:15:46 +0000

Are you an obsessive compulsive web site optimization nut? Are you willing to sacrifice maintainability for one less HTTP request? Have people ever asked you to seek a mental health professional because of your OCD? If so, then you are in good company. Us optimization nuts may not be very well received when we start building a project, but we become very appreciated when someone asks how to make it scale. Depending on who you ask, web optimization is either the most useless talent ever or the single most important skill on a web developers resume.

Allow me to broaden your optimization toolbox with the ultimate HTTP connection killer, the data URI scheme. Most web developers have never heard of data URIs but they can dramatically reduce the number of HTTP connections required to download your web site.

This article will explain what data URIs are, how to use them, and how to properly implement them.

What exactly are data URIs?

The data URI scheme (aka data: URLs) is a method to include data in-line in a URI.

I don’t suppose that makes much sense, let me elaborate; Simply put, data URIs allow you to include a file (or multiple files) inside of another file. The most obvious and practical use is to embed background images inside a CSS file. You do this by base64 encoding a file and embedding the contents, with some basic meta data, into a URI.

It is important to note that while the data URI scheme was created in 1998 it never reached the status of Standard Protocol. It is still on the list of “Proposed Standards”. However, all of the most popular modern browsers have implemented the data URI scheme.

Due to the IE problem (explained further down) data URIs can only be fully supported in CSS files.

Data URI scheme

The format is pretty straight forward, the official data URL scheme as defined in the RFC is:

data:[][;charset=""][;base64],

Okay, let me break that down for you and provide some real world examples.

Like any URI it starts with a protocol identifier, data. Just like http, ftp, mailto or gopher this tells the browser how to use the following information.

The first piece the browser needs to know is the MIME type of the data you are including (e/g image/png or text/html). Over normal HTTP transfers the server identifies the MIME type from the file extension and sends it back with the file in the response headers. Since we are not transferring the data through an HTTP connection there is no automatic MIME-type data, and we have to manually identify the type of data we are giving to the browser.

Next is the charset identifier, which is exactly what it sounds like. If you are passing any kind of text document (e.g. plain text, html, xml) then you should tell the browser what kind of character encoding your are using for that text (e.g. UTF-8 or US-ASCII).

And the last piece of possible meta data is the base64 keyword. This tells the browser that the following data is base64 encoded.

All of this meta data is optional depending on the data you are embedding. If you are passing an image you won’t include the charset information, likewise if you’re passing a text document then your probably wont be encoding it and will omit the base64 keyword.

Useful examples

Here are a few examples of data URIs at work.

CSS background

In the real world, CSS backgrounds are about the only place where data URIs can be used (see: IE problem below). In my opinion this is the single best use for this technique anyway. We can embed all of our icons and backgrounds inside of the very style sheet that defines the rules that use those images.

#logo {
	background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEkAAABJCAYAAABxcw[*about 2000 more characters*]AAAABJRU5ErkJggg==');
}

JavaScript

If you don’t have to worry about old versions of IE then you can use data URIs anywhere, even in JavaScript for any kind of URL destination. If you need to open a new window with a basic layout but don’t want to host an otherwise useless file then embed the layout in a data URI.

window.open('data:text/html;charset=utf-8,%3C%21DOCTYPE%20html%3E%0D%0A%3Cht' +
	'ml%20lang%3D%22en%22%3E%0D%0A%3Chead%3E%3Ctitle%3EEmbedded%20Window%3C%' +
	'2Ftitle%3E%3C%2Fhead%3E%0D%0A%3Cbody%3E%3Ch1%3EDATA%3C%2Fh1%3E%3C%2Fbod' +
	'y%3E%0A%3C%2Fhtml%3E%0A%0D%0A', '_blank', 'height=300,width=400');

Embedded images

If you don’t need to support IE 6/7 then you can embed an image inside of the image tag itself with a data URI. This probably isn’t very useful, but it is a good demonstration of the technique.

Embedded links

Even Internet Explorer 8 won’t let you embed data in links because of security considerations, but it is another good demonstration of the technique.


	Link to Twitter logo

The Internet Explorer problem

As usual, Microsoft Internet Explorer screws up everything, data URLs are not supported in any version prior to Internet Explorer version 8. As much as I may want to add this to the hate list, I honestly cannot fault Microsoft for not supporting this until version 8. As I said earlier the data URI scheme is still on the list of Proposed Standards. So the fault rests entirely on the shoulders of the W3C for not looking at this RFC for the last 10 years.

In addition to the problem of old versions, Internet Explorer 8 has some additional restrictions.

data must be smaller than 32KB
data URIs cannot be used for JavaScript
only object, img, input and link HTML tags can use data URIs

However, you can use data URIs in any CSS url statements, so there is still plenty of use for data URIs in the latest version of Internet Explorer.

Graceful degradation options for older versions of IE

There are two options for graceful degradation:

Make regular images available and use regular url links as a fall-back.
Or, use the MHTML technique.

Personally, I’m not a fan of the MHTML technique because it will greatly increases the size of your CSS file (doubling the number of embedded images). In my opinion, this is just too high of a price to pay. It is far simpler to make the images available on your server and just link to them with regular CSS url statements. At least it is for me, if for some reason you would rather take the file size hit instead of publishing the image files then there is a good MHTML how-to article here.

For the this article I will use CSS url fall-backs for graceful degradation.

IE 6-7 specific CSS degradation

There are two ways to implement your CSS fall-back statements to target IE 6 and 7:

Use Internet Explorer revision targeting with conditional comments
Or, use the asterisk CSS hack.

Both of these are good choices, with simple pros and cons. IE revision targeting with conditional comments will require an additional CSS file specifically for older versions of IE, but in return it allows you to craft a style sheet that validates. Whereas the asterisk hack saves you the trouble of having an extra file but creates an invalid CSS statement.

Personally, I am anal about writing valid CSS and often have a separate CSS file for IE anyway (for IE PNG fix behavior statement) so I use IE revision targeting statements.

I’ll leave the choice up to you, so here are examples of both techniques:

Internet Explorer revision targeting

I’m sure you seen this technique countless times before, but here it is again. Internet Explorer supports version targeting statements in HTML comment blocks. These conditional comments are quite smooth because they produce valid markup and guarantee, for all time, that your code will only be parsed by the versions of Internet Explorer that you target.

The following snippet demonstrates how you include a CSS file that will only be seen and downloaded by Internet Explorer versions 7 and older:

In this example base.css is seen by all browsers but old_ie.css is only seen by IE 7 and older. Place this code in your HTML head block and the statements you place in old_ie.css will override the rules from base.css in old versions of Internet Explorer. For example:

base.css

#logo {
	background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEkAAABJCAYAAABxcw[*about 2000 more characters*]AAAABJRU5ErkJggg==');
}

old_ie.css

#logo {
	background-image: url(/images/logo.gif);
}

Now when a modern browser visits the page they will properly download and use the embedded background image but when an old version of IE visits the site it will use the rule from old_ie.css.

The asterisk hack

A less well known technique for targeting old versions of IE is the so called Asterisk Hack. This is a nifty little trick that someone, somewhere figured out. If you prepend an asterisk to a CSS rule then that rule will only be processed by Internet Explorer versions 6 and 7.

Here is an example of this hack:

#wrapper {

	/* Processed by all browsers */
	background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEkAAABJCAYAAABxcw[*about 2000 more characters*]AAAABJRU5ErkJggg==');

	/* Only seen by IE 6 & 7 */
	*background-image: url('/images/graphic.png');

}

Simple! Unfortunately this is invalid CSS, which irks me, but it is a clean and efficient hack. Only IE 6 and 7 will see the second background statement, and since it comes after the default data URI statement, it will override it with a link to the image.

How to get encoded files for data URIs

I built a little tool that allows me to quickly encode files on my computer. Here is the download if you would like to use it. This isn’t my finest work, it is just a basic tool to do the job in 20 lines of code, don’t expect a lot of bells and whistles.

Steve’s Encode64 Utility

Click here to download

This tool is for almost any version of Microsoft Windows (sorry Mac users).
There is no file size check so be careful. If you try to process a big file it will hang your computer for some time, or lock up entirely. Files less than 32KB (IE8 limit) will be fine.
You can get the source code here: Steve’s Encode64 Utility Source Code (VS 2008 C#)

How to generate base64 strings in code

If you need to dynamically generate base64 strings then you’re in luck, this task is quite simple. Here are a couple code examples to get you on your way:

PHP

$encoded_string = base64_encode(file_get_contents('logo.png'));

C#

private string Base64Encode(string fileToEncode)
{
	byte[] EncodeBuffer;

	using (FileStream fs = new FileStream(fileToEncode, FileMode.Open))
	{
		int Length = Convert.ToInt32(fs.Length);
		EncodeBuffer = new byte[Length];

		fs.Read(EncodeBuffer, 0, Length);
	}

	return Convert.ToBase64String(EncodeBuffer);
}

Conclution

I think data URIs are an awesome, and powerful optimization trick. You can save many extra HTTP connections, and the associated latency and overhead for all of your users who are running a modern browser. This will make your site much snappier, especially if you use a lot of icons.

Using these techniques you can have a fully compatible web site that will gracefully degrade to traditional external resources if you can accept the following restrictions:

Only images
Only in CSS files
Less than 32KB each

I can see this working well for background images and for icons. With data URIs you can embed all of your presentation resources into the same CSS file that uses them. This file will grow much larger than normal but it will cache well and save in overall bandwidth.

How to circumvent internet filtering and enhance privacy

Steven Benner — Sun, 09 May 2010 23:30:36 +0000

The whole world knows about the so called “Great Wall” that protects the Chinese people from dangerous western ideals, the serious concerns about internet privacy in the United Kingdom, now we have Australia making a real push towards internet filtering legislation as well as all of North America and most of Europe signing on with the ACTA legislation. Of course we still have Iran blocking news, social media and even gmail. But even countries that we wouldn’t expect to support censorship are signing bad legislation, such as France, New Zealand, and Ireland.

For at least a decade the very idea of filtering information was something first world countries laughed at as a tactic only dictatorships and xenophobes would seriously consider, now it seems the stage is changing and governments and ISPs are becoming more open to the idea of national censorship.

This is a terrible, and dangerous trend. I for one hope the aussies block the forced internet filtering law and I hope the whole world sees ACTA as the scam that it is. There is a quote that I often like to cite when the topic of internet filtering comes up, “The internet sees censorship as damage, and automatically routes around it”. Unfortunately, when censorship happens at the ISP level, that automatic routing is broken, so you have to do some manual routing to circumvent the broken tubes.

I think it is important to know how to get around censorship schemes and I believe that the greater the number of people with this knowledge the better off the internet, and the world will be. So here’s how to circumvent internet censorship, the VPN.

In several countries it is illegal to even attempt to circumvent government authorized internet censorship. You can and will go to jail if you get caught bypassing official filters in such countries. As much as I want to promote internet free speech I do not want any of my readers to go to jail. Please read up on your countries laws regarding net filtering and, of course, I do not advise you to attempt anything illegal.

Disclaimer: This site is not intended for distribution to, or use by, any person or entity in any jurisdiction or country where such distribution or use would be contrary to local law or regulation.

VPN tunnel through a country that supports internet freedom

The idea is simple, do your internet surfing through a VPN tunnel that ends in a country that has a good record of supporting a free internet and protecting privacy (such as Sweden).

What is a VPN?

A VPN (or Virtual Private Network) is a way to securely extend a private network across the internet to another location. The client computer (in this case your computer) makes an encrypted connection to a server which acts as a normal network connection. This technique is commonly used at tech companies to allow employees to connect to their work network from anywhere in the world so they can access files on the corporate network or even open a remote desktop connection to their office computer.

This system is very secure. Since VPN users don’t want any nefarious types or compromised systems along the way to be able to sniff out what files you are reading or what information you are sending, hence “Private”. When a VPN connection is open it is just like having an Ethernet cable connected to the other machine, just slower since it is going over public networks.

Sounds great! So how do I get a VPN?

It’s not free, but it is cheap. There are actually several companies offering this service already, such as PRQ which offers an OpenVPN tunnel service for as little as $10 per month. There, your VPN server is all ready to go! I mention PRQ only because they are famous for protecting their customers, for more VPN providers check out the VPN list on the Internet censorship wiki.

You can also set up a VPN server of your very own anywhere you like, but you will need to find a host to house the system and keep it online. I highly recommend a VPS (Virtual Private Server) system for this because it is cheap, fast and you don’t need much computing power to run a VPN. Of course a dedicated server is fine too, but a lot more expensive. No matter what you will need a system that gives you root access so you can install and configure the VPN server.

Alright, I have my VPN server! Now, how do I use it?

Making a VPN connection is easy with OpenVPN, and even easier with a Windows Server VPN. For this article I’ll assume you are running OpenVPN because a Windows Server VPN would be prohibitively expensive for a home user who just wants to browse the internet in peace. I will also assume you are using the Microsoft Windows operating system (because that’s what I use). If you are a Mac or Linux user you’ll find similar guides with a quick Google search.

The first thing you need to do is to download a copy of the OpenVPN GUI for Windows. This is a free application and the Installation Package includes everything you need, even the base OpenVPN software.

If you are getting your VPN access from a company that provides OpenVPN services then you should receive all of your configuration information from them. If you’re setting up your own VPN server then you will need to read the OpenVPN documentation for your particular system.

The guide for setting up your VPN will be the topic of another article, I’ll write up a nice how-to with screen shots and such eventually. For now, if you’re looking fo a good walk-through on setting up the OpenVPN GUI check out How to configure OpenVPN.

The other option

I believe that a VPN system like what I have described is the best solution because it is fast and nearly impossible to eavesdrop on, however it isn’t the only solution.

The simplest option is the good old fashioned proxy server. A proxy server, as it’s name implies is a server that bounces internet traffic from your system, through the proxy server, to the destination. Setting up your computer to work with a proxy server is easy, every web browser has a configuration section for proxies. They are free and readily available, however they have some serious drawbacks. Proxies are notoriously slow, often have injected ads, offer little to no security and do not encrypt the connection.

There are more advanced proxy systems such as Tor and picidae that aim to remedy some of those problems. However in general proxies are designed to enhance privacy, not circumvent filtering.

In completely cut off countries there is the sneakernet phenomena. A sneakernet is literally a network of people who pass media around by hand, such a CDs and flash drives. This is the last line of defense for free speech because it doesn’t use any computer network to transmit data. However I understand that the latency is quite poor.

Why ACTA is bad for the internet and you

Since I’m American and the only thing really threatening American internet is the upcoming ACTA treaty. So I feel that I must add a note here about ACTA.

ACTA (The Anti-Counterfeiting Trade Agreement) is an international agreement being pushed through many nations. There are some parts of ACTA that are good, basically the anti-counterfeiting portions, however most of ACTA is about intellectual property. In fact ACTA, if it is accepted, will put forth some of the most sweeping international copyright policies ever made.

Primarily backed by media giant and organizations such as the infamous RIAA and MPAA, ACTA will force countries to enact legislation very similar to the American DMCA (Digital Media Communications Act) except with even more stringent rules and harsher penalties.

Quick ACTA facts:

ACTA will require ISPs to permanently sever users internet connections if they get 3 copyright infringement complaints.
Complaints need not have any evidence nor see a court of any kind, all it takes is a complaint.
Several sections of the ACTA draft show that rightsholders can obtain an injunction just by showing that infringement is “imminent,” even if it hasn’t happened yet. (source: Ars Technica)
ACTA will allow border agents to search and seize any equipment that they believe may contain copies of copyrighted material. (e.g. laptops, iPods, hard drives, etc.)
ACTA would ban “the unauthorized circumvention of an effective technological measure.” No-CD cracks will become illegal.

Basically, ACTA is a very bad thing. It sacrifices a great deal of your privacy and threatens to get you permanently disconnected from the internet without any evidence or legal action.

Stop ACTA, support the EFF and countless human rights organizations.

Support free speech, privacy and net neutrality

Now that you have the power to get around censorship I want you to help prevent censorship. Take action, join free speech groups, donate to influential organizations, write your government representatives and vote for candidates who support free speech. It is important to know how to get information even when the powers that be try to stop you, but it is even more important to work with those powers to make the world a better place.

Support the watchdog organizations

Here are some influential organizations that are fighting for internet freedom:

Learn more

These are a few informational sites on topics of freedom of speech and internet censorship:

Calculate page size and view port position in JavaScript

Steven Benner — Sun, 25 Apr 2010 23:45:50 +0000

Have you ever had to try to figure out the dimensions of a page or browser in JavaScript? This task seems very simple at first glace, every browser has simple properties that will give you this information. Unfortunately, not every browser agrees what object these properties belong to, or even what the names are.

I’ve built a couple nifty tools in JavaScript that need to know the exact dimensions of the document as well as the exact dimensions of the browser view port. Since these scripts are uses on very high traffic sites with a very wide audience I have had to make sure they work in a multitude of browsers.

In this article I will show you how I support all modern (and not so modern) browsers with one function and I will explain a little about exactly what techniques I’m using to accomplish this.

When do you need to know page and view port dimensions?

Basically, whenever you are going to dynamically place absolute positioned elements on the document. I have built several JavaScript features that needed to know this information.

One good example is a Lightbox/Thickbox style image viewer. You need to place a box in the exact middle of the view port and have the ability to shrink it down to fit within the view port constraints.

An even better example was a pop up image thumbnail feature. When you put your mouse over certain links a thumbnail pops up to show you a picture or the logo of the link you’re about to click on. This image follows the mouse around, hanging just below and to the left of the cursor. Since I could never say for sure where the links will be on the page I have to know when the flip the image to the other side of the mouse to keep it from extending past the view port.

Dimensions needed for placement

To do any kind of dynamic absolute placement you need to know what your constraints will be. These bounds are generally going to be the browser view port, which is literally the browsers current view of your document, but it is a little more complex than that.

Think of the web browser as a window and the document as a paper underneath it. When you scroll down a page you are simply moving the document up in the view port. This means that if you want to position something based on the view port you will need to consider where the view port is on the page.

Horizontal bounds are simple (assuming you aren’t dealing with a very small browser), your constraints will be X=0 to X=View Port Width. Vertical bounds are a little more complicated. To compute vertical bounds on a page you have to factor in the current scrolled position of the view port in the document. So your vertical constraints will be Y=View Port Offset to Y=View Port Offset + View Port Height. Simple.

However those formulas are missing one critical piece that most browsers do not give you, scroll bar width. Most browsers consider the scroll bars to be inside of the view port so they are not factored in to the view port dimensions that the browsers will give you. If you create an element and push it in to a scroll bar it will increase the size of the document as the browser renders it, this leads to the “infinite scrolling phenomena”. Unfortunately the width of the scroll bars depends on the operating system.

It is basically impossible to compute the size of the scroll bars in such situations, so I picked an arbitrary number that seems to have appropriate padding on every systems that I’ve tested, 17 pixels.

Taking the 17 pixel padding into consideration our bounding formula now looks like this:

X = 0 to X = View Port Width – 17
Y = View Port Offset to Y = View Port Offset + View Port Width – 17

The browser properties

Basically every browser except Internet Explorer agrees where the dimensions information is located. Can’t say that I’m particularly surprised, Microsoft has made it their mission to complicate these things.

For a nice visual list of available properties and browser support check out the W3C DOM CSS Object Model Compatibility article on QuirksMode. It’s a very nice list of many useful properties and methods as well as a chart of what browsers support them and how effective their support is.

View port dimensions

I found three possible objects that contain the view port size information. In most browsers the self object has the properties of innerWidth and innerHeight. However Internet Explorer has two other objects, IE6 keeps this information in the document.documentElement object and all new IEs use the document.body object. These objects have a clientWidth and clientHeight property.

Document dimensions

Once again, I found three different ways to get the document measurements. Most browsers agree that this information is stored in the document.body object. However Firefox’s best measurement of total document height seems to be window.innerHeight + window.scrollMaxY. The rest of the browsers respond to the scrollWidth/scrollHeight or offsetWidth/offsetHeight.

Scroll offset

Last but not least is the scroll offset, and again three different possible locations for this information. The general consensus is that scroll offset can be found in the pageYOffset and pageXOffset properties of the self or window objects. However IE uses scrollTop and scrollLeft properties on the document.body or document.documentElement objects.

Short documents

One scenario that we need to be able to handle is that of a small document. If the document is smaller than the view port, such as a short page that doesn’t extend further down past the browser view, then we don’t want to use the document as our bounding constraints, we want to use the view port.

We do this by adding a simple test to see if if the view port height is greater than the document height and another test to see if the view port width is greater than the document width. In each case if it is greater then use the view port dimension instead of the document dimension.

The code

Here is the JavaScript code that I came up with. It is fairly short and simple and uses object detection, because you should always check for objects, never check for browsers.

This code is based on the work posted on QuirksMode some 6 years ago. The article has long since disappeared so I can’t give a better credit.

// Page Size and View Port Dimension Tools
// http://stevenbenner.com/2010/04/calculate-page-size-and-view-port-position-in-javascript/
if (!sb_windowTools) { var sb_windowTools = new Object(); };

sb_windowTools = {
	scrollBarPadding: 17, // padding to assume for scroll bars

	// EXAMPLE METHODS

	// center an element in the viewport
	centerElementOnScreen: function(element) {
		var pageDimensions = this.updateDimensions();
		element.style.top = ((this.pageDimensions.verticalOffset() + this.pageDimensions.windowHeight() / 2) - (this.scrollBarPadding + element.offsetHeight / 2)) + 'px';
		element.style.left = ((this.pageDimensions.windowWidth() / 2) - (this.scrollBarPadding + element.offsetWidth / 2)) + 'px';
		element.style.position = 'absolute';
	},

	// INFORMATION GETTERS

	// load the page size, view port position and vertical scroll offset
	updateDimensions: function() {
		this.updatePageSize();
		this.updateWindowSize();
		this.updateScrollOffset();
	},

	// load page size information
	updatePageSize: function() {
		// document dimensions
		var viewportWidth, viewportHeight;
		if (window.innerHeight && window.scrollMaxY) {
			viewportWidth = document.body.scrollWidth;
			viewportHeight = window.innerHeight + window.scrollMaxY;
		} else if (document.body.scrollHeight > document.body.offsetHeight) {
			// all but explorer mac
			viewportWidth = document.body.scrollWidth;
			viewportHeight = document.body.scrollHeight;
		} else {
			// explorer mac...would also work in explorer 6 strict, mozilla and safari
			viewportWidth = document.body.offsetWidth;
			viewportHeight = document.body.offsetHeight;
		};
		this.pageSize = {
			viewportWidth: viewportWidth,
			viewportHeight: viewportHeight
		};
	},

	// load window size information
	updateWindowSize: function() {
		// view port dimensions
		var windowWidth, windowHeight;
		if (self.innerHeight) {
			// all except explorer
			windowWidth = self.innerWidth;
			windowHeight = self.innerHeight;
		} else if (document.documentElement && document.documentElement.clientHeight) {
			// explorer 6 strict mode
			windowWidth = document.documentElement.clientWidth;
			windowHeight = document.documentElement.clientHeight;
		} else if (document.body) {
			// other explorers
			windowWidth = document.body.clientWidth;
			windowHeight = document.body.clientHeight;
		};
		this.windowSize = {
			windowWidth: windowWidth,
			windowHeight: windowHeight
		};
	},

	// load scroll offset information
	updateScrollOffset: function() {
		// viewport vertical scroll offset
		var horizontalOffset, verticalOffset;
		if (self.pageYOffset) {
			horizontalOffset = self.pageXOffset;
			verticalOffset = self.pageYOffset;
		} else if (document.documentElement && document.documentElement.scrollTop) {
			// Explorer 6 Strict
			horizontalOffset = document.documentElement.scrollLeft;
			verticalOffset = document.documentElement.scrollTop;
		} else if (document.body) {
			// all other Explorers
			horizontalOffset = document.body.scrollLeft;
			verticalOffset = document.body.scrollTop;
		};
		this.scrollOffset = {
			horizontalOffset: horizontalOffset,
			verticalOffset: verticalOffset
		};
	},

	// INFORMATION CONTAINERS

	// raw data containers
	pageSize: {},
	windowSize: {},
	scrollOffset: {},

	// combined dimensions object with bounding logic
	pageDimensions: {
		pageWidth: function() {
			return sb_windowTools.pageSize.viewportWidth > sb_windowTools.windowSize.windowWidth ?
				sb_windowTools.pageSize.viewportWidth :
				sb_windowTools.windowSize.windowWidth;
		},
		pageHeight: function() {
			return sb_windowTools.pageSize.viewportHeight > sb_windowTools.windowSize.windowHeight ?
				sb_windowTools.pageSize.viewportHeight :
				sb_windowTools.windowSize.windowHeight;
		},
		windowWidth: function() {
			return sb_windowTools.windowSize.windowWidth;
		},
		windowHeight: function() {
			return sb_windowTools.windowSize.windowHeight;
		},
		horizontalOffset: function() {
			return sb_windowTools.scrollOffset.horizontalOffset;
		},
		verticalOffset: function() {
			return sb_windowTools.scrollOffset.verticalOffset;
		}
	}
};

As you can see this object has several functions that go through all of our object checks and dimension calculations then provides an object with 5 simple properties that have every dimension we need for any kind of absolute placement.

Perhaps this code is more complex that is actually necessary, but I like to make my scripts very portable. I can drop this in anywhere and be able to get exactly what I need out of it.

Using the code

I have created a simple centerElementOnScreen function to demonstrate how all this works.

The first the you need to do before using any of the dimensions data is to get that data (or update it). You can load all of the dimensions data at once by calling the updateDimensions method.
Once you have called that method all of the information is loaded and made available through the pageDimensions methods. You call these methods directly to retrieve the information. So if you want to know how far the viewer has scrolled down the page you call sb_windowTools.pageDimensions.verticalOffset(). I know it’s kind of long, but I like descriptive names. Do not forget the parentheses, since I need to perform some logic on the pageHeight and pageWidth properties I had to make all of them functions.

Here is an example that will center a test element on the screen when the window loads:

window.onload = function() {
	// center the test element on the screen
	sb_windowTools.centerElementOnScreen(document.getElementById('test'));
}

Handling browser scrolling and resizing

Ideally, you will only need to compute these dimensions once, when you build whatever element you are trying to build. However you may need to handle the browser scrolling or resizing after you have done whatever work you needed to do.

The simplest solution is to call the function again and update your interface whenever the window.onscroll or window.onresize events are fired. I’ve tried to make this function as efficient as possible for a stand-alone function but it will have to recalculate information that surely hasn’t changed. Of course there is a more efficient way, but it requires some additional code.

What we can do is add just the needed code to the window events. Here is an example that will update just the scroll offset and reposition the element whenever the user scrolls the page.

window.onscroll = function() {
	// update the scroll information
	sb_windowTools.updateScrollOffset();
	// update the vertical position of the element
	var element = document.getElementById('test');
	element.style.top = ((sb_windowTools.pageDimensions.verticalOffset() + sb_windowTools.pageDimensions.windowHeight() / 2) - (sb_windowTools.scrollBarPadding + element.offsetHeight / 2)) + 'px';
};

Now, when the user scrolls it will only update the needed information. This is really unnecessary, simply calling the centerElementOnScreen method in the onscroll event is perfectly acceptable. The execution time is so minimal that you really shouldn’t even bother. But if you are incredibly obsessive about optimizing your JavaScript code then this is the way to do it.

Working example

Here is a working example with this code.

Click “Center Me!” and scroll. Click “Cancel” to reset.

Please note that this is not the best way to keep an element in a fixed position on the page. It’s just a good example of the code.

Conclution

There, I’ve successfully made a simple task complex and difficult to understand. I hope at least some of this information will be useful to someone out there! If you have any questions or comments please leave them in the form below and I will respond asap.

Building a home Network Attached Storage server. Part 3: Wrapping up and product reviews

Steven Benner — Tue, 06 Apr 2010 23:05:15 +0000

It’s taken me a quarter of a year to write this series, partially because I wanted to make sure I got it right, partially because I’m a lazy writer. But now the wait is over, here is the third and final installment of the NAS series about my home NAS server build.

80MB/sec across the network!

In this article I will discuss the final garnishing that you need to do to get you NAS to fire and forget server status. I will also give you my reviews of the various pieces of hardware that I used and offer a general wrap up statement on the project.

Surprises

During the course of building and using my NAS I discovered a couple of interesting things that I was not expecting to see.

Drives spin down: I was quite surprised to see that the drives on the ICH10R will power down. This was not something that I was expecting, but was happy to see. Just setting the regular power controls in Windows will allow your system to turn off the hard drives when they haven’t been used for a while. Doing this will not only save your electric bill, but will also increase the life span of the hard drives.
Full network saturation: To be honest I was expecting 40-50MB/sec, and I would have been able to live with that. However I often see network transfers moving at 70-80MB/sec! Fully saturating my gigabit network. Of course a lot of this speed is due to the SMB2 performance, so don’t expect to see these speeds on XP or Linux machines.
Low power consumption: The whole system only consumes 68 watts when the hard drives are powered down and the server is just sitting idle. I can probably reduce this even further with an undervolt and more aggressive power saving, but it isn’t worth the trouble. I sleep quite well knowing my NAS is costing me less than that light I left on in the kitchen.

Final tinkering and optimizations

Before we walk away and call the server done we should take care of a couple small items that will help keep the server up, running and problem free for a very long time.

Add a UPS with a decent battery backup time
Disable the swap file
Set up RAID notifications

Add a battery backup

Running the file server off of a UPS (Uninterruptible Power Supply) is perhaps the single most important thing you can do. I would say this is not only necessary, but absolutely required for this kind of RAID server. If you run your server simply plugged into the wall then you are placing your data at great risk. A risk that can be all but removed by spending $50-$250 on a UPS.

A good battery backup can make the difference between a fully functional system and an instantly corrupted RAID array. If the power on the system were to fail during a parity write then the RAID array could be corrupted, i.e. catastrophic data loss.

This bears repeating:
Do not run a NAS with a RAID setup like this one without a good UPS!

Really, the choice of UPS isn’t all that important so long as it has a data connection that Windows Vista has native support for. Any somewhat modern UPS with a USB connection should do fine. You should be able to just plug the UPS USB cable in and Windows will recognize it as a battery. For the record I chose the APC Back-UPS XS 1300 (second generation) because it was a quality UPS that was available locally.

The bigger the battery the better, but you’re not going for length of run time, you just want a nice thick buffer of power for when there is a brown out or extended black out. I would recommend 1000VA or better, but you could probably get buy even with a 500VA so long as you’re not running a monitor and a bunch of peripherals. Go as big as your budget allows.

Configure the system to automatically power down

You want the server to automatically turn itself off (safely) if you are experiencing an extended black out or brown out. To do this we will configure Windows to shut down when a low battery status is detected. Here is a brief walk-through for Windows Vista:

Open up the Power Options from the Windows Control Panel.
Click the “Change plan settings” link under your current power plan.
Pick an appropriate time for sleep.
Click the the “Change advanced power settings link”.
Under the Battery controls (at the bottom of the list), set both Low battery action and Critical battery action to “Shut Down”. Also verify that the Low battery level and Critical battery level setting are appropriate for your UPS.

The default battery level settings should be fine for 1000VA or better, but if you’re running an underpowered battery jack these values up to 70-80% because a small battery will run out much faster and you want to leave ample time for the system to shut down.

That’s it for settings. Your server should automatically shut down when there is a power failure without any assistance, even if you’re far away from home. When you are at home you will surely hear your UPS screaming at you if there is a power failure. If you’re there then go ahead and shut the server down, don’t take any risks with your precious data (and quiet that loud alarm).

A note on power actions: I believe that hibernate and sleep should also protect your RAID array, but I don’t know enough about the inner workings of these functions to say for sure. All you are trying to do is to get the drives to finish any current I/O activity and power down safely.

The only time a power failure can damage your system is if it happens during a parity write. So if you’re in the middle of a wind storm and see your lights flickering, do yourself a favor and do not save a bunch of files to the NAS.

Test it

Once you’ve got your UPS charged and running you should do a little test to verify that it is all correctly configured and working properly. Make sure that no one will be writing to the NAS for a while (or just unplug the ethernet), also make sure all of the drives are fully powered up and spinning. Then unplug the UPS from the wall.

Verify that it instantly switches to battery power and the system stays up an running.
Verify that the system automatically powers down according to your settings.

Disable the swap file

We want our hard drives to last as long as possible, the way we do that is by keeping disk I/O to a minimum and powering down the drives when they’re not needed.

The first thing we can do to lower I/O is to disable the swap file (a.k.a. the paging file). In case you’re not familiar with this, the swap file is basically on-disk RAM for the operating system. If you have it enabled then Windows is always pinging the drives for the extra memory.

If you’re running Windows Vista then you should have at least 4GB of RAM before you disable the swap file. Without the benefit of virtual memory the system will crash if you max out your RAM, which could put your data at risk. So skip this step if you’re only running 1 or 2GB of RAM.

Disabling the swap file will not only greatly decrease disk I/O, but it will also improve the performance of your system because it forces Windows to use RAM for its temporary storage, which is about ten-bazillion times faster than hard drives.

To disable the swap file in Windows Vista:

Open “System” in the control panel
Open “Advanced system settings”, on the left menu in the System screen.
Under the “Advanced” tab, click the “Settings” button in the “Performance” section.
Click the “Advanced” tab.
Click the “Change” button in the “Virtual memory” section.
Select the “No paging file” option and click “Set”.

Disable the swap file on Windows Vista

The screen shot above shows the windows and configuration settings that you want to disable your swap file in the Microsoft Windows Vista operating system.

Setup RAID notifications

Something that is woefully missing from the Intel Matrix RAID controller software is notifications. Sure, it will pop a little message on the desktop, but I don’t spend much time looking at the desktop on a machine that doesn’t even have a monitor.

No, what you need to have is email notifications, throw on some SMS and maybe a small fog horn for good measure. However for this article I will only cover email notifications and skip the instructions for setting up SMS or a fog horn.

Now as I said, the Intel Matrix software doesn’t have any way to send notifications, however, it does post events to the Windows event log, and the Windows event log does have the ability to attach actions to log events. So let’s use this feature which comes with Windows.

RAID event log information

Whenever a RAID event occurs a message will be sent to the Windows event log. These events are logged on the Application log as a message from the IAANTmon source. The events that we’re looking for (RAID events) are given the Warning level. This is all of the information we need to attach an action.

The big thing we are looking for is when a RAID array goes from “Normal” status to “Degraded” status. This means that a hard drive has failed and the array is in it’s emergency status, working hard to keep the data available but on the verge of total data loss.

A Degraded notification is logged with an event ID of 7202 (on my system, may be standard but I can’t say that for sure). Here is an example of the first couple lines of such a notification:

Status for volume ‘Data’ changed from ‘Normal’ to ‘Degraded’.
System Report:-
Intel RAID Controller: Intel(R) ICH8R/ICH9R/ICH10R SATA RAID Controller
Number of Serial ATA ports: 6

When you see this message you should be very concerned and do everything possible to replace the failed hard drive. The information that follows this message will give you the details of all of the drives so you can identify which SATA port has the failed drive.

Attaching actions to Windows events

Creating a task that will be run whenever this event happens is really easy, it’s built right in to the event system. It even has a “Send email” feature, unfortunately it is really dumb and requires a simple SMTP server which doesn’t require a log in. If you have such a SMTP server (possibly from your ISP) then you can use this feature and call it done.

If you do not have access to an SMTP server that doesn’t require authentication then you will need to do find another way to send messages. In my case I created a little program that uses my gmail account. I’ll post more on this and provide a download in another article.

Hardware reviews

I have now logged 3,000 hours of uptime on this machine and feel qualified to review the hardware that I picked. Overall, I am exceedingly happy with my investment, but I still wish I had been able to spend the money on a hardware RAID controller. Not for performance mind you, this system still completely saturates my gigabit network, but for peace of mind, rebuilds take a long time with the ICH10R.

Antec 1200 computer case

This case has been wonderful for the NAS server. I love how the hard drives and internals stay cool and clean. Taking the filters out for cleaning is a bit annoying but I can live with that. Overall I’ve been very satisfied with this case.

Seagate Barracuda LP ST32000542AS 2TB Hard Drives

The price that I paid and the vast storage capacity has made these drives a great choice. I purchased six of them and all of them are still working fine to this day. It seems that the reports of DOAs have been greatly exaggerated. Just goes to show, people are 50 times more likely to leave a negative review when something goes wrong then they are to leave a positive review when everything works fine.

Western Digital Caviar Black hard drives

Fine drives, they work well, were reasonable priced and are still chugging along. But everyone already knows that WD makes good hard drives.

Gigabyte EP45-UD3R motherboard

The first one I got was DOA, it would try to boot, fail and reboot over and over. Apparently this was a common occurrence with these boards. The second one I got worked fine, however two of the CPU pins were slightly bent. I don’t know if it was something that happened at the factory or during shipping, but they were easily repaired with a pair of needle-nose pliers. Other than those issues the board has been working great.

APC Back-UPS XS 1300 (second generation)

APC has a long history of making good UPS systems and supporting them for a very long time. You can probably still purchase replacement batteries for a UPS that you bought from APC 10 years ago.

I chose APC because I have had good luck with them personally, and I chose the second generation Back-UPS XS 1300 because it was a high quality system that I’ve used before and was available locally. These can be quite heavy so I didn’t want to pay shipping and I didn’t want to have to do a mail RMA if I had the bad luck of getting a dead one.

The XS 1300, as it’s name implies, is a 1300VA UPS, which is more than enough juice for my low-power NAS. I can get well over 30 minutes of battery run time on this unit and it has a nice LCD screen that reports load (wattage and percent) as well as estimated run time. It has AVR (Automatic Voltage Regulation) which is a feature I would recommend. AVR keeps the voltage constant during a brown out which will help protect sensitive (and expensive) computer components. It also looks good, a tall and slim black unit that looks great sitting next to the Antec 1200.

This is a quality unit and I am happy to give it a big thumbs up.

Conclusion

This has been a great project and I have received an absolutely amazing file server for my time, effort and money. I leave this machine on 24 hours a day, 365 days a year and use it for all of my file storage. It’s wicked fast, has massive storage capacity and doesn’t even seem to hurt my electric bill.

Since I started this project the Intel i5/i7 processors and motherboards have come down in price a lot, but have yet to completely replace the old Core 2 series. If anything the Core 2 series is an even better deal today and I would still recommend building your home NAS on that platform.

I hope that this series of articles will be of some help to anyone out there looking to set up their own file storage solution. Please, if you have any questions leave them in the comment field below and I will answer them to the best of my ability.

MS Ajax 4.0 DataView Templates vs. jTemplates and PURE

Steven Benner — Sat, 27 Mar 2010 03:00:18 +0000

There are many occasions where you’ll want a JavaScript template system to handle the rendering of data. If you’re building any kind of JavaScript widget or user interface that will consume JSON data, then build presentation with that data, you really should consider using a JavaScript template engine. It can save a lot of future headaches and make updates a much easier task.

Up until now there has really been only two good option for doing maintainable presentation templates in JavaScript, jTemplates, and PURE. These two template engines allow you to build layout with data.

jTemplates is a plugin for jQuery, and of course jQuery is by far my favorite JavaScript library, the only one I’ll ever recommend. jTemplates is an abstraction for data and presentation. It even has its own pseudo-language syntax for the template engine.
PURE (Pure Unobtrusive Rendering Engine) is a template engine that works with several JavaScript libraries, including jQuery. This system is designed from the ground up to use valid, unobtrusive techniques to build out it’s templates. PURE templates are normal in-page markup that the JavaScript replaces data in to. A very elegant system indeed.

Now there is a new option coming out along with the new Microsoft .NET Framework release.

Version 4.0 of the Microsoft Ajax Library (aka ASP.NET AJAX) includes a new client side template system using the DataView control which is actually quite good. Frankly, it feels like it was inspired by taking the best of jTemplates and PURE but it is enhanced for ASP.NET developers.

I’ve been playing around with this new system and have been pleasantly surprised. Templates are relatively easy to construct, implementation is straight forward, and it works quite well. There are some pros, and some cons when compared to the other template systems.

Of course there are actually many other template plugins for jQuery, however PURE and jTemplates have the best documentation, best syntax and the most recent updates. Almost all of the other template systems have been abandoned or deprecated by their creators.

Comparison table

Basically all of these template systems have similar features and capabilities. They are all great choices for JavaScript templating, they only really differ in template structure and syntax.

	jTemplates	PURE	MS Ajax DataView
JSON Data	Yes	Yes	Yes
In-page templates	Yes	Yes	Yes
External Templates	Yes (as js files)	No (but can be done)	No (but can be done)
Valid XHTML	Yes (hacky)	Yes	Yes
Fast	Yes	Yes	Yes
File Size	10kB	9kB	50kB

The Microsoft Ajax Library has some nifty features for power users but that is something that I will save for another article. Suffice it to say that any template you care to make can be done in any of these systems.

Comparison details

Supported data

PURE, jTemplates and the DataView control can only understand and parse JSON data.

In-page templates

When I say “in-page templates” I am talking about markup in the HTML document that gets evaluated as the template. It may be far more convenient for you to build your template as part of the page that will be using it (or it may not). All of these systems support in-page templates, but they have different ways of accomplishing this technique.

jTemplates grabs the template by using the jQuery .html() method to read the contents of a node. Because jTemplates templates are usually not valid markup the common practice is to store them in a


Customization
The AddThis widget is also the most flexible in terms of layout and design. There are several basic templates and styles to choose from and the JavaScript that runs the widget supports numerous customizations. With a little CSS trickery you can completely control the look of your widget.
Performance
This widget scored very well in the performance testing for initial load, with low initial download and only 4 initial HTTP requests. However it has the second largest total download weight when your mouse crosses it with a total of 71KB of script, CSS and images.
Thoughts
I do like the look and feel of AddThis and love the extra levels of customization you get with this product. They have good documentation and have good performance. They’ve recently added clickback analytics which is a very nice feature indeed. Saves you the time of building bit.ly links and integrates click tracking back in to the analytics system.
If you care about your visitors privacy, do not use AddThis, the    primary reason this widget exists it so mine data. They use black-hat    tracking tactics and their privacy policy says that they can   resell  the data to anyone who wants it.
ShareThis
ShareThis Widget
The second most popular social bookmarking widget out there is ShareThis. The ShareThis widget is not as flexible as the AddThis widget and can be very difficult to customize beyond their widget creator. But it has other strong points.  The ShareThis widget is supported by a service that users can sign up for. People logged into a ShareThis account will see their favorite social sites at the top of the widget. This is a pretty nice feature, for people who decide to sign up.
Try it

Customization
You can customize a couple levels display with a few different display options. You can of course choose and order the social site links that display in the hover view. However since the popup menu is displayed via an iframe there is absolutely no way to change the look and feel other than the very basic controls you get in the widget creator.
Performance
Unfortunately ShareThis has done quite poorly in the performance test. It has the largest initial download weight (by far) and the most initial HTTP requests. The initial weight is a whopping 83KB! Most of the initial weight comes from one large sprite and the MooTools JavaScript library. The sprite is fine, a JavaScript library is not. Granted both of these items will cache well, but you should not build a simple little widget on top of a JavaScript library, it’s a huge dependency that is 95% wasted on this product.
It takes a total of 18 HTTP requests to get the widget if the user does not have it in browser cache. Since it is so popular many people will have it cached but these requests will delay the loading of your page.
What really hurt was the initial load time. I should point out that the load time is skewed by the Google ad services and analytics calls that the script calls. It will render and display before the 1.69 second average initial load time, but since I tested load time not render time I have to factor in the tracking and ad code.
ShareThis sends two logging GET request as well as an AJAX POST request when it opens the sharing dialog, the ShareThis server requests tend to take around 220ms each which hurt the total load time even more. The Google analytics request only takes 45ms, which is they kind of speed they should be shooting for if they want this extra logging.
In my opinion even through the loading time doesn’t really reflect the render time, they still deserve a bad score because of this unnecessary logging and fat weight. Do you really need two separate logs every time someone happens to mouse over the button? Do you really need any?
Thoughts
Aside from their overzealous logging, ShareThis was actually quite a good contender. The widget has a good look and good functionality. However, the bad load time and the lack of customizations hurt.
AddToAny
AddToAny Widget
The AddToAny brings a couple nifty features to the table that are quite nice, it tracks how many shares a particular page has for each service and shows them in the Share / Save list.
They also have a “personalization” feature that displays services that individual visitors actually use up top. This is rather dubious because if they are doing what I think they are doing, then they are exploiting a security hole in the browser. Regardless, it is conceptually a brilliant feature that can be very useful.
Try it

Customization
The button itself is an image tag, so it’s easy to change that to anything you like. The widget creator lets you change the colors, but that’s about it. However, since the popup menu is not in a iframe you can control it’s look with CSS. The markup is well structured and has good ID and class identifiers. Their API offers some additional customization elements as well.
Performance
AddToAny did very well in the performance testing. It has the second highest initial weight, but that is due to one huge sprite with every logo imaginable. Its initial load time was right in the middle, not super-fast, not slow. However it had the fastest average total load time.  The performance is fast and it is consistent. I had to give it 3 DNS requests because that’s how many it takes to load it, however one of those DNS requests is google-analytics.com, which basically every person on the internet will already have cached before they ever reach your site.  Overall, the performance was very good.
Thoughts
If you ask me the popup share view is way too big. The bookmark tab is not very useful, in IE it gives you an “Add to Favorites” button, but in every other browser it simply says “Press Ctrl+D or ⌘+D to bookmark this page”, there isn’t much point. The search form? Why is it there? Who will search for their social site? If you press that little down-arrow under the social links it will display every social site that has been enabled. With the default settings this is a massive list which can mess with your site layout if you don’t expect it.
Tell-a-Friend
Tell-a-Friend Widget
The Tell-a-Friend widget from SocialTwist is the least known bookmarking widget in this review, but those who use it tend to like it so I felt they deserved a place in this article. Of all the widgets I’ve been giving serious consideration to, this is the only one with a pay version. There is of course a free version but the pay version does have several features that power users will want.  It would seem that the primary focus of this widget is branding control for the client. You don’t get these advanced features in the free version so I cannot give you more details, but they have acquired some rather large clients (Disney, Barnes & Noble, Kraft Foods, etc.) who use the more advanced branding features.
Try it

Customization
The widget customization is pretty much limited to what you can build in the widget creator. The button itself is an img tag so you can replace that quite easily with any image you want. This widget doesn’t use an iframe for the hover windows so you can do a little bit with CSS, however the social media icons are inline styles so they cannot be changed.
They offer more customization in the pay version, but I wasn’t willing to shell out any cash for this test so I can’t comment on the quality or any additional flexibility.
They chose to go a different direction for the bookmarking and sharing functionality. When you click on an icon to share the widget opens a kind of large modal over the page, instead of simply clicking an image and getting redirected to the share form on the social site. This feature is part of their branding focus which allows paying clients to customize the sharing screen. But it absolutely dominates the window once it opens.
Performance
Surprisingly, the Tell-a-Friend widget was the one of the best in the performance comparison. It had the lowest initial download weight and the second fastest average initial load time. The images are served from a different sub domain than the JavaScript, they could literally cut their DNS lookups in half by putting the images and JS on the same CDN sub domain. But I can’t really complain about 2 DNS lookups. For some reason their script seems to have the longest execution time, average window.onload fires around 100ms later than AddThis or ShareThis.
The performance tests did not take into consideration the modal window. Since you have to take an action to open the window I didn’t feel it needed to be rated for performance. However I do want to note that it is quite large, with lots of scripts, CSS and images.
Thoughts
Tell-a-Friend is the only TRUSTe certified widget. They seem to have the best privacy practices.
“We take user privacy very serious. We do not profile users, store any  personal information. Even the address book feature, others “import” we  only let users view.”
Indeed, they don’t do any cookie based tracking and their low weight and fast  response suggest that they are not doing any profiling on the back end. When it comes to user privacy they seem to be the winners.
I understand why they have a pay system in place, it’s the  only business model that works for them. Unfortunately their free  implementation feels like they’ve had to do extra work to reduce the  usefulness of their product. The basic vs detailed analytics, not  offering button customization and not offering modal customization. It  all just feels too forced, especially when their competition is offering  all of those features and more for free.
Personally, I refuse to enter my Twitter login information anywhere other than Twitter or my Twitter desktop application. However if you want to share to twitter via the Tell-a-Friend form you will have to enter your login information. It asks you to login to share for several other services as well. I can see this stopping many users dead in their tracks.
Features chart
All of the services that I’ve tested include support for just about every social media site ever made. Since I haven’t used every social media site ever made I can’t comment on the ones they are missing, however I will say that they all support every social media site you care about.
Every service also has it’s own integrated analytics system. All of these analytics systems are pretty good with pretty much the same metrics (shares, locations, services used, etc) and are easy to navigate. All except for Tell-a-Friend can support Google analytics integration and none except for ShareThis can support Omniture integration.




AddThis
ShareThis
AddToAny
Tell-a-Friend


Every service you care about






Language Localization
 (55)
 (5)
 (51)
 (18)


Analytics






Google Analytics






Omniture Reporting


(?)



SSL Support







To be honest I never found any value in having the ability to share SSL pages, but it has been a deciding factor for some people. Really, why is a page that you want to share with everyone behind a secure connection?
Other mentions

OnlyWire is another social bookmarking widget service that wants to play with the big boys. Unfortunately their free version has ads and I don’t think anyone will accept ads on their site just for a bookmarking widget. It also blew the performance tests with 42 initial HTTP requests and an initial weight of over 100KB.
SocialList is a super-simple bookmarking widget. It doesn’t have analytics or any creation interface and near as I can tell it hasn’t been updated in over a year. But it is small and simple, so it does deserve a mention.

Winners
Of course each widget is better in different areas, so I can really only compare them with the performance tests and features.
Best performance – AddToAny
This was a difficult choice. For initial load performance Tell-a-Friend won for having the lightest weight, least HTTP requests and fastest response. But they failed the overall widget performance test because their form system is just so big and heavy. AddToAny wins the best overall performance despite having the heaviest total weight of all the widgets. Most of that weight is from a 26.8KB sprite with every logo ever made. But it caches well and does not seem to hurt performance.
Most customizable – AddThis
AddThis wins the customization contest hands-down. In some ways AddToAny is as good or better, but because of their “Toolbox” and list layouts you really can get a very good level of control over how the widget renders. They have more standard buttons and layouts than any of the other services and have more customization features.
Best end-user privacy – Tell-a-Friend
Of course, Tell-a-Friend with their decent privacy policy and lack of profiling wins the privacy competition. Most web masters don’t concern themselves with matters like this, but internet privacy is something that I consider very important. As much as I want to know who you are, I don’t presume the right to know everything about you just because you viewed my site. At the same time servers and dark-fiber aint cheap, and I need to make money off of you somehow.
Do you really need a bookmarking widget?
Before you go implementing one right this moment; please take a moment to ask yourself, do you really need a social bookmarking widget?
There are literally hundreds of social media sites out there, most of them you have never heard of and neither have any of your visitors. If your site serves a particular niche then it should be easy to identify the social media sites that you need to target. Just put up a couple icons and links. This will make your site load faster and make it easier to follow your presence on those social media sites. If someone wants to link your site on a social media service then not having a widget will not stop them.
If you do need to have an easy link for more than just a few social media sites then perhaps you do need one of these widgets.
Final thoughts
Any of these widgets are good choices, depending on your needs. Personally, I think AddThis has the best look and feel, but I hate their aggressive tracking strategy. I like the look and functionality of ShareThis but it’s too fat and users have to register to get the most out of it. AddToAny has good performance and is intuitive, but it is just so big. And  Tell-a-Friend has nice branding tools but their big modal window is a bit cumbersome and it feels neutered unless you pay.
If I had to pick a widget right now it would probably be AddToAny, with a lot of thought given to AddThis.
Update (3/16): Re-tested all widgets and updated all of the performance numbers. Updated AddToAny customization information. Added thoughts on Tell-a-Friend privacy.
Update (3/19): Update language localization information. Added SSL support.
Related articles
Building a home Network Attached Storage server. Part 3: Wrapping up and product reviews
MS Ajax 4.0 DataView Templates vs. jTemplates and PURE
Custom link click tracking using Omniture
AddThis: friendly widget and brilliant viral tracking scheme



Custom link click tracking using Omniture
Steven Benner — Sun, 07 Mar 2010 20:55:28 +0000
Omniture is the de facto standard tracking and analytics  system that most online retailers use. It has a suite of reporting metrics and allows for custom reporting variables. It is primarily designed for online stores to track usage, conversions and sales.
Admittedly, I am not that big a fan of Omniture. Their SiteCatalyst reporting application is slower than Google Analytics and not as flexible as it could be. My biggest gripe is their tracking JavaScript code, it’s just plain terrible. It’s slow, obfuscated (really? why do this?), bloated, impossible to debug and not built using modern practices. Oh, and they’re also insanely expensive if you want to get at all fancy.
This article is simply a straight how-to for building a custom link tracking JavaScript in the hopes that I can save some other JavaScript developer out there the headaches and tears usually associated with learning advanced Omniture implementations.

Omniture basics
The Omniture JavaScript does all of its work through the s object. This object has many different methods and properties that are all ingeniously designed to confuse and torment would-be implementers.
For custom tracking metrics you will be using prop codes, which are called “Custom Insights” in the SiteCatalyst reporting software. You associate a specific prop code to a specific piece of data. So you may associate s.prop1 to be the category of a product and s.prop2 to be the name of the product.
The way we track is by loading up prop variables with strings then telling the Omniture code to run. This is done by calling s.t() for a pageview track or s.tl() for a link click track.
How to track a click action with custom prop data
The code to track an action is actually quite simple, loading prop values and calling the tracking method. Here is what a tracking script looks like. This example is the basic custom click tracking code that you would expect to see in an onclick event attached to an anchor tag.
s.linkTrackVars = 'prop1,prop2';
s.linkTrackEvents = 'None';
s.prop1 = 'Televisions';
s.prop2 = 'Samsung - LN40B530 - 40" LCD TV - 1080p (FullHD)';
s.tl(this, 'o');

Now, this is fairly self-explanatory, but I’ll break this down line for line so you know exactly what’s happening and what these statement mean.

s.linkTrackVars declares which prop codes you will be using in this event. This property is a comma-separated list of each prop code that you want tracked.
s.linkTrackEvents are the “events” that an action may represent. Usually you will use events like “purchase” or “save” but you can put just about anything you want in here. It’s just a different way of looking at the same information. I typically set it to none and use custom insights for reports.
s.prop1 and s.prop2 are the custom metrics that will be attached to this click event. In this case prop1 is the product category and prop2 is the product name.
s.tl(this, ‘o’) calls the tracking action. The tl() method (a.k.a. the trackLinks method) requires two parameters, this anchor object to be tracked (should always be this) and the type of link we are tracking. There are three types of basic links that is method understands: o is the “Other/General” link type, d is a “File Download” link, and e which is an “Exit Link”. Anything that isn’t a download or exit link should be given the o tag.

There you have it, not incredibly complex, but not obviously simple either. There are a bunch of other properties that you can attach to a trackLinks event, but this is what 99% of them look like.
Building a script to attach link tracking events automatically
There are two standard ways to attach these custom link tracking events to the anchors that you want to track: the stupid method, and the smart method.

The stupid method is the most common. This involves adding an onclick tag to every anchor and filling it with the appropriate code.

Pros: Stupid, simple, always works
Cons: Stupid, unmaintainable, ugly, increases markup, adds non-semantic markup


The smart method is to build a single script that will attach the code to every link you want to track automatically on page load.

Pros: Smart, not-stupid, simple, maintainable
Cons: Links don’t get the tracking events attached until the whole page has finished loading.



For this rest of this article I will assume that you want to use the smart method. Basically, we will build a script that will look at every single anchor tag in the DOM and attach the click tracking events to any links matching whatever criteria you choose. This script will automatically run when the page loads.
Defining the criteria and search pattern
First off, you must be able to identify the links that you want to track. I have used two different systems for this: Searching href tags and adding rel tags.
Searching href tags is generally the best option. This means matching the link href and attaching the event based on that data. However I have found situations where I didn’t want to track every link that had the string “products.aspx” in it, so I chose to add rel tags to those links that I did want to track. This way I could just search for links that had rel=”product” in them.
The technique is still basically the same, we will gather all of the anchors in the document into an array, then iterate through the array, attaching the appropriate events and data to those anchor which meet our criteria for each type of page.
The basic script structure
No matter what you choose as your search criteria and trackable data you will need a script that has at least the following features:

Iterates through all anchors on a page
Allows you to attach tracking events without overriding any existing events
Automatically runs as soon as the page loads
Highly optimized for minimum execution time
Cannot throw an error that will break the page
Unobtrusive design that doesn’t have any dependencies (other than Omniture)
Namespaced in object literal notation so that it can never have any confilcts

The basic JavaScript object that I use for this
// Omniture click tracking
// http://stevenbenner.com/2010/03/custom-link-click-tracking-using-omniture/

if(!sb_trackLinks) { var sb_trackLinks = new Object(); };

sb_trackLinks = {
	// initialize the click-tracking system and attach events
	init: function() {
		// verify that we are running in a modern browser, if not give up silently
		if (!document.getElementsByTagName) return false;

		// iterate through all of the links in the document
		var anchors = document.getElementsByTagName('a');
		for (var i=anchors.length-1; i>=0; i--) {

			// SEARCH STATEMENTS AND TRACKING VARIABLE BUILDING GOES HERE

		};
	},

	// tracking function
	track: function(obj, propCodes, linkType = 'o') {
		s = s_gi(s_account);
		// reset tracking properties
		s.linkTrackVars = '';
		s.linkTrackEvents = 'None';
		// iterate through each property in propCodes
		for (code in propCodes) {
			s.linkTrackVars += code + ','; // build the CSV list
			s[code] = propCodes[code];     // attach the prop code
		}
		// track it
		s.tl(obj, linkType);
	},

	// non-breaking/non-overriding/cross-browser event attachment
	addEvent: function(obj, evType, fn, useCapture) {
		if (obj.addEventListener) {
			obj.addEventListener(evType, fn, useCapture);
			return true;
		} else if (obj.attachEvent) {
			var r = obj.attachEvent('on' + evType, fn);
			return r;
		} else {
			obj['on' + evType] = fn;
		};
	}
};

// auto-init on load
sb_trackLinks.addEvent(window, 'load', sb_trackLinks.init);

This is the basic outline of my JavaScript click tracking object. It has three methods and automatically runs itself in the window.onload event.
I've put a lot of thought into optimizing this script. For instance, notice that I reverse iterate through the anchors array. This technique has been proven to be faster because the array length is only calculated once when the for statement begins. If you do the while i style then the length is calculated once for every iteration in the loop.

When you build any kind of tracking or analytics JavaScript your highest priority should be to shave every nanosecond from it's operations. It is acceptable to sacrifice readability for performance here. After all, as a web developer your primary job should be to improve user experience, tracking scripts are a necessity but do not benefit users in any way, so you do your best to reduce their impact on user experience.
The sb_trackLinks methods
There are three methods in the sb_trackLinks object:

init()
The init() method is where the startup, initialization and event attachment will occur. We still need to build the search criteria, tracking variables and event attachment code before this will do anything.
track(obj, propCodes, linkType)
The track() method is the code that will be attached to the onclick events for the anchors that we want to track. It requires two parameters and will accept a third.

obj: The first parameter is the anchor object that we will be tracking, in the context of our for statement in the init() method this will always be anchors[i].
propCodes: The second parameter is an object with the prop codes that we want to use. This should be a standard JavaScript object with the conventions as the s.propXX codes. So if you want to track prop1 you should set propCodes['prop1'] to the value that you want and pass it through to the track() method.
linkType: The third parameter is the link type that will be passed to s.tl() and is optional. It will default to 'o'. As I said earlier, it will accept o, d and e as valid values.

You may have to modify this function if your Omniture code has been customized.

addEvent(obj, evType, fn, useCapture)
The addEvent() method is the standard addEvent that you see everywhere on the internet.

Crafting the search and tracking variables
Now that we have a basic structure that we can use, all that we need to build is the anchor search statements, tracking variable and event attachments.
This needs to be custom built for your specific needs. The application, URL structure and tracking needs are all critical parts of this equation. If you are using friendly URLs and need to parse data from them then you will may have to deconstruct the URI to grab the specific data that you want to track in prop codes. If you need data from somewhere else on the page you will have to figure out out to grab it. Etcetera.
I'll provide you with an example based on our previous examples. In this case we are tracking clicks in an online store application. We are using standard query string parameters in our URLs and do not need any other special data.
This is what our for statement will look like for this site:
for (var i=anchors.length-1; i>=0; i--) {
	// parse the query string into an object
	var queryString = {};
	anchors[i].href.replace(
		new RegExp("([^?=&]+)(=([^&]*))?", "g"),
		function($0, $1, $2, $3) { queryString[$1] = $3; }
	);

	// conditionals to match link href targets
	if (anchors[i].href == 'http://your.domain/' || anchors[i].href.indexOf('home.aspx') > -1) {
		// home page
		sb_trackLinks.addEvent(anchors[i], 'click', (function() {
				return function() {
					try {
						sb_trackLinks.track(
							this,
							{
								'prop1': 'home'
							}
						);
					} catch(e) {}
					return true;
				};
			})()
		);
	} else if (anchors[i].href.indexOf('category.aspx') > -1) {
		// category pages
		sb_trackLinks.addEvent(anchors[i], 'click', (function(catId) {
				return function() {
					try {
						sb_trackLinks.track(
							this,
							{
								'prop1': 'category',
								'prop2': catId
							}
						);
					} catch(e) {}
					return true;
				};
			})(queryString['category_id'])
		);
	} else if (anchors[i].href.indexOf('product.aspx') > -1) {
		// product pages
		sb_trackLinks.addEvent(anchors[i], 'click', (function(catId, productId, productName) {
				return function() {
					try {
						sb_trackLinks.track(
							this,
							{
								'prop1': 'product',
								'prop2': catId,
								'prop3': productId,
								'prop4': productName
							}
						);
					} catch(e) {}
					return true;
				};
			})(queryString['category_id'], queryString['product_id'], queryString['product_name'])
		);
	};
};

This is a bit technical, and uses some advanced JavaScript techniques. However, it should be fairly understandable if you know your way around JavaScript.
Since we need to grab query string values for most of our tracked pages I start off by parsing the query string into an object.
I then proceed down a list of if statements that try to match a specific page. Inside each if statement I attach an onclick event to the current anchor. That event passes our current data down through the JavaScript scope via a (function(v){})(data) statement. Inside that event I call the track() method with the data and return true so that the browser follows the link once the code as been executed.
This can get very long indeed if you have a lot of different  types of pages in your web application that need click tracking. As  always, order your if statements by commonality, putting the most  commonly matched statements first. The script engine will have to fail  every if check to get to the last statement. This minor optimization may  save you a millisecond or two.
It is very important that you wrap all of the code inside the event in a try{}catch(e){} just in case something somewhere breaks. If you do not do this then the links on your site will stop working if anything goes wrong. There is nothing quite as asinine as links not working because the tracking script is broken.
Putting it all together
Now that we've built the search and attach code all we have to do is insert it into our base script. The result looks like this:
// Omniture click tracking
// http://stevenbenner.com/2010/03/custom-link-click-tracking-using-omniture/

if(!sb_trackLinks) { var sb_trackLinks = new Object(); };

sb_trackLinks = {
	// initialize the click-tracking system and attach events
	init: function() {
		// verify that we are running in a modern browser, if not give up silently
		if (!document.getElementsByTagName) return false;

		// iterate through all of the links in the document
		var anchors = document.getElementsByTagName('a');
		for (var i=anchors.length-1; i>=0; i--) {
			// parse the query string into an object
			var queryString = {};
			anchors[i].href.replace(
				new RegExp("([^?=&]+)(=([^&]*))?", "g"),
				function($0, $1, $2, $3) { queryString[$1] = $3; }
			);

			// conditionals to match link href targets
			if (anchors[i].href == 'http://your.domain/' || anchors[i].href.indexOf('home.aspx') > -1) {
				// home page
				sb_trackLinks.addEvent(anchors[i], 'click', (function() {
						return function() {
							try {
								sb_trackLinks.track(
									this,
									{
										'prop1': 'home'
									}
								);
							} catch(e) {}
							return true;
						};
					})()
				);
			} else if (anchors[i].href.indexOf('category.aspx') > -1) {
				// category pages
				sb_trackLinks.addEvent(anchors[i], 'click', (function(catId) {
						return function() {
							try {
								sb_trackLinks.track(
									this,
									{
										'prop1': 'category',
										'prop2': catId
									}
								);
							} catch(e) {}
							return true;
						};
					})(queryString['category_id'])
				);
			} else if (anchors[i].href.indexOf('product.aspx') > -1) {
				// product pages
				sb_trackLinks.addEvent(anchors[i], 'click', (function(catId, productId, productName) {
						return function() {
							try {
								sb_trackLinks.track(
									this,
									{
										'prop1': 'product',
										'prop2': catId,
										'prop3': productId,
										'prop4': productName
									}
								);
							} catch(e) {}
							return true;
						};
					})(queryString['category_id'], queryString['product_id'], queryString['product_name'])
				);
			};
		};
	},

	// tracking function
	track: function(obj, propCodes, linkType = 'o') {
		s = s_gi(s_account);
		// reset tracking properties
		s.linkTrackVars = '';
		s.linkTrackEvents = 'None';
		// iterate through each property in propCodes
		for (code in propCodes) {
			s.linkTrackVars += code + ','; // build the CSV list
			s[code] = propCodes[code];     // attach the prop code
		}
		// track it
		s.tl(obj, linkType);
	},

	// non-breaking/non-overriding/cross-browser event attachment
	addEvent: function(obj, evType, fn, useCapture) {
		if (obj.addEventListener) {
			obj.addEventListener(evType, fn, useCapture);
			return true;
		} else if (obj.attachEvent) {
			var r = obj.attachEvent('on' + evType, fn);
			return r;
		} else {
			obj['on' + evType] = fn;
		};
	}
};

// auto-init on load
sb_trackLinks.addEvent(window, 'load', sb_trackLinks.init);

Save this code in the s_code.js (or whatever you named it) script for your Omniture-enabled web site and it will start sending click data back to your Omniture account.
Gotchas
There are some possible problems that may crop up using this system.

Page never stops loading

I have seen pages that simply never stop loading because of poor use of AJAX, slow servers, or broken dependencies. The click tracking events will not be attached to links until the loading icon in the browser stops.
Links added after page load

If there are any links that are created after the page load event is fired, such as AJAX widgets or dynamic links, then they will not exist when this script is run and therefor will not have any tracking events attached to them.
Omniture conflicts

There have been several occasions where I've seen people try to use two (or more) instances of Omniture on the same page. If you have to deal with this then you need to modify the track() method to use your account name.
500ms link delay

The s.tl() function has a half-second delay built into it. This is because the tracking data is sent by requesting a 1 pixel web beacon from the Omniture servers and the script needs to give the browser enough time to send the request. Unfortunately, this also adds an annoying delay to every link that you attach tracking code to.

Advanced tracking (more than just links)
Omniture link tracking can be used for more than just links. You can use the s.tl() method to send any data at any time for any reason. Do you want to track when someone puts their mouse over an image, or how long they waited before scrolling, or even what text they selected? You can do all of that, and more if you really want to.
If you ask me those fine-grain viewer eyeball focus type of reports are not only invasive but genuinely useless. However, I've been asked to do even worse invasive and pointless tracking before. If you need to you can send any data you want on any DOM event, or even with timers.
The only thing that you must overcome is that the object you pass to s.tl() must be an anchor, or more specifically, something with an href attribute. There is nothing about this in the documentation, but it simply doesn't seem to work unless it has an anchor with an href passed to it. You can get around this by creating an element or simply passing true.
This is, however, the subject of a different article.
Conclusion
As you can see it's not that difficult to implement custom link tracking with Omniture and you can get some very powerful reports without that much work. I still cannot give my glowing stamp of approval for Omniture as a reporting solution, for various reasons, but if you are using it then hopefully this tutorial will save you some time and headaches.
If you have any questions or comments please leave them in the form below and I will do my best to answer them. You can also take a look at the Custom Link Tracking post in the Omniture blog. It's a year old, but the author is still responding to questions.
Related articles
Calculate page size and view port position in JavaScript
AddThis: friendly widget and brilliant viral tracking scheme
External templates for the DataView control: Microsoft Ajax Library 4.0
How to circumvent internet filtering and enhance privacy
MS Ajax 4.0 DataView Templates vs. jTemplates and PURE

	AddThis	ShareThis	AddToAny	Tell-a-Friend
Every service you care about
Language Localization	(55)	(5)	(51)	(18)
Analytics
Google Analytics
Omniture Reporting			(?)
SSL Support