Ok, so if you followed the previous link and checked out our app, you’ll see that it’s somewhat pretty (for a ripoff), and barely functional! Man we rock… but really, the competition wasn’t just about winning for our team as much as it was about fiddling around with some new JavaScript hawtsawce. So here’s what I learned:

Pick your 48 hour project Wisely

Yeah, so our idea was to actually build a NodeJS backend which could scan the directories input by the user, read the id3 tag info from the files within those directories (as a start, we could support ogg and flac and other format metadata schemes later), and then put the metadata into a CouchDB.

Then the client could connect to your server, grab the catalog from NodeJS, and play songs that NodeJS would convert on the fly into the proper format for your browser depending on capability (probably through a child process exec to some other encoder — haven’t figured that part out yet).

Unfortunately, we hit a snag on step 1 of this process: reading the data from an id3 tag, which is binary data stored at the beginning on an mp3 file. Node can read streams of data, like so:

  var s = fs.createReadStream(file, {start: 1, end: 2});

  s.on('data', function(data){
    console.log(data.toString());
  });

This is actually the code I was using to check whether a particular file begins with “ID3,” which would thus signify ID3 version 2 data. All the fun technical details behind ID3 can be read about here: http://www.id3.org/Developer_Information

But this is about when things started to break down. After this, and I’m literally talking about the first 10 bytes here, became totally beyond my grasp for a 48 hour competition. WTF does this mean, for example:

The ID3v2 tag size is stored as a 32 bit synchsafe integer (section
6.2), making a total of 28 effective bits (representing up to 256MB).

I mean, I somewhat get what that’s saying. If you just read section 6.2, you can clear as day see…

All this led me to one conclusion after a few hours: Choose your project wisely. Dylan Bathurst, who I’m officially dubbing my “Node Wingman,” is a Front end guy. I’m a front end guy. We probably should not have picked a project that not only required us to pick up server-side programming, but also get all Computer Science major trying to decipher binary data (I have a degree in Sociology — super helpful for that kind of thing). It seems like a simple game using WebSockets might have been a better call… but oh well, just pick a project that helps you learn Node and plays to your strengths!

Plan a bit in advance…

We did a bit of preliminary planning, but we really didn’t get into the nitty gritty of our roles and responsibilities until the competition began. Not that this isn’t a totally fun way to approach a 48 hour competition, and if you think Animal from the Muppets is the coolest guy ever, then just jumping headfirst into the competition might be the best thing for you.

But, it seemed like some of the best ideas had been well thought out a in advance, including splitting up roles for each team member so they could focus on their tasks. With only 2 days of actual design and programming time, getting the prep out of the way so you can focus on code is clutch.

The community rocks

Yeah, I’m talking to you. I hadn’t spent much time in IRC before that weekend, but I was able to ask questions and get prompt, helpful responses. Plugin creators took their time to explain basic stuff to me, and it was even easy to hit up the creator of NodeJS. Plus, the vibe was just electric. Man, it was just fun being in there with everyone.

I also gotta say that I was really impressed with the Joyent setup overall. Thanks for providing such an awesome platform on which to host the competition!

Even if you don’t know anything about NodeJS, you should still totally participate

I wasn’t completely ignorant of Node, but I’m by no means an expert, and I haven’t built any serious/real apps on it. Heck, I hadn’t even built much of a toy app with it. This was my first real attempt at anything of substance. And even though we didn’t get a fully-functional app going, I learned more in two days than I have in a long time.

I would encourage anyone who has even the smallest inclination to give Node a shot to join this competition next time it comes around. If you haven’t learned Node because you don’t really know what to do with it, this competition ends that excuse! Just dive in. The water is warm — I promise!

The future is lookin pretty good…

And you know what was so cool about this competition? Knowledge wasn’t the best thing I gained. I walked away with a sense of optimism I haven’t felt in a while. When you get into the daily grind, a lot of the knowledge we accumulate seems to be incremental rather than revolutionary. NodeJS is a revolution.

Now, of course, there are a couple of secrets I’ve gotta save for myself until next year to give myself a bit of a competitive edge (heyo!), but I gotta end this post thanking everyone who put the competition together, Joyent again for hosting, the awesome people who were hangin out in IRC, and my buddy Dylan for putting the app together with me! It was a blast, and you’d better believe we’re gonna win it next year, so be ready to bring it!

Bookmark

So if you’re going to go with a base theme, which to chose? There are a ton out there, but the two I was most intrigued by were Zen and Tao, and I’m going to tell you why Tao totally beat the heck outa Zen in a very Buddhist way.

Zen – the path to (record scratch)

I started with Zen. It was very nice at first. It made me feel welcome in a foreign land. It has nice documentation. It walked me step-by-step through creating a sub-theme. I felt good. I felt confident.

After my sub-theme was established, I checked it into our SVN repository, and began coding. This was going well! I saw a lot of things I liked about Zen — A focus on accessibility, an attempt to break down CSS into logical files so you don’t have one monster CSS file, logical content regions, and did I mention the documentation!?

Heart of Darkness

But after a few days of working with Zen, theming became maddening. What had been sensible defaults now turned into a Heart of Darkness journey to find and override styles. I was spending far more time in Firebug tracking down exactly what styles were doing so they could be overridden instead of coding my site. And I couldn’t use CSScaffold, which is my CSS framework of choice, due to the way the comments were formatted throughout (this really isn’t Zen’s fault).

And while the CSS files were broken down, they are broken down in a very illogical way which requires redefining selectors in multiple files, and also mixing your screen and print styles. For example, zen has “layout.css” where you’re supposed to put all layout-related styles, such as display, margin, padding, floating, etc. But then it encourages you to put background and font styles in other stylesheets — yuck! That decision alone makes it very hard to quickly visualize what an element will look like in your “Minds Compiler”, and requires you to shift between files just to make edits to a single element. Overall, bad news.

Not so sure about ems

Furthermore, within Zen, fonts and many margins are sized in ems. I decided to go with the flow and keep using ems, which is nice if you want to support text resizing in IE6, but is a complete mess if you’re trying to establish a baseline grid. Ems turn into nasty fractions you have to constantly calculate (and recalculate if you want to do something simple like make the page text slightly larger), and the rounding errors that occur cross browser are simply unavoidable. So scratch that. IE6 only accounts for 1.5% of our users, and my brain accounts for 100% of my sanity.

No reset — the deal breaker

And then, in my opinion, the dagger in the heart: There’s just no reset. This was an intentional design decision by the creators, but in my opinion, a HUGE mistake. I simply could not get my Zen sub-theme to render consistently in all browsers, which is a direct result of no reset.

So I added a reset. But you know what? It was still inconsistent! Ack, why? Because of Drupal’s core CSS files. It was like a dream within a dream… I just couldn’t escape Firebug purgatory, and almost lost my cool with Drupal. Once again, I had found myself in assumption land, and I didn’t like it.

Tao rescues me. Or did I rescue myself?

So I went back to the drawing board. I went through a lot of the starter themes again, and still saw a bunch of stuff I was concerned about. This time around, I wanted a theme which did one thing well — reset pretty much everything so that my Drupal page looks just like an XHTML document with a reset CSS applied to it. No weird spacing, no more Drupal CSS files effing with me, and I didn’t want ANY STYLING. I literally didn’t even want default columns, tabs, yellow backgrounds for blocks on the edit-block page, NOTHING. I just wanted some very clean HTML with a reset style that made everything consistent between all our favorite browsers.

And that’s when I remembered reading about Rubik theme from Development Seed. If you don’t know, Development Seed makes the best looking Drupal sites. The Best. No question in my mind about it.

Rubik is a fantastic administration theme. It dramatically improves the drupal administrative interface by reducing visual clutter, and it looks just wonderful. It’s worth it to download and install just for it’s own functionality. And best of all: it relies on a fantastic base theme maintained by Development Seed: Tao.

As Tao says: “Tao is a base theme for Drupal that is all about going with the flow. It is not meant for general-purpose use, but instead takes care of several tasks so that sub-themes can get on with their job.” And this is exactly what it does.

It has an “aggressive reset,” provides preprocessing functions for “core annoyances,” properly splits up its files between screen and print, and unifies theme files within a “templates” directory (thank you for this alone — makes the directory structure much cleaner!).

Basically, it resets the ugly in Drupal, and kindly opens the door to beautiful design for you.

Now, I can use CSScaffold. This gives my page a beautiful grid without needing to further pollute my HTML with even more extraneous classes, as you have to do to employ 960gs or Blueprint.

I can split CSS files up into an order I think is much more logical (CSS files for the header, footer, content, blocks, etc). This means no more switching between files to edit one selector, and also prevents me from slipping into Firebug Purgatory.

And best of all — my site renders almost perfectly across browsers now.

Conclusion

If you’re just starting off, Install Zen and play with it. If you’re a casual designer and don’t care too much about whether the line-height of your fonts is off, and don’t particularly care about rigid control of you grid system (which is too bad, cus your page probably looks ugly), go ahead and use Zen.

But if you want to avoid Firebug Purgatory, clean up your code, organize it in a way that’s logical to you, and make sites that are truly spectacularrrrr, the choice is clear: use Tao.

And if you don’t believe me, take a look at Lullabot (creators of Zen), and then compare it to Development Seed. Lullabot’s site isn’t bad — I’m not saying that, but it clearly isn’t the same caliber as Development Seed. And if that’s the case, which one do you trust to make the best base theme? I know where I’m putting my money.

Bookmark

After cloning and examining a lot of his repos (Use the Source!), I found out in an endnote that TJ had published a book entitled Advanced JavaScript from DevMag. It’s a short book, but one that opened my eyes in a surprising number of ways.

Overview

The books Table of Contents are as follows:

• Ch 1: Closures, Scope and Context
• Ch 2: Prototypal Inheritance
• Ch 3: Advanced Meta-programming techniques
• Ch 4: Behavior Driven Development
• Ch 5: Creating a jQuery Clone
• Ch 6: Tools of the trade

The meat of this review will focus on Chapters 3-5, which made me rethink the way I develop JavaScript. No offense to the author(s), but the contents in Chapters 1 and 2 have been better explained in other texts, notably JavaScript: The Good Parts by Douglas Crockford, and Chapter 6 should have probably just been an Appendix.

But back to the good stuff: Chapter 3 goes through the creation of a JavaScript clone of some of the most basic functionality found in Sinatra, namely creating routes and executing them when matched. While the concept of DRY is often promoted as The Thing To Do, this chapter clearly explained some of the ways to make your programs smarter so that you can in fact avoid repetition! Very nice.

The revolution will not be televised, it will be tracked on GitHub

The revolution started for me in Chapter 4. I’d read about some agile programming techniques, but I’d never heard of BDD, or Behavior Driven Development. Chapter 4 talks about how to implement BDD with JavaScript using JSpec, an awesome unit testing framework, and one that totally changed how I think about my projects.

An aside on BDD

Essentially, BDD is an “outside-in” methodology, meaning that you start by identifying outcomes with your clients (internal or external), and then map out the features that will achieve such outcomes. This often happens in a story format.

Overall, it’s fantastic because it allows your clients to stay on the same page as you, the developer. You define concrete outcomes with your clients, and which steps needed to be taken for each feature to work. This means that both you and your clients have a solid understand of what needs to be done, and what “done” truly means. To read more about BDD specifically, check out a great explanation of BDD from the inventor, Dan North.

Using JSpec to implement BDD

With regard to chapter 4, JSpec helps us implement BDD from the developers perspective. Once the outcomes and features have been hashed out into specific steps, or “units,” then the developer can begin doing his/her job! Now, BDD is an extension of test driven development, and therefore implies that we should begin not by writing our implementation, but by writing Unit Tests to ensure that the functionality defined within our story is properly executed! By working this way, developers get the myriad of benefits that come from proper unit testing, such as avoiding regressions (breaking your code).

However, it had never really clicked in my mind how to really do convenient / good unit testing with JavaScript, and that’s what Chapters 4 and 5 go into in detail. There are a ton of libraries out there, but none of them encourage Unit Testing as part of the workflow the way JSpec does, because most of them are just simple unit testing frameworks written in JavaScript. But JSpec is so much more!

Jspec doesn’t just include a unit testing JavaScript file, you download a whole Ruby gem and get amazing command-line utilities for creating new projects with testing and documentation all set up, and then very convenient methods for running the unit tests in all your browsers! For example, if you execute the following command:

jspec init mySpecTest

You’ll get a new folder called mySpecTest with a directory full of goodies, such as a convenient folder structure for creating consistent projects, and all you need to run the JSpec unit tests. With the specs from your client in hand, you could then quickly begin writing your unit tests to achieve the desired functionality! Then all you have to do is write your JavaScript to pass all of your tests, and you have a very discrete piece of functionality “done!”

Wrap Up

The book does a great job of going through how to use JSpec in a number of ways, and how to build a cool, limited jQuery clone using the principles of BDD. Overall, as I said, this has literally changed the way I think about writing JavaScript in the future. This will certainly be the approach I take (I had never really written rigorous tests for my JS before), but one that I can immediately see will provide a ton of benefits to the quality of my code.

If you’re like me, and you thought “Unit testing is pretty cool, but I really have no idea how to properly implement it,” you will walk away from this book knowing exactly how to use unit tests, and you’ll also probably be bursting with ideas about how you can improve your code and your development methodologies.

“Advanced JavaScript” rules, and for $4 it is well worth the price. Go buy it.

Addendum/Nitpicks

Now, while the book totally kicks ass, I did have some minor problems with it. I really think the book needs a good editor. There are some frankly just kind of sloppy grammatical and spelling errors throughout the book, and the writing in general could use a good solid critique to make it a bit better. Also, there’s a lot of source code presented in the book, but the book didn’t come with all of the source in separate files. It had a few files, but it was missing some of the JSpec files that I would have found most interesting.

Bookmark

But when I could start hooking up my designs to really cool functionality, such as dmMonocle, my jQuery plugin for viewing images, I loved the combination that was possible. I could not only make beautiful designs, I could make beautiful web applications. My code could react to the actions of users — or in JavaScript lingo, the events of users.

Server side JS?

I’ve since learned to branch out, and have an affection for Python as well at this point! But I’m still in that weird designer/front end dev gray area, so I don’t get as much Python time as I’d like. However, with the advent of Node.js, I can now do some pretty awesome server side stuff in my favorite language. That’s right, you heard me: JavaScript is going server side — and even further in the future! Here’s the code I wrote to implement a very simple web server with Node.JS:

var fs = require('fs'),
    sys = require('sys'),
    http = require('http');

// Simple Server Function
// Pulls the url of the request and responds with the appropriate HMTL file
// If no HTML file found in the request url, returns a 404 error. Boosh!

http.createServer(function (req, res) {

    var HTML;

    if(req.url.search(/\.html$/) > -1) { // Searches to see if the URL entered by the user ends in .html
        fs.readFile('../' + req.url, encoding='utf8', function(err, data) { // If it does, it tries to read the file
            if (err) { // Oh noes! The file doesn't exist! 404-butter time!
                _404(err);
            } else {
                HTML = data;
                res.writeHead(200, {'Content-Type': 'text/html'});
                res.write(HTML);
                res.close();
            }
        });
    } else { // Hey, they didn't enter a valid URL! You know what that means -- 404 BUTTER TIME!
       _404();
    }

    function _404(err) {
        if (err) sys.log(err);
        fs.readFile('../404.html', encoding='utf8', function(err, data) {
           HTML = data;
           res.writeHead(200, {'Content-Type': 'text/html'});
           res.write(HTML);
           res.close();
        });
    }

}).listen(8080); // Listen on port 8080
sys.puts("Brian's HTML Server running at http://127.0.0.1:8080/"); // Output a string to STDOUT (the command line in most cases)

Lame, but WTF is require()?

Now, this server isn’t very durable. For example, the html files have to be one directory down from the node server — not awesome, and it serves no CSS. But it was just a simple test script I wrote in 5 minutes to see what could be done with Node, and was a heck of a lot of fun. It also introduced me to the CommonJS project, and their implementation of modules. First of all, the CommonJS project aims to produce a common spec for server-side and application-building JavaScript — i.e. JavaScript’s not just for browsers anymore!

Now on to modules: at the top of the code, you’ll see I’m declaring 3 variables, all of which call a require() function. That require function is an example of the CommonJS module implementation, a way for JavaScript to pull in the contents from another file in a similar fashion to Python’s import or Ruby’s require.

The way of the future?

Pretty cool stuff for a few minutes of compilation time, reading through the Node.JS API (which only takes about 45 minutes even if you’re slow like me), and then getting a little dirty — and by that, I mean not dirty at all.

There’s some real potential here, and I’m going to take some time to play around with Express. It’s a Node.JS web framework inspired by a Sinatra, a Ruby web framework. Sinatra is another intriguing piece of software, and combined with HAML and SASS, two great technologies for rapidly developing HTML and CSS, it’s a very nice package indeed. Express seems to have adopted those parts of Sinatra, and should be just a grand old time to tinker with.

Bookmark

Flame war! (Intro)

I was actually a sociology major in college, so I kind of enjoy the rigor of trying to test things well. This is not a rigorous test — but I did learn some fun stuff!

Over at NETTUTS there was a discussion about the best way to update the DOM using jQuery. The author suggested this method, and was promptly called an idiot:


var updatedText = "Value1
Value2";
$("#container").html(updatedText);


I actually used a variation on this method when writing dmMonocle, my jQuery plugin which makes viewing images within library digital collections, such as the Southern Nevada Boomtowns Collection, a bit easier. What dmMonocle does is generate a series of tiles (divs) and then injects them all together at once, just as the author suggested above. Others claimed that the built-in jQuery methods (.html() & .append()) were more readable, and should therefore be used.

Test me once, shame on you (Methods)

In these cases, I like to let the numbers speak for themselves, and so I created a series of tests using JSFiddle that test the performance of three methods. I posted a couple of simple tests to NETTUTS this morning, but I thought it would be fun to get more in-depth and play around with loops of console profiles and timers to test performance (you know, how every sane person likes to spend their time)!

I wanted to time and profile three different functions:

• Build a string of spans and inject it once into a container using the .html() method
• Update each one of the spans individually with $(#span).html("new html")
• Building the spans within the contain using the .appendTo() method.

As the bad joke of a subheading should tell you, we don’t simply want to run these timers and profiles once, we want to run them 100 times (more would probably be better). To run them once would be a grave mistake. Don’t you remember doing 10 trial runs in science class to eliminate error? Did you think the cheeky lab report workflow of this post was a mistake?

I’m also running these tests on a Summer 2009 Macbook Pro, and would love to hear about results from other machines.

Show me the code! (Data)

• Timer Tests
• Profile Tests

As I said, these tests run each method 100 times. Scratch that, Firefox ate it when I tried that =P I’ll just run em a bunch manually and record numbers the good ol’ fashioned way. I’ll work on best practices for averaging results in a future post

Chrome

My new browser of choice:

1. First Test Average: 2-3ms
2. Second Test Average: 6-8ms
3. Third Test Average: 8-10ms

Firefox 3.6

Tried and true (but slooooooow):

1. First Test Average: 11-12ms
2. Second Test Average: 21-23ms
3. Third Test Average: 37-39ms

I mean look at that. 3-5x slower than Chrome 4 on simple operations.

Safari

Safari was… creepy with how consistent it was. I think I may be doing something wrong

1. First Test Average: 3ms
2. Second Test Average: 6ms
3. Third Test Average: 16ms

Internet Explorer

I can only run IE on a virtual machine at the moment, and I’m not sure how reliable that is. Give me some data!

Blah Blah Blah, what’s the bottom line (Results und Discussion)

Bottom line is that, according to my tests, using native JavaScript strings to build the html and injecting it once is roughly twice as fast as the updating method, and 3-5x times faster than append method (which is the most similar to the original string function). These results are due to the fact that the string method skips over calling not only the jQuery constructor $() on each iteration, but also the jQuery methods .html() and .append(), which themselves call multiple functions (as demonstrated by the profiling tests).

Based on those facts, when I need to inject any significant amount of code, I’m going to try to do as much building as I can in regular old JavaScript, avoiding the DOM and unnecessary jQuery methods as much as possible. Look, jQuery is great and really fast; I mean, I’m literally talking about 2-5x slower than 2 miliseconds). But, I’ll basically take any speed advantage I can get, and in this instance there is a clear winner. I personally don’t think the JavaScript is too much uglier or less maintainable than the jQuery fashion, but that is really a matter of taste. When it comes to the data, there’s no debate.

Bookmark

Starting with SEO

I usually don’t start with SEO, but I have a few friends who run a blog called Scrappy Marketing, and they put up a great post entitled SEO 101 for your WordPress site – In 7 easy steps. Because I had never even installed a plugin before, going through this tutorial was a great introduction to some of the UI features, such as easy plugin installation,  configuring said plugins through the “Settings” menu, getting setup with Akismet, etc. Essentially, they have you:

1. Activate Akismet, the spam blocker for WordPress. Done and done.
2. Download and configure “All in one SEO” for better meta information and title tags
3. Get Feedburner rockin for better stats about my readership
4. Download Evermore, which adds the “more” tags automatically. (I actually skipped this step)
5. Download and Install Social Bookmarking Reloaded. I did, but I’ll probably look at other options or figure out how to customize this one a little more to my liking.
6. Install XML-Sitemap, making it easier for teh robotz to evaluate your worth
7. Setup a permalink structure. Clutch.

What a great way to begin. I got to work with WordPress a bit, and also configured some important things up front, such as my meta data, permalink structure, and Feeburner. Not a bad way to spend 30 minutes!

Local Dev Environment

After instlalling those plugins on the server, I realized that I was probably going to need a local dev environment to do my theming. No worries there: I just grabbed the free version of MAMP (Windows users can grab xampp, Linux users — you know what to do =P). I also use a program called Navicat Light, which is basically a desktop version of phpMyAdmin, to create databases and users.

After firing up MAMP, using Navicat Light to get my database ready, and installing WordPress locally, I was ready to move onto playing with some themes!

So… how does this work?

I’ve worked with PHP and Django themes enough to know the basics of how to pull the content I need, but I thought it would make sense to start by looking at other porfolio themes to see how the structured their theme folder, categorize content, and pull it through with specialized templates. While I thought about buying a professional theme, I found some good free ones from the following links:

• Using WordPress as a Portfolio: Examples & Themes
• 100 Excellend Free WordPress Themes
• 83 Beautiful WordPress Themes

After installing 10 or so from those sites, I feel like I have a pretty good idea of how I’ll proceed from here. In my next post, I’ll discuss what I’m doing in the background, from my initial brain dump, to sketching compositions, to my first photoshop mockups. Stay tuned!

Bookmark