Somehow you have ended up in a technical leadership role and the first thing you need to do is build a team. Alternatively, you are looking to grow an existing one. But where the heck do you start?

This is a challenge I am facing in my current role as CTO of Higher, a SaaS start-up in Perth, Western Australia, where we are building a platform for digital freelancers to better manage their client relationships and projects.

I have also faced this issue a few times previously in my career, and it can be surprisingly nuanced based on the company, team, culture, economy, and locality in which you are situated. Having worked at large corporates and start-ups alike, I can honestly say that there are always new and interesting problems to get in your way. However, there are also repeated themes that, if handled in the right way, will lead you down a path of success. With that in mind, this article is an attempt to condense my experiences as a CTO/VP Engineering at start-ups, as a Senior Development Manager at Amazon, as an Engineering Manager at Betfair and Yahoo! UK, and overall simply as a hiring manager into something that somebody out there might find helpful.

Personality goes a long way

Remember you are hiring people. People just like you, with their own aspirations and concerns. Walk a mile in their shoes and remember that every person is fighting battles you know nothing about.

It horrifies me how often I have to bring this up, usually because someone has abstracted the idea of a team and its members away as a “resource” or “tool”. It might not be you; it could be your manager or the business as a whole. In this case it is down to you to bring back the humanity to the process. There are no excuses.

If you treat people with respect, honesty, empathy, and transparency, you will sow the seeds of success. Do not let an over-zealous leader or business process prevent you from representing both the candidates’ and the team’s best interests above everything else. Believe me, I have made those mistakes and they are some of my deepest regrets. Be open. Be approachable. Help wherever you can, even if that help is not in the best interests of the company but is in the best interests of the candidate. Strong relationships and respect are built on these foundations.

Next we need to do some preparation…

Understand the needs of the individual

Understanding the needs of the individual is important so that you can ensure you are meeting those needs, both as an employer and as a leader. My boss in a previous role—whom I’m lucky enough to now consider a mentor and friend—liked to re-purpose Maslow’s Hierarchy of Needs for this. (He liked to re-purpose it for customer/user interactions too, but that’s a post for another time.)

Maslow’s Hierarchy of Needs is a psychological theory that visualises five layers of human motivation in a pyramid with base needs at the bottom. Those motivations are, ordered from most needed to least:

1. Physiological
2. Safety
3. Belonging
4. Esteem
5. Self-actualisation

For those able to see it, here’s a visualisation:

If we re-purpose these motivations towards a person’s job, we might better specify them as:

1. Physiological: The need for income to survive.
2. Safety: Security of job to protect income, family, health, property.
3. Belonging: Liking the people we work with, friendship, team.
4. Esteem: Confidence, achievement, respect by others, respect of others.
5. Self-actualisation: Creativity, problem solving, freedom to make a difference.

If you want to build and manage great teams, it is imperative you ensure all of these things are available to the individuals that make up those teams. Often I find the first three (physiological, safety, and belonging) are taken care of, but the last two (esteem, self-actualisation) are over-looked or poorly managed. These are the basis of your working culture, not what you wear, or the music you listen to, or your out-of-work activities, and definitely not your propensity for consumption of food or drink (yes, I’m talking about after-work drinking, which many people do not enjoy, or cannot be a part of for various reasons).

It is important to provide definition for team members. They should have a clear understanding of their roles, their career progression, and the differences between each stage of that progression. It is these differences that will provide them with goals for individual growth, and allow them to understand why other team members are classified they way they are. This removes ambiguity and helps folks understand the expectations upon them and others, and sets-up clear boundaries and responsibilities.

What is my role?

Understanding what you do within an organisation is the foundation of those first two layers of Maslow’s hierarchy. Any form of ambiguity here introduces uncertainty, and uncertainty leads to a lack of security and potential employee churn. To avoid that, it is important we can show all employees—both new and current—that we have spent time ensuring they step into a well understood role, and that they also have room to grow and change however they choose.

Job specifications are required for all the roles in your business so that the individuals within those roles know how they are evaluated. Their use in hiring should be their secondary value. In my experience, actually defining the difference between an entry-level developer, a staff-level developer, and a senior developer—and discussing and amending those differences based on feedback from the teams themselves—will fix misconceptions and set the business up with a better understanding of roles and responsibilities. This is particularly useful where boundaries are muddied.

There are a lot of public resources available to help with defining job roles and expectations. I’m a big fan of the GovUK Digital, Data, and Technology Professional Capability Framework amongst others. I’m also a big fan of Performance Profiles as an improved method of definition. Use these resources as a guideline to define your own roles, and work with the people already in the business to better understand what they think their roles and responsibilities include.

What are my opportunities?

As stated previously, it is important that employees understand their room for growth within both their role and organisation. This means laying out clear career paths for progression, and allowing them to figure out what direction they want to take in their own time. This does not mean only moving down the paths they are on right now; they should also have opportunities to move sideways into different disciplines or specialisations.

When dealing with engineering teams, it is normal to see two career tracks; one for managers (e.g. senior -> lead -> senior manager) and one for individual contributors (e.g. senior -> principal -> architect—where there is no expectation of people management). This is fantastic when thinking in terms of moving up the ladder, but it doesn’t translate to specialisations such as front-end/UX/UI design and accessibility, client-side architecture and state management, data processing and consumption, machine learning, network programming, system operations, developer tooling etc. Most engineers will have one or two of these areas that they are passionate about and allowing them to become the subject-matter experts within your team or organisation is an equally compelling story for their personal development.

Most mature organisations will have some form of career path matrix that aligns completely across the entire business. Creating such a matrix is key to ensuring fairness in terms of compensation and benefit across all career paths within your organisation, and enables staff to understand levels of seniority. Mapping this matrix to your already defined job specifications then provides clear communication of the qualifications and expectations of each role, and how an individual might then change role or career path within it—even if they are jumping from being a junior sales person to a junior developer or vice versa. The key is to ensure your employees aspirations are only limited by their own choices and commitment, and that there is clear definition of requirements to support a move in any direction they choose.

Once you have done this, I recommend you map clear open salary bands to each level within your career matrix. The aim here is to make all salaries ultimately visible, thus limiting any potential variation that is the antithesis of diversity within a business and focusing very clearly on building trust with all employees. The best organisations have public salaries, because individuals are paid fairly within understood banding that maps to the requirements of their roles and responsibilities. Unfortunately, this isn’t easy to achieve, and requires constant market management to avoid problems of limited flexibility during new hire salary negotiations, and internal salary adjustments. Despite this, it is my personal belief that this model should be the aim of any mature organisation. The management effort is outweighed by the trust and engagement it inspires within the team, and it completely nullifies any whispered politics around pay.

Understand the needs of the team

The next thing to understand is the structure of your teams, and how you intend supporting cross-functional development and support. Often this will depend on the methodologies you are intending to use for project/programme management, with team design evolving organically from the way your business handles the flow of work from inception through to delivery.

By “team design”, I specifically mean how many teams you have, how many people are on those teams, what their individual roles are, and what those teams are going to be responsible for. Ownership and accountability will be defined by this, so it’s important you give it a fair amount of your time unless you’re able to grow an ownership model organically as your business grows. Disclaimer: This task can be really difficult. I have failed at this even recently, and retrospectively I think the changes I needed to make happen were outside the remit of my role—I needed to work more intensively with the wider business to achieve a fundamental change to the business operating model which could support a more successful organisational design within Engineering.

Function vs. product focus

In 1967, a programmer named Melvin Conway introduced the following idea (now known as Conway’s Law):

organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.

To translate into slightly less impenetrable language:

Any piece of software reflects the organizational structure that produced it.

A study of multiple large codebases was undertaken at Harvard Business School to validate this particular law. The study took multiple examples of software created to solve a particular problem (e.g. word processing, financial management etc.), and compared the codebases of loosely-coupled open source teams, and those of tightly-coupled teams. They found that the often co-located, focused product teams created software that tended more towards tightly-coupled, monolithic codebases. Whereas the open source projects resulted in more modular, decomposed code bases. This illustrates the need to design your organisation backwards from how you wish to deliver code.

Having worked in several larger organisations, its interesting to me that the ones with the greater success of consistent delivery are those that allow autonomy within their product and engineering divisions based on direct access to customers. Amazon, as an example, encourages a technical leader to treat his organisation as its own start-up, creating an environment where a team is forced to consider their users as customers, even if they are internal, and to plan and deliver their roadmap accordingly. In addition, operational performance is constantly measured and assessed using specifically identified key metrics for each subdivision. You cannot have function-focused teams working for a customer/product driven business unless there is some form of further adaptation in place which understands how to divide work for each function.

Tech start-ups, on the other hand, are often intensely product-focused due to growth around a limited use-case or customer cadre. In this case, it is more likely that the team will make the mistake of becoming too focused on the needs of one specific big customer, in order to keep the business. This can result in a loss of grip on the boundaries of a product or service, or worse, it turns the team into an outsourced pseudo-agency development resource for that one controlling customer. Strong product management is important here, but that product management needs to be the adapter between customer demand and intelligent technical delivery. Knowing how to convert requirements into user stories that make sense for functional teams is an undervalued skill. However, that’s a topic for another time.

Avoid skill silos

Enabling autonomy in a team requires a reduction in the opportunities for blockers. Back in the dark ages we suffered through a divide between ops, back-end, front-end, and QA skills. By grouping teams by skills we created artificial walls between stages of development and delivery. This enforced waterfall working—even though we claimed to be agile—because we had to wait for dependencies to complete further down the stack. The end result was that iterations were focused more on getting work over these boundaries, rather than rapid feedback loops with users.

In my opinion, development teams perform better when they are truly cross-functional. However, it is also important to realise that no two cross-functional teams will be exactly the same. Cross-functionality comes from understanding that a good team is organically built-up around the skills of its members. If we imagine each skill “specialisation” as a circle in a gigantic Venn diagram, each team member will appear on that diagram in quite different places, dependent on their key specialisations and competence in other areas. With that in mind, it makes sense to balance skills using complimentary team members, allowing for some degree of overlap. If I do not have the skills to complete delivery of a specific task, it is important that I can rely on another member of my team to support me through it. A team of complimentary peers is a powerful thing, especially if each team member has a little imposter-syndrome—this is a sure sign they’re in an environment of learning and growth, supported by everyone around them and not just their mentors or leaders.

This also relates to the concept of “full-stack” engineers—or what Amazonians often call fungible engineers. These are engineers that are equally capable across the entire stack—from the toughest cross-device-and-browser front-end challenge, through API and system architectural separation woes, to the gnarliest operation tooling problem—therefore making them fungible or flexible to move around, solving the business problems with the highest priority. In my opinion, however, this is a recruitment-driven misidentification that belittles the unique talents and experiences of each individual. I have an entire other post in progress on this subject. There are plenty of jack-of-all-trades developers, but they will always have a tendency towards that one particular specialisation. Which means they are actually unlikely to be “fungibly” valuable (otherwise described as flexibly valuable) across a wide array of problems, and are far more likely to create more issues than they solve. Any capable engineer knows what it is to grow through iterative failure; to learn from their mistakes, and to lead others with this invaluable knowledge. The process of doing this will always result in a specifically targeted skill-set. Also, I believe any truly self-aware engineer understands the effect their team has on them as an enabler. As previously stated, it is cross-functional teams that become flexible. Do not move individuals between problem spaces; move the entire team. This means you gain all that valuable talent and working culture without having to rebuild relationships between team members.

An additional method to avoid skill silos in teams is to encourage cross-team expertise, either as working groups—where individuals in a team also work with like-minded specialists in other teams to create standards and improve quality—or as satellite teams who are specifically designed to provide support in a given proficiency alongside any team that needs it for a given deliverable. My best experience with this has been in testing, where a “roaming” team of test-automation engineers spend time seated with a given team, helping them ramp up their skills and quality standards before moving on to another team. I have also seen this employed as “tiger teams”, who are a cross-functional team with significant maturity who may not have any one specific area of ownership, but can be deployed to help solve high priority problems within your entire stack. This is the absolute peak of an effective cross-functional team, as spoken about before.

Team size

Despite being the topic of great debate, I believe Amazon have significant cultural maturity, driven in no small part by Jeff Bezos’s own pearls of wisdom. A good example is the “two pizza team” rule, which advises not to have any teams larger than can share two pizzas together. This rule is based entirely on the rapidly increasing number of relationships between individuals that need to be subconsciously managed with each added member of the team. This can be modelled with the equation:

l = n(n-1)/2

And if we visualise that with a graph:

Not only do larger team sizes make for significantly greater relationships to manage, but they have also been proven to make people overconfident. Teams have a tendency to increasingly underestimate task completion time as their size grows, due to the optimism of the group. In their paper, “The Team Scaling Fallacy”, Bradley R. Staats, Katherine L. Milkman, and Craig R. Fox observe that when tasked to build the same Lego kit, teams of two people took 36 minutes while teams of four took 52 minutes to finish. However, the larger teams were almost twice as confident about how long they’d take. That kind of error in estimation can be extremely costly for a growing business or team.

In my experience, the sweet spot is between 5-8 folks, but opinions differ—even within Amazon itself. Based on the equation (and graph) above, 8 folks is a maximum of 28 relations to maintain between the individuals in the team, which makes for about the right balance between delivery speed, accuracy of estimation, and efficient communications. At this point, if you need to scale further, add another team and divide up the problem-space so that each team can have very specific ownership, control, and accountability.

Seniority

The other question I am asked a lot is, “what does a good team make-up look like in terms of seniority?” Unfortunately, there’s no single answer to this. It really does depend on the people. As an aside, it is my opinion that seniority is usually contextual to a team and business anyway, and those that are considered senior in one company and role are unlikely to be senior in their next role. However, they will have a greater potential to excel and will likely rise in seniority faster than similar peers who have not been senior before.

So what is seniority? Basically two things:

1. Autonomy; the less guidance an individual needs in applying themselves to delivering team and company goals (without creating confusion or ambiguity), the greater their seniority within a team.
2. Altruism, support, mentoring; accepting that seniority relates to greater autonomy, it follows that more senior folks spend more time working with team members that need more guidance and support. Senior team members invest significant effort in making the whole team better at what they do.

It is also worth noting that seniority at one employer—in one position—is very specifically contextual. There should be no expectation to transfer seniority, earned through experience or longevity, to a completely different role and company. Seniority needs to be consistently demonstrated through action. With that in mind, it is important as a hiring manager to make real use of a probation period to ensure a new hire is meeting your expectations. This really isn’t something you can confirm simply through interviews alone.

In terms of team structure, my personal preference is to have one or two senior folks per two-pizza team. However, I can’t emphasise enough how important it is to ensure those two folks are mutually complimentary. Having two senior folks on a team who are often at odds with one another is going to be one of the most destructive things you can do to your business. The major aim of any team structure design is to ensure that every member is complimentary to their peers, and that each individual brings something new and lifts the performance of the team as a whole. With senior folks, this effect is obviously going to be even more impactful as they will spend more of their time mentoring those with less autonomy.

As another aside, whilst I was writing this post over the past few weeks I came across this great and relevant post on an anonymous blog/newsletter called LittleBlah (whoever you are, I’m loving your work). Needless to say, I’ve subscribed:

A Senior Engineer’s CheckList

The right make-up of seniority across your teams is largely going to depend on how urgently you need deeper hands-on experience (like managing operational systems in production, as an example) vs. whether you have the flex to grow a team’s experience and skills organically over time. In my experience, organically grown teams perform better 80% of the time because they have developed together over time, and are more able to second-guess their colleagues and team mates. In comparison, if you hire too many senior level folks too early, you may have problems of contention and disagreement over the best ways to solve certain problems. It’s also worth noting that hiring more senior folks is going to take time, so you may not be able to have one or two per team at any given moment. In this situation, you need to ensure the senior folks are given flexibility to work at mentoring across all teams.

All that said, I’ve worked in very successful teams of both types. I do not think there is any one correct answer here, but I would always favour less senior roles and more entry-level. There are always more entry-level folks than seniors over time, and it’s incredibly rewarding to take folks on and help them “level up” before such time as they choose to leave your employment, for whatever reason. It doesn’t matter what your business is, or what you’re trying to achieve; ultimately it should always come back to the people you employ, and doing right by them. In many ways you’re setting an example as to how you expect them to treat other people, which will include your customers or clients too.

Understand the needs of the company

So now we understand the needs of the individuals we’re hiring, and the roles and team structures we’re hiring them into. We’ve also figured out that we need to be able to translate customer/product requirements into a set of well-bounded ownership teams with specific deliverables. But what about ensuring the teams are adding true value to the company? And how can we ensure we’re attractive and engaging as an employer? This is where company culture becomes important.

Many large organisations spend time defining the characteristics and culture they think make them successful, and share them publicly. This is an important tool in allowing current employees to understand what the company is expecting of them, but also in attracting new employees. A few good examples are:

• Netflix: https://jobs.netflix.com/culture
• Amazon: https://www.amazon.jobs/en/principles
• Facebook:https://www.facebook.com/careers/facebook-life/

The folks over at Tettra have also collated a whole bunch of culture decks here.

At a bare minimum, you should think about defining out a business summary, vision statement, and mission statement; even if you’re only generating them for a single internal ownership team. These will help you formalise the language and imagery you use to describe what you do, and what you’re trying to achieve as an ownership group. This also helps unify these ideas across all your team members, which is super important if you want them to work as a coherent delivery organisation.

This stuff can sound very “soft and fluffy” (for want of a better description) to development teams, which means they often overlook the immediate value. Believe me, I have never delivered this information to a team and not received feedback that they had somehow misconstrued what they were building, or that it had somehow bridged the gaps in their understanding. People like to have a strong grasp on what it is they’re doing day-to-day. Literally anything we can do to help formalise or realise that should be considered priority.

In the longer term, setting out a set of standard behaviours or values for the team will prove to be an important group task. It can be the difference that enables confident decisions and strong plans and avoids confusion and ambiguity. Moreover, it helps provide team identity which affects our sense of belonging, as discussed way back in Maslow’s Hierarchy of Needs.

Key Performance Indicators

Another important tool in helping a team understand their place in the wider business are key performance indicators. At their most basic, these are a set of 1-3 metrics that the team agree will be used to measure their success. It’s important these metrics are;

• well-defined and quantifiable;
• fundamental to achieving the team or organisation goals;
• applicable specifically to the teams affect on the business needs;
• communicated throughout the organisation and the team.

At a higher level, it’s usually better to ensure there are KPIs defined at the business level, and that they are then repeated and transformed down to more suitable KPIs for each team within the business.

Once these metrics have been decided on, the team should then set about constantly measuring and reporting on them, allowing an iterative feedback and adjustment loop. The sooner a team have this sort of process in place, the sooner they will be able to both grow in effectiveness, but also to learn from their mistakes and share that knowledge with the wider organisation.

All this said, I have personally found that trying to wrestle KPIs all the way down to the personal level is a lost cause. Personal goals can certainly reflect the premise of a particular KPI, but they need to be individually tailored to personal growth, rather than that of the business. KPIs provide leadership and direction to teams, but they are difficult to deliver against for individuals, who should be working to a slightly different but complimentary agenda.

Summary

So if you want to build better teams, you have a bunch of things to think about and deliver yourself:

1. Generate a business summary, mission statement, and vision statement.
2. Build a career matrix showing career paths and job specifications.
3. Balance the skill and seniority split of your two-pizza teams.
4. Figure out the ownership of those teams, to support the architecture of your product.
5. Work together to come up with some company values.
6. Define business and team level KPIs.

Generally, you’ll find once you’ve got all this sorted out, it will be much easier to sell your culture and brand during hiring activities, and your team will naturally attract like-minded individuals. By representing the best interests of the individuals, they are more likely to talk favourably about you as an employer in social situations which will spread through networks quickly.

In addition, by having this all sorted out in your own head, it becomes easier to provide documentation and points of reference for your team. This means they will feel better supported in their growth, and are therefore likely to be more engaged over time.

This is, of course, only the start. Once you’ve got this all worked out, you’re going to need to start writing performance profile job specifications; figure out a sensible interview process that allows you to identify technical skills, autonomy, altruism, experience, and anything else you come to value; and design a probation process and ongoing scheme of 1:1s that provides the right environment for goal setting and support, allowing your employees to grow significantly during their tenure with you and your business. But that’s a whole other blog post really, and I’ve already taken up enough of your time.

Finally, and unashamedly, if you like the sound of the above and you’re a React/RoR developer, and you’d like to come work with me at Higher in Perth, WA, by all means reach out to me via LinkedIn.

Software developers like lists. They like to log-on and focus on working their way through a list of deliverables, checking them off one-by-one as completed. Developers like it even more if they are given clear requirements for each of those deliverables, because having to dig up or define your own requirements requires context switching away from development. Context or task switching away from development is a form of process waste—see my previous post, “Eliminating Web Development Waste“—and software developers do not like wasted time.

When we adopt an agile process for software development, the product backlog becomes the list of things to complete. It is at this point I often see developers exhibiting frustration because the backlog is no longer as clear as the to-do list they had made for themselves. When faced with this frustration, my advice is to remember that the backlog is a list of work items for greater audiences than the development team.

The backlog provides a product source-of-truth for everybody; from sales through to support. Each item in the backlog needs care and attention to ensure it is easy to read, contains all relevant reference materials, and provides enough explanation that anyone in the business can understand the user need and the proposed solution. That said, it’s also important that everyone accepts that each item in the backlog is a work in progress, and that gaps should be acknowledged (but filled in as soon as possible). Being agile means we should recognise it is a pipe dream to expect perfect requirements for every user story or task we undertake. Change is inevitable, and the agile manifesto acknowledged that when it stated:

Customer collaboration over contract negotiation
Responding to change over following a plan

Over the past few years, I’ve worked with a few businesses that have started down the agile path for the wrong reasons. They believe it will make them faster, because they’re delivering incrementally. Inevitably, these businesses then get into all sorts of issues because they fail to adapt their processes to allow clear conversation between the actual users and the developers. Agile is not about speed of delivery but speed of correction, and every layer of business function between the folks using the software and the folks designing and building it is a layer of misdirection in that correction.

The whole point of iterative development and incremental improvement is that we have a shorter feedback loop. We avoid veering off course from the user’s requirements and deliver small improvements, instead of one massive release that somehow lost its way and missed the brief. What’s more, good feedback requires direct exposure to users, who can tell us whether we’ve hit the brief or whether we’ve missed something.

Which brings me back to the backlog being a list of stuff to do. Being agile also means recognising that one user-story or epic does not have to be the final word for a given feature. Stuff gets missed. Stuff ends up broken. It is entirely acceptable to create further tasks and stories to patch stuff up, even if we’ve already delivered an epic for that feature. The best agile backlogs are made up of a lot of tiny deliverables with very simple acceptance criteria that help us avoid over-engineering but that iteratively build up to a whole that is greater than its parts. Write your stories and tasks to deliver value against a clear user need, but with the minimum viable acceptance criteria. Anything else can come later.

So next time you’re irked about the backlog, remember:

1. It’s not just for you; it’s for everyone, including the users.
2. It’ll work better if you keep user stories and tasks simple and direct.
3. Improvement between iterations requires feedback and conversation with everyone, including the users.

Secured connections require a certificate, which needs to be issued by a trusted third-party certificate authority (CA). This used to be costly and slow and required a great deal of manual jiggery-pokery. However, thanks to Let’s Encrypt, a free automated CA, obtaining certificates for SSL/TLS is now very easy indeed.

With that in mind, let me walk you through activating SSL/TLS for your web server using CertBot, an automated tool that will implement SSL and regularly update the required certificates via Let’s Encrypt.

Certificates

The SSL/TLS certificate verifies and creates secure sessions between client and server over the SSL protocol. This secure connection cannot be established without the certificate, which digitally connects company information to a cryptographic key. Certificates are issued by a Certificate Authority, who certify ownership usually through payment and then a further validation process against the domain in question. However, these days there is a better option.

Let’s Encrypt

Let’s Encrypt is a free, automated, and open [certificate authority, run for the public’s benefit. It is a service provided by the Internet Security Research Group (ISRG), [and they issue certificates for free simply because they believe in a more [secure internet for everyone.

It’s worth noting that Let’s Encrypt only issue Domain Validation (DV) certificates, and not the more secure Organisation Validation (OV) or Extended Validation (EV) certs as issuance is more complicated for these. Be sure to understand the difference between these certificate types because you may well do better implementing the alternatives for more trust in your business. There’s a good explanation of the differences between certificates over at SSL.com. There’s also a great explanation of how Let’s Encrypt works over on their site, so I won’t waste too much time on it here. In short, by making the issue of DV certificates free, the ISRG have made the entire process of implementing SSL/TLS on your servers almost painless.

Certbot

Certbot is an easy-to-use automatic client for Let’s [Encrypt that fetches and deploys SSL/TLS certificates for your webserver. If [you’ve ever undertaken the manual struggle of certificate management, Certbot [will be a breath of fresh air.

As a workflow, you simply install Certbot on your server, run through a short configuration process to identify the sites you wish to turn SSL/TLS on for, and then it will do all the hard work requesting certificates, installing them, setting up your webserver configuration, and restarting the hosts. It’ll even create redirects from HTTP to HTTPS if you want them. What’s more, it’ll keep an eye on your certificates and update them when the time comes, so you almost never have to think about it again.

Installation and configuration

If you refer to the Certbot documentation, they have walkthroughs for common cross-sections of webservers and hosts. In my case, I was working with Ubuntu Bionic and Nginx:

$ sudo apt-get update
$ sudo apt-get install software-properties-common
$ sudo add-apt-repository ppa:certbot/certbot
$ sudo apt-get update $ sudo apt-get install python-certbot-nginx

This got Certbot installed on the host, and then all I needed to do was kick-off the configuration process for Nginx:

$ sudo certbot --nginx

Certbot will then ask which sites you want to activate SSL/TLS for in your Nginx configuration, will reach out to Let’s Encrypt to get certificates and install them, update your Nginx configuration, and restart the Nginx service.

If you’re a little more precious about your Nginx configuration files, Certbot can be directed to only obtain certificates with the following command:

$ sudo certbot --nginx certonly

Obviously, you’ll then need to update your Nginx configuration yourself.

At this point you have successfully set-up your site(s) for SSL/TLS and can happily go about your day, safe in the knowledge your users’ connections are encrypted.

Automatic Renewal

The Certbot packages also installs a cron job that will renew your certificates automatically before they expire. This is a great feature because Let’s Encrypt certificates last for only 90 days.

You can test automatic renewal for your certificates by running this command:

$ sudo certbot renew --dry-run

Wildcards

It’s also possible to have Certbot obtain a wildcard certificate that will also cover all subdomains for your domain. However, this requires installation of DNS plugins and use of their ACMEv2 server. You can find information about how to do all this in the excellent Certbot documentation.

Summary

As you can see, going the extra mile for your users and encrypting their connections is actually pretty painless when you adopt Certbot and Let’s Encrypt. Hopefully this short post will help a few folks get over the fear of implementing SSL/TLS, and help to ensure more of the web is secure!

In a recent conversation with a bunch of old web development and design friends, discussion got around to lamenting that a bunch of us seem to have stopped writing posts and articles on our personal blogs. With that in mind, I’ve decided to resurrect this old thing and start writing again.

In many ways, Matthew Pennell’s “Slight Return” post (potentially an artefact of the same conversation mentioned before) really resonated with me. I believe one reason for my being AWOL is the fact that each of my articles took a great deal of commitment. I would spend a lot of time double checking my sources, and crafting my tutorials, deeply aware of the intelligent scrutiny of my peers. Five years later, and I realise that there is just as much value in the simple personal gratification of documenting stuff I’ve been playing with, so that is probably mostly what I am going to do.

First things first, though, I need to enable some HTTPS on this baby.

Which gives me a great idea for the next post…

Eliminating waste is the first—and most fundamental—principle of Lean Software Development; something that was introduced to me by an exceptionally talented Project Manager whom it was my pleasure to work with. Lean Software Development is a set of seven fundamental principles that really work hand in hand with agile development to improve software delivery. Those principles are: eliminate waste, amplify learning, decide as late as possible, deliver as fast as possible, empower the team, build integrity in, and see the whole. In this article I’m going to look at how we can improve our ability to identify waste and iteratively remove it from our development chain.

Lean manufacturing

The principles of Lean Software Development are basically reinterpretations of the principles of Lean Manufacturing as designed and tested by Mary and Tom Poppendieck. Those original manufacturing principles—formulated by Sakichi Toyoda, the founder of Toyota Industries, and his son Kiichiro Toyoda, the founder of the Toyota Motor Corporation—were realised as an actual method of production by the Toyota engineer Taiichi Ohno. This later came to be known as both Just In Time Manufacturing and the Toyota Production System.

Lean Manufacturing is a production practice that considers the expenditure of resources for any goal other than the creation of value for the end customer to be wasteful, and thus a target for elimination. Working from the perspective of the customer who consumes a product or service, “value” is defined as any action or process that a customer would be willing to pay for. At its most simple, Lean Manufacturing can be considered a move towards manufacturing to order rather than manufacturing to build stock since stock is essentially waste.

In reading descriptions of American supermarkets, Ohno saw the supermarket as model for what he was trying to accomplish in the factory. A customer in a supermarket takes the desired amount of goods off the shelf and purchases them. The store restocks the shelf with enough new product to fill up the shelf space. Similarly, a Toyota work-center that needed parts would go to the inventory storage point for the particular part and withdraw the quantity it needed. The storage point would then be restocked by the work-center that produced the part, making only enough to replace the inventory that had been withdrawn.

The Toyota Production System

The key objective of the TPS was to design out overburden (muri) and inconsistency (mura), and to eliminate waste (muda). A manufacturing process that removes inconsistency is capable of delivering smoothly. However, that process also needs to be as flexible as possible so as not to introduce overburden which creates waste. Also there is clear value in the elimination of waste in any process.

TPS Reports

Anyone who’s worked in the industry for a while, or has seen the film Office Space, will be familiar with the term “TPS reports” being synonymous to pointless paperwork. TPS reports have nothing to do with the Toyota Production System, the TPS stands for “Testing Procedure Specification”.

That being said, TPS reports as pointless paperwork is entirely relevant as waste that could be removed from your processes. We’ll come back to that later.

Web vs. Software Development

Before we get too involved in investigating the waste element of this post, let’s just examine the difference between general software development and web development:

I consider myself to be a web developer and a web development manager. I am used to working with the internet as part of my development landscape and also as part of my toolkit. This means I am used to dealing with high numbers of users, a vast array of user-agents which interpret my code and therefore need to be tested against, a degree of uncertainty as to the capabilities of the user-agents and the users themselves, and a wide number of technologies. This has been described by Douglas Crockford—the inventor of JSON, and often described as the godfather of JavaScript—as the “most hostile environment imaginable”.

This is not to say that software development is any less hard or varied, just that there are a number of specific complications put on me as a web developer; and there are a number of very specific web-sourced problems that I’m used to solving. It’s simply different because it specifically involves the epic WAN that is the Internet.

Lean Software Development translates very well when dealing specifically with web development since the same problems exist in both development life-cycles. What really varies is how we might implement a solution to those problems. Taking that into consideration, be aware that I’m writing this article as a manager of web developers not as a manager of software developers.

Waste in Web Development

The TPS identifies seven types of manufacturing waste:

1. Overproduction
2. Waiting
3. Transportation
4. Extra processes
5. Overstock
6. Motion
7. Defects

In relation to software development, however, it’s clear that not all these types of waste translate. Lean Software Development identifies the following types of waste which translate perfectly between software and web development:

1. Partially complete work
2. Extra processes
3. Extra features
4. Task switching
5. Waiting
6. Motion
7. Defects

Since some of those are not immediately clear as waste, let’s examine them in more detail…

Partially complete work

It’s fair to say that all web development is partially complete at some point in its lifecycle. The priority here is to minimise the amount of partially complete work that is within a project at any given moment.

Partially done web development can quickly become obsolete as it is replaced with a differing method or solution. The only way to be sure what problems exist in a piece of code are to have it fully integrated with the rest of the project. If you are working behind feature branches or feature toggles, or some other method for cloistering partially done work, you are suffering from your isolation as much as you are benefiting from it.

In fact, the only place where you will capitalise on your investment in a feature is when it is in production. Partially done development ties up resources in investments that have yet to yield results. You need to provide integration test coverage, end-to-end test coverage, integration code, and a whole host of other methods of ensuring the minimum of bugs exist when you finally release it from its isolation. If your feature never actually makes it to production, you have some significant investment to write off. Reducing the amount of partially complete work within your project is both a reduction of waste and risk.

Extra processes

We all love our processes and documentation, right? I’d be willing to bet that just about everyone who has worked in an agile organisation of some form has become disillusioned with the amount of meetings or breaks from focus they have to attend as agile “ceremonies”, and the added work dealing with documenting user stories, their requirements, and how they might be tested and accepted. In fact, two of the tennets of the agile manifesto are specifically related to process and documentation:

We have come to value:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation

At every opportunity you should be assessing whether a particular process or document is entirely necessary for your development. Extra processes and documentation consume resource, slow down response times, and can mask problems elsewhere. What’s more they break focus, which is something I’ll come back to when I explain “motion” as a type of waste.

Many development processes require paperwork of one form or another. Most of the companies I’ve worked for recently have replaced actual hard-copy with wiki documentation these days, but those documents are still there. Documents for release processes, documents for sign-off, documents for traceability, not to mention documentation for security or compliance.

The real test of these documents is whether someone is actually waiting for it as a deliverable, and whether it then adds value to their process. If not, dump it. Otherwise, find ways to simplify that process. Reduce the amount of input required by using templates or pre-designed input forms. Condense the amount of information required. In fact, there should always be an ongoing process of refinement and efficiency evaluation for all documents and processes. Any required process should be as efficient and effective as possible. Rapid yet easy to validate.

In terms of documenting code, do it as you go. Write clean, easily understandable, easy to maintain code and enforce style with static-code analysis and code reviews. Make sure code is commented effectively and you’ll cut down on cost. It’s important to remember that sometimes documentation is required for code when knowledge needs to be shared over a wide base, but if you’ve minimised what other documentation is required this will be the exception to the norm.

Extra features

Adding in extra features to a piece of development is over-engineering; don’t do it. There will be many times that you foresee the need for an extra feature before you actually have the requirement. There are other times that a developer might see the opportunity to add something extra that may save time for others later on. Whilst at first glance this may seem relatively harmless and of benefit in the long run, it is almost always a source of waste.

All code in the system should be tracked, reviewed, tested, and integrated every time it is touched. What’s more it then needs to be maintained for the life of the project. With that in mind, it is important to remember that every bit of code adds complexity and a new possible point of failure. It also introduces something new for all the developers to understand and—worse—misunderstand and misuse.

If there is no requirement for a piece of code now, don’t write it. Putting it in is waste. Resist your urge to over-engineer.

Task switching

The fastest way to complete two projects using the same people is to do it one at a time, sequentially. Trying to run two projects in parallel using the same resources introduces a large amount of waste. Likewise, sharing people between teams is also wasteful.

Much like extra process, task switching breaks focus. Every time a developer, QA, IA, or designer have to switch tasks, there is a fair overhead to pay. These sorts of tasks require enough focus to warrant a significant amount of time to get into “the zone” or “flow”. Thus, switching between tasks incurs waste through time.

Similarly, being a member of two teams introduces interruptions that might be entirely unrelated to the person’s current task. This means even more task switching, and probably at an even more granular level. For this reason alone it is unwise to assign people to multiple teams.

To illustrate how much task switching can affect delivery time, list 6 peoples names. Firstly, time how long it takes to write each name one after the other. Next, time how long it takes to write a letter of each in turn until you’ve written all 6 names completely. You’ll find the latter takes significantly longer than the former. This is only the simplest form of task switching and yet the waste incurred is quite clear.

Back in 2009, Paul Graham wrote a fantastic piece on focus entitled “Maker’s schedule, Manager’s schedule”. In it he examined the difference between time schedules for developers and managers. It’s important that all managers understand that their schedule is significantly different to the developers that report to them.

A developer works in larger units of time than a manager, with much longer between breaks of focus; usually something in the region of a half-day unit. They dive into work as soon as they arrive at the office and will break focus at lunch. Likewise, when they return for lunch they’ll dive back in and only come back to the surface when it’s time to leave the office. Usually they’ll arrange their tasks in chunks that align to this and therefore will complete tasks shortly before lunch or shortly before they’re ready to leave for the evening.

Managers work in shorter units; usually around 1 hour in length. This allows time for meetings and context switches in between. As a manager it’s your job to context switch effectively, although even the greatest manager in the world will struggle to do it for too many subjects. This is why it’s important to make sure you don’t bite off more than you can chew in terms of projects to run. If you have, it’s time to start delegating.

If you’re a manager and you want effective makers, you need to arrange your days to suit them, not you. Allow 30 minutes in the morning for email checking and an added 15 minutes for the daily stand-up meeting. Don’t bother them again until directly before or after the break for lunch or at the end of the day. Trust me; they’ll cope with this much better than continuous interruptions throughout the day.

Likewise, if you’re a developer, be strict about what you do at these key times during the day. Make sure you don’t kick off development before your stand-up or you’ll fail to cope with the interruption. If people can’t agree to arrive at the office at around the same time in the morning, then you’ll be better off as a team scheduling the stand-up around lunchtime.

All this might sound obvious but you’ll be surprised how many teams let this sort of misalignment between team members screw up their daily flow. It’s also worth noting that the larger the team, the harder it is to align this stuff. There’s definitely a sweet spot to be found in team size and I personally believe it’s around 6-8 members.

Waiting

Day to day development is thick with delays. It’s a standard part of our working lives for projects to be delayed at the start, awaiting requirements or documentation; for the actual development to be blocked awaiting testing or deliverables from IA and UX; or to delay delivery through an inefficient release process.

In web development, delays can be a serious form of waste. They prevent the completion of work and they prevent the capitalisation of investment in the development itself. The speed at which we can respond to a customer-driven requirement is directly related to the number of delays in our development life-cycle.

As soon as we can accept that delays are a significant form of waste, we can identify where those delays occur and optimise them out of our processes. One of the seven principles of Lean Development is “decide as late as possible”. Taking that into account, it is better to re-prioritise tasks based on the delays you are likely to experience than suffer significant blockages in your delivery. This might mean putting tasks off until you have more information or free resource, or it might mean rearranging tasks so that developers are working on minimising those delays before delivering their original priority.

The most common delays in web development are when work passes between individuals. There’s nothing worse than finding out you’re prevented from completing a task because the wireframe or design hasn’t been completed or the requirements haven’t been finalised due to poor prioritisation. It is important that, as a team, you don’t agree to work on more than you can complete and similarly, it’s also important that you don’t agree to work on stories or tasks that have outstanding dependencies or requirements. Sometimes taking the time to groom your backlog as a team can mean the difference between wasting time on waiting or a user story or task flowing through your process painlessly.

Further to that, you can often reduce the time waiting for a deliverable by eradicating the deliverable altogether. I’ve had great success in the past by removing the need for wireframes and design visuals from the IAs and designers and getting them interacting with the developers on a regular basis. If the designers and developers maintain a clear style-guide with clear links to the front-end code—and at best a library of corresponding modules of code—they can easily relate new features to this through “pair programming” for an hour or two. Remove the requirement for a deliverable and you’re only left waiting for the developer and designer to have some time together in their calendars. This should be significantly easier to achieve since both parties are no longer required to deliver anything but their expertise during the interaction.

Finally, it’s important to remember that implementation is everything. If you can’t implement quickly following a key decision then you can’t delay that decision either. Empowering your team to deliver quickly and efficiently is your primary concern at all times, whether you are a manager or a team member.

Motion

Agile software development favours colocation for teams. Likewise, the agile manifesto states that we should value customer collaboration; involving them in all discussions and decisions throughout the development process. The point of both these principles is the reduction of “motion” within the team.

It is important to make sure that developers, QAs, IAs, designers, product owners, project managers, and any other involved parties are immediately available to each other. Reducing the amount of time it takes for specific development-related questions to be answered is vital. Having everyone next to each other reduces that overhead.

If you can’t colocate people, the next best thing is to make sure they have easy communications as soon as they need them. Get everyone hanging out in a Jabber or IRC room, and make sure that’s as much a social experience as it is a work experience or they won’t come back. It should really imitate the local working environment so is likely to involve both work-related conversations and more water-cooler-type discussions and jokes. Expect that; nurture it; it’ll bring the team closer together.

It’s also important to remember that it’s good to put a name to a face. If some of your team are remote, make sure you get them over for social occasions once in a while. It’s good to break bread with your colleagues, and it’s easier to foster feelings of camaraderie when you’ve all been out to the pub together.

Similarly to social interactions, artefact interactions can also suffer from waste in motion. Reducing the amount of time it takes to pass a particular artefact of work from team member to team member can significantly improve the efficiency of the whole team. Good examples are the hand-off of design deliverables like style guides, wireframes, and visuals. Placing these in a known shared location using source control means all interested parties can have access to them immediately. Similarly, making sure tests are written and run in the same places regardless of whether they are written by developers or QAs means that there will be a clear vision of coverage for the whole team.

Motion might seem like a fairly inconsequential form of waste with a minimum of impact, but really it combines with task switching to be quite a serious problem. A competent team will work with this at a minimum from the outset, and most teams can address this issue quickly to improve their performance.

Defects

Defects are an obvious form of waste, but really it’s only the undetected defects that cause a project to suffer. That might sound obvious at first, but it’s important to remember that the amount of waste generate by a defect is directly proportional to both the bug’s severity and the amount of time it goes undetected. A critical show-stopping defect that is identified and addressed quickly generates significantly less waste than a small insipid bug that hangs around for months undetected.

Reduce the impact of defects by finding them quickly through thorough testing practices and QA. Make sure the whole team are involved in that; quality is too important to be the responsibility of anyone except the whole team. Practices such as test-driven development and continuous integration can massively improve the time it takes to detect issues. It is important to understand what you are trying to achieve by adopting these practices and implement what you can.

Summary

Understanding waste in your development process can help you improve the efficiency of your delivery. Being able to take the principles of lean development and integrate them into your agile teams are a clear way to improve your life as a manager. In my experience it’s almost impossible to implement agile within an organisation without truly understanding the whys and hows. Further to this, it’s everyone in the team that need to buy-in and understand, not just the managers or the organisation. This rings true for Lean Development as well.

This has really been a study of the types of waste discussed in the Poppendieck’s various books on Lean Software Development. If you’re interested, I recommend you buy at least one of the various volumes. They’re absolutely worth it.

This is the first of a couple of articles I’ve started writing around agile and lean development practices and hopefully it’s at least got you thinking about how you can improve your life as a developer or manager. In the future I want to look at the complications around writing user stories, around implementing good testing practices, and how systems-thinking should be a primary goal of all teams.

For a good while I used defined width floats for my grids. This means you have to calculate the width of each column and fix it (either with pixels or something more flexible like ems or percentages). That method is flawed due to rounding errors and general problems with flow when resizing. A better method is to use a mix of floats and negative margins for positioning. When I need to create a CSS column grid, I use this method.

However, when I was recently asked to share my method—which was outlined to me by a good friend and CSS guru—I had to remind myself how to do it. These are the perils of becoming a manager and developing less. In the interests of remembering next time (and also sharing the knowledge), I’ve decided to document the method in this article.

A word on frameworks

Just a quick word really. There are frameworks out there that will do your grids for you. I know they exist but I do not use them. Often they result in bloated mark-up and esoteric element identifiers that create unmaintainable code. Your mileage, of course, may vary.

I’m capable of creating a grid with the minimum possible code required. Hence, for production purposes, that’s what I do. In a prototyping situation, when I need something quickly and painlessly, I may choose to favour a framework; but it hasn’t happened yet.

If you choose to use a framework to set-up your grid, this article probably isn’t for you.

Source order is important

It’s good to make sure your main content is first in your source order. It’s helpful for accessibility, and it’s (a bit) helpful for SEO. This is something that isn’t always easy to enforce when using traditional grid techniques. The technique I use is actually built around this concept.

Building the mark-up

The simplest possible mark-up we would want for a three column layout would be something like this:

<div>
    Main content
</div>
<div>
    Secondary content
</div>
<div>
    Navigation
</div>

To make it a little easier to target, and to make it a bit more readable, we can add some element identifiers:

<div class="main container">
    Main content
</div>
<div class="related container">
    Secondary content
</div>
<div class="navigation container">
    Navigation
</div>

This gives us three containers for our columns. If we were to use the simplest possible method of floating each column left and fixing their widths, we’d have ourselves a column grid. However, that grid would not respond reliably cross-browser and could pick up wrapping errors. It also requires exact knowledge of the required width of ALL columns. Depending on how we’d like our grid to respond to change, this may be suboptimal.

Floats

A floated element is shifted to either the left or right of its containing box and surrounding content then flows around it. Floating multiple boxes stacks them up either left or right, which is useful behaviour when attempting to simulate columns. However, if there is not enough horizontal room for the float, it is shifted downward until either it fits or there are no more floats present.

Floats are taken out of the normal flow of the document. This means the positioning model is changed somewhat and we can use that to our advantage.

The trick

Applying a negative margin to a floated element allows us to control how it makes use of empty space around it. We can make use of this if we add an extra wrapper to our main content:

<div class="container">
    <div class="main">
        Main content
    </div>
</div>
<div class="related container">
    Secondary content
</div>
<div class="navigation container">
    Navigation
</div>

To make two satellite floats react correctly we need to make sure that our newly added container is also taken out of the normal flow of the document with a float. However, we don’t want surrounding content to flow around it so we should fix its width to 100%.

.container {
    float: left;
    width: 100%;
}

Next we want to put some space around our main content using standard margins:

.main {
    margin-left: 150px;
    margin-right: 200px;
}

Now let’s fix the width of those satellite columns:

.navigation {
    width: 150px;
}
.related {
    width: 200px;
}

This will result in our three containers floating alongside each other. However, because the first container is set to a width of 100%, the two satellites drop down beneath it.

We want the satellites to take up that nice clear space we’ve created with the normal margins on our main content within the first container. To do that, we shift them left across the first container using negative margins:

.navigation {
    width: 150px;
    margin-left: -100%;
}
.related {
    width: 200px;
    margin-left: -200px;
}

Here we’ve shifted the navigation container 100% across the width of the container of our floats (in this case, the body of our page). This puts it flush to the left, filling the space to the left of our first floated container.

We then shift the related container only 200px to the left, matching its width and therefore leaving it flush to the right of the first floated container.

This leaves us with a perfect three-column grid. What’s more, since we’re only fixed the width of our satellites and have left the main content in normal flow, it will respond to the width of its container, as will the position of the satellite floats.

Demo and source

I’ve taken the above and scrubbed it up a bit, to create a nice flexible responsive CSS grid demo page. Further to that, I’ve also stuck the code up on GitHub in a “flexible CSS grid” repo. Feel free to do with it as you wish.

The vision of the project is to create a tool to transparently manage all the complex parts of modern development within a virtual environment without affecting the everyday workflow of the developer too much.

Vagrant automates creation and provisioning of virtual machines within VirtualBox via the command line, whilst also allowing for easy distribution and reuse across multiple projects. In this article, I’m going to look at how it can make life considerably easier when dealing with development environments.

Previously

Previously I’ve written a couple of articles on virtualised development environments; one on VMWare and one on VirtualBox. Since then I’ve upgraded my model to use Vagrant, which allows me to control my environment per project, and also fixes me up with painless access to Puppet provisioning.

This has completely altered my workflow for development. To help you to understand why, I’m going to walk through how Vagrant works and then look at setting up a Vagrant-powered development environment.

How it works

The base box

The true power of Vagrant lies within its design. The basis of a Vagrant VM is its “base box” which is simply a VirtualBox VM that represents a base install of your system. This base install has the following basic requirements:

• VirtualBox Guest Additions for shared folders, port forwarding, etc.
• SSH with key-based auto. support for the vagrant user
• Ruby and RubyGems to install Chef and Puppet
• Chef and Puppet for provisioning support

Thankfully the creation of base boxes has been done for you a thousand times over, and the subsequent base box files are available for download at various places across the interwebs. However, should you require something more specific, creating a base box is pretty undemanding and there are even programs like VeeWee which will automate this process for you.

For more information on building your own base box, take a look at the Vagrant base box documentation.

If you’d rather download one that someone else has created, take a look here:

http://vagrantbox.es

This is a really useful resource set-up by Gareth Rushgrove, the man behind DevOps weekly and also a passionate advocate of Vagrant. If you can’t find the box you need here, create it, host it somewhere, and let Gareth know so he can list it.

The VagrantFile

Once you’ve got a suitable base box available within Vagrant, you’re ready to start using it as the basis for your development environments. The plural is important here; Vagrant allows you to reuse the base box in generating VMs for multiple projects, each with their own set-up. To best understand that, I’ve drawn a very simple diagram:

Whilst you’ll probably use the same base box for multiple projects–my box of choice at the moment is a base install Ubuntu 11.10–you probably want a very different set-up per project. For instance, you might have a PHP project that requires a box running Apache and mod_php whilst a Python project better suits nginx proxying to gunicorn. This is entirely possible in Vagrant.

For each project you create, you’ll have a base source directory. It’s here that we’ll create a VagrantFile to control the environment for that project. This means you’ll end up with a VagrantFile per project:

/my_application/
    bin/
    etc/
    lib/
    var/
    VagrantFile
    …

/my_site/
    etc/
    public/
    templates/
    var/
    VagrantFile
    …

To create a VagrantFile you simply run the following command in your project directory:

vagrant init

This will create a nicely annotated VagrantFile. I recommend reading through it to understand the various settings available for you to configure.

The VagrantFile will allow you to change settings on the VM that vagrant will generate for you; for instance, you might want your VM to only have 512Mb of RAM, whereas another might need 2Gb of RAM. You can also specify what base box to use, what ports are forwarded to the VM, what IP address it is configured with, how the VM should handle networking (bridged, NAT, Host Only etc.), and many other system settings besides.

Provisioning

The VagrantFile also allows you to specify whether you want to provision your VM using Chef or Puppet. In OpsWorld™ [possibly the worst theme-park in the world] this means setting up the software on your VM. Personally, I prefer Puppet for this, simply because it uses an external DSL for configuration which I find easier to read than the internal Ruby DSL that Chef uses. (For an explanation of Domain Specific Languages, see Martin Fowler’s excellent blog post, book, and guide page.)

Puppet automates the provisioning, patching, and configuration of operating system and application components across multiple servers. Vagrant uses Puppet to do exactly that; to set-up, patch, and configure the base box to suit your project. To do this, you need to specify that configuration in Puppet manifests–which are basically scripts or programs for Puppet (hence the .pp filename).

The core of the Puppet language is the resource declaration, which represents the desired state of one resource. Manifests can also use conditional statements, group resources into collections, generate text with functions, reference code in other manifests, and do many other things, but it all ultimately comes down to making sure the right resources are being managed the right way.

For more on Puppet and its manifests, I recommend reading the Learning Puppet documentation. You can probably even skip straight to the manifests section. Also, the resource types cheat sheet is invaluable.

Another useful resource for Puppet is the Puppet Cookbook, which contains a collection of task oriented solutions in Puppet.

I’ve recently started keeping a git repository of the various manifests I use for Vagrant. This way, I can copy them and share them between Vagrant projects depending on what software or set-up I require. You can find that repo on GitHub:

http://github.com/nefarioustim/puppet-manifests/

Be aware, I tend to use Ubuntu server hence package management is handled via apt. This means these manifests will need a bit of editing should you want to use them on CentOS or some other linux distro.

The Vagrant VM

The Vagrant VM will be created according to your VagrantFile. This means there will be one per project. This granularity affords you a degree of independence with the VM, and allows you to treat it as a really powerful isolated root for your project. If you’ve ever used virtualenv when working with Python, imagine Vagrant to be virtualenv squared. Since you’re not sharing this environment with any other project, you can set it up, alter it, reconfigure it, and destroy it as much as you please; and since it’s entirely controlled by the VagrantFile and the provisioning scripts, those processes are easy and powerful.

Once set-up, your vagrant VM contains a default user of vagrant. That user has full sudo access and is generally the user you’ll use for everything within your development environment. When you think about it, there’s no real need for complex user set-up within this VM since you could be setting it up and tearing it down as much as you’re doing development on it. If you do need complex user set-ups, there’s nothing stopping you doing that with Puppet or Chef during the provisioning stage.

The Vagrant VM itself can be controlled at the command line through a number of commands:

vagrant up

The first time you run this command, it will copy across the base box specified in your VagrantFile, boot it up with the specified networking set-up, and then provision it according to your Puppet manifests or Chef recipes. Once the box has been set up, and as long as it isn’t destroy, this command will boot up the box and confirm the provisioning. You can deactivate provisioning on this command using the --no-provision switch.

vagrant ssh

Logs into your project VM as the vagrant user, using SSH keys. This circumvents the need for complicated SSH set-ups on your host machine and affords you all the power you need.

vagrant destroy

This will remove all trace of your project VM from within VirtualBox. This means, the next time you run vagrant up, Vagrant will need to recopy across the base box and reprovision it. Most of the time you’ll only really want to do this if you’ve b0rked your VM somehow, or you’ve made a significant change to your VagrantFile; like specifying a different base box for example.

vagrant suspend

Takes a snapshot of your VM and suspends it, allowing you to vagrant resume later. This saves you having to perform a length vagrant destroy && vagrant up cycle, but also stops the VM from taking up valuable memory and CPU cycles.

vagrant resume

Resumes a suspended VM from its snapshot state. The vagrant up command will also resume if it detects a suspended VM.

vagrant halt

Forces shutdown of a VM. This is basically a last resort shut down of your VM and the folks over at Vagrant recommend attempting a graceful shutdown via vagrant ssh first.

vagrant status

Shows the current status of the VM related to the VagrantFile in the current directory.

vagrant provision

Runs the provisioning scripts without reloading the entire Vagrant environment. If you’re just tweaking or adding some cookbooks, this command can save you a lot of time.

There are a few other Vagrant commands available to you but the above are pretty much the majority of what you’ll use on a daily basis. For the complete list, see the Vagrant commands documentation.

An example Vagrant environment

Now that I’ve run through the workings hopefully you’ve got a better understanding of the power of Vagrant and how it can improve your development workflow. To further highlight that, let’s set-up a basic example Vagrant environment on a Mac, via the command line.

Installation the best way

With the release of v1.0 of Vagrant comes a new set of installation packages, available from the Vagrant downloads section. This is by far the best way to install Vagrant and will sort you out with all the prerequisites automatically. These are now the recommended and supported method of installation for Vagrant.

When I began writing this article, these packages did not exist and, as such, I covered the slightly more complicated installation with all its prerequisites (for Mac). With these new packages, you should be able to follow along with the majority of this tutorial in Windows (albeit with slightly differing CLI commands).

Installation the old way

Apple Developer Tools

You’ll need to either install Xcode or the newly created Command Line Tools (significantly smaller than the full Xcode install and designed for stuff like this), which you can download from Apple (you’ll need a free Apple ID to log in):

https://developer.apple.com/downloads/

Homebrew

Firstly, I’d recommend you install some package management for which I use Homebrew. Whilst this is not required—you can always install from source—it is lightyears ahead of MacPorts (the alternative package manager for Mac) and is actively supported on GitHub. If you don’t have it already, do this:

$ /usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"

This is the recommended install for Homebrew. If you’re unsure, read the script first. It installs Homebrew to /usr/local so that you don’t need sudo when you brew install. It is a careful script, it can be run even if you have stuff installed to /usr/local already. It tells you exactly what it will do before it does it too. And you have to confirm everything it will do before it starts.

You can then use Homebrew to update your Ruby and RubyGems installs

Ruby and RubyGems

Vagrant is powered by Ruby and is installed as a RubyGem. This means you require both Ruby and RubyGems on your development VM host machine. Both are preinstalled on Mac OS, so we only need to update them:

$ sudo gem update --system

Install the gem

Once you’ve made sure you’ve installed all the requirements for Vagrant, the actual installation is really simple. Just install the Vagrant RubyGem:

$ gem install vagrant

Installing this gem will also install VirtualBox (if you haven’t already).

Bob’s your uncle, Mary’s your aunt, and Dave’s your vicar’s love-slave. You are now ready to start building Vagrant environments, but first you’ll need at least one base box.

Base box

You now have the option of downloading a base box, or you can build your own in VirtualBox using your operating system of choice. Since this is supposed to be a development environment, I’d recommend choosing an OS that is as close as possible to the one you’re going to use in production. For me that tends to be Debian or Ubuntu (which is really just another flavour of Debian), so I’m working with an Ubuntu base box.

Building your own base box

Creating a base box is straightforward as long as you’re confident building and provisioning servers. If you’re a bit unsure but you’re not scared to give it a go, I heartily recommend trying to build your own because it’s certainly a learning experience.

To begin with, get yourself a disk image of your chosen OS. For me, that’s Ubuntu 11.10 Server AMD64. Once you’ve got your image, you’re ready to start setting up a VM in VirtualBox.

My template for new VMs is:

• 360Mb RAM
• 10GB VDI disk, dynamically sized
• 64 bit
  • In General > Basic choose Linux > Ubuntu (64 bit) as your OS
  • Set System > Motherboard > Enable IO APIC to true
  • Set System > Processor > Enable PAE/NX to true
• Disable audio
• Disable floppy drive

Once that’s set-up, make sure you’ve “inserted” your OS disk image to the CD/DVD drive on the VM, and fire it up. I generally opt for a standard install for my base box, but if your OS of choice offers a low footprint install intended for virtualisation, you may want to opt to use that instead. If you do, make sure you know the differences; it may use a different kernel and therefore not support some of the motherboard and processor settings we discussed earlier.

When the time comes, make sure you use a default user of:

• Fullname: vagrant
• Username: vagrant
• Password: vagrant

Post install

I’ve written a script to handle post install in a similar method to VeeWee. To use this you’ll need to create a .vbox_version file in your home dir:

$ echo "4.1.8" > ~/.vbox_version

You’ll want to replace the “4.1.8” with the relevant version of your VirtualBox installation because this will be used to work out which Guest Additions to install. Next let’s switch to root for ease:

$ sudo -i

You’ll also want to make sure cUrl is installed:

$ apt-get install -y curl

Once you’ve done that, we can download and run the script (raw gist URL shortened for convenience here):

$ curl -L http://git.io/cSGCgw | bash

If you read through the script you’ll see it does the following:

1. Creates a log of the date and time we built the box in /etc
2. Installs aptitude
3. Updates the package list
4. Upgrades the installed packages
5. Installs a bunch of packages that are useful for set-up
6. Installs SSH
7. Installs the NFS client
8. Sets-up the correct permissions in sudoers
9. Installs Ruby and RubyGems
10. Installs the Chef and Puppet gems and updates the JSON gem
11. Removes the the gemspec for an older version of JSON because it’s broken
12. Installs the insecure Vagrant public keys supplied by Mitchell Hashimoto
13. Makes sure SSH doesn’t use DNS look-ups
14. Installs the VirtualBox Guest Additions
15. Zeros out the free space in the image to make it easier to compress
16. Removes leftover DHCP and network rules

Once the script completes you should be ready to rock. Better shut down that VM:

$ sudo shutdown -P now

Package up your self-built base box

Assuming the VM you created in VirtualBox is called “ubuntu-11.10”, we package it up using the following command:

$ vagrant package --base ubuntu-11.10 --out ubuntu-11.10.box

The --base parameter refers to the VM in VirtualBox and the --out parameter is what we want our newly packaged .box file to be called. Once you have a .box file, you’re ready to start using it with Vagrant–and you’re also ready to share it with other people should you so wish.

Using a pre-built base box

In true Blue Peter fashion, here’s one I made earlier:

http://timhuegdon.com/vagrant-boxes/ubuntu-11.10.box

For other options, as previously mentioned, have a look at Vagrantbox.es.

Making Vagrant aware of your base box

Assuming you’re in the same directory as your .box file, you can add it to Vagrant using the following command:

$ vagrant box add ubuntu-11.10 ubuntu-11.10.box

The first parameter is the name Vagrant will use to refer to the box in Vagrantfiles, and the last is the path to your .box file.

Configuring your VagrantFile

Now that you’ve downloaded or created a base box, and have made Vagrant aware of it, our next step is to configure a VagrantFile for whatever project we’re working on. It’s here, amongst other things, that we’ll specify which base box we want to use.

Change directory to your project and run vagrant init:

$ cd ~/projects/myproject
$ vagrant init

Vagrant will have now created a VagrantFile at the root of that directory. Let’s open that in the editor of your choice–which for me is Sublime Text 2:

$ subl VagrantFile

If you’ve ever worked with Chef before, or indeed Ruby, you’ll recognise that the VagrantFile is Ruby. You’ll also notice that the folks at Vagrant have made newly created VagrantFiles wonderfully verbose, and if you read through the annotations, all the settings are explained.

For our Vagrant VM we’re going to want to use our base box, so update that line in the Vagrant file to:

config.vm.box = "ubuntu-11.10"

Since we’ve already installed this base box, we don’t need the VagrantFile to be aware of a hosting URL, so leave that line commented.

The next line defines whether you want to run your VM headless (i.e. without firing up the VirtualBox GUI). Generally the only time you’ll want to uncomment this line is if you’re debugging a problem with the VM. Most of the time you won’t want to waste system resources on the VirtualBox GUI. With that in mind, leave it commented.

Now we can choose networking options. For the purposes of this example I’m assuming we’re working on a local development machine and, as such, host-only networking will suit our purposes. If we were working on a separate dev box that may host multiple VMs, we’d probably want to use bridged networking to expose them to the network. For now though, just uncomment the following line:

config.vm.network :hostonly, "33.33.33.10"

This means the VM will be exposed to the host machine on the IP address 33.33.33.10, and will be able to access your network. However, machines other than your host machine will not be able to see the VM.

Since we’ve uncommented this line, leave the bridged networking line commented.

Now we have the opportunity to forward some ports from the host to the guest OS. This can be useful if you want to temporarily expose a host-only network VM to machines on your network, however, for the purposes of this tutorial, please leave this line commented too. Later on we’re going to install a web server and connect to this VM as if it were any other server on our network, so port forwarding is not required here.

Next we need to define some folders to share with the VM, and how to share them. It’s a well recorded fact that when using VirtualBox on a unix-based system it struggles as the number of files within any shared folder increases. For this reason, if you’re running Vagrant on a Mac, it’s better to use NFS to mount your directories—Windows does not suffer the same issue so NFS is not required.

To set up a directory to share on your Vagrant VM, simply declare it as follows:

config.vm.share_folder("v-root", "/vagrant", ".", :nfs => true)

This will create a directory /vagrant at the root of your VM. That folder will be shared with the directory on the host containing your Vagrantfile, implied by the . directory reference. For a Windows host, just set nfs: => false.

Now we can choose how the Vagrant VM is provisioned. As I said earlier, I personally prefer to use Puppet, so uncomment the following lines and update the manifests path. I find “puppet” is more obvious than “manifests”:

config.vm.provision :puppet do |puppet|
  puppet.manifests_path = "puppet"
  puppet.manifest_file  = "base.pp"
end

This will look for our base manifest, “base.pp”, in a directory called “puppet” at the same directory level as our VagrantFile. We’ll set that up later.

Finally, let’s add a last line to our VagrantFile to tell it to set-up a VM with 512Mb of memory:

config.vm.customize ["modifyvm", :id, "--memory", "512"]

You’ll notice this particular command isn’t annotated in the VagrantFile. For more information about customising the specific characteristics of the VM, take a look at the Vagrant docs.

Once you’ve completed these changes, your VagrantFile should look something like this (if you remove all the commented lines):

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant::Config.run do |config|
    config.vm.box = "ubuntu-11.10"

    config.vm.network :hostonly, "33.33.33.10"

    config.vm.share_folder("v-root", "/vagrant", ".", :nfs => true)

    config.vm.provision :puppet do |puppet|
        puppet.manifests_path = "puppet"
        puppet.manifest_file  = "base.pp"
    end

    config.vm.customize ["modifyvm", :id, "--memory", "512"]
end

Our Vagrantfile will now create exactly the VM we require when we run the vagrant up command. However, the VM itself only contains the software installed as part of the base box. Since this VM is serving our project, we probably want some more stuff on there like a web server or a particular version of a language. To set that up, we’ll need to use a provisioner which is where Puppet comes in. (If you want to use Chef, don’t let me stop you. I just won’t be covering Chef here.)

Provisioning with Puppet

Provisioning happens in Vagrant every time we run vagrant up or vagrant provision. This sets up the software on your VM according to a set of manifests (or “recipes” in Chef). Those manifests basically tell Puppet what it should script on the VM. This means you can update anything, including copying across files, updating files already on the system, installing and uninstalling packages, changing permissions, stopping and starting services, and a whole host of other software changes.

To do this, you’ll need to learn Puppet’s manifest language. This is a custom DSL that is quite easy to learn; although the Puppet documentation can be a little difficult to navigate. I’d recommend looking at:

• Learning Puppet
• Puppet Docs Reference Shelf
• Basic Features and Use
• Core Types Cheat Sheet PDF

To begin, let’s set up a directory in our project folder to contain our manifests, and another folder within that to contain any files we may want to copy across to our guest OS with Puppet. From the directory on your host that contains your Vagrantfile do:

$ mkdir -p ./puppet/files

Now in the newly created puppet folder, create a file entitled base.pp:

$ cd puppet && touch base.pp

Note that Puppet manifests have the .pp extension. You should be able to find syntax rules for all good editors to deal with this extension and file type.

Open up the newly created base.pp file in your editor of choice. Puppet manifests have a number of constructs, chief of which is the concept of a “resource”. Resources can be packages, files, executed commands, users, groups and services. These resources can then be organised within classes. So let’s start with a class for our dev box:

class devbox {
}

As an example, let’s set-up nginx as a web server. To do that, the first thing we’re going to need to do is install the package:

package { "nginx":
    ensure => latest
}

That was quite simple, right? Puppet already knows to use apt as the default package installer on Ubuntu, so this command checks the apt-cache to see if the latest version of the package is installed, and if it isn’t installs it.

Now we should make sure the nginx service is running:

service { "nginx":
    ensure => running,
    hasrestart => true
}

Equally simple. Here we’ve told Puppet to check the service nginx is running, and we’ve also informed it that the service has a restart function which Puppet can make use of.

I’ve stuck a bunch of hacked together Puppet manifests into a repository on GitHub. Feel free to poke about in them; I’ll be updating them as I do more stuff with Vagrant. For now, at least, take a look at the files directory because we can repurpose some of my set-up for this example. Save the following from my repo to your puppet/files directory which we created earlier:

• files/nginx.conf
• files/vhost.conf

Now we need to get these files from your host computer to the guest OS in the Vagrant VM. Once again, this is pretty simple:

file { "/etc/nginx/nginx.conf":
    owner  => root,
    group  => root,
    mode   => 644,
    source => "puppet:////vagrant/puppet/files/nginx.conf"
}

file { "/etc/nginx/sites-available/vagrantsite":
    owner  => root,
    group  => root,
    mode   => 644,
    source => "puppet:////vagrant/puppet/files/vhost.conf"
}

Notice the source exists within our /vagrant directory, which we’ve already set-up as a NFS shared directory. This means we’re effectively copying the file from there to the /etc/nginx/ directory on the guest. Also, if the destination directory does not already exist, Puppet will create it. Notice how I’m also setting the owner, group, and mode of the file during the copy.

So far we’re getting along fine, but wouldn’t it be nice if we could let Puppet know we’d like to handle these stages in some semblance of an order? We can do that in the resources using the require parameter:

file { "/etc/nginx/nginx.conf":
    owner  => root,
    group  => root,
    mode   => 644,
    source => "puppet:////vagrant/puppet/files/nginx.conf",
    require => Package["nginx"]
}

Note the reference to the resource we require is capitalised? This is a subtle difference between declaring and reference a resource.

What would also be nice is if we could tell the nginx service that, by changing the configuration files, we’d like it to restart. To do that we use the notify param:

file { "/etc/nginx/nginx.conf":
    owner  => root,
    group  => root,
    mode   => 644,
    source => "puppet:////vagrant/puppet/files/nginx.conf",
    require => Package["nginx"],
    notify => Service["nginx"]
}

We just have a couple more things to do now. Firstly, let’s create a symbolic link to sites-available from within sites-enabled so that our newly copied across vhost is picked up by the nginx configuration:

file { "/etc/nginx/sites-enabled/vagrantsite":
    ensure => symlink,
    target => "/etc/nginx/sites-available/vagrantsite",
    require => Package["nginx"],
    notify => Service["nginx"]
}

Notice I haven’t required the original file for this. That’s because it’s not a requirement for creating the symlink in the first place.

Secondly, let’s remove the default vhost because we don’t need it:

file { "/etc/nginx/sites-enabled/default":
    ensure => absent,
    require => Package["nginx"],
    notify => Service["nginx"]
}

With all that in place, and adding a full set of require and notify parameters, our final class should look like this:

class devbox {
    package { "nginx":
        ensure => latest,
    }

    file { "/etc/nginx/nginx.conf":
        owner  => root,
        group  => root,
        mode   => 644,
        source => "puppet:////vagrant/puppet/files/nginx.conf",
        require => Package["nginx"],
        notify => Service["nginx"]
    }

    file { "/etc/nginx/sites-available/vagrantsite":
        owner  => root,
        group  => root,
        mode   => 644,
        source => "puppet:////vagrant/puppet/files/vhost.conf",
        require => Package["nginx"],
        notify => Service["nginx"]
    }

    file { "/etc/nginx/sites-enabled/vagrantsite":
        ensure => symlink,
        target => "/etc/nginx/sites-available/vagrantsite",
        require => Package["nginx"],
        notify => Service["nginx"]
    }

    file { "/etc/nginx/sites-enabled/default":
        ensure => absent,
        require => Package["nginx"],
        notify => Service["nginx"]
    }

    service { "nginx":
        ensure => running,
        hasrestart => true,
        require => Package["nginx"]
    }
}

The next time we fire up our Vagrant VM, nginx will be installed, set-up, and running, and attempting to serve a website from the /vagrant/static directory on your guest OS. This will be the static directory within the same directory as your Vagrantfile on the host.

On the host, let’s create that directory and stick a very basic file in it:

$ mkdir static && cd $_
$ echo 'Hello world from my Vagrant VM!'' > index.html

Not the most well-formed example, but it’ll do. Now let’s fire up the Vagrant VM:

$ vagrant up

This should copy across the base box we installed, set-up all the things we specified in the Vagrantfile, and begin provisioning from our manifest. You should see messaged that Puppet has successfully installed nginx and the relevant files, and that the service is running. Now point your browser to:

33.33.33.10

Ta da! You have successfully set-up a vagrant development environment with specific software requirements for a specific project. Congratulations! Now go away and hack it.

Summary

So this has been an epic 5000 word look at why Vagrant is now my development environment tool of choice. I hope you can see that it makes working with multiple projects—each with their own requirements for an environment—as simple as possible. With a good understanding of the features of Vagrant, and also of the provisioning tool of your choice, you really do have the power to fire up and destroy environments on a whim.

Apologies for the sheer size of this article but I wanted to make sure I covered as much as possible to help newbies get up and running quickly. It doesn’t take much effort, and it really empowers developers to be able to spin up new environments quickly for hacking around in.

It’s more of a realignment rather than a redesign. I’ve trimmed back the cruft, and improved what I believe is important—and I’ve also made some more drastic changes which can be reversed should I want to.

So, without further ado, let me just take some time to explain my reasons…

Technology

The site is now statically generated by Jekyll in Ruby instead of dynamically on request via Wordpress in PHP. This has allowed me to rebuild my server and ditch a bunch of technologies I no longer work with. Key amongst those are MySQL and PHP, but I also no longer need Apache. It’s also made the site faster and less in need of a caching solution.

Statically generating the site has also meant I’ve ditched some more dynamic functionality like the contact form and—more importantly—comments. Considering the ratio of signal to noise in the comments posted to my site, and the sheer weight of spam, I’m reasonably confident I won’t miss them. What’s more, with the constant discussion on Twitter and the fact that I can easily bring comments back later with Disqus, I have the option for resurrection should I desire it.

Jekyll has some pretty cool extensions and plug-ins, and if you take a look at some of my more code filled posts you’ll notice the code samples are now styled up with the help of Pygments, a python library that marks-up the code for styling in CSS. It’s the same system used by GitHub.

I’ve used an HTML5 doctype so I can add modern features as they become more widely supported cross-browser. I’ve also used a fair amount of CSS3 so I hope you’re browsing using an up to date browser rather than something as flawed as Internet Explorer.

The styling now requires nothing but CSS and web fonts. I have significantly reduced the number of images required to make the site look pretty. This means the number of HTTP requests made by the page is significantly smaller and overall the site performs better in YSlow and PageSpeed. I have to say, this wasn’t a primary concern but it’s nice to get it without trying too hard.

My new site is now much easier to deploy and update and I’ve made use of all the good stuff I’ve learnt about server administration over the past few years. You could say it’s all much more DevOps-like in approach and makes use of stuff like Puppet, Rake, and SASS.

Design

I wanted to make sure I was designing for my content and not trying to fill space with extra information that nobody really wanted. Hence, the right hand menu has been replaced by much wider blog posts. Hopefully that’ll do away with a few of the nasty horizontal scrollbars that crept into some of my code examples whilst also giving the main content of the site room to breathe.

The typography of the site should now be significantly more predictable cross-browser. I’ve used @font-face before but it’s nice to use it for one’s own devices. Hopefully this new iteration is much easier to read than the last, and makes better use of the space available.

I’ve improved everything for SEO; flattening the site architecture, making sure to include descriptions, moving navigation below the main content, and defining canonical URIs. I’ve also got plans for the category pages but need some more time to plan out the work and felt it wasn’t really a blocker for launch.

The future

So apart from the SEO stuff I’d still like to work in, I’ve still got some work to do on the responsiveness of the site. Currently it’s not looking great for screens less than 480px in width and I need to rethink some positional stuff to make it work.

For the moment I’m reasonably happy with the site, and with some more work on it over the coming months I’m sure I can turn it into something I’m much more comfortable with. The real target was to make the core content more readable and I think I’ve achieved that. I only hope you, as a reader, agree.

Personally I think the new designs are excellent and really appreciate the fact that they address my need to access my Google Apps from a variety of devices. I also like the fact that they allow me my own preference for the visual density of content within the page.

In fact this is such a killer feature that I reverse engineered it to better understand it. I’ll be looking to use this knowledge in some of my future work. Whilst I wouldn’t really refer to this deconstruction as such, plagiarism is the sincerest form of flattery. Unfortunately, due to extensive work commitments, I’ve only just gotten around to writing up my work…

Responsive

It’s my opinion that this technique relates quite closely to the responsive design ethos. The visual content density should respond to both the user’s available screen real-estate, their chosen font-size, and their own preference—which should always override any other external actors.

The majority of responsive design discussions and tutorials seem to talk extensively around how a design responds only to the client width. True responsiveness, however, equates to so much more than this.

Visual density

In this article I am referring to visual content density. This refers to the actual physical space around the content and is not to be confused with keyword density, which is concerned with the proliferation of words and phrases within said content. Balanced visual content density is an important factor towards improved typography, improved legibility, and an improved interface overall. Most designers will wax lyrical on the benefits of plenty of whitespace and how giving content room to breathe can improve the usability of the interface.

The visual density of content within your web-based documents can easily be controlled with an extensive set of CSS properties. However, there are only a few that give entirely predictable results across all browsers:

• font-size
• line-height
• padding
• margin
• width
• height

Importantly, these properties control both content size and the amount of whitespace that surrounds it. Together these two factors can sway the balance between spacious legibility and the efficient use of limited screen real-estate.

A demonstration

Here’s a quick and dirty prototype that I’ve knocked up to demonstrate visual density variance at work. You can vary the density via the buttons at the top, or by resizing your browser window (which will override the former):

http://timhuegdon.com/test-cases/density/

The source code for this example is available here:

http://github.com/nefarioustim/density-demo/

How it works

In my demo I am applying a class to the root element of the document that controls the level of content density selected by the user. I also repeat those styles within media queries for different client widths to allow denser content on smaller screens—try resizing the browser window to watch it respond.

To make this more maintainable in the future I’d probably invoke those styles through some form of SASS or LESS mix-in to cut down on the obvious duplication of styles. However, be aware that the use of CSS preprocessors like SASS and LESS can complicate things when attempting to get browsers that do not support media queries to detect the styles using a JavaScript polyfill like Respond.js.

Summary

So there you have it; a simple technique that has a pretty significant impact to the readability of your content. Hey, and the demo is kinda fun to play with too.

By a total coincidence a discussion in a recent front-end code review and a discussion over the past couple of days on Twitter both relate to the use of HTML element identifiers. The former, a discussion on semantic value and remembering that HTML should not prognosticate styling; the latter, a discussion on the validity of using IDs to target CSS. With both of these considerations I’m going to study the fundamentals of writing good element identifiers within this article.

What’s an element identifier?

HTML has two element identifiers; the id attribute and the class attribute. These attributes can be applied to most of the elements within a modern HTML document.

What do they do?

The id attribute assigns a name to an element. This name must be unique within the document and each element can have only one.

The class attribute assigns a class or set of classes to an element. Any number of elements may be assigned the same class name or names and each element can have many.

In relationship terms this means IDs and elements are a one-to-one relationship, and classes and elements are a many-to-many relationship. IDs are very much an absolute index system and classes are more of a tag-style classification aide.

Make note of those words: "name" and "classification”. I’ll come back to that in a bit.

Mislaid mastery

Put in those simple terms it’s all pretty straightforward, and I think that most front-end developers have a good general grasp of their usage. However, it’s the detail that introduces the issues, and if you take time to step back it’s easy to see why:

Element identifiers are the perfect answer for cross-linking other front-end technologies to your HTML. They are the hooks on which to hang your styling and behaviours, and because of this they take on that personality in your mind. This is where so many developers go wrong.

When front-end developers are first introduced to the id and class attributes, it’s often so the identifiers can be targeted in CSS selectors as a means of styling the developers’ documents. They will also inevitably start using them to hook progressive enhancement into their documents via DOM scripting. These are the two most common uses of element identifiers:

<ul id="site-navigation" class="navigation">
    …
</ul>

#site-navigation {
    border-bottom: 2px solid red;
}
.navigation a:focus,
.navigation a:hover,
.navigation a:active {
    outline: 2px solid yellow;
}

var site-nav = document.getElementById("site-navigation");
site-nav.invokeOarsumNavWidget();

// Use some modern browser native DOM + JS win
var navCollection = document.getElementsByClassName("navigation");
navCollection.forEach(function(element, index, array) {
    // Do something to every element
    // with a class of "navigation"
});

Also, if you’ve ever wanted to make use of the URI fragment identifier in a non-destructive way, you may have used the ID element identifier instead of the old-skool named anchor () approach:

<a href="#navigation">Skip to navigation</a>

In each of the above examples, the element identifiers were added specifically to be used by some other language; for styling it is CSS, for dynamic behaviour it is JavaScript, and for linking it is the URI RFC 3305. However, that association also occurs psychologically in our minds as developers and we often have trouble seeing past it.

Names and classifications

Back to those "name" and "classification” descriptions:

It seems we have carelessly forgotten the fact that element identifiers are a feature of HTML and not the other languages that use them as targets. We should be using them as they were originally intended; to name and classify the content within our mark-up.

We already use HTML elements to describe the content which they contain. This is the very heart of semantic HTML and should result in content like heading text being wrapped in h1-h6 tags; lists in ul, ol, or dl tags; and tabular data in table elements. Since IDs and classes are attributes of these elements, we should use them to improve the semantic value of their hosts and provide further descriptiveness or context.

If we return to our previous list example, the following mark-up is reasonably semantic:

<ul>
    <li><a href="/index.html">Home</a></li>
    <li><a href="/about/">About</a></li>
    <li><a href="/blog/">Blog</a></li>
    <li><a href="/blog/archive/">Archive</a></li>
    <li><a href="/contact/">Contact</a></li>
</ul>

However, we can improve it further by giving it some context with an ID:

<ul id="site-navigation">
    <li><a href="/index.html">Home</a></li>
    <li><a href="/about/">About</a></li>
    <li><a href="/blog/">Blog</a></li>
    <li><a href="/blog/archive/">Archive</a></li>
    <li><a href="/contact/">Contact</a></li>
</ul>

Now it’s more obvious what this list of links actually is without seeing it styled up and rendered in the browser.

We could provide even more information with a single class:

<ul id="site-navigation">
    <li><a href="/index.html" class="current-page">Home</a></li>
    <li><a href="/about/">About</a></li>
    <li><a href="/blog/">Blog</a></li>
    <li><a href="/blog/archive/">Archive</a></li>
    <li><a href="/contact/">Contact</a></li>
</ul>

I’m sure most web developers have done something similar in the past, but it’s a very good example of applying context with a class the right way. The HTML now gives us a very good description of the content it contains. In fact, you now only need glance at this markup to understand its function.

Let’s take it a little further. Imagine we have several types of navigation in the page; we can better describe these elements by classifying them as navigation:

<ul id="site-navigation" class="navigation">
    <li><a href="/index.html" class="current-page">Home</a></li>
    <li><a href="/about/">About</a></li>
    <li><a href="/blog/">Blog</a></li>
    <li><a href="/blog/archive/">Archive</a></li>
    <li><a href="/contact/">Contact</a></li>
</ul>

<p id="breadcrumb" class="navigation">
    <a href="/index.html">Home</a>
    &gt; <a href="/blog/">Blog</a>
    &gt; My Blog Post
</p>

<div id="page-footer">
    <p>© 2011 Oarsum Corp, all rights reserved.</p>
    <ul class="navigation">
        <li><a href="terms.html">Terms and conditions</a></li>
        <li><a href="privacy.html">Privacy policy</a></li>
    </ul>
</div>

Simple, right? We’re using the element identifiers to improve the semantics of the markup and gaining improved readability as a side-effect. What’s more the added context can be targeted by whatever technology you so desire.

Anti-patterns

I’m pretty confident that most web developers will end up with markup similar to this without necessarily understanding my point. To that end, let’s take a look at some common anti-patterns (or misuses) that may go some way towards forwarding that understanding:.

The "clearfix" class

Way back in 2004 [ya rly!], Tony Aslett and Position is Everything published a clever method for containing floats that involved using the pseudo :after selector to insert some content and use that as an anchor for clear: both. It was a clever technique and was quickly adopted by the community, often incorrectly bound to a class entitled "clearfix":

.clearfix:after {
    content: ".";
    display: block;
    height: 0;
    clear: both;
    visibility: hidden;
}

/* IE targeted property */
.ie .clearfix {
    zoom: 1;
}

Quite apart from the somewhat erroneous name (it should really be "contain-float") this is absolutely the wrong place to be addressing this sort of styling issue. A styling-specific meta-class such as this is no different to writing a class entitled "red-text" or "big-blue-heading". In my opinion, the small amount of repetition it saves you in the CSS is not worth the non-semantic clutter in your HTML.

There are several ways to contain floats in CSS and the method you should use absolutely depends on the context. What’s more, some of those methods are a single CSS property declaration. For this reason, declaring a class of "clearfix" and implementing a single float clearing solution within it is actually potentially detrimental to the overall efficiency of your CSS.

It’s worth noting also that providing a meta-class tool like this results in a disassociation between the problem and the "fix". You’re likely to experience maintenance issues when you change any float behaviours and the containing mechanism is abstracted from the target’s styles. In addition to this you’re likely to find developers over using the solution to address problems they’ve introduced themselves.

The "hide-offscreen" class

When developers are aware of accessibility issues, and are attempting to provide textual description for otherwise potentially inaccessible content, you will often find a class that looks something like this:

.hide-offscreen {
    text-indent: -999em;
    font-size: 0;
}

/* or */

.hide-offscreen {
    position: absolute;
    left: -999em;
    font-size: 0;
}

This moves text off the screen, thus making it invisible for those not using a screenreader. This is preferable to display: none (which you might use to hide content from all users) because the content is only hidden within the browser viewport, but not removed from the complete rendering of the DOM.

However, what we’ve done here is define our CSS first and then apply that classification to our content. We’re supposed to be doing it the other way around. This class is unhelpful in describing the content that it wraps because it describes the behaviour the CSS should apply. This results in it providing little explanation of the developer’s intent, resulting in HTML like:

<ul class="hide-offscreen">
    <li><a href="#navigation">Skip to navigation</a></li>
</ul>

Not a good class at all. This is an improvement:

<ul class="accessibility navigation">
    <li><a href="#navigation">Skip to navigation</a></li>
</ul>

I've added the accessibility class here to signal intent: it's navigation for improving accessibility. I could have been more specific here, but overall I predict a high likelihood I'll want to signal a similar intent on other features within my document.

However, the double class will cause problems if you need to target both classes at once (e.g. .accessibility.navigation {}) in IE6. That browser will only target the final class in the selector. To fix this, you could combine the classes to create a new one:

<ul class="accessibility-navigation accessibility navigation">
    <li><a href="#navigation">Skip to navigation</a></li>
</ul>

Or a less ugly solution would be to include an ID on the navigation:

<ul id="header-skip-links" class="accessibility navigation">
    <li><a href="#navigation">Skip to navigation</a></li>
</ul>

Or to use a wrapping element and target with the descendant selector:

<div class="accessibility">
    <ul class="navigation">
        <li><a href="#navigation">Skip to navigation</a></li>
    </ul>
</div>

Here we’ve named the content and classified it well. It would be difficult to argue that we haven’t provided any targets for styling or behaviours now too. Unfortunately we’ve had to jump through hoops for IE6 and have broken our rule of changing the HTML for the CSS, but I just wanted to illustrate that there are more options available given a little thought.

Avoiding repetition

It’s entirely fair to argue that both the anti-patterns above avoid repetition within the CSS itself. However, this shouldn’t necessarily be the only reason to adopt one of these types of solution. There are other options available to us to abstract away this kind of anti-DRY maintenance nightmare:

• LESS Mix-ins
• SASS Mix-ins
• Place the CSS higher up the cascade

Indeed most CSS preprocessors will allow you similar functionality as it is clearly a "missing feature" of CSS as a whole. The alternative, of course, is to make your "fix" a part of the relevant cascade, so that all selectors beneath inherit that behaviour.

A nice example

A really nice example of classification is this simple design pattern for pagination:

<p id="paginglabel" class="audible">Pagination</p>
<ul role="navigation" aria-labelledby="paginglabel">
    <li><a href="#"><span class="audible">%TYPE% Page</span>1</a></li>
    <li><a href="#" rel="prev"><span class="prev">Previous<span class="audible">: %TYPE% Page</span></span>2</a></li>
    <li><p><span class="audible">You're currently reading %TYPE% page </span>3</p></li>
    <li><a href="#" rel="next"><span class="next">Next<span class="audible">: %TYPE% Page</span></span>4</a></li>
    <li><a href="#"><span class="audible">%TYPE% Page </span>5</a></li>
</ul>

Here we see the "audible" class describing content that should only be exposed to non-visual devices, such as screen readers and content scrapers (like search engine spiders). This is a good classification of a subset of the content.

It’s also worth noting the good use of WAI-ARIA roles and the rel attributes on the next and previous links. These are two other ways you can improve the semantics of your HTML and abstract away some of the classification to a more suitable annotation system.

Clean Code

Markup languages can often add to the obfuscation of code simply by their structure. If we use the element identifiers as they are intended we should end up reducing the overhead to grok relatively clean code.

Clean code is, amongst other things:

• Readable
• Understandable
• Predictable
• Maintainable

Use of element identifiers as semantic descriptors improves each one of these points. This is especially obvious when dealing with somebody else’s code, and in a team environment I personally feel it is imperative to include this in coding standards and best practices.

Good use of names and classification is just as effective as good commenting. When the code is effectively self documenting through its element identifier annotation, the need for commenting is significantly reduced.

On CSS best practice proposals

Now that I’ve covered the "hows" and "whys", I’d like to highlight why I think certain recommendations for CSS best practice directly contravene the correct use of element identifiers:

There has been some discussion within the front-end community regarding the use of IDs when targeting CSS. The ID attribute has particularly strong specificity in CSS, due to its uniqueness. When used incorrectly, it can cause maintenance problems and specificity bugs. However, the idea that targeting the ID property with CSS should be considered harmful—as seen in CSSLint, for example—is extreme to say the least.

It is reasonably simple to clobber other rules on an element by incorporating an ID in the selector, but this should really be an expected outcome. The use of an ID in a selector should denote the intent that the rule targets only a single occurrence of something within the page. If that is not what the developer intended, then they’ve wildly misunderstood the need for that selector. Certainly it makes sense that targeting an ID should be considered harmful when the developer actually meant to target something less specific, but then that’s just using CSS properly, isn’t it?

The argument that it is less valid to target IDs when working in a truly modular environment is even worse. A truly modular front-end environment will always require instance targeting alongside classification targeting. Sure, it’s likely to be the minority of situations, but there will always be cases in your system where it makes much more sense to target an ID than a class, and since they’re already in place on your HTML by this point [they are, right? We’ve already talked about this. If they’re not, go back and read it again.] you don’t have to worry about your CSS enforcing your HTML.

The developers that make these sorts of recommendations must be aware that they, as leaders in their field, have a responsibility not to mis-sell their solutions. Problems born of misuse do not emphasise an inherent flaw with the technology but perhaps an inherent flaw with the expertise of the user. As an exercise in correlation, it’s worth noting that Chefs do not veto the use of sharp knives because someone once cut their fingertip off with one.

Summary

HTML, CSS, and JavaScript are separate technologies. They are quite tightly related in the world of client-side development, but they should always remain decoupled. The element identifier is an artefact of HTML and should therefore not attempt to predict the hooks required for other technologies; it should simply be a property of effective semantic markup.

Further to this, avoiding the use of an element identifier, simply because it is so often misused by otherwise able front-end developers, is—at the risk of being blunt—ridiculous. Be proud of your HTML, spend as much time perfecting it as you would on any other language, and, most of all, don’t pollute it with the litter produced by a solution, in a related language, that lacks finesse.

Final note

I realise, however, that—despite my banging on about it—there are some developers you just can’t reach. As Jared Spool so eloquently puts it in his blog post, "Beans and noses":

No matter how much you try, you can’t stop people from sticking beans up their nose.

Of all the standards that a front-end team might want to implement, those that concern the identification and graceful degradation of cross-browser feature sets can be the hardest to define.

With that in mind, I’ve been poking around the front-end community looking for possible solutions. By far the most common approach—and one that gains an astounding level of attention in the community—is to implement Modernizr, a JavaScript feature sniffer created by Faruk Ateş, Paul Irish and Alex Sexton.

Unfortunately, despite my respect for the developers involved, I just can’t advocate Modernizr as a solution. Let me explain why; but first, let’s revisit some concepts that are going to be quite relevant…

The web standards trifle

The separation of structure, presentation, and dynamic behaviour is imperative to web standards. To that end, Andy “Malarkey” Clarke wrote a fantastic post detailing a method he liked to use to explain this fact to clients. He entitled it “the web standards trifle”.

In Andy’s metaphor, the separation of concerns can be imagined as follows:

1. Sponge

   This layer is your content, marked-up with well structured and valid semantic HTML. This HTML provides more information about each piece of content and allows any device reading the code to output as appropriate.

2. Fruity jelly

   This is your presentational layer. This separates all your presentational code into CSS, potentially allowing a user to override it with their own, or for you to serve something different based on context, e.g. a print or mobile stylesheet.

3. Custard

   This is the layer that accommodates any on-page behaviours. In the world of web standards we use this layer to progressively enhance our HTML and CSS layers with more interactive magic. I’ll come back to this later.

The true virtue of this divide is that the presentational (jelly) and behavioural (custard) layers can potentially be removed or overridden without affecting each other, or the content and markup beneath. This is hugely important if we want our websites to maintain client agnosticism and improve accessibility.

Progressive enhancement

Following the publication of this article there have been several suggestions that I explain the difference between graceful degradation and progressive enhancement. Suffice to say, this is something I have covered previously in another article.

Browser vs. Feature Sniffing

Browser “sniffing” is the act of attempting to discern a user’s browser by some means. This “awareness” can then be used in conditional code to control output or interaction for a specific browser. This technique was used a great deal in days of yore, when browsers had very different rendering engines and didn’t all conform to web standards. However, it was only ever really required in JavaScript.

The old school way:

if (document.all) {
    // IE4
    height = document.body.offsetHeight;
} else if (document.layers) {
    // NN4
    height = window.innerHeight;
} else {
    // other
    height = 0;
}

Not a good solution

There are problems with the method used above; for one thing, it is not the most direct method of identifying a browser. However, putting that aside for the moment, there are issues even with the theory of browser sniffing:

It’s quite obvious that browser sniffing does not scale. As new browsers are released, the sniffing code will need updating.

To make matters worse, many manufacturers developed their browsers to identify themselves incorrectly to get around legacy browser sniffing code. This ultimately meant that sometimes browsers were misidentified.

Ultimately browser sniffing is not a good solution. If you adopt it, you will usually end up maintaining separate streams of development whenever you need to add or update features. There are ways to mitigate that pain, but what if there was a better solution?

Feature sniffing

A more scalable approach is to use object detection in JavaScript to discover if a feature is available before you use it. An example of this might be:

if (document.querySelector) {
    element = document.querySelector(selectors);
}

The obvious advantage here is that it is browser agnostic; there are no assumptions as to which browser is involved at all. Using this technique, we overcome the scalability and maintenance headaches incurred through browser sniffing, and we can develop for graceful degradation from the outset.

CSS is a DSL that lacks features

Now, that’s all very well and good when we’re talking about JavaScript, but what happens when we need to affect the presentation layer?

CSS is a very simple domain specific language (DSL) that functions as a series of rules that override each other in a variety of ways (the cascade). There is no mechanism to detect a specific feature or a user-agent; only media types and potentially the odd CSS hack where a user-agent has imperfectly implemented an interpreter. There are also no conditionals; you just have to use the cascade to provide alternatives based on specificity or even just source order.

With these limitations, it is impossible to implement either browser or feature sniffing in CSS. This means we need to find an alternative method to affect the cascade based on the user’s feature set.

The Modernizr method

Step in Modernizr. In the words of the documentation:

Modernizr aims to bring an end to the UA sniffing practice. Using feature detection is a much more reliable mechanic to establish what you can and cannot do in the current browser, and Modernizr makes it convenient for you in a variety of ways:

1. It tests for over 20 next-generation features, all in a matter of milliseconds.

2. It creates a JavaScript object (named Modernizr) that contains the results of these tests as boolean properties.

3. It adds classes to the html element that explain precisely what features are and are not natively supported.

Following along so far? For the most part, that all seems fairly good. What’s more, Modernizr is definitely being pushed as the solution of choice amongst the front-end community:

• Mark Pilgrim’s Dive into HTML5 “Detection” chapter
• Faruk’s “Taking Advantage of HTML5 and CSS3 with Modernizr” on A List Apart
• Web Designer Notebook: How to use Modernizr
• Image Gallery tutorial on Sitepoint

So with all that in mind, let’s go back to why I will not be recommending Modernizr as a best practice:

Sniffing CSS features with JavaScript

The real issue is that Modernizr uses JavaScript to do the feature sniffing. This method is absolutely fine if we were only looking to test for the features in JavaScript. However, Modernizr is marketed on its ability to detect CSS3 and HTML5 features so that you can write gracefully degrading CSS:

Modernizr is a small and simple JavaScript library that helps you take advantage of emerging web technologies (CSS3, HTML 5) while still maintaining a fine level of control over older browsers that may not yet support these new technologies.

If Modernizr had sold itself as a JavaScript feature sniffing solution that also detected CSS features perhaps I could be more forgiving. In that context, it would be the developers’ responsibility to use Modernizr correctly. However, since the Modernizr site and documentation are selling it specifically as a CSS feature sniffing solution, for primary use within CSS, the error is clearly with the Modernizr team themselves.

It’s all down to the final step outlined in the documentation I quoted earlier:

It adds classes to the html element that explain precisely what features are and are not natively supported.

Modernizr uses JavaScript to add those classes to the html element; one for each of the features detected in the current browser. If we examine the generated source we see the following:

<html lang="en" dir="ltr" id="modernizr-com" class=" js flexbox canvas
canvastext webgl no-touch geolocation postmessage websqldatabase indexeddb
hashchange history draganddrop websockets rgba hsla multiplebgs backgroundsize
borderimage borderradius boxshadow textshadow opacity cssanimations csscolumns
cssgradients cssreflections csstransforms csstransforms3d csstransitions
fontface video audio localstorage sessionstorage webworkers applicationcache
svg inlinesvg smil svgclippaths">

When I first saw that, I was horrified. Not only is it ridiculously cluttered, it also feels decidedly unsemantic and looks like a terminal case of classitis. However, let’s try to be pragmatic here; it’s a means to an end, potentially a minor side-effect to a process that will win us some power in the bigger picture. In fact, it’s one step on from Paul Irish’s proposed work around to the FOUC (flash of unstyled content) which I’ve advocated as a reasonable fallback to marking progressively enhanced objects with an “enhanced” class.

Making use of the classes

The Modernizr method allows you to assign styles based on these new classes on the document’s root element like so (example lifted from the Modernizr docs):

button.glossy {
   background: #ccc url(gloss.png) 50% 50% repeat-x;
}
.cssgradients button.glossy {
   background: #ccc -webkit-gradient(linear, left top, left bottom,
         from(rgba(255,255,255, .4)),
         color-stop(0.5, rgba(255,255,255, .7)),
         color-stop(0.5, rgba(0,0,0, .2)),
         to(rgba(0,0,0, .1)));
}
.cssgradients button.glossy:hover {
   background-color: #fff;
}

Here we can see that button.glossy is first defined with a standard png as its background. The Modernizr-added .cssgradients class is then used to increase specificity and assign a WebKit gradient instead.

What happens if we approach the problem without the .cssgradients class:

CSS allows us to define a property twice in the same rule. It also contains built-in error handling that will ignore a property it doesn’t understand; so browsers that do not understand -webkit-gradient(), for example, will simply ignore a property that uses it. This allows us to progressively enhance our CSS like so:

button.glossy {
   background: #ccc url(gloss.png) 50% 50% repeat-x;
   background: #ccc -webkit-gradient(linear, left top, left bottom,
         from(rgba(255,255,255, .4)),
         color-stop(0.5, rgba(255,255,255, .7)),
         color-stop(0.5, rgba(0,0,0, .2)),
         to(rgba(0,0,0, .1)));
}
button.glossy:hover {
   background-color: #fff;
}

However, because we can no longer target the .cssgradients class specifically, we cannot target the button:hover styles only when CSS gradients are detected. Clearly this is an issue we’ll have to find some way around and is exactly what Modernizr is attempting to solve.

This all seems as if Modernizr is giving us a great deal of power, however, it comes with a significant cost.

Dependency on JavaScript is bad

Fellow web developer and good friend Mike Davies wrote a really great article containing a good summary of obstacles potentially preventing JavaScript from executing back in October 2010. This followed a somewhat presumptuous post by Nicholas Zakas on the YDN regarding the number of users with JavaScript disabled.

In short, there are a large number of obstacles that can affect the execution—or even rewrite the source of—JavaScript included in your pages. Hence, dependancy on JavaScript is bad because it introduces many significant points of failure.

As an example of why dependency on JavaScript can be catastrophic, I recommend you read another of Mike’s posts on Gawker’s JavaScript dependent URIs and how it broke their sites quite considerably. It’s worth noting that the Gawker sites have since seen the error of their ways and are attempting to fix the damage.

Styling dependant on JavaScript

By adding styles that are specific to classes that have been inserted with JavaScript, we are making those styles dependant on JavaScript. This means those styles will fail to apply if JavaScript itself has been disabled or, more commonly, the JavaScript file applying the classes has failed to load or execute successfully due to one of the reasons above. In essence, we may just as well add those styles with JavaScript directly because we’ve already broken the web standards separation of concerns.

Do we really want to remove the majority of CSS items covered by Modernizr if the Modernizr JavaScript include fails for some reason?

It would only be the feature-sniff-dependant CSS rules that would fail, since those classes would now be missing. Potentially this only means the loss of a few nice in-browser rendering optimisations. However, it also means there is a significant loss in the overall structure of specificity within your stylesheets, which could easily cascade unintended effects through to other elements. This could easily result in some content being styled to be unreadable, or worse, incorrectly hidden. This would be very bad indeed.

Alternative solutions

Modernizr is clearly unfit for purpose. If it were simply a JavaScript feature sniffing library for use only in JavaScript, it would be a suitable solution although only really useful for JavaScript progressive enhancement.

With that in mind, what other options are available that allow us to target our CSS?

Server-side browser sniffing

We could browser sniff on the server and add a class to the html element as we generate the HTML that is delivered in the HTTP response. This could result in something like the following:

<html lang="en" dir="ltr" class="firefox">

This means we can target CSS with the .firefox class in much the same way that we used the feature classes that Modernizr provided. There is no way for us to perform feature sniffing on the server; the best we could offer is a feature lookup based on the user-agent string that we’ve used to sniff the browser, but that would ultimately still be browser sniffing.

In fact, the best browser sniffing solution would allow us a little more flexibility on browser versions:

<html lang="en" dir="ltr" class="moz ff ff3-6">

Note I’ve added several classes this time; one for the rendering engine, one for the browser, and one that contains the major and minor versions. This would allow us more specificity.

Update: Matthew Pennell has written an interesting post on server-side browser sniffing with .htaccess and environment variables. Definitely worth a read.

Cache issues

However, server-side browser sniffing means that you’ll run into issues if your pages are publicly cacheable. This server-side solution results in a different HTTP response (the HTML) for each differing browser.

If you utilise any intermediary caching (Squid, Varnish, or a custom origin CDN) for static pages, and you use server-side browser detection, you need to make sure those caches don’t inadvertently send the wrong content to the wrong browser. To do this you’ll need to Vary: User-Agent header in the HTTP response. This instructs any intermediary caches to store multiple copies of the page (one for each User-Agent string that it sees) and to inspect the incoming User-Agent string when looking for cached responses to the current request.

IE targeting with conditional comments

If we’re entirely honest with ourselves as web developers, we can probably admit that the majority of woes we experience in CSS are as a direct result of features that any given version of Internet Explorer has not implemented. In fact, I regularly interview developers who advocate maintaining a separate stylesheet for IE that is included through the use of IE’s conditional comments. Having attempted to use this method in the past, I can confidently say that it is a maintenance nightmare and quickly becomes pretty unmanageable.

A better technique would be to adopt conditional comments to add IE specific classes in much the same way we have with Modernizr or server-side browser sniffing. This method was first proposed by none other than Paul Irish back in 2008 in his article "Conditional stylesheets vs. CSS hacks? Answer: neither". With that method, our html element would look like this:

<!--[if IE ]>
<html lang="en" dir="ltr" class="ie">
<![endif]-->
<!--[if !IE]>-->
<html lang="en" dir="ltr">
<!--<![endif]-->

This looks a bit odd, and often (for the true markup perfectionist) takes a bit of getting used to. However, it does give us what we need without requiring JavaScript or multiple cache versions.

We can even make it more specific if we so desire:

<!--[if lt IE 7 ]>
<html lang="en" dir="ltr" class="ie ie6">
<![endif]-->
<!--[if IE 7 ]>
<html lang="en" dir="ltr" class="ie ie7">
<![endif]-->
<!--[if IE 8 ]>
<html lang="en" dir="ltr" class="ie ie8">
<![endif]-->
<!--[if IE 9 ]>
<html lang="en" dir="ltr" class="ie ie9">
<![endif]-->
<!--[if gt IE 9]>
<html lang="en" dir="ltr" class="ie">
<![endif]-->
<!--[if !IE]><!-->
<html lang="en" dir="ltr">
<!--<![endif]-->

Quite clearly this gets ever uglier, but the solution is entirely encompassed in the structural layer; we will always have the correct class, even if CSS or JavaScript are unavailable for some reason. We also have the same markup regardless of user-agent string, which means caching is not a problem.

The only flaw with this solution is that we can only target versions of Internet Explorer. No other browser implements conditional comments.

Use CSS properly and make use of the cascade

As previously stated, CSS is designed to cascade. This means the built-in error handling will ignore properties the interpreter does not understand, and we can override rules based on specificity and source order. In most cases, these key features are more than adequate to develop gracefully degrading stylesheets.

Most capable front end developers are already doing this and coping just fine. Add this to the previous IE targeting approach and you’ll find that you do not need to over-engineer a solution.

In summary

So, in summary then, I absolutely cannot recommend implementing Modernizr as a best practice for front end development. It attempts to solve a problem from the wrong direction, and introduces a new potential point of failure. What’s more it breaks the fundamental ethic of web standards; the separation of concerns.

I continue to recommend well crafted, gracefully degrading CSS, backed up by conditional commented classes on the html element for targeting IE.

Modernizr may still be a worthy solution for JavaScript-only feature sniffing, if only it allowed the developer to disable the injection of classes on the html element.

Further to that, it’s important to know what the search bots are looking for when they spider your pages. From URL structure through to semantic markup and page specific metadata, there are a multitude of features you can build-in from the start that will improve the search engines’ insight into your content.

A note on ranking factors

Every two years, SEOmoz conduct a survey across various SEO experts which they use to publish statistical findings in relation to search engine rankings. This data is exceptionally useful in judging what elements to concentrate your efforts on during development.

In 2009 link metrics were worth a whopping 43% of value when calculating rankings. However, in the latest figures, that value has dropped to only 22%, which brings it in line with similar values for domain-level link authority (i.e. your domain is a “trusted” domain for quality). This reduction in size hasn’t resulted in other factors growing in value, but rather new factors have been introduced. These additions are domain-level keyword usage (how the keywords are relevant across the site), domain-level brand metrics (highlighting the importance of “brands” as a whole), and page-level traffic/query metrics.

Understanding the bots

Understanding how the search engine bots evaluate your pages and content is key to learning how page construction affects SEO.

Page vs Site

I have previously written—in my article “Semantics and Structure“—of the importance of remembering that the web is simply a network of single pages. This network has only a vague understanding of the human concept of site; via the comprehension of URLs beneath a fixed domain—and even that doesn’t necessarily translate to a single site, despite being used for authority rankings. Large sections of search bot AI has been dedicated to discerning site structure from the links within your pages.

Whilst we can see from the metrics above that domain-level factors are now more prevalent in the calculations for ranking, it is still important to imagine each page of your site as an independent unit, and structure your content and code accordingly. This means that the page title should be situated in an h1 tag and not the site title. That is unless you’re developing the home page, in which case the site title probably does belong in an h1 element.

Site-wide architecture should be situated within non-pertinent markup (i.e. markup that doesn’t apply any semantic emphasis on the content) and placed accordingly in the source order. I've heard very peculiar things about content placed in paragraph tags being more pertinent than content that isn’t. In my experience this is a significant fallacy. For more information about “pertinent markup”, read the “On-page optimisation” section later in this article.

Block-level analysis

Search engines, for the most part, follow an algorithm of block-level analysis. This means they will break a page down into sections (e.g. masthead, navigation, footer, main content, secondary content etc.) as a signal towards ranking the content within.

It is a good idea to make sure that the content you want to rank for is situated within your main content area. This may sound obvious but in a brave new world of modular development, and pages made up of “modules” of content, it is surprisingly easy to confuse the search engines and suffer for it in the rankings.

Source order is important!

It’s also important that you are prioritising your main content in your source order. Search engines will only evaluate the link text of the first use of a URI in a page. If you’re repeating a link with more valuable and relevant link text in your content than you are in your navigation, you’ll need to make sure the navigation comes after the main content. This is often the exact opposite of good UI design which will attempt to place navigation in the most obvious place; across the top, or down the left.

Take the following code as an example:

<ul>
    <li><a href="/about/">About me</a></li>
    <li><a href="/blog/">Blog</a></li>
    <li><a href="/contact/">Contact</a></li>
</ul>

<h1>My site</h1>

<p>Welcome to my site. In here you can find
<a href="/about/">information about me and
my career</a> and <a href="/blog/">my personal
blog about web development and the
internet</a>.</p>

Here the navigation links will be the first things evaluated by the bots, but they’re probably not the most contextually relevant links to the content in question. In fact, the second set of links include good keywords and may add significant value. For this reason the better option would be this:

<h1>My site</h1>

<p>Welcome to my site. In here you can find
<a href="/about/">information about me and
my career</a> and <a href="/blog/">my personal
blog about web development and the
internet</a>.</p>

<ul>
    <li><a href="/about/">About me</a></li>
    <li><a href="/blog/">Blog</a></li>
    <li><a href="/contact/">Contact</a></li>
</ul>

Obviously you can change the visual position of this content with CSS. Whilst the more sophisticated search bots will render CSS in an attempt to detect invisible content (i.e. that which is hidden either through display or visibility rules, or that which is moved offscreen), they are less bothered about the reordering of content visually.

Creating content

It may sound entirely obvious but it’s exceptionally important to write good textual content for search engines. You will always rank higher if you include a good balance of text and links—with a high relevancy to each other—on your pages.

Duplication is bad

Try to avoid duplication of content. By this I mean avoiding the same content on two different URIs. If your content is duplicated then you will be diluting its value by placing it in two places, even if it’s actually the same page served from two different URIs. I’ll cover the method you should use to declare the One True Version™ of your pages later on.

It’s also important to make sure your pages don’t duplicate content that is elsewhere on the internet. Good examples of this sort of repetition would be travel brochure text or product descriptions which are highly likely to be used on a multitude of affiliate sites; especially if that content is included in some sort of feed. Wherever possible try and create your own content; it will always serve you better and will ultimately separate you from the crowd.

Dynamic vs. static content

Dynamic content is that which updates regularly and hardly ever stays the same from day to day. Good examples of this might be a list of blog posts on a blog index page (where the blog items update regularly), news item indexes, feeds from other sites (e.g. RSS, Twitter etc.), and regularly moderated lists of links.

Static content is that which hardly ever changes. Good examples might be “about me” text on your blog, the description on a product detail page, and the article text on a blog article page.

It is very important to find the right balance of both dynamic and static content on your pages. Some pages will suit more static content (e.g. the article page of a blog) whereas others will suit more dynamic content (e.g. the index of that blog). In either case, make sure you’re including some of both types of content. On the article include links to the top articles on the blog, or feeds from Delicious or Twitter; on the index page include some static “about this blog/author” text.

Design your URIs

Since links are the most important part of your SEO strategy, it’s hardly surprising that the design of the URI is a fundamental ranking factor. It’s important that you use just as much care with the URI as you do with the text of those links. Here are the important factors in good SEO-friendly URI design:

Keywords in your domain

If you can, try to get some keywords in your domain. Often the most successful domains will have one or more relevant keywords in their domain name, and the closer to the left the better. A good example of this might be something like travelsupermarket.com, which is currently number one in Google for “travel”.

Rather interestingly, _exact_ keyword match domains seem to perform marginally better than domains with _some_ keywords. By this I mean domains that are entirely formed of keywords in the order for which they are searched (e.g. cheapmajorcaholidays.com). Hyphenated exact match domains (e.g. cheap-majorca-holidays.com) seem to perform slightly worse than those without hyphens, and domains that contain all query terms but are not an exact match are marginally worse again.

Choose the right TLD

Rather notably the .com TLD (top-level domain) appears to perform better than any other TLD. That’s not to say that you can’t easily out rank a .com domain, but they do seem to gain an advantage in a like-for-like ranking test with pretty much every other TLD I tried.

Subdomains

Subdomains do not count as part of the domain. Cross subdomain links, as previously stated, are deemed internal and URIs containing a keyword subdomain appear to rank similarly to URIs with the same keyword as the first path element. Example:

http://games.mygamessite.com/

Ranks the same as:

http://mygamessite.com/games/

Keep paths shallow

Try to keep your URI path to a minimum. Each level you go down the hierarchy, the less value you give to keywords within it. This is another good reason to maintain a flat site architecture. Also, remember that the keywords on the left of each path segment (i.e. each section delimited with /) are the ones with the highest value at that level.

The maximum number of layers (or segments) you should use in a path is about 3. Any more than that and you’ve basically lost any SEO benefit from that section of the URI.

Be strict with your characters

There are a limited subset of characters that are permissible in URI syntax. However, that subset still allows a great variation in the style of your characters. In general it is best to keep your URIs strictly lowercase so as to cut down on the chance of creating or generating duplicates through case sensitivity. Also, try and internationalise the character set in your URIs; it’s always going to be easier to match a UTF-8 search term against a UTF-8 keyword in your URI.

Space separators

There are several characters that can be decoded as a space in URI syntax, but only two that will work successfully as part of your path: “%20”, and “-“. It is best to enforce the use of a hyphen as a word separator in your paths. Google understands this as a space, and it will ensure your URIs remain readable. Underscores (“_”) are not recognised as a word separator and are therefore of little use. Matt Cutts has previously discussed this in his "dashes vs. underscores" blog post.

Branches and leaves

Do not underestimate the value of human readability when designing your URIs. I have personally experienced, through extensive user research and user testing, that users afford a certain level of confidence to a readable URI. What’s more, some SEO consultants whom I have spoken with recommend applying slightly different rules to branch and leaf URIs.

A branch URI is a node that could potentially lead to more branch URIs or leaf URIs. The recommendation is that these URIs should end with a /.

Branches:

http://sportsnews.com/football/
http://sportsnews.com/football/bundesliga/

A leaf URI is one that is the final node in the path tree. The recommendation is that these URIs should not end with a /, and should include some kind of filetype extension (usually .html). I prefer leaving out the extension myself, since it feels a bit old school and almost enforces a file-type assumption on the resource in question, but I accept I may be clouded by a developer’s understanding of HTTP and REST.

Leaves:

http://sportsnews.com/football/international/teams/england.html
http://sportsnews.com/contact-us.html
http://sportsnews.com/terms-and-conditions

These recommendations amount to a more traditional OS directory-style vision, which makes them more familiar to the general non-techie user. It’s worth remembering that the URI is displayed prominently in the SERPs and as such inform the users’ confidence in clicking the item.

On-page optimisation

Now you’ve hosted the pages on good SEO friendly URIs, and you’ve built a good network of links to those pages, it’s about time we looked at improving the way the search bots spider and evaluate your content:

Page title

The page title should sum up the content in as few words as possible. On left-to-right reading pages the left most words are deemed more significant (I’ve no research on right-to-left languages, but one would assume the opposite). A good title will have a healthy sprinkling of keywords whilst remaining human readable:

<title>SEO for Web Developers - Nefarious Designs</title>

Note that, in my titles, I’ve chosen to include the site title following the page title. Firstly, this means the site title will be included in the page title on the SERP, but it also means that the site title is associated as a keyword or keywords.

Semantic content markup

The majority of semantic markup won’t give you a significant boost in rankings. Sorry folks, it’s true; despite the fact that we web devs love some good semantic markup, the search bots are less bothered. Let’s face it, the internet is still full of badly constructed web pages and the bots have to spider, analyse, and rank those too.

However there are some semantic elements (and some presentational elements) that will have a greater influence on denoting keywords in your content to the search engines. These are:

• h1-h6: Well structured headings, that are relevant to your content (and not used as way-finders such as “navigation”) are highly important. The h1 on your page is arguably the second most important element short of your page title and should contain relevant keywords accordingly.

• em, strong, i, b: Emphasising textual content with these tags will infer the content has some degree of added value. However, it’s important not to saturate your content with this as it will inevitably be flagged as keyword spam. Overall my tests were inconclusive in quantifying the effect of emphasised keywords, but there is clearly value in it.

• a: I’ve previously waxed lyrical on the importance of links within your content, and the textual content of those links is just as important. Linking to similar pages with text that involves your keywords, or is simply related, can significantly boost your pages’ performance in the SERPs.

  As a side note, I’ve been told by so-called SEO “consultants” that all internal links should be fully qualified (i.e. be absolute links including the domain) to prevent scrapers from duplicating your content elsewhere on the internet. This is absolute nonsense. There’s really no reason that a scraper won’t be able to remove the domain from any links it identifies as internal to your site (i.e. on the same domain as the page it’s scraping).

Meta elements

There are plenty of articles misrepresenting the value of meta elements in terms of SEO. For pretty much the last ten years, they have been mostly irrelevant for SEO. In fact, at the moment, this very site isn’t using them (more through laziness than intention—the site templates are old and I intend on migrating away from them sooner or later).

Keywords

The keywords meta element is all but abandoned by todays’ search engines. You will see little effect in rankings if you remove it altogether. I have noticed some smaller search engines using it, and it’s conceivable that other search engines may use links from these search engines as authority on your content. In short, you might see a small amount of benefit by proxy, but possibly not enough to warrant making the effort to keep the keywords list up to date and matching your content.

Description

The description meta element is used as the first port of call for the description of your page in the search engine. There is also a minor ranking effect from including keywords in this description but as ever, it’s important not to spam them here.

Link element

The link element is a sneaky little devil. Most web developers use it solely for linking stylesheets to their pages, but it has so many other uses that can help the search engines understand your site architecture better.

The link element allows you to specify another URI that is linked to your page, and also what relationship that URI has with the current one. To specify the relationship, you declare it in the rel attribute. The rel attribute has many possible values, but only a few that are relevant to SEO:

Canonical

At a minimum it’s important to understand the canonical link element. This allows you to declare the One True Version™ of any page on your site. This is probably easier to explain through example. Consider the following two URIs:

http://sportsnews.com/football/

http://sportsnews.com/football/?ads=false

Assuming these are actually the same page, the search engine will consider these to be unique URIs and could penalise you for duplication of your content. To avoid this, it’s important to declare one version as the canonical version. To do this you add the following to the page:

<link rel="canonical" href="http://sportsnews.com/football/">

This method is also true of something more subtle like “topic” style pages. Imagine the following URIs:

http://sportsnews.com/teams/tottenham-hotspur/peter-crouch

http://sportsnews.com/teams/england/peter-crouch

By adding a canonical link to the first item we can declare that, although the pages are exactly the same, the second is actually the same page on a slightly different URI.

Others

There are a whole host of available options to use in the rel attribute of the link element that may improve the search engines’ understanding of your pages and site structure. I really can be bothered listing them all out here, but you can find an interesting explanation of SEO relevant rel values here.

Search bots as knowledge-based systems

Modern search engine bots are incredibly complex programs. Considering the amount of information they gather on each pass of your pages, they should certainly be considered to be knowledge-based systems. As the web changes, so do the methods the bots adopt to navigate your pages in as close to a humanised way as possible.

To this end, both Googlebot and Bingbot are capable of understanding fairly complex navigation systems. What’s more, they are very good at evaluating navigation links within your pages to ascertain different routes to your content. Through this understanding they are able to apply value calculations to specific URIs.

Breadcrumbs

Google announced at the end of 2009 that they were attempting to display site hierarchies as an alternative to the URL in the SERPs where they could surmise it from on-page breadcrumbs. For more information on this, see “New site hierarchies display in search” on the Googleblog.

To do this, Googlebot simply analyses any constructs that look like a breadcrumb within your pages. This can be either a list of links identified by “breadcrumb” as an ID or class, or a set of links separated by the “>” character. In my tests both of these methods have been successfully picked up by Googlebot and displayed as site hierarchies on the Google SERP.

There are other methods of marking up breadcrumbs using “rich snippets” which I will talk about later.

Learning new interaction paradigms

Googlebot has been designed to understand modern in-page navigation paradigms, such as tab, carousel, and accordion widgets. It does this by searching for the presence of known markup configurations and included scripts.

For this reason, it’s sometimes better to use commonly used scripts such as popular jQuery or jQuery UI widgets which are likely to have a large user base across the web. Sadly this often means that badly written front-end interaction scripts may perform better than more bespoke options.

You can also improve Googlebot’s understanding of your pages by adopting common element identifiers on your JavaScript enhanced elements. This means using classnames such as “open”, “closed”, “active”, or “selected” etc.

It is worth noting, however, that in my tests rankings seemed better when pages used prebuilt widgets (i.e. those already in its knowledge base) than with bespoke techniques using predictable classes or IDs.

Rich snippets

These days Google is capable of rendering a large number of “rich snippets”. These are generally pieces of code specifically designed for marking-up data of a particular type, e.g. microformats, microdata, and RDFa. Often these snippets simply involve a range of predictable element identifiers or tags.

When displaying search results, Google will try and handle these rich snippets in the best way it sees fit. For example, reviews will appear as a 5 star rating widget just below the title of the search item. Recipes, on the other hand, are awarded their very own search utility:

Google Recipe search for “Chicken Madras”

Currently, Google supports the following types of rich snippet:

• Reviews
• People
• Products
• Businesses and organisations
• Recipes
• Events

For more information on implementing these snippets for your content, take a look at Google’s excellent rich snippet explanation page.

JavaScript

Googlebot currently runs a headless browser as it navigates pages. A headless browser is basically a browser with no user interface and, in this context, it means Googlebot constructs the correct DOM, and attempts to render all CSS and some JavaScript. It also means that many DOM events are fired as it follows links through your site, potentially exposing functionality it wouldn’t normally find.

The JavaScript related tests I have run to analyse the bots’ paths through my sites have largely been quite inconclusive; it appears that sometimes Googlebot simply evaluates embedded script in the page (i.e. code inside ). It also appears to find URLs manipulated into links on click events. However, it is definitely finding links in embedded script tags more regularly than those in external script files.

So far I haven’t managed to get Googlebot to spider content or links added during a DOMready or window.onload event. My tests are ongoing and I shall endeavour to document my findings on this blog once I have a clearer picture.

Summary

Apologies for the length of this article, it just kept growing! There was so much to cover that I really didn’t want to have to split it out into several sub-articles; especially considering the fact that it’s already part of a series.

Hopefully you’ve now got a head-start when it comes to building your pages for SEO. Tie this to the link building and keyword targeting you’ve been doing following my first article and you’ll be ranking well in the SERPs.

In the following article I’ll be looking at what you can do to help the search engine robots specifically by providing metadata for them, and how proper handling of HTTP can improve crawlability. I’ll also cover some basic tools that can aid your monitoring and improvement of SEO.

What’s more the proliferation of false knowledge, the equally charlatan-populated worlds of SEO consultancy and web development, and the sheer abundance of SEO false positives have meant that snippets of true SEO wisdom are drowning in a sea of irrelevant balderdash.

It is no wonder then that your average web developer is not armed with the erudition required to fight the good fight; knowledge that will allow them to articulate the message of their content to the search engines and therefore be exposed to a wider audience.

Fear ye not, weary web devs, for here is your codex of power. I hereby commit to blog my personal experiences and knowledge of Getting SEO To Actually Work™. Everything I discuss in this series of articles has been learned through experimentation, careful analysis, and search-bot honey-pot test sites. Oh, and mummyfrakkin’ science, bitches.

Keywords

One term you will hear more than others when researching or improving SEO is “keyword”.

A keyword is a word or phrase that contains high relevance to your content’s subject matter. A keyword is best imagined as a term a user might enter into a search engine in the hope of finding content like yours. For example, on a travel website generalised keywords might include “travel”, “vacation”, and “holiday”, whereas more specific keywords might be “5 star holidays” or “Greek holidays”.

As the search engines crawl and index the content on your pages they keep track of those pages in keyword-based indices. Thus, rather than storing 25 billion web pages all in one database, the engines maintain a vast array of smaller databases, each centred on a particular keyword term or phrase. This denormalisation makes data retrieval much faster, reducing it to mere fractions of a second.

Use of keywords

The search engines analyse the embedded keywords within your content to ascertain their relevance to the subject matter. This means a good way to optimise your content is to make sure the keywords are situated in key content structures, such as URLs, page titles, headings, and areas of emphasis (<strong>, or <b>; your non-presentational mark-up bears little relevance here). I’ll cover this in more detail when I talk about on-page optimisation later in this series.

It is also of note that a generalised keyword—such as “car insurance”, for example—will have a broader use, and therefore considerably more competition in terms of rankings. Something more specific like “women’s small car insurance” will limit your competition but might not have such a large search density. There is a clear balance to be found in your market and it’s important to identify the keywords you wish to target for a given page as early as possible.

A word on keyword density

The term “keyword density” describes the density (i.e. occurrence) of a keyword within your content. The term is a veritable minefield of misinformation and should set off alarm bells as soon as it is mentioned.

The density of your keywords bears no relation to the quality, semantics, and relevancy of your content; and it is these characteristics that glean the highest return. Do not be tempted into thinking that a page saturated with keywords will perform better in search rankings than a page that seats fewer keywords in higher quality, higher relevancy content.

Keyword targeting

As I said earlier, it’s important to identify the keywords you wish to target before you do anything else. To be effective, SEO requires that keywords are associated with and contained in your content at as many levels as possible.

Most SEO consultancies will invest significant amounts of time researching keywords for a given market, and also the competition against those keywords. What’s more, that analysis is ongoing, as the value of your chosen keywords will fluctuate wildly due to changes in the market, target saturation (i.e. too many people using it), and other such external factors (including black-hat devaluation which I will cover later).

Links

Let’s make sure we’re not under any false illusions before we get too far in; the two most important things about any page—in terms of SEO—are the links into it, and the keywords associated with those links. You could serve up the most hideous pile of non-web-standards, non-semantic, tables-for-layout, text-in-images content, and if the links into it are plentiful, and also contain the right keywords and/or link juice (more on that later), it will perform well in the rankings. Links are how the search engines will begin to calculate the relevancy of your page. Once they have a relevancy rating, it will then be affected by the analysis of the content of that page.

That’s not to say that there isn’t some artistry in the placement, design, and variance of those links, but since hyperlinking is the very essence of the world wide web, it is unsurprising that it is also the very essence of SEO. A high quality link will be featured on a page containing high relevance to the link’s subject matter, will be situated in an area of highly relevant content, and will have link text containing relevant keywords. It’s all about relevance.

Link Juice (aka PageRank)

Search engines apply a numeric score to a spidered page known as PageRank. This is often called “Link Juice” in SEO circles and is a calculated number based on the links—and the quality of those links—into, and out of, your page. This score is then used as a factor in calculating the ranking of your page against keyword relevancy.

As the search-bots spider the links within a page they pass a proportion of the host page’s score to each link. The page at the other end of the link is therefore afforded a total based on the score flowing into it from links elsewhere and how it passes that score onward. The best way of imagining this is as a complex network of pipes and sluice gates (links), with the score being the liquid within. High quality pages will have higher page rank due to incoming links with a higher score.

Bear in mind, however, that “link juice” is calculated differently based on “internal” and “external” links. Most search engines appear to deem links within a domain (internal) of less importance than those from a different domain entirely (external). For the most part, my linking tests seems to infer that cross-sub-domain links are also deemed internal. However that sort of testing is fairly difficult to quantify, so I’d recommend investigating for yourself if you’re intending doing something like that.

As an aside, it’s worth noting that a high link score (juice, PageRank, call it what you will) does not necessarily mean high SERP rank.

The “nofollow” rel attribute value

Following the dawn of the “blogosphere” and other such UGC (user-generated content), such as forums and guestbooks, there was an explosion of link-spam. These are links that have been placed on as many different sites as possible—either manually, or through automated processes known as spam-bots—with little relevance to the intended page content or conversation. This link spam is a black-hat method of rustling link juice into a given target page.

To combat this, the search engines implemented a scheme that allows a developer to mark a link as non-pertinent in the flow of link juice. To do this, you apply a rel attribute of nofollow:

<a href="http://somewhere.com/" rel="nofollow">Link</a>

nofollow sculpting

The rel="nofollow" attribute used to allow you to sculpt the score that would pass between links on your site, by preventing score flowing through no-followed links. This is no longer the case.

Before Google changed their algorithm, rel="nofollow" used to be handled like this:

Link 1 - Link score/4
Link 2 - Link score/4
Link 3 - Link score/4
Link 4 - Link score/4

Link 1 - Link score/3
Link 2 - No follow
Link 3 - Link score/3
Link 4 - Link score/3

Now it works like this:

Link 1 - Link score/4
Link 2 - No follow
Link 3 - Link score/4
Link 4 - Link score/4

This means that you get significantly less benefit from no-following a link than you used to. However, all the search engines (with Ask being an exception) state that they do take rel="nofollow" into account when weighting rankings:

Google, Yahoo!, and Bing say no-followed links are treated as plain text and therefore have no added emphasis when calculating keyword relevancy. Y! and Bing, however, do state that they use no-followed links for discovery and may follow them without affecting your rankings.

It’s all very complicated but suffice to say, it’s probably better to mark unimportant links on your pages as rel="nofollow" and let the search engines sort out amongst themselves what they want to do with it.

Improving “crawlability”

To improve the efficiency of spidering (i.e. how the search bots crawl your site) it is advisable to maintain a flat site architecture. In this context, site architecture is simply a means to describe the link depth of your pages. “Link depth” is the number of links it takes to navigate from the home page of your site to any given page within it. The shorter the maximum link depth, the better the “crawlability”.

For the most part, search engines understand a “site” to be a confined graph of internal links as mapped by the search bot when it spiders your site. Each pass through the site will give the bot a better understanding of your site-map. There are several ways to inform this process, such as sitemap XML files and the robots.txt file. I’ll cover those later in this series.

Maximum links per page

Most search engines recommend hosting less than 100 links on any page. This is primarily because, above a certain number, link score will deteriorate due to a search engine defined degenerative quality equation. This will result eventually in links that are not spidered at all. Secondly, a large number of links on any page is a signal of diminished content quality and will directly affect SERP rankings as a result.

The arbitrary value of 100 is misleading though as a high quality landing page, such as your homepage, can easily maintain 250-300 good quality links without penalty. The pages at the next level can then host ~200 quality links and so on. With this architecture, a site with a link depth of 4 or 5 can easily maintain millions of pages within its structure with little or no penalty.

Link building

Link building is the art of obtaining external links back to your content from high quality (i.e. high relevance to your content subject matter) pages elsewhere on the web. There is a fine line to walk between link building and creating link spam; it is the difference between the relevance of the surrounding content on the link hosting page.

Many SEO companies employ link building specialists who will inundate the web with links, either posted manually or accrued through link sharing contracts, registration with pay-to-link directories, and other such socially manufactured solutions.

Third-party linking

To build effective links it also a good idea to offset the effort. In some cases, placing a link on a site with high traffic, but that isn’t particularly high quality—or isn’t even spidered effectively—can work in your favour. A good example of this is Facebook which hides a large majority of its content from search engine spiders. A link placed on Facebook has a high likelihood of being reposted elsewhere, where the link quality will be vastly improved.

In today’s internet company you will find many Social Media “Experts” who’s sole responsibility is to use social networking to market the brand, the company, and, more importantly, links to the website. Whilst the quality of links on social networks is debatable, the benefit of reposts elsewhere cannot be ignored, regardless of quality.

Link bait

Another technique when link building is to create content that can be classed as “link bait”. This is content that has a high likelihood of being reposted elsewhere and is therefore subject to significant third-party linking.

In my personal opinion the concept of link bait is very hypothetical; one man’s link bait is another man’s quality content. In web development particularly there are any number of cynical parties waiting in the wings to troll a link as link bait. This may, however, have something to do with the inherent cynicism in the development world in general.

Summary

Understanding the concept of keywords, and then applying that concept to link building and your internal link structure, are really the two base concepts of SEO. Once you have mastered them you should be well on the way to improving your performance in search engine rankings.

The next article in this series will cover SEO in page construction, including information about on-page optimisation, designing your URLs, correct use of HTTP, and other such technical information.

Following a small refactor—guided by some more Pipes-knowledgable friends of mine (cheers Carlo and Dirk)—I privatised my token, and allowed customisation of the GitHub account used.

So, without further ado, here’s the pipe; feel free to use it to stalk your friends on GitHub:

GitHub Friend Stalker

As the front-end development team where I currently work are going through a definition process, I have been reminded of a few things I’ve learnt over the course of my career. With that in mind, I thought I’d type up some of that knowledge in the hope that it might benefit other developers out there.

Some things to remember

It’s important to remember that a developer takes great pride in his or her work, and as a result has a significant emotional attachment to the code they output. When that code is submitted for a code review—where the implementation of any coding standards is going to be assessed by the developer’s peers (obviously alongside other things)—it is important not to tear the code apart without respecting that emotional attachment.

Further to this, it is important that the reasoning behind the rules in the coding standard are rigorously explained and well documented, along with good and bad examples. If the developers understand why they should be doing something, you’re more likely to get them doing it as second nature.

Finally, always check to see if there is a more general coding standard you can utilise, such as Python’s PEP-8 standard or the PHP PEAR standard. It makes far more sense to start with one of these and then build your own standards upon it.

Why have a coding standard?

There is really only one reason to adopt a coding standard; to maintain a predictable base of easily maintainable, understandable, and legible code. Ultimately, you are aiming to develop code that can be picked up by any developer—either in an internal team, or some other, wider community—and easily read, understood, and either fixed or appended.

Further to that, it is not the opportunity to enforce your own particular coding style on others; in fact, you should probably expect to change your own style as there will invariably be some decisions made that will alter your usual syntax or method. This should be embraced as a good thing, as you may learn something.

Creating

When creating a new coding standard, it’s important not to dictate rules to the wider team or community. The best standards are defined in agreement, not by dictation. It’s important to make the definition of team standards a team task.

When a standard—and the reasoning behind that standard—are offered to the team, discussion will invariable ensue. That discussion needs to be controlled and constructive, without dragging on. Nobody wants to sit and listen to two or three developers arguing; sooner or later someone will need to step in and make a decision, despite potentially disagreeing (and therefore upsetting) multiple parties involved.

A good way to avoid massive discussion on mailing lists, or in team meetings, is to make sure your recommendation and acceptance processes are not too complex. As soon as you instigate a complicated system of recommendation, confirmation, voting etc. you will instantly regret it. Developers are adverse to change, and you are potentially changing the very fundamentals of their day to day work. You just can’t expect the process to be quick and painless unless you enforce quick and painless at every possible stage.

The absolute best way to implement and maintain a coding standard is based on the outcome of code reviews. These are the perfect forum for discussion of style and method, and a good follow-up on that discussion is to update the coding standard.

The last thing you should bear in mind when developing a coding standard is that it will usually work best if it’s fairly language agnostic. Similar rules should be applied universally across languages; if you break this rule, you’ll quickly find yourself in a heated discussion about some minutia of syntax that is entirely deconstructive. Having said that, there are always going to be particular features and conventions of a specific language that should be applied to aid legibility or maintenance.

Comments

I hate badly commented code. I also hate writing comments. Ergo, I hate my own code.

Comment as you go

Commenting code is usually way outside the modus operandi of most developers. If we feel a piece of code should be commented, we’ll go back and do it afterwards, which means we’ve automatically given ourselves a mind-numbingly dull job that will, without fail, fall by the wayside upon release. To comment effectively, it’s important to do it as you go, even if you find yourself changing the comments as you refactor your code.

Comment sensibly

Comments should be added in distinct blocks, rather than commenting every single line. It’s important to remember what is important in a block of code and explain that. You’re documenting it for maintenance, so at least afford the next developer a modicum of intelligence. Of course, if you come back to it 6 months down the line, that next developer could very well be you!

When I say “distinct blocks”, I mean functions, methods, objects, classes, event callbacks, blocks of CSS rules, and anything else that requires explanation e.g. particularly complicated or confusing sections of functionality that might not otherwise be entirely obvious. Also, be sure to include references to other sections / files that may be relevant.

Pick your audience

As I’ve said previously, comments are there for the code maintainer, so if you can hide them from the end user, you may as well do so. Comments should be added in the template language rather than the HTML; likewise, any other code generation should keep the comments in the generator not the generated file (unless it is specifically pertinent to do so). If you comment JavaScript and CSS, make sure the comments are removed as part of your build process/combo handler, alongside minification.

Commenting standards

Commenting standards, such as JSDoc, are a good thing. You’re already trying to implement a coding standard, you might as well leverage [manager++] someone else’s.

Naming conventions

Language constructs may allow you the freedom of your given language, but please adopt a little restraint. It’s important to make them readable and decipherable by the next developer along.

Style

Try and adopt a naming convention that suits the style of the language. Here are some examples:

• HTML and CSS — Use hyphenated lowercase IDs and classes:

  <div id="live-scores" class="tab-module">
      …
  </div>

• JavaScript — Use camelCase variable naming, with constructors in UpperCamelCase:

  var myVar = true,
      myFunction = function() {
          // Wicked awesome codez here.
      },
      MyConstructor = function(id) {
          this.id = id;
      };

• PHP, Python — Use lowercase with underscore separation, with UpperCamelCase for classes:

  $dev_name = "Timmah";
  
  
  function get_foo() {
      return "foo";
  }
  
  
  class SimpleClass {
      // property declaration
      public $var = 'a default value';
      // method declaration
      public function displayVar() {
          echo $this->var;
          }
  }

Grace

It is important to keep names concise yet explanatory. There’s no need to write a variable name that’s fifty characters long, if you could do it in ten. Likewise, it’s important you aren’t just writing a nonsensical string of characters that ends up looking like some kind of demented Countdown Conundrum.

Bad:

var boolean_flag_for_portrait_view = true;

Also bad:

var bl_flg_pview = true;

Good:

var is_portrait = true;

Prefixing / suffixing type

Type prefixes and suffixes are a bad idea; loosely typed languages allow the developer the ability to change things up, so you’d better expect it once in a while. Further to that, any intelligent developer can work out what type a variable is if they absolutely have to, so why waste the characters?

Final note: If type prefixes and suffixes are a bad idea, Hungarian notation is a very very bad idea. Don’t go there. SRSLY.

Style

As I’ve stated previously, a coding standard is not a way for you to enforce your own coding style on others. Try and keep any such rules to a minimum.

Tabs vs. spaces

One of the first conversations you’re going to have, when defining a coding standard, is whether you should be using tabs or soft-tabs (spaces) for indentation. In a modern development environment, this is irrelevant—even in languages where whitespace indentation is important, such as Python.

A developer should be able to set-up their IDE to use either tabs or spaces when indenting, thus they shouldn’t have to worry about it. The only thing that is important in this area is that it remains uniform in any given file. There is nothing worse than opening a file to find some things indented by 4 spaces, and others indented by 3 [deviants!].

Legibility

It’s important to write code that is legible; the more so, the better. To aid comprehension of blocks of code, it is useful to separate them with new lines.

It’s also important to try to keep lines of code to a reasonable length. Many developers and coding standards recommend a length of around 80 characters, with some even enforcing it. Python’s PEP-8 certainly falls under this heading. Personally, I’d prefer going over that length if the line remains legible; often breaking a line with carriage returns can have a negative effect on legibility.

Similarly, Lines that are too long are often difficult to read. If this is the case, then you must attempt to find the most legible form. Legibility is always the most important factor.

Whitespace

Liberal use of whitespace in your syntax can mean the difference between readable and unreadable code. Having said that, over use of whitespace can be entirely detrimental. Like my previous notes on style, it is important to find a balance when wrapping parentheses, expressions, assignments, and other such code with whitespace:

Readable:

var myVar = 'foo';

for ( var i = 0, j = myArray.length; i < j; i++ ) {
    …
}

Less readable:

var myVar='foo';

for(var i=0,j=myArray.length;i<j;i++){
    …
}

Accepted standards

There are many accepted standards used across multiple languages. It is important to make sure you don’t break these with your coding standard, even if you don’t specifically declare them anywhere. Good examples might be:

• Use of i, j, k for simple iterators.

  for ( var i = 0, j = myArray.length; i < j; i++ ) {
      …
  }

• Use of _ to denote private members.

  var myConstructor = function() {
      this._private = 'something private';
  };

• Use of UPPERCASE to denote constants.

  var VERSION = 'v1.1';

Expressions

Expressions are often where the complex logic within your code can be found. As such, to aid assimilation, you may like to break them down to their constituent parts.

A complex expression:

if ( myArray.length < 4 || ( myString.length > 2 && myString !== 'foobar' ) ) {
    …
}

Might become:

var isArrayShort = ( myArray.length < 4 ),
    isStringCorrectLength = ( myString.length > 2 ),
    isStringMatched = ( myString !== 'foobar' ),
    hasCorrectString = ( isStringCorrectLength && isStringMatched ),
    isPassed = ( isArrayShort || hasCorrectString );

if ( isPassed ) {
    …
}

If we clean it up further:

var isArrayShort = myArray.length < 4,
    isStringCorrectLength = myString.length > 2,
    isStringMatched = myString !== 'foobar',
    hasCorrectString = isStringCorrectLength
                       && isStringMatched,
    isPassed = isArrayShort
               || hasCorrectString;

if ( isPassed ) {
    …
}

Note the fact that I have broken the logical operators onto their own lines. This is another technique that can sometimes aid the readability of an expression, should it adopt multiple ANDs and ORs.

This last example may look more complex to begin with, but should a second developer read your original code, they will be left confused. The second example breaks the expression down into constituent boolean values to aid legibility.

Now imagine we have a similar example, but adopted using ternary notation:

var output = ( myArray.length < 4 || ( myString.length > 2 &&; myString !== 'foobar' ) ) ? 'win' : 'fail';

This would become:

var isArrayShort = myArray.length < 4,
    isStringCorrectLength = myString.length > 2,
    isStringMatched = myString !== 'foobar',
    hasCorrectString = isStringCorrectLength
                       &&; isStringMatched,
    isPassed = isArrayShort
               || hasCorrectString,
    output = isPassed ? 'win' : 'fail';

Ternary becomes much easier to read by a second party when used in this way; particularly when dealing with nested ternary statements. Of course, a good rule of thumb is to just avoid nesting ternary statements altogether.

Concerns over file size

In some languages, the size of the source code files you are creating can be pertinent to the performance of the final software. The obvious example of this is JavaScript files being served to a client.

In this instance, it is important not to sacrifice maintainability for optimisation. Should your files need optimising for final consumption, factor that optimisation away into your build or delivery mechanism. With JavaScript, this means minifying your code using an algorithm that can understand how to refactor it. Google’s Closure Compiler is a good example of one such tool; however, it still has limitations, and you may find it beneficial to write your own based around your coding standards.

Enforcing

There is really only one way to enforce a coding standard, and that is exactly the same way you should be developing your coding standard; through code reviews.

Summary

As my final word, I’d just like to point out that your mileage with this may vary wildly. This article is really only based on my own experience. However, whether you agree with everything here or not, I would hope you agree that easily maintainable code is the most important outcome of any good coding standard.

Without further a-do, let’s jump in with the technical stuff:

A quick note on variable declaration

The var declaration allows you to chain definitions in a comma separated list; what’s more that list can reference variables declared earlier in the same list.

Take this code as an example:

var el = document.getElementById(“my-element”);
var elWidth = el.offsetWidth;
var elHeight = el.offsetHeight;
var elArea = elWidth * elHeight;

This could alternatively be written like so:

var el = document.getElementById(“my-element”),
    elWidth = el.offsetWidth,
    elHeight = el.offsetHeight,
    elArea = elWidth * elHeight;

Personally, I prefer the latter method simply because it keeps my var declarations in one place (where possible) without endless repetition of the var keyword. However, this is simply a minor note on coding style, rather than something that can improve the operation of your code.

Understanding object-oriented scope

As any experienced JavaScript coder knows, JavaScript is functionally scoped; this means any variables created within an object or function are only available within that object or function.

Simple, right?

Well no, there are actually a couple of complications to this behaviour that you should really understand:

Passing by reference

When passing arguments in JavaScript, it’s not immediately clear how those arguments are being handled internally. In fact, it is entirely dependant on the data-type of those arguments.

When passing arguments of type Number, String, or Boolean, JavaScript will pass them in by value. This means that the value stored in the argument will actually be a copy of the original value:

function accidentalFall(count) {
    count = count - 1; // Remove one
}

var greenBottles = 10;

alert(greenBottles); // greenBottles is 10

accidentalFall(greenBottles); // Pass by value

alert(greenBottles); // greenBottles still 10

Passing in an object, however, passes the argument by reference. This means that, within the function, the argument is not a copy, but a reference—or link, if you like—to the original object. This means any members of that object are available within our function, and any changes to the object passed as an argument will be reflected outside of the scope of the function.

Actually, the way JavaScript handles references is another blog article in itself. Watch this space.

Here’s our example again:

function accidentalFall(obj) {
    obj.count = obj.count - 1; // Remove one
}

var greenBottles = {
    count: 10
};

alert(greenBottles.count); // 10

accidentalFall(greenBottles); // Pass by reference

alert(greenBottles.count); // 9

This is a subtle difference, but one, I think you’ll agree, that is important to understand.

The this special operator

The this operator can be used to obtain a reference to the current context object and allows properties and methods within that execution context to be referenced. The context object can be considered a “hidden” parameter that is passed to any function.

There are four ways the context is made available to a function:

Implicitly with a method

var myObject.method = function(arg1, arg2…) {
    // Context here is myObject, and has been passed
    // implicitly.
}

Implicitly with new

var myConstructor = function(arg1, arg2…) {
    // Here we have a new anonymous context
    // understandable as the instance…
}
// …so the context here will be myObj
var myObj = new myConstructor(arg1, arg2…);

Explicitly with Function.call

var myObject.method = function(arg1, arg2…) {
    // Implicit context is myObject
}
// Explicitly force context to anotherObject
myObject.method.call(anotherObject, arg1, arg2…);

Explicitly with Function.apply

var myObject.method = function(arg1, arg2…) {
    // Implicit context is myObject
}
// Explicitly force context to anotherObject
myObject.method.apply(anotherObject, [arg1, arg2…]);

In general, JavaScript developers tend to rely on implicit context passing rather than explicit. However, there is one situation where this reliance falls down…

A common problem

When registering event handlers against DOM nodes in JavaScript, the this context no longer references the method’s parent object, but rather the DOM node to which the event handler is registered.

The easiest way around this is to make sure the reference to the context object originally stored in this is maintained before defining the event handler:

var widget.init = function () {
    var myButton = document.getElementById(“my-button”),
        counter = 0,
        that = this;

    myButton.onClick = function(event) {
        that.counter++;
        this.innerHTML = “Clicked!”;
    }
}

Further to this, according to Douglas Crockford, there is an error in the ECMAScript spec. that causes this to be set incorrectly for inner functions; something that he discusses in his article Private Members in JavaScript:

function Container(param) {
    function dec() {
        if (secret > 0) {
            secret -= 1;
            return true;
        } else {
            return false;
        }
    }

    this.member = param;
    var secret = 3;
    var that = this;

    this.service = function () {
        if (dec()) {
            return that.member;
        } else {
            return null;
        }
    };
}

Now, the "var that = this" technique is certainly a bone of contention amongst members of the JavaScript community, but I’m firmly on the fence in the whole discussion. Certainly my second example could be solved with a complex network of Function.call and Function.apply instead of that.member, but my first example would be somewhat more difficult.

Personally, I feel if Doug Crockford—a developer with many years more experience and infinitely more JS knowledge than myself—says it’s ok, then I’m fine with it.

Garbage collection

As JavaScript is a scripting language, it implicitly allocates memory for objects, strings, variables, and so on as it runs. Garbage collection is the process by which the JavaScript engine detects when those pieces of memory are no longer reachable—that is, they could not possibly ever be used again—and reclaims the memory.

The ECMAScript specification (of which JavaScript is an implementation) doesn’t include a definition on how a JS engine should handle garbage collection, so although each JavaScript engine includes garbage collection—memory usage would quickly snowball if they didn’t—they all handle it somewhat differently.

Some engines are better than others; Google’s V8 is exceptionally good, where as the IE JScript engine is notoriously bad (often resulting in memory leaks).

Nested functions and closures

JavaScript allows the nesting of functions, creating nested scope blocks that inherit scope from their parents all the way up to the global scope.

Take this code for example (there are much better ways of doing this; this method is only in the interests of illustration):

function appendList(list, itemPrefix) {
    var getListItem = function(x) {
        return document.getElementById(itemPrefix + x);
    }

    for (var i = 0, j = list.length; i < j; i++) {
        getListItem(i).innerHtml += ‘ [done]’;
    }
}

In the above example getListItem is a nested function within appendList. This means it inherits everything within the scope of appendList. As a result of this, the inner function is able to make use of the itemPrefix parameter that is passed to its parent without it needing to be passed in as another parameter on getListItem.

When JS automatically garbage collects the appendList function, it will find no external references to the getListItem function and will garbage collect that as well.

This is a useful feature of JavaScript, and allows for fairly advanced lambda functions (anonymous functions), as well as providing a useful tool for namespacing library functions.

However, if, upon garbage collection, JavaScript finds an external reference to a nested function, it creates a closure of scope; thus maintaining access to all the variables required within the scope of that function.

This is known as a closure.

Common usage of closures

The most common occurrence of a closure in JavaScript is when declaring event callbacks. This is simply due to the fact that the callback function is referenced within the event listener once registered.

However, it’s important to understand that many modern libraries, plugin architectures, and plugins themselves also make use of closures, and understanding how they use them can mean the difference between controlled and uncontrolled memory usage.

Invisible pitfalls are invisible

It’s obvious when you think about it, but all closures have a potentially high memory imprint as they are maintaining much more than just their constituent parts. What’s more, because of their inherent avoidance of garbage collection, they are maintained until they are manually destroyed either by the code, or by a page refresh.

Prototypal inheritance

I did discuss prototypal inheritance in my original Object-Oriented Javascript article, however I just want to go into a little more detail here.

In a prototypal system, objects are supposed to inherit from objects; unlike in a classical inheritance system where classes inherit from classes and instantiate objects. JavaScript has no class (pun intended).

Prototypal inheritance is actually simpler than classical inheritance once you get your head around it. You don’t need to define classification, so your code is smaller and less redundant since objects inherit from other more general objects. It is a model of differential inheritance; i.e. each level of inheritance only adds the differences with its parent.

Unfortunately, JavaScript is a bit confused about its prototypal nature, and wants to fit in with the other scripting languages by pretending to like The Beatles and by making itself more attractive to classically trained programmers by introducing the new operator.

This means that:

new myConstructor();

produces an object that inherits from myConstructor.prototype.

This indirection means that JavaScript actually has a pretty ugly constructor pattern and most new JavaScript developers never really get to grips with the inheritance model at all.

However, all that has changed with the all singing, all dancing advent of JavaScript 1.8.5 (The New Shit™), which introduces the Object.create method:

var human = {
    legs: 2,
    arms: 2
};

var tim = Object.create(human, {
    tall: true,
    fat: true,
    overInflatedSenseOfSelfImportance: true
});

Mmm tasty, but since JS 1.8.5 only seems to be implemented in Firefox 4 (still in beta), don’t get all excited just yet.

If you want something similar in your own code, you can implement the following for a similar approach (and still kiss goodbye to ever having to use the new operator again):

if (typeof Object.inherits !== 'function') {
    Object.inherits = function(parent, child) {
        function temp() {};
        temp.prototype = parent.prototype;
        child.super_ = parent.prototype;
        child.prototype = new temp();
        child.prototype.constructor = child;
    };
}

Object.inherits(parentConstructor, childConstructor);

What’s more, this DIY version will allow you access to any superclass’ implementations of a particular method like so:

childConstructor.prototype.foo = function(a) {
    // Here’s some of that scope correction we discussed
    // earlier…
    childConstructor.super_.foo.call(this, a);

    // Other code
};

This code is a mix of the goog.inherits function—included in the Closure library—and the Object.create outline by Crockford in his article on prototypal inheritance.

JavaScript design patterns

Taking all this into account, we can start developing some incredibly useful design patterns that make use of prototypal inheritance, closures, and JavaScripts nuances of scope.

In fact, several have already been developed within the JavaScript community:

JavaScript namespacing

To avoid the perils of globally scoped variables and functions, it is advisable to create a single global object for the purposes of namespacing the rest of your code. This is relatively simple to achieve:

var NEF = window.NEF || {};

This command tests for the existence of a NEF object at the global scope, and if evaluated to true, returns that object. Otherwise, it creates a new object using the object literal syntax.

You may now assign any further variables or modules to your namespace:

NEF.myAttribute = ‘foo’;
NEF.myFunction = function() {
    …
};

Module pattern

The JavaScript module pattern was first proposed by Doug Crockford as a means of enforcing public and private object members through the use of closures.

My preferred variation on the pattern is such:

NEF.Module = function() {
    // Private members
    var privateAttribute;

    function privateMethod() {
        alert('Private method called.');
    }

    var pub = {
        // Public members
        publicAttribute: true,

        publicMethod: function() {
            // Private members are in scope here

            privateMethod();

            // Public members are addressable through the 'pub' object.

            pub.publicAttribute = false;
        }
    }

    return pub;
}();

This method simple uses a closure to maintain private variables in the scope of the returned pub object. The trick of this technique is the use of the (), right at the end of the code, after the parent function definition, which causes the function to run immediately and return the pub object.

jQuery plugins

The jQuery plugin architecture also makes use of closures to do something quite similar to the Module Pattern above. However, jQuery plugins are specifically namespaced to the jQuery object itself:

(function($) {
    // Private members
    var debugMode = false;

    function debug(msg) {
        if (!debugMode) { return; }
        if (window.console && window.console.log){
            window.console.log(msg);
        } else {
            alert(msg);
        }
    }

    $.fn.extend({
        myPlugIn: function(config) {
            var defaults = {
                …
            };

            if (config) {
                $.extend(defaults, config);
            }

            this.each(function() {

            });

            return this;
        },

        publicMethod: function() {
            …
        }
    });
})(jQuery);

Alternatively, should you wish to only include a single public method, you can avoid jQuery.fn.extend by instead making use of jQuery.fn for definition:

(function($) {
    // Private members
    var debugMode = false;

    function debug(msg) {
        if(debugMode && window.console && window.console.log){
            window.console.log(msg);
        } else {
            alert(msg);
        }
    }

    $.fn.myPlugIn = function(config) {
        var defaults = {
            …
        };

        if (config) {
            $.extend(defaults, config);
        }

        this.each(function() {

        });

        return this;
    };
})(jQuery);

In this code we can see that jQuery is passed to an anonymous function, that is run immediately, as the parameter $. The anonymous function provides a scope block for the plug-in code, thus preventing potential headaches from variable naming clashes across shared code. The plug-in functions themselves are then created as closures so that they have access to the private variables and functions within that outer anonymous function. These plug-in variables are then bound to the passed in jQuery instance $ using jQuery’s own built in extension functions.

This is actually quite a clever use of closures, although it does have the potential for memory leaks and excessive memory usage if variables and references aren’t kept in check. With that in mind, it’s often a very good idea to profile your jQuery plugins to understand what might need tidying up.

Custom events

Most modern JavaScript libraries include a mechanism for defining your own custom events. A custom event is merely a bespoke event, defined in your code, that other event-handler functions may be bound to. This custom event may then be triggered by you at any given point of execution in the code.

The custom event architecture is an implementation of the Observer Pattern, in which an object maintains a list of dependant “observers” which it notifies automatically of any state changes. In JavaScript, the object maintaining the list is our event, and the “observers” are the event handlers.

Using custom events in your own code provides a useful binding for other developers’ code. The event itself is loosely coupled to the event handlers, thus, additional event handlers can be added or removed by third-party code.

For more information on custom events, I’d recommend having a look at the documentation for the library of your choice. Here are few links:

• jQuery custom events: .bind()
• YUI2 custom events: The CustomEvent object
• YUI3 custom events

Summary

So that’s the technical addendum to my 2006 post. Hopefully, that should serve to bring the article up to date, and correct a few of my previous errors. As always, comments and corrections are welcome via the comment form below.

Originally I had intended posting this article in two parts, however, it has continued to grow even as I write it. For this reason, the third and final part of this follow-up series will look at what’s on the horizon for JavaScript development, including how GoogleBot deals with JavaScript, the importance of supporting a core experience (where JavaScript is unavailable), server-side JavaScript, and architecting JavaScript applications.

When I read that article now, however, it bothers me. The tutorial is still good although the information within is somewhat dated; and my personal experience, knowledge, and general approach to writing JavaScript has changed quite drastically. The fact that the article still receives a consistently high number of unique visitors on a daily basis, when I no longer agree with all the content within it, bothers me even more. With that in mind, I have decided to write an update that looks at OOJS as I would currently use it.

The resulting post has become quite extensive, so I’ve decided to split it into two separate articles. This first one will cover some development theory, whilst the second will dive into JavaScript coding specifics.

A note on the name

A friend of mine recently quoted this little gem in his Twitter stream:

Java is to JavaScript as Ham is to Hamster.

Enough said.

Is it object-oriented?

In my original article I stated that JavaScript was not an object-oriented language, but rather an object-based language. This was entirely unfair on JavaScript, and I’d like to fix that:

JavaScript is an object-oriented language. However, it breaks away from more conventional object-oriented languages by implementing prototypal inheritance, instead of classical inheritance. This is a Very Good Thing™ (although that seems to be a matter of opinion). More on that later.

First principles for JavaScript programming

Before I start, let me be clear that there are first principles of programming that are language agnostic. We’re going to look at a couple of those, but I’d also like to discuss a few of the more JavaScript specific principles first; those that relate to the cross-browser web standards environment. Finally, I’m aware that I’m really only scratching the surface of some of these topics. I don’t really want to muddy the water here by waxing lyrical on good development practices, so if you’re after more depth, I’d recommend researching them via the search engine of your choice:

Proper separation of the behavioural layer

Above everything else, it’s important to keep your content and markup (HTML), presentation (CSS), and behavioural (JavaScript) layers separate. Why is that important? Well if one or more of those layers are missing or broken (with the obvious exception of the core HTML), the others should still function as intended. Ultimately this makes presentation and behaviour optional, and a user might turn them off—via their browser settings—should they so desire. Alternatively, the user may include custom presentation or behaviour to aid a particular use-case.

In JavaScript, proper separation requires making use of the DOM to bind behaviour to HTML elements via element identifiers (i.e. the id or class attribute) in your source code. The alternative would be to write JS directly into the HTML, via event attributes such as onClick=”” or via the javascript: pseudo-protocol in the href attribute. This is not a good practice and I heartily recommend against it.

Graceful degradation

Graceful degradation is a well known software engineering concept which involves writing your code to take advantage of awesome feature X, unless awesome feature X is unavailable, in which case degrade to use slightly-less-awesome feature Y (assuming that too is available).

In web development terms this might mean making use of an amazing JavaScript date picker widget on a text-based input box, and just degrading to server-side date validation if JS is not turned on in the browser. It could also mean making use of the latest CSS3 styling techniques but degrading to use CSS2 techniques in less capable browsers.

In short; build everything in the most up to date browser, making use of all it’s wonderful modern features, and then go back and make sure the site is still usable when any of those features are not.

The problem with approaching the issue from this direction, however, is that you are saying to your users: “You (or your browser) are incapable of handling the full experience we want to present, so here’s a cut-down version.” This pretty much flies in the face of user-centric design, and good user experience. With that in mind, we’d be better using…

Progressive enhancement

Progressive enhancement involves approaching the problem from the other direction. You should design for the user-agent with the least capabilities and add functionality for more capable browsers afterwards.

In web terms this means developing and designing with nothing but the base features of HTML in mind, and then enhancing those features (or the features of the browser interpreting that HTML) by attaching CSS and JS to the HTML via element identifiers like IDs and classes.

The final outcome of these two approaches might be very similar, but the subtle difference is that you are giving more attention to the less capable user-agent when you progressively enhance.

The most common use-case for progressive enhancement is the support of a non-JS experience.

DRY (Don’t Repeat Yourself)

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system. This means abstracting code into shared functions and objects whenever you find you are repeating yourself, and attempting to maintain a single source of truth in every aspect of your system.

DRY is a well known paradigm in software engineering, and as such, should be applied in every detail of your JavaScript code.

Refactor early, refactor often

Further to the principle of DRY, it is important that if you spot a potential issue in your JavaScript code you fix it. The more small refactors you do, the less time-consuming big refactors you are likely to need to do later on.

Once you’re refactoring often, you’ll be able to…

Make it work, make it right, make it fast

The first iteration of your code should only look to reach the level of functionality defined in the functional specification. Once this has been achieved, refactor to improve the way it works—so the code is “right” in your opinion. This may very well mean a complete overhaul of architecture, but you will be able to concentrate much more specifically on that since you already have it working. The final stage in this development cycle is to optimise your code for performance and efficiency; premature optimisation is a fundamental mistake in development.

Useful resources

When it comes to general development, there are a veritable plethora of resources out there, but my recommended starting point is the book The Pragmatic Programmer, by Andrew Hunt and David Thomas. This book outlines quite a few paradigms that should get you thinking about the way you develop.

Important things to understand

Pure JavaScript

In this age of JavaScript libraries and frameworks, it’s important to make sure you understand core JavaScript functions, objects, and methods. Although your library of choice may clean up any cross-browser issues you may encounter, it is always important to understand exactly how you are interacting with JavaScript as a language.

Useful resources:

The Mozilla Developer Centre section on JavaScript is an invaluable reference resource when writing pure JavaScript. Also, make sure you own a copy of JavaScript: The Definitive Guide by David Flanagan.

The DOM

The Document Object Model is an entirely separate beast to JavaScript, but is implemented natively within it. There are also DOM libraries for other languages that implement the W3 DOM specification when dealing with XML and SGML files.

Understanding the DOM in JavaScript is essential when designing and developing interactions with your HTML and CSS. Thankfully all JavaScript libraries abstract out the pitfalls of DOM scripting so that you don’t have to, but as before, it’s still important for you to understand how so that you can maximise your efficiency in using said library.

Useful resources:

Once again the Mozilla Developer Centre provides an excellent DOM reference section, although it is slightly skewed in favour of the Gecko browser engine. You can also find plenty of information on the DOM at the W3C site.

Interaction with the browser

When writing client-side JavaScript, it is important to take into account how HTML, CSS, and JavaScript interact with the browser directly. At every available opportunity, you should attempt to make sure you are not contradicting the browser, and that you are not rewriting interactions that are already well handled by the browser and operating system.

A good example of this is to make sure you do not break the “back” button. Often this can be achieved by simply using the window.location.replace() method to control which pages appear in the browser history.

Libraries

Using libraries in JavaScript has become standard practice. A JavaScript library exists to save time and effort, and to provide a single source of truth (as discussed earlier) for many common tasks within that language.

A JavaScript library provides you a toolbox full of handy devices to make your life easier. It needs to be treated as such, and not as a language in its own right. It is very important to understand the core structures of JavaScript, and how the library of your choice interacts within those structures and your own code.

Every library is different and has its own pros and cons. I’m not going to go into my opinion of those here; perhaps that’s something for another post. They do, however, all interact somewhat differently with JavaScript, and it’s important to understand how the library you choose does that specifically.

Summary

Hopefully that should get you in the right frame of mind to develop high quality JavaScript code. In the second half of this article I will be looking at specific use of JavaScript in regard to object-orientation.

Continue reading part 2 of my object-oriented JavaScript follow-up…

Following on from my post about my jQuery Carousel, here’s another widget: an accordion.

Until recently, I’ve usually used my friend and ex-colleague Marco’s jQuery accordion plug-in. However, whilst prototyping some stuff for work, I noticed a rather uncomfortable animation jump which, after some investigation, seems to be a well documented issue with jQuery’s slideDown() method. Since I was prototyping at the time, I quickly knocked up my own very basic accordion and, over time, it grew into something more.

The current version of my accordion follows a recent refactor–there are still several features I’d like to add–but for now I’m happy to let it out into the wild.

Demo

As was the case with my carousel, I’d definitely recommend taking a look at my page of example accordion implementations. This should give you a reasonable idea of the capabilities of the script; and as I update the widget, I’ll update this page.

Source Code

All the source code for the accordion–and the example page above–is available in the following GitHub repository:

jQuery Accordion – GitHub repository

Implementation

HTML

Basic HTML for the accordion is as follows:

<ul class="accordion">
    <li>
        <h3>Handle 1</h3>
        <div class="panel">
            &hellip;
        </div>
    </li>
    <li>
        <h3>Handle 2</h3>
        <ul class="panel">
            &hellip;
        </ul>
    </li>
    <li>
        <h3>Handle 3</h3>
        <p class="panel">
            &hellip;
        </p>
    </li>
</ul>

For each panel in the accordion, you will require both a panel (the content of our accordion pane) AND a handle (an item that is clicked to open a panel). These can be anything you want as you can specify the jQuery selector used for each as part of your configuration. By default, the selector for handles is “h3”, and the selector for panels is “.panel”. Obviously that was fairly specific to my original implementation, so I’ll look to fix that in the near future.

Notice that, because we’re specifying our own selectors, we’re able to use flexible HTML for either the panel or the handle. However, it’s also worth remembering that the accordion script injects a link around the contents of whatever you specify as the handle, so that the handle becomes focusable and keyboard navigable. If you are likely to end up with invalid markup (i.e. the inline link wrapped around a block level element), you may get unpredictable behaviour cross-browser.

CSS

The included CSS is really only a reset style-sheet for the list elements within the accordion. You will probably already be achieving this with your own reset stylesheet, but you can add the following link to the head of your document:

<link rel="stylesheet" href="css/accordion.core.css" type="text/css" charset="utf-8">

JavaScript

To activate the accordion in your pages, you will require both jQuery and the accordion script itself to be included at the bottom of your page, just before the closing tag, but before any script that applies the accordion to a jQuery object.

A default implementation would look like this:

    …
    <script type="text/javascript" src="js/jquery-1.4.2.min.js" charset="utf-8"></script>
    <script type="text/javascript" src="js/jquery.accordion.2.0.js" charset="utf-8"></script>
    <script type="text/javascript">
        $(".accordion").accordion();
    </script>
</body>

As with my carousel, you can configure the accordion more specifically by passing a configuration object to the .accordion() method:

<script type="text/javascript">
    $(".accordion").accordion({
        "handle":           "h3",
        "panel":            ".panel",
        "speed":            200,
        "easing":           "swing",
        "accordion":        true,
        "toggle":           false,
        "activeClassPanel": "open",
        "activeClassLi":    "active",
        "lockedClass":      "locked"
    });
</script>

Note: The above options are all the defaults.

Parameters for the configuration object are as follows:

Recently I’ve had the need to build several jQuery widgets, because, no matter how many already exist in the wild, they never seem to have all the features I require. Reinventing the wheel? Well yes, but there’s no harm if the current wheel doesn’t fit the vehicle.

Further to that, building widgets with various JavaScript libraries is actually pretty good fun, and is definitely a good way of familiarising yourself with that particular library. Having recently moved from using YUI to the more widely used jQuery, these widgets have aided my education.

So without further ado, here’s the first: the carousel.

Demo

Take a look at the page of example carousels first, to see the various implementation options.

Source Code

You can find all the source code hosted on GitHub here, with tarballs and zips:

Modern Carousel — GitHub repository

Implementation

HTML

The core HTML of the carousel should look something like this:

<div class="carousel">
    <div class="clip">
        <ul class="horizontal">
            <li>…</li>
            <li>…</li>
            <li>…</li>
            <li>…</li>
        </ul>
    </div>
</div>

The carousel, clip, and horizontal classes are all used by both the core CSS and the JavaScript to build the carousel. As a result, they are required features.

The carousel div must be the direct parent of the clip div, which in turn must be the direct parent of the ul element. The current version of the carousel script does not support the use of an ol, but it’s one of the features on my todo list.

For a vertical carousel, you should substitute the horizontal class with vertical.

CSS

The only required CSS for the carousel, is the carousel.core.css file:

<link rel="stylesheet" href="css/carousel.core.css" type="text/css" charset="utf-8">

My examples also include a CSS skin file that applies some styling to the carousel controls. Feel free to reuse this, or poke at it to come up with your own styles:

<link rel="stylesheet" href="css/carousel.skin.css" type="text/css" charset="utf-8">

JavaScript

To add the JavaScript, you simply need to include the script—along with jQuery—somewhere on your page. It’s best to include scripts at the bottom of your pages, before the closing body tag, as some browsers can delay loading of the rest of the page whilst the script loads.

<script type="text/javascript" src="js/jquery-1.4.2.min.js" charset="utf-8"></script>
<script type="text/javascript" src="js/jquery.carousel.2.0.js" charset="utf-8"></script>

Once you’ve done that, you can simply apply the carousel() method to any jQuery object, like so, to get the default options:

<script type="text/javascript">
            $(".carousel").carousel();
</script>

Alternatively, you can configure the carousel by passing it a configuration object like so:

<script type="text/javascript">
    $(".carousel").carousel({
        "loop": true,
        "autoplay": true,
        "hovercontrols": true,
        "hoverpause": true
    });
</script>

Parameters for the configuration object are as follows:

For more on the transition hooks, it’s probably best to take a look at the code. When I have more time, I’ll look at updating the GitHub wiki with details myself.

When I’ve used VMWare Fusion in the past, my VMs have all been assigned IPv4 addresses in the range 172.16.0.0 through 172.31.255.255, via DHCP (the 20-bit block of RFC1918). This has always suited me fine, and has meant I’ve been able to assign host file mapping entries for those IP addresses with relative confidence since the addresses have never changed, even after reboots.

However, on the new Mac, I was surprised to find my VMWare VMs being assigned IPv4 addresses in the range 192.168.0.0 through 192.168.255.255 (the 16-bit block of RFC1918). As you can imagine, this could cause a certain degree of havoc if you’re LAN is also assigning DHCP IP addresses in that range. This is exactly what was happening in my case—and on both my work LAN and my home LAN—so VMWare was having to change the IPs each time I changed network.

To fix this issue, all I needed to do was tell VMWare to use a specific range of IP addresses when assigning DHCP. Unfortunately, VMWare doesn’t make this particularly easy (despite including a handy network configuration script). So here’s what you have to do…

A quick word on VMWare Fusion networking

Before I jump in with the “how to”, let me just take a moment to explain how networking works in VMWare Fusion:

VMWare sets up two network interfaces during installation, plus a bridge from Fusion to your host’s active network interface. The two network interfaces that are configured are “vmnet1”–the host-only network–and “vmnet8”–the NAT network interface. It is the latter of the two that we need to configure for control over VM DHCP.

Resetting the network configuration

Firstly, make sure you have shut down all your VMs (even the headless ones), and that the VMWare GUI isn’t running. We’ll also need to make sure, before we start our configuration, that we’re in a predictable state for all the configuration files used by VMWare. Handily, VMWare Fusion supplies a network interface configuration script that will allow us to do exactly that.

VMWare records a bunch of information–in regard to DHCP and the vmnets–in a file called “locations”. To begin with, we’ll need to delete that file so that it is regenerated later:

$ cd /Library/Application\ Support/VMWare\ Fusion/
$ rm -f locations

Following that, we run the handy configuration Perl script, which will automatically answer it’s own questions with the default answers:

$ sudo ./vmware-config-net.pl

Configuring a NAT network for vmnet8.

'ping' -q -t 10 192.168.119.1 > /dev/null status = 2
The subnet 192.168.119.0/255.255.255.0 appears to be unused.

The file /Library/Application Support/VMware Fusion/vmnet8/dhcpd.conf that this program was about to install already exists. Overwrite? [yes]

The file /var/db/vmware/vmnet-dhcpd-vmnet8.leases that this program was about to install already exists. Overwrite? [yes]

The file /var/db/vmware/vmnet-dhcpd-vmnet8.leases~ that this program was about to install already exists. Overwrite? [yes]

…

I haven’t included all the output here, but it continues on for a while.

Once you have a prompt again, it’s time to restart the network services with these update values (even though they’re not going to be final):

$ sudo ./boot.sh --restart

This will give you a “Shutting down VMware Fusion:” message, followed by a “Starting VMware Fusion:” message.

Once you get your prompt back, make sure the network interfaces have been set up correctly by running the interface configurator:

$ ifconfig

…

vmnet8:	flags=8863 mtu 1500
		inet 192.168.119.1 netmask 0xffffff00 broadcast 192.168.119.255
		ether 00:50:56:c0:00:08
vmnet1: flags=8863 mtu 1500
		inet 172.16.225.1 netmask 0xffffff00 broadcast 172.16.225.255
		ether 00:50:56:c0:00:01

…

You should see something like the above, even if the addresses are slightly different. As long as the vmnet8 and vmnet1 interfaces are showing, we’re up and running again.

Defining your IP range

Remember that “locations” file we junked earlier? That’s where we need to define our IP ranges so that the network configuration script can work its magic. Following our little clean-up, that file should now be nice and simple, and ready for our changes. To do that, you’ll need to edit with root permissions:

$ mate locations

I’m using TextMate, which will prompt me for my admin password when I save the file. If you’re using an editor like vi or vim, you’ll need to sudo:

$ sudo vim locations

The settings you need to define in the locations file are as follows:

VNET_8_HOSTONLY_HOSTADDR
VNET_8_HOSTONLY_NETMASK
VNET_1_HOSTONLY_HOSTADDR
VNET_1_HOSTONLY_NETMASK
VNET_1_HOSTONLY_SUBNET

I defined mine like so, with the third digit matching the vmnet to which it is assigned, just so I have a point of reference if I need it:

…
answer VNET_8_HOSTONLY_HOSTADDR 172.16.8.1
answer VNET_8_HOSTONLY_NETMASK 255.255.255.0
…
answer VNET_1_HOSTONLY_HOSTADDR 172.16.1.1
answer VNET_1_HOSTONLY_NETMASK 255.255.255.0
answer VNET_1_HOSTONLY_SUBNET 172.16.1.0
…

Once we’ve saved our changes, we simply need to reset the network configuration using the same Perl script as earlier:

$ sudo ./vmware-config-net.pl

Configuring a NAT network for vmnet8.

Configuring a host-only network for vmnet1.

The host-only network is currently configured to use the private subnet 172.16.1.0/255.255.255.0. Do you want to keep these settings? [yes]

Finally, restart VMWare Fusion daemons and reconfigure the network interfaces:

$ sudo ./boot.sh --restart

Once that is complete, VMWare is configured to use your defined range of IP addresses for both its vmnets. You can check this by running the interface configurator again:

$ ifconfig
…
vmnet8: flags=8863 mtu 1500
        inet 172.16.8.1 netmask 0xffffff00 broadcast 172.16.8.255
        ether 00:50:56:c0:00:08 
vmnet1: flags=8863 mtu 1500
        inet 172.16.1.1 netmask 0xffffff00 broadcast 172.16.1.255
        ether 00:50:56:c0:00:01

Summary

So now you can configure your vmnets in VMWare Fusion to use a predictable range of IP addresses. That, as they say, is that.