Four Kitchens blog

Co-presenting "960.gs theming overview" at DrupalCon Paris

Todd Ross Nienkerk — Wed, 24 Jun 2009 16:19:37 -0500

Jake Strawn of Drupal Dynamics and I are teaming up to propose a 960.gs theming overview session for DrupalCon Paris. (To be fair, he beat me to it and was gracious enough to add me as a co-presenter.)

Our session will cover the following basics of the 960 grid system:

What is 960.gs?
Using the NineSixty theme as your starting point / parent theme
Understanding the grid-x, push-x, pull-x classes
Why a grid-based system can help speed up theme development
How to break the 960-pixel limit

In the last month, I have presented sessions on 960.gs and the NineSixty theme at DrupalCamps in Copenhagen, Helsinki, and Stockholm. Most recently, I co-presented a session at Drupal Design Camp Boston with Nathan Smith, creator of the 960 grid system. You can download the slide decks on our “Presentations” page.
<!—break—>
Why should you care about 960.gs and grid-based theming? Easy: It will save you time, money, and frustration. Grid systems contain trim, robust, and reusable CSS that can handle virtually all positioning and cross-browser compatibility. Standardized design grids provide themers and developers a more solid jumping-off point when creating a new layout or design.

Interested? Vote for our session!

Improvements to the Materialized View API

David Strauss — Sun, 21 Jun 2009 03:41:11 -0500

<!—break—>
An eye-catching graphic, largely irrelevant to this blog post.

Introduction

The Materialized View API (related posts) provides resources for pre-aggregation and indexing of data for use in complex queries. It does this by managing denormalized tables based on data living elsewhere in the database (and possibly elsewhere). As such, materialized views (MVs) must be populated and updated using large amounts of data. As users change data on the site, MVs must be intelligently updated to avoid complete (read: very slow) rebuilds. Part of performing these intelligent updates is calculating how user changes to data affect MVs in use. Until now, these updates had limitations in scalability and capability.

Update and deletion propagation

In the first iteration of the Materialized View API (MV API), which is currently deployed to Drupal.org, update and deletion propagation were rather naïve: the hooks used to trap changes (hook_nodeapi and hook_comment) simply called the updater for both the entity itself and the updater for anything related. For example, hook_comment() called both the updaters for the comment itself and the node parent of the comment:

function materialized_view_comment($a1, $op) {
  $comment = (array) $a1;
  switch ($op) {
    case 'insert':
    case 'update':
    case 'publish':
    case 'unpublish':
    case 'delete':
      MVJobQueue::update('comment', $comment['cid']);
      // Also "update" the node for the comment.
      MVJobQueue::update('node', $comment['nid']);
  }
}

Calling updaters for related entities is important for aggregation-based data sources, like one that, for a given node, determines the later of when the node was changed and when the latest comment to the node was posted. A change to either the node or a comment related to the node may change the aggregated value:

class MVLastNodeActivityTimestamp extends MVColumn {
  public function getValue($entity_type, $entity_id) {
    $timestamp = db_result(db_query('SELECT MAX(c.timestamp) FROM {comments} c
      WHERE c.nid = %d', $entity_id));
    if (!$timestamp) {
      $timestamp = db_result(db_query('SELECT n.changed FROM {node} n
        WHERE n.nid = %d', $entity_id));
    }
    return $timestamp;
  }
  [...]
}

The design of building propagation into the change-capture hooks proved sufficient for the initial MV API uses, which were forum-centric. The design was sufficient because update propagation was highly predictable: nodes to themselves and comments to themselves and their parent nodes.

But a limitation quickly became apparent: this would not scale to more entity-entity relationships and introducing more MV-supported entity types.

Here’s why:

Update notifications quickly became noisy: MVs based on purely node data would be updated whenever comments for the node changed, even if the node-based MV didn’t rely on comment data.
Mapping change propagation created misplaced burdens. It’s impossible for a change-capture hook to predict all the possible relationships MV data sources might introduce. For example, if we wanted an MV based on the number of replies to a comment, we would have to trigger updates for every parent comment walking up the tree. Do we update hook_comment yet again?

The solution was to put the change-propagation burden on the data sources, with the default change-propagation algorithm being “changes to X require updating rows related to X in the MVs.”

The default covers the standard entity attribute (e.g. published status for a node) data sources while allowing aggregated sources to become much smarter.

The default change mapper in the MVColumn abstract class:

abstract class MVColumn {
  [...]
  public function getChangeMapping($entity_type, $entity_id) {
    $changed = array();
    $changed[$entity_type] = array($entity_id);
    return $changed;
  }
  [...]
}

But for data sources like MVLastNodeActivityTimestamp — which provides a data sources for nodes which is the later of the last comment posting and the node change timestamp — has more complex change-propagation logic. (This code admittedly assumes that comments will post-date the last node changes.)

MVLastNodeActivityTimestamp’s change propagation logic:

class MVLastNodeActivityTimestamp extends MVColumn {
  [...]
  public function getChangeMapping($entity_type, $entity_id) {
    $changed = array();
    if ($entity_type == 'node') {
      // A change to a node only affects its own value.
      $changed['node'] = array($entity_id);
    }
    else if ($entity_type == 'comment') {
      $comment = MVEntityCache::get('comment', $entity_id, '_comment_load');
      
      // A change to a comment affects the value of the node it's attached to.
      $changed['node'] = array($comment['nid']);
    }
    
    return $changed;
  }
  [...]
}

getChangeMapping() effectively says:

This data source changes whenever a node changes or a comment changes.
A node change affects the value of this data source for that node.
A comment change affect the value of this data source for the parent node.

Now when an entity changes, the Materialized View API walks through data sources in use on any MV and establishes the unique set of entities needing updating. If a node-based MV doesn’t use any data based on comments, comment changes won’t trigger any changes in that MV. (See the new update() method for class MaterializedView.)

But this caused a problem (already solved in the code above): while hook_comment() gets passed any comments being deleted, it’s not possible for a data source to later load the comments and look up the related nodes to calculate propagation. The solution for this also became a useful overall optimization, the entity cache.

The entity cache

The disconnection between change-capture hooks and data sources used to result in excessive object loading. For example, changing the timestamp on a comment would pass the comment to hook_comment(), but a data source relying on the timestamp for the comment would load the comment fresh from the DB while updating MVs at the end of the page request (when MV updates currently occur).

Now, change-capture hooks populate the entity cache, allowing most data sources to use statically cached entity data. The entity cache also transparently loads entities in the background, keeping the data source code clean.

Of course, the entity cache was originally created to solve the change propagation for deleted items problem. It solves that problem by caching deleted items in the change-capture hooks. MV data sources are then able to load basic data for deleted items despite running after the items disappear from the database.

Challenges ahead

Change propagation can be expensive: for modifying a taxonomy term, it’s O(n), where n is the number of nodes with the term. Eventually, change propagation will have to be batched and handled offline, which raises the next issue.

It’s now more complex to queue MV updates to happen offline (read: during cron). The data necessary to calculate propagations lives in a static cache that disappears at the end of each page request. The only truly scalable option now is to have a persistent entity cache. That way, change propagation can happen offline, especially for large sets.

Some sets are so large that the most reasonable option may be to trigger reindexing for affected MVs. Changing a taxonomy term will fall under this category until change propagation can be batched.

The end scalability goal is to have the real-time overhead for running MVs be very small and linearly proportional to the number of entity changes being requested by the user, but the challenges above will need implemented solutions to reach this goal.

Opportunities

This new architecture opens the door for an explosion of MV data sources and supported entity types. Particularly, MV should expose every CCK field (or Field API field in D7) as an MV data source.

The long-awaited automatic “Views to MV” conversion system work is now underway. It will be possible to automatically convert many common Views to run on MV-based data sets, dramatically improving scalability for converted Views without requiring any external system (like Solr, Lucene, or CouchDB).

Bazaar 1.16 RPMs for RHEL 5 and CentOS 5

David Strauss — Thu, 18 Jun 2009 16:39:26 -0500

Fresh packages, 64-bit only. There are release notes for the curious.

<!—break—>

OpenSolaris 2009.06 first impressions

David Strauss — Sun, 14 Jun 2009 19:07:53 -0500

I downloaded and installed OpenSolaris to experiment with its LAMP stack implementation. I eventually want to set up a headless, SSH-accessible box I can use to experiment with unique OpenSolaris features like dtrace on PHP and MySQL.

I have no experience using Solaris or OpenSolaris. Nearly all of my *nix experience is with BSD and GNU/Linux. I understand that anyone even basically familiar with OpenSolaris would not have some of the problems I’ve had.

Downloading the CD

It’s a little too easy to start downloading an OpenSolaris ISO. Many “download” links go right to the x86 ISO. When downloading an operating system install CD, a landing page (like Ubuntu’s) is helpful to get new users oriented; all “download” links on the site should link there.

The landing page is also helpful because people downloading CD images don’t always want to download directly to the local machine, so a basic “click here to actually download the ISO” is helpful.

I do later find the download landing page, but I’m initially confused what to download to install OpenSolaris. It shouldn’t merely be in the descriptive text on the download landing page that the live CD is also the installation CD. “Live CD” does not imply “installation CD.”

First boot

The live CD boot goes quite smoothly, though GRUB asks users to pick among some confusing options. My questions: “Do I pick the ‘SSH-enabled’ option? What does SSH have to do with my boot choice?”

It’s also unnerving to see a terminal login prompt while the GUI login is loading. It gives the impression that X has failed to load. There’s also no indication that a user should wait for the graphical login to load, other than some screen blanks.

Eventually, though, the GDM login displays.

First login

Ubuntu’s live CD automatically logs in a basic user on boot. OpenSolaris gives users a standard login prompt. That wouldn’t be so much of a problem if the download landing page mentioned the username and password. But it doesn’t, at least for the latest release. Users actually wanting to use their live CDs either have to go off-site or view the landing page for a previous edition.

So, the username and password are both “jack.” I’ll file that alongside Oracle’s default included schema for arbitrary naming after someone who probably hasn’t worked on the project for years.

Installation

Starting the installation is straightforward: there’s a nice icon on the desktop. Running the installer is easy and successful.

But like most people trying OpenSolaris, I’m installing it for use as a server, not a desktop. For one, I want the time zone configured as UTC, not a local one. There seems to be no option to do this at installation time. I also don’t want to install a GUI, but “no GUI” is not an option in the installer. Nor is there an obvious “server” installation CD. I guess making it a real server will have to happen post-install.

While running the installation, I try to open the “getting started” link on the desktop, which launches Firefox. Well, at least it tries to. It actually fails, but I hold off investigating in case it starts working after completing the installation.

First login to the installed instance

Completing the first boot and first login is easy. I used the username and password I had set during installation. A GNOME desktop very similar to the one from the live CD loads effortlessly.

There seem to be two network-focused icons in the tray near the clock, and it’s not clear what the difference is. Upon further investigation, one is Network Manager and the other is a simple status icon for the ethernet adapter. Really, only one is necessary, especially given the visual confusion of having both and redundancy in what they do.

Firefox still doesn’t start. It gets a “704 Illegal Instruction” error when I try from the shell. Considering that Thunderbird and the rest of GNOME run fine, this isn’t a problem with the hardware. I guess I’ll withhold judgment until at least updating.

Given the problems with Firefox, the first thing I seriously try is updating the OS. The package manager GUI, which is sort of like Ubuntu’s, is linked from approximately three places (desktop, toolbar, and application menu). It starts quickly and provides a mostly sane interface.

Time to search for packages. Typing in the search box doesn’t immediately do anything, so I eventually press ENTER, which starts the search. There isn’t an obvious visual indicator of the search being in progress, so it’s not clear when all the results have been displayed. Clicking the “clear search box” (broom) button clears the search box and takes the user out of search mode. Other GNOME applications typically have live search in combination with the broom button, which usually makes this behavior less confusing.

The update buttons on the toolbar seem disabled, so I click “refresh” and use the standard menus to initiate the update process. It’s not very clear whether anything actually needs to be updated. The updater takes me through creating a boot system snapshot. This feature is very cool, and I imagine it’s implemented using ZFS under the hood. The installer offers to show me consolidated release notes for the update, which of course uses Firefox. I shrug and reboot.

First login after update

The post-update boot happens smoothly all the way to GNOME, but Firefox still doesn’t start. Same error.

Oh well, this wasn’t going to be used as a desktop, anyway.

I try to set the static IP address for the testing subnet of the Four Kitchens office network. After clicking on the non-Network Manager tray icon and setting the IP, OpenSolaris solves the redundant tray icon problem by having both widgets crash. I try to open the GNOME network connections configuration tool to at least see if the static IP took effect, but that spends about 30 seconds loading the interfaces before also crashing.

Granted all of these problems are GUI-related and shouldn’t affect my final work, but setting the IP and loading Firefox are both pretty basic.

Next adventures

Install Bazaar on Solaris (not expected to be difficult, but necessary for my application deployments)
Set up remote SSH logins
Remove the GUI
Set up the Sun’s packaged LAMP stack
Explore dtrace on PHP and MySQL

Advanced Drupal form theming: Take control of error styling with a form-item-error class

Todd Ross Nienkerk — Wed, 10 Jun 2009 13:11:52 -0500

Note: This HOWTO covers Drupal 6.x.

By default, Drupal adds an .error class to the form element itself: textarea, select, input, and so on. Sometimes, that’s not good enough. Maybe a client needs the label’s color changes — or a big, red border encompassing both the label and input elements.

This can be achieved by overriding theme_form_element() to add an error class to .form-item, the div that wraps all elements with a form component.
<!—break—>
Add these lines to the top of your theme_form_element() override:

<?php
function MYTHEME_form_element($element, $value) {
  // This is also used in the installer, pre-database setup.
  $t = get_t();
  // Start compiling classes for the .form-item wrapper
  $classes[] = 'form-item';
  // Add an error class to the .form-item wrapper
  $exempt_elements = array('checkbox', 'radio', 'password_confirm');
  if (form_get_error($element) && !in_array($element['#type'], $exempt_elements)) {
    $classes[] = 'form-item-error';
    $classes[] = 'form-item-error-'. $element['#type']; // Optional
  }
  // Build the .form-item wrapper
  $output = '<div class="'. implode(' ', $classes) .'"';
  if (!empty($element['#id'])) {
    $output .= ' id="'. $element['#id'] .'-wrapper"';
  }
  $output .= ">\n";
?>

The lines above should replace the following:

<?php
function theme_form_element($element, $value) {
  // This is also used in the installer, pre-database setup.
  $t = get_t();
  $output = '<div class="form-item"';
  if (!empty($element['#id'])) {
    $output .= ' id="'. $element['#id'] .'-wrapper"';
  }
  $output .= ">\n";
?>

All form elements except those listed in the $exempt_elements array will have two classes applied to div.form-item: .form-item-error and .form-item-error-ELEMENT_TYPE. Feel free to changes these as you like.

Why are some elements exempt? Checkboxes and radio button usually come in groups of two or more wrapped inside a single div.form-item, and each individual box or button is wrapped in yet another div.form-item. This nesting can make theming difficult, so I’ve exempted them here. (You can, of course, customize this override to fit your needs.)

Here’s what the markup looked like before adding wrapping error classes. Note that the .error class is applied to the form elements themselves only, which makes theming some elements like radio buttons impossible in some browsers:

<div class="form-item" id="edit-first-name-wrapper">
  <label for="edit-first-name">First name: <span class="form-required" title="This field is required.">*</span></label>
  <input maxlength="255" name="first_name" id="edit-first-name" size="60" value="" class="form-text required error" type="text" />
</div>
<div class="form-item" id="edit-last-name-wrapper">
  <label for="edit-last-name">Last name: <span class="form-required" title="This field is required.">*</span></label>
  <input maxlength="255" name="last_name" id="edit-last-name" size="60" value="" class="form-text required error" type="text" />
</div>
<div class="form-item">
  <label>Gender: <span class="form-required" title="This field is required.">*</span></label>
  <div class="form-radios"><div class="form-item" id="edit-gender-m-wrapper">
  <label class="option" for="edit-gender-m"><input id="edit-gender-m" name="gender" value="m" class="form-radio error" type="radio" /> Male</label>
  </div>
  <div class="form-item" id="edit-gender-f-wrapper">
  <label class="option" for="edit-gender-f"><input id="edit-gender-f" name="gender" value="f" class="form-radio error" type="radio" /> Female</label>
  </div>
</div>

And here’s the same markup after applying the override above:

<div class="form-item form-item-error form-item-error-textfield" id="edit-first-name-wrapper">
  <label for="edit-first-name"><span class="form-required" title="This field is required.">*</span> First name:</label>
  <input maxlength="255" name="first_name" id="edit-first-name" size="60" value="" class="form-text required error" type="text" />
</div>
<div class="form-item form-item-error form-item-error-textfield" id="edit-last-name-wrapper">
  <label for="edit-last-name"><span class="form-required" title="This field is required.">*</span> Last name:</label>
  <input maxlength="255" name="family_name" id="edit-last-name" size="60" value="" class="form-text required error" type="text" />
</div>
<div class="form-item form-item-error form-item-error-radios">
  <label><span class="form-required" title="This field is required.">*</span> Gender:</label>
  <div class="form-radios"><div class="form-item" id="edit-gender-m-wrapper">
  <label class="option" for="edit-gender-m"><input id="edit-gender-m" name="gender" value="m" class="form-radio error" type="radio" /> Male</label>
</div>
<div class="form-item" id="edit-gender-f-wrapper">
  <label class="option" for="edit-gender-f"><input id="edit-gender-f" name="gender" value="f" class="form-radio error" type="radio" /> Female</label>
</div>

The wrapping .form-item divs now have error classes: form-item form-item-error form-item-error-textfield.

The technique described above was inspired by the function _form_set_class, which is responsible for adding .required and .error classes to form elements.

Four Kitchens' website featured on 960.gs

Todd Ross Nienkerk — Wed, 03 Jun 2009 14:02:21 -0500

Our website has been featured on 960.gs, home of the 960 grid system! This is quite an honor, as we’re big fans of grid-based design — especially 960.gs — and have begun implementing its principles and techniques in virtually every project.

<!—break—>
12-column grid overlay

To see grid-based design in action, go to 960.gs and click the “show grid” button above the screenshot.

Note that every region on the page is contained neatly within the overlaid columns. The understated simplicity of the layout and the remarkably trim CSS used to achieve it are at the core of grid-based design.

Last month, I presented sessions at DrupalCamps in Copenhagen, Helsinki, and Stockholm on Drupal theming using 960.gs. You can download the slide deck on our presentations page.

In a few weeks, 960.gs creator Nathan Smith and I will co-present a session on accelerated grid theming using NineSixty, the Drupal port of 960.gs, at Drupal Design Camp Boston.

Thanks to Nathan Smith for featuring us. I’d also like to thank Joon Park, whose NineSixty theme has made major strides towards improving the functionality of grid-based design. (The Four Kitchens theme is a subtheme of NineSixty.)

David's Epic Presentation Megapost

David Strauss — Tue, 02 Jun 2009 19:40:29 -0500

UPDATE 2009-06-08: You can download these slide decks (and more!) from our Presentations page.
—The Web Chefs

DrupalCamp Stockholm 2009

Drupalcon DC 2009

Four Kitchens and GeekAustin present an evening of Drinks and Drupal

Aaron Stanush — Tue, 19 May 2009 15:35:25 -0500

We’re co-hosting a party and you’re invited!

Photo by Antonio Zugaldia on Flickr (CC-Attribution)

On Wednesday, May 20th at 6:30pm, Four Kitchens is teaming up with GeekAustin to spread Drupal love in Austin. This free event is a chance for local Drupal professionals to share their passion with the curious and uninitiated masses of our fair city.
<!—break—>
From Lynn Bender at GeekAustin:

We’ll have drinks out front and presentations in back. We’ve been sending personal invites to Drupalistas throughout Texas. So, if you have any unanswered Drupal questions, this will be the place to find the answers.

Master web chef David Strauss will be on hand to give a presentation on “Quick and maintainable site-building for charities and non-profits using Drupal and CiviCRM.”

So if you’re interested in learning how the Drupal CMS can benefit your project, come and meet the Austin and Texas folks that make it happen every day. And remember, just like Drupal, this event is FREE

Drinks and Drupal
Co-hosted by Four Kitchens and GeekAustin
Date: Wednesday, May 20, 2009
Time: 6:30pm - 10:30pm
Venue: Union Park Austin
Street: 612 W Sixth St. Austin, Texas 78701
RSVP on Facebook

Update: The slide deck is attached below.

Downloads

Zero to CiviCRM.pdf

Zero to CiviCRM.pdf 697.92 KB

Bazaar 1.14 RPMs for RHEL 5 and CentOS 5

David Strauss — Wed, 29 Apr 2009 04:41:56 -0500

Fresh packages, 64-bit only (again). I can make 32-bit ones if anyone would like some. I’d say that this is the most exciting release yet.

<!—break—>

Alternatives to rebasing in Bazaar

David Strauss — Mon, 20 Apr 2009 05:02:38 -0500

A discussion recently arose on the Bazaar mailing list asking, “Why isn’t rebase support in core?” Rebase support is currently packaged as a plugin. This plugin is widely distributed, even in the standard Mac OS X installation bundle.

There are boring reasons that rebase support isn’t in core, like the lack of strong test coverage. More interesting are questions about the necessity of rebasing in typical workflows.

What is rebasing, and why should I care?

In large projects, there’s a mainline branch representing the current, global, coordinated development. In Drupal’s case, this is CVS HEAD. This mainline might not always be in perfect condition, but there’s a general sense that the mainline is not a sandbox for untested changes. Many changes are small enough that the developers simply work on and test a patch, but this workflow is inadequate for larger development projects like Fields in Core. Such large features require their own branch for development, a feature branch.
<!—break—>
A feature branch allows development of a feature in isolation from the mainline but with the eventual intent of merging the changes back into the mainline. Because feature branches are created to foster long-term, divergent development from the mainline, it’s common for both feature development and mainline development to happen in parallel. This parallel development creates a problem: How do developers on the feature branch prepare for the eventual re-integration of their feature code into the mainline?

There are a few options:

Don’t sync changes. This option makes merging the feature back into the mainline painful. This option also defeats the purpose of developing and testing the feature in isolation because merging two tested (but divergent) branches often results in one broken (but converged) branch.
Merge the feature into the mainline before making any changes to the mainline and then re-branch for more feature work after making mainline changes. Merging an untested or incomplete feature into the mainline makes this option unattractive and impractical. This option is so silly, I only included it for completeness.
Periodically update the feature branch from the mainline. This is ideal because the feature branch continually answers the question “What if we merged this feature into the mainline?” and is ready for quick merging into the mainline without any disruption to mainline work.

The third option is the only practical one. But how should it work? What should the feature branch history look like after synching from the mainline?

Back to rebasing…

Rebasing integrates the updates to the mainline as ancestors to the changes on the feature branch. The commit history is reorganized (read: rebased) as if the feature branch were freshly created from the mainline and all work were done on top of that. There are many theoretical objections to rebasing, and I won’t rehash them here. There’s general consensus that rebasing is sort of icky.

I find that many rebase users use the tool because they’re not aware of better workflows. I’ll address each (supposed) reason to use rebase in its own section.

“I want to keep my feature branch updated from the mainline.”

The better choice is to run bzr merge [mainline] on the feature branch. This command will update the common ancestry between the feature and mainline branches so that the feature branch includes the latest changes from the mainline and is ready for smooth merging back into the mainline.

“I want to view only the revisions that make up the feature I’ve been working on.”

With a rebase, it’s reasonably clear which revisions constitute the feature work: they’re the top ones. But rebasing is not the best choice for reviewing this list. Run bzr missing --mine-only [mainline] from the feature branch, and Bazaar will output all the feature branch’s unique revisions without mangling the actual history (the way rebasing does).

“I want a human-readable summary of how merging the feature into the mainline will affect the code.”

For background, a rebase user would run a diff from the oldest feature-specific commit to the latest commit, but there’s a better way. Instead, run bzr diff --old=[mainline], and Bazaar will provide the net diff for merging the feature into the mainline. Now, don’t use this diff for anything but human review; you should still use bzr merge from the mainline to integrate the feature branch’s changes and preserve all history.

Creating a merge directive with bzr send provides an identical human-readable diff to the method above, but a merge directive also includes all the binary data Bazaar needs to perform a history-preserving merge.

“I want to maintain a patch set on top of the mainline.”

Rebasing commits is an ugly way to do this because you don’t retain your own history of work on each patch or the history of how rebasing has changed each patch. Bazaar has a plug-in called “Looms” that provides direct support for a much better patch set workflow. I’m a touch skeptical of Looms’ stability, so I just do what Looms does under the hood: maintain multiple branches, each derived (branched) from the one below. Each branch represents a patch. This method retains full, original history, including any changes I’ve made to the patches. When the mainline updates, I simply merge the mainline changes up through my patches.

“I want to clean up my commit history prior to submitting my changes to the mainline.”

Rebasing may group the feature commits, but it doesn’t make them coherent or pretty. It’s more effective to do the following:

bzr merge [mainline]
Use bzr diff --old=[mainline] on the feature branch to create a net diff.
Get a fresh branch from the mainline.
Apply the net diff as a patch.
Shelve all changes.
Work through unshelving the changes and committing them to create a coherent, pretty history.
Create a merge directive using bzr send.
Submit the merge directive.

“[Your reason here]”

I’d like to hear from users of any distributed version-control system why they use “rebase” in their workflows, even if their reason is one I’ve discussed above.

Drupal.org redesign sprint San Francisco: Day 4

Todd Ross Nienkerk — Sat, 11 Apr 2009 20:17:19 -0500

Photo by Franco Folini on Flickr (CC-Attribution-ShareAlike)

Despite being held on a Saturday, more than 15 dedicated Drupalers showed up for Day 4 of the San Francisco Drupal.org redesign sprint. Here’s what was achieved.
<!—break—>
Josh Koenig:

Made a “token” contribution. (Get it? Tokens.)
Developed a UI for setting up activity logging (the following/tracking utility see on the Dashboard).
Worked on a cross-site delivery system across the *.drupal.org infrastructure.

David Strauss:

Built the sever-side components for activity reporting and logging.
Built a microformat-based facet system for activity reporting.
Finished the single sign-on system for all *.drupal.org properties.

Chris Bryant:

Developed content and site architecture maps comparing the Drupal.org structure to what we think Mark Boulton is suggesting in his prototypes.
Reviewed the list of Dashboard widget and where they will appear throughout the site.
Mapped features needed to implement project-related functionality.

Mark Burdett:

Worked on the project browsing form.
Developed some tools for tracking and displaying CVS statistics.

Derek Wright:

Delegated tasks to other developers.
Reviewed several patches.
Fixed some project UI problems.

Neil Drumm:

Deleted more code than he added. (Nice.)
Worked on the “Add to Dashboard” links for widgets.
General UI/UX work for adding widgets to the Dashboard.
Figure out how cross-site widgets will work across *.drupal.org.
Ordered pizza. (Nice.)

Károly Négyesi (chx):

Worked on a new parser for the API module. His work will eventually allow commenting on api.drupal.org.

Dmitri Gaskin:

Build the jQuery-powered map on the front page.
Started work on remote Dashboard widgets.

Jerad Bitner:

Worked on the jQuery side of the Dashboard.
Recruited Nate Haug to help.

Courtney Miller:

Changed drop-down search form CSS to properly style the checkboxes after they were changed from radio buttons. She also made room for the “Popular Searches” feature in that form.
Worked on the user picture feature.
Fixed formatting and grid layout of user profiles.
Created logic to place section titles as non-header text while styling them to appear like <h1> elements.
Documented her side of the comment titles debate. (We DO NOT AGREE.)
Added styling of book navigation menus.

Erik Hopp:

Fixed $links styling for nodes.
Consolidated CSS.
Fixed the Views administration interface.
Added node type classes to node output.
Fixed styling of the “Develop with Drupal” block on the front page.
Researching styling of the Project module. Later, he will propose a new, more maintainable markup style.
Improved accessibility of navigation tabs.
Improved skip links.

Todd Nienkerk (that’s me!):

Created graphical elements for the world map on the front page.
Mapped all colors from the prototypes into a master color palette.
Created color palette for status, error, and help messages.
Added comments support to the Permalink module. The module now adds an anchored permalink for each comment.
Created comps for documentation pages. I still need to style classes for good versus bad examples of code, style, language, etc.

Keiran Lal: Photo by lolg42 on Flickr

Thanks to the omniscient Kieran Lal for posting these earlier San Francisco sprint updates:

Day 1
Day 2

Kieran enjoys fine food, wine, and dispensing knowledge of San Francisco’s microclimates and underground economies. In his spare time, he works tirelessly to build momentum for the Drupal.org redesign.

Speaking of the redesign…

You can help!

We need you to help us launch the new Drupal.org. Here’s what you can do:

Join the Drupal.org redesign implementers group.
Read the TODO list to get an idea of what we need done.
Visit the issue queues to find specific tasks. Assign them to yourself and fix ‘em!
- All redesign issues (tagged drupal.org redesign).
- Bluecheese theme (tagged Bluecheese theme). “Bluecheese” is the name of the new design. Morten.dk is responsible.
Most importantly, don’t wait for an invitation! Dive right in. If you have any questions, we’re in #drupal-infrastructure on irc.freenode.net. (Here’s how to use IRC.)

Photo by wili_hybrid on Flickr (CC-Attribution)

Drupal's vulnerability reports are not signs of security weakness

David Strauss — Thu, 02 Apr 2009 22:47:44 -0500

Photo by loop_oh on Flickr.

I’ve been tweeting back and forth with Alex Limi, one of the founders of Plone, about the validity of the security analysis from a CMS comparison report that includes Plone and Drupal. He’s proud of Plone’s infrequent vulnerability notices; it had two in the last year. Drupal had 26. Alex also cited a related IBM report on security in a later tweet.

While both reports above seem to identify Drupal (and Joomla! and WordPress, to be fair) as having notably bad security, they’re also both based on one superficial metric: self-reported vulnerabilities. Neither severity nor response time nor history of actual exploitation factored in.
<!—break—>
The vulnerabilities in question have all (long) been fixed in Drupal, so Alex’s argument could only be that past occurrences of vulnerability reports are a predictor of future security problems. Unfortunately, he merely begs the question of correlation without answering it, and that’s only the beginning of the problems with his argument.

Even if vulnerability reports were perfect indicators of future risk, vulnerability self-reporting carries a high conflict-of-interest. This conflict is especially strong when, like Alex, you argue that the quantity of reports you issue should be held against your project.

The Drupal community (both in developers and users) is much larger than the Plone one, and the two continue to diverge:

Drupal is red, and Plone is blue (to state the obvious).

Many of us in the free software community are familiar with Linus’s Law: “given enough eyeballs, all bugs are shallow.” Vulnerabilities are merely a special class of bugs. All other things being equal, Drupal’s larger developer and user base should be expected to find and publish more vulnerability reports than Plone’s.

But Drupal had more than just community growth in 2008; it also experienced unprecedented security review thanks to work by Barry Jaspan, who presented his findings at Drupalcon Szeged 2008. Barry subjected Drupal’s core code to static and dynamic analysis, resulting in the discovery of several vulnerabilities. Has Plone undergone similar scrutiny? A quick search on Google didn’t uncover anything of the sort.

Despite Alex’s thoughts on my stubbornness, I am open to an honest evaluation of Drupal’s security versus similar tools. I’m just not willing to base the debate on a superficial metric of such questionable importance.

Check out Four Kitchens' hot new logos!

Todd Ross Nienkerk — Wed, 01 Apr 2009 17:17:24 -0500

After many months of deliberation, we’ve decided to totally rebrand Four Kitchens. It was a tough decision — there’s so much work that needs to be done — but we decided, in the end, that our firm needed a new look.

Our goals for the rebranding are:

Identify Four Kitchens as a leading Drupal consulting firm.
Raise awareness of our design skills and portfolio. (We’re not just scalability experts!)
Create an iconic brand that associates the Four Kitchens with quality, respect, and community involvement.

Please check out our ideas below. Any feedback is welcome. We really need your help!

Four Kitchens logo: version 1

Inspiration: Building a website is like reading a book. First, you ride your fixed-gear bike to a locally owned, vegan bookstore and pick out something about World Trade Organization-sponsored coups. Then you turn the book over to see how much it costs. Finally, you pedal home — uphill — and buy it on Amazon in a hot minute.
<!—break—>

Four Kitchens logo: version 2

Inspiration: Building a website is like growing a plant. If you leave it in the sun too long, it will wither up and die. Then you have a dried up, dead plant, dummy. What a waste of five bucks. Oh, and you have to make a geospatial/SMS mashup that visualizes demographic data based on your proximity to a mailbox, squirrel population, volcanic activity, and roadside historical markers.

Four Kitchens logo: version 3

Inspiration: Building a website is a lot like owning a giant, red robot. Everybody thinks it’s cute, but you’re the one that has to follow it around with plastic bags on your hands, cleaning up its piles of instructional DVDs.

UPDATE: Version 4 submitted by a loyal reader

We’re not sure who made this, but we like it! They deserve a ping, whoever they are…

Project Manager Shannon Hinshaw's "SCRUM-ptious" birthday cake

Todd Ross Nienkerk — Tue, 31 Mar 2009 13:50:15 -0500

We at Four Kitchens are devotees of the scrum method of development. When we celebrated the birthday of our Project Manager and Certified Scrum Master Shannon Hinshaw, we could think of nothing more appropriate than a SCRUM-ptious cake:

Task: Eat the cake (999 points)
<!—break—>
The cake will be eaten in two-minute increments during our daily standing meetings.

(In case you’re wondering, that’s a chocolate Italian cream cake from the Upper Crust Bakery in Austin, Texas. Special thanks to their crew, who always handle our unusual syntactical requests without hesitation.)

The Transatlantic Tacky Swag Swap has begun!

Todd Ross Nienkerk — Tue, 31 Mar 2009 11:37:28 -0500

Web Chef Aaron Stanush “mugs” for the camera. Get it?

Drupal themer extraordinaire Morten.dk, currently ranked #7 on Google for “king of Denmark”, has been bugging us for a Don’t Mess with Texas mug. Well, “bugging” may not be the right word. “Profanely demanding” is more appropriate.

Finding one was surprisingly difficult. While (lesser) cities like Dallas and Houston are lined with shops hawking rattlesnake heads and scorpions encased in plastic, there doesn’t seem to be much demand for Texas memorabilia in Austin.

Except at the airport, where you can find your name stamped on a fake Texas license plate or worn chunk of fencepost.

So, after scoring the great city of Austin for tacky crap, we proudly present Morten.dk’s Don’t Mess with Texas mug:

Morten.dk’s Don’t Mess with Texas mug
<!—break—>
Denmark or bust!

In return, we demand Morten.dk send us the tackiest Danish thing he can get his hands on. (I seem to remember him saying something about mermaids. Is that a Danish thing? Is Denmark known for mermaids or mer-related activity?)

Morten, the ball’s in your court. The gloves are off, and I’ve thrown down the gauntlet. There’s a line in the sand. You’re walking a tightrope.

That is to say, you have been challenged. The Transatlantic Tacky Swag Swap has begun!

The hidden costs of proprietary software: #2 your vendor is an adversary

David Strauss — Wed, 25 Mar 2009 17:13:19 -0500

On December 2, 2008, customers of SonicWALL woke up to broken firewalls. This wasn’t the result of a real problem in the firewalls; it was a result of SonicWALL’s DRM server malfunctioning and deactivating all customer firewalls.

The relationship between customers and vendors of proprietary software is fundamentally adversarial: proprietary vendors have business models where customer activity (like installing Windows on a desktop) requires payment to the vendor. Because the activity happens entirely on the customer side and paying the vendor conflicts with the customer’s desire to save money, proprietary software vendors don’t trust their customers to pay them.
<!—break—>
So, they’ve developed strategies based on customer distrust. One of these strategies is embedding DRM, software their customers run that looks out for the vendor’s interests. DRM systems, like Microsoft’s activation tools, continually threaten to disable the software customers rely on. And, with few exceptions, they run on code the customers cannot inspect.

Microsoft has removed activation code from many of their server products, but it remains common in their desktop products, like Windows and Office. Other proprietary vendors (like SonicWALL) still have it in their server and infrastructure code. Any software that has secret failure mechanisms integrated for the vendor’s sake has no place in important business infrastructure.

Software fails often enough without being defective by design.

Bazaar 1.13 RPMs for RHEL 5 and CentOS 5

David Strauss — Sat, 21 Mar 2009 23:03:00 -0500

Fresh packages, 64-bit only this time. I can make 32-bit ones if anyone would like some.

BarCampAustin 4 graphics are awesome

Todd Ross Nienkerk — Thu, 12 Mar 2009 15:40:44 -0500

BarCampAustin 4 graphics have arrived, and they are totally rad. Congrats all around to the fine folks who made them happen.

DJ Dillo: BarCampAustin 4 logo
<!—break—>
BarCampAustin 4 badge (front)

BarCampAustin 4 badge (back)

Here are some handy BarCampAustin links:

BarCampAustin 4 Facebook event page (RSVP here!)
BarCampAustin Facebook page

Oh, and there will be a special guest…

"CVS Instructions" tab now available for all Drupal.org projects

Todd Ross Nienkerk — Thu, 12 Mar 2009 09:38:36 -0500

“CVS Instructions” tab on the Author Taxonomy module

Drupal’s CVS is now more user-friendly!

As part of the Documentation Sprint at Drupalcon DC 2009, web chef David Strauss built a “CVS Instructions” tab for Drupal.org. The tab provides concise, step-by-step instructions on how to check out, commit, patch, tag, and branch any module or theme. A simple drop-down box at the top of the page allows the user to select the version of the module or theme they want to work with, and the instructions are updated to display exact, copy-and-pastable commands.
<!—break—>
Here’s an example of the CVS Instructions tab for the 6.x-1.x branch of Author Taxonomy.

This tiny feature represents a huge step forward in Drupal’s approach to opening its doors to contributors of all skill levels. This is especially important for those designers among us, many of whom cannot contribute themes — something Drupal sorely lacks — because they do not understand Drupal’s arcane CVS or command-line interaction in general. At Drupalcon DC, we designers pledged to learn and use the developer-oriented tools used by the Drupal community — namely IRC and CVS. Simple steps like the “CVS Instructions” tab move mountains for those who would otherwise give up and not contribute anything at all.

The hidden costs of proprietary software: #1 optimizing around licensing

David Strauss — Sun, 01 Mar 2009 20:35:48 -0500

Articles abound about the “hidden costs” of using free, open-source software. Many of them are sponsored by companies with a stake in their own proprietary solutions — and they’re responding to the threat of increasing enthusiasm about free alternatives. Some of the claims are legitimate; others are FUD.

Here at Four Kitchens, we’re on the opposite side. We advocate using free software like Drupal (and our own free-software derivative, Pressflow) whenever possible. When it’s not immediately possible, it’s a hard decision between writing a free solution and going proprietary. We enjoy the freedom of free software for many reasons, especially because it doesn’t feel like we’re fighting the company behind the software in order to get the most out of it.
<!—break—>
The problem with proprietary licensing

Proprietary software usually requires licensing fees; this is hardly a “hidden cost.” The hidden cost is identifying the optimal licenses and developing around them. Technical people suddenly get involved in the business decisions of what licenses the the organization needs to buy. Business people put constraints on the technical team with what licenses they approve. They’re both operating outside their areas of expertise, something that should be minimized.

Whether it be differences between Oracle editions, Lotus’s processor value unit formulas, or what restrictions IBM puts on its IFL cards, they’re all technically arbitrary restrictions designed to maximize price discrimination. In other words, the vendors want to make different customers each pay as much as possible for the same basic product.

Companies are even up-front that these licensing schemes are wholly for settings price tiers:

The attractively priced IFL processor enables you to purchase additional processing capacity exclusively for Linux workloads, without affecting the MSU rating or the IBM System z™ model designation. This means that an IFL will not increase charges for System z software running on general purpose (standard) processors in the server.

For readers unfamiliar with the System z IFL scheme, IFLs are identical except for the microcode restrictions IBM installs to prevent running an unapproved “workload” on a restricted IFL. So, instead of simply buying the number of processors the server needs, systems architects have to choose how many Java cards, Linux cards, and general-purpose cards will go into their mainframe, despite them all being physically identical.

The cost to the team is engineering around these arbitrary restrictions, restrictions that may be vaguely defined, as in some editions of Microsoft SQL Server (.doc), where “…performance degrades when more than five Transact-SQL batches are executed concurrently.”

As a developer and systems architect, I don’t like designing systems around restrictions artificially imposed by sales and marketing teams, especially when they’re vague. I’d much rather design around real restrictions and spend the rest of the time building great software. Don’t systems architects have enough to worry about?

What makes Pressflow scale: #1 faster core queries

David Strauss — Sun, 01 Mar 2009 16:36:15 -0500

Drupal has a number of queries with unfortunate scalability profiles.

URL alias counting (one instance in core)

The biggest offender in Drupal 5 and Drupal 6 is the query counting the number of URL aliases: SELECT COUNT() FROM url_alias. This query dates back to when nearly every Drupal site ran on MyISAM, which is important because MyISAM keeps an exact count of the number of rows in every table, making SELECT COUNT () FROM [table] an O(1) (read: fast, constant-time) operation.

But InnoDB, the engine of choice for high-scale Drupal sites, does not keep an exact row count for tables because its multiversion concurrency control (MVCC) makes such a count difficult and inefficient. But, MySQL with InnoDB still faithfully runs the query, but by counting every row in the table, an O(n) operation, meaning it is proportionally slow to number of URL aliases on the site.

Such counting is particularly unfortunate because the URL alias system only cares whether the number is zero or the number is greater than zero. In Pressflow 5 and Pressflow 6, we replace this query with SELECT pid FROM url_alias LIMIT 1 (or equivalent), giving us just the information we need (“Is there at least one alias?”) in a way that runs in O(1) on both MyISAM and InnoDB.

Use of LOWER() for case-insensitivity (many places in core)

Drupal 5, 6, and 7 all currently use LOWER() on both sides of some queries to create database-agnostic, case-insensitive string matches. The uses of LOWER() in Drupal on the a table column prior to comparison in a query automatically degrades queries to O(n) with respect to the number of users on the site. The most users a site has, the more time login and other frequent user operations take.

The reason Drupal 5 and 6 use LOWER() is because PostgreSQL’s LIKE operation performs case-sensitive comparisons. (And PostgreSQL’s ILIKE operation is not cross-platform.) By using LOWER(), the same query can run on MySQL and PostgreSQL without modification.

But Pressflow 5 and Pressflow 6 only explicitly support MySQL, so they can take advantage of MySQL’s case-insensitive collations and seamlessly drop the LOWER(). Dropping LOWER() results in user lookups happening in O(log(n)) time, which is very fast for even the largest sites.

Real results from the Materialized View API

David Strauss — Mon, 23 Feb 2009 14:41:17 -0500

I’ve faced a lot of skepticism (rightfully so) over my Materialized View module, which I’m pushing for inclusion in Drupal 7 as a solution to the overhead of table-per-field storage in Field API, as well as many other scalability issues.

I could have responded with contrived benchmarks, but I wanted real results. With Drupal 7 far from release and large projects on Drupal 7 even farther from release, I decided to rewrite MV for Drupal 6 and install it on Drupal.org to replace some of the worst queries.

I used MV-based tables to rewrite #1 and #4 of Drupal.org’s slowest, most common queries. This required the creation (and indexing) of two materialized view tables. The rewritten queries went live on Saturday at around 19:00 UTC.

The drop in load is visible at 19:00 on Cacti graphs from Drupal.org’s DB2:

Temporary disk tables, one of the worst causes of scalability issues, showed a small drop:

The execution plans of SELECT queries also improved. “Read next,” an indicator of table-scan behavior, dropped significantly following the MV switch-over:

And a commenter below requested the standard “load average” stats from the server:

If anything, the results are pessimistic because MV indexed data (a one-time process) from 19:00 UTC through Sunday morning. You can see the effect of MV indexing on load by comparing Sat 12:00-19:00 and Sun 04:00 - 05:00, where MV was not indexing, to surrounding times, where MV was indexing.

This indexing load is visible on the “Volatile Queries” graph:

Weekday vs. Weekday (scale change)

The most interesting graphs, in my opinion, are some Friday versus Monday ones:

Schema changes should be lazy

David Strauss — Wed, 18 Feb 2009 18:44:23 -0500

We’re in the middle of upgrading Drupal.org, and many of the longest-running upgrades involve schema changes. Unfortunately, MySQL with InnoDB has a very unfriendly method of doing schema changes: it rebuilds the table and blocks all writes until the new table is ready. A more sensible approach would be schema versioning that allows different parts of a table to have different schema versions. This would minimize blocking, allowing schema changes to happen without downtime.
<!—break—>
David’s rules for efficient schema changes:

Don’t block: Take no immediate action when the admin requests the schema change, except for recording the change that needs to occur.
Be flexible: Have a versioning model in place that allows different rows to have different on-disk formats. This versioning may not be per-row, but it should be sufficiently high-resolution that upgrading the smallest unit does not cause troublesome lock delays on a live system.
Be lazy: Upgrade the on-disk data format as rows are written. Transform row formats as they are read.
Hide the mess: Despite having rows in various formats, the application should read from and write to the tables as if they are all in the latest format once the change occurs. Combined with the “be lazy” rule, this abstraction should be possible with minimal performance impact.
Be thorough: Actively upgrade on-disk data in the background during periods of low load. Tables should converge on having all rows in the latest format.
Stay correct and atomic: Check that the conversion will be successful on a current table snapshot (you’re using MVCC, right?) and any incoming writes that happen during the check. Before the atomic moment of schema change, the change should fail if any rows won’t convert, including the ones written during the check. After the atomic moment of schema change, writes should fail if they don’t match the new schema.
Play loose with indexes: Indexes are optimizations. A database shouldn’t block while adding, removing, or modifying indexes. There’s no need to. Just make sure an index, if used, is complete. Even MyISAM follows this rule. (Granted, the external InnoDB plugin handles this.)

Is there any free, open-source database that observes all of these rules? I know PostgreSQL has transactional DDL, which leads me to believe it implements something similar.

I’m aware there may be objections to delaying the atomic schema changeover until after tables have been checked because it doesn’t perfectly preserve transaction sequencing. But, there’s no sensible alternative. If we assume the schema change will succeed and immediately reject illegal writes, we risk false rejection of the writes if the schema change later fails. That would violate isolation and atomicity for the schema change, a far greater ACID sin. (Give penance to the Oracle?)

While the delay for the schema change in my model may be considerable, administrators would have a choice of taking the application offline (as they do now) or making the application aware of the upcoming schema change (which cannot easily happen now). Such application awareness could involve as little as a check each request to see if the change occurred, choosing alternate code paths for before and after. The next application update removes the old code path. Properly handled, there would be no downtime.

This “lazy schema change” model isn’t just applicable to relational behemoths. It maps well to document databases like CouchDB where records do not generally get cluster-wide updates, even as formats change. Document database systems can be aware of what changes need to occur when reading data in old formats, and they can update documents to the latest formats while reading and writing them.

David Strauss elected as a Permanent Member of the Drupal Association

Todd Ross Nienkerk — Wed, 18 Feb 2009 14:40:31 -0500

Congratulations to David Strauss, Four Kitchens co-founder and Drupal scalability guru, who was elected yesterday as a Permanent Member of the Drupal Association.

David’s goals as a member focus largely on improvements to infrastructure, community-building, and reaching out to other open-source projects. Details can be found in his his application:

What are the primary goals you would like to work on?

I would like to advance the infrastructure for development and sprints by working with the community to drive development and deployment of next-generation (read: not CVS) tools, both for issue tracking and version control. I would like to participate in discussions surrounding the membership software for Drupal Association membership, including CiviCRM (the current tool) and alternatives. I would like to work with major free culture and free software organizations to establish partnerships.
<!—break—>
What strategy will you employ in order to accomplish said goals?

I would like to host a series of official online meetings to discuss options for development infrastructure and prepare an Association-endorsed roadmap. For membership management, I would use my experience implementing and managing CiviCRM and other non-profit-focused tools to inform and guide the Association’s decisions. For partnership building, I would draw on my relationships with other projects and developers to start productive discussions about shared goals and their projects’ stake in Drupal’s success.

Read more »

Congratulations are also in order for the other elected Permanent Members and the new Board Members, all of whom are donating huge amounts of time towards promoting and supporting the Drupal project.

Developer preview of Materialized Views

David Strauss — Mon, 16 Feb 2009 20:16:32 -0500

I’ve posted a developer preview of Materialized Views for Drupal 6. I’d like ambitious Drupal developers to try it out so I can get feedback on the developer experience.

From sites/all/modules:
bzr branch bzr://vcs.fourkitchens.com/drupal/modules/materialized_view/6 materialized_view (all on one line)

To get them running:

Install all three MV-related modules.
Run cron.

This will create the mv_forum_topic materialized view, which is populated and indexed for fast Forum module topic listings.

Specific developer experience areas I’d like feedback on:

Creating new materialized views.
Creating new data sources.

If you create or modify MVs, run cron to generate the tables and index the data.

No-brainer shared branch storage on your workstation

David Strauss — Mon, 09 Feb 2009 03:08:57 -0500

<!—break—>
This post is a follow-up to my post about the new Drupal HEAD Bazaar branch hosted at Four Kitchens. This post is written specifically for users of Unix-like systems. You can do the same thing on Windows, but you’ll have to adapt the directions a bit.

For me, my projects are code-based and, hence, mostly a collection of version-controlled branches. So, my “Projects” directory (located at ~/Projects) contains a smattering of branches organized into multi-level directory structures.

In my last post, I noted that shared branch storage is the key to fast branching, good offline access, and efficient disk usage. I even covered a quick way to get it rolling for Drupal branches. But it’s one thing to know how and another to know “best practices,” so I’ll share how I do things.

If you’re like me, you want minimal hassle and maximum flexibility for organizing all of your branches. Fortunately, Bazaar doesn’t just look at the next directory up to find a viable shared branch storage database.

Here’s what happens when you create a branch:

Bazaar checks if the new branch is created using the --standalone option. If so, it doesn’t even worry about the possibility of shared storage, and it stores the branch right there. (I’ve included this option for completeness, and you should only worry about it if you need branch permission control, which is typically not an issue on a workstation.)
Bazaar traverses up the directory tree looking for a viable shared storage location. If one is available, it stores the data there, taking advantage of shared revision history between branches.
If all else fails (and this is the default case), Bazaar stores the data right there with the branch.

Based on rule two, all you need is shared storage somewhere in a writable parent directory. I like to put my branches in ~/Projects, so I create shared storage in ~/Projects by running bzr init-repository ~/Projects. Then, I’m set for any branches created in or below ~/Projects.

If you don’t like to centralize your branches one place, you can run bzr init-repository ~ and have shared storage for any branch, anywhere in your home directory. For those used to Subversion and CVS spewing directories everywhere, I’ll explicitly note that this only creates one directory at the specified shared-storage location.

Once you’ve set up shared storage, you don’t have to think about using it; Bazaar will manage it automatically and transparently. If you’re curious whether it’s working, you can run bzr info from within any branch and look for “shared repository” in the output.

If you want to move your branch
If your branch uses shared storage, it references revisions in the shared storage repository. You can’t move it to another computer using normal filesystem commands and still have it work. If you’d like to make the branch completely independent of its shared storage so you can move it anywhere, run bzr reconfigure --standalone from a directory within the branch. It will copy its revisions to local storage.

A better option for moving branches is to use Bazaar’s own bzr branch command to clone the branch and then delete the original. Then, you don’t have to worry about the possibility of shared storage.

Theory note: This is Bazaar’s version of “cheap branching.” Creating additional branches with shared storage only takes the additional disk space required to create the branch’s new working tree (and even that additional space is only necessary if you choose to create a working tree for the new branch).

A Bazaar branch of Drupal HEAD with all history

David Strauss — Sun, 08 Feb 2009 22:12:42 -0500

I originally created a Bazaar branch of Drupal HEAD with hourly snapshots of upstream updates (commits to CVS HEAD) to streamline work on patches to Drupal HEAD.

This snapshot method had a few advantages over using Launchpad’s Drupal branch:

Because of the shallow history, branching from the Four Kitchens server was relatively fast compared to branching from Launchpad.
The Four Kitchens server has more reserve bandwidth and capacity, making initial branching and updates faster.
Launchpad goes down more often than the Four Kitchens server.

The snapshot-based branch also had a big disadvantage: it didn’t keep a true commit-by-commit history that mapped to CVS HEAD commits, making it hard to understand ongoing changes. After some discussions with chx, I’ve decided to combine the best of both worlds: a mirror of Launchpad’s Drupal HEAD branch on the Four Kitchens server with instructions (provided below) to avoid the trouble of pulling thousands of revisions every time you branch.
<!—break—>
Why not branch directly from Launchpad? Aside from occasional Launchpad reliability issues, downloading all Drupal revisions takes takes 2m37s from Launchpad versus 1m15s from Four Kitchens (both tested from my home connection).

I will also continue to maintain the old snapshot-based branch until Drupal 7 is released.

Method 0: Plain, old branching

Pros

Runs with older Bazaar versions
Performs all post-branch operations locally with no network access
Easy to set up

Cons

Downloads all upstream revisions each time you branch (takes 1m25s)
Stores all upstream revisions each time you branch (takes 51M each branch)

Run this: bzr branch bzr://vcs.fourkitchens.com/drupal/7-all-history

Method 1: Stacked branches

Pros

Fast initial branching: only downloads basic branch data (takes 42s)
Most space efficient: only stores basic branch data (takes 6.6M total)
Easy to set up

Cons

Requires Bazaar 1.6 or later
Requires internet access to perform most history-based operations

Run this: bzr branch --stacked bzr://vcs.fourkitchens.com/drupal/7-all-history

Method 2: Shared branch storage

Pros

Runs with older Bazaar versions
Downloads upstream revisions once for all branches
Stores upstream revisions once for all branches (takes 51M once)
Performs all post-branch operations locally with no network access

Cons

Still stores upstream revisions once
Still downloads all upstream revisions once
You have to run a few more commands to create your first local branch

From a new directory that will be a parent of your future branch directories, run this:
bzr init-repository .

From within the parent directory you just created, run this to create your local branch:
bzr branch bzr://vcs.fourkitchens.com/drupal/7-all-history

Using your branch

Presumably, you’ve gone through all this to work on patches for Drupal HEAD. I’ve updated my earlier instructions for how to use your new branch.

Case study preview: Four Kitchens' new business cards

Todd Ross Nienkerk — Thu, 05 Feb 2009 18:23:47 -0500

Assignment: New business cards for our staff.

Goals: A clean, professional, and friendly design with room to write on. (People like to make notes on business cards — namely, when and where they met that person.)

Process: After several iterations — I’ll post ‘em in the full case study later on — we arrived at a minimalist, two-color design. What’s most striking, perhaps, is the front side doesn’t actually say “Four Kitchens.” Instead, we opted to show only the logo and slogan to create a sense of mystery: “What’s this all about? Who makes big websites? Who are these people?” Simply flipping over the card reveals the answer.

To create a friendly, informal tone and demonstrate our sense of humor, we have the logo “speaking” our slogan, as if to say: “Hi! Nice to meet you. I like you. Let’s hang out!” Just like I do in real life.

Front

The back of the card displays three important chunks of information, the placement and design of which each tell a story about our company.

First, the employee’s name and contact information is placed above the company’s. Each of our web chefs posses unique and invaluable skills. Everyone here contributes to the direction of each project, both internal and client-driven.

Second, each card includes GPL and Creative Commons watermarks. Open-source development and collaborative culture are central to our business model. We do not work with closed-source software, and we contractually obligate our clients to license our work under the GPL and similar licenses.

Third, each card reflects the web chef’s personality. When designing the cards, we asked each chef to choose four areas of professional and personal interest and created a set of icons that symbolized each.

Robbie, for example, is really into tuning his car. (He’s installed a supercharger, added USB hookups to tune the onboard computer, and has all kinds of extra gauges to monitor… whatever.) So we made him an engine icon. I, however, have a much more reasonable and interesting hobby: reading and writing. So I made myself a pilcrow.

Aaron likes rock climbing. Shannon Lucas likes SCIENCE! You get the idea.

By placing these icons in talk bubbles to the right of each web chef’s name, the cards seem to say: “This is what I’m good at, and this is what motivates me.” These cards communicate so much more than simple phone numbers and email addresses. (How boring!)

Here’s the entire set:

Back / Todd Ross Nienkerk

Back / Aaron Stanush

Back / David Timothy Strauss

Back / Shannon Hinshaw

Back / Shannon Lucas

Back / Robbie Sternenberg

Enforcing branch commit atomicity (or, why the git staging area is bad)

David Strauss — Mon, 02 Feb 2009 23:26:35 -0500

With CVS, one of the only repository-wide atomic operations is tagging a local checkout. And not all that long ago, Subversion introduced mainstream users of free, open-source version control systems to full-scale atomicity. Or, at least the ability to be atomic.

Subversion’s approach to atomicity is rooted in its centralization and hybrid branch/directory model. Because Subversion makes it hard to merge from other repositories, there’s a strong incentive to combine many projects and branches into one repository. Subversion therefore offers directory-level checkouts as well as convenient, repository-wide checkouts for developers working on multiple projects. To fit this project and branching model, Subversion performs most operations at the level of the current directory and below. (The only alternative would be performing the operations checkout-wide, which would cause behavior confusingly dependent on the choice of checkout root.)

Subversion’s model operates atomically if users run the commands from the root of a project or branch, but the this-directory-and-below model encourages bad development behavior. For example, when working on a Drupal project, it’s easy to commit just the changes for one module or theme, which creates a revision in the repository that may never have existed as a working copy and may not work. Administrators can mitigate the problem with repository-side continuous integration (CI), but even CI still doesn’t guarantee true project coherence and atomicity.

Bazaar, on the other hand, performs its operations (at least by default) on the entire branch, encouraging real atomic commits. This default, branch-level behavior tends to annoy developers used to separating a project’s changes into different directories of a Subversion checkout and checking in by directory. This Subversion-based workflow works reasonably well in practice, and for systems like Bazaar and git to enforce high levels of atomicity and remain usable, they must provide convenient tools to separate the changes intended for each commit.

Bazaar and git have different approaches providing such tools. Bazaar has shelve and unshelve. git has the staging area.

The most obvious way they differ is in workflow. Bazaar’s commands are optional and remove changes from the upcoming commit. git’s commands are mandatory and add changes to the upcoming commit. Here, git’s choice seems very sensible. Encouraging manual approval of each change in each commit reduces mistakes. On the other hand, Bazaar provides a convenient uncommit command that allows reversal of erroneous commits (which are often only obvious when seeing the file list as the commit is happening). All considered, I slightly prefer git’s workflow here.

Where git fails is in the theoretical foundations of the staging area. The staging area encourages the same bad behavior as in Subversion, just with more surgical control of what gets committed. Committing in git with only some changes added to the staging area still results in an “atomic” revision that may never have existed as a working copy and may not work.

Along these lines, one of the most atomicity-busting aspects of git’s staging area is that it doesn’t just mark code that needs to go into the next commit; it actually saves the hunk into the staging index. So, a developer could add code to the staging area, modify her working copy, and end up with a commit containing code that’s neither in her working copy nor in her stash. The code only ends up in the commit just made, silently filed away for someone else to get in their next merge:

This command [add] can be performed multiple times before a commit. It only adds the content of the specified file(s) at the time the add command is run; if you want subsequent changes included in the next commit, then you must run git add again to add the new content to the index.

In contrast, shelving a change in Bazaar reverts the change in the working copy. (It does save the change for later restoration with unshelve.) Because shelved changes are not in the working copy, Bazaar encourages the ultimate in atomicity: what a developer commits represents an atomic snapshot of the entire branch as represented by her working copy. And if tests pass when she commits, the same tests will pass if another developer pulls the same revision of the same branch.

DrupalCon DC swag is here!

Todd Ross Nienkerk — Thu, 29 Jan 2009 17:39:03 -0500

Here’s a taste of what you’ll be getting at DrupalCon DC.

DrupalCon DC swag

The stickers and button were designed by Four Kitchens. Development Seed very graciously printed and paid for the buttons. Thanks, Development Seed!
<!—break—>
The sticker in the lower left is based on a DrupalCon T-shirt design we submitted. (If you like it, vote for it! And our other designs, too…)

The button is a more polished version of a DrupalCon “campaign button” design we released last month under the GPLv3.

Todd’s bag o’ buttons

Here’s my bag wearing a series of hip buttons and a very classy Oktoberfest scarf. It’s probably the best scarf you will ever see — and you’ll see it at DrupalCon DC!

Bazaar 1.11 and BzrTools 1.11.0 RPMs for Red Hat Enterprise Linux 5 and CentOS 5

David Strauss — Tue, 20 Jan 2009 20:04:09 -0500

These RPMs are now obsolete. Please check this blog for the latest ones.

The EPEL RPMs for Bazaar and BzrTools are quite out of date, so I rolled some experimental packages based on the Fedora 10 spec files.

<!—break—>

Bazaar i386	Bazaar x86_64	Bazaar SRPM	Bazaar Spec
BzrTools i386	BzrTools x86_64	BzrTools SRPM	BzrTools Spec

One more requirement

If you don’t have EPEL installed as a repository, you may have to manually download and install the python-pycurl (i386 or x86_64) package.

Creating common branch ancestry is a hard problem

David Strauss — Mon, 19 Jan 2009 01:08:06 -0500

One of the key features of distributed version control systems (DVCS) is support for divergent development (branching) and then merging. Most DVCS tools, including git and Bazaar, include rather elegant support for such workflows by embedding metadata about common ancestry into branches. In this post, I’ll be focusing on Bazaar.

“Common ancestry” means an identical revision shared by two branches. The most recent common ancestor typically indicates the point of branching, unless there has been a more recent merge. A successful merge between two branches establishes a new, more recent common ancestor. Identifying a common ancestor is a required step for performing automatic three-way merges to integrate changes from a foreign, divergent branch.

Typically, the branching metadata allows Bazaar to automatically determine the most recent common ancestor to use as the base revision for three-way merging. The problem comes when you need to merge two branches that do not share any common ancestors. (For the purpose of this post, I am not counting revision zero, the universal common ancestor, as a common ancestor. It is generally useless for merging.)

Creating common ancestry
If you try to merge two branches without common ancestry and without any revision identifiers, Bazaar will complain and do nothing. You can specify a revision range (like -r5..-1, which would be everything from the fifth revision to the latest on the foreign branch) and then apply the merge. Bazaar will then merge the changes in the specified revision range into your local branch and establish the last merged revision as the latest common ancestor, making future merges a breeze. Unfortunately, finishing this initial merge is where dragons lie. But before I can get into the difficulty of creating common ancestry by merging two unrelated branches, I have to briefly discuss how Bazaar handles files.

How Bazaar tracks files
Bazaar maps file paths to globally unique file IDs, and two branches without prior common ancestry will have different file IDs for the same paths, even if the files are really the same. Every time a file is added to a Bazaar branch, it gets a unique ID. As files are moved and renamed, they keep their unique IDs. So, if two people download Drupal, extract it, and independently “bzr init”, “bzr add”, and “bzr commit”, their respective README.txt files (for example) will have different file IDs.

Creating common ancestry (continued)
These globally unique file IDs cause trouble when merging from unrelated branches. When Bazaar merges two branches (related or not), two files with the same path but different file IDs create a conflict even if they contain identical content. Merging two unrelated branches with lots of shared files creates a mess of conflict files and conflict directories, and there is currently no convenient way to resolve these conflicts.

A concrete example with Drupal
A developer downloads Drupal and puts the project under version control in a fresh Bazaar branch. She discovers that Four Kitchens maintains a Bazaar branch of stable Drupal releases and decides she would like to use the Four Kitchens branch to automate installation of minor Drupal updates. She attempts to merge in the revision range from the Four Kitchens branch that would perform the upgrade. Because the file IDs in the Four Kitchens branch differ from hers, every Drupal core file and directory has a conflict despite having identical content for the base merge revision. After a bit of tedious conflict cleanup, she commits the merge and goes back to enjoying Bazaar’s generally elegant architecture.

I thought this blog post was going to give me a solution!
Nope, there’s not one out there yet. I’m currently looking at writing a custom merge handler (subclassed from the standard merge3 in Bazaar) that would intelligently handle merges where file paths do represent the same files, regardless of file IDs. Unfortunately, the file ID/path conflict is low-level in Bazaar and occurs before reaching the most modular part of merge conflict resolution.

Thanks to Robert Collins on the Bazaar project for walking me through the Bazaar internals necessary for me to explain this issue and, hopefully, solve it.

Quick Drupal version control with Bazaar

David Strauss — Sat, 17 Jan 2009 18:53:13 -0500

We all have the Drupal projects we work on that ought to be under some sort of version control system but aren’t. There are many reasons why this might be the case: the small size of the project, the lack of central version-control infrastructure, or the annoyance of “.svn” or “CVS” directories littering your working copy.

With Bazaar, none of these excuses should prevent a project from having solid version control.

Here’s why Bazaar avoids those problems:

It requires minimal effort to bring code under control.
Bazaar does not require (but still supports) centralized infrastructure.
A single .bzr directory at the root of the project contains everything Bazaar needs, minimizing disruption.

Bringing a Drupal project into Bazaar

Note: If you want to automate your Drupal updates, you may want to branch from the Four Kitchens repository instead of initializing your own branch. Branching from the Four Kitchens repository is more time consuming, but it’s worthwhile for large projects.

Install Bazaar.
Go to the root directory of your project.
bzr init
bzr ignore ./files # Drupal 5 only
bzr ignore ./sites/default/files/ # Drupal 6 only
bzr ignore ./sites/default/settings.php
bzr add
bzr commit -m “Initial import”

That’s it! Learn to use Bazaar and enjoy powerful version-control capabilities for even your smallest projects.

Distributed version control provides a streamlined alternative to vendor branches

David Strauss — Sat, 17 Jan 2009 18:35:04 -0500

Anyone who’s worked with a sufficiently large project eventually ends up establishing vendor branches to track and merge upstream releases. Maintaining these branches is time-consuming, redundant work because everyone who needs a vendor branch of a project needs approximately the same thing.

With centralized version control tools, you have two choices:

Apply patches integrating the changes from upstream releases. This approach avoids the need to set up vendor branches, but it will not allow three-way merges, increasing the developers time spent resolving conflicts. Patches also cannot represent many types of changes, requiring more manual work to upgrade.
Download, extract, and commit each upstream release to your project’s own vendor branch. This is also time consuming and requires regular work checking for upstream releases. It does, however, allow three-way merges and applying all types of changes.

Distributed version control systems, like Bazaar, allow you to branch your project from a remote “vendor” branch, providing all the advantages of a local vendor branch with none of the headaches. Moreover, tools like Bazaar track the last merges from remote vendor branches and automatically apply changes made since your last merge.

For Drupal, Four Kitchens maintains vendor branches of stable Drupal 5 and 6 releases:

bzr://vcs.fourkitchens.com/drupal/5
bzr://vcs.fourkitchens.com/drupal/6

We also maintain vendor branches for Pressflow:

bzr://vcs.fourkitchens.com/pressflow/5
bzr://vcs.fourkitchens.com/pressflow/6

Branching from these allows you to upgrade your project with a simple “bzr merge [vendor-branch-url]”. The popular vendor branch URLs are shown above.

Using Bazaar to collaborate with other patch developers

David Strauss — Tue, 30 Dec 2008 14:50:01 -0500

In my earlier post on using Bazaar for Drupal core development, I explained how to use the Four Kitchens Bazaar repository to streamline development of core patches. For patches you’re developing on your own, those instructions work great. For patches involving a team of developers, you’d want to have a shared mainline branch, which is beyond my last post’s scope and this one’s.

But, what about when two to three people are collaborating, you’re using Bazaar, and someone else has posted an updated patch that you’d like to work from?

First, you’ll want to install BzrTools. If you installed pre-packaged Bazaar for Mac OS X, you already have these plugins installed.

These instructions also assume you’re working on a patch for Drupal 7. If not, replace the number “7” below with the appropriate version.

Warning: Continuing will erase your local changes and replace them with the changes in the patch you apply.

Copy the patch URL to your clipboard. Don’t bother downloading; Bazaar’s patch utility will download the patch itself.
Change directories to be somewhere in your development branch for the issue you’re working on.
Run as one command: bzr pull —overwrite —revision=date:[date-of-patch] bzr://vcs.fourkitchens.com/drupal/7
The —revision part is optional, but specifying a date will help Bazaar apply the patch against the revision the patch was created against (or something very close), which is more likely to work than applying the patch to CVS HEAD directly. Use a date specifier like “yesterday”, “2008-12-30”, or “2008-12-30,13:41:14”. See Bazaar’s complete documentation on revision specifiers.
bzr patch [url-of-patch]
bzr commit -m “Applied patch [url-of-patch].”
bzr merge bzr://vcs.fourkitchens.com/drupal/7
Resolve conflicts, if any.
bzr commit -m “Merged in changes to CVS HEAD since patch.”

You’ll have a refreshed working copy with the patch applied against the latest Drupal 7. You can now go back to using the instructions on my earlier post to continue working or roll a fresh patch.

Dynamically attribute content in Drupal using the Author Taxonomy module

Todd Ross Nienkerk — Tue, 16 Dec 2008 16:21:17 -0500

Author Taxonomy settings screen

Attributing a story, image, or blog post to more than one person can pose a problem on many web platforms. In the print publishing world, it’s simply a matter of adding another name to the byline or tacking “Additional reporting by Sue” to the end of a piece. (Nowhere does the poor designer or typesetter get credit for laying out the page!)

On the web, priorities are flipped. Instead of attributing a piece of online content to its author, most web software attributes content to the user who posted it on the site. Countless sites display ugly bylines like “nyeditor5” or “harry_henderson” instead of a properly formatted name.

And what to do about stories with multiple authors? They’re usually given credit in the first or last line of the content itself — a surefire way to reduce a semantic web evangelist to XML-encased tears.

Enter the Author Taxonomy module. We initially developed this module for That Other Paper, an Austin-area, online magazine we used to publish. To maintain a cohesive voice and layout style, we required all content to be vetted by one of our few web-savvy editors prior to publications. As a result, virtually none of the content on the site was placed there by its original author.

We experimented at first with creating a properly formatted username for each contributor: “Sue Smith,” “Hilarious Jackson,” and so on. This turned out to be a hassle, as users are unaccustomed to Usernames with Caps and Spaces. Even then, we couldn’t properly attribute content to more than one author. It was time for a custom module.

Author Taxonomy combines the power of taxonomies with the real-world demands of professional publications. Features include:

New authors can be created on the fly with “Tags” (“Free tagging” in Drupal 5.x) enabled.
Multiple authors are displayed using serialized text: “Sue and Jan” or “Sue, Jan, and Bobby.”
Names link to their taxonomy term pages, providing readers with a quick shortcut to all posts attributed to a particular author.
Site admins have the option of automatically overwriting the “real” node author — that is, the user who posted the node — with an author term if a matching username is found.
The date display is fully customizable and can be disabled completely.

If anybody has any suggestions for improving the functionality of Author Taxonomy, please add them to the module’s issue queue. We’d love to hear `em.

Screenshots

Node editing interface

How Author Taxonomy renders node output

A year in open source

Shannon Lucas — Mon, 15 Dec 2008 11:51:26 -0500

Mid-October marked my one-year anniversary with Four Kitchens and, consequently, the same anniversary of being an open-source software contributor. Because my career up to that point had been limited to proprietary software development, I had intended to write a one-year retrospective on the experience. But, as often happens, one gets busy, and now October, November, and half of December have passed. So without further ado, my self-indulgent retrospective on moving to open source development (complete with pontifications).
<!—break—>

Working for “The Man”

I have no idea how many lines of code I’ve written in my career. A lot? Maybe. I can’t actually prove it since all the evidence is locked away. And, to make it worse, there’s not even a product on a store shelf I can point to and say, “Hey! I wrote some of the code in there!”

That was always the hardest part of the job: working my ass off and having nothing to show for it but a paycheck. For some people, the pay is all the satisfaction required. For others, the activity itself provides the satisfaction. For me the satisfaction comes from the end result: being able to feel good about that result and being able to show it to other people. I rarely found that satisfaction while I was doing closed-source work.

Closed-source companies usually have no qualms using open-source software. Contributing back is usually a different story. Often it’s limited to reporting bugs. Even at commercial software companies that participate in open source, employees often need permission to contribute (Big Blue, I’m calling you out). This is true even at some universities since, in the United States, universities can own software patents. Yes, I could have still done it undercover, but I like to keep the agreements I make.

Rose tint my world

So what attracted me to doing open-source work in the first place? The biggest appeal for me was being able to show my work. That sounds really selfish on the surface, but the reality is that unless you are getting what you need out of your work, you’re not actually doing anybody any good. The next appeal was from the non-profits and grass-roots organizations I often saw making use of open-source projects. I saw it as an opportunity to contribute in some way to the things I cared about and could believe in. At least, that was the romantic view.

But there was also a cynical view. Several years of reading comment threads on Slashdot had convinced me that the open-source community was mean-spirited and hostile. Browsing support forums and seeing such helpful responses as “Why don’t you just use Linux?” reinforced that idea.

When I had to opportunity to take a job doing open-source work, I let the romantic view win. I committed my first Drupal module with a lot of anxiety and wondered if my ego would survive that first “damn, you suck at this” email. I’m sure that email has been delayed somewhere in that series of tubes.

The Drupal tribe

I don’t use the word tribe in jest, but rather to emphasize the feel of the open-source community that I’m currently involved with. Tribes are made up of individuals with distinct identities and distinct goals. The tribe facilitates those goals in order to reach its goals, and the individuals contribute their time and skills in order to reach their personal goals. All of the tribe folk are in the tribe by their choice and contributing in the way they want; it is not the selfless commune from Ayn Rand’s Anthem. The cycle of life and death in traditional human tribes is replaced by the arrival of new members who want to contribute and the departure of existing members that choose to move on to other things.

Being able to function as a member of the tribe has significant importance. It’s at least as important as technical skill. The tribe survives by nurturing those members who add to it and ostracizing those that don’t. I observed an instance of this when someone with strong industry credentials tried, rather arrogantly, to assert himself on those credentials alone. Several members of the tribe tried, without success, to persuade him to change his approach. Eventually, he stopped trying to participate after the group started ignoring him. I’ve heard stories of other times this has happened.

Tribal participation isn’t the same as being a “team player” at typical software companies. A tribe holds itself together by the will of its members; the typical corporate team is held together by the will of its management. This difference is subtle, but I believe that understanding it is essential in order to function in open-source projects. Coming from an industry background, it can feel like there must be someone in charge. Sometimes that can lead to the newcomer trying to take charge and causing friction.

There is a strong mentor contingent in the tribe. There is the occasional snarky answer to a question, but the general vibe of the tribe is that everyone starts out knowing nothing. No one is viewed as stupid for having a basic question.

Drupal is the only-open source community I’ve been actively involved with so far, so I don’t know how universal these observations are. I would guess that there are more similarities than differences with other open source communities. My expectations at this point, however, are rather high.

Perils and thrills

Open source development is not without danger. It has certain addictive properties. It tickles my ego whenever I get an issue on one of my Drupal themes. Solving the issue provides another tickle, and I find myself contemplating what to do to get more of those thrills. There are warning signs for addiction. For me, when I had a dream in which I was writing Drupal code while webchick YELLED AT ME on IRC, I decided it was time to take a break.

Coda

In writing this, I realize I am preaching to the converted. One of the easiest things to loose sight of in any human activity is the beginner’s mind. What motivated us to get involved? What did it feel like when we got started? What were the joys? And what were the frustrations? My personal experience getting involved with open-source was overwhelmingly positive, but there was trepidation in the beginning, and I don’t think I’m unique in that respect.

Open-source projects thrive through an infusion of fresh ideas and fresh minds. I believe it’s important as we seek out those new minds to remember what it was like for us when we were new and wanting to contribute. It’s not something that can be addressed in an FAQ. Newcomers will see it in our interactions with them and in our interactions with each other. It is subtle, but it’s never unseen.

Using Bazaar to work on Drupal core patches

David Strauss — Sat, 06 Dec 2008 23:11:36 -0500

As anyone who’s developed core patches knows, it’s not the writing of initial patch that takes the work, it’s the combination of revising the patch and keeping up to date with HEAD. With Drupal.org’s CVS, this is difficult because you cannot commit your core changes to checkpoint your work. CVS’s merge algorithms are also relatively poor for maintaining large divergence from CVS HEAD.

One option — which several community members used to create the huge DB-TNG patch — was to create a Subversion repository. This is far too much overhead for smaller core patches, and it still required person-by-person approval to commit to the Subversion repository.

Four Kitchens is now hosting a much more flexible system for users of Bazaar. Every hour, we synchronize core changes from CVS HEAD into our Drupal 7 branch, which is available for anonymous branching by anyone.

Simply download and install Bazaar, which has packages or installers for Linux, Windows, Mac OS X, and BSD. We’re using a repository format that should work with Bazaar 1.0 and newer, at a minimum.

Then, when you’re ready to develop, create a local branch and make its new directory your working directory.

Then, where you’re ready to develop, run this:
bzr branch bzr://vcs.fourkitchens.com/drupal/7 [optional-working-copy-directory]

You’ll have a fresh working copy with a number of benefits over a CVS checkout:

You can commit to your local branch to checkpoint your work: bzr commit
You can integrate in changes to HEAD using Bazaar’s superior merge algorithms:
bzr merge
You can branch from your own branch: bzr branch [existing] [new]
If you’re working with others and they have Bazaar branches, too, you can merge from their branches to collaborate.

And when you’re ready to post a patch, run
bzr diff --old bzr://vcs.fourkitchens.com/drupal/7-all-history > mypatch.patch
to create a patch reflecting all of your changes relative to CVS HEAD.

~~And when you’re ready to post a patch, run bzr diff --old bzr://vcs.fourkitchens.com/drupal/7 to create a patch reflecting all of your changes relative to CVS HEAD.~~

It will be a while before Drupal.org moves to anything other than CVS. Until then, we can foster decentralized development quite effectively using something like the Four Kitchens repository.

Note (2009-12-07): Yes, I realize Launchpad has a similar service that stores more detailed revision data, but I often find Launchpad slow or unresponsive. We also maintain Bazaar branches of stable Drupal 5 and 6; just replace the “7” above with “5” or “6”.

Note (2009-02-09): I have updated this post to use the new Bazaar branch for Drupal HEAD. We also maintain Bazaar branches of stable Drupal 5 and 6; just replace the “7-all-history” above with “5” or “6”. All obsolete information is now ~~crossed out~~ and replaced. I’ve maintained the obsolete instructions in case anyone is actively using them; they will continue to function.

Drupalers for Drupal: DrupalCon campaign buttons

Todd Ross Nienkerk — Fri, 05 Dec 2008 12:22:28 -0500

Just for kicks, here are some DrupalCon “campaign buttons” we cooked up while brainstorming T-shirt ideas. We’re releasing them under the GPLv3, so feel free to use `em as you see fit!

SVG and PNG formats are linked below. If you’d rather not work with a vector program, you can also download all three buttons as a layered, smart object PSD file.

Button #1: Drupalers for Drupal

SVG format | 150px PNG | 300px PNG

Button #2: Stars `n’ stripes

SVG format | 150px PNG | 300px PNG

Button #3: America’s ticket

SVG format | 150px PNG | 300px PNG

Snazzy! Our DrupalCon DC T-shirt submissions

Todd Ross Nienkerk — Thu, 04 Dec 2008 13:47:02 -0500

<!—break—>

We’ve been busy preparing submissions for the DrupalCon DC T-shirt contest. Here’s what we’ve come up with so far.

Retro-fitted!

Full-size graphics | Vote for Retro-fitted! »

Retro-fitted! part two: Refitted

Full-size graphics | Vote for Retro-fitted! part two »

Drupal arcade

This is my personal favorite (*cough* *cough*).

Full-size graphics | Vote for Drupal arcade »

Project Lincoln

Full-size graphics | Vote for Project Lincoln »

Robbie’s shirt

Robbie’s one of our developer web chefs. He begged us to release him from his coding cage long enough to design a T-shirt. This is what he came up with:

According to Robbie, the only reason he didn’t submit the design is because he got the year wrong. Poor guy.

A plethora of DrupalCon DC session proposals

Aaron Stanush — Thu, 04 Dec 2008 10:55:26 -0500

Photo by Don LaVange

Besides sponsoring next year’s DrupalCon Four Kitchens is also offering to share our know-how in the form of five session proposals and a MEGA session series on theming! Oh yes, we have a plethora. Read the rundowns below and if you find one you want to see on the schedule — go vote for it!

Business

Build your own open source business
This session will share the real experiences and hurdles encountered in the first two and half years of starting a Drupal consulting business. It’s aimed to enlighten attendees who are interested in a) building their first business with little or no funding and b) who prefer to embrace open source solutions.

Project Management for Agile/Scrum Development
Project Management is essential to successful projects and successful companies. Learn how the Agile/Scrum method can enhance your current project management skills. Or, learn about Project Management for the first time! This session will cover basic PM skills, organization, communication, and tools to make your life as a project manager run smoothly.

Core

Building infrastructure you can scale, monitor, and maintain
It’s easy to set up a single GNU/Linux server, but there’s no obvious way to take those same skills and build functional clusters. If you want to do that, you need a whole new class of tools. At Four Kitchens, we’ve done the hard work: we maintain an in-office lab to develop and apply cluster-level integration and management techniques to the challenging problems we encounter managing our clients’ infrastructure. We’d like to show you what we’ve learned in our lab and in the field.

Flexible development and deployment with Bazaar
Bazaar represents a departure from the dichotomy of “centralized” versus “distributed” because it directly supports both types of workflows, as well as hybrids. While other distributed version control systems also claim to do this, none support it quite so elegantly (and with such good cross-platform support) as Bazaar. Learn how Four Kitchens uses Bazaar to have unprecedented capability in development and deployment — from our smallest personal projects to our largest enterprise installations.

Design

MEGA SESSION: Theming Drupal from the ground up
This series of sessions will cover the full spectrum of Drupal theming. The first session will cover webdesign basics: HTML, CSS, and three-column layouts. Subsequent sessions will demonstrate how to port a theme to Drupal, aspects of design unique to Drupal, and advanced techniques such as implementing preprocessing functions and overriding theme functions.

Update: Our resident UX expert Shannon Lucas will be lending his expertise to the newly submitted usability session

Usability testing at the University of Baltimore
A report back from the third round of formal usability testing of Drupal core - held at the University of Baltimore, a week before Drupalcon DC. We’ll give an overview of the state of usability in Drupal core, see where improvements have been made, where there’s still work to be done. Building on the lessons learned from previous testing at UB and the University of Minnesota.

The great Drupal module hunt

Aaron Stanush — Wed, 19 Nov 2008 14:07:59 -0500

If you’ve keep up with Drupal news, you know there’s a hefty ongoing process of redesigning Drupal.org. Leisa Reichelt has played a huge part in documenting this process and has started some great discussions as well as set up some super useful cardsorting exercises to crowdsource site architecture concerns.

One of the most important examples of these cardsorting exercises is happening right now! It’s to help organize the somewhat chaotic page for finding Drupal modules. The creation of DrupalModules.com was no doubt formed as a solution to categorize and rate the available modules, but no one would argue the problem needs to be fixed at home as well.

Drupal’s huge set of freely available community modules is what makes it so versatile. Join the fight, ensure that finding modules is a painless experience for all time!

Extra credit: Here’s what the redesign team learned from the last card sort.

Photo by Caitlin

Four Kitchens at DrupalCon DC 2009

Shannon Lucas — Thu, 23 Oct 2008 10:32:27 -0500

Four Kitchens is proud to announce that we are attending and helping sponsor DrupalCon DC. From March 4th through March 7th, the chefs will be in Washington, D.C. to discuss their recipes for scalability, theme development, module development, design, usability, project management, and business strategy. And we promise we don’t use any secret sauce!

If there is a particular topic you would like to see the chefs present a session on, please add a comment to this entry, contact us, or send us a tweet on Twitter.

Photo by Ryan McFarland

<!—break—>

Formal usability testing of Drupal 7.x/8.x/9.x

Shannon Lucas — Wed, 01 Oct 2008 19:10:17 -0500

As part of an ongoing effort to enhance user experience, the Drupal User Exeprience Team is submitting a grant proposal to the Knight Drupal Initiative to receive funding for several rounds of formal usability testing on Drupal. The working draft of the proposal is currently open to community discussion, and we encourage anyone who is interested to participate.

KDI Usability Proposal

About the Knight Foundation

Since 1950, the Knight Foundation has invested more than $300 million to advance quality journalism and freedom of expression worldwide. Today, the program focuses on leading journalism excellence into the digital age. We define journalism excellence as the fair, accurate, contextual pursuit of truth. The Foundation is a Miami-based non-profit organization.

About the Knight Drupal Initiative

Drupal and the Knight Foundation agree that open source digital publishing can enable powerful agents of transformation in their communities.

Knight Foundation is working with the Drupal community on the Knight Drupal Initiative with the following goals in mind:

To enable more people to enter the digital conversation by lowering the technical barriers to entry.
To provide powerful tools for digital publication, free and open to all.
To encourage people to improve their communities by supporting the free exchange of information and ideas.

Download our free Drupal topic icons pack

Aaron Stanush — Tue, 30 Sep 2008 12:46:34 -0500

<!—break—>

These icons were originally created to correspond with session tracks for the Drupalcon Szeged 2008 conference. They didn’t end up being used, but that doesn’t mean we’re going to forget about them. Now you can download and use these hot icons for your own Drupal-related events, projects, and websites. These icons are released under the GPLv3 so they are yours to use, modify, and rerelease.

Download the Drupal topic icons pack (1.5MB)

Decorators and directories

David Strauss — Sat, 27 Sep 2008 17:45:40 -0500

Nodes have evolved remarkably over Drupal’s history. In Drupal 4.7, node types were typically created by modules that “owned” their node types. There was no way to create a node type without a module behind it. Modules creating node types would implement hook_node_info() and directly handle the the main loading, saving, and editing of the node type. Drupal core handled the loading and saving of the title and body. Modules doing this were effectively subclassing a pseudo-abstract node class (a class containing title and body only) in core and adding their own fields.

Drupal 4.7 was also the dawn of the Form API and its hook_form_alter(). Combined with the ability (beginning in Drupal 5) to create node types directly in core, the dominant pattern of node type development changed. The decorator pattern emerged as the preferred approach. This allowed multiple modules to simultaneously create fields on the same node types.

Using decorators with node types is straightforward:

Create your node types using Drupal’s content type administration tool.
Create a module that implements hook_form_alter() and hook_nodeapi() that adds fields to selected content types.
Configure the module to add its fields to the selected content types.

Even when it would be equally straightforward to have a module implement its own node type, implementing the fields with a decorator is superior because the approach maintains consistency. All fields are saved and loaded through hook_nodeapi(), and all form elements are defined through hook_form_alter(). No module has any special claim to particular node types. This approach should completely obsolete the implementation of hook_node_info() and other “I own this node type” hooks by modules.

But this model has some downsides, at least in Drupal:

When modules directly created node types, data tended to be better consolidated. Because we now prefer to decorate node types using the fields from multiple modules, data is more scattered. We’re typically implementing each decorator as a table with a foreign key to nid and vid. This has massive performance implications.
Configuration for managing the mapping of decorators to node types is wildly inconsistent, and there’s no way to see, globally, which decorators apply to which node types.
Managing interaction between decorators is inconsistent or absent. This interaction includes namespace conflicts on $node objects.
The node editing form and $node object are the only places where decorators all come together in a consistent way. This makes data importing and exporting nearly impossible without custom code for each module performing decoration.

Fortunately, we don’t have to solve this problem on our own. Systems like Sun’s OpenDS have sophisticated, well-reasoned data models that allow decorators to elegantly combine to form coherent, node-like objects. OpenDS discusses its schema model on its wiki, and I’ll use it as my example.

The OpenDS schema contains basic layers:

Attributes, which are fields
Abstract and structural classes, which contain attributes
Object classes, which are the set of classes assigned to an object

Objects (which are node-like) can be assigned multiple object classes, each of which functions like a decorator. The objects may contain values (often even multiple values) for the attributes provided by their object classes.

Drupal modules would create attributes and directory classes, and Drupal core would contain a unified interface for assigning the classes to node types.

What this gets us:

Asynchronous multi-master replication support. Right now, node data is scattered all over the database, and there’s no way to “package” it for coherent, asynchronous replication across multiple hosts without a PHP-level implementation. In OpenDS, objects are fundamentally understood and managed by the directory’s data storage layer. It’s easy for it to replicate whole nodes.
Similarly, this ability to “package” objects gives us importing and exporting for free. OpenDS can import and export LDIF-formatted data, and this would allow nodes to be transported to dissimilar systems, even different directory servers. You would simply need the same classes supported on the destination system.
The “packaged” objects make sharding and partitioning data much easier.
We get tools like Apache Directory Studio that give a coherent object view, including the list of classes for each object. There’s no way to view a node in MySQL without a painful number of joins.
Built-in protection against namespace collisions for attributes.
We get unified indexing across decorators. Because decorators currently store their data in multiple tables, we can’t index, say, a city and the title for a node without denormalization. In OpenDS, you can create VLV indexes that span any set of attributes and selectable subsets of nodes. It basically allows creation of a comprehensive index for anything configured in the Views module. The only comparable features in relational databases are indexed views in SQL Server and materialized views in Oracle. MySQL does not support such indexes.
We can change a field from being single-valued to multi-valued without a fundamental change in the way we access the data.

I’ll be experimenting with using OpenDS as a node back-end in the upcoming weeks. It would be great to have a robust, multi-master, free/open-source node storage system.

We've arrived in Szeged for Drupalcon

Four Kitchens — Thu, 28 Aug 2008 04:03:16 -0500

After all flying into Europe on different days, we finally converged in Munich. From there we hopped trains to Vienna, Budapest, and finally Szeged. It’s a beautiful city and its modest size is a welcome change.

There’s more coverage to come, but for now check out David’s sessions that he’s presenting or co-presenting:

Indexes and denormalization: keys to scaling sites with massive content

RDF storage back-ends

Performance tuning expert panel

Photo by Todd Ross Nienkerk

Four Kitchens merch is here

Four Kitchens — Tue, 26 Aug 2008 13:42:26 -0500

Just in time for Drupalcon Szeged 2008, Four Kitchens has a few shirts and lots of stickers with our new logo to give away to our fellow Drupalers meeting up in Szeged. We’ll also be giving away some sweet Drupal stickers to put on your laptop, bike, dog, chair, or grandma. To get your hands on one, simply find us—before we find you. photo by Todd Ross Nienkerk

Drupalcon Szeged 2008 track icons preview

Aaron Stanush — Wed, 30 Jul 2008 12:00:00 -0500

Four Kitchens is proud to supply the icons for the tracks of this years Drupalcon conference. The tracks include:

Drupal core contributions and trends which will focus on Drupal 6 and 7, as well as killer and upcoming contributed modules with some spice added on trends we follow (like the semantic web effort).
Showcases, user experience and design will focus on the site builders, themers and those interested in delivering a killer user experience both for Drupal itself and for their own sites.
Code development is our hardcore developer track on topics like simpletests, coding practices, scalability, deployment and so on.
Growing Drupal will be our business and marketing oriented track with a good amount of local community building and focus on efforts such as the Knight Drupal Initiative, Google Summer of Code and GHOP.
BoF (Birds of a Feather) According to good tradition, Drupalcon will also have its full fledged (unmoderated) unconference in the form of BoFs.

When the icons are finalized, we will make them freely available to the public under a GPL license.

And to our fellow Drupalers out there, if you’re going to Szeged we’ll see you August!

Find out more about Drupalcon Szeged 2008.

BarCamp Austin 3 is this Saturday

Aaron Stanush — Tue, 04 Mar 2008 12:00:00 -0500

Four Kitchens is proud to help organize this year’s BarCamp Austin alongside our good friend Whurley and our new friend Richard Goodwin. If you’re going to be in Austin this weekend (either because you live here or you’ll be in town for SXSW) be sure to stop by GSD&M Idea City anytime on Saturday, March 8th. This year is going to be bigger and better with a lot more room for demos, presentations, and killer robots. This event is free, open to the public, and everyone is invited to participate! More info at barcampaustin.org.

Drupal 6.0 released!

Aaron Stanush — Wed, 13 Feb 2008 12:00:00 -0500

Here’s a few of the new features:

Drag-and-drop administration
OpenID support
Update status for core, modules, and themes
Theme developer module

Be sure to check out the full list of features and when you’re finished drooling, go download it!

Who is the best candidate for the open source community?

Todd Ross Nienkerk — Thu, 07 Feb 2008 12:00:00 -0500

whurley at BMC has posed an interesting question: Do the candidates have an opinion on open source software? Are they even aware of it? And if not, shouldn’t they be? So whurley created a poll to find out who we, the open source community, feel best represents our common interests. (Yes, “None of them” is an option!)

Assume for a moment that a knowledge of open source is prerequisite (or integral) in solving issues like patent reform. Who would you vote for? In other words, who would be the best candidate for the open source community?

Please vote! The more buzz we can build, the better chance we have to educate our elected representatives on the important, complicated, and philosophical questions surround open source, free culture, intellectual property.

Relaunching GetOpenID.com using a server-side Drupal module

Todd Ross Nienkerk — Thu, 07 Feb 2008 12:00:00 -0500

OpenID is blowing up!

This morning, the OpenID Foundation announced that Google, IBM, Microsoft, VeriSign, and Yahoo! have joined as its first corporate board members. But don’t worry: OpenID.net was quick to note that the members are not decision-makers:

While the OpenID Foundation serves a stewardship role around the community’s intellectual property, the Foundation’s board itself does not make any decisions about the specifications the community is collaboratively building. (Emphasis theirs.)

Well, we have an announcement of our own: We’re completely overhauling our OpenID registrar GetOpenID.com and expect to re-launch the site sometime in April 2008 — and we’ll do it on Drupal!

OpenID support for Drupal 5.x and 6.x is currently limited to clients. We plan to build a module to enable server-side support, allowing any Drupal site to become an OpenID registrar. The module will be released to the Drupal community under the GPL.

Viva free software and open standards!

Four Kitchens' new office open for business

Todd Ross Nienkerk — Fri, 18 May 2007 12:00:00 -0500

Four Kitchen Studios and That Other Paper have relocated to a new facility in North Austin at 183 and MoPac. Here are some pictures of the work in progress courtesy of Kristin (or can view the entire Flickr set):

An empty, soulless office pre-painting

Albert, Tim, and I discuss the finer points of laying drop cloths

Christine, Stephen, and Chad paint a wall green — the color of MONEY

Christine and Stephen pretend to like each other long enough to make a meaningful contribution to the greater good

An action shot of the paint drying

Excelsior!

David and I contemplate the sublime nature of Swedish office furnishings

That’s me. I’m working.

Four Kitchens builds Wikimedia's new fundraising system

David Strauss — Tue, 19 Dec 2006 13:00:00 -0500

What do you do when you’re one of the world’s top-ten websites, and you support 50 languages, 30 currencies, and four payment systems? And you have, at any given time, about 700 users hitting “reload” to find out up-to-the-minute results of worldwide contributions. You also need a meter to show progress on a site that serves 20,000 requests per second.

You’re Wikimedia, the charity behind Wikipedia.

In collaboration with Wikimedia’s technical staff and its community, Four Kitchens has built an integrated system to meet these demands. It’s called Fundraising C.O.R.E. (Central Online Reporting Engine).

Fundraising C.O.R.E. is built with Drupal. It uses the Drupal modules CiviCRM, i18n, and a custom module built just for this project. The custom Fundraising C.O.R.E. module integrates with CiviContribute’s fundraising management system to display privacy-conscious real-time data to community members.

With hundreds of contributions coming in every few hours, the small staff at Wikimedia doesn’t have the time to enter them all. For that Fundraising C.O.R.E. integrates with the instant notification features of PayPal and Moneybookers. In case anything gets lost between the payment processer and the fundraising system, Fundraising C.O.R.E. provides the ability to synchronize downloads of records.

SANE: Scanner drivers for MacOS X

Todd Ross Nienkerk — Wed, 06 Dec 2006 13:00:00 -0500

If you use an old USB scanner with a Mac, you’re probably unhappy with the out-of-date drivers offered by the manufacturer. (Chances are it runs in the “Classic” interface, famously replete with Chicago typeface.) Luckily, the opensource community is developing a solution: the TWAIN SANE interface for MacOS X. “SANE” stands for”Scanner Access Now Easy.” (Looks like someone put the acronym before the horse, if you catch my meaning.) I’ve installed it myself, and it works well enough. I have two major gripes, however: (1) the preview image does not automatically update as I adjust contrast and gamma, requiring me to sit through a “Preview” each time I nudge the settings; and (2) the gamma table interface is so geeked out that it displays the mathematical formulas of the curves instead of a human-readable legend. Here’s a quick installation guide:

download and install libusb
after installation, libusb will unpack sane-backends.pkg; install it
install TWAIN SANE Interface

SANE should now appear in whatever program you normally use to scan your junk. In Photoshop, you can find it under File » Import » SANE....

Read more about SANE project SANE-project.org.

That Other Paper (Austin’s ONLY Paper) launches!

Todd Ross Nienkerk — Sun, 03 Dec 2006 13:00:00 -0500

Four Kitchen Studios is proud to announce the launch of That Other Paper — Austin’s FIRST and ONLY publication!

<!—break—>
From our Declaration of Principles:

We will do our best never to bore our readers

Years of publishing experience have taught us a simple lesson that seems to evade most writers and editors: Readers have extremely short attention spans. What’s the value of a 10,000-word exposé on the backroom dealings of Senator McJerk if the reader is stifling yawns by the fifth paragr—

Oh, too late. She already turned the page.

We will infuse everything we do with a sense of humor. Every article, review, and blurb will tell a story. And we’ll always keep it short.

We will cover all forms of entertainment — not just music

Austin may be called the “Live Music Capital of the World,” but that doesn’t mean music is the only entertainment in town. Most notably, Austin has a vibrant live comedy scene on the cusp of national recognition.

Our restaurant reviews will be written with the average Austinite in mind

You aren’t rich, and neither are we. Most of us don’t have the cash to blow on so-called “fine dining.” So why waste your time reading pretentious reviews of stuff you can’t afford?

We’re gonna do things differently. If we can’t afford it, we won’t review it. Value — the quality of food and service we get for our money — will be paramount. And though we have a broad palette, we’ll deliver our reviews with a plain tongue. If the BBQ kicks ass, we’ll say so.

We’ll dig deeper, too. We’ll publish restaurants’ health inspection scores and interview the staff regarding the management’s treatment of its employees. (Prepare to be shocked at the state of Austin’s kitchens.)

Above all, we will serve you, the citizens of Austin

This is for you, Austin. You’ve educated, employed, and entertained us. Now it’s time we gave something back, and we know nothing better than entertainment.

Beware StockApplets.com!

Todd Ross Nienkerk — Mon, 02 Oct 2006 12:00:00 -0500

Recently, one of our clients wanted to install a stock ticker on her company’s website. It was a simple request; the only difficulty lay in finding a data feed of stock prices. After several hours of research, we found StockApplets.com, an online purveyor of Java-based stock ticker functions. We plunked down $50 and purchased an applet. Unfortunately, it didn’t work. Why? Because StockApplets.com doesn’t provide a data feed. We requested a refund. This was their reply:

Dear Sir Can you please let me know where we state that we are selling content?

Sarcasm. That’s a good sign, right? I asked about the status of the refund. They replied:

Dear Sir Your refund request is not accepted because the reason you are mentioning is not real. We are not stating anywhere on our site that we are selling content. For more details, please read our order page: [link removed]

I visited their order page. Since we had purchased the applet, StockApplets.com had revamped their website and added this disclaimer: [… P]lease note price does not include any data source, we only sell software, not data. In other words, if you’re hungry, they’ll sell you an empty box. But it’s your job to fill it with Lucky Charms. What inexplicable timing! I thought. I certainly didn’t remember seeing anything like that beforehand. I replied:

Why would you sell a product whose purpose is to deliver content, then refuse to deliver the content? Refund my purchase, or I will contact my credit card company and dispute the charge.

What’s the first rule of doing business? Tell the customer he made a mistake by purchasing your product! Their reply:

When you buy a car, do you expect the car company to offer you free gas? It is the same logic here. We state everywhere on our site that we are not data providers and we only sell tools. Please pay atention next time you make a purchase, we are not stupid nor full of money to pay for your mistakes. We are working very hard to develop tools and to sell them to make a living, and your complains are very ofending to our efforts.

The second rule of doing business is, of course, to tell the customer that his legitimate problems with your service are “offensive” to your efforts. After all, a company’s products should be met with unconditional support, right? HOW DARE YOU SAY OUR PRODUCT DOESN’T MEET YOUR NEEDS! WE WORKED HARD ON IT, AND YOU WILL LIKE IT! I decided they needed some good advice:

What’s offensive is your complete lack of concern for your customers’ satisfaction. The product you offered doesn’t meet our needs. We’re not using it, and we don’t intend to pay for it. You’ll have to deal with the credit card companies now. Is this really how you intend to operate your business? By making your customers angry and refusing to issue a refund for something that doesn’t meet their needs? This will be a very expensive fifty dollars.

That pretty much sealed the deal—or so I thought. I contacted our bank and requested a chargeback. A few minutes later, I received one of the most melodramatic, logically flawed emails in human history:

I asked Shareit to send you refund. I also think this might be a refund it will be very hard to bear for you. It bears all our work and hopes for a peaceful and better world for everyone that you distroyed. Because of you we decided to stop selling this product and we will probably stop selling all our stock based products.

I’m not making that up. According to StockApplets.com, I “distroyed” (sic) “all [their] work and hopes for a peaceful and better world for everyone”. And if I’m to blame for StockApplets.com shutting down—the obvious implication of that last line—then I’ve done the world a favor, and I should be greeted as a hero. Statues should be commissioned in my honor. Beware StockApplets.com. If you do purchase a stock ticker from them, you’ll need to supply your own stock data feed as well as your own programmer to write the code that can pull quotes from the web, scrape the data, and hand it to the applet for display. And by that point, you’re better off simply writing your own ticker. If that sounds like too much work, consider this: it’s better the burden of “destroying their work and hopes for a peaceful and better world for everyone”!

UPDATE: I’ve found proof that StockApplets.com did indeed market their product as offering a data feed. From their freshmeat.net page:

The financial market data is transferred to the applet using an HTTP client-pull connection. PHP and ASP scripts are provided that feed the applet with data from the Yahoo! Financial section. The data connection works without the need to modify proxy servers or firewalls.

Sounds like they used to support a data feed, doesn’t it?

'Alcalde' article highlights Four Kitchens' approach to creativity and publishing

Todd Ross Nienkerk — Tue, 29 Aug 2006 12:22:00 -0500

“Original Thought”
by Alcalde assistant editor Tim Taliaferro

Update: With Alcalde’s permission, I’ve posted the entire text of the article.

The Alcalde, UT-Austin’s alumni magazine, interviewed me about the formation of our company and our approach to working creatively. Wonderfully assembled by assistant editor Tim Taliaferro, the article explains Four Kitchens’ conception and philosophy (and why we’ll never again work a desk job!). Here’s a scan of the full article in PDF format: download “Original Thought” by Tim Taliaferro.

Tim did a great job assembling my meandering, self-indulgent ramblings into something approaching coherence. Here’s the full text, added August 30, 2006:

<!—break—>

Original Thought:
Four Kitchen Studios Offers Creativity Consulting

Todd Nienkerk lasted exactly one year and one day in the world of cubicles, staff meetings, and pressed khaki pants. Stifled by the doldrums of an 8-to-5er, he co-founded Four Kitchen Studios, a “creative collective” that does “anything and everything involving creativity and publishing.” He and his comrades are creativity consultants, hired guns sought for their technical skill and restless imaginations. They’ll spruce up your Web page, brighten your documents, and come up with that type of clever radio or TV ad that always makes people wonder, “Who thinks of this stuff?” Four Kitchen Studios operates as a hub for quirky people with specific skill sets, a one-stop shop for companies desperately in need of talent. Between the four founding members there’s a freelance writer, a computer scientist/political campaigner, a cartoonist, and a graphic designer/Webmaster/writer. Then there’s the talent pool — a small army of freelance writers, editors, designers, comics, ninjas, and artists with other full-time jobs but whom the founders can call on when a project requires their expertise.

Their collective inspiration for the company came from working on the Texas Travesty, UT’s humor magazine. The four met there as students and decided, over the many, many late nights they worked together, that it was basically what they wanted to do for the rest of their lives — have this sort of creative freedom and apply it toward something profitable, something that will pay the bills while they do the fun stuff.

What’s in a Name
Our first client was this guy out in Bastrop who had built a nine-bedroom, four-kitchen house in the middle of the woods. It was like, “Eccentric old man builds Ewok village.” That’s what this place looked like. Yet he left the place wide open, and no one had ever lived there. So the place was infested with mud daubers and all kinds of nasty little vermin. We were trying to strike a deal with this guy where we would do some Web design for him in return for free rent, and it turned into a really bad business deal. A really bad business deal. But we kept the name because it’s snappy. Mostly, 99 percent, it’s a catchy title for a company, and we really like it. One percent is: vet your clients, do a little bit of research, and don’t let them entice you with lame-brain schemes of “you can live in my house and do Web design.”

Collective Consciousness
If one person sits down and does something, it can usually be pretty good. But if that person is constantly getting feedback from other people who know what they’re talking about, it’s going to be excellent. And that’s a big part of what we’re trying to do. We’re drawing from that entire talent pool. Everyone there has a really good design aesthetic. They may not be able to do what needs to be done because they don’t know the software or they don’t know the technique. But they’ll know it when they see it. I trust everybody there to give me really good feedback. And if they don’t like something, they just say it. There’s none of this political couching. The way we approach it is, if something isn’t good, speak up now and be brutal. And if you can’t take it, you shouldn’t be here.

Motto
We don’t waste our time with figuring out what other people charge or how other companies go about doing their business. The motto that we’re all sort of investing in is: do what you love, and the money will follow. And so far, we haven’t had any trouble finding work. It just falls into our lap, left and right. I think it’s because people pick up on this vibe that we have as a company, that we’re all just bristling to do creative stuff.

Web Business
The importance of having an office with a lounge with sofas and receptionists and stuff like that is increasingly unimportant. However, I hope that we get to the point very quickly that we will have an office downtown with a receptionist and a waiting area, because when you reach that point, you’re a firm. For the time being, we are operating out of our apartment, and we do that to save a lot of money and, as a result, we can charge people a lot less. So I would say that the Web has almost eliminated the barrier to entry for a lot of creative and entrepreneurial types like myself and the people I’m working with.

One-Year Goal
Boy, I’m barely thinking two months from now.

Two-Month Goal
Doing pretty well. Self-sufficient. I guess a year from now I would like to see us in an office with creative space. My real inspiration as far as proof that this can be done is GSD&M. Either G, S, D, or M — I don’t know which one — spoke at my commencement ceremony when I graduated a year ago, and he just struck me as such a cool guy. [It was S — Roy Spence, BS ‘71.-Ed] And I was already sort of in awe of GSD&M and the fact that they were all recent UT grads who said, “You know what, we don’t want to work for anybody; let’s start our own company.” And against all odds, they did it. I very much see ourselves doing exactly that, but in a slightly different realm. They’re doing advertising, and we’re going to be doing publishing. I would like to turn our company into the GSD&M of the publishing business.

No More Khakis
I love the fact that now, not only can I work until I’m tired, and then go to bed and wake up when I’m rested, but I can now take off in the middle of the day. If I decide, “You know, I’m just not feeling it today, I’m not getting the kind of work done that I need to and there’s no use trying,” I’m going to go down to the pool at my apartment complex and hang out for a few hours and just relax.

Nudginess
This kind of business model entirely relies on people being completely self-motivated. And none of us are. So the idea that we are a collective and that we are all relying on each other provides a nudge. We are all, just by our mere presence in the same similar area, nudging each other towards, “Hey, let’s get this thing done. Let’s do this.” It’s nudginess. And it works. But we need to know each other really well. This extremely informal approach would not work in a company where people have been cobbled together from résumés.

What It’s Like to Love Your Job
I love my job. My job is my life. I wake up in the morning, and I sit down at my desk, and I just start to work. And I work until I can’t work any longer. Then, if I want to take the evening off and hang out with my fiancé, that’s just what we do. I’m the kind of person who loves taking work home with him if it’s good. I’ve always been that way. If I’m really into a project, and if I really enjoy it, it’s going to consume me. And that’s exactly what we’re trying to do. We’re trying to filter out all the garbage and take in work that will consume us.

Stirring Finale to Interview
What we’re trying to do is merge work and heart. Our attitude is: we’re going to do stuff and interact with our clients in a way that’s fun and different and totally not at all stodgy. Yet not sloppy. We’re T-shirt and jeans; we’re not stained wife-beater and cut-offs. We try to be fun but still be taken very seriously. I want people to walk away from dealing with our company saying, “They were a lot of fun to work with, and they were highly professional.” And that kind of mix you just don’t really see in the business place. To continue the kitchen analogy, it’s the pinch of humor, the zest, the dash — whatever your spice or seasoning metaphor is for such a thing — that’s what we are.

Originally published in the September/October 2006 edition of the Alcalde (vol. 95, num. 1). Words and photos by Tim Taliaferro.

Thanks, Tim, for a great interview. You’re doing great work at the Alcalde.

OpenID group on StumbleUpon

Todd Ross Nienkerk — Mon, 14 Aug 2006 12:00:00 -0500

Do you Stumble? Do you like OpenID? Do you love this cheesy, anaphoric introduction?

Even if obscure rhetorical devices aren’t your bag, you can still join the brand-spanking-new OpenID StumbleUpon group. StumbleUpon, in case you’re unaware, is a social bookmarking tool and Firefox toolbar with a very strong community backbone. You can post your favorite links, rank others’, and post message board-style in groups and on individual users’ pages. But here’s the best part: whenever you get bored, hit the “Stumble!” button in the toolbar, and you’ll be whisked away to a site recommended by users who share your interests.

Help us build OpenID awareness among StumbleUpon’s rapidly growing ranks! Download the StumbleUpon toolbar and join the OpenID group. Oh, and don’t forget to tell the Stumblers about the $5,000 bounties up for grabs…

Why do we all possess only half a brain?

Todd Ross Nienkerk — Sat, 12 Aug 2006 12:00:00 -0500

It’s an old question—if not a touch psuedo-scientific—but it points to a stark divide in human psychology: are you right-brained or left-brained?

Artists and scientists rarely share the same skin, and those who do are heralded as geniuses, eccentrics, or Renaissance ~~Men~~ Persons of Indeterminate Gender. This split is especially frustrating when working on the web, a sprawling medium that requires two very different talents: programming and design.

Now here’s a good idea: Programmer Meet Designer, a community site that introduces programmers to designers (and vice versa) to collaborate and build better websites. Here’s what PMD has to say:

This site was created to unite programmers and designers because rarely is a person good at both programming and designing. PMD helps programmers and designers partner up to make websites and web applications that look and work great. It also lets entrepreneurs and writers find people to work with.

It seems like a useful tool for finding the Yin to your Yang, the Bernie Taupin to your Elton John, or the Da Vinci to your, uh, Da Vinci.

(Of course, if you’d rather not hunt the web for your left- or right-brained counterpart, you can always call Four Kitchens instead. We pride ourselves, after all, in delivering the gamut of talent: from web to print, creative to technical! ;))

Source: Lifehacker

Olive Garden, Best Buy, and Coca-Cola 2.0?

Todd Ross Nienkerk — Fri, 11 Aug 2006 12:00:00 -0500

Corporate logos redesigned in Web 2.0 style

Brilliant. Kinda makes me wonder when the whole “web 2.0” thing will burn itself out. I mean, haven’t we collectively satisfied the web’s considerable thirst for shiny gradients and Trebuchet MS?

OpenID development bounties announced at OSCON 2006

David Strauss — Thu, 27 Jul 2006 12:00:00 -0500

Four Kitchen Studios is proud to co-sponsor over $50,000 worth of bounties to support OpenID development and inclusion in popular open-source Internet software. We’re contributing both in cash and in kind by designing the handouts for OSCON 2006 and writing the press releases. After OSCON, we’ve agreed to handle the site redesign for iwantmyopenid.org, the bounty headquarters.

I’ll also be personally travelling to Wikimania 2006 to distribute more handouts and answer questions about OpenID’s future in MediaWiki, the software that runs Wikipedia.

GetOpenID.com launch signals our full support of the OpenID standard

Todd Ross Nienkerk — Mon, 17 Jul 2006 12:00:00 -0500

We love OpenID. We’re so hopelessly, head-over-heels, eye-battingly infatuated with the possibility of single sign-ons (SSOs), in fact, that we’ve launched GetOpenID.com, our very own OpenID server and registrar. Oh, and we’ll use it to fully integrate OpenID with all our projects and contracts.

Six Apart (creators of TypeKey and Movable Type), LiveJournal, and VeriSign have already integrated OpenID, and the Wikimedia Foundation (creator of Wikipedia) has announced their intent to support OpenID (see the video).

So what is OpenID? It’s a decentralized identity management system that allows you to sign on to OpenID-enabled sites without having to register or create a new account. If you’re the type of user who comments or posts to several blogs, message boards, and wikis, you know what a pain remembering and managing dozens of usernames and passwords can be. With OpenID, however, you only need one username—your OpenID “identity”—and it’ll work on any site that accepts OpenID.

Still not convinced? OpenID allows you to delegate your identity to a URL simply by adding two lines of code to the header. Instead of getopenid.com/fourkitchens, I could log in to any OpenID-enabled site as simply FourKitchens.com, and Four Kitchens’ URL will appear as the byline for any post or comment I make. (Hey, marketing professionals: are the branding possibilities making you drool yet?)

Want to know more? Get your own OpenID, then check out OpenID.net for specs and documentation.

I Make Mistakes elevates schadenfreude to a new (more pronounceable) level

Todd Ross Nienkerk — Fri, 07 Jul 2006 12:00:00 -0500

scha·den·freu·de (n) Pleasure derived from the misfortunes of others.

Mistakes. Everybody makes ‘em. Everybody loves to hear about other people making ‘em. But why can’t we vote on them according to prevalence and severity? Why??

Calm down, imaginary person in my head! At I Make Mistakes, you can anonymously post your flubs, flaps, and fu—uh, faults—and rate those of others. Part group hug, part justcurio.us, I Make Mistakes is a social experiment-meets-confessional. It’s a window into the human psyche. No, wait—it’s a bathroom stall next to the window of the psyche that overlooks the confessional inside the social experiment.

Well, whatever it is, it’s totally anonymous, ad-free, and zero-profit. (And for you web geeks out there, we’ve integrated the OpenID single sign-on (SSO) standard.)

We’re adding several features in the coming weeks, including user moderation (to flag spam posts for deletion), work-safe mode, and sydication of your personal Mistake Threat Level on your website or blog.

Vote for I Make Mistakes on Listible:

Updates:

Dumping pure MVC for task flows

David Strauss — Wed, 24 May 2006 12:00:00 -0500

I’ve talked a lot about task flows on web sites but not much about how I’d implement them. Here’s a brain dump.

Here’s how I’d structure the human side of URLs:
http://example.com/verb_that_maps_to_task/options

This would generate the task flow, associate it with the user in the database, determine how the user can accomplish the task (if security allows it), and send the user to the first necessary step of the task. The new task would have a unique ID in the database, and the site would forward the user to a URL constructed for that task:
http://example.com/taskflow/task_flow_id

These URLs would also be linked from the task bar in the site’s interface. The user would click on the task bar to jump among independent task flows.

Tasks would be transactional. More technically, they would be ACID-like. At the site administrator’s option, tasks could expire, or users could have a maximum number of concurrent tasks.

Now, MVCs still have an abstracted role in this system. Task flows actually hop among controller actions, and many task flows would only need to use one controller’s action to finish. But as systems become more complicated, one controller’s action doesn’t always do the job. That’s where this system would shine.

How sessions and variable passing fail

David Strauss — Wed, 24 May 2006 12:00:00 -0500

Another common problem in web development is session management. Basically, web programmers have two choices, use global sessions or precariously pass variables between forms and links. Both have serious problems:

Global sessions
- Users can’t perform the same basic task in multiple windows on the same site simultaneously.
- If a user quits a task and returns to a page related to the task later, the system will show the old task’s information. This can be good, but it can also be confusing.
Passing variables
- It’s hard to constrain the user to a path where the programmer can control passing the variables.
- The number of ways programmers must use to pass variables is daunting. PHP’s non-cookie session method is a good example.

The solution is a true “task bar” (not in the Windows sense). The state of the user’s actions should be encompassed in tasks, and the tasks should be associated with the user. Clicking on a task would bring the user to the first incomplete step. Deleting a task would remove that branch of work. Ideally, the user could jump between tasks whenever, with the system saving any incomplete state. This would function well for single or multiple browser windows on the same site. And while variables would still have to be passed between immediate steps in a task, the programmer needn’t worry about the user getting off track, as the user would only need to click on the task to resume work on it.

Superscalar Task Management: A Different Approach to User Routing

David Strauss — Wed, 24 May 2006 12:00:00 -0500

Users and programmers don’t think in terms of means; they think in terms of goals. No one cares that they have to open Thunderbird, click Address Book, and then search to find a phone number. They just want to find a phone number. So, why are we programming and using web sites any other way? Right now, as programmers, we have to manually route users around a site. On the OpenID server I’m currently programming for Four Kitchens, routing a user through the authentication process is tedious. When a user first arrives to authenticate, he or she has one goal, authentication to the “OpenID Consumer” site. This goal has two dependencies, authentication with the OpenID Server and authorization of the OpenID Consumer site through the user’s trust roots. A user can have any combination satisfied. That means I have to consider four different routes (2^2):

State: User is authorized with OpenID Server
State: OpenID Consumer is under the user’s trust roots
Action: Pass-through
State: User is authorized with OpenID Server
State: OpenID Consumer is not under the user’s trust roots
Action: Authorize trust root
State: User is not authorized with OpenID Server
State: OpenID Consumer is under the user’s trust roots
Action: Authenticate
State: User is not authorized with OpenID Server
State: OpenID Consumer is not under the user’s trust roots
Action: Authenticate
Action: Then, authorize trust root (must be in this order)

If we had a superscalar-style task management system, we would have a set of tasks with dependencies. The system would be responsible for scheduling the dependencies to satisfy them. We can break authentication to an OpenID Consumer into three tasks (in no particular order):

Task: Authenticate the user to the OpenID Consumer site
Dependency: Authentication to the OpenID Server
Dependency: Authorization of the OpenID Consumer under the user’s trust root
Task: Authenticate the user to the OpenID Server
Dependencies: None
Task: Authorize the OpenID Consumer under the user’s trust root
Dependency: Authentication to the OpenID Server

But, unlike a regular superscalar architecture, the programmer would (as directly above) establish tasks, dependencies, and means of accomplishing tasks beforehand. The running system would only need to set goals. The system would even determine a viable, short task path to accomplish the goal. With such a system, authenticating a user to an OpenID Consumer would be simple. The programmer would establish the controllers (as in MVC) that accomplish the tasks. When a user needs to authenticate, the running system would merely add a goal (authenticate to a given OpenID Consumer) and tell the system, “Go.” No more complex routing logic in every controller.

Four Kitchen Studios formed

Todd Ross Nienkerk — Fri, 21 Apr 2006 12:00:00 -0500

It’s official: we’re a real company! Today I received a letter from the Texas Secretary of State announcing the formation of Four Kitchen Studios, LLC.

There’s much more to be done, of course, so stay tuned for many exciting announcements in the coming days and weeks.

So what exactly is Four Kitchens? We’re a creative collective based in Austin, TX. Our goal is deceptively simple: to create fun things that people enjoy.