Josh Benner

In Search of a Notes App

2017-09-16T00:00:00-04:00

Over the years I've used various apps to capture notes as I code, troubleshoot, prepare for moving to a new home, or plan a party. I continually have this dream that I'll some day find the perfect note-taking app that will satisfy all my needs in the multitude of scenarios in which I take notes... but to date that hasn't happened.

What I'm Looking For

Structured organization (notebooks, nesting, tags)
Outline-based note taking
Code highlighting
Markdown-capable (or similar)
Tables (preferably visually edited)
Diagrams (either markup-driven or with a diagram editor)
Basic spreadsheet/calculation features
Code evaluation
Image attach & insert
Sync-capable
Cross-platform
Open source is a bonus
Extensible
In-page table of contents
In-page and cross-page linking

What I've Used

Evernote

I used EverNote years ago. It was decent. It was also the only real game in town in my cross-platform existence. It barely checks a couple of my requirements above. So, once other apps started joining the party, I left EverNote in the dust, and I've only taken a peek every year or so to confirm it hasn't changed that much.

Workflowy

I used Workflowy back when it first launched, and for about a year after. I even used it at work with a team, and it was great. If you really like outline-based notes, it really nails that use case. That's about all it does, though.

OneNote

I've made very heavy use of OneNote, starting with when it landed on OS X a few years ago. It's fantastic. I love the notebook/tab/group/page organization. It's great for capturing images, editing tables, and will even occasionally do some basic math. I still use it for some things, but it is just terrible for technical notes.

Org Mode

I used Emacs and Org Mode steadily for two years while working on a Linux desktop every day, including its org-babel and a host of other add-ons. It's really incredible. Emacs and I don't get along on Mac for various reasons, so I haven't used Org Mode in the better part of a year. Org Mode is the gold standard in Literate Programming, outline-based notes, and organization in general. If I were writing a note app, I'd take heavy inspiration from Org Mode.

Quiver

I used Quiver briefly, but it deserves mention because it's very representative of the modern crop of programmer-oriented note taking apps that are catching headlines today. It's really a great little app, and the author is improving it. I'm going to keep my eye on it, but for now it lacks a few too many of the things that are important to me.

Boostnote

An open-source note taking app for programmers. Boostnote has caught my attention because it is elegant, has a nice mix of features, and is getting constant work that's painting a bright, extensible, open-source future for programmer-oriented note taking. This is the note app I'm trying to use now... and so far, it's a pretty pleasant experience. There are a lot of my boxes it doesn't check yet, but there are possibilities on the way as it approaches increased extensibility.

Honorable Mention: Jupyter et al

Jupyter, and projects that try to extend it like Beaker, are in a family of apps trying to solve problems for professionals dealing with large datasets. They are really impressive collections of technology, and solve their target problems well. I just can't help but feel like they are only a couple steps from being the most killer programmer-oriented, literate programming note taking apps in the world.

Diverging from Best Practices

2017-07-26T00:00:00-04:00

I have often used the phrase "Best Practice" to bolster my opinion about a technical approach, or been swayed by someone else's use of the term. There's probably a blog post worth of ranting about that on its own, but for now, I'd like to talk about diverging from "Best Practice".

The idea is that there is a set of "best" ways to do something, solve a given challenge, or handle some situation. I think it's important to draw a distinction here, though -- these practices are not the best because they're necessarily the best suited for every situation. These so-called best practices are usually the most widely accepted, officially approved, or initially intended approach.

Best Practices are often ignorant of context. Granted... they're usually still the best idea in most contexts; however, they shouldn't be applied blindly. Even if you're convinced you should adhere to a best practice, you should know why you are choosing to do so. What benefits does the approach offer? Why is it better than alternatives?

Asking these types of questions (and writing down their answers!) is a great way to always be clear on when and why you decide not to adhere to a best practice. Choosing to shirk best practice isn't an engineering sin... as long as you have good, documented, rational reasons. Like... really good reasons!

Work Around Docker Devicemapper Busy Filesystem

2017-07-18T00:00:00-04:00

If you are using Docker on Red Hat Enterprise Linux 7 (or one of its derivatives, such as CentOS 7), you may run into an error when you attempt to remove a container:

# docker rm my-container
Error: Error response from daemon: Driver devicemapper failed to remove root filesystem 4f872301a939db6eb8dd14d296bc152dbb97774e0da77e0b4585c06f7d4f5f1e: Device is Busy

I hit this and I went googling, to discover this GitHub issue, where people are working on a fix. The problem appears to be some sort of file or mount leakage out of docker into other processes running on the host. A little alarming!

It seems there is a temporary fix (though with some unclear impact). You can configure the docker daemon to run with an alternate mount configuration by placing MountFlags=slave in /etc/systemd/system/docker.service. It also sounds like they are trying to have this fixed in RHEL7.4 and/or the next Docker release. We'll see!

However, if you don't want to do that, there is a handy workaround:

Step 1: Find file hash

Inspect the container that refuses to be removed:

docker inspect 4f872301a

Look for the section that mentions the GraphDriver, which is likely to look something like this:

"GraphDriver": {
    "Name": "devicemapper",
    "Data": {
        "DeviceId": "23",
        "DeviceName": "docker-9:127-303967-c32dd5e9af3c0c15dc8c6ec476759a701619186845fde7638d0ffcfd93fb05ad",
        "DeviceSize": "10737418240"
    }
}

You are interested in the long hash at the end of the DeviceName attribute. In my example, that's c32dd5e9af3c0c15dc8c6ec476759a701619186845fde7638d0ffcfd93fb05ad.

Step 2: Identify Processes Hanging On

You need to find the processes that think they have that file open.

find /proc/*/mounts | xargs grep -E "c32dd5e9af3c0c15dc8c6ec476759a701619186845fde7638d0ffcfd93fb05ad"

Hopefully, you'll see a few lines like this:

/proc/23007/mounts:/dev/mapper/docker-9:127-303967-c32dd5e9af3c0c15dc8c6ec476759a701619186845fde7638d0ffcfd93fb05ad /var/lib/docker/devicemapper/mnt/c32dd5e9af3c0c15dc8c6ec476759a701619186845fde7638d0ffcfd93fb05ad xfs rw,seclabel,relatime,nouuid,attr2,inode64,logbsize=64k,sunit=128,swidth=128,noquota 0 0
/proc/23008/mounts:/dev/mapper/docker-9:127-303967-c32dd5e9af3c0c15dc8c6ec476759a701619186845fde7638d0ffcfd93fb05ad /var/lib/docker/devicemapper/mnt/c32dd5e9af3c0c15dc8c6ec476759a701619186845fde7638d0ffcfd93fb05ad xfs rw,seclabel,relatime,nouuid,attr2,inode64,logbsize=64k,sunit=128,swidth=128,noquota 0 0
/proc/23009/mounts:/dev/mapper/docker-9:127-303967-c32dd5e9af3c0c15dc8c6ec476759a701619186845fde7638d0ffcfd93fb05ad /var/lib/docker/devicemapper/mnt/c32dd5e9af3c0c15dc8c6ec476759a701619186845fde7638d0ffcfd93fb05ad xfs rw,seclabel,relatime,nouuid,attr2,inode64,logbsize=64k,sunit=128,swidth=128,noquota 0 0

Step 3: Restart or Kill the Processes

For each of the PIDs you find (the PID is just after /proc/ in the output), use ps aux | grep <PID> to figure out what's running at that PID, then restart or kill the processes as you deem appropriate. In my case, I had to restart a few random processes that had nothing to do with docker or the container that was refusing to be removed.

After that, another docker rm should do the trick.

Happy troubleshooting!

Merge Variables from Different Groups in Ansible

2017-07-13T00:00:00-04:00

Now and then it would be handy if Ansible let you merge variables from multiple sources. I had a use case where an ansible role would configure VRRP for floating IPs on servers. Most of the time, this works great: I set a variable or two in a group or host var context, and the role uses those to accomplish its purpose. However, I had a case where I wanted two floating IPs on one server, and these were two logically unrelated reasons.

Because I had two unrelated reasons for doing the same config, I didn't want to just mash the variables together -- which context do I pollute? Will I understand why I had to do that later? Yuck. I found myself coveting Chef's ability to set attributes that get merged. Could I do that with Ansible?

I had found some people with the same question in an Ansible GitHub Issue, where I came across an interesting piece of code by Leapfrog Online: an Ansible action plugin called merge_vars.

merge_vars lets you merge other vars with a common suffix into a new variable using an Ansible action, which is executed on the executor, not remotely. So, my VRRP variable setup was able to look something like this:

# nfs-cluster-a.yml
---
nfs_a_vip__to_merge:
  nfs_a_vip:
    virtual_router_id: 51
    shared_iface: "{{ ansible_default_ipv4.interface }}"
    auth_pass: "{{ vault_nfs_vip_pass }}"
    shared_vips:
      - "10.3.4.11"

# ipa-failover.yml
---
ipa_vip__to_merge:
  ipa_vip:
    virtual_router_id: 50
    shared_iface: "{{ ansible_default_ipv4.interface }}"
    auth_pass: "{{ vault_ipa_vip_pass }}"
    shared_vips:
      - "10.3.0.40"

I set floating IP configuration in two unrelated group_vars files.

Then, I was able to bring them together in a single play:

# floating-ips.yml
---
- hosts: all
  become: yes
  pre_tasks:
    - name: Merge floating IP configs
      merge_vars:
        suffix_to_merge: vip__to_merge
        merged_var_name: merged_vips
        expected_type: 'dict'
  roles:
    - role: uZer.keepalived
      keepalived_vrrp_instances: "{{ merged_vips }}"

Note the use of the merge_vars action. When this is called, it creates a new variable merged_vips, which is equivalent to if I had constructed it like this manually:

merged_vips:
  nfs_a_vip:
    virtual_router_id: 51
    shared_iface: "{{ ansible_default_ipv4.interface }}"
    auth_pass: "{{ vault_nfs_vip_pass }}"
    shared_vips:
      - "10.3.4.11"
  ipa_vip:
    virtual_router_id: 50
    shared_iface: "{{ ansible_default_ipv4.interface }}"
    auth_pass: "{{ vault_ipa_vip_pass }}"
    shared_vips:
      - "10.3.0.40"

Keep it Simple

2013-09-04T00:00:00-04:00

As a younger programmer, the complexity of a solution appeared to be a measure of the skill of the developer. The more complex, sophisticated, or enigmatic the solution, I reasoned, the more clever, skilled, and impressive the architect.

I learned the truth the hard way. Because when the time came to hand-off, enhance, debug, or even just re-understand a complex system a few months later... I found myself struggling against the complexity. It was too hard to load up an understanding of everything into my brain to form a mental model for moving forward.

Eventually I realized that the role of a good developer was not to develop complex solutions, but rather to develop simple solutions. In fact, the more simple the solution, the better.

Simple solutions have a lot of virtues. First, they are cheaper (easier) to explain, change, fix, and grok. Second, they tend to only do what is necessary, and nothing more. This helps reduce unintended consequences like regressions and bugs in the new functionality.

But the best thing about simple solutions is that they leave capacity for the system to do more. I'm not talking so much about technology resources as I am about cognitive capacity. If your system is constructed from a series of simple, minimally coupled subsystems, you can comprehend discrete chunks at a time, understand the operation of the whole thing working together, and dream up new capabilities with more ease and confidence.

A good developer is one who isn't happy until he's satisfied that he's found the simplest possible way to solve the problem he's been given. A good developer reduces complexity.

Blogging With Pelican

2013-09-02T00:00:00-04:00

As a web developer, I spend a lot of time with Drupal, which is what formerly powered my blog. Maintaining the site (which I generally neglect) seemed burdensome. Occasionally anti-spam would fail, cache files would need to be cleaned, or some other issue would require my attention. It all felt too much like my 9-5 work. So, I've converted the blog to use Pelican.

From what I can tell, Jekyll sparked renewed interest in static websites -- although static websites that are dynamically generated. Simply write static, semantic content, and have some scripts pump out HTML. The benefits are reduced complexity, increased security, increased performance, and so on.

Many similar systems have popped up. They all have similar capabilities and varying degrees of complexity. Many have poor or modest documentation. Jekyll is built in Ruby, as are several other static blog generators. However, I haven't been into ruby much... but rather Python! Turns out Python has a plethora of options for static site generation. After a brief survey of the available options, I decided to go with Pelican.

Pelican has a nice balance of simplicity, capability, readable (and extendable) code, and contributed content (such as theme and plugin repositories).

Getting started with Pelican was pretty straightforward. I invested most of my time into porting my previous content (which needed a little massaging), and getting some additional features working properly, such as thumbnails and lightboxes. I've heaviliy adapted a contributed theme and I've put in place commenting and other forms. All-in-all: I like it!

Blogging with ReSTructured Text, storing the content in source control, and extending functionalities with simple methods just feels right. More right than depending on a heavy, dynamic site.

Links:

Prevent "AHAH the Right Way" in Drupal from Breaking with Validation

2010-11-05T00:00:00-04:00

So there has been a lot of blogging and documentation in the past about the "right" way to do AHAH in Drupal. I think these people make excellent points, provide good documentation, and are pushing these types of dynamic interactions in Drupal in the right direction. I think a lot what's out there about this "right way" to do AHAH in Drupal misses some critical issues, though. Maybe someone else has blogged about it already, but I'll share one pitfall that I learned to avoid while doing AHAH.

Specifically, if you have a form using AHAH, there are a series of events that can lead to the form breaking completely:

Load form
Activate AHAH element (which calls AHAH callback, rebuilds form, then re-caches form)
Submit form with an error
Validation reloads the page with a re-rendered copy of the form and errors
Fix form errors, and submit form
BOOM!

What happened? The form submits and you probably get a load of JSON output, and your browser's URL is at the AHAH callback location. WHAT?

So, what happened is this: during the AHAH callback, the form is rebuilt. However, during the form rebuild, the #action element on the form is set to the default... which is the current request URI, which will be the path to the AHAH callback. This is bad, because now when you have a form validation error and the form is re-rendered in its entirety (as opposed to just using a section of re-rendered form as during the AHAH callback), the form now has its #action property set to the path of the AHAH callback.

That is bad. There have been mentions of how to fix it, but here is how I did it:

So there is this semi-utility-function-like block of code that AHAH the "right way" depends on existing in the AHAH callback, which looks something like this:

// From http://drupal.org/node/331941
$form_state = array('storage' => NULL, 'submitted' => FALSE);
$form_build_id = $_POST['form_build_id'];
$form = form_get_cache($form_build_id, $form_state);
$args = $form['#parameters'];
$form_id = array_shift($args);
$form_state['post'] = $form['#post'] = $_POST;
$form['#programmed'] = $form['#redirect'] = FALSE;
drupal_process_form($form_id, $form, $form_state);
$form = drupal_rebuild_form($form_id, $form_state, $args, $form_build_id);

That final call to drupal_rebuild_form() does a couple things: it rebuilds the form array, and it caches the result. This means that we can't just change the $form variable here after our call to drupal_rebuild_form() -- we need to make our form generator function smart! To do that, I made an important change to the above code:

// From http://drupal.org/node/331941
$form_state = array('storage' => NULL, 'submitted' => FALSE);
$form_build_id = $_POST['form_build_id'];
$form = form_get_cache($form_build_id, $form_state);
$args = $form['#parameters'];
$form_id = array_shift($args);
$form_state['post'] = $form['#post'] = $_POST;
$form['#programmed'] = $form['#redirect'] = FALSE;
// Stash original form action to avoid overwriting with drupal_rebuild_form().
$form_state['action'] = $form['#action'];
drupal_process_form($form_id, $form, $form_state);
$form = drupal_rebuild_form($form_id, $form_state, $args, $form_build_id);

Notice that I stashed the form's original #action property (as loaded from the previously-cached version of the form) in the $form_state variable. Then, when drupal_rebuild_form() goes to work, this little bit of code in my form function makes sure that my #action is correct through AHAH+Validation:

$form['#cache'] = TRUE; // Make sure the form is cached.

// Pull the correct action out of form_state if it's there to avoid AHAH+Validation action-rewrite.
if (isset($form_state['action'])) {
  $form['#action'] = $form_state['action'];
}

Now when the form is rebuilt during my AHAH call, it explicitly sets the #action property, which prevents the #action overwrite when drupal_prepare_form() (which is called by drupal_rebuild_form()) adds the result of _element_info('form') to the form (adding an array to another will add keys from the second array to the first, but not overwrite keys in the first that are also present in the second -- so $form['#action'] being present already prevents it from being overwritten).

Quick Tip: Detect and Encode Curly Brackets in URL Validation

2009-10-26T00:00:00-04:00

Validating user input is always a great idea from a usability and security point of view. However, when it comes to things like URLs, the data is complex and there is a very strict pattern that the data has to adhere to. From a data perspective, this is great news, since we can validate for what we want, not try to detect what we don't.

However, a lot of modern URLs don't always do a great job following RFC 1738. Specifically, I'm looking at you .Net guys who insist on putting UUIDs wrapped in curly brackets in query strings and the like. According to RFC 1738, curly brackets are "unsafe" within URLs and should be encoded to their URL-encoded entities.

So, technically, curly brackets are fine in URLs (if encoded), but when a user pastes their URL with curly brackets into your site and you pass it through your likely regex-based validation algorithm, you are likely to experience a validation failure since the curly brackets aren't allowed. Now, sure, you could pass the entire URL through something like urlencode() or rawurlencode() -- but that encodes everything! What I do is simple: I just replace any brackets and then continue on my merry way:

$replace = array('{' => '%7B', '}' => '%7D');
$newval = str_replace(array_keys($replace), array_values($replace), $user_url);
if (!my_url_validation_func($newval)) {
  my_error_function("Hey, you need a valid URL!");
}

If you're saving the URL, you should probably save the encoded version, but you don't have to, as long as you adhere to RFC 1738 on output (although most modern browsers are fine if you don't).

I've found this especially useful while doing URL validation in Drupal, as its built-in (and contributed) URL validation routine all seem to be pretty adherent to RFC 1738.

Quick Tip: Get proper DOCUMENT_ROOT When Using mod_vhost_alias

2009-10-25T00:00:00-04:00

The Apache module mod_vhost_alias and its VirtualDocumentRoot directive can really be a great time saver for local development (some googling will explain why in more deapth). Basically, my local dev is set up so that I just have to create a directory in my aliases directory, and I just then navigate my browser to a URL matching the name of that new directory, and apache knows exactly what to serve automagically.

However, there are a few evil gotchas when using mod_vhost_alias, one of which is that the PHP global $_SERVER['DOCUMENT_ROOT'] remains set to the apache default DOCUMENT_ROOT environment variable rather than being re-assigned to the document root activated by the VirtualDocumentRoot directive for the current URL. This can cause some PHP applications (that are too trusting) to die for one reason or another.

I found a great solution to this in the related apache bug report: Simply add the following line to your apache configuration inside the VirtualDocumentRoot vhost definition:

php_admin_value auto_prepend_file /path/setdocroot.php

Then, create the referenced PHP file, and put set this as its contents:

<?php
$_SERVER['DOCUMENT_ROOT'] = str_replace($_SERVER['SCRIPT_NAME'], '', $_SERVER['SCRIPT_FILENAME']);

Now, every page load has this file executed, which properly sets DOCUMENT_ROOT.

Review: NetBeans IDE 6.7.1

2009-10-16T00:00:00-04:00

I think my previous posts have already indicated that my development environment, specifically my IDE, is important to me, and that I make a habit of exploring my options on a regular basis. For the last 7 months I've been using NuSphere's PhpED, which I've really enjoyed. The only things that I've been just a little dissatisfied with is that it only runs on windows and that its window arranging capabilities are not as robust as Eclipse. But, having seen a few tweets about NetBeans, I decided to visit this old acquaintance once again.

I had used NetBeans ages ago when I was learning a little bit of Java. It was good enough... for Java. Then, some time ago I had tried NetBeans 6 (I don't recall which minor version), because I learned they were working on building a PHP IDE out of NetBeans, which intrigued me. At the time, I simply felt NetBeans didn't stack up to Eclipse, and I hadn't yet discovered PhpED. However, with some recent twitter traffic about NetBeans, I figured I'd give the latest incarnation a spin to kick the wheels a little.

For a quick summary of what I look for in IDEs and how I evaluate them, see my review of PhpED. Based on that, I'll try to be succinct:

Customizable Interface - NetBeans does well here, allowing me to dock panes, minimize things for single-click fly-out, and even position code windows next to eachother. Also nice is the ability to "undock" a window to have it handled by the window manager as a separate window appearing in the task bar. NetBeans beats PhpED (which can't put code windows next to eachother) and Eclipse (which treats side-by-side code in a silly manner) in this category. Only downside is that dragging windows can be a little awkward compared to the other IDEs.
Stability - NetBeans has yet to crash for me on Ubuntu 9.04 x64 or Vista 32-bit (used heavily on both).
Fast - Startup takes a little, but that's faster than Eclipse, and doesn't bother me. Normal operation is good. Scanning projects to build auto-complete and such is faster than Eclipse (though hideously slower than Eclipse when scanning files on a Samba share), though slightly slower than PhpED. Speed of UI is good enough. Overall, PhpED wins in this category, but NetBeans is "good enough" to be perfectly usable.
Project/File Management - NetBeans has the all-too-familiar project management pane. It has some nice features, good integration with the language, and nice right-click options. Nothing special, and it works well enough. IDEs tie here.
Syntax Highlighting - Highlighting is fast and modestly customizable -- easier, I might say, than Eclipse. Add this to the ease of importing/exporting segments of settings, and you NetBeans just barely beats PhpED here as well. Only thing extra I want in NetBeans is separate highlight category for built-in PHP functions.
Real-time, Remote Debugging - NetBeans works very well with Xdebug, using the IDE-triggered debug session model. I feel it does this slightly better than Eclipse, which always wants you to be creating debug profiles. Also, NetBeans path mapping has yet to get confused, while I'm always fighting with Eclipse in this regard. NetBeans is not as easy as PhpED with debugging. PhpED still wins here, but NetBeans is a close runner-up.
Configurability - I haven't hit anything yet I wanted to configure but couldn't in NetBeans. IDEs tie here, but kudos to NetBeans for having the simplest yet still powerful configuration UI.
Version Control Integration - NetBeans has a pluggable version control system and comes bundled with several widely-used VCS modules, including Subversion, which is my poison of choice. SVN module uses native binaries and is fast. VCS UI provides nice options and gives you pertinent info. NetBeans definitely beats PhpED's cop-out approach to VCS, but loses slightly to Eclipse with its better file "decorations" and full-fledged SVN browser.
Other Integration - NetBeans has database integration, but it's clumsy and difficult to use if you work with a lot of variable databases (as I do). I honestly don't know about NetBeans' abilities with things like SSH or [S]FTP as I haven't tried to find them. PhpED wins for DB integration, and Eclipse wins for other integrations. I may stop considering this since I use external tools for most of this stuff now (mostly command-line).
Updates - NetBeans hasn't been updated since I've been using it, but it's Open Source and seems to be building in popularity (just my perception). PhpED wins in this category, as they update very regularly.

All-in-all, I've come to be comfortable using NetBeans as my day-to-day environment. However, I keep PhpED open on my windows box to handle debugging, since PhpED just rules in that category still. Honestly, if PhpED had a Linux version comparable to its Windows version (PhpED for Linux is out of date), then I might abandon NetBeans. But, for now, NetBeans has earned a place in my workflow.

Fix mod_auth_mysql on Ubuntu 64-bit

2009-09-13T00:00:00-04:00

At least twice, now, I've encountered problems (read: apache segfaults, oh noes!) with mod_auth_mysql on Ubuntu 64-bit installs, most recently using Jaunty Jackalope (9.04) -- the most recent release (at least until 9.10 comes out soon). In searching for solutions, I discovered that this issue has been reported at least a couple times, the issue has been identified (and is a super-simple fix), and a working patch exists... but as of the writing of this article, the fix has not yet been compiled into the repo packages.

So then this becomes one of those relatively rare times that I will compile my own package. I prefer to compile binary packages (deb files) rather than simply compiling from source and doing a "make install," since packages leverage the package management system for things like dependencies and upgrades. For instance, if Ubuntu does release the fix in the next version of mod_auth_mysql, I want to be able to seemlessly upgrade to the next version.

I'm going to describe the steps to fix mod_auth_mysql 4.3.9-11, but this general process should work for applying community patches and building deb files in general.

Uninstall mod_auth_mysql:

$ sudo apt-get remove libapache2-mod-auth-mysql

Get mod_auth_mysql source (drops it in the current directory, so be where you want it, such as ~/src):
```
$ sudo apt-get source libapache2-mod-auth-mysql
```

Get dependencies for building from source:

$ sudo apt-get build-dep libapache2-mod-auth-mysql

Get a dependencies that didn't install with build-dep:
```
$ sudo apt-get install apache2-threaded-dev
```
(could be different for your system)
Get devscripts package to make this all much easier:
```
$ sudo apt-get install devscripts
```
(installs postfix -- annoying)
Set file perms on the source files you downloaded:
```
$ sudo chown -R josh:josh *
```
Enter src dir:
```
$ cd mod-auth-mysql-4.3.9
```
Prepare to apply patch. This actually starts a new shell with the current directory containing a copy of the code so you can edit it.
```
$ dpatch-edit-patch -a fix_segfaults
```
Apply patch, a one-line edit to mod_auth-mysql.c described here.
Exit the shell.
Add fix_segfaults to the end of debian/patches/00list.
Record the changes and change the package version info. Change the version at the top of the file. I usually just add something at the end of the existing version string.
```
$ dch -i
```
Build the deb file. This will compile the source, package the binaries, and put the deb file one directory up.
```
$ dpkg-buildpackage -rfakeroot
```
Install the new package. Note: Your deb file name will be different based on how you changed the version string above.
```
$ sudo dpkg -i ../libapache2_mod_auth_mysql-4.3.9-11_amd64.deb
```

Seems like a lot, but after you do it, it doesn't seem too bad, and you can bask in the glory of creating your own package -- which is reusable on all same-type systems.

Windows Home Server FTW

2009-08-25T00:00:00-04:00

I tend to consider myself a technical user, and for years, I operated various Linux distributions and setups to facilitate our home network. Mostly this meant a place to dump our backups in an intelligent manner, but I would also setup things like easy file sharing between our computers and even get fancy sometimes and so music servers or video streaming (though we never actually used any of that). However, after years of putting a lot of effort into maintaining that type of setup (keeping things organized, re-configuring everything after a workstation re-install, reconfiguring the server after a Linux re-install, etc), I decided I wanted something simple, that "just works." To be fair, part of my problem with using Linux was that the Linux box doubled as my development and testbed server, so I was always tinkering with it. Instead, I needed a box that would serve our family home network and not be my tinker box.

Enter Windows Home Server.

Since I had extra hardware, I decided to get a copy of WHS and install it myself. This works just fine, though the install takes just about forever since WHS is basically a cleverly hacked and enhanced version of Small Business Server 2003. WHS mostly sells pre-installed on hardware, such as HP's MediaSmart server line, so my bet is Microsoft isn't too worried about the install experience. Though, other than the time taken, the install was easy and seamless.

Once everything was installed and I ran through windows update (got the power packs, too), I set about linking the computers on our home network to the server. This process was pleasantly simple, requiring that a small installer be run. The installer was quick and, might I say, "just worked." It finds your WHS on the network, helps you manage your password sync between computer and server, and places useful but unobtrusive shortcuts and a tray icon on the workstation. I did this with three Vista machines and an XP machine, all without a hitch.

Once a machine is connected to the WHS, you can use the convenient WHS console (WHS is designed to operate headless, BTW), which is basically the console application piped through a terminal services window, or you can login via terminal services the "normal" way. In the console, you can configure the server, see connected computers, visualize hard drive usage, and configure the automated full-computer backups.

Yes, automated, full-computer backups, a-la time-machine (though not hourly, which really isn't a big deal to me at all). Essentially, when you configure a computer to be backed up by WHS, every day during the time window that you configure, WHS will pull a complete backup of your machine. But, WHS is smart about it, and just does differential backups, meaning that while the first backup might take quite a while, subsequent backups are much faster (just a few minutes usually). I have my network configured to backup to the WHS every day between 1am and 6am. It keeps a week of daily backups, and several weeks of weekly snapshots. So, should a hard drive fail in the family, WHS has all the data... and I mean all the data, including the windows install, applications, everything that's not specifically excluded by me.

If this was all WHS did, in my opinion, this would be worth the $100. But, consider the growing multitude of add-ins you can acquire for WHS, and the value continues to grow. On my list of personal favorites is an online backup service called KeepVault. For $100/year, we get 100gb of backup space to which WHS will automatically backup any of the shared folders we choose. We keep all of our multimedia files on the WHS (and access them via mapped network drives), so the moment we import pictures from our digital camera, they are on the server, being backed up securely online. Perfect!

I could continue to go on about other capabilities, but I don't want to sound like too much of a fan boi at this point. In short: I recommend WHS because it just works and offers key features that make keeping your home network running a breeze.

Drupal Performance on SliceHost with Boost Module

2009-08-21T00:00:00-04:00

If you know me, you know that I work on a lot of Drupal sites. You might also know that SliceHost (now a part of RackSpace) is one of the most affordable and simple VPS hosting solutions out there. That said, you might not know that I run several of my own Drupal sites on a single SliceHost 256 slice (256mb ram, that is).

Since I put up my personal blog site using Drupal on a slice, I've noticed that the site, despite being very straightforward as far as Drupal sites go, was not performing very well (3-5 second page loads were common, 400-500ms before initial HTTP response codes). So, I decided to take a closer look to determine what was going on and try to resolve the performance issue.

The first thing I did was to create a duplicate instance of my site on my local Linux development box. I was mildly surprised to find that the local instance of the site was blazingly fast. Understanding that something was causing poor performance in the live version, I continued on and ran an execution profile of a few pages of the site... no glaring performance issues compared to typical Drupal site execution.

That lead me to suspect SliceHost itself. As a virtual server, the system naturally shares physical resources with several other virtual systems. In addition, the CPU and disk IO resources that the VPS has access to are probably highly dependent on the activity of the other virtual servers on the physical server. After watching top on my VPS for a while and reading the observations of others using SliceHost, I came to the conclusion that a good portion of my problem was likely due to the IO delays when the web server was trying to load files or the DB was reading/writing to the disk for whatever reason.

Among the other things I already know are these things: I dont' have the time or inclination to switch to a different VPS vendor, and I don't have the money to pay for more than a 256 slice. So, my solution was to try to work around the issue... using the boost module.

The Boost module is a project to add a static page cache mechanism to Drupal. While Drupal core includes a database-based cache for anonymous users, sometimes performance gains can be realized by the web server loading only a single, flat HTML file instead of executing all sorts of PHP, hitting the database a bunch of times, then outputting the resulting HTML. If the output will be the same each time anyway, a static page cache makes a lot of sense.

Installing boost on Drupal 6 wasn't too bad. Its configuration pages can be a little overwhelming to parse at first, but once I started to gain an understanding of what it was trying to do, things started making a lot more sense. Specifically, boost not only has a static page caching mechanism, but it also features a cron-based crawler that refreshes cached pages as well as pages specifically flagged for cache generating (using boost's block) to keep the static cache nice and preemptively fresh.

Once I deployed boost and made the requisite configuration changes, I re-ran my performance tests (me clicking around with YSlow). Instead of seeing the average page load times around 3-5 seconds, pages (once cached) were loading in less than a second (0.5 - 0.9 seconds specifically). During first-time cache generation, I still see some higher times -- but since I've got my cron-based boost crawler doing the work in the background, most pages should render rather quickly now.

Review: NuSphere PhpED 5.6

2009-03-20T00:00:00-04:00

As I explained in my previous post, I've been on the hunt for a new PHP IDE. During past searches for this type of tool, I've always taken a glance at NuSphere's PhpED and usually went away screaming rather quickly because of how clunky, ugly, and just plain annoying the interface was (keep reading, though, they fixed this!). My last look at PhpED was verison 5.2 quite some time ago -- it was a good improvement over previous PhpED versions, but it couldn't compete with the Eclipse-based options (at least in the terms that mattered to me).

Which brings me to an important note: what I look for in an IDE is probably not exactly what another coder will look for in an IDE. From what I've experienced, choice of tools for most coders is 50% features/capabilities and 90% preference (oh hai, math!). For me, here is a list of features that are important:

Clean, tabbed, highly customizable interface
Stability
Fast (operation, not necessarily start-up)
Good project/file management
Good/customizable syntax highlighting
Real-time, remote debugging
Code insight/navigation/etc
High degree of configurability (keyboard shortcuts, behavior of UI elements, etc)
Version control integration (SVN specifically)
Integration with other parts of my workflow (SSH, MySQL, [S]FTP, etc)
And as we saw with Zend, update regularity is important to me (especially if I pay for it)

My First Day with PhpED

I went to NuSphere's site, which still feels a little underwhelming for some reason, and downloaded, installed, and activated the PhpED Professional trial. PhpED installed easily and quickly, and getting an activation code for the trial was simple and fast.

PhpED Debugger in Action

I knew I wanted to be able to do some real-time debugging on the Linux box under my desk, so I followed NuSphere's instructions on how to install their DBG debugger extension for PHP. I was pleased that their debugger installed without any trouble on PHP 5.2 on my Ubuntu 8.04 server. When starting a new project, you drop a dbg-wizard.php file into the docroot and PhpED test debugger functionality, providing you with feedback on any changes you should make to your server configuration -- such as disabling APC. I was impressed at the ease of the debugger setup. Oh, and compared to Zend's remote debugger, DBG is fast.

PhpED New Project Wizard

Next I wanted to pull my existing projects into PhpED. I work on a Vista machine that mounts a samba share containing the various docroots from my local Linux web server so I can edit files and test the local version of the site immediately without any bothersome file transfers to interrupt the process. PhpED's workspace/project setup easily and specifically accounted for this arrangement with some simple questions during the new project wizard. I just told PhpED that I work with local files that the web server also has access to, gave PhpED details about the docroot and URL, and PhpED pulled in each project tree flawlessly.

My next stop was syntax highlighting. During the very brief sting that I worked with TextMate, I fell in love with their default syntax highlight color schemes because they are easy on my poor coder eyes. PhpED's syntax highlight settings were easy to find and configure, resulting in a code interface that I was familiar and comfortable with.

PhpED Pop-out Windows

One of the things I love about Eclipse is the incredibly customizable nature of its window interface. You can dock just about any window anywhere in the IDE, collapse them, temporarily expand them, split views -- just about anything I want. While Eclipse has spoiled me in this regard, I was pleased to find that PhpED is fairly customizable as well. While PhpED can't do everything Eclipse can as far as customizing the interface, PhpED does enough for me to get the interface arrangement I like (only code is visible with pop-out tabs on either side for supporting features like project navigation, reference, etc).

PhpED doesn't include SVN integration out-of-the-box, but their workspace/project navigator provides easy access to the Windows shell menu. This means that since I have TortoiseSVN installed, I was able to easily work with my SVN working copies. While I don't feel that this is an ideal arrangement, it works and doesn't slow me down. I'd like to see tighter integration between PhpED and SVN -- even if it's in the form of more tightly integrating with TortoiseSVN. While I haven't tried this yet, it looks like there may be ways to get some tighter integration now.

PhpED Terminal Window

Another way in which I'm spoiled with code editors is how some will do little but meaningful things to help you code faster. Of particular interest and use to me is when the editor will auto-close brackets, parenthases, and quotes, then intelligently handle the code when you close them yourself (ie: not adding any additional, unwanted closing characters). Zend Studio does this fairly nicely (though it gets confused sometimes), and a lot of PHP editors out there that I've tried make an attempt at this but end up being more annoying than with the feature off. PhpED, however, has implemented this type of code typing help very well. PhpED intelligent closes and manages brackets and such, pops up useful help when typing a function, and has fast, effective completion options. PhpED for another win here.

PhpED Database Client

I was pretty happy with PhpED having discovered all the things I've already covered... but I became somewhat thrilled with PhpED when I discovered its DB Client and Terminal windows. PhpED lets you easily connect to many databases, browser their structure and data, and even directly query them right from within the IDE. Their terminal integration allows you to SSH (or telnet) and have the terminal window exist seamlessly inside the IDE as well. To be fair, Zend Studio has these features as well in its Remote System Management module -- but that feature is clumsy, hard to use, and often doesn't quite work right. PhpED's implementation of these features, however, is sensible, fast, and easy to use.

Conclusion

I'm pleased to say that through my whole week working with PhpED, I continued to discover new little shortcuts and features that made my life as a coder just a little bit better. PhpED has some very minor annoyances (ie: left arrow at beginning of line does not move cursor to end of previous line UPDATE: you can configure this to work like other editors!), but these are things that I've been able to quickly overlook. At the end of the week, I'm very happy with PhpED -- in fact, I think I'm almost ready to give NuSphere my money.

Summary

I kept a text document open all week and have collected pros, cons, better than Zend Studio for Eclipse (ZSE), worse than ZSE, nifty, and silly features.

Pros:

Syntax highlight pretty customizable
Decent filesystem-based workspace/project management
Configurable shortcuts
Configurable IDE layout (can hide stuff for maximum code area)
Smart handling of brackets, parenthases, quotes, etc
Excellent debugger/profiler that just works
Intelligent project configuration
Good external tool integration
Excellent DB client
Excellent terminal/SSH client
Excellent and extensible help integration
Find declaration
Shows PHPDoc data
Code folding that doesn't make me want to shoot myself
Good code navigator/explorer
Handy features available on side of every editor tab
Floating of tabs is handy, esp. with multiple monitors
Fast
Pretty configurable (lets me turn annoying stuff off)
Lets you scroll past EOL
Intelligent PHPDoc start (auto-populates param statements)

Cons:

No plugin system that I can see
Would rather see built-in SVN support than relying on Tortoise
No line number color change on changed lines to show diffs from saved version
No X on tabs
Can't lock toolbars
No code formatting? Maybe I just haven't found it?
Behavior of unpinned windows seems a little off
Left arrow at beginning of line does not go to end of previous line

Better than ZSE:

Faster refresh of code warnings/errors
Syntax highlight settings doesn't mess with popups
CSS editing faster, better
Debugger install was relatively easy and worked the first time

Worse than ZSE:

No plugin repository for adding nifty features

Nifty:

Dynamic highlight mode

Silly:

"Full Screen" mode
Popup PHPDoc help when using a function destroys PHPDoc formatting (indents, etc)
Popup function help doesn't show defaults used in function declaration

Disappointed with Zend

2009-03-19T00:00:00-04:00

Every few months as I sit at my desk and code away, I can't help but wonder to myself, "Is there anything better out there?" I'm not being metaphysical or introspective -- I'm talking about my IDE. Most coders I've met are the same way: we find a tool we like, use it for a while, but usually end up wanting something more or different. Perhaps discontent is a core part of human nature, but every few months I go looking for better tools.

I've been happily using Zend Studio for Eclipse (ZSE) for just over a year now. I had loved Eclipse + PHPEclipse for a long while before moving on to Eclipse + PDT, and had finally paid for ZSE to get a few extra features, support, and install packages that didn't take me hours to assemble and install. I've grown to know and understand Eclipse well, which is part of why I've been using Eclipse in one form or another for about 4 years now.

In truth, I was fine using ZSE for now. Work is keeping me busy, there is plenty to do at our new house... honestly, I didn't care to spend time looking into another solution for my daily-use PHP IDE. Then my ZSE license expired, and while I can still use ZSE, I cannot get any updates or bug fixes until I pay Zend's extortion fee of $200.

Let me be clear: I understand and accept paying recurring licensing fees. Except that in a year, Zend has only managed two updates that pass for anything more than the most minor bug fixes. And $200 for yearly maintenance for a product I bought for $300 a year ago (they were running a promo) just seems too high.

If Zend was offering me some great new feature, or even fixes to all of the mild annoying bugs that run a little rampant through ZSE -- I'd pay the fee! Yes, even $200! However, given the current state of ZSE and the paucity of updates from Zend, I'd have to strongly consider even putting $100 toward ZSE at this point.

Why I Cancelled My Mozy.com Subscription

2009-03-02T00:00:00-05:00

Mozy is an online backup solution that has grown in popularity recently. My wife and I have a lot of pictures, music, and generally important files on our computers that we know need to get backed up in a reliable and secure fashion. Mozy seemed like a natural choice.

Now, my profession involves computers, servers, backups, etc. Our home setup consists of my main Vista workstation, my Vista laptop, my OS X MacBook, my wife's Vista laptop, the central Ubuntu server, and the Mac Mini that handles all of our in-house backups (time machine as well as file sync, etc). We don't have more music or pictures than the typical computer user -- ours is just dispersed among many machines and all gets synchronized or backed up to the Mac Mini running OS X Server.

So, I signed up for MozyHome, their personal-use, unlimited backup plan, and went happily on my way downloading the OS X client to set up the online backup for our files. Imagine my surprise and disappointment when MozyHome refused to install, for no other reason than I was running a "Server" SKU product, which MozyHome refuses to operate on. There was no technical reason, no software capability limitation, and no resource problem causing this -- the MozyHome client is specifically disabled for Operating Systems it thinks should use their "Pro" product.

I contact Mozy support, and their only answer is, "You must use MozPro to backup a server system." Great. Now what?

Plan B: I'll mount the backup drive attached to my Mac Mini to my windows machine and run the backups from there.

Fail again! MozyHome does not support backing up network drives. Not because the Mozy technology doesn't support this -- because their "Server" line of client software does this no problem. Again, this limitation is put in place specifically for marketing reasons.

Sure, I could probably come up with a Plan C that would involve duplicating all of my backups to a windows box and pushing the Mozy updates from there, but that's not what my windows machines are set up to be spending their time and resources on. I have a home network setup that is great for backup -- but Mozy's silly limitations prevent me from taking advantage of them.

So, at least until I have a spare box that runs windows or OS X (non server SKUs of course!) available to house all of our consolidated files, we won't be using Mozy. In the meantime, I'm in the market for a decent backup solution, preferably compatible with OS X and/or Linux... I'd even settle for a good rsync-based backup solution.

Any suggestions?

The Essentials: What I Install on a Fresh Workstation

2008-10-22T00:00:00-04:00

This past weekend, for a number of reasons I won't get into now, I performed a complete "re-install" of my primary workstation. Previously, I was using XP Pro SP2, which had been installed and running smoothly on the machine for nearly two solid years. This also meant that my machine had two years worth of my customized configurations, applications, tools, etc., so a re-install also meant bringing a fresh Windows (Vista this time) installation up to speed and in sync with my preferences.

During the process of re-installing all the various programs and utilities that I make regular use of, it occurred to me that I was installing the things that I just couldn't do without -- the tools I keep in my toolbelt at all times. I don't know if I've got everything installed yet, but I know I've installed all the primary tools that I use on a regular basis... and they are listed here:

AVG Free Anti-Virus

http://free.grisoft.com

Regardless of the enhanced security of Windows Vista, the very first thing I installed was AVG Free. This free little anti-virus is great for the price tag and reasonably effective at catching threats.

Firefox 3

http://getfirefox.com

Firefox is by far my browser of choice due to its features, speed, extensibility, standards compliance, and cross-platform support. Along with Firefox I also install a standard battery of extensions, including:

Firebug
Web Developer Toolbar
YSlow
Download status bar
Foxmarks

Notepad++

http://notepad-plus.sourceforge.net/uk/site.htm

For my simple text-editing and random coding, Notepad++ is my weapon of choice. I spent some time taking a look at the editors highlighted over at LifeHacker, but Notepad++ has just the right mix of everything for my taste.

PuTTY/PuTTY Tray

PuTTY:	http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
PuTTY Tray:	http://www.xs4all.nl/~whaa/putty/

I've looked at other programs for Windows, but they are either too expensive, too bloated, or just too limited. PuTTY is by far my favorite SSH/terminal program for Windows. I install the full PuTTY package including PuTTYGen and the like, then replace the putty.exe file with the exe from the PuTTY Tray project, to get just a few more nice features that make it that much better.

Zend Studio for Eclipse

http://www.zend.com/en/products/studio/

I use ZSE as my primary IDE for my web development work.

MySQL GUI Tools

http://dev.mysql.com/downloads/gui-tools/5.0.html

During the course of web development, you usually end up interacting with databases a good bit. I've found that the MySQL Query Browser is a handy little tool, especially when working in a local development environment. This is probably a tool I could do without or even find a better fit, but it got installed quickly nonetheless -- because I'm familiar with it.

P.S.: It's a rather large shame that MySQL decided to cripple the free version of MySQL Workbench and charge you $100 for some of the best features. I don't mind paying for software, but the value provided by Workbench is not worth $100, especially when many of those features used to be free.

KatMouse

http://ehiti.de/katmouse/

KatMouse is a little Windows interface tweak that makes the cursor behave a little more like Linux desktops or OS X by allowing you to use the scroll wheel to scroll a window does not have the focus but is directly under the cursor. Once you have this functionality you never want to go back... imagine not having to click in the list of files to scroll through it or not having to switch windows to scroll that web page you have open behind your terminal...

Taskix

http://taskix.robustit.com/

Taskix is simple and to the point: it lets you drag taskbar items to reorganize them. It is lightweight and just works, so I love it. Thanks LifeHacker for turning me on to this little gem.

FileZilla

http://filezilla-project.org/

I have been using FileZilla for (S)FTP transfers for as long as I can remember -- long before it was a slick as it is now. Many agree that FileZilla is the best FTP program out there.

eWallet

http://www.iliumsoft.com/wallet.htm

eWallet is a little addiction I picked up a couple years ago at my last job. This is where I keep all of my passwords and important information, safely encrypted behind my master password. As an added bonus, Illium software (the maker of eWallet) just released an iPhone version of eWallet that syncs over WiFi with your Windows eWallet!

Launchy

http://www.launchy.net/

Since I am using Vista this time around, I thought Vista's nifty search bar in the start menu would replace Launchy. However, my love for Launchy's power, customizability, and great plugins brought me running back into Launchy's arms. Launchy is a simple (although not very light-weight) application launcher that indexes your Start Menu (and whatever you want) and provides a great little hotkey-induced interface for launching your programs, documents, or whatever you want. The excellent PuTTY Plugin makes a great companion to PuTTY as well!

TUGzip

http://www.tugzip.com/

After a long time of using 7-Zip, I found TUGZip about a year ago. TUGZip is extremely fast, has an excellent shell extension (right click on a file and extract it in place), and supports every compression format I've ever heard of... and many that I've never heard of. Best of all, it's free!

CrossClip

http://www.stretchedout.com/products/crossclip/crossclip.php

I started using CrossClip when I started working from home a year ago. My Windows machine sits front and center, flanked by my Ubuntu laptop running virtual machines on the left and my MacBook Pro with OS X on the right. Swivelling back and forth between the platforms is great fun until you realize that you would give just about anything to be able to quickly and easily copy from one computer and paste to the other. CrossClip lets you do that, and it works on Windows, Mac, and Linux (though I don't use it on Linux right now because of synergy), and it turns out I only had to sacrifice $19.95.

Synergy

http://synergy2.sourceforge.net/

Synergy, with all its quirks and oddities, is still the only cross-platform way I've found to use the keyboard/mouse of one computer to control another computer. I use the keyboard/mouse connected to my Windows workstation to also use my Ubuntu workstation to the left.

Note: If you need this functionality between two Windows computers, check out Stardock's excellent Multiplicity.

Jing

http://www.jingproject.com/

Jing is awesome. Never has it been so easy to make a screen capture (with or without voice-over), take a screenshot of just part of the screen, or share your captures with the rest of the world. You have to try it to understand -- I use it regularly to collaborate with coworkers or explain things to clients.

Quick Tip: Route All PHP E-Mails for Development

2008-10-15T00:00:00-04:00

I do a lot of work involving PHP-based web development. I often find myself needing to test e-mail functionality of a complex PHP application (such as Drupal), but I want to be sure that emails won't be sent to any of the unsuspecting users in the database of the application I'm testing.

Do accomplish this, I make a small but important change to my php.ini (actually in my conf.d/dev_mail.ini -- but it's the same thing):

sendmail_path="/usr/sbin/sendmail -i user@host.tld"

This tells PHP to use this command line whenever the mail() function is used to send an email. PHP's default here is "sendmail -i -t". The -t tells sendmail to scan the message text for To:, Cc:, and Bcc: headers to determine where to deliver the mail. By eliminating the -t we tell sendmail to use the email address(es) on the command line to determine delivery, thus making sure that no matter what PHP's mail() tells sendmail, it will deliver to the address you want.

Zend Studio for Eclipse 6.1

2008-08-26T00:00:00-04:00

I'm one of "those" people that paid for Eclipse, an open source IDE. I originally bought it because I love and use Eclipse a lot for my work and I wanted commercially available support for the product on which I depend so heavily.

Zend recently released version 6.1 of Studio for Eclipse, with very little (if any) fanfare or announcements (I didn't know about it until I looked at the download site), compared to the release of 6.0.1 which was heralded in their newsletter, all over their website, and everywhere Zend's press team has any influence. I find this ironic, because 6.1 contains huge improvements and more fixes than the upgrade to 6.0.1 did.

Clearly the biggest change is Studio 6.1's move to being based on Eclipse 3.4, the most recent edition of the IDE. This brings with it a whole slew of fixes and improvements on its own, such as a (marginally) improved install/update engine, more efficiency with how the thousands of Eclipse files are handled, some better over-the-shoulder security, icons and more colors now in various interfaces (such as search), and a whole slew of enhancements to SWT, the graphics engine that renders Eclipse.

Anyone who has used Eclipse/PDT knows that there is a lurking bug that manifests seemingly at random during which the PHP builder/parser will get stuck in some sort of infinite loop. This tied up the workspace task queue, which meant that many other functions were useless when this bug reared its ugly head. The only solution was to kill the Eclipse process and restart it, hoping that your workspace didn't require any Notepad surgery to recover. All that is over in the new version of Studio for Eclipse. Let the villagers rejoice!

A minor annoyance I encountered while updating to 6.1 is that it's not actually an update. Instead, I had to download the full 6.1 release and install it separate from 6.0.x. Zend claims this is because 6.1 uses Eclipse 3.4 -- apparently there isn't an easy upgrade path from 3.3 to 3.4 -- oh well. Overall, though, just installing 6.1 was fine as it was able to load up and understand my workspace just fine. The worst part was re-installing my favorite plugins, like QuickRex.

While Zend solved my biggest gripe about Studio for Eclipse (the building PHP problem mentioned above), they've managed to incorporate a new, really annoying issue. If the Code Analyzer marks a part of your code with a warning or error, in some cases the syntax highlighting just stops after that line. The only way I've been able to get the syntax highlighting to return is to disable the specific error that the Code Analyzer was flagging (usually something like using the value of a function that might not return a value -- duh, then NULL is a possible return!) and rebuild the file. The next time I encounter it, I'm reporting it to Zend to see if that commercial support I paid for really works...

All-in-all -- I'm very pleased with Studio 6.1, and at this point, I'll continue to be a Zend customer, and Studio for Eclipse will continue to be my IDE of choice.

Drupal for Firebug

2008-08-09T00:00:00-04:00

Last month Chapter Three announced they were working on a Firefox plugin and Drupal module that would allow Firebug to talk with Drupal. As a Web Developer who uses Firebug heavily every day and who specializes in Drupal, this announcement from Chapter Three got me a little excited -- so I installed the plugin, enabled the module, and gave it a shot.

Installing the DrupalForFirebug Firefox Extension

Downloading and installing the DrupalForFirebug Firefox extension was basically like installing any other extension for Firefox: find it, click the link, let Firefox install it, then restart Firefox. Except one little oddity: in order to download/install from addons.mozilla.org, you have to have an account on the site and be logged in because DrupalForFirebug is in a pre-release status. Not a big deal, so I signed up, clicked the link in my confirmation email, and went on my way downloading the extension.

Installing the DrupalForFirebug Drupal Module

Installing the Drupal module, on the other hand, involved nothing special at all. I just downloaded the module from the Drupal.org project page, unzipped it into my Drupal install's sites/all/modules directory, and enabled the module in the Drupal module administration page (note there are two modules, both required). Also, if you plan to use DrupalForFirebug with a user other than user 1, then you may have to grant the appropriate permissions (I did this for testing things as an anonymous user).

Using DrupalForFirebug

One of the unfortunately rare, but pleasantly surprising events when working with pre-release software, especially software targeted at other developers, is when it works the way it is intended the first time. I took my newly-enhanced Firefox and navigated to the login page of my newly-enhanced Drupal instance and was very pleased to see this section available in my Firebug window:

Drupal for Firebug tab showing a form array.

The DrupalForFirebug module processed the forms on the page (the login form in this case) and output useful information inside a hidden div at the bottom of the page. The Firefox/Firebug extension then pulls that hidden data and puts it into a convenient spot inside the ever-open Firebug interface. And the fun doesn't just stop with outputting FormsAPI arrays. DrupalForFirebug will also tell you useful bits of info, such as:

List of SQL queries, their exection times, and what function called the query (just grabs output from Devel module)
Dump of any user objects loaded during page execution
Dump of any node loaded during page execution
Dump of any view built during page execution

DrupalForFirebug also provides a quick and easy wrapper around Devel's execute PHP code feature, letting you execute arbitrary PHP code on the server from within the Firebug window.

Getting the Most of out DrupalForFirebug

If DrupalForFirebug just had the features already mentioned, it would be worth using. However, the authors took some initiative and included a means for module developers (or debuggers) to output information from their code in a meaningful and well-organized fashion. If you're a seasoned Drupal developer, you've probably done your share of print_r($node) and the like when trying to track down an odd behavior or figure out what data is available to you at the theme level (let's just ignore real-time debugging for now, even though that's still by far my preferred method). The DrupalForFirebug module provides a simple function, firep(), that lets you send in any data, and DrupalForFirebug will wrap it in PRE tags and output it in the General tab of Firebug.

Here's an example I had while working on a site just recently. I had some data stored in the session while going through a wizard-like series of pages. I was debugging the process and I wanted to be able to quickly see what was in the session on each page load. Real-time debugging wasn't ideal for this as there were many page loads during the series of tests I was performing, so instead, I used firep():

function my_module_menu($may_cache) {
  if (!$may_cache) {
    if (module_exists('drupalforfirebug')) {
      firep($_SESSION, 'Session');
    }
  }
}

This bit of code put the entire $_SESSION variable dump into the General tab in Firebug on every page load, so I could get a quick insight into what data was being maintained in the session. Using this I was able to quickly track down a silly typo that was putting the data I wanted into the wrong element of the session variable. If you're wondering what the optional second argument is, it is a label placed above the data output in the Firebug window. This is so you can identify various dumps if there are more than one during a page load.

Conclusion

DrupalForFirebug has earned itself a place in my Drupal toolkit for now. I'm curious to see how often I'll use it over real-time debugging, and I'm anxious to see if the developers of this tool maintain it well and/or take it any further than it already is.

Google Chrome: First Impressions

2008-08-05T00:00:00-04:00

Sometime near the beginning of this week I heard something in passing about Google releasing a web browser. At first, I thought that was a cute idea. After a couple seconds of letting it soak in, however, I began to wonder what Google would try to bring to the table that they felt was not already present. I also wondered if Google's banning, then re-instatement of the Mozilla Public License had anything to do with their new browser -- but they're not evil, so that can't be it!

I wasn't really worried about trying it out at all, but a few things conspired to change my mind. First, seeing the CEO of Google being interviewed about Chrome during prime-time. Second, all the blog coverage of Chrome; and, third, glancing over the shoulder of someone trying it out and thinking to myself, "Huh -- looks interesting."

So I downloaded Chrome and set about evaluating it:

Installation

Installation was surprisingly simple and quick. Chrome made things convenient by pulling in all my relevant data from Firefox (including stored passwords -- a reminder that malicious software could get this too!).

Startup

I had heard that Chrome was based largely on WebKit, an excellent Open Source browser engine that is used by Apple's Safari browser. If you've used Safari in Windows, you've probably come to expect to wait a few seconds longer than you're accustomed to waiting while Safari starts up for the first time. This is what I half-expected when starting Chrome, but I was pleasantly surprised when Chrome started up very quickly and cleanly.

Appearance

Having already seen Chrome at a glance, I knew basically what to expect. In general, the interface looks very clean and quick -- just like what we've all come to expect from Google.

Usability

You can't help but feel very unhindered by Chrome's interface. Moving tabs into the title bar is, in my opinion, a cool idea that I jive with (especially nice when the browser window is maximized). The button layout is similar to other browsers, especially Safari, and the configurations and options are easy to find under two buttons to the right of the address bar. I continue to be a little bothered by Chrome responding more than other applications to a single notch on my mouse wheel. Also, scrolling in general sometimes goes noticeably slower than Firefox/IE and shoots the CPU usage pretty high.

Page Rendering

I clicked around for a little while in some of the recent sites I'd worked on, and I couldn't find any rendering errors. After some clicking around to the sites that I frequent I was able to count two instances of not-quite-right rendering of sites that appear fine in FF2/3, IE7, and Safari3.

Often during page loading I would see the various layers get rendered in order. This isn't necessarily bad, because Chrome seems to go pretty quick (at least on my machine) -- but I found this interesting and wonder if it implies something about Chrome's differences even from its WebKit cousins.

Conclusion

I like Chrome. I don't expect to be using it as my primary browser just yet, and I'm not sure if I'd be ready to give up my Firefox extensions. I'll probably use Chrome for quick testing and possibly as just another browser session when testing sites, etc. If you want a simple, quick browser, though -- Chrome doesn't seem like a bad choice. And who knows -- maybe Google is only getting started!

Make Views More Flexible/Maintainable

2008-05-24T00:00:00-04:00

Any Drupal developer worth his salt at least knows of the Views module. The shear usefulness and time-saving nature of views has earned the module a place in every Drupal site I've developed. However, because views are stored in the database and their presentation is controlled by the views module itself, managing views between staging/live sites, handling updates, and tracking revisions become difficult issues when dealing with Views. On the other hand, source code is easily managed between copies of a site, updates are a breeze, and revisions are natural -- all of this with revision control, such as Subversion. Also, what if you want to implement complicated access rules for your view?

Views documentation has a section on how to programmatically build and render views. This is great, but building a view programmatically takes a lot more time than using the Views UI because you're constantly trying to figure out the table/field names used by the various modules that provide views integration, let alone filter, field, and argument settings. If your development cycle is anything like mine, taking more time on something is not always a luxury you can afford.

Views module also provides an interesting Import/Export feature that allows you to copy/paste a view definition between sites. This helps with moving a view between two copies of a site and even helps a little dealing with updates. If you're diligent, this could be used to deal with revision control as well, but it's still a little clunky if you're maintaining and building sites all the time.

In my opinion, the sweet spot when dealing with views is combining views exports with embedding views in your own PHP code.

Every Drupal site I've developed as ended up with a "custom" or "glue_code" module that does various minor modifications to the site, such as hook_form_alter, hook_nodeapi, hook_menu, etc. I've also started to use this type of module to render and provide default views using the following process:

First, during development of your site, go ahead and design the view using the Views UI as you normally would. We'll call this view example_view.
Once you have the view's page display the way you like it, keep the page display enabled, but clear the URL so that the view page isn't really accessible anywhere on your site.
Now, export the view and copy the large export array that is provided.
Implement hook_views_default_views() in your custom module. Simply paste the exported views array into this function. Be sure your code makes sense and returns something according to the hook spec.
Delete the view from your site, because your custom module will be providing it now.
In your custom module, add a new hook_menu item that points to a callback that will provide the view. For the purpose of this example, our callback is called 'custom_view_page'.

The custom_view_page() function will then look something like this:

function custom_view_page() {
  $view = views_get_view('example_view');
  return views_build_view('embed', $view);
}

That will retrieve the view (the default one provided by your implementation of hook_views_default_views OR the overridden version if you've defined a view in the UI by the same name), then render it for output. If you need to send arguments to the view, pass them as elements of an array in the third parameter sent to views_build_view().

Benefits:

View definition is no longer in the database.
Updates are performed by updating code base. Simply build updated version of view on dev/staging site, export, and update the hook_views_default_views() implementation.
Revisions of changes to view are kept in source control.
All benefits and flexibility of views is maintained.
Finer control over things like access to the URL that provides the view.
Possible performance gains (untested).

Ecto and Drupal

2008-05-12T00:00:00-04:00

A few weeks ago I read a brief but interesting article by Jeff Whatcott on Getting Ecto Set Up with Drupal. I had heard of Ecto in passing but had never really looked into using it. With all the hubbub around WYSIWYG tools for Drupal and the like recently, I wondered if a good desktop application would be a viable solution, so I figured I'd give Ecto a try for myself.

I was initially very impressed with the relatively simple interface sported by Ecto. After quickly configuring the blog_api module on Drupal and just a few clicks in Ecto I had connected to my Drupal blog and was presented with a list of the blog posts already published. I explored the various configurations and was pleased to find a lot of configuration options -- this isn't just a handy app, Adriaan Tijsseling has done a fine job putting together a fully-featured application workspace.

I decided that I owed Ecto at least a couple blog posts to see if I could use it effectively on my Drupal installation found at jbenner.net. I set about writing my first post with Ecto, and was very comfortable at first, typing away and easily setting basic font styles, adding links, and breaking paragraphs. I was a little confused by the interface to select "Categories" and "Tags" -- as I was entirely sure how they translated into my Drupal environment.

I published the post, then went to my site to check it out. For the most part, things looked okay... there were a few minor formatting things here and there that most likely resulted from oversights on my part while preparing the post. The problems came in when I started inspecting the post in more detail. First, Ecto didn't seem to set my topic tags at all -- for future reference, ignore Ecto's tags, and just use Categories.

Also, I use the pathauto module to automate Drupal's path aliases -- but posting using Drupal's blog_api seems to go right around pathauto. From a Drupal developer's perspective, this will probably make some sense, as I imagine that pathauto does its magic with a combination of hook_form_alter() and hook_nodeapi(). Does blog_api skip various hooks? This might be worth more investigation.

Add to this Drupal's lack of proper time zone handling and you get a rather confusing set of configurations between Drupal and Ecto to get both Ecto and Drupal to show the same (and correct) time. I eventually got a pair of settings that seemed to almost work properly, but the journey was ripe with confusion and lacked a satisfying end. In fairness, I think this may be a shortcoming of Drupal, though I'm not entirely sure in this instance.

Ecto has an option to set the input format to use when it sends the post to Drupal; however, it did not seem to set this correctly in every case. I don't know what was causing the difference in behavior in different instances, but this is an issue worthy of note.

As a note -- don't touch Ecto's summary feature if you're using it to post to Drupal. For some reason, it sends Ecto into a spiral of arguing with Drupal over the contents of your post, and seems to always end up adding what it wants your summary to be at the beginning of your post with broken HTML. You're better either setting your break tag manually in the HTML or creating a custom HTML tag in Ecto and inserting it that way.

Overall, Ecto did a fair job of posting. However, between skipping pathauto and the time zone issues, I can't see myself using Ecto regularly, as I'd find myself in Drupal's node form fixing things up anyway. More importantly, Ecto doesn't really simplify the process of posting blog entries in general, though it does offer some nice tools for performing more advanced blog actions, such as integrating images, or Amazon and Flickr resources. If anything, I could see myself using Ecto to write drafts and performing cleanup and publishing in the web interface. This might especially be handy if I want to start a post while offline.

I don't see myself encouraging any clients to use Ecto with Drupal at this point. I want to emphasize, however, that this isn't necessarily because of problems with Ecto as much as some problems between Ecto and Drupal.

My OS X Toolkit

2008-04-16T00:00:00-04:00

After the first day of having the new Mac and playing with iChat, it was time to hunker down and start accomplishing things with the Mac. In the last few years one of the biggest reasons I've been reluctant to get a Mac has been a sense (whether justified or not) that the software selection wouldn't be sufficient for my wants, needs, and tastes. Part of my recent decision to get the Mac was the general feeling that this was no longer the case. I've found this to be true.

I spend a lot of time on my computers. Most of that time is spent using or writing web pages and web applications (and the occasional video game). Web development isn't what it used to be, and tackling everything requires a lot of tools. I'll summarize my needs and the tools that I've settled on using to meet them.

SSH Terminal

Windows:	puTTY
OS X:	iTerm

Most of the stuff I found on the web about doing SSH in OS X said "just use the terminal!" While I like the OS X terminal, I was hoping to find something to help me manage sessions and generally make all the SSH connections I juggle in a day a little easier to use. Enter iTerm -- it does just enough using its 'bookmarks' and connection and keyboard profiles to make managing multiple console or SSH sessions graceful, and its tabbed interface is another step in keeping my workspace navigable. Oh, and it's free.

Text Editor

Windows:	Notepad++
OS X:	TextWrangler

OS X comes with a nice little text editor aptly named "TextEdit" that is pretty good, but I like text editors that go the extra mile. I like to search and replace with advanced regular expressions. I like a text editor that implements a good system to manage a bunch of open text files. I like a text editor to do basic syntax highlighting in the event I don't open a file in my IDE. TextWrangler does all of this and so much more. And its free!

Web Browser

Windows:	Firefox
OS X:	Safari and Firefox

I love Firefox, and it was the very first thing I downloaded on the new Mac. However, I felt I owed it to myself to give Safari a fair shake, especially since I've been impressed with the betas (and recent release) of Safari 3 on windows. Safari on OS X is fast, clean, and nice to use. If Safari had the plugin base of Firefox, rendered a few more sites better, and had some minor usability tweaks... okay and a bunch of other things, I'd use it as my primary browser all the time. For now, I enjoy using both of the browsers side by side.

Development Environment (IDE)

Windows:	Eclipse
OS X:	Eclipse

I have been very happy with Eclipse on Windows (and Linux) for a few years now, and it just keeps getting better. When I started working on the Mac, I wanted to use whatever was widely used by the Mac development community. A lot of web developers I know who use Mac as their primary OS use TextMate. While I liked TextMate (a lot), between the TextMate price tag ($64), the effort of learning a totally different interface, and the fact that Eclipse works great on OS X... I stuck with what I knew in this department. Expect more on this later.

Local Web Application Stack

Windows:	XAMPP
OS X:	MAMP

When I develop a web application I almost always work locally and migrate the changes to the live server. This development pattern has so many advantages that I can't believe there are web developers that still don't use it. A few years ago the way to do this was downloading and installing (sometimes compiling) the various components of the stack (usually Apache, MySQL, and PHP). More recently groups like apachefriends have put in great effort to package the components in easy-to-use, point-and-click packages. XAMPP is a great packages for Windows and is also available for OS X. I tried XAMPP for OS X, but when I tried to configure it with things like vhost aliasing and such, it kept resulting in errors and odd behavior. I then tried the free version of MAMP, which I found to be much easier to configure.

A Voyage Home: My Return to Mac OS

2008-04-13T00:00:00-04:00

People often ask me how or when I started working with computers, programming, etc. My answers usually hark back to my childhood and my first ventures with my parents' Macintosh Plus, then our family Performa, and my Powerbooks 180c and 540c. Basically, all of my earlier computer encounters were on a Mac, from learning to type to learning my first programming language to discovering the internet for the first time.

Sometime during the late 90s I became frustrated and angry with working in a Mac world and I "converted" to my first Windows machine, vowing to never look back. Since that day I've been working primarily in Windows and Linux, only touching a Macintosh when absolutely necessary... that is, until this last week.

When my new 15" Mac Pro arrived and I opened it and started it up, I was consistently impressed with how easy it was to use, and how well it "just worked." The experience has been a far cry from the agonizing memories I have of Mac OS 8 & 9. I spent some time playing with the software that Leopard comes with, such as iChat, iCal, etc. I have to say... iChat is effective, Photobooth is a blast, and syncing the iPhone with the Mac is an order of magnitude better than with my Windows box.

I've spent a respectable amount of time making the Mac work the way that I like, such as managing hotkeys, setting up my command line environment, and trying to find a software to meet my many needs. The process has been fun and I've been very pleased to find that the solutions available for Mac are closely related to (and sometimes better than) the Windows software that I regularly use (more posts on that later?).

All of this to say that I'm not only happy with the Mac, but I've been overwhelmingly surprised at how much I'm enjoying the experience. I won't be getting rid of my windows or linux boxes any time soon, but the Mac has quickly secured its place in my toolkit.

Adventures with VersionOne

2008-04-07T00:00:00-04:00

A few months ago the primary project to which I'm assigned espoused an Agile/XP approach for development... well, a heavily modified Agile/XP approach as there is only one developer (me) working on features/tasks/etc that come from a team of people, funneled through a project manager of sorts before they get to me. Because of the unique relationship we've enjoyed with the client, managing the project has been an interesting challenge since client users are involved closely with the project and are sometimes responsible for tasks such as testing.

We've moved between a few task tracking/project management systems in the process, trying to find software that fit our needs, the latest of which is VersionOne. The client actually came across VersionOne, which I'd only heard of in passing before that point, and asked if it would be a viable system to use during development. After a quick glance by several of our team members, we agreed that the system looked pretty good and was unique in that it was designed with Agile in mind so it supported the Agile/XP workflow in a manner than other ticketing systems didn't. So, over the course of a couple days we migrated to VersionOne and started using it.

The first thing that impressed me about VersionOne (aside from its nice, Web 2.0 shiny look) was how you can customize every view you have access to. You can specify which columns to show, and even which you would like to be able to edit inline (great feature!). This allows for a customized experienced and access to information the way you want it.

It wasn't long before we started running into the challenge of figuring out how to pass information back and forth. VersionOne has nice "Notes" that can be attached to any object (story/defect/issue/request/task/etc); however, as far as I've been able to tell, the overly-complicated but barely useful subscriptions/notifications feature does not allow emails to be sent to let me know when there's been a new note on an object. In fact, the notification system won't send emails at all -- the only option you have is a personalized RSS feed that pushes nearly useless notifications about the various objects (not including when there is a new note).

In pursuit of a workable setup, I discovered that VersionOne has an Eclipse plug-in. I rejoiced. That is, until I installed and used it. The plug-in is horribly buggy and exposes an abhorrently limited set of abilities to the user in the Eclipse environment. All in all, the plugin should be abandoned and they should put some honest time into writing a good Mylyn connector.

Lacking an email option wouldn't even be so bad if the notification system's personalized feed provided useful information about the objects for which it's updating feed items. After a couple days of tinkering with notifications, I've all but stopped paying attention to the notification feed and have been forced to rely on a combination of clever custom statuses, a custom 'status note' text field, and pure memory to determine if an object has been updated in a manner that needs more attention from me.

Overall, however, I've been really pleased with the way VersionOne works and its slick interface. I'm holding out hope that with time I'll either get used to how VersionOne wants me to communicate or uncover a feature I'm not yet aware of that will help bring important information to my attention.

Don't Use It, You Lose It

2008-03-31T00:00:00-04:00

A friend of mine has been using CakePHP to solve some problems where he works lately, and has been asking me for some help here and there. I always love to help as much as I can, but lately I've found my advice when it comes to CakePHP to be less helpful than it was in the past.

Now, ask me about Drupal stuff and I can answer most questions that you could come up with (I think?), but that's because I use Drupal day-in and day-out, for both my job and the hobby stuff I do on the side. The side effect is that I find my skills using other technologies suffer since I don't flex those muscles as much. I even worry about some basic coding skills -- since Drupal manages so much of the application execution process, I don't have to worry about a lot of the more technical (cool) stuff.

Don't get me wrong -- I love Drupal. I think it's the best CMS out there. I just wish I had more opportunity, time, and possibly motivation to use some of the other technologies I enjoy... like CakePHP.

Technical Debt -- The Best Laid Plans...

2008-03-28T00:00:00-04:00

Over the last couple months I've been hearing the term "Technical Debt" get thrown around frequently, describing the effect that results when decisions are made in a project or process that cause problems down the line.

As any good developer will tell you, there are usually more than one way to solve the problem. Once you've narrowed down your options to a few viable courses of action, it's usually the fastest or cheapest that gets chosen. I'm not here to say this is an incorrect way to choose between good options, but I'd say that when we're planning the development of a project, or even just a feature, we ought to also say to ourselves, "Which approach will leave us with the least technical debt?"

The problem is that this isn't an easy question to answer, because we don't always know what new technology, approach, service, or totally different alternative is coming down the road. So what seems like the best idea today may be what we lose sleep over a year from now. What are our options, then?

After jumbling it around in my head and thinking through some of the real-life examples I encounter in my every day work, I think the best answer is to consider which is the most flexible approach. Which solution ties us down the least? Which method is the easiest to rip out, expand, convert, or replace at a later date? What does approach A prohibit that approach B does not? What technologies or methodologies are we committing to by choosing one option over another?

I don't think that I'd go so far as to assert that the most flexible option is always the best, but rather I'd stress that giving flexibility heavy consideration in the decision process is warranted... because you may be paying out the time and/or money later to the Technical Debt accrued by your decisions.

Analysis of Drupal as a Framework

2007-11-18T00:00:00-05:00

I recently presented at an event where several web application frameworks were discussed, including Drupal. Around that time I was also working on several projects written as Drupal modules at a level that I had not done before. These series of events caused me to take a step back and consider Drupal's merit as a web application framework.

I've often fought with myself in my head about whether a large web application would be best developed from scratch, based on a loose framework (such as CodeIgniter or CakePHP), or based on a large, existing platform such as a CMS. Typically I would have said that if you're writing a relatively complex application, that it might be best to use a loose framework for pure time saving and best practice adherence.

However, I feel that my opinion may be shifting. Many web application share basic components: authentication, authorization, URL routing, user management, etc. The features that typical web applications share read almost as a feature list for a CMS. What Drupal manages to do is not only implement those typical web application features (well), but also expose a considerable, well-thought-out set of APIs that allow programmers to take full advantage of Drupal's core features and wield them for their own purposes.

Node API

One of my favorite subset of Drupal's features is the node api. This system allows the programmer to create modular content types for which Drupal will control all aspects of CRUD, requiring the programmer to write only the code which will dictate that content's behavior that is different than the norm, such as the layout of a specialized form or storing extra data in the database.

Form API

The Form API is another example of one of those systems that I've found myself often imagining programming for myself. I've often though, "There has to be a way to write a system that will make doing forms and the related HTML not suck so much." The Drupal Form API is that system that I've desired for so long. Creating a form is as simple as populating a rather simple array, describing each field's properties in short form.

Conclusion

I would encourage anyone looking for a framework, nay, a platform upon which to build their web application to carefully consider Drupal. I'd venture to say that just about anything you'd want to accomplish is either already implemented or is (relatively) easily accomplished using Drupal's API and core modules, let alone the community-contributed content.

Write a Reasonably Complex Application with CakePHP/Drake/Drupal?

2007-07-24T00:00:00-04:00

This post is in reply to a comment on a previous article that I thought contained some good questions:

Hi Josh,

Saw your post on the cakePHP list regarding Drake, here http://groups.google.com/group/cake-php/browse_thread/thread/4d0ad8b73a5... .

I'm hoping you won't mind answering a couple of questions.

In particular, I am concerned with the performance hit of running a framework within a framework, and all the associated redundancies. Was performance something you looked at in any detail?

How much harder is it to work with Drupal's API from within cake than it is to write a native drupal module?

Did you feel the benefits of using cake overcame the increased complexity of having to bridge using drake?

In your opinion, if one wanted to develop a (reasonably complex) module for drupal, would you recommend writing a native drupal module, or writing a cake app and bridging it with Drupal?

Thanks very much.

Hopefully you will be kind enough to reply.

Sorry for taking some time to get back to you! Been busy!!

Yes, I looked at performance issues with Cake in Drupal (Framework in a Framework as you said). For our application, we're weren't looking at really high traffic. The initial box I had for the project was an old dual P-III system running RHEL5. It performed pretty well with some unexplained pauses here and there (may have been hardware related).

We recently migrated the site to a Virtual Machine hosted on a VMWare ESX server -- the performance is excellent! Drupal itself is very snappy and the Cake applications are very responsive. All page loads are under 3 seconds (my target), with the exception being pages with a LOT of data.

You can work with Drupal's API perfectly fine as Cake is executed in the same space as Drupal. We use this fact in some of our small apps that need to know what user they're dealing with, so Cake just pulls data out of the $USER global from Drupal.

The benefits of using cake definitely outweighed the complexity increase. In fact, the complexity increase was pretty negligible as I no longer had to do much consideration of authentication and authorization against our systems (LDAP), as Drupal takes care of that for me. Also, it was easy to integrate the Cake applications with Drupal styles and have a very consistent look, which took care of a lot of the normal UI considerations we were having with other custom apps (ie: things are too ugly :) ).

I see a Drupal Module and a Cake app running inside Drupal as two different beasts. If I wanted to add functionality to Drupal, like add a new content type with custom handling, I would use a module. If I wanted to write an application that I wanted to "live" within a Drupal site, I would use Drake/CakePHP.

Now, your question was mostly about a reasonably complex application. My assumption is that a reasonably complex application's benefit to existing inside Drupal would come primarily from relying on Drupal's many charms, including authentication/authorization handling via user/role-based security, content type management, views (I love views!), exposing data in blocks, etc. Saying this, my initial answer would be to write it as a Drupal Module, because I would likely want my data to be visible to Drupal as a content type to open a world of powerful managing of the data, and CakePHP models will want to talk to the database directly. While this isn't an insurmountable problem, I'd think at first that this alone would be enough to dissuade me.

You could override AppModel and teach it to talk to Drupal's database layer, though (which might actually be a VERY interesting experiment with Drake). Or, if you weren't worried about your data being visible to other Drupal modules, this wouldn't be a consideration. In our applications, we've (so far) not been worried about exposing our data to Drupal (probably quite the opposite, both for security reasons and based on the fact that CakePHP talks with a completely different database server than Drupal), but I can foresee the day that we might want to.

Could you write an effective, reasonably complex application with CakePHP in Drupal via Drake? Absolutely. But you might have to invest some time on integration between the frameworks if that's a concern of yours. All in all, that's a question best answered in your context, not broadly.

Understanding CakePHP Associations

2007-06-06T00:00:00-04:00

CakePHP uses the Model-View-Controller (MVC) concept as a framework to encapsulate the three basic layers of an application. Models, the component of the MVC framework that connect the application to the data, are especially good at describing relational databases. One of the questions that seems to come up alot on the CakePHP Group in one form or another, is what the practical difference is between the various relationship types (or associations) that CakePHP models can implement. I'll try to explain what's helped me keep them straight...

NOTE: This is not a tutorial on writing a CakePHP application, but rather a few tips for understanding associations. For more complete information on writing a CakePHP application, see the manual.

I remember studying basic genetics in biology in high school, and our teacher said that genetics is one of those topics that people have an extremely hard time understanding... until it just clicks -- then you can't understand why you didn't get it before. I feel that learning associations in CakePHP can be a similar issue. I've been writing relational database applications for several years, but I remember when I was defining models in my first Cake app... I couldn't for the life of me get a grip on what the difference between "hasOne" and "belongsTo" was, let alone the infamous hasAndBelongsToMany! That was, until I took a closer look at the CakePHP Manual chapter on Models and made an important discovery: the difference between hasOne and belongsTo lies in which table in the relationship does the 'pointing.'

belongsTo:

A -> B

If table A has a field that references table B, then table A is said to "belongTo" table B.

Just imagine that every record in the A table says "I belong to a record in table B!"

In the movie database example below, since for each movie there is one director, and the movie table has a field that points to a director, each movie belongs to a director. With belongsTo, each record can only be associated with one foreign record, but multiple records can be associated with any given foreign record. For instance, all three Matrix movies were directed by the Wachowski brothers, so each movie would belong to the Brothers' entry in the directors table.

hasOne:

A <- B

If only a single record in table B has a field that references table A, then table A is said to "hasOne" table B.

Every record in table A says: "I have one record in table B that points to me!"

hasOne is basically belongsTo, but the pointing goes in the other direction. If table A hasOne table B, then it can also be said that table B belongsTo table A (except only one B record would be associated with any A record).

In all honesty, I don't think I've ever used this association. This is because if every record in table A only has one record associated in table B, the tables may be able to be combined. Where the use for this association comes in is when you have 1-to-1 relations, but the data may be best kept in different tables for logical or technical reasons. For instance, in the CakePHP manual blog example, every user hasOne profile -- in theory, the tables could be combined, but it probably is easier to keep them apart and accessible through association.

hasMany:

A <- B (multiple)

If table B has a field that references table A, and multiple records in table B can point to the same A record, then table A is said to "hasMany" table B.

Every record in table A says: "I have many records in table B that point to me!"

hasMany is the true sibling to belongsTo. If table A has many B, then table B can be said to belongTo A (and the multiple works, since A has MANY). hasMany and belongsTo are very complimentary if you need to get to a relationship from each side of it.

hasAndBelongsToMany:

A <-> C <-> B (multiple)

If every record in table A can link to multiple references in table B, and every record in table B can be linked to by multiple A records, table A is said to "hasAndBelongToMany" table B.

HABTM is a little more complicated, because it uses a link table. A link table will have at least two fields, one to point to the id of the first table, and another to point to the id of the linked record in the second table. In our movie database example below, since a single movie can have multiple genres, and you can have more than one movie in any given genre, the movie table has many and belongs to the genre table. HABTM associations can go both ways since the link table will work in each direction, so they can be (and often are) defined in both models (as it is done below).

Let's consider a sample application where we're developing a small movie database (an extremely small step-brother to IMDb). This application will be a database of movies categorized by genre with directors and movie reviews. Each movie can be in multiple genres (Action/Horror, Sci-Fi/Thriller, etc), can have one director (we'll ignore multiple directors for now), and have multiple reviews.

We'll start by outlining our models:

Movie:	belongsTo Director hasMany Reviews hasAndBelongsToMany Genres
Director:	hasMany Movies
Review:	belongsTo Movie
Genre:	hasAndBelongsToMany Movies

And now some code:

class Movie extends AppModel {
    var $name = 'Movie';
    var $belongsTo = array( 'Director' => array( 'className' => 'Director' ) );
    var $hasMany = array( 'Reviews' => array( 'className' => 'Review' ) );

    // We don't need many settings in this association, because we'll stick to Cake's naming conventions
    var $hasAndBelongsToMany = array( 'Genres' => array( 'className' => 'Genre' ) );
}

class Director extends AppModel {
    var $name = 'Director' ;
    $hasMany = array( 'Movie' => array( 'className' => 'Movie' ) );
}

class Review extends AppModel {
    var $name = 'Review';
    $belongsTo = array( 'Movie' => array( 'className' => 'Movie' ) );
}

class Genre extends AppModel {
    var $name = 'Genre';

    // We specify the join table here because Cake would expect the table to be called genres_movies from this side
    $hasAndBelongsToMany = array( 'Movies' => array( 'className' => 'Movie',
                                                     'joinTable' => 'movies_genres'
                                                   );
}

And for good measure, here's a basic outline of the table structure:

movies:	id title release_date director_id
directors:	id first_name last_name
reviews:	id movie_id author_name body_text
genres:	id name
genres_movies:	id movie_id genre_id

Using these models and tables, we can use this line of code in a controller:

$movie = $this->Movie->read(null, $id); //assuming $id contains a movie id...

Will gives us a data structure like this in the $movie variable:

Array (
    [id] =>
    [title] =>
    [release_date] =>
    [director_id] =>
    [Director] => Array (
        [id] =>
        [first_name] =>
        [last_name] =>
    )
    [Reviews] => Array (
        [0] => Array (
            [id] =>
            [movie_id] =>
            [author_name] =>
        )
        [1] => Array (
            [id] =>
            [movie_id] =>
            [author_name] =>
        )
    )
    [Genres] => Array (
        [0] => Array (
            [id] =>
            [name] =>
        )
        [1] => Array (
            [id] =>
            [name] =>
        )
    )
)

Mssql_Table Behavior for CakePHP 1.2.x

2007-06-05T00:00:00-04:00

I've been using CakePHP 1.2.x for projects at work, where our primary database back-end is Microsoft SQL Server. Maybe I'm just missing something, but it seems to me that SQL Server outputs datetime values in an odd way (by odd, I mean not parsable by strtotime). When pulling datetime values out of SQL Server, CakePHP will render them as PHP strings just as they are returned from the server, in this format:

Jan 10 2008 12:25:07:000PM

Now, strtotime parses incoming values according to GNU Date Input Format, which is incompatible with the output from SQL Server (note the milliseconds). To make life easier when working woth models that describe MSSQL tables with datetime fields, using this behavior can make things a little easier.

To use this behavior, put the code in app/model/behaviors/mssql_table.php and including the following line in your model:

var $actsAs = array('MssqlTable' => array());

... and datetime fields will come through as strings formatted according to your current locale (which is parsable by strtotime -- well, at least here in en_us).

<?php
/**
 * $Id: mssql_table.php 127 2007-06-04 13:11:09Z jbenner $
 *
 * This behavior will automatically handle some of SQL Server's oddities. Right
 * now all it does is handle parsing of datetime fields automatically, and
 * converts them to GNU datetime compliant strings.
 *
 * @package      pbuapps
 * @subpackage   pbuapps.base
 * @author       Joshua Benner
 * @copyright    2007 Philadelphia Biblical University
 * @version      $Revision: 127 $
 * @modifiedby   $LastChangedBy: jbenner $
 * @lastmodified $Date: 2007-06-04 08:11:09 -0500 (Mon, 04 Jun 2007) $
 */

/**
 * Behavior that will deal with the oddities of SQL Server models.
 */
class MssqlTableBehavior extends ModelBehavior {

    protected $__defaultParsers = array( 'datetime' => array('parser' => 'parse_datetime'));

    /**
     * Initialize behavior
     *
     * @param object $model
     * @param array $config
     */
    function setup(&$model, $config = array()) {
        $settings = am(array('parsers' => $this->__defaultParsers), $config);
        $this->settings[$model->name] = $settings;
    }

    /**
     * After a find is performed, this function will check if any of the fields
     * returned are datetime fields and reformat them. This is necessary to
     * deal with microsoft's odd datetime output in SQL Server.
     *
     * @param AppModel  $model
     * @param array   $results
     * @param boolean $primary Whether or not this is primary query
     */
    function afterFind(&$model, $results, $primary) {
        // First: find all the offending fields
        $parseFields = array();
        // Load parsers from settings
        $parsers = $this->settings[$model->name]['parsers'];
        // Iterate through fields in table info and check if their type is in our parser list
        foreach ($model->_tableInfo->value as $field) {
            if (array_key_exists($field['type'], $parsers)) {
                // We found a juicy field to parse, so assign a parser function
                $parseFields[$field['name']] = $parsers[$field['type']];
            }
        }
        // Now we know which fields to parse, let's parse them!
        if (count($parseFields)) {
            // Iterate through results rows
            foreach ($results as $i=>$r) {
                // We don't iterate through models, because they can worry about
                // their own datetime parsing if they must.
                $fields = $r[$model->name];
                // Iterate through parsable fields for each row
                foreach ($parseFields as $field=>$func) {
                    // If this row has this field
                    if (isset($fields[$field])) {
                        // Execute associated parser function
                        $results[$i][$model->name][$field] = $this->$func['parser']($fields[$field]);
                    }
                }
            }
        }
        return $results;
    }

    /**
     * Parses the output from MSSQL datetime field and returns it in a format
     * compliant with GNU datetime standards (ie: strtotime can parse it).
     *
     * @param string $datetime
     * @return string GNU-compliant date string
     */
    protected function parse_datetime($datetime) {
        // MSSQL Format: Jan 10 2008 12:25:07:000PM
        return strftime('%c', strtotime(str_replace(':000', ' ', $datetime)));
    }
}

Cake, Drake, and Drupal: My Search for the Ultimate PHP Framework

2007-06-03T00:00:00-04:00

At work a few months ago, I was directed to start looking into some options for developing small database applications. So, I set out on my quest to find the best solution out there. Several weeks, and at least as many frameworks later, I finally found a solution that seems to work.

The first thing I did while considering where we should go in the future was to sit down and purposefully consider the past, asking questions and trying to pull from the experience of my coworkers (who have all been working there far longer than me). Many of our old solutions were implemented in MS Access or Paradox. While these are powerful programs that definitely have their place, the presence of many independent databases all over the place was causing problems, from (incorrect) duplication of data, wrong data, difficult-to-support solutions, and frustration on the part of the users. For this reason, we decided that a centrally-based solution would be far more manageable for a small team.

As a result of a centralized system, we would need to make sure that access control was done right. As such, it was decided that the solution should integrate into our existing authentication framework: Novell eDirectory. Sure, that should be easy enough -- we'll just be sure the solution can authenticate to LDAP. Another issue with centralization is that we'd have to be sure that the solution could have centralized data, and more importantly, integrate with our current "official" data sources. Again, no problem -- everything talks to SQL Server, right?

One of the other things I learned from my colleagues, is that with any new little "custom" application, there is a chance that the people requesting it might end up not using it, or might discover that what they thought they needed doesn't in fact fulfill their requirements. While sometimes it's hard to fight to urge to just say, "Decide EXACTLY what you want, then talk to me," that just doesn't work -- because most end users don't have the exposure to the technologies we're bringing to bear to make informed decisions about how they want it to look... exactly. This scenario combined again with the low manpower variable meant that we needed to make sure that a solution would have a rapid development cycle, so we could build it, tear it down, and rebuild it in a timely fashion without putting (too many) things on hold.

Finally, we came up with the following list of criteria for a solution for our custom applications:

Simple, easy-to-learn user interface
Rapid development cycle (ie: under two weeks for a small application; under four weeks for a larger one) [1]
Centralized
Integration with our Novell eDirectory
Backend support for Microsoft SQL Server, MySQL, and Oracle (primarily MS-SQL)
Solution should be free or relatively low cost, with good support and active community

There were a few initial Commercial software packages I considered. Most of them were companies I'd not heard of before that offered proprietary, Windows-only web server software that generated web interfaces in a similar manner to Access or Paradox. The familiarity of the development seemed nice, but there were limitations all over the place, and while the cost was not prohibitive, I felt that what those solutions offered could be accomplished for far less with much more flexibility and power.

About the same time I started my search, a coworker was preparing to deploy a new intranet based on Drupal, a rather popular open source content management system. While talking with him, we found that our project goals had a lot in common: LDAP integration, simple interface, low cost, low maintenance, etc. This got us thinking that integrating our applications into Drupal would be ideal -- a user would log into our intranet, and have a link on the side to access all the applications that their LDAP group memberships grant them.

Since Drupal was based on PHP, and PHP is an almost pervasive language, I started going through lists on Wikipedia of PHP Frameworks. I was looking for something that would basically write itself for me. After installing and playing with a few of the lesser-known frameworks, I soon discovered that they offered far less than they claimed, or were too buggy to use. Then I happened upon PHP 4 Applications. I was excited, because getting P4A running was very simple, writing forms involved no HTML, it was fairly attractive, and the system just worked. For a while, I thought I had the solution we needed -- that is, until I tried to integrate it with Drupal. Now, I'll be honest and say right now, that I'm not expert at integrating disparate systems with each other -- but getting P4A to run inside of Drupal was nothing short of a nightmare. Even by the time I managed to overcome (most of) the session challenges, the interfaces were completely incompatible. After a couple days, I moved on.

I moved on right into the arms of CakePHP.

I had taken a look at CakePHP about 6 months ago. I had been considering beginning an Open Source project to write a web-based University Security Department management system (more on this some other time), and CakePHP showed up on my radar. I took a closer look while considering it for our needs this time, though, and discovered the riches that CakePHP framework offers. After not even 30 minutes of playing with Cake, I had a functioning web-based database application. I was sold. Then, about an hour later, I happened upon Drake, a Drupal module that integrates CakePHP with Drupal. By the end of the day, I had a proof-of-concept CakePHP application running inside of Drupal via Drake. With some minor modifications to the Drake module, I had per-application permissions based on Drupal's module permission system.

The final piece of the puzzle was a bit of wrestling with the CakePHP MSSQL driver (only needed when using custom finderQuery in a hasManyAndBelongsTo relationship), and we had a working solution that met all the criteria. The first small application was finished in about 7 work days, and now I'm on the second (much larger) project.

[1]	Development times based on a single programmer working on the project "as he can" through the day, as he is regularly interrupted for support calls, walk-in repairs, etc. -- we are a small IT department.