Dougal Campbell's geek ramblings | Posts

Bug Chasing

2010-03-06T22:33:15Z

Okay, so in my post about Code Spelunking I mentioned about how working on a project can lead you to explore the code because you need to become more familiar with how the code works. But it can also lead you to explore the code to figure out why code doesn’t work. In this particular case, I spent many hours puzzling over why something didn’t work correctly, chasing down the root cause, and eventually finding a bug in the WordPress core. I documented the bug in Ticket #12394, provided a patch, and it was committed to core in Changeset [13561], which will be part of WordPress 3.0.

And how did I find this little buglet? As usual, it’s because I was doing something a little off the beaten track. I was working on some code which imports XML data into WordPress, on a scheduled basis (hourly, daily, weekly, etc). During testing, sometimes the images in the imported content would come through fine, and other times, they would be missing the src attribute, without which, there really isn’t an image, is there? So you’d view the post and there would be this big 300-pixel square hole with just the alt text where the image should have been.

At first, I didn’t know why it worked only some of the time. Then I saw the pattern that when I ran the code “manually” via a “Run now” button in my options screen, the images worked. But when the code ran via WP-Cron, they didn’t. At first, I thought it was a timing issue, and that maybe when the cron action hooks fired, maybe there was some piece of WordPress functionality that wasn’t loaded yet. But shunting my execution hook to run at a later point didn’t fix anything.

Next, I decided that one key difference when running manually versus running from cron was me — I was logged in as an admin. And, in fact, after some debugging, I determined that there was no user context at all when running from cron. When I modified the code to run as myself, the image tags came through cleanly. Well, I didn’t want to hard-code the program to always run as me, so I added a user selector to the options so that the owner of the posts could be set.

But then when I started testing again, with users of various roles, the problem cropped up again. In particular, it worked great for a user with the Editor role, but not for the Author role. Digging a little deeper into the differences between the two roles, the thing that jumped out at me is that Editors (and Admins) have the “unfiltered_html” capability.

You see, normally, when you write a post, it is sent through a series of filters which take your free-form writing, and turn it into cleaner HTML. One of these filters is called ‘kses‘ (which stands for ‘kses strips evil scripts’). This filter is especially important on multi-author blogs where you might not be able to give 100% trust to the other authors. Otherwise, one of them would be able to (for instance) put javascript in a post which would steal the cookie information from another user who reads the post. So it is the job of kses to ensure that only “safe” HTML is kept. This would also keep you from embedding things like YouTube videos, Java applets, and other fun useful things. So users with the unfiltered_html capability set in their profiles are able to post without this filtering.

This certainly seemed like a likely culprit, except for one thing: even when post content is filtered through kses, the HTML img tag is not filtered out. And neither is the src attribute on an image. That is specifically supposed to be allowed. An image is a perfectly normal thing to have in a post. So why, oh why, was my src attribute being stripped?

I started looking very closely at the kses library. It’s a rather hairy bit of code, full of complex regular expressions and state-machine logic. But when reverse-engineering how the attribute-cleaning bits work, I noticed something in one of the regular expressions: it was hardcoded to expect a space between the end of an attribute and the closing of a tag. In other words, it expected an image tag to look something like this:

<img width='400' height='300' src='people.jpg' />

But, since my data was coming from an XML source, there was no extraneous space. My image tags looked like this:

<img width='400' height='300' src='people.jpg'/>

Notice the subtle difference? There is no space between the final single-quote around 'people.jpg' and the /> which closes the tag. And because of the way the match was being done, kses was throwing away any attribute that abutted the tag-close in that fashion.

The next question was: was this (technically) a bug, or was kses just being strict about some rules of formatting? A quick search turned up the Empty Elements section of the XHTML spec, which covers the syntax for empty tags like img, br, and hr. The examples given there do not include a space before the end of the elements. Furthermore, this section points to the HTML Compatibility Guidelines, which show that adding a space is for compatibility with older HTML browsers. So, since the XHTML spec does not require the space, and WordPress is supposed to render XHTML code, the behavior in kses was definitely a bug, and not just bad manners. I quickly worked up a patch, submitted it on Trac, and brought it to the attention of the core team.

Fortunately, the WordPress system of filters allows you to alter just about anything on the fly, so I was able to “trick” the system into thinking that the posting user selected in my plugin had the unfiltered_html capability, even when they really didn’t. This allowed me to work around the bug while my plugin is running.

This bug was pretty minor in the grand scheme of things. Probably not many people had ever run into it. But after hours of puzzling over those broken image tags, it felt darned good to find it, and — more importantly — squash it. And after the release of WordPress 3.0, nobody will have to scratch their heads over it again. Yay me!

No related posts.

Dougal on WordPress Community Podcast

2010-02-24T02:33:50Z

I was the guest tonight on the WordPress Community Podcast, hosted by Joost de Valk and Frederick Townes. We talked about upcoming features in WordPress 3.0, my past work as a WordPress core developer, my WordPress plugins, microformats, and a smattering of other topics. My interview should be up later tonight.

Joost and Frederick do a great job with the interviews, so if you haven’t already, you should subscribe to the WordPress Community Podcast and give it a listen!

UPDATE 9:30 p.m. EST: The show notes are up now!

WordPress Code Spelunking

2010-02-19T04:56:17Z

Earlier today, I finished up a WordPress plugin that I’ve been working on for a client. Without divulging too many details, I’ll say that it wasn’t a huge project, nor was it overly complicated, but it was interesting. One fun thing about working on projects for clients, is that it often gives you a chance to explore areas that you might not have looked too closely at on your own.

For example, some of the areas of WordPress (and PHP in general) that I researched and learned more about were:

The WordPress Settings API
The WP Cron scheduling system
Using the WP_Http class to communicate with a third-party web service
Using the wp_upload_bits() function to create an image file
Learning some of the newer bits that were added to wp_insert_post() since I originally created it as part of my XML-RPC API work over 6 years ago
PHP’s SimpleXML functions (including some specifics around using xpath() with namespaced elements)

I used a combination of the links above, other articles I found, plus some good old exploration in the WordPress core to learn more about these topics. Exploring the core code was most helpful in learning about WP_Http, wp_upload_bits(), and some extras for wp_insert_post() that aren’t documented on the Codex page.

Digging around in these corners of the code can be much like spelunking. You’re digging down into places never seen by most people, and often finding little gems that you suspected might exist, but you weren’t sure until you actually saw them. If you haven’t poked around in the core code lately, maybe you should. I usually prefer to use command-line tools like grep and a text editor. But the online WordPress phpdoc xref is a nice way to look at the code, too.

Bing Maps Photosynth Mashup

2010-02-12T17:53:52Z

I’ve always been a big fan of Google Maps. But stuff like this is could sway me into using Bing Maps on a more regular basis:

Microsoft is combining geo-tagged, Creative Commons licensed photographs from Flickr, and integrating them with Bing Maps using their awesome Photosynth technology.

To try this feature yourself, you’ll need the Silverlight browser plugin.

WordPress 3.0 Multisite Terminology

2010-02-04T15:08:34Z

One of the big changes coming in WordPress 3.0 is the merge of the WordPress MU code into the standard stand-alone WordPress codebase. When WordPress 3.0 is released, you will be able to choose to install it either as a single site, or in multi-site mode. With the migration from WordPress MU (MultiUser) to WordPress 3.0 (Multisite), I think there is a need for us to clarify some terminology. On the wp-hackers mailing list, I’ve seen several instances where people used the word “site” to mean different things, depending on who is writing, and the context of what they are trying to say.There is some overlap currently between how we discuss an individual site within the setup, versus the overall system which contains those sites.

In WordPress MU, the term “site” tended to refer to the overall installation, covering multiple “blogs”. With WordPress 3.0 in Multisite mode, some people are conflating the terms “blog” and “site”, and it appears that “site” would be the preferred term. The new term for the overall system is “network”, as evidenced by the new Network Settings page. So instead of a “site” which contains multiple “blogs”, we have a “network” which contains multiple “sites”.

The change in terminology might cause some confusion at first, especially among people who are already familiar with WordPress MU, and are used to the old terms. But I think that saying “site” instead of “blog” is better, because it avoids confusion for people who are building websites that are not blogs. With that in mind, it makes a lot of sense to talk about a WordPress installation as a network of sites.

Recipe: Moroccan Chicken with Rice

2010-02-04T15:07:24Z

Let’s take a break from the tech talk, and I’ll share a recipe with you. No, not a programming recipe, but the food kind. Yes, sometimes I cook. I generally like cooking, though I’m not so fond of cleaning up the mess afterward. Recently, we had some split chicken breasts that we needed to cook up. I thought about just doing a standard baked chicken, but I wanted something a little. different. After perusing a selection of chicken recipes on SimplyRecipes.com, and reviewing the ingredients we had on hand, I decided to try combining ideas from two different recipes: Moroccan Chicken with Lemon and Olives and Chicken and Rice Casserole.

In our case, I was cooking for seven people (myself, my wife, her parents, and our three kids). We had two family packs of chicken breasts (4 split breasts per pack, at about a pound per breast), so I knew I was going to have to adjust some measurements. I had to make some best-guess changes based on the amount of chicken I was cooking, and the ingredients I had available. First, here’s the recipe as I actually prepared it, and afterwards I’ll discuss what changes I’ll probably make next time we make this dish.

Moroccan Chicken with Rice

A chicken and rice casserole with a Moroccan flair.

Created by Dougal Campbell, based on recipes from Simply Recipes.

Published Jan 22, 2010

Ingredients

8 split chicken breasts (approx 1 pound each).
4 tsp paprika
2 tsp cumin
2 tsp ginger
2 tsp turmeric
1 tsp cinnamon
1/2 tsp black pepper
1 cup Italian seasoned bread crumbs
6 Tbsp olive oil
6 cloves garlic, minced
1 onion, chopped
1/2 cup dried peaches, chopped
2 lemons
1 cup olives (pitted)
3 cups rice
5 cups water
1 cup red wine

Instructions

Coat a large baking dish (4 quarts, 10.5″ by 14.75″) with 2 Tbsp of olive oil.
Put the rice, water, and olives into the baking dish, then set aside.
Mix the dry spices and bread crumbs together in a large bowl.
Heat 4 Tbsp of olive oil in a large skillet on medium-high heat.
Pat each chicken breast dry, then coat them in the spice mixture.
Brown each breast in the skillet, about 5 minutes on each side (just brown them, you aren’t trying to cook through), then arrange the breasts on top of the rice in the baking dish.
Slice lemons thinly, and place lemon slices on top of the chicken breasts.
Reduce the skillet heat to medium, then saute the chopped onion until translucent, about 3 minutes (add more olive oil, if necessary).
Add the minced garlic to the skillet, and saute about another minute.
Add the red wine to the skillet, and stir until the wine reduces down almost completely.
Spoon onion mixture over the chicken breasts evenly.
Bake at 375F for about 50 – 60 minutes

Further details

Enough for approximately 16 servings.

Preparation and cooking time is approximately 2 hours

This recipe is medium difficulty and delicious. Even our kids liked it, though they picked out the olives. We served it with a side of steamed broccoli.

As prepared above, it came out quite good. The onion mixture baked into a sort of crispy topping with a pretty good flavor. The spices from the chicken seeped into the rice, giving it a pretty yellow tint and a subtle, exotic flavor. However, when we cook this again, I’ll probably make some adjustments.

The original Moroccan recipe that I based much of this on called for raisins, but we didn’t have any in our pantry. I figured that the sweetness of the raisins was supposed to balance out some of the sharper flavors from the lemon, ginger, and olives, so I substituted the dried peaches (that being the closest thing we had). But I think the peaches were a little too sweet, so next time, we’ll use raisins (about a cup would be right for the above measurements).

Also, I only cooked so much because we already had those family packs of chicken thawed out, and we needed to go ahead and get them cooked up. I could only fit six of the breasts in the baking dish, and I ended up cooking the other two in my skillet, with some water added. I think it might be interesting to repurpose the leftover chicken, perhaps as an exotically spiced chicken salad. Next time, I’ll probably cook a more appropriate amount, and I’ll probably start with skinless, boneless chicken breasts, and cut them up into bite-sized chunks before spicing and braising them.

I also didn’t have enough olives on hand. I used a mixture of some large green olives and kalamata olives, and only had about a cup. I would have liked about twice that amount. Also, the original recipe left the olives whole, but I sliced mine in half, lengthwise, just so they’d distribute better.

I think in the next iteration, I’ll probably leave out the red wine, and I might skip the lemons. I added the wine on a whim, just because it was in the fridge, but I think it detracted from the flavor of the caramelized onions. The lemons were okay, but in places where the peel cooked on the chicken meat, it tended to be slightly bitter. However, I might consider marinating the chicken in a little bit of olive oil mixed with lemon juice.

All-in-all, this dish was a success, and I think there is potential for lots of tweaking. One last change I might make would be to cover the dish with foil before baking. The rice was just a little dry on top after baking (though not annoyingly so), and the foil would help keep steam in. I’m also thinking that toasted, slivered almonds might be a nice addition to the rice.

Sound good? What do you think? Should I post more recipes?

Oh, and this article isn’t completely without a tech factor: I marked up the recipe with the experimental hRecipe microformat.

jQuery API site using WordPress as CMS

2010-01-19T17:12:13Z

I mentioned before that the jQuery project was going to switch from Mediawiki to using WordPress for their online documentation. They’ve recently pulled the trigger on that change, and you can now visit the new WordPress-powered site at: api.jquery.com

Performance and Backend

As mentioned before, we’ve switched away from using a MediaWiki backend to a new WordPress-powered backend. So far we’ve been very impressed with the performance. Every single page on the site is heavily cached and gzipped – with all media being served up from our CDN. We think you’ll find it to be a significant performance improvement over the old docs site.

via Pre Release Day 1 – The 14 Days of jQuery.

This seems to be an excellent example of a site using WordPress as a CMS. Each API method is created as a post, and they use categories liberally to make it easier for users to locate the methods they need based on usage. There’s also a nice AJAX live-search that narrows down results as you type. Obviously, they are using a custom tailored theme, and a permalink structure that produces easy to remember bookmarks like “api.jquery.com/slideDown/“. As an added bonus, they get user comments and an easier way to manage spam. Nice!

I imagine that there is more going on behind the scenes to help them manage the documentation structure of each API method, code examples, and demonstrations. Maybe some day we can coax them into giving us some more details about that.

Plugin Pages

2010-01-14T04:38:32Z

I have finally gotten around to adding proper pages on this site to provide information about my WordPress plugins. It’s still a work-in-progress, but it’s at least at a point that they are useful. When I get a chance, I’ll document the methods I used to automate things. But basically, I’ve created a couple of page templates, and I’m using the Plugin Info plugin to automagically bring in plugin metadata from the wordpress.org plugin repository.

The main Plugins page lists each plugin, with the plugin name and version, number of downloads so far, and an excerpt of the plugin description. On each plugin detail page, you get the full description, changelog, download link, and other stats, such as the minimum and last-tested WordPress versions.

WordCamp Atlanta

2010-01-09T15:54:44Z

As I begin writing this, Jane Wells is delivering her keynote at WordCamp Atlanta, speaking about the upcoming changes in WordPress 3.0. On the one hand, WordPress releases are ’serial’, so an 3.0 release doesn’t necessarily have any bigger changes than when we move from 2.8 to 2.9. It’s just the next step after the 2.9 release. But, that said, there are big changes coming. One of the biggest is going to be the merge of the single-site version of the code and the WordPress-MU code, which will now be referred to as “multi-site” capability. Other new bits will include the “Core Plugins”, which will be a set of plugins cultivated and updated along with the main core code, changes to menu management, more support for custom post types, and a new default theme, dubbed ‘2010′. There will probably be a handful of other smaller changes, but not included at this time is the revamp of the Media Manager, which will probably wait until version 3.1.

With this release, the core team is trying to set a more structured, focused standard for how to get everything done. At the last developer chat, the scope of development was set to those features mentioned above. Setting a strict feature scope this early in the kickoff of a new release is a new thing, and I think it will turn out to be a good thing.

I’ve been wanting for a while to get back into more involvement with WordPress development, so I’ve volunteered (along with several others) to help out with the “Healthcheck” Core Plugin. This plugin will examine the configuration of your server and alert you to anything that might prevent certain features from working correctly, or that might cause problems in the future (such as old versions of PHP or MySQL).

In other news, there are going to be some updates to the wordpress.org website, including improvement to the Ideas forum, and some BuddyPress functionality, including user profiles for all regiestered users. WordCamp.org will also get some BuddyPress love and see new features.

As usual, the future holds some very exciting changes. But I’m particularly happy to see that there is not only progress happening in how the code is handled, but in how the community is shepherded. I think just about everyone would agree that WordPress would not be where it is today without its great user and developer community. So, if you have ever contributed a patch, reported a bug, suggested a new feature, or even just thanked somebody who has — you’re a part of the community. Thanks!

Merry Christmas!

2009-12-24T15:31:57Z

I had hoped to post about some things before things got too hectic, but that didn’t happen. But I wanted to at least wish everybody a Merry Christmas (or which ever holiday variant you prefer), and a Happy New Year!