Developer Blog Posts

Review Board 1.0 Released!

chipx86@chipx86.com (chipx86) — Sun, 21 Jun 2009 07:47:36 -0700

Tonight, we hit a milestone in the Review Board project that we’ve been working toward for over two years. We finally pushed out our 1.0 release. A lot of blood, sweat and tears went into this release (okay, so not literally, but it was A LOT OF WORK!). The last few months in particular have been challenging, as we’ve had to solve some tricky bugs and scalability problems, but the end result is pretty great.

Just a short while ago, we announced the release and put up an overview of the entire release and product. We’ve already had some nice congratulatory e-mails and tweets, which is really nice :)

Some stats for this release:

2 years, 9 months, 25 days have passed since our first commit.
120 contributors have contributed to Review Board so far (in terms of code contributions).
2,019 commits were made.
899 review requests have been posted to our project’s actual Review Board server. 1,650 users are registered on there.
Our demo server, in comparison, has 2,082 review requests filed and 10,154 users.
938 bugs were filed. 812 were fixed.
232 feature requests were filed. 101 were implemented. Most remaining ones are scheduled for releases.
An estimated 200+ companies are now using Review Board. 26 have let us list them publicly.
The largest known Review Board install has over 83,000 filed review requests and over 2,000 users, doing upwards of 10GB of traffic per day.
5 presentations on Review Board are known to have been given, 3 by us, 2 by others.
552 users have joined our main mailing list, and 3,674 e-mails have been sent.

Now that Review Board 1.0 is out, we can get started on some awesome new features we’ve had planned. I have a little notebook full of ideas for our 1.1 and 1.5 releases (which may become 1.5 and 2.0, respectively, as this list grows). Some of the new features are actually ready to be committed within the next couple of days, so those of you using nightlies will start to see them soon.

We were accepted into this year’s Summer of Code, and have three students working on exciting projects for us, so hopefully we’ll start to see these trickle into the upcoming nightlies as well. Among these projects include diff viewer improvements (moved region detection, better whitespace-only change detection), IDE integration with Eclipse, and improved notification hooks and e-mail support.

We’re also working on providing support for third-party extensions, which will allow developers to extend Review Board in new, exciting ways without having to modify Review Board itself. This is especially handy for companies who wish to integrate better with their sandboxes, bug trackers or unit testing services. This will likely land in 1.5 (2.0?) at the earliest, as it’s a large change, but the code for this mostly works today. It’s just a matter of getting the codebase ready and figuring out what APIs we want to stabilize and expose.

As I mentioned in the release announcement, we’re planning a release party, tentatively on July 11th, 2009, in the Bay Area (somewhere around Palo Alto, CA). If any Review Board users want to join us, please RSVP!

Review Board: Summer of Code, Roadmap and Future Plans

chipx86@chipx86.com (chipx86) — Thu, 02 Apr 2009 21:53:39 -0700

Summer of Code

This year, we (the Review Board project) was given the opportunity to participate in Google’s Summer of Code. We’ve received some great student proposals so far, and I think we’ll see exciting work done on Review Board this summer.

The deadline for Summer of Code is coming up fast (April 3rd, 19:00 UTC). If you’re interested in working on Review Board and haven’t yet applied, it’s not too late, but you’ll want to hurry. Skim through our ideas page and, if you find something interesting or have a great idea not listed here, then apply and tell us your plans. I can say we’ve received several proposals so far for the installer and admin UI, so unless you feel strongly about either of those, you’ll increase your chances with other proposals.

We’re also offering free Review Board hosting for open source projects participating in Summer of Code. If you’re a mentoring organization and would like to give Review Board a try for reviewing and managing student code, go ahead and contact us and we’ll get you set up.

Roadmap

We’re finally nearing 1.0. We recently put out our 1.0 beta 2 release and are now in a feature freeze. We’re working to get some bug, performance and usability fixes in for beta 3, which I’m shooting for in a few weeks. Then we’ll branch for 1.0, put out a Release Candidate or two, and then finally release 1.0!

There’s a lot of really cool features planned after 1.0, namely extensions and policy customization.

Extensions

Our bug tracker is filled with feature requests for all kinds of things, ranging from bug tracker integration, instant messaging, a method for offering bribes for code reviews, and so on. We clearly can’t put all the requested features in the codebase, so we’ve decided instead to add support for third-party extensions. Coming soon, developers will be able to write extensions to Review Board in the form of Python modules to extend or alter the functionality of Review Board. The extension framework will allow them to do the following:

Access the database using the existing Review Board database models.
Add new database models for storing data.
Listen for signals (new review request published, review request submitted, etc.) and act on them.
Add custom URLs.
Replace existing URLs, for advanced capabilities such as replacing the diff viewer.
Add new API handlers.
Add “action” links to existing review requests and reviews.
Add columns and sidebar entries to the dashboard.
Add pages to the administration UI.
Communicate with other extensions.
Provide a settings page, which stores data in Review Board-provided models (we even auto-generate the settings page for the extension by default).
And more!

A lot of this already exists in a private development branch, and it will be one of our primary focuses as soon as 1.0 goes out.

In time, we’ll add a new section to the Review Board website where developers can list their extensions for download and for sale. Administrators will be able to browse and search for extensions directly from the administration UI and install them without having to even open a terminal (in most cases).

We’re hoping this will solve a lot of in-house integration issues. For example, many companies have custom sandbox architectures, bug trackers, and statistics software which they’ll now be able to tie in with Review Board.

Policy Customization

We’ve found that a lot of companies have very specific ways they want to handle policy and access restrictions. For example, many companies want to limit who can see certain parts of a repository (and therefore certain diffs), or want to allow anybody to create review groups, or want to disallow people from joining review groups. Some also want to dictate what constitutes approval for submitting a change.

We’re looking into the various requests and attempting to come up with a policy model that is flexible enough to handle these needs. One of the ideas is to provide some basic level of access control on a per-repository, per-path, and per-group basis. We’d then piggy-back on the extension framework to allow for more specific policy control. The advantage is that developers could write their own policy rules that interface with some part of their company’s infrastructure.

If people have any input on this, we’d love to hear it.

Improving browser performance in Review Board

chipx86@chipx86.com (chipx86) — Tue, 27 Jan 2009 12:14:29 -0700

This past Sunday, I landed a set of changes into Review Board that provide improved performance, such as aggressive browser-side caching of media and pages. It’s just a start, but has already significantly reduced page load times in all of my tests, in some case by several seconds. We implemented these methods for Review Board, but they’re methods that can be applied to any Django project out there.

There are several key things that Review Board now does to improve performance:

Tells browsers to cache all media for one year.
Only sends page data if new data is available.
Compresses all media files to reduce transfer time.
Parallelizes media downloads.
Loads CSS files as early as possible.
Loads JavaScript files as late as possible.
Progressively loads expensive data.

A lot of the performance improvements come straight from Yahoo!’s Best Practices for Speeding Up Your Site. We’re not doing everything there yet, but we’re working toward it. That’s a great resource, by the way, and I recommend that everyone who has even made a website before go and read it.

So what do the above techniques buy us, and how are we doing them? Let me go into more details…

Caching all media for a year

The average site has one or more CSS files, JavaScript files, and several images. This translates to a lot of requests to the server, which may leave the site feeling slow. On top of this, a browser only makes a few requests to a server at a time, in order to avoid swamping the server, which will further hinder load times. This happens every time a user visits a page on your site.

Aggressive caching makes a huge difference and can greatly reduce load times for users. Review Board now tells the browser to cache media files for a year. Once a user downloads a JavaScript or CSS file, they won’t have to download it again, meaning that in general the only requests the browser needs to make is for the page requests and AJAX requests.

The big pitfall with long-term caching is that the cached resources can go stale. For example, if a new version of an image was uploaded, the browser wouldn’t even know about it, since it was told it should keep its old version for a year before checking again.

We solve this by introducing “media serials,” timestamps that are appended to all media paths. Instead of caching /js/myscript.js, the browser would cache /js/myscript.js?1273618736.

These media serials are computed on the first page request by our djblets.util.context_processors.ajaxSerial context processor. This quickly scans all media files known to the program, finding out the latest modification timestamp. It then provides a {{MEDIA_SERIAL}} variable for templates to append to media URLs as part of the query string.

The benefit to this method is that we can cache media files for a year and not worry about users having stale cached resources the next time we upgrade a copy of Review Board. The filenames requested will be different, browsers will see that the new file is not in the cache, and make a request, caching the new file for a year.

Only send page data if new data is available

Aggressive caching of media files is great and saves a lot of time, but it doesn’t help for dynamically generated content. For this, we need a new strategy.

When a browser makes a request, it can send a If-Modified-Since header to the server containing the Last-Modified value it received the last time it downloaded that page. This is a very valuable header, and there’s some things we can do with it to save both the server and the browser a lot of trouble.

If the browser sends If-Modified-Since, and we know that no new data has been generated since the timestamp provided, we can send an HttpResponseNotModified (HTTP response code 304). This will tell the browser it already has the newest version of the page. The sooner we do this, the better, as it means we don’t have to waste time building templates or doing any expensive database queries.

Djblets, once again, provides some functions to help out here: djblets.util.http.set_last_modified and djblets.util.http.get_modified_since.

The general usage pattern is that we first build a timestamp representing the latest version of the page. This could be the timestamp for a particular object that the page represents. We then check if we can bail early by calling:

if get_modified_since(request, timestamp):
    return HttpResponseNotModified()

Further down, after building the page, we must set the Last-Modified timestamp, using the same timestamp as above, like so:

set_last_modified(response, timestamp)

We’re using this in only a few places right now, such as the review request details page, but it drastically improves load times. If the review request hasn’t changed and nobody’s commented on it since the browser last loaded the page, a reload of the page will be almost instant.

Compress all media files

Our Apache and lighttpd config files now enable compression by default. By compressing these files, we can turn a relatively large JavaScript file (such as the jquery and jquery-ui files) into a very small file before sending it over to the browser. This reduces transfer times at the expense of compression/decompression time (which is small enough to not worry for deployments of this size, and can be offset by caching of compressed files server-side).

Parallelize media downloads

It’s important to not mix loads of media files of different types. The browser parallelizes media downloads of the same type, in page load order, but if you load one CSS file, one JavaScript file, another CSS file, and then another JavaScript file, the browser will only attempt one load at a time. If you load all the CSS files before all JavaScript files, it will parallelize the CSS file download and then the JavaScript downloads. By enforcing the separation of loads, we can achieve faster page download/render times.

Load CSS files as soon as possible

Loading CSS files before the browser starts to display the page can make the page appear to load smoother. The browser will already know how things should look and will lay the page out accordingly, instead of laying the page out once and then updating that once the CSS files have loaded.

Load JavaScript files as late as possible

JavaScript loads block the browser, as the browser must parse and interpet the JavaScript before it can continue. Sometimes it’s necessary to load a JavaScript file early, but in many cases the files can be loaded late. When possible, we load JavaScript files at the very end of the document body so that they won’t even begin downloading until the page has rendered. This provides noticeable performance for script-heavy pages.

Progressively load expensive data

There are types of data that are just too expensive to load along with the rest of the page. For a long time, Review Board would parse and render fragments of a diff for display in the review request page, but that meant that before the page could load, Review Board would need to do the following:

Query the list of all comments.
Fetch every file commented on.
Apply the stored patch to each file.
Diff between the original and patched files.
Render the portion of the diff commented on into the page.

This became very time-consuming, and if a server was down, the page wasn’t available until everything timed out. The solution to this was to lazily load each of these diff fragments in order.

We now display a placeholder table for each diff fragment in roughly the same size of the rendered fragment (to avoid excessive page scrolling on loads). The table contains a spinner showing that something is happening, and, one-by-one (to avoid dogpiling) we load each diff fragment.

The code to render the diff fragment, by the way, takes advantage of the If-Modified-Since header and is also cached for a year. We use an AJAX_SERIAL (same principal as the MEDIA_SERIAL above) to allow for changes in new deployments.

With these caching mechanisms in place, the review request page now loads in roughly a second in many cases (or less once cached), with diff fragments coming in lazily (and then almost immediately on future loads).

More to come…

This was a great first step, but there’s more we can do. Before we hit our 1.0 release, we’re going to batch together all our CSS files and JavaScript files into a couple of combined files and then “minify” them (basically compressing them in such a way to allow for smaller files and faster load times of the interpreted data).

Again, these are techniques we’re now making use of in Review Board, but they’re not in any way specific to Review Board. Anyone out there developing websites or web applications should seriously look into ways to improve performance. I hope this was a good starting point, but seriously, do read Yahoo!’s article as well.

Infect your application with Parasite!

chipx86@chipx86.com (chipx86) — Tue, 20 Jan 2009 02:52:47 -0700

Ever find yourself stuck debugging an application because the UI is just doing something weird that you can’t track down? Maybe a widget isn’t appearing correctly, or you just need more information about the overall structure and logging statements aren’t doing you much good. Debugging complex UIs can be a pain.

We’ve had some real challenges at VMware, due to the complexity of our applications. It was enough to drive me mad one day, so rather than write more logging statements, I wrote Parasite.

Parasite is a debugging tool that David Trowbridge and I have been working on to give developers an interactive view of their entire application’s UI. It provides a number of really useful features, including:

See the entire widget hierarchy of your UI.
Watch properties update live.
Modify existing properties on a widget.
View all registered GtkActions.
Toggle GTK+’s debugging of graphic updates.
Inject custom code while the application is running.

Yes, you can inject new code into an application. With Python. Parasite runs in-process as a GTK+ module, so it has access to some internals of your application. We provide a Python shell equipped with PyGTK support for creating and modifying your UI on-the-fly, regardless of the language it was written in. Handy when you want to test out new concepts for a UI without writing new C code.

David has a nice screencast available showing some of what Parasite can do.

For more information on Parasite, including screenshots, a mailing list, and where to get the source code, see the Parasite homepage.

Review Board 1.0 alpha 1 released

chipx86@chipx86.com (chipx86) — Mon, 19 Jan 2009 23:20:13 -0700

Roughly two years ago, David Trowbridge and I began development of Review Board for use in our open source projects and our team at VMware. During that time, we’ve turned Review Board into a powerful code review tool that works with a variety of version control systems. Most of VMware has moved over to it, as have an estimated 50-100 companies world-wide. We’ve had over 100 contributors to the project, people providing volunteer support on the mailing list, and people have developed third party tools for integrating with Review Board.

After all this time in development, with this many people contributing, we decided it’s probably time to get a release out there. Sure, we could have done this a long time ago, but there’s a number of large things we were hoping to get in (a recently-committed UI rewrite, for instance). Now that we have most of the major features we want for our 1.0 release, we decided it was time for an alpha.

Over the coming months, we’ll be working on stabilizing the codebase, fixing a few large remaining usability quirks, enhancing performance, and writing some proper documentation (which is coming along nicely).

We’re eager to get a quality product out there and to begin development on the next release. There’s a lot of neat things planned:

Support for writing extensions to Review Board.
A fully-featured API covering every operation you’ll need to perform.
Some degree of policy support (specifying which users/groups can see which parts of a repository, for instance).
Reviews with statuses other than “Ship It”. This will probably be customizable to some degree.
Possibly some theme customization to allow Review Board to blend in better with corporate sites, Trac installs, etc.

Along with this, I plan to roll out a new website for the project that will have a browseable list of third party extensions, apps, Greasemonkey scripts, and more.

We have more information on our release on our release announcement.

Armed robbery in progress

chipx86@chipx86.com (chipx86) — Fri, 26 Dec 2008 06:04:46 -0700

Our Christmas this year was very nice, with lots of good presents, time spent with family, delicious food, and an opportunity to foil an armed robbery. Yes, an armed robbery. How do I continually get myself into these situations?

While heading back home from my stepdad’s parents’ house, I spotted a guy standing in a small field at a street corner near my parents’ house, holding what looked to be a gun. The guy was probably late teens/early adult, Caucasian, and dressed in all black. His actions looked instantly suspicious. He was pointing the gun to the ground and made some motion as if he was checking the ammo or something. He seemed pretty lost in his own world, apparently not even realizing he seemed very suspicious.

My Mom was driving, and decided to slowly drive away from home instead of toward it, and then turn around, giving us time to watch and see what was happening. We knew there was a gas station across the street and wanted to see if he was going to head in there. If so, there was a good chance we’d be witnessing a robbery.

Sure enough, he started walking across the street to the TowerMart, a gas station/convenience store. We parked the car and my Mom grabbed my phone and called 911 while my brothers and I carefully watched from behind a building, making sure we weren’t noticed. We saw the guy walking back and forth at the side of the building, looking very nervous. He was wearing long sleeves with his left hand exposed and his right hand (which was holding the gun) completely covered. As my Mom talked to the police, we continued to watch, and the guy eventually psyched himself into going into the store. He put the gun in his pocket and went in.

The store was pretty crowded, and we weren’t sure what to expect. Would people be running out screaming? Would people be locked in? Would we hear gun fire? Or would he just leave?

We never saw him actually leave, but we could only see the one side of the building. A couple minutes later, six police cars drove up, two right beside us. Two cop cars pulled up alongside us. One of them hopped out, grabbed a big ol’ semi-automatic, and approached the building, pointing the gun, ready to fire. Three other cops did the same, surrounding the building. A couple other cops went in and got people out of there, with another couple cops asking those people if they had seen anyone matching the description we gave, or saw other suspicious activity.

They spent some time going through the TowerMart and eventually came back out once they were sure he wasn’t hiding in there. It seems at some point, he had left the building. We weren’t able to see when. However, he was definitely in there. Practically everyone they talked to noticed him, as he was nervously walking in circles around the building, completely covered. He had either grown nervous with the number of people in there, or heard the sirens come. Either way, he got out of there before doing anything.

As most of the cops started to exit the building, the one who had pulled up closest to us walked up and asked what we saw. Then he said, “Oh, I talked to you before.” We all thought that was strange, since he hadn’t actually talked to us, but then he pointed out that he remembered us from the assault back in May. Good memory.

We walked with him to the field and pointed out roughly where we saw him standing. He found the footprints and was calling out the detectives to take pictures or whatever. We chit-chatted briefly before returning home. He told us we very well may have saved the store and a lot of people from experiencing a robbery on Christmas Day. I just hope the guy didn’t make another attempt elsewhere.

We found out that an hour or two later, the police were still all over that place. About an hour ago, they apparently had arrested someone not too far from here. Whether related or not, I don’t know. Given that they had video footage of the guy, testimonies and descriptions from a bunch of people in the store, and his footprints, it’s probably only a matter of time.

The rest of Christmas proceeded without police intervention. Merry Christmas!

Ubuntu and Desktop Notifications

chipx86@chipx86.com (chipx86) — Thu, 25 Dec 2008 07:09:12 -0700

This past Monday, Mark Shuttleworth wrote about their plans for an overhaul of desktop notifications (the little popup bubbles telling you someone IM’d you or there’s updates available). Many people have asked me about this, some concerned, and wanted to know what I thought. Being the maintainer of libnotify, notification-daemon and the Desktop Notifications specification, some people were concerned that this work would supersede my own.

The reality is, this isn’t a replacement of my project. This is a new joint effort between Ubuntu, KDE, and myself. What’s been written in Mark’s post may not end up being the end result. We’re still deciding how this will progress, and Ubuntu wants to experiment a bit. Aaron Seigo’s post on the subject sums up a lot of my thoughts on this, and I recommend reading it, though I’ll go further and discuss where this all started and how it’ll affect the project.

First of all, libnotify/notification-daemon isn’t going anywhere. It’s not being replaced, nor is an alternate spec being drafted. Ubuntu’s User Experience team has some new things they want to try, and that’s fine. The plan is to see how this stuff goes in their codebase, with the intention of migrating code back upstream.

A couple of weeks ago, during the Ubuntu Summit, I was invited to speak with some of the developers and User Experience guys from Ubuntu about their plans. They showed me the mockup that’s on Mark’s blog and told me about their plans. They have things they want to do that could definitely improve the experience. Some things are pretty controversial, such as the removal of actions on notifications. We sat down, discussed and debated various aspects of the proposals for a while, and I believe reached a general course of action for the project. The highlights include:

Actions will be removed for applications packaged in Ubuntu. The developers will try to replace them with something that they feel makes more sense, with the hope of pushing these changes upstream.
The released notification-daemon will, I believe, support actions, since many non-packaged applications do use them. They will, however, appear as old-style notifications and not appear in the new window stack demoed in Mark’s blog post.
We’ll be drafting new additions to the notification spec in order to address the needs of KDE and Mozilla.
The work will be done by their developers in either a fork or a whole new notification-daemon implementation, allowing them a greater ability to experiment. These changes (or parts of them) will make their way upstream. It will be spec-compatible so users won’t have to worry too much about losing features (aside from actions, perhaps).
In the end, it’ll likely be that the Ubuntu theme specifically works this new way, and that other themes will work differently (they may support actions more directly, for example).

Now I should point out that I don’t believe actions are a bad thing. There’s many use cases where actions are very much warranted, and it seems Aaron agrees. While the Ubuntu team has discussed the possibility of deprecating this in the spec, I believe the end result will be that actions will live on. I’m also pretty adamant that upstream notification-daemon will still support actions and some other features. Ubuntu can choose what experience to give their packaged applications, but that may differ a bit from what we decide upstream.

Time will tell how this all turns out. I personally think it’s great that there’s some momentum on this project. I know I haven’t had much time to work on it as of late, between VMware and Review Board, which is why getting some new people on board with fresh thoughts will probably be a good thing.

On a related note, I’d like to welcome Andrew Walton to the project. He’s going to be working as a co-maintainer and helping out when I’m busy (which will be a lot of the time for a while).

Over the next month or two, the project should start to pick up. We’re beginning to look at ways to improve the spec and at what work needs to be done in the near future for the project. These discussions will take place on xdg-list.

And with that, I wish everyone a Merry Christmas (or just a very good December 25th for those who don’t celebrate Christmas)!

Fixing broken key codes in VMware on Ubuntu 8.10

chipx86@chipx86.com (chipx86) — Tue, 23 Dec 2008 09:43:22 -0700

I recently upgraded a system to both Ubuntu 8.10 and VMware Workstation 6.5, and while using it, I realized a number of keys were broken. The down arrow invoked the Windows Start Menu, for instance. Pressing Alt caused the key to be stuck. I knew this stuff worked in 8.04, but it certainly wasn’t in 8.10.

After some digging around, I found a forum post (lost the link, sorry) that fixed this. It’s worked pretty well for me, and so I thought I’d share for all those who have hit this issue.

Edit your $HOME/.vmware/preferences file and add:

xkeymap.keycode.108 = 0x138 # Alt_R
xkeymap.keycode.106 = 0x135 # KP_Divide
xkeymap.keycode.104 = 0x11c # KP_Enter
xkeymap.keycode.111 = 0x148 # Up
xkeymap.keycode.116 = 0x150 # Down
xkeymap.keycode.113 = 0x14b # Left
xkeymap.keycode.114 = 0x14d # Right
xkeymap.keycode.105 = 0x11d # Control_R
xkeymap.keycode.118 = 0x152 # Insert
xkeymap.keycode.119 = 0x153 # Delete
xkeymap.keycode.110 = 0x147 # Home
xkeymap.keycode.115 = 0x14f # End
xkeymap.keycode.112 = 0x149 # Prior
xkeymap.keycode.117 = 0x151 # Next
xkeymap.keycode.78  = 0x46  # Scroll_Lock
xkeymap.keycode.127 = 0x100 # Pause
xkeymap.keycode.133 = 0x15b # Meta_L
xkeymap.keycode.134 = 0x15c # Meta_R
xkeymap.keycode.135 = 0x15d # Menu

That should do it!

Twittering as Review Board Approaches 1.0

chipx86@chipx86.com (chipx86) — Sun, 07 Dec 2008 06:58:23 -0700

On the road to 1.0

We’re getting very close to feature freeze for Review Board 1.0. The last couple of major features are up for review. These consist largely of a UI rewrite that simplifies a lot of Review Board’s operations and moves us over to using jQuery. This will go in once it’s been reviewed and tested in Firefox 3, IE 6/7, Opera and Safari.

There are some preliminary screenshots up of the UI rewrite. Some things will be changing before this goes in, but it should give a good idea as to the major changes (if you’re already a Review Board user).

In the meantime, we’re working to get some other fixes and small features in, and I’m beginning work on a user manual. I’m not sure how much will get done for 1.0, but with any luck I’ll have a decent chunk done.

Twittering the night away

I’ve just set up a @reviewboard user on Twitter that I’m going to try to keep up-to-date as progress is made. This should give people a decent way of passively keeping track of updates if they’re Twitter users.

Barter system?

Britt Selvitelle of Twitter fame just sent me a great screenshot of a barter system for Review Board. Can’t get someone to review your code? Offer them something in exchange!

As some people know, we’re planning to have extensions in the next major release (1.5 or 2.0). This would be a fun little extension to have :) Maybe I’ll write it as part of a tutorial.

libnotify 0.4.5 and notification-daemon 0.4.0 released

chipx86@chipx86.com (chipx86) — Thu, 20 Nov 2008 11:58:11 -0700

It’s been a long time since I’ve really put out a libnotify or notification-daemon release. Review Board and VMware have taken up a lot of time the past year, so development has been slow, but I think these releases are worth it.

libnotify 0.4.5 [Release Notes] [Downloads]

This is mostly a small bug fix release, but introduces support for associating a notification with a GtkStatusIcon. This allows for better position tracking when notification icons are jumping around.

notification-daemon 0.4.0 [Release Notes] [Downloads]

This is a large feature and bug fix release. The major two features are multi-head support and a new control panel applet for specifying the notification theme and corner origin.

The multi-head support will show standard (non-positioned) notifications on the monitor the mouse is currently on. Previously, they would only appear on the primary head. This helps to notice new notifications, and fixes problems with some people who have a multi-head setup but only have one monitor on or in view. Also, notifications that appear on the edge of a monitor in a multi-head setup will no longer cross the monitor boundary, and will instead just stay on the correct monitor.

The new control panel applet makes it easy to switch notification themes or to specify which corner notifications should appear from. This will be expanded in the future.

As mentioned above with libnotify, notification-daemon can now track when status icons have moved so that the notifications are no longer in the wrong position during startup.

There’s a bunch of other nice little bug fixes that are mentioned in the release notes.

Thanks for the patience, everyone!

Racism, Sexism, and now Prop 8

chipx86@chipx86.com (chipx86) — Tue, 04 Nov 2008 07:40:30 -0700

I found out this evening, to my dismay, that my site was littered with “Yes On Prop 8″ banners. Now, for those who live outside California and haven’t been following this, Prop 8 is a measure designed to introduce an amendment to the California constitution to ban gay marriage, basically ensuring that certain people would never have the same rights as others in this state.

Now I normally try to stay away from politics on my blog, but I want to talk about two points.

First, I don’t mind banners on my site that are designed to sell a product. People generally understand that an ad for an online web service or a product of some sort is not necessarily endorsed by the site it’s running on. Ads are everywhere and most people generally get that it’s provided by an ad service, and just ignore them.

What bothered me about the Yes On Prop 8 ads is that it felt as if I’m endorsing Prop 8. Somehow, it feels wrong to me. I’m not morally outraged about Sun Microsystems wanting to sell a server system or Microsoft wanting to sell an office suite. I am outraged about Prop 8. Products are fine to advertise on my site. Controversial freedom-limiting propositions I’m completely against are not.

I look back in our history and see that by and large, our generation is regretful of how we’ve mistreated people in the past. Shooting Native Americans used to be fine. Stripping away their rights and making them unequal was socially accepted. It was completely understood that if you’re black, you’re property. If you’re a women, you had no rights to vote and your opinion didn’t matter.

I like to think we’ve come a long way from that. People pride themselves on how we’re more mature now. Black, white, red, men, women. It doesn’t matter. This is the land of the free, the land of equality. So why is it that it’s still okay to discriminate against someone because their love of someone makes you feel uncomfortable?

It’s okay to not feel comfortable with gay marriage. A lot of people don’t. But do you feel more comfortable being part of a group of people that knowingly discriminated against another group, stripped them of certain rights that you yourself enjoy, simply because something you don’t have to deal with on a daily basis makes you feel uncomfortable to think about? Are you going to be okay with the thought of your grandkids or your great-grandkids feeling embarrassed because of how you voted, like how you feel about your great-grandparents’ racism? How much is preventing marriage for two people who love each other, in order to feel less uncomfortable, worth to you?

The Yes On Prop 8 advertisements often show the clip with the mayor of San Francisco saying “It’s going to happen, whether you like it or not!” It’s a good strategic clip for them to have chosen, as it can be interpreted as him saying “you have no say, we’re forcing gay marriage on all of you.”

I see it another way. I see gay marriage being inevitable not as an attack, but as the inevitable rise in tolerance that, over time, we’ve come to develop in this country. As a country, we don’t have the best track record of tolerance to new things, but we always mature in the end. This is not the last time we’ll face such mass intolerance and the limiting of rights of a group of people, just as this will not be the first time that we as a people will overcome our fears and begin to see us all as being equal.

So this is important. It’s not just about your level of comfort with those who live a different lifestyle. It’s about equality. It’s about overcoming personal fears. It’s about making an effort to keep this country on a path of freedom. Because if we start going back to our old ways of discrimination and fear, all we’re doing is regressing and limiting the rights of others out of some fear of the world spiraling into chaos. We’ve worked to abolish racism. We’ve worked to abolish sexism. The world is still here. We can do this again.

Vote no on Prop 8.

Djblets and Review Board moving to jQuery

chipx86@chipx86.com (chipx86) — Thu, 18 Sep 2008 23:46:49 -0700

When we first began development of Review Board and its Django utility package Djblets almost two years ago, we needed to pick a JavaScript library to use. There were several good ones available at the time, and after evaluating several options we chose the Yahoo! UI Library (YUI) and YUI-Ext, an extension library to YUI. Both were excellent and have served us well over the past two years.

However, YUI-Ext itself is really no more, which means we needed to choose something new. We could have continued to bundle it and YUI with Review Board, but users of Djblets would have to hunt down a copy and load in all of YUI and YUI-Ext just to use such features as datagrids. This was, to say the least, inconvenient.

YUI-Ext and the Licensing Situation

Now I said YUI-Ext is no more, but really it’s still around, just in a new form. It had been renamed to ExtJS and became its own independent library. While compatible with YUI, most of the functionality we needed was really provided by ExtJS, meaning we didn’t really need both. Over time, YUI evolved as well, so we began looking into the differences and figuring out which to go with.

ExtJS, it turns out, was a non-starter for us. Unlike YUI-Ext before it, the ExtJS licensing terms were a bit restrictive and, frankly, confusing. Open source projects could use it under the GPL3, but commercial products required a special license per developer, which is $289 per developer. The GPL3 itself is unclear with regards to our situation. We’re an open source project, but we’re MIT-licensed. What if a company wants to modify something for their own use and not release it to the public? What if down the road someone wants to offer commercial extensions to Review Board? We’d have to choose the commercial license to be safe, but then contributors would also need to pay $289 for a license, wouldn’t they?

Looking Again at YUI

We didn’t want to step down a dangerous road with ExtJS, so we started to look again at YUI. It’s definitely come a long way and I must recommend it for developers looking for a strong toolkit with good UI support. They provide good documentation, many examples, and base functionality for nearly everything.

However, there’s a lot we’d have to rewrite to move to it, and if we had to rewrite the code anyway, I wanted to look around a bit at the current generation of JavaScript toolkits. Especially since our needs have changed over the years and we’re looking for something with a lighter footprint. We’re also moving away from the dialog-centered UI we have on some pages, which is where YUI shines.

jQuery

We ended up deciding to go with jQuery, partially because of the footprint and partially the clean way in which scripts can be written. As an experiment, I converted our Djblets datagrid code to jQuery. The result is a smaller, much more readable, reusable, cross-browser script. I was impressed by what jQuery let me do, and by the size.

I did some comparisons on the number of files downloaded and the size of the files, using the Review Board dashboard as a test.

With YUI and YUI-Ext, we were loading 7 JavaScript files, excluding our own scripts, at a total size of 376KB (when minified).

With jQuery, we loaded only 2 JavaScript files, at a total size of 128KB.

Our datagrid.js file also went down from 13KB to 9KB largely due to some of the niceties of jQuery’s API.

This is a pretty significant savings, a whole 252KB, which could be a couple of seconds on an average DSL line. And it’s not just the file size but the number of requests, which can make a big difference on a heavily accessed web server.

What This Means for Djblets

Developers using Djblets can now use datagrids without needing to do anything special. Djblets bundles jQuery, which it will use by default unless you choose to override which jQuery script is being used. This means all you need is an install of Djblets set up and you’re ready to use it!

Going Forward

We’re working on migrating the rest of Review Board to jQuery. This will take place over the next few weeks, and is one of the last major things we hope to do for our 1.0 release.

Review Board Roadmap and Donations

chipx86@chipx86.com (chipx86) — Sun, 17 Aug 2008 23:31:47 -0700

Roadmap

With the upcoming release of Django 1.0 in the next few weeks, we decided it was time to formalize a roadmap for Review Board 1.0. The roadmap provides a good overview of what users can expect for our release, and what it will take to get there.

At this point we’re asking for people to contribute wherever possible. The big thing is fixing bugs targeted for the 1.0 release. We’d also like some help in finalizing unit tests.

Quality Control

We’re doing what we can to improve quality control in Review Board. For a lot of people, Review Board works great, though setups often differ and some users hit issues that others never see. For this, we’re trying to improve our unit tests to catch these various cases. When people submit patches, we’d greatly appreciate unit tests to cover the new code, and in some cases will require them for the code submission.

Selenium

We will soon start using Selenium in our unit test process to simulate user action in various web browsers. Selenium allows for remote-controlling a web browser, simulating clicks, text input, and other user actions and checking the results. Over time, when our Selenium test suite is more complete, we should be able to catch browser-specific problems a lot more easily.

Buildbot Server

Another issue users have hit lately is breakages due to changes in Django for the 1.0 release. As things calm down there, this will become less of an issue, but we’ve put things in place to catch these problems before users do.

We have just set up a buildbot server that will perform a full build and run the test suite whenever there’s a code check-in to Review Board, Django, Djblets or Django-Evolution. It will then notify us when there’s a new breakage. Users can check the build page before updating just to make sure they won’t hit a major problem. Later on, our buildbot server will generate nightly builds and handle Selenium tests.

We have a limited number of servers to test with. If you have server space and resources to donate and would like to run a BuildBot Slave server, let me know. We’re looking to set up slaves to test various combinations of the following:

Python 2.5
Python 2.4
Django SVN trunk
Django 1.0
Windows 2000, XP and Vista
Internet Explorer 6 and 7 (for Selenium tests)
Opera (for Selenium tests)
Firefox 2 and 3 (for Selenium tests)

Sandbox

Our BuildBot server is also set up to allow us to test code changes before we submit the code. Running a sandbox build of our pending code will cause all build slaves to run the entire test suite. This ensures that we don’t break things accidentally.

If you’re a contributor working on large patches for Review Board and would like to have access to the sandbox, please post to the mailing list and we can work with you on getting an account set up.

Installation Improvements

We’re working to make the installation experience much easier. I’m in the process of creating Python easy_install packages for Review Board and Djblets. Soon, users will be able to simply easy_install ReviewBoard to get going instead of checking out the development tree. I’m hoping to create both nightly builds and release builds.

Code will soon go in to move the entire project configuration into the administration interface. Modifying settings_local.py and restarting the server will be a thing of the past. All that will be left there will be a few site-specific settings and the database settings. Expect this to go in real soon.

A tool is in development for helping to generate the initial Review Board server tree based on an installed reviewboard Python module (using easy_install) and generating the web server configuration files. This will hopefully take care of a lot of problems people hit when trying to get their server configuration right the first time.

And last but not least, before 1.0 we will have a first-time installation page that handles the creation of the initial settings_local.py and the adding and checking of repositories.

So in the end, the installation process will be something along the lines of:

sudo easy_install ReviewBoard
sudo rb-install-site /var/www/reviews.mycompany.com
Fill out the fields presented.
Hand-tweak the configuration files if needed.
Go to the page for the new Review Board server, fill out the fields and finish the install.

This is the goal, anyway. We’re going to try to get as close to this as possible for 1.0.

Donations

Review Board has become a full-time project for us. Though it got its start at VMware, it’s really a personal project developed in our spare time, not a project run by VMware. As the project grows, we’ve been putting more time, energy and money into it.

Hosting fees have started to become large, given that we’re now hosting the main project website, the main Review Board server for our code reviews, the demo server, the Google Summer of Code review server and the BuildBot server and slaves. Down the road, we have many plans that will also require funding.

To help cover our costs, we’re now made it easy to donate to the project. If Review Board has helped you, your company, team, or project and saved you money or time over alternative solutions, maybe you’d like to help give back to keep our project going. Every bit helps.

Happy birthday Review Board!

chipx86@chipx86.com (chipx86) — Sun, 18 May 2008 10:37:24 -0700

One whole year…

While attending SuperHappyDevHouse, David and I realized that it was the one year anniversary of Review Board’s public announcement. (This was the evening of May 17th. I’m just a few hours late in getting this post up.)

It’s been a pretty awesome year for Review Board. What begun as a small project intended for use in our team at VMware and in our personal open source projects turned into a large project with a great community of developers and users. It’s now being used by dozens of companies and projects (some of which have given us permission to list them publicly) and has had code contributions from over 35 users. To everyone who has contributed to the project, we’d like to give our thanks. Review Board wouldn’t be the tool it is today without your help.

I’d like to list some of the major things that have happened in the past year.

Review Board has been adopted by one or more teams at around 40 companies (including Yahoo! Search, VMware, and Tripwire).
Received contributions from over 35 users.
Added support for CVS, Subversion, Perforce, Git, Mercurial, and Bazaar.
Added support for useful features such as interdiffs, screenshot commenting, syntax highlighted diffs, customizable dashboard for keeping track of review requests, status reports, a full JSON API, and more.
Several presentations were given at various companies and conferences on Review Board by third parties.
One public presentation was given by us at LugRadio Live USA 2008 (view the presentation)
We’ve started hosting projects for this year’s Google Summer of Code on our Summer of Code server.

That’s just a small sampling. There have been many changes made to stabilize the codebase, improve usability, and generally make the product more awesome.

And of course there’s a number of things happening in the near future:

Monotone support.
A new, improved administration UI.
Extensions support, allowing Review Board to be extended in a variety of ways and integrated into other products.
A new set of command-line tools and a Python library for working with Review Board servers.
Support for P4V (for Windows Perforce users).

Database Migration and Parent Diffs

Today, a change went in that improves Review Board in two fundamental ways.

First, we now have a fast, powerful way of doing database migrations in-place, without dumping and loading the database. While this should make life easier for users and allow us to modify the database without fear of breaking people’s installs as often, it will also save administrators of large Review Board servers a lot of time. Before this change, the VMware database (of over 25,000 review requests) took at least 30 minutes to do a full migration. Now it shouldn’t take more than a minute.

Second, we’ve just landed the initial support for diffs based on parent diffs. If you’re a user of a distributed version control system (such as Git or Mercurial) you’ll appreciate this. Before, your change had to apply against revisions in the master repository, making it impossible to put up a review request for a change on a sub-branch of a branch when the parent branch didn’t exist on the server. Now, a diff of the parent branch can be uploaded along with your sub-branch’s diff.

Confused? Maybe an example will help. Say you’re working on a large restructuring change in your “code-restructure” branch in your local Git checkout. You have a topic branch off of your “code-restructure” branch with changes you want to put up for review. This used to be impossible without including the whole “code-restructure” branch’s changes in your diff as well, but now, you can put up a diff of the topic branch along with a diff of the “code-restructure” branch, and the topic branch’s diff will appear on Review Board, ready for review.

The backend code for this is now in Subversion. We’ll be adding support to the post-review tool shortly, making this accessible to anyone. Review Board will become fully usable for distributed version control systems.

Are you using Review Board?

Review Board is gaining popularity, and more companies are beginning to use it. If you work for a company or an open source project that’s using Review Board and can give us permission to list you on our Happy Users page, please let us know!

And here’s to an even better year for Review Board.

Free Review Board Hosting for Summer of Code

chipx86@chipx86.com (chipx86) — Thu, 01 May 2008 08:45:00 -0700

Google Summer of Code 2008 is on, and soon students will begin coding, which means mentors will be reviewing code. While the Review Board project is not a participating project in this year’s Summer of Code, we felt Review Board could help with the whole process, improving things for both the students and the mentors.

Starting today, and for the duration of this year’s Summer of Code, we at the Review Board project would like to supply up to 30 projects with free Review Board hosting at gsoc.review-board.org. We’ll handle maintenance of the server, support, and provide assistance to get students and mentors set up.

Doing code reviews with Review Board is simple and saves time over reviewing standard raw diffs, with features such as a powerful diff viewer with syntax highlighting and inline commenting, interdiffs, status reports, and support for a wide variety of revision control systems. We believe it can be as effective a tool for open source development as Bugzilla and Trac have become.

Th Summer of Code server is intended for Summer of Code-related changes only. While we’d love to provide hosting for projects in general, we have limited resources. However, should your project decide to set up its own Review Board server in the future, we’ll be able to assist in migrating your Review Board history to your new server.

If you’re interested in trying out Review Board for your Summer of Code project, you can find out how to apply on our Summer of Code Hosting page.

To learn more about Review Board and how it works, you can look at our project website or watch our presentation from this year’s LugRadio Live USA.

Review Board presentation at LugRadio Live 2008

chipx86@chipx86.com (chipx86) — Tue, 15 Apr 2008 03:19:32 -0700

This year’s LugRadio Live USA conference was a blast. There were some great talks, interesting booths, and fun swag.

The Review Board presentation went pretty well. I think in my next presentation I’ll demo more often instead of just at the end, but all in all people seemed to find it interesting. The presentation focused on what Review Board is, how it works, and how it could benefit open source. With luck, it will encourage projects to begin using a real code review system, making life easier for all involved.

For those that missed it, the slides and the presentation video are now available. Enjoy!

Django Development with Djblets: Unrooting your URLs

chipx86@chipx86.com (chipx86) — Fri, 04 Apr 2008 06:35:11 -0700

Table of contents for Django Development with Djblets

Django Development with Djblets
Django Development with Djblets: Custom Tag Helpers
Django Development with Djblets: Unrooting your URLs
Django Development with Djblets: Data Grids
Django Development with Djblets: Dynamic Site Configuration

Typically, Django sites are designed with the assumption that they’ll have a domain or subdomain to themselves. Often times this is fine, but if you’re developing a web application designed for redistribution, sometimes you can’t make that assumption.

During development of Review Board, many of our users wanted the ability to install Review Board into a subdirectory of one of their domains, rather than a subdomain.

There’s a few rules that are important when making your site relocatable:

Always use MEDIA_URL when referring to your media directory, so that people can customize where they put their media files.
Don’t hard-code URLs in templates. Use the {% url %} tag or get_absolute_url() when possible.

These solve some of the issues, but doesn’t address relocating a Django site to a subdirectory.

Djblets fills in this gap by providing a special URL handler designed to act as a prefix for all your projects’ URLs. To make use of this, you need to modify your settings.py file as follows:

settings.py

TEMPLATE_CONTEXT_PROCESSORS = (
    ...
    'djblets.util.context_processors.siteRoot',
)
 
SITE_ROOT_URLCONF = 'yourproject.urls'
ROOT_URLCONF = 'djblets.util.rooturl'
 
SITE_ROOT = '/'

SITE_ROOT specifies where the site is located. By default, this should be “/”, but this can be changed to point to any path. For example, “/myproject/”. Note that it should always have a leading and a trailing slash.

The custom template context processor (djblets.util.context_processors.siteRoot) will make SITE_ROOT accessible to templates.

SITE_ROOT should be used in templates when you need to refer to URLs that aren’t designed to respect SITE_ROOT (such as User.get_absolute_url). Your own custom applications should always respect SITE_ROOT whenever providing a URL.

ROOT_URLCONF is typically what you would set to point to your project’s URL. However, in this case, you’ll be pointing it to djblets.util.rooturl. This in turn will forward all URLs to your project’s handler, defined in SITE_ROOT_URLCONF.

This is all you need to have a fully relocatable Django site!

To sum up:

Add djblets.util.context_processors.siteRoot to your TEMPLATE_CONTEXT_PROCESSORS.
Set SITE_ROOT_URLCONF to your project’s URL handler.
Set ROOT_URLCONF to ‘djblets.util.rooturl’
Prefix any URLs with SITE_ROOT in your templates, unless the URL would already take SITE_ROOT into account.

This is functionality that will hopefully make its way into Django at some point. For now, you have an easy way of unrooting your Django project.

Previous in series Next in series

Django Development with Djblets: Custom Tag Helpers

chipx86@chipx86.com (chipx86) — Fri, 29 Feb 2008 12:37:46 -0700

Table of contents for Django Development with Djblets

Django Development with Djblets
Django Development with Djblets: Custom Tag Helpers
Django Development with Djblets: Unrooting your URLs
Django Development with Djblets: Data Grids
Django Development with Djblets: Dynamic Site Configuration

I’m planning to cover all of what Django can do, but for now, let’s start simple with something most Django developers spend way too much time creating: Custom tags.

Django’s nice enough to provide a @register.simple_tag decorator for creating very basic tags that don’t take a context but do take parameters. This is great, but what if you want more? Many Django applications use the same boilerplate time and time again to create their tags, but we make it much easier.

Introducing @basictag and @blocktag.

@basictag

Ever wanted to use Django’s @register.simple_tag but needed access to the context? I’ve found far too many cases where this would be useful, but Django doesn’t make this easy. Your tag code would end up looking like this:

class MyTagNode(template.Node):
    def __init__(self, arg1, arg2):
        self.arg1 = arg1
        self.arg2 = arg2
 
    def render(self, context):
        arg1 = Variable(self.arg1).resolve(context)
        arg2 = Variable(self.arg2).resolve(context)
 
        return context['user']
 
@register.tag
def mytag(parser, token):
    bits = token.split_contents()
    return MyTagNode(bits[1], bits[2])

Do this a few times and it’s bound to drive you nuts. How about this instead?

from djblets.util.decorators import basictag
 
@register.tag
@basictag(takes_context=True)
def mytag(context, arg1, arg2):
    return context['user']

Far less code and increased readability. Hooray!

@blocktag

@blocktag aims to do the same thing @basictag does but for block tags. A block tag is a tag that contains nested content, like @spaceless or @for. This usually requires even more boilerplate than the above code fragment, except with the added complexity of having to grab the contents of the block.

We’ve condensed it down to this:

from djblets.util.decorators import blocktag
 
@register.tag
@blocktag
def blinkblock(context, nodelist, arg1, arg2):
    return "&lt;blink&gt;%s&lt;/blink&gt;" % nodelist.render(context)

If you’ve built block tags in the past, you’ll appreciate how simple that was.

Previous in series Next in series