zpao.com | blog

Undo Close Window Has Landed!

paul@zpao.com (Paul O’Shannessy) — Fri, 24 Apr 2009 21:32:21 GMT

I’m happy to announce the landing of Undo Close Window. This adds a menu & keyboard shortcut for reopening your previously closed windows. By default we store 3 windows (though sometimes more due to special cases involving pop-up windows). I’m not going to go into further implementation details (because really, I know you don’t care), but if you’re interested, you can take a look at the patches on the bug.

A Little History

Sometime before Firefox 2, there were APIs in place for handling recently closed windows. The only thing missing at that point was the front-end work to tie it together. But that never happened. Instead, since the code wasn’t even being published for extensions to use, all that was happening is that we were eating memory for no good reason. So the code was taken out late in the 2.0 development cycle.

Bug 360408 was added for Firefox 3, making it possible for extensions to be able to implement their own Undo Close Window if they wanted. The description of that bug was unclear though, since it didn’t actually adding any APIs, it just made sure the data was kept around long enough to be saved by somebody else.

Along comes Bug 394759 – Add undo close window feature, which sat untouched for almost a year. After I finished the primary project I worked on during my internship, I still had a week left, so I looked for bugs that I found interesting and allowed me to explore a new area of the code. I found this bug and wrote a “working” patch over a weekend. At that point I was really surprised that we didn’t have this already. We’ve had Undo Close Tab for a while, and this seemed like the sensible next step. In all honesty, somebody familiar with the code involved here could have written that patch in an hour or two. That’s when I found out that I had essentially reimplemented the code that had been taken out and added the front-end to it.

I worked on it a bit more before I was done interning, then went back to school and time to work on this became increasingly rare. I dropped the ball a bit, and then suddenly we were at string freeze, and this wasn’t done. Fast forward a few months; I graduated & started working for Mozilla full time. I found out we were still going to have another string freeze for beta 4, so I got my patch out of bitrot and continued working on it. It needed more work and testing by then due to Private Browsing and a few other changes that were made. Strings went in right before we froze them & the actual patch went in very last minute, and only because code freeze had slipped for other reasons.

A huge thanks to Simon for all the reviewing he did and help he gave me, Dietrich for the heroic “today is code freeze” review and help, Dão for fixing the problems that popped up as this had a rocky landing (there was no time to bake on trunk), and Beltzner for approving this without the normal bake time.

Changing Tab Load Order

paul@zpao.com (Paul O’Shannessy) — Wed, 01 Apr 2009 17:38:48 GMT

During the Firefox work week last month, somebody had the idea of loading visible tabs first. It’s one of those things that just make sense. Since I had done some work in the SessionStore component before, I volunteered to head up this task.

What does it do exactly?

When you open Firefox using session restore (and through some other paths), you often load several tabs. I know I’ve personally restored windows with 50+ open tabs. It takes a lot of time to load 50 tabs. One optimization that was done previously was to (roughly speaking) prioritize the selected tab, so that it would in theory load first. This change make it so that the tabs visible in your tab strip are also prioritized.

How does it work?

When we were doing this for just the selected tab, it was extremely simple – we just reordered the array of tab data so that the selected one was first. Now we prioritize a set of tabs (keeping the selected tab first). That’s it.

To be clear, this does not actually improve performance. All it does is make the browser feel faster. But hey, isn’t that just as almost as important.

Go ahead and try it out. It’s been in Minefield (mozilla-central) for over a week and landed in Shiretoko (mozilla-1.9.1) yesterday.

Is The Tree Green? For Your Desktop

paul@zpao.com (Paul O’Shannessy) — Tue, 24 Mar 2009 23:08:39 GMT

GeekTool is awesome. Is the Tree Green? is awesome. So together they are SUPER AWESOME!

Python?

Historically, I’m a Ruby guy, but I started learning Python for a Django project I’m (barely) working on. So I picked up a book here at work & decided to flex my pythons a little bit. You can laugh, it’s punny.

The first pass of this took me ~20 minutes, and I just tidied it up this morning so it would take command line options.

How?

If you’re reading this and actually need to know whether or not the tree is green, then you should be able to figure it out. GeekTool is OS X only, but there are similar programs for Windows and Linux.

It’s easy enough to do what I did in the image above. The program can take 2 arguments treename and output. Run it with -h for usage.

Code?

A big thanks to Justin Dolske for Is the Tree Green?. He deserves more credit for this than I.

Your Browser is Special

paul@zpao.com (Paul O’Shannessy) — Mon, 23 Mar 2009 00:03:52 GMT

Roughly 6% of the people visiting this site in the past month were using IE6. That’s mostly insignificant, but I despise IE6 and had to do my part to put an end to it. So, following in the footsteps of The Rissington Podcast (and surely others), I’m serving up an IE6 stylesheet complete with Comic Sans. I also put together a super tiny script that puts a special message up top.

I’d been wanting to do this for a while, but it’s finally done. I’m using conditional comments to differentiate browsers. I used JavaScript to display the message so I wouldn’t be adding clutter to my source, even if nobody else would see it. View the source if you’re interested.

And yes, I’m biased about the browser I’m suggesting, but I’m allowed to be.

Here I Am, Back Again

paul@zpao.com (Paul O’Shannessy) — Tue, 10 Mar 2009 16:52:12 GMT

I’m now into my third week here at Mozilla, working on Firefox. It’s been a tiring 2 weeks so far, but I’ve enjoyed every second.

For those of you who don’t know, I was an intern on the Firefox team last summer. I primarily worked on the password manager and a performance analysis tool. I was offered a full-time position soon after the summer ended, and while I did some looking around, I realized that I wanted to be at Mozilla. I loved the company, the people, and the work I would do—so overall it was right for me.

After an extended winter break, I flew out here with Amanda on February 17^th, a Tuesday. Amanda had been looking at apartment stuff like a fiend, and while I helped, she found a bunch of great places for us to see and did the real work with that. So we spent Wednesday & Thursday looking at ~20 apartments. CitiApartments may (apparently) be the spawn of Satan when it comes to housing in SF, but Hilary was an incredibly helpful agent, if perhaps a little unconventional. It turns out that the apartment we really wanted was the first one we saw on Wednesday morning. Friday was spent making sure our application went through, and jumping through hoops to get my money from my bank in Pennsylvania in order to get certified checks. It all worked out; we signed our lease on Saturday and moved in Monday/Tuesday. We’re working on making the place a bit more homey, but it’s coming along and hopefully we’ll have a little housewarming when there are places for people to sit.

Back to Work

I joined the team as we’re wrapping up Firefox ~~3.1~~ 3.5, and since I’m already relatively familiar with the code base, I got to start working on fixing blockers & participating in sprints. My first week was also a Firefox “work week,” where the Firefox team physically gets together (we’re spread out around the world), and gets stuff done face-to-face. We came up with a bunch of good ideas for what we want to be and how to make people excited about Firefox—that’s where our sprints came from.

I look forward to working with everybody here and meeting those of you who I haven’t met yet. Remember, I’m not an intern this time around :)

YELL AT THE TV!

paul@zpao.com (Paul O’Shannessy) — Thu, 22 Jan 2009 01:20:08 GMT

FRESHLY LAUNCHED: Yell At The TV!

I just launched a new app that lets you yell at your TV via twitter. It’s nothing fancy, but it was a good little bit of fun.

Is LOST on yet?

While this was an idea I had been kicking around for a while, it was the work done by Rick Olsen on Is LOST on yet? that made me decide to do this.

Feature Poor

It’s worse than feature poor right now. There’s only the first page right now.

Hopefully I’ll be adding features over the next week or so. The primary thing I want to add is some support for hashtags, allowing you to view tweets for a show, season, or episode. Something like #lost-s05e01. It shouldn’t be that much work, so hopefully I’ll find some time to do that.

Dreamhost Pains

I originally made this in Sinatra, but that didn’t work out so well on Dreamhost for a few reasons. I fucked around for a bit trying to get it to work, but gave up and instead rewrote it using Ruby on Rails. I really do like Dreamhost, but for projects like this, it’d be nice to host it with ease, using the framework of my choice.

So give it a try: http://yellatthetv.com

Hey Hey-ey, Goodbye

paul@zpao.com (Paul O’Shannessy) — Mon, 15 Dec 2008 23:22:06 GMT

Today I took my last final exam as a Carnegie Mellon student. It’s been four and a half years, but it feels like just yesterday I got here. Pending that one last grade, today I became a Master of Information Systems Management. That degree joins my Bachelor’s degree I finished up in May. I could go on and on about the value of my education or spend time evaluating the programs I was in, but I won’t. I have a lot to say, so I’ll save it for another time. I mean this to be brief.

It’s often said that college changes you. It’s true. I really grew up here; or maybe matured. Either way, Carnegie Mellon has had a huge impact on my life. I figured out what I want to do with my life; I made good friends; I fell in love; I learned.

Now it’s time to move on, into the scary unknown of the real world. Working the ol’ daily grind, paying the bills, and so on. I’ll be far from my family and friends. I look forward to it though—it’s the next big adventure.

(Cue sappy music: “A Whole New World”)

Oh yea, did I mention that I’ll be working on the best damn browser out there?

Yahoo! HackU 2008

paul@zpao.com (Paul O’Shannessy) — Sun, 12 Oct 2008 18:50:09 GMT

This past week Yahoo! was on campus for their annual HackU event. This is the third year they have been doing it and each year it has gotten progressively larger.

The Past

The first year (2006), I think there was a single entry. I made it in time for the presentation and was pretty sad I hadn’t had a chance to make anything.

Last year, I made a determined effort and hacked together my own entry – yWeather – and contributed to flolcatr. These were among the six or so entries from Carnegie Mellon. I didn’t win anything, though I’m still pretty proud of yWeather. There was a 24-hour hack session on the last night, but I’m pretty sure there was nobody there after midnight.

This year

This year there were 28 hacks! I was simply astounded at the number and variation of the entries this year. There were technically impressive hacks, humorous hacks, hacks that could change the web, and even some creepy hacks. We had some big hacker names from Yahoo! show up – Rasmus Lerdof (creator of PHP), Kent Brewster (hacker extraordinaire), and my new pal Paul Tarjan (creator of Search Monkey). I think there were people who actually pulled off the 24-hour hackathon, staying up the whole night. I know Paul was there when I left around 6am.

Last year I took advantage of the time and spent the week working on my hack. This year I didn’t start until after midnight. I contributed a little bit to a hack (“lolmonkey”) with Mattt Thompson which uses Search Monkey and creates an infobar in Yahoo! search results lolifying an image on the linked page. While not the craziest hack, we had fun with it. I think at this point we are both officially done creating tools/hacks that have anything to do with lolcats.

Busses & the Blueprint Framework

The other hack that I worked on uses the Yahoo! Blueprint framework for creating mobile applications. While not the most technically advanced hack, I think it was one with the most immediate value. It was called “CMU Busses”, and simply tells you the next 5 busses leaving Carnegie Mellon in either direction. You can see a screenshot on the project page. This is something I’ve been wanting to make for a long time, and something I would actually use (no offense to the other hacks). This was a true “hack” – I’m scraping the PAT website with Ruby. My Ruby code is outputting PHP code (the departure times are hardcoded instead of doing a DB lookup). Then the rest of the script is using the PHP Blueprint framework that Yahoo! provides (after spending a while working it out).

I had originally planned to make this an iPhone application, but having never done Objective-C, I figured now wasn’t the time to start. I had also wanted this application to be able to detect the user’s GPS coordinates and find the closes 5 bus stops before showing departure times, but that had some barriers as well. In a future iteration hopefully we’ll see that. The Blueprint framework is supposedly also able to convert a Blueprint application to an iPhone application, but that has not been released to the public yet. When it is released, I think I may publish what I have now as a first iteration.

Some Other Highlights

I got to hang out with some great people throughout the week. I also got treated to a dinner for all of the former Yahoo! interns at Fuel & Fuddle. It was a good time, getting to talk some technology and design with smart people.

As a result of working on the lolmonkey hack, Mattt and I were invited to dinner by Alicia at Bug Labs. They have a really cool product called BUG, which as they say on their website, “is a collection of easy-to-use electronic modules that snap together to build any gadget you can imagine”. I got to play with one and it’s really neat. I wish I had some spare cash to get one. They also have some of the coolest package design I’ve seen in a while. So if you’re interested in hacking hardware, take a look at them – they’re cool people with a cool product, hacking on open source hardware and software.

Making the New Password Manager Storage Faster

paul@zpao.com (Paul O’Shannessy) — Fri, 29 Aug 2008 01:01:47 GMT

When I first wrote the new storage module for the Password Manager, I took a few “shortcuts,” trying to keep my code DRY. Partially this was because of the first patch by Mrinal Kant, but mostly it was because I like to reuse code. This bit us just a bit.

I mentioned in my first post about this change that we were initially considerably slower in the critical countLogins method. While it got improved before being checked in, it was still marginally slower (milliseconds on an abnormally large dataset).

As I said before, this was most likely since we were doing a SELECT * on the moz_logins table, and looping over the results and counting. This allowed me to reuse more code. Loops are a kind of slow, and since this was so important, I decided to speed it up.

I filed a bug just over a week ago entitled “storage-mozStorage should use COUNT in countLogins” – which pretty much explains what the solution to the above problem. I created a patch which essentially just switched the mosStorage module to use SELECT COUNT(1). I reran the performance tests I created and we’re doing much better now. There’s still a miniscule loss in speed from the legacy storage module, but at this point, we’ve done all that we can, and where the difference was milliseconds, its closer to millisecond.

This was checked in today (thanks Justin!).

And that’s it. I have another patch in the pipeline and hopefully I’ll have time to get it finished, approved, and reviewed for the freeze (whenever that is now).

Password Manager now uses mozStorage

paul@zpao.com (Paul O’Shannessy) — Tue, 19 Aug 2008 03:55:43 GMT

The other day my patch landed switching the Password Manager to use mozStorage (our wrapper around SQLite). The bug had been up on Bugzilla for a long time, over 3 years, when I came across it at the end of June. I had been doing some Password Manager related work already by that point, so I decided I would do it. This is the story of that bug: the process, the hardships, the code (at least a bit). Keep in mind I was also doing work on my DTrace Treemaps at the time, went to Summit, and encountered more edge cases than I wanted, so this took longer than expected.

Quick Features & Change Summary

Some of this is discussed further down, so bear with me. One of the primary differences in the switch to using mozStorage is that we now store data in a database. Previously we were storing data in a text file, using lines and periods to separate data fields. Open signons3.txt in your profile directory and take a look at it (assuming you’ve saved a password before). All information was kept in memory, and when a new password was saved, the whole file would need to be rewritten. The same thing happened if you ever removed a saved password. Using a database means that we don’t have to keep any (potentially) sensitive information in memory. It also means faster reading and writing since we don’t have to read the whole file every time. These speed boosts are apparent especially in the speed tests, attached to the bug and summarized below.

Since this is really just a drop in change that must implement an API, to the outside world nothing has changed. Although the inner workings are different, it’s the same to anybody who happens to use it (extensions or other parts of Firefox).

v0.1 – The beginning

I began work by really taking a look at the legacy storage module to make sure I knew what was supposed to be happening. Then I looked at the initial attempts by other developers. The first attempts were made before Password Manager got rewritten, so those weren’t relevant. Mrinal Kant came in (over 2 years after the first patches) and wrote what I used as the basis for my code. I don’t think I ran it as it was, but it looked functional, at least at the core.

I actually started by just copying the legacy storage file, removing all of the code from methods that would need to be changed, and started fresh. I copied in some code from Mrinal’s work and used some of the conventions, but the bulk of it was rewritten. I opted to use the wrapper we have for Storage, which makes it easy to do parameter replacement. This also automatically binds the parameters to a type – so when you give it a string, it will ensure it’s treated as a string. It’s very handy.

This first version “worked” (at least as far as I remember), though it definitely had problems. I brought it up at our weekly status meeting and it became one of the “nice to have” features for 3.1. That gave it some attention it needed and I got some quick feedback from Shawn and Justin.

v0.2 – Database details

One of the primary changes here was some of the database stuff. Shawn had pointed out that I needed a way to version the database. All I had was a method to create the tables. In order to future-proof this, I needed to make sure the schema was stored somewhere and there was a procedure for migrating the database.

I took a look at nsContentPrefService (which stores your preferences for specific sites, like remembering zoom settings) since that was another component using storage and written in JavaScript. I “hijacked” the code related to the database stuff, and modified it a bit to fit my needs.

The other major change here was to replace var with let – “let is the new var” as my shirt says. A number of other changes were also made – cleaning up queries, hard-coding the table names, and making sure I was using statements correctly.

v0.3, v0.4 – Cleanup & Optimization

These versions were pretty light – mostly involving cleanup. modifyLogin and the process involved got improved. I also reduced the number of queries we were making by hand and so all SELECTs were done from just 2 places (one for each table).

v0.5 – Importing

v0.5 focused on importing from the legacy module. The basics were in the original code Mrinal wrote, but all of the edge cases were difficult to handle. The one case that caused a lot of problems were the “user has a master password, but presses cancel when we import”. We needed to handle that gracefully, and Justin and I decided the best way was to introduce a _deferredInit method, which did the bulk of the initialization work. At the beginning of each public method, we then check the initialization state and try to import again. It can get annoying, but everything about the master password is annoying.

This also resulted in a couple bugs being spun off to help us with importing: one bug was just a few lines added to the legacy module and the password manager UI to handle an additional error, and the other was just to ensure the legacy module didn’t create an empty file if it was never used.

v0.6 – Tests

I finally got to writing the tests, which involved a lot of copy & paste from the legacy tests, then making small modifications. It’s not the most efficient way, but it works ok. Since the mozStorage module works slightly differently, the code duplication is a necessity for now. In theory they can be cleaned up, but that’ll be a task for the next intern :)

At this point I thought I was pretty much done, and I was. There were still a few problems though, and also not quite enough test coverage.

v0.7, v0.8 – Cleanup, Tests, Corrupt Databases

v0.7 involved a lot of cleanup and adding tests. v0.8 was an important milestone in that I finally added the handling of a corrupt database. Before this point, if we encountered a corrupt database, we would fail and then as with a failed import, just try again and again. This was bad, really bad. So a “thank you” to Shawn for catching that. Now we backup the corrupt database and just create a new one. It should be difficult to get a corrupt database, but just in case (and to cover the case when people think they know what they’re doing, but don’t).

v0.9, v1.0 – Performance

In one of the recent status meetings, Mike Shaver asked me about performance. At that point I hadn’t really done much except throw it up on the try server. The try server gives decent ball park figures, but it’s not perfect. So Justin worked on getting Standalone Talos working while I wrote some XPC shell “tests”. I discovered that while we were generally faster – faster init, faster add, faster remove – we were considerably slower for countLogin, which is a critical path, since it gets called on every page. Over 90% of the that time was actually spent initializing nsLoginInfo objects (since I tried to reuse code). This got improved, though we are still a couple milliseconds slower. This could likely be improved a little bit since we are still doing a SELECT and looping over the results. A little bit more work needs to be done that way, so making the query use COUNT would cut that out. Maybe I’ll write another patch to do that before Beta 1. For now though, we’ll keep doing it how we’re doing it.

And that’s all… almost

After this got checked in, the Windows boxes turned orange on Tinderbox. This was because of the tests (trying to delete files). Justin and I thought we had a quick fix, committed that. As I was packing to come home, I got pinged on IRC since the boxes were still orange and it was my fault. So the tests for my changeset got backed out and we switched back to the legacy module, but the code was still in there.

On Sunday, my module was re-enabled (thanks Justin!). There was another hiccup related to packages-static, but that was fixed as well. If you have any problems please report them on Bugzilla.