a Chris Smith on Software

On Google's 20% time

2011-10-31T14:58:00.000-07:00

Recently I had a long conversation explaining the ins-and-outs of Google's 20% time, and I figured it would be appropriate to share it publicly.

TL;DR: 20% time is awesome and you should get your company to do it.

Caveat Emptor: The Google employee handbook defines the company's official policies concerning 20% time. This blog post is my own personal interpretation.

Back when I was interviewing, I was very suspicious about Google's 20% time. I've heard rumors about products like GMail coming from 20% time. Marissa Mayer even quipped that to half of Google's launches in 2005 came from 20% time efforts.

Seriously!?! If a company spends one-fifth of its time futzing around how can it get anything done?

The truth is that while every Googler has the opportunity to take 20% time to work on whatever they wish, it isn't carte blanche to take Fridays off and play X-Box. Having a 20% project doesn't mean you are responsible for 80% of your job; more like you are volunteering to put in 120% the effort.

At Google performance is measured by impact: benefit to Google and its customers. If an engineer is able to have a big impact, then spending some cycles on a 20% project is no big deal. However, if someone was having difficultly meeting expectations as-is, then contributing to a 20% project would be a poor decision.

In short, the attitude at Google is: "We trust you to do what you think is important with your time."

20% Projects in Practice

Not everybody takes on a 20% project, but those that do usually fall into a few board categories:

"I have something interesting to do"

Some people use 20% projects to pursue their passions. For example, there are Googlers who teach classes at high schools and colleges. Some have desks, go to PTA/faculty meetings, the whole bit.

Other people spend time organizing projects such as GoogleServe or facilitating classes internally. (I, for example, am proctoring a course titled Having Difficult Conversations tomorrow to help Googlers, well, have difficult conversations.)

This has negligible impact as far as engineering efforts are concerned, so it's unlikely that teaching Introduction to Algorithms is going to get someone a promotion. But the people engaged in these types of projects are likely happier and likely more engaged in their work.

"We could use help" or "Team dating"

Other times, 20% projects are used to describe temporary within the company.

Imagine GMail and Google News and a hypothetical feature. The GMail guys have a great idea -- delivering a daily newspaper to your inbox. That way as soon as you log in you could see the highlights of the past 24 hours.

The Google News folks might be excited to add this feature, but are likely already working on other big projects for the quarter.

Rather than GMail simply waiting the weeks or months it takes for Google News to get to that feature, some GMail engineers might start working on Google News in a 20% capacity to add the changes necessary to support the daily digest scenario.

Of course large teams are more likely to work with one another, but the point is that if you have a good feature to contribute then just add it! 20% contributors are welcome to add features and address low-hanging fruit.

Also, volunteering to work for another team in a 20% capacity is a low-cost way of doing transfers. Rather than interviewing to change groups, an engineer can simply try working with them and see if its a good fit first.

"I have an itch to scratch"

Occasionally there are problems, sometimes endemic, that nobody has the time or authority to fix. The classic example is the origin of GMail. A lot of Googlers were unhappy with the email software and figured they could do better. Rather than waiting for a VP to staff a team and issue a mandate, a few engineers just banded together and started writing code.

A few years later GMail was released publicly and used by millions worldwide.

This is the basis for that "Half of Google's launches in 2005" reference. If you see a problem, 20% time gives you the freedom to be proactive and just solve it.

Personal Experiences with 20% Time

As nice as 20% time sounds, does it really work? Since I've been at Google, I've take advantage of 20% time on a few occasions. Here are my experiences; your mileage may vary.

Project A - Dating an engineering team

Shortly after I started I wasn't super-excited about the project I was working on, and was really intrigued by Project A. So I met with their tech lead and signed up to do deliver low-hanging fruit that quarter. (Bug fixes, a few independent features, etc.)

The next quarter, I transferred to the other project as my full time responsibility and stuck around on my old project as a 20% project as a transition / finish up remaining work.

Project B - Gathering hard data

About a year ago, a lot of people were talking about "creating a death star" and how easy it would be able to implement one using "lightsaber crystals". Such a project would be amazingly successful if we added killer features like a "trash compactor inhabited by a Dianoga".

Sick of informal meetings, cloud-talk, and brainstorming I started writing code. About two full weeks working full time on my prototype I had a demo. As it turns out "trash compactor inhabited by a Dianoga" wasn't as exciting as we hoped and this projects "hypermatter reactor core" was kind of a deal-breaker. So alas, Google's plans for "creating a death star" had to be scrapped.

The project didn't have any impact, so it didn't necessarily advance my career any. But the experienced enabled me to learn some new technologies and provide some concrete data points for future experiments.

Project C - Trying to seize an opportunity

When WebGL was checked into Chrome's experimental branch, I knew that a slick, 3D, social game on Google+ would be a hit. So I figured I had a few months to come up with a project and cause a big splash when WebGL was finally generally available.

I met with some friends and we started working on the project. We got pretty far on creating a 3D engine, asset management, and so on. But unfortunately we weren't making progress fast enough. In the few months it took us to get a semi-playable demo there already were a glut of existing games ported to WebGL. (Example 1, example 2, example 3, etc.)

With our small window of opportunity gone -- and the crushing reality of how much of an investment modern games are -- we decided to kill the project.

But again, it was a great learning experience and now there is a game design and prototype there, perhaps only in need of a pivot to be revived! And, had this minor risk had panned out there could have been a huge upside to Google. (Specifically, thousands of man-hours wasted in pursuit of "points".)

So there you have it. In my experiences 20% time hasn't been about zoning out of work, but rather staying engaged and having the freedom to take advantage of opportunities. At Google there is a strong sense of trust between the company and its engineers, and 20% time really exemplifies that phenomenon.

What to do when you aren’t getting promoted

2011-08-22T10:53:00.000-07:00

Edit: A reader on Google+ pointed out a pretty serious omission: this post makes no mention of your manager. AKA the person who is tracking whether or not you will actually get that promotion. I'll save my comments on "managing up" for a later post, but in short they play a pretty important role in this process as well.

For many people, it is nearing annual performance review time. A time when some lucky engineers get promotions -- along with fame and fortune -- while the rest of us are left to languish for another year, slaving away with more responsibilities and fewer perks.

Obviously the tech industry isn’t that cold and gray, but it can seem pretty disappointing when you are feeling ready for a promotion and don’t get it. This post is about the sorts of activities you can do in-between promotion cycles. Rather than, say, sulking.

But first I need to explain what a promotion actually signifies. Promotions are a recognition of the level of work you are already doing. The operative word in that previous sentence is already. With the exception of Texas politics, it makes no sense to promote someone into a role they can’t perform. Instead, people should earn the title of “senior” or “principal” when they have already demonstrated that they can successfully perform in that capacity. (This promotion-after-demonstration concept mitigates the impact of ineptitude due to the Peter Principle.)

You might be wondering then, “How can I perform a job if I don’t have the associated title? People won’t listen to me if I’m not an architect!” Let me spill the beans on another secret regarding promotions: a promotion does not give you any more authority than you already have; it may however add credibility. If nobody likes your ideas now, they still won’t like them if you have “senior”, “principal”, or even “partner” in your job title.

This revelation leads me to the first thing you should do when seeking a promotion: observe.

If you want to join the people higher up the corporate ladder than you, take a step back and observe them. What do they do well? What are their strengths? Why do people listen to them?

Occasionally you will find a bozo with a fancy title, but more often than not those people have skills and experiences you can learn from.

Even better, consider putting yourself in their shoes. What strengths do you have that you could be utilizing more? Are there weaknesses you should address?

If you aren’t sure what the steps are for reaching that next career stage, you should first gather data. Once you have an idea of the work required, go ahead and ask for help. The fancy business-school word for this is networking.

Networking does not mean hanging out in a smoke-filled bar with people you don’t like. Having a good rapport with your coworkers is a form of networking. And taping into that network doesn’t mean asking them for money. It can be something as simple as asking to pair-program a tricky interface, or getting an early code review for something you are working on. Or even something as general as “What should I do when I’m not getting promoted?”

There is an immense value in knowing people outside your team, group, company, or even walk of life. You never know when having an extended network will come in handy. Perhaps your team needs to hire someone with a specific skill. Maybe you need to reach out for advice. Or perhaps you just have a crazy idea in need of a home.

Arguably a person’s personal network shouldn’t be linked to whether or not they get a promotion. But since a person’s effectiveness, influence, and impact is tied to their network the two are at the very least correlated.

But how do you go about building a network? As it turns out, people have already solved this problem for you, inviting others just like you to get together to share ideas and experiences. All you need to do is show up! A few quick Google searches can get you info about conferences, user groups, give camps, and startup weekends. All which serve as fantastic opportunities to network.

And not every conference requires spending several thousand dollars on travel and lodging. There are many local conferences and user groups. These are sometimes harder to find information about -- so it helps to network with developers in your local area.

Even better than going to a conference is presenting at one. Presenting is much easier than you think and is a great way to build expertise and credibility in a particular area.

This brings us to our third thing to do while you aren’t getting promoted: study.

Let me take a moment to give you the best tech career advice you will ever hear: don’t do the same thing for 20 years. No matter how good you are at your given technology and how much your employer loves you to just do the same thing every day: don’t. In other words, actually get 20-years experience; don’t get the first year’s worth of experience twenty times.

Beyond promotions, if you allow your raw skills to atrophy you are crippling your ability to find a new job and limiting your future opportunities. Constantly incorporating new ideas is a must.

So where do you go to learn new skills? First, you can start with the many other tech-related blogs out there. But in case you are looking for something meatier, let me first shamelessly plug my book.

If spending 30-minutes every night reading isn’t for you, consider starting a reading group with some coworkers. Pick a book (or a paper) and meet every week to discuss a chapter or two. It’s a great way to be accountable for the material as well as start the discussion about how to apply it to your office or work environment.

However, IMHO the best way to learn is to simply experiment. I’ve benefited immensely from Google’s 20% time. For example, I wrote a simple voxel editor in WebGL. I never got around to finishing my 3D game, but it was a great opportunity to learn the Google Closure and use JavaScript for more than DOM manipulation.

Remember, experimentation doesn’t have to be about what you are working on. Follow your passions!

Observing, network, and studying may sound a bit overwhelming. Fortunately my last piece of advice for you promotion seekers is to simply relax.

Sometimes the best way to grow is to step away from the keyboard and live your life. Perhaps the thing holding you back isn’t your work but rather just how people perceive you. Would you want to give that guy or girl who is chronically stressed out more responsibilities?

Your life isn’t actually what happens during the hours you are at a keyboard. And a promotion, no matter how big a raise it entails, won’t alter your life.

Anyways, hopefully you have some ideas to keep you busy for the next few months. If you do end up getting promoted next year because of this advice, feel free to donate part of your fame and/or fortune to me.

On the importance of other people’s hard work

2011-07-20T22:27:00.000-07:00

NOTE: As I return from a long hiatus from blogging, please excuse the overly agressive tone of this post. I don't want to disparage any of the teams I worked with at Microsoft, just point out some opportunities we had for getting more done by doing less.

At the start of the Visual Studio 2008 product cycle I was working on the Indigo Tools team, tasked with adding WCF tooling to Visual Studio. On what would later be known as the “Add Service Reference” dialog, or this guy:

The hardest decision we had to make during the product cycle was the very first: how will we implement our feature? We had to choose between two very different approaches.

The first was to build a wrapper on top of the existing WCF tools, specifically svcutil.exe. Svcutil.exe did everything users would want from the Add Service Reference dialog and more. So all we would conceivably need to do is add some hooks to the tool and put a UI on top of it.

The second option was to start from scratch and duplicate the functionality of svcutil.exe. This would mean rewriting the guts of the tool, but make it much easier to integrate into Visual Studio.

After much debate we went with option B: coding our own thing from scratch. Ultimately we erred on the side of making it easier to integrate into Visual Studio.

As you can probably tell from the title of this post, in hindsight that was a mistake. Visual Studio doesn’t support nearly as many WCF service types as we would have liked, it generates poor code, and has a poor user experience to boot. (Click “Show All Files” in Solution Explorer and just see how much crap is added when adding a service reference.)

The problem was that we did exactly what Joel Spolsky claims is “the single worst strategic mistake that any software company can make”, and that is to decide to rewrite code from scratch. Instead of building off of an existing code base and refactoring it to suit our new needs, we instead chose to rewrite the stack from scratch.

Let me expound on the perceived pros and cons of using existing code (EC) or rewriting from scratch and using new code (NC).

The aversion to reusing hypothetical existing code or EC is that it never does quite does exactly what you want it to do. (Because, obviously, if it did you wouldn’t have a problem using it.) EC either doesn’t support the features you need, doesn’t implement them in the way you want, or perhaps is just formatted in a way you don’t like. Also, it would be naive to expect the documentation for EC to be complete, well written, and thorough.

In reality however, EC is actually perfectly fine. (Think about it -- someone used it for something, so it can’t be fundamentally flawed.) The real problem with EC has is that you didn’t write it, and therefore it seems too complicated to work with. But, more on EC later...

Suppose that you bought into the assumption that reusing / refactoring EC was a bad idea. So you instead introduce new code or NC. Hypothetical NC is a well-researched solution, engineered to solve your problem elegantly and efficiently.

Most likely you will start writing NC with lots of unit tests, but even so, eventually bugs will creep in. (Most likely from features NC must support which you didn’t envision.) You will fix these bugs, but leave TODO comments for the larger ones. And of course, since you are busy coding you don’t have any time for real documentation. So you scale back your vision for NC and the second it half-works you forget about NC and move onto the next problem. (Thereby abandoning NC and perpetuating the EC vs. NC struggle.)

NC always seems like a great idea at first, but it runs into the same problems as EC: writing clear code is hard. Documenting code is hard. Finishing code is hard.

My thesis is that reimplementing anything is a waste of time. That’s right, I used the totally bold and flame-bait word "anything".

If you work to improve existing code, then you and the code’s existing developers/users benefit. You fix some bugs, contribute some documentation, and leave the world better than you found it. The word here is synergy -- everybody wins.

But in the case that you try to go about things on your own, nobody wins. Even if NC has more bells and whistles than EC, the world is worse off. Why?

Because you’ve made an even bigger mess for the next guy. Previously the choice was just between reusing EC and writing NC. Now, they have to chose between EC, NC, and newer code (NC++?).

And, even if this EC, NC, and NC++ situation wasn’t bad enough think of the critical fixes that now need to be made to all the duplicate code in the world. Security updates, breaking library changes, compiler updates, and so on. By introducing new projects you increase the overall cost of housekeeping.

Let me break away from the hypothetical and talk about real examples. Look through these lists and think to yourself: does every one of these contribute something novel to the universe? Or would mankind be better off if the effort spent in creating that new thing had been spent improving the old one? http://en.wikipedia.org/wiki/Category:Open_source_content_management_systems

http://en.wikipedia.org/wiki/List_of_build_automation_software
http://en.wikipedia.org/wiki/List_of_text_editors
http://en.wikipedia.org/wiki/List_of_compiler-compilers

Especially for the text editors list. Is there anything you can’t do in Emacs? What else could you possibly want?

Despite what Richard Stallman and the hippies at the FSF tell you, I think the real benefit of FOSS comes form people not having to worry about solved problems.

Certainly licenses, politics, and egos may make it difficult to follow another developer’s lead and work on an existing project. But I urge you to think twice before writing any new code. Ask yourself, has this function already been written in the history of mankind? And if so, how can you take advantage of that and not write that function. Just go search for it!

Let me provide a specific example of the benefit of not rewriting code with another Microsoft war story. Fast forward three years from my WCF experience and I’m working on the F# team trying to ship Visual Studio 2010. We were a small and agile team with an immense amount of work to do. But due to Don Syme’s amazing leadership we were able to produce a great v1.0 product on time and with a few more features than we expected.

That is not a typeo. We actually able to ship more language features than we thought we would.

One such feature came in during a weekend Don Syme was in town. The team decided to spend a Saturday afternoon hacking away on the codebase, but rather than just fixing bugs we wanted to add something bigger and meatier. Of all the options we considered, we settled on a new language feature of dynamic support.

One of the big features the C# and VB.NET teams were adding to their compilers for Visual Studio 2010 was support for dynamic types.

It took a team of people -- dev and test -- working on adding this feature to the C# and VB compilers months, and yet we were able to add it to F# in an afternoon*.

How? By simply doing the easy part and reusing all of their hard work.

Implementing these dynamic types in F# required two pieces. First, syntax support for differentiating dynamic calls from regular ones. We did this in F# by adding the (?) and (?<-) operators. Second, you need a runtime binder to determine how dynamic calls get dispatched. (I.e. how to deal with method overloading, name resolution, and so on.)

Adding syntax support for two operators is quite easy, the bulk of the work for adding dynamic support would have been in implementing the F# runtime binder. However, the key observation is that we didn’t actually need to write a new runtime binder. There already was an excellent dynamic method binder for C# which worked just fine.

All users need to do is copy and paste this code (by Matt Podwysocki) and you are set. While the corner cases for name/method overloading for F# don’t exactly match up with C#, for 99% of cases you won’t notice.

That, in a nutshell is the importance of leveraging other people’s hard work.

It is easy to get distracted with ‘how’ to implement something, and forget it is ‘what’ you want to enable. It is important to leverage any and all pre-existing work. Not only do you have an opportunity to save time, but you also get to build up on the insights, innovations, and efforts of those before you. This sets you up for long term success, both in terms of maintenance.

putCharacter vs. putPixel and my new blog

2010-12-31T13:15:00.000-08:00

A few years ago, and I regrettably don't even remember the context, Miguel de Icaza quipped that there were two types of programmers in the world: those who used putCharacter and those who used putPixel.

For the putCharacter programmers the end goal is productivity and getting the job done. putCharacter programmers don't care about the nuts and bolts of how it works, they just want it to work.

Contrast that with the other type of programmer. putPixel programmers don't care so much about productivity as they do understanding how the system works. If you have a good understanding of how to put and read pixel data to the console, putting characters is just a matter of abstraction. However, that understanding never comes cheap. What would have been one line of code, putCharacter('x'), now becomes a labyrinth of font destriptions, VGA adapter settings, and other low-level nuts and bolts that are hardly documented and seldom intuitive.

Right now you are probably thinking, "There is no way I'm a putCharacter programmer. I'm far too { smart | experienced | hip }." But you are probably wrong. Unless you are writing a compiler, operating system, or database you are probably using putCharacter. I would even go so far to say that if you are using C# then you definitely are a putCharacter programmer.

Think about C#/VB.NET's Edit-and-Continue feature. You can write/edit program code while it is running under a debugger. Features like that don't exist for people who measure twice and cut once.

But this post isn't about ragging on putCharacter programmers. I'll admit that I too was a putCharacter programmer, and loved it. In college while my friends were always working on labs until the wee hours of the morning, while I could get by with devoting only an hour at most. My peers were struggling with learning Linux, Emacs, and command-line debugging with GDB. I was doing everything in Visual Studio. I had a IntelliSense, I had a graphical debugger, I had... no need to learn anything.

The problem with the putCharacter mentality is that it forces you to be dependent on someone else for their tools or libraries. This, in my opinion is the danger of using the Microsoft stack. If you never look beyond putCharacter then you are immediately limited by the functionality it provides. What if you need to print in color? Or a unicode character? If the putCharacter system you are using is missing a feature and you aren't willing to resort to putPixel, then you are probably screwed.

Anyways, the real intent of this post was to announce that my blog has moved (yet again) to a permanent home:

http://blog.aChrisSmith.com/

In my own effort to become less of a putCharacter programmer I've started working on a blogging package of my own. (Called GAEB.) To address your first two concerns:

Yes, it is missing (important) features
Yes, it has (horrible) bugs

But I'm not writing GAEB because I actually care about the blogging software. It's more like learning how to putPixel on my own terms.

30-day challenge for October decided

2010-10-05T22:03:00.000-07:00

Quick update on the 30-day challenge front. The votes are in and "meditate 10-15 minutes a day" won by a strong margin. So far, so good. I'll keep you posted with any insights I can gain from my inner-self later this month.

September recap and 30 day challenge

2010-09-30T14:08:00.000-07:00

September has been a bit busy for me and in order to get through October, I'm going to need a little help from you. But first, let me recap.

The month started off with PAX, the Penny Arcade expo. It's a conference devoted to all things gaming. For me, I was super excited by two things. First, the news that after a decade's wait I might actually play Duke Nukem Forever.

Second, was the chance to see Fallout 3: New Vegas. If you haven't every played the Fallout series I would highly recommend it. In the picture is my girlfriend posing with a giant T-Rex statue at the New Vegas booth.

The weekend after PAX I was at a Magic: The Gathering tournament. Specifically, the Portland Grand Prix. I am only a casual player of the game, so finishing ~860/1400 wasn't to bad. I did however win a few side draft tournaments.

Pictured is my buddy Matt Ellis, whose wife made the hat. In case you don't play Magic, Doom Blade is a good way to crush noobs and clearly Matt has an unfair psychological advantage due to the hat. (In case you were wondering though, his sealed pool didn't contain a single Doom Blade.)

Next stop was a trip to Japan. It was half vacation half international press tour. For the vacation half I visited a friend who teaches English in Fukushima. As a long time student of the Japanese language and culture, I enjoyed a chance to better understand the Japanese way of life. For example, I learned that in Japan everyone does their part. Even dogs are expected to pick up their own poop.

However, the other half was part of an international press tour. Evidently Programming F# has been translated to Japanese! Personally, this is exciting news but it marks an even more important milestone for the language itself.

O'Reilly Japan setup a talk session in Akihabara so I could answer questions and sign copies of the book. The event was a ton of fun, and worth the stress of getting lost in the Tokyo subway system.

Of course, that was last week and September still has plenty of more juice!

Currently, I am writing this post this from San Diego where I am at StarWest, a conference about software quality. The reason I am here is to give a talk about Google Test Analytics, the tool my grand boss Dr. James Whittaker has been hinting at on the Google Testing Blog for months.

I'll save the explanation of what Google Test Analytics and the "ACC methodology" is for a future, technical blog post. For now you should just know that this is exciting news :)

Anyways, this post isn't actually about chronicling my travels. It's about setting the stage for a far more important fact: I don't have any plans for October. (Thank God.) No traveling, no work-related deadlines, nothing.

So the question becomes them, what will I do with my new found free time? Well, I figured I would blatantly steal an idea from Matt Cutts, the unofficial Google spokesman, and try out a 30-day challenge for October.

I'm going to do one thing every day for 30 days in an attempt to somehow better my life. But I need your help to determine what that thing will be. The reason you need to choose it is about motivation. It's super easy to let myself down. For example I've wanted to lose weight my entire life. But if you, my esteemed readers, ask me to do something then I am on the hook to actually do it.

So, without further adieu please fill out my one-question survey and give me some guidance. In October I'll provide an update for what the challenge is and keep you updated on the progress. Thank you.

What should I do in October?

I have been outed! Sorta.

2010-07-22T17:10:00.000-07:00

If you follow the Google Testing Blog you might have read the latest post by Dr. James Whittaker's about 'antenna gate'.

The gist of the post was that Apple missed an important bug -- holding the phone too hard will ruin the antenna signal -- and what the testing community can learn from it. One part of the post of particular interest to me was:

Risk analysis is so important that we've built an internal tool to help guide testers in performing it. Code-named "Testify" this tool streamlines the process of risk analysis, at least the way we do it at Google. We're working on open-sourcing an early prototype in time for GTAC 2010 (I can hear my team cringing now ... "you promised it when?").

The mention of a mysterious risk analysis tool sounds interesting and I like the humorous anecdote about the team cringing about their newfound deadline. Well, I would have found that humorous if I wasn't on that team. (And too busy writing code for that newfound deadline.)

I'm one of the people working on that "internal test tool" for performing Risk Analysis, and am especially glad that James outed the tool so I can talk about it publicly. Risk Analysis is an interesting topic that needs to be mentioned more in software testing circles.

In my experience a classic mistake testers make is being too focused on finding bugs. That's right, you can be too focused on finding bugs. Because what really matters is answering the following questions:

Is your software ready to ship?
If not now, when?
When it does ship, will it solve the problems it set out to solve?

You can find bugs until I'm blue in the face, but systematically finding and removing defects doesn't get you any closer to answering those questions. What matters is understanding where you are testing and what areas need that testing. Polishing the bottom of a banister isn't going to do you any favors.

Risk Analysis is the process of looking at the distribution of possible outcomes and understanding your acceptable risk. If the worst thing that can happen if component X breaks is that the user gets annoyed, then that might be acceptable. Certainly a $500 phone being rendered inoperable is not an acceptable risk.

Risk Analysis wouldn't have said, "Hey, Apple, make sure holding the phone doesn't impact the signal!" But it would have uncovered that there is a piece of the system that has potentially devastating risk, and therefore needs some more investigation or testing.

I don't think Risk Analysis or tool like Testify will revolutionize the way people develop software, but I do think that if used judiciously they can help prevent the next 'gate'.

F# active pattern seven-tag limit

2010-06-03T09:42:00.000-07:00

Edit: Corrected a minor technical detail. Thanks James for pointing it out.

This morning on Twitter there were some questions about the seemingly arbitrary limit of only seven tags per active pattern. (Active patterns being a really awesome F# language feature.) While this limit seems unreasonable and arbitrary, I contend that the F# language is working as intended. Let me explain...

As a tester on the F# team I always took pleasure at finding the capabilities and limits of the compiler. For example, did you know if you are pattern matching against an integer type and have rules for all four billion possibilities the F# compiler stack overflows? Shame, I know.

Anyways, back when I first learned about active patterns I discovered this seven tag limit right away. Seven? There don't seem to be other restrictions in the language like this; for example, you can have tuples of arbitrary length, and discriminated unions with any number of tags. So why not let active patterns run free?

One of the perks of working at Microsoft on the F# team is having access to legendary Cambridge researcher Don Syme. So I simply asked about the design.

To paraphrase his answer to me, "if you need more than seven tags, you are probably doing it wrong." Once he went into the theory behind active patterns I agreed that the limit was reasonable and didn't push for us to change the design. This doesn't mean you can't use active patterns with greater than seven tags, it just means that you need to use a different language construct -- the partial active pattern -- instead.

So what is this 'theory' behind active patterns? Well, to quickly recap there are several types of active patterns -- single, multi-case, and partial -- and each serves a specific role. The only one where this limit applies is the multi-case scenario.

Multi-case active patterns are where the active pattern converts the data into one type of a set of different types. So I give you an X and you convert it into an Alpha, Beta, or Gamma. The right way to think about this is partitioning the input space (all possible values of X) into several categories (if X is an Alpha, if X is a Beta, or X is a Gamma). The word 'type' here is used loosely. Alpha, Beta, and Gamma are more like classifiers than actual .NET types.

For example, I give you an Email Message object and your active pattern returns 'Spam', 'NotImportant', or 'Important'. So then I can pattern match against that email message. For the 'Spam' case I can call 'Delete', for the 'Important' case set the status flag, and so on. You can associate data with each pattern tag too. For example, the 'NotImportant' tag might carry with it a score indicating just how critical it is.

Once you wrap your head around this, active patterns become super useful and are the best way to clean up unruly pattern matches. The rule of thumb I go by is when I see more than one 'when guard' in a pattern match I create an active pattern.

However, multi-case active patterns can quickly can get out of hand. Building off of this Email example, what if the active pattern partitioned the input Email into:

- Spam
- LolCatVideo
- Bill
- FromMom
- FacebookFriendRequest
- TwitterMessage
- FromWork
- SocialCorrespondence
- etc.

If the input space needs to be partitioned into such granular areas, then you should probably consider breaking down the input some more first. For example, being able to pattern match against 'NotImportant' Emails and use a second active pattern to further classify the email as from Facebook, Twitter, LolCat, and so on. This is exactly the sort of thing that partial active patterns were designed for.

So there you have it. A solid reason why there is a seven tag limit to active patterns. If you aren't convinced that this limit is reasonable, here are two more, personal, reasons why I like it..

1. We can always change it later.
It's easy to add features to a programming language and neigh impossible to take them away. So if enough people complain, Microsoft can always increase-or-remove the limit. (In fact, Microsoft loves customer feedback!) The reasons stated above are why the language feature is there in the first place.

2. I love the compiler telling me to do the right thing.
To go on a mini-rant here, I really hate clever code. Consider this C gem (related to 'Duff's Device'):

while (*src++ = *dst++) ;

That one line of C code is particularly awe-inspiring, it is code like that which made Windows 1.0 through XP so insecure. Nobody should ever write code like that, ever. Even if you claim that that is the only way to get performance, run a profiler and be proven wrong.

In my humble opinion, having an active pattern with 8 or 12 tags just seems wrong. Much like having a function taking 8 or 12 parameters. It makes code harder to understand, debug, maintain, and so on. Having the F# compiler slap you on the wrist is just like having that big-ol' long ugly 'mutable' keyword. It's there to help you write better code.

Limiting active patterns to seven tags is F#'s way of saying too much of a good thing isn't good.

One month in

2010-05-20T10:47:00.000-07:00

I’ve been a Noogler for a little over three weeks now and I would certainly characterize the experience as intense. It’s not too common to literally relearn everything you know in a month. I would like to share some of the things I’ve learned so far at Google.

Working at Google
Google’s culture is what makes it so unique, and is absolutely the reason why I left Microsoft. So is the grass as green as I hoped it would be? To be totally honest, yes. It really is that good. I’m sure there will be some bureaucratic bullshit I’ll have to deal with in the future, but I haven’t seen it yet. In speaking to a fellow Noogler, he described Google as a “Geek’s Disney land.” I agree with this statement and would add, “... except you also get paid to ride the rides.”

Google has a huge focus on individuals and their productivity. As I said in my last post I was given a MacBook Pro on day one. As soon as I found myself needing some headphones I just walkted to the Tech Stop and picked one up. At Google you are given everything you need to be successful, and I really appreciate it.

Another thing I find surprising -- not really after you think about it -- is how tied into the Internet Google is. Everything is on a wiki, a public Google doc, or simply open source. One thing I never liked about Microsoft was how hard it was to find information. Even if you worked three offices down I’d have no idea what you worked on, where to get the latest build of your product, etc. However, imagine having Google.com wired to your corporate infrastructure. Within a few keystrokes I can find out what other people are working on or who the local expert is for a technology.

Earlier I said Google wasn’t a transparent company; I was wrong, sort of. For customers, Google is quite opaque until they make a big announcement. However internally Google is crazy-open. Eric Schmit, Google’s CEO, came to the Kirkland office a couple weeks ago and fielded questions about all sorts of sensitive topics: allegations of international espionage and Ninja infiltration by the Chinese government, world-wide privacy ass-plosions due to the Buzz launch, and how Greek financial woes relate to Google. (For the record, I was very impressed with Eric and his answers.)

Life in the Trenches
So in short, live at Google is great. But what is it like actually doing my job? Well, it isn’t all rainbows and ponies, but it’s still fun too. The biggest change for me is the switch to using open source software. Here’s my current setup for the project I work on:

OS: Linux desktop for dev work and Mac OS X for Twitter

Tools: Eclipse IDE for Java, FireBug for all things HTML

Frameworks: Google Web Toolkit and Google App Engine

I’ll be honest with you. I miss Windows 7. I miss being able to press WIN+Left to quickly dock a window to the left side of the screen. I miss being able to use Visual Studio and the F# Interactive window. But, I’m able to get a lot of things done on F/OSS despite it being completely counter-intuitive and difficult to use. C'est la vie.

On the plus side, I really do like the ‘automated refactoring’ tools of Eclipse. While I dislike how verbose idiomatic Java code can be, the fact I can leverage the IDE to do so many otherwise mundane tasks is a plus.

The Future
Unfortunately I can’t publicly talk about my project until we announce the Google Space Deathray(tm) later this fall. However, if you want to speculate what I’m up to I will tell you that my two-boss is (evil genius) Dr. James Whittaker. He’s got a lot of great ideas and hopefully I can put some of them into practice. Stay tuned.

First Day at Google

2010-04-26T20:55:00.000-07:00

So I'm a Noogler! (New-Googler) This means several things:
* I get a sexy new laptop, for work of course
* A very sexy Noogler hat
* The proud right to say I work for Google

It's been quite a busy day with non-stop information overload. But I've finally gotten back to my hotel to reflect upon the day. Here are some initial thoughts.

What I like about Google
Culture. Absolutely and unequivocally Google's culture is their biggest asset and a major reason if why I choose to join the company. I won't say that Microsoft doesn't treat it's employees well, in fact it is rated one of the top places to work at for a reason. However, Google intentionally goes out of its way to make their work environment fun, festive, and most of all, productive.

There is a lot of trust in the Google environment; people are trusted to do their best work and make the right decisions. During orientation one of the speakers quipped (my wording) "if you bitch about a problem 3 times, it's your problem to take ownership and fix." Being empowered to fix problems as you see them - not pass them off to another team, group, or organization - is huge. And to be honest, it has paid dividends to Google as a company.

For example, some Google employees a few back didn't like their current Email system. They found it difficult to search their inboxes. So as a 20% project on the side they wrote GMail. They saw a problem and fixed it. A lot of companies will say they support that level of proactivity, but I've never seen a company actually fund it. (Make allowances for 20% time, actually staff people full time on long-term prototypes like GMail or Wave, etc.)

What I dislike about Google
Coming to Google from Microsoft gives me a unique perspective. Microsoft is absolutely not the opaque and faceless company many people make it out to be. If you want to know what's going on at Microsoft, simply check http://blogs.msdn.com/. Microsoft is an incredibly transparent company and really deserves to get more recognition for that.

Google on the other hand, to me at least, seems much more secretive. Microsoft spends years developing products and so providing a steady stream of information is crucial to stay relevant. Google operates at 'internet speed' and goes dark on products before having a surprise announcement.

Contrast Microsoft's Oslo / Quadrant / M with Google's Wave.

I don't want to accuse Google on whole as being as secretive as, say, Apple. However I am not a huge fan of not being able to state directly what I work on. (However, if you do a search for "Google's secret space death ray" you won't be too far off...)

Also, something I already dislike at Google is writing software for the web. In case you didn't already know, it sucks. Writing desktop applications for Windows was not only easy, but fun. F# is hands down the greatest language on Earth. Moving from C# and Visual Studio to Java and Eclipse is like going from keyboard and Word to pen and paper. ::sigh::

What I look forward to at Google
Challenge. Right now there is plenty to learn, lots to do, and a huge, dark cloud of failure on the horizon. I don't know what the next problem I face will be, but I will rest assured that I am empowered to fix it whatever it may be.

FSLex and FSYacc examples updated

2010-04-20T12:20:00.001-07:00

This morning I gave a talk titled Writing a Java to x86 Compiler in F#, detailing my adventures while in CSE P 501. While I will post the slides and overview here in a few days, I wanted to post an update to my simple FSLex and FSYacc example in the mean time.

For background on fslex and fsyacc, refer to some earlier blog posts on the subject.

While fslex and fsyacc haven’t changed much in the past 18 months, getting things setup has. Here are the current things required to get fslex and fsyacc working as of 4/2010.

F# CTP v2.0.0.0 or Visual Studio 2010
F# PowerPack v1.9.9.9

Once v2.0.0.0 gets released (in a few days) things will still work but you will have to tweak the project references to PowerPack assemblies.

When you add the MSBuild targets for the F# PowerPack note that it is order dependent you must import the F# PowerPack.targets file after Microsoft.FSharp.targets.

e.g:
<Import Project="$(MSBuildExtensionsPath32)\FSharp\1.0\Microsoft.FSharp.Targets" />

<Import Project="$(MSBuildExtensionsPath32)\FSharp\1.0\FSharp.PowerPack.targets" />

And that’s it. I’ve attached the updated MiniCalc source code here.

Evidently I'm an Authority on Tail Recursion

2010-04-18T19:15:00.000-07:00

Evidently my post on Tail Recrusion is the #2 page second only to the entry on Wikipedia. While writing a well reviewed book on the F# programming language gives me some credibility on the subject of functional programming language topics, I find it a little disconcerting that Google's search algorithm has determined that my blog post is one of the most 'authoritative' descriptions of the subject.

I don't want to belittle that blog post however, as it is a pretty great write up on tail recursion. What I find interesting however is that it's my blog post. Three years ago I had absolutely no idea what tail recursion was. And now, according to the world's best search algorithm, I'm consider somewhat of an expert. (To be clear, in the past three years I have become somewhat of an expert in programming languages.)

Anyways, as more internet pages get created and updated I'm sure my 'tail recursion authority' will go away. But it does make you wonder what the state of the search index will be a few years from now. Perhaps this page will be the top result for "Bill Gates." One can only hope.

Running Quake II in a Browser... Seriously?

2010-04-03T11:21:00.000-07:00

Some ~~robot super engineers sent from the future~~ Google employees recently ported Quake II to run in the browser.

While Quake 3 has been 'in the browser' for a few months, this is actually pretty significant. Here is how they did it:

Quake II was written in highly-optimized C
'The Community' ported it to Java
Google Web Toolkit was used to port Java to JavaScript
HTML5 extensions were used to fill in holes.

Realistically all the Googlers did was #4 and #5, put some Java code into a tool to spit out JavaScript. And then fix bugs in the JavaScript to remove any non-existent API calls. While this certainly took some technical know-how, I wouldn't classify this port as rocket science.

What makes this so significance is that in addition to being straight forward port, note that any computer which could run Quake II back in the day could probably run Windows XP as well. And, Windows XP is still the most widely used operating system.

So if you can put enough processing power to put an operating system in a frackin webpage then imagine what's next? We are not talking AJAX to add progress bars here; HTML5 is offering a full frame buffer, audio stack, and embedded video.

If this is a glimpse of things to come, I look forward to the future. One small leap for the Strogg, one giant leap for mankind.

My second meeting with Bill Gates

2010-03-31T22:45:00.000-07:00

One thing Google doesn't have is Bill Gates. During my tenure at Microsoft I've not only had a chance to interact with him, I've had a chance to interact with him twice! That's right. I'm like two steps away from having Billy over to play bridge on the weekends.

My second interaction with Bill Gates was at Tech Fest, an internal conference where Microsoft Research shows off their wares. I was walking around the expo, like you do, when out of the corner of my eye I see Bill-frackin-Gates! He's just walking around with some technical assistant giving him a guided tour of the show.

Most people would freak out and maybe shout OMG SQEEE! I however, decided to play it cool. So I casually turned around - knowing full well that Bill Gates was facing me - and slowly walked forward. There was absolutely no way I was going act different or strange just because Bill Gates was behind me, and coming closer. Of course I was a bit nervous, so I walked a little stiff. To be honest, you wouldn't be incorrect for using the term 'awkward'.

Anyways, I then I felt someone tap me on the shoulder. Initially I was kind of pissed. I'm like, "Do you know that Bill Gates is behind us? Act cool brother!" I turn around and it wasn't some yahoo, but Bill Gates himself, in the flesh, had it was he who touched my shoulder.

Certainly this was so that he could give me an award. Or tell me how much he appreciates my work testing Visual Studio. Bill Gates probably has a track record of greeting random employees and telling them how awesome they are. Probably.

For a moment, which in my mind lasted a few days, we met eye to eye. And that is when Bill Gates spoke to me. Bill Gates literally spoke directly to me. He said, in his angelic, computer-visionary voice, "Excuse me."

He and his technical assistant then proceeded to walk around me and continue on through the crowd. I bet he still remembers me. I mean, I was wearing a name tag. He definitely wrote it down in his mind "Chris Smith, remember to give that guy his award."

Well Bill, I waited a few years, but you'll have to find me at a new address.

It's official, I'm going to become a Noogler

2010-03-29T20:53:00.000-07:00

Today I accepted a job offer from Google. It actually was a very difficult decision. While Google offers quirky perks like free food, an on-site rock wall, and razor scooters in the hall, Microsoft has the distinct advantage of being called 'home'.

My time at Microsoft started as an intern back in 2002, and I've been planting roots in the company ever since. If you hang around anywhere long enough you can see the system for how it really works; and at Microsoft I'd like to say that I am at the top of my game. I know who to ask for help when I needed it. I know what to do when tools or services when down. I know how to get things done quickly, and to get them done right the first time.

So why accept a job with Google? Well, it certainly isn't because of the razor scooters. (I'll be frank and say they are not only impractical they are downright annoying!) What brings me to Google is the opportunity for failure, seriously. At Microsoft my career was growing and I was learning new skills each and every day. However, it was becoming too comfortable. Currently I can tell you the top problems facing my group and exactly how to go about addressing them. But, personally, I don't want my career to be about conquering the known. I want to spend my time exploring the unknown.

What Google offers is a chance to help define what the future of software looks like. How does a ubiquitous internet impact the way we interact? Google's mission of making the world's data searchable does much more than provide a platform for displaying ads. It provides a near infinite amount of data to anyone willing to do something with it. There's a lot of power in that, and I look forward to harnessing it towards changing the world.

Anyways, my first day is April 26th. Until then I get to walk around the halls of Microsoft like a ghost, finishing up loose ends, and pass down my ancient wisdom to the next generation. Hopefully I can make things a little more comfortable for those that remain.

Hello, world!

2010-03-17T17:29:00.000-07:00

What the world needs is another blog, and I am so happy to be bringing it to you. If it weren't for all these people, like me, bitching about their ennui for life you simply wouldn't know just how terrible it is to live these days. Global warming, peak oil, religious persecution - all of these problems pail in comparison to whether or not they had enough mustard for my hotdog today at the cafeteria. (Which, I am pleased to report they did.)

Anyways, I've setup this blog as a placeholder should I choose to move in.