flow|state

UI component whiteboard talk: The Web Still Needs a Vibrant UI Ecosystem

2013-02-11T08:00:00-08:00

I've been considering putting together a video screencast sharing some thoughts on web UI components, covering the case for why we need them, some limited solutions today, the prospects for the Web Components standard, and the principles behind my own work on the QuickUI web component framework. It's been my experiencing that producing even a very short video takes gobs of time, but it occurred to me that I could break up the talk into short segments, perhaps 5 minutes or less in length. As an experiment, I created this short (4 minute) intro video:

WheneverI speak in group settings, I always find myself sketching ideas on the whiteboard, so I thought that a whiteboard talk might be a reasonable format for a web video – perhaps that can strike a good balance between something that's engaging while also not incredibly time-consuming to produce. In the course of considering how to make such a whiteboard talk, I came across an interesting iPad app called Doceri that lets you project your iPad sketches onto a desktop computer. Doceri includes its own complete screencast production tool, but unfortunately the audio quality of an iPad's built-in mic isn't great, and editing on an iPad is currently cumbersome. I ended up projecting the Doceri project onto a desktop window, capturing the window contents with Camtasia, then laying down the audio narration in Camtasia and doing final editing there.

I hope you find this short video interesting. If this seems worthwhile, I might follow up with additional segments on a semi-regular schedule.

A 2013 wall calendar optimized for project management discussions

2013-01-16T19:26:00-08:00

In many project discussions, it’s been my experience that two calendar-related questions constantly arise:

On what day of the week will date x fall? Example: We’d like to make our next release on or around the 15th of next month. Is that a weekend?
What is the date of a given day of the week n weeks from now? Example: We usually publish releases on Thursdays. If we want to release Thursday of next week, what date is that?

For several years, I searched for a large wall calendar I could hang in meeting rooms used for project discussions, but couldn’t find anything suitable. Calendar companies still living in the dark ages make the calendar dates unnecessarily small to leave extra room for writing information directly on the calendar by hand. That’s useless to me. I don’t want to track vital project information on the physical wall of a meeting room; I want to track it online where everyone can see it. Nor does a meeting room need a calendar decorated with scenic pictures. Finally, a giant wall calendar that only shows a month at a time is too limited in scope. All I want is a wall calendar that answers the two questions above.

If you want to design a calendar with only the above two goals in mind, I think the logical conclusions are:

The dates should be as large as possible so that they can be read from far away. Ideally, it should be possible for someone sitting across the room to answer the questions above.
A corollary of the above point is that the calendar should have a minimum amount of border junk. Borders can be helpful where you’re scrawling information onto a calendar, so you can keep track of which notes go with which dates, but aren’t necessary if the only information on the calendar is the dates themselves.
The calendar should display several months at once. Project management discussions look beyond the current month, often several months into the future.

Since I couldn’t find a wall calendar that incorporated these principles, I put one together myself:

I’ve now used such calendars for a number of years, and they work great. It’s very easy for anyone in a meeting to quickly answer the key date questions above. I also find it handy to keep one at my desk — glancing at the wall is faster than launching a calendar gadget.

Since other people like these, I thought I’d put these into the public domain. If you find them useful as is, great. Otherwise, feel free to remix to suit your own needs.

PDF version (U.S.). For a small room, a standard printer will do fine. If you want to use this in a large room, print the PDF at a copy service such as FedEx Office which offers poster printing. I’ve printed these at poster size and they work great.
Microsoft Word (U.S.). The gray weekend dates show up in Google Docs’ preview, but they’re there if you download the file.

The calendars above are U.S.-centric, with the first day of the week on Sunday, and with U.S. federal holidays highlighted. A friend asked for an international English version, with the first day of the week on Monday, so I’ve also created PDF and Microsoft Word versions of that. These have no holiday highlighting.

Designing by Making: your process for arranging furniture can point toward a good process for UI design

2012-11-13T08:00:00-08:00

The last time you had to arrange the furniture in your home — did you create a design first? No. You had a design idea, and then immediately jumped into implementing your idea by moving the sofa and table around until the result felt good.

Hmm… let’s try putting this over here…

Consider these attributes of the typical process for arranging furniture:

You do it yourself. If you have enough money, you might tell movers where to put the heavy things first, but you’re still directly involved, and you’ll end up pushing things around yourself before it’s all over.
You work directly with the furniture and the space, without recourse to a single design artifact. Think about it: in the time it would take to create a scale model of a room and the furniture to sufficient accuracy that it could actually inform your decisions, you can finish the task of moving the real furniture into place.
You can never predict whether a layout will completely work until you’ve actually gotten things in place. Once the pieces are in place, you always discover something unexpected. You move your desk so it faces the door, then sit in the desk chair and realize you can’t see the view out the window. So you turn the desk around to face the window, then get a creepy feeling that someone might sneak in the door and creep up behind you without your knowledge. Each layout you try teaches you something.
The process is inherently iterative. You start with an idea, and iterate through various layouts until you converge on an acceptable result (or you’re tired of moving stuff around).

You can design software user interfaces this way too.

I had a chance to speak about my own design process at a talk I gave last month at the California College of the Arts in San Francisco, to an engaged audience of interesting students in the school’s MBA in Design Strategy program. There I discussed how my own design process has changed substantially in the last five years to become something I might call designing by making. In this process, the design of a software experience is inseparable from the coding of that experience. In this regard, the process has a lot in common with arranging furniture.

Many contemporary design process artifacts like field interviews, a wall of post-it notes, and paper prototypes reflect an increasingly antiquated premise: that building a real thing is much more expensive than producing a design. Historically, it has been true that designing software with a complex user interface was a minor cost compared to the labor of actually writing the code. In my early days at Microsoft, one might have seen a ratio of one designer to five to eight engineers (developers and testers), because even primitive tasks like obtaining user input or positioning interface controls in a window entailed such extensive, labor-intensive coding. It seemed sensible to invest considerable thought and time in the design phase because it could be many months before the designer would get to experience the actual product for the first time. Unfortunately, that moment of enlightenment often didn’t come until the fully-functional pre-beta builds arrived roughly two-thirds of the way through the product cycle. At that point, when the designer inevitably has new insights into the best design, any big design changes would often needed to be deferred until the next version.

Much software is still designed this way, even though the economics of user interface implementation have changed radically. The effort required to create useful, functional, beautiful, reliable, and performant software application user interfaces has been dropping for years, and this trend will continue for the foreseeable future. About five years ago, the technology reached the point where it became possible for me to create web applications directly. Rather than working in Photoshop, Microsoft Word, or a prototyping tool as before, and handing these designs off to an engineer, I can now directly create the user interface design in code myself.

This is roughly as expensive as the old way of doing things, but with the significant advance that I am now working with a functional artifact — a working user interface — from the very beginning. This turns out to be a transformative difference. Just as you can never predict all the ramifications of a particular furniture layout, you can never fully predict the strengths and weaknesses of a UI design.

Instead, I currently believe it’s best to design something by making it. This means it’s generally not worth a great deal of time to consider the hypothetical implications of a theoretical design. (“Will the user find this clear?”, “Will this meet the user’s needs?”) It’s faster to just build something that actually works, then immediately observe whether it is good or not. Instead of viewing design as a predecessor to making, this is designing by making. The process looks just like the process above:

Do both the design and coding yourself.
Work directly in code, without recourse to other design artifacts. If you’re working with good tools, in the time it would take to create an accurate static image of what you want, with all the specs that would go along with that, you can instead create a functional design that actually does what you want.
Know that you will be unable to predict whether a design will completely work until you actually having a working interface.
Build your schedule around iteration. You start with an idea, and iterate through various approaches until you converge on an acceptable result (or you’re tired of moving stuff around).

This process isn’t for everyone. There are software domains that don’t entail a user interface (Mars landers, say), where a traditional, process-heavy design phase obviously holds true. And not all designers can code, nor can all coders design. But I believe that designing by making does allows someone who can do both well to iterate much faster from an idea to a usable interface than a designer who is forced to rely on someone else to write the code.

I believe that in the near future, most software application design will look like this. The trends simplifying the coding of user interfaces will continue and accelerate, as better design/coding tools permit the construction of better design/coding tools. Component-oriented user interface frameworks will allow people to spend less time designing and coding the details of common patterns.

Furthermore, companies with experience in creating tools like Adobe are now waking up to the realities of a post-Flash world, in which the open web is the real application platform to focus on. (Microsoft is also slowly waking up to the prospect of a post-Windows client world, although that change will take much longer, and I’m not sure they’ll be able to change fast enough to stay relevant.) Generally speaking, I have high hopes for innovation in the realm of tools and frameworks, all of which should make it more and more practical for someone like you to do both the design and coding yourself.

Today, it is already possible to have a design process built around coding that is as efficient — or, often, more efficient — than a traditional, artifact-heavy, pre-coding design process. What’s more, the tool chain will ultimately improve to the point where designing a user interface will be as fast as arranging furniture. In the time it takes you to say, “Let’s try moving the bookcase over there”, and actually move the bookcase, you’ll be able to say, “Let’s try a tabbed navigation approach”, and actually switch a design to using tabbed navigation. Imagine what it will be like to design software like that.

An axiomatic approach to defining user interface elements: building complex elements from simple ones

2012-09-17T09:11:59-07:00

Just as geometry builds up complex results from simple axioms, and programming languages build up complex constructs from simple primitives, it should be possible to create complex user interface elements from simple elements. But the lack of great building blocks for web user interface components causes people to waste a colossal amount of time reproducing common behaviors or, worse, forces them to settle for something quick but suboptimal.

Take something as basic as tabs. Every web UI package includes a widget or component that produces a set of tabs, such as the typical example from jQuery UI:

While a tab set may seem to be an irreducible unit of user interface complexity, we can actually decompose its behavior into smaller, simpler chunks:

Ensuring a single element within a set is “active” at any given time. Here, only one of the tab buttons is in the active state. There are many other manifestations of this behavior. Single-selection list boxes, for example, also have a notion that a single item in the list is active.
Showing a single element a time. The main region of the tab set shows a single page which corresponds to the active tab button. The active page is the only one that’s shown; the non-active pages are hidden. This behavior comes up in situations other than tabs. For example, photo “carousel” controls let a user page through photos one at a time, generally with previous/next buttons instead of a tab strip.
Showing a set of identical elements that correspond to items in a list. The strip of tab buttons across the top has an internal consistency: every tab button is represented with the same type of button.
Positioning one collection of elements directly above another. Here, the strip of tab buttons is stacked on top of the tabbed pages. This kind of layout seems so simple as to not deserve consideration. However, in current web browsers, this can be frustratingly difficult to achieve in the common cases where the size of the tab set is flexible. Suppose you want the tab set to fill the viewport, or a region of the viewport. The tab strip should consume a given height (which for a variety of reasons should not be fixed beforehand in, say, pixels), and the remainder of the space given over to the tabbed pages. This type of layout can be achieved with a CSS flexbox, but at least for a little while longer, many app developers will need to support older browsers (i.e., IE).
Giving UI elements a description which can shown elsewhere. The pages shown within the tab set are rectangular regions, but the name of the tab is shown outside. It’s fairly common to want to give a UI element a user-friendly name like this.
Letting a user tap or click a button to achieve a result. That is, the elements in the tab strip behave like buttons.

It should be possible to create UI classes that implement each of these more fundamental behaviors or aspects. It should then be possible to exploit these behaviors on their own, or recombine them with other behaviors to produce other recognizable user interface controls. In effect, we should be able to arrive at fundamental behaviors that behave like the axioms in a mathematical domain or, alternatively, like atoms in a physical system of elements.

The domain of computer science has much to say on the topic of axiomatic design. Programming languages are often predicated on the notion that you can boil down everything you’d want to do in the language to a tiny set of primitive functions. It’s only this small set of primitives which must be written in a lower-level language (e.g., a machine language). Everything else can be built up in the language itself. This not only keeps things clean, it ensures the language’s popularity and survival by facilitating the porting of the language to new platforms — only the primitives must be rewritten, and all the remaining code built on top of the primitives can be used as is. The original example of this axiomatic principle in language design was Lisp, whose story Paul Graham recounts in his article The Roots of Lisp. (The full article is available on his site in the original Postscript version, or in various converted PDF versions.) From his article:

In 1960, John McCarthy… showed how, given a handful of simple operators and a notation for functions, you can build a whole programming language.

[McCarthy’s] ideas are still the semantic core of Lisp today. It’s not something that McCarthy designed so much as something he discovered. It’s not intrinsically a language for AI [artificial intelligence] or for rapid prototyping, or for any other task at that level. It’s what you get (or one thing you get) when you try to axiomatize computation. … By understanding [Lisp] you’re understand what will probably the main model of computation well into the future.

Can we determine a similar axiomatic deconstruction of user interface elements? That’s a topic I’m acutely interested in, and I believe the answer is yes. Even through graphical user interfaces span a range of devices, platforms, and frameworks, the underlying collection of distinct user interface behaviors is quite consistent: clicking one thing something makes something else appear; items in lists are given consistent representations and behavior; modes (for both better and worse) constrain the user’s attention and powers; and so on. It should be possible to boil those consistent behaviors into reusable code.

The result of this decomposition is a set of UI primitives which is significantly bigger than the canonical tiny set of user interface controls: a push button, a radio button, a check box, a text box. Of all the aspects numbered above, only #6 (push buttons) are available as a native browser control. Web developers are generally forced to recreate all the other aspects through combinations of CSS and JavaScript. That's inefficient and error-prone. As noted above, even something as seemingly straightforward as stacking two regions on top of one another can prove unexpectedly complex.

The actual set of web UI primitives is probably an order of magnitude larger than what browsers expose as interactive UI controls. At the same time, the set of really general purpose contemporary UI (see this article for a breakdown of UI elements by context-specificity) is not so large it can't be enumerated or understood. For today’s typical mobile or web application, I believe a reasonably comprehensive collection of UI primitives would number in the 100 – 200 range.

What would those primitives be? My work on the QuickUI Catalog is an attempt this question. It’s a work in progress, and is by no means complete. It currently includes controls which shouldn’t be there (they’re really just sample uses of an underlying component), and on the other hand doesn’t (yet) include enough controls for common situations like mobile. Nor is the set of controls completely stable yet. I occasionally realize two controls exhibit similar behavior whose implementation should (or shouldn’t) be shared, which results in both minor and major refactorings. Nevertheless, the Catalog already represents a highly useful starting point for creating application UIs.

Let’s return to the tab set example above. The QuickUI Catalog includes a Tabs control for this purpose, which can be used as is. But that Tabs control is simply a combination of lower-level components corresponding to the attributes listed above:

A Sequence base class. A Sequence control keeps track of which one (and only one) of its children is currently active.
A Modes control. Extends the Sequence class to hide everything but the active child.
A List control. Maps an array of internal data items to an array of user-visible controls.
A VerticalPanels control. Stacks things vertically. This inherits from SimpleFlexBox, a user interface polyfill which uses a CSS flexbox for layout on modern browsers, and a manual engine for layout on older browsers.
A Tab control. Associates a description property with an arbitrary block of content. It's this description the Tabs control displays in a List of buttons across the top.
A BasicButton control. This wraps the browser’s native

All these derive from a common Control base class.

We can show the relationships between all these classes in a graph, where a solid line represents an “is a” relationship (one class derives from another) and a dotted line shows a “has a” relationship (one class makes use of instances of another class):

This arrangement entails a lot more pieces than a typical web user interface platform. The browser itself only provides a native button. Most existing web user interface frameworks provide some button class wrapper (such as BasicButton here) and a tab set class (Tabs). They may or may not expose a general purpose UI component base class (here, Control). The tab set class is typically fixed in a monolithic implementation, and can only be modified via parameters the framework designers have anticipated beforehand.

Traditional client-side UI frameworks (e.g., Windows Presentation Foundation) do have rich class hierarchies, although even their UI primitives tend to be too course-grained. And contemporary web UI frameworks rarely have good building blocks. (Some people claim the Sencha framework does, but it's unfortunately encumbered with developer licensing fees, and requires you to build your app on top of a proprietary substrate. To me, that's moving in the exact opposite direction of web development trends.)

The main obstacles to UI like this on the web may have multiple causes, including the fact that the web's primary client-side programming language JavaScript, still has no native support for traditional object-oriented classes. Moreover, the browser doesn't yet expose a model for modular component composition, which creates a lot of work for a UI framework's creators.

In the above implementation of a tab set, all the lower-level pieces are directly available to the user interface designer and developer. These can be used on their own, or combined with other types of elements to create other user interface elements. And, significantly, new elements constructed with this approach are, by default, extensible and recombinable in their own right. In a subsequent post, I plan to show some of the other sorts of UI controls which can be created by combining some of the pieces above in different ways.

As noted above, this Catalog implementation isn’t perfect. Among other things, there are inherent limitations on what you can achieve with a classic single inheritance hierarchy. But, overall, this feels like a promising direction, and in practice is a highly efficient way to create web apps. Above all, this axiomatic approach feels like the right paradigm for building UI.

McCarthy's big advance with Lisp wasn't to create programming language primitives — all programming langauges have primitives. His insight was that the primitives in programming languages of the time weren't primitive enough. Instead, you should break a language into irreducible axioms, and let people combine those axioms to create any new language functions they need. The functions you create with those Lisp primitives are just as powerful as any pre-packaged functions created with those same primitives by the language's designers. That is, there's nothing special the language designer can do you which you cannot also do.

Similarly, a UI platform should give you a powerful set of axiomatic appearances and behaviors and a means to combine them to create new elements which are every bit as powerful as those elements that come bundled with the platform. This is why attempts to build a tiny handful of new controls into web browsers is almost completely uninteresting to me. A new date picker in the browser, to take just one example, is just never going to solve your date picker needs. It's like the FORTRAN committee adding yet another hard-baked statement to the language. What's infinitely more interesting is a UI platform that gives you the building blocks you need to build a date picker of your own that's as powerful as anything in the browser itself.

Evidence suggesting more than half of web app UI code is reinventing results already achieved many times before

2012-06-19T08:00:00-07:00

Web app designers and developers spend a staggering amount of time recreating common effects and behavior that have already been done many times before on other sites, or within their own organization, or in their own code on previous projects, or — worse yet — in their own code on the same project. You may spend days and days carefully reproducing common UI behavior that can readily be found in other apps: menus, dialogs, in-place editing, progress feedback, and on and on. The web wasn’t built to solve those problems, so you have to solve them — over and over again.

This situation is already at least partially avoidable with current web frameworks that permit the creation of reusable UI components. As a case in point, I recently created a sample Contacts application in the QuickUI framework. The app sports a reasonably interesting user interface, but the bulk of its behavior is driven by shared components from the QuickUI Catalog that provide layout, visual effects, editing behavior, list management, and keyboard navigation.

Having built a handful of web apps in QuickUI now, there’s a pretty clear pattern to the balance of UI components used in these apps: about half of the UI code is comprised of components directly from the Catalog or from previous projects. And, in every case, the project itself has generated new, sharable UI components.

Look at your app’s UI elements — at every scale, from page, to region, to widget, to tiny little visual element — and ask yourself: has anyone done this before? Will someone do it again? If this were a component, could I be sharing it with someone down the hall, or at another company? In asking these questions, you’ll generally need to scrape away purely stylistic attributes such as color and typography, and focus more closely on behavior.

As you consider these question of UI reusability, it becomes apparent that the audience for a reusable UI element varies in size, depending on the degree to which the UI is solving a problem that comes up in other contexts. Some UI is completely specific to the context of a single feature, while some UI patterns are extremely general and come up everywhere.

It’s possible to categorize your UI elements according to this aspect of context-specificity. Having created a half dozen or so web apps of reasonable complexity in the component-orient QuickUI framework, the proportional breakdown across these categories has been very consistent. This leads me to hypothesize that the general proportions of these categories are roughly consistent across most web apps.

Categories of reusable user interface components across apps

Such a breakdown might look like this, ordered from most context-specific to most general:

30% Feature-specific UI. These are elements you create to define the UI for a specific feature: an Update Account Settings page in a web app, or a custom popup that applies to just one list. You take more basic controls (usually drawn from the categories below), compose them together in a unique combination, and wire them up with context-specific interactivity to achieve a specific task. By definition, this category of UI code is not reusable. If you find an opportunity for reuse here, you can factor that code out, but then you should group it one of the other categories.
10% App-specific UI. Any app with more than one feature will have UI elements which are consistent across those features, and those consistencies can be implemented as reusable components. UI elements you might use across multiple features within a given app might be: page templates, templates or controls for table or list elements, a custom type of touch menu used in multiple situations, and so on. You can think of this set of UI as your app’s design language: a more focused expression of your organization’s overall design language (below).If you work on a good team, it should be straightforward to find and take advantage of such opportunities.
10% Company-specific UI. Everything your company or organization does has some (maybe not enough?) consistency in its user interfaces. Perhaps you all follow a convention for app home pages, or a standard way to handle user commenting, or maybe your company prefers using multi-step wizards for complex tasks. These are the UI elements that distinguish your company’s output from that of other companies working in your industry. That is, this category defines your company’s design language: the UI solutions that make your apps recognizable to your users. (If your company makes only one app, then you can lump this category together with the App-specific UI category above.) While in company leaders may assume that everything in this category should be freely leveraged across the company as a strategic advantage, in practice this category often presents the most vexing practical challenges to reuse: office politics, conflicting project schedules, and a lack of way to secure or account for funding on shared work.
20% Domain-specific UI. Everyone working in your industry works in the same problem domain. If you’re struggling to figure out the best way to visually represent a complex data set, or to get a credit card number from a customer, then others in your industry are too. You may be lucky enough to work in a cooperative domain, but chances are, those other people will be your competitors, and so for business reasons your company may not be inclined to share implementations, and may in fact fight tooth-and-nail to avoid their replication in competitive products. If you’re in that boat, then this category of UI code can effectively be combined with the Organization-specific UI category above. That is, your company will end up with private implementations of solutions that could be shared in theory, but in practice is company-specific. But occasionally even competitors may recognize the value of sharing work. For example, a shared solution might benefit your industry’s customers, and the result payoff for all your companies may be great enough to overcome corporate resistance to sharing.
30% General purpose UI. These are the common UI patterns that everyone spends time coding up today: context menus, paginated search results, docking toolbars, and so on. Very few companies want to spend time on this stuff, because it’s just too far removed from any company’s core competencies. Everyone wants to focus on the categories above; no company believes they are going to beat their competitors with their excellent implementation of tab buttons. So most companies rush through the creation of these components, getting many of the details wrong. This UI category contains everything that should have been baked into the web, if only the web had been designed for creating real applications instead of sharing scientific research documents. As browsers evolve, the set of shared solutions here is expanding, but only at a glacial pace. In the meantime, we all have this chunk of UI problems to solve, and there is an enormous opportunity to share UI code here. At the same time, the broad set of possible consumers of any given UI component implies a significant challenge in establishing consensus. The UI code in this category should be written once (or maybe, because we could never get everyone to agree on anything, written a tiny handful of times) and never written from scratch again.

The percentages I’ve given above are rough, but drawn from examining the UI code in apps I’ve written over the last few years. Those apps were already carefully componentized, and focused on code reuse, so I expect a more thorough analysis of more web apps would confirm that the numbers above are conservative. That is, the actual degree of unnecessary reimplementation in a typical web application is probably far higher. Without a component foundation, the most expedient way to replicate a given behavior is often to cut-and-paste it from somewhere else in the app’s source, then hack on the result to fit the new context. The app may not only be reinventing the UI wheel, but doing so multiple times in the same codebase.

If the above breakdown is even roughly correct, then consider a new web company entering an existing market who writes their app UI entirely from scratch. Even if it were extremely well-factored, 50% of all the UI code they write would be reinventing the wheel, solving domain-specific or general purpose UI problems which have already been solved before. While that sounds extreme, it’s probably not that far off the mark for most companies. While most apps consume at least some third-party UI elements (to implement a Facebook “Like” button, say), in many cases the typical company is just nibbling at the edges of the problem. And, if we assume that office politics and other factors prevent them from sharing code internally, the percentage of unnecessary re-invention may be much higher.

No matter how you slice it, chances are that most app teams are writing way too much UI code. Because the web lacks a real component model, most companies write reams and reams of non-modular, non-reusable UI code. If they were to build their apps on a UI framework that let them use and extend components, they could probably avoid writing much of the UI code they write today. To put this in business terms: if they were to componentize their UI effectively, they could get the same amount done in half the time or with half the resources. Obviously adopting a component strategy and reusing components have costs of their own, but I expect those are dwarfed by the mind-numbing scale of solving the same problems again and again.

There already are component frameworks for developing web app user interfaces. I’m obviously heavily invested in QuickUI, but you can find others out there as well. Given the huge savings they can make possible, they’re worth a look.

Update

2012-05-22T08:00:00-07:00

For the past two months or so, I’ve left off from my weekly blogging habit here to focus on some behind-the-scenes aspect of QuickUI. I post about those updates on the separate QuickUI blog. That blog is more technically-oriented, but I though it was worth sharing a roundup of those posts here:

I’ve made a number of improvements to the QuickUI runtime, including a significant version update. One interesting new feature is support for creating UI in CoffeeScript (in addition to plain JavaScript).
A developer asked for sample application code showing how to use QuickUI as the “View” in an application with an MVC (Model-View-Controller) architecture. That’s a great idea, and to date I haven’t had such a sample I could offer. Cozi’s Meal Planner is actually a Model-View-Presenter that uses QuickUI for the View, but the source for that application is proprietary. It’ll be useful to have an interesting MVC/MVP sample application that shows off how to use QuickUI; I’ll post back here when I have something worth looking at. Thanks for the suggestion, Chris!
I continue to be interested in making sure the emerging Web Components spec is well-suited to the scenarios routinely faced by UI designers and developers, and have articulated a vision for how QuickUI and Web Components could co-evolve. This has included some time analyzing the QuickUI Catalog controls in light of the Web Components spec. On that note, I’m looking forward to a meeting with the spec’s author, Dimitri Glazkov, later this week.
A designer friend suggested creating a new QuickUI screencast. The few QuickUI screencasts I’ve done in the past are now out-of-date, and my ideas about how to explain the value of component-based UI development have evolved, so it’s a good time for a new one.
Along those lines, I’ve invested some time improving the framework documentation, including an overview of how QuickUI controls render themselves.
I also continue to improve the QuickUI Catalog controls, although at a slower pace. The above work on the fundamentals and explaining them is taking precedence for the time being.

Thanks to those who have shared suggestions with me — they’re very helpful. If you take a look at any of the above and have feedback, please let me know.

From MacPaint to FiftyThree's Paper: Someday all our apps will be this great

2012-04-05T05:00:00-07:00

Imagine, for a moment, that you’re living way back in the early 1980s, maybe 1984. You have access to a computer, and on that computer, you use a top-end DOS app like Lotus 1-2-3:

Then, one day, you see a marketing campaign for a new computer. Your eye catches on this image:

Your mind is completely blown. The user interface for this application, which you learn is called MacPaint, seems utterly unlike any application you’ve ever seen. In fact, the entire premise of this image speaks to a proposition that’s never before even occurred to you: a computer can be used to create art.

• • •

Looking back, it’s hard to convey now how stunning both this image and that proposition were at the time. When the original Macintosh was released, this was probably the first vaguely art-like computer-rendered illustration most people had ever seen. Before that moment, when (or if) the average person thought about a computer, they considered it a tool for crunching numbers or typing documents.

In retrospect, this Japanese woodcut was probably the most sophisticated illustration most people ever saw on an original Macintosh. As groundbreaking as the application was, it was simply impossible for the average user, even a fairly artistic person, to create something of this quality with a mouse. Drawing with a mouse in MacPaint was said to be, “like drawing with a bar of soap”. If you tried to create something like the above yourself, the results were laughable. You could indeed create interesting works in MacPaint, but only by relying on text, lines, polygons, and those paint bucket textures along the bottom. That is, you got the most interesting results with tools that were well-suited to software implementation and which produced effects you couldn’t easily achieve on paper.

The designer behind this image, Susan Kare, discussed it in an interview:

With the Japanese lady, [Apple developer] Bill Atkinson was experimenting with scanning, and Steve [Jobs] brought in an actual woodcut that he had bought: it was big and colorful, and that was one of the first things that we scanned. And I took the scan, which was kind of rough, and refined it to make the final illustration. It looks so crude now — in terms of scanning technology — but it seemed amazing at the time that you could get a “real” image into your computer.

The fact that this image started from a scan was both a surprise and something of a disappointment. Ah, no wonder we never saw illustrations like this — fundamentally, this was marketing! Not to detract from the groundbreaking impact of this work, but this image was clearly meant to suggest to users that they could create art freehand, using only the tools in MacPaint.

Nevertheless, MacPaint represented a watershed in application user interfaces that had broad impact far beyond its users or market. When such an event occurs, it’s possible to look at the app and say something remarkable: Someday all our apps will be this great.

The only reason the MacPaint woodblock image is no longer jaw-dropping to us today is because, within a relatively short time, nearly every application acquired a user interface that in many ways looked and worked as as well as (or, eventually, better than) the interface in MacPaint. Apps simply had to improve to stay competitive, and users everywhere reaped the benefits.

• • •

Such a moment has now happened again — or at any rate, it has now happened again to me. The moment came when I saw a post on Beautiful Pixels about Paper, an app by a company called FiftyThree:

Paper is beautiful, and I find it a joy to use. Like MacPaint before it, I think Paper represents a new watershed in user interfaces.

Earlier I’d tried Adobe Ideas, a vaguely similar sketch pad app. It’s a fairly typical touch-enabled iPad application, and follows many (but not all) iPad conventions. Judging by app store reviews of Adobe Ideas, some of its users love it, and find it very useful. I myself was underwhelmed. Adobe Ideas feels utilitarian, like a dead thing. Using it to create a sketch feels like work. After a few attempts, I stopped using Adobe Ideas.

Paper, in contrast, feels like something tangible and alive. It’s delightfully fun. Since I installed Paper, I look forward to using it every day. Paper’s interface is beautiful at every level. Zooming out from a drawing (above) shows a sketchbook (below left) containing multiple drawings, and zooming out further shows your collection of sketchbooks (below right):

A stunning amount of detail has gone into every aspect of Paper’s design. Just a sampling of the tiny details I’ve noticed:

The “paper” background of a sketch isn’t pure white; it’s very slightly off-white. This lets the pure white ink and paint appear extra bright.
If you’re in the middle of drawing and drag your finger (or stylus) over the tool palette, the palette automatically drops out of the way so you can continue your line into the space the palette had just been occupying.
You can paint with white watercolor to lighten things. While Paper carefully models the physics of real inks and paints, in various places it breaks with those limitations to let you do things which useful but not possible in the real world. This seems to be done judiciously; the drawing tools still feel very much like their physical analogues.
The tools respond to the speed of your movement — but not always the same way. The pen gets thinner the faster you go, which makes physical sense, but the calligraphy pen gets thicker when you go fast. This is another case in which the physical metaphor has been judiciously broken. I’m not sure of the precise rationale behind these differences, but the result feels right.
As you flip through a sketchbook, not only do the pages animate in 3D, their shadows do as well. Paper is built on the OpenGL 3D library, but it probably still was a lot of work to get these effects to look this good and this smooth.

Surprisingly, Paper actually delivers on the original MacPaint premise: you can create beautiful art. I’m no artist, but I was able to quickly sketch the still life with fruit shown above, and the cat in the smaller image. It turns out you can add watercolors to pretty much any pen or pencil drawing in Paper and get something that looks pretty good. My children think so too — yesterday evening I had to read the Kindle edition of Angelmaker on my phone because I couldn’t pry the iPad away from my four year-old.

(Aside: Paper is free, but you’ll have to pay to get the watercolors. You should just bite the bullet and buy all the tools — you will in the end, anyway. I think Paper’s pricing model is as clever as their interface design.)

As amazing as the artistic results are, I don’t think they represent Paper’s greatest accomplishment. At the highest level, I think the best thing Paper has really done is let you feel like an artist. I haven’t regularly sketched anything since Drawing 101 in college, and now I find I’ve bought an iPad stylus so I can do more with Paper.

FiftyThree carries this message throughout out the Paper app, as well as through their site and brand. Everything about this product is designed to lead you to believe, “I am the kind of cool latter-day renaissance person who carries around a Moleskine notebook because my free aesthetic soul may encounter a beautiful scene I want to render as art. I am that awesome.” This is, in fact, the very image in the Paper promotional video: a guy wandering around New York City sketching stuff. The video is shot from first-person perspective. That guy is you.

I think the term “user experience design” is often overblown puffery — when I get to observe someone working through an app design problem, they’re usually focused on the feature set and interface. I rarely witness someone actually thinking directly about the experience their user will have. But with Paper, I think “experience design” is an apt term for what they’ve done. Maybe even that term sells it short. It could be called something like “self-conception design”.

But, wait! Here’s the best part. Someday all our apps will be this great.

Think about that. In the not-too-distant future, every bit of software you currently use (and maybe swear at) — an online store, the Settings area for your latest device, a random tool from an IT department, the production app you spend your workday in — all those things will someday be as beautiful to look at and joyful to use as this Paper app.

And those apps will make you feel great. When send a message, you will feel like a great communicator (or socialite). When you follow a treasure map to an out-of-the-way restaurant in a new town, you will feel like a great explorer. When you follow a recipe, you will feel like a great chef. And when you create a bit of software, you will feel like a great designer.

Make web menu bars more usable: open a menu on hover only if another menu is already open

2012-04-03T05:00:00-07:00

The history of user interface design isn’t terribly long, but it’s long enough that designers who ignore it do so at their users’ peril. In the transition from clients apps to the web, a lot of UI history has been forgotten, ignored, or reluctantly set aside because its lessons were too expensive (if not impossible) to preserve in early browsers.

For example, it’s hard to find a web application with a menu bar as usable as the standard system menu bars in OS/X and Windows. Consider the basic tasks of opening and closing a menu in a menu bar. Last week’s post on popups listed a number of ways in which a user can cancel a menu: clicking outside of it (while not accidentally triggering whatever one clicks on), pressing Escape, resizing the window, scrolling, or moving the focus to a different window. Web implementations often overlook these aspects of closing a menu.

If we now turn our attention to the task of opening a menu, we find most web apps give this basic act similarly blunt treatment. The choices you’ll typically see in web menus are one of these:

Menus open when the user clicks on a menu title. This is straightforward for a single menu, but problematic in a menu bar with multiple menus. Users need to scan a set of menus if they’re exploring their options, or if they’re hunting for a particular command. In these situations, having to click on each menu in turn feels clunky. And if the menu developer has done the fundamentally right thing in absorbing mouse clicks outside the menu (so the user doesn't accidentally trigger something when canceling the menu), the user must click twice to open up the next menu.
Menus open as soon as the user hovers over a menu title. This feels responsive, and lets the user quickly scan a set of menus. On the downside, it can be incredibly distracting to have menus pop open when they’re unwanted. Consider a user who clicks in a text field, and then has to move the cursor away from the text field because the cursor doesn’t automatically disappear when they start typing. (Another bit of UI history that’s been forgotten!) Knocking the mouse out of the way, the user happens to end up parking the cursor over the menu bar, and now a completely unwanted, giant mega menu pops up, covering up their work surface. (That menu article suggests using careful timing to avoid irritating the user, but to me that seems like a band-aid on what’s fundamentally the wrong solution.) Open-on-hover does offer the ability to have a click on the menu title perform navigation, but as discussed in Why Hover Menus Do Users More Harm Than Good, users may not discover that they can click on the title like a link — if hovering into the title popped it up, then the user can easily conclude that the menu has already performed the only job it’s there for.

The odd thing is that a completely smooth way to finesse the problems of both these methods is right in front of the designer and developer, in the very same OS/X and Windows client applications they are likely using to design or code one of these problematic approaches.

Key attributes of menu riffing behavior

For ages both OS/X and Windows have used the following menu behavior:

When no menu is open, hovering over a menu title can provide hover feedback on the title (Windows does this), but does not pop up the menu.
Clicking a menu opens it. This required click prevents accidental menu invocation.
Once a menu is open, hovering into the title of another menu closes the previous menu and implicitly opens the new one. This lets the user quickly riff through a menu bar’s menus to familiarize themselves with their contents or to find a specific command.

[UPDATE: A commenter correctly points out that client OSes actually open menus immediately on mouse down, instead of waiting for mouse up. This makes it possible to riff through menus with the mouse down. If I recall, Mac OS menus originally only worked on mouse down; letting go of the mouse while over a menu title closed the menu. Windows, in contrast, would keep the menu open even after the user released the mouse button, which was easier to use. The user didn't have to hold the mouse down throughout the whole menu exploration and command selection operation. This approach was eventually adopted by the Mac OS. But both Windows and OS/X still support mouse down opening and riffing of menus.]

To me, this resolution seems about perfect, and I wish all web app menus worked this way. In contrast, how often have you used one of the clunky always-click-to-open or twitchy open-on-hover web menu implementations and said to yourself, “I wish all my OS/X (or Windows) apps worked this way!”?

To be fair, simply knowing the UI history (or being very observant) isn’t enough — there’s still the question of cost. One could argue that Apple and Microsoft have greater control over the environment than a web site within the constraints of the browser, which is true, but I think that explanation falls short. The fundamental problem seems to be the economics of homegrown UI: for most companies, it’s hard to justify the return on investment to get these details right in order to make a really usable menu bar. (Which, if they get it right, their users won’t even notice.) Apple and Microsoft can each build a perfect menu bar once that many developers can benefit from, so it’s worth their taking the time to get it right.

Google Docs is one web app that has taken the time to sweat the details. Their document editing suite carefully follows the same menu riffing behavior described above: you open the first menu with a click, and subsequent menus with hover:

I’m not sure if Google acquired this finely-tuned menu through Writely or one of the other predecessors to Google Docs, or if they’ve more recently decided that a good way to displace with Microsoft Office is with great usability at a much cheaper price. Either way, it’s details like this that make Google Docs feels like such a reasonable replacement for a desktop application suite. (Thought not perfect yet: Google Docs gets the menu open behavior right, but gets points off for menu closing behavior because they don’t absorb background mouse clicks. And, as referenced above, it doesn’t hide the mouse when you start to type, the way most client text editors or word processors do.)

MenuBar control

I’ve added a MenuBar control to the QuickUI Catalog, along with the usual companions of Menu, MenuItem, and MenuSeparator classes. A Menu can be used on its own, or as part of a MenuBar. When placed inside a MenuBar, the menus will exhibit the riffing behavior described above.

I like the way Google’s visual style puts both the menu title and an open menu on the same seamless surface to visually connect the two regions, so I’ve used that style for a Menu’s generic appearance (the one you get if you don’t want to do any of your own styling).

Although the MenuItem and MenuSeparator classes assume a traditional vertically-oriented list of commands, use of those classes isn’t required, and the Menu class could just as easily be used to present commands in multiple columns or any other arrangement.

Implementation notes

The tricky bit here was making the entire MenuBar and its menus accessible to the mouse, while simultaneously absorbing any background mouse click outside the menu bar or its menus. By default, an individual Menu control supplies its own Overlay so that a Menu can be used on its own or in some other menu bar-like UI construct. The problem is that an Overlay behind a single Menu control will prevent the user from hovering into other menus in the menu bar. So the MenuBar creates its own Overlay control, and turns off the Overlays of the individual Menu controls. The result is the entire menu bar and its menus sit above a shared overlay. The user can hover from one menu to the next, and any clicks on the background overlay are absorbed and cancel the currently-opened menu.

As always, it’s my hope that delivering this behavior in an open, reusable component can eventually change the economics of web usability so that anyone can benefit from the UI design history baked into a component — whether they realize that history is there or not.

There must be 50 ways to close a popup: menus, dropdowns, tooltips, palettes, dialogs, and more

2012-03-26T08:00:00-07:00

Apps often need to pop up something over the main UI; common examples would be menus and dialogs. Unfortunately, while apps need popups, documents don’t, and until recently HTML was relentlessly document-focused. It’s frustratingly difficult to do a popup well in a contemporary web app, and so it’s not surprising to see so many apps do them poorly or inconsistently.

As a case in point, consider the ways a user might want to dismiss a UI element which has popped up over the main UI. Depending on the specific form of popup, there are a surprisingly large number of methods the popup might support for leaving it:

Click outside the popup. This is the most common means to dismiss a lightweight popup like a menu. The user is saying, “I see you, popup, but don’t want to interact with you; let me get back to the main UI.” When the user clicks on the main UI in the background, a key question arises: what happens with that click? This isn’t an easy question to answer; see below.
Click inside it. Perhaps the user has hovered into an element that’s popped up a tooltip, and maybe the tooltip’s under the mouse. If the tooltip is nothing but static content, the user can generally click anywhere within the popup to dismiss it.
Make a selection. This is a special case of the above point. If the user’s dropped down a combo box and has clicked in an item in the resulting list, they’re not only making a selection, they’re also saying they’re done with the dropdown.
Click a button that explicitly indicates completion. Another special case of point #3. A classic example would be an OK button in a modal dialog, which is essentially a heavyweight form of popup.
Click a close box. A modeless dialog or persistent palette window often relies on a small “×” icon in the upper-right corner as the primary means to dismiss it.
Press Esc. Popups of many flavors can be dismissed by pressing the Escape key.
Wait. A tooltip or transient message may go away all on its own after giving the user time to read its contents.
Hover into another control that produces a popup. The classic example here is menu riffing in Windows or OS/X menu bar. The user must click a menu to open it, but once that first menu is opened, the user can open the next menu simply by hovering into it. (This aspect of menus is worth a closer look in a subsequent blog post.)
Move the focus to another window. Most forms of pop ups are temporary enough that the user doesn’t expect them to stick around. If the user opens a right-click context menu in Google Docs, and then switches to work in a different window, they don’t expect to come back to Google Docs later and find the context menu still open.
Press the ALT key. On Windows, the ALT key or (considerably more obscurely) Shift+F10 are used as the keyboard shortcuts to activate the menu bar (or, in some cases, the selection’s context menu). If the user already has a menu or other popup open, this generally dismisses the popup before activating the menu bar.
Scroll the page with the mouse wheel. Some apps handle this, some don’t. But if a tooltip or context menu was invoked from something that’s being scrolled out of view, there’s probably no reason for the popup to remain open.

[… Are there other ways? There are a wide range of other user actions that could dismiss a popup, but the others I can think of close the popup as a side effect of a click outside the popup or a loss of window focus.]

Most web apps that create popups seem to consider only a small fraction of these cases. For example, it’s common to have a web app’s menu stay open even when the Escape key is pressed (point #6 above) or the tab or window has lost focus (#9 above).

Some of the above cases have complexities that get overlooked. Point #1 above — handling a click outside the popup — raises the question of what should happen with that outside click. The choices are: a) absorb the click so that it has no effect other than closing the popup, or b) let the click affect as usual whatever element outside the popup the user clicked on. On the web, the latter choice can be easier to handle, but this raises a significant usability risk: if the user clicks outside a menu, and just so happens to do so by clicking on a link, do they really intend to trigger the link’s normal navigational response?

As an illustration, suppose a Facebook user has dropped down the menu on the right side of their current toolbar, and then they decide to close the menu by clicking outside it:

Careful!

That click outside the menu isn’t just going to dismiss the menu—the click is also going to activate the partially obscured “app request” link. If the mouse were just a few pixels lower, the user would end up launching the process to create an ad.

Most OSes and client apps will absorb a click made outside a popup like a menu so that the user doesn’t accidentally trigger an unintended action. Web apps usually don’t absorb the click. It’s hard to know whether this is intentional or not. I think it’s simply a reflection of the fact that absorbing the outside click in a web app takes more effort. I personally think that effort is worth the trouble to avoid usability issues that can arise if, in the course of dismissing a popup, the user ends up accidentally triggering a background UI element. I think that work is even more worthwhile if it can be folded into a shareable component so that most designers and developers don’t have to ever think about this issue.

Related to the concept of a popup is that of an overlay. To help the user see a heavyweight popup like a modal dialog, many web apps display a “lightbox effect” or other visual treatment. This layer sits visually behind the popup but over the main UI in the background. This overlay is really a distinct UI element, albeit one whose existence is probably seldom noticed. The overlay may not even be visible — it may be entirely transparent! But a transparent overlay is precisely the means one would typically use to absorb clicks outside a popup: a click on the overlay prevents the click from reaching a UI element in the background.

The Popup control and its related classes

Over the past week, I’ve overhauled the Popup base class as part of work towards a new Menu control. One of my goals was to create a base class that handled more of the cases above automatically. For example, I wanted a Popup to absorb outside clicks by default so that most designers won’t have to even think about this, while still leaving the option of letting the outside click go through if the designer really wants that behavior. Similarly, I wanted the various Popup subclasses (like Dialog) and related classes to handle their respective situations better so that anyone using them has an edge in producing UI with good usability.

The base Popup class now gives the designer and developer the ability to smoothly handle many of the dismissal cases above: outside click, inside click, loss of window focus, pressing Esc, etc. Special cases like menu bar hover behavior can be addressed in subclasses (like the forthcoming Menu control).

A Popup control will normally create a companion overlay control to absorb outside clicks. This overlay is generally an instance of the Overlay class. By default, the first click on an overlay dismisses the popup and removes the overlay. A subclass called ModalOverlay can be used for modal dialogs that want to absorb all outside clicks (not just the first), so as to force the user to explicitly dismiss the dialog. The generic appearance of the ModalOverlay class includes a basic lightbox effect. A Popup can also be created with no overlay in situations where it’s important to let outside clicks have their normal effect.

A related class called PopupSource is available for the common case where a persistent UI element (a button, say) invokes an attached popup. PopupSource takes care of positioning the popup in relation to the button which invokes the popup. If space allows, the popup is shown below the button and left-aligned, but if this would cause the popup to extend beyond the viewport, the popup may appear above the button or right-aligned as necessary. PopupSource is used as the base class for ComboBox, so a dropdown produced by a combo box may actually drop up if there’s more room above the combo box and not enough below. This is standard behavior on client OSes, but rare in web sites that have created their own combo box-like elements.

Implementation notes

In dealing with popups, one naturally has to dive into the details of how browsers render one element on top of the other. In this study I was aided by an excellent summary of how DOM elements stack. Having read that, it now seems likely to me that any occurrence of the CSS rule, “z-index: 1000”, can also be read as, “z-index: I don’t really know how this works”.

Predictably, creating a general-purpose Popup class that works reasonably well in a wide variety of configurations on all the mainstream browsers entailed a substantial amount of testing and debugging. IE8 was particularly problematic in this regard.

Controls of the Week: HorizontalPanels and VerticalPanels for basic CSS3 flexbox layouts today

2012-03-19T08:00:00-07:00

It’s really, really common in UI to place a panel on one or both sides of a main content area, on the left and right or on the top and bottom:

As ubiquitous as these layouts are, until recently it wasn’t easy to create them in HTML and CSS alone. You were either forced to hard-code the heights or widths of the panels, which is gross and hard to maintain — measuring the rendered dimension of a UI element is a task best left to the browser. You could write JavaScript to calculate the dimensions at runtime, but that’s a bunch of work many have avoided.

The CSS Flexible Box Layout Module, a.k.a. “flexbox”, is intended to address layouts like the ones above. For a general introduction to flexbox layout, see CSS3 Flexible Box Layout Explained. This feature hasn’t gotten as much use as it could; as shown on When can I use, it’s not supported on the current (as of this writing) versions of Internet Explorer. Moreover, the flexbox spec changed a while back; only Chrome supports the final spec.

To address older browsers, it’s possible to use a polyfill to support new CSS features. In this case, I wanted to create QuickUI controls to serve as a polyfill for flexbox layout. That is, these should take advantage of flexbox on browsers that support it. On older browsers, they should fall back to simpler flexbox-less CSS in cases where that is sufficient, and otherwise fall back to JavaScript-based layout.

Key attributes

The flexbox layout module can handle many layouts and needs beyond the two shown above, but the two above are common enough that they represent a good starting point.

Each layout has a stretchable main content panel.
A horizontal layout can have a panel on the left, right, or both. Similarly, a vertical layout can have a panel on the top, bottom, or both.
The control needs to be able to handle arbitrary content in the panels. If the content changes, the layout should adjust in response.
Each layout comes in two forms: one with a constrained height (in which the content is generally scrollable) and one with no height constraint (i.e., grows as tall as necessary). In practice, the unconstrained form comes up much more often in the horizontal layout. (In the vertical case, the unconstrained form is really just a stack of divs, so no special layout is necessary. However, controls such as TabSet come in both height-constrained and unconstrained forms, and it’d be nice to be able to position the tabs using a vertical layout in either case. So even the unconstrained vertical layout comes up in some, albeit rare, situations.)

HorizontalPanels and VerticalPanels controls

I’ve posted HorizontalPanels and VerticalPanels controls that address the layouts described above. They can each handle up to one panel on either side of the content area.

As browser implementations come up to snuff, the components can be updated to take advantage of native CSS flexbox support (including, eventually, the new syntax). You can build a UI using these layout components that will work today (as far back as IE 8), knowing that your UI will capitalize on flexbox support as it become more available.

Implementation notes

The HorizontalPanels and VerticalPanels controls derive from a base class called SimpleFlexBox, which sniffs out support for display: box and its variants. In testing, it seemed only WebKit’s flexbox implementation is worth using today. As of this writing, the Mozilla implementation seems too flaky overall to depend upon. And even on WebKit, I hit what looks like a bug preventing the use of automatic scroll bars in a height-constrained flexbox panel with horizontal orientation, which is a pretty common use case. This means HorizontalPanels can’t always use flexbox layout, even on Chrome. And while I’m interested in testing these controls on IE 10, Microsoft has tied the IE 10 preview to the Windows 8 preview, and I’ve already wasted too much of my life fiddling with Windows betas to care about trying Windows 8 before it’s ready. (Weren’t all the tying-IE-to-Windows shenanigans supposed to end with the DOJ consent decree?)

The height-unconstrained cases can be emulated on older browsers using other CSS (i.e., without doing layout in JavaScript), so again there’s no price to pay unless its necessary. If the only way to perform the layout is JavaScript, the control binds to a newly-overhauled pair of events in the QuickUI framework. There’s now a general-purpose layout event to which any control class can bind if it wants to be notified when the control’s dimensions have changed in response to a window resize. There’s a companion sizeChanged event a control can listen to for changes in the dimensions of its children. This is used by the SimpleFlexBox base class, for example, to listen for any changes in the size of controls in its side panels so it can determine whether it needs to adjust the size of the center content area. SimpleFlexBox only binds to these events in the cases where it needs to manually lay things out, so you’re only paying the price of those events when it’s necessary.

I did hit a weird cross-browser issue in IE9: when I view the VerticalPanels demo in IE9 under Large Fonts, the border for the main content area doesn't quite touch the border for the bottom panel. This can happen in IE9 because elements that size to text content can end up with fractional pixel heights. Since IE9 doesn't support flexbox, in the constrained height scenario SimpleFlexBox needs to examine the height of the top and bottom panels so it can adjust the metrics on the main content area. SimpleFlexBox requires on jQuery's height() function to do this, which turns out to always report integral pixel values. Under certain cases, then, it's possible to end up with a sub-pixel gap between the main content area and the panels — and the gap can become visible if the browser or display is scaling things up (as with Large Fonts). IE9 can report fractional heights via window.getComputedStyle(), but it doesn't seem worth this trouble just to support IE9 under various display edge cases. IE8 reports integral heights, and IE10 should support flexbox, leaving only IE9 with this issue. A simple workaround would be to avoid setting interior borders on the main content area if you're also setting them on the panels.

In any event, it’s nice to be able to wrap up a bunch of browser-dependent styling or code into a reusable component that can handle the details so the component user doesn’t have to. And, IMO, I’m not altogether sure that universal flexbox support will actually eliminate all need for controls like HorizontalPanels or VerticalPanels. Use of those controls in your code can arguably make it easier to clearly state your intent. While the CSS flexbox spec is very, um, flexible, the resulting CSS is not particularly easy to read. I preferred the Dock=“Left” syntax of Microsoft’s DockPanel control to the flexbox syntax, and have tried to mirror the former in designing the API for HorizontalPanels and VerticalPanels. Compare: to set the content of the left panel of HorizontalPanels control, you can stuff that content into a property called “left”. To achieve the same result in CSS3, you omit the “box-flex:” property to ensure the panel won’t stretch. I think the former is easier to read and maintain. Even once everyone has a flexbox-capable browser, these controls might still find use as more legible wrappers around the underlying CSS.