The project followed the off-canvas menu metaphor for handling navigation on narrow screens. The baseline, mobile-first layout keeps the navigation out of the way—initially as footer nav with a button-ish jump link, then progressively enhanced to convert the link into a trigger element for the off-canvas menu—while a media query for wider screens formats the navigation on-screen.

A sketch of this layout adaptation:

In the past we’ve approached implementation of this as:

1. Implement a baseline CSS layout that has a menu button.
2. Use a CSS media query to adapt the layout for wider screens, hide the button, reflow the nav.
3. Use JavaScript, often bound on window.resize, to check media query applicability and adapt behavior accordingly (sometimes alternately handled by looking at the width of the viewport and comparing it to media-query-defined widths).

Bolting on Behavior

Often by the time we get to the behavior implementation of our sites and apps, the process involves bolting on JavaScript as a follower of the CSS-based breakpoints that we’ve generated during the design process. We define the visual breakpoints of our stuff and the JavaScript is expected to use those to indicate how it should behave.

So we bump into things. We want the JavaScript to “know” about the breakpoints defined in CSS, and which media queries are presently active. This leads to heartache, and also some pretty clever hacks. We use matchMedia (or a polyfill to support same) to determine whether a particular, specific media query is active.

Thus: duplicating media queries in two places (CSS and JavaScript), a situation that makes a whole lot of us twitchy. I have definitely fantasized about future CSS module spec revisions that allow for naming and scoping of media queries, to make stuff like this better.

But hold on a minute. What are we trying to accomplish here? Does CSS really hold a monopoly on breakpoints? Should our behavioral components be entirely beholden to the specific formulae of our visual layouts?

(Re-)Defining Breakpoints

But why is the breakpoint for menu/navigation behavioral adaptation entirely linked to a CSS concern? Could there be better indications in the browser that the behavior change is appropriate, beyond a CSS media query or screen width? And, in that case, what does define the breakpoint?

This sort of thing was already on my mind when I started reading Stephen Hay’s excellent new book, Responsive Design Workflow. And Stephen is thinking about these things, too—except he’s already a lot further along!

Stephen’s way of defining a breakpoint is: “the points where certain aspects of a website or web application change depending on specified conditions.”

No mention of CSS there—because the picture is bigger than that. As Stephen continues on to say, “[a]nother reason to consider a more full definition of breakpoints is that CSS is not the only method used to implement changes when a breakpoint has been reached.”

Exactly!

Breakpoint Graphs

To express and plan breakpoints, Stephen advocates the use of breakpoint graphs, an adaptation on bullet graphs he’s invented to communicate both visual and behavioral aspects of breakpoints.

Along the “qualitative” axis (horizontal in these examples), one charts a scale. Often, this is a range of screen width resolutions—the way we tend to think about responsive design and breakpoints.

Using the process from above, we might have a breakpoint graph that looks like this:

Thus, we’ve expressed that we have a breakpoint at 40em that alters the page layout.

But Stephen’s graphs go further than this visual design element. Using qualitative “bands,” Stephen’s graphs can communicate behavior or other aspects, like so:

This is starting to move toward thinking about behavioral changes as well as visual ones, expressing explicitly that we want to adapt navigation behavior, but the breakpoint is still owned and defined in terms of the CSS breakpoint: 40em. So it’s natural that we’ve been creating media queries:


@screen only and (min-width: 40em) {}

and the JavaScript corollary

if (window.matchMedia( "screen and (min-width:40em)" )) { }

Behavioral Breakpoints

Looking at breakpoints in such a clear way inspired me. I’d seen the screen-width-resolution-style graphs before, but the qualitative dimension was new and exciting. In fact, it frees us from tying our breakpoints to a visual or CSS source at all.

What is the breakpoint, in the case of the navigation menu example here?

When implementing the navigation behavior, Matt chose to use the state of the triggering button as the indicator for which kind of menu behavior to apply. Button extant and visible? Convert the navigation behavior to the corresponding off-canvas menu. Button gone? Deactivate the off-canvas menu and use on-page navigation. Matt does this by observing the state of that button and reacting accordingly, not by duplicating or checking on the status of the CSS media query that put it there in the first place.

Building Behavior into the Process

A behavior-centric breakpoint graph for this could look something like this:

where the breakpoint is the state of the trigger button. Yes, the state changes at 40em as a result of a CSS media query, but it’s the state we care about, not the media query (or window width) that did it.

Stephen’s book does an excellent job of pushing the notion that behavior needs to be a part of our responsive design processes, integrated and partnered with visual design, not just adjunct to or beholden to it. Breakpoints span various aspects of the overall experience, and I’m glad Stephen helped me really understand this.

Thanks!: my gratitude to Stephen Hay for his personalized help in making breakpoint graphs, Matt Gifford for the off-canvas menu idea that got me thinking and Tyler Sticka for a bit of sketching help and proofreading.

Personally, I dig Dave Rupert’s idea of “Tiny Bootstraps, for Every Client”:

Responsive deliverables should look a lot like fully-functioning Twitter Bootstrap-style systems custom tailored for your clients’ needs. These living code samples are self-documenting style guides that extend to accommodate a client’s needs as well as the needs of the ever-evolving multi-device web.

The whole post is great, and it got me thinking… along with solid content strategy, design and engineering processes, what steps can we take to insure our “tiny bootstraps” are comprehensive enough to remain relevant and useful long after launch?

Cue Jason with a cool idea: We could document patterns in existing frameworks. A list of what’s typically included might serve as a good starting point, something to measure our client deliverables against to make sure we haven’t overlooked anything obvious.

In which I willingly make a spreadsheet

I combed through a (non-exhaustive) collection of suitably broad or noteworthy links from Anna Debenham‘s list of front end styleguides and pattern libraries, recording instances of observed patterns and adding new ones as necessary. I skipped over anything that seemed too idiosyncratic, and grouped elements of similar intent even if their description or implementation differed.

The results are contained in this handy Google Doc.

Lessons learned

I found this to be a worthwhile exercise. It helped me wrap my head around the elastic scope of a “tiny bootstrap.”

I thought there’d be more overlap between frameworks than there is. I recorded over 160 distinct patterns, none of them ubiquitous. Some came pretty close, especially headings 2 through 4, typographic elements and pre-HTML5 form elements. No single framework included even half of the total recorded patterns (Bootstrap had the most).

Sometimes the most infrequent elements surprised me with how obvious they seemed in retrospect. For example, color guides and font stacks only occur in a couple of places.

The thought of maintaining the document indefinitely makes me queasy, but I’ve already started referring to it frequently. I’d love to know if anyone finds it as interesting or useful as I have.

But I have a new idea on how we might be able to define responsive image breakpoints that is based on a performance budget.

Before I begin, I should note that a lot of this is a thought experiment. I don’t yet know how practical this approach would be.

Brief refresher on responsive image breakpoints

Without going into all of the details about responsive image breakpoints, the short version is that most people are picking the breakpoints for responsive images based on one of two criteria:

1. Based on what Scott Jehl referred to as “sensible jumps in file size to match screen dimension and/or density” OR
2. Simply matching the image breakpoints to the major breakpoints being used for the design.

While the first method is more efficient and will probably result in better image sizes, my suspicion is that defining “sensible jumps in file size” is so nebulous that most web developers are going to choose to do the second, easier option.

That is unless we can find a formula to calculate what constitutes a sensible jump in file size and that’s what got me thinking about performance budgets.

What is a performance budget?

I’m not sure how long the idea of a performance budget has been around, but I first became cognizant of the idea when Steve Souders talked about creating a culture of performance on the Breaking Development podcast.

Tim Kadlec expanded on the idea in a recent blog post. He cites the BBC which determined that “each page to be usable within 10 seconds on a GPRS connection and then based their goals for page weight and request count on that.”

So that’s the basic idea. Establish a performance budget and stick to it. If you add a new feature to the page and you go over budget, then you have a three options according to Steve (and transcribed by Tim):

1. Optimize an existing feature or asset on the page.
2. Remove an existing feature or asset from the page.
3. Don’t add the new feature or asset.

What is the performance budget for flexible images?

Let’s apply this idea of a performance budget to responsive design. In particular, let’s treat the idea of flexible images as a feature. Because flexible images are a feature, we need a budget for that feature.

And as long as we’re making up the rules, let’s establish a few more hypotheticals:

• The page we’re working with has 10 images on it of varying formats and visual content.
• We haven’t reached our performance budget yet so we don’t have to remove other features, but we still need to make sure that flexible images do not add too much to the page weight.
• We’ve concluded that flexible images can add up to 200k to the page above what the size of the page would be if we provided fixed width images. We picked 200k because it is ~1 second at HSDPA (recent mobile) speeds. And well, 200k is a nice even number for this thought experiment.
• Because this page has 10 images on it, each image has a 20k budget for flexible images.

One thing to keep in mind, 200k isn’t the cap for the file size of all ten images combined. Instead, it is the price we’re willing to pay for using flexible images instead of images that are perfectly sized for the size they are used in the page.

For example, say you had a responsive web page with the following image on it:

That image is 500×333 pixels and 58K in file size.

Now imagine a visitor views that web page and based on the size of their viewport, the image is displayed at 300×200 pixels, but the source image is still the same. The cost of using flexible images is the difference in file size between what the image would be saved and optimized at 300×200 versus the file size of the image downloaded at 500×333.

In this case, I’ve taken that example image and resized it to 300×200 and saved it with the same compression level as the 500×333 image to see what the file size cost is of using that flexible image.

In this example, the visitor is downloading an extra 34k of image data because they are downloading a flexible image instead of downloading one that had been resized to the exact size being used in the page.

Translating the budget into breakpoints

Let’s go back to the page we want to optimize—the one with ten images on it and a total performance budget of 200k for flexible images. How do we translate that into image breakpoints?

Thinking back to the example above, the price for using flexible images is the difference between the size of the file that is downloaded and the size the file would have been if perfectly sized for its use in the page.

Our budget says that we can only download up to an extra 20K per image. Therefore, we need to make sure that we have a new image breakpoint every time the size of the image increases 20K.

We now have a methodology for picking sensible jumps in image file size that is tied to user experience instead of picking them arbitrarily.

Finding the breakpoints

How would this translate into a heuristic that could be used to find the breakpoints? You would need the following:

• What is the minimum size this image will be used at? (In our example, let’s say 320×213)
• What is the file size between breakpoints? In other words, what is your per image budget? (20k for our example)
• A high quality source file to use for resizing.
• Optionally, the largest size the image will be used at. (990×660 for this example).

Once you have this information, the basic logic looks like this:

1. Take the source image and resize it to the smallest size the image will be used at.
2. Keep the file size of that image handy.
3. Start a series of tests that create new image files from the source that are gradually getting bigger.
4. Check each image created. If the difference between the file size of the new image and the image file size you stored is less than your budget, discard the new image.
5. When you find an image that hits your budget, save that image and replace the previous file size that you stored with the new file size.
6. Repeat steps 2 through 5 until you reach either the maximum resolution of the source image or the largest size the image will be used at.

I’m tickled to say that my co-founder John Keith got excited by this idea and built a rough prototype of how this might work.

Sample Page

Using the script that John built, I created a demo page containing ten images. The source images were 990 by 660 pixels and all but one of them were saved as JPEGs at 50% quality. The one exception is a PNG8 image with an optimized color palette.

I tried to pick a variety of images so we can see how each image might have different breakpoints using our budget. Let’s take a look at three sample images.

Time Square — 8 Image Breakpoints

This image has a lot of visual diversity. The variations in colors and textures means that JPEG’s lossy compression cannot do as much without damaging the image quality.

Because of this, there are eight image breakpoints—set at 20k intervals—between the smallest size of the image (320×213) and the largest size of the image (990×660).

Morning in Kettering — 3 Image Breakpoints

Unlike the Times Square image, this image has a lot of areas with very similar colors and little variation. Because of this, JPEG can compress the image better.

On an image that can be compressed better, our 20K budget goes farther. For this image, we only need three image breakpoints to cover the full range of sizes that the image will be used at.

Microsoft Logo — 1 Image Breakpoint

This is a simple PNG8 file. At its largest size (990×660), it is only 13K. Because of this, it fits into our 20K budget without any modifications.

On a recent consulting project with a company that has over 800,000 images on its site, we identified a class of images—some icons, little badges, etc.—where the size of the image on desktop retina was not much different than the size used on mobile either because the image resolution doesn’t vary much or because the image compresses well. For those images, we decided to deliver the same size image to all screen sizes.

Every image is different in the way it compresses

Take a look at the other images on the sample page we created. See how the number of breakpoints vary even through all the images start with the same resolution end points.

This diversity exists despite the fact that with the exception of the Microsoft logo, all of the images start at the same size with the same compression settings. On a real site, we would see even more diversity with varying levels of JPEG quality, PNG8 with gradients going horizontally instead of vertically, and PNG32 images in the mix.

But what intrigues me about this approach to setting breakpoints is that it we wouldn’t be setting one-size-fits-all image breakpoints. Instead, we would make decisions about where the breakpoints should exist based on the our goals for user experience—the performance budget—and the unique characteristics of the image and how will it can be compressed.

What conclusions can we draw from this thought experiment?

The point of this thought experiment wasn’t to provide a complete methodology to set responsive image breakpoints. I started by simply asking the question about whether we might be able to use performance budgets to come up with a way to calculate what are sensible jumps in image sizes.

But the outcome of this exercise has caused me to draw some interesting conclusions as well as sparking more questions about responsive images:

• Images do contain clues that can tell us where the breakpoints should be.
  Last year I wrote that “the problem is there is nothing intrinsic to the image that would guide you in deciding where you should switch from one size of the image to another.” But this experiment shows that images do have intrinsic information—how well the image compresses, what type of compression is being used, the range in size between the smallest and largest use of an image—that can be used to decide when you should switch from one source file to another.
• We can set a performance budget for flexible images.
  There’s no reason why we can’t treat the use of flexible images like any other feature that we add to a page and define a performance budget for its use. In fact, setting a performance budget for flexible images could be the key to making informed decisions about where image breakpoints should be set.
• Automated image resizing and compression is a must.
  I have already written about how automated image services will likely be a must for sites in the future. If a company wanted to use a technique like this to set their breakpoints, they will need an automated way to do it.
• Can we set a performance budget for flexible images across an entire site?
  For our sample page, setting the performance budget to 200K for the whole page worked well. But in the real world, we often don’t know how many images are going to be on a given page. Similarly, we may not know what pages a given image is going to be added to. It seems like it would be useful to be able to say that for any given flexible image on the site, we’ve established a 20K budget. It would be less precise than a per page limit, but it may be the only practical way to translate this thought experiment into a production environment.
• An image and its breakpoints could be stored as a bundle.
  The outcome of this approach to image breakpoints is that the breakpoints could be specific to the image no matter what context the image is used in. You could store the calculated breakpoints with the image and whenever the image is displayed on a page, no matter what size the image is used at within the page, the same breakpoints could be used.
• An image and breakpoint bundle would be difficult to use with the proposed picture and srcset standards
  Image breakpoints calculated this way depend on knowing the size of the element in the page. Both picture and srcset make the switching of image sources contingent on the size of the viewport instead of the element. This means that you’d have to find a way to translate your image breakpoints to viewport sizes which would undermine a lot of the utility of storing breakpoints with the image.

Crazy? Or crazy like a fox?

Phew, you made it to the end. So what do you think? Is there merit in using performance budgets for flexible images to determine responsive image breakpoints?

Thanks to John for creating the sample script and for being my partner in crime on this crazy idea and to Lyza for being an amazing photographer and publishing her photos under creative commons.

I agree. Back in 2011, I wrote that Google needed to step up. Thankfully, they have. Chrome for Mobile is a great browser.

But Chrome for Mobile is still a small percentage of what people on Android devices are using. A lot of this has to do with the fact that older versions of Android that cannot run Chrome still account for 44.1% of Android’s installed base.

Last month at the Breaking Development conference, many of the speakers talked about large installed base of Android browsers as being an impediment to pushing the web forward on mobile. The discussion reminded me of the efforts that the web community undertook to convince users to move off of IE6.

Web developers started encouraging users to upgrade to a newer version. Microsoft helped out by creating the IE6 Countdown web site which helped web developers figure out when IE6’s market share had gotten low enough that they no longer had to worry about it.

Microsoft and others created banners and warnings that encouraged people to install newer, more standards-compliant browsers.

I’ve thought a few times about how we should encourage Android users to upgrade their browsers, but in this case, there is no way for them to update to a new version of the Android browser and they can’t install Chrome on any device not running Android 4.x.

But while discussing this at Breaking Development, it was pointed out to me that even if people cannot update their browser to Chrome, many do have other options. They can install Opera Mobile or Firefox.

Maybe it is time to change our mindset towards the Android 2.x Browsers and instead of working around its many limitations, perhaps we should actively encourage people to switch to a better browser.

What do you think?

Mobile is fast-paced and constantly changing, and we’re changing constantly along with it. Our team is built of flexible, smart folks who enjoy learning new stuff all the time and see opportunity in the challenges presented by all of those mobile devices out there (and their sometimes-finicky, inconsistent behavior).

We’re a small agency with big aspirations, focused on building mobile and web solutions for our customers. We believe in cross-platform solutions and advocate a mixture of mobile web, native, and hybrid approaches to mobile development depending on the project objectives.

Cloud Four was founded in November, 2007 by four mobile and Web enthusiasts. Our mission is to create usable, inspired mobile and web applications using standards-based technologies. Our clients range from Fortune 500 companies to local businesses, and our projects vary in audience and scope accordingly.

This is a full time position on-site in lovely, lively Portland, Ore. We offer benefits including medical, dental, vision, and IRA.

Job Description

• Research, identify and document client technical requirements.
• Determine and identify appropriate technologies to be used.
• Assist with developing technical project schedules, plans, task assignments and time estimates.
• Assist in strategic planning and requirements gathering.
• Program mobile applications and build mobile web sites.
• Be a positive and enthusiastic contributor to our team.

Our ideal candidate is:

• Able to create a concise, clear plan of action from multiple input sources and stakeholders; flexible and responsive to changes in requirements and scope.
• Self-directed and takes an ownership role of complex projects.
• Strategy-focused and creative; excited to face new challenges and learn new skills.
• Deadline-driven and steadfast about meeting commitments to customers.
• Excellent communicator, ability to comprehend and articulate technical concepts both internally and to customers.
• Able to learn and be productive quickly.
• Passionate about the job and enjoys solving customer needs.
• Straightforward, honest, team player.
• Able to effectively prioritize multiple tasks.
• Comfortable and enthusiastic in a small, start-up environment.

Experience and skills we’re seeking

• 3-5+ years of relevant technical experience or related background.
• Keen focus on JavaScript, with a firm expertise in application design, architecture, performance, testing and security.
• Extensive experience with additional core front-end technologies (HTML5, HTML5 APIs, CSS), tools and components.
• Some programming experience in at least one additional web language: PHP and/or Ruby are ideal.
• A background of mobile web application implementation and/or experience with PhoneGap or other hybrid-native
  technologies strongly preferred.
• Strong problem solving and analysis skills.
• BA/BS or equivalent.
• Not required, but excellent:
  • Native iOS or Android development
  • Node.js experience
  • Back-end web development experience

Drop us a line

Interested? Send us a resume and cover letter at jobs@cloudfour.com.

That was my reaction last week as I listened to the audio from Bruce’s presentation at Responsive Day Out conference.

What had Bruce said that blew my mind? It was the fact that you can embed media queries inside SVG images.

Maybe this is common knowledge for everyone else, but I was stunned by the news. Today I finally got a moment to research this further and found this fantastic video from Andreas Bovens showing off media queries in SVG.

I recommend starting the video at the 3 minute 25 second mark.

The really cool thing about the way media queries work inside SVG is that they react to the viewport of the image itself, not the viewport of the browser. This means you can make decisions about how an image should look at different sizes without needing to know how that image will be used in a given page.

Here is the source from one of the example images that Andreas uses:

 
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="400" height="400">
 <g>
  <title>Simple SVG + mediaqueries</title>
  <defs>
  <style type="text/css">
	#circle {
		fill: #fce57e;
		stroke: #fff;
		stroke-width: 100px;
		stroke-opacity: 0.5;
		fill-opacity: 1;
	}
	@media screen and (max-width: 350px) {
	#circle {
		fill:  #879758;
	}
	}
	@media screen and (max-width: 200px) {
	#circle {
		fill: #3b9557;
	}
	}
	@media screen and (max-width: 100px) {
	#circle {
		fill: #d8f935;
	}
	}
	@media screen and (max-width: 50px) {
	#circle {
		fill: #a8c45f;
	}
	}
	@media screen and (max-width: 25px) {
	#circle {
		fill: #2c3c0c;
	}
	}
  </style>
  </defs>
  <circle cx="200" cy="200" r="150" id="circle" />
 </g>
</svg>

SVG images with media queries embedded in them seem perfect for the responsive images art direction use case.

The examples that Andreas shows in the video can be found at:

• Colored Circle
• One SVG image, with media queries triggering different colors
• Logo, turning into a simplified logo when smaller
• Gold trend chart, turning into a sparkline when smaller

And I would be remiss if I didn’t also share his post from 2009(!) where I found the examples and the video. Andreas was so far ahead of us on this.

Finally, I highly recommend listening to all of the audio from Responsive Day Out. There’s a ton of good stuff in there.

One of the big lessons is that we spend a lot of time discussing what the frontend solutions for responsive images should be—the new picture element or srcset or a magical new image format—and little time discussing how backend systems and human processes might need to change.

The company I consulted has procedures in place for images. These processes include having the images enhanced if necessary, cropped to fit, resized, and then saved in a web optimized format to fit exactly how the image is used on the page. That process makes a lot of sense for the current fixed width site, but will break down when the site becomes responsive.

These conversations led me to document my ideal scenario. If I could wave a magic wand for organizations, what would I put in place for their responsive images processes and tools?

The result was eight guidelines and one rule for responsive images.

8 Guidelines for Responsive Images

1. Use vector-based images or font icons whenever you can

Wherever possible, avoid the problems of pixel-based images by using vector alternatives.

SVG has decent browser support. The Filament Group has a project called Grunticon that will generate SVG and fallback PNGs along with the CSS necessary for you.

Icon fonts are also a big win with fairly wide support. You can create your own fonts with just the icons you need. A bunch of images can be contained in a single HTTP Request which means icon fonts provide similar performance benefits as CSS sprites used to.

Just make sure you use progressive enhancement for your icon fonts and use them in an accessible manner.

Resources

• Can I Use: SVG
• Can I Use: Web Fonts
• Grunticon
• A List Apart: The Era of Symbol Fonts
• IcoMoon
• Pictos
• FontForge

2. Encourage people to upload the highest quality source possible

As prices for high-density screens go down, they are starting to show up in more devices and at larger sizes. It isn’t hard to imagine Apple releasing Retina Cinema Displays in the not so distant future.

Having content authors upload the highest quality image they have into whatever system you use to manage content means that you’ll be ready for whatever resolutions are needed in the future. Managing and keeping track of these assets will make life easier if you redesign the site in the future.

Storing the highest quality source files will likely mean an increase in storage requirements, but storage is cheap these days.

3. Provide an automatic image resizing and compression service

If you have a lot of images on your site, it will no longer make sense to resize them and optimize them for the web by hand.

For example, if the client I was consulting with had decided that they needed three responsive image breakpoints, that would mean three times their current 800,000 images. And that is just standard definition. Multiple the number of images by two if you want to support retina displays. The end result would be 4.8 million images.

Now they wouldn’t have to do all the images at one time, but it still a lot of work. So instead, companies need to build or buy centralized systems that will resize and compress images.

4. Images can be resized to any size with URL parameters

That image resizing service? It should be able to take information that identifies a source image as well as the measurements that the image is needed at and resize the image on the fly.

A good example of a service like this is Sencha SRC. See how you can declare in the URL just the width or the height and the service will resize the image proportionately. That’s what you want.

<img
  src="http://src.sencha.io/320/http://sencha.com/files/u.jpg"
  alt="My constrained image"
/>

If you’re an Akamai customer, the Akamai Edge Image Manipulation service might do the trick for you. It is currently in beta and offers a ton of useful functionality.

Whatever you do, you’ll want to make sure you’re providing smart caching and not breaking external caches.

5. Provide automated output of your image markup

It doesn’t really matter what solution you decide to use in your markup for responsive images. My personal preference is the PictureFill solution, but there are many options to choose from.

But no matter what solution you pick, you should centralize the markup so that it will be easier to change in the future.

For example, in your templates, you might not put any direct HTML, but instead define a series of source images and breakpoints like so:

{
"source":"/source.jpg",
"breakpoints": [
{ "max-width":"30em","pixel-density":1,"width":360px}, 
{ "max-width":"30em","pixel-density":2,"width":720px},
{ "max-width":"30em","pixel-density":1,"width":800px},
{ "max-width":"30em","pixel-density":2,"width":1600px},
{ "pixel-density":1,"width":800px},
{ "pixel-density":2,"width":1600px},
]
}

And then pass that to a central function that would output the correct PictureFill markup or whatever markup you’re going to use.

Don’t get hung up on the syntax or the data contained in the example. The key is that by centralizing image markup in a function, you can change it quickly when the standards evolve.

6. Provide a way to override resized images for art direction needs

For most sites, the most important use of responsive images will be to provide different resolutions of images based on the size that the image is used on the page at a particular viewport size. That is what we call the resolution switching use case.

Occasionally you will find that simply resizing an image isn’t sufficient. A smaller version of the same image becomes unrecognizable.

But at a smaller size, the image can be hard to discern.

If you make changes to the image source at smaller sizes, the image can be easier to see.

This is what we call the art direction use case.

Another place where you may need art direction is for images that contain a combination of text and photography. Ideally, you can separate the photography from the text, but sometimes you can’t get the control you need and when you shrink the image, the text becomes unreadable.

For this recent project, we were able to identify a handful of templates where the need for art direction might be necessary. For those templates, the CMS will be modified to allow authors to upload different sources and define at what breakpoints those sources should be used.

7. Integrate image compression best practices

If you’re going to centralize your image resizing as a service, this is the perfect opportunity to incorporate the best tools in image compression.

Google has some great information on ways to optimize images. It should also go without saying that you need to set far future expires headers for your images.

8. Bonus: Detect support for WebP image format and use it

As long as you’re centralizing image delivery, why not look into supporting WebP?

The average WebP file size is 25% – 34% smaller compared to JPEG file size. WebP compresses 34% better than libpng, and 26% better than pngout for loseless images.

Google wrote recently about how they’re checking for WebP support when an image is requested and automatically serving up the WebP alternative if the browser is capable of displaying that image.

Ideally, browsers would simply support WebP, but in the meantime, there are some performance gains that could be had and it might not add too much to your image resizing service to support it.

The one and only rule for responsive images:

Plan for the fact that whatever you implement will be deprecated

No one knows what the future of responsive images holds. A few years from now, we will probably look back on the hacks we’re using and laugh at our naivety.

The one thing we can be certain about is that we’re going to need to replace what we implement now when standards catch up with responsive images.

So whatever you build, make sure it is flexible and can be changed easily when the ultimate solution for responsive images becomes apparent.

After working on the expense reimbursement form app, I realized that the end result was very similar to what Apple and Microsoft are doing in trying to provide a cohesive experience across devices.

Desktop operating systems are leading the way

One of the big shifts that both Apple and Microsoft have undertaken has been to incorporate the lessons that they’ve learned from mobile and start to apply them to their desktop operating systems.

The features of OS X Lion centered on innovation from iOS being taken “Back to the Mac” and Mountain Lion continued that trend.

While Apple has taken a gradual approach, Microsoft has made a radical shift by taking the design developed for Windows Phone 7, formerly called the Metro UI, and applying it to all devices.

We’re seeing how both Apple and Microsoft are reusing patterns across all their devices and how those patterns have their roots in the mobile design. As I mentioned in Responsive Design for Apps — Part 1, it is much easier to go from something designed for small screens and touch and adapt it to large screens with keyboard and mouse than it is to go in the opposite direction.

Let’s take a look at Apple’s Mail App as an example.

iPhone Mail App: Nested Doll Design Pattern

The iPhone’s Mail App uses what Rachel Hinman described as the nested doll design pattern in her book The Mobile Frontier: A Guide for Designing Mobile Experiences.

Rachel writes:

Mobile experiences that employ the nested doll pattern are all about funneling users to detailed content…It’s a pattern that has a strong sense of forward/back movement.

You can see pattern in action in this screenshot of the iPhone’s Mail App:

There are thousands of apps that use this same design pattern.

iPad Mail App: Bento Box

The Mail App on iPad uses a different design pattern—one that Rachel describes as the Bento Box pattern:

Just like a bento box from a Japanese restaurant, this pattern carves up the surface area of a mobile device’s screen into small compartments, each portion contributing to the overall experience. This pattern is a good way to express sets of information that are strongly related to each other, and it is more commonly used on tablets than smartphone experiences.

You can see how the selection of information on the left side of the screen affects what is visible on the right side in the example below:

One of the things to notice in the screenshot above is that the panel on the left is still acting like the nested doll pattern until an individual email is selected. Once a single email is selected, the details of the email show up on the right.

In fact, the left panel is almost exactly the same both in look and experience as the iPhone version of the app. The panel is even the same width as the iPhone screens. The only difference is that the panel has become taller to fit the size of the screen.

Looking at the Mail App in portrait orientation on the iPad finds a similar pattern:

Again, the nested dolls pattern exists, but is now operating inside a pull down menu instead of in a left panel. And again, the size of the panel is consistent with the width utilized on the iPhone, but the height has been adjusted to take advantage of the larger screen.

Mail App on Mac

I’m not sure what to call the design pattern on the Mac version of the Mail App. We can see the same patterns that we saw on iPhone and iPad have been extended to take advantage of the fact that there is more available screen real estate.

The mail accounts and folder screens have be combined into a single view on this screen. The nested doll pattern is gone. But the email list and view an email screens are almost exactly the same as what we saw on the iOS devices.

The similarities between all three versions of the apps are striking.

How different is the Mail App from a Responsive Design?

When I look at the way Apple’s Mail App uses the same elements across the different screen sizes, I see many similarities to responsive design. While the design is probably more accurately described as an adaptive layout because of the multiple fixed sizes, it seems pretty clear how the Mail App design could easily become responsive if necessary.

Were I to rebuild Apple’s Mail App as a responsive design, I would start with the mobile screens as modules. Then as the screen got bigger, I’d use something akin to the AJAX include pattern to bring more modules into view.

And in fact, this is what we have in mind for the expense reimbursement form that I discussed in Part 2. It starts with a nested doll pattern design pattern that shifts to a bento box pattern as the screen gets bigger.

The future of web design

I believe this is where we will be heading not only for apps, but also for web pages as well. We’re going to evolve beyond designing pages to building discrete building blocks (or modules) that are inherently responsive unto themselves. Then we’ll build systems and rules that define when and where those modules are pulled into the page and displayed.

This is one of the reasons why I think JavaScript UI frameworks will follow the lead of Apple and Microsoft and start to seek consistency between their mobile and desktop frameworks. If the widgets in these frameworks were designed to be responsive and accomodate different inputs, then they would be ideal building blocks for a responsive app.

How we build the systems to pull in the necessary building blocks will shift from project to project. I suspect many will use solutions like the aforementioned AJAX Include pattern to dynamically add modules to a page based on screen size and capabilities. But some may require device experiences or RESS-based solutions.

But regardless of the techniques used, I firmly believe we will be building apps that are responsive in nature. As Stephanie Rieger said, “choosing responsiveness, as a characteristic shouldn’t necessarily define the wider implementation approach”.

Responsiveness for apps is inevitable

When I look at the landscape of devices and the diversity within form factors, I find it inconceivable that even solutions built around delivering different code to different form factors won’t also find it important to be responsive.

The lines are blurring so quickly now that when we design web apps, we need to take into consideration how that app will respond to different screen sizes and inputs even if the app isn’t built exclusively using the three main technical pieces of responsive design (media queries, fluid grids and flexible images).

So it’s time to give up our collective hallucination and embrace the inherent nature of the web for apps as well as for pages.

These aren’t easy problems to solve and the implementation may require more than “pure” responsive design, but when we do take the time to think about our apps in a responsive manner, we build better, more future-friendly solutions for everyone.

But it is one thing to say that you believe that JavaScript UI frameworks will eventually be responsive. It is another thing altogether to figure out what should be done with an existing app built to desktop UI conventions that would need to be made responsive.

Basically, it wouldn’t make sense for our client to bet their future on responsive design unless it could be demonstrated that the types of things they are building with these JavaScript UI frameworks could also be built in a responsive manner.

So I picked a web app that was representative of the way in which they’ve been using the Telerik widgets and decided to figure out how I would make it responsive.

Expense Reimbursement Form

The app that I chose is one that you might find on any large company’s intranet: a reimbursement form. Because this was an existing app, my goal was to find a way to keep the app as close to its original form as possible. If we were starting from scratch, we might find a different solution.

It’s also important to note that this isn’t the most complex application you’ll ever see. But it does represent the types of things they’re using the UI widgets for.

The Existing App

The existing app is designed around a couple of Telerik’s widgets: the TabStrip and Grid tools.

The app starts with a screen containing some instructions, tabs to switch to different areas of the expense report, and a total of the current report at the bottom.

Clicking on each individual tab takes you to a screen for that type of expense. The information collected for a mileage expense is different than the information that is collected for lodging. Each tab has a table showing the current list of expenses.

On this screen, you can begin to see the benefits that the developers are getting from the widgets. The grid widget is used to layout the table. It also makes the table sortable and selectable. Each row can be selected and then edited, copied or deleted. Behind the scenes, the grid widget is allowing the developers to retrieve JSON results and populate the table automatically.

If you select a row to edit or add a new expense, you get a modal with a web form. There is nothing terribly special about the form other than their use of some handy widgets for things like the date picker.

There are other screens for the app, but this is the core of the interaction. And while it may seem simple up front, the backend systems that this app integrates with are terribly complex so building it was no small feat.

How to convert this to a responsive design?

After finding that the majority of the JavaScript UI frameworks were not designed to support responsive design, I decided to try to figure out what they might do with something like the grid widget if it were built responsively.

I looked at the tabs and the tables and tried to figure out how to shrink them to fit on mobile.

And I got stuck. Gloriously stuck. Banging my head on the desk stuck.

Mobile first design

After a couple of hours of struggle to envision the app as responsive, I decided to give up on making the desktop app responsive. I was beginning to think that the people who say that web apps can’t be responsive were right.

I figured if I can’t make it responsive, I better figure out what the mobile version of the app should look like because that will be the next step.

Now I’ve seen a lot of people argue against mobile first design. Most recently, Henry Blodget called mobile first a dumb strategy. He wrote:

As a result, the prevailing wisdom is that companies should become:

• “Mobile Only” (just forget big screens, keyboards, and mice–they’re dead)

Or, at least,

• “Mobile First” (design everything for smartphones first and big screens as an afterthought)

Frankly, the only people who have ever argued that mobile first means that other form factors are an afterthought are people building straw man arguments and ignoring what mobile first actually says.

Luke Wroblewski’s mobile first arguments can be summarized simply as mobile is a huge opportunity, mobile has some capabilities that aren’t practical on other platforms, and the constraints of mobile cause you to focus and build better designs.

From my perspective, the opportunity on mobile should be obvious. Using the capabilities of mobile (like geolocation) are great enhancements. But the real power comes from designing the mobile interface first.

Every time I design the mobile interface first, I find that I learn things that make the wide screen version better. Embracing constraints, particularly in design, is tremendously powerful.

How do you apply mobile first design to a desktop app?

The most common complaint I’ve heard about mobile first design is that it sounds great if you don’t already have a desktop site or app, but not everyone has that luxury.

But even if you have an existing desktop design, mobile first thinking can be useful. That mindset is what helped me see how the reimbursement form could be retooled to be a responsive design.

After I got stuck, I decided to set aside the desktop app entirely and design the app for mobile. I wasn’t convinced that responsive design was going to work so I decided to assume that the mobile app would be a separate application. What would make a great experience for mobile?

To aid me, I used an iPad application called AppCooker to build mobile wireframes. I’ve become a big fan of this application because I can quickly build wireframes and simple prototypes of interactions. Plus, it is nice to get away from the keyboard and mouse and do actual work on an iPad.

While AppCooker works for me, it doesn’t matter what tool you use. A pencil and paper would work just as well. The point is to use the mobile design as a forcing function to make you think through what really matters.

A mobile reimbursement form

So I started designing a mobile app version of the reimbursement form. As I thought about the form, I realized that the desktop app didn’t feel much like a form. I wanted to put make it clear that this was something you are filling out and submitting.

I realized that the little summary table at the bottom of the desktop app was the core of the form. You add a bunch of expenses in each category, it totals it for you, and then you hit submit.

Designing it this way meant that another tenet of mobile design was being used. The content itself was the interface. Instead of using tabs to navigate, each row in the form both served to summarize the amount for that category as well as a link to go edit the expenses in that category.

With the home screen done, I had to figure out what someone would see when they went to edit an expense category. On desktop, each row in the grid had multiple columns of information. There wasn’t enough room for that level of detail on the smaller screen.

I had to think through what really mattered for these expenses. While all of the data being collected is important, what information would make it possible for someone using the app to quickly identify which expense was which and also determine how much they were being compensated.

In this case, the title of the expense, date of expense, miles traveled, and cost seemed like the most important things.

Again, seeing the details, editing or deleting each expense became easy. Tap on the expense to get to the details of that expense and act on them.

Now, these wireframes aren’t perfect. Looking at them a few months later, I can see things that I would love to change. But they serve their purpose. They describe how the reimbursement form could be done on mobile in an efficient manner.

Mobile interfaces map well to the desktop app

When I had finished with these wireframes, I woke my computer from sleep and decided to compare what I had done to the original application. I was surprised at how well the mobile app mapped to the original application.

The widget at the bottom of the screen that tracked the expense total had become the home screen of the mobile app. The tabs had been replaced by each row in the form.

The mileage screen was almost a direct comparison except with less information in each row. Also, instead of the less intuitive select a row and then hit buttons. Selecting a row took you to the detail page.

And of course the detail screen mapped almost directly to the expense modal on desktop. It also made it see the current values and edit them.

Once I saw how well mobile was mapping to desktop, I began to think that responsive design might be possible. Not only that, but I was beginning to believe the mobile interface was superior to the desktop one.

Rethinking the desktop version

At this point, I decided to take a look at what the desktop version might look like if it was influenced by the mobile version. My goal was to keep the grid widget in the design and minimize changes to the desktop functionality as much as possible.

I would be the first to admit that this interface feels off. It is because we’re mixing widgets that are part of the mobile UI framework with ones that are designed for the desktop framework. I think the app would feel more cohesive if we ditched the desktop widgets and focused on using the mobile widgets for all screen resolutions.

But for our purposes, the wireframe proved what we needed to see. It was possible for us to envision a design that could start with mobile screens and add functionality as the screen got larger. It was now possible for us to see how we might make apps responsive even if we weren’t exactly certain how we would handle converting the current application to a responsive design.

So how did the app turn out?

On the one hand, the exercise accomplished our goal. It was clear how responsive design could be applied to their applications. More importantly, we identified the process by which designers should think through how to convert existing apps.

For this particular app, the assessment was that it wouldn’t be worth making the current code base responsive. While we might be able to convert the Telerik-based app to be responsive, we know that Telerik’s framework has been declared end-of-life so it doesn’t make sense to invest time and money into a system that is deprecated.

Instead, the device detection will be used to route people on mobile devices to a mobile app. The mobile app will be built responsively. And when they are confident that the app works as expected, it will replace the desktop version.

In Part 3, I look at similar changes that are happening with native desktop apps and what we might learn from them.

Tyler is someone whose work I have long admired. His portfolio speaks for itself. He has designed games, created illustrations, guided projects, and teaches a course on web standards.

Most importantly, he is passionate about the web and pushing the boundaries of what it can do. I think he will fit in nicely. :-)

On an unrelated note, when I started to write this post, I searched to see what I had written when Matt Gifford joined Cloud Four back in 2011. I realized that we had been remiss and had forgotten to write something back then.

So in hopes that it is not too late to remedy that oversight, I wanted to belatedly say that like Tyler, Matt was someone we knew and admired from his role in the tech community. Not only has he become a critical member of our team building complex JavaScript applications, but he has stepped up as the primary organizer of Mobile Portland’s monthly meetings.

The best thing about starting Cloud Four has been the opportunity to work with an amazing team. I feel truly privileged to get to work with some of the smartest people I know.