Functioning Form: Interface Design

Follow-ups: 07-10-2009

LukeW — 2009-07-10

Follow-ups for Hardware Becomes Software

�Apple fanboys might still be drooling from the iPhone 3Gs announcement, but I bet they never saw this coming: an electric superbike with an iPhone for all its instrumentation.� - Electric Superbike Uses iPhone For Its Dashboard, Gas 2.0 (thanks Vincent)

�Now the smartphone is beginning to displace yet another stand-alone device � the GPS receiver � as a convenient way for drivers to get directions to unknown destinations.� - Sending GPS Devices the Way of the Tape Deck?, New York Times

Follow-ups for More Sensors Coming to the iPhone...

�This is a prototype of an iPhone media player that uses physical objects to control media playback. It is based on Radio Frequency Identification (RFID) that triggers various iPhone interactions when in the range of a wireless tag embedded inside a physical object.� - iPhone RFID: object-based media, Touch (thanks Andres)

"On April 15th Nokia announced the 6212 'classic' phone that incorporates Near Field Communication technology. This phone is the fourth NFC-capable phone from Nokia in as many years and it is the first NFC device that supports 3G data connections.� - Thoughts on Nokia�s NFC developments, Touch (thanks Arran)

Tags: sensors, apple, technology, innovative UIs, product concepts

Hardware Becomes Software

LukeW — 2009-07-09

Though it's not a new trend, the rate at which hardware solutions become just software solutions is interesting to watch. As consumer device platforms (like the iPhone) integrate a rich enough set of capabilities to develop on, the transition from unique hardware for every consumer device to a set of screens that can run many software solutions accelerates. Here's just a few examples:

Tags: apple, sensors, technology, convergence, interface

Coming iPhone Software Enhancements...

LukeW — 2009-07-09

In recent articles, I've been discussing the impact of sensors on design and the transition from hardware to software. Apple's iPhone platform is a shining example of these trends in action. Today, the Patent Office published several Apple patent applications related to iPhone software. From Mac Rumors:

Object Recognition: Describes methods by which a mobile device user could detect an object via camera, RFID sensor or other means and have their device automatically identify and provide additional information on the object.

Facial Detection: Utilizes automatic facial detection via a camera on the device to determine that a user is present or passively interacting with the device. The method may also be used to grant or deny users access to restricted applications or other data.

Voice Modulation: Describes methods for modulating voice audio output to simulate additional voices. Apple's examples describe a mother who could have an audiobook read in her voice to her child or a student who wishes to alter a section of a podcast lecture for emphasis.

More on these fillings...

Tags: apple, technology, sensors, communication, innovative UIs

Common Capabilities, Different Screens

LukeW — 2009-07-08

When personal computing companies build up a compelling platform of hardware and/or software capabilities, they can bring these to life across a range of screens that align with the different facets of people's lives.

Palm-sized screens: mobile devices, PDAs, cell phones
Lap-sized screens: laptop computers, netbooks, tablets
Desk-sized screens: workstations, desktops (perhaps increasingly less important for the home)
Wall-sized screens: TVs, wall projectors

Consider the example of Apple Inc. illustrated below. The company has a set of platform capabilities that have been integrated into palm-sized phones and media players, lap-sized computers, and desk-sized workstations. In speculation, they are said to be developing an additional lap-sized screen and perhaps a networked TV (wall-sized screen) as well.

Why not? once you have a compelling set of capabilities, you can simply develop additional screens to deliver a multitude of experiences: mobile, 10 ft, work, etc.

Tags: apple, technology, convergence, sensors

Wardrobe Manager: It's Just Software

LukeW — 2009-07-07

As part of a series of articles on interface concepts five years ago, I outlined a networked device for wardrobe management. Wardrobe Manager was a WiFi connected wall-mountable display that used the RFID tags embedded in clothing to maintain digital inventory, usage patterns and recommendations for your wardrobe.

Wardrobe Manager accessed information about each of the RFID tagged items in your closet including: product photos, warrantees, care instructions, purchase dates, cost, and more. You could also set Wardrobe Manager to track your daily clothing selections. The interface would then make suggestions based on what you wore in the past, what remained in your closet, what could potentially match your current selection, what a trusted friend has suggested (through a shared social network), and what was most appropriate given today�s weather forecast. You'd never wear the same shirt to work twice in one week again!

At the time, Wardrobe Manager was conceived of as a networked consumer device. In other words, you bought the hardware and got the software. Today, this assertion seems foolish as hardware platforms are so sensor-rich that products like Wardrobe manager only need code to run.

In fact, if Apple's recent iPhone patents come to life, Wardrobe Manager can simply run as software on a networked smart phone with an RFID reader. Just install an application on your phone that makes use of the RFID reader and network connection.

For me, this example illustrates an interesting trend. When consumer device platforms integrate a rich enough set of capabilities to develop on, many consumer devices become just software.

Tags: apple, product concepts, sensors, technology, convergence

Apple Devices with Built-in Micro Projectors

LukeW — 2009-07-06

Expanding even further on my recent articles about understanding capabilities, Mac Rumors reports that Apple is expected to launch devices with built-in projectors later this year. The projectors would allow the iPhone and possibly the iPod touch to directly project video output onto an external surface.

To better understand the impact of this capability on consumer devices, take a look at this video of Pattie Maes' talk at the TED conference this year. In it she demos a wearable device with a projector that paves the way for rich interaction with our environment.

Tags: apple, product concepts, technology, innovative UIs, interface

Web Form Design at UX Book Clubs

LukeW — 2009-07-06

A number of UX Book Clubs around the world have taken up Web From Design: Filling in the Blanks as group reading material. Most recently book clubs in Ithaca, Toronto, Warsaw (photo from ndemi), and Atlanta. In support, I've video conferenced with a number of groups and my publisher, Rosenfeld Media, has been sending books for free to club organizers (paperback to US clubs, digital editions to clubs outside the US) and providing 30% discounts to book club members.
If you are interested in joining a UX Book Club in your area, check out the resources available online .

Tags: forms

First Person UI: Nearest Tube

LukeW — 2009-07-05

One of the first first person UI apps to go live in the iPhone AppStore. Nearest Tube tells Londoners where their nearest tube station is via their iPhone's video function.

When you load the app, holding it flat, all 13 lines of the London underground are displayed in coloured arrows. By tilting the phone upwards, you will see the nearest stations: what direction they are in relation to your location, how many kilometres and miles away they are and what tube lines they are on. If you continue to tilt the phone upwards, you will see stations further away, as stacked icons. Thanks for the tip Julian!

Tags: apple, navigation, sensors, innovative UIs, interface

Understanding Capabilities

LukeW — 2009-07-03

In a number of recent articles, I've been cataloging the capabilities of new consumer device product platforms with deep interest. One might wonder: why so many articles on the technology within smart phones, consoles, media players, and netbooks/pads?

It boils down to a simple mantra: know your medium. As a designer, understanding the capabilities of technology empowers you to deliver interfaces that empower people in new ways. It was this belief that lead Frank Ramirez and I to author Web Application Solutions: A Designer�s Guide more than 4 years ago (April 2005).

Web Application Solutions: A Designer�s Guide (PDF)

In the document, we compared some of the most popular Web application presentation layer solutions available through consistent criteria (deployment & reach, user interactions, processing, interface components & customization, back-end integration, future proofing, staffing & cost, unique features) and provided an overview, set of examples, and references for each. This guide helped us to better understand the technologies supporting our Web application designs. It was a way to better understand our medium.

Since the digital product design medium keeps changing, keeping up with capabilities is essential.

Tags: technology, innovative UIs, interface, product concepts, mobile

More Sensors Coming to the iPhone...

LukeW — 2009-07-02

In Designing for Sensors, I outlined how ubiquitous integration of sensor technologies in consumer devices shifts how we think about designing digital products. Soon, instead of just designing for computing (productivity and data management) or communication (email, social networks), we'll be designing for sensors.

Today Apple Inc. revealed new patent applications that cover additional sensors which could be used in future versions of the iPhone. From Mac Rumors:

Haptic Tactile Feedback: Adoption of "haptic" display technologies which allow the user to "feel" different surfaces as their finger moves across a touchscreen. As an example, a display could include a virtual click wheel which vibrates at a different frequency at the center. Users could easily sense the difference and use the click wheel without having to look at it.

Fingerprint Identification as an Input Method: Fingerprints have already been used in computers for security purposes, but Apple's research involves the use of fingerprint patterns to actually identify distinct fingers. This could then be used to produce specific functions depending on which finger is being used.

RFID Reader: Apple suggests that an RFID antenna can be placed in the touch sensor panel itself, allowing it to also be used as a RFID reader. RFID tags are tiny chips that chips that label objects with unique identities and can be used to track items at a distance.

These three capabilities would be added to an already impressive list of sensor integrations.

Location sensor: precise location coordinates from GPS
User orientation sensor: directional heading from a digital compass
Touch sensors: Multi-touch input from one or more simultaneous gestures
Light/dark sensor: Ambient light detection
Device orientation & motion sensor: from built-in accelerometer
Proximity sensor: device closeness to other objects or people
Audio sensor: input from a microphone
Image & video sensors: capture/input from a camera (all kinds of signals can be found in real-time visual information)
Device sensor: through Bluetooth
Audio broadcast sensor: FM transmitter (rumored on iPhone)

Very relevant related reading (back from 2004 and 2005): The Future of the Object and Designers Shaping Things.

Tags: apple, mobile, sensors, technology, convergence

Stacks on the iPhone

LukeW — 2009-06-29

In my Apple�s Next Integrations article, I showcased several examples of how the company integrates new capabilities across its product line. Developer Steven Troughton-Smith recently did the same himself by building an implementation of OS X's Stacks feature on the iPhone.

On a related note... is a three-dimensional remote controller (much like Nintendo's Wii) next on the list of integrations?

Tags: apple, innovative UIs, UI components, interaction design, navigation

First Person UIs on Android

LukeW — 2009-06-25

In my iPhone with a Compass = First Person UIs article, I discussed how location and orientation awareness in mobile devices opens up a set of new interface possibilities that are designed from the user�s current perspective. In other words, first person user interfaces that are built knowing where you are and where you are facing.

Recently I found three such applications have made their way to Google's Android Market. Though in their early stages, these apps demonstrate the potential of creating user interfaces from a natural, first person perspective: how we actually see the world.

IBM Seer beta
The IBM Seer beta is an application designed to help you navigate you way through a better time at Wimbledon. Point your phone at anything the app will tell you what it is. In camera mode what you're looking at and what's happening there now. For example, concessions reveal themselves to you and then tell you how long you can expect to queue. Check out screenshots of the app in action.

Layar
Layar shows what is around you by displaying real time digital information on top of reality through the camera of your mobile phone (Augmented Reality). This version is available in the Netherlands only.

Wikitude
See the world through your phone�s camera view overlaid with Wikipedia content. What you see is an annotated landscape, mountain names, landmark descriptions, and interesting stories: Augmented-reality for everyday use.

Tags: mobile, sensors, technology, innovative UIs

Web Form Design: Korean Edition

LukeW — 2009-06-24

Insight Publishing has translated my latest book Web Form Design: Filling In the Blanks into Korean and the book is now available for sale in Korea. If you are interested, take a look at the book announcement from Insight and the product listing on their site. Or peek inside the book at the Korean book site: Aladdin.

Tags: forms, global design, UI components, guidelines

Apple�s Next Integrations

LukeW — 2009-06-23

Not only does Apple Inc. develop a lot of new hardware and software capabilities, they do a great job of integrating these new capabilities where it makes sense.

Consider the lifecycle of Coverflow: a software capability that allows fluid visual browsing of digital assets. Coverflow first appeared in the iTunes media application, then made its way to the iPod line of portable media players, then became an optional way to browse files in the OSX Finder, and now serves as a visual search results list for your browsing history in the Safari Web browser.

The iSight video camera, which began life as a stand-alone hardware peripheral, was integrated into the laptop line, then recently made its way into the latest version of the iPhone. Even the video editing software on the new iPhone started life elsewhere like iLife, Final Cut, and Quicktime software.

Other notable integrations include multi-touch (phone to ipod to laptop), non-removable extended life batteries (ipod to phone to laptop), glass & aluminum, and more. Given the consistent level of integration of Apple�s hardware and software integrations, it may be reasonable to look for clues of what�s next by considering integration opportunities. For instance, face recognition was shipped in the 2009 version of iPhoto. Chances are it will show up in the iPhone soon.

Other potential integrations include:

iChat video messaging on iPhone & iPod Touch
Voice Control and Voice Memos for Macbook laptops
CoverFlow for iPhone application management
Quick Look for files on the iPhone based on OS X improvements
And so on...

Tags: apple, product concepts, technology

UX London: Parti and the Design Sandwich

LukeW — 2009-06-15

In my Parti & the Design Sandwich talk at UX London 2009, I outlined a structure to enable interaction designers to move toward a holistic product design in the face of many stakeholders, cross-functional groups, and diverse audiences.

Parti & the design sandwich is a structure for projects that allows you to go from concepts to products using the tools of design.
Parti (as defined by Matthew Frederick) is the central idea or concept of your project. A parti is expressed as a diagram depicting general structure and experiential & aesthetic sensibility.
In interaction design, that means a core interaction model. The inclusion of a core interaction model to represent the big idea is how a parti differs from product vision -though there are many parallels.
Getting a parti requires the consideration of factors outside of design such as technology opportunity, market factors, resources, alignment, etc.
Parti should focus on the aspects of an interaction that are unique to a project.
The whole team needs to buy in to a parti.
Parti is a guidepost for designing the many aspects of a project but the design sandwich helps us make informed decisions that bring a parti to life.
At the top of the sandwich are design principles. These are filters for making decisions.
At the bottom of the sandwich are design considerations. These are the factors you learn about and weigh when considering options.
The middle of the sandwich is where decisions about the design happen. Patterns, best practices, and testing -all can help inform decisions.
The distinction for me between best practices and patterns is nuanced �a pattern is a way you can do things in a specific context. A best practice is the way you SHOULD do things in a specific context.

Using parti & the design sandwich to structure a project means:

Considering all the things that are relevant to a design decision (design considerations).
Making use of patterns, best practices, and testing (when we can) to help inform decisions.
Evaluating those decisions by passing them through a filter of design principles so that all the decisions we make support the central idea of what we are making.
The point of which is to create a holistic design.
Your parti can change but don't give up on having a parti for your project.

Update: A listing of live Twitter messages from my presentation at UX London at disambiguity.com

Tags: decision making, design vision, interaction design, user experience, models, patterns