For more information on the Innovation publication, visit http://www.idsa.org/innovation

Important peer-reviewed and informally published recent research on user interface design and user experience (UX) design.

For the benefit of clients and colleagues we have culled a list of approximately 50 curated recent research publications dealing with user interface design, UX design and e-commerce optimization.

In our opinion these publications represent some of the best formal research thinking on UI and UX design. These papers are also among the most widely downloaded and cited formal research on UI / UX design. We have referenced many of these studies in our work at MauroNewMedia.

Pay walls: As you will note in reviewing the following links and abstracts, most of the serious research on UI / UX design and optimization is located behind pay walls controlled by major publishers. However, in the end, good data is well worth the investment. Many links and other cited references are, of course, free.

Important disclaimer: We do not receive any form of compensation for citing any of the following content. Either Charles L Mauro CHFP or Paul Thurman MBA has personally reviewed all papers and links in this list. Some of these references were utilized in the recent NYTECH UX talk given by Paul Thurman MBA titled: Critical New UX Design Optimization Research

Title: The influence of hedonic and utilitarian motivations on user engagement: The case of online shopping experiences

Abstract
User experience seeks to promote rich, engaging interactions between users and systems. In order for this experience to unfold, the user must be motivated to initiate an interaction with the technology. This study explored hedonic and utilitarian motivations in the context of user engagement with online shopping. Factor analysis was performed to identify a parsimonious set of factors from the Hedonic and Utilitarian Shopping Motivation Scale and the User Engagement Scale based on responses from 802 shoppers. Multiple linear regression was used to test hypotheses with hedonic and utilitarian motivations (Idea, Social, Adventure/Gratification, Value and Achievement Shopping) and attributes of user engagement (Aesthetics, Focused Attention, Perceived Usability, and Endurability). Results demonstrate the salience of Adventure/Gratification Shopping and Achievement Shopping Motivations to specific variables of user engagement in the e-commerce environment and provide considerations for the inclusion of different types of motivation into models of engaging user experiences.
Abstract Copyright © 2010 Elsevier B.V. All rights reserved.

Title: New Support for Marketing Analytics

Abstract
Consumer surveys and myriad other forms of research have long been the grist for marketing decisions at large companies. But many firms have been reluctant to embrace the high-tech approach to data gathering and number crunching that falls under the rubric of marketing analytics, which uses advanced techniques to transform the tracking of promotional efforts, customer preferences, and industry developments into sophisticated branding and advertising campaigns. Fueled in part by Tom Peters and Robert Waterman’s seminal 1982 book In Search of Excellence, which coined the phrase “paralysis through analysis,” skepticism about the approach remains widespread, even in the face of a number of positive research results over the years. This new study, involving Fortune 1000 companies, offers yet more ammunition for supporters of marketing analytics.
Abstract Copyright © 2013 Booz & Company Inc. All rights reserved.

Title: Video game values: Human-computer interaction and games

Abstract
Current human–computer interaction (HCI) research into video games rarely considers how they are different from other forms of software. This leads to research that, while useful concerning standard issues of interface design, does not address the nature of video games as games specifically. Unlike most software, video games are not made to support external, user-defined tasks, but instead define their own activities for players to engage in. We argue that video games contain systems of values which players perceive and adopt, and which shape the play of the game. A focus on video game values promotes a holistic view of video games as software, media, and as games specifically, which leads to a genuine video game HCI.
Abstract Copyright © 2006 Elsevier B.V. All rights reserved.

Title: When fingers do the talking: a study of text messaging

Abstract
SMS or text messaging is an area of growth in the communications field. The studies described below consisted of a questionnaire and a diary study. The questionnaire was designed to examine texting activities in 565 users of the mobile phone. The diary study was carried out by 24 subjects over a period of 2 weeks. The findings suggest that text messaging is being used by a wide range of people for all kinds of activities and that for some people it is the preferred means of communication. These studies should prove interesting for those examining the use and impact of SMS.
Abstract Copyright © 2004 Elsevier B.V. All rights reserved.

Title: Understanding factors affecting trust in and satisfaction with mobile banking in Korea: A modified DeLone and McLean’s model perspective

Abstract
As mobile technology has developed, mobile banking has become accepted as part of daily life. Although many studies have been conducted to assess users’ satisfaction with mobile applications, none has focused on the ways in which the three quality factors associated with mobile banking – system quality, information quality and interface design quality – affect consumers’ trust and satisfaction. Our proposed research model, based on DeLone and McLean’s model, assesses how these three external quality factors can impact satisfaction and trust. We collected 276 valid questionnaires from mobile banking customers, then analyzed them using structural equation modeling. Our results show that system quality and information quality significantly influence customers’ trust and satisfaction, and that interface design quality does not. We present herein implications and suggestions for further research.
Abstract Copyright © 2009 Elsevier B.V. All rights reserved.

Title: What is beautiful is usable

Abstract
An experiment was conducted to test the relationships between users’ perceptions of a computerized system’s beauty and usability. The experiment used a computerized application as a surrogate for an Automated Teller Machine (ATM). Perceptions were elicited before and after the participants used the system. Pre-experimental measures indicate strong correlations between system’s perceived aesthetics and perceived usability. Post-experimental measures indicated that the strong correlation remained intact. A multivariate analysis of covariance revealed that the degree of system’s aesthetics affected the post-use perceptions of both aesthetics and usability, whereas the degree of actual usability had no such effect. The results resemble those found by social psychologists regarding the effect of physical attractiveness on the valuation of other personality attributes. The findings stress the importance of studying the aesthetic aspect of human–computer interaction (HCI) design and its relationships to other design dimensions.
Abstract Copyright © 2000 Elsevier Science B.V. All rights reserved.

Title: UX Curve: A method for evaluating long-term user experience

Abstract
The goal of user experience design in industry is to improve customer satisfaction and loyalty through the utility, ease of use, and pleasure provided in the interaction with a product. So far, user experience studies have mostly focused on short-term evaluations and consequently on aspects relating to the initial adoption of new product designs. Nevertheless, the relationship between the user and the product evolves over long periods of time and the relevance of prolonged use for market success has been recently highlighted. In this paper, we argue for the cost-effective elicitation of longitudinal user experience data. We propose a method called the “UX Curve” which aims at assisting users in retrospectively reporting how and why their experience with a product has changed over time. The usefulness of the UX Curve method was assessed in a qualitative study with 20 mobile phone users. In particular, we investigated how users’ specific memories of their experiences with their mobile phones guide their behavior and their willingness to recommend the product to others. The results suggest that the UX Curve method enables users and researchers to determine the quality of long-term user experience and the influences that improve user experience over time or cause it to deteriorate. The method provided rich qualitative data and we found that an improving trend of perceived attractiveness of mobile phones was related to user satisfaction and willingness to recommend their phone to friends. This highlights that sustaining perceived attractiveness can be a differentiating factor in the user acceptance of personal interactive products such as mobile phones. The study suggests that the proposed method can be used as a straightforward tool for understanding the reasons why user experience improves or worsens in long-term product use and how these reasons relate to customer loyalty.
Abstract Copyright 2011 British Informatics Society Limited. Published by Elsevier B.V. All rights reserved.

Title: Heuristic evaluation: Comparing ways of finding and reporting usability problems

Abstract
Research on heuristic evaluation in recent years has focused on improving its effectiveness and efficiency with respect to user testing. The aim of this paper is to refine a research agenda for comparing and contrasting evaluation methods. To reach this goal, a framework is presented to evaluate the effectiveness of different types of support for structured usability problem reporting. This paper reports on an empirical study of this framework that compares two sets of heuristics, Nielsen’s heuristics and the cognitive principles of Gerhardt-Powals, and two media of reporting a usability problem, i.e. either using a web tool or paper. The study found that there were no significant differences between any of the four groups in effectiveness, efficiency and inter-evaluator reliability. A more significant contribution of this research is that the framework used for the experiments proved successful and should be reusable by other researchers because of its thorough structure.
Abstract Copyright © 2006 Elsevier B.V. All rights reserved.

Title: Socio-technical systems: From design methods to systems engineering

Abstract
It is widely acknowledged that adopting a socio-technical approach to system development leads to systems that are more acceptable to end users and deliver better value to stakeholders. Despite this, such approaches are not widely practised. We analyse the reasons for this, highlighting some of the problems with the better known socio-technical design methods. Based on this analysis we propose a new pragmatic framework for socio-technical systems engineering (STSE) which builds on the (largely independent) research of groups investigating work design, information systems, computer-supported cooperative work, and cognitive systems engineering. STSE bridges the traditional gap between organisational change and system development using two main types of activity: sensitisation and awareness; and constructive engagement. From the framework, we identify an initial set of interdisciplinary research problems that address how to apply socio-technical approaches in a cost-effective way, and how to facilitate the integration of STSE with existing systems and software engineering approaches.
Abstract Copyright © 2010 Elsevier B.V. All rights reserved.

Title: Five reasons for scenario-based design

Abstract
Scenarios of human–computer interaction help us to understand and to create computer systems and applications as artifacts of human activity—as things to learn from, as tools to use in one’s work, as media for interacting with other people. Scenario-based design of information technology addresses five technical challenges: scenarios evoke reflection in the content of design work, helping developers coordinate design action and reflection. Scenarios are at once concrete and flexible, helping developers manage the fluidity of design situations. Scenarios afford multiple views of an interaction, diverse kinds and amounts of detailing, helping developers manage the many consequences entailed by any given design move. Scenarios can also be abstracted and categorized, helping designers to recognize, capture and reuse generalizations and to address the challenge that technical knowledge often lags the needs of technical design. Finally, scenarios promote work-oriented communication among stakeholders, helping to make design activities more accessible to the great variety of expertise that can contribute to design, and addressing the challenge that external constraints designers and clients face often distract attention from the needs and concerns of the people who will use the technology.
Abstract Copyright © 2000 Elsevier Science B.V. All rights reserved.

Title: Needs, affect, and interactive products – Facets of user experience

Abstract
Subsumed under the umbrella of User Experience (UX), practitioners and academics of Human–Computer Interaction look for ways to broaden their understanding of what constitutes “pleasurable experiences” with technology. The present study considered the fulfilment of universal psychological needs, such as competence, relatedness, popularity, stimulation, meaning, security, or autonomy, to be the major source of positive experience with interactive technologies. To explore this, we collected over 500 positive experiences with interactive products (e.g., mobile phones, computers). As expected, we found a clear relationship between need fulfilment and positive affect, with stimulation, relatedness, competence and popularity being especially salient needs. Experiences could be further categorized by the primary need they fulfil, with apparent qualitative differences among some of the categories in terms of the emotions involved. Need fulfilment was clearly linked to hedonic quality perceptions, but not as strongly to pragmatic quality (i.e., perceived usability), which supports the notion of hedonic quality as “motivator” and pragmatic quality as “hygiene factor.” Whether hedonic quality ratings reflected need fulfilment depended on the belief that the product was responsible for the experience (i.e., attribution).
Abstract Copyright © 2010 Elsevier B.V. All rights reserved.

Title: The role of social presence in establishing loyalty in e-Service environments

Abstract
Compared to offline shopping, the online shopping experience may be viewed as lacking human warmth and sociability as it is more impersonal, anonymous, automated and generally devoid of face-to-face interactions. Thus, understanding how to create customer loyalty in online environments (e-Loyalty) is a complex process. In this paper a model for e-Loyalty is proposed and used to examine how varied conditions of social presence in a B2C e-Services context influence e-Loyalty and its antecedents of perceived usefulness, trust and enjoyment. This model is examined through an empirical study involving 185 subjects using structural equation modeling techniques. Further analysis is conducted to reveal gender differences concerning hedonic elements in the model on e-Loyalty.
Abstract Copyright © 2006 Elsevier B.V. All rights reserved.

Title: A framework for evaluating the usability of mobile phones based on multi-level, hierarchical model of usability factors

Abstract
As a mobile phone has various advanced functionalities or features, usability issues are increasingly challenging. Due to the particular characteristics of a mobile phone, typical usability evaluation methods and heuristics, most of which are relevant to a software system, might not effectively be applied to a mobile phone. Another point to consider is that usability evaluation activities should help designers find usability problems easily and produce better design solutions. To support usability practitioners of the mobile phone industry, we propose a framework for evaluating the usability of a mobile phone, based on a multi-level, hierarchical model of usability factors, in an analytic way. The model was developed on the basis of a set of collected usability problems and our previous study on a conceptual framework for identifying usability impact factors. It has multi-abstraction levels, each of which considers the usability of a mobile phone from a particular perspective. As there are goal-means relationships between adjacent levels, a range of usability issues can be interpreted in a holistic as well as diagnostic way. Another advantage is that it supports two different types of evaluation approaches: task-based and interface-based. To support both evaluation approaches, we developed four sets of checklists, each of which is concerned, respectively, with task-based evaluation and three different interface types: Logical User Interface (LUI), Physical User Interface (PUI) and Graphical User Interface (GUI). The proposed framework specifies an approach to quantifying usability so that several usability aspects are collectively measured to give a single score with the use of the checklists. A small case study was conducted in order to examine the applicability of the framework and to identify the aspects of the framework to be improved. It showed that it could be a useful tool for evaluating the usability of a mobile phone. Based on the case study, we improved the framework in order that usability practitioners can use it more easily and consistently.
Abstract Copyright © 2011 British Informatics Society Limited. Published by Elsevier B.V. All rights reserved.

Title: Understanding the most satisfying and unsatisfying user experiences: Emotions, psychological needs, and context

Abstract
The aim of this research was to study the structure of the most satisfying and unsatisfying user experiences in terms of experienced emotions, psychological needs, and contextual factors. 45 university students wrote descriptions of their most satisfying and unsatisfying recent user experiences and analyzed those experiences using the Positive and Negative Affect Schedule (PANAS) method for experienced emotions, a questionnaire probing the salience of 10 psychological needs, and a self-made set of rating scales for analyzing context. The results suggested that it was possible to capture variations in user experiences in terms of experienced emotions, fulfillment of psychological needs, and context effectively by using psychometric rating scales. The results for emotional experiences showed significant differences in 16 out of 20 PANAS emotions between the most satisfying and unsatisfying experiences. The results for psychological needs indicated that feelings of autonomy and competence emerged as highly salient in the most satisfying experiences and missing in the unsatisfying experiences. High self-esteem was also notably salient in the most satisfying experiences. The qualitative results indicated that most of the participants’ free-form qualitative descriptions, especially for the most unsatisfying user experiences, gave important information about the pragmatic aspects of the interaction, but often omitted information about hedonic and social aspects of user experience.
Abstract Copyright © 2011 British Informatics Society Limited. Published by Elsevier B.V. All rights reserved.

Title: The Usability Metric for User Experience

Abstract
The Usability Metric for User Experience (UMUX) is a four-item Likert scale used for the subjective assessment of an application’s perceived usability. It is designed to provide results similar to those obtained with the 10-item System Usability Scale, and is organized around the ISO 9241-11 definition of usability. A pilot version was assembled from candidate items, which was then tested alongside the System Usability Scale during usability testing. It was shown that the two scales correlate well, are reliable, and both align on one underlying usability factor. In addition, the Usability Metric for User Experience is compact enough to serve as a usability module in a broader user experience metric.
Abstract Copyright © 2010 Elsevier B.V. All rights reserved.

Title: User acceptance of mobile Internet: Implication for convergence technologies

Abstract
Using the Technology Acceptance Model as a conceptual framework and a method of structural equation modeling, this study analyzes the consumer attitude toward Wi-Bro drawing data from 515 consumers. Individuals’ responses to questions about whether they use/accept Wi-Bro were collected and combined with various factors modified from the Technology Acceptance Model.

The result of this study show that users’ perceptions are significantly associated with their motivation to use Wi-Bro. Specifically, perceived quality and perceived availability are found to have significant effect on users’ extrinsic and intrinsic motivation. These new factors are found to be Wi-Bro-specific factors, playing as enhancing factors to attitudes and intention.
Abstract Copyright © 2007 Elsevier B.V. All rights reserved.

Title: Understanding purchasing behaviors in a virtual economy: Consumer behavior involving virtual currency in Web 2.0 communities

Abstract
This study analyzes consumer purchasing behavior in Web 2.0, expanding the technology acceptance model (TAM), focusing on which variables influence the intention to transact with virtual currency. Individuals’ responses to questions about attitude and intention to transact in Web 2.0 were collected and analyzed with various factors modified from the TAM. The results of the proposed model show that subjective norm is a key behavioral antecedent to using virtual currency. In the extended model, the moderating effects of subjective norm on the relations among the variables were found to be significant. The new set of variables is virtual environment-specific, acting as factors enhancing attitudes and behavioral intentions in Web 2.0 transactions.
Abstract Copyright © 2008 Elsevier B.V. All rights reserved.

Title: Fundamentals of physiological computing

Abstract
This review paper is concerned with the development of physiological computing systems that employ real-time measures of psychophysiology to communicate the psychological state of the user to an adaptive system. It is argued that physiological computing has enormous potential to innovate human–computer interaction by extending the communication bandwidth to enable the development of ‘smart’ technology. This paper focuses on six fundamental issues for physiological computing systems through a review and synthesis of existing literature, these are (1) the complexity of the psychophysiological inference, (2) validating the psychophysiological inference, (3) representing the psychological state of the user, (4) designing explicit and implicit system interventions, (5) defining the biocybernetic loop that controls system adaptation, and (6) ethical implications. The paper concludes that physiological computing provides opportunities to innovate HCI but complex methodological/conceptual issues must be fully tackled during the research and development phase if this nascent technology is to achieve its potential.
Abstract Copyright © 2008 Elsevier B.V. All rights reserved.

Title: Modelling user experience with web sites: Usability, hedonic value, beauty and goodness

Abstract
Recent research into user experience has identified the need for a theoretical model to build cumulative knowledge in research addressing how the overall quality or ‘goodness’ of an interactive product is formed. An experiment tested and extended Hassenzahl’s model of aesthetic experience. The study used a 2 × 2 × (2) experimental design with three factors: principles of screen design, principles for organizing information on a web page and experience of using a web site. Dependent variables included hedonic perceptions and evaluations of a web site as well as measures of task performance, navigation behaviour and mental effort. Measures, except Beauty, were sensitive to manipulation of web design. Beauty was influenced by hedonic attributes (identification and stimulation), but Goodness by both hedonic and pragmatic (user-perceived usability) attributes as well as task performance and mental effort. Hedonic quality was more stable with experience of web-site use than pragmatic quality and Beauty was more stable than Goodness.
Abstract Copyright © 2008 Elsevier B.V. All rights reserved.

Title: Sample Size In Usability Studies

Abstract
Usability studies are a cornerstone activity for developing usable products. Their effectiveness depends on sample size, and determining sample sizehas been a research issue in usability engineering for the past 30 years. In 2010, Hwang and Salvendy reported a meta study on the effectiveness of usability evaluation, concluding that a sample size of 10±2 is sufficient for discovering 80% of usability problems (not five, as suggested earlier by Nielsen in 2000). Here, I show the Hwang and Salvendy study ignored fundamental mathematical properties of the problem, severely limiting the validity of the 10±2 rule, then look to reframe the issue of effectiveness and sample-size estimation to the practices and requirements commonly encountered in industrial-scale usability studies.
Abstract Copyright © 2013 ACM, Inc.

Title: An experimental study of learner perceptions of the interactivity of web-based instruction

Abstract
An effectively designed interaction mechanism creates a shortcut for human–computer interaction. Most studies in this area have concluded that the higher the level of interactivity, the better, especially regarding interactive websites applied in the fields of business and education. Previous studies have also suggested that designs with a higher level of interactivity result in higher learner evaluations of websites. However, little research has examined learner perceptions as they interact with web-based instruction (WBI) systems in a situation with limited time. To assist learners in acquiring knowledge quickly, the interactivity design must make the web learning environment easier to use by reducing the complexity of the interface. The aim of the present study is to explore learner perceptions of three WBI systems with different interaction levels under time limitations. This study was therefore designed to provide a new framework to design systems with different degrees of interactivity, and to examine learners’ perceptions of these interaction elements. Three WBI systems were developed with different degrees of interactivity from high to low, and a between-subject experiment was conducted with 45 subjects. The results of the experiment indicate that a higher level of interactivity does not necessarily guarantee a higher perception of interactivity in a short-term learning situation. Therefore, the instructors must pay attention to modifying or selecting appropriate interactive elements that are more suitable for various learning stages. The findings provide insights for designers to adopt different degrees of interactivity in their designs that will best fulfill various learners’ needs.
Abstract Copyright © 2011 British Informatics Society Limited. Published by Elsevier B.V. All rights reserved.

Title: Age differences in the perception of social presence in the use of 3D virtual world for social interaction

Abstract
3D virtual worlds are becoming increasingly popular as tool for social interaction, with the potential of augmenting the user’s perception of physical and social presence. Thus, this technology could be of great benefit to older people, providing home-bound older users with access to social, educational and recreational resources. However, so far there have been few studies looking into how older people engage with virtual worlds, as most research in this area focuses on younger users. In this study, an online experiment was conducted with 30 older and 30 younger users to investigate age differences in the perception of presence in the use of virtual worlds for social interaction. Overall, we found that factors such as navigation and prior experience with text messaging tools played a key role in older people’s perception of presence. Both physical and social presence was found to be linked to the quality of social interaction for users of both age groups. In addition, older people displayed proxemic behavior which was more similar to proxemic behavior in the physical world when compared to younger users.
Abstract Copyright © 2012 British Informatics Society Limited. Published by Elsevier B.V. All rights reserved.

Title: Human error and information systems failure: the case of the London ambulance service computer-aided despatch system project

Abstract
Human error and systems failure have been two constructs that have become linked in many contexts. In this paper we particularly focus on the issue of failure in relation to that group of software systems known as information systems. We first review the extant theoretical and empirical work on this topic. Then we discuss one particular well-known case — that of the London ambulance service computer-aided despatch system (Lascad) project — and use it as a particularly cogent example of the features of information systems failure. We maintain that the tendency to analyse information systems failure solely from a technological standpoint is limiting, that the nature of information systems failure is multi-faceted, and hence cannot be adequately understood purely in terms of the immediate problems of systems construction. Our purpose is also to use the generic material on IS failure and the specific details of this particular case study to critique the issues of safety, criticality, human error and risk in relation to systems not currently well considered in relation to these areas.
Abstract Copyright © 1999 Elsevier B.V. All rights reserved.

Title: Feminist HCI meets facebook: Performativity and social networking sites

Abstract
In this paper, I reflect on a specific product of interaction design, social networking sites. The goals of this paper are twofold. One is to bring a feminist reflexivity, to HCI, drawing on the work of Judith Butler and her concepts of peformativity, citationality, and interpellation. Her approach is, I argue, highly relevant to issues of identity and self-representation on social networking sites; and to the co-constitution of the subject and technology. A critical, feminist HCI must ask how social media and other HCI institutions, practices, and discourses are part of the processes by which sociotechnical configurations are constructed. My second goal is to examine the implications of such an approach by applying it to social networking sites (SNSs) drawing the empirical research literature on SNSs, to show how SNS structures and policies help shape the subject and hide the contingency of subject categories.
Abstract Copyright © 2011 British Informatics Society Limited. Published by Elsevier B.V. All rights reserved.

Title: A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing

Abstract
Physiological computing represents a mode of human–computer interaction where the computer monitors, analyzes and responds to the user’s psychophysiological activity in real-time. Within the field, autonomic nervous system responses have been studied extensively since they can be measured quickly and unobtrusively. However, despite a vast body of literature available on the subject, there is still no universally accepted set of rules that would translate physiological data to psychological states. This paper surveys the work performed on data fusion and system adaptation using autonomic nervous system responses in psychophysiology and physiological computing during the last ten years. First, five prerequisites for data fusion are examined: psychological model selection, training set preparation, feature extraction, normalization and dimension reduction. Then, different methods for either classification or estimation of psychological states from the extracted features are presented and compared. Finally, implementations of system adaptation are reviewed: changing the system that the user is interacting with in response to cognitive or affective information inferred from autonomic nervous system responses. The paper is aimed primarily at psychologists and computer scientists who have already recorded autonomic nervous system responses and now need to create algorithms to determine the subject’s psychological state.
Abstract Copyright © 2012 British Informatics Society Limited. Published by Elsevier B.V. All rights reserved.

Title: Positive mood induction procedures for virtual environments designed for elderly people

Abstract
Positive emotions have a significant influence on mental and physical health. Their role in the elderly’s wellbeing has been established in numerous studies. It is therefore worthwhile to explore ways in which elderly people can increase the number of positive experiences in their daily lives. This paper describes two Virtual Environments (VEs) that were used as mood induction procedures (MIPs) for this population. In addition, the VEs’ efficacy at increasing joy and relaxation in elderly users is analyzed. The VEs contain exercises for generating positive-autobiographic memories, mindfulness and slow breathing rhythms. The total sample comprised 18 participants over 55 years old who used the VEs on two occasions. Twelve of them used the joy environment, while 16 used the relaxation environment. Moods before and after each session were assessed using Visual Analogical Scales. After using both VEs, results indicated significant increases in joy and relaxation and significant decreases in sadness and anxiety. The participants also indicated low levels of difficulty of use and high levels of satisfaction and sense of presence. Hence, the VEs demonstrate their usefulness at promoting positive affects and enhancing the wellbeing of elderly people.
Abstract Copyright © 2012 British Informatics Society Limited. Published by Elsevier B.V. All rights reserved.

Title: The effects of trust, security and privacy in social networking: A security-based approach to understand the pattern of adoption

Abstract
Social network services (SNS) focus on building online communities of people who share interests and/or activities, or who are interested in exploring the interests and activities of others. This study examines security, trust, and privacy concerns with regard to social networking Websites among consumers using both reliable scales and measures. It proposes an SNS acceptance model by integrating cognitive as well as affective attitudes as primary influencing factors, which are driven by underlying beliefs, perceived security, perceived privacy, trust, attitude, and intention. Results from a survey of SNS users validate that the proposed theoretical model explains and predicts user acceptance of SNS substantially well. The model shows excellent measurement properties and establishes perceived privacy and perceived security of SNS as distinct constructs. The finding also reveals that perceived security moderates the effect of perceived privacy on trust. Based on the results of this study, practical implications for marketing strategies in SNS markets and theoretical implications are recommended accordingly.
Abstract Copyright © 2010 Elsevier B.V. All rights reserved.

Title: Usability testing: what have we overlooked?

Abstract
For more than a decade, the number of usability test participants has been a major theme of debate among usability practitioners and researchers keen to improve usability test performance. This paper provides evidence suggesting that the focus be shifted to task coverage instead. Our data analysis of nine commercial usability test teams participating in the CUE-4 study revealed no significant correlation between the percentage of problems found or of new problems and number of test users, but correlations of both variables and number of user tasks used by each usability team were significant. The role of participant recruitment on usability test performance and future research directions are discussed.
Abstract Copyright © 2013 ACM, Inc.

Title: Predicting online grocery buying intention: a comparison of the theory of reasoned action and the theory of planned behavior

Abstract
This paper tests the ability of two consumer theories—the theory of reasoned action and the theory of planned behavior—in predicting consumer online grocery buying intention. In addition, a comparison of the two theories is conducted. Data were collected from two web-based surveys of Danish (n=1222) and Swedish (n=1038) consumers using self-administered questionnaires. These results suggest that the theory of planned behavior (with the inclusion of a path from subjective norm to attitude) provides the best fit to the data and explains the highest proportion of variation in online grocery buying intention.
Abstract Copyright © 2013 Elsevier B.V. All rights reserved.

Title: Decomposition and crossover effects in the theory of planned behavior: A study of consumer adoption intentions

Abstract
The Theory of Planned Behavior, an extension of the well-known Theory of Reasoned Action, is proposed as a model to predict consumer adoption intention. Three variations of the Theory of Planned Behavior are examined and compared to the Theory of Reasoned Action. The appropriateness of each model is assessed with data from a consumer setting. Structural equation modelling using maximum likelihood estimation for the four models revealed that the traditional forms of the Theory of Reasoned Action and the Theory of Planned Behavior fit the data adequately. Decomposing the belief structures and allowing for crossover effects in the Theory of Planned Behavior resulted in improvements in model prediction. The application of each model to theory development and management intervention is explored.
Abstract Copyright © 1995 Elsevier B.V. All rights reserved.

Title: Knowledge and the Prediction of Behavior: The Role of Information Accuracy in the Theory of Planned Behavior

Abstract
The results of the present research question the common assumption that being well informed is a prerequisite for effective action to produce desired outcomes. In Study 1 (N = 79), environmental knowledge had no effect on energy conservation, and in Study 2 (N = 79), alcohol knowledge was unrelated to drinking behavior. Such disappointing correlations may result from an inappropriate focus on accuracy of information at the expense of its relevance to and support for the behavior. Study 3 (N = 85) obtained a positive correlation between knowledge and pro-Muslim behavior, but Study 4 (N = 89) confirmed the proposition that this correlation arose because responses on the knowledge test reflected underlying attitudes. Study 4 also showed that the correlation could become positive or negative by appropriate selection of questions for the knowledge test. The theory of planned behavior (Ajzen, 1991), with its focus on specific actions, predicted intentions and behavior in all four studies.
Abstract Copyright © 2013 Informa plc

Link: http://www.businessinsider.com/ron-johnson-apple-store-j-c-penney-2011-11

People come to the Apple Store for the experience — and they’re willing to pay a premium for that. There are lots of components to that experience, but maybe the most important — and this is something that can translate to any retailer — is that the staff isn’t focused on selling stuff, it’s focused on building relationships and trying to make people’s lives better.
Abstract Copyright © 2013 Business Insider, Inc. All rights reserved.

Title: Naturalizing aesthetics: Brain areas for aesthetic appraisal across sensory modalities

Abstract
We present here the most comprehensive analysis to date of neuroaesthetic processing by reporting the results of voxel-based meta-analyses of 93 neuroimaging studies of positive-valence aesthetic appraisal across four sensory modalities. The results demonstrate that the most concordant area of activation across all four modalities is the right anterior insula, an area typically associated with visceral perception, especially of negative valence (disgust, pain, etc.). We argue that aesthetic processing is, at its core, the appraisal of the valence of perceived objects. This appraisal is in no way limited to artworks but is instead applicable to all types of perceived objects. Therefore, one way to naturalize aesthetics is to argue that such a system evolved first for the appraisal of objects of survival advantage, such as food sources, and was later co-opted in humans for the experience of artworks for the satisfaction of social needs.
Abstract Copyright © 2011 Elsevier Inc. All rights reserved.

Link: http://www.scientificamerican.com/article.cfm?id=the-neuroscience-of-beauty

Studies from neuroscience and evolutionary biology challenge this separation of art from non-art. Human neuroimaging studies have convincingly shown that the brain areas involved in aesthetic responses to artworks overlap with those that mediate the appraisal of objects of evolutionary importance, such as the desirability of foods or the attractiveness of potential mates. Hence, it is unlikely that there are brain systems specific to the appreciation of artworks; instead there are general aesthetic systems that determine how appealing an object is, be that a piece of cake or a piece of music.
Abstract © 2013 Scientific American, a Division of Nature America, Inc.

Link: http://blogs.scientificamerican.com/symbiartic/2011/10/03/need-proof-that-were-visual-beings/

This video offers proof that humans are visual beings.
Abstract © 2013 Scientific American, a Division of Nature America, Inc.

Link: http://hbr.org/web/slideshows/five-charts-that-changed-business/1-slide

Once in a while, a chart so deftly captures an important strategic insight that it becomes an iconic part of management thinking and a tool that shows up in MBA classrooms and corporate boardrooms for years to come. As HBR prepares for its 90th anniversary, in 2012, their editors have combed the magazine archives and other sources to select five charts that changed the shape of strategy.
Abstract Copyright © 2013 Harvard Business School Publishing. All rights reserved.

Link: http://www.strategy-business.com/article/04412

It is a widely accepted and rarely challenged tenet of marketing that companies can sustain competitive advantage only through “new and improved” product differentiation based on unique features and benefits. What a mistake. By paying attention to what consumers really want, companies can attract new customers and create a distinctive brand.
Abstract © 2013 Booz & Company Inc. All rights reserved.

Link: http://www.economist.com/node/17723028

If you can have everything in 57 varieties, making decisions becomes hard work. Many of these options have improved life immeasurably in the rich world, and to a lesser extent in poorer parts. They are testimony to human ingenuity and innovation. Free choice is the basis on which markets work, driving competition and generating economic growth. It is the cornerstone of liberal democracy. The 20th century bears the scars of too many failed experiments in which people had no choice. But amid all the dizzying possibilities, a nagging question lurks: is so much extra choice unambiguously a good thing?
Abstract Copyright © The Economist Newspaper Limited 2013. All rights reserved.

Link: http://e.businessinsider.com/public/1099804

Mobile apps are becoming more important to people, not less important, according to this chart plucked from a big presentation on the internet. It’s an interesting trend because it shows how mobile behavior is different than traditional desktop computing behavior when it comes to the web.
Abstract Copyright © 2013 Business Insider, Inc. All rights reserved.

Link: http://blogs.scientificamerican.com/scicurious-brain/2012/07/30/you-want-that-well-i-want-it-too-the-neuroscience-of-mimetic-desire/

Mimetic desire is more than jealously wanting something because someone else has it. Rather, it’s about valuing something because someone else values it. And it’s pretty easy to transmit the value. Just writing about Person A’s activities and habits and showing it to Person B will make Person B start to think Person A must have seen something good about the Toyota Camry…maybe his next car…

But what is behind this contagion of desires?
Abstract © 2013 Scientific American, a Division of Nature America, Inc.

Link: http://www.united-academics.org/magazine/27212/visual-memory-blindness/?utm_source=feedburner&utm_medium=twitter&utm_campaign=Feed%3A+ResearchBloggingAllEnglish+%28Research+Blogging+-+English+-+All+Topics%29

A well-known pheonomenon in psychology has been the ‘inattentional blindness’ principle. In fact, you might know it from experience: it means that people tend to fail seeing things in their visible fields when they have to focus on a task. Until now, it was thought that in order to cause the effect, a cluttered visual field is required. Recent research shows that the effect is present though in many more situations.
Abstract Copyright United Academics 2012 Coypright – All rights Reserved

Link: http://www.businessinsider.com/18-24-texting-2011-9?nr_email_referer=1&utm_source=Triggermail&utm_medium=email&utm_term=SAI%20Chart%20Of%20The%20Day&utm_campaign=SAI_COTD_092111

Chart of the Day: According to the Pew Internet project, people in the 18-24 year-old range are sending and receiving 110 texts per day on average. The median number of texts sent/received by that group is 50 per day.
Abstract Copyright © 2013 Business Insider, Inc. All rights reserved.

Link: http://www.businessinsider.com/chart-of-the-day-facebook-time-2011-9?nr_email_referer=1&utm_source=Triggermail&utm_medium=email&utm_term=SAI%20Chart%20Of%20The%20Day&utm_campaign=SAI_COTD_091211

Chart of the Day: A new report on social media from Nielsen shows U.S. users spent 53.5 billion minutes on Facebook in May, which is more time than was spent on the next four biggest sites.
Abstract Copyright © 2013 Business Insider, Inc. All rights reserved.

Link: http://www.scientificamerican.com/article.cfm?id=your-brain-on-facebook

A recent study showed that certain brain areas expand in people who have greater numbers of friends on Facebook. There was a problem, though. The study, inProceedings of the Royal Society B, was unable to resolve the question of whether “friending” plumps up the brain areas or whether people with a type of robustness in brain physiology are just natural social butterflies. But with the help of a few monkeys in England, teenagers everywhere may now have more ammunition to use against parents.
Abstract © 2013 Scientific American, a Division of Nature America, Inc.

Link: http://techcrunch.com/2013/02/05/amazon-to-launch-virtual-currency-amazon-coins-in-its-appstore-in-may/

Amazon has just announced a new virtual currency for Kindle Fire owners to use on in-app purchases, app purchases, etc. in the Amazon Appstore.
Abstract © 2013 AOL Inc. All rights reserved.

Link: http://www.thinkwithgoogle.com/insights/library/studies/the-new-multi-screen-world-study/

Today 90% of our media consumption occurs in front of a screen. As consumers balance their time between smartphones, tablets, PCs and televisions, they are learning to use these devices together to achieve their goals. This multi-screen behavior is quickly becoming the norm, and understanding it has become an imperative for businesses. This article offers a few insights from Google’s latest research study.
Abstract Copyright Google Inc

Link: Interesting social media, a/b testing, e-commerce, and other stats from 2012 (via Twitter):  ow.ly/gQ5OO

updated on 5/13

Title: Developing elements of user experience for mobile phones and services: survey, interview, and observation approaches

Abstract
The term user experience (UX) encompasses the concepts of usability and affective engineering. However, UX has not been defined clearly. In this study, a literature survey, user interview and indirect observation were conducted to develop definitions of UX and its elements. A literature survey investigated 127 articles that were considered to be helpful to define the concept of UX. An in-depth interview targeted 14 hands-on workers in the Korean mobile phone industry. An indirect observation captured daily experiences of eight end-users with mobile phones. This study collected various views on UX from academia, industry, and end-users using these three approaches. As a result, this article proposes definitions of UX and its elements: usability, affect, and user value. These results are expected to help design products or services with greater levels of UX.
Abstract Copyright 2011 Wiley Periodicals, Inc.

Title: Why different people prefer different systems for different tasks: An activity perspective on technology adoption in a dynamic user environment

Abstract
In a contemporary user environment, there are often multiple information systems available for a certain type of task. Based on the premises of Activity Theory, this study examines how user characteristics, system experiences, and task situations influence an individual’s preferences among different systems in terms of user readiness to interact with each. It hypothesizes that system experiences directly shape specific user readiness at the within-subject level, user characteristics and task situations make differences in general user readiness at the between-subject level, and task situations also affect specific user readiness through the mediation of system experiences. An empirical study was conducted, and the results supported the hypothesized relationships. The findings provide insights on how to enhance technology adoption by tailoring system development and management to various task contexts and different user groups.
Abstract Copyright 2011 ASIS&T

Title: A review of factors influencing user satisfaction in information retrieval

Abstract
The authors investigate factors influencing user satisfaction in information retrieval. It is evident from this study that user satisfaction is a subjective variable, which can be influenced by several factors such as system effectiveness, user effectiveness, user effort, and user characteristics and expectations. Therefore, information retrieval evaluators should consider all these factors in obtaining user satisfaction and in using it as a criterion of system effectiveness. Previous studies have conflicting conclusions on the relationship between user satisfaction and system effectiveness; this study has substantiated these findings and supports using user satisfaction as a criterion of system effectiveness.
Abstract Copyright 2010 ASIS&T

Title: The development and evaluation of a survey to measure user engagement

Abstract
Facilitating engaging user experiences is essential in the design of interactive systems. To accomplish this, it is necessary to understand the composition of this construct and how to evaluate it. Building on previous work that posited a theory of engagement and identified a core set of attributes that operationalized this construct, we constructed and evaluated a multidimensional scale to measure user engagement. In this paper we describe the development of the scale, as well as two large-scale studies (N=440 and N=802) that were undertaken to assess its reliability and validity in online shopping environments. In the first we used Reliability Analysis and Exploratory Factor Analysis to identify six attributes of engagement: Perceived Usability, Aesthetics, Focused Attention, Felt Involvement, Novelty, and Endurability. In the second we tested the validity of and relationships among those attributes using Structural Equation Modeling. The result of this research is a multidimensional scale that may be used to test the engagement of software applications. In addition, findings indicate that attributes of engagement are highly intertwined, a complex interplay of user-system interaction variables. Notably, Perceived Usability played a mediating role in the relationship between Endurability and Novelty, Aesthetics, Felt Involvement, and Focused Attention.
Abstract Copyright 2009 ASIS&T

Title: Exploring user engagement in online news interactions

Abstract
This paper describes a qualitative study of online news reading and browsing. Thirty people participated in a quasi-experimental study in which they were asked to browse a news website and select three stories to discuss at a social gathering. Semi-structured interviews were conducted post-task to understand participants’ perceptions of what makes online news reading and browsing engaging or non-engaging. Findings as presented within the experience-based framework of user engagement and demonstrate the complexity of users’ interactions with information content and systems in online news environments. This study extends the model of user engagement and contributes new insights into user’s experience in casual-leisure settings, such as online news, which has implications for other information domains.
Abstract Copyright 2011 by American Society for Information Science and Technology

Title: Exploring user engagement in online news interactions

Abstract
This chapter of The Fabric of Mobile Services: Software Paradigms and Business Demands contains sections titled: New Services and User Experience, User-Centered Simplicity and Experience, Methodologies for Simplicity and User Experience, and Case Studies: Simplifying Paradigms
Abstract Copyright 2009 John Wiley & Sons, Inc.

Title: The Right Angle: Visual Portrayal of Products Affects Observers’ Impressions of Owners

Abstract
Consumer products have long been known to influence observers’ impressions of product owners. The angle at which products are visually portrayed in advertisements, however, may be an overlooked factor in these effects. We hypothesize and find that portrayals of the same product from different viewpoints can prime different associations that color impressions of product and owner in parallel ways. In Study 1, automobiles were rated higher on status- and power-related traits (e.g., dominant, powerful) when portrayed head-on versus in side profile, an effect found for sport utility vehicles (SUVs)—a category with a reputation for dominance—but not sedans. In Study 2, these portrayal-based associations influenced the impressions formed about the product’s owner: a target person was rated higher on status- and power-related traits when his SUV was portrayed head-on versus in side profile. These results suggest that the influence of visual portrayal extends beyond general evaluations of products to affect more specific impressions of products and owners alike, and highlight that primed traits are likely to influence impressions when compatible with other knowledge about the target.
Abstract Copyright 2012 Wiley Periodicals, Inc

Title: The Counterfeit Self: The Deceptive Costs of Faking It

Abstract
Although people buy counterfeit products to signal positive traits, we show that wearing counterfeit products makes individuals feel less authentic and increases their likelihood of both behaving dishonestly and judging others as unethical. In four experiments, participants wore purportedly fake or authentically branded sunglasses. Those wearing fake sunglasses cheated more across multiple tasks than did participants wearing authentic sunglasses, both when they believed they had a preference for counterfeits (Experiment 1a) and when they were randomly assigned to wear them (Experiment 1b). Experiment 2 shows that the effects of wearing counterfeit sunglasses extend beyond the self, influencing judgments of other people’s unethical behavior. Experiment 3 demonstrates that the feelings of inauthenticity that wearing fake products engenders—what we term the counterfeit selfmediate the impact of counterfeits on unethical behavior. Finally, we show that people do not predict the impact of counterfeits on ethicality; thus, the costs of counterfeits are deceptive.
Abstract Copyright 2010 Francesca Gino, Michael I. Norton, and Dan Ariely3

Send Us Your Research References: If you have interesting and relevant research references post, post content as comment below for possible inclusion in next year’s updated list.

Other Content from PulseUX: Here are 2 other references from widely read and quoted long-form posts you may find interesting.

Angry Birds UX: Why Angry Birds is so successful and popular: a cognitive teardown of the user experience (1.5 million page views).

http://www.mauronewmedia.com/blog/why-angry-birds-is-so-successful-a-cognitive-teardown-of-the-user-experience/

Apple v. Samsung: Impact and Implications for Product Design, User Interface Design (UX), Software Development and the Future of High-Technology Consumer Products

http://www.mauronewmedia.com/blog/apple-v-samsung-implications-for-product-design-user-interface-ux-design-software-development-and-the-future-of-high-technology-consumer-products/

Charles L Mauro CHFP
President / Founder
MauroNewMedia

Find out more about Charles L Mauro
Find out more about MauroNewMedia
Follow Pulse>UX on Twitter @PulseUX

A review of factors influencing user satisfaction in information retrieval

   http://360vr.com/2011/06/22-discovery-flight-deck-opf_6236/index.html

MauroNewMedia Sponsor
PulseUX Blog / NYTECH UX Design Series 2013 / IDSA Design Protection Section

How Important Is This Case?

No other U.S. patent case in recent history has consumed as much media bandwidth and generated more discussion than the recent Apple v. Samsung case in the Northern District of California⁶. That case resulted in the largest jury award for patent infringement in history (1.05 billion dollars). However, this is by no means the first patent case with potentially massive implications for the technology we have come to rely on to improve our daily lives. Other cases have had similar impact including the famous Wright Brothers⁷ patent litigation involving fundamental aircraft design and the battle between Alexander Graham Bell and Western Union over misuse of Bell’s telephone patent⁸. Over the past 100 years there have been a handful of patent cases that have fundamentally impacted how products in major commercial sectors are designed and commercialized in the U.S. market. While it is too early to place the Apple vs. Samsung case at the same level of impact as the Wright Brothers litigation, it is likely that the case has altered how smart companies going forward will create and protect innovations in the mobile platform space.

More Than a High Profile Legal Case

Lawyers see this case as simply a series of patents and related protections litigated brilliantly to the financial benefit of Apple. Surprisingly, the legal position is not the most important fall-out from the verdict. The case will certainly have an impact on Product Design, User Interface (UX) Design and Software Development. This fact will remain valid whether or not aspects of the case are overturned on appeal. Specifically, when one peels back the veneer of the case and views the judgment from the perspective of Product Design, User Interface (UX) Design and Software Development, surprising insights emerge which a legal analysis fails entirely to frame. There is far more to the case than simple financial or legal precedent.

What Was Apple Really Trying to Protect?

The answer to this question is found in detailed examination of the patents that Apple chose to assert against Samsung. Below is a list of the patents that the parties asserted against each other in this matter⁹.

Apple initially accused Samsung of infringing:

• `381 patent: touchscreen interactions
• `915 patent: using an API to scroll through documents
• `163 patent: tap-to-zoom
• `677 patent: general outline and ornamental design
• `087 patent: general outline and ornamental design
• `889 patent: ornamental design
• `305 patent: GUI for a display screen

Samsung countersued, accusing Apple of infringing:

• `941 patent: mobile phone 3G capabilities
• `516 patent: mobile phone 3G capabilities
• `711 patent: MP3 playback on a mobile device
• `460 patent: device that functions as a phone and a camera
• `893 patent: method for remembering user’s place in a gallery

When one decomposes the patents asserted by Apple against Samsung it is clear that Apple was intensely focused on protecting more than individual patents taken at random from its portfolio of about 1,300 patents¹⁰ related to mobile devices. The palette of patents asserted against Samsung is a clear and convincing example of the concept that “the whole is greater than the sum of its parts.” Based on a detailed analysis of these patents and their relatedness, it is clear that when combined the Apple patents create, as closely as possible given the complexity of the litigation, the “Total User Experience (TUX)“¹¹ of the iPhone and related Apple iOS devices.

Protecting More Than Patents Claim

The asserted Apple patents covered the look of the iPhone, the style of the screen-based interface, design of the icons and related screens, and some of the essential gesture-based interaction behaviors of the iPhone product. The sum of these patents covered essentially the iPhone you may or may not have in your hand from the standpoint of what you cognitively experience when you use the device. In retrospect this was a brilliant strategy because it gave Apple the opportunity to secure (conceptually) protections well beyond those that accrue to any single patent related to iPhone technology.

This was likely an essential legal strategy of Apple since it is the Total User Experience (TUX) of the iPhone that drives commercial success, not any individual feature or function as presented in a single patent asserted. It is interesting to note that Samsung asserted against Apple a series of garden-variety utility patents, which the jury found Apple had not infringed. The Samsung patents did not appear to have any specific rationale for selection other than they might have been broadly applicable to cellphone infrastructure. In this regard Samsung’s legal team seemed clueless with respect to the larger strategy by Apple to fence in the TUX of the iPhone through careful selection of key patents that combined to create a far broader protective screen around the iPhone product offering.

Mind Math: 1+1=5

This approach of conceptualizing the iPhone as comprised of many highly engaging features which combine to create an experience which is greater that the sum of its parts fits exactly with advanced research¹² in UX design which posits the view that a company’s most valuable corporate asset in the future will be the total user experience of its products, not a subset of features covered by discrete, individual patents or related IP. The cognitive science behind this approach is well understood by those who develop such products but ignored entirely by the current legal IP system. The human mind does not approach the world as a series of disconnected features but assembles experiences from the world we inhabit to form a mental model that is robust and determines what we find engaging and meaningful.

⁴⁰

Smart companies going forward will work with their IP counsel to frame patent applications and related litigation toward protecting the total user experience of their products. But the road to an IP strategy like Apple’s that focuses on the total user experience may not be an easy one. The current legal system works in exactly the opposite direction by requiring inventors to slice up their products into many features that can be described in patent claims or in other forms of narrowly defined legal descriptions. Trade dress¹³ protection is a notable exception but is understood to be, in its current state, both unwieldy and difficult to assert.

The Fundamental Problem with the Current IP System

Some of have said this “divide and define” legal framework is a fundamental conceptual and structural problem with the current patent system and its ability to protect these new massively interconnected, screen-based, software-driven, globally distributed, high-technology products and services. Furthermore, patent litigation currently requires attorneys, working with the aid of economic experts, to place a dollar value on features that have been infringed. One can see immediately the friction between the value of individual features and the value of the combined total user experience where a given feature or features may reside. This is not a new concern and has recently become a source of considerable debate and rancor within the legal system and the larger software development community.

⁴¹

Dummy Economics?

In the future, it may be conceptually impossible, from a business perspective, to decouple individual features from a highly successful product, like the iPhone, in a manner that allows the courts to assign a meaningful dollar value to infringement of piecemeal features. This is a heated topic of debate in the legal community that has surprisingly brought out opinions ranging from lawyers to federal court judges. Nowhere has this issue been made more obvious than in the recent Federal Circuit rulings and writings of Judge Richard Posner¹⁴ in the case of Apple vs. Motorola¹⁵. In his recent rulings in the Apple v. Motorola case, Judge Posner was so upset by the testimony of the economic experts of both sides when establishing the value of features found infringing that he called the experts in formal legal opinions “Dummy” and “Dummkopf”¹⁶.

Judge Posner went on to write a surprisingly candid review in The Atlantic¹⁷ of the current US Patent system that has been characterized alternatively as a “rant”¹⁸ and the opinions of “an influential loose canon.”¹⁹

Judging Judge Posner

When viewed in the context of the new forms of technology that allow companies to create highly engaging and successful total user experiences like those found in the iPhone, Judge Posner had it exactly right. In the context of business impact Judge Posner’s frustration with piece-meal valuations of asserted and infringing features is both deeply insightful and functionally correct. Assigning an asset value to individual features of an overall user experience like that of the iPhone makes little practical sense in the context of how such features actually contribute to the overall user experience of the product. All truly successful products are carefully conceived combinations of features, which, as we have said, can create engagement at a level that makes customers the world over stand in line to purchase an iPhone.

The difficult problem is determining how one assigns the proper value to a given feature or set of features in a manner that is objectively related to the commercial success of a product like the iPhone.

The marketing sciences have been dealing with this question for over a decade as marketing experts are constantly testing the marketplace to identify clusters of features that will produce enhanced purchase decision behaviors. The current patent litigation system has no such sophistication either conceptually or technically, hence Judge Posner’s frustration. As previously noted, in the Apple v. Samsung case, Apple focused on protecting the total user experience by asserting a cluster of patents that combine to drive high levels of user engagement. This raises the profoundly interesting question of exactly how many features and functions protected by patents does one have to remove before the iPhone user experience is also removed. This is probably a conversation Samsung bet the farm on but somehow failed to understand completely. The finding of infringement based on this palette of IP protections is a massive win for Apple both tactically and strategically. In fact, in terms of long-term business impact, this aspect of the win is vastly more important than a billion dollars in cash. It is also a strategy that others can learn from in a big way.

The Long Ball

Ultimately what Apple has to be reaching for in the future is a way to protect the iPhone TUX under the legal concept of a comprehensive trade dress²⁰ judgment. Under that legal framework, Apple will have virtually no time limit on protection and will be able to objectively tie the iPhone TUX to core consumer purchase decision behaviors and product recognition. It is important to note that some measure of trade dress protection was achieved in this litigation but when taken in the larger context Apple can objectively claim a much larger field of user engagement or customer touch points when defining its overall user experience. Such trade dress claims can easily be conceptualized as including all primary and secondary consumer touch points such as retail, Genius Bar, web site, etc., all integrated into a truly Total TUX.

While there do not appear to have been patents claiming the total user experience design as patentable subject matter it is only a matter of time before companies with highly engaging and successful TUX solutions file for such patents. In a critical way this is the future of product and service design protective strategies as it allows companies to protect the combined “whole” of their user experience solutions across all relevant customer touch points.

Judge Koh’s Strategy and Opinions on UX Design and Infringement

Judge Koh²¹ was, by most measures, an outspoken jurist with her own clearly stated opinions accompanied by a well defined strategy for keeping the case on track and both sides within bounds. This was no simple task. Her formal statements from the bench resulted in surprisingly strong opinions on the merits of the case along with decisions that allowed for a case of such profound impact. Specifically, Judge Koh allowed a wide range of legal IP protection forms to be litigated in the same case at the same time. She allowed Apple to assert infringement based on several major forms of IP protection (utility patents, design patents and trade dress)²² in the same trial, a fact that some have said was a dramatic benefit to Apple in implementing its ultimate legal objective of producing a “walled garden” of IP protection which combined design patents, utility patents, and trade dress.

By allowing consideration of such a wide palette of IP protection in this single matter Judge Koh may have established a critical precedent that will impact both legal strategy and product development. The legal teams and product development staff of companies large and small involved in UX design and software development should clearly benefit in a big way by making use of the widest range of IP protections possible and UX design should be conceptualized as an effort which optimizes against the total UX experience, not just individual features and functions and related infrastructure.

Judge Koh’s Opinions and the Design Experts

It is important to note that much of the coverage of the case in the mass media focused on the look of the iPhone or its “overall shape, style and appearance” as compared to that of the Samsung products. Even though Apple successfully executed a broader strategy of protecting the overall UX design of the iPhone, the design patent aspects of the case did generate by far the largest portion of the total financial award of 1.05 billion dollars²³. The design patent aspects of the case were of special interest to Judge Koh as she expressed her opinion related to the similarity of the Apple and Samsung products during various pre-trial hearings. To say that she believed the products to be similar is an understatement. In one exchange with the two legal teams present she essentially held up the two products and challenged the Samsung team to tell the difference.

This exchange²⁴ apparently left the Samsung lawyers speechless if not totally flummoxed. The surprising aspect of this exchange and the judge’s expression of likeness between the competing products was that Samsung lawyers gave the Judge a complete free pass on her opinion and never offered up expert analysis or testimony for the judge or jury on the cognitive science of shape perception. It turns out that such science reveals surprising insights on the question of shape equivalence. There was a great deal of expertise and science available to Samsung related to shape perception that could have helped educate the judge and jury on the realities of the ordinary observer test²⁵. Surprisingly, throughout the entire trial, Samsung did not use expert testimony to educate the judge and jury on the science of shape perception.

Certainly such lack of insight and expertise on the part of Samsung’s legal team gave Apple a free ride on the design patent infringement. Would such testimony have altered the findings of the jury? One will never know, but clearly Samsung could have provided the judge and jury with a far more reasoned and research-based understanding for reaching the finding of infringement on the design patent issues.

Why Protecting the Total User Experience (TUX) is Critical for Apple

While the media frenzy surrounding this case created surprising awareness and discussion ranging from skits on Saturday Night Live²⁶ to Conan O’Brien²⁷, the fact is that this case was not only about the current products in question but more specifically about future products to be created by Apple, Samsung and many other corporations that will populate our world in the next 10-15 years. In this regard the case is more about what is not on the table than what is.

Looking Beyond This Case to the Larger Technology Trends

There are a number of important trends in technology that are critically important when determining the impact that this case will have on product and software development over the longer term. Of the several important trends, the “connectedness” of devices is the most compelling in the context of this case. This trend estimates the number of products across eight major product categories that will be connected by some form of interactive network over the next 10-15 years. Research from numerous sources suggests that the so-called “Internet of Things” is going to take place on a massive scale, unlike anything we can imagine today. When one examines this data, for example shown in the chart below, it can be seen that the largest number of devices that will likely be networked are not computer clients but consumer products, followed by pallets and cases, appliances, machinery, vehicles, and hand held devices²⁸.

The Numbers Game of UX Design in the Future

While the impact of this trend is beyond the scope and purpose of this discussion, it is clear that companies like Apple, Samsung and thousands of other entities large and small will soon create products under their own brands which will be connected to a common universal network. Each of these products, devices or even entire processes will require a user experience that informs the customer of that specific product’s status and features.

What Matters Most
In this new world of connectedness between all manner of devices, one key factor, more than any other, will drive success, and that is the delivery of a common, highly engaging and easy-to-learn user experience that resides cognitively in the same form across many customer touch-points. The future goes to those who have the greatest control over the most productive user experience design running on the greatest number of devices.

This is a fundamental structural concept for a world of connected things. When companies have a compelling and easy-to-learn user experience running across many devices they provide the customer/user with one huge benefit: positive transfer-of-learning from device to device. This dramatically reduces the cognitive workload of customers and reduces friction at the point of sale. One can see this principle in action when walking into an Apple retail store where the iPhone behaves much like the iPad which in turn behaves like the iPod…and so on.

Samsung’s Thousands of Products vs. Apple’s Twenty

Samsung and other large corporations are peering into a future of connectedness without a TUX interaction model that will allow them to control the mind, wallet and learning profile of the consumer across many, many product categories. In fact, the chart above, which shows the number of products to be connected by product category, reads like the global product catalog of Samsung.

The sheer number of devices, products and entire systems that Samsung produces and the fact that many will be connected in coming iterations places far more demand on Samsung’s management than Apple’s in terms of creating a unified UX design framework that will map across not a dozen or two dozen devices like Apple but across thousands of SKUs ranging from smartphones to vacuum cleaners²⁹. It is therefore somewhat of a mystery as to why Samsung played this case so badly on several levels given the strategic impact of the verdict.

Does Anyone Have All the Pieces?

This specific matter of Apple v. Samsung is only one of hundreds of cases being litigated in the smartphone/tablet space at this time. The chart below shows some of the recent activity in terms of litigation, patent purchases, and licensing agreements taking place between parties in this technology space.

The most interesting point is that there are many other parties including non-practicing entities (NPEs), private inventors and other corporations that may hold key patents related to smartphone product design, UI design and infrastructure. Neither Apple, Samsung nor any other entity has the ultimate treasure trove of patents necessary to control this space.

Where Is the Prior Art?

Many of the patents that may be relevant in terms of prior art³¹ references flow from the mid to late 1990s. Searching this older body of prior art is not only time consuming, complex and costly but it leaves even the best, most recent intellectual property protection vulnerable to questions of both obviousness and invalidity. In fact, prior art accumulated by Samsung in this case, which was not allowed due to an error on the part of Samsung’s counsel³², is a case in point.

It is interesting to think about how that prior art found by Samsung might have or have not influenced the outcome of the trial had it been properly dealt with and allowed into evidence. In the end what this means is a landscape of scorched earth litigation that is not only complex, time-consuming and costly but highly unpredictable given that there are many patents and other forms of viable prior art still valid and relevant but that have not surfaced in the process of this litigation or other matters of strategic importance. This is a major problem for legal and development teams striving to create and protect innovative user experience design solutions like those at the center of this litigation.

Who Owns What?

The question of who owns the necessary IP to control the smartphone market is a point of considerable interest to lawyers and corporate executives, not to mention UX design and software development teams. The technical term for the underlying technology required to produce a successful (or even unsuccessful) smartphone system in known as “the stack”. The stack is divided into 4 primary layers, which together create technology to do what smartphones do. In his broadly read and widely quoted article in ArsTechnica titled Owning the stack: The legal war to control the smartphone platform³³, Law Professor James Grimmelmann divides the stack into essentially 4 layers: the UI/application layer, operating system layer, the device layer and the cellular network layer.

He goes on to describe how the current wave of litigation is focused on two factors: 1) owning or controlling the entire stack (not possible, but every player’s fantasy) and 2) owning a critical portion of the stack that other layers touch or pass through, so that a patentee can hold others participating in the smartphone market hostage by way of licensing fees.

Up and Down the Stack

Professor Grimmelmann goes on to say essentially that the current litigation is bi-directional (within a layer or between layers). For example, Oracle vs. Google in operating systems as in this litigation where Apple is trying to control two critical layers of the stack: the UI/Application and the Product layers. Currently no single company has anything like control of the entire stack, although Apple has the greatest proprietary control of any smartphone player through control of three levels of the stack: the UI/Application layer (iOS interface design), Operating System layer (iOS) and Device layer (iPhone). The only factor keeping Apple from owning a complete stack is control of the cellular network layer held by the likes of Verizon, AT&T and other telecommunications companies. However, were Apple to use its massive cash balance (70+ billion dollars at last count) to purchase one of these cellular networks, Apple would then have the only complete smartphone stack.

When Cognitive Science Trumps Computer Science

Ultimately the lower layers of the stack will become commoditized and less valuable than what the customer ultimately interacts with: the overall user experience design (UX + Device). It is the combination at the center of this case that makes possible actual user engagement with the new, massively connected world around us. The most advanced thinking related to UX design suggests that operating systems are in most ways irrelevant when considering whether or not a given UX design is infringed.

It is a technical reality that most modern operating systems (iOS, Android, Java, etc. have a robust GUI layer that can be employed in the implementation of advanced UX design solutions. Against this conceptual backdrop the measure of infringement is a cognitive science problem, not a computer science problem.

In this regard there is going to be a shift in the value of IP toward UX design and device design and away from deep-level smartphone infrastructure. Ultimately, several billion users couldn’t care less about anything going on behind the glass. All they care about, and what they employ in making purchase decisions, is what happens at the glass, hence at the UX Design and Product Design layers. In this regard this case may be a precursor to a new process for determining core asset value from IP going forward.

The Current Business Realities of the Smartphone Industry

While the legal issues of this case are interesting, the more important question for those who create technological innovations and seek protection is: what is the business rationale for this litigation? One might assume that the case is simply about two corporations who have a strong dislike for each other. That is not the real story. Apple and Samsung share massive production commitments outside of this litigation. A look into the underlying business environment of smartphones shows instantly that this case is about two companies that together totally dominate the smartphone market and more specifically capture between them a staggering level of profitability from this product category. This is shown in stark relief by the chart below, which shows that Apple and Samsung combined capture over 95% of the profit from the sale of smartphones.

In this competitive landscape Apple’s market share combined with Samsung’s market share makes up essentially the entire market. All other manufactures combined realize a negative percentage of the profitability of this market, when taking into account the current negative operating profit value of once prolific companies like Motorola, RIM, and Nokia. One can see the staggering impact that failing to create a compelling user experience can have on a previous world leader in smartphones by tracing the path of Nokia in the chart above as they went from essentially owning the profitability of the smartphone market to almost zero. There is one key player not at the table in this litigation but one whose products go to the heart of the case. That is, of course, Google³⁴.

The Massive Migration to Mobile

Over the past two years there has been considerable discussion about the trend toward consumers engaging on an increasing level with their mobile devices for tasks previously undertaken on desktop devices. This migration to mobile for more complex tasks by the consumer has been driven to some extent by the success of the iPhone User Experience and other devices of similar inherent ease of use. However, this trend known as “Mobile-Migration” has not been well documented in objective terms that could be mapped to reliable business implications. However, all that changed when Google recently released an important study titled The New Multi-screen World³⁵.

The Future Is Not What It Used To Be

In that study one can discern both the structure and extent of mobile-migration and the importance of protecting and optimizing the user experience for mobile devices like that of the iPhone. Based on the Google study one can see in stark relief the staggering shift in user behavior toward engaging with smartphones first as their primary entry point for a wide range of tasks that have critical business impact. Nowhere in the study data is this behavioral change more apparent than in basic search, where now 65% of all tasks involving “Searching for Info” start on the smartphone. As can be seen from the summary chart below, other critical tasks are also being started on mobile smartphones and then continued on the computer and to a much smaller percentage on a tablet.

This data suggests (although we have not seen the raw data) that those who own a smartphone have made a fundamental transition to the mobile user experience for critical tasks. This removes, for the most part, the question of whether or not consumers will adopt mobile user experiences for tasks that are known to fundamentally impact business performance for companies that rely on connected user behaviors. Therefore, when taken from a strategic point of view one can see that the design and protection of a robust mobile user experience like the iPhone’s may actually determine the success or failure of a given business entity, even one as large as Apple.

The Perfect Storm

In this regard the case happens to have surfaced at exactly the same time that mobile user experience design has transitioned to a critically important interaction framework for users of such technology. One can also glean from this data the apparent connection between mobile and desktop user experience design and the related human performance requirement to minimize negative transfer between these two hardware platforms. This awareness is likely at the heart of recent developments by Apple and Microsoft related to common interaction frameworks for the PC desktop/laptop, tablet and smartphone.

With the now documented “mobile-migration” it is clear that both UX design professionals and IP attorneys have objective proof from the marketplace that the future belongs to those who make mobile-migration their first and most important priority in terms of user experience design and IP protection. Will this insight lead to more or less litigation?

Controlling the Trial Narrative in the Context of Brand Messaging:

At the highest level the case has been framed in the media based on the narrative of “The Good” (Apple: market leader and innovator) versus “The Bad” (Samsung: market follower and copier). The creation and propagation of this narrative was a primary and well executed legal strategy of the Apple litigation team³⁷. This simple narrative abounds in Apple trial material including filings, trial testimony, visual aids and summation arguments.

An Old Idea Well Executed

There is nothing especially new or insightful about this approach since creating such a narrative is a basic litigation technique. However what is unusual in this case is the extent to which Apple was able to structure and control this narrative in the courtroom in a manner consistent with Apple’s brand strategy. The surprising fact is that this ability to control the highest-level narrative of the case was simply a reflection of how Apple has aggressively, and one can say brilliantly, controlled its brand narrative in the actual consumer marketplace. This clarity related to brand positioning in the marketplace, some have said, is responsible for premium pricing of Apple products.

However, in this litigation, it is clear that Apple’s control of the trial narrative was simply a extension of its branding strategy which has focused on delivering attributes of innovation and leadership to the market place through its brand delivery channels and the actual design of its overall user experience.

When Branding Meets the Legal System

Mapping this same brand positioning narrative into such a complex litigation is no simple matter and was likely a key factor in Apple’s success as the jury received a trial narrative consistent with Apple’s advertising and brand positioning strategy in the actual marketplace. This is not a trivial matter as jury members, no matter how media averse, had easily encountered the Apple brand messaging thousands of times prior to the case in various passive forms through the mainstream media including TV ads, billboards in the Valley, magazine advertisements, Apple retail store experiences or mall/store walk-bys. It is safe to assume that the jury had ample exposure to the global brand messaging of Apple for years prior to the case. It is well known from brand research studies that Apple has one of the top 5 simplest brands in the world, and that this simplicity is a desirable brand trait for which consumers are willing to pay a premium²⁸. The psychology behind delivery of a consistent brand image is well known and when extended to trial-based communications reduces the jury’s likelihood of confusion during deliberation by removing a question of conflict between market brand positioning and the way a party is presented in litigation.

Giving Up the High Ground

Conversely, Samsung’s legal team³⁹ failed to either alter the trial narrative or call into question Apple’s positioning of the “good” versus “bad” structure. Samsung had ample opportunity to provide an alternate narrative or to weaken Apple’s high-level structural messaging but seemed unable or even unaware of the importance of this aspect of the case. It is safe to say that this case demonstrates that in order to prevail in the protection of critical user experience IP, brand positioning needs to extend to trial strategy at the highest level.

What this means to those who develop software and UX Design innovations is that ultimately your brand will be on trial along with your products. Therefore, having a clearly defined and well articulated brand strategy in the consumer or commercial marketplace is likely to be very helpful when defending against issues of infringement or when asserting your IP against possible infringers. To be clear this means that those developing UX Design software-based solutions should not overlook the importance of developing a simple, clear and meaningful brand strategy based on a simple messaging structure.

All of this assumes your legal team can carry the narrative to the jury as Apple did in this case and that you have a positive narrative that is supported by brand positioning that actually delivers attributes that consumers find aligned with the functionality of your total user experience. In other words the best brand strategy possible will not replace a dysfunctional user experience either in the marketplace or at trial.

The Persistent Question: Did the Case Damage Innovation?

In the final analysis we are left with the question that has consumed more blog bytes than any other issue related to this case: “will the verdict have a negative impact on innovation?” The answer is almost certainly no. There is one important reason.

From this case a new strategy for product development and IP protection has emerged. The change dictates a shift to innovation to total user experience design. This is a fundamental change that has now been shown to provide companies with a massive core asset in a world that is rapidly becoming an interconnected lattice of billions of devices where product design, usability and ease of learning are the most important factors in achieving business success.

Ease of Use Matters, the Operating System Does Not

As noted before, consumers couldn’t care less whether their device runs on iOS or Android or a host of other operating systems. In this new world, UX/product design innovation is a given and basic requirement of business success. This case teaches the value of user experience design as a new corporate asset. It does not in any way limit the available solutions to such problems. Samsung will not stand aside and Apple will not stop evolving in terms of user interface and product design innovations. This is a model for others to follow but not to duplicate. The case does not restrict innovation but more accurately demonstrates the objective value of such solutions. The current iPhone interface is by current measures a successful innovation but it is by no means the end game as entirely new technology and cognitive science are showing the way to solutions well beyond those at the center of this litigation.

What the Case Teaches

This case marks the beginning of a very different future for teams tasked with product and software development for this category of devices. Below is a partial list of key legal and development strategies flowing from this case.

It is important to note that this litigation surfaced in parallel with several important trends that together are setting in place a perfect storm of new technology and massive consumer demand for the next generation of mobile devices and related systems. In the end this case is simply a mileage marker on a new and exciting road to a world of massively connected mobile devices where the user experience is all that ultimately matters. Protecting such innovations, in light of the available protections, will require as much innovation as the design of the devices themselves.

8 Key Insights from the Apple v. Samsung Case

1. Develop products and services in this space that focus on the total user experience (TUX): The future of robust IP protection for products in this category is going be determined by how well your total user experience is protected. In this regard, focus innovations on ALL critical customer touch points covering hardware and software interactions.
2. Create clusters of IP protection that capture the total user experience (TUX): IP practitioners should make use of all available forms of IP protection including design patents, utility patents, copyright, trademark and in litigation, trade dress, to create an interlocking set of protections that cover the look, feel and function of your products and services.
3. Develop a robust interactive brand that supports your overall TUX strategy: Consider carefully protecting core brand positioning elements and messaging as part of the overall IP protection strategy. In the best user experience design solutions in the future your brand will be delivered by (and be indistinguishable from) your total user experience.
4. Practice tighter integration between legal and development teams: Waiting until an innovative new user experience is complete before developing a legal strategy for protection will no longer be productive given the large body of prior art, rapid release schedules and litigious nature of the marketplace.
5. Focus on “mobile-first” design strategy: The larger trends which are surfacing in parallel with this litigation strongly suggest that business-critical user interface and product design solutions should implement a “design-for-mobile-first” strategy that is then migrated across the other 3-4 screen types known to capture user behaviors.
6. Do not rely on specific software operating systems or platforms for user experience design IP innovations: This is more complex than it sounds since good IP strategy may well include utility patent filings. However, it is critical to think of TUX design innovations as platform agnostic.
7. To the extent possible secure trade dress rights through litigation: This is an issue for the legal team. If you are going to go for litigation consider doing so based on trade dress as this form of legal protection is likely the most robust option. Be aware of the complexity of such cases before proceeding.
8. Use design patents to secure visual design protection on UX design solutions: Consider the visual (non-functional) aspects of your solutions absolutely worth protecting and make sure that your IP counsel understands design patent filings and related methods. There are many ways to employ design patents beyond those adopted by Apple in this litigation.

Charles L Mauro CHFP
President / Founder
MauroNewMedia

Find out more about Charles L Mauro
Find out more about MauroNewMedia
Follow Pulse>UX on Twitter @PulseUX

Detailed Assistance by Emily Fisher MSE

We would like to thank the following colleagues for review of drafts of this post: Rob Tannen PhD, Chris Carani Esq, Rob Katz, James Grimmelmann PhD, Erik Grimmelmann PhD, Mark Moore Esq, Eric Hopkins PhD, Hilary Hinzmann, John Blake

Other posts on PulseUX you might find interesting

• Why Angry Birds is so successful and popular: a cognitive teardown of the user experience (over 1 million views)
• Why the music industry doesn’t have a prayer against iTunes (over 100,000 views)
• How new theories in human information processing explain the meltdown on Wall Street (over 50,000 views)
• What US Airways Flight 1549′s ditching in the Hudson River teaches companies about how to create world-class user interface design solutions (over 50,000 views)

To comment on this post, go to the comment link at the end of the references.

Everywhere we turn in the development of technology-based business models the world is focusing on data. Big Data, Structured Data, Unstructured Data, Fast data, Slow data, even Data about Data. This transformation to a research and data-driven decision-making process started with web analytics and has now migrated to the design and development of all manner of user experiences for high-technology products and services.

No Data

It is more common than most development executives wish to admit that large, complex, product design and web development projects are often undertaken with minimal professional user experience UX testing during development. This approach, while prevalent in the past, is changing rapidly as more advanced forms of user testing are finding a place in complex product design and web development efforts. The risk has simply become to great for pushing into production UX solutions without objective and professionally structured user testing much earlier in the development process.

The Real Data

Unfortunately there is a great deal of misinformation about how to objectively test and optimize a given user experience with a specific focus on improving business performance. More important is the need for development teams to understand the pallet of tools available for such testing in a way that goes beyond the usual sales demo provided by UX testing vendors and services. As an aid to clients and development teams PulseUX and Charles L Mauro CHFP has teamed with another UX testing expert, Rob Tannen Ph.D in the creation of the following tightly crafted review of the 12 most important testing methods for both product design and web/screen based user experiences. In the following video we summarize each of 12 relevant testing methodology and also identify the limitations and benefits in a way that adds depth to your understanding of UX testing processes. The video is structured according qualitative research methods as they apply to the design of physical products (Rob Tannen section) and quantitative methods and how those methods apply to web/screen-based user experiences (Charles Mauro CHFP section).

Timeline of Data Content

The following timeline chart indicates the general structure of the video content and is provided for your reference based on a given area of interest. However, if you have time, we suggest watching the entire video as there are many interesting insights provided in all sections of the video that go beyond testing.

Thank you
Charles L Mauro CHFP
Rob Tannen Ph.D

Our newest post is a recent in-depth live interview recording of Charles L Mauro covering his more than 30 years as a leading usability scientist. If you are wondering why some products are easy to use and others much less so, why Apple products are so successful, what does it take to create a world-class user experience you will find the interview eye opening if not highly thought provoking.

Listen to the interview below.

This interview was conducted by Robynn McCarthy, Co-host, Skepticality: The Official Podcast of Skeptic Magazine dedicated to critical thinking and science.

Download an MP3 of the interview here.
Download the podcast from iTunes here.

The unusual question: Surprisingly, it is a rare client indeed who asks the opposing question: why is an interface so engaging that users cannot stop interacting with it? This is a difficult question because it requires cognitive reverse engineering to determine what interaction attributes a successful interface embodies that result in a psychologically engaging user experience. This question pops up when products become massively successful based on their user experience design – think iPhone, iPad, Google Instant Search, Nintendo Wii, Microsoft Kinect.

The interesting question: Recently clients have asked about the phenomenally successful casual computer game Angry Birds, designed for mobile phones, tablets and other platforms. For those who don’t have a clue what Angry Birds is all about, here is a quick synopsis. The game involves employing a sling shot to propel small cannonball-shaped birds with really bad attitudes at rather fragile glass and timber houses populated by basically catatonic green pigs. The basic thrust of the game is to bring about the demise of the pigs as quickly and expertly as possible by collapsing the pigs’ houses on top of their (sometimes) helmeted heads. Obviously, this sounds like a truly dumb concept. However, there is a catch.

Why is it that over 50 million individuals have downloaded this simple game? Many paid a few dollars or more for the advanced version. More compelling is the fact that not only do huge numbers download this game, they play it with such focus that the total number of hours consumed by Angry Birds players world-wide is roughly 200 million minutes a DAY, which translates into 1.2 billion hours a year. To compare, all person-hours spent creating and updating Wikipedia totals about 100 million hours over the entire life span of Wikipedia (Neiman Journalism Lab). I say these Angry Birds are clearly up to something worth looking into. Why is this seemly simple game so massively compelling? Creating truly engaging software experiences is far more complex than one might assume, even in the simplest of computer games. Here is some of the cognitive science behind why Angry Birds is a truly winning user experience.

Simple yet engaging interaction concept: This seems an obvious point, but few realize that a simple interaction model need not be, and rarely is, procedurally simple. Simplification means once users have a relatively brief period of experience with the software, their mental model of how the interface behaves is well formed and fully embedded. This is known technically as schema formation. In truly great user interfaces, this critical bit of skill acquisition takes place during a specific use cycle known as the First User Experience or FUE. When users are able to construct a robust schema quickly, they routinely rate the user interface as “simple”. However, simple does not equal engaging. It is possible to create a user interface solution that is initially perceived by users as simple. However, the challenge is to create a desire by users to continue interaction with a system over time, what we call user “engagement”.

What makes a user interface engaging is adding more detail to the user’s mental model at just the right time. Angry Birds’ simple interaction model is easy to learn because it allows the user to quickly develop a mental model of the game’s interaction methodology, core strategy and scoring processes. It is engaging, in fact addictive, due to the carefully scripted expansion of the user’s mental model of the strategy component and incremental increases in problem/solution methodology. These little birds are packed with clever behaviors that expand the user’s mental model at just the point when game-level complexity is increased. The process of creating simple, engaging interaction models turns out to be exceedingly complex. Most groups developing software today think expansion of the user’s mental model is for the birds. Not necessarily so.

Cleverly managed response time: A universal law of user interface design is “the faster the response time, the better”. True enough, there are applications where this is patently true. For example, Google has made this a mantra for their systems. However, surprisingly few software developers realize that response time management is actually a resource that can be leveraged to add to the quality and depth of engagement of a user interface. The surprising point that is often misunderstood is that not every aspect of the user interface needs to be or should be as fast as possible. Programmers uniformly have a really hard time with this one and few game designers take advantage of this potent variable. In most commercial software interfaces, response time management is completely overlooked even by those who claim to be UI design experts. The developers of Angry Birds managed response time in a way that goes far beyond simply “faster is better”.

For example, in Angry Birds, it was possible for the programmers to have made the flight of the birds fast – very fast, but they didn’t. Instead they programmed the flight of the angry flock to be leisure pace as they arc across the sky heading for the pigs’ glass houses. This slowed response time, combined with a carefully crafted trajectory trace (the flight path of the bird), solves one huge problem for all user interfaces – error correction. The vast majority of software user interfaces have no consideration for how users can be taught by experience with the system to improve their performance. This problem is a vast and complex issue for screen-based trading systems where error correction is not only essential, but also career threatening.

In Angry Birds game play the pigs also take a long time to expire once their houses are sent to bits. In many play sequences, seconds are consumed as the pigs teeter, slide and roll off planks or are crushed under slow falling debris. This response time of  3-5 seconds, in most user interfaces, brings users to the point of exasperation, but not with Angry Birds. Again, really smart response time management gives the user time to relax and think about how lame they are compared to their 4 year old who is already at the 26th level. It also gives the user time to structure an error correction strategy (more arc, more speed, better strategy) to improve performance on the next shot. The bottom line on how Angry Birds manages response time: fast is good, clever is better.

Short-term memory management: It is a well-known fact of cognitive science that human short-term memory (SM), when compared to other attributes of our memory systems, is exceedingly limited. This fact has been the focus of thousands of studies over the last 50 years. Scientists have poked and prodded this aspect of human cognition to determine exactly how SM operates and what impacts SM effectiveness. As we go about our daily lives, short-term memory makes it possible for you to engage with all manner of technology and the environment in general. SM is a temporary memory that allows us to remember a very limited number of discrete items, behaviors, or patterns for a short period of time. SM makes it possible for you to operate without constant referral to long-term memory, a much more complex and time-consuming process. This is critical because SM is fast and easily configured, which allows one to adapt instantly to situations that might otherwise be fatal if one were required to access long-term memory. In computer-speak, human short-term memory is also highly volatile. This means it can be erased instantly, or more importantly, it can be overwritten by other information coming into the human perceptual system. Where things get interesting is the point where poor user interface design impacts the demand placed on SM. For example, a user interface design solution that requires the user to view information on one screen, store it in short-term memory, and then reenter that same information in a data field on another screen seems like a trivial task. Research shows that it is difficult to do accurately, especially if some other form of stimulus flows between the memorization of the data from the first screen and before the user enters the data in the second. This disruptive data flow can be in almost any form, but as a general rule, anything that is engaging, such as conversation, noise, motion, or worst of all, a combination of all three, is likely to totally erase SM. When you encounter this type of data flow before you complete transfer of data using short-term memory, chances are very good that when you go back to retrieve important information from short-term memory, it is gone!

One would logically assume that any aspect of user interface design that taxes short-term memory is a really bad idea. As was the case with response time, a more refined view leads to surprising insights into how one can use the degradation of short-term memory to actually improve game play engagement. Angry Birds is a surprisingly smart manager of the player’s short-term memory.

By simple manipulation of the user interface, Angry Birds designers created significant short-term memory loss, which in turn increases game play complexity but in a way that is not perceived by the player as negative and adds to the addictive nature of the game itself. The subtle, yet powerful concept employed in Angry Birds is to bend short-term memory but not to actually break it. If you do break SM, make sure you give the user a very simple, fast way to accurately reload. There are many examples in the Angry Birds game model of this principle in action. Probably one of the most compelling is the simple screen flow manipulation at the beginning of each new play sequence. When the screen first loads, the user is shown a very quick view of the structure that is protecting the pigs. Just as quickly, the structure is moved off screen to the right in a simple sliding motion.

Coming into view on the left is a bevy of bouncing, chatting and flipping birds sitting behind the slingshot. These little characters are engaging in a way that for the most part erases the player’s memory of the structure design, which is critical to determining a strategy for demolishing the pig’s house. Predictably, the user scrolls the interface back to the right to get another look at the structure. The game allows the user to reload short-term memory easily and quickly. Watch almost anyone play Angry Birds and you see this behavior repeated time and again. One of the main benefits of playing Angry Birds on the iPad is the ability to pinch down the window size so you can keep the entire game space (birds & pigs in houses) in full view all the time. Keeping all aspects of the game’s interface in full view prevents short-term memory loss and improves the rate at which you acquire skills necessary to move up to a higher game level. Side note: If you want the ultimate Angry Birds experience use a POGO pen on the iPad with the display pinched down to view the entire game space. This gives you finer control, better targeting and rapidly changing game play. The net impact in cognitive terms is a vastly superior skill acquisition profile. However, you will also find that the game is less interesting to play over extended periods. Why does this happen?

Mystery: You probably do not know how to recognize it, but Angry Birds has it. To add context to this idea, mystery is all around us in the things we find truly compelling. The element or attribute of mystery is present in all great art, advertising, movies, products, and not surprisingly, interactive games. The idea of mystery in a user experience as an attribute for increasing user engagement is embedded in the idea of mystery (conceptual depth). We all experience the impact of mystery when we view a cubist period Picasso, recall the famous Apple 1984 super bowl ad, or listen to Miles Davis.  He is said to have described jazz as playing the spaces between the notes, not the notes themselves. Mystery is present when you pick up an iPad for the first time. Why are the icons spaced out across the screen when they could be clustered much closer together to save space. Why does the default screen saver look like water on the inside of the screen?

Mystery is that second layer of attributes that are present but undefined explicitly, yet somehow created with just enough context to consume mental resources in subtle and compelling ways. At its most basic level, experiencing mystery in what we interact with makes you ask the question, “Why did they do that?”.  What we mean here is, “Why did they do that? – A good thing, not “What were they thinking? – A bad thing.  If you think carefully about the experiences you have in the ebb and flow of life, you realize that the most compelling are those that force you to think long and hard about why a given thing is the way it is. For example, why did Frank Gehry create the Guggenheim Museum Bilboa using the shapes he did? The famous architect could have created any shape concept, but why did he choose those shapes? It’s a mystery – we do not know and probably neither does he. What we do know is that his creation is cited as one of the most important works of contemporary architecture. In the same way that a building can captivate millions of sightseers, the element of mystery (conceptual depth) can help sell a few million copies of a simple interactive game.

Angry Birds is full of these little mysteries. For example, why are tiny bananas suddenly strewn about in some play sequences and not in others? Why do the houses containing pigs shake ever so slightly at the beginning of each game play sequence? Why is the game’s play space showing a cross section of underground rocks and dirt? Why do the birds somersault into the sling shot sometimes and not others? One can spend a lot of time on the Acela processing these little clues, consciously or subconsciously. When users of technology process information in this way, it is very likely that they are more deeply engaged than without these small questions.

How things sound: Over the past 15 years, the neuroscience of music has taken a huge leap forward. This new research is just beginning to tell us why music adds such a strong emotional component to movies, advertising, theater, and of course, new media of all types, including casual computer games. Employing the power of audio stimuli including structured music often adds a critical level of engagement for users of all forms of technology. Angry Birds’ audio effects and music seem simple but are, in fact, very complex. The use of audio effects and carefully varied melodic music lines works to enhance the game play engagement level. Many games do this but few do it expertly. The audio in Angry Birds serves to enhance the user’s experience by mapping tightly to the user’s simple mental model of conflict between the angry birds and the loathsome pigs. This concept, known in film production as “action syncing”, provides enhanced levels of the feedback for users at just the right time. For example, in Angry Birds, we hear the birds chatter angry encouragement to their colleagues as each prepares for launch. We hear avian dialogue as the birds arc toward their targets and hear the pained response from their victims when they strike their targets. The pigs are by no means silent. When the avian interlopers fail, they are often egged on to try just one more time by the snickering, grinning pigs. These consistently applied audio elements reinforce the player’s interactions and deepen engagement by emphasizing the anthropomorphic qualities of the main characters of the game and providing clever enhanced feedback during critical on-screen behaviors. What about the actual melodic music shifting from the foreground to the background without apparent reason? This musical thread running through the game play experience is mysteriously familiar and easily understood in the context of the overall theme of the game. Where have I heard that melody before? This combination of audio feedback is varied just enough that parents sitting in the next room are rarely prone to demanding an end to game play based on distracting musical repetition. Perhaps this explains the high number of hours spent playing the game!

How things look: Angry Birds has a look. One might characterize the visual style of Angry Birds as a combination of “high-camp cartoon” with a bit of greeting card graphics tossed in for good measure.

This leads to a more interesting question: How does visual design impact success in the marketplace? I routinely get this question from clients who are undertaking large redesign or new development projects. Decades after it first surfaced in automobile design, visual design is still the most contentious aspect of designing compelling user experiences. Designers (mostly of the UX stripe) routinely sell clients on the concept that the visual design (graphic style) of a given interface solution is a critical factor in success. This assumption seems to make good intuitive sense. However, the actual working principle is counter-intuitive. In most user experience design solutions, visual design (how things look) is technically a hygiene factor. You get serious negative points if it is missing, but minimal positive lift beyond first impression, if a user interface has great visual design. When we conduct user engagement studies for clients (not the same as usability testing), we routinely see data that strongly supports this theory. This concept does not apply to all user experience design problems, but in most cases it holds well. The ultimate question is how much visual design is enough?  Even more important than good or bad visual design is appropriate visual design. On this metric, Angry Birds again has just the right set of attributes. The concept of appropriate visual design is in itself complex as designers generally apply too much rendering and engineers apply none, which often leaves the actual user staring at the equivalent of an engineering prototype (Google) or alternatively, World of Warcraft. After decades of experience in user interface design, I can predict fairly accurately the corporate software development bias of clients by simply examining the user interfaces of their products. I cannot imagine Google as anything but engineering-driven, despite the apparently large number of UX designers hired in recent years.

Measuring that which some say cannot be measured: How does one measure visual design in this context? There are several well-understood methodologies for assessing the appropriateness of visual design that we employ in development projects. These research methods make objective that which is thought to be only subjective. Visual design can be measured, rated, and scaled to the benefit of users and those who develop such interfaces. The actual dimensions of appropriate and winning visual design vary widely, depending on the application but in game design two factors reign supreme. First, the visual design must be memorable and second, it must convey the desired attributes of the game play model.

So memorable is Angry Birds that the developers have deals for real world “brand extensions”, including Angry Birds stuffed toys, t-shirts, and all matter of off-the-wall consumer goods that make BIG profits. The simple visual design of those tiny cartoon-ish birds is so compelling and simple, it brings an additional level of continuous interest to the game play experience. Of note too is the world the birds and pigs inhabit which changes in strange and subtle ways with every level. Visual design is another critical dimension of the success of Angry Birds, which leads to the ultimate question: Is Angry Birds the best it can be? Not by a long shot!

We are left with the notion that a cognitive teardown of a truly compelling user experience is vastly more interesting and insightful than simply answering the opposite question: why is a given user interface dysfunctional? To summarize, in the context of Angry Birds, success is bound up in slowing down that which could be fast, erasing that which is easily renewable, and making visual that which is mysterious and memorable. Over the past 10 years, our firm has conducted user engagement studies on hundreds of user interfaces. The vast number did not get one principle right, much less six.  You go Birds! Your success certainly makes others Angry and envious.

1. Lack of aircrew situation awareness (SA): In the exceedingly well executed study conducted by Basic Usability Testing Global Unlimited International Corporation (BUT-GUI), it was discovered that use of the new Google Maps interface was resulting in high levels of on-board stress on the part of the air crew. This was due primarily to a near complete lack of global situation awareness (SA) caused by the need to repeatedly enter local delivery addresses into the on-board Google point-to-point map system. The problem was apparently made worse by the large number of local addresses that required constant updating during in-flight operations. The Google Maps address storage functions apparently tanked at 5000 addresses when the real-time universe was expected to be about 500 million over the actual 2 hour trip period.

2. High cognitive workload for the air-crew: It was also learned in the final (pre-trip) usability study that the aircrew’s cognitive workload reached exceedingly high levels due to the need to process delivery route location points while simultaneously sorting packages for delivery to each location in real-time. When the air-crew simulated navigation and spot-to-spot delivery of packages it was found that cognitive workload exceeded acceptable levels and high error rates (wrong packages to the wrong houses) ensued. Apparently, not a winning situation.

3. Lost way-finding based on constant scrolling and zooming of Google map display: In a second section of the final test of the new Google Maps navigation system, it was discovered that constantly scrolling and zooming the map display on the primary navigation heads-up interface caused the primary navigation officer (designated in the study as EL2) to completely lose orientation because he was required to both scroll and zoom the Google display at the same time resulting in disorientation and further loss of situation awareness.

4. Small sample size leads to unreliable data: The CTO further commented that the final “pre-flight” research study may have been totally invalid because of the very small sample size involved in the study. In fact it seems as if only one crew-chief  (designated as SC) and 3 crew members (EL 1, 2, and 3) constituted the entire sample for the study, thus leading one to conclude that this may be an unrepresentative sample of users. Surprisingly the crew chief (SC), which apparently has made over 500 similar, successful sorties, seemed very upset by this section of the final report.

5. The final decision: At the time of this report it is believed that the final decision to either adopt the new navigation system or return to the prior, human-based, dead reckoning technology (or lack there of) was in the hands of the crew chief who reportedly was leaning toward dumping the whole idea. He said in an interview last night on The Next Big Thing, in an apparent tip of the hat to Bob Lutz: “This whole idea of improving on my abilities by using this new web 2.0 mapping stuff seems like a “Crock”. Finally, according to unconfirmed sources the final decision will apparently be made by the crew chief at 11:59 December 24th…!

The real deal: If you want to watch the flight crew’s actual progress on Christmas Eve go to the NORAD (North American Aerospace Defense Command) site, http://www.noradsanta.org/. There you will find a bevy of cool tracking stuff including Santa trackers for your cell phone, desktop and even a 3D Santa tracker using…Yes…you guessed it….Google Earth.

Charles L. Mauro

What stimulates creation of simple user experiences is not what it used to be

Let’s be clear, this push toward operational simplicity is not motivated by IT’s desire to do the right thing but is a result of a more recent economic imperative that says simply that customers want products and services that can be acquired, set up, used, and maintained by the lowest level of technical expertise possible while sustaining suitable reliability. For example if 2 manufacturers make the same category of routers and one can be serviced by technical school grads and a 3 page user manual and the other by employees with an MS in computer science and a massive call center in India, guess whose stock price is eventually dropping like a stone?

How “expertise consumption” is becoming the core of expense management

Expertise, in all its various forms, is the new currency of corporate profitability. In a world where IT is the CENTERPIECE of business processes “expertise = expense” and nothing eats up expertise like IT hardware and software.  So, for all the years that usability and user experience design has been proffered as a soft benefit that demonstrated customer awareness, it is now clear that usability and user experience design are actually key factors in expense management. With the new focus being on management by metrics, the actual cost of expertise is now finally being measured objectively. For example: one router (Company A) costs $2,000 but consumes $10,000 in set up and maintenance costs. A second router (Company B) sells for $3,000 and costs $300 dollars to set up and maintain. It only takes one quarterly board presentation on IT expertise costs to understand that Company A is no longer the router of choice. So, whereas usability was previously thought to be no more than a “nice” attribute, not having serious usability performance is now a “nasty” liability.

The interesting question is how do corporations which have previously been disdainful of formal usability science and user experience design actually make the transition to a development culture that values above all the cost (cognitively and technically) demanded of customers using its products? Give this question to an investment banking group and the answer will be to acquire simplicity in the form of a company that has such a development culture. The idea here is that simplicity and the ability to create products which actually are less complex to learn and use can be bought and/or transferred between corporate cultures. Sorry, it does not really work that way because developing a culture that creates simplicity as a primary corporate strategy is not the same as creating one or two simple products. One is massively complex; the other is trivially simple.

Why simplicity is fundamentally an innovation diffusion problem

The key to moving from a development culture that is technology-centered to one that is actually user-centered depends on the corporation’s willingness to support innovation as a core concept. Based on research conducted in the writing of a chapter for the book titled: “Cost-Justifying Usability: An Update for the Internet Age”, we described the factors that must be present for formal user-centered design to be successfully adopted into an existing corporate structure. Below are the key list of attributes from that chapter. Note: this list is adopted from the excellent work of Everett Rogers, Diffusion of Innovations, which deals with innovation dispersion within corporate cultures. We view the dispersion of usability science and user-centered design as fundamentally an innovation dispersion problem and not a factor that can be acquired through M&A.

Factors that impact dispersion of innovations and the creation of simplicity

1. Centralization of decision-making is negatively correlated with adoption of innovations such as usability science. If all decisions are made in a centralized manner, usability science is less likely to be effectively adopted within the organization.
2. Complexity of the organization’s staff background and educations can be positively correlated with adoption of innovations. If an organization has many highly qualified and professional individuals from varied disciplines, it is more likely to adopt innovations such as usability science. It is especially important that no one discipline dominate product development decision-making.
3. Formalization of rules and procedures is negatively correlated with adoption of innovations. The more structured the organization’s development rules and procedures are, the less likely the corporation will be to adopt an innovation.
4. Interconnectedness is highly correlated with adoption of innovations such as usability science. This means that the level of network utilization and interpersonal communications that take place between and across divisions and departments and even within individual teams of corporate employees and executives are strong predictors of willingness to adopt innovation.
5. Organizational slack is simply the amount of uncommitted resources available to use in the adoption of new and powerful ideas. It is important to remember that all corporations run on a yearly development budget and rarely is there a line item for “open innovation” resources, those resources which can be deployed easily and quickly to problems that often drive simplicity into product design solutions.

If one were to apply the attributes listed above to Cisco Systems (and many other leading IT companies) it is likely you will find that while these large IT entities may have the desire to sell products that demonstrate simplicity and are paying a lot in an attempt to do so, they have some serious obstacles to overcome before they will be able to successfully create truly simple and empowering user experiences on a corporate wide basis.

Why creating simplicity is more than flipping a switch

It is important to note that creation of a consumer product like the Flip Video Camera is relatively easy compared to reducing the complexity of a major piece of IT infrastructure. The difference between these 2 types of problems can be found in the depth of the user experience layer that must be modeled when creating a product or system that is truly easy to learn and use. The Flip Video Camera can be optimized in many ways by trial and error modeling of the user experience. However, creating higher levels of simplicity in truly complex, software-based interfaces requires the application of formal and highly structured cognitive modeling of the relationship between the user/customer and the deep feature sets of more complex products. This is exactly the point at which buying simplicity and creating it comes off the rails for companies like Cisco and others companies who produce products with deep feature sets.

The ability to successfully employ new design processes that may result in the development of products with simplicity attributes like that of the Flip Video Camera requires essentially altering the structure of the product design process itself. Traditional processes focus almost exclusively on the design teams’ vision of what the product should be with little acknowledgement of the underlying cognitive processes of the end user and how such processes affect acceptance of a product in the marketplace. In fact, one of the most difficult problems facing design teams is realistically accepting the fact that they cannot represent the cognitive state of the end user as they develop new products and services.

Why the simplicity game has changed

A development process that successfully creates simplicity in the face of increasing complexity shifts the perspective of the design process to the actual customer through the application of usability science. This shift allows the design team to develop a product that matches how the user thinks about and interacts with the product resulting in designs that are more engaging, innovative, and less complex to learn. But this is not exactly a ground breaking insight since the concept of “user-centered” development has been around for decades. So what is the important story here, what has changed?

If you are not measuring simplicity, you are not creating it

The difference today is that to apply professional usability science properly to increasingly complex products requires the establishment of formal user-based “simplicity metrics” which a new product must objectively meet in order to deliver a truly engaging user experience. It is this shift toward basing design decisions on hard user experience performance metrics that is a sea change for companies like Cisco and many others struggling with the “Expertise-Consumption” problem. Any company today (large or small) that wishes to develop game-changing simplicity in a market category and does not have a set of tightly structured simplicity metrics is simply kidding itself…simplicity is not going to happen. If you are not basing your design on initial cognitive modeling of the user experience supported by repeated user testing it is highly unlikely that simplicity will be a claim you can make in the marketplace.

Why development executives feel uncomfortable with simplicity as an objective

To clarify our point, what we are talking about is NOT the same as a corporation running a usability study when the product or system is in late Beta. Such an approach will never deliver insights that result in major reduction in learning and operational complexity. It is simply too late. If true simplicity is a goal it is necessary to test design concepts repeatedly against “simplicity metrics” during development. This means building much more robust simulations and constant user testing with unbiased respondent pools. As any experienced development manager can see immediately, the requirement of constant user testing of simulated user experiences virtually reverses their existing development model…not a comfortable position given the nature of the economic environment we face today. Some, like Cisco, may feel that it is easier to purchase simplicity than attempt to create it on a corporate wide level.

Why cognitive capital is the new measure of business success

In our experience, without rigorously (and repeatedly) evaluating the cognitive workload placed on the end user during product development it is nearly impossible to create solutions that result in empowering and game changing user experiences. The simple truth is that it’s not easy for most design teams to accept that the user’s perspective on what is actually simple and empowering is vastly more valuable than their own. Unfortunately for companies like Cisco and others, this shift in perspective can’t be guaranteed by acquiring a company whose culture may have created a terrific product like the Flip Video Camera and hoping that simplicity will occur by “osmosis” or for that matter even by plan. In the near future business success will turn on the cost of “cognitive capital” not “financial capital” and nothing consumes cognitive capital like IT product complexity. Just ask anyone who has tried to install a Cisco router recently.

Charles L. Mauro

Photo by Greg L. available under a Creative Commons Attribution Non-commercial license.

One might reasonably ask what does an A320 Airbus have to do with the design of products ranging from MP3 players to popular social networking websites? For example had MP3 player manufacturers applied even a small portion of the user interface design methods employed by the commercial aircraft companies iTunes would have been a non-starter and the music companies would be a happy bunch today. The more surprising point is that the best insights come not from how contemporary commercial aircraft fly, but how they crash.

No example in recent history is more helpful in this regard than the ditching of US Airways 1549 into the Hudson River on January 15th, 2009. Here is a condensed analysis of what this incident teaches all who strive to build compelling user interfaces for high technology products and services in the future. You will not find these insights combined in any currently available source on user interface design.

To set these leanings in context, below is a link to the audio recording of communications between US Airways Flight 1549 (using the call designation “Cactus 1549”), the New York TRACON flight controller, and the controller from Teterboro airport in New Jersey. The entire clip is about 3 min. in length. The tape picks up where Cactus 1549 reports a bird strike to the TRACON controller and continues until Cactus 1549 vanishes from his radar view. The brief conversations at the end of the tape are communications between the TRACON controller and several other planes in TRACON’s space a short time after Cactus 1549 vanished into the Hudson River. (Note: If you are really into this sort of thing, you will get the most from the audio if you listen to it 3 times; first for overall flow, second focusing on the pilot, and a final pass focusing on the TRACON controller.)

Listen to the cockpit conversation below:

While you listen to the audio follow the flight path on the diagram below. Note that the entire event string consumed only about 3 min on the time line. The bird strike takes place at about 3:27 on the chart below.

Photo by S.Bollman, available under a Creative Commons Attribution Non-commercial license.

User Interface design insights from Cactus 1549…why did everyone survive?

Insight 1: Design for automation, plan for user control
The push to automation in all forms of high technology development is inevitable. This thrust is wide-ranging from simple products like the iPod and extending to the A320 Airbus as it bobbed in the Hudson River. It is a fact that automation is at the center of all future user interface design problems and solutions. Some industries deal with this variable better than others. For example the history of the airline industry shows that no other commercial sector has undertaken such a massive and aggressive stance on the use of automation related to its technology. Over 2 decades ago the airlines, working in collaboration with the aircraft manufacturers launched an effort known essentially as the “Cockpit Automation Program”. That effort coincided with the development of a new generation of commercial aircraft. As with most major transitions in technology, the program was controversial from the start and produced a pitched battle between the Air Line Pilots Association (ALPA) and the airlines and manufacturers for control of cockpit flight systems (user interface) for planes of the future. On the one side were pilots rejecting nearly all additional automation and on the opposing side were airlines and aircraft manufacturers demanding nearly complete automation which they believed was necessary to achieve smaller crew sizes that would result in major operational cost reductions. In the end the ALPA gave up the third seat in nearly all commercial aircraft as functions previously spread across three professional pilots were reallocated to two pilots supported by new levels of aircrew automation. The pilots however retained control of some critical functions and most importantly the ability to take complete control from cockpit automation at anytime. Captain Sullenberger in Cactus 1549 made use of this allocation design to maintain control of the A320 during its entire descent into the Hudson River. He did so without being stuffed in the gap between automation and lower operating costs.

Think of this insight in another context. Had the A320 been designed by Facebook one can envision a large crater in Queens with a terse statement on the Facebook website reading “we have changed our flight policy…crashing is now allowed as part of our terms of use”. In the Facebook A320 Captain Sully would have spent the last 3 minutes of his life scanning pages of obscure text looking for the automation control check boxes. Clearly, a commercial aircraft has a different set of usability and user engagement objectives than a Social Networking website but in the end success is determined by applying the same concepts. It is a fact that you can design for 90% automation but realistically you will never achieve more than 60% automation. Therefore, it is more productive to design for automation but plan continually for giving the user control of the functions of the product when your clever software-based automation simply does not work properly or gives the user more complexity to deal with not less. This fact is just as valid on a Social Networking site as it is on the Airbus A320.

Insight 2: Simulate everything, build almost nothing

No industry has made better use of simulations during both development and user training than commercial airlines and the military. Today pilots undergo massive levels of required simulation training to make procedures routine and to experience the actual handling attributes of the planes they fly in sometimes dangerous and dysfunctional situations such as a “total loss of power” descent with a full fuel and passenger load. Professional pilots today can achieve up to 90% of their required training for a commercial license in flight simulations. There are those who believe that modern commercial aviation has been possible essentially through the use of flight simulations for training. Imagine if you will the amount of jet fuel saved based on simulator certification allowances. You are talking about millions of gallons of jet fuel a year. Both Captain Sully and Patrick Harten (the TRACON flight controller), and the tower controller at Teterboro ALL spent time in simulations. When one listens to the dialogue between these parties it is clear that their responses were essentially automatic. These automatic responses would have been impossible without being subjected to the system failure scenarios made possible by simulations. What does this have to do with the user interface design solutions for commercial and consumer products and services in the future?

Surprisingly this insight may be the most important lesson available. To be clear there are several relevant factors in using simulations to create compelling and empowering user interfaces for high technology products and services. The form we are discussing here is known as RS or Robust Simulations. These are simulations that are high fidelity interfaces created during development to allow users to exercise the user interface in ALL critical ways. They are also “robust” because they are created before a single line of actual code has been written. Their entire purpose is to simulate new user interfaces comprehensively BEFORE alpha, beta or launch. Why is this insight important for those who develop new user interfaces in the future? Well, ponder these numbers. This year software development expenditures will total about 270 billion dollars on a global basis. Less than 10 million will be spent on simulations before code is written. About 90% of all large software projects can be termed failures due to cost overruns, failure to deliver functions as specified or flat out user rejection due to dysfunctional user interface design. This is not an especially promising track record for the future even without the current economic crisis. Clearly something is broken here that indicates the need to plan before we build.

Today in the development of commercial aircraft, every aspect of the design is simulated including all flight systems and pilot control and automation configurations and interfaces. Years before the first prototype leaves the runway commercial aircraft manufacturers know exactly how the commercial aircraft will behave and more importantly how it will actually feel to the pilots as they interface with new hardware and software cockpit interface designs. Not every industry sector is so enlightened.

It is curious that no other industry has made less use of simulations than those corporations and startups that develop complex high technology products and services ranging from Social Networking websites to new high-tech consumer products. These companies routinely pound out a million lines of code before sitting a user in front of the system. This is old news. However the lesson in Cactus 1549 for all those who hope to develop successful high technology products and services is now a bit more refined. Here are some additional interesting numbers.

To develop a robust simulation of your product or service is 1/10 the cost of coding the same interface to early alpha level. If you have to make changes in beta or in early launch it is 100 times more costly than the same changes resulting from the simulation. The major benefit of simulation that Cactus 1549 teaches is that with the proper attention to detail simulated user experiences can be made virtually indistinguishable from the real experience. This allows you to simulate almost everything and build almost nothing except that which will objectively drive user adoption and profitability. It is a mystery why VCs routinely fund startups to create early production products based on real code when they could get vastly better return by requiring robust simulations and carefully structured user testing. In the context of how most user interfaces are designed today, let’s think of this whole idea of robust simulations in another way.

Had the A320 been designed by Facebook there would have been no simulation because Facebook flies all of its planes by pure automation. Such automation would have likely lead Cactus 1549 back to LaGuardia runway 1, which almost certainly would have produced a large crater in Queens a mile short of the runway. This would have been followed by another terse statement on the Facebook website reading: “To all our pilots flying the new Facebook A320: whether you like it or not we are turning on the automation, so suffer the consequences!”

Insight 3: The strength of your user’s mental model predicts product success.

This is a concept that is important and not well understood but one that the flight of 1549 teaches all those involved in development of high technology products and services. This concept is revealed in the analysis of  the TRACON controller, Patrick Harten, who plotted alternate landing options for Cactus 1549 at three area airports in less time than it takes most of us to check our email on a Blackberry Storm. When we speak about developing a mental model of a system what we mean is the user’s ability to create a robust mental picture and functional understanding of the system with which they are interacting. The design of all high technology user interfaces virtually determines the strength and clarity of the customer’s mental model. A well designed user interface helps users develop such mental models by DESIGN. Patrick Harten demonstrated an amazingly well developed mental model of his transition air space and 3 major NY area airports. This was present in his rapid suggestions of alternate airports but further included his recall of actual runway numbers, spatial orientations, and distances. Mr. Harten maintained a robust mental model of his operational environment. In the current leading theory in systems design this is known technically as “situation awareness” and was reflected in the operational profile of the pilot and the involved air traffic controllers. This unquestionably contributed to the successful end to a potentially tragic series of events. Designing a user interface to support and create situation awareness will be a fundamental requirement of all successful high technology user interfaces in the future.

Most new high technology products and services are based on user interface design concepts that have virtually no focus on supporting the development of the user’s or customer’s mental model of the system. Yet, this is the single most effective way to test a new system for usability and more importantly the best way to ensure commercial success. If, after appropriate exposure, your customers do not have a firmly established mental model of your new product or service they will never understand either its benefits or its functions. Such products cannot become successful. In the end all of the key players in the Cactus 1549 ditching had highly developed mental models of the systems they were interacting with, where they were, and what actions could and should be taken to produce a productive outcome. Very few new high technology products offer customers interfaces that build such robust mental models or support such decision making performance. Cactus 1549 teaches this important lesson and it is not a trivial learning.

Photo by jkrums, posted on Twitter.

The big lesson

It has been said that we learn more from our failures than from our successes. While Flight 1549 is formally classified by the FAA as a crash it was none the less an amazing example of how an industry (even one crippled by mismanagement) employed development methods that led to the development of a truly world-altering technology, the commercially viable, mass produced, commercial airliner. Possibly the big lesson from 1549 for all who aspire to creating world-altering technology is that by employing automation, simulation and cognitive modeling it is possible to fly 10 billion miles a year without an incident. Even more important may be that, when your time comes (and it will) a soft glide to the Hudson where everyone survives and your corporate brand value skyrockets is a solution worth simulating and ultimately designing for.

Charles L Mauro CHFP

___________________________________________________________________________

Additional observations of interest from Flight 1549 incident.

Captain Sully: What made him extraordinary in this situation? Captain Sully had 20 + years of commercial airline experience and previously flew fighter aircraft in the military, but his most interesting and relevant background may have been his sailplane experience. Did this contribute to his situation awareness and flight domain knowledge as the A320 descended in unpowered flight? Did his final flare out and use of ground effect upon landing result in a vastly more reliable final water landing? Almost certainly this played a role as can be seen in the images of the final water landing taken from a security camera in New Jersey. Was Captain Sully experiencing extreme stress?…almost certainly. How do we know? He failed to activate the “Ditch Switch” once in the water which is designed to close water inlet valves and reduce flooding of the aircraft. This oversight led to more rapid flooding of the aircraft once in the water. This was a classic omission error correlated with high stress. Did he deserve the keys to New York presented to him by Mayor Bloomberg?…without question.

Patrick Harten: how “situation aware” was he really?Was Patrick Harten experiencing high levels of stress during the 3 minutes of the event? Without question he was. How do we know? If you listen carefully to the audio tape you will hear Mr. Harten misstate the actual flight number of Cactus 1549 as 1529 multiple times. This form of error is known as a “commission error” and is understood to be a predictor in most cases of excessive cognitive workload and stress. How “situation aware” was he really? The audio file shows he was extraordinarily “situation-aware”…almost hyper-aware. This is demonstrated by his maintaining control of the emergency flow as well as communications with at least 6 other planes in his direct airspace during the actual crash sequence and by his attending to the location and heading of these other aircraft within seconds of losing Cactus 1549 from radar view. Mr. Harten’s hyper-awareness can likely be attributed to his unique experience as a TRACON controller. His father held the same position at TRACON 30 years prior and young Harten would often come with him to work. The 5 year old was even occasionally allowed to guide planes to turn left or right. From a young  age Mr. Harten has been building awareness of the area he covers as a TRACON controller and on Jan. 15th, he was at the absolute top of his game.