User Experience in ArnoLand

We've moved: https://arnoland.wordpress.com

noreply@blogger.com (Jonathan) — Wed, 22 Apr 2015 03:06:00 +0000

Hello everyone, this is the swansong post for this site, arnoland has moved to the 21st century at: http://arnoland.wordpress.com. See you there ...

The User Experience Designer’s Charlatan Test

noreply@blogger.com (Jonathan) — Tue, 21 Apr 2015 16:34:00 +0000

A First Step towards UX Sanity Checking

Below is the CHI paper I wrote on the UX Designer Charlatan’s Test. It is very much a work in progress and I invite people to help me iterate on it: what questions should be added or subtracted. Each question is not arbitrarily chosen but reflect actual UX practices in the field as practiced by companies I or my colleagues have witnessed throughout the years. Multiple companies/designers/managers have been guilty of the points made here so no one should feel any one company/designer/manager is singled out here. The answer options are agree/disagree/no opinion. I tell you in advance, No Opinion is always wrong. No opinion essentially means you do not care. If too many people chose this option for a given question, I will take that as a indication of a possible low quality question that should be replaced by a better question.

Please enjoy, think of it as the UX Follies, so no chip on the shoulder implied or warranted. I do make the serious point that UX accountability and oversight are sorely lacking but do so with tongue firmly planted in cheek.

To give me feedback on the test comment here or send me an email at arnoland -at- gmail.
I know this paper (some 10 pages long) is longer than most blog posts, I will develop a new, shorter version of this post based on initial feedback I get from readers.

ABSTRACT

This paper proposes an introspective test to see whether the reader of this paper is a charlatan UX practitioner. If so, it points ways the reader can professionalize their practice. For the non-UX professional this questionnaire can act as an interview script to ascertain how professional a potential UX candidate is during a job interview. This paper also infers a need for a governing body of some kind to assure the quality of the practitioner. The test can be taken and evaluated. Future versions of this test look to include a wider coverage of UX best practices, techniques and methods.

INTRODUCTION

In my 20+ years of experience in the User Experience/HCI Design field I have seen not only a large amount of beautiful UX (User Experience) work ship but also an incredible amount of wasted UX effort and pointless design work. Early in my career as a consultant I heartily took part in such practices because it impressed the customer but did not serve the user nor the company hiring my work. The deep point was reached with one of my most lauded designs, a business software generator whose UI’s were based on the paintings of Mondrian. [1] Without engaging me—or any other designer—for any follow up work, the company went broke trying to implement these designs on their own.

Since then, I have been sensitive to user experience designer charlatans. Surprisingly this represents the majority of our profession that I have observed over the years, including design bureaus lauded for their so-called UX excellence, whose work served more to stuff their website portfolio, rather than be any practical help or added value to the companies they claim to be assisting.

Internal teams, even for many large companies, are teeming with design mediocrities that systematically see that they maintain their safe mediocrities through charlatan hiring and design practices. This paper tries to ferret out those charlatans and unmask them for who they really are. I propose that we force everyone in our profession to take The User Experience Designer’s Charlatan Test. This is a personal test that one cannot take publicly to get the correct answers. Instead this is an introspective test the UX Practitioner must take to heart and test themself. Of course, this does not take into account the power of those who lie with such ferocity that they lie even to themselves (and believe their own lies) regarding their personal competence with no or scant evidence. For those of goodwill, taking this test with an honest introspection will point out the errors of your ways and show you how to take corrective action before anyone finds out you are a charlatan. UX is perhaps one of the most curious, frivolous and unprofessional professions I have witnessed.

How is this possible? I know of no profession more allergic to any accountability, professional standards or even some basic tenets of solid design execution. I can’t tell you how many times I have heard even allegedly experienced UX managers say, as a design direction, “Go ahead, knock yourself out.” Or my favorite, “Use your own best practices” (for the reason that you, as the manager, are devoid of them). Most UX managers are overpaid traffic cops and most designers are simply incompetent. Don’t believe me? Take the test. If you can’t get at least 80% correct (16 of 20 or using five points per question, 80 points), then you are a charlatan and part of the problem. But don’t worry; you are in very good company and the answers you got wrong can point the way to your redemption.

One word on what this paper is not: this paper does not single out any individual company or person. I mean to accuse us all and almost without exception. There is no one company, organization or designer more culpable than another. My observations cover the work of many colleagues in many companies for whom I have not worked. So everyone should feel equally distressed.

And for further clarity, here is the definition of what I am accusing you of:

Charlatan — a person falsely claiming to have a special knowledge or skill; a fraud

In answering these questions none are aimed at how things are in your company, university or organization but rather, on your practices in them. In answering the questions, answer the way you strive to behave or practice if allowed. In other words you are not a charlatan if work forces you to perform unprofessionally—that makes you something else beyond the scope of this paper.

For example, let’s pretend one of the questions is: “Agree or disagree: You believe users are irrelevant to user experience design.” You may very well—indeed probably do—work for a company where this is generally accepted. However, if you are actively fighting against this you should answer that you disagree rather than agree with a company policy. The test is all about you, not the company or organization in which you work.

There is only one correct answer. When in doubt, choose the best answer among multiple choices. For example, if the question arises: “All users are …” though you may be tempted to answer in the affirmative, the negative is the more likely answer.

Please use non-erasable pen to mark your answers. Freudian slips are as important as a considered answer. Without further ado, sharpen your wits, and here comes the test:

The UX Designer’s Charlatan Test

Test Questions (5 points each question):

1. Design is purely data determined
☐ Agree ☐ Disagree ☐ No Opinion

2. UX has an ROI and that is it’s sole reason to exist.
☐ Agree ☐ Disagree ☐ No Opinion

3. You know what a great User Experience is when you see it.
☐ Agree ☐ Disagree ☐ No Opinion

4. I weight positively a UX resume that shows a lot of experience and impressive UX titles.
☐ Agree ☐ Disagree ☐ No Opinion

5. I consistently follow the most trendy or popular UX methods or tools. So-called timeless ones are too academic.
☐ Agree ☐ Disagree ☐ No Opinion

6. I can judge a UX portfolio by looking at it.
☐ Agree ☐ Disagree ☐ No Opinion

7. I am intrinsically opposed to the formation of a generally (State, National or Domain specific) accepted UX certifying agency.
☐ Agree ☐ Disagree ☐ No Opinion

8. I can name what UX should own and be held accountable for.
☐ Agree ☐ Disagree ☐ No Opinion

9. Aesthetics drives the UX, consequently, someone with a good aesthetic sensibility can naturally create a great user experience? Top designers know the profession in their gut and between their ears.
☐ Agree ☐ Disagree ☐ No Opinion

10. Agile and/or lean startup are design methods.
☐ Agree ☐ Disagree ☐ No Opinion

11. I believe engineering or development is the most important aspect to creating successful digital products.
☐ Agree ☐ Disagree ☐ No Opinion

12. I can sum up what a User Experience does in a short succinct sentence or two.
☐ Agree ☐ Disagree ☐ No Opinion

13. I can name three books on cognitive psychology that I have read, relevant to user experience.
☐ Agree ☐ Disagree ☐ No Opinion

14. Excluding Steve Jobs or Jonathan Ive, when asked who is my favorite designer, I can name a designer as well as their lasting artifact of digital design excellence.
☐ Agree ☐ Disagree ☐ No Opinion

15. Without objection, I work on projects I do not believe in, it’s a job after all.
☐ Agree ☐ Disagree ☐ No Opinion

16. I believe that only designers can come up with good designs.
☐ Agree ☐ Disagree ☐ No Opinion

17. I am specialized into a single domain.
☐ Agree ☐ Disagree ☐ No Opinion

18. I am specialized in a single set of design tools.
☐ Agree ☐ Disagree ☐ No Opinion

19. No one really understands UX design. It is best if I can go off and just do it without interference from engineering or product management. Then just show them what a good job we can do.
☐ Agree ☐ Disagree ☐ No Opinion

20. When designing, I show multiple and significantly alternative concepts.
☐ Agree ☐ Disagree ☐ No Opinion

The correct answers

1. Design is purely data determined

DISAGREE — data can inform design but data cannot make a design decision. Well collected and analyzed data can point out a problem but there are infinite number of ways to address the problem for that a designer—at the end of the day the designer is always trusting their gut, their gut feelings maybe supported by data and with the help of UX researchers that gut can be inspired or directed but not dictated to. Moreover poor quality data (bad sampling, asking the wrong questions, naive or inaccurate analysis) can have a detrimental effect on design. [2] A leading UX researcher once said, tell me what you want to prove and I can design a research protocol to prove it.

2. UX has an ROI and that is it’s sole reason to exist.

DISAGREE — Software development is one of the most complex activities. It requires work from many different backgrounds, domains and resources. Their success is inevitably a sum of their parts rather than the isolation of the whole. Are there instances where an improved UX appears to have had a positive effect on the success of a product? No doubt, but that can also be said of good product management, good engineering, none of which operate on an ROI so why should UX? UX has more vital role than improving a company’s bottom line it is what a company is really about to the end user.

With so many competing interests and stakeholders in the software or digital product process, trying to prove an ROI on just UX is dubious at best. Instead the argument should be that UX is an essential part of the product just like the code just like marketing and just like the sales. [3]

3. You know what a great User Experience is when you see it.

DISAGREE — A user experience is intangible and abstract. You cannot see this thing; it is based on visuals, behavior and user perceptions. Without knowing those elements you cannot judge a UX full stop. So get off your hobby-horses! The UX award shows are mostly shams and temples to charlatanism.

4. I weight positively a UX resume that shows a lot of experience and impressive UX titles.

DISAGREE — It is possible to be a total failure at UX and inhabit all kinds of responsible UX positions for major and minor companies. Since very few people understand UX, the chances of any engineering-driven company giving out meaningful UX titles are very low. When evaluating a UX title you need to do your research and see who bestowed the title; was it simply a manager? a Vice President? Who they reported to will tell you a lot more than their title. As to a UX resume, the gobbledy-gook in them should set expectations for what the prospect will be grilled on in an interview. None of it should be taken for granted. Saying does not make it so. Read your Descartes.

5. I consistently follow the most trendy or popular UX methods or tools. So-called timeless ones are too academic.

DISAGREE — Popular methods can never compete with timeless ones. There is a reason why they are timeless. [4] There are all sorts of sketching techniques, user research ideas and entire design ideologies, like Lean Startup, that have specific applicability. When looking for the right design tools, check out case studies and see how these tools work or do not work in your situation. Case studies that are simply “we came, we saw, we conquered” without exposing any of the involved trade-offs are useless. Our profession is nothing if not trade-offs and only timeless design methods will really get you to make the right ones.

6. I can judge a UX portfolio by looking at it.

Disagree — By looking at a job applicant’s portfolio you are reviewing the work of the visual design’s initial impression, which may not have been a primary consideration in the design at all; moreover, you have no idea:

what role the applicant may have played in that design

what their achievements were,

what the ambitions of the project were,

their inter-team dynamics,

their design rationale, etc.

I think most people in this room would be astonished at how many even seasoned “Experienced” Charlatan UX Managers there are. I have observed many a hiring committee in many different companies and I have been appalled at the ease with which these charlatan glance at portfolios and quickly judge whether the candidate is a great candidate or not, even when the role was researcher, interaction designer, visual designer, or UX designer. The result has been mediocre and arbitrary hiring practices that lead to mediocre teams, which further ruin our professional credibility.

7. I am intrinsically opposed to the formation of a generally (State, National or Domain specific) accepted UX certifying agency.

DISAGREE — Except for trepidation on the quality of the certification process, why should you be afraid to show off your skills and prove you are a certifiable worthy designer? As Dan Rosenberg once observed, you need a license to practice architecture, why not also UX?

Personally, I would love to see a study done on the thousands of people who were fired not because of errors in judgment but errors in UX design and data presentation, which lead them to make the wrong decision. Charlatans reign supreme when no one is minding the professional store and no objective information is gathered about a designer’s competencies, and in specific UX areas.

All too often, people are touting themselves as UX one man bands. This is unrealistic. Each designer has a specialty (IxD, VD, etc.) Designers should be certified – at the very least – in their specialties. However, if you have your suspicions about how a board would be comprised and what they would use to analyze and certify applicants, that is fair enough; if you know your profession so well, get involved in the certifying committee. I don’t like the image of a certifying authority being like the Guild Meistersingers, simply observing adherence to rules of the trade and not their creative application. However, I prefer that authority to the UX Charlatans who would rather think of our profession as a mystical exercise that defies objective evaluation, observation or accountability.

8. I can name what UX should own and be held accountable for.

Agree — You must want to be held accountable to the design and its implementation. Even if in your current organization, you are not accountable, you should still be able to articulate the accountability as a desirable goal. Among the things you can mention are:

1. Accountable for the design concepts

2. Accountable for quality and effectiveness of all UX artifacts

3. Partner in the Development process, not a service

4. Empowered to log high priority bugs against bad UX implementations, which warrant high priority

9. Aesthetics drives the UX, consequently, someone with a good aesthetic sensibility can naturally create a great user experience? Top designers know the profession in their gut and between their ears.

DISAGREE — Aesthetics can be surprisingly of little relevance because they are mostly subjective, except when they are based on the user’s aesthetic sensibilities, not the designer’s. Design rationales should rule the top priority design decisions instead of pure matters of taste. There should also be a reason for every ‘gut’ level design decision that the designer makes even if the reason is a simple design guideline important to the product, such as Fitt’s Law. You should be able to argue to defend your overall design with objective data and when disproven or in doubt should be courageous enough to change your mind. Great designers often have great instincts, but great designers use more of their brain than just their instincts. Incredible guts are only half the story as the difference between guts and gall is a thin line, without objective arguments. [5]

10. Agile and/or lean startup are design methods.

Disagree — These are software engineering methods. UX charlatans are constantly coming up with books, articles, and cults that conflate software engineering with design. The two have little to do with each other, except that they need to be scaled and coordinated with each other. If software development scaled to UX, instead of the other way around, there would be many fewer unsuccessful products. But that would assume there exists sufficiently competent designers and developers to trust in a real iterative design and development method. [6]

All too often, charlatan UX’ers use agile or lean as an excuse for turning in shoddy work they can ‘fix later’. Agile, as commonly practiced, is largely nothing more than incremental waterfalls, and lean startup—again as practiced—is for people who only vaguely know what they are doing, Anorexic UX would be a better title for what they practice. Neither Agile nor Lean is an excuse for UX to turn in substandard work. I have never met a developer who can output code faster than a competent designer can paint pixels or generate a prototype. [7]

11. I believe engineering or development is the most important aspect to creating successful digital products.

Disagree — Software Development is not the most important factor in product development. There are others equally important, all of which are essential parts of the digital solution: UX, Product, Business, etc. The digital solution process (software, services or digital products of any kind) starts well before development, with an idea. The process also ends with development taking a back seat to marketing. Moreover, most of the time the product is successful because of the team approach, not because of the software engineer as hero.

12. I can sum up what a User Experience does in a short succinct sentence or two.

Agree — If you cannot sum up what you do every day of your life for work then you have no idea what you are doing. There are many ways to define UX. Whatever way you pick, it is a statement of your vision of the great user experience.

My particularly favorite definition—among many possible—is: User Experience designers take products that force a user to think like a computer and turn them into products that mimic the way their users think. The UX Charlatan has no idea what User Experience is. They resort to a mish-mash of visual models, abstract visualizations and a grab bag of jargon where the recipient is none the wiser, which is the hidden agenda of the charlatan: obfuscate so they can just do whatever they want and call it good.

13. I can name three books on cognitive psychology that I have read, relevant to user experience.

Agree — if you do not understand basic tenets of the human brain, you have no business being in this profession. You do not need to be a cognitive psychologist, and in many cases I prefer that you aren’t; however, in the list of books below that you should have read I include some popular titles as well as scientific tomes, so there is really no excuse.

The Psychology of HCI by Stuart K. Card, Thomas P. Moran, Allen Newell
Designing with the Mind in Mind, by Jeff Johnson
Why we Make Mistakes, by Joseph T Hallinan
How We Decide, by Jonah Lehrer
The Design of Everyday Things (original title was The Psychology of Everyday Things) by Don Norman

Some designers with no inkling of what Cognitive Psychology is claim that you do not need to be a psychologist in order to design. I will actually go out on a limb and tell them they are wrong. So much of what we do as designers is based on psychology: Cognitive, Gestalt, and Social all playing an important role, whether we know it or not. So, if you know what you are doing instead of intuiting it you are, by definition, a more professional designer.

14. Excluding Steve Jobs or Jonathan Ive, when asked who is my favorite designer, I can name a designer as well as their lasting artifact of digital design excellence.

AGREE — I cannot tell you how many job interviews I have suffered when a potential candidate names as their favorite designer either Donald Norman or Bill Buxton. Mr. Norman has never claimed to be a designer. As for Mr. Buxton, when I ask the candidate what Mr. Buxton has designed that they love so much they cannot tell me. Other people great and small have also been named but, surprisingly, rarely is there a design heavyweight with a proven track record—because most of us wouldn’t know one if we saw one, nor know how to recognize them or their outstanding products. [8] This situation is understandable given the non-transparent nature of our industry, the lack of personal acknowledgment for the designer for anything and most organizations’ allergy for UX accountability.

15. Without objection, I work on projects I do not believe in, it’s a job afterall.

DISAGREE — It is alarming how many consultants and even UX internal employees gleefully perform jobs they know are senseless. I have seen many famous consultancies do absolutely schlock work. When confronted about that, they reply, “That’s what the client wanted.” Employees also do work they think is meaningless, just shrugging their shoulders, “That’s what management wants.” Now sometimes you have to do work you do not agree with, and certainly you should pick your battles. But wasting a quarter of million dollars in a project doomed from the start is morally reprehensible, especially when the due diligence was not done to change the direction of the project into a professionally responsible direction. If consultants work with in-house UX teams, or employees work with their internal stakeholders to change the direction of a project and management still goes is a direction the designer does not agree with, then the matter is one of personal ethics not charlatanism.

16. I believe that only designers can come up with good designs.

DISAGREE — You impoverish your design thinking when you restrict yourself to the ideas that only you or your UX colleagues generated. A real designer does not fear great ideas from developers, product managers, marketers, etc. Rather, the designer learns to embrace these ideas and learn from them.

17. I am specialized into a single domain.

Disagree — A designer who is specialized in just consumer Electronics, or medical e-commerce, or mobile music apps, or enterprise analytics software, or … (fill in the blank) are usually the most inflexible and unimaginative designers around. They tend to see things from a single perspective – the mainstream of the domain – which in turn leads to failing to see problems that others with a fresh pair of eyes can see. These UX Charlatans have, at best, mastered trial-by-error (woe betide the early employers of these guys) a single domain’s challenges and learned by rote how to solve problems by what worked in the past, instead of analyzing the situation and coming up with a fresh new idea. My most engaging work was often my first foray into a domain, or the first foray back into a domain after a long absence. In both cases I had fresh eyes able to learn and apply lessons from other domains. Another good example is how all the specialized mobile designers of Erickson, Nokia, Motorola, etc., failed not only to produce the iPhone first but even failed to come up with a credible answer to it. It took another non-phone company, Google, to do it.

18. I am specialized in a single set of design tools.

Disagree — A good designer is open to learning new skills and techniques. This often means adding new tools to your list of trusted tools of the trade. I am not against anyone having a favorite tool, but one should still be open to see if other tools might be more suitable for a given project. One should also be flexible enough that if required, you could design in new tools you have not previously used. The UX Charlatan is a one trick pony, like to a hammer all problems are a nail. [9] Conversely it is also true that organizations should not force a specialization in a single UX tool, but rather in a set of UX deliverables for their projects (but that’s a different topic).

19. No one really understands UX design. It is best if I can go off and just do it without interference from engineering or product management. Then just show them what a good job we can do.

DISAGREE — The best way to impoverish design is through walling yourself off from your product development team, and, by far the worst offense of UX Charlatans is not being transparent with your product teams. Walling yourself off means just going off and designing, getting feedback from stakeholders and then iterating in isolation from other team members. Lack of transparency means hiding the design process, your decision-making rationale, and your design activities from stakeholders. Stakeholders should be enabled to manage and assist you better if you are up front with your process and planned activities. You should also be upfront regarding what deliverables or outcomes will result from these activities and their trade offs if you need to compromise This empowers everyone to come up with a workable and understandable UX plan. Lastly, by iterating in isolation, instead of in participatory, collaborative or interactive design sessions, you mask your lack of knowledge or skills at the expense of getting essential feedback from other stakeholders.

20. When designing, I show multiple and significantly alternative concepts.

YES — Just presenting a single UX solution and getting feedback is more an engineering than a design exercise. While appropriate for developers, for designers iteration must involve interactive collaboration as well as the use of multiple design concepts and synthesizing them into a single unified concept. This allows you to include innovative ideas you would not otherwise have considered.

Quick Summary Results Table

Question Answer Reply

1 Disagree

2 Disagree

3 Disagree

4 Disagree

5 Disagree

6 Disagree

7 Disagree

8 Agree

9 Disagree

10 Disagree

11 Disagree

12 Agree

13 Agree

14 Agree

15 Disagree

16 Disagree

17 Disagree

18 Disagree

19 Disagree

20 Agree

Total

SCORE

Table 1. Answer sheet for the quiz. How well did you do?

In Summary Table 1 displays the answers to the quiz. Add 5 points for every correct answer. Give yourself an honest score. Please do not tell anyone what it is; this only works as an introspective reality-check that only you need to enjoy. Moreover, if you are a charlatan chances are we know that already. But cheer up, as I said – you are in good company in this profession. However, given your results, does the looming possibility of a universally accepted UX certification board make your tremble or shout with delight.

Conclusion

The purpose of this exam is an introspection for the benefit of our profession. I hope people undertake this test honestly in their hearts and let it serve as a guide for where they may wish to work on their profession in order to become less and less a charlatan and more and more a professional.

For those who wish to hire UX professionals, I hope this quiz can also serve as a guide to ask questions that will lead you to a competent candidate. Let me just repeat what this paper is not: this paper does not single out any single company or person. I mean to accuse us all and almost without exception. There is no one company, organization or designer more culpable than another. My observations cover the work of many colleagues in many companies for whom I have not worked. So everyone should feel equally distressed.

ACKNOWLEDGMENTS

There are also many non-charlatans: designers from whom I have learned and by whom I have been inspired. I am also blessed to have known many excellent UX Managers with whom it has been a privilege to work, and who have helped me grow out of my own charlatanism. My professional persona is also a sum total of the amazing product managers, developers, marketers and the occasional CEO it has been my honor to know. To all of them I am greatly indebted and without whom I could not have made these hopefully pithy, wise, and (for those who can appreciate such things) humorous observations. I think I do them a greater favor by not mentioning them here.

REFERENCES

Priester, Ruurd, Arnowitz, Jonathan , Willems, Eric, Faber, Laura, Mahler, Mondriaan, and Bauhaus: using artistic ideas to improve application usability, DIS '97 Proceedings of the 2nd conference on Designing interactive systems: processes, practices, methods, and techniques, Pages 12-21
Albert, Bill, Tullis, Thomas, Measuring the User Experience, Elsevier Books book 2013, this book covers many different ways data can be collected and analyzed.
Rosenberg, Daniel, The myths of usability ROI, interactions - Volume 11 Issue 5, September + October 2004, Pages 22-29
Arnowitz, Jonathan and Dykstra-Erickson, Elizabeth, Masters of our process, interactions Volume 14 Issue 5, September + October 2007 Pages 56-ff.
Mackay, Wendy, Dykstra-Erickson, Elizabeth, and Arnowitz, Jonathan. Perspectives: trialogue on design (of) interactions, Volume 8 Issue 2, March 2001 Pages 109-117
Arnowitz, Jonathan Taking the fast RIDE: designing while being agile, interactions Interactions Homepage archive, Volume 20 Issue 4, July + August 2013, Pages 76-79
Enter the chief design officer! hail to the chief! interactions - Volume 14 Issue 4, July + August 2007, Pages 56-ff
Arnowitz, Jonathan, Arent, Michael, Berger, Nevin, book, Effective Prototyping for Software Makers. Morgan Kaufmann Publishers Inc. San Francisco, CA, USA ©2006

Excel Protptyping Templates

noreply@blogger.com (Jonathan) — Mon, 03 Feb 2014 14:55:00 +0000

A long time ago, in a galaxy far far away, I co-authored a book with Nevin Berger, Michael Arent and Fred Sampson on Excel Prototyping for Software Makers. Yes, really using Excel as a software prototyping software. It is still a good idea, although the version of excel have evolved so it makes the book not as helpful as it once was. Nevertheless, for some people look for a simple fast prototyping tool Excel is still attractive for some.
There even seems to be renewed interest in our book: http://www.amazon.com/Effective-Prototyping-Excel-Interactive-Technologies-ebook/dp/B003FK5PWA/ref=sr_1_3?ie=UTF8&qid=1391438838&sr=8-3&keywords=excel+prototyping. Unlike the book link on the left this one will make me no money via Amazon, which is only fair since the book is dated and no plans are made to update it.
The problem with this renewed interest in our book is our web site has died a slow painful death. The web site is where we made our prototype templates available.
So to facilitate a place to get these templates, I am posting them right here, enjoy:
https://drive.google.com/file/d/0ByaeNqsDdjGXQVNwbXl4UjQwUEE/edit?usp=sharing
If you have any problems downloading post a message here, but please be polite.
Lastly, I am using Google Drive so if you have a problem with the way Google Drive works that's not an issue I can solve.

Taking the Fast RIDE: Designing While Being Agile

noreply@blogger.com (Jonathan) — Wed, 22 Jan 2014 19:44:00 +0000

A new year and a new post in my less than active blog, here is an article that originally appeared in interactions magazine.

While many design methods are practiced “in the wild,” the most prevalent one appears to be “Design first and ask questions later”—also known as “Throw it over the wall and see if anybody salutes,” “Launch first, fix later,” and so on. Whatever you call them, these approaches are all responses to the pressure for rapid turnaround man- dated by Agile and other high-speed development environments. These design approaches are all proven methods—that is, proven to create Frankenstein UIs within a mere two to three iterations: That’s speed.

A single-minded focus on speed guarantees that these methods produce poor user experiences, because they do not allow for the reflection and deliberation necessary to achieve high-quality, coherent design. Instead, speed breeds pragmatic short-term solutions. An interaction style gets locked down in the early sprints. Then other ad hoc interaction styles emerge for parts added later. Too much emphasis on reactive speed reduces design to puzzle fitting: How can I put this new square-peg function into my round-hole application?

I am concerned that in an attempt to adapt to the pressure for speed, designers are failing to uphold what we know is good design practice. We compromise too much of the interaction strategy y in order to “ just get down to it.” The end result is competing styles, competing imagery, and unintelligible system/conceptual models.

Agile and the Emperor’s New Clothes

How do you get out of this cycle and still survive in the context of fast Agile or pseudo-Agile (aka reckless) release cycles?

This question may sound odd, coming after years of our learning to live in Agile environments. Agile’s true believers may argue that the whole point of Agile is not about producing speed, but instead about creating a structured process to produce high- quality results in the face of the pressure toward speed (which Agile itself did not create).

But there are obvious incompatibilities between Agile and the needs of good design practice. Unfortunately, an emperor’s new clothes mentality has developed in which it is not acceptable for user experience people to point these out. Instead, we become mock up

monkeys scurrying to meet the next sprint deadline, throwing in features and widgets at whim.

Part of the problem is that Agile’s own best practices are often not followed—for example, and most important, the regression testing, which introduces some flexibility in Agile software development. People tend to claim that Agile is iterative, but without regression testing, it is too often practiced in a piecemeal manner, wherein once something is developed it cannot easily be changed. This makes the emerging Agile practices look a lot like an incremental Waterfall development process. Once sprints begin, the train has left the station. You are then stuck with design decisions made at Sprint 0 or 1, with little ability to iterate on basic concepts.

Perhaps you have heard statements like this or experienced things like this in Agile environments:

Sprint 1: “Designer, just do something, anything, right now so we can get some feedback—we can always change it later.”
Sprint 2: “Designer, sorry, we can’t make changes anymore—that would affect the back end.”
Sprint 3: “Oh, we can’t do that because of lack of resources. You have to stick with your current design. Of course, you can always tack on a new visual design.”

As common and frustrating as this sequence of events sounds, the main point is not just that things keep changing; it is that the serial structure of Agile sprints, which may make sense for engineering, does not fit for design. Often with the best of intentions, designers are either too lazy to push back or intimidated into thinking serially instead of conceptually. This fundamental mistake causes poor design.

Design does not proceed by dividing a complex problem into parts and then working on them sequentially. Design is more of a layered process, moving from broad concepts to devilishly (or heavenly) detailed design. Broad design concepts, overall IA, and key interaction models need to be established first. This conceptual design then guides the detailed designs. Then, as this detailed design progresses, some detailed decisions may fracture the existing overall concepts, showing their limitations. This speaks of the need to supply feedback from the detailed design work back to the underlying conceptual layer.

Establishing a successful conceptual design calls for intensive collaboration between designers and researchers. I would advocate (hence my article’s inclusion in this section of interactions) that in Agile environments, designers and researchers need to be joined at the hip more than ever. They need to work in close concert in order to be as strategically and tactically “agile” as possible.

What About RITE?

Rapid Iterative Testing and Evaluation (RITE) is one of the main ways of trying to incorporate an iterative user-centered design mind-set into an Agile development process. It also joins research and design by coupling testing with quick design revisions. (See Medlock, M.C., Wixon, D., McGee, M., and Welsh, D. The Rapid Iterative Test and Evaluation Method: Better products in less time. In Cost Justifying Usability. G. Bias and D. Mayhew, eds. Morgan Kaufmann, San Francisco, 2005, 489-517.] for more information on RITE.) RITE does ensure some feedback from testing into design. However, in my observation, it does not address the mismatch between Agile and good design practice.

RITE is too reactive and tactical; it also does not address the need to support early, deep design work. It actually separates the testing activity from the interaction design activity by fragmenting the team, and puts both design and research in a reactive, entirely tactical mode. I have seen teams where the usability researchers are consumed with scrambling to plan and set up a test of features on parts of a couple of pages.

Meanwhile the design team is moving on to design some other part. Soon the usability researchers are scrambling equally reactively to test features from the next pages under development. The result is that no one ever evaluates the overall concept or architecture, or even the contextual fit of the application as a whole.

RITE’s testing-led approach to design improves design incrementally. There is nothing inherently wrong with this, as long as one is testing the right things. Unfortunately, this practice will never lead to that, especially as testing will tend to focus on what is being worked on in a given sprint. RITE also can lead to a kind of tyranny of testing.

Testing is an important evaluation technique. But a good researcher knows to mix a cocktail of different evaluative techniques to come up with a far richer view of the system and its user. RITE’s pragmatism never gets to the level of sophistication needed for a holistic design evaluation.

The RIDE Alternative

There are better ways. In this design-hostile environment, I advocate a design method I call Rapid Iterative Design and Evaluation (RIDE). In addition to rapidness, it emphasizes inter play between design and research, beginning with conceptual design, where every evaluation is not necessarily a test. The method also allows for alter native evaluation methods for specific iterations. Moreover, it includes partnering with engineering and product management to rapidly work through multiple concepts. This allows the team to identify the backbone of concepts. These backbones (as opposed to key user stories) get developed/ evaluated first. This places UX strategic design decisions up front, where they belong. While these strategic decisions are still made in the context of a sprint, RIDE respects the need for the layered design thinking needed to design a system holistically.

RIDE strives to do this by:

• encouraging the design team (by which I mean to include ever y- body with design input—interaction designers, visual designers, user experience researchers, and, yes, even engineers) to work faster through collaborating rather than working in isolation;

respecting the best practices of UCD/HCI/design; and
encouraging the development and evaluation of multiple design concepts at each stage.

The main ingredients of RIDE are:

Understanding the product context
UX Planning—establishing cross-disciplinary collaboration
Defining UX goals
Rapidly generating and evaluating multiple design concepts
Holistic iteration.

The first two steps here belong in the product-definition phase before the sprint cycles begin. Since Agile promotes fragmentation, a holistic view of the product should first be developed. Unlike a traditional UCD project, the concept is developed in broad strokes, leaving the further definition to the sprints.

Understanding product context

Understanding product context— taking time to understand the users and usage context.

The product context is an essential element in the definition of the product. This effort, led by product management, includes development, design, and research. It helps clarify the product landscape, the users, and their environment. Design helps to visualize this definition through the rapid sketching of multiple concepts. These concepts are done quickly and iteratively, as more and

more information is gained about the product. The end result is a basis for a product requirements document (PRD) and two to three credible alternative UX directions.

UX planning

UX planning—establishing cross- disciplinary collaboration at each phase.

Design and research create a UX plan, which is meant to span the design and sprint iterations. The plan anticipates the possibility that some research and design

activities stretch over sprints. This is the strategic plan to achieve

the design goal of the product and includes identifying the types of evaluation, research, and design activities that will take place and when. Again, this plan does not follow sprint planning but will take it into account. After every sprint, the plan can be reiterated based on the usually unexpected outcomes of the sprint.

The RIDE UX plan also creates the possibility for each stakeholder to influence, guide, and inspire the design at all levels. RIDE acknowledges that design is not done in a vacuum. The more isolated it is from other disciplines, the more discontinuity and signal interference will arise in the UX. Rather than separating UX design into individual sub-disciplines (visual, information, interaction, engineering, product management, and other stakeholders), these all need to work together, because each part of UX design informs, feeds, and inspires the others.

For example, there is nothing to prevent a researcher or engineer from coming up with a great interaction design solution. Nor a designer coming up with a better analysis of the data. Quite to the contrary—combining their differing perspectives almost guarantees new, innovative ideas. We need not be afraid of a plethora of ideas. Some designers in fact fear engineers making design decisions. Yet if the developer is in tune with the larger design concepts, the chances of their having a good point are significantly increased over when they do not. And further, good designers should be able to defend and persuade stakeholders of the value of their designs; otherwise they might have to face the possibility that they may be wrong.

Outside of these core stakeholders, there is another equally important outer circle. These people include anyone who is taking vital interest in the user experience

and has the power to influence it for good or evil—anyone from developers to marketers to CEOs. Without cultivating their support from the beginning and working to keep them on board, you risk having progress derailed later by random changes of direction. In this planning phase, one should engage them early and on the most abstract level they can stomach: product definition. Get them to wrestle with the big conceptual design choices before development starts. Support from them also strengthens your mandate to execute on the concept.

Of course, nothing guarantees that the CEO won’t insist on purple buttons late in the game; however, in an Agile environment, this is less likely to happen if the stakeholders are on board when the conceptual train leaves the station.

Without this plan, you have to find some way of dealing with these outer-circle UX inputs. Research may give you a chance to resolve design debates objectively, but the Agile timeline typically does not allow for this. The best defense is to make them partners proactively in the design process. Include as many stakeholders early on in brain- storming sessions where the conceptual design is worked out.

Defining UX goals.

Before the beginning of a sprint, it is important to establish the UX goals. These goals require four types of iteration: product definition, conceptual design, detailed design, and evaluation activities. I don’t mean to suggest that these are serial types of goals; rather, they are all interconnected. In the early sprints, the accent may be on product definition, moving to conceptual design. But even these should be done in service of the detailed design. This way, activities do double duty: iterating the practical short-term goals and informing /evaluating the strategic longer-term goals.

These goals are also planned along three time scales:

• The project end—progress to product release

• The current UX plan timeline— the current planned and ad hoc UX activities

• The current sprint—a time snapshot of the current state of the UX goals.

Rapidly generating and evaluating multiple design concepts.

Designer and researcher work on a coordinated effort at designing, evaluating, and then iterating during the sprint. Many parallel activities are driven by the design strategy, not by a reactive “test and see what happens” approach. It is much different from RITE in that the UX goals will determine the strategy for the sprint. The evaluations may or may not trigger reiteration; they may just inform. They can also split off another design variation for exploration. This depends on the agreed-upon evaluation activities: Are they formative or evaluative? Are they abstract or concrete? Will they help find a synthesis among competing ideas? Researcher and designer are partners in this effort. In parallel, longer and different activities (focus groups, interviews, cognitive walk-throughs, etc.) are being done to triangulate with data from other evaluations.

Holistic iteration.

This involves evaluation at the strategic level in parallel with detailed design. Detailed designs sometimes will trigger refinement of the conceptual design and vice-versa. Moreover, evaluations can touch visual, interaction, information, and system design issues— one cannot predict which. Therefore, many design disciplines involved can lead to vastly different results. For example, when a particular design tests poorly, an interaction designer might change the interaction. A visual designer might change typography, colors, and so on. This leads again to incremental and non-holistic revisions. Real iteration would involve these different design disciplines and find ways to distribute the answer among all the design elements, thereby coming up with a far more robust iteration. Holistic iteration also means assuring the deliver y of what engineering needs to meet their goals and their management objectives. Meeting this need will, of course, mean trade-offs on design. This is why having engineering in the core team is essential. Any development method that ignores or frustrates the engineering team’s management goals will fail.

Conclusion

RIDE is a richer design methodology because it leverages a collaboration of all stakeholders. It encourages exploration through a reliance on multiple design options that are synthesized, as opposed to a single option that is puzzle-fitted with additional features. RIDE seeks to find a holistic solution for Agile design and development, not just a tool for rapid changes to a single concept. But most important, it tries to find the right way for design (interaction, visual,

and information) to work with researchers: joined at the hip. It also requires strength and energy, because--as with any quality UX--there is no free RIDE.

Prototyping 3: What is a prototype, part 2 other dimensions

noreply@blogger.com (Jonathan) — Mon, 22 Mar 2010 08:00:00 +0000

In Part 1 of what is a prototype we discussed many dimensions of what a prototype is, but this covers only part of the story, actually half there are many more layers of complexity and if you do not understand them, instead of controlling a prototype, the prototype will control you or worse victimize you. So let’s begin with some prototyping characteristics, for lack of a better term.

Prototyping characteristics

Prototypes have many more important characteristics than just content and fidelity. Knowing what these characteristics are will also help you plan and execute the prototype to the right level of effort. Too numerous to name them all here, here are just a few examples:
Longevity -- what is the lifecycle of the prototype. Is it something to be presented and thrown away, or is it part of an evolutionary prototyping design cycle? How long a prototype will continue to haunt you, should effect how much effort you are willing to put into it.
Stage -- what stage of development is the product? Usually, the more mature the more detailed the prototype should be.
Speed -- how much time do you have? If you have one week, it probably isn’t enough time to make as thorough a prototype as you would like, you may have to adjust your content-fidelity ambitions based on how fast you must work.
Style -- will the prototype be narrative (e.g. demo’d) or interactive (e.g. used). Interactive prototypes are more difficult and time consuming than narrative ones.
Medium -- will the prototype be in a digital media or physical, if digital will it be on the web, mobile or a desktop application, etc.
Being aware of the characteristics of a prototype, empower you to make much more professional judgement as to what kind of prototype you can make.

Defined audience (s)

Audience -- who is the the prototype for? Unlike the end product which is meant for an end user a prototype is meant for certain stakeholders, which may or may not include end users. The prototype should be designed to communicate clearly with the stakeholders. For example, this usually means that a prototype meant for the CEO of the company, will probably look different than a prototype meant for a domain expert

Prototyping tool(s)

Prototyping tools are like tools of the trade, the more you know the better. Likewise for many the simplest software tools suffice for most purposes.
There is no one single prototyping tool that can do everything. Prototyping tools are as varied as there are types of prototypes. Prototypes can just as easily be made in Excel, Powerpoint, Visio, even Word as they can be made in Axure, Dreamweaver, Visual Studio etc.
The point is to match 2 things: First, match the prototyping characteristics with the right toolset. Secondly, of those tools, use the tools you know best. Chances are, your talents in software you know well will outstrip the added functionality of other software tools.
Personally, I no longer use a single tool, but quickly jump between Graphics editors, html editors, scripting tools, layout tools, and yes the occasional prototyping tool.
...but having said that there are some types of tools

dedicated prototyping tools
Programming tools with prototyping capabilities
graphical tools
layout tools
presentation tools

Dedicated prototyping tools, tools that are only for the creation of prototypes not working software or any other purpose. Examples:

Axure
Denim
Balsamiq

Programming tools with prototyping capabilities -- tools that can create full functioning software, but due to their efficient interfaces can allow users to also create prototypes. The theory, or rather myth is if a designer uses one of these tools, a programmer can take over the design and implement it without recreating it. This is rarely true as the html code, or programming code used by a designer (focusing on visualizing something) is completely different in nature to that of a programmer (focusing on implementing something). Examples:

Dreamweaver
Visual studio
Flash

Graphical tools -- tools that help you create the visuals of an interface, ideal for wireframes. Sometimes these tools can also mimic interaction making them suitable for a variety of prototypes. Examples:

Photoshop
Fireworks
Paintshop pro

Layout tools -- tools that help you layout content. Sometimes these tools include interactivity such as hyperlinks or programming scripts that help create a variety of prototypes.

Word
Pages
Excel
Numbers
Visio

Presentation tools -- tools that have some built in narrative capabilities that make it particularly (though not exclusively) suited for narrative prototypes.

Powerpoint
Keynote
Acrobat

Method

Prototyping is much more than just wireframes or a ‘dumbed down’ version of real software. The Methods are many, and in addition to the methods below, there are all sorts of hybrid methods which combine features of other methods. Just to give you a flavor here are some examples of some of my favorite methods:

Wireframe Prototyping -- A wireframe is a narrative prototype, usually created in the beginning of the design process. This prototype shows high-level sketches, visualizing conceptual assumptions about the product structure and general interaction.

Storyboard Prototyping -- A storyboard is a narrative prototype, usually created in the early stages of the software-making process to articulate business and marketing requirements in the form of a usage scenario or story. These stories narrate the user actions needed to perform tasks as specified by marketplace, customer, and user requirements.

Paper Prototyping -- A paper prototype is an interactive prototype that consists of a paper mockup of the user interface. The interface is usually fully functional, even if all the functionality is mocked up on paper. Paper prototypes allow you to test a design with many different stakeholders, including end users.

Digital Prototyping -- A digital prototype is an interactive prototype that consists of a digital mockup of the user interface. The interface is usually partially functional, even if the functionality is implemented by hyperlinking, screen switching and other methods of mocking up actual interaction. Digital prototypes like paper prototypes allow you to test a design with many different stakeholders, including end users. Unlike paper prototypes, digital prototypes can be tested remotely.

Blank Model Prototyping -- Blank models are low-fidelity prototypes produced quickly by user study participants using readily available arts and crafts materials to represent their notions about what an intended hardware/software design could be like. This method is used in the early stages of product design to elicit user perceptions and mental models about hardware form factors and interaction controls in conjunction with a software user interface.

And with the prototyping methods that covers the definition of a prototype. Now was that so painful? Now you understand at least to some degree the richness of prototyping. Instead of being victimized by these dimensions you should be wielding them like a weapon. So hopefully know you can understand the basic concepts of effective prototyping: that a prototype is:

purpose
content
content fidelity
requirements and assumption
prototyping characteristics
defined audience (s)
toolset
method

If any of these concepts are still not clear, I can discuss them in subsequent postings. Next week I will discuss the so-called benefits of prototyping, which probably could better be labelled: the myths of prototyping.

Prototyping 2: What is a prototype, part 1

noreply@blogger.com (Jonathan) — Tue, 16 Mar 2010 11:08:00 +0000

In my last post I discussed what a prototype does. Now here comes a far trickier question: what is a prototype. A prototype turns out to be quite complex, and rightly so. Because to get the benefits of prototyping (the subject of my next post), a prototyper must understand these vital concepts, otherwise you are just shooting arrow into the air.

A prototype is deservedly complex since it is by definition the coming together of many different disciplines. Whether you like it or not every prototype has an either implied or explicit:

visual design
interaction design
technical implementation
information design
editorial content
and my personal favorite: a reason to exist

But those are all vague terms and do not really help you get control of your prototype. And getting control is the point of the definition of a prototype that I want to discuss. This definition will provide you with everything you need to control your prototype, so it does not control you. Likewise, for you non-prototypers, this will also give you enough information to fight what I call the razzle-dazzle effect: a prototyper who over-delivers a slick prototype and uses the wow factor to cover up a paucity of good ideas.

To begin, we need a prototype definition that covers what are the parts that make up a prototype and not what a prototype does (that was covered in the last post).

The Effective Prototyping definition of a prototype

A prototype is a model of a design that is:

utilized for a specific planned purpose
illustrating specific content and fidelity
articulating defined requirements and assumption
specified with prototyping characteristics
customized for a specific audience(s)
created with a specific tool
performed in a specific method

Here is a less verbose but more specific version of the same definition:
A prototype is a model of a design with:

purpose
content
content fidelity
requirements and assumption
prototyping characteristics
defined audience (s)
toolset
method

Below we will discuss them briefly, for more thorough details, you can always consult the full book, Effective Prototyping for Software Makers .

Purpose

A prototype will be created for a specific purpose. Whether it is a proof of concept, or a demonstration of a product’s interaction or a visual direction, it is important to know what the purpose(s) is (are).

Content

Based on what the purpose of the prototype is, you will want to prioritize the content in the prototype.
A prototype consists chiefly of 4 different types of content:

Interaction -- how a user will interact with it
Visual design -- how the prototype will visually appear
Editorial content -- what information will be on the prototype
Information Design/Architecture -- what will be the structure of the information

Fidelity

Generally, only in late stages do you want the content all at a high fidelity. Consequently, a prototyper will strategically set the fidelity of any given content higher or lower depending on what they want the prototype to focus on. The higher the fidelity, the more prominent the content. The lower the fidelity, the more the content will fade into the background.
Setting the wrong level of fidelity is the most common error. It results in discussions getting bogged down on visual design, when in fact the interaction design was the only intended goal of the prototype.
Contrary to what most prototyping texts state you can play with fidelity within a content type. For example you can raise the fidelity on the visual design for the chrome of an application and lower the fidelity of the content in order to discuss the visual structure of a given. You can also de-emphasize a content type completely, for example by showing all text in greeked text format you for your audience to concentrate on the visuals or interactions instead of trying to read editorial content which usually grabs their attention.
However, the issue is more nuanced than it appears. For example, let’s say you want to test the interaction design. Then, if you set the visual design level to lowest and editorial content to lowest fidelity, it will be impossible to really test the interaction: you need just enough editorial and visual design content to test the interaction. Likewise, say for example, the visual design is already finished and agreed on by stakeholders, then there is no real reason not to use a high fidelity visual design.
In general, the rule is, lower the fidelity of the content you are both less sure of and do not want to evaluate. At any rate a professional prototyper should be able to justify their choices.

requirements and assumption

The whole point of a prototype, when used as part of a digital product or service creation process is to validate requirements, or rather separate the requirements from the assumptions. Requirements are some function or feature that is necessary for the success of the product or service. An assumption is something that is presupposed to be a requirement, but has never actually been proven or tested. A prototype usually consists of proven requirements, requirements to be validated in the current iteration and assumptions. In general, the higher the assumptions the more risky a prototype is. Whether something is a requirement or an assumption will help prioritize content and set its fidelity.
I see know the post is over 1,000 words, so let’s stop here and resume with prototyping characteristics next week.

Prototyping 1: What does a prototype do

noreply@blogger.com (Jonathan) — Mon, 08 Mar 2010 07:51:00 +0000

A series on prototyping
In the 4 years since our book on prototyping first came on the scene there was precious little written about the professional way to prototype. Today prototyping seems to be the hot topic, unfortunately most of the current stuff available on the internet only give an isolated tip or trick. What is especially harmful is that most of these articles rush into how to prototype without really understanding what it is. These works are rife with unquestioned assumptions and and uncritical approach to prototyping.

The book I co-wrote with Michael Arent and Nevin Berger remains still the only thorough attempt to understand prototyping. In the coming series of posts on prototyping, I want to make a compact discussion of what a prototype is and how it works from our book Effective Prototyping for Software Makers . This post will it a little more approachable and if you want the full details, by all means you can buy the book

In this series of posts I want to address 3 broad topics:

What does a prototype do
What is a prototype
Raising the bar in prototyping

In this first post I want to discuss what a prototype does. For that I want to use a definition of prototyping that restricts itself to what it does not what it is. For that I turn to a definition from the book “Universal Design Principles” by William Lidwell and others:

A prototype is “The use of simplified and incomplete models of a design to explore ideas, elaborate requirements, refine specifications, and test functionality.”

For ease of discussion, I will break this definition down into its components. First, I will throw out the models business because that goes into what a prototype is, which will be the subject of the next post. That leaves us with the following uses of a prototype:

to explore ideas
to elaborate requirements
to refine specifications
to test functionality

All prototypes attempt to do at least one of the above purposes and usually more than one simultaneously, often without the prototyper even being aware of it. What is essential to know is that the prototype is first and foremost a communication medium. A prototype communicates the above 4 concepts.

Explores ideas

Here the accent is on if the idea is desirable. Prototypes are at their best when they explore abstract concepts or ideas and makes them concrete. It is easy enough for a group of technocrats to discuss their new idea for a killer document registration, yet being able to both rapidly and interactively visualize with a prototype makes the idea come alive and often inspires and informs the whole ideation process. Any software idea can be visualized with a prototype. But here are just a few examples (a fuller list comes in a future post discussing prototyping content):

Interactions design
Application functionality
Visual design
Information design/architecture
Rough concepts and ideas

Among the many means of using prototyping to explore are:

a single prototyper visualizes the idea
a group prototypes through participatory design practices
members of a group each sketch out their ideas as a group
a group brainstorms a prototype with a designer as facilitator

Elaborates requirements

Here the accent is on, whether the prototype is possible. A prototype elaborates requirements by often illustrating what is necessary to actually put an idea into action. For example, and idea of having a running total in web interface when illustrated will make a developer realizes they need Web 2.0 technology. Or it could make the business analyst realize that discounts or other items that effect the total also need to be known upfront or somehow communicated to the end user. Once an idea is explored, software makers often look at a prototype differently. Among the types of requirements that are elaborated by a prototype include:

Business
Organizational
Functional
Technical
End user

Refines specifications

Here the accent is on, whether the prototype is feasible and if so how. One the idea is desirable and deemed possible, then the detailed design comes in. A prototype is often a superior form of specification than a large paper document with lots of verbiage where the requirements are difficult to ascertain let alone visualize. Furthermore, the prototype speaks in the visual language of the product itself and cross cultural and language concerns are not so big an issue with today’s global development teams.
A prototype can be stand alone documentation if it is a totally complete model. Otherwise, often some form of annotation or some lightweight document is needed to accompany it.

Tests functionality

Here the accent is on evaluattion, for example whether the prototyped design is usable for the end user.
A working prototype (paper, digital or whatever form) can be shown to stakeholders and they can test it to see if they can work with it. If it works the way it should or they way it needs to. This way corrections to the design will cost no redevelopment costs.

Summary

So in a nutshell, this covers what a prototype does. In essence it communicates four things:

to explore ideas -- is it desirable?
to elaborate requirements -- is it possible?
to refine specifications -- how do we do it?
to test functionality -- does it work?

How and what it communicates will be discussed in my next post on what a prototype is. In that post I will discuss the characteristic parts of a prototype. Understanding these characteristic parts allows you to control how your prototype will come across to your audience.

UX Strategy III: The UX Declaration of Independence from Engineering

noreply@blogger.com (Jonathan) — Tue, 02 Mar 2010 13:46:00 +0000

(Co-written by Thomas Jefferson, I hope my non-American friends will indulge an American metaphor of the declaration of independence.)

When in the course of human events, it becomes necessary for one profession (UXD) to dissolve the bonds which have connected them with another profession (Software Engineering), and to assume their own powers, separate and equal from other professions to which the Laws of Nature entitle them. A decent respect to the opinions of software engineering requires that User Experience should declare the causes which impel them to the separation.

We hold these truths to be self-evident, that all products are endowed by their creators with user experiences with certain unalienable Rights, that among these are usability, satisfaction, and business feasibility. Furthermore the user has a right to a user experience which is derived from the entire company/organization not just what is technically feasible at a given moment.

That to secure these rights, User Experience professionals are engaged by companies and businesses. These professionals derive their just powers from a professional integrity that must not be compromised, otherwise the User Experience design loses whatever rights they have to exist.

Software Engineering as a process has had a tyrannical effect on the User Experience professional, forcing them to shorter and shorter deadlines with less and less available resources that the point is reached that UX Professionals often find themselves going through motions rather than truly designing professional products the way they are truly capable of creating them.

The history of Software Engineering processes are a history of repeated injuries and usurpations of UX terrain, all having in direct effect the establishment of an absolute Tyranny over this profession. To prove this, let Facts be submitted to a candid world:

Development methods are continually shortening their design process and their delivery deadlines

This makes it impossible to do a thorough and adequate design process, forcing us to take all kinds of irresponsible and inappropriate short cuts.
Specifically, Agile development processes attempt to preclude any upfront design or research as good UX processes demand

Development do not use UX metrics as a measure for their success

Consequently there is no business case for following UX best practices

Development keeps the UX bar purposefully low so that UX accountability is

non-existent -- even when it is clear that products are failing because of their poor user experiences
an afterthought -- the product is a success or failure and after the fact UX is blamed or ignored
an anecdote -- the arbitrary story or urban legend of use becomes definitional for the user experience
unprofessional -- as long as the bar is low, poor UX design will yield equal results making the establishment of UX best practices very difficult

Development’s near fetish-like fascination with a release puts artificial blinders on the UX processes, resulting in:

assuring structurally sub-optimal results
cutting corners when it really is not necessary
giving undue credence to an artificial argument against UX additional processes
obscuring the value of user experience design by forcing it into the release focus of software engineering.

UX quality is now reliant on the kindness of strangers, that will say the extent to which a Software Engineering team is or is not enlightened to the value and processes of User Experience Design.

We, therefore, the Representatives of the united User Experience Designers hold that instead of working under the hegemony of engineering, User Experience activities should work in coordination, not in tandem with Software Engineering.

Among the ongoing process which User Experience should be working on independent of Software Engineering include (partial list for the longer list of UX processes see the previous post in this blog):

User Research
Design Research
Requirements gathering (SE’s are needed for technical requirements but that is only one part of the whole requirements picture
Product design
Conceptual design which may cover multiple products/channels and multiple releases.

Places where software engineers and user experience should closely together is

translating a conceptual design to a specific product release cycle

product definition
product detailed design
product design reviews and iterations

mentoring developers through a product release
evaluating software engineer work for fidelity to UX concept using appropriate UX metrics
release planning

Software engineering in turn should act as mentors in the UX processes assuring technical feasibility for short and medium term are tracked and noted. In this way Software Engineering, Product Management and User Experience are truly equal partners in the creation of great products and product experiences.

Signed 2 March 2010

UX Strategy II: About the iterative diagram: What is it?

noreply@blogger.com (Jonathan) — Tue, 16 Feb 2010 14:53:00 +0000

In the second part of this Strategy discussion, I will concentrate on the Strategy diagram from the previous post. This post will cover what the diagram is and who is it for. There are more issues than that to be complete, but I can always add an additional post if there is a desire to read more detailed information about it. [Note this post, like all my posts are revised based on user comments or feedback.]

Just to review from my previous presentation (see post below): this diagram is a way of anchoring the design process to key strategic activities thereby assuring both a true design process as well as a strategic execution of this User Experience design process. The alternatives that are in vogue now are either

seeing the User Experience as a bolt-on to engineering processes

'Bolt-on' being American for: just embedding a UX process in to a software engineering process
A software engineering process which is already cumbersome and unpredictable
In general adding design process to software engineering process is like forcing the square peg into a round hole.

or at best its own independent process that mimics a software engineering process

Where the UX process eventually turns into something that looks like some variation of

a waterfall
incremental design
Some other variation of the straightest line between two points approach

The above points coupled with my belief that software engineering process is a contradiction in terms pleads for the necessity of this new diagram.

Figure 1: the UX STrategy Iteration Diagram

In general terms you can think of the diagram as a planning tool one can talk over with a program manager or client or even all key stakeholders during a workshop. You can also think of it like a hula hoop, somewhere, anywhere in the hoop you can cut it flatten out and make a project manager or software engineer happy to see a simplified overview of what activities you are going to do for the current cycle.
These diagrams can be stacked on top of each other and connected at key points to plan multiple user experiences among different channels, products or services. This would allow planning and illustrating hos a mobile product project can inform a web application project. Likewise a strategy iteration can inform a tactical one, etc.
The strategy diagram and the planned activities should be revisited after each activity and see if it assumptions are still valid or if it is time to iterate the activities. In this way the very strategy is iterative just as the User Experience. But before going into too much details, I want to discuss two points here:

What is the diagram
Who is the diagram for

What is this diagram

This diagram is an attempt to create a model for User Experience Strategy and in so doing create also an instrument for both:

understanding User Experience Strategy
planning an User Experience project for your
company organization
or heaven forbid for a client if you are one of those charlatan UX consultants like me.

The diagram consists of the following (names are provisional):

Circles
Elements
Activity
Properties

Circles

The circles represent iteration cycles. But iterations are centered on an element or two or more, but they have iterative effects also on its neighboring elements and then even ripple effects through the entire UX element landscape (see below). Even when an iteration confirms an already existing UX element it still strengthens that element and thereby changing it. The circles show the interdependent nature of the User Experience as an expression of a series of elements.

Elements

The elements are a major area of the User Experience, usually with one or more associated deliverables. In order to qualify as a major element in the User experience it must meet the following criteria”

Plays an essential role in UX products, services, and other expressions (brochures, ads, etc.).
Major risk to the resulting product and/or organization if this element is not ready.

With this definition it speaks for itself that each project/company/organization may have a slightly different diagram, but complete coverage is essential.

We (my colleagues at Stroomt and the helpful people who kindly mailed in their suggestions) identified a generic set of UX elements, namely:

Mission Statement
Vision
Goals and principles
Channels
Brand design
Business Case
Business Plan
Requirements
Define product/service(s)
Conceptual Design
Detailed Iterative Design
Evaluate and refine design
Release product and plan for next iteration

Each of these elements must have a sufficient level of maturity and stability in order to release a product or service to the world. The User Experience Strategist is obliged to review the state of each of these elements. It is not the job of the User Experience Strategist to be the person who delivers or executes on these elements, UX is by nature multi, or I would say macro-discplinary. The UX Strategist is a facilitator first and foremost.
Figure 2 UX Strategy Diagram with activities

Activities

If these elements are not in an acceptable state then activities should be planned to bring them up to the appropriate level. It is not the User Experience Strategists job to perform all of these activities, or even any of these activities. Like the elements, the activities also require many different disciplines. The UX Strategist may be able to assist and find and support the right people to perform the activities. However the strategist is primarily concerned that all the information is available, up to date, stable and mature.

Properties

Both activities and elements have properties, these depend on the need of the organization, but can include things such as:

Staffing
Resources
Start and end dates
Deliverable requirements
Budgets
Etc.

Who is this for: Multi-disciplinary vs Macro-disciplinary

Last topic for this week: Who is this diagram for?
Well definitely not for the faint of heart.
The User Experience Strategist, Designer, Project Sponsors, Program Manager, Project Manager are those most to gain from getting this overview as well as wanting to be able to plan on a macro level. But the fact is, this is one way of getting all of the multi, or rather Macro-disciplinary team literally on the same page about who is doing what and how it all fits together.
I use the term Macro-disciplinary because unfortunately too often the word multi-disciplinary is bandied about to mean multiple disciplines without recognizing that these are mostly separate people. Most UI multi-disciplinary projects means the designer--or whoever the UI one man band is called-- is up late at night and weekends. They are also often caught talking to themselves in a desperate attempt to bring in another disciplines or perspective into their work. By Macro-disciplinary I want to show that it is impossible not to include many talented people with many complementary, but more often contradictory perspectives.
This last concept: contradictory perspectives is essential to every successful design project I have ever worked on. This diagram allows these contradictory perspectives to elegantly be laid plain in a map. It also allows you to plan activities for incorporating those perspectives back into the larger UX iteration so contradiction are resolved rather than brushed under the carpet.

Next week the UX Declaration of Engineering Independence.

UX Strategy is different than UI strategy Part I

noreply@blogger.com (Jonathan) — Tue, 09 Feb 2010 13:31:00 +0000

[Note: First of three parts.
Next post Part II, a detailed discussion of the diagram below.
Part III. the UX Declaration of Independence from Engineering.]

Here is some big news: UX strategy is not UI strategy. This must be big news since the two seem identical in how they are practiced. There seems to be a fundamental flaw in our ability to make a difference between UX practice and UI practice. However, there does not seem to be a shortage of differences between defining the two that is covered and almost written to death (so I won’t cover it here if you are interested in that wikipedia is a fun to place to start). Yet when the rubber meets the road, most strategists, designers, usability engineers and other nefarious UX practitioners like me, explain a process that looks awfully similar to UI design best practices anno 1989.

So here is some other big news, amazing news for everyone in the UX business: design is not engineering. What? You knew that already? That is strange since I am yet to see a single UX strategy or UI process that is actually iterative let alone independent of a development cycle. Oh I am sure they are out there, its just the secret sauce of a chosen few who really get it right? Not likely. It seems to me that most people who claim to be UX designers are in fact UI designers.

That fact is so few people understand UI design that no one really notices when it goes by another name, especially one that sounds more expensive like UX design does. The reality is UI designers have nothing to be ashamed of: it is one of the most difficult and nuanced professions due to its inherent multi-disciplinary focus. The inclusion of Interaction Design, Information Architecture, Graphic Design, User Research etc is all classical UI design not User Experience design. Because of this confusion, too many people are spinning their wheels in UX design when they are really discussing the important and essential issues around UI design. Here I would like to discuss my take on the practice of User Experience design.

Here is the good news: there is a solution
I recently gave a talk in Utrecht UX Strategy. I include the slides with this post.

UX Strategy as told by the paintings of Jan Steen

View more presentations from Jonathan Arnowitz.

I think my most important point in that talk is a real iterative UX Strategy that is based on Design practice not software engineering practice. A subsidiary thesis to that talk could be: if you are fixated on how UX fits in a development method (e.g. Agile, RUP, Waterfall, etc.) than you are not a User Experience Designer at all but a UI Designer. There is no shame in doing UI design but then let’s not muddy the UX waters with it.
Moreover a real UX strategy should not only not resemble an engineering process, but should also be independent of it. To not accept this reality, is to concede the hegemony of engineering in both process and decision making on UX. That hegemony is not the reality except in engineering driven companies. UX is inherently strategic, whereas UI design is inherently tactical requiring a close association with engineering toward realization. Perhaps it bears noting that not all User Experiences have UI’s or have UI’s as their most important component. UI Design is in fact the place where Engineering and UX meet but UI design is not the end of the UX strategy, it is rather one of its many expressions.
The presentation above identified 3 goals of UX Strategy, they are not the three goals rather just the three I am concerned about:
1. Keep the client/business/organization focused on their business goals
2. Keep the Design and Technical teams focused on the Conceptual Design
3. Provide a predictable repeatable process
4. Maintain a UX reality check that is at once iterative, open-ended, and reliant on solid analysis (some may call this trusting one's gut) as any good design process should.

A good UX strategy is therefore better represented by a loop, as it is in the slide presentation above. A loop unlike any engineering process. A loop with no beginning, because we invariable enter at some arbitrary moment: when we contracted, when we were hired, etc. It reflects reality that we start anywhere in the process and it also reflects the interdependent nature that one step will invariably influence the other, if not for this product than maybe the next. Moreover there maybe multiple simultaneous iterations occuring at the same time.
A much improved version of the image from the presentation would look like the image below. This is what looks like a ferris wheel approach. Each node on the wheel below represents a common analytical element of a User Experience. This is a 1.0 release so hopefully some helpful comments will come forward and allow me to iterate on it.
This image recognizes the interconnectivity of an organization to the user experience and the ripple effect of one UX element will have on the rest. One error in the drawing is that everything appears to be the same weight and magnitude, which is not true. A gear like metaphor would be better. Each element, mission, goals, etc. could be represented as a gear that turns the larger iteration gear. A gear, whose size could change with each of the UX elements could have a bigger or smaller gear depending on the character of the company or organization.

Each element can have a series of activities associated with them. These activities will help continue the iteration cycle. The activity can vary with organization, its needs and (the weak link in the chain) the talents of whom they hired to iterate the User Experience.
Another important aspect to the drawing is the iteration cycle does not end. There is no ultimate goal with which life starts and then ends. The reality is relases/successes are temporary and no sooner is one goal achieved than the next goal must take over, the next quarter’s numbers must be met, the next new thing must be created to stay ahead, etc. In this way the UX evolves throughout the life of the organization.
A few examples of a chart completely filled in are given below. The first example is a complete cycle refresh. The next one is a product oriented iteration. The last one an organizational iteration with a proof of concept product at the end of the iteration cycle.
After the images below i welcome your comments. I will refine the presentation and the drawing (maybe a good visual designer would volunteer?). The the drawing will start to live, and will it ever be finished? I hope not, or we will be out of a job.
UX Strategy wheel template

UX Strategy wheel completely filled out

UX Strategy wheel completely filled out for a product oriented iteration

UX Strategy wheel completely filled out for a company iteration with a proof of concept product or service

Agile & UX

noreply@blogger.com (Jonathan) — Wed, 21 Oct 2009 06:26:00 +0000

The presentation below started out as a short talk at the UX Cocktail Hour. However, the presentation has been gaining in popularity. So I decided to post it here. Because it does not really stand alone to people not at the presentation some misunderstandings have resulted. I will record the presentation with audio, but in the mean time, let a few words here suffice:

It is a design-centric view of Agile, or rather a War weary design centric view of agile.

The main point being that Agile is not a development method as much as it is a way of setting aggressive deadlines. What happens in one Agile project does not predict success for another. Instead the designer needs to be Agile in figuring out how they can best fit in. However, that agility should not extend to your design process. Designs still need to be well thought out concepts not something grown together in piecemeal increments.

The bottom line message (found on the last slide) is to be truly successful in Agile, you need to follow your own design process but be intimately involved in the Scrum process, preferably as a Scrum Master. This is essential for maintaining an overview of what is going on with your design. At the very least, own the user facing stories/requirements in the product backlog.

And Sherlock Holmes and Dr. Watson were meant to personify regression testing.

Agile & UX

View more presentations from Jonathan Arnowitz.

Halcyon days at the EuroIA Conference

noreply@blogger.com (Jonathan) — Thu, 01 Oct 2009 07:29:00 +0000

Last week I attended the EuroIA conference. I was there primarily to give a talk with my former Google colleague, Greg Hochmuth, on a project we did on on-line privacy. To be honest I had low expectations for the conference, thinking it was not going to be very professional. That was my estimation of the IA movement in general. I favored the more rigorous CHI model. This reliance/faith in CHI is why I have been working so hard to bring practitioners into CHI with the design track work and of course the DUX conference series, etc. I assume that CHI was where the interesting professional UX work would be done. I did not expect any such thing at an IA conference, which I thought was too narrow and too niche to be interesting.
I was wrong and closed minded, both of which I find annoying.
I was quite surprised to attend a very fine conference with a strong practitioner focus with competent representatives from industry giving case studies and thought provoking discussions. There were, of course, more than a few missers. However, when you attend a CHI conference misser you really wasted your time at some inapplicable pedantic presentation. These were all interesting even if not earth shattering.
I was also pleased to see that the attendees had a kind of willful confusion of IA with UX. Eric Reiss one of the leaders in the conference series said early on he was proud that they would have no debates on terminology or definitions.
What is IA
It seems to me that IA (Information Architecture) and HCI (Human-Computer Interaction) are two ways to achieve the same effect. One is information driven, the other is interaction driven. Both strive for but don’t quite achieve UCD. To borrow a Mahler analogy, these two movements seem to dig from opposite sides of the mountain to reach the center.
Setting the stage for the conference was an interesting case study keynote given by Scott Thomas on his work for the Obama presidential campaign web site. A refreshing talk, one would probably never hear at CHI, charting the work he did as both designer and web developer and IA for one of the most successful and high profile web presences.
It was clear at the conference that there are those who do specialize in IA and don’t touch interaction design with a ten foot pole; however the majority seem to blissfully switch between IA, ID, and UX designer labels based on what will get them the job or the most influence. The resulting conference content is interesting and competent, usually not pedantic (there were a couple regrettable forrays into pedantia--oh I am being pedantic aren't I?). I will hasten to add that probably 10% of these presentations would have been accepted at CHI.
CHI Bashing
Not that I am in anyway bashing CHI (well I guess i am sort of). CHI continues to be dominated by Academia, it is its reason to exist. So it makes sense that more practioner oriented organizations thrive and offer better conference experiences like EuroIA, SXSW is another such conference. However, there are some design heavy weights very active and present at CHI. People like Bill Gaver, Bill Verplank, Bill Buxton--hey are all of them named Bill? So I guess we should also include Bill Card and Bill Dray...
Still going to a CHI conference is daunting and if you do not stick to the Design or practitioner focussed papers it is really hit and miss. Then there is also the unfortunate academic who strays into a design paper and lambastes a practitioner for not holding double blind studies on a project with a limited client budget. Ah, it is always embarassing when people can't check their egos at the door.
So, it is good there are several credible alternatives to CHI. I guess this means I need to attend the next IA Summit and see what that’s all about. I don’t think I can take anymore good stuff...
This profession
In the end, I had a friendly familiar feeling at EuroIA. A feeling like I had met these people before. It seems that regardless of whether you are at CHI or EuroIA or UPA or wherever, people of our profession(s) share this common empathic passion for our stakeholders. This makes us a particularly caring and sympathetic tribe.

Measuring the User Experience

noreply@blogger.com (Jonathan) — Fri, 04 Sep 2009 12:53:00 +0000

This weeks post is a review of the book Measuring the User Experience by Tom Tullis and Bill Albert. From time to time other book reviews will follow.

Why a book review

The current state of books on UX is deplorable. Many UX books can’t make up their mind if they are about a given subject or the UX world according to Garp. Just looking at my UX bookshelves, I notice there are, for example, many books with authors who have a narrow or focussed expertise. These authors write books supposedly over a narrow subject, which they sustain for about a chapter or two before they deteriorate into their own homemade version of the User Centered Design process that has little if anything to do with the subject of the book they intended to write. The result is a book with grains of truth in a stew of platitudes. A review of just three books one claiming to be on prototyping, one on designing and another on UX communications, reveals that all of these books cover more or less the same material such as user research, task analysis, persona’s and prototyping; but it does it in such a way that they use both conflicting terminology and conflicting methods.

My more ideal UX books are those on a subject and stick to that subject. They explain their topic in a way that is process independent so that they can plug into whatever processes companies or organizations utilize. The fact of the matter is that no two organizations adopt the same software development process. What they all have in common whether they are called agile or waterfall, iterative or serial, is that they are all machiavellian. Therefore if a book's material cannot fit into the current machiavellian software development processes, then the book is largely worthless; even if entertaining (though probably not as entertaining as E.M. Forester).

I think one of the best services I can do then is to help people navigate around these literary cataracts and start a series of book reviews. These reviews will try and highlight the best of the UX literary corpus.

Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics by Tom Tullis and Bill Albert

I want to start with one of the brighter lights in our industry Tom Tullis. I have often wondered why he had not earlier written a book, given the high quality of contributions he has made to our profession. Well the wait is over.

It's true it is a book on usability metrics. Now I realize there are some people who hate metrics. These people particularly hate any accountability for their design work. I can’t tell you the hate mail i received, even from large design firms, when as Interactions Editor we did a special issue on measuring usability that was guest edited by Jeff Sauro. Well, I purchased Measuring the User Experience (MUX if you will) expecting a more thorough version of that special edition that went into the statistical significance of usability testing. I was in for a very welcomed surprise: this book does not just cover summative usability statistics but many different ways to collect user experience metrics and the also discuss proper analysis techniques.

The book empowers the user to make the right decision regarding what methods you can use and what you can expect the metrics to be able to tell you or not tell you. As the book states metrics can help you to answer questions such as:

Will the users like the product?
Is this new product more efficient to use than the current product?
How does the usability of this product compare to the competition?
What are the most significant usability problems with this product?
Are improvements being made from one design iteration to the next?

This is a refreshing change from just looking at time on task, error rates and task success rates. Though of course these play a role they are but ends to the means of answering these larger questions. Furthermore, the book also points out that there is also an analysis step that can greatly alter the seemingly obvious findings.

I cannot tell you the amount of time and money I have seen wasted as perfectly reasonable and wonderful user research was conducted, only to have its results obfuscated and mutilated beyond use. This book will not just enable the usability tester or researcher to avoid such mistakes it also empowers a project manager to see to it that a development project designs the solid usability study that will fit in the goals and needs of the development team.

In their discussion of designing the right usability study. The authors guide you in choosing the right metrics.

First you need to establish if the goal of your study is what the goal of the user’s are. Then on that basis you can look at which metrics, the authors identify 10 common types of usability studies:

Completing a transaction
Comparing products
Evaluating frequent use of the same product
Evaluating navigation and/or information architecture
Increasing awareness
Problem discovery
Maximizing usability for a critical product
Creating an overall positive user experience
Evaluating the impact of subtle changes
Comparing alternative designs

Then, a key issue they discuss is looking at the budgets and timelines, aka, the Machiavellian business case for the study. Then you can tailor the type of study: how many participants, will it be tests, or reviews or focus groups or a combination thereof.

In the conduct of these studies it is also important to track the right metrics. Tullis and Albert identify the following types of metrics:

Performance Metrics -- time on task error rates, etc.
Issue-based metrics -- particular problems or successes in the interface along with severity and frequency
Self-reported metrics -- how a user can report their experience with questionnaires or interviews
Behavior or physical metrics -- facial expressions, eye-tracking etc.

It handles these metrics as they should be as part of an overall strategy not favoring one over another as being innately superior. All too often usability testing consultants are one trick ponies, prisoners of whatever limited toolset they happen to have learned.

This book allows the user to assemble all the needed metrics across types to achieve a more holistic view of the user experience, or at least sensitize them that they are not looking at the whole picture.

What is also amazing is the focus and discipline in the book. I think many other authors would not be able to fight the temptation to then expand the book to include how to perform the different types of evaluations, usability tests, etc. These authors acknowledge there are already books that cover these other related aspects and keep their emphasis purely on the subject matter of their book: measuring the user experience.

Yes the book does also get into statistics and evens hows you how to do simple straightforward statistical analysis using that panacea to the world’s known problems’ excel (but that is next week’s topic).

And just in case your wondering the usability score for Amazon is 3.25, while Google’s is 4.13 and the Apple iPhone is a mere 2.97. While the web application suite I just finished designing got a perfect 4.627333.

Confusing a Heuristic with a Moral Imperative

noreply@blogger.com (Jonathan) — Wed, 29 Jul 2009 07:06:00 +0000

Heuristics are excellent assistance in identifying potential problems with a given user interface design. The trouble lies when people come to rely on these as the sole input, that somehow they can come and overtake the more rigorous and far more accurate methods of evaluation. So please don't read below as being anti-heuristic but rather anti-misuse of heuristics.

I have been working more and more with consultants and pseudo-designers who have been working on evaluating web applications with a ton of heuristics in their hands. I can hear them clear across cubeville with clipboards in their hands:

"This is terrible, you are inconsistent between these pages, those pages ignore web standards, these other pages behave differently than the others, and oh my gosh look at all these unnecessary graphics, rip these all out. Get rid of the background coors, and ugh those button colors!"

Concept and user groups can trump heuristics

The fact is there could be a valid reason for violating every single one of these heuristics. Worse yet, there are these type of evaluators who without so much as learning the context go in a tear apart a site for violating, standards, UI conventions and other heuristics of all sorts.

A well defined and innovative concept will often require breaking a few rules. Moreover, if a concept is tailored to a specific user group, who is not the evaluator, then all the heuristics are almost invalid.

Heuristics are defined as (according to my Mac dictionary, and why should we doubt Apple?):

Heuristic (/hjʊˈrɪs.tɪk/) is an adjective for experience-based techniquesthat help in problem solving, learning and discovery. A heuristic method is particularly used to rapidly come to a solution that is hoped to be close tothe best possible answer, or 'optimal solution'. Heuristics are "rules of thumb, educated guesses, intuitive judgments or simply common sense.

Well here are some of these so-called common sense rules of thumb with some food for thought to think about along side of them, I am using the list from Jakob Nielsen's site, just to pick 10 basic one (http://www.useit.com/papers/heuristic/heuristic_list.html) This is not to pick on Jakob, as the point here is to discuss the pitfalls when heuristics are used as the sole means for evaluation, as such every heuristic can be picked apart and discredited, these are just 10 examples:

Heuristic	Justification (From Nielsen)	Yes but...
Visibility of system status	The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.	Maybe the user doesn't and shouldn't care. This heuristic assumes a user population actually cares about what is going on. Many user's could careless unless it's going to cause them a problem. You should have some basic trust built with your users and that trust may mean only informing them in the case of a problem, or handling the back end status problems yourself.
Match between system and the real world	The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.	Unless the purpose of the site is teaching the user a domain, or new task. An example would be Google Ad Words, where a novice user does need to learn some basic Advertising terminology or the advanced features will be lost on them.
User control and freedom	Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.	This heuristic seems to justify poor design. User control and freedom come more from safety is more than just redo or undo, its the ability to let the user explore and play around with the system. This is done through facile interaction design, a heuristic I have never seen listed.
Consistency and standards	Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.	This assumes 1. the user has no other reference point than platform standards 2. the platform has standards or usable ones Again this justifies lazy design. Standards are a fall back (I say this as someone who has written UX standards for 3 major software companies); the conceptual design should be leading.
Error prevention	Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.	Here is a useless Heuristic. What is an error? One man's error is another man's exploration. Maybe you should enable errors?
Recognition rather than recall	Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.	Indeed the memory load should be lightened for a user. However the better way to do this is to employ well established visual and interaction patterns. Worse, this explanation can be very misleading for the naive reader. Indeed I have experienced many a designer and developer use this heuristic explanation to 1. attack progressive disclosure and 2. To create a ridiculously busy screen throwing all functionality with equal visibility into a "one-stop shopping" kind of screen. Or worse a screen with a huge amount of text explaining how to use the screen. All of which are from a cognitive ergonomic perspective completely unusable.
Flexibility and efficiency of use	Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.	Building in redundancy to support multiple styles of interaction would be a better way of putting this. However, this needs to be seen in the context of a broader design concept. For example, there is often this designer fetish for drag and drop, when often it is only the designer who wants to perform this action. Also, implementing drag and drop in one place, invites the user to try it everywhere and very annoying when it does not work as they expect it to. So pick these accelerators well and not just for their own sake.
Aesthetic and minimalist design	Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.	The explanation is here at odds with the heuristic. The heuristic seems to cry for everything to be a Scandinavian styled minimalist design; whereas the explanation goes on about text. The visual design should leverage the brand and ability to communicate. Gratuitous graphics are supposedly bad unless the delight the target users (think of Google's doodles on their home page). As far as minimalism, I recall Tufte who said anyone can pull information out, how you pack information into something and keep it intelligible and usable is the real challenge.
Help users recognize, diagnose, and recover from errors	Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.	My only problem here is the "precisely indicate the problem." I am sure Jakob did not mean go into gory technical details of the problem, but rather concisely describe the issue. E.g. "Your data was not saved." not "Your data was sent to the application layer and experienced a time out longer than 3 ms and the system sent back the data in an unusable format." My formula for error message writing: "Short sentence what happened. (forget why) Short sentence how to fix it. A link can be added "Learn more" or "Why did this happen to such an undeserving user as me" for the morbidly curious.
Help and documentation	Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.	Far from apologizing for help we should revel in it.Help and documentation should be electronic and in context. For example, micro help (a question mark icon or What's this link which work on mouseover or a small popup) often assist the user without interruption.

The mythical 80/20 rule

noreply@blogger.com (Jonathan) — Tue, 07 Jul 2009 12:57:00 +0000

In a sad day for most digital products and services, an italian economist, Vilfredo Pareto, observed that 80% of Italy's wealth was owned by 20% of the population. From that economic observation has come a torrent of the most far flung interpretations of a non-existent 80-20 rule. There is no 80-20 rule. There never was and never will be. Yet so many developers, designers, product managers evoke this mythical rule to justify the most outlandish pipe dreams, shoddy work or just plain laziness. Which is a pity because it ruins the credibility of a principle that in 20% of the circumstances can be 80% helpful.

The crux of when the 80/20 principle is helpful is when you need to fend off perfectionists. The the 80/20 principle helps you to illustrate that a minority of factors can result in a majority of effects, the aka 'biggest bang for your buck.' But how can you tell when the principle is being used wisely or not.

80/20 Pipe dreams
Some G-d forsaken Gui Guru once said that 80% of screens could be driven by templates; while only 20% of the screens needed to be designed. This completely unsubstantiated drivel has lead to many efforts to "automatically generate a UI". It has lead to millions of wasted dollars and development effort in worthless tools and idiotic processes all aimed at designing without designers. If the Guru was right then about 80% of the screens could be generated, reasoned the technocrati, you hit the 80/20 rule and you applications will be fine except for 20% of the time. Some more thoughtful product managers then would hire in an army of designers to cover the 20% they thought 'really needed design.' But even then, as one Product manager in just such a project told me, in how own pipedream: "I want you to design templates with such a narrow path of movement that a designer can only make the right choice."
The reality of the matter is more along the lines that 80% of a given screen could be generated while 20% needed to be designed, but oh the devil is in the details and often that 20% is where the most difficult design challenges lay. Therefore, the 20& should end up driving the other 80%, not the other way around. [Never mind the fact that this pipe dream totally negates the necessity and power of the conceptual design (see It's all about the concept).]

80/20 Shoddy work
Often someone will deliver (or even ask for) 80% of the work they really need to get done. This is usually done to purposefully keep the quality low. Example. A software engineer asks the designer for rough documentation that is quick and easy to read, just giving the 20% key interactions and let the 'no brainers' to the engineer himself. This assures the design bar remains low. With it this low shoddy work can triumph the design goals being so low. The design fails to deliver but it was set up to fail and no one even expected it to succeed in the first place. This way the technology can triumph, reason the developer, while the design has a systematic back place.

80/20 Laziness
All to often a designer themselves will end with a rough sketch and miss some of the finer details of the design they need to deliver again claiming to deliver according to the 80/20 rule. Often the excuse is: "No need to over deliver those developers won't design it to spec anyway." Or: "Stuff always comes up during development it will change anyway. I will just give them 80% and give them 20% margin to play with." This is pure laziness. As any good designer will tell you that the devil is in the details. Or as some of the better desigers have pointed out G-d is in the detail; because those small details are often what separates an ordinary design from a truly excellent well thought out design. That last 20% is again, the last thing you want to leave to a developer or other non-designer. Furthermore, those gnarly details you have solved will go a lot further in helping developers improvise when they have to than if you just leave them a blank space to fill in all on their own.

80/20 rawks
The 80/20 principle works excellently when you need to stop someone going off into the weeds of perfectionism. The software should be bug free. The software should please every user to do everything. The software should have perfect tests. Anything that reeks of perfectionist is liable for the 80/20 rule. For as we all know at one moment, the waters can get murky. e.g. one user's bug is another user's feature. Just make sure whenever some pulls an 80/20 on you, or you pull it on someone, that you have an objective measure to back up your 80/20. Yes, 20 percent of the people own 80% of the wealth. Yes, if we provide 20% of the functionality we will make 80% of the users happy as we can see in these usability tests, etc.

Usability Engineers vs Designer: the process problem.

noreply@blogger.com (Jonathan) — Tue, 30 Jun 2009 08:16:00 +0000

Another week has come (mine starts on Tuesday, you can do those things when you live in Europe). More and more, we see problems surfacing not from having the wrong people in place but rather the wrong process. My previous post discusses the rampant wrong process with prototyping. Here I want to touch on the process issues with usability testing and design. I want you to consider this familiar and completely unnecessary scenario:

A Designer works on a conceptual design with the customer. Then he works out a detailed design into a prototype that can be tested. So far so good. But what goes wrong is that the Usability Engineer is often disconnected to either the design concept or the detailed design. The usability engineer ends up suggesting new designs that totally contradict the conceptual design. The designer is gone. The engineering team implements the changes and the result is a Frankenstein's monster that despite the best UX resources, fails in the marketplace.

The obvious problem is the process disconnect between Designer and Usability. And the problems are serious. I want to discuss two aspects to these problems and how to resolve them: namely how False Negatives in a usability test and how to deal with Usability Engineer's design advice.

False negatives

When usability testing is conducted without input from the designer, this can lead to many false negative issues in the usability test. Examples of the errors that can result include:

Early tests will report usability issues with conventions that a user is expected to learn over time or with a different task flow than being tested.
Early tests especially lower fidelity ones may not catch learnability/system feedback issues due to lack of visual fidelity needed to communicate with the user
Test moderators, not knowing the underlying concept may inadvertently introduce the topic or task in a way that is at odds with the design, thereby confusing the test subject.

This list is just the tip of the iceberg. These negative side effects are completely avoidable by making sure the Designer and Usability Engineer work together on the Usability Test script, identifying tasks and their importance. Also the task order when that is appropriate for a test (for example one step is a required gateway: e.g. sign up). Also let the Usability Engineer attend some of the conceptual design sessions and, OMG let them participate in the conceptual design; so they gain a thorough understand of it. Conversely, Designers should observe the usability tests whenever possible. The tests themselves can be so much more inspiring and vivid than even the best written report.

Usability Engineer's design advice

It is an expected part of the Usability Engineer's work to include not just data and analysis; but also design advice or alternative designs. This does not need to be a problem. But without setting expectations, the innocent Product Manager or software engineer confronted with new contradictory designs can quickly conclude that the UX profession is a screwed up group who cannot make up their minds.

Among the possible problems with blindly taking Usability Engineer design advice is:

Designs may not be an ideal solution for the problems they have discovered
Designs often recommend things that will cause usability problems elsewhere by introducing conceptually non-standard interactions
Designs ignore larger issues since their advice focuses on the testing and not the larger issues (e.g. Business and Technical requirements which may lead to a different solution than suggested).

A common example of this is when the Usability Engineer suggests something that is technically impossible for the requirements or constraints.

These and other issues with the Usability Engineer design advice harms everyone's credibility both designer and usability engineer. This is not to say that Usability Engineers should not give their advice. But it is absolutely important to set the right expectations. Usability Engineer design advice should be viewed as input to the problem not the solution. If the Usability Engineer includes the design rationale this will often give the vital information for coming up with a more ideal solution.

The design rationale should enumerate the objective reasons for the alternative design. This allows the designer to bridge the problem design with a solution based on objective criteria instead of personal taste.

[Objective information is one that either refers to the usability data itself (e.g. only 2 out of 12 users understood this command) or conceptual data based on requirements, (This design does not appeal to our target users or is not constant with the image/branding of the company). Both types of information can lead to a solution. Comments like, "I don't like that color" or "It doesn't look right to me." do not lead to workable solutions.]

Usability data misinforming design

Usability data is rarely communicated with the limitations or short-comings in the data and this is a real pity. All too often a usability engineering report reads like a set of demands and commandments without stipulating under what conditions this advice or anaylsis should be given. Things like the significance, persistence, sampling issues, etc. are often underplayed. Again a faulty process is the problem. Many usability engineers are under pressure to work quickly and also find dramatic and significant results. This can put a Usability Engineer between a rock and a hard place: asked to review a product with three of Janitor's friends and then come up with a list of "just the most important recommendations." Ah if life were only so easy? Yet we are constantly being put in this position. The client maybe always king, but findings that can include a little context setting would help the end-users of the usability reports.

Lessons learned

Designers and Usability Engineers should insist on working together in projects. Meaning the Engineer is available during the concept design phase and the Designer is available during the testing design phase. (With iterative testing the designer must be available with each design cycle.)
The customers should require design and usability engineers to work together. This will often require the usability engineer to come in early for 1-2 days in the conceptual design phase before their main work begins week(s) later. (Yes that also means if the engineer is an external contractor, the customer must pay the Usability Engineer for this work.)
Customers should also realize the usability engineers do not provide solutions they propose challenges and problems that need to be solved.
Usability Engineers may be great designers or maybe crap designers but as long as they include objective design rationale for their proposed solutions they will always be helpful

Prototyping software tools that work

noreply@blogger.com (Jonathan) — Fri, 26 Jun 2009 07:16:00 +0000

The Blogosphere is full of posts over what is the best prototyping tool. This fight over Axure, Flash, Dreamweaver, PowerPoint, etc., is misplaced and looks at prototyping as something that is purely a single concrete one size its all activity, or at least one tool fits all. This is misplaced for two reasons. First, this discussion disempowers the designer, product manager, developer who wants to use software tools already at their disposal. Secondly, ignores the fact that prototypes are sometimes created extremely rapidly (like in an hour) or more thoroughly (in a month). The two prototypes on the opposite of these extremes have far different tool requirements. Therefore, it is far better to think of prototyping toolsets.

A prototyper should have many tools just like a mechanic has a toolbox. Otherwise the prototype who knows just one tool is not very effective. Like to the hammer every problem looks like a nail, a designer who knows just one prototyping tool offers just one type of solution when many many more are possible. If you look for a different perspective mainly what are the specific prototyping needs, some surprising prototyping tools emerge.

Specialities
Prototyping tools can usually do some kinds prototyping better than other kinds. Furthermore, you may also prefer to use a certain tool simply because you know it's special features better than another tool.

General Availability
A prototyping tool that is not generally available to the design team makes the team reliant on a chosen few with access. If the resulting file format is also unreadable outside the prototyping tool then access to the prototype is further limited. In some organizations this is a good thing; but when you want to maximize the knowledge of your design team having a prototyping tool set hat empowers people is important.

Interaction styles & Functionality

Tools support certain platforms, interaction styles and functionalities. These actually limit what types of prototypes you can create. For example, if the widget set in your prototyping tool does not support spinners, then spinners will not appear so easily on your prototype. Moreover, the more sophisticated the UX widget set and the interaction capabilities the more conservative and less innovative your prototype will be. Conservative because you are cornered into the widget set and interactive capabilities of the tool.

Design Team Talents
If a design team loves one product, making the switch over to another product robs them of their talent they already have. Of course that can be counterbalanced if there are long term advantages in adding the prototyping to the prototyping toolset.

Deliverable formats

How a prototype will be used is also essential. If the resulting prototype will be a paper prototype. For example, choosing a tool which can print out designs and portions of designs easily would be a factor.

Tools review

There are many different dimensions to look at since prototyping involves multiple dimensions (fidelity, audience, style, etc.) However as one handy table (I can make more if they are helpful, here is a grouping of prototyping tools and what they are most appropriate for. (I will update this table based on user comments.) The table is more or less an overview of a possible look on similar tools. For your personal organization, it might be helpful to list all the tools you know and list for each: advantages, disadvantages, etc.

Methods	Tools
Static wireframe	PowerPoint, Excel, Photoshop, Illustrator, Fireworks, Visio, OmniGraffle, HTML editors, Word
Storyboard	PowerPoint, or other presentation software, Acrobat, Excel
Paper	Word, PowerPoint, Excel, Photoshop, Illustrator, Fireworks, Visio, OmniGraffle, HTML editors
Wizard of Oz	WebEx or other video conferencing software or synchronous sharing tools.
Digital partial interactive	Excel, PowerPoint, HTML Editors, Acrobat, Visio, OmniGraffle, Axure
Digital fully interactive	Flash, Dreamweaver and other HTML Editors, Visual Studio, Director, Axure

It's all about the Concept

noreply@blogger.com (Jonathan) — Wed, 24 Jun 2009 14:23:00 +0000

This editorial is one very dear to my heart, as it touches on the cornerstone of good design and something I miss in all to many HCI designer's work: the concept. In order to keep to the same points as in the original, I have done little editing from the original.

How can a design review be conducted on static interfaces?

What constitutes good design?

Good design is harder and harder to find these days. It is disheartening when people present a single window or Web page and ask for an evaluation, especially when the question is: "Is this good design?" How can a design review be conducted on static interfaces? What is possible to evaluate? What constitutes good design? Is it possible to judge a design from static screens?
As we said earlier—in fact way earlier in an article co-written with Wendy Mackay in March 2001—when discussing the importance of contextualizing design, the issue is not whether a design is good, but is it a good design of...what?
When starting to review a single screen, your heuristic-laden ego itches to pour out criticisms. A copy command on the file menu! Are you nuts?! You don't put buttons on the bottom left! Serif fonts! Are you mad? Where did you get that typeface? Spinners are so 1990s! Script typefaces on mobile devices? Split buttons! Are you sure about that? But instead of continuing that tirade, let's pause for a moment and ask: design of what?
First off, let's suggest some good questions. Do you really know the context to start judging a design correctly? What aspect of design are you reviewing? The visual design? The information design? The layout design? The interaction design? How do you "see" an interaction design in a static page?
There are of course many ways to represent all of the above. We're interested in how you do it. Do you divorce these aspects of design, or do you combine them in certain ways? Which combinations have been most successful for you?
It's difficult to divorce one from the others: All aspects of design must work together in a unified concept. That concept involves rich knowledge preceding the design activity: of the end users, their background, their tasks, their mental models. It doesn't stop there: You then need to understand your engineering constraints: what your developers' toolkit can and can't do, what sort of custom code your design will require, and whether your design needs to absolutely follow standards for future evolution and code maintenance, or if you're able to leap into new territory and design a new widget or two. Further rich knowledge that can and should influence the conceptual design: understanding the business model of the company producing the software. Is this a demo? Can it cut corners, or is it production quality? Is it a step in a long line of .dot releases? Is it the first version? Can you take risks? Do you need to reach feature and usability parity with other competitors, or do you need to excel and claim best-in-class?
Before you can understand how to design, you need to understand design. The conceptual design is more than any one facet of design; it's a gestalt that is more than the sum of the parts. Taking this perspective, how do you evaluate that single screen?—Jonathan Arnowitz and Elizabeth Dykstra-Erickson

This is a draft version (preprint) of material which was later published in substantially the same form in my Rant column in ACM’s magazine. The published version is a copyrighted feature of Communications of the ACM. All requests for reproduction and/or distribution of the published version should be directed to the ACM.

The Designer’s Hippocratic Oath—A Reformulation

noreply@blogger.com (Jonathan) — Sat, 30 May 2009 03:18:00 +0000

The practitioner is constantly barraged with new guidelines: platform guidelines, guru guidelines, papers and articles on heuristics, case studies and anecdotes that promote practice directives. And then there’s the occasional Web forum, workshop, or hallway conversation that suggests there is an overarching method to our madness. We enter the fray with this, our May-June Rave, honoring a long tradition and making it our own: The Designer’s Hippocratic Oath. Say it out loud: I Believe!

I swear by Don Norman, Douglas Engelbart, the Eames, ergonomics and all the powers of cognitive psychology and good design, that, according to my ability and judgment, I will keep this Oath and this stipulation—to reckon the teachings of this Art equally meaningful as my own experiences, to share my experiences with my teachers and colleagues and free their perspectives from bias; to look upon all design in the same footing, and to share my thinking, if they shall wish to learn it, without condescension or confusion; and that by precept, lecture, and every other mode of instruction, I will impart a knowledge of the Art to my peers, my colleagues, and to designers similarly bound to the Oath, but to none others who are not willing to endure it.

I will follow that system of regimen which, according to my ability and judgment, I consider for the benefit of my users, and abstain from whatever is deleterious and mischievous.

I will give no crash or unrecoverable error to any one if asked, nor suggest any such counsel; and in like manner I will not give to a single person the means to procure confidential company information or illegal copies of my product.
With openness to learning, to constructive criticism, and to negotiation I will pass my life and practice my Art.

I will not write code or falsely pretend to be a visual designer, but will leave this to be done by those who are practitioners of this work. Into whatever projects I enter, I will go into them with balance for the benefit of the unwitting user and the bottom line, and will abstain from every voluntary act of mischief and corruption; and, further from the temptation of awkward and incomprehensible user experiences, even as practical jokes or “experiments.” Whatever, in connection with my professional practice or not, in connection with it, I see or hear in the life of usability participants, which ought not to be spoken of publicly, I will not divulge, as reckoning that all such should be kept secret, sometimes even when they have signed an NDA and release form. While I continue to keep this Oath unviolated, may it be granted to me to enjoy life and the practice of the art, respected by all men, in all times! But should I trespass and violate this Oath, may the reverse be my lot!

— Hippocrates with some updates by Jonathan Arnowitz & Elizabeth Dykstra-Erikson

This is a draft version (preprint) of material which was later published in substantially the same form in my Rant column in ACM’s magazine. The published version is a copyrighted feature of Communications of the ACM. All requests for reproduction and/or distribution of the published version should be directed to the ACM.

Still not ready for prime time

noreply@blogger.com (Jonathan) — Mon, 16 Mar 2009 02:18:00 +0000

It's been almost 2 years since Elizabeth and I first wrote this article, and it is sadly more topical than ever. Until e-voting has a standard of usability, and open source accountability, this will stay an anti-democratic development. E-voting has the power to increase democratic participation; unfortunately as it is currently being abused it is still not ready for prime time. --Jonathan Arnowitz

Not to mince words: November 2, 2004 was a low point in American technology. Let’s get this straight: There is a current system that is the lifeblood of democracy, voting. There is a new technology, e-voting, that is being deployed, which has not been thoroughly system tested, not checked for security bugs, not even tested with the wide variety of users who must use the system (many of whom are computer illiterate, more of whom are computer skeptical). If this technology fails there is no way to determine the scope of its failure: There is no paper trail to trace down the existence of a problem. And to add insult to injury, one leading system vendor’s machines can be cracked with a secret two-digit code. (Though we don’t think you can call a two-digit code very secret or difficult to crack.)
To make matters worse, the leading vendor of e-voting machines, Diebold, Inc., has been unashamedly activist-conservative. Their methods have been shoddy at best, and at worst, not in good faith. The Cleveland Plain Dealer reported on August 28th that Walden O’Dell, Chief Executive of Diebold, Inc., who became active in U.S. President George W. Bush’s re-election campaign, was quoted saying he was committed to helping Ohio deliver its electoral votes to the sitting president. Several other publications note similar qualms about the Diebold system, including Federal Computer Week, ComputerWorld, Investor’s Business Daily, and others, although an inspection of partisan publishing will, of course, surface opposing views.
Complicate system problems with human decisions and you have, shall we say, an interesting twist. In Santa Clara, California, where voters were given a choice (“paper” or “digital?”), users suspicious of the touch-screen systems opted to cast their ballots in the more traditional physical way. No problem ... until it became clear that these paper ballots were frequently placed in pink “provisional ballot” sleeves, introducing another layer of fragility on the business of counting votes.
How do smart-election officials make bad decisions? Simple, we say: Rush into a technological push before basic security standards, let alone ethical standards, can be developed. If a software company attempted this, the entire development team would be rightfully out of a job.
Yet, this scenario has just played itself out in the U.S. presidential elections. The voices of HCI professionals in this e-voting debacle have been loud, strong, vocal and completely ignored. On reflection: At least we know where we stand.

—Jonathan Arnowitz and Elizabeth Dykstra-Erickson

This is a draft version (preprint) of material which was later published in substantially the same form in my Rant column in ACM’s magazine. The published version is a copyrighted feature of Communications of the ACM. All requests for reproduction and/or distribution of the published version should be directed to the ACM.

Results Are In: Fidelity Deception Ranks High on Usability Problems

noreply@blogger.com (Jonathan) — Mon, 23 Jul 2007 05:52:00 +0000

(Adapted from a previous editorial I wrote for interactions Magazine)

This editorial is from a controversial issue on measuring usability. This special issue covered CIF usability testing (a seemingly innocuous if important topic). Upon publication many took strong esxception to our coverage of CIF testing. But perhaps these critics also took exception to our criticizing a practice all too often employed: how to lie with statistics and its designer equivalent using too high a fidelity to overwhelm an audience. --Jonathan

This issue is primarily about usability and metrics, measurements, and what we do with them. And thus our rant: deceiving with numbers.
At university you may have had the pleasure of a reading assignment instructing you to read and memorize a slim volume called How to Lie with Statistics [1]. This is a classic that presents the unimaginable- convincing people of something by manipulating unfamiliar concepts that sound scientific, and thus authoritative. One could, of course, achieve the same end via bullying or mesmerizing charisma. But lying with statistics is a venerable process engaged in by politicians, advertisers, and scientists- and even usability professionals.
One of the ingenious cheats of statistics is to "prove" that you are correct by accentuating the positive and eliminating the negative. To be a master of deception you do, of course, need tools. Not of the rabbit-and the-magic-hat variety. A simple high-fidelity prototype can do the trick. We can call this type of deceiving with numbers High Fidelity Deception, and it goes something like this:
Reduce the overall feature set of a target product.
Design a prototype that covers some of the features.
Ensure that it is attractive enough to resemble a finished product with a dose of glamor: nice animations, transitions, professional color palette (for extra points, make it blue-you do know about that, right?), refined font, and upscale branding elements.
Show it to a few high-placed individuals who (this part is important) don't have the time or inclination to dive into the details.
Get their enthusiastic endorsement.
Proceed to formalize this into a final product regardless of the paucity of credible detail.

You can get excellent marks on the prototype: high levels of user satisfaction, compliments on the prototype's refined sensibilities, and unequivocal approval to proceed. This prototype is, in fact, merely an attractive shell, but it passes muster, and you can quantify its success by repeating what percentage of users who have seen it were "wowed" by it (don't mention the actual number of users, though-after all, two out of two gives you 100 percent!).

Why does this happen?

A designer can impress stakeholders by producing an impressive-looking artifact. If it is, in fact, your product, but it isn't finished yet, you can call it a demo and all will be forgiven if your scripted click-through falls off track and your prototype crashes badly. Because it is lovely, it can even give the false sense of being usable.
Using early prototypes for usability evaluations is an important option for testing design concepts. However, if the fidelity of your prototype is too high, it may not be clear what you are really testing, and you risk gathering false positives on the concept.
Sometimes the consequences are high stakes: You convince a customer that you are much further along than you actually are, and deadline negotiations for delivering the actual product reflect the customer's reasonable demand for early delivery while you struggle to meet impossibly close deadlines. Is offering an unlovely collection of half-baked ideas to a user the only viable source of critique? One could say it's much better to go for a half-baked look without the bells and whistles, and simply test the concept with a low-fidelity prototype. It may not look nice, but it will result in a better product at the end of the day. But you can also do both: Show an attractive limited subset to illustrate the look and feel of the final product, and then get to work evaluating that drab gray collection of input fields and checkboxes.
In any case, the map is not the territory and the prototype is not the product.
Confusing the two will result in an impossible entanglement that no quantitative data can unravel.
-- Jonathan Arnowitz and Elizabeth Dykstra-Erickson

REFERENCE 1. Huff, D. (1954). How to
Lie with Statistics. New York: W-W Norton
and Company Inc.

This is a draft version (preprint) of material which was later published in substantially the same form in my Rant column in ACM’s magazine. The published version is a copyrighted feature of Communications of the ACM. All requests for reproduction and/or distribution of the published version should be directed to the ACM.

Software that Behaves Like Spoiled Brats

noreply@blogger.com (Jonathan) — Sun, 24 Jun 2007 14:08:00 +0000

As many people know, one way of analyzing interaction design and system behavior is to do something that has many different names, but usually sounds something like dialog analysis. Dialog analysis is the process of analyzing the software interaction as if the system is a human being who is performing a service for the client, aka the

woe-besieged user. After careful analysis of many products, it seems to us that if most commercial software and Web products were humans, they would be characterized as greedy spoiled brats. Just for an example, look at the way desktop products, customer services, and Web products behave.

Desktop Products

Office productivity tools make you do somersaults to undo their automatic formatting. They also drive you crazy with the collision of features they never knew would be used together. One word processor we were using to create a large document from many sub-documents refused to behave—auto-indenting, bulleting, or renumbering tables into items. It so destroyed the formatting and twisted the style usage that we had to resort to using flattened pictures of some tables and formats. When you try to do something that a product's designers didn't anticipate, some products exact their revenge: They do the digital equivalent of throwing everything into a heap, leaving you to clean up the mess all by yourself. This is marginally acceptable behavior from a two-year-old child learning the concept of patience. It is reprehensible behavior in adult human-computer interactions, and it turns the product into something we call a spoiled brat.

Customer Service Products

How ironic that some of the most abusive products around are software products designed for customer service. Let's just take call-center experiences as an example. Phone trees in service calls exhibit "do what I say or else" behavior. The user follows patiently along only to find 17 steps later that the end point is closed, or does not give you the information you expected, or transfers you to a busy line, or leaves you no way to return anywhere but to the beginning of the long depressing tree. Brat.

Web Sites and Services

There are also hidden brat interfaces that reflect the bratty intentions of their owners. All too many Web services abuse users by demanding all sorts of personal information, much of it completely unrelated to buying the service. They then turn around and sell your name and information to other people to spam and telemarket you to no end. These sites even brashly claim that your data is safe and that they have privacy rules and policies. Here the HCI brat is the very small print and the checkmark you will overlook that permits them to share your information with other companies with whom they have a relationship (even though that relationship only exists to sell your personal information!).

User experience as a field has a long way to go. Our bratty products do us little credit. Thank goodness we're not all brats.... There are certainly things to rave about as well!—eic, Jonathan Arnowitz and Elizabeth Dykstra-Erickson

This is a draft version (preprint) of material which was later published in substantially the same form in my Rant column in ACM’s magazine. The published version is a copyrighted feature of Communications of the ACM. All requests for reproduction and/or distribution of the published version should be directed to the ACM.

Whose profession is it? It’s Mine!

noreply@blogger.com (Jonathan) — Mon, 04 Jun 2007 09:04:00 +0000

This article was the front editorial for Pabini Gabriel-Petit’s special issue on different stakeholders in the user experience.

We must step up to the plate and take ownership of experience design.

Focusing on "Whose profession is it, anyway?" you might ask, "Which profession are we discussing?" The professional interest is in "user-centered design," wherein professionals design and evaluate user experiences of human-computer interactions. There seems little consensus on who is responsible for that user experience.

Variously, we find that responsibility can fall to engineering, or marketing, or some collaborative group ownership. It seems like everyone's responsibility except our own. One might become resigned to the realization that it is politically inappropriate to claim our own expertise; but in the end this is shirking our obligation.

Taking responsibility

Attributing responsibility or accountability is different than attributing participation: We believe that UX (user experience) design is by its nature collaborative. Working with multi-disciplinary teams is a significant part of our work life. So why is it that UX design, unlike other disciplines, seems to promote a sense of collective work? Could it be that our conduct reflects our discipline's standards, rather than corporate standards?

Product management doesn't build or design products; their job is to own product vision and strategy (naturally with the other stakeholders' input). Engineers own code development and code quality, with a wide range of specialties (architecture, code design, QA, and release management, to name a few). Product marketers take clear ownership of marketing communications and product campaigns, keeping the pulse of the marketplace, and trying to detect what it will buy. Therefore, it's only logical that human-computer interaction professionals take ownership of the user experience. We are, after all, user experience experts, despite the fact that we depend on other development participants to meet user and business needs.

C-level ownership?

Why is it that we see few c-level managers for user experience? Chief User Experience Officer (CUXO) could be a formidable title. In most large software development organizations, user experience folks report in through engineering or marketing, whether there is a special group for HCI or not. That implies, however, that at the end of the day and after all possible design dilemma escalations, it is squarely the call of engineering or marketing (heaven forfend it should be our call) on what to create. We need our own high-level management who can directly compete with marketing and engineering management. (Please read that part again: This time and at this level we didn't say collaborate). Our agenda needs to be given the same respect as other disciplines. But this won't happen if we don't do our part. We must step up to the plate and take ownership of user experience design.

We've seen some positive changes in org charts: An HCI executive is no longer unheard of as we see more directors and senior vice presidents of User Experience. Little by little our agenda is slowly rising to the surface of formerly engineering-driven companies. But this puts all the more pressure on us to seize the moment and take control of our user experience.

If we aren't accountable for design, why does a software-making organization need us? If we don't fight for that responsibility, and we accept that our role is simply to suggest possible outcomes, our voices become weak and our protests unheard. We have a sneaking suspicion that our credibility is low because it's not our voices, but our rationales, that are weak. Insisting on ownership of the discipline comes with it a concomitant obligation to do the work with the highest possible standards, and that includes, like it or not, accommodating the quirks of marketing, the insanity of deadlines, and the caterwauling of engineers who have already thought of a cheaper, faster way to implement a requirement than the investment our designs require.

So what's the problem? Why don't they trust us with ownership? Are we inflexible, unschooled, uninformed, or just plain too crabby when asked to truly collaborate? Even in companies that proudly proclaim themselves "user-centered," all too often we see HCI activities included as an afterthought. When our work is conducted too late in the cycle it is pointless—if it has no effect on outcomes, it's a waste of time. Is HCI in your company just a checklist item deliverable? "Did you finish your usability testing? Great, check it off the project chart. You can look at those usability tapes when we're done with this project; we'd like you to focus on something else just now."

Some parting shots

Herewith we offer a few small pearls of truly sensible advice from some of the most difficult engineering managers ever:

"Why bother with a usability test? It is what it is and it's too late; if you test it you'll expect us to do something about your results, and that isn't going to happen. Save the time for when it matters."
"Please stop designing things. We've run out of implementation time."
"Look, we're after a decent band-aid here, not elegance. Just get on with it or we'll do it ourselves."

You can always pack these concepts away for version n+1 (we call that type of cache of well-constructed design think-ahead as "blue sky"). Be a hero (if you're still there for the next release); revenge is sweet.

Other disciplines should be accountable for appreciating and abiding by our design and evaluation deliverables. It is not enough to require a design spec or a usability test's results. We've seen teams use lab testing to resolve internal disagreements when the number of test participants is barely larger than the team size; this is abdication of design responsibility to an extreme. We should not delude ourselves: There is a big difference between so-called discount methods and Machiavellian ones.
It could be that we all just need better persuasive powers, but we suggest that designating an accountable party (that would be us) in advance would (we guarantee) be much cheaper than extended debate. Engineers, product strategists, and others involved in delivering software experiences must commit to acting on our work. We'd like it if they understand it, too. Our commitment as a support to code development does not mean we can't have a controlling vote in a product partnership where consensus does not always rule.

Hold the soap

That is not to say that what a designer says is always the end of the line. We don't want to make a soapbox petition for control; we want a true dialogue on user advocacy and design sensibility. Designers need a thick enough skin to deal with critique, compromise, and collaboration, just like the developers, the marketers and the product managers. In each of our personal lives, someone, somewhere, told each and every one of us that people are supposed to just get along. This implies that the natural state of things is easy consensus. We beg to differ! It isn't supposed to be easy! You can admire and respect your competitors and your collaborators, but you do not always have to agree with them. And for the times that the discussion is on your turf, calling for your expertise, and demanding intelligent solutions from you, you need to be able to grab the ball and run with it.

When called for, we do need to be able to look any of our collaborative colleagues square in the eye and say: "I hear you. I understand your arguments; but we're going to do it this way."—eic Jonathan Arnowitz and Elizabeth Dykstra-Erickson

Loser-Centered Design or let it be: the Frankenstein Persona's

noreply@blogger.com (Jonathan) — Tue, 29 May 2007 16:49:00 +0000

This first entry is an editorial that covers the phenomenon of creating persona's without user research, which made quite a buzz at CHI2007, even though it first appeared in interaction in 2006. Frankenstein persona's, a particular egregious example of user-uncentered design, is something Elizabeth and I have witnessed not only in practice but also as reviewers for many conferences and journals (all of which incidentally have no name).

All too often, terms are misappropriated, if not outright hijacked, and their meaning becomes either obfuscated or totally changed. Have you noticed that user-centered design is quickly becoming such a victim? We feel the only way to protect this saintly term is to fight fire with fire and provide a counter weapon whenever the term user-centered design is misused: Enter the term loser-centered design.

Loser-centered design focuses on the wrong things, and worse, ignores the user-yet it still uses the term user-centered or user-focused; either to justify some Machiavellian ends or to cover up poor process. Unfortunately, loser-centered design, loser-focused design or loser-centric development (the terms are sadly synonymous) take many, many different forms. We want to give you some examples here; surely you have your own. (Feel free to add them as comments to this entry and you will have the immense gratification of commiserating other designers worldwide.)

Frankenstein Personas
One obvious example of loser-centered design is pandering to a marketing department's market research instead of conducting and using real user research (e.g., talk to someone who isn't in your company, on your team, or co-habitating with one of the aforementioned). You can smell these projects a mile away—the task analysis or information architecture just mimics the research, and neither questions basic assumptions nor adds anything that doesn't fit the pre-conceived marketing model. Loser-centered design in these projects also takes the form of user segmentation based on personas that were defined largely by internal focus groups and a few friends of the marketing department (or anyone else who has read too many in-flight magazines). All too often these personas are created from magazine clippings illustrating brand desires and wants based entirely on marketing mumbo-jumbo. Yet they become so tangible and visually vivid that designers rush in and design for these modern persona Frankensteins as if they really existed. Tell the truth, are you guilty?

The Loser Researcher The loser researcher is a charlatan or inexperienced user researcher, with just enough knowledge to be dangerous, who conducts user research for a product and instructs designers on what problems to solve. Instead of observing what users actually do, they ask users for instructions on what they should design. Loser research has some tell-tale signs: out-of-context telephone interviews, open-ended survey questions, and site visits conducted only in the on-site lunch room (assuming the software has nothing to with employee lunches).

One example of such folly follows. A designer was asked to look at improving communications since something was obviously wrong with the intranet. Indeed, upon being interviewed, senior management offered up numerous complaints that the junior members of the staff could not find the right information on their intranet. These complaints were addressed at great cost with much changing around of data layout and information architecture, but the results did not improve the situation.

A user (not loser) research firm came along and observed what the users actually did. The real problem, as became apparent by observation of the office, had nothing to do with the intranet and everything to do with a recent reorganization and the physical traffic through the work area. During the reorganization the company decided to reward high performers with window cubes, centralizing all the junior and new employees several walls away from the more senior employees. Not wanting to appear as uninformed as they felt, junior folks stopped asking for advice, since they had to expose their ignorance by making a public trek to a senior level staff cube to seek advice. A room redesign solved the problem.

Designing the Emperor's New Clothes
Reducing call-center-response time forms the crux of another loser research opportunity. Convinced that shaving seconds off of each support call would add up to entire individuals that could be dismissed, the user researcher was called in to analyze the business process and observe why results were still below expectations. Through interviewing the call-center staff, the user researcher discovered a huge number of problems—none of which management expected. Told that mistrust, disrespect, lack of information, and unfair treatment was the root of the problem, management reacted with outraged disbelief. "We already solved that problem!" Evidently not. The user research results were uncomfortably human. Subsequent loser research focused on technical issues; while the results showed no promise of improvement, focusing on machines rather than social practices "felt" better to management. After all, one should only have to "fix" people once. When loser research results were available, management breathed a sigh of relief and felt that the problem was well in control—and that its own management style was no longer to blame.

So the next time you hear the word user-centered design applied incorrectly, jump on the offensive and whip out the loser-centered design retort: "user or loser?" There is, indeed, a difference.—eic - Jonathan Arnowitz & Elizabeth Dykstra-Erickson

This is a draft version (preprint) of material which was later published in substantially the same form in my Rant column in ACM’s magazine. The published version is a copyrighted feature of Communications of the ACM. All requests for reproduction and/or distribution of the published version should be directed to the ACM.

Welcome to User Experience in ArnoLand

noreply@blogger.com (Jonathan) — Tue, 29 May 2007 15:37:00 +0000

Hello and Welcome,

This Blog is dedicated to a discussion of the state of the User Experience profession. For the coming entries, we will be reprinting the commentaries on HCI that have appeared in interactions magazine for the past 3 years as written by myself and Elizabeth Dykstra-Erickson (as slightly edited/updated by me). These commentaries, though tongue in cheek sometimes, or sometimes very provocative were intended to draw a response from our audience. As a print magazine in an e-world, this was difficult to realize. Now putting this information in blog format I hope to generate the discussion our articles were intended.

Needless to say, this is also the space to discuss/confront one of the authors of Effective Prototyping.

I hope you enjoy.

Jonathan