Bringing user studies in form of a game

Wohoooo! First exam successfully finished, a lot of preparation, hours of user study evangelism and a lot of insights into importance of user studies into design and innovation. I understand more and more that role of designer is becoming more than just creating a beautiful wrap. It is also something about discovering a user need, smartly involving them in creation process and giving a soul to the concept that could be fulfilled.

I would like to share my ideas of how I imagine user studies brought into the team that is not familiar with such. Most of the insights came from our Participatory innovation project with Dimaps, but is not limited to it.

I have used this synopsis in exam presentation, and of course – this is not a complete scientific paper, just a guide for myself. The basic idea is to make studying users as a game (3rd poster) and building knowledge about them from the methods applied that would be made in attractive form.

In quick glance – it is important to challenge teams’ assumptions about users that at least I have met quite often. In my opinion these assumptions lower the motivation of conducting user studies. Of course, there always can be good guesses, but studying users I think is actually a safer way of making innovation that would be accepted on the market.

More about challenging these assumptions in a synopsis. That’s at least from my own experience

Fortunately memorization without understanding is over for me in education for a long time, but still in daily life it is often necessary to remember some things, as portable devices tend to scare me sometimes about giving them too much of my brain tasks. You lose what you don’t use, right? Luckily I have discovered that memorization can be fun and want to share with you my favorite technique. It really works!

The technique that I have read from pocket book (Latvian version) is actually quite simple and incredibly effective. First - try to remember at back of your head 10 objects below and associate them with respecting numbers. Check if you are able to recall objects in any sequence. Should be easy as appearance of these objects is similar to respective numbers.

From Roland Geisselhart and Christiane Burkart

Second, pick up-to 10 things that you have to remember (shopping list works well). Then remember these things by making absurd and fun stories that associate things with objects above like “I saw a really big candle made from a cheese.” and “A snow-white swan was bringing a beer for me on it’s back” etc.

Finally - try to remember above objects and they will remind about associated thing. I have tried – works, in any sequence!

If you need more than 10 things to remember, just memorize more objects. Works also not only for individual terms, but also sentences (you just have to pick up most important keywords).

What is your favorite memorization technique?

Riga Technical University has made a new and very interesting Master’s program for students that wants to study Information Communication Technologies in a business context. Program is intended for students with diverse backgrounds and supposed to run in the atmosphere close to real business environment.

The aim of the Business Informatics study programme is to prepare professionals who master systems thinking and engineering sciences; who are able to use, choose, develop, and acquire ICT solutions that enable enterprise development; who can design intra- and inter-organizational information systems; and are capable of participating in corresponding inter-disciplinary and international projects.

This is a 2 year full-time program starting with 2010/11 study year. Studies will be in English.

It’s time to write about our last study project together with Innovation&Business students.  Writing is such a good way how to save an experience for a reference with a high level of details The course we are taking now is called Participatory innovation - one of major user-driven innovation approaches. And this project was a complementary part to lectures and discussions. We had to 1) find a real company, 2) get an innovation project and 3) fulfill it with methods that we were taught of. Project was meant to be 8 weeks long.

To sum up we worked together with company Dimaps on their route planning software. We made reasonable ideas and scenarios which were based on our user studies. However, there were some problems with inner motivation, communication (different mind-sets) and in our case – presentation of results. There is one of the scenarios above that we have extracted from user studies and co-ideation events.

In this project I have learned:

1. There is  need for one leader in a team, who helps to make teams’ decisions in a way that everyone is satisfied. That person needs to be charismatic, knowledgeable, a fast-thinker (to evaluate everyones’ ideas) and has to have a big picture in a head. Such person must not lose composure. By having many people who are willing to be only the one leader, makes decision making hard. Having bad or no leader in team, results in low motivation and incoherent group-work,
2. Team of 7 was too big in our case, just 4 people were working,
3. Presentation is how others perceive your work, good presentation means for most that group had a good work together (and it is mostly true),
4. It is important to be quick in capturing and interpretating data. Aim could be having 1-2 user-study videos per day, that would help to better and faster co-ideate in a group leading to richer ideas,
5. There is little value in having a shallow look at user-study materials (videos) and interpreting them 1:1. Question why they are doing that is important in order to create good user-centered products and services.
6. I have got a good insight in participatory innovation,
7. Consensus based group-work is slower, especially if members have different mind-sets, that requires to discuss each decision.

I am going into more details below.

Looking for a company

We started by dividing in teams (or groups would be more precise title) of 6-7 that took a lot of discussions. In the end our group consisted of 1 design student (me) and 6 business students.

Then we had a group meeting discussing what expectations/skills we have and what companies we will look for. It ended in a list of 10 companies what we contacted through email. I had to argue that only an email is not effective way of establishing connections that I experienced while working at WSI (only about 3% respond rate).  So we decided to call our contacted companies after couple of days will be passed since sending emails. Well, from 10 companies indeed 2 responded (rejected), that makes 2% respond rate But calling helped to established a contact, as managers seems to be so busy that they delete such emails or in best case – goes through very briefly and then forgets about them.

In the beginning we thought it shouldn’t be difficult to find companies because of many reasons including a cheap labor doing a project in favor of company for free Another reason that we imagined was possible connection with a university, that is a good source of constant innovation. However, in reality not many companies wanted to work with students as such a project in their opinion was for too short time or they didn’t though that such project would give them sufficient benefit. Looking for companies took more time than we expected so a project suddenly shrinked from 8 week project to 4 week long.

I guess our biggest mistake was waiting on one contact until they would give an answer, we should have had at least 1-2 meetings each day in order to find someone who would be more open and interested in such innovation project.

Starting the project

Dimaps GIS route planning (dimaps.com)

We were lucky, because one of the other groups did a better job in finding companies. They have got 5 options that have agreed on project, and they were happy to share with us. I must admit that they have sent e-mails to more than 60 companies. This company we were lucky with was Dimaps, quite small but very flexible software engineering company with concentration on route planning software and geographic information systems.

They were quite open, interested in new ideas how their route planning system could be improved – how users could interact with a map, how people could report if routes were incomplete (hey dude, I have found a better route!) etc.

Soon we have conducted a first group meeting. We decided on a next plan – in having user studies to understand the use of route planning software in practice. We have got some serious disagreements as well – do we have to spend a lot of time in order to understand an existing Dimaps software or concentrate on user studies and their needs. That was a point where I missed another design student who understands a need of studying users. However, both points about software and users were right. We needed to understand sofware that we are making suggestions for and we need to understand users in order to suggest user-centered ideas. However, communication often was not fruitful.

Thing that we missed from a beginning was one strong project leader, because at least three of us wanted to lead and had strong arguments. That made it difficult to achieving consensus and making strong decisions which should have leaded towards specified tasks for each team member and coherent team-work.  How do you pick up only project leader? Everyone has ideas, but who should be the one, who makes decisions while having everybody satisfied?

Conducting user studies

Parcel delivery experiment

In the beginning we decided to conduct a delivery experiment. One of our group mates, Eleonora, had to deliver a parcel to a flat in student dorms. The reason for choosing this building was because it is challenging (many entrances, most of them closed) and that our group mate hasn’t been there before. We followed Eleonora with video cameras capturing real-life experience (being confused, calling, asking, looking). That helped us to get our minds into the project and put ourselves in the shoes of users.

After capturing this video, we made another one. This time our group mate Margus was a main actor, having a route calculated by Dimaps software. This case was straightforward as software showed right entrance and route to the right room. We found out that Dimaps route planning software really helped. However, we have also found out that there are some potential problems – how someone who has a delivery room on a map knows in which floor this room is?

Next there was a delicious idea – to order a pizza and follow their delivery in order to see how local pizza delivery businesses work. The only one local pizza place used to advanced methods of planning a route. Basically they just 1) got an order,  2) made a pizza, 3) delivered by memory (or paper map).

We also conducted interviews with ambulance and truck driver about their route planning systems. Both of them use GPS devices (in ambulance case updated every 2-3 days).

Discussion / tangible stuff

User workshop

We conducted a workshop having a manager of Dimaps (who is also an expert in software engineering), manager of customer department in large media company and three of our group-mates on board. Using tangible, abstract elements and mock-up of building where delivery experiment was done, we provoked discussions about real-life delivery problems, possible solutions and a future of route planning systems. We have got many ideas there.

Challenge in such workshops is to give a specific, reasonable task for participants to solve. Tangible parts is a good way in helping participants to build  solutions. However, as I experienced facilating workshop, there should be a clear introduction of task and parts that should be used. Besides, such seemingly tiny thing as easiness to reach tangible parts, also affects very much participant willingness to

Scenarios

Using knowledge that we had from user studies and a workshop, we generated different ideas of how and where Dimaps route planning software could be used which includes 1) mobile phones, 2) smartphones, 3) special our-made device Obox. We saw these three as cost-effective and mobile environments of having route planning software in. Besides such devices could support feedback from users in order to improve routes. We also saw an opportunity for Dimaps for selling their information for GPS software developers.

Report & presentation

Mock up of Obox & Kweedo

In the end we made a report and presentation. All groups including us got some critics from teachers about a report. Main points are that it should be a joy to be read which includes having graphics not only text. And graphics should be in one common style.

Also presentation of findings is very important. That includes basic things 1) are materials (posters, slides) readable by everyone? 2) is presentation coherent? 3) Does somebody get something out of presentation (information, fun) 4) Does presentation fits in the time allowed?. Some of the teams did a great job in having enjoyable presentation by making it in a form of little theater/storytelling. That makes it easier to be perceived.

All the teams had interesting prototypes, just have a look in my Flickr account or at our courses’ common one.

Some longer time ago I have discovered an incredible source of amazing sounds that helps to contemplate the meaning of my life, heh For me it is a non-banal interaction device. Look at the facts, dude! –  132 different notes that makes a very large opportunities in playing intervals. If we look a the chords (consists of 3 or more notes) then 374660 options of triads and 12082785 combinations of 4-note chords are available there. Hope my math is not failing me and that I used the right formula: 132C3 and 132C4

Anyway, for everyone who has tried a guitar it is obvious that there a lot of options. And if we look at the math, it communicates that this beautiful piece of instrument can’t be learned fully in ones’ lifetime. But it is sure that guitar is sufficient for all tastes of music – let it be rock, jazz, bosa nova … And all tastes of playing – strumming, fingering and flatpicking. We are different, we can’t learn everything (oh no, does it mean that I will never read all books I have on my list???), but we can pick what suits us best and do it as good as we can. Once we know what we want, we can throw out without regrets everything that doesn’t fit in our life and future goals.

So this incredible instrument promises us a potential reward for interaction with it. Just imagine your favorite melody and double the feeling of it when you make it by yourself. Unlimited possibilities, a skill level to be mastered whole lifetime makes this instrument appealing for centuries. Even in our fast-paced age there are things that doesn’t change. I am glad for that.

Here is my first attempt to show people what I am actually doing with a guitar. Well, I still consider myself as a beginner with a lot of things to be learned, but I think that I should end up with my conservative nature by not hiding the things I like anymore

Post-it notes - a cognitive artifact

Fridge full of post-it notes (what a variety there!) and a shopping list represents something about our mind. In this case we reduce a workload from a memory by externalizing our memory to the outside environment. With these artifacts humans represent information taken from their mind trying to improve their cognitive abilities. External cognition topic is important topic in Human Computer Interaction field as a complement to mental model theory in order to explain human cognition.

As it is suggested in Interaction design book, we use such external representations in order to:

Cognitive artifacts are different from cognitive tools in a way that artifacts are in static representational form while tools require more interaction, just like a modelling software which outputs different results relating to the input data.

Examples

I am amazed how different these artifacts are in various cultures. For instance the quipu recording devices used by Inca civilization – strings containing knots helping to remember something.

As another example a solar clock (sundial) could be mentioned where time is being measured by the position of the sun. First used probably by Egyptians on about 1500 B.C.

Future

As our manufacturing tools improve, cognition representing is becoming more complex. Take visualization of complex networks library or another data-driven vizualization library as an inspiration. What a variety of information design! As the amount of (recognized and processed) data is increasing there is a demand of relevant ways how to visualize them.

Youtube changes voting system

You probably have experienced brand new changes in user interface of Youtube. If not, you can opt-in there. In case, if you dislike it just opt-out.

First thing that i noticed – they have removed 5 star rating system and replaced it with a 2 rating system – like / dislike. Is the 2 rate system better? Well, generally it depends, but Youtube data shows that the 5 star rating system is not as effective as it could be, as most of people use marginal ratings – either 1 (dislike) or 5 (like).

For those, who are facing a decision of what kind of feedback solution use into web interfaces, Yahoo has a good library of those. But I am sure there is always a room for innovative user interface solutions solving a particular dilemma

Youtube angagement with users

It is possible to gather an interesting information by researching what happened in the past (looking back). However, many interesting things (innovation, improvements) happen by going forward into the unknown. It is interesting how Youtube engages with users online (and I am sure – offline as well). That includes posting on blog and listening at feedback (that includes republished articles at other places), providing platform for users to express their product ideas, roundtable discussions, one-on-one conversations with the community. Data from a support forum could be used as well.

I am thrilled to discover more and more channels in receiving user feedback both online and offline. However, such corporations as Google (and Youtube), Microsoft etc. has advantages in data available leading towards better data-driven decisions and innovations. For companies with limited resources offline qualitative research techniques (including provoking discussions with provotypes, engaging users in workshops, testing prototypes) could help better.

From http://www.creativeuncut.com

I remember early days of my childhood, when I was able to appreciate the power of stories. Fairy tales took a large part of my free time and the tower of books was growing more and more reaching the critical height, threatening to make a big bang by falling on the ground. This was time full of reading stories that kept my attention for hours, gently leading me through an imagination path and involving me in the magical world of kings and queens.

Well, I am glad to discover that stories haven’t disappeared even in academic and professional life. Not only they have remained, they are so useful in communicating ideas. The greatest example are my inspiring teachers at Syddansk university where in each lecture examples from professional and academic fields are being actively involved in form of stories of their professional experience. I am sure that the power that non-linear information that for instance Wikipedia gives is invaluable. However, I would like to argue that good (inspiring, catching) stories is a good way in efficiently (and effectively) leading listeners into the topic. How much do you remember from your all-time most favorite teacher/hero/movie, and how much do you remember the content of the last encyclopedia you read recently?

Communicating ideas in a design team

I have experienced that it is not enough to have really great solution and idea when working in team. Actually there is not much sense of having one if I can’t communicate it in a way that other people can understand it and add to it (or argue against it). Badly communicated idea is being listened (by patient audience), then accompanied with blank eye-winks and  forgotten immediately leaving a feel of emptiness and disappointment. On the other side, telling your ideas as a fairy tale, with a lot of imagination, examples, in expressive and coherent manner can really attract people attention and share your point with them.

Jacob is explaining us a Video Card game

In a field of design, communicating ideas is an essential part of teamwork. There are a lot of ideas flying around, and there is a significant need to share these ideas so that everybody understands and can evaluate them. In hard sciences there is often one common truth (like laws of physics) and involved people understand that. While in design ideas are arguable, both designs could make a sense, but which is better?

Is storytelling just about talent?

Some people seems to have natural grasp on communicating ideas clearly, convincingly and often inspiring the audience. That includes my fellow mates at university (Smaranda among them), friends and of course – professional speakers like Robin Sharma (whose books I am reading with a great enthusiasm), powerful Gary Vaynerchuk – the wine expert, amazing/incredible/unbelievable Steve Jobs and many more.

Such people I call  good storytellers. Passioned about their topic, courageous,  imaginative, to some extent – talkative. Are you born with such skills?

Well, as some people seems to be more communicative and imaginative as other, I think it could be learned (by doing). In a high school I could die when I had to communicate my work or other ideas in public. And many people have fear of that, even a special term has been introduced to describe that. So in that time I thought that good public speech is just about losing your fear and using your natural improvisation skills. As this is true to same extent (even the best have a little bit of anxiety), there is something more. A time invested.

Good storytellers and presenters respect the audience. They have prepared their story (or know topic very well, or if not – then have reaaaly great imagination skills) and they are passioned about sharing it with audience. They are proud of their story and seems that they are seeing a story in their mind visually making their speech a reflection of that cognitive movie. They are keen on seeing audience inspired.

I wish everyone, including me, to develop such passion of sharing good ideas to others in an attractive manner. A good way to start is to watch people who are good at it. And then try it by ourselves.

The idea of the mental models seems very useful to me, as it is important to take into account what is going on in people’s heads  in order to make a successful user interface design (and of course other kind of design). With the successful design I mean one that user can learn easily and naturaly and use it effectively.

Quick attempt to make OpenID interface easy to use (HCI classes)

OpenID interface proposal

As the task for discussions in HCI class we needed to prepare proposals for a successful OpenID interface. We had to test the design on one user at least. In my case I used the Skype screen-sharing feature to watch and listen how my girlfriend is using various OpenID log in designs, such as Facebook, Twitter (at RWW to comment an article).

I immediately noticed a problem that different services puts different contexts on log in feature. In other words, users might think that by logging in by, for instance, Twitter account, it not only allows to post a comment in RWW, but also affects her Twitter accout (will a note appear in my Twitter? will I be required to follow someone?).

So I tried to develop common interface for all services (Facebook, Google, Twitter etc.), meaning that there won’t be differently designed pop-up windows, only similar username/password/url text boxes. However, I guess this violates the idea of OpenID, that you are redirected to the authentic website (e.g. Facebook) to confirm your data in order to log into website (such as RWW). Otherwise, I personally would be hesitant  to share my Facebook password with other website I don’t know.

So, my design was more like attempt to highlight a problem of different contexts in OpenID log in design.

Different understandings of OpenID

In the class we tried our designs on each other. I was surprised how diverse understandings (mental models) about OpenID were. And I was more surprised that each person defended their own mental model (belief?) quite convincingly, even if their mental model was incorrect. For instance, some people claimed that OpenID is a big database containing all password, while some people said that in their understanding it is decentralized collection of databases.

Mental models (http://middledigit.net/)

What is a mental model?

Before reading books, doing excercises and having discussions I had very blurred understanding of mental models. I knew that this is something that happens in people’s heads, but that’s all. How can that be used in design, and why? Does it change by the time? Do everyone have their own mental model and does it differ a lot? There were many questions I had no clue about (actually I didn’t have questions before that ).

From a book I understood that main concepts of mental models are information processing and external cognition.

Information processing

The mind is thought as information processor. Information comes into mind, goes through various processing stages (comparing, matching) and goes out in form of behaviour (including, no reaction at information). There is much interest in calculating how long does it take to perform a certain task.

Of course, there is limited amount of information that is possible to process. That creates bottlenecks in procession information.

Information overload is a common concept nowadays, so there is a need for structuring information, so I guess librarians and information designers (in physical world) together with information architects (in digital world) are doing a great job in making a complex clear.

As I am personally interested in information architecture and am following community for a longer time, I have noticed that there are some difficulties in defining this practice. However, I am glad that effort in explaining IA is being made and on right side there is a pretty neat explanation if IA. I guess something the same could be with information design practice.

External cognition

This concept refers to thought that people interact with or create information through using a variety of external representations including books, maps, diagrams, notes, images.

This is for reducing cognitive load:

1. memory load,
2. computing,
3. and to reflect changes (crossing out what’s done, manipulating models).

No more than 7 elements in User Interface?

Have you heard of  7 +/- 2 rule? It argues that average person can hold from 5 up to 9 unrelated objects in his/her working memory at one time. That comes from George Miller’s paper The Magical Number Seven, Plus or Minus Two.

This rule is sometimes applied to design including user interface design (UID). Even some recognized books devoted to UID refer to that rule. Read (old) post by Edward Tufte. There are lot of suggestions on the internet around rule of 7.

So, now we know that people can store  about 7 objects in their working memories, should we use only 7 elements in design?  7 options on menu? Well, as it appears, not really.

Well, when operating in UI, user doesn’t use only memory as the only one cognitive function. We can’t forget also about recognition. And there are a lot more cognitive functions that would help user to operate. For instance there are some techniques that helps people to memorize more by using their imagination. For instance I have personally found a very useful technique of memorizing things by relating them to constant pictures in my mind.

Well, in short words, usually there are more complicated things going out in user interfaces than some general rules (which might, and also might not be true in our specific cases). And as there is something about not having too many elements in user interface, it is always better to be critical and have real tests on users to tell whether our claims are true or not. Of course, there is always a problem of having a good testing in fast paced development environments where pressure from clients is an everyday event. Some people are talking about integrating user centred practices in such agile environments, but this is a story for other time.

Thank’s for our HCI course teacher Jared, for bringing attention upon that. I was one the people who sometimes blindly believed in such sticky-attractive-number-rule.