Grieved over what Navigation 2.0 does to the Quiz editing UI I designed and usability tested in summer 2008 (discussion, tracker item), throwing away much of the tested design without testing the changes (update, later the same day: this is still being discussed; see the tracker item). How I wish I could be there when core moodlers design and decide about big UI-related changes like this Navigation 2.0! (spec for the Quiz change)

• Worked with Eloy, Sam and others to get Moodle’s first proper Wizard in. Yay!  Sam did a great job of reacting to my feedback quite late in the process, though apparently he did not know of the specifications I had made for the wizard last autumn.
• Discussed Database module exporting UI
• Discussed Forum search UI with Anthony
• Commented on The database module’s UI on Anthony’s request – hope to have a chance to do some research and help in redesigning the Database module UI at some point
• Added MediaWiki categories to Moodle UI guidelines for faster browsing, cleaned up the guidelines so unfinished things are less in the way of usage
• In the Moodle sprint, took a look at the toolbar of the rich text editor; reviewed all the comments the question had received since last summer, and created a new patch. Petr told me it is okay to assign the bug to him, as he will be working on the editor before Moodle 2.0 release.
• Commented on issues, such as MDL-6820 MDL-20461,  found on the CANnect Accessibility report by Randall Hansen (MDL-20409)
• Commented on the new Moodle 2.0 Dock navigation’s interactive behaviours: MDL-21529

Also, jotted some usability related bugs down:

• MDL-22393: password recoverz functionality tweak
• MDL-22249: wrong mime type for an image file

Part II to What is a course & the tools for having a great one, with exciting new UI design for an old challenge is coming out at the beginning of June. Stay tuned!

Inspired by Tomaz’ blog post, I did an informal interview with a business and marketing teacher I know. There are two separate points I want to make about the interview, so this article starts a series of two articles.

I wanted to go thinking on a very general level of what are the tools that can be used for helping individuals learn on a given theme. I will here call the place to do such learning, a course.

The questions I presented:

What constitutes a course?
What are the defining factors; what do you do on a course, how, and why? In other words, we playfully tried to generate a definition of a course.

What kinds of tools can be used in order to facilitate learning of individuals on a course?
Then I asked the interviewee to list the tools that can be used for learning in each aspect of the course’s definition. “Tools” are defined very widely here, as anything that can facilitate learning on the theme. They may sometimes have natural hierarchy, but here I want to perceive them such that each we can each still see the relations differently.

The definition here is of necessity more narrow than that discussed by Tomaz – I believe that restriction helps when thinking about the design of a platform for courses.

The main bullets of the list below are what defines a course. The sub-bullets are tools for reaching the goal/factor of definition in question. Text in italics is my reflection on what a given point could mean to Moodle.

A course (instance) is defined by:

• The defined content learning process of the course – (however loosely “content” is defined); taking into account the gradual learning of an individual in the course. She described one such process she uses to me (below)
  • Schedule (how much more emphasized could it be in the course frontpage UI for a student that there is an explicit schedule for a course?)
  • Teacher/facilitator as the one who defines or facilitates definition of the process and content
  • Learning theory being used
  • “Traditional” teaching methods: lecture, guidance counseling, …
  • Theme discussions
  • Study circle (a specific kind of forum)
  • Course community itself (as a tool for learning)
  • Private discussions for gaining trust between teacher and participant
  • Deadline for returning assignment (as deadlines are seen as something that is defines the course, should they be associated to the course on the UI level so teacher can see all deadlines she has defined at once, in addition to being defined in the module itself?)
  • Student awareness of eventual evaluation
• Community
  • Teacher as facilitator of building common trust
  • Introductory forum, sharing what expertise participant can bring into the community
• Evaluation
  • Guiding/counceling discussion (chat; private messages?)
  • Calculator - could calculator or excel be used as a stronger metaphor in the UI whenever Moodle assists the teacher by calculating grades or when the db module estimates the reliability of a student grade?
  • Exam
  • Peer review (apparently Blackboard does not allow anonymous peer review?)
    
• Learning goals of course: generic learning goals, specific learning goals
• Content - a clearly defined theme
• The external representation of the course: the criteria a student uses for determining if they want to take part on the course
• The intended position of the course in the curriculum: If the course is given as a part of a curriculum, including the prerequisite courses/understanding of student
• In the personal learning process of the student, where does the course fit; i.e. how what is learned in the context of the course fits in the mental model of the student

(I exhausted both myself and the interviewee before getting to the tools for the last parts of the definition.)

The point here is when designing a learning management system, we need to give teachers tools that are needed, in the context of their goals, taking into account their perception/definition of what is a course. The reason for defining a tool so broadly, allowing both very abstract to very concrete tools, is that whatever a teacher uses, is an expression of the teacher’s need in reaching a goal. So anytime a teacher states to use a tool, it is something we can consider as a potential need Moodle can fulfill, one way or another.

She also outlined a content learning process on an online course she has:

1. Before course: Reading assignment of book(s) chosen by participants on the topic, two-page reflection writing assignment
2. Making sure communication channels are open (valid e-mail addresses), clarifying the rules of the course
3. Introduction: members present themselves and the existing knowledge they bring into the community (forum)
4. Questions assignment: Think of two or three questions one should be able to answer after reading the books you read for the first assignment (forum)
5. Based on questions assignment, teacher defines themes that are central to the participants. Participants are asked to give feedback if themes they think are central are missing
6. Teacher creates forums for each theme selected. Teacher asks students to discuss the themes. Teacher may participate herself.
7. Teacher asks students to summarize the discussion in one of the forums with a theme.
8. Assigment to apply the gained learning in a real enterprise
9. Learning diary
10. Feedback form on the course

Here, the principal lesson to learn is that the modules that the teacher requires are not that many, but each time the teacher uses a module they do so in a specific context. It is these contexts Moodle needs to understand if it wants to be distinguished as

• a tool that really supports learning and those facilitating that learning, or
• a pile of tools you can use, if you learn a sufficient number of combinations of arcane settings by heart.

Presets of module settings, intended for a given purpose, labeled according to the pedagogical purpose, is the main means to get there. Instead of/in addition to just allowing the teacher to label a forum according to the pedagogical purpose they want to use it for (“Introductions”), have a Moodle preset module that is exact fit for that user goal.

Of course, knowing what the presets teachers actually need so we can serve, say, 80% of teachers, requires both interviews and ethnographies to really know what most teachers think is meaningful.

Why? Because the guy promotes the idea of having tests and running them all the time to prevent your code from becoming an enormous, unholy mess. Because when you have tests, you are not afraid of making changes. (In fact, you are effectively improving the user experience of programming¹.) You can play all you want, because you know exactly when anything in your code breaks as a result of you changing the code.

Guess what? It applies to usability, too. Three points:

What is a test? Essentially, it embodies what *should* happen. If, after having changed your code, that something doesn’t happen, you know you are in trouble.

Likewise, When you test usability, you expect the person looking at your UI to do something. When they don’t, you know you have two options: go back to the original, or make it better.

Debugging. When you test code, you may spot that oh, there is an error: the test didn’t pass. Ideally, debugging is so built into the process that you don’t really think of it as separate: since you have tests for every little part of the program, the bug may be pretty easy to spot.

When you usability test, when you notice an issue (and try to keep your calm so the test participant does not notice your frustration) and if your test participant is talking out loud like you have told them to, you learn the reason there and then, and the solution is often more or less obvious.

Lastly, tests are not something that lead to great design. Your code may still suck, but at least you have the courage to improve it since you can test whether your new fix breaks anything else. The important thing is that the tests exist so you can rely on them.

The same thing with usability. A test does not do anything for the design in itself. If you have failed to understand the user’s goals in the first place, a usability test will only show you how the user gets confused while doing the wrong thing in the first place, or while doing it in an unrealistic setting².

However, a test does describe what the UI was designed to do. When you have comprehensive usability tests for a UI, you can use those test tasks against any new version of the UI, and see if it still serves the purpose it was originally created for, and how well.

In ways, usability testing is just like unit testing: When you have tests defined and you regularly run them alongside development, you know your stuff is good.

If you don’t run them, you don’t know. More likely than not, what you create just does not hold together.

1. How Test-Driven Development Increases Overall Usability
   Programmers are People, Too (thanks to Iikku for this)
2. For example, Richard E. Cordes discusses having tasks that fit actual user needs in Task-Selection Bias: A Case for User-Defined Tasks. But the most important thing is to get started with the tasks that are more or less obviously the UI’s purpose.

During the iMoot conference I also got a couple of precious chances to hear about different community members’ usability efforts within their organisations. Turns out there are a bunch of people already doing usability related to Moodle, mostly inside their organizations. Even more people are interested, but the environment to discuss usability in the context of Moodle does not exist.

We need an environment where community members can make their usability efforts visible to the rest of the community:

• A place where people are encouraged to brag about what they have done for usability in their organisation, and to share what has been learned (usability test tasks, results, …).
• A site that could propose you directions to take with usability: give instructions interactively for usability testing, for instance.
• A corner in the community to chat in about usability, where you could share your frustrations with Moodle, and with doing usability work, and with usability issues in general.

Another aspect of this effort would be to visualize actual concrete usability data about Moodle.

• Have a hierarchy on the site for the high level to low level goals, and for red routes of the Moodle UI (of course, these need to be defined first).
• Allow people to link user interfaces (cvs/git), tracker issues and usability tests (containing test tasks and results) into these goals, although keeping the user goals as the starting point for everything.

The slogan? How about… We are all about the goals of the learners!

The magic I want to make happen: make usability visible socially since it is, at its foundation, a phenomenon of social artifacts. Engineers are creating artifacts to users, when they would be better off with other kinds of artifacts. If we can make this disrepancy obvious in the community’s social sphere, there would perhaps no longer be a need to try to convince software engineers of the concrete need for the work. As it is currently, it seems many perceive usability as something too abstract and distant for them to actually do something about it.

Before any of this though, I believe the first milestone is to do usability testing to determine the current level of usability. This is to set measurable goals for Moodle usability, and to prioritize the things a given part of Moodle is primarily intended for. The fun thing about the above vision is that it is easy to start small: first start filling in data for one activity module while it is usability tested, and then build on the vision I am proposing here, as we go. Even if we end up just creating a site for documenting Moodle’s current level of usability (as a side effect of doing usability testing), it is still worth it.

1. Silvia is someone who has since summer encouraged me in work with Moodle in a great deal. She is working for CVA, a Moodle partner, herself and we also met in EuroIA09 in Copenhagen to discuss where to take Moodle’s usability efforts.

Martin’s keynote also inspired me to create a small bug report in the tracker, my first one in months.

#imoot2010 Twitter channel

If you are interested, you can still register now for access to ALL the conference presentations, and today you can still join the discussions for a couple of hours.

Usability and Users’ Experiences in Moodleland

Quotations in the presentation:

• The Hotel Story: Henrik Werdelin: Flow over feature
• Jared Spool: Journey To The Center Of Design
• Leisa Reichelt:
  • Improving your site’s usability – what users really want
  • Practical User Research – UA Conference 08
• Steve Krug: Rocket Surgery Made Easy

Once the concepts have been defined like this, the rest is implementation.

Software engineers are indeed aware of the fact that we must understand what are the concepts of the target domain, and their relationships. (They may express this understanding in different terms, though.) When we do, we can turn them into programming constructs, be they classes and objects, or procedural code (in Moodle, PHP pages and functions).

What is missing from this image? The people using the system: the actual dynamics of what happens, the characteristics and the goals of the users (?), and the circumstances of the users. All of these are more abstract than implementation details and can not be described using programming code, yet taking or not taking them into account can dramatically affect whether software meets the actual needs of the people it is supposed to serve.

I had eight test subjects during three days usability tests last week. The tests took about 15 minutes each, but gave plenty of data. The main conclusions:

• Uploading images to the rich text editor in Moodle 2.0 has too many steps and they are well hidden. All 8 of the users really struggled in one or more of the steps (different users in different ones). If it weren’t for a test situation where users typically try more (since they assume the task is possible to do and there is social pressure), even more of them would have likely failed the task.
• The old ‘forgotten password’ form failed or caused struggles in 4 out of 5 tests. The new form caused no confusion in any of the three tests. (I intended to have an equal number of tests for both. This was a mistake on my part.)

This is discussed in the tracker item MDL-16597, along with proposed solutions (Updated September 2nd).

File Upload in the Rich Text Editor

In the foreseen Moodle 2.0, it is possible to upload images whenever there is a rich text field. Except for the [Choose...] button in the Insert/Edit Image dialog, this diagram/mockup is a very close image of the functionality in Moodle 2.0 HEAD I tested, although there were only three sections int

• Getting from the text editor to the dialog where you can select which image to upload takes five clicks. Users got lost in all four steps except the last one (pressing the “Browse…” button), most of them in several steps.
  • 1. Get to the add image dialog (4 users found toolbar button, 1 found the item in the right click menu for opening the dialog)
    • Failure: 2 subjects (needed my help to proceed)
    • Struggles: 3 subjects (searched how to do it for several minutes, trying clipboard and drag&drop etc. but continued on their own)
    • Passed this step without struggles: 3 subjects
  • 2. Click the ‘browse’ button (URL is a technical abbreviation and got many users lost; some complained that they do not know what it means; one user complained that URLs have nothing to do with uploading an image from the local computer and was confused due to that. Some users wrote the image name in the description/title field or both; they did not understand the difference between the fields)
    • Failure: 2 (1 needed my help to proceed; 1 gave up at this point)
    • Struggle: 2
    • Passed this step: 4
  • 3. Click “Upload a file” (after this clicking “Browse…” and finding desktop where the file was, was quick)
    • Failures: 0
    • Struggles: 4 (In general, users assumed ‘Local files’ to mean the local computer and first thought they should look there. Thus they  were not looking for upload anymore – apparently they assumed they were uploading already.)
    • Passed this step: 3
  • 4. Press “Browse…”
  • Passed this step: 7

Notes:

• One or two of the users did not recognize the rectangle under the file upload field as a button so got stuck for a moment there.
• One user did not understand they still had to click the Insert button in the first dialog to actually get the image in the document.
• Users ignored “Current files” section, some of them wondered what that is

Forgotten password form

The old form was confirmed misleading since it made 80% of the subjects fill both of the fields. Some of the subjects still did not understand what to do after the form gave the error message to fill one field or the other – apparently the visual (erroneous) message given by the form was a stronger clue to them. What the original forum thread reported was clearly right. I am really surprised such an issue has not been spotted before.

When it was too late, I noticed the Wordpress people have designed it even better, though than I did. How bitter: I did not believe it when someone told me having a single field might be a better solution. Now that I see it I think that might have worked better. (Especially due to the formslib bug we have that makes it impossible to have two forms on the same page and thus probably makes the current solution not accessible). Nonetheless, since we now have promising usability test results for my design and none for that of Wordpress, I still recommend keeping my design (patch) for now.

Other notes

• TinyMCE and other parts of the file uploading were in English, although the browser and rest of Moodle were in Finnish since all the test subjects were Finnish. This may have slowed users down but all the test subjects demonstrated during the tests that they understood the English they encountered.
• All test subjects were my friends and in theory this might have introduced a bias. However, as they were unfamiliar with the UI in question and I did not help them during their test taking, the issues they encountered seemed genuine. This can be disputed though and I welcome discussion about this style of low-fidelity testing.
• The test subjects were Finnish young people between ages 20 and 30. Three males, five females. Occupations: software development basic degree graduate, musician, social worker student, industrial engineering student, unemployed, interactive technology student, college teacher, kindergarten teacher

The videos taken (screen image and recorded voice in Finnish) are available on request.

I hope to explain how I did this round of testing and which were the parts where I made it easier for myself. There will hopefully be a blog posting or a video, to show the community how little work this can be.

The test preparatory talks and setting were roughly the same as in last summer’s Quiz UI tests, though less formal.

The test tasks

1. Käynnistä internet-selain.

Start up the internet browser.

2. Kirjaudu sisään Moodleen. Tunnukset löydät näytön alareunaan kiinnitetystä lapusta.

Sign in to Moodle. You can find the credentials in the piece of paper attached to the lower border of the screen.

[The username and password are invalid in reality]

3. Näyttää siltä, että vaikka sinulla on käyttäjätunnus ja salasana, toinen niistä on virheellinen. Ota selvää, mitkä oikea käyttäjätunnus/salasana ovat, jotta voit taas kirjautua sisään.

It seems, that although you have a username and password, either of them is incorrect. Find out, which are the right username/password, so that you can sign in again.

4. Testin järjestäjä avasi eteesi dokumentin. Lisää tietokoneen työpöydällä oleva kuva tähän dokumenttiin.

The organizer of the test opened a document in front of you. Add the image on the desktop of the computer into this document.