The group blog features an interesting post by Tessa Lau, titled Three Misconceptions About Human-Computer Interaction, which raises a few interesting points. In my opinion, HCI is much more fundamental to creating interactive systems than people usually believe. In this context, I would like to refer to an interview with Patrick Baudisch that I recently read, in which he explains how he got started in HCI:

Doantam: How did you get started working on human-computer interaction?

Patrick: Without knowing it. I was a Ph.D. student in Darmstadt, Germany and worked on user interfaces for information filtering systems. A friend of mine saw my work and said “oh, I did not know you were in HCI, too”.

That was the first time I heard of that field.

A few months ago I decided to look for a better solution. I experimented with various solutions, including Drupal (with a lot of help from Wim Leers, thanks Wim!) and WikkaWiki (with a few tips from Dario Tarborelli) before finally settling on PmWiki. In this post, I will describe my rationale for choosing PmWiki over Drupal and WikkaWiki.

An advantage of Drupal is that it is extremely flexible. For instance, the website’s menu and corresponding URL structure can be completely customized. It also allows you to define content types such as “Course” or “Student” in a graphical way with the CCK module and visualize them with the Views module. However, this takes quite some initial work. Moreover, changes to the structure (e.g. allowing courses to have associated pictures of student projects) requires changing all this again. I believe that I should be able to change the structure of my homepage at any time. Adding a related publication to a software project or to a Master’s thesis should not involve a whole day of editing content types. Although I could use the Page content type for all my content, this is considered bad practice since Drupal only uses Page for static pages that almost never change. Of course, using Drupal only to provide a shared template and a global menu is massive overkill

In the same period, I read an interesting blog post on Academic Productivity about using a wiki to invisibly power a (research) homepage by Dario Tarborelli. A wiki seemed to provide a solution to structural flexibility, since it allows users to quickly rearrange content. However, as Dario points out, there are two problems that often make people suspicious of using a wiki for their website:

1. anyone can edit the content of a wiki
2. wikis look wikish

Dario demonstrates in the blog post that WikkaWiki can easily overcome both issues with access control lists and an easy-to-use skin system. He continues by giving a few advantages of using wikis for content management:

• they allow you to edit and modify content at the speed of light
• they allow you to easily embed all sort of contents in a page
• they allow you to set granular access privileges for specific pages, which is a very powerful solution for collaborative work with colleagues and coauthors

I started to experiment with a WikkaWiki-based homepage and was able to very quickly migrate my content and create a custom skin. I was impressed! There was a catch though. WikkaWiki did not support file uploads in an easy way, and custom markup (e.g. lists with a certain CSS class) seemed to be hard to achieve. Having the ability to upload files was essential for me, since I did not want to use an (S)FTP client everytime I wanted to add a publication or screenshot. Custom markup would be nice to style my publications list. Although WikkaWiki supports embedding HTML directly, it is obviously not ideal to have large amounts of embedded HTML in the content of a page.

Then I decided to have a look at PmWiki . I already heard about PmWiki before. Jori used it to power his homepage, and Peter also used it to power the Mobile and Pervasive Computing course website. Of course, the fact that 6th most frequent author in HCI uses PmWiki to power his personal homepage and the homepage of his research group helps as well A few other examples of PmWiki usage by HCI researchers or groups:

• personal website of Jakob E. Bardram
• the website of the in|situ research group in Paris, France
• personal website of Sean Gustafson

But the question was of course: how can I use PmWiki to invisibly power my website? I decided to post a message on the PmWiki mailing list asking for help, and got many helpful responses. One of the things I especially like about PmWiki is the helpful and lively community. Everything I wanted to do with PmWiki was either explained in the Cookbook, or pointed out by someone on the mailing list. As an example of a very nice website invisibly powered by PmWiki, take a look at the Village School website.

These are the steps that I followed:

1. Install PmWiki: this is very easy, just extract the archive and point a web browser to pmwiki.php (detailed instructions). PmWiki does not require a database, it uses flat files (yes, flat files!). That means you can easily copy a PmWiki installation from one directory to another (or even from one server to another).
2. Create a new skin by copying the default pmwiki skin in a new directory and give it an appropriate name.
3. Start editing the page template and CSS file to suit your liking. Hide typical wiki elements with CSS or by removing them from the template. PmWiki supports conditional markup so you can change the displayed page actions based on your login status. For more information, see this post on the mailing list and the documentation about page actions. You can also add conditional markup to hide the PmWiki documentation pages from the sidebar. If you refer to images in your CSS file, just keep in mind that the path is relative (e.g. refer to pmwiki/pub/skins/myskin/icons/trash.gif as background: url(icons/trash.gif);).
4. Set a password for editing pages.
5. Set up file uploads (disabled by default). Once uploads are enabled, PmWiki will automatically create the uploads/ directory when you upload a file. You might have to change the maximum upload size.
6. Set up clean URLs like Main/Homepage instead of pmwiki.php?n=Main.Homepage (preferably with mod_rewrite). This is a lot easier than it sounds due to the excellent documentation.
7. Done! Now you can start adding your content.

A few other random tips:

• You delete a page by replacing its contents with the word “delete”.
• Conditional markup is extremely useful to hide content that you don’t want to show yet. Just surround it with (:if false:) and (:ifend:) directives.
• PmWiki’s markup is very flexible. You can easily add CSS classes to elements (see WikiStyles) and create advanced tables (see Table directives).
• PmWiki supports wiki groups. This allows you to structure your pages in namespaces, so you can have for example both a Research.Activities and a Teaching.Activities page.

Finally, here is the result:

All in all, I am very happy with it. I especially like the fact that I can add or change content quickly, wherever I am. All I need is a browser.

No big surprises in the list of course, but nevertheless interesting to have a look at. The problem (at least in Belgium) seems to be that the rankings used for university funding are focused mostly on journals. The top conferences are often only ranked slightly higher than average conferences, meaning that two or three papers at a mid-level conference will be better in terms of funding than one paper at a top conference. Of course, there is always the prestige and international recognition you receive when getting a paper accepted at a top conference

In their prototype, the user carries a wearable projector that projects information on physical surfaces or objects. This is essentially augmented reality but without having to use VR helmets/goggles or other devices (e.g. mobile phones) to view digital annotations. Although there are already systems that use wall-mounted projectors to augment rooms with digital annotations (Jonathan Slenders also looked into this for his Master’s thesis), in this prototype the projector is mobile as it is simply worn around the user’s neck. The device could project useful annotations on physical objects, such as Amazon reviews on books or flight information on boarding passes:

<p><em>There is embedded content here that you cannot see. Please <a href="http://blog.jozilla.net/?p=239">open the post in a web browser</a> to see this.</em></p>

This reminds me of the topics that my student Ruben Thys explored for his Master’s thesis (paper, video). Our solution was a bit more clumsy than this one though (it required a mobile device to view information attached to physical objects). I also remember reading a related UIST’06 paper that described multi-user interaction techniques with handheld projectors.

Wearable projectors seem to be a promising approach to provide feedback in ubiquitous computing, and might help to further bridge the physical and virtual worlds.

I got the idea to document the way I read papers from a discussion with Lode (he uses mostly the same approach as I do). I’m pretty certain this is obvious for many people, but I hope it may still be useful for others.

In this post I focus on processing lots of (related) papers in the least amount of time. I will not discuss how you find interesting papers, since this is fairly easy. Reading prestigious conference proceedings or journals in your field usually allows you to end up with a couple of seed papers that are representative for your specific area of interest. If you do research in ubiquitous computing, reading the proceedings of Ubicomp, Pervasive or Percom, and journals such as Personal and Ubiquitous Computing will get you along quite nicely. If you focus on the human-computer interaction aspects of ubiquitous computing, throw in the proceedings of CHI and UIST, together with the TOCHI and Human-Computer Interaction journals. The papers you find here will be stepping stones to related papers (and conferences) using the simple technique I will discuss in this post.

I estimate that of all papers I read, I only process 1 out of 5 in full detail. Most of the time when I find a paper that seems interesting, I start with skimming the paper briefly. I focus mostly on the abstract, the last part of the introduction (which should list the contributions), relevant pictures in the main matter, and the conclusion/discussion. I do several passes over the paper to validate my early understanding of what the authors did. I also go through the related work and references. This allows me to see if the authors refer to work that I don’t know but which could be interesting (which I mark for later processing). This will usually take little more than 5 minutes.

Now I have a pretty good idea whether the paper is interesting to me. If it’s not, I just go through the interesting references (if any) using the same process and stop there. If it is interesting, I go one step further, treating the paper as a seed paper that allows me to find other relevant work.

If the paper is not too recent, there will probably be (lots of) other articles that refer to this paper. Even if the paper is fairly new, it is still worth looking up papers that already refer to it. Looking up these articles allows me to discover relevant related work and to see how others critically describe this paper. A critical review could contradict of the results of the paper (e.g. through replication, such as Shumin Zhai’s work on target expansion), or might offer new insights (such as suggesting a new direction for research related to the original paper, as Yvonne Roger’s Ubicomp’06 paper did for Mark Weiser’s original Ubicomp paper). Both types of critical reviews are very useful for your own understanding of the paper.

I usually use Google Scholar to find citations (and downloadable versions of articles), since it indexes the major publisher databases (e.g. the ACM Digital Library, IEEE Xplore or SpringerLink):

After entering a title in Google Scholar, it presents you with a list of matching papers. Below each item, there is a link with the number of papers that cite it (indicated in orange in the figure below). In this case there are 118 citations:

After clicking on this link, Google Scholar shows all articles that cite the previous one, sorted by their relevance (in other words, the number of times these articles are themselves cited):

This approach to reading papers has one disadvantage: your reading list will quickly grow. You can end up with 30 more papers to read starting from just 1 interesting seed paper (which is what happened to me when I looked for papers that cited Intelligibility and Accountability: Human Considerations in Context-Aware Systems by Victoria Bellotti and Keith Edwards). In my opinion, this is actually an advantage as it allows you to quickly find many related (and hopefully interesting) papers. Furthermore, because you’re skimming papers, you never spend more than 5 minutes on papers that are not relevant or interesting.

After I have found a significant number of papers that I want to read, I of course will go through them in more detail. Papers that are very relevant for my work or give me the feeling they deserve a more thorough reading (e.g. because they provide many new insights), will be read from beginning to end. A few examples that I also discussed in my blog are: Range: Exploring Implicit Interaction through Electronic Whiteboard Design, Evaluating User Interface Systems Research and two chapters from Beyond the Desktop Metaphor: Designing Integrated Digital Work Environments.

A few related guides:
* Research Techniques by Alan Dix (especially the slides about gathering information)
* Reading AI from How to do Research At the MIT AI Lab

Thanks to Lode for having a look at an early draft of this post, and for providing the example of replication in HCI by Shumin Zhai.

The paper is situated in the field of ubiquitous computing. The goal of Mark Weiser’s vision of ubiquitous computing was calm computing, where calm reflects the desired state of mind of the user. Invisibility in ubicomp is more about enabling seamless accomplishment of task than staying beneath notice. Just as a good, well-balanced hammer “disappears” in the hands of a carpenter and allows him or her to concentrate on the big picture, computers should participate in a similar magic disappearing act. Calm computing moves between the center and the periphery of attention. The periphery informs without overwhelming the user, but the user can still move to the center to get control. The implicit interaction framework presented in this paper contributes to how calm computing can be effectively realized. It allows to reason about the way users can mitigate actions the system takes, and is a good complement to Eric Horvitz’s work on mixed-initiative interaction.

Implicit interactions enable communication and action without explicit input or output.One way that an action can be implicit is if the exchange occurs outside the attentional foreground of the user (e.g. auto-saving files, filtering spam, and ubicomp interaction). The other way is if the exchange is initiated by the computer system rather than by the user (e.g. an email alert, screen saver, etc.). Although it may seem strange that something that grabs attention is implicit, the key factor is that the interaction is based on an implied demand for information or action, not an explicit one.

The implicit interaction framework divides the space of possible interactions along the axes of attentional demand and initiative:

Attentional demand is the degree of cognitive and perceptual load imposed on the user by the interactive system. Foreground interactions require a greater degree of focus, concentration and consciousness and are exclusive of other focal targets. Background interactions are peripheral, have less demand and can occur in parallel with other interactions.

Initiative is an indicator of how much presumption the interactive system uses in the interaction. Interactions that are initiated and driven by the user explicitly are called reactive interactions, while interactions initiated by the system based on inferred desire or demand are proactive interactions.

The implicit interaction framework builds on Bill Buxton’s background/foreground model. Buxton’s model assumes attention and initiative are inherently linked. On the contrary, this framework decouples attention and initiative in two separate axes. Buxton’s foreground corresponds to the reactive/foregound quadrant, while his background corresponds to the proactive/background quadrant.

An example: a word processing program that …

• auto-saves because you command it to is situated in the reactive/foreground quadrant
• auto-saves because you have set it to do so every 10 minutes is situated in the reactive/background quadrant
• auto-saves because it feels that a lot of changes have been made is situated in the proactive/background quadrant

Being proactive means the word processing program is acting with greater presumption with respect to the needs and desires of the user.

Designers can manipulate the proactivity and reactivity by (1) dictating the order of actions (does the system act first or wait for the user to act?); or by (2) choosing the degree of initiative (does the sytem act, offer to act, ask if it should act, or merely indicate that it can act?); or by (3) gathering more data to ensure the certainty of the need for an action or when they design features to mitigate the potential cost of error for the action. Even in the reactive realm, the degree of initiative can vary based on the amount that the user needs to maintain ongoing control and oversight of an action in progress.

Ju discusses 3 implicit interaction techniques:

• user reflection
• system demonstration
• override

User reflection is how the system indicates what it feels the users are doing or would like to have done.

A good example are modern spell-checking programs. Early versions had to be invoked explicitly, and engaged the user in an explicit dialog about potentially misspelled words to repair. Current spell-checking programs run continuously in the background allowing users to more easily notice potential errors. The implicit alert of this interaction is far more seamless than that of earlier spell-check programs. A similar example is the continous compilation used by modern IDEs such as Eclipse and Visual Studio. Earlier programs only showed compile errors when the user explicitly activated the compile command.

System demonstration is how the system shows the user what it is doing or what it is going to do.

In Range, the whiteboard animates its transition from ambient display mode (where it displays a set of images related to the workspace) to drawing mode (where users can make sketches and diagrams) as a demonstration-of-action that calls more attention to the mode change than a sudden switch would, and provides a handle for override (see later).

Override techniques allow users to repair misinterpretation of the user’s state or to interrupt or stop the system from engaging in proactive action.

This usually occurs after one of the previous techniques (user reflection or system demonstration) alert the user to some inference or action that is undesirable. Override is distinct from “undo” because it is targeted at countering the action of the system rather than reverting a command by the user.

An example of override in Range is that in the transition between modes users are able to “grab” digital content to use it as part of the whiteboard contents, or to stop the motion of objects that are being moved to make space for drawings.

The main contribution of this framework compared to prior models for implicit interaction lies in the key variable of initiative. Without this variable, it would not be possible to distinguish user reflection techniques from system demonstration techniques or to map the role of override.

In conclusion, a very interesting paper that offers a framework to reason about proactive user interfaces and make sure that users are always in control.

One of the samples is the script inquiry.py which allowed me to detect nearby devices in just a few lines of code. Here it detected mine and Petr’s cell phones:

I had to position the camera a bit lower, because it would sometimes not recognize my face when I was looking at the bottom of my screen. Having a dual-monitor setup doesn’t help either, since you should ideally position the webcam in the middle of the two displays to have good coverage. I positioned the webcam in the middle of the two displays at the bottom and tilted it upwards, which gives good results:

I guesstimated that if the system couldn’t detect a face for a period of 10 seconds, I would be away from my desk (or at least not paying attention to the screen). I played around a bit with pygtk and notify-python to create a status icon and show notification bubbles whenever my status changed. Here’s what happens when it detects that I’m away from my desk (notice the bubble in the upper right corner):

When I return, the system will change its status icon accordingly and notify me again:

OpenCV is fun!

I didn’t know where to start, so I searched for existing OpenCV face detection examples. I found a blog post by Nirav Patel explaining how to use OpenCV’s official Python bindings to perform face detection. Nirav will be working on a webcam module for Pygame for the Google Summer of Code.

I managed to rewrite Nirav’s example to get it working with CVtypes:

Here’s the code. Although it’s just a quick and dirty hack, it might be useful to others. It requires CVtypes and OpenCV, and was tested on Ubuntu Hardy with a Logitech QuickCam Communicate Deluxe webcam. You will need Nirav’s Haar cascade file as well.

import sys
from CVtypes import cv
 
def detect(image):
    image_size = cv.GetSize(image)
 
    # create grayscale version
    grayscale = cv.CreateImage(image_size, 8, 1)
    cv.CvtColor(image, grayscale, cv.BGR2GRAY)
 
    # create storage
    storage = cv.CreateMemStorage(0)
    cv.ClearMemStorage(storage)
 
    # equalize histogram
    cv.EqualizeHist(grayscale, grayscale)
 
    # detect objects
    cascade = cv.LoadHaarClassifierCascade('haarcascade_frontalface_alt.xml', cv.Size(1,1))
    faces = cv.HaarDetectObjects(grayscale, cascade, storage, 1.2, 2, cv.HAAR_DO_CANNY_PRUNING, cv.Size(50, 50))
 
    if faces:
        print 'face detected!'
        for i in faces:
            cv.Rectangle(image, cv.Point( int(i.x), int(i.y)),
                         cv.Point(int(i.x + i.width), int(i.y + i.height)),
                         cv.RGB(0, 255, 0), 3, 8, 0)
 
if __name__ == "__main__":
    print "OpenCV version: %s (%d, %d, %d)" % (cv.VERSION,
                                               cv.MAJOR_VERSION,
                                               cv.MINOR_VERSION,
                                               cv.SUBMINOR_VERSION)
 
    print "Press ESC to exit ..."
 
    # create windows
    cv.NamedWindow('Camera', cv.WINDOW_AUTOSIZE)
 
    # create capture device
    device = 0 # assume we want first device
    capture = cv.CreateCameraCapture(0)
    cv.SetCaptureProperty(capture, cv.CAP_PROP_FRAME_WIDTH, 640)
    cv.SetCaptureProperty(capture, cv.CAP_PROP_FRAME_HEIGHT, 480)    
 
    # check if capture device is OK
    if not capture:
        print "Error opening capture device"
        sys.exit(1)
 
    while 1:
        # do forever
 
        # capture the current frame
        frame = cv.QueryFrame(capture)
        if frame is None:
            break
 
        # mirror
        cv.Flip(frame, None, 1)
 
        # face detection
        detect(frame)
 
        # display webcam image
        cv.ShowImage('Camera', frame)
 
        # handle events
        k = cv.WaitKey(10)
 
        if k == 0x1b: # ESC
            print 'ESC pressed. Exiting ...'
            break

A known problem is that pressing the escape key doesn’t quit the program. Might be something wrong in my use of the cv.WaitKey function. Meanwhile you can just use Ctrl+C. All in all, the face detection works pretty well. It doesn’t recognize multiple faces yet, but that might be due to the training data. It would be interesting to experiment with OpenCV’s support for eye tracking in the future.

Update: the script does recognize multiple faces in a frame. Yesterday when Alex stood at my desk, it recognized his face as well. I think it didn’t work before because I used cv.Size(100, 100) for the last parameter of cv.HaarDetectObjects instead of cv.Size(50, 50). This parameter indicates the minimum face size (in pixels). When people were standing around my desk, they were usually farther away from the camera. Their face was then probably smaller than 100×100 pixels.

Just a quick note on ctypes. I remember when I created PydgetRFID that I tried to use libphidgets’ SWIG-generated Python bindings, but couldn’t get them to work properly. I had read about ctypes, and decided to use it for creating my own wrapper around libphidgets. Within a few hours I had a working prototype. When you’re struggling with SWIG-generated Python bindings, or have some C library without bindings that you would like to use, give ctypes a try. Gary Bishop wrote about a couple of interesting ctypes tricks to make the process easier.

I was curious to see if things changed. It seemed that both systems have improved in speed, but there were no shocking results.

There are some differences between our experiments though. Jordan wonders how long it took for me to commit my changes after adding. In fact, I didn’t do that directly but did a diff before committing. I don’t know why, I think I just forgot to commit. This clearly showed the big difference between bzr and git before and after committing. Bzr was a lot faster than git before committing, while git outperforms bzr after committing. I guess this is because git makes heavy use of data structures created during a commit.

Bzr took almost 4 minutes to diff after committing, while git only took about a second. This clearly showed a problem with bzr, it started walking the entire tree again, while git could immediately see that nothing changed. Bzr exacerbated the same problem when performing a status, it was unnecessarily slow. Jordan’s tests showed that the status problem was solved, but doing a diff when nothing has changed is still a major performance issue on big trees.

Committing a small change was a lot faster for both git and bzr. And it should! This is after all a really common operation.

Jordan did another comparison with bzr, git and hg, which showed that hg is actually fairly close to git in terms of performance.