Readers are acclimated to ads, but you have to use good taste. “Many readers may blame you for misleading claims or ugliness in ads, and they may think the ads undermine your editorial integrity,” says Rennie. In a few fields, readers enjoy ads (e.g., SuperBowl ads, or fashion magazines), but in science and cultural fields, that’s rarely the case. They best you can hope for is that your ads are benign.

This article examines the ad business on a broad level, and looks closely at how to use or launch an ad network.  

Online advertising is big business

In 2011, global online advertising was ~$73 billion, comprising 16% of the $464 billion spent on ads of all kinds (print newspapers, magazines, television, radio, cinema, outdoor, internet). In four markets (Denmark, Norway, Sweden and the UK), internet advertising already accounts for more than 25% of total ad expenditure.

In a sign of the times, U.S. online advertising will exceed print for the first time in 2012. Print advertising continues to erode in the U.S., and is projected to fall from $36 billion in 2011 to $33.8 billion in 2012. Meanwhile U.S. online advertising is projected to grow 23.3% to $39.5 billion this year. Mobile ad spending is growing rapidly, $1.5 billion in the U.S. in 2011.

 Google is the top dog of online advertising, earning half of all global online advertising revenue, $36.5 billion in 2011. Another 11% of the world’s digital ad spending is earned by four other major companies, Microsoft, Yahoo!, AOL and Facebook. Collectively, these top search engines and portals have 61% of the ad market.

The other 39% of the market is most often served by ad networks. Publishers work with one of more ad middlemen — ad networks — who recruit and manage advertisers, serve ads, bill advertisers, and pay publishers. Google is also in this business, paying publishers ~70% of ad revenue. In 2011, Google paid out $7.3 billion to publishers who allow Google to place ads on their network. Other major ad networks include: ValueClick,  Tribal Fusion, AdBrite, Burst Media, Casale Media, isocket, and Blogads. (See top 50 ad networks from ComScore.)

Finding appropriate advertisers

In the realms of science and culture, these major ad networks may pay too little, or have irrelevant ads for your audience. Smaller advertiser networks “should be a good model for culture blogs, since it’s a valuable niche audience if you carefully manage the list of sites,” says Barry Hoggard, co-founder of an arts/culture ad network, Culture Pundits, which ran from 2007 to 2012. Hoggard says a custom ad network “allows the sites to have more control over the kind of ads that appear. General networks like AdSense can show some pretty lousy ads — credit cards, correspondence schools, etc.” Further, “when it’s working,” a custom ad network can pay higher revenue than a general network.

But if your primary audience is students and their educators, and the context is educational settings, ads may be inappropriate. If you run ads on these sites, since your site is positioned as an expert source, and you have a young, potentially naive audience, you have an extra responsibility to have nonintrusive ads that are not misleading.

If your audience is the general public, a large ad network like Google’s AdSense can be fine. A good quality ad network excludes misleading and scammy advertisers. We use AdSense on WebExhibits.org, and the ads tend to be nonobjectionable, drawing $0.64 CPC and $1.60 RPM in 2011. (Acronyms defined below.)

If you have a niche audience, of professionals in your field, or knowledgeable enthusiasts, you could  build your own pool of advertisers. Hoggard says a good minimum for a culture-oriented audience is 1 million page views a month. He says, “Below that, it can be hard to get the attention of a lot of media buyers.” However, ”if you have a niche audience with good demographics that is attractive to advertisers… you can be a lot smaller.” Rennie agrees with that ballpark, for science-related sites, and estimates that a “few million” monthly page views is the minimum for interesting major media buyers directly.

Culture Pundits ad network

Culture Pundits was a well regarded arts & culture ad network that operated from 2007 until January 2012. Culture Pundits was founded by Barry Hoggard and Tom Schreiber. Their sites served an audience of arts professionals and enthusiasts. Their member sites were all cultural blogs or art, film, literature calendars. They excluded sites that focused on design or fashion, although those sites have higher traffic. Their advertisers tended to be art-related institutions, galleries, publishers, museums and businesses.

They ad network launched in August 2007 with ten sites, expanding to 40 websites, and 1 million page views per months, by September 2008. By May 2009, managing the ad network became unwieldy, with advertisers requesting targeting of specific web sites, and geo targeting. To make things more manageable, Culture Pundits tried reducing the pool of advertisers to 10 sponsors each month, cut back to 25 sites, and reduced visual clutter by delivering a single medium-size ad (300×250 pixels) across all of the sites in the network.

The network paid 60% of revenues to its publishers. The balance of 40% paid sales commissions and administrative costs, with a little left over to pay the two founders’ salary.

Ultimately, the network, while profitable, was not a good business. Over 5 years, ad sales fluctuated. They sold out their full ad inventory 3 times, but most months, they sold only a third. When they closed down the ad network, they were drawing in ~$2k/month, paying out ~$1200/month to member sites — each site earning less than $100/month, on average.

Hoggard says, “It’s a lot of work to educate organizations about the value of a niche network like ours, versus just doing targeted Google AdWords campaigns. There is a lot of hand-holding.” His colleague, Schreiber, did most of the negotiations. Other tasks included managing the ad serving software, managing finances, and providing technical support to publishers and advertisers. There were often delays to get creative assets for ads, and also to get paid. In all, it became too time consuming.

While they had found a good niche, with satisfied publishers and advertisers, many of whom made repeat ad buys, Hoggard says, “In the last six months [of 2011], our revenue was really down compared to any other six-month period of the last 3-4 years.” Given the other projects he is working on, Hoggard decided to spend his time on making those more successful.

More is more: History News Network

George Mason University’s History News Network (HNN) has been running for over a decade, and has had ads the whole time. The site has over 300k unique visitors a month. Rick Shenkman, the site’s founder and publisher, says that up until three years ago,  ”publishers bought up most of our ad space. Today the publishers are barely advertising and we are mostly using ad networks.” Shenkman spends 6-8 hours a month managing his advertising. His approach uses a lot of ads:

The top ads are managed in-house, and link to pages at George Mason, or well respected TV, magazine and retail advertisers. They charge $400/month for  banner ads on the lower sections of their pages. When they don’t have interested advertisers, they are at the mercy of their ad network, which was promoting two misleading sites that hustle readers to get on mailing lists for commercial online and technical colleges, a scammy site selling questionable health supplements, and Johns Hopkins. At the bottom of their page, they run irrelevant text ads for discount men’s watches, ways to make money fast!, and a misleading site for Green Card applicants.

Less is more: The Deck and InfluAds

The opposite approach is to have a single, small ad on a site. Small ads usually pay 3-5 times better, and deliver a better reader experience than the the “crappy ad clutter culture that exists on the web today,” says Anibal Damião, founder and CEO of the ad network InfluAds. But total revenue can be less than packing a site with 5-10 ads.  This is what one ad looks like:

Single, small ads. Left: reformrevolution.com with an ad via InfluAds; Right: alistapart.com with an ad via The Deck.

The single ad approach was pioneered by The Deck network, which delivered 114 million ad impressions in January 2012. The ads are 120 pixels wide by 90 pixels tall and also allow for up to 80 characters of text to accompany the image. It’s an appealing model. The Deck charges $8300 USD per ad per month, and rotates 33 ads a month.

“[The Deck] inspired us and our objective is to democratize that model,” says Damião, whose network offers a number of niche topics, and runs small ad sizes (130×100, 154×70, and 250×250).  At Culture Pundits, Hoggard also used single ads, though he found that a slightly larger size (300×250) was “better-suited” for their arts & culture advertisers.

It’s an issue of quality of quantity, says Damião, “We are publisher + quality focused, while other networks are advertiser-demand + quantity focused.” For business-related sites, InfluAds pays around $4-5 CPM, and design-related sites tend to earn around $1-1.6 CPM. InfluAds has a full dashboard for publishers to monitor their accounts, and pays 10 days after the month.

Freethought & 3 Quarks

To see how this works, let’s look at how two blog networks have set up their advertising…

Freethought blogs is a blog network of 31 blogs, centered on a theme of independent thinking and skepticism. In December 2011, the site earned ~$10k. The blog network had 6.4 million pageviews in January, averaging 200k pageviews daily. The site is a spinoff of Scienceblogs begun by PZ Myers and Ed Brayton, and has some of the same contributors. They had ads from when they launched. “I spent several thousand dollars to get the network off the ground and it was the intent from the start to derive revenue from the network and to pay all of our bloggers based on their percentage of the site’s full revenue,” says Brayton.

When they first launched in August 2011, Brayton tried to manage the ads himself using Google Adsense and  other networks, putting them into a Doubleclick server. “But it was very time consuming and our revenue was pathetically low,” so he decided to turn advertising over to a private ad manager who runs the ads for independent news and politics sites, Raw Story, Alternet, Crooks and Liars, and some others — which immediately tripled their revenue. Brayton occasionally does direct ad sales, augmenting the advertisers solicited by his ad manager.

Freethought has five ads on each page: A  wider banner (728×90) on the top, and four medium size squares (300×250). In December 2011, Freethought had about 5.7 million page views, and their revenue through the Raw Story ad manager was about $6800, with a total CPM rate of ~$1.20. They draw additional revenue from a popunder ad from 3I Interactive, that earns about $800 a month, and they also run their own Google Adsense ads on our RSS feed and search feeds, which bring in anywhere from $400 to $900 a month. And he has some direct ad sales of ~$500 a month.

3 Quarks is a filter blog with diverse contributors, collecting information from around the web on science, design, literature, current affairs, and art. S. Abbas Raza, the founder and editor, says they had 430,000 impressions last month. The site launched in 2004, and they’ve had ads for over 5 years. Running ads was an easy decision. Raza says, “We didn’t wring our hands over it at all. We wanted to make some money, and so we ran ads.” He spends 2-4 hours a week managing his advertising. Their ad revenue varies widely.

The advertisements run in a strip on the right side of the screen. Raza is satisfied with his advertisers. They use Blogads, Google Adsense, and a few direct text link ads. Today, they have text ads for tickets at Las Vegas shows and Florida theme parks, and image ads for several books, a professional society, a legitimate institution promoting online courses on social research, and five nonobjectionable Google text ads.

Sponsored posts

Another form of advertising is sponsored posts, which can be appropriate for some sites. A good example of sponsored posts are these posts at business analysis blog Asymco for Carnegie Mellon University, and mobile app Scrivener. The sponsored posts on are topics the blogger would not normally cover.

The sponsorship network, The Syndicate, charges advertisers $2,750 USD for a single post on sites with combined 1,500,000 monthly page views. The Syndicate sends the blogger possible topics to approve, and draft text for the post. The blogger edits a short description of the product or service she or he’s willing to endorse, adding a personal note. Sponsored posts can be paid a flat fee, or the publisher can receive a commission for each click.

Advertisers like sponsored posts because they are an endorsement of sorts, and also catch readers attention. They also workaround the reality that many web readers use ad blockers, so some traditional ads get ignored. Brayton notes, “I don’t know how many total impressions we serve each month. Some of our visitors have ad blockers, of course, so it’s lower than the number of page views.”

Selling your soul?

More broadly, if you are thinking about adding ads, will you lose your  integrity? Not necessarily. There is vast precedent you can draw on.

“The first, most important principle is to avoid not just conflicts of interest but the perception of conflicts of interest, in so far as that’s reasonably possible,” says Rennie. “Conflicts of interest are bad for business because they make the editorial look untrustworthy and the advertisers look dishonest. Depending on the sensitivity of a topic and the audiences coming to read about it, the bar for avoiding those conflicts may get higher or lower.”

Traditional publishers have been dealing with conflict of interest for decades. The American Society of Magazine Editors outlines principles for the dance between editorial and advertising content:

• Readers are entitled to fair and accurate news and information
• The value of your site to advertisers depends on reader trust
• The difference between editorial content and marketing messages must be transparent
• Editorial integrity must not be compromised by advertiser influence

See their detailed guidelines, most of which apply to your site, or at least offer good food for thought.

“Most of the time when online publishers get into trouble with editorial-advertising conflicts, they’ve done something that goes against these longstanding rules,” says Rennie. “For example, in the notorious Pepsi-gate fiasco at Scienceblogs a few years ago, the publishers made the mistake of letting the sponsored blog have the same look and feel as the rest of the editorial network, without adequate labeling. (They made a bunch of subsequent mistakes, too, but this was the completely avoidable root error.)”

Talking the talk

The world of online advertising has its own language. A quick primer of important terms:

• Impression - Each time an ad is viewed. There can be multiple ad impressions per page.
• Publisher - This is you. The term is a holdover from the days of print marketing.
• Flat-rate - An ad runs on a site for a specific time period. Ad could be 100% of the time, or rotate with others, to appear with a specific probability.
• CPM - Cost per 1,000 impressions. (M is the Roman Numeral for one thousand.)
• CPC - Cost per click. Advertisers only pay when a reader clicks their ad, regardless of how many impressions. This is how Google’s ad system, and many others work.
• CTR - Clickthrough rate. A high CTR means ads are more appealing to readers. Number of clicks ÷ number of impressions.
• CPA - Cost per action or acquisition. Advertisers pay a commission for an action, such as becoming a new customer. e.g., if NetFlix paid a $20 commission for new customers. Similar to CPL, or cost per lead.
• RPM – Page revenue per thousand impressions. Earnings ÷ number of page views * 1000.
• eCPM - Same as RPM, but from the advertisers’ point of view. Effective cost per 1,000 impressions. CPC/A * CTR * 1,000.
• Text vs. Image or rich media - Text ads tend to be 1-2 sentences plus a link; Images are a graphic; Rich media are animated or have other interactivity.
• Mini, small - Small, subtle ads. They don’t annoy readers, and fit into many layouts.
• Skyscraper, Hi-rise - Vertically oriented ads, typically tall and skinny.
• Leaderboard, Marquee - Horizontally oriented ads, typically full screen width, and short.
• Roadblock - Exclusivity. Advertisers gets 100% of the advertisements on a site for a short time, useful for important announcements and product launches.
• Sponsored post - An alternative to ads; a sponsored post is part of the site’s content, and is clearly marked. Analogous to an advertorial, but not misleading.
• Above the fold – Newspaper display space is more valuable “above the fold” on the front page, as is the section of a website visible without having to scroll down.

Making your own

It’s possible to run your own ad network, but profit margins are tight if you focus on a niche audience. A new ad network could serve multiple publishers, or one blog network. While there are already a lot of science blog networks (see my post on blog networks last year, or this list from scienceblogging.org), there’s room for more blog networks that focus on specific fields (e.g., neuroscience, mathematics, etc.), or new networks that focus more on the general public and/or content that will be useful for educators to use in a classroom.

In culture and the humanities, there is a lot of potential for new blog networks, and many untapped advertisers. Culture Pundits solicited advertisers who were museums, galleries, shows and retailers. There may be potential interest from retailers (e.g., a retailer selling antiques for history-themed sites), online courses, and industry-specific vendors for sites that target professionals, and other advertisers.

To launch an ad network, your publishers will insert a piece of web code (HTML and JavaScript into their web pages, and that code will load the ads. You can make it as simple as one code = one ad, or run a complex system. Google’s DoubleClick for Publishers is free for small and medium-size sites, and provides extensive options for scheduling ads.  Hoggard started using a commercial ad manager, OpenX, for ad serving, but there were regular security/hacking problems. Later, they switched to DoubleClick, and it “worked very well for us.”

If you want to partner in making new ad network for a science or culture-related sites, and have over 1 million (preferably 2-3 million) views a month, InfluAds wants to expand their networks, and is a possible place to start.

Over 12k students received a 'statement of accomplishment' from Ng’s Fall 2011 course.

Over 100,000 students signed up for his free, fall 2011 course on machine learning. The impacts were huge. Over 12% of the students completed the course, and received a statement of accomplishment. Ng says he “heard many stories from students about how they’re using it at work, about how it’s inspired them to go back to school, and so on.” In contrast, Ng’s normal, for-credit course at Stanford, one of the most popular on campus, would enroll 350 students.

It’s part of a new revolution in higher education, and it’s serious learning. They deliver complete courses where students are not only watching web-based lectures, but also actively participating, doing exercises, and deeply learning the material. Students are expected to devote ~12 hours a week to the course, to read and watch course materials, complete assignments, and take quizzes and an exam. Online students did not receive one-on-one interaction with professors, the full content of lectures, or a Stanford degree — those who completed the course received a statement of accomplishment. Course materials include prerecorded lectures (with in-video quizzes) and demos, multiple-choice quiz assignments, automatically-checked programming exercises with an interactive workbench, midterm and final exams, a discussion forum, optional additional exercises with solutions, and pointers to readings and resources.

Ng promotes the Fall 2011 course, on machine learning:

Other Stanford professors experimented with massive enrollments in free, online courses.

In Fall 2011, professor Jennifer Widom offered a version of her introductory database course. Over 90,000 accounts were created, 25,000 students submitted at least some work for grading, and 7% of students (6,500) did well enough to receive a “statement of accomplishment.” Most of the material was a subset of what the Stanford students did. Widom created different exams for the public students, and led weekly video chat. One bonus, Widom notes, is that they made everything “100% error-free as the 90,000 public students found every conceivable bug.”

Similarly, over 160,000 students enrolled in professor Sebastian Thrun’s college course on Artificial Intelligence in Fall 2011, co-taught with Google’s Peter Norvig. They graduated 14% (23,000 students) from 190 countries.

Thrun promotes his latest course, on building a search engine:

The success of these courses have convinced Thrun and Ng that online courses are the future.

Since the success of his course, Thrun has spent roughly $200,000 of his own money, and raised venture capital, to create Udacity, a new online institution of higher learning independent of Stanford. (He had resigned from Stanford in April 2011 to focus more on his work at Google, where he has a senior position.) Last week, Udacity announced two classes — building a search engine and programing a self-driving car — with plans to eventually offer a full suite of computer science courses.

Udacity will be set up as a teaching institution, not a research institution. “At Stanford, priority is your research career,” says Thrun to Reuters. “That is counter to teaching 100,000 students, who generate 100,000 emails.” Stanford is the institution being disrupted, it’s not the institution doing the disrupting. Similarly, Thrun isn’t doing Udacity in direct association with Google. He told Reuters that that Udacity does fit quite easily into Google’s mission of making the world’s information available for free. “Having a clean slate is a better way to start … The last thing I want is people asking whether Google is disrupting education. Better to ask if Sebastian is trying to disrupt education.”

To read more about Udacity, and speculation on it’s impacts, see two recent articles:

• “Udacity’s model” by Felix Salmon, the finance blogger at Reuters on 31-Jan-2012.
• “Lessons and Legacies from Stanford’s Free Online Classes” by Tina Barseghian a blogger for KQED MindSift on 31-jan-2012.

Similarly, Ng and Stanford professor Daphne Koller have funded and launched a new company, Coursera, and are currently hiring staff, with “16 courses launching in winter and spring, with more on the way.”  These new courses are intended provide people with “access to world-leading education that has so far been available only to a tiny few. We see them using this education to improve their lives, the lives of their families, and the communities they live in,” they say on the interim Coursera site.

The focus on ‘world class’ education on a post-secondary level is a distinct difference from other successful online learning sites, such as Khan Academy, which also reach massive numbers of students. Khan has 2,800+ video lessons on academic topics that range from arithmetic to physics, finance, and history and 290 practice exercises. So far, Khan Academy has delivered these lessons over 116 million times, and is a popular tutoring site for K-12 course subjects like algebra. Khan Academy is also pushing the boundaries, recently expanding into humanities and cultural subjects, but they are still focused on concise lessons that consume a few hours at most, not a full course or degree.

Here’s an example of a Khan video about the geology of the Hawaiian islands:

There are other experiments with eliminating direct communication with an instructor. For example, University of the People, launched in 2009, is a tuition-free, online university targeting developing countries. The University has accepted 1,200 students (3% of applicants) from 121 countries so far, and charges a $10-50 enrollment fee, depending on the GDP of the country. They use existing, free online materials (not audio/video because of low bandwidth in the 3rd world), paired with extra structure and discussion.

MIT recently announced plans to launch an open platform for free online classes, MITx, offering certification for those who demonstrate mastery. This is an expansion on MIT’s last decade of work in creating a robust free online library of its course materials called OpenCourseWare (OCW), which includes 2,100 MIT courses and has been used by more than 100 million people. In May 2011, New York University began allowing students from University of the People to use their online credentials to apply to study at its Abu Dhabi campus.

This new breed of online courses — run by stellar educators with intense technical abilities, striving to serve massive numbers of students — is exploding the possibilities of online education. These courses eliminate traditional constraints: Students do not come to a physical classroom, they do not have class at the same time, and they do not pay for their courses. The instructors also have a lot of slack, in terms of the technical ease-of-use. Computer science assignments are good candidates for online assessment, and the self-selected pool of students is technologically savvy.

As these massive online courses evolve, the traditional education industry will have to respond.

See also my related post, on online courses for learning skills, which I recently wrote about. That article looks at courses at MoMA, NYT, and several for knitting. It’s different because the courses focus on delighting learners with personal — not professional — skills.

Structuring a course

The Museum of Modern Art (MoMA) currently has six 8 or 10 week online courses. The cost is $200 for self-guided courses, or $350 for instructor-led. The latter enroll 30-45 students. MoMA offers both knowledge classes, e.g., “Modern and Contemporary Art: 1945–1989,” and knowledge/skill courses, e.g., “Materials and Techniques of Postwar Abstract Painting,” in which students do hands-on work at home.

The instructor-led classes offer structure, socialization and personalization; whereas, the self-guided courses are about individual freedom, providing access to curated articles and video, with no live instructor facilitation nor social interaction with other students.

The studio-art offerings have weekly assignments. For example, students paint canvases using the  materials and techniques of iconic artists. They photograph their works in progress and finished, and upload them to discuss with other students and the instructor. Wendy Woon directs MoMA’s education department. She feels the 10-week timeframe has worked well for studio art, allowing enough time for a sense of trust and community to develop in the discussion forums so that students are willing to have “critical conversations” criticizing each other’s work.

At the completion of the course, MoMA students receive a certificate by email.

Each week’s lesson includes a Art History video lecture on a specific artist, 3-6 videos from the gallery with a ‘visual unwinding’ of the artists’ techniques, articles, slideshows, suggested readings, links to artwork on the MoMA site, and a detailed studio video demonstrating the techniques of an artist. The studio course totals about 20 hours of video footage. The art history survey course has 25% less video since there’s no studio component.

Texas artist Carol Wickenhiser-Schaudt says: ”Unlike being in the museum, [we were] able to view the pieces up close, and things were pointed out to us that we may never see if we were there in person.”

The New York Times Knowledge Network offers a variety of online courses, generally taught by Times staff. The network is a good example of a traditional publisher exploring new revenue models and ways to inform the public. The courses are diverse, ranging from the arts to business to science. Some courses offer credit via partner schools.

Many NYT courses focus on personal skills, typically various kinds of writing (e.g., writing medical memoirs), last 4-10 weeks, and cost $200-1,000.

The 4-week “Writing A Medical Memoir” course ($225) was created and taught last year by Times reporter Denise Grady. The outcome of the course is that students produce a personal memoir of their experience with illness or injury. A good friend of this blogger took the course and found it very meaningful.

The course began by reviewing four essays that illustrate superb writing, and then the instructor coached students in writing their own essays. There were daily self-paced lessons, online discussion forums and web links, and scheduled online chats. The chat sessions and course contents were archived for students to review any time in the future.

Online learning is flourishing in the knitting and crocheting community (see my post about the Ravelry online knitting community).

Craftsy/Sympoz is an internet startup based in Denver, Colorado, offering 35 courses on various crafty topics, from knitting socks to quilting. Tens of thousands of students have taken their online courses, priced from $15 to $60 for unlimited access. They also offer occasional free courses to spike interest.

Craftsy courses consist of a series of video lectures, written documents, open Q&A-style discussion, plus video bookmarking, note-taking and private messaging. There is no fixed schedule, students come and go freely.

User interfaces

The best user interfaces are specifically designed to delight learners, make content easily accessible, and facilitate easy discussion. The Craftsy courses are the best of the three. Here are some screenshots:

In Craftsy, there’s a clear progression of steps through the course, easy access to course materials and notes, and an active, threaded discussion. The videos are designed to be displayed on screen, as part of the course.

The MoMA course interface is more cluttered and dense, but they have a lot more content to cover, with many videos, text, and images for students to review in each course. The course’s web software runs on the Haiku learning system. Screenshots follow:

The NYT Knowledge Network user interface is hard to use, ugly, and inefficient. They use the Epsilen course management system, which is designed for school administrators, so it includes lots of irrelevant features like grade books and quizzes, while key functions are buried or broken: It takes multiple clicks to read single messages from fellow students, and uploading documents breaks their formatting so they are hard to read. Following are two screenshots, personal details obscured. As a work-around, Grady says she used email to directly communicate with her students.

Course development

The NYT medical memoir course had the simplest development process because there was not extensive curricula to prepare before the course launched. Grady took the better part of a day to plan four short intro videos, and a few hours for a casual shoot. Unlike the knitting videos, which need high quality closeups, the purpose of Grady’s videos was to set the scene, and they kept it “very quick and casual. Not a huge big deal. My hair looks awful.” Grady says she ”taught it the way I would have taught a regular classroom course, except they could see me and I couldn’t see them.” Once the course started, however, the workload became very substantial, consuming all her free time and more.

Shooting a video of El Lissitzky at MoMA for an upcoming course.

The MoMA courses are much more complex to prepare. The most time consuming part of creating courses is planning, video production and editing, says Woon. Shooting videos in a large museum like MoMA requires a lot of coordination between departments. It takes around 10 months from concept to launch. The development team is two full-time digital learning staff plus an adult & academic programs staff member, and an intern. The team collaborates with the instructors to build the course and manage an outside crew who shoots and edits the videos.

They start by running an instructor-led course, and then spin off a self-guided course the following semester, with some improvements based on first semester evaluations. The self-guided courses are edited to be flexible, since self-guided students may take a nonlinear sequence through the material.

Students have a range of experience levels, from complete novices to working artists with long exhibition histories. Students are predominantly female, aged 30 and older, and 55% of them are from the U.S. Woon says student’s primary motivation is “their belief in MoMA’s authority in modern art” and “they expect a high-quality experience.” Secondary motivations are “flexibility and accessibility”. Wickenhiser-Schaudt, the online student, said the classes are for serious students who do not live close enough to the MoMA to take in-person courses.

The knitting courses have a shorter production cycle. Courses are planned by the instructors, who typically work in the craft field as authors and teachers. Instructor Linda Permann said it took her a few months to outline and develop her course. Once the course is developed, there is a ~3-day video shoot (using HD video) with the Craftsy team and a film crew. Professional video editors produce finished videos. Above is the studio of Carol Ann Waugh during a Craftsy filming.

Craftsy loads the content into their proprietary course management system, developed by their parent company, Sympoz, which handles the technical details of course registration, displaying video and written materials, and online community. Sympoz is expanding their formula to other topics, such as cooking and personal finance. Nearly 400,000 people have created accounts with their site. At this time, the Sympoz backend is not available to use for your own projects.

Human touch

These courses may be online, but the human experience is vital for all of them.

At MoMA, charisma carries the day. Students praised instructor Corey D’Augustine’s charm and the fact he brought real-life knowledge as a practicing artist, art historian and conservator. Here’s a promo video, to get a feel for his style:

Student interaction is a vital too. The online discussion forums were buzzing with approximately a dozen active student participants. D’Augustine said that, “week to week, I’m amazed at the discussions that have taken place between students in, say, Moscow and Miami, Caracas in Colombia, or Croton-on-Hudson in New York.” Dallas artist Deborah Rhee said in a blog post that it was “a full-time job keeping up” with discussions in a section of the web site known as ‘The Cedar Bar.’ UK artist and art teacher Mania Row says she is still in touch with her fellow students.

The availability of the instructor has a big impact: in one D’Augustine course, he had other obligations at the same time, and students complained he was aloof. In another course, students said he was attentive and responsive.

Rhee said the course was “actually more intensive than being in class for a few hours a day,” and said her “personal art practice has most definitely matured” from the first MoMA online course she took. Rhee feels “it is the best investment I have made for myself and my Arts practice since beginning my journey in Art a decade ago.” Similarly, Wisconsin artist Myrna Leigh said in a blog post that the painting class she took in Fall 2011 was “informative and fun,” and she introduced the new techniques she learned into her work. Enthusiastic students continue the discussion on an official MoMA alumni Facebook page.

Craftsy’s instructors are charming and idiosyncratic. They directly address the viewer, and convey sincere enthusiasm. They’ve honed their stage presence with years of in-person classes at local stores, craft conferences and retreats. The good instructors are diligent about logging in daily to answer student questions.

Meanwhile, each Craftsy course builds an archive of questions and answers, and general chit-chat. There’s enough usage volume, and students will often reply to other students. In crafts, everyone is an expert on their personal experiences, and knitters love to talk about their favorite yarns and colors, and commiserate over small mistakes. The chat in the paywalled communities of Craftsy differs from that on the free community at Ravelry in that the discussions tend to have a much more on-topic “course” type of feel.

An instructive experience

Here are some views of the teaching experience:

Denise Grady is an experienced science and health reporter at The New York Times. She says she “dreamed up the course and got excited about the idea of helping people tell stories about life-changing events. Forgive me if this sounds trite but it is kind of a labor of love. It’s very intense, and rewarding. Quite fascinating. People show up with remarkable stories. It sells out every time.” The course doesn’t bring in much revenue (approximately $6,500), and Grady still had her full-time newsroom responsibilities while teaching. “The time commitment was huge,” Grady says. “Tens of hours, is the best way I can put it. Hours of preparation for each online session even though they were only an hour apiece. Many many hours reading drafts, commenting on them and returning.” Due to the workload and drain, she declined to teach this spring, and another Times journalist, David Corcoran, is teaching the course instead. Grady hopes to teach again in Fall 2012.

Donna Druchunas teaches three knitting courses (all priced at $60), and has over 5000 students enrolled. Craftsy is a significant income source for Druchunas (the $300k+ revenue is shared between her and Craftsy) and also a way to indirectly promote her books. She has authored several knitting books, and teaches in-person classes at local yarn shops and at knitting conferences and retreats on topics such as rug knitting, making yarn, and knitting socks and sweaters. She acknowledges that online courses make it harder to detect confused or frustrated students, but that’s mitigated by the clearer instructions in an online setting where students “can see things close up and replay the videos over and over again as often as they want.” While she can’t directly observe students working, she can comment on photos students upload. Druchunas spends ~15 minutes a day answering new questions in the courses.

Linda Permann teaches two crochet courses (priced at $30 & $60). She is a full-time crochet designer, writer and teacher. She has a few books, an ongoing column in Crochet Today magazine, and writes for other outlets. She finds teaching personally rewarding, and also useful to “stay in touch with crocheters and what is ‘hard’ for them.” She says that many communities lack convenient local crochet classes, “so I really wanted to jump in and provide a couple of intensive classes that would help people grow their skills.” Permann logs in once or twice a day to answer student questions, and her daily workload is ~20 minutes.

Stefanie Japel teaches three courses (priced $30-60) and a free mini-course, with a total of 16,000 paid and unpaid students. Japel is an author and teacher, as well as a staffer with Sympoz. A few years ago, she was a new mom, with a toddler and an infant, and she was unable to keep up with traveling to events for teaching. She started teaching online using Ning, where she started with shorter (4-8 week) courses, and amateur quality video. At Craftsy, the video production is a high level, courses never expire, and there’s a support team (designers, tech support, help desk, customer support). Japel invests an hour a week to maintain her three classes and the free workshop, answering questions and talking with students.

Japel has found the online and in-person courses complement each other. For example, in recent in-person courses she taught with Interweave Knitting Labs,  20% of the 25 enrolled students were already taking the online course and wanted to see the class live. Conversely, after the in-person course, several of the in-person students signed up for the online class.

Launching your own course…

Your organization could launch your own courses, adapting the models above as a starting point. The classes can have the side benefit of institutional branding; Woon says students report they feel connected to MoMA after taking the course.

Courses that require heavy involvement from an instructor don’t scale beyond a few dozen students, but they can be very meaningful learning experiences. The MoMA and NYT examples above involve large amounts of time and commitment from both students and the instructor. (Both MoMA and NYT also have self-guided courses that are more like online lecture series, and are more analogous to the Craftsy video series.) In the case of MoMA, there’s also a large up-front investment in video production. The Craftsy model is a slick and appealing presentation when most of the information can be taught by video. Economies of scale with the Craftsy approach allow for lower course fees, and substantially higher revenue.

Updates: 24-Jan: Corrections from Grady and Woon clarifying some details.

“The debate about which is better, face-to-face learning or online learning is fast becoming obsolete,” says Jennifer Berghage, an instructional designer at Pennsylvania State University. The common goal is that ”an online course should be, above all, engaging, so that the learner enjoys the learning and is able to not only assimilate it but retain it and apply it.”

Online courses are revolutionizing formal education, and have opened a new genre of outreach on cultural and scientific topics. These courses deliver a series of lessons to a web browser or mobile device, to be conveniently accessed anytime, anyplace.

An “online course is designed as a built environment for learning. It’s constructed as an experience that can be followed sequentially or can be accessed throughout the designated time period,” says Wendy Woon, director of education at the Museum of Modern Art.

It’s a directed learning process, comprised of educational information (articles, videos, images, web links), communication (messaging, discussion forums) and some way to measure students’ achievement.

“There is no single formula for what constitutes an excellent online course,” says Berghage, though in good courses the “student feels a great sense of community and investment in the endeavor.” But an online course is more than the presentation of information or lectures. “Online courses require interaction, direction, and feedback,” says Jean Mandernach, a psychology professor and director of the Center for Innovation in Research and Teaching at Grand Canyon University.

The following are some overall dimensions to consider for teaching your target audience or professionals in your field, and possibly creating a new revenue stream.

Academic/professional 

In academic (K-12 and higher ed) and professional (e.g., continuing education) settings, the student’s primary motivation is to advance their professional or academic career. Courses are designed around the objectives of the school administration.

The number of students in kindergarten through 12th grade in the U.S. taking an online course as part of their school was estimated by the U.S. Dept. of Education to be 1.8 million enrollments in 2009-10, 74% of which was High School. Of those, 62% were to recover course credits from classes missed or failed, 47% were dual High School and college credit, 29% were AP, and 27 were career and technical education. In Canada, ~4% of  K–12 students were engaged distance education. Full time K-12 cyber schools are also gaining popularity, currently estimated to be 225,000 in the U.S. In China, more than 200 online schools serve 600,000+ students (iNACOL PDF).

Soldiers stationed overseas can complete their coursework via online courses.

Enrollment is higher for higher-ed, where 31% of U.S. students (over 6.1 million students) took at least one course online in fall 2010, and annual growth continues at ~10%. Some higher-ed degree programs are entirely bought online, and are ranked by U.S. News. For-profit institutions such as University of Phoenix, DeVry University, and American Public University draw the highest enrollments, and have helped scare and propel the industry online, but have not yet earned academic respect.

Social studies teacher in a recorded video.

For academic online courses to work, parents and students have to be extremely self motivated. ”Unfocused, passive learners do much better in an environment where the adult owns the responsibility for the learning,” says David Haglund, Principal of the Riverside Virtual School.

Some K-12 school systems create their own courses, but most contract them from private companies like K12, and Connections Academy, a unit of Pearson. Some courses are fantastic, but many are poor, or worse. The NYT drew attention to the shortcomings of online learning and of K12 last month in “Profits and Questions at Online Charter Schools,” and the WSJ recently profiled two families using virtual schools, with and without success. In higher-ed there are also commercial providers, e.g., Straighterline charges $99 to register for a month, plus $39 a course, and thousands of less-selective schools now accept 23 of their accredited courses for transfer credit.

Professional learning and continuing education are great for online courses because students are motivated and focused. Startups like Treehouse, CodeAcademy, Lynda.com  target the professional skills market. Training marketplace OpenSesame aggregates training content from 100+ providers with 10,000+ courses. “We’re creating Amazon.com for courses,” said Tom Turnbull VP at OpenSesame.

Personal learning & happy students

For outreach, the best models of online courses are coming from the realm of personal learning. These courses are free of the constraints of K-12 and higher education, and are designed to delight and satisfy students.

Substantial numbers of learners have taken online courses offering personal knowledge. This kind lifelong learning is for “anyone who wants to continue learning and building new skills once their formal education is done — no matter where they live, what their degree of education, how old they are, or whether they do it formally in the classroom or use less traditional resources,” says Dan Colman, founder of Open Culture and director of the Continuing Studies program at Stanford.

There’s an expanding market for personal learning. DailyPath delivers “actionable lessons” by email, MightyBell lets users post experiences and goals that others can achieve by completing a series of steps. Craftsy offers delightful video-based courses on crafty topics.

Credit or not

Credit introduces many constraints, with standardized curricula, accreditation, and requirements to work with the ‘learning management systems” large institutions use to manage students and teachers. When credit is offered, the course builds toward student’s graduation, qualification or long-term career goals, thus, students (or whomever funds their education) are willing to pay substantial enrollment fees, and endure crummy experiences.

If credit and grading are not a concern, there’s a lot more flexibility. Several schools have been experimenting with putting quality audio and video recordings of charismatic professors from top colleges online. These can be viewed in a web page (e.g., YouTube), or downloaded to listen later (as iTunes podcasts). iTunes makes it easy for learners to access 350,000+ free lectures, videos, films, and other educational resources. To find the gems, Colman’s Open Culture site indexes 400 free online courses from top universities.

Stanford Engineering professors recently offered three of the school’s most popular computer science courses for free online in Fall 2011. These included assignments and online discussion, and thousands of students who completed the course received a statement of accomplishment.

Measuring up

In formal learning, assessment is the currency of progress. Good scores on assignments and tests are central. Creative teachers have invented other ways to assess learning, often by having students create projects and presentations.

Assessment is more challenging in a virtual world, where it’s harder to measure quantitative and intangible aspects of learning, and it’s easy for student to cheat with conventional assessments. Assessment should include “intangible aspects, and should also offer multiple means of demonstrating learning, including individual and group projects and online presentations,” says Liz Pape, president of VHS, a nonprofit virtual school and consultancy.

The more interesting question is how to incorporate assessment into personal learning, when there’s no credit. Assessment, if done in an appealing way, can encourage students, help them track their progress, and gain a sense of accomplishment. See my recent article on gamification for a variety of ideas on how to follow student’s achievements and progress.

Assessment is also vital feedback loop so you can improve your course.

Human touch

The more human, or human-like interaction the better.

With the instructor — Feedback from the instructor on an essay, assignment or quiz, or answers to questions prompted from a lecture. Online, the feedback can be private (by email or direct message), or public (discussion forum). The instructor can moderate the online discussions, “inserting some comments into the discussions to keep students on task, add clarity to a discussion, or ask another question to get students to think deeper,” says Pape.

With other students – Students can interact with the rest of their cohort in the discussion areas of a course. “This kind of course cannot be self-paced, because you need all the students interacting together around the course content at approximately the same time,” says Pape. Students enjoy working with other classmates in discussions, or perhaps group projects. Online discussion allows more students to discuss than could in a short in-person class, gives a space for timid students to speak up, and helps mute overbearing or bullying students.

A personal style — Lectures should give a personal feel, as if directly talking to the student via video chat. One approach is to use second-person phrases that directly address “you” the student, as in, “Now I’m going to show you…” and “so you’ve made a new discovery, that…” Use of second person has a long history in guidebooks, self-help books, do-it-yourself manuals, and other forms of writing that intend to address the reader without the instructor actually being present. Likewise, online assessment can lend a human touch, “Nice job! You got 3/4 correct!”

The sky’s the limit?

There are some practical limits. For example, an instructor can only provide personalized feedback to a relatively small number of students. Five minutes per question, for 25 student questions, is over two hours of work. If the feedback requires more critical analysis, online teaching quickly becomes a full time job. There are also practical limits as to the size of online discussion communities. Too small, and there’s no discussion, but over a few hundred students, and conversation gets swamped. For some courses, it’s appropriate to answer the most popular questions. Google’s moderator service allows students to vote on the most popular questions, allowing thousands of students to participate in choosing questions. Questions about assignments or technical issues can be public questions.

Despite these scalability hiccups, growth of all kinds of online courses will be explosive. The big money will be in for-credit courses, but there’s still huge potential for personal knowledge courses in areas of arts, culture and science.

Harvard professor Clay Christensen, well-known for his academic work on disruptive innovations, wrote a book, Disrupting Class, looking at changes in how people learn. In a recent interview, he said:

I think that not only are we ready but adoption is occurring at a faster rate than we had thought… We believe that by the year 2019 half of all classes for grades K-12 will be taught online… The rise of online learning carries with it an unprecedented opportunity to transform the schooling system into a student-centric one that can affordably customize for different student needs by allowing all students to learn at their appropriate pace and path, thereby allowing each student to realize his or her fullest potential….”

Sources: Lede robot illustration adapted from Stanford School of Engineering Initiative.

The delightful app presents 19 different objects in 3D, to spin and zoom, providing an immediacy that rivals seeing an object in real life. In fact, it’s better in many ways than peering at an object through a protective case because the objects can be spun through a full 360°, view under bright lighting, at high resolution.

Personal exploration

The free app presents the mobile visitor with a grid of objects (below, left):

Tapping any object brings up a spinnable 3D detailed view (above middle). The visitor can spin the object with their fingertip, or pinch to zoom in. Dragging up displays more of the caption, and tapping the corners or the bottom selects other objects.

A brilliant feature (above right) is a display of wax seal from the rear side of a 17th century Milanese painting. Wax seals are hard to make out clearly, but the app allows the user to drag the light source. Moving the light reveals the contours of the wax seal in a way that would otherwise be impossible. The visitor can see a two-headed eagle clutching a sword in one talon, and a royal orb in the other.

For more of a flavor of the app, watch the following 1:45 minute demo from the museum:

“Our hope is that this app provides a level of access and interaction that isn’t always available to the public,” says Nicole Anastasi, Assistant Registrar at UVaM who spearheaded the project.

Escape from behind glass

Anastasi says that museum staff have the “luxury” of handling objects and inspecting them closely from all angles, but visitors can only peer through glass enclosed storage (case at right).

“The iPad offers a degree of access that we’ve never before been able to extend to our audience. Rather than inspecting static images, virtual visitors are now able to “touch” the art and interact with it on a more personal level.  By downloading the UVAM app, iPad users are able to take our collection home with them.”

The app clearly fulfills it’s promise from museum director Bruce Boucher to “give users better access to the objects than simple, static photography, and will enabled details to be appreciated in a way that would challenge the naked eye.” The team did a good job, and the app has been well received. In the last two weeks since launch, it has 5 out of 5 stars in the U.S. app store, and enthusiastic reviews — a real accomplishment since most museum tour apps have been disappointing users.

See the app at iTunes.

How they did it

The app emerged out of discussions between Anastasi and two computer science professors at the university (Abhi Shelat and Jason Lawrence). The computer science professors were in the early stages of launching an app development company, Arqball. “We felt that interactive 3D graphics would help create amazing new educational materials,” says Shelat.

The app was funded internally by the university for an undisclosed budget, and took half a year from idea to completion. The app is free because “as a state institution, we are not allowed to make a profit on such things,” says Boucher. “In addition, we don’t want to limit our audience by naming a price–everyone should have access to the arts without worrying about fees,” adds Anastasi.

Arqball developed a photography rig with a stage, camera and lighting that were all computer-controlled. They capture images of the piece, rotating it a few degrees between shots, and then process the stream of images into a format that can be displayed within the app. Below, the team is photographing a Greek Aryballos, c 600-575 BCE. The Aryballos (flask) is on a white turntable, flanked by two light boxes, and the camera between.

Spinning an object on screen is effective, Shelat explains, because “people are very good at reading depth cues from motion. So a rotating model like ours lets then subconsciously develop their own ’3D’ model of the piece.”

Other kinds of 3D

The concept of spinning 3D (virtual reality) has been used on the web for years, but it’s surprisingly more intuitive with a fingertip on a tablet than dragging a mouse. Also, an app can contain high-resolution images that would be impractical to display in realtime on the web. UVaM’s app is the first of its kind from a cultural heritage museum.

In the sciences, TouchPress has created a series of well-recieved apps. Shown below are iPad screens from TouchPress’ eBooks about the chemical element gold (left), gold jewelry (middle), and the planet Jupiter (right). Many of the images are fully-rotatable and pinch-zoomable, as in the UVaM app.

Another kind of 3D has recently emerged on handheld devices, using “augmented reality” apps to superimpose 3D into a video of a real scene. A recent app (below) from the Asian Civilisations Museum of Singapore used augmented reality to advertise their summer 2011 Terracotta Warriors exhibition.

The museum promoted the exhibition via publicity flyers, newspaper advertisements, and a display at the Dhoby Ghaut MRT train station, with instructions for commuters to download the free app, and aim their camera at a specially-constructed symbol. Above are four screens from the app. At the train station, a commuter can download the app, point the phone at the symbol. The app translates the symbol into the statue, which can appear alongside real life. Instead of manipulating the 3D using a fingertip, the user tilts and turns the camera, and the statue appears to be standing beside the symbol. The app has 4.5/5 star ratings in the Singapore App Store.

A downside of augmented reality (vs. 3D images like the UVaM app) is that it is designed using realtime video game graphics, not real photography, so the visitors can not critically study the object in detail.

Future directions

Cultural heritage comes to life when the public is given access to the fascinating objects of the past, along with compelling backstories and context. But we can’t wait for publishers like TouchPress to expand their offerings. TouchPress is an interactive eBook publisher with high internal costs, and they need to partner with private collectors or traditional publishers to obtain their content. Their Solar System app cost approx $250,000 to produce, so unlike UVaM, they have to choose sensationalist topics presented at middle school to high school levels, to turn a profit.

We need more organizations like UVaM to create apps that present less common objects of cultural heritage, whether for the general public, or serious scholarship. Over time, as companies like Arqball hone their formula, the development costs will continue to fall, to be within reach of hundreds of museums.

The UVaM app is better than any other cultural heritage app on the market because it was designed to give the public a valuable experience that uses the capabilities of the device. It’s not merely replicating a paper brochure or audio tour.

If you are looking to connect the public with your collection, a great starting point would be to emulate the UVaM app. Building on their approach, you could improve on the model in a few ways:

• Start with an optional, simple, pictorial help screen (example at right), not a letter from the museum director, nor a help video.
• Include more objects, including unique visualizations like the wax seal, or microscopic or X-Ray views. (Using Arqball’s approach, an object takes approximately 20 MB for a high res set of 3D images, so a maximum app size of 2GB corresponds to 100 high res objects.)
• Adapt for the smaller screens of the iPhone or iPod Touch. (UVaM is planning this.)
• Make the captions more interesting (perhaps with a lay-person’s version, and a more academic caption for deeper information). Irrelevant information like accession numbers and copyright reminders can minimized.
• Add a toolbar and use obvious-looking icons.
• Provide adjustable font sizes. This is vital for visually-disabled visitors.
• Get social. Apps should not be closed silos, rather they should allow easy sharing of still images or text by email, Tumblr, Twitter, etc., or saving images to the device’s library or DropBox.
• Collaborate with cultural heritage scientists (who will probably share their content for free). Including some science broadens the appeal of the objects to a wider audience, and enriches visitor’s understanding. See some of the ways of examining artworks at WebExhibits.
• Link your data. Pool together content with other institutions — let’s have thousands of interactive cultural objects by 2015!

Want to make your own 3D app?

“The iPad provides a fantastic platform to showcase the arts. As advances in technology change the way we view and research the arts, its critical to stay current and to be as innovative as possible,” says Anastasi. “Our intent is to expand the 3D collection to hundreds of objects.”

Arqball wants to develop more educational materials that use 3D technology. They have reusable software from the UVaM app, and Shelat says they are trying to drive down costs further “to make this technology available to all museums.” The cost for an app works out to $1-2k per object, for 10-20 objects, and there are economies of scale if there are a lot of objects. Museums can further cut costs if their staff can operate the photography rig themselves.

Source: View of the Dhoby Ghaut MRT station from the terracotta warrior app publisher, Magma Studio. Pictorial help from the Discover app. 

To make some sense of the sea of metadata, Jenn Riley mapped 105 metadata standards for cultural heritage. 

A Tower of Babel

Standards are needed because metadata is only useful if we all use the same system. Imagine that you labeled your music collection with genres of “Classical” and “Hip Hop”, but your friend used the terms “Baroque” and “Rap.” You’d have trouble merging your collections.

A “taxonomy” defines terms, providing a common way to group things together. For example, the Linnaean taxonomy is a biological classification (taxonomy) set up by Carl Linnaeus, as set forth in his 1735 Systema Naturæ, with kingdoms, orders, families, genera. Nearly four centuries later, Linnaeus’ system is still generally in place for the Animal Kingdom, providing a common language for classifying the world’s animals. Many industries have unique, standardized taxonomies which are widely used.

“Search engines now have the incredible power to serve up millions of documents that contain words or phrases, and to do so almost instantaneously as users type. Taxonomy has a different power, the power to serve up a focussed selection of documents that best match the meaning of the ideas that users are searching for,” says Dave Clarke, CEO of Synaptica. “Search terminates with a set of results, but taxonomy never dead-ends; it constantly exposes alternative pathways and associated ideas.”

In the library field, taxonomies and metadata are a big deal, but “the sheer number of metadata standards in the cultural heritage sector is overwhelming, and their inter-relationships further complicate the situation,” says Jenn Riley, Metadata Librarian in the Indiana University Digital Library Program. Martyn Daniels, president of a publishing consultancy, notes that ”we find ourselves drowning is a sea of acronyms and names. We are familiar with some we have no clue about others.” To shed some light, working with designer Devin Becker in 2010, Riley created a map of the 105 most heavily used or publicized metadata standards. [Her diagram, a PDF, is summarized below.]

Map of metadata

Here’s an overview of the standards that Riley maps, with four ways to group the standards, by: domain, community, function or purpose. This is a useful starting point for finding a standard for your own taxonomy, or for thinking about how to classify your data.

Domains

The domain is the type of material the standard used for.

• Cultural Objects are works of art, architecture, and other creative endeavor.
• Datasets are collections of primary data, typically before they are interpreted. They may be collected by scientific instruments, or by researchers in the sciences, social sciences, humanities, or other disciplines.
• Geospatial Data is information about the geographic location, either as the data about geographic places themselves or the relationship of a resource to a specific location.
• Moving Images are resources expressed as film, video, or digital moving images.
• Musical Materials express music in any form, including as audio, notation, and moving image.
• Scholarly Texts are produced as part of a research or scholastic process, and include both book-length and article-length material.
• Visual Resources are materials presented in fixed visual form, and can be either artistic or documentary in nature.

Community

The community is the groups that currently or potentially use the standard.

• Libraries collect and preserve both primary and secondary material in support of research, scholarship, teaching, and leisure. Can include academic, public, special, and corporate libraries.
• Archives are organizations that collect and preserve the natural outputs of the daily work of individuals and other organizational entities, including traditional records management processes. Their emphasis is frequently on the context of the creation of the materials and their relationship to one another.
• Museums  collect and preserve artifacts from a given field with an emphasis on their curation and interpretation. Can include art, science, natural history, and many other types of museums.
• Information Industry are diverse organizations that make up both the public and the commercial Web. Can include technologies that support inventory and knowledge management, e-commerce, and the workings of the Internet.

Function

The function is the role a standard plays in the creation and storage of metadata. Some functions define the basic entities to be described, others define specific fields, others give guidance on how to record a specific data element, and still others define concrete data structures for the storage of information.

• Conceptual Models provide a high-level approach to resource description in a certain domain. They typically define the entities of description and their relationship to one another.
• Content Standards guide the creation of data for certain fields or metadata elements, sometimes defining what the source of a given data element should be.
• Controlled Vocabularies are enumerated (either fully or by stated patterns) lists of allowable values for elements for a specific use or domain. Includes classification schemes that use codes for values — such as the Dewey Decimal System.
• Framework/Technology encompasses models and protocols for the encoding and/or transmission of information, regardless of its specific format.
• Markup Languages feature specific aspects of a resource, typically in XML. They are unlike other “metadata” formats in that they provide not a surrogate for or other representation of a resource, but rather an enhanced version of the full resource itself.
• Record Formats are specific encodings for a set of data elements. Many structure standards are defined together with a record format that implements them.
• Structure Standards define, at a conceptual level, the data elements applicable for a certain purpose or for a certain type of material. These may be defined anew or borrowed from other standards. This category includes formal data dictionaries.

Purpose

The purpose is the general type of metadata the standard is designed to record.

• Data are standards whose purpose is to enclose the resource itself, possibly together with metadata or with added value such as markup.
• Descriptive Metadata standards include information to facilitate the discovery (via search or browse) of resources, or provide contextual information useful in the understanding or interpretation of a resource.
• Metadata Wrappers package together metadata of different forms, or metadata together with the resource itself.
• Preservation Metadata is the information needed to preserve, keep readable, and keep useful a digital or physical resource over time. Technical metadata is one type of preservation metadata, but preservation metadata also includes information about actions taken on a resource over time and the actors who take these actions. — e.g., information about conservation of a painting.
• Rights Metadata is the information a human or machine needs to provide appropriate access to a resource, provide appropriate notification and compensation to rights holders, and to inform end users of any use restrictions that may exist.
• Structural Metadata makes connections between different versions of the same resource, makes connections between hierarchical parts of a resource, records necessary sequences of resources, and flags important points within a resource.
• Technical Metadata documents the digital and physical features of a resource necessary to use it and understand when it is necessary to migrate it to a new format.

Riley says she created the map to “assist planners with the selection and implementation of metadata standards.” It’s an awesome starting point, and the map places the standards in the pie slices, depending on their connection to a category. See the diagram PDF, where you can zoom in and see the 105 standards mapped on the pies, and see how standards are interrelated. Riley also made a glossary about each standard.

There are many other metadata standards for the sciences and industry.

Keeping it simple

Apps just need to be good enough. No need to get too fancy or reinvent the wheel. While custom apps run from $25-100k, many vendors will create an app for you for less than $25k, and some for well under $5k. This is a summary of the vendors offering apps for less than $25k at yesterday’s 3rd Museums & Mobile online conference.

The key to a tour app on a budget is creating a ‘templated’ app. You upload content (“tour stops”) to the vendor’s online content management system (similar to creating blog posts), and then the vendor packages your content into their pre-existing framework, creating an app that gets submitted to the Apple or Android app stores.

Summary of vendors: 

TourBuddy: List of tour stops from 'Savannah Walking Tour', detailed view of a stop, and their GPS view. Good for large outdoor areas.

TourBuddy - $1-2k per app, with reasonably priced updates, and options for additional features. Inspired by the IKEA model of anchoring on a low price, yet delivering good design, Yvonne Jouffrault, says Tour Buddy is creating apps for organizations who want an affordable app, and don’t need a lot of customization. Jouffrault says GPS maps are one of their best features. The GPS works well for any large, outdoor location with a clear view of the sky. The map mode allows users to pan over a map to choose a tour spot, or see their location on a Google map. TourBuddy has created approximately a dozen apps so far, and their apps run on both Apple iOS and Android. Coming soon, TourBuddy is adding a web app option, a better organization of tour stops, and more visual styles. Later in 2012, Tour Buddy may add support for tablets and video. They have two apps with many ratings: their “Savannah Walking Tour” has a free and $10 version, and 4/5 stars in the Apple App Store; “Dutch Utopia” is free and has 3.5/5 stars. See pricing.

OnCell iPhone App: On the left, list of stops, and map view. On the right, a view from their sample web app.

OnCell – $1.5k for audio tours, $3.8k for multimedia. OnCell built their business creating audio tours run from cell phones, i.e., visitors call a phone number and enter stop numbers into the phone to hear recorded tour messages. Their call-in business has hundreds of clients in the US and Canada, including many major institutions like the Grand Canyon and 80+ other National Parks, the Met, the Smithsonian, etc. The call-in market is being disrupted by the tour apps, which don’t require visitors to make a phone call and burn their cell phone minutes. They have two apps so far on the “OnCell App” platform, the “New Orleans Jazz” app was published this summer, but has no reviews yet. See pricing.

TourSphere: Underground Railroad app, list of stops, a stop, and a keypad mode. Not shown is a non-GPS map mode.

TourSphere – $500-900/month for native apps. $400/month for web apps. They have a CMS for uploading content. No upfront fees, no contracts. The price depends on which combination of phone and tablet sized iOS or Android apps you make. Their ‘National Underground Railroad Freedom Center’ has 3/5 stars. See pricing.

Tristan: Browsing stops and viewing detailed information in the 'Canadian Museum of Civilization' and the 'Infinity of Nations'

Tristan – $8-15k per app for an exhibition guide and walking tour. Tristan’s authoring platform is called Autor. Vanessa Vanzieleghem, Tristan’s Global Sales Manager, notes that Autor supports up to 7 major languages, and publishes to iPhone, Android, Blackberry, iPad and creates a web app. Their more expensive, custom apps have ratings in the range from 2.5/5 stars (“Phillips Collection,” U.S. app store), to 3.5/5 (“The Art at Cowboys Stadium” and “Smithsonian: Infinity of Nations,” U.S. app store), to 4.5/5 (Canadian Museum of Civilization, Canadian app store).

GuideOne: Alaska Native Heritage Center app. Alternating between portrait and landscape orientations, the app has coverflow-like and map methods to browse stops. The stops play as videos. At right is a scavenger hunt, visitors go find the object and answer a quiz question.

GuideOne - $15k+ per app. Their “plug and play” apps draw on a number of preset modules: audio tours, zoomable floor maps and images, scavenger hunts and quizzes, membership/donation features, and links to social networks. GuideOne has lowered their prices and improved their offerings since the Spring. According to product manager, Juan Sanabria, their Alaska Native Heritage Center app launched in May (4/5 starts), the Inupiat Heritage Center app launching soon for iOS and Android. Their app for the Chrysler Museum of Art (Norfolk, VA) was released in February, but has only garnered one review. Coming soon is a Boston Historic Sites iPad kiosk and Android and iOS app launching in launching in Spring 2012. Their apps have an offline mode, but the option to download content into the Alaskan app didn’t work for me on a high-speed network.

KanvaSys' Biosphere app: List of tours, map mode (non-GPS), a tour stop, and empty search results for "ecosystem" with no alternative suggestions.

KanvaSys - $20k+ one-time, or $8k+/year if hosted. KanvaSys is part of Ideeclic, a Canadian company which began creating web sites and educational games for the Alberta and Quebec school boards. Their platform is called “inSitu,” and they can skin it for different apps. KanvaSys’ latest app, “Biosphere” for the Environment Museum is free, but doesn’t have many reviews in the app store yet. Their “Cleveland Metroparks Zoo” app has a 3/5 rating.

Adlib – €2k EUR/ year for a 3 year contract, plus the cost for underlying Adlib Museum system, a local collection management system. The advantage is that the app content comes from the collection management system, so it does not need to be re-entered.

CMS and custom hardware from Antenna International.

Antenna International - Full pricing is not disclosed. Antenna is a big player in audio tours. They have two billing models. Their ‘Universal’ plan involves Antenna taking the risk, owning all the content, and handling all the logistics of the hardware. All visitors have access to the tour as part of their admission fee, and the museum commits to pay Antenna approximately 50 cents per head for 3-5 years. So a million visitors = $500k to Antenna, with no upfront cost to the museum. Their other main option is ‘leasing’ which costs $17-20/month per device. As with the Universal plan, Antenna handles logistics (headphones, maintenance, repainting damage, staff training on best practices, and delivers sufficient devices to meet demand). Often museums have seasonable demand, so Antenna will ship over a few hundred (rarely as many as 1000) additional devices for a few months. A baseline of 250 devices would cost around $55k per year. Prior to launching their proprietary XP Iris device, which is a touch screen and number pad, they had options to buy iPod Touches, but Antenna is phasing out commodity hardware, and pushing customers to use their devices. A downside of working with Antenna is that you usually don’t own your own tour, and especially not any celebrity voices they might recruit, so you can’t spin the content into less expensive tours, post them to YouTube, or use them for any other purposes in the future. Antenna has a new CMS, called “publisher” which allows organization staff to rearrange a tour, or upload a new piece of content. Examples of the CMS in use, offered by Ken Husband, was the marketing department adding a “stop” promoting a sale in the museum shop, or the curators removing a stop for an item which is on loan.

Other vendors at the conference are higher-end, full service firms who create custom apps. These vendors are often vague about their pricing, though  vendors try to keep costs down by reusing features between apps. Full service firms at the conference were: Art Processors, EarPrint, Espro Acoustiguide Group, GVAM, and Imagineear.

Why native apps?

With a native app, visitors don’t have to struggle to read your web site into their small screen. And while you should have a mobile-formatted version of your web site (a web app), native apps give organizations better branding with an icon directly on audience’s home screen, and provides a smoother and richer user experience. Plus, a native app can work when there’s no internet connection.

One important detail: Make sure you are the official “publisher” of your app, so that when you improve the app in the future, or change vendors, your visitors automatically get the new version of your app.

Update: 27-Oct-11. Added new information provided by Tristan.

Three suggestions

For a real-world example on a recent project, designer Ben Snyder said on his blog, “the purpose of the user test is to get feedback about the new design to understand if there are any parts of the website that are confusing to users, and to test the site for hidden bugs that might prevent a user from getting the information they need” or prevent them from completing actions on the site.

Snyder considered three constraints: (a) Two week time frame; (b) Evaluation limited to Facebook fans and current users; (c) Inexpensive.

His conclusion was to use: Feng-GUI, Inspectlet, and Usabilla. Snyder says, “With Feng-GUI we will get some great data that approximates an eye-tracking study. With Usabilla we have a tool where we can actually conduct a user test. We can write the instructions for the user and they can take the test without need for us to moderate it. It will provide us with great feedback based on the outcome of the user tests. We will use Inspectlet to record the user studies and will generate additional data and heatmaps that will show us more directly how the users behaved on the website.” Their costs came in under $100, with $25 for 10 Feng-GUI tests; $7.99 for Inspectlet (the first week is free); and $49 for 100 tests on Usabilla (or 10 for free).

More options

Here’s a broader view of testing options, adapted from a recent list by Cameron Chapman, plus several additional suggested by commentors on her blog:

“Usability testing differs from focus groups in that it involves the observation of participants as they actually use the product,” said Ian David Moss, a development consultant who works in the Arts. “They key feature of usability testing that makes it different from most other kinds of feedback-gathering methods is that it is based on direct observation rather than self-reporting….So, rather than have people sit around a room and talk about (for example) how they might react to a new feature or what challenges they face in their daily work, you have people sitting in front of a computer and trying to navigate a website’s capabilities while staff members look over their shoulders and take notes.”

To do testing… see my recent post about usability testing vendors and systems.

In addition to collecting user feedback with testing, there are other approaches to good usability:

• Intuition (instinct) - Idiosyncrasy is valuable. Create a site or app that you really like, while also trying to think about how outsiders and users will perceive it. Try to clear your mind (for a fresh look) and ask yourself how you would want it to work. Many of the biggest ideas, e.g., Facebook, and many products from Apple started this way.
• Casual testing - You can catch the majority of usability problems by testing with just a few people. Use your friends, family and colleagues as Guinea pigs — and ask them for quick 5- minute feedbacks.
• Heuristics (abstract theories) - Follow a series of rules and best practice guidelines. Jakob Nielsen’s list of ten heuristics are a famous starting point, but there are many variations. Apple also has a classic list of human interface guidelines for engineers. Heuristics are useful framework and checklist, but merely following heuristics misses the forest for the trees.
• Tracking - Review statistics of what they click and tap. Services like Google Analytics show the most popular pages, and the sequence users take. Review this to make sure that important screens are reached, and that there are not deadens. Eye tracking and heat maps are also useful to make sure that important parts of your design are not totally ignored.
• Accessibility - Make sure people with different abilities (e.g., blind, or with cataracts) can user your resource. Often this means adjustable font sizes, or including audio versions.
• Internationalization - Adding translations/localizations for users to who don’t read English. Mandarin Chinese is the most widely spoken native language, followed by Spanish, English, Hindi-Urdu, and Arabic. See a list of languages by number of native speakers at Wikipedia for a reality-check about how many non-English speakers there are on earth.

These concepts apply to programs (e.g., real-life outreach projects) as well as web sites and apps. Arts management consultant David Dombrosky said, “I love the idea of re-contextualizing “usability testing” as a feedback mechanism for arts audiences. As with online usability testing, the devil will be in the design, but the concept is well worth pursuing.”

Gameplay has a lot to teach us about motivating participation through joy. ‘Gamification’ is a new term, coined in 2008, for adapting game mechanics into non-game setting — such as building online communities, education and outreach, marketing, or building educational apps. Here are some ideas for how to do it.

Achievements

Badges, trophies and points represent having accomplished something. Since antiquity, people have been honored with medals, crowns and other decorations. Wreaths made of bay laurel were awarded to Greek athletes, and worn by Roman poets (e.g., Ovid, at left).

Judd Antin, at Yahoo! Research, in a talk this summer noted, “Closer to home, the Boy Scouts of America’s iconic merit badges promote the acquisition of specific skill-sets as diverse as nuclear science and basketry. One of the first large-scale implementations of badges in online games began in 2002 with Microsoft’s Xbox Live service. Since that time, badges have become a fixture in many games.”

Achievements can be easy, difficult, surprising, funny, accomplished alone or as a group. FourSquare uses badges to promote location-sharing via “check-ins,” StackOverflow and Quora use rating systems to encourage members to write quality answers to posted questions, KhanAcademy uses cute meteorite badges to reward correct answers (see KhanAcademy ”Going Transonic” badge above left), and Wikipedia encourages hardcore contributors to post barnstars and WikiLove to other member’s profiles. Many shopping and social media sites have some form of member ranking.

“This has already occurred in education for a long time with things such as merit certificates and awards,” says Australian science teacher Alice Leung, but “gamification is more than that “because the game guides learners towards those goals, and gives constant feedback.”

It’s not about winners and losers, says Leung. Gamification leads to fewer “losers” because the education is personalized for each learner, and “students feel safe to take risks in their learning.” Rather than most students having to work at the same pace, with gamification, “students work at their own pace to gain achievements.”

Judd and his colleague Elizabeth Churchill outline five key psychological functions of badges:

• Setting goals: Badges challenge participants to reach a higher mark, and are best when they are just outside of comfortable reach, and when participants can see their progress.
• Instruction: Badges embody the social norms of a system, exemplifying activities and interactions that are valued — i.e., what participants should do. — In a social setting, a party organizer could reward positive social behaviors by assigning roles to event attendees (e.g., matchmaker, deep talker, explorer) and awarding prizes for fulfilling their roles.
• Reputation: Badges encapsulate a participant’s interests, expertise and past interactions — providing an easy way to gauge the trustworthiness of other people, the reliability of content, and assess whether a participant is a casual or fanatical community member.
• Status & affirmation: Badges serve as a status symbol, advertising a participant’s achievements and accomplishments without explicit bragging. Some people are highly driven by status rewards (displayed in leaderboards, class rankings, etc.), but most people are more driven more when their work interacts with others’ and when their recognition creates enduring artefacts (e.g., school newsletters, posters, wikis, blogs, etc.).
• Group identification: Badges bind a group together around their shared experiences, lend a sense of solidarity, and promote collaboration.

A limitation of achievements is that they are external motivators, and only a subset of people really care about external recognition, so don’t rely on achievements alone to drive interest in your project.

In a classroom setting, Leung cautions, “If gamification is implemented in a superficial way (just points and badges), it is just a layer of extrinsic motivation, which may work well for younger students but not for older students…” it needs to include “strong narratives, goal-orientated lessons and personalized learning.”

Other game mechanics

Many other game dynamics can help engage your audience. Dynamics that draw on the human psyche, create feedback loops, or lead participants to accumulate skills or accomplishments. Here are some more:

• Appointments are specific times/places a participants must participate. (e.g., FourSquare and geocaching are based on physical places; Farmville requires players to return to harvest their crops after a specific amount of time has passed after planting; last summer, nine Smithsonian museums cooperated in a mobile game based on SCVNGR called goSmithsonian Trek, played on iPhones or Android phones.)
• Behavioral momentum is people’s tendency to keep doing what they have been doing.
• Blissful productivity is a sense of accomplishment, which might be missing elsewhere in someone’s life.
• Cascading Information Theory says information should be released in the minimum possible snippets, as not to overwhelm.
• Community collaboration rallies people to work together to solve a problem or a challenge. Learners are more motivated if their success at tasks is dependent on other group members, not just their own scores. Cooperative motivators should be stronger than competitive motivators. (e.g., DARPA balloon challenge.)
• Countdowns give participants a short amount of time to do something, and can spike participation. Arcade games often have a countdown. (e.g., Bejeweled Blitz gives players 30 seconds to get as many points as they can.)
• Discovery or Exploration delight participants with the surprise of something new, sparking their curiosity. The element of surprise can come from unraveling a complex subject, or challenging preconceived notions. A slick presentation will attract attention from its technical novelty, but thoughtful curiosity comes from sustained engagement that makes learners think, gain productivity, filter information, or create. Discovery works because it is mostly an internal driver, but some people can be encouraged by giving them a bonus for exploring, e.g., how many new pages they read each week.
• Epic meaning lends a sense of achieving something great, awe-inspiring, and bigger than oneself. Meaning can drive people to participate in citizen science, or other crowd sourcing projects like Zooninverse. Meaning also comes from creating an environment that does exist, such as inventing characters, locations, objects; and from applying a skill to that environment (e.g., simulation and roll employing games). Richer learning happens when learners connect new learning to prior knowledge through their narrative structure. (e.g., The online game, War of Warcraft’s ongoing story line motivates players to devote hours to the game, and also work outside the game, where volunteers have created a huge wiki to help them achieve their individual quests and collectively their epic meanings.) What’s challenging or an interesting fantasy will vary from person to person, and vary over the course of  a person’s learning life.
• Free lunch is when when participants feel they are getting something for free due to someone else having done work. (e.g., Groupon gives participants the sense of a great deal because other people have also signed up.)
• Infinite gameplay does do not have an explicit end. (e.g., Casual games like Farmville have a static, positive state.)
• Levels are a system, or “ramp,” by which participants are rewarded for accumulating points. Often features or abilities are unlocked as participants progress to higher levels. Leveling is one of the highest components of motivation for gamers. There are typically three types of leveling ramps: flat, exponential and wave function. — An example in an online community is giving frequent contributors special perks, like the capability to moderate, or the ability to unlock new content.
• Loss aversion is the drive to avoid punishment. (e.g., In Farmville, player receive alerts so they remember to log in and harvest their crops, other games have decays of points which require active participation to maintain.)
• Lottery determines winners based solely on chance. This can create a high level of anticipation, but can quickly alienate losers.
• Ownership gives participants a sense of control, and fosters loyalty. In the game world, participants’ decisions have consequences; winning isn’t dependent on completely random factors. Empowering learning environments depend on making learner’s choices tied to significant and meaningful outcomes. Learners must feel they are capable of succeeding. Conversely, too many choices can swamp and frustrate a learner.
• Points are a running numerical value given for any single action or combination of actions. They are a form of achievement, and can indicate a participant’s progression in completing itemized tasks. Points can be delivered as virtual currency. Here’s a video of adding points to a recycling bin — making it an arcade game — dramatically increasing recycling.

  http://www.youtube.com/watch?v=zSiHjMU-MUo

• Quests & challenges - Challenges usually have a time limit or a competition, and Quests are a journey of obstacles which participants must overcome. — Learners prefer the right level of challenge, with clear goals and feedback on performance. Goals can be relevant for allowing a learner to do something new (functionally useful), feel emotional connection (fantasy relevance), or social relevance. Uncertainty also matters. If you know you will triumph, you stop caring. Uncertainty can be boosted by varying difficulty levels, hiding information, or otherwise randomizing.
• Reward schedules are a timeframe and delivery mechanisms through which rewards (points, prizes, level ups) are delivered. Three main parts exist in a reward schedule; contingency, response and reinforcer.
• Urgent optimism is extreme self motivation. A desire to act immediately to tackle an obstacle combined with the belief that we have a reasonable hope of success.
• Virality is a game element that requires multiple people to play (or that can be played better with multiple people).

You can combine these mechanics. In the following 20 minute video from TED, Jane McGonigal talks about the lure of the ‘Epic win,’ and how gaming can make a better world:

Keep in mind, gamification does not necessarily mean playing games, though there is certainly a place for games in outreach & education. Gamification is not serious games, and it is not playful interactions (see chart at left). Though, there is continuum from games with a purpose to subtly incorporating some principles of gameplay into other projects.

Gamification “really has little to do with games or video games,” rather it is about giving people proper, faster feedback says Ryan Elkins, an entrepreneur who started gamification platform company IActionable. “It helps new people learn what is expected of them and that they are on the right track. It gives experienced people reasons to continue by quantifying their intrinsic motivation. It helps provide context to users so they can make better decisions. It helps individuals track personal growth and progress with measurable goals and a path to mastery.”

No size fits all. People might be driven by (a) a desire for achievement and the prestige of accomplishment; (b) the joy and delight of exploration, satisfying their curiosity; (c) a draw to socialize and connect with other people; or (d) a thirst for competition. – Your audience (e.g., students, the public or your community) will all have their own unique motivations for learning, participating in you projects, or using your resource. — And you don’t want to accidentally alienate some of your participants who don’t care about petty tokens, or make the game elements overwhelm the core job to be done.

Two examples 

A classroom example is from Leung, who created a unit called ‘The Great Science Race‘ with game mechanics like narrative, quests and achievement badges. See her post with positive data on student responses.

The LA Times ran a story last year: “Michael Pusateri is a 43-year-old senior vice president at the Disney-ABC Television Group, but he still doesn’t eat his vegetables. So in October he joined Health Month, an online game that allows him to compete against 16,000 other users in striving toward his goals — which include cycling 80 miles a week and going on a weekly date with his wife… When he made progress, he earned life points and raised his ranking. When he failed, he lost points but could ask other players to take pity and ‘heal’ him by giving him virtual ‘fruit.’ The game prepared him for his first triathlon. ‘My wife has been after me for years to eat more fruit and vegetables and bring my lunch to work, and it was, ‘Next week, I’ll do it next week,” says Pusateri, an avid video game player and father of two. ‘Just because it was on this dumb website I actually did it.’”

Gaining popularity

Gamification is gaining traction as a word. The term was first used in 2008, and became more popular in late 2010 (see Google Trends graph at right). In online marketing circles, gamification tends to focus on achievements because they can be readily added to web sites and apps. Vendors like Badgeville, Bunchball, Bigdoor Media, and GetGlue jumped to deliver a service layer of reward and reputation systems with points, badges, levels and leader boards.

But gamification is much more, and is a useful mental framework for planning how to incentivize your audience to be active and productive.

Leung says, “You don’t fail in games. If you don’t pass a stage, you reflect back on what you need to change and improve on and you play again. This is a vital element of gaming that will vastly change students’ academic achievements.”

Sources: The structure and much of this article comes from  a list at the gamification wiki. Some background from instructional designer and blogger Dianne Rees, who writes about education psychologist Jerome Bruner’s work on intrinsic motivation in 1966, and Malone and Lepper’s 1987 taxonomy of intrinsic motivation. See also papers from a 2011 CHI workshop (PDF). 

Update 20-Oct-2011: Added several quotes from Alice Leung.

The codes are easy to generate. Several web sites and software programs will make the codes for you.

To read the codes, users need a QR reader app to take a snapshot of the code with their device’s camera. The app returns the decoded text or web URL.

In the photo at left, a pedestrian takes a photo of a QR code promoting an Andy Warhol show.

There are hundreds of barcode-reader apps (e.g., RedLaser and QuickMark for iOS and Android devices, and the Kaywa reader for dumber smartphones), and code-reading can be included in custom apps, e.g., a museum tour.

QR codes on the rise

QR codes are currently the best way to transmit information from the real world into an app. The codes were originally designed in 1994 for tracking car parts at Toyota, but they have recently exploded in popularity, and are becoming more familiar to the public. ComScore reported that  14 million American mobile users scanned a QR code in June 2011 (6% percent of all mobile device users). Linda Kelly at the Australian Museum recently surveyed 100 visitors, and 15% said they have scanned a QR code.

The EXPLORA Science Center in Frankfurt is using QR Codes to promote their exhibition (photo at right), and extensively within the exhibition. QR codes in the exhibits at Michigan’s Cranbrook Institute of Science link to Wikipedia entries and relevant websites like the census bureau.  The Heard Natural Science Museum and Wildlife Sanctuary posts QR codes to help its visitors find maps of the sanctuary. (See more expamples in the comments on this MuseumNext page.) Last year, NYC’s central park had a display of QR codes. Here’s a 4 minute promo video:

http://www.youtube.com/watch?v=7OCyfV_k2_g

Major advertising campaigns are helping propel QR codes into the public view. For example, this clever billboard shows you the rest of a Victoria’s Secret model:

Give visitors something extra

As an example, the Love Lace Exhibition (July 2011 – April 2012) at Sydney’s Powerhouse Museum has a free app for Apple iOS and Android devices. Visitors can scan the QR code on an object’s label, and see more information in the app

The QR codes are used by Love Lace visitors, but are not a hit. In the first month, the exhibit reported 572 iOS and 165 Android app downloads. People used the app an average of 5 times on iOS and 3 on Android (including when they are not in the exhibition). The QR feature was only used 844 times in the first month, with a 5% failure rate, and 2% of scanned code unrelated to the exhibiton. Visitors scanned fewer than 2 objects per app download, suggesting that visitor did not think it was worthwhile. Many objects were not scanned at all. App developer Carlos Arroyo says they intend to add more interactivity.

Annelies Valgaeren of the MAS in Antwerp says they use QR codes extensively to provide translations, and give objects a ‘voice.’

Nina Simon of The Museum of Art & History in Santa Cruz noted last month that there should be consistency in what the QR codes do, and visitors should know what they will get. In developing a woodworking exhibition which included QRCodes, her team added a sentence next to each QR code, explaining that it would yield. e.g., “Scan the QR code to see the inside of this cabinet (1 min slideshow)” or “Scan the QR code to watch the artist carving these pieces (9 min video).”

What do QR codes look like?

The QR codes (short for “Quick Response code”) are a two-dimensional version of bar codes, consisting of a box of black & white squares.  The codes can vary in complexity.

Encoding up to 15 alphanumeric characters does not require many pixels (top left), while encoding up to 395 characters requires a more densely packed matrix (top right). Also, the codes can be customized (bottom row). The codes employ error correction, so your designers can replace up to 30% of the space, and the code is still readable. However, if you get too artsy, your audience might not realize it’s a QR code.

Try them out 

QR codes are useful for promoting your work because they provide an easy way for the public to learn more about your project, and they are rapidly being adopted by many industries. Book publishers are using QR codes to link to additional content. Academics like Katy Meyers have added QR codes to her poster at a professional conference, where she got 30 more hits to her web site than usual. Washington, D.C. middle school teacher librarian Gwyneth Jones has created cool library scavenger hunts with QR codes.

If your organization is thinking about creating mobile content to complement a new exhibition, whether in an app or a mobile-friendly web site, think about what kinds of additional content, information, audio and video, interactivity or social media you can link to, so as to enrich your visitors’ experiences.

Wireframes and sketches are the intermediary between a conceptual plan, and the actual, detailed prototypes and specifications needed to build a project.

Use quick, coarsely-drawn sketches to plan and discuss your next project. Create several views (e.g., a home view, and interior/secondary views) to flesh out the overall design of your site or app. The sketch embodies how it will work, and what the overall user experience will be.

Be fast and sloppy — Create the wireframes or sketches on paper, the back of a napkin, or on the computer. Use any tool you are good at: A pencil, PowerPoint, Photoshop, etc. Specialized “wireframing” software may be useful, but are not necessary.  There are many styles (see below).

Cost effective — Creating a wireframe can be free (if you do it yourself), or a few hundred dollars (if you hire a freelance designer to work with you for a few hours). It pays for itself many times over by:

• (a) increasing the chance of a good result,
• (b) decreasing cost for a designer to create the final, meticulous prototypes,
• (c) reducing the chance of having to re-work the project later.

A concrete deliverable from a meeting — If you are having a meeting, consider having a designer (or visually-oriented person) work throughout the meeting to churn out a new wireframe every 5-10 minutes. Thus, the output of your meeting will be a concrete combination of a short written plan, plus some simple sketches —  the two key ingredients you need to launch your project.

Sketchy sketches — Working quickly tends to lends a sketch-like appearance. Here’s some examples, of varying degrees of artistry. These were created by designers, but you can make a rougher version yourself.

(Sources: Rachel Hinman’s 90 Mobiles in 90 Days, Pixelhalunke, Harold Emsheimer, Anthony Armendariz)

Paper mockups — Print out web sites, and bits of text from a word processor, cut them with scissors, and stick them on a board with tape. Use sticky notes and markets. Since mobile apps need to be operated by fingertips, it’s useful to make them real-size to test your ideas.

(Sources: Nat Geo GPS,  Nick Romanos, inTouch Weight Watchers, a game app, DogVibes.)

Schematics on your computer — Draw simple line diagrams using drawing software. Here’s a nice example for a Washington Post app:

You can mix and match styles. Also, it’s useful to cut up screenshots from existing sites. For example, if you are planning a new blog network, look at ten existing blog networks and news sites, and copy pieces you like from each place, and stitch them together with some simple boxes and lines.

It’s just a sketch — Just like a charcoal sketch can be the precursor to many styles of art (realistic, impressionistic, etc.), the purpose of your wireframes is to guide the overall functionality and appearance. At a later step in the process, a designer will methodically refine the layout and flesh out the right feel and design. Often the final prototypes will evolve, but the wireframe will have embodied the core functionality you need. Below is van Gogh’s sketch and painting of his bedroom in Arles.

Only in the beginning — Wireframes and sketches are great for brainstorming and are vital in the early stages of planning. They help realize what needs to go into a project, and how it will work. But once the overall plan is in shape, it’s time to graduate. The sketches that helped launch and plan your project will now be a hinderance. Simple sketches obscure the user experience, and most people have trouble giving meaningful feedback to overly-abstract sketches.

So once your overall plan and sketches are done, have a designer flesh out visually-pleasing prototypes — with real sample text, real images, and a real design.

Great examples online — For more examples and inspiration, look at  Jakub Linowski’s Wireframes blog, Hannah Milan’s collection of mobile wireframes at Moobleframes, a nice set of 25 wireframes and mockups compiles by Igor Ovsyannykov last week, Wolf Becvar’s Wireframe Wednesday, or the Flickr pool “I ♥ wireframes.”

LEGO models of every shape and size were spread on tables. The models were created by kids and geeky adults, on topics from fantasy, to trains, to animals, to space. Front and center, near the entrance, the NASA Aeronautics directorate set up a display with a large banner, some astronaut paraphernalia (e.g., gloves), and an assembly of LEGO models created by enthusiasts.

A steady stream of kids poured by the NASA booth. The LEGO event had 941 participants showing their models, and 16,500+ paying attendees.

Origins of the partnership

According to fair organizer Todd Webb, in 2009, NASA suggested some involvement in the show, but ”NASA involvement was small and most wondered ‘what is that guy doing sitting there?’” Webb realized that to make it work, they had to kick it up a notch.

From NASA’s point of view, summer 2011 was good timing. The Space Shuttle Atlantis recently landed, and the Juno mission would launch during the LEGO event. NASA looks for opportunities to promote science, technology, engineering and mathematics. It’s a priority for both President Obama and NASA Administrator Charles Bolden. According to Tony Springer, the head of communications and education for NASA’s Aeronautics directorate, it was also logistically easy: NASA Headquarters was a short drive away, so “it made sense for us to participate.” Springer’s department is interested in reaching both adult attendees, and youth.

Webb says, “I suggested NASA would make a cool ‘theme’ one year, then we’d have a special logo, t-shirts, a trophy, etc.” At the time, 2010′s theme was planned, so they agreed to do it in 2011. Webb has themes for the LEGO fair each year. Past themes have been “food and drink,” and “music.” He says, “Normally I wouldn’t be quick to accept a ‘corporate’ type of entity as our theme.  That’s not very fun.  But NASA isn’t too corporate.  And it’s generic enough — spacey — that we could encourage people to participate, build models for the theme.”

The NASA booth was much more professional looking in 2011, said Webb, and they had “even more cool stuff… and made a nice presentation.” NASA was responsible for their display, and brought their own table clothes, display cases, and props.

For Webb, the motivation was the “neato-factor,” and also providing appealing mood for his events. He said the NASA presence did not boost attendance. For LEGO enthusiasts, “LEGO is all the attraction needed, and is all we feed off of.”

Audiences & impact

According to Springer, NASA looks for events that are “STEM-appropriate.” Anything from an education convention to an air show. In general, it’s a nerdy crowd at a LEGO event. Springer says, “Many of those attending BrickFair events are already predisposed to STEM disciplines: “self-interested,” according to the term of art. We are trying to amplify that interest. Since they are so attentive to design and construction, devoted LEGO builders seem to have an innate STEM interest. We hope that one day they may make the decision to transition that personal fascination into a STEM-related career, which we think would be a very positive thing.”

Springer is looking for “quality and not necessarily quantity. That translates during a given event into solid interactions with attendees; substantive conversations with educators and students that we then follow up with actions like emails, phone calls, and further exchange of educational materials. Leveraging strong initial contact with subsequent, sustained relationships (online or in person) is how we define success.”

Implementation details

To make it work, NASA had to do some leg work and provide some swag. At LEGO events, the majority of participants are fiercely independent in their interests. Webb says annual theme tends to draw just a few builders. Adult LEGO fans “don’t like to be told what to build, and most just build what’s in their genes.  So I bribed them with big prizes (an autographed book donated by NASA and a large LEGO set).  The turnout of NASA MOCs was pretty good I think.”

In terms of promotion, there were a few ground rules. Administratively, NASA created a Space Act Agreement with the Festival. Due to NASA’s strict regulations on external use of its logo or those of any its research partners (e.g., Boeing), Webb’s event was required do something special for its NASA-themed Event Kit. While the kit itself could not include the NASA logo on its label, instructions or any photographs, Webb was able to include a special tile with the NASA logo and a plain white 2×2 LEGO tile as choices for “the very last piece to be assembled.” The instructions stated: “So now you have 2 tiles left.  We leave it to you to decide which to add to your model.”

Reaching other audiences? 

The LEGO event was a great fit, but can they push the envelope? Springer says NASA Aeronautics has no current plans for more daring events, like Comic-Con. That’s a shame, since there’s a huge overlap between science and science-fiction geeks (see post by  Whitney Clavin of NASA’s Jet Propulsion Lab  from the 2010 Comic-Con.)

Meanwhile, the folks at Goddard are exploring collaborations that cross over into the arts. NASA and Tor/Forge recently announced a new publishing collaboration to meld the sensibilities of strong science fiction with equally strong space science. It will team up Tor/Forge authors with scientists from the Goddard Space Flight Center. Tom Doherty, publisher of Tor Books, said in a statement, “When I was a boy, books by Isaac Asimov, Robert Heinlein and their colleagues excited me, inspiring a lifelong fascination with space and the science and technology that would get us there.”

Do it yourself

To do community outreach in the sciences or humanities, take a fresh look at local/regional events. Who can you partner with?

The following is the verbatim article…

Do We Really Need a National Weather Service?

By Iain Murray  and David Bier — Published August 27, 2011 | FoxNews.com

As Hurricane Irene bears down on the East Coast, news stations bombard our televisions with constant updates from the National Hurricane Center.

While Americans ought to prepare for the coming storm, federal dollars need not subsidize their preparations. Although it might sound outrageous, the truth is that the National Hurricane Center and its parent agency, the National Weather Service, are relics from America’s past that have actually outlived their usefulness.

The National Weather Service (NWS) was founded in 1870. Originally, the NWS was not a public information agency. It was a national security agency and placed under the Department of War. The Service’s national security function has long since disappeared, but as agencies often do, however, it stuck around and managed to increase its budget.

Today the NWS justifies itself on public interest grounds. It issues severe weather advisories and hijacks local radio and television stations to get the message out. It presumes that citizens do not pay attention to the weather and so it must force important, perhaps lifesaving, information upon them. A few seconds’ thought reveals how silly this is. The weather might be the subject people care most about on a daily basis. There is a very successful private TV channel dedicated to it, 24 hours a day, as well as any number of phone and PC apps. Americans need not be forced to turn over part of their earnings to support weather reporting.

The NWS claims that it supports industries like aviation and shipping, but if they provide a valuable contribution to business, it stands to reason business would willingly support their services. If that is the case, the Service is just corporate welfare. If they would not, it is just a waste.

As for hurricanes, the insurance industry has a compelling interest in understanding them. In a world without a National Weather Service, the insurance industry would probably have sponsored something very like the National Hurricane Center at one or more universities. Those replacements would also not be exploited for political purposes.

As it stands today, the public is forced to pay more than $1 billion per year for the NWS.  With the federal deficit exceeding a trillion dollars, the NWS is easily overlooked, but it shouldn’t be. It may actually be dangerous.

Relying on inaccurate government reports can endanger lives. Last year the Service failed to predict major flooding in Nashville because it miscalculated the rate at which water was releasing from dams there. The NWS continued to rely on bad information, even after forecasters knew the data were inaccurate. The flooding resulted in 22 deaths.

Private weather services do exist, and unsurprisingly, they are better than the NWS. When Hurricane Katrina hit New Orleans in 2005, the National Weather Service was twelve hours behind AccuWeather in predicting that New Orleans would be affected. Unlike the NWS, AccuWeather provides precise hour-by-hour storm predictions, one of the reasons private industry supports them.

It is not just random mistakes in crises either. Forecast Watch has found that the National Weather Service predictions of snow and rain have an error rate 20 percent higher than their private alternatives. “All private forecasting companies did much better than the National Weather Service,” their report concludes. In 2008, they found that the NWS’s temperature predictions were worse than every private-sector competitor including the Weather Channel, Intellicast, and Weather Underground. Even NWS’s online ZIP code search for weather reports is in some cases totally inaccurate, giving reports for areas hundreds of miles away.

NWS claims to spread information, but when the topic of budget cuts came up earlier this year, all they spread was fear. “There is a very heightened risk for loss of life if these cuts go through,” NWS forecasters said, “The inability for warnings to be disseminated to the public, whether due to staffing inadequacies, radar maintenance problems or weather radio transmitter difficulties, would be disastrous.”

Disastrous? The $126 million in cuts would still have left the Service with a larger budget than it had a decade ago. The massive bloat in government should not get a pass just because it’s wrapped in good-of-the-community clothing. NWS services can and are better provided by the private sector. Americans will invest in weather forecasting because if there is one thing we can be certain of, people will want to protect their property and their lives.

Iain Murray is Vice President at the Competitive Enterprise Institute and author of “Stealing You Blind: How Government Fatcats Are Getting Rich Off of You.” David Bier is a Research Associate at CEI.

Update 28-August: Updated introduction, added context.

It’s not “merely a larger audience, but a different audience,” says Sara Snyder, webmaster for the Smithsonian Institution’s Archives of American Art, who has recently started to use Wikipedia more. She says, “Our main website is geared towards an academic-minded or university-level student, researcher, curator, or professional art historian. Yet we have information and collections that may inform or appeal to a broader set of folks, such as younger students and art enthusiasts.  Wikipedia is a platform for trying to start serving those researchers too, without overhauling the current way we do business or our existing website.”

Go where the people are

Search Google for “dinosaurs“, “Jupiter,” “Genome,” “WWII“, “Impressionism“, or “Monet,” and the first hit is usually not a museum, school, or commercial publisher. It is Wikipedia.  Wikipedia ranks high in search engines for most non-commercial queries.

Wikipedia currently has 3.7 million articles in English, and an additional 15.7 million articles in other languages. The popularity of Wikipedia articles is a long tail distribution. A few hot topics are viewed millions of times a day. For example, on 23 July 2011, when soul singer Amy Winehouse died, her Wikipedia page was viewed 8 million times. More typical, the top 100 articles (topics like Steve Jobs, United States, Harry Potter, Miley Cyrus are read around 25k times a day, with traffic depending on spikes in public interest from current events. The end of the long tail includes hundreds of thousands of esoterica which are read only a few times a year.

Content lives on

Articles in Wikipedia have a long shelf life, and most last indefinitely. Whereas content on your web site can die of obsolescence, be buried in the archives of a blog, or otherwise lost through staffing and technology changes; content added to Wikipedia sticks around for years, and will further improved and reused by scholars and amateurs.

Wikipedia articles evolve. The median Wikipedia article has 16 edits, and 30% of articles have fewer than 10 edits. Articles are occasionally subject to vandals, ignorant editors, or editors with an axe to grind. But vandalism is usually fixed quickly, and most scientific and cultural topics are monitored by well-meaning volunteers. You can use Wikipedia’s “watch list” tool to keep track of your articles and fix problems.

Your organization’s name is not closely affiliated with the article, and potential flaws introduced to an article in the future will not be blamed on you; they will be blamed on Wikipedia.

Contribution logistics

Wikipedia’s rules require that contributors be single individuals, so your organization’s outreach department can not have a shared account. Wikipedia has grass roots beginnings, and has a systemic bias against organizations, corporations, and public relations. The intention is to maintain a neutral, objective approach.

Giving to Wikipedia means giving to the public. Content in Wikipedia, by definition, loses copyright. Any content you contribute to Wikipedia needs to be new or be public domain. Wikipedia must scrupulously avoid copyright infringement, so many volunteers watch to prevent content from being copied from other sites. The result is that educational and outreach staff can not re-post your own institution’s existing content to Wikipedia verbatim unless you release the source content (and label it as such) under Creative Commons Attribution-ShareAlike. Or you can follow steps to officially donate your own material.

You also need to disclose potential conflict of interest, but that’s easily manageable.

Keep with it

The process of contributing to Wikipedia is not user friendly, and there are a number of rude volunteers. These two problems have led to recent decline in the number of volunteers for the first time since Wikipedia launched in 2001. “A lot of it is convoluted,” said Wikipedia founder Jimmy Wales at a conference last week, “A lot of editorial guidelines … are impenetrable to new users.”

The learning curve to do meaningful work with Wikipedia is steep but short. But the personality problems are pervasive. Wales says the typical profile of a contributor is “a 26-year-old geeky male,” not professional educators, scholars, writers nor outreach professionals. By the time you contribute a few dozen articles, you will encounter obstructionist personalities. Keep in mind that these antisocial people play an important part of the Wikipedia ecosystem because they remove the thousands of petty vandalisms which happen every day. You will have to ignore bad attitude of occasional bad apples, and not get discouraged.

Think like an encyclopedia

Wikipedia is built on secondary and tertiary sources. Virtually everything you say in an article has to be cited, and verifiable in some other source. You can cite books, news articles, journals and magazines, but Wikipedia discourages citing primary sources. This policy is intended to make sure that all statements have passed through some kind of institutional filter. But it doesn’t make much sense for culture and science, which often have peer-reviewed research, artwork, objects, or data. The guidelines say, “secondary or tertiary sources are needed to establish the topic’s notability and to avoid novel interpretations of primary sources, though primary sources are permitted if used carefully.” Ironically, peer review (e.g., a paper in Science magazine) does not count; but if your institution has a complex internal editorial review, your in-house publications might count as secondary sources.

Getting started

It’s easy to get started on a small scale. In minutes, you can make an account, and start contributing. If you want to make large contributions or add pages, look over some of the orientation guides. These guides vary in quality and are occasionally contradictory — but there are “no rules” and most good contributions, backed up with secondary sources, are welcomed.

The best starting point for cultural and scientific projects is the Wikipedia GLAM pages. (GLAM is an acronym for Galleries, Libraries, Archives and Museums. It also incorporates other institutions such as theatres, zoos, botanical gardens, public broadcasters, etc.) Those pages will point you toward relevant regulations, and give you tips on getting started. Many of the GLAM guidelines also apply to science and technology related outreach.

The challenge remains of finding staff time and resources. The key here is to draw in volunteers. The GLAM pages have several ideas about how to connect with existing “Wikipedians,” get your own volunteers involved in Wikipedia, make”content donations” so there are more raw materials available for volunteers to work with, or set up internships dedicated to Wikipedia.

Internships

If you set up an internship, recruit a Wikipedian, and give your intern the job title “Wikipedian-in-Residence.” Archives of American Art currently has a summer Wikipedian intern, Sarah Stierch.

The biggest advantage, says Sara Snyder, of a Wikipedian intern, is that it’s hard to learn the ins-and-outs of Wikipedia without bringing in an Wikipedia insider who has already learned the ropes. Snyder says, “I have lurked and done light editing on Wikipedia for years, but I never understood what it was really about, how it functioned, why things are the way they are, until I had some people who were able to speak to me face and face and start explaining and answering my questions. Having [our Wikipedian-in-Residence] Sarah here has made me a thousand times more confident/competent as an editor and advocate.” A secondary benefit is that the residency raises awareness of Wikipedia among institutional leadership.

Since summer 2010, there have been nine volunteer interns (e.g., a grad student) or staff given the job title “Wikipedian-in-Residence.” (Four are in the photo at right.) The title is useful for the intern or staff person because it defines their job, and helps them build collaborations with other Wikipedians.

Your intern can collaborate with your organization’s staff to improve articles, organize local events which help excite volunteers, organize challenges or competitions which motivate more volunteers, or serve as your bridge to teach in-house staff about Wikipedia practices and values. Snyder says she would love to have more Wikipedians intern at her institution in the next few years.

Local events

Events like meetups and/or edit-a-thons boost energy of volunteers and staff. Snyder co-organized an event with Stierch in late July. Snyder was inspired by the enthusiasm of the ten Wikipedians who attended, and impressed by the intelligent questions and engagement. The event also puts a human face on Wikipedia for the Archive’s staff. Events have trickle-down effects, from attendees, to their colleagues in cyberspace, ultimately increasing the volume of work done by volunteers.

Traffic boost

Working with Wikipedia also boosts inbound web traffic. Wikipedia has long been a top-5 referrer for the Archives of American Art web site, but since June 2011 when Sarah started working, Wikipedia has been the #1 inbound referrer.

Sources not noted above: Wikipedia stats from WikiRoll; April Wikipedia Report card; and I attended the Backstage Pass Tour of the Archives of American Art, on Friday, 29-July-2011, organized by Wikipedian-in-Residence Sarah Stierch, and webmaster Sara Snyder.