O'Reilly Radar - Insight, analysis, and research about emerging technologies.

Visualization of the Week: 30 years of tech IPOs

2012-05-25T15:00:00Z

This week's visualization comes from The New York Times, which tries to shed a little light on Facebook's initial public offering by showing how it compares to the 2,400 technology IPOs that have occurred since 1980.

The visualization begins with a timeline of tech IPOs that runs up until last week. Until then, Google had the largest market capitalization with a value of $28 billion at its launch. The next image in the visualization series then adds Facebook to the mix — the animation makes the other IPOs literally shrink in comparison. Facebook's value at launch was $104 billion.

Screenshot from the New York Times' tech IPO visualization.

The bubbles for each company reveal their value at the time of IPO, the percentage change after one day of trading, and their value three years later.

Found a great visualization? Tell us about it

This post is part of an ongoing series exploring visualizations. We're always looking for leads, so please drop a line if there's a visualization you think we should know about.

OSCON 2012 — Join the world's open source pioneers, builders, and innovators July 16-20 in Portland, Oregon. Learn about open development, challenge your assumptions, and fire up your brain.

Save 20% on registration with the code RADAR

More Visualizations:

Publishing News: Kindle Fire and "your ad here"

2012-05-25T14:30:00Z

Here's what caught my attention this week in the publishing space:

Kindle Fire home screen may be for sale

Rumors flew this week saying Amazon plans to launch an ad campaign in which it will sell ads on its Kindle Fire home screen. Jason Del Rey at AdAge reports:

"Amazon is pitching ads on the device's welcome screen, according to an executive at an agency that Amazon has pitched. The company has been telling ad agency execs that they must spend about $600,000 for any package that includes such an ad.

"The ad campaigns would run for two months and also include inventory from Amazon's 'Special Offers' product. For $1 million, advertisers would get more ad inventory and be included in Amazon's public-relations push, according to this executive and an exec at another ad agency.'"

Del Rey says that "[b]oth agency executives have so far declined to participate, citing several concerns. For one, Amazon isn't guaranteeing the number of devices that the welcome-screen ads will reach, telling agencies that it hasn't decided whether the ads will start popping up on devices that have already been purchased or just on new devices."

O'Reilly GM and publisher Joe Wikert assessed the situation on his Publishing 2020 blog. He says this is just the beginning and that other ebook retailers are going to suffer:

"Given that Amazon's goal is to offer customers the lowest prices on everything, what's the next logical step? How about even lower prices on ebooks where Amazon starts making money on in-book ads? Think Google AdWords, built right into the book ... At some point in the not too distant future I believe we'll see ebooks on Amazon at fire sale prices. I'm not just talking about self-published titles or books nobody wants. I'll bet this happens with some bestsellers and midlist titles too. Amazon will make a big deal out of it and note how these cheaper prices are only available thru Amazon's in-book advertising program. ... Imagine B&N trying to compete if a large portion of Amazon's ebook list drops from $9.99 to $4.99 or less. Even with Microsoft's cash injection, B&N simply doesn't have deep enough pockets to compete on losses like this, at least not for very long."

Wikert concludes by asking: "Why wouldn't Amazon follow this strategy, especially since it helps eliminate competitors, leads to market dominance and fixes the loss leader problem they currently have with many ebook sales?"

The future of publishing has a busy schedule.
Stay up to date with Tools of Change for Publishing events, publications, research and resources. Visit us at oreilly.com/toc.

Apple calls foul on the DOJ

Apple this week filed a reply to the Department of Justice's antitrust lawsuit that was filed in April against Apple and five major publishers. PCWorld reports:

"Apple's reply to the court is in line with a statement issued by Apple in April after the DOJ filed its case, in which it said that 'the launch of the iBookstore in 2010 fostered innovation and competition, breaking Amazon's monopolistic grip on the publishing industry.' The company added: 'Just as we've allowed developers to set prices on the App Store, publishers set prices on the iBookstore.'"

The filing, entitled "APPLE INC.'S ANSWER," opens:

The Government's Complaint against Apple is fundamentally flawed as a matter of fact and law. Apple has not 'conspired' with anyone, was not aware of any alleged 'conspiracy' by others, and never 'fixed prices.' ... The Government sides with monopoly, rather than competition, in bringing this case. The Government starts from the false premise that an eBooks 'market' was characterized by 'robust price competition' prior to Apple's entry. This ignores a simple and incontrovertible fact: Before 2010, there was no real competition, there was only Amazon.

Reuters reports that in the filing, "Apple also denied that the government 'accurately characterized' the comment attributed to [Steve] Jobs." The DOJ's complaint (PDF) states:

"77. Apple understood that the final Apple Agency Agreements ensured that the Publisher Defendants would raise their retail e-book prices to the ostensible limits set by the Apple price tiers not only in Apple's forthcoming iBookstore, but on Amazon.com and all other consumer sites as well. When asked by a Wall Street Journal reporter at the January 27, 2010 iPad unveiling event, 'Why should she buy a book for ... $14.99 from your device when she could buy one for $9.99 from Amazon on the Kindle or from Barnes & Noble on the Nook?' Apple CEO Steve Jobs responded, 'that won't be the case .... the prices will be the same.'"

Apple's filing responds:

Apple denies the allegations of paragraph 77. The Government mischaracterizes on its face the alleged statement of Steve Jobs to the press on January 27, 2010, which simply conveyed that a publisher would not have a particular eBook title priced at $9.99 through one distributor and $14.99 through another. Apple's MFN provision would allow it to require the publisher to lower the price to $9.99 on the iBookstore. Apple had no contractual rights to require a publisher to require that it, or any distributor of its products, charge more for eBooks than it chose in a competitive market." [Reference link added.]

You can read Apple's reply in its entirety at Scribd.

It's time to hack digital covers

Craig Mod (@craigmod) mused on book covers recently in a piece on his website called "Hack the Cover," which also is available as a Kindle Single. He says the way we search for and discover books has changed:

"The covers ... on Amazon.com are tiny on the search results page. Minuscule on new books page. And they're all but lost in the datum slush of the individual item pages. Great covers like Mendelsund's design for The Information disappear entirely.

"Why? Because — What do we now hunt when buying books? Data.

"The cover image may help quickly ground us, but our eyes are drawn by habit to number and quality of reviews. We’re looking for metrics other than images — real metrics — not artificial marketing signifiers. Blurbs from humans. Perhaps even humans we know! And within the jumble of the Amazon.com interface, the cover feels all but an afterthought."

Mod argues that since readers can approach a book from any number of entry points, the entire book should be viewed and treated like a "cover":

"The covers for our digital editions need not yell. Need not sell. Heck, they may very well never been seen. The reality is, entire books need to be treated as covers. Entry points into digital editions aren't strictly defined and they're only getting fuzzier. Internet readers don't casually stumble upon books set atop tables. They're exposed through digital chance: a friend tweeting about a particular passage — and linking, directly, into that chapter ... To treat an entire book as a cover means to fold the typographic and design love usually reserved for covers into everything. Type choices. Illustration styles. Margins and page balance."

Mod's piece is a must read this week.

Related:

Amazon is, indeed, a disruptive publishing competitor

Amazon does a little Snoopy dance

Agile content models better address audience wants and needs

What publishing can learn from tech startups

More Publishing Week in Review coverage

Developer Week in Review: Oracle's big bet fails to pay off

2012-05-25T13:00:00Z

I've been taking the opportunity this week to do some spring office cleaning. Unfortunately, I clean my home office infrequently enough that at a certain point, cleaning it becomes more an exercise in archeology than organization. There's nothing like finding a six-month-old check you never deposited to encourage more frequent cleaning.

The same can be said for code, of course. It's far too easy to let crufty code build up in an application, and then be faced with the mother of all refactoring efforts six months down the road, when your code finally reaches a critical mass of flaky behavior. It's worth the effort to continually refactor and improve your code, assuming you can convince your product management that quality is as important as new features.

Android is almost out of the woods

It wouldn't be a Week in Review without the latest in ~~Godzilla vs. Gamera~~ Oracle vs. Google. Things aren't looking all too sunny for Oracle at the moment, as the jury in the case just threw out all the patent-related claims in the lawsuit. This doesn't leave Oracle with much left on the plate, as the case now boils down to the question of whether the Java APIs are copyrightable. That's a matter the jury is deadlocked on.

Like all things legal, this is going to drag on for years as there are appeals and retrials and the like. But for the moment, it appears that Android is out of the woods, at least as far as the use of Java is concerned. Of course, there's still all those pesky International Trade Commission issues keeping many Android handsets waiting at the border, but that's a battle for another day ...

Scripters of the world, rejoice!

For Perl developers, a point release of the language is a major event, as it only occurs roughly once a year. This year's edition has just been released, and Perl 5.16 packs a ton of improvements (nearly 600,000 lines' worth!).

Since Perl is such a mature language, most of the changes are incremental. Probably the most significant is further enhancements in Unicode support. Nonetheless, there should be something useful for the serious Perl developer.

FreeBSD bids GCC farewell

As the licensing on the GCC compiler has become increasingly restrictive, some of us have been wondering when the fallout would start. Wait no longer: The FreeBSD team has ditched GCC for the more BSD-friendly licensing of Clang.

GCC has spent decades as the compiler of choice for just about everything, but recent changes in the GPL have made it less attractive to use, especially in commercial development. With the Apple-sponsored Clang compiler now seen as a viable (and perhaps even superior) alternative, with a much less restrictive license, the Free Software Foundation may need to decide if it would rather stand on principle, or avoid becoming marginalized.

Got news?

Please send tips and leads here.

Related:

Intellectual property gone mad

Mobile's embedded irony

More Developer Week in Review coverage

Four short links: 25 May 2012

2012-05-25T10:00:00Z

Meet The New Boss, Worse Than The Old Boss -- transcript of a thoughtful music industry insider considering the effect of the net on the business. The other problem? I’ve been expecting for years now to see aggregate revenue flowing to artist increase. Disintermediation promised us this. It hasn’t happened. Everywhere I look artists seem to be working more for less money. And every time I come across aggregate data that is positive it turns out to have a black cloud inside. Example: Touring revenues up since 1999. Because more bands are touring, staying on the road longer and playing for fewer people. Surely you all can see Malthusian trajectory?
Kottke on Quarterly -- I eyed TED's book club and thought "hmm, interesting business model: you like my taste, sign up and I'll send you things". Quarterly is a "my taste as a service" service. (via Sacha Judd)
Pipe Viewer -- clever little command-line utility to show progress of pipes.
Sheryl Sandberg's HBS Class Day Speech -- two things stood out, beyond the honesty of the talk: If you’re offered a seat on a rocket ship, don’t ask what seat (that's her quoting Eric Schmidt) and [careers] are not a ladder; they’re a jungle gym (her quoting Facebook's head of HR). (via Sacha Judd)

Strata Week: Visualizing a better life

2012-05-24T15:30:00Z

Here are a few of the data stories that caught my attention this week:

Visualizing a better life

How do you compare the quality of life in different countries? As The Guardian's Simon Rogers points out, GDP has commonly been the indicator used to show a country's economic strength, but it's insufficient for comparing the quality of life and happiness of people.

To help build a better picture of what quality of life means to people, the Organization for Economic Cooperation and Development OECD built the Your Better Life Index. The index lets people select the things that matter to them: housing, income, jobs, community, education, environment, governance, health, life satisfaction, safety and work-life balance. The OECD launched the tool last year and offered an update this week, adding data on gender and inequality.

Screenshot from OECD's Your Better Life Index.

"It's counted as a major success by the OECD," writes Rogers, "particularly as users consistently rank quality of life indicators such as education, environment, governance, health, life satisfaction, safety and work-life balance above more traditional ones. Designed by Moritz Stefaner and Raureif, it's also rather beautiful."

The countries that come out on top most often based on users' rankings: "Denmark (life satisfaction and work-life balance), Switzerland (health and jobs), Finland (education), Japan (safety), Sweden (environment), and the USA (income)."

Researchers' access to data

The New York Times' John Markoff examines social science research and the growing problem of datasets that are not made available to other scholars. Opening data helps make sure that research results can be verified. But Markoff suggests that in many cases, data is being kept private and proprietary.

Much of the data he's talking about here is:

"... gathered by researchers at companies like Facebook, Google and Microsoft from patterns of cellphone calls, text messages and Internet clicks by millions of users around the world. Companies often refuse to make such information public, sometimes for competitive reasons and sometimes to protect customers' privacy. But to many scientists, the practice is an invitation to bad science, secrecy and even potential fraud."

"The debate will only intensify as large companies with deep pockets do more research about their users," Markoff predicts.

Updates to Hadoop

Apache has released the alpha version of Hadoop 2.0.0. We should stress "alpha" here, and as Hortonworks' Arun Murthy notes, it's "not ready to run in production." However, he adds the update "is still an important step forward, as it represents the very first release that delivers new and important capabilities," including: HDFS HA (manual failover) and next generation MapReduce.

In other Hadoop news, MapR has unveiled a series of new features and initiatives for its Hadoop distribution, including release of a fully compliant ODBC 3.52 driver, support for the Linux Pluggable Authentication Modules (PAM), and the availability of the source code for several of its components.

Have data news to share?

Feel free to email me.

Related:

Visualization of the Week series

Data science is a pipeline between academic disciplines

Hadoop: What it is, how it works, and what it can do

Knight Foundation grants $2 million for data journalism research

2012-05-24T14:00:00Z

Every day, the public hears more about technology and media entrepreneurs, from when they started in the garages and the dorm rooms, all the way up until when they go public, get acquired or go spectacularly bust. The way that the world mourned the passing of Steve Jobs last year and that young people now look to Mark Zuckerberg as a model for what's possible offer some insight into that dynamic.

For those who want to follow in their footsteps, the most interesting elements of those stories will be the muddy details of who came up with the idea, who wrote the first lines of code, who funded them, how they were mentored and then how the startup executed upon their ideas.

Today, foundations and institutions alike are getting involved in the startup ecosystem, but with a different hook than the venture capitalists on Sand Hill Road in California or Y Combinator: They're looking for smart, ambitious social entrepreneurs who want to start civic startups and increase the social capital of the world. From the Code for America Civic Accelerator to the Omidyar Foundation to Google.org to the Knight Foundation's News Challenge, there's more access to seed capital than ever before.

There are many reasons to watch what the Knight Foundation is doing, in particular, as it shifts how it funds digital journalism projects. The foundation's grants are going toward supporting many elements of the broader open government movement, from civic media to government transparency projects to data journalism platforms.

Many of these projects — or elements and code from them — have a chance at becoming part of the plumbing of digital democracy in the 21st century, although we're still on the first steps of the long road of that development.

This model for catalyzing civic innovation in the public interest is, in the broader sweep of history, still relatively new. (Then again, so is the medium you're reading this post on.) One barrier that the Internet has helped lower is in the process of discovering and selecting good ideas to fund and letting bad ideas fall to the wayside. Another is changing how ideas are capitalized through microfunding approaches or how distributing opportunities for participation in helping products or services go to market now can happen though crowdfunding platforms like Kickstarter.

When the Pebble smartwatch received $10 million through Kickstarter this year, it offered a notable data point into how this model could work. We'll see how others follow.

These models could contribute to the development of small pieces of civic architecture around the world, loosely joining networks in civil society with mobile technology, lightweight programming languages and open data.

After years of watching how the winners of the Knight News Challenges have — or have not — contributed to this potential future, its architects are looking at big questions: How should resources be allocated in newsrooms? What should be measured? Are governments more transparent and accountable due to the use of public data by journalists? What data is available? What isn't? What's useful and relevant to the lives of citizens? How can data visualization, news applications and interactive maps inform and engage readers?

In the context of these questions, the fact that the next Knight News Challenge will focus on data will create important new opportunities to augment the practice of journalism and accelerate the pace of open government. John Bracken (@jsb), the Knight Foundation's program director for journalism and media innovation, offered an explanation for this focus on the foundation's blog:

"Knight News Challenge: Data is a call for making sense of this onslaught of information. 'As data sits teetering between opportunity and crisis, we need people who can shift the scales and transform data into real assets,' wrote Roger Ehrenberg earlier this year.

"Or, as danah boyd has put it, 'Data is cheap, but making sense of it is not.'

"The CIA, the NBA's Houston Rockets, startups like BrightTag and Personal ('every detail of your life is data') — they're all trying to make sense out of data. We hope that this News Challenge will uncover similar innovators discovering ways for applying data towards informing citizens and communities."

Regardless of what happens with this News Challenge, some of those big data questions stand a much better chance of being answered because of the Knight Foundation's $2 million grant to Columbia University to research and distribute best practices for digital reporting, data visualizations and measuring impact.

Earlier this spring, I spoke with Emily Bell, the director of the Tow Center for Digital Journalism, about how this data journalism research at Columbia will close the data science "skills gap" in newsrooms. Bell is now entrusted with creating the architecture for learning that will teach the next generation of data journalists at Columbia University.

In search of the reasoning behind the grant, I talked to Michael Maness (@MichaelManess), vice president of journalism and media innovations at the Knight Foundation. Our interview, lightly edited for content and clarity, follows.

The last time I checked, you're in charge of funding ideas that will make the world better through journalism and technology. Is that about right?

Michael Maness: That's the hope. What we're trying to do is make sure that we're accelerating innovation in the journalism and media space that continues to help inform and engage communities. We think that's vital for democracy. What I do is work on those issues and fund ideas around that to not only make it easier for journalists to do their work, but citizens to engage in that same practice.

The Knight News Challenge has changed a bit over the last couple of years. How has the new process been going?

Michael Maness: I've been in the job a little bit more than a year. I came in at the tail end of 2011 and the News Challenge of 2011. We had some great winners, but we noticed that in the amount of time from when you applied in the News Challenge to when you were funded could be up to 10 months, by the time everything was done, and certainly eight months in terms of the process. So we reduced that to about 10 weeks. It's intense for the judges to do that, but we wanted to move more quickly, recognizing the speed of disruption and the energy of innovation and how fast it's moving.

We've also switched to a thematic theme. We're going to do three [themes] this year. The point of it is to fund as fast as possible those ideas that we think are interesting and that we think will have a big impact.

This last round was around networks. The reason we focused on networks is the apparent rise of network power. The second reason is we get people, for example, that say, "This is the new Twitter for X" or "This is the new Facebook for journalists." Our point is actually, you should be using and leveraging existing things for that.

We found when we looked back at the last five years of the News Challenge that people who came in with networks or built networks in accordance with what they're doing had a higher and faster scaling rate. We want to start targeting areas to do that, too.

We hear a lot about entrepreneurs, young people and the technology itself, but schools and libraries seem really important to me. How will existing institutions be part of the future that you're funding and building?

Michael Maness: One of the things that we're doing is moving into more "prototyping" types of grants and then finding ways of scaling those out, helping get ideas into a proof-of-concept phase so users kick the tires and look for scaling afterward.

In terms of the institutions, one of the things that we've seen that's been a bit of a frustration point is making sure that when we have innovations, [we're] finding the best ways to parlay those into absorption in these kinds of institutions.

A really good standout for that, from a couple years ago as a News Challenge winner, is DocumentCloud, which has been adopted by a lot of the larger legacy media institutions. From a university standpoint, we know one of the things that is key is getting involvement with students as practitioners. They're trying these things out and they're doing the two kinds of modeling that we're talking about. They're using the newest tools in the curriculum.

That's one of the reasons we made the grant [to Columbia.] They have a good track record. The other reason is that you have a real practitioner there with Emily Bell, doing all of her digital work from The Guardian and really knowing how to implement understandings and new ways of reporting. She's been vital. We see her as someone who has lived in an actual newsroom, pulling in those digital projects and finding new ways for journalists to implement them.

The other aspect is that there are just a lot of unknowns in this space. As we move forward, using these new tools for data visualization, for database reporting, what are the things that work? What are the things that are hard to do? What are the ideas that make the most impact? What efficiencies can we find to help newsrooms do it? We didn't really have a great body of knowledge around that, and that's one of the things that's really exciting about the project at Columbia.

How will you make sure the results of the research go beyond Columbia's ivy-covered walls?

Michael Maness: That was a big thing that we talked about, too, because it's not in us to do a lot of white papers around something like this. It doesn't really disseminate. A lot of this grant is around making sure that there are convocations.

We talk a lot about the creation of content objects. If you're studying data visualization, we should be making sure that we're producing that as well. This will be something that's ongoing and emerging. Definitely, a part of it is that some of these resources will go to hold gatherings, to send people out from Columbia to disseminate [research] and also to produce findings in a way that can be moved very easily around a digital ecosystem.

We want to make sure that you're running into this work a lot. This is something that we've baked into the grant, and we're going to be experimenting with, I think, as it moves forward. But I hear you, that if we did all of this — and it got captured behind ivy walls — it's not beneficial to the industry.

Related:

Commerce Weekly: Facebook continues its mobile acquisition spree

2012-05-24T13:00:00Z

Here are the commerce stories that caught my attention this week.

Facebook acquires social gift-giving app Karma

The media focus on Facebook's IPO might be missing the point — as Michael V. Copeland points out in a post at Wired, the IPO is really about the cash Facebook now has on hand to continue its acquisition spree. And that's just what it did. On the heels of the market's closing bell last Friday (May 18), Facebook announced its acquisition of Karma, a mobile ecommerce app that facilitates social gift giving — what some analysts are calling the next big mobile commerce boom.

IDG News' Cameron Scott points out in a post at PCWorld that mobile is where Facebook needs to focus now, as "it currently generates less revenue per user [in mobile] than it does on the desktop." He notes that Karma already is integrated with Facebook and describes how the app works:

"Karma allows users to buy gifts from its catalogue from their mobile phones. Recipients receive a text message notifying them of the gift and directing them to a website where they can exchange it if they want to and enter their shipping address."

Chris Preimesberger over at eWeek.com reiterates Facebook's need to "monetize interaction with its subscribers in better fashion on mobile devices" and provides a nice roundup of Facebook's mobile-related acquisitions to date.

X.commerce harnesses the technologies of eBay, PayPal and Magento to create the first end-to-end multi-channel commerce technology platform. Our vision is to enable merchants of every size, service providers and developers to thrive in a marketplace where in-store, online, mobile and social selling are all mission critical to business success. Learn more at x.com.

If you can't beat "showrooming" ...

Raise your hand if you've showroomed. I'm certainly guilty, and much to retailers' dismay, this type of shopping behavior is becoming more common. Mobile Commerce Daily reports this week on a new study from the Interactive Advertising Bureau (IAB) that shows "53 percent of mobile commerce users have stopped an in-store purchase as a result of using their mobile phone." Joe Lazlo, senior director of the Mobile Marketing Center of Excellence at the IAB told Mobile Commerce Daily the practice of showrooming isn't likely to go away:

"Mobile shoppers today use their mobiles to save them time or money or both — mobile shopping helps make them more efficient. And since we all love to make the best use of our time and money, these kinds of behaviors are likely to spread and become mainstream really quickly.

"Fighting this trend is certainly going to be a losing strategy, so I predict that savvy marketers will learn to live with, and even court, these newly empowered consumers, who can and do leave a store if they discover a better deal somewhere else."

Staff contributors from McKinsey & Company call this situation a "retail store apocalypse" in a post over at Forbes. The group says, however, that "showrooming shouldn't be a show stopper. These digital shoppers are ready to buy. Excelling at multichannel sales is today's must-have capability, and retailers must adapt if they want to survive." They offer seven practical strategies to avoid succumbing to the apocalypse. Some highlights include establishing your store as an authority on your products, making use of data — or learning to — to better target consumers on an individual level, and re-imagining the role of your retail store as more of a service hub. You can read more on these ideas and their other suggestions here.

Mobile disrupts the grocery store checkout line

More and more, consumers are incorporating mobile into retail shopping, but what about grocery shopping? According to Kunur Patel in a post at AdAge Digital, the focus of mobile in grocery stores has more to do with in-store shopping behavior and convenience rather than online grocery competition. She points to loyalty program company Catalina Mobile and describes how it's testing the mobile grocery shopping waters, in part by targeting the big grocery store dice roll — choosing the fastest checkout line. Patel writes:

"[Catalina Mobile] is behind mobile apps in 110 of Ahold USA's Stop & Shop stores in Massachusetts, Rhode Island and Connecticut. Here's how it works: A shopper walks into the store, opens the store-branded app and receives offers based on their shopping history. To skip the line at checkout, shoppers can scan barcodes of items they put into the cart to buy the haul right on the phone. Catalina says the app will be available in 200 more Stop & Shops this summer."

Patel also highlights Coupons.com's app Grocery iQ:

"Shoppers create lists in the app by scanning barcodes or typing in entries, then the app organizes those items by aisle. The most recent [app] update ports coupons into the search tool so that, say, a shopper may add one brand of yogurt to the list over another because Coupons.com offers 50¢ off."

Unless the consumer is shopping at a participating store with a loyalty card that syncs with the app, the coupons will need to be emailed and printed.

Tip us off

News tips and suggestions are always welcome, so please send them along.

Related:

Facebook finds a mobile commerce partner

Facebook's shopping spree continues

Target doesn't want to be the showroom for online retailers

More Commerce Weekly coverage

Four short links: 24 May 2012

2012-05-24T10:00:00Z

Last Saturday My Son Found His People at the Maker Faire -- aww to the power of INFINITY.
Dictionaries Linking Words to Concepts (Google Research) -- Wikipedia entries for concepts, text strings from searches and the oppressed workers down the Text Mines, and a count indicating how often the two were related.
Magic Wand (Kickstarter) -- I don't want the game, I want a Bluetooth magic wand. I don't want to click the OK button, I want to wave a wand and make it so! (via Pete Warden)
E-Commerce Performance (Luke Wroblewski) -- If a page load takes more than two seconds, 40% are likely to abandon that site. This is why you should follow Steve Souders like a hawk: if your site is slower than it could be, you're leaving money on the table.

Jon Loeliger offers some practices to use with Git

2012-05-24T08:00:00Z

After finishing the second edition of "Version Control with Git," author Jon Loeliger talked to me about some of the advice he offers and how to use Git effectively as changes to code pile up.

Highlights from the full video interview include:

What's new in Git since the first edition of the book? [Discussed at the 0:38 mark]

Importance of understanding concepts behind Git [Discussed at the 2:40 mark]

How to manage complicated branching [Discussed at the 3:33 mark]

Aspects of Github beyond storage [Discussed at the 6:22 mark]

You can view the entire conversation in the following video:

Related:

Clojure's advantage: Immediate feedback with REPL

2012-05-23T15:00:00Z

Chas Emerick (@cemerick) is the co-author of "Clojure Programming" along with Brian Carper and Christophe Grand. He maintains a busy blog at cemerick.com and he also produces "Mostly Lazy … a Clojure podcast".

I asked Chas to enumerate some of the topics that would make a difference to developers: something that would attract attention to Clojure as a language, so we wouldn't spend time talking about yet another syntax. One of the first things he immediately mentioned was REPL. Writing code in Clojure is often about making changes and immediately seeing your results. Clojure has emphasized this shell-like approach to development and created an environment that allows for immediate evaluation of code and incorporating changes into a running process.

Clojure differs from other languages in that this interactive shell isn't an afterthought. Ruby's IRB or Java's beanshell are similar attempts at interactivity, but they are not primary features of each language. With Clojure, REPL is built in, and you can connect to any running Clojure process and modify and execute code. In this interview we discuss some of the possibilities that this introduces for Clojure developers.

The full interview is embedded below and available here. For the entire interview transcript, click here.

Highlights from interview:

On what is unique about REPL

"...what's really unique about Clojure is that most people's workflow when developing and using Clojure is tightly tied to the REPL, using the dynamic interactive development capabilities that the REPL provides to really boost your productivity and give you a very immediate sense of control over both the Clojure runtime, the application, programming or service that you're building." [Discussed 00:52]

How does REPL affect your development?

"Generally what you do in Clojure is you start up a JVM instance that is running the Clojure runtime and starts a REPL that you connect to ... then you stay connected to that running Clojure runtime for hours, days. I've had Clojure environments running for weeks in an interactive development setting, where you gradually massage the code that's running within that runtime that corresponds to files you have on disc.

"You can choose what code to load into that environment at any time, massage the data that's being processed by your application. It's a great feedback mechanism and gives you an immediate, fine‑grain sense of control over what you're doing [Discussed 01:18]

On using REPL to deploy production patches

"It's true, you can start up a REPL running on Heroku right now, connect to it and modify your application. Everything will work as you would expect if you happen to be running the application using Foreman locally.

"You don't want to be, in general, modifying your production environments from an interactive standpoint. You want to have a repeatable process ... Depending on the circumstances, it can be reasonable in a 'fire drill' situation to push a critical, time‑sensitive patch out to production ... 99% of the time you probably shouldn't reach for it, but it's very good to know that it's there." [Discussed 05:44]

On using REPL for direct access to production statistics

"JMX is great in terms of providing a structured interface for doing monitoring. But you need to plan ahead of time for the things you're going to monitor for and make sure you have the right extensions to monitor the things that you care about.

"Having a REPL available to connect to in every environment — whether it's development, user acceptance, functional testing or production — means that when you need to, you can get in there and write a one‑off function. Pull some data from this database, see what's going on with this function, capture some data that you wouldn't normally be capturing, stuff that may be far too large to log or to practically get access through JMX. It's a great tool." [Discussed 05:44]

Related:

White House launches new digital government strategy

2012-05-23T14:33:40Z

There's a long history of people who have tried to transform the United States federal government through better use of information technology and data. It extends back to the early days of Alexander Hamilton's ledgers of financial transaction, continues through information transmitted through telegraph, radio, telephone, and comes up to the introduction of the Internet, which has been driving dreams of better e-government for decades.

Vivek Kundra, the first U.S. chief information officer, and Aneesh Chopra, the nation's first chief technology officer, were chosen by President Barack Obama to try to bring the federal government's IT infrastructure and process into the 21st century, closing the IT gap that had opened between the private sector and public sector.

Today, President Obama issued a presidential memorandum on building a 21st century digital government.

In this memorandum, the president directs each major federal agency in the United States to make two key services that American citizens depend upon available on mobile devices within the next 12 months and to make "applicable" government information open and machine-readable by default. President Obama directed federal agencies to do two specific things: comply with the elements of the strategy by May 23, 2013 and to create a "/developer" page on ever major federal agency's website. Here's an excerpt from President Obama's memo:

The innovative use of technology is fundamentally transforming how the American people do business and live their daily lives. Exponential increases in computing power, the rise of high-speed networks, and the growing mobile revolution have put the Internet at our fingertips, encouraging innovations that are giving rise to new industries and reshaping existing ones.

Innovators in the private sector and the Federal Government have used these technological advances to fundamentally change how they serve their customers. However, it is time for the Federal Government to do more. For far too long, the American people have been forced to navigate a labyrinth of information across different Government programs in order to find the services they need. In addition, at a time when Americans increasingly pay bills and buy tickets on mobile devices, Government services often are not optimized for smartphones or tablets, assuming the services are even available online.

On April 27, 2011, I issued Executive Order 13571 (Streamlining Service Delivery and Improving Customer Service), requiring executive departments and agencies (agencies) to, among other things, identify ways to use innovative technologies to streamline their delivery of services to lower costs, decrease service delivery times, and improve the customer experience. As the next step toward modernizing the way Government works, I charged my Federal Chief Information Officer (CIO) with developing a comprehensive Government-wide strategy to build a 21st century digital Government that delivers better digital services to the American people.

Today, the CIO is releasing that strategy, entitled "Digital Government: Building a 21st Century Platform to Better Serve the American People" (Strategy), which provides agencies with a 12-month roadmap that focuses on several priority areas.The Strategy will enable more efficient and coordinated digital service delivery by requiring agencies to establish specific, measurable goals for delivering better digital services; encouraging agencies to deliver information in new ways that fully utilize the power and potential of mobile and web-based technologies; ensuring the safe and secure delivery and use of digital services to protect information and privacy; requiring agencies to establish central online resources for outside developers and to adopt new standards for making applicable Government information open and machine-readable by default;

aggregating agencies' online resource pages for developers in a centralized catalogue on www.Data.gov; and requiring agencies to use web performance analytics and customer satisfaction measurement tools on all ".gov" websites.

Ultimately, this Strategy will ensure that agencies use emerging technologies to serve the public as effectively as possible. As a Government, and as a trusted provider of services, we must never forget who our customers are — the American people.

While Kundra and Chopra set in a motion of series of reforms, from more transparency on IT spending and waste to an ambitious open government data program to adoption of cloud computing to improved IT security and a modern approach to open innovation, they left an immense portfolio and set of challenges for Steven VanRoekel and Todd Park to take on and implement against. Many of those challenges remain the same, including attracting talent, reforming procurement, data quality, and the reality of agency mainframes that are still running on COBOL.

"There are many things the federal government should do to improve IT performance and efficiency," said Darrell West, vice president for government studies and director of the Center for Tech Innovation at the Brookings Institute, when asked for comment. "It can quit adopting expensive legacy systems that are obsolete from the moment they are purchased and move towards more nimble strategies. There are many new apps that are available through the federal Apps Store and agencies can use them to improve performance and cut costs. In the medical area, there are 40,000 mobile health applications. Technology should be a money saver, not a money waster. The key problem federal officials face is overcoming economic interests that are vested in the past, not the future of technology innovation."

Big visions matter, in terms of inspiring the country to action or a historic course, from building transcontinental railroads to sending men to the Moon to starting up a new government agency. Implementing against that vision, however, in a time of great budget pressure, increased demands for government services online and falling trust in institutions, is just as important.

Today, VanRoekel and Park have put their own stamp on the future of digital government in the United States with the introduction of a digital strategy for 21st century government, which went online this morning. (In a notable upgrade to the way such policies have been released, this digital strategy was coded in HTML5.)

They introduced it to the nation in an decidedly non-Washingtonian sort of way, traveling north to New York City, a city that has become one the global epicenters for data-centric digital government under Mayor Michael Bloomberg, to directly engage Gotham City's community of tech entrepreneurs, venture capitalists and (civic) developers at TechCrunch's Disrupt Conference. Video of their announcement at Disruptis embedded below:

Whether their "pitch" for smart government gets funded and supported is an open question. One measure will be whether this trip results in new recruits: Park and VanRoekel urged the developers and entrepreneurs in the room to "join a startup called the U.S. government" as one of 15 "Presidential Innovation Fellows." Park said in New York that the fellowships will focus on five areas: a USAID campaign, open data across the federal government, releasing more health information, an online system they're calling "MyGov," a request for proposals program to help startups with procurement.

You can watch Park and VanRoekel answer follow up questions from writer Greg Ferenstein (@ferenstein) on procurement, the role of the private sector and emerging technology in the video below:

Park and VanRoekel took questions from the press on a call following their presentation. Molly Walker published audio of their discussion of the digital strategy, which you can listen to in the player below:

What's next for open data?

There have been longstanding questions about what constitute "high quality datasets" for years. VanRoekel, in past conversation, has identified an activity that many Americans engage in — looking for real estate. Buying a house is the single biggest purchase many citizens will ever make.

Now, VanRoekel and Park are talking about releasing more data that could actually inform that purchase. The challenge is that the third parties that they want to have use it need to have it be high quality, regularly updated, standardized and accessible. For instance, developers making energy apps don't want data published every quarter: they want it every week, every day or, if possible, every hour.

To learn more about how they're thinking about these challenges, I interviewed Park and VanRoekel yesterday about the components and thinking behind this strategy. My discussion with VanRoekel follows. Look for an extended discussion with Park and video from TechCrunch Disrupt later today.

This digital strategy has been coming for a long time. What's in it?

VanRoekel: Over the last 10 months or so we've been hard at work on a bunch of divergent strategies, thinking about our web presence as a federal government, managing the 1,800 .gov domain names we have out there, and tens of thousands of websites and millions of pages. We've been focused on mobile and the consumerization of technology and that impact on government. And as you know, I come storming in with a passion around open data, making data machine-readable and accessible, unlocking the potential of that data, as you and I talked about last time, and many times at the FCC. We had a moment this winter when we realized these divergent strategies were really converging into one thing, where they were all parts of a bigger strategy, which breaks down into several areas.

This first is an information-centric approach. That pillar is about making data openly available as the new default across government.

The second is around a shared platform, where we can share resources across government, removing duplication and leveraging existing projects that we're doing.

The third layer is all about customer centric-government. Typically in the government, we very tightly couple the presentation of stuff with the stuff itself. Finding that stuff is nearly impossible, if you have to navigate a quagmire. The nature of this is embracing the new application delivery model, looking at how we deliver content and data more effectively to Americans. Also, how do we create, across the customer-centric platform, a way of creating government as a platform, where we have a common approach to building applications on top of government data. Our goal is to bring government to Americans where they live, versus expect Americans to come to government, where we are. We saw this in the past, with weather and GPS. Combined, there's over a $100=billion dollar industry, if not more. We now, as Americans, probably take that for granted, in the ability to just connect in and use that data from the government. We have unending examples of places where this government data can be unleashed to the private sector and to American citizens to provide better e-services.

The last pillar, which I won't go into depth now but that it's very important that we're doing, is that we're building consistent methodology around security and privacy. We're calling on a bunch of groups in this strategy to help us on that front.

How will government become more data-centric?

VanRoekel: Yesterday was the three-year anniversary of Data.gov. It was an important first step to raise the awareness of people around data. It largely was a bulk upload, bulk download system. It didn't manifest in data being digital by default through the entire lifecycle of that data. It was sort of an add-on, to take the step of getting it to Data.gov. The key difference in what we're doing here and evolving Data.gov — which will be involved, as part of this — is architecting data for openness.

There are some very specific deliverables in the strategy itself, including a government-wide open data content and web API policy and then work with NIST and others to identify standards and best practices for interoperability to scale that across government. We will have, from its creation to its dissemination, open data as the new normal. We're going to, as part of this, inspire investing in IT systems that respect open data and think about open data.

This also bridges into big data and that roadmap as well. The next step of that is thinking about web APIs, how we build for modularity. It's about how we deliver our content and manifest systems in a way that allow APIs to do that. Part of this deliverable will also be about expanding Data.gov to also be the metadata catalog of all data in the federal government. It won't be a place where you use data but it will be a place where you find data, discover data or discover the relatedness of data. We also intend to take Data.gov and add some API key management and some other technology there, so it is the developer platform of all this stuff.

As you remember from the Open Government Directive, where we directed all agencies to create a "/open" page. Part of this deliverable will be all agencies creating a "/developer" page, to start to build the catalog of citizen-accessible web APIs for their systems.

This may all be a bit abstract to a non-technical citizen. Why is open data important to the American people? Why is this a strategy that's important enough to elevate to this level?

VanRoekel: For citizen-facing discussions and the conversation I'll have with consumer publications, I start with the customer-centric element and think about what are those applications that will be delivered by government and go drive a new set of normals and value systems. I always harken back to examples like GPS, like weather, but we can also look at early agencies that have done a better job, in the realm of health, tax, or travel, including that MyTSA app on your smartphone giving you wait times and security gates at the airport nearest you, or travel tips. Those are possible from unlocking government data and making that government data available to application providers.

To the TechCrunch crowd, I'm going to talk about real estate. When you're buying a home, why doesn't it manifest to you the myriad of data that the government has locked up about school quality, healthcare quality, infrastructure investments, broadband, everything else that people really care about when they're picking a place to live? We don't do that — we do roof composition and the number of bathrooms, and that's typically the extent of it. Some services are doing a better job with other government data but largely it's pretty silo'ed and not very specific to what Americans really care about.

How will this strategy result in those releases happening in the format and timeliness that making that kind of data useful would require? How do you drive the timely release of data in a consumable format from agencies, given the technical challenges they have?

VanRoekel: That's why we're launching the strategy now. It couldn't more important to manifest those things across all of the opportunities we have. From declaring that open data is now the new default — that you have to, when you create datasets, do so in an open way — when we say that when you buy IT systems that interface with data, they need to be purchased in a way that respect the aspects of open data, when you talk about dissemination of data, we're going to inspire in the strategy different aspects of agencies building and delivering open data solutions. The first tranche of this is that each agency has to do two of them. That's a "crawl, walk, run" approach. You have to have the magic mix of all of these elements: great policy, great technology, and great people leading the way. That's what we're trying to bring to bear to get there. We've been kind of haphazard. There's been no one at the top saying "thou shalt do this." Now we have that with this strategy.

President Obama has talked about information technology with some frequency, in the context of the importance of American innovation, improved government services and job creation. How much have you talked with the president about this strategy? How closely has he been involved in helping to shape it? How much support do you have from him?

VanRoekel: We have an incredible amount of support, both from the president and his staff. He was excited enough about this strategy that he insisted in actually issuing a memo as a cover page for it. This presidential memorandum is basically telling agencies "go do this stuff. It's important," talking about the necessity of unlocking this government technology on behalf of citizens. That doesn't happen very often, with him weighing in on the importance of what's happening here.

If you look at how well agencies have complied with the "/open" requirements, in terms of simply updating their open government plans, you'll see some laggards. What policy hooks will you have, with respect to making sure the deliverables in this strategy come through?

VanRoekel: The important part of this is that we have one policy out there. Policy doesn't go all the way. Policy should be a place that gives framing and direction, and inspires people to act. It explains the art of the possible. Part of this is just education on what is the possible and how we do this. I think the important part that will make this stick long term includes three things.

One is oversight. Part of this strategy will be working with agencies to create reporting mechanisms — and those reporting mechanisms will be built in open, standard ways that allow people to create easily accessible dashboards that track agency progress on these deliverables, so we can put some pressure on people from an oversight standpoint. The new normal for dashboards needs to be open data as the feeds, and allowing people to build on top of them.

The second part is regular accountability in meetings with us. We've got other mechanisms to touch base with people, too. I care a lot about this. At the FCC, I mandated this stuff and watched and saw it through to some really successful endpoints.

The third — and this is probably one of the most important elements — is injecting this into the different parts of doing business within government agencies. It's getting the permit team to do new things. It's getting the IT team to buy and deploy things in new ways.

Luckily, we're not starting from scratch. I think agencies generally —- and IT professional in those agencies — want to deliver against mission. They also care a lot about this stuff. They know this is the new way of doing things. They just need to be empowered in a way that gets them to that end point. We're not going to get a huge tide of resistance. It's going to bring them measurable value.

Part of this strategy is around mobile consolidation, including contracts and other things which are going to save them a lot of money in this fiscal environment. We've got a nice forcing function there.

How much is the "bring your own device" (BYOD) wave in government and enterprise driving this data-centricism of this digital strategy? CIOs are stressed about not being able to get mobile app development talent to create multiple applications for multiple platforms.

VanRoekel: It definitely is part of it. We have to walk that careful walk of being device and vendor-agnostic, in terms of the strategies we lay out. We don't want to be creating a marketplace winner through policy. We also have to be cognizant that we really have to open up the doors here for us to utilize these government systems in new ways. I know that's not going to be a sweeping phenomenon right away. We need to get stuff moving in the right direction. I think this is the way we do it. We do it by getting agencies to deliver solutions that are decoupled from the underlying data and ride government like a platform. We rapidly prototype and create single websites presence for the government. We get the private sector to step up and deliver solutions as well.

So you want agencies to be "dogfooding" the same data and web services that will be consumed by third parties?

VanRoekel: I think there have to be places where, when we unlock the data, it makes sense for us to actively deliver solutions against that data. That's either proof-of-concept work or actually deliver solutions against it.

This post has been updated to add audio and video when it became available online.

Schlomo Schapiro on continuous delivery platforms

2012-05-23T13:05:14Z

In this new Velocity Podcast, I had a conversation with Schlomo Schapiro (@schlomoschapiro), a Systems Architect at ImmobilienScout24, and a Velocity Europe committee member. This conversation centers mostly on building a continuous delivery platform. Schlomo has some interesting insights into how to deploy a platform that delivers 24x7.

Our conversation lasted 00:11:15 and if you want to pinpoint any particular topic, you can find the specific timing below. Schlomo talks a little about ImmobilienScout24, the tech stack they use, and his role at about 40 seconds into the conversation. The rest of the conversation is outlined below.

Can you explain what a continuous delivery platform embodies? 00:01:37

What role can DevOps play with a continuous delivery platform? 00:03:29

How do you demonstrate the business value in building a continuous delivery platform and that it is working? 00:04:40

What does a typical stack for a continuous delivery platform look like? 00:05:42

What kind of of organizational changes are needed for a continuous delivery platform? 00:06:46

What does the technical stack look like for a continuous delivery platform? 00:07:49

So an open source stack is the way you would build a continuous delivery platform? 00:08:52

Are we going to see you at Velocity in Santa Clara in June and London in October? 00:10:20

Related:

Subscribe to the free Velocity podcast through iTunes

See more Velocity podcasts

Four short links: 23 May 2012

2012-05-23T10:00:00Z

Tale of Two Pwnies (Chromium Blog) -- So, how does one get full remote code execution in Chrome? In the case of Pinkie Pie’s exploit, it took a chain of six different bugs in order to successfully break out of the Chrome sandbox. Lest you think all attacks come from mouth-breathing script kiddies, this is how the pros do it. (via Bryan O'Sullivan)
The Future is Specific (Chris Granger) -- In traditional web-MVC, the code necessary to serve a single route is spread across many files in many different folders. In a normal editor this means you need to do a lot of context switching to get a sense for everything going on. Instead, this mode replaces the file picker with a route picker, as routes seem like the best logical unit for a website. There's a revolution coming in web dev tools: we've had the programmer adapting to the frameworks with little but textual assistance from the IDE. I am loving this flood of creativity because it has the promise to reduce bugs and increase the speed by which we generate good code.
Best Online Editors For Teaching HTML/CSS/JS (Pamela Fox) -- Over the past few months, I've been teaching in-person classes on HTML, CSS, and JavaScript, as part of GirlDevelopIt San Francisco. Along the way, I've experimented with various online consoles and editors, and I thought I'd share my experience with using them for teaching.
Makie -- design a doll online, they'll 3d-print and ship it to you. Hello, future of manufacturing, fancy seeing you in a dollhouse!

A gaming revolution, minus the hype

2012-05-22T14:00:00Z

In the following interview, "Playful Design" author John Ferrara (@PlayfulDesign) explains what he sees as the real gaming revolution — not "gamification," or the application of gaming characteristics to existing applications and processes, but how games themselves can and will be a "force of cultural transformation." Ferrera also reveals five universal principles of good game design.

Our interview follows.

How are mobile and social technologies affecting game design and the evolution of gaming technology?

John Ferrara: One of the really surprising things about modern smartphones and tablets is that the've turned out to be such credible gaming platforms. They open doors to new ways of experiencing games by giving designers access to touchscreens, accelerometers, cameras, microphones, GPS, and Internet connectivity through a single device. They also allow games to be experienced in new contexts, enjoyed on the train to work, in the minutes between meetings, and while you're out with friends. The traditional gaming model, where players sit passively in one place in the home and stare at a fixed screen, seems stodgy and limiting by comparison.

The funny thing about social technology is that before we had video games, gaming was almost always a social activity. You needed to have multiple people to play most board games, card games, and sports — in fact, the game was often just a pretense for people to get together. But then video games made solitary experiences more of the norm. Now social technology is bringing gaming back to its multiplayer roots, but it's also going beyond what was ever possible before by enabling hyper-social experiences where you're playing with dozens of friends and family at once. Even though you may be separated from these people in space and time, you have an intimate sense of shared presence and community when you're playing. That's revolutionary.

How do you see the social media aspects of gaming seeping into day-to-day life?

John Ferrara: Games certainly can transform the workplace, though I want to caution that it's very easy to make the mistake of dressing up everyday work activities as games by just tacking on some points and badges. That's not game design, and people will recognize that it's not. In the process of failing, approaches like this generate cynicism toward the effort. Games need to be designed to be games first and foremost. They must be intrinsically rewarding, enjoyed for their own sake.

That said, I absolutely believe that games can work at work. As you suggest, for example, they have great strengths for training. Games create a safe space for people to test out their mastery of a set of skills in ways that aren't possible or practical in the real world. They can also help people figure out how best to handle different situations. Say, for example, that you created a game to develop management skills. You might allow players to assign values to their in-game avatars like "nurturing," "autocratic," or "optimistic," which lead to different behavior paths. Players could then examine how these traits play out in a situation filled with characters who have different values like "dependability," "autonomy," and "efficiency." A structure like this could not only impart insight about management styles, but also invite introspection about how an individual's own personality traits may lead to success and failure in the real world.

In your book's introduction, you say, "I hope to start moving toward a post-hype discussion of how games can most effectively achieve great things in the real world." Who is leading the way — or at least moving in the right direction — and what are they doing?

John Ferrara: You know, there's so much really inventive work being done right now. Recently, I've been playing a lot of "Zombies, Run!," and I think it's great. This is a game for smartphones that overlays a narrative about survivors in a zombie apocalypse onto your daily run. As you're out getting your exercise, you're listening to the game events as they unfold, and you can hear the zombies closing in. It's a great use of fantasy, and it plays as a true game with meaningful choices and conflict.

There's also a great group at the University of Wisconsin-Madison that's developed a smartphone app called ARIS, which builds game scenarios into physical locations, and they've developed dozens of applications for it. One of them is being developed as a museum tour for the Minnesota Historical Center, giving people quests to complete by scanning objects in the exhibit and then using them to complete objectives in a story line. The museum is actually changing the way the exhibit is laid out to better accommodate the gameplay, moving away from the traditional snaking path to more of an open layout that allows players to move more freely between the interacting displays to solve the game's challenges.

Some of the thought leaders who I really admire include Eric Klopfer and Scot Osterweil at MIT, Ian Bogost at the Georgia Institute of Technology, and Jane McGonigal. A common current among these thinkers is their emphasis on games themselves as a force of cultural transformation, rather than simplistic "gamification" of software applications that lead to little or no meaningful change.

What about engineering games like "Foldit" — with improved UX, could this type of crowdsourced gaming become a viable research tool?

John Ferrara: This is what's been called "human computation," where a group of people work together to solve some complex problem as a by-product of some other action, like playing a game. Luis Von Ahn at Carnegie Mellon describes games as algorithms that are executed by people rather than machines, and I think that's a really fascinating idea. Foldit is a great example. This is a puzzle game where players try to figure out how to fold chains of proteins. This is a problem that's very well suited to human computation because it requires a type of intuitive reasoning that's very difficult for actual computers. Foldit made a big news last fall when the people playing it decoded the structure of a protein related to a virus that causes AIDS in monkeys, which had eluded researchers for years.

This is a wonderful demonstration of how this type of game can be really valuable to researchers. At the same time, I'm very critical of Foldit because I think its gameplay experience is kind of awful. It's very difficult to figure out which actions lead to the results you see on-screen — like why you're awarded points the way you are — and there's not a strong sense of objectives or conflict. These design issues place limits on the appeal of Foldit, and that's a big problem because human computation works better the more people you have playing. If the gameplay were really compelling and fun, then the sky would be the limit.

How do you see the collection and use of gaming data evolving?

John Ferrara: Games can produce enormous volumes of data because it's really simple to gather every little interaction the player has in the game and report it all back to a central server. This has immediate applications for game design itself. Zynga, for example, uses data to determine which design choices create greater tendencies for players to stay engaged longer, involve more friends, or pay to enhance the game experience. I expect this kind of data collection and analysis to become the norm because companies will be more successful the better they can do it.

I would suggest that financial services could be one of the biggest secondary beneficiaries of such data because there's so much to learn about how people make financial decisions under different circumstances. Staying with the Zynga theme, suppose players have the option of investing in any of a variety of different farm crops, each of which has different strengths and vulnerabilities to environmental conditions. How do players choose which ones they should purchase? How do they appraise risk and reward? Which presentations of information lead to a better understanding of a crop's attributes? Which lead people to make more appropriate choices for their goals? All of these questions can be examined quantitatively through games and can lead to greater insights into the innate qualities of human psychology that drive investor behavior and decision making.

What are some emerging best practices for game technology?

John Ferrara: Best practices vary widely depending on the game and the type of player motivations to which it appeals. For example, games meant to promote a sense of immersion like "Red Dead Redemption" remove as much of the user interface elements from immediate view as possible. Data-intensive games like "Tiny Tower" benefit by compressing as much information and as many functional controls as they can into the smallest possible space.

With that in mind, there are some clear universal principles for the design of all games:

Skip the manual and embed as much instruction into the gameplay as you can.
Fit the game into the player's lifestyle so that he or she can play when and where it's convenient.
Don't cheat — people recognize when a game unfairly stacks the odds against them and they resent it.
Make sure players always have a clear sense of cause and effect, and that they understand what actions are available to them.
Above all, playtest, playtest, playtest. It's impossible to fully anticipate how people will react to a game short of actually watching them play it.

In the book, you argue that games should be used as instruments of persuasion. Why is this?

John Ferrara: To be clear, it's not that all games should be persuasive but that people who want to persuade should look at games very seriously; I believe they present an ideal way to convince people to adopt a particular point of view or to move them to action in the real world. Ian Bogost describes games as a form of "procedural rhetoric," meaning that they communicate messages through participation in the experience. This creates a lot of advantages for persuasion. For example, it allows a kind of self-directed discovery where people adopt the designer's message as a working hypothesis and then test its truthfulness through the gameplay. That's a really powerful way to get your point across. Furthermore, it builds a sense of personal ownership of the insight the player has uncovered.

Are there ethical concerns related to persuasion in gaming environments?

John Ferrara: As there are for any medium, certainly. Film, television, books, billboards, oratory, and posters have all been appropriated for less-than-above-board purposes. Whether it's propaganda, demagoguery, misleading advertising, or dirty politics, you'd expect that games would be subject to the same kinds of unethical practices. It's especially important to be aware of this in the case of games, considering how compelling a procedural rhetoric can be. Rather than casting a negative light on games, however, I think that speaks to their power to effect meaningful change in the real world. I believe that games can achieve great things, and I expect that over the next decade we'll see them doing a lot of good.

This interview was edited and condensed.

Related:

Gamification has issues, but they aren't the ones everyone focuses on

7 areas beyond gaming where Kinect could play a role

Three game characteristics that can be applied to education

The purpose of gamification

In defense of games in the workplace

O'Reilly Radar - Insight, analysis, and research about emerging technologies.

Top Stories: May 21-25, 2012

Visualization of the Week: 30 years of tech IPOs

Found a great visualization? Tell us about it

Publishing News: Kindle Fire and "your ad here"

Kindle Fire home screen may be for sale

Apple calls foul on the DOJ

It's time to hack digital covers

Developer Week in Review: Oracle's big bet fails to pay off

Android is almost out of the woods

Scripters of the world, rejoice!

FreeBSD bids GCC farewell

Got news?

Four short links: 25 May 2012

Strata Week: Visualizing a better life

Visualizing a better life

Researchers' access to data

Updates to Hadoop

Have data news to share?

Knight Foundation grants $2 million for data journalism research

The last time I checked, you're in charge of funding ideas that will make the world better through journalism and technology. Is that about right?

The Knight News Challenge has changed a bit over the last couple of years. How has the new process been going?

We hear a lot about entrepreneurs, young people and the technology itself, but schools and libraries seem really important to me. How will existing institutions be part of the future that you're funding and building?

How will you make sure the results of the research go beyond Columbia's ivy-covered walls?

Commerce Weekly: Facebook continues its mobile acquisition spree

Facebook acquires social gift-giving app Karma

If you can't beat "showrooming" ...

Mobile disrupts the grocery store checkout line

Tip us off

Four short links: 24 May 2012

Jon Loeliger offers some practices to use with Git

Clojure's advantage: Immediate feedback with REPL

On what is unique about REPL

How does REPL affect your development?

On using REPL to deploy production patches

On using REPL for direct access to production statistics

White House launches new digital government strategy

What's next for open data?

This digital strategy has been coming for a long time. What's in it?

How will government become more data-centric?

This may all be a bit abstract to a non-technical citizen. Why is open data important to the American people? Why is this a strategy that's important enough to elevate to this level?

How will this strategy result in those releases happening in the format and timeliness that making that kind of data useful would require? How do you drive the timely release of data in a consumable format from agencies, given the technical challenges they have?

If you look at how well agencies have complied with the "/open" requirements, in terms of simply updating their open government plans, you'll see some laggards. What policy hooks will you have, with respect to making sure the deliverables in this strategy come through?

How much is the "bring your own device" (BYOD) wave in government and enterprise driving this data-centricism of this digital strategy? CIOs are stressed about not being able to get mobile app development talent to create multiple applications for multiple platforms.

So you want agencies to be "dogfooding" the same data and web services that will be consumed by third parties?

Schlomo Schapiro on continuous delivery platforms

Four short links: 23 May 2012

A gaming revolution, minus the hype

How are mobile and social technologies affecting game design and the evolution of gaming technology?

How do you see the social media aspects of gaming seeping into day-to-day life?

In your book's introduction, you say, "I hope to start moving toward a post-hype discussion of how games can most effectively achieve great things in the real world." Who is leading the way — or at least moving in the right direction — and what are they doing?

What about engineering games like "Foldit" — with improved UX, could this type of crowdsourced gaming become a viable research tool?

How do you see the collection and use of gaming data evolving?

What are some emerging best practices for game technology?

In the book, you argue that games should be used as instruments of persuasion. Why is this?

Are there ethical concerns related to persuasion in gaming environments?