The startup world has gone mad for it. And though I think Microsoft’s Chief Software Architect Ray Ozzie played down the depth of Microsoft’s commitment to it in his recent interview with Steve Gillmor, it’s apparent that it’s at the very least a top-of-mind subject for the people at the highest levels of the biggest companies in the Internet world. As it should be.

The live web started to feel more tangible in shape and clearer for me to see because of Google Wave. Two of the Guardian developers here, Lisa van Gelder and Martyn Inglis, recently shared the results of a DevLab they did on Wave.

My brain has been spinning on the idea ever since.

(A DevLab is an internal research project where an individual or team pull out of the development cycle for a week and study an idea or a technology. There’s a grant associated with the study. They then share their findings with the entire team, and they share the grant with the individual who writes the most insightful peer review of the research.)

Many before me have noted the ambition and tremendous scale of the Wave effort. But I also find it fascinating how Google is approaching the development of the platform as a service.

The tendency when designing a platform is to create the rules and restrictions that prevent worst-case scenario behavior from ruining everything for you and your key partners. You release capability gradually as you understand its impact.

You then have to manage the constant demand from customers to release more and more capability.

Google turned this upside down and enabled a wide breadth of capability with no apologies for the unknowns. Developers won’t complain about lack of functionality. Instead it will probably have the opposite effect and invite the developers to tell Google how to close down the risks so their work won’t get damaged by the lawlessness of the ecosystem.

That’s a very exciting proposition, as if new land has been found where gold might be discovered.

But on the other hand, is it also a bit lazy or even irresponsible to put the task of creating the rules of the world that your service defines on the customers of your service? And do those partners then get a false sense of security because of that, as if they could influence the evolution of the platform in their favor when really it’s all about Google?

Google takes no responsibility for the bad things that may happen in the world they’ve created, yet they have retained full authority on their own for decisions about the service.

They’ve mitigated much of their risk by releasing the code as “open source” and allowing Wave to run in your own hosted environment as you choose. It’s a good PR move, but it may not have the effect they want it to have if they aren’t also sharing the way contributions to the code are managed and sharing in the governance.

They list the principles for the project on the site:

• Wave is an open network: anyone should be able to become a wave provider and interoperate with the public network
• Wave is a distributed network model: traffic is routed peer-to-peer, not through a central server
• Make rapid progress, together: a shared commitment to contribute to the evolution and timely deployment of protocol improvements
• Community contributions are fundamental: everyone is invited to participate in the public development process
• Decisions are made in public: all protocol specification discussions are recorded in a public archive

Those are definitions, not principles. Interestingly, there’s no commitment to opening decision-making itself, only sharing the results of decisions. Contrast that with Apache Foundation projects which have different layers of engagement and specific responsibilities for the different roles in a project. For example,

“a Project Management Committee member is a developer or a committer that was elected due to merit for the evolution of the project and demonstration of commitment. They have write access to the code repository, an apache.org mail address, the right to vote for the community-related decisions and the right to propose an active user for committership.”

That model may be too open for Google, but it would help a lot to have a team of self-interested supporters when things go wrong, particularly as there are so many security risks with Wave. If they are still the sole sponsor of the platform when the first damage appears then they will have to take responsibility for the problem. They can only use the “we don’t control the apps, only the platform” excuse for so long before it starts to look like a cop out.

Maybe they should’ve chosen a market they thought would run with it and offer it in preview exclusively for key partners in that market until Google understood how to position it. With a team of launch partners they would have seemed less autocratic and more trustworthy.

Shared ownership of the launch might also have resulted in a better first use-case app than the Wave client they invented for the platform. The Google Wave client may take a long time to catch on, if ever.

As Ray Ozzie noted,

“When you create something that people don’t know what it is, when they can’t describe it exactly, and you have to teach them, it’s hard…all of the systems, as long as I’ve been working in this area, the picture that I’ve always had in my mind is kind of three overlapping circles of technology, social dynamics, and organizational dynamics. And any two of those is relatively straightforward and understandable.”

I might even argue that perhaps Google actually made a very bad decision to offer a client at all. This was likely the result of failing to have a home for OpenSocial when it launched. Plus, it’s never a good idea to launch a platform without a principle customer app that can drive the initial requirements.

In my opinion, open conference-style IM and email or live collaborative editing within docs is just not groundbreaking enough as an end-user offering.

But the live web is fractionally about the client app.

The live web that matters, in my mind, harnesses real-time message interplay via multiple open networks between people and machines.

There’s not one app that runs on top of it. I can imagine there could be millions of client apps.

The Wave idea, whether it’s most potent incarnation is Wave itself or some combination of a Twitter/RabbitMQ mesh or an open XML P2P server or some other new approach to sharing data, is going to blow open the Internet for people once again.

I remember trying very hard to convince people that RSS was going to change the Internet and how publishing works several years ago. But the killer RSS app never happened.

It’s obvious why it feels like RSS didn’t take off. RSS is fabric. Most people won’t get that, nor should they have to.

In hindsight, I think I overvalued RSS but undervalued the importance of the idea…lubricating the path for data to get wherever it is needed.

I suspect Wave will suffer from many of the same issues.

Wave is fabric, too.

When people and things create data on a network that machines can do stuff with, the world gets really interesting. It gets particularly interesting when those machines unlock connections between people.

And while the race is on to come up with the next Twitter-like service, I just hope that the frantic Silicon Valley Internet platform architects don’t forget that it’s about people in the end.

One of the things many technology innovators forget to do is to talk to people. More developers should ask people about their day and watch them work. You may be able to breakthrough by solving real problems that real people have.

That’s a much better place to start than by inventing strategic points of leverage in order to challenge your real and perceived competitors.

But the key questions haven’t yet been played out enough. What does the Internet mean? How far will the changes go? Which aspects of civilization itself will become something different, perhaps even unrecognizable to us today through the pervasive effect of the network?

This is what we want to surface with The Guardian’s Activate Summit. Activate is an event about the people who are uncovering the answers to those questions.

Who are the ‘Activators’?

We’ve designed the event to get into the heads of the people driving the most important changes in politics, society, technology and the economy. Here are a few examples of the types of people and the things they are doing that we’ll see at the event…

• There are new ways to elect our government leaders demonstrated by people like Thomas Gensemer of Blue State Digital who orchestrated Obama’s digital campaign.
• Adam Afriyie MP is leading innovation across the public sector for David Cameron. He said in an interview about Activate:
  

  “I’ve started looking at cumbersome Whitehall IT and the way IT policy can be improved to strengthen society and kick-start the digital economy. [Dormant Whitehall data sets] can be re-used by the public, adding both commercial and social value to these public assets.”

• Tom Steinberg of MySociety is forcing a new kind of transparency in our government, a perspective we now expect of the publicly funded institutions that serve us in a way that we could only hope for before the Internet existed. And William Heath of Mydex and the Open Rights Group, among other things, is surfacing some of the implications of these changes and how to protect the individual. As he stated in an interview:
  

  “In UK public services it’s clear to me that feedback, transparency and a stronger voice for the individual are all healthy. So I’m very optimistic, but I think we’re only half way there. In e-commerce we’ve tooled up the big organisations. Now we need to get properly tooled up ourselves.”

• Iqbal Qadir, Charles Leadbeater and Umair Haque are demonstrating new forms of capitalism and the shape of the new economy for an age of scarcity..
• Sugatra Mitra’s Hole in the Wall research that inspired Vikas Swarup to write Slumdog Millionaire demonstrates that education can be refactored into more self organized learning environments. Similarly, Richard Baraniuk is developing new open educational resources to revolutionize knowledge sharing.
• Nobel Peace Prize winner Dr. R.K. Pachauri is driving policy change and promoting sustainable development around the globe through his research on climate change at The Energy and Resources Institute.
• Innovators like Arianna Huffington and Gerry Jackson are reinventing the news business. Huffington is developing a next generation distributed news organisation, and Jackson, who operates the only non-state run radio station for Zimbabweans, is finding ways to use technology as an invisible medium to bypass censors and tell the important stories on the global stage that would otherwise never be heard.
• Researchers like Andy Baio and Jon Udell are uncovering brilliant ways people can use tools to connect with other communities near them both physically and intellectually.
• Channel 4’s Matt Locke is empowering young people to deal with issues they face with projects like the International Digital Emmy winner Battlefront. Similarly, William Perrin of the Kings Cross Environment and Talk About Local is networking together community campaigners across the country to help people get things done more effectively.
• There are some amazing data-driven projects that are changing the world such as Steve Coast’s OpenStreetMap, a sort of wikipedia of location information which grows richer every day by 10’s of thousands of active volunteers who are creating a collaborative view of the world. And there’s also Gavin Stark’s AMEE project which aims to measure the carbon footprint of everything on earth.
• Dr Ian Lipkin is identifying, studying and tracking the trajectory of infectious diseases throughout the globe. And Jay Parkinson is revolutionizing healthcare by changing the way people communicate with their doctors:
  

  “Technology will not solve healthcare’s problems. New business models combined with today’s technology and transparent market forces will…Healthcare needs to be Amazoned, Zipcarred, Facebooked, Etsyed, Tumblred, Appled, and Zapposed.”

• Forward thinking designers like Matt Webb are reintegrating the networked and physical worlds. And Ryan Carson is innovating on the concepts of the social web.
• And while John Van Oudenaren is using the Internet to preserve the past, Nik Bostrom is challenging where we’re going at the Future of Humanity Institute and Oxford University.
• Of course, the foundation services enabling these visionaries to do their work are in many cases is powered by the accomplishments of people like Werner Vogels at Amazon and Bradley Horowitz at Google who are opening the vast technological capabilities and resources of their organizations.

Crucially, though, the technology behind all these movements is a tool in a larger agenda than the technology itself.

And this is why the event matters now. We’re tying to focus heavily on the do-ers, the type of people who break things to see how they work, people who are committed to larger agendas in life, leaders with global perspectives and deep concerns for the future.

It’s about the people actively changing the world and showing us all how to do it, too, hence the name – ‘Activate’.

Why now?

Brian Eno painted the picture that I hope Activate will convey when he described his Sydney Opera House light display:

“To imply ‘Oh God, there’s a crisis, no time for imagining any more’ – it’s not true. This is the time for imagining…The human ability to imagine made people capable of surviving. By allowing ourselves to let go of the world that we have to be part of every day, and to surrender to another kind of world, we’re allowing imaginative processes to take place.”

But perhaps a more tangible answer to ‘why now’ was captured by John Heilmann who observed via twitter:

“Amazing how much important campaign 08 stuff happend in 06. More amazing how oblivious I was at the time – and I was paying attention!”

I suspect a lot of people feel the same way and wish to recalibrate their perspective of what this revolution is all about. Hopefully, Activate will be the platform for people to reset and point forward again.

• in hard times people look for someone to blame for what’s wrong in the world
• people are learning to expect more transparency from their government
• the availability of the data compells the curious to dive into all the detail and look for trends and interesting nuggets
• activities are surfacing that people want to understand
• the prospect of uncovering abuses that result in the downfall of a politician is too exciting for people who crave gossip to resist
• …and on and on

The Internet is optimized for this kind of story.

A big pile of personal data was posted publicly in a usable format. (This data has been available via parliament.co.uk as PDFs for years, but once it’s in a convenient spreadsheet format it suddenly becomes meaningful and very shareable.) People then started finding interesting trends with very little effort. And then we got a very public flame war.

Here’s a snapshot of some of the recent triggers around this issue:

Now, despite the fact that it’s incredibly important for this kind of thing to be possible, I think the scale of the conversation about it is very much a distraction. Stephen Fry captured this sentiment in a quip for a BBC journalist:

“Let’s not confuse what politicians get really wrong with things like wars with the rather tedious obsessions about whether or not they charged for wisteria. It’s not that important. It really isn’t. It isn’t what we’re fighting for. It’s a journalistic made up frenzy.” (Only, I disagree that’s a journalistic made up frenzy. The MPs, the public and the mainstream media organizations are all contributing to the noise together.)

If twitter activity can be considered an insight into what MP’s are spending their valuable time thinking about, then this issue is definitely becoming too big. This Tweetminster chart shows that ‘expenses’ are much more relevant today than ‘banks’ amongst MP’s who use twitter, for example:

Again, I’m happy that we live in a world where our politicians who we pay to represent us are accountable in a very open and public way and that we have the ability to ask them hard questions directly from wherever we sit in the social hierarchy.

Interestingly, this case also provides a view into the cost of openness.

Here are some of my favorite Twitter posts about the launch (more here):

• @IanYorston “Guardian Open Platform may be the most interesting thing to happen to newspapers in ages”
• @netspaze “When major newspapers are closing down, UK’s Guardian is opening up.”
• @lisov “Oh my! Take a look at what The Guardian’s done! An open platform!”
• @newscred “The Guardian’s Open Platform is an awesome initiative, their support & developer engagement is impressive.”
• @dreamingspires “The Guardian open platform is a genius idea. maybe the way forward for newspapers.”
• @estragon “Is this the future of journalism?”
• @evirtus “Seriously impressed by The Guardian Open Platform! The future of news?”
• @kate_butler “THANK YOU http://www.guardian.co.uk/open-platform i love you more than ever.”
• @matlock “Guardian Open Platform is a fantastic piece of work. not jealous at all, honestly. ”
• @r1tz “Serious kudos to the Guardian for launching the open API”
• @tomskitomski “Thinking that Guardian’s Open Platform is what BBC Backstage could and should have become.”
• @adrianholovaty “Super impressed with the Guardian’s API.”
• @SamShepherd “Guardian Open Platform – I am a) impressed and b) disheartened. the gulf between the likely-to-survive and the soon-to-be-bankrupt widens”

And here are some of the best quotes from the media and blogosphere (more here):

• “The Guardian is showing some guts by embracing new business models instead of clinging on to old, defunct ones.” (Mashable)
• “The media brand is less a destination and a magnet to draw people there than a label once you’ve found the content, wherever and however you found it. So the more places you can find it, the better.” (Jeff Jarvis)
• “a whole bunch of smart, enthusiastic developers are going to leap on the Open Platform API and use it to make products that use news content in interesting ways. And the insights won’t cost Guardian a cent (penny?) in testing or development costs.” (LiveMint)
• “Guardian Open Platform is a chasmic leap into the future. It is a work of simplistic beauty that I’m sure will have a dramatic impact in the news market. The Guardian is already a market leader in the online space but Open Platform is revolutionary. It makes all of their major competitors look timid.” (Tom Watson, MP)

This is my personal favorite…

• “If content is king, then this is service is a hundred of the king’s best horses, and thousands of his best messengers, sending the Guardian far and wide. A misstep online is unlikely to cost the Guardian much, and should only encourage competitors innovation—the industry sure needs it.

  With this move, the Guardian redraw of where the boundaries of the newspaper industry lie, using to technology to reach as far as possible. It’s enough to make Conrad Black spit his prison breakfast all over his email-inbox.” (Bad Idea Magazine)

The feedback we’ve heard while participating in various events this month has also been very very positive.

• Phil Wills and Mat Wall were both part of QCon London.
• Simon Willison was on a panel at SXSW with The New York Times, NPR, Wired and DayLife talking about “the technical hurdles, the internal arguments, the surprising ways in which people have discovered new ways of looking at the news.”

  Simon was also part of Future of Web Apps in Dublin which was a few days before our launch.

• I shared some of the thinking behind the Open Platform at Changing Media Summit in London last week in a talk called “The Open Strategy“. That presentation is here:

  (Changing Media Summit is “for anyone concerned with creative and commercial success in the digital age. It is aimed at senior executives responsible for strategies in digital, online, new media, mobile, marketing, branding, finance, comms, content, audio and more.”)

• We also hosted the Rewired State event earlier in the month, and BarCamp London is going to be in our Kings Place offices this weekend which we’re really looking forward to.
  

There’s a ton of work to do to move closer to our vision for the platform. If we are going to be successful in weaving the Guardian into the fabric of the Internet, then we need to grow the services we rolled out already and develop the additional services that will round out the offering.

But if the Open Platform launch was about creating the conditions for positive things to happen, then I think we’re off to a really good start.

First, infographics editor Paddy Allen explains the global recession through a very elegant interactive piece “Where did all the money go?“. Paddy has quite a collection of brilliant work from his interactive infographics such as the Energy-hungry houses piece to his storytelling through interactive visualization like the map of Heathrow’s planned 3rd runway.

Second, a strong team led by editor David Leigh has begun posting their investigations into “The tax gap,” a study of tax avoidance by big business.

“It has taken a team of specialists more than three months and involved checking scores of trademark registers and sets of company accounts in Luxembourg, the Netherlands, Switzerland and Ireland.”

One of the many ouputs of the investigation is the raw data that is informing some of the work, such as the interactive guide to corporate tax. For example, you can see what British Airways has reported paying compared with what is notionally due against their stated profits. The information is available in XML format, such as this year-by-year feed.

Third, and this is my personal favorite, the Football guys have outdone themselves with a new feature called Chalkboards. The Guardian’s head of sport Ben Clissett explains:

“No football debate will ever be the same again – it’s not about opinion any more, it’s about facts. And our chalkboards give you the ammunition to settle the argument. You can also compare two players side by side – if you want to compare Robbie Keane and Steven Gerrard in the same position for Liverpool, or Michael Essien and Mikel John Obi for Chelsea.

And when you have built your chalkboard, you can save it and start a discussion with your mates simply by pressing the save button and explaining your point. You can also embed images you have created on your blog, and use the tool with social networking sites.”

For example, I can see clearly for myself that Aston Villa’s draw against Wigan on Saturday was not due to a lack of offense. They had 16 attempts on goal, in fact, 4 on target and 3 shots blocked. The level of detail is amazing. I can also see where the teams focused their passing during the game.

 by Guardian Chalkboards

This is the kind of data that typically only team owners and managers have access to. And even though the super fans can keep much of this in their heads, they can’t watch every game.

Now, perhaps the best part of this is the embeddable Chalkboard image. Since much of the Premier League discussion is happening in places all over the Internet, it makes sense to share the Chalkboards that both editors and users are creating both on and off guardian.co.uk.

Simple but very clever.

I love that each of these is so different. But there can be no doubt that data is starting to drive a lot of very creative approaches to the journalism process.

• “it is noisy outside. my daughter said, ‘the texans are shooting their guns!’ no, dear. just fireworks“
• “Asked my son to hold my unfinished snack while I drove and said, ‘I’m full, don’t let me have any more.’ He said, ‘OK. I spit in it.’“
• “Son asked me to get yardstick from garage. Told him to use ruler from his room. He said, I am measuring your width. I need a yardstick.“

So then I wondered whether there was a way to capture, prioritize and then syndicate the best Twitter posts into a ‘kiddie quote of the day’ or something like that.

My experiment only sort of works, but there are some lessons here that may be useful for community builders out there. Here’s what I did:

1. Get the quotes: I ran some searches through Twitter Search and collected the RSS feeds from those results to create the pool of content to use for the project. In this case, I used ‘daughter said‘ and ‘son said‘. I put those feeds into Yahoo! Pipes and filtered out any posts with swear words. Then I had a basic RSS feed of quotes to work with.
2. Prioritize the quotes: I’m not sure the best way to prioritize a collection of sources and content, but the group voting method may do what you want. Jon Udell has another approach for capturing trusted sources using Del.icio.us. For voting, there’s an open source Digg clone called Pligg. I set it up on a domain at Dreamhost (I called it KidTwits…Dreamhost has a one-click Pligg installer that works great) and then pumped the RSS feed I just made into it. In no time I had a view into all the Twitter posts which were wrapped in all the typical social media features I needed (voting, comments, RSS, bookmarking, etc.).
3. Resyndicate the quotes to Twitter: While you might be able to draw people into the web site, it made more sense in this case to be present where interested people might be socializing already. First, I created a Twitter account called KidTwits. Then I took a feed from the web site and sent it through an auto-post utility called twitterfeed. Now the KidTwits Twitter account gets updated when new posts bubble up to the home page of kidtwits.com.
4. Link everywhere possible: When building the feed into Pligg I made sure that the twitter ID of each post was captured. This then made it possible to “retweet” with their IDs intact. Thus, the source of the quote would then see the KidTwit posts in their Twitter replies. It works really well. People were showing up at the web site and replying to me on Twitter the same day I began the project.

   Again, I used Yahoo! Pipes to clean up and format the feed back out to Twitter to include the ‘RT’ and @userid prefix to each entry. I played around a bit before arriving at this format.
   
   I also included a Creative Commons copyright on all the pages of the web site to make sure the rights ownership issues were clear.

   Lastly, I added a search criteria for my feed collector that looks for references to KidTwits. This means people can post directly to the web site either by adding @kidtwits to their posts or #kt. There was already a New Zealand Twitter community forming who began using ‘kt’ to join their posts (short for kiwitweets), but they gave it up. I then had to filter out references to the kidtwits Twitter posts to avoid an infinite loop.

5. Improve post quality: Now, here’s where things have been failing for me. I can’t think of better search terms to capture the pool of quotes I want, but there are so many extraneous Twitter posts using those words that it seems like I’m getting between 5% and 10% accuracy. Not bad, but certainly not good enough. The good news is that it’s pretty easy to kill the posts you don’t want through the Pligg interfaces. I just don’t have the time or desire to maintain that.
6. Optimize the site: I then did a bunch of the little things that wrapped up the project. I added Google Analytics tracking, created a simple logo and favicon, customized the Twitter background, and configured Pligg to import the Twitter Search pipe automatically.

There are several things I like and a few I dislike about this little project.

• I really like the fluidity of Twitter’s platform. It’s amazingly easy to capture and resyndicate Twitter posts.
• I love the effects of the @reply mechanism. I can essentially notify anyone who gets their Twitter post listed on the home page of kidtwits.com without lifting a finger. And they get credit automatically for their post.
• I already knew this, but Yahoo! Pipes is just brilliant. I can’t imagine I would have even considered this project without it.
• Pligg is pretty good, too. It does everything I want it to do.
• I would love to hand over the management of the voting and quality checks to someone else. Voting naturally invites gaming. At the end of the day, however, the quality control and community management function is what makes a community service interesting to people. You can’t automate everything.
• I’m actually not a fan of voting approaches to prioritizing content. It will ultimately result in dumbing down the quality. That’s less of an issue for highly niched topics like this, though.
• The rights issues are a little weird. This wouldn’t be a problem in forming a community whose purpose is noncommercial naturally. But I’m not sure the Twitterverse would respond well to aggregators that make money off their posts without their knowledge or consent. (To be clear, KidTwits is not and never will be a commercial project…it’s just a fun experiment.)
• Auto-retweeting feels a bit wrong. I wouldn’t be surprised if the KidTwits account gets banned. But I have explicitly included the source and clearly labeled each Twitter post with ‘RT’ to be clear about what I’m doing. I’m not building traffic to my account, the web site, nor am I intentionally misrepresenting anything.
• By adding “RT @userid” I’ve killed the first 10 or so characters of the post that I’m retweeting. This means the punchline is often dropped which kills the meaning of the retweeted post.
• Some conversational Twitter posts get through which include @replies to another user. When the KidTwits retweet of that post goes out it’s very confusing.

The potential here, among other things, is in creating cohesive topical communities around what people are saying on Twitter. You can easily imagine thousands of communities forming in similar ways around highly focused interest areas.

In this method the community doesn’t necessarily have the typical collective or person-to-person dynamics to it, but the core Twitter account can act as a facilitator of connections. It can actually create some of the authority dynamics people have been wanting to see. It becomes a broker of contextually relevant connections.

In a very similar way the web site serves as a service broker or activity driver. It’s a functional tool for filtering and fine-tuning the community experience at the edge. The web site is not a destination but more of a dashboard or a control panel for the network.

The experiment feels very unfinished to me still. There’s much more that can be done to create better activity brokering dynamics across the network through the combination of a Twitter account and a web site, I’m sure.

“The decentralized nature of the net means that it will never be physically full. As long as we can keep making hard drives, we won’t run out of space to store those inane videos of your Aunt Sally. What is full is our attention.”

I just refuse to believe that we’ve hit the ceiling of what the human brain can deal with.

There is no doubt that we have a lot of useless information available to us, much of it pushed at us, cluttering our lives in really irritating ways. But information overload is a symptom of some bigger issues that we can and should resolve.

I think it’s about better linguistics, technologies and education, to begin with. More broadly, it’s about how we collectively understand and apply abstraction layers to manage a more complex world.

Like everyone, I hit my attention limit nearly every day. Seth is right when he says “You can’t read every important blog… you can’t even read all the blogs that tell you what the important blogs are saying.”

That’s a reason to explore some more, not to give up. We shouldn’t become fatalistic about the future of information or look down our noses at all that messy stuff strewn about the Internet. I never want the flow of information to slow down or, worse, retract, no matter how much mess gets in the way of finding the stuff that matters to me.

What we may need are more dramatic changes in our language, more effective information discovery services, more experience-based education programs both for kids and adults, and, perhaps even more important than all that, an altered world view that can accommodate and make the most of the vast resources that are now part of our culture forever.

Mat Wall details what happened behind the scenes:

“[Simon Willison] wrote a piece of code to extract the 12,000 BNP member’s postcodes through the They Work For You constituency API. Now he had a voting constitency for each person on the list. He then injected this data back into his hack day project to plot this information onto the map obtained from Wikipedia. This took about an hour.”

Update: The project didn’t end there, it turns out. The infographics guys used the concept for the newspaper the next day. Here’s a picture of what was printed:

Here’s a 15min highlight reel:

We did a lot of the standard stuff that makes Hack Day so interesting, but there were a few innovations to the event format itself that I thought worked really well, too:

1. DabbleDB. Simon Willison setup a simple hack submission queue using DabbleDB, a handy online database tool. It’s as if the software was designed for this purpose. Two nice benefits: 1) you can upload a screenshot with your submission which it displays nicely, and 2) it prints beautifully. I handed out a hardcopy of the hack demo queue for each judge who then used the list to take notes.
2. Double Screens. We setup 2 projectors so we could jump back and forth between presentation locations and save some time. While one person was presenting, the next person was setting up on the other screen. I was a little worried it would be distracting, but that wasn’t a problem at all.

   I think this is primarily what kept the pace up. We got through 37 hacks in just about an hour. At that pace you couldn’t really afford to look away. Oh, and Simon’s lightning timer was hugely helpful, too.

   This then had the nice effect of giving the judges more time to deliberate…

3. Comprehensive recognition. The judges went through every single hack and found a way to acknowledge each participant. Emily Bell did a sort of improv act dishing out the jokes. She first went through all the hacks that “we would have given an award to”. Then she handed out the trophies…
4. Silly trophies. These worked perfectly. You can keep it on your desk. It makes no sense to anyone else. And it reinforces the idea that the recognition is for the work itself, not for winning a competition. We did hand out a couple of Flip cameras and Make Magazine generously offered some free subscriptions for the hardware hacks, but the emphasis was clearly on the hackers and their hacks, not the idea of ‘winning’.

Otherwise, it seemed to operate much like other Hack Days, except for the refreshing focus on hacks that mean something. I wasn’t sure what kind of hack quality to expect which was in fact very high, but I loved the fact that most of the hacks had the added dimension of context.

Many times a Hack Day results in a lot of amazing technology solutions for problems that don’t exist. I would never challenge the value of creativity for creativity sake, as that’s a big part of what Hack Day is about. But I was really happy to see that in addition to the impressive technical hacks things like Ben Griffiths’, Rob McKinnon’s and Simon Willison’s hacks (to name a few) presented data and information in new ways that could influence the way people think about what they are reading or interacting with.

Anyhow, the event was fantastic, and I’m really looking forward to doing it again.

This is what we saw (1 min):

Some other accounts of the event:

Neil Girling: “Extra Action Marching Band led the crowd in jubilation, and there were many cheers and chants of “Obama” and “U S A.” The mood was ecstatic, and the cops were polite and extremely hands-off; a little after midnight, when they finally asked Extra Action Marching Band — who have a reputation for chaos and noise — to start to shut down, they did so with smiles and were met with the same.”

Tim Redmond, SFBay Guardian: “San Francisco is going crazy. I haven’t seen this much excitement in the streets since we shut down the city when the Iraq war began. But this time, we actually have something to celebrate.”

Sean Bonner, SF Metblogs: “My neighborhood went 9 kinds of insane last night.”

Strange Things Will Happen: “It was like Italy had won the World Cup, only with less mopeds and more high-fiving.”