O'Reilly Radar - Insight, analysis, and research about emerging technologies.

Google Buzz re-invents Gmail

2010-02-09T19:00:00Z

When I first heard about Google Buzz, I was worried that I might be seeing the birth of another "me too" product. After all, everyone wants a piece of the Twitter halo. But with the release of Buzz today, you can see how Google has taken the social media lessons of Twitter and applied them to their own core products.

I'm especially fond of Gmail Buzz, which adds the power of asymmetric following to email.

AWESOME idea. There are many of us for whom email is still our core information console, and our most powerful and reliable vehicle for sharing ideas, links, pictures, and conversations with the people who constitute our real social network. But up till now, we could only share with explicitly specified individuals or groups. Now, we can post messages to be read by anyone. Sergey Brin said that Buzz gives the ability "to post a message without a 'to' line." That's exactly right - something that in retrospect is so brilliantly obvious that it will soon no doubt be emulated by every other cloud-based email system.

Buzz items can be shared directly in Gmail, but are also pulled in from other social sharing sites, including Twitter, Picasa, YouTube, and Flickr.

What's particularly cool is that the people you "follow" are auto-generated for you out of your email-based social network. If you communicate with them, they are the seed for your buzz cloud. Over time, as you like or dislike buzz entries from that network, the buzz cloud adapts.

Google has also done a neat hack on the Twitter @name syntax, allowing you to prefix @ to an email address to have a message show up for sure in that user's Gmail Inbox. Saying @foo@gmail.com (or @foo@bar.com) will put a message into foo's Buzz cloud in the same way as saying @foo does on Twitter, but it will also show up in their Gmail Inbox, to make sure they see it. You can also make messages private to only named recipients or groups. (I love this - right now, I have two Twitter accounts, one for public sharing, and another for private sharing.)

I've always found it perplexing that vendors who manage pieces of our communications network for us - our email, IM, and phone - have failed to build social networking features into their products. Google is clearly now tackling that job, increasingly making its communication products into a powerful social media platform. Gmail already includes IM and some automatic social learning in the address book; adding Buzz makes it that much more powerful. And the fact that whatever you buzz is added to your Google profile (and immediately picked up in Google search) will turn those seemingly vestigial Google profiles into something that might just become the next generation personal home page.

You can begin to see where all this is going: the integration of Gmail, Buzz, Reader, Voice, Geo, Blogger, YouTube, Calendar, Contacts... Buzz is a game-changing first step, but when you think about where Google will take this over the next year it gets exciting...

There's a real lesson here for anyone who wants to enter a crowded market: play to your strengths. Think through what job that hot new startup does for its users. Don't copy what they look like. Apply what they've taught you to your own business.

There are real benefits to using email as a social media platform. Just about everyone knows how to use it. (Despite claims that young millenials look down on email, it's just too useful to go away anytime soon.) It's incredibly flexible - you can share anything you want, and comment on it at any length, from 140 characters to as many as it takes to get your point across. It has a global address space that allows you to find almost anyone, an address space that links people to content. It's multi-platform, and accessible from anywhere.

In some ways, Gmail Buzz brings many of the benefits of Google Wave to Gmail. Every Buzz item can be turned into a conversation (much as in Wave or Friendfeed.) People can comment on your Buzz, comment on your comments, or @ reply you. Sure, it lacks the hyper-cool wiki-style shared editing features (though those perhaps could be added in a future release), but it also lacks the critical flaw that made Wave into more of a "concept car" than a real product: I don't have to adopt a new tool or build a new social network. It just adds rich new capabilities into the tool and network that I already use.

Google has also done a terrific job of giving inline preview to links you share. This is especially awesome for photos and videos. The inline slideshows are terrific - actually better than you get in most native photo or video sharing apps. And I love that you can share a Flickr link as easily as you can share one from Picasa (bucking the trend of vendors to try to lock you in to their own services.) Google says it's committed to Buzz being "the poster child for what it means to build an open, standards-compliant social product that serves the interests of users..." I'm looking forward to seeing more signs of this commitment as Buzz (and other Google products) evolve.

You can read more about the functionality behind Buzz at O'Reilly Answers: "Google Buzz: 5 Things You Need to Know."

P.S. There's also a great, related Buzz announcement for Mobile, which shows off Google's platform thinking. On the mobile phone, Buzz is automatically "snapped" to your location, also using metrics like time of day to figure out the most relevant location (e.g. during the day you might be at Google, but if it's nighttime, it may be more likely that you're at the Shoreline Amphitheater across the street.) Buzz related to a location will show up on the relevant Google Placepage, and in a new geotagged Buzz layer on Google Maps. What we're seeing is the application of algorithmic relevance to buzz - and the power of what I've long been calling "the internet operating system."

P.P.S. Buzz will be rolled out starting at 11 pm today. Apparently, it will take 2-3 days to show up in every Gmail account; if you don't have it right away, be patient.

Ignite, Syndicated Events, and Social Media Marketing

2010-02-09T13:41:32Z

As we approach Global Ignite Week, a collection of Ignite events around the world during the first week of March, I can't help but think about the future of conferences, one of O'Reilly's major businesses. Here are some of the things we're learning from Ignite.

People love the rapid-fire format. Steven Levy once said that Foo Camp is the wiki of conferences, an unstructured space where the attendees make things happen. Well, by that measure, Ignite is the Twitter of conferences, a way to quickly share information and spark enthusiasm. The Ignite slogan: "Enlighten us, but make it quick." is a great way to force speakers to focus on the essentials. It's amazing how much you can pack into five minutes when you're on the clock.

We're increasingly using the Ignite format at our traditional conferences as a way to highlight lots of great ideas that people can dig down into later. We've had 5 minute "Lightning Talks" at the Open Source Convention since 2003, but Ignite has a social environment halfway between structured sessions on stage and the "hallway track" that is so exciting at many conferences. As a result we're now holding Ignites in conjunction with many of our conferences, both as part of the program and as a social event in addition to the regular conference content. We've also organized Ignites at other events, such as Google I/O and Adobe Max, and upcoming at SMX West.
Self-organization enables amazing scale. Since Brady Forrest and Bre Pettis launched the first Ignite event in Seattle in December 2006, there have been over 180 Ignites held around the world, with over 80 of those held in the past six months. Each Ignite has the same format: an evening event, often in a bar or other informal meeting place, starting out with a Make: contest, followed by a series of short talks, with 10-15 speakers given five minutes to speak on the subject of their choice, each with 20 slides auto-advancing every 15 seconds. Organizers invite speakers and, like any event organizer, pick people who will engage the audience. Events usually draw at least 100 attendees, and the largest Ignite to date has had 800 attendees. 2-300 is the average.

We learned a lesson when Foo Camp led to Bar Camp, and hundreds of other "camps" (City Camp and Crisis Camp being two fabulous recent incarnations), and so, with Ignite, we set out from the first to make it a self-organized event, providing instructions and a mailing list for Ignite organizers.
Syndication allows a decentralized event to gain some of the benefits of aggregation. Accordingly, we provide a central calendar listing of upcoming events, a video portal, and other opportunities for organizers and participants to share what happens at their local event.

We're working to develop additional mechanisms to support local Ignites, including social networking tools, and a much improved video portal (to be released in time for Global Ignite Week). Each Ignite provides its own factory for innovation, so we're looking for the best ideas from local organizers and working to spread them more widely.

We're particularly interested in developing mechanisms for syndicated sponsorship.
Up till now, there has been some local sponsorship of Ignite events. Local sponsors might provide beer in exchange for a banner, or give away product from the stage. Ignite Portland began showing short sponsor videos during the socializing breaks. Here's an example:

With Global Ignite Week, we realized that we've reached the scale where we can engage major sponsors. Global Ignite Week will have the reach of a large trade show, with 15-20,000 participants. Across all Ignite events this year, there may well be significantly more than 50,000 participants.

We've come up with a sponsorship model in which major sponsors can contribute a video to be shown across all participating events. If sponsors understand the format and deliver entertaining, informative video rather than traditional marketing spam, this can be a huge opportunity to engage passionate, interesting (and often highly technical) audiences. (In the future, we hope to have these video sponsorships in the form of actual syndicated Ignite sessions.)

There's a particularly interesting aspect to Ignite that we've come to realize. It's a social event, and so sponsorship at Ignite is fundamentally social media marketing. In addition to the people who attend each Ignite event, millions more are exposed to the event via Twitter and Facebook. We've been working with PeopleBrowsr Analytics and O'Reilly Research to understand the social media impact of Ignite events.

We compared the tweet count and reach from the Web 2.0 Expo NY and Web 2.0 Summit events last fall with the tweet count and reach from the Ignite events happening in the same timeframe. (For purposes of comparison, we decided to use a date range from 10 days before the Web 2.0 Expo till 20 days after the Web 2.0 Summit.) For the Web 2.0 Expo, we counted tweets using the #w2e and #w2expo hashtags; for the Web 2.0 Summit, we counted tweets using #w2s and #web2summit; for the various Ignite events, we counted tweets using either #ignite and the individual hashtags recommended by the organizers of the Ignite events held between 12 October to 24 November. As you can see from the figure below, the Web 2.0 events each generated a huge, concentrated spike, while the Ignite events provided a repetitive series of spikes, each much smaller, but important in the aggregate.

The Web 2.0 Summit generated 8,723 tweets from 2,356 individual users with a combined reach (aggregate of all followers of unique tweeters using one or more of the hashtags) of over 11 million, with 74 million potential tweet impressions (aggregate of all tweets seen by all followers.) The Web 2.0 Expo NY generated 11,950 tweets from 2,953 users with a combined reach of 6.4 million and nearly 42 million potential tweet impressions. Meanwhile, the 26 Ignite events held around the world during October and November generated 8,026 tweets from 2,585 with a combined reach of 3.5 million and over 11 million potential tweet impressions.

Clearly, the numbers were stronger for the traditional events - especially the Web 2.0 Summit, whose tweeters included a much higher proportion of "influentials" with high follower counts. But the Ignite movement is gaining steam. While the numbers for the sample period were smaller than those for the traditional events, when you use the Ignite data to project the expected tweet count from Global Ignite Week, the numbers are quite comparable. The sample period included 26 Ignite events spread over two months, and a total of perhaps 6000 participants. With more than 79 events currently scheduled (and perhaps as many as a hundred, as more are added each day) over a period of a week, Global Ignite Week (#giw) should generate more than 3 times the attendance and the tweet traffic that we saw during the sample period - as many as 25,000 tweets with a combined reach of 10 million followers and 35 million potential tweet impressions. Over the course of a year, several hundred Ignite events will have an attendance and a social media impact that exceeds that of even large traditional events.

We're still working out how to manage the syndicated sponsorship opportunity. Challenges include finding sponsors (prospectus pdf here) who understand the opportunity, making sure that those sponsors understand the Ignite culture and provide valuable content, developing mechanisms for sharing sponsorship benefits with local organizers (for example, we're talking with Facebook about providing in-kind advertising that organizers can use to bring attendees to their events), and working to understand the demographics and interests of the attendees. With tools like PeopleBrowsr analytics, it's increasingly possible to measure these things (and much more, including attendee sentiment) via the twitter "data exhaust."

There's an important twist to this story. A recent study showed that 70% of companies plan to spend more on Twitter & Facebook marketing rather than traditional marketing channels. Given the new social media marketing disclosure rules put forward by the Federal Trade Commission, you've either got to do explicit ads, or sponsor content that will spread on its own. Ignite is a great way to do social media marketing right.

There's a nifty Bing map from Global Ignite sponsor Microsoft that makes it easy to find an Ignite near you. Click on the heading of the map below to see a larger version that lists all the Ignite locations alphabetically beside the map.

Four short links: 9 February 2010

2010-02-09T11:00:00Z

Track DC -- informative drill-down report from Washington DC government about the different departments. (via Sunlight Labs blog)
Errors in Scientific Software -- a 1994 study of scientific software that found inconsistent interfaces (1 in 7 for Fortran, 1 in 37 for C) and poor use of arithmetic such that significant figures declined from 6sf in the data to 1sf in the result. (via "If you're going to do good science, release the computer code too" in the Guardian)
How Farmville Scales -- 75M players/month (28M/day), 1/4 of disk activity is writes, 50% higher load spikes, 3G/s traffic go between Farmville and Facebook at peak, LAMP stack, nagios+munin+puppet. (via Hacker News)
Mathematical Philology -- when two manuscripts of the same text differ, which is correct? This PLoSONE paper looked at all such discrepancies in Lucretius's De Rerum Natura and found that the traditional principle of choosing the more difficult reading (on the grounds that errors are from humans unconsciously simplifying) has a strong information theory justification for it. Interesting to see this less than a week after an MIT Technology Review article on quantum teleportation remarked, There is a growing sense that the properties of the universe are best described not by the laws that govern matter but by the laws that govern information.

Flickr Photos In Google Street View

2010-02-09T06:05:38Z

Google Maps has added more user photos to its Street View (above). Now the Yahoo-owned Flickr is joining the Google-owned Panoramio and Picasa photo sites as a supplier of alternative street views. GeoBloggers reported it earlier today and also noted that the photos are available in the Panoramio 3D view (below).

This is significant for two reasons:
1) Flickr has millions of geotagged photos (2.3 million photos with location data were uploaded this month; 95,634,285 in total as this writing). These photos document the earth and with the addition of location metadata they can become useful for more than just photo-lovers. User-generated data and content is being used in significant ways to represent the earth -- especially online. Human contributions show up in base mapping data (in products like Google MapMaker and Open Street Map) and in routing data (in products like Tele Atlas MapShare). This is another proving point in the case for the human built map.

2) The web is a platform and it is great to see excellent, rival services able to work together to build a superior product. I have put out some questions to the Flickr team about how this came about and some of the inner workings of the deal, but I am pretty sure that it would have only been done if the Flickr and Google Maps teams were working together. I am curious if any money was exchanged (none is my guess), how often the Flickr photos get updated, where else these Flickr photos are going to show up in Google's services (Google Goggles perhaps?) and will they show up in new search partner Bing? I am doubly curious if Facebook will ever let its photos be used in a similar way.

We'll be discussing Mapping, Mobile and Local trends with Google and Yahoo! (and others) at Where 2.0. The three day conference runs March 30- April 1 in San Jose. Radar readers can register with this discount code for 25% whr10pcb.

Four Steps to Gov 2.0: A Guide for Agencies

2010-02-08T14:15:09Z

What Does the World Look Like When the Work of Government is Driven by the People?

Gov 2.0 has a lot of definitions, but in observing the exciting breadth of projects currently being built, it feels a little like the Blind Men and the Elephant, where everyone defines it based on their first hand experience, but not from a holistic view. In its essence, Tim O’Reilly’s definition of Gov 2.0 is where government acts as the catalyst to let others build upon its work h — and most importantly, to multiply its impact.

For the first time in history, we’re really at a point where this is technologically feasible. Even if you have no specific tie to government, Gov 2.0 envisions a world in which — just by having experience and interests — ordinary members of the public willingly contribute to the knowledge, facts and policies that comprise our government. It might be as easy as carrying your cell phone. And it might take just 30 seconds.

In December, the Obama Administration released its long awaited Open Government Directive, which was met with enthusiasm from some, and an underwhelmed “meh” from others. The Administration has asked state and local government to adopt the Directive, but it still begs the question:

If I am an agency head and want to embrace Gov 2.0, what should I do first?

Right now it’s a confusing whirlwind of options: Create raw datafeeds in machine readable formats? Create iPhone apps? Use a wiki internally? Create a Facebook group, a Facebook page? Start posting to Twitter? The choices are infinite, but the resources are most definitely limited.

Below is a starting discussion, a "Four Steps to Gov 2.0," designed to align the various Gov 2.0 stakeholders - individuals, governments, private companies, elected officials - toward the same goal in pursuit of open and participatory government. It applies to all levels of government at the federal, state, and local level. It attempts to structure an agency’s actions as prioritized consecutive steps, in a way that will reward those that adhere to it with more power, better engagement, and future compatibility with other government agencies, private companies, experts, and the general public. Even a few years ago, it would have been technologically impossible or at least prohibitively expensive. Now, the biggest obstacle is simply a plan and the political will.

It's most definitely an amalgam of many different ideas, especially Clay Shirky’s idea of convening the conversation, and the Obama Administration’s ideas around releasing high value datafeeds and making government transparent, participatory and collaborative. It prioritizes the steps, and finally, introduces the idea of an API that creates a virtuous cycle by returning crowdsourced value back to the agency.

Four Steps to Gov 2.0

1. First and foremost, “convene the conversation.” Governments that want to win should first maximize the free contributions of the general public and experts for issues handled by that agency. Focus on creating the systems to foster self-organization and moderation (think user voting, forum moderation, and social reputation).

Before all else, this should be the first — and only — goal of agencies at every level. The original Obama campaign site and Peer to Patent are great examples, and several other early examples are starting to emerge.

2. Next, examine your agency’s data and put it into three “buckets”. If you have not completed #1, go back and do that first because you’re leaving a valuable resource on the table. The buckets are:

Define high value data sets that can be shared in machine-readable format. This is data that is not updated frequently, never anticipates the need for improvement, and is generally referential in nature. Examples might include historical spending, infrastructure details, and census-like data.
Define high value data sets than can be interacted with via an API. This is data that anticipates improvement from the public, and/or which regularly needs to stay updated by the agency. Examples might include permits, locations of buildings, and crime data.
Define the data types that are not shared, period. Shine a bright light on these data types, and make very clear statements as to why they are not shared. If “getting to the data” is the reason for not sharing, put that to the community and you will be able to find someone to help you get that data out for free. Examples are data that is already protected by law, or which contains personally identifiable information.

3. Next, build the datafeeds, because they will help maximize the public’s information and contribution in Step #1. Push this data to the public in machine-readable formats: XML, RSS, or CSV, accessible via Web services.

4. After the datafeeds are complete, build the API. Look at this as a social compact, where as part of the exchange, companies and members of the public are able to return value back to the agency, creating an infinite loop of ever improving data. Use it to generate mechanical turk-like assistance from the public. I’ll explain some of the key components of an effective Gov 2.0 API in a future post.

After these steps have been accomplished, look at building a regular Web site, specific applications, and services. Agencies that prioritize in this order won’t put themselves at risk of building social silos (these are social networks that end at the boundary of the town, state, or agency).

I’ll consider each of these steps individually in subsequent blog posts. If you have more ideas, please let me know here or send a note at greg [at] crimereports.com.

Google Book Settlement Round 2

2010-02-08T08:06:00Z

The US government filed its Statement of Interest regarding the revised Google settlement yesterday with the District Court in New York. While the statement was signed by an attorney from the Antitrust Division of the Justice Department, several agencies including the Copyright Office reportedly contributed to it.

As you may recall, the judge has only 2 choices: he can approve the settlement, or send it back to the parties for revision. He cannot modify it himself.

The US government statement advises the judge that the public interest would be best served by sending the settlement back, and points out that the revised version still suffers from the "same core problem" that afflicted the first version: "an attempt to use the class action mechanism to implement forward-looking business arrangements that go far beyond the dispute before the Court in this litigation."

The press reports that I've seen take the government's statement as an emphatic thumbs down.

The judge has scheduled a hearing for February 18 in his Manhattan courtroom.

It is very unlikely that the judge will approve this version of the settlement. Also, he may once again decide to postpone a full-fledged fairness hearing-although the many objectors, large and small, are eager to have their day in court. Because the parties withdrew the proposed settlement before the originally scheduled fairness hearing occurred in October 2009, the judge has not yet formally considered the many objections filed to date on the revised settlement or those filed in anticipation of the fairness hearing cancelled last October.

Bottom line for the long term: even if the judge sends the settlement back, and even if the parties agree to deadlines as short as the deadlines for this presumably ill-fated revision, there is no resolution in sight for the litigation.

Whether the case is tried or the settlement discussions continue, the legal end point will not be the trial judgment or settlement approval issued by the district court judge. The end point will be the disposition of the final appeal from that district court judgment or approved settlement, and that disposition is years away.

It's hard to imagine what relevance the final legal disposition would have then, as public and private innovators are not sitting on their hands, waiting for the judge to sort this out.

Feedback and analysis: the missing ingredients in local's recipe

2010-02-08T08:05:00Z

There's plenty of enthusiasm for local / hyperlocal projects, but the sweepstakes has yet to be won. PaperG CEO Victor Wong digs in to some of the missed opportunities in a paidContent.org guest column.

I found this excerpt intriguing:

How useful would it be to know when local used-car dealerships have a large increase in inventory (and thus are probably more willing to sell at a lower price)? Other data like new-car listings could show what the local population is buying by examining what is posted and taken down by the dealers. Publishers can even create new content by encouraging users to input data about what sorts of deals and treatment they got, which would be useful for other local buyers and could be turned into a local car-buying guide.

Wong has a stake in the local game -- PaperG focuses on local advertising -- but that doesn't diminish the point he alludes to in the excerpt: feedback and analysis are the missing parameters in the local equation.

So many of these local efforts rely on traditional information delivery through news articles or databases. That material has use, no doubt. Yet few projects take the extra step and put that data into context. They don't explain why the information is important. They don't connect the dots.

A lot of this reminds me of web analytics. It's easy to grant access to traffic data, and the access itself has a low level of value. But the insight that guides decisions comes from deeper analysis. You need to know why a particular keyword or topic is resonating.

Four short links: 8 February 2010

2010-02-08T08:04:00Z

Kindle Development Kit APIs -- Amazon will release a Kindle SDK. These are the API docs. (via obra on Twitter)
rePublish -- all-Javascript ebook reader. (via kellan on Twitter)
Peer Review: What's it Good For? (Cameron Neylon) -- harsh and honest review of peer review with some important questions for the future of science. But there is perhaps an even more important procedural issue around peer review. Whatever value it might have we largely throw away. Few journals make referee’s reports available, virtually none track the changes made in response to referee’s comments enabling a reader to make their own judgement as to whether a paper was improved or made worse. Referees get no public credit for good work, and no public opprobrium for poor or even malicious work. And in most cases a paper rejected from one journal starts completely afresh when submitted to a new journal, the work of the previous referees simply thrown out of the window. Some lessons in here for social software, too.
Analog IMDB -- The transition is moving slowly, but it’s moving. It’s a fascinating thing to watch. The technology is the dull part: what’s interesting is the shift in perception. You know how sometimes you turn off a certain section of your brain and force yourself to see a word not as a piece of language with meaning, but as a sequence of black shapes and white spaces? It’s like you’re seeing that image for the very first time and suddenly “bird” seems like a very odd thing. I’ve been buying all of my in-print books electronically for a couple of years. Physical books aren’t weird to me yet. But damn, that old copy of the Maltin guide was a freaky and bizarre object. It’s the first time I looked at a book and didn’t see a container for information. I saw dead wood.

Four short links: 5 February 2010

2010-02-05T11:00:00Z

The Public Domain Manifesto -- eloquent argument in favour of the public domain. (via BoingBoing)
Clear Climate Code -- project to write and maintain software for climate science, with an emphasis on clarity and correctness. What a wonderful way for coders who aren't scientists to contribute to open and better science. (via the interesting OKFN blog)
Don't Hash Secrets -- One area of secure protocol development that seems to consistently yield poor design choices is the use of hash functions. What I’m going to say is not 100% correct, but it is on the conservative side of correct, so if you follow the rule, you (probably) can’t go wrong. You might be considered overly paranoid, but as they say, just because you’re paranoid doesn’t mean they’re not after you. So here it is: Don’t hash secrets. Never. No, sorry, I know you think your case is special but it’s not. No. Stop it. Just don’t do it. You’re making the cryptographers cry.
Javascript Grid Editors -- nice wrapup of available Javascript editable grid components, divided into "data driven", "light edit", and "spreadsheet". (via joshua on Delicious)

One hundred eighty degrees of freedom: signs of how open platforms are spreading

2010-02-05T02:17:59Z

I was talking recently with Bob Frankston, who has a href="http://frankston.com/public/Bob_Frankston_Bio.asp">distinguished
history in computing that goes back to work on Multics, VisiCalc,
and Lotus Notes. We were discussing some of the dreams of the Internet
visionaries, such as total decentralization (no mobile-system walls,
no DNS) and bandwidth too cheap to meter. While these seem impossibly
far off, I realized that computing and networking have come a long way
already, making things normal that not too far in the past would have
seemed utopian.

Flat-rate long distance calls

I remember waiting past my bedtime to make long-distance calls, and
getting down to business real quick to avoid high charges.
Conventional carriers were forced to flat-rate pricing by competition
from VoIP (which I'll return to later in the blog). International
calls are still overpriced, but with penny-per-minute cards available
in any convenience store, I don't imagine any consumers are paying
those high prices.

Mobile phone app stores

Not that long ago, the few phones that offered Internet access did so
as a novelty. Hardly anybody seriously considered downloading an
application to their phones--what are you asking for, spam and
fraudulent charges? So the iPhone and Android stores teaming with
third-party apps are a 180-degree turn for the mobile field. I
attribute the iPhone app store once again to competition: the href="http://www.oreillynet.com/onlamp/blog/2008/01/the_unflappable_free_software.html">uncovering
of the iPhone SDK by a free software community.

Downloadable TV segments

While the studios strike deals with Internet providers, send out
take-down notices by the ream, and calculate how to derive revenue
from television-on-demand, people are already getting the most popular
segments from Oprah Winfrey or Saturday Night Live whenever they want,
wherever they want.

Good-enough generic devices

People no longer look down on cheap, generic tools and devices. Both
in software and in hardware, people are realizing that in the long run
they can do more with simple, flexible, interchangeable parts than
with complex and closed offerings. There will probably always be a
market for exquisitely designed premium products--the success of Apple
proves that--but the leading edge goes to products that are just "good
enough," and the DIY movement especially ensures a growing market for
building blocks of that quality.

I won't even start to summarize Frankston's own href="http://frankston.com/public/">writings, which start with
premises so far from what the Internet is like today that you won't be
able to make complete sense of any one article on its own. I'd
recommend the mind-blowing href="http://frankston.com/public/?n=Sidewalks">Sidewalks: Paying by
the Stroll if you want to venture into his world.

But I'll mention one sign of Frankston's optimism: he reminded me that
in the early 1990s, technologists were agonizing over arcane
quality-of-service systems in the hope of permitting VoIP over
ordinary phone connections. Now we take VoIP for granted and are
heading toward ubiquitous video. Why? Two things happened in parallel:
the technologists figured out much more efficient encodings, and
normal demand led to faster transmission technologies even over
copper. We didn't need QoS and all the noxious control and overhead it
entails. More generally, it's impossible to determine where progress
will come from or how fast it can happen.

Apple vs. Adobe vs. Content Creators

2010-02-04T15:30:00Z

Remember when Wired's fancy tablet demo made the rounds a few months ago? That Adobe Air-driven prototype certainly stoked the fires of iPad enthusiasm.

There's just one problem: ~~It won't work on the iPad.~~ It won't work natively on the iPad.

Leander Kahney at Cult of Mac explains why:

Apple has rejected Adobe technologies like Flash and Air — with extreme prejudice. No one at Condé Nast appears to have seen that coming, even though the iPhone OS hasn’t supported Flash since its launch in 2007.

~~Maybe Condé Nast developers thought the iPad would run Mac OS. Or maybe they just got ahead of themselves.~~

Update 2/5: Wired editor-in-chief Chris Anderson notes in the comments, and in a follow-up piece at Cult of Mac, that the iPad's Flash limitations were known from the start. Wired will be available on the iPad, as well as Android and Windows.

Time Inc. ran into a similar problem just before the iPad's launch. Its Sports Illustrated tablet prototype was constructed around a wish list, not tech specs.

This is the first sign I've seen that the Apple vs. Adobe spat is spilling beyond the tech space. Content creators accustomed to the Adobe toolset -- particularly Air and Flash -- will have to recalibrate if they want to be on the iPad (and really, who doesn't want to be on that thing?). That means more development and a longer wait for consumers.

Four short links: 4 February 2010

2010-02-04T11:00:00Z

Google Ad Preferences -- my defaults look reasonable and tailored to my interest. Creepy but kinda cool: I guess that if I have to have ads, they should be ones I'm not going to hate. (via rabble on Twitter)
Android and the Linux Kernel -- the Android kernel is forked from the standard Linux kernel, and a Linux kernel maintainer says that Google has made no efforts to integrate. (via Slashdot)
On Amazon EC2's Underlying Architecture -- fascinating deconstruction of the EC2 physical and virtual servers, without resorting to breaking NDAs. (via Hacker News)
First Full Open Source Symbian Release (BBC) -- source code will be available for download from the Symbian Foundation web site as of 1400GMT. Nokia bought Symbian for US$410M in 2008 (for comparison, AOL bought Netscape for $4.2B in 1999 but the source code tarball had been escape-podded from the company a year before the deal closed). This makes Symbian more open than Android, says the head of the foundation: "About a third of the Android code base is open and nothing more,” says Williams. “And what is open is a collection of middleware. Everything else is closed or proprietary.” (quote from Wired's story).

What Facebook's HipHop means for developers and businesses

2010-02-03T18:18:23Z

Facebook's PHP overhaul, HipHop, reportedly cut CPU usage on the company's servers by around 50 percent. You don't have to be a programmer to understand that kind of result.

Facebook says it wants to share that efficiency -- and presumably influence a few things along the way -- so it's setting HipHop loose as an open source project. I asked Kevin Tatroe, co-author of O'Reilly's Programming PHP, to weigh in on HipHop's functionality and its broader applications.

Mac Slocum: How will HipHop help programmers?

Kevin Tatroe: One of PHP's greatest strengths is its expansive leniency. But for very large code bases, it can also be somewhat problematic. For example, while you can change the type of data stored in a variable mid-script, I'd wager that the vast majority of the time, it's a mistake.

Those are the kinds of things HipHop's analysis and type inference steps will find. For that reason, I can even see running sites through HipHop that don't need footprint savings, just as a sanity check.

MS: How about businesses that rely on PHP. Does HipHop offer them any utility?

KT: Certainly. At its best, PHP scales very well by running on teeming hordes of cheap servers. As great as that strategy has turned out to be, running on half as many teeming hordes of cheap servers has clear benefits in deployment costs and costs to maintain.

There's also benefits at the other end of the scale. I've seen more companies than I can count stick with one deployment server when they really ought to be scaling up to two or three. There's a "leap of faith" barrier there. Companies say: "Obviously, we can't deploy on zero servers, so one seems fine. But two? We're just a small operation. Can't you make your code work better?"

MS: Facebook calls HipHop a "source code transformer." In plain terms, what is that?

KT: It takes the PHP code written by PHP programmers and converts it to C++ code, which is then compiled by g++ into machine code.

In doing so, it has to disallow certain PHP language features, like eval(). And it runs a pass to determine what type each variable in your PHP code should be in C++.

PHP does not require you to state up front what kind of data you're going to store in a variable. It lets you change the type of data stored in a given variable willy-nilly, which is not necessarily the best idea.

MS: Facebook says HipHop reduced CPU load on its servers by about 50 percent. Any idea how it does that?

KT: Running native, compiled C++ takes less processing effort to run than PHP's scripts via an opcode virtual machine (such as using Zend Engine). That's because it's skipping the virtual machine entirely.

It's telling that Facebook didn't mention any HipHop speed increases. It'd be surprising if there weren't any measurable speed improvements, but their primary focus seems to be: use less cheap hardware to run the same site.

MS: How easy -- or hard -- do you think it will be for other companies to take advantage of HipHop?

KT: This all depends on the tools. Certainly, any organization with deployments large enough to really notice much out of this will have devs comfortable with the more traditional "write, compile, test, deploy" cycle.

PHP also attracts a large percentage of folks who've never had to compile anything in their lives. But these same people aren't necessarily afraid of the command line. If the tools are simple enough that people aren't scared off, it should be pretty simple to get a site up and running using HipHop.

Note: Kevin's comments were condensed and edited from a longer interview.

Four short links: 3 February 2010

2010-02-03T11:00:00Z

Bad Census Data for The Last Decade (Freakonomics blog) -- the "representative sample" of statistics data that the Census Bureau releases has apparently been flawed. It's been used in thousands of studies, and the Census Bureau has refused to correct it.
Modern Telephone Fraud -- it's actually an old fraud updated: an insecure digital PBX used to route expensive calls. Innocent company is whacked with bill at end of month. Interesting questions raised about what we expect company to do (pay?) and telco to do (forgive?). It's a good reminder that every electronic product is now an avenue for fraud or intrusion, but we don't plan or contract for these situations.
Found Functions -- Nikki Graziano adds mathematics to photographs. Her photos let me see the world through a mathematician's eyes. (via sciblogs)
Getting Past Good-Enough E-Books -- fantastic list of TODOs for ebook publishers.

Forget Google, social search is all about mobile

2010-02-02T14:17:47Z

There's considerable chatter about a seismic shift in search. A lot of it's overblown, but the central idea is intriguing: Google's biggest problem -- the one that keeps company execs up at night -- isn't Bing or Ask or traditional search. The real threat is social search.

Or so the thinking goes ...

I've always dismissed the notion that Twitter or Facebook could knock Google from its throne. Those services are built for speed, not depth. And even though Google is a huge organization, it still has the agility and forward-thinking to fend off attackers.

Earlier today, I ran across a data point in Aardvark's new social search report that I find way more interesting than Google's theoretical downfall. It's not whether social search will displace Google. It's how -- and where -- social search can actually be useful

TechCrunch's breakdown of the Aardvark report includes this bit of analysis:

[Aardvark's] average query volume was 3,167.2 questions per day, with the median active user asking 3.1 questions per month. Interestingly, mobile users are more active than desktop users. The Aardvark team attributes this to users wanting quick, short answers on their phones without having to dig for anything. They also think people are more used to using more natural language patterns on their phones. [Emphasis added.]

The real seismic shift in social search will come from its commingling with mobile applications.

Why? Because mobile is a different animal than the desktop. No one wants to fumble around for queries. People on the go don't have time to scan listings. The screens are too small, and the input mechanisms -- improved as they are -- are way too clunky.

Mobile search has to be concise and targeted. Results that emanate from a trusted network of friends and associates certainly fit that bill. Toss in more geolocation features and improved speech recognition, and the utility of mobile-based social search could get really interesting.