It’s a fact that most newspapers are dealing with significant declines in traditional readership. The internet has become ubiquitous, and people have fully embraced using free online sources for keeping up with the news.

Most news organizations viewed this shift over to the net as a real opportunity, and began to make their content available online. Unfortunately, the projected ad revenue they had hoped would come from this move never materialized, and they were unable to offset the declines that happened in their traditional revenue sources as a result. They now find themselves left with declining interest in their traditional, revenue producing marketplace, strong growth in online ventures that they have been unable to monetize effectively, and a balance sheet that is bleeding red ink. These organizations are now growing restive, and calls have been coming from within the industry for publishers to begin charging for online content. Rupert Murdoch, Chairman and CEO of News Corp, seems to have become the voice of this effort:

“The digital revolution has opened many new and inexpensive distribution channels but it has not made content free. We intend to charge for all our news websites.”

I believe that moving to a pure “for fee” model with online content simply wont work for most news organizations. It ignores some fundamental changes that have taken place in the market. The uncomfortable truth is that the industry wont succeed by looking backwards for business inspiration. The news world has changed.

There was a song written by Joe Young and Sam M. Lewis after the first world war called How Ya Gonna Keep ’Em Down on the Farm (After They’ve Seen Paree). It was basically a lament that after experiencing a more cosmopolitan world, it would be hard for the young men that fought over in Europe to come back home and be content in their former, simpler lives. And though this song is nearly a century old, it applies equally well to the dilemma the newspaper industry finds itself in.

The traditional news consumption model involved buying a newspaper with everything packaged inside, and then reading through it in whatever order and depth appealed to you. It was a closed but comfortable view of the world around you. To draw the analogy, that newspaper was The Farm. In contrast, the internet is the Paree of information. Everything you could want is there. Every topic. Every perspective. Every form of media.

The real challenge facing the news industry isn’t that people don’t want to pay for a paper like the New York Time because they can get it for free on the internet. If that were the case, charging for it online would make complete sense. The challenge they have is that people don’t want to pay for just the New York Times because they can get EVERYTHING online for free.

And while the “free” part is great, it’s the “everything” part that’s essential.

The fact is, we just don’t go to a single site to read the news. We graze for news across the entire internet, following links and references as we go. We share articles with each other via IM’s, tweets and blogs, and constantly discover new sources that way. News consumption has moved from being linear and static into being both interactive and social. We depend on having a constant connection to an almost endless flow of information. This is something no one is willing to give up.

That said, having access to news does have real value, and I believe people would be willing to pay something for it. What has changed is that people are far less willing to pay for specific sources of news anymore. While some publications may be able to charge for their content uniquely, the industry as a whole will die if it tries to go that route. People are no longer satisfied with limiting what they read to just a few publishers.

They’ve been to Paree and are not going back to the farm.

One way to address this challenge would be for the newspaper industry to form a new not-for-profit news cooperative designed around the distribution of news. In many ways this could act as a compliment the Associated Press - the industry cooperative formed to help share the high cost of gathering news. This new cooperative could act as the clearing house for all news articles published by it’s member firms. It could charge a single subscription fee for access to all of the content it cleared, and pay it’s members a percentage of those fees based on some formula linked to the popularity of their specific content. This could at least provide a business framework the industry could build on. Getting a viable commercial model in place is critical, and this could be a big help.

But it will do nothing to address the news industry’s other big problem - massive oversupply.

Historically, newspapers were local affairs. They usually served the town or a city they were founded in and didn’t get much further than that. These papers covered everything from world and national affairs to regional and local events. Since distributing newspapers was a physical affair with real costs and logistics, these organizations ended up becoming localized monopolies of a kind. They serviced a captive audience of both advertisers and readers, and were very profitable endeavors.

With the shift to online news, however, all of the distribution constraints went away. The internet is global and everyone can reach everywhere. Reporters that covered national news for their local audience are now competing with hundreds or even thousands of other news sources with nearly identical coverage. This radically changed the economics of the business.

The market simply can’t support the number of news sources that exist today, and some significant changes will need to happen. Many news organizations will need to scale back and just produce something that is uniquely local or otherwise differentiated. Many more will simply need to fold their tents and close. The rightsizing of news supply will be a very painful transition for the industry, but there is simply no way of avoiding it.

Overall, the status-quo in the news industry is unsustainable, and the traditional industry business model is on the fast track to extinction. As that plays out, I believe we’ll see very different news provider community start to emerge that is far more in tune with the web - leaner, better integrated, more collaborative, and hopefully much more profitable.

But to get there, the industry needs to stop clinging wistfully to its past.

Let me start by saying that I own a Sony ebook reader, and have enjoyed using it for many years. While all of Sony’s readers are well designed and engineered, they have been held back by a lackluster (and somewhat expensive) ebook store, and a lack of hunger on Sony’s part to really push the ebook market where it has the potential to go. Their Connect store is poorly thought out, and despite their recent introduction of several new reading devices, they are still lagging behind the current ebook market leader - Amazon.

Amazon’s reading device - the Kindle - offers wireless access to their ebook store, allowing books to be purchased and downloaded directly from the device. It also supports the automatic delivery of subscription based digital content like newspapers and magazines. In addition, it also offers basic web browsing capability with access to knowledge sites like Wikipedia.

Sony has none of these features now, and probably won’t for the rest of this year.

On the other hand, B&N’s Nook has almost all of Kindle’s capabilities, a few unique ones of its own, and dangles the possibility of some interesting software enhancements in the future. The Nook is a dual screen device. It has a large e-ink display on top for reading, and a small touch enabled color display below it for navigation. While it is hard to make usability judgments without living with something for a while, its design seems better thought out and more appealing than the joystick/menu navigation model used by the Kindle. The Nook also has ePub and eReader format support, making it open to connection with other ebook stores as well as public domain titles from Google (as the Sony can as well). The Nook also brings a new concept to the ebook space - lending books to friends. When it comes to the basics, Nook seems to be a good mix of both feature refinement and clever innovation.

What is most exciting about the Nook is that it is based on Google’s Android OS. Barnes & Noble has implied that they may be interested in opening up the platform in some way to third party developers. I really hope they do. This could extend the interface with useful utilities and widgets to access various web based services. This openness could give Nook the potential to become much more that just and ebook reader. And that is exciting. If there are no negative surprises once it starts to ship, the Nook will be a serious threat to Amazon’s Kindle.

As for Sony, they are clearly outgunned at this point. I don’t see a strategy outside of serious price cuts that will allow them compete with Amazon and B&N this holiday season. If they are committed to being in this market long term, Sony will need to invest a lot more next year into improving their entire ebook ecosystem. Time isn’t on their side.

The thing is, I just not sure that Sony is really committed to making that kind of investment in this market.

They have been boxed in at this point and don’t have too many options available to them. It will be difficult for them to find a partner in this space. Both Amazon and B&N are clearly off the table, leaving Borders as the only full catalog bookstore that might have the scale to do something with them (assuming Apple hasn’t already locked them down!). It would also be difficult for Sony to just shutter their store and be successful as a device-only provider. Open ebook standards aren’t widely adopted at this point, and there just aren’t enough independent sellers available. Without an industry wide adoption of common open ebook formats, it will be difficult for any unaligned ebook provider to have a critical mass of titles to build a viable business around.

Apple proved that it can be good for the bottom line to tie a device to a store, and both Amazon and B&N have taken that lesson to heart. And that leaves Sony in an unenviable position: double down or fold and get out.

I hope they find a way to stick it out.

One aspect of the Sidekick that has made it unique is it’s cloud based architecture. By default, all of a person’s contacts, emails, photos, & messages are stored up in the cloud. The storage on the device is used more like a local cache, with data persistence becoming a centralized service. The big advantage of this approach was security - if a Sidekick device ended up being lost or stolen, all of a person’s data would still be safe and easily sync-able with a replacement unit.

At least that was the theory.

The reality, described in this announcement from T-Mobile, has ended up being quite different:

Regrettably, based on Microsoft/Danger’s latest recovery assessment of their systems, we must now inform you that personal information stored on your device - such as contacts, calendar entries, to-do lists or photos - that is no longer on your Sidekick almost certainly has been lost as a result of a server failure at Microsoft/Danger. That said, our teams continue to work around-the-clock in hopes of discovering some way to recover this information. However, the likelihood of a successful outcome is extremely low.

I have never, in my experience, seen an outage like this happen - even in tiny start-ups that are running pretty lean and mean data centers. In situations where a catastrophic outage does happen, there will typically be a roll back to an earlier version of the system, with the loss of only recent updates. And that really is a worst case outage.

But somehow, in this outage, everyone’s’ data is just gone. All of it. No backup seems to be available.

For what’s it’s worth, Microsoft does have some really sharp engineering talent - a few that I know personally. They’ve been running massive data centers for quite a while and clearly understand operational best practices. That makes what happened here, at least to an outside observer, a complete enigma. Losing everything is simply unheard of in professional circles. Whatever the cause, this is just a screw-up of unprecedented proportions.

So what’s next for Microsoft?

They need to take public ownership of the situation. This means more than just getting to the bottom of what happened here and fixing it. Microsoft also needs to be completely transparent about what occurred. No matter how ugly or unflattering it may be, they need to discuss what went on openly and honestly. Most importantly, they need to communicate what steps they are taking to be sure that this type of event won’t happen with ANY Microsoft service again.

Ultimately, the biggest loss that took place as a result of this outage was the loss of trust - trust both in Microsoft and in their cloud based architectures. It may not be fair, but that’s the reality of the situation they find themselves in. This isn’t the time for Microsoft to just crank up the PR machine and try to spin this. They also shouldn’t try to blame the folks from Danger as a way to distance themselves from this mess. Neither of those approaches will repair the damage. When events like this occur, there is no quick fix. Rebuilding marketplace confidence will be a process that takes both time and effort.

There are a lot of people watching how Redmond responds to all of this. How they handle this situation will be critical to establishing the success or failure of their entire cloud based strategy - Azure, Office Live, and MyPhone.

The future of the company depends on them getting it right.

While some of the concepts being explored here are interesting, I don’t see them offering much of an advantage over a large multi-touch surface like the track-pad on the MacBook Pros. That said, if Microsoft were to combine this work with some of the innovative things going on through their XBox 360 Project Natal, that could open up some exciting interface design possibilities:

The continued innovations around low cost processing power, remote sensor technologies, and real-time software will certainly push the introduction of some interesting computer-human interface advancements.

Experimentation here will be key.

Multi-touch is a great starting point, but we have a long way to go…

Here is a clip of the XML for a track from my own library:

There really isn’t anything hidden here. Anyone that wanted to sync this track to their device could read all of the meta data, format it for their own library structure, then follow the link to the media file and copy it all over to the device. This is a fully open way for any third part application to interact with iTunes stored media.

Instead of leveraging this open format, Palm has made the decision to try and impersonate an iPod and use Apple’s undocumented protocols directly with iTunes sync. Assuming they are not totally strapped for cash, they could have easily written their own sync code. My guess is that they thought they could make some headlines for themselves and generate some bad press for Apple if they tried to make this into an issue.

It doesn’t need to be.

As I’ve said before, the Palm Pre looks like an excellent device and has some very innovative features. I just can’t understand why Palm thinks it makes good business sense to try and score point against Apple by providing a terrible media experience for their customers. These are the folks they need to have in their corner. “Word of mouth” advertising is critical to any newly launched device - especially for a company in Palm’s position.

And the word that comes to mind right now is “unprofessional”…

What if every peripheral someone would want to connect to a computer could hook up to it via a single strand of fiber optic cable? This would include monitors, cameras, keyboards, printers, networks - basically everything short of power. The concept was that the myriad of connectors built into systems today could end up being replaced by a single thin cable.

Intel’s newly announced “Light Peak” hopes to achieve exactly that. Take a look:

Not only could adoption of Light Peak clean up the ‘cable clutter’ associate with the typical computer setup (the same way HDMI has cleaned up the cabling of AV systems), but it could also de-clutter the design of systems themselves! Computers, especially laptops and other portable form factors, would no longer need to be designed around the requirements of a wide range of bulky connectors. A single tiny connector would be the only consideration impacting their design.

This would let system designers avoid the connectivity compromises Apple had to make to in developing an incredibly thin or small device like the Macbook Air:

Enclosures could finally become sleeker and more rounded without forcing people to give up the connectivity they depend on. Even small handheld devices like the iPhone could easily connect to a range of devices without changing its size or form factor.

Apple has always pushed the envelope when it comes to new storage and connectivity options - everything from eliminating floppy drives, to support for USB and Firewire, to ExpressPort, to the ADC video connector and recently the new Mini DisplayPort. In large part this has been driven in service of better, sleeker device designs.

I can’t wait to see what they come out with once Light Peak is available…

Their own offering - “Hulu Desktop” - is described in this video:

Taking a cue from Apple’s Front Row media design, Hulu Desktop lets you use a simple 6 button remote to navigate to and play any content they make available. While it lacks any of the socially anchored features provided by Boxee (beyond ratings), Hulu Desktop does provide a simpler interface for those focused solely on content.

And content is Hulu’s biggest asset.

The interface works smoothly and intuitively, and shows a great deal of refinement for a “version 1″ release. The one MAJOR thing lacking that I had expected in Hulu Desktop is a way to ‘record’ shows for playback when disconnected from the internet. Maybe in a future release…

What Hulu Desktop ISN’T is a full media portal. You can’t integrate your Netflix account or stream media you have stored locally. It’s just about the programming that’s available through Hulu - but for many that won’t be much of a limitation.

If you haven’t tried it out, it’s definitely worth a look.

You can download it here for either Mac and PC.

While many are wizards at adding and extending the features and capabilities of the systems they work with (and in some pretty amazing ways), they can be almost dysfunctional when it really comes to getting rid of code and infrastructure that has outlived its original purpose. Some of the best developed systems around seem to just collect screens, functionality, subsystems, API calls, database tables, etc that - while possibly important a one time - add almost no value to the end user today. That fact that the most significant feature of Apple’s newly released “Snow Leopard” version of their operating system is it’s cleaned up, slimmed down code base speaks volumes about the state of complex code packages these days.

There are lots of reasons systems get fat. Some of it comes from engineers simply over-engineering things and making things more complicated than they really need to be - usually by choosing purity over practicality. A LOT more of it comes from the “need” for companies to continue adding new features to their platforms - no matter how marginal - to generate upgrade revenue and justify support contract costs.

Some of it also comes from designers that like to keep the product fresh, programmers that want to add ‘cool new things’ they are interested in, and sale folks that push for one-off additions to try and win new sales.

When it comes to bloat, there’s plenty of blame to go around.

But wherever it comes from, all of this extra code (and the infrastructure that goes into supporting it) typically ends up surviving release after release. And while there may be someone out there that is actually still using it, support for marginally used functionality comes at a steep price. Some areas impacted by this are:

• Complexity: People are already complaining that many technology based systems and devices are confusing and difficult to use. Years of legacy functionality only adds to this problem.
• Mobility: Feature heavy products don’t translate well to small footprint mobile devices. And mobile is where the money is heading.
• Reliability: The more that is put in to a release, the greater the odds that something will fail. There are bits of code living in any complex system that no one really understands, and changing things around it can cause all type of reliability problems that are difficult to diagnose and fix. That’s why there are so many “work-arounds” out there.
• Cost: It costs a lot to add new code (especially when it isn’t really needed by the marketplace). If you also add in the even larger costs needed to maintain it, code around it, and QA it over the life of the product, the ROI starts looking pretty sad.
• Performance: Bloated systems run slower and take longer to load than optimized systems. And sorry - no system is going to get faster by adding more code to it.

This boils down to one simple thing: the need to a more disciplined approach to designing systems. Designers need to place the same value to pruning marginal features from a release that they do adding new ones to it. They need to know their clients, know their markets, and have the guts to make the near term tough calls that will result in a better product for everyone over time.

For those not familiar with the technology, a CableCARD is effectively a digital settop box on a card. It can be plugged into a slot available on many digital televisions and some DVR’s, allowing them to decode all of a premium channels available from the cable provider.

Prior to this announcement, only devices built by CableLabs’ certified manufactures were allowed to include a CableCARD slot. While not a problem when it came to television sets, the lack of cableCARD support was a very big deal to anyone looking to build a computer based DVR for themselves. No manufacturers were allow to sell cableCARD compatible cable tuners to individuals on an OEM basis, limiting this market only to those companies selling certified, completely bundled systems (most of which were relatively expensive and fairly limited in power and storage).

Good news, right?

Well - sorta. At this point, it seems the only systems this will work with will be those built around Microsoft’s Windows 7. While I think Microsoft’s latest version of Windows Media Center is really elegant, I would much prefer to building a solution around the open sourced MythTV instead:

The main reason I would go with MythTV is that it doesn’t included any form of content protection. It is an open sourced effort built by folks with no concerns about keeping any media interests happy as part of a larger business strategy. It’s a product where the user comes first. And that means you get the most flexibility on how you can use the content - record what you want, create archives of shows you want, and sync them with any portable devices you choose.

And when it come down to it, that’s the reason you’d want to build your own DVR in the first place.

We still lack a universal means of asserting and communicating identity online.

It would be hard to imagine a functioning modern society without a broadly accepted means of establishing identity. And as more components of our social interactions move online, our lack of a singular, verifiable online identity is devolving from being merely cumbersome to becoming a major liability. It fractures the web into isolated communities that do not play nice together, and hinders the development of many commercial and personal forms of sharing and interaction. What’s most frustrating is that in the digital realm, universal identity is something that could be implemented in a way that’s far more convenient and efficient than any analog world equivalent.

In the physical world, identity depends on a complex network of ‘trusted agencies” that provide various levels of assurance that a person actually is who they claim to be. As individuals, we start with our birth records, and use that to begin establishing who we are. In the United States, the issuance of a birth certificate allows a person to get a uniquely identifying social security number. Issued by the federal government, this social security number is then used by schools, financial institutions, employers, and other government agencies as a means of linking records to that specific individual. These records build up on an historical basis to provide a comprehensive picture of who a person is.

Pieces of this historical record can be used as forms of identification: utility bills, bank statements, active credit cards, a driver’s license, a passport. Some ‘higher’ forms of identification are based on having multiple ‘lesser’ forms of identification. For example, getting a driver’s license in some states requires at least 4 other forms of identification. Because getting these higher forms of identification involve a more rigorous verification process, they often become a more broadly accepted means of establishing identity. For instance, if an institution trusts the screening process done by a state to issue a driver’s license, they can simply accept the license as a form of ID.

For an identity system to work in the real world, it is essential that the sources that need to check identity have trust in the sources that are issuing the identity information. While not originally intended as such, government issued passports and driver’s licenses have become the most trusted forms of identity in our society. As a practical matter, our government has become the ‘trust authority’ for establishing identity in the physical world.

Identity is a key element of so many things government is responsible for - immigration, taxation, permits, licensing, contracts, security, law enforcement, and social programs. Given the key role that government already plays in establishing identity, they are the logical body to address this issue in the online space. This is not an area that benefits from competition. Individuals need to be able to claim their digital identity once - for minimal cost - and have it accepted universally.

Probably the biggest obstacle standing in the way of having the government take on the role of issuing digital identity is the erosion of trust we have in the motives and integrity of our elected representatives. That won’t be easy to change, but establishing clearer regulations around online privacy, ownership of information, and protections from government snooping and misuse could go a long way to making government involvement in this space more acceptable.

We really don’t have another choice.

As we increasingly become a digital society, we can’t continue to ignore the importance of digital identity. What a I am advocating is letting the government issue digital certificates with public/private key pairs that would have the same legal recognition as a notarized signature does today. That’s it. The free markets can take it from there.

It’s been proven time and again that no society can flourish without a foundation of strong personal property rights. And the most fundamental element of personal property rights is your identity.

We need to act.

There are some control systems available in the marketplace that attempt to provide a ‘whole house’ control experience. However, beyond being expensive to customize and install, they tend to have pretty primitive control interfaces (mostly virtual menus and buttons on a touch screen) and are not very intuitive to operate - even for technically inclined people.

Well the folks from the Media Interaction Lab at the Upper Austria University of Applied Science have come up with a new control interface design called CRISTAL - Control of Remotely Interfaced Systems using Touch-based Actions in Living spaces. It is built around a multitouch table (reminiscent of Microsoft’s Surface computing device) that presents a live room image controllable via multi-touch. This video provides a good overview of how the system operates:

This looks like the type of system I would love to have in my house. Touch based, intuitive, and with one panel controlling everything. Even though this is just a lab based demonstration, all of the technologies it is built on already exist in the consumer marketplace today. I could see something like this going commercial in the next few years if it can get a big enough backer behind it to launch it with the scale it needs to be successful. Both Apple and Microsoft comes to mind as potential providers of this kind of home experience. In fact, if Apple does release a tablet at some point in the near future, I believe providing this kind of control surface will be one of the motivating factors.

This is definitely the kind of experience people are looking for. It’s not about navigating menus and touching buttons. It’s about interacting with intuitive proxies for the environment around them. Without a doubt, the army of traditional remotes most people struggle with today have overstayed their welcome.

And no one I know would be sorry to see them go.

The program - officially called Car Allowance Rebate System (CARS) - was passed by congress and implemented by the Department of Transportation. A download from their website describe the program as follows:

The Car Allowance Rebate System is a new program from the government that will help you pay for a new, more fuel-efficient vehicle from a participating dealer when you trade in a less fuel efficient one.

While this program has had the benefit of stimulating new car sales, it really hasn’t done enough to achieve it’s key objective - improving fuel efficiency and reducing greenhouse gas emmisions. New cars being purchased through this program only need to have a fuel economy rating of 22 miles per gallon - 3 MPG below the currently mandated 25 MPG average that an automaker’s entire fleet must achieve. This is simply not a very impressive goal for a program that taxpayers are being asked to underwrite - especially when it is being sold to the public as a green initiative and not an industry bailout.

The concept behind the program is a good one, and is worth pursuing. But the current program is very expensive, and won’t achieve the results an initiative of this size should. I would like to see a new version of this program set up to replace it that leverages both incentives and disincentives, is simpler to administer, and reduces the burden to taxpayers.

The new program would be based around how a new vehicle’s milage compares to the currently mandated fleet average. Based on today’s average, 25 MPG would be the initial benchmark. Any car with better gas mileage would receive an incentive rebate of $1000 for each 10% of MPG improvement it has over that benchmark. The total rebate would cap out at $5000. Helping to subsidize this rebate would be a tax on vehicles falling below the 25 MPG rating. This will take a similar approach of adding $1000 to the cost of a vehicle for each 10% of MPG it falls below that average mark. It would be capped at $5000 or 10% of a vehicle’s total cost - whichever is less. The benchmark could potentially rise each year. If the actual fleet average MPG from previous year end’s up being higher than the benchmark 25 MPG, it will become the new benchmark for the next year. If it is lower, the current benchmark would remain unchanged.

A program like this - with escalating incentives and disincentives - has the potential to shift consumer interest from larger cars over to hybrids and other alternative fuel technology vehicles. It can also shift the focus of the auto manufactures as well. The market advantage for any company that ‘out-innovates’ its competitors in the fuel efficiency area could be significant, and the penalty for falling behind could be severe. This will force every auto company to invest more and prioritize innovations in technologies that will keep them competitive in this space. They simply can’t take on the risk of neglecting it.

As for getting rid of the clunkers that are out there on the road today, I would create a standing offer of $1500 for any insured car in working condition regardless of age or particulars - no new car purchase required. That could put cash into people’s pockets that they could spend anywhere, or save for a rainy day. Any business could be approved to accept these vehicles - not just car dealers. They could receive a $500 processing fee, and would need to follow the same terms around destroying the vehicles and recycling them for scrap.

By being marketplace driven, these approaches could end up improving fuel efficiency standards far more effectively than any government mandated standard could. Not only that, they could also help the US auto industry begin to produce vehicles that are globally competitive.

As a nation we need to become leaders in green technology. It is not just an imperative from an ecological perspective, but also from an economic one. Success in green tech will define the winners over next 25 years in the same way that success in digital technology has defined the winners over the last 25.

It’s too important for us - and for the world - to just pay lip service to it.

In the update I wrote a couple of weeks ago on the Amazon Kindle, I commented on the price of readers:

…the recent Kindle price cut, bringing it down to $299, is another step in the right direction. Though I personally believe it will need to move below $100 to really start to gain mainstream traction, breaking below the $300 price crosses a psychological threshold that makes it easier to bring in that next level of interested buyer.

Well Sony just moved the bar a little closer to that tipping point price, breaking the $200 barrier today with the announcement of their new Reader Pocket Edition. It has a somewhat smaller 5″ e-ink screen and can hold about 350 books. Sony has also announced that they will be matching Amazon’s price of $9.99 for recent best sellers.

Though it doesn’t have any way to let you buy books wirelessly like the Kindle does, the Reader Pocket Edition does cost $100 less - probably a fair trade off for many people. By having an under-$200 reader, along with lower ebook prices, Sony should be able open up the ebook market to a much wider audience. This is a significant move down the price curve, and will hopefully keep pressure on Amazon to continue moving their own prices lower.

Another bit of good news in the ebook world is that Barnes & Noble has decided to jump back into the business. For those that don’t remember, B&N was the supplier of digital books for the pioneering Nuvomedia Rocket eBook in the early 2000’s. After acquiring Fictionwise earlier this year (relaunched as ereader.com), they are now starting to pull their digital book strategy together. Unlike Amazon and Sony, B&N decided not to launch their own dedicated ebook device to go along with their new digital store. Instead, they are starting out by releasing a free software reader that runs on both the iPhone and iPod Touch, with an eye toward partnering with potentially multiple ebook device makers in the near future. This could be an excellent strategy for them. Given the lead Amazon and Sony have in this market, it makes sense for B&N to become the ‘open platform’ in the ebook world with the broadest choice of reading options available. Backing up this effort, they have launched with a significant number of ebooks already available for sale, as well as around 500k free public domain books available for download. Though not as splashy as Amazon’s launch of the Kindle 2, B&N has made a very credible entry into the ebook market here. And like Amazon, they have the buying power and focus necessary to evolve, become successful, and turn this into a viable component of their overall business.

With three major competitors in the ebook space (and many smaller ones as well), it’s clear that this market isn’t going to fade away this time like it has in the past. Between the introduction of new reading devices and the continually expanding catalog of books now available in digital format, the ebook business shows every sign of being a young, healthy consumer product segment.

But there could be one big shakeup coming in the near future.

While there isn’t a lot of clarity around their intentions, Apple is shaping up to become a possible “800lb Gorilla” in the ebook space. With mobile reader apps available from both Amazon and B&N, the iPhone/Touch already offers a great platform for digital book readers. Rumors are also making the rounds that Apple with be launching a color “tablet device” with a 10″ screen - a general portable media platform that could easily include ebooks in the mix. What lets Apple cast such an long shadow over this space is the power of their iTunes ecosystem. They have the store. They have the desktop footprint. They have the device footprint. The introduction of a larger form factor ‘tablet device’ could place them in the perfect position to subsume the ebook market within the already significant digital media market they dominate today.

While even the launch of a new device from Apple is only speculative at this point, their ability to dominate a market has a clear precedent in the way they have moved from music, to audio books, to podcasts, to television shows, and recently to movies. They started out small in each of these areas, but over time have managed to become the dominating force in all of them.

Whatever ends up happening with Apple, it’s great to see so much new activity going on in the ebook space. It seems to be moving into the mainstream on the consumer side even faster than I thought it would.

Hopefully publishers will take note and finally start to ramp up their digital efforts.

It feels like the tipping point is finally getting close.

With InfoNgen, I spend a great deal of time thinking about potentially new and innovative ways to analyze and classify content - including a broad range of web based content. Without a doubt, the research going on around the semantic web is some of the most interesting in this field. While there has been some really exciting progress in applying this research to many constrained information domains, creating this self-describing, intelligent network of information on an “internet wide scale” is still an incredibly daunting task.

And as Fred points out, it isn’t one we are making a lot of progress in.

I am struck by the similarities between the efforts happening here, and the work that took place from the 70’s to the early 90’s in the field of artificial intelligence. In computer circles, A.I. was the cutting edge discipline of it’s day. Until the arrival of the Internet, it was a magnet for creative engineers and scientific talent. People saw it as the next great revolution in technology. Encouraged by successes like chess playing computers that could beat grand masters and medical expert systems that demonstrated real value in clinical situations, expectations were high that we would soon see computers that would be able to interact with us conversationally - personal assistants that could carry out spoken directions and provide us with relevant advice and information. This video - done by Apple in 1987 - is a great example of what people were hoping computers would soon be able to do for them:

It’s more than 15 years later, and we’re still a very long way off from the promise shown in this video.

Today’s efforts to create the foundation of the semantic web are in some ways like a reemergence of artificial intelligence - but now repackaged for a web centric world. Many of the concepts and technical disciplines that were sitting behind A.I. - Bayesian inference, natural language processing, weighted decision trees, classifiers, and knowledge bases just to name a few - are now in some form or fashion powering various commercial and open efforts to realize the semantic web. And while they do share a common set of technologies, that doesn’t mean they need to share a common fate.

But to be successful, things will need to start coming together in a different way.

This time around, these technologies will need to leverage the core social fabric inherent in the web architecture. Analysis needs to be pushed out to the edge and become an integral and interactive part of the content creation process. This would not only be able to suggest tags or other meta level markups, but also offer potential summaries for quick display, highlight ambiguous terms or content blocks for refinement, and suggest unique topical terms that could be included in the content to improve discoverability. The human generated editorial insights that exist in trusted content sets need to be leveraged to mine for relationships in other content sets that exist more broadly. (Fair use/copyright law will need to be updated and clarified keep up with innovations in this area.) Most importantly, the creation of public databases, taxonomies, and ontologies need to become a priority for open source efforts, potentially leveraging a DBpedia style model of publication and quality control. Freely available datasets will be the fuel that powers many of these efforts going forward. Overall, any successful approach here needs to blend the things people do well with technologies that can amplify and extend it, producing something neither could accomplish well on its own.

With all of that said, I’m not naive. I don’t believe we will ever have a truly global, harmoniously classified semantic web. There are simply too many perspectives to rationalize in a way everyone can agree on, and too many people looking to game the process for their own gain. The Utopian model discussed academically is really an idealized goal that isn’t achievable on a practical level. But I strongly believe that it will be possible to offer to the broad web community the same improved web experience currently provided by vertically focused solution providers like InfoNgen. Meaningful progress at this level will require more than the isolated technological breakthroughs of any single company or organization. Though it can be anchored around the same core semantic concepts, getting the scale and scope needed to succeed here will require some kind of cooperative framework to share and enhance the currently disconnected efforts and innovations that are taking place today. Without having some mutually beneficial relationship exist between the various commercial and open sourced initiatives, it is likely that the global semantic web will end up hitting the same kind of wall that the original efforts in A.I. did.

While a technical discussion of the various solutions in this space may be interesting, the end goal of the semantic web is to make it easier for for individuals and organizations to discover and apply information that is relevant to them. This means that access to content needs to become more flexible, and conform to the variety ways people may think about it and want to consume it. This is in sharp contract to the traditionally rigid way publishers have wanted to package and present it in the past.

None of this will be easy, but getting publishers to embrace this kind of change may be the biggest challenge of all.

Aside for their recent (and rather dumb) move of erasing some books people already purchased due to a copyright issue, Amazon has done a masterful job of supporting and extending their Kindle ebook reader ecosystem. I am starting to see far more Kindles on the trains and subways I take each days, and they seem to be appealing to a fairly broad demographic. I am also hearing more people talk about possibly getting a Kindle for themselves, a sign that it’s starting to move a little more mainstream than the original Kindle 1 ever managed to do.

Amazon’s Kindle library continues to grow, providing prospective buyers with a sense of confidence that their choices will continue to expand. On top of that, the recent Kindle price cut, bringing it down to $299, is another step in the right direction. Though I personally believe it will need to move below $100 to really start to gain mainstream traction, breaking below the $300 price crosses a psychological threshold that makes it easier to bring in that next level of interested buyer.

I have had the chance over the last several weeks to borrow my wife’s Kindle and use it on a daily basis. The device itself is incredibly convenient to carry and hold, and really does become transparent once you start reading on it. You just see the words without the hardware getting in the way.

Another great move on Amazon’s part was the introduction of a Kindle reader application for iPhone. Being able to read on the iPhone definitely extends the usefulness of Kindle ebooks for me. When I’m stuck on a line or waiting for a train, I can easily sync with the Kindle 2 and continue reading on the smaller device. It is even a viable reader without having a Kindle at all. Since I don’t actually own the Kindle myself, most of my reading prior to this has been on the iPhone. While certainly not as nice to read on as a regular Kindle, it is a more than acceptable experience. I have read four books (over 1500 ‘real’ pages) this way already, and wouldn’t hesitate recommending this to anyone who isn’t ready to purchase the physical reader yet.

What excites me most, however, is the new Kindle DX. Short of having a color e-Ink screen (which probably won’t be available for another two years or so) it is my ideal ebook platform. It is light and thin, making it easy to hold, but has a large enough screen that I can read it comfortably without my glasses.

Equally important, the Kindle DX includes a native PDF renderer. This means that I could use it to store the incredible number of documents - mostly technical manuals, journals, and brochures - that I end up carrying around with me for work. While being pitched more in academic circles as a device for textbooks, I think it shines as a reader that can address the needs of any technical professional.

Amazon is making all the right moves with new Kindle, and is really starting to build out the ecosystem needed to support it. The demand is there for Kindle, even if the devices are still a bit pricey for most people. As production costs for the readers fall along the natural technology price curve, Amazon should be well positioned to dominate in this space.

I continue to be impressed by what they have accomplished here.

Starting today, anyone connecting with any generation of iPhone or iPod Touch will see a slimmed down rendering of the regular site when they connect:

You can now scroll through a summary of posts on the home page, and click on the “Read Entire Post>” to view the full story - including all images and comments. Controls are also available to let you share posts on popular social sites.

I have been working - albeit slowly - on a complete refresh of this site that is going to include a more customized version of this iPhone template. However, with interest in a solution increasing, I figured it made sense to go ahead and get this basic iPhone support rolled out now. I appreciate your patience.

Let me know what you think.

Thanks to the folks over at iWPhone for the plugin that makes this possible.

Speed, simplicity and security are the key aspects of Google Chrome OS. We’re designing the OS to be fast and lightweight, to start up and get you onto the web in a few seconds. The user interface is minimal to stay out of your way, and most of the user experience takes place on the web. And as we did for the Google Chrome browser, we are going back to the basics and completely redesigning the underlying security architecture of the OS so that users don’t have to deal with viruses, malware and security updates. It should just work.

I have no doubt that Google will try to make Chrome OS a fairly complete solution out of the box. They can certainly roll together all of their own web applications with popular 3rd party web apps to cover most of the key functionality people would look to have when they power a system on. I also expect that Google will extend their Android “App Store” and fold it in to this new OS. This would let new applications download and install just like browser plug-ins instead of like traditional windows applications. If Google can combine that simplicity with ‘instant on’ functionality, Chrome OS will offer a clearly differentiated computing model from any of the “old-school” operating systems.

This is an exciting and important move by Google. Microsoft’s “Windows” is the crown jewel of tech industry franchises. Even for a company the size of Google, grabbing just a small piece of Windows total market share - even an overlapping piece - would be significant. Chrome OS has a lot of potential here.

But…

While the move to a web centric operating may appear conceptually correct and even inevitable, Google will still need to overcome a lot of challenges if they want to make Chrome OS a success:

• Time To Market: Chrome OS won’t be out for another year. In technology circles, a year is forever. Neither Microsoft nor Apple are passively waiting for this to arrive. Windows 7 should be able to support Netbook systems, and more of the Office suite will be available as web based applications. Apple has already claimed a big chunk of this mobile web space with their iPhone, and will likely be releasing a new device this year that will probably appeal to the same audience Chrome OS is targeting. And innovation continues to come from every corner.
• Market Momentum: Windows is everywhere. People are comfortable with it and pretty much know how to work with it. For all it’s well publicized issues, it’s the devil everyone already knows. Getting people to take a chance on something new is tough, and Google will need deliver more than a ‘Field of Dreams’ marketing strategy if they want to get any mind share/traction with Chrome OS. Unfortunately, that’s not an area they’ve shown themselves to be particularly adroit in.
• Mobile Connectivity: Anyone that depends on any of the US wireless carriers for mobile data services already knows just how bad service can be in some places. If I had a hard drive that was as unreliable as these services are, I would need to get it replaced. In a mobile, internet centric computing device, the web is my new “hard drive”. It’s where I store my data and load my applications from. To overcome this Google will need to offer a system that presents a meaningful level of functionality even when users are disconnected from the web, or when connectivity is intermittent.
• Device Support: Beyond everything else, this could be the make or break item for Chrome OS. People have significant investments in all sorts of devices: printers, phones, cameras, scanners, media players, etc. If Google can’t figure out a way to get support ready for the most popular of these devices by the time it launches, it will end up being just an interesting experiment that most people ignore. And it needs to do it without making Chrome OS a slow starting or virus prone mess.

At this point, Google’s Chrome OS is just an idea with potential. It’s success will depend on focus, attention to detail and flawless execution. They will need to articulate clearly how this fits in with their seemingly competitive investment in Android, and actively work with partners in the market place to make sure support is there for it on launch day. Even though Chrome OS will be open sourced upon release, Google needs to take ownership of getting penetration in the market. This is different from any other product they have launched. Google will be asking people to depend on Chrome OS for everything they want to do, and will even need to convince new system buyers to bet their entire purchase on it. It needs to be a complete, fully functional, well supported offering.

I’m excited to see how well Google rises to the challenge…

Ironically, these types of events also remind us of the many limitations that still remain around web content delivery, and the broad challenges the web will face in supporting the kinds of things we are expecting it to support one day.

So how did the web hold up?…

As events unfolded, the LA Times - who broke the Jackson story - had its website crash after several million visitors hit it in less than an hour. Many other major news sites slowed down significantly due to high volume. Even Google, a company used to dealing with access on a massive scale, had problems handling the load of people searching on its Google News section for information about Jackson’s death. And Twitter, a site that has effectively become the web’s real-time “newswire”, was running at least 5 minutes behind in getting tweets posted. While not an infrastructure disaster, this event certainly pushed most news/gossip sites close to the edge of their capacity, and not many had a graceful way to degrade.

Problems seemed to be even worse for the mobile web. Unaware of what was going on that night, I had posted the following on Twitter:

This wasn’t the first time I have had problems connecting to the web wirelessly through AT&T, but clearly it was more than just typical AT&T issues that ended up causing it this time. In general, I believe the adoption of web enabled mobile devices is outpacing even the fairly aggressive growth in mobile data capacity. Combine that with both a spike in demand and many unresponsive news sites, and the results were no doubt frustrating for many others as well.

The other big traffic spike happened this past Tuesday.

Many sites decided to set up live streams of the memorial service held for Michael Jackson. Though it didn’t go off completely trouble free, Akamai, the leading video distribution/streaming provider on the web, ended up serving about 20 million live video streams during that time frame. That is nearly 10 times the number of streams they typically handle - by any measure a huge spike. While this number is way short of the 100 Million+ viewers that watch live events like the Superbowl, it still represents around 3 times the audience that watches a typical top rated TV show every week. This was impressive.

Unlike the bit starved mobile web, the wired web didn’t seem to have an issue with overall bandwidth or routing. There were no reports of general slowdowns or serious bottlenecks occurring because of this event, which is great news. What didn’t seem to scale up as well were individual sites. Some of those issues could probably be addressed with a more aggressive adoption of cloud based site deployments. If designed correctly, cloud-based deployments could help these types of sites scale capacity dynamically to better handle unexpected spikes in demand. In effect, this is exactly what all of the major news organizations did by using Akamai to deliver their video, an aspect of their delivery that seemed to work pretty well. They produced and packaged the content itself and then leveraged Akamai’s shared global infrastructure to handle delivery - something they would never be able to do well on their own.

We need to start thinking differently about how to build out the web going forward, especially around the optimal use of shared vs proprietary resources. I also think we are probably getting to the point where we should look more closely at what role fundamental network management technologies like QOS, packet prioritization, deep packet inspection and traffic shaping should play on the internet, and how we can make sure they aren’t abused. This is an issue I am somewhat torn about. I am a big proponent of network neutrality, but recognize the very real negatives of an ‘all packets are equal’ approach to managing traffic. Though it tends to be an issue that evokes passion from all sides, we will need to have a rational, dispassionate discussion about it if we are serious about making the web into a truly global media backbone - something it has the potential to become.

Events like these remind us that we still have a lot more to do.

Last week, Opera Software announced the result of that effort - a browser based collaboration platform called Opera Unite. Here is the video they put together to introduce their new offering:

I’m really torn about Unite. While I’m a big proponent of seeing choice, capability, and control pushed out to the edge of the web, I’m just not sure how well Unite will be able to deliver on this promise in practical terms.

First, there are three big marketplace trends that are riding against this.

• The growing acceptance of cloud based services. Web email is the best example of this. Almost everyone I know is comfortable with having a web based email address. It’s not that they don’t understand the level of personal information that gets communicated via email, they simply find it to be the most efficient way for them to integrate what they see as a “must have” capability. But even beyond email, the continued growth of services like Facebook and LinkedIn speaks to the markets’ acceptance of using intermediary service providers to deliver a broad range of socially anchored functionality. This is not a trend I see reversing any time soon, so selling against it may fail to resonate with most people.
• Mobile computing has become mainstream. While there may be “always on” broadband Internet connectivity running to many homes globally, there is typically not an equivalent ‘always on’ computer sitting behind it. For most people, the only device they keep on all the time is an iPhone or Blackberry. And with a growing consumer preference for laptops over desktops, there may not even be a computer in the house most times. If a model for sharing demands coordination - requiring people to be online at the same time - it creates a barrier to adoption. This will be a big challenge for Unite to get past.
• Search is the dominant way to navigate the web. Most people today struggle with the volume of content available to them that they need to deal with. Web search is one of the key tools they use to discover, sort through, and manage it. And while search engines do a pretty good job crawling the public web, they have no way of crawling the ‘deep web’ - the private servers that live behind firewalls or have passwords. And this presents a unique problem for Unite users. While Unite does allow for sharing content at a ‘public’ level, one of it’s key value propositions is that it also lets people control access to more specific content they may have. The implication here is that these “access controlled” items won’t be discoverable via the normal search tools most people use - even if they have the rights to access them. And given the way things work in practice, if someone can’t find something on Google, to them it doesn’t exist.

These trends tap in to the way people work and interact with technology. Overcoming them will require Unite to deliver something so compelling and unique that people would be willing to go out of their way to adopt it. Any hope for that would most likely to come in the form of applications people develop on the Unite platform.

And that is a tough position for Opera to be in.

On top of that, there are also a few significant technical issues that can weigh on adoption of an offering like Unite. The two biggest ones I see are in architecture and security:

• There is still a middleman. From what I can tell, Unite isn’t a hubless P2P service architecture. While files are not stored on a centralized server, Opera still controls all of the routing in the service. The only way anyone can connect to a peer is via a subdomain that needs to be resolved through Opera’s local DNS servers. This means that governmental agencies that wish to limit free speech and access to peers will only need to subvert access to a single domain to shut the network down.
• Security is light weight. Running a web server on a computer creates a much broader surface area for attack. Unlike sharing that takes place in a cloud computing model, outside people will be connecting directly to your computer and reading files directly off your hard drive. Given the vulnerabilities that have been found to exist even in more mature web connected software, I would not be comfortable placing a new release like this on any system that contained confidential files, passwords, etc. It would be terrible to have your hard drive wiped. It would be even worse to have your identity stolen. It’s unfortunate, but this is the kind of world we live in.

With all of this said, I really do like the concept being promoted by Opera Unite. True edge based connectivity has the potential to change the nature of many things we do on the web. Creating a common platform for social applications is also a compelling concept. Unfortunately, it think these ambitious goals are simply too big for any single company to take on alone.

For Unite to be successful on a broad level, I think that Opera will need to make it open source, and let the market work through the myriad issues that would have otherwise conspired to thwart a single vendor approach. Alternatively, they could package it as an internal corporate collaboration solution, and develop a more conventional business model around selling it.

Unfortunately, I don’t think Opera Software isn’t planing on doing either of these things. And while I would love to see a positive outcome for Unite, I just don’t see success coming from the path they are on.

I’m excited to dig in to it in depth, and once I do, I’ll post in detail on my experience with it. Before that, I want to share some photos of my unboxing of it:

I’m already impressed by the 3.0 software update which I’ve had for a couple of days now, and I’m really looking forward to the speed increase that comes with the new hardware.

For those also getting your new iPhones today - enjoy!