Why does everything suck?

Tech industry silence is deafening on #BlackInAmerica

2011-11-16T13:43:00.000-05:00

Last sunday night Black In America 4, the documentary that chronicled the summer that I and seven other black entrepreneurs spent in Silicon Valley, aired. (note it will re air Sat Nov 19th at 8pm) The aftermath has been, in some parts exciting. I have been incredibly busy doing panels and interviews. #BlackInAmerica was even a trending topic on Twitter on Sunday evening. In some sense it felt like lots of people were paying attention.

This is important to me not because I am in the documentary but because the lack of significant African-American presence in the tech economy is, I believe, critically important. In fact, If we don’t fix it, its going to accelerate an already dangerous level of wealth inequality in the country.

As I said in the documentary, not fixing this problem ultimately leads to a permanent underclass. And if you think Occupy Wall Street is a troubling signal regarding dissatisfaction around wealth distribution, you ain’t seen nothing yet. I fear the growing wealth disparity, particularly along racial and ethnic lines, will be catalyst for significant civil unrest.

If we are going to change course, in my view, the most valuable potential outcome of the documentary would be a willingness to more openly discuss the issue of race in technology. And since Twitter is a great proxy for engagement on any issue, that’s where I looked. I was hoping that given the heavy discussion in the tech blogosphere and press that the issue had finally broken into the mainstream.

But the Twitter stream said something else. Initially my sense was purely anecdotal, but I saw none of the tech industry “players” participating in the conversation.

So at my company, Kloudco, we decided to do some quick analytics. We pulled down all 150,000 #BlackInAmerica tweets between 9am est, the morning of the Black In America 4 airing, and 9am the next day. Then we cross referenced that list with industry mega-pundit Robert Scoble’s important tech people lists. These include his Twitter lists for press, VCs, and others.

Unfortunately, the results were just as I feared.

Across all of Scoble’s lists, there were only three participants in the discussion: @lekanB, @rachelsklar, and @venturebeat. The tech industry either wasn’t watching, was totally unengaged or worse, uninterested.

For whatever the reasons the tech industry is silent.

The CNN American Morning Interview

2011-11-14T14:14:00.001-05:00

The Soledad O'Brien Interview

2011-11-12T21:43:00.000-05:00

Soledad O'Brien interviewing me about the meritocracy of Silicon Valley.

The death and rebirth of useful interface affordances

2011-11-10T10:43:00.001-05:00

Mainstream interface design, as a discipline, was born during the era of the early Macintosh circa 1984, and died during the early era of the Web. In some quarters, Flash helped bring a bit of real interface design back, but now with HTML5 and iOS and Android I think people are starting again to really focus on and understand what real interface is about again.

Let me explain.

For those of you not steeped in the language of design, an “affordance” is a characteristic of an interface element that leads one, through its nature, to understand what to do. This is a little more subtle than it seems. For example a button in a user interface has a “perceived affordance” in that when we see the button we know that we can click on it and something will happen.

During the early days of the graphical user interface, we had a beautiful collection of new affordances provided by the operating system. Windows that had draggable headers, buttons that were clickable and did something, dialog boxes that could be modal, meaning that they had to be dismissed before further action could be taken, etc.

This allowed for lots of innovation. But because we also, as developers, had access to low level graphical primitives, we could make new interface objects that had new perceivable affordances. For example, in the first drawing programs I ever saw, LisaDraw and MacDraw back in 1983 and 1984, clicking on an object caused “handles” to appear around the sides of the object. These handles told you that the object was selected, but they also made it clear that the given object could be resized. This was an incredibly intuitive and obvious affordance. We, as software developers, had the capacity to create new objects with new affordances like this that could be incredibly powerful.

Make no mistake, this could be abused, and was, by software developers that had more programming chops than taste, but the best and most important software during these golden years of graphical interfaces always created new interface objects and introduced new perceivable affordances.

This was important not for the sheer act of adding new interface widgets to the built-in palette of operating system widgets, but because it allowed us to do something which for the most part was, in the ensuing years, almost entirely lost from software design. It was the act of creating software that allowed users to understand the data model of the application and to interact with it in an intuitive way.

In an ideal world you want an application to expose its nature, its data model, to you through its objects. This means that without having someone describe what it is you need to do or provide menus of options, that you understand what to do by the nature of how the data model is exposed.

As a counter example, coming back to MacDraw, imagine if drawing programs required you to create a new web page for each object you wanted to add to your canvas. A “new” button would allow you to create a new circle or square or line, and then a new page would come up that would ask you to enter the coordinates of the object. Then imagine when you wanted to see your creation, you would press the “render” button and your canvas would be rendered on the screen in some “under glass” manner that would not allow any direct manipulation.

In a drawing program, the data model is a list of objects with types such as line, circle and square, where each such drawing has characteristics such as color and dimensions. So there is no reason the web page model could not be used to represent the data model.

Except that it would suck.

The point is that good interface objects and metaphors and perceivable affordances make software vastly more useable.

But what happened is as the web browser ascended to the preferred software platform is that software developers lost their palette. Not only did web browsers not have the ability to express rich interfaces in the way that applications did, but a whole generation of user interface designers for the web have no idea about most of these subjects, or if they do, it is as some long lost art, and not a part of their actual toolbox.

Interface and interaction designers today tend to think in terms of pages and flow where the user is a mouse that must be guided through a maze. This is fine for certain categories of applications, for example content management. Web browsers were intended for display of text anyway so the web browser never proved to be an impediment for that class of application. No new affordances are needed, people just need to be guided to their content. 

But as I see it, many application categories could benefit from a bit more creativity. A good interface designer is someone that can think without the constraints of a limited palette. This is, more often than not, a programmer (these days this includes CSS3), because a programmer is much more likely to understand what is possible that may never have been done, and how to make it happen.

This whole subject came up because, in the last year I have had a variety of people lecture me about user interface as if I somehow “didn’t get it.” As we have been iterating Kloudco I have been working inside out, and art has really not, for most of that time, been a focus. I always listen politely, but most of these folks were barely even born when the first Mac’s came out and we were trying to explore what the real potential of man/machine interface and design really was. So such lectures have never sat well with me, but I haven’t been able to put my finger on what the deeper problem was. I finally realized that that the majority of newly minted design professionals don’t realize that they are being asked to create Rembrand style art with house painting rollers. (Do they even require reading Don Norman?- serious question).

The primary design palette is a series of pretty screens (or pages) that walk you through choices. But this just doesn’t work (or isn’t best) for apps that aspire to the level of problem solving of the early days of the graphical interface.

To be clear, there are lots of websites for which a limited set of objects is totally fine. You really don’t need anything beyond links and text to create a e-commerce site. But when we think about applications that allow us to more deeply understand our data models and interact with those data models in an intuitive way, we need the flexibility of a canvas with which we can cook up fresh new interface widgets and affordances that speak to the user without yelling at her. Applications like drawing, or calendars, or text editing, or collaboration or a myriad of potential applications that don’t fit the web page model demand this type of fresh thinking.

Another area where design options can and often should be more constrained is with smart phones. Most smart phones apps provide sequences of menus to navigate the data and the command sets (our mouse in the maze). This is necessary because not only are they on-the-go devices, but smartphones have limited screen real estate and input resolution. So creating a drag-and-drop calendar or drawing program might not be a very good idea on a phone. But if you have the input resolution and screen real estate of a laptop or a tablet, I would much rather drag an appointment to change its date than to click on the appointment, then click on the date and flick a roller.

What is exciting to see now is that HTML5 is going mainstream and is an acceptable platform for making web applications, so we are back to having rich tools with which to build. Still, the Mac Toolbox from the 80’s is vastly more powerful (though admittedly not easier) than today’s web browser. But Apple’s iOS and Google’s Android have an almost totally unlimited design palette. And while the potential for interfaces on phones is more limited because of the i/o issues, on tablets I see a vast potential to create intuitive and yet powerful experiences. One early example of this is Apple’s iPad GarageBand.

So the point of all this is that I would really like to see application designers today looking back at the history of the art of interface design. The golden era was only from 1984 to 1994. There are things that were pioneered that are as relevant today as they were then. With the right design, it is not necessary to sacrifice all power in an attempt to achieve simplicity. New affordances can make hard things easy. With the new tools available my hope is that we can win back some of the ground lost in the last 15 years of the web revolution and its unnecessarily dumbed down interfaces.

Arrington's not a racist (who's said that anyway?)... he's just being dishonest

2011-11-03T09:37:00.000-04:00

In his last two blog posts on the subject of the CNN Black In America documentary airing next Sunday, November 13th, Michael Arrington has been running around like a wounded dove claiming people are calling him racist.

Let's get something straight. No one credible or substantive has said that.

The fact that Mike can't discern the complex and important arguments about this from "people are calling me a racist" is incredible. The other thing he is doing is accusing Soledad O'Brien and CNN of sandbagging and tricking him and accusing them of starting a "race war".

From where I sit, asking him about the black entrepreneurs he knows is nothing of the sort. It's not a crazy question, it's not unfair, and it's certainly not a trigger for or indicia of a race war. In my view, Mike shows his insensitivity to the issue but certainly not ill intent, with the on-camera answers we've seen in the clips. There is a big difference between having a lack of understanding or awareness of the issue and of being a racist.

But what Arrington is doing now is deflecting a hugely important issue and discussion by trying to generate sympathy based on non-existant racism accusations. He is diminishing and minimizing the life experiences of all of us who are arguing with him who, to be honest, have far more experience with this issue than he does (i.e. apparently almost none).

For example one of my housemates in the documentary, and the co-organizer of the NewMe accelerator, Wayne Sutton, was stopped this summer by the Mountain View police at night and checked for warrants for doing nothing more than walking down the street and being black. The police's after-the-fact excuse for the stop was "they didn't recognize him" and it was a "voluntary" stop. For those of you who may not realize, this is *very* common. I've been stopped three times by the police for just walking around.

But the biggest problem I have right now with Arrington is, to bend an old political cliché for my own purposes, the coverup is far worse than the crime. What Mike is now spewing is really bad because it shows him to be either purposefully or "in the fog of war" dishonest. I think the "they are calling me racist" defense is intellectually dishonest, but some of the specifics he uses to defend himself are flat out lies.

In his latest post, Mike characterizes my last post by saying I am criticizing him because his coverage of NewMe or African-American entrepreneurs was not enough.

Specifically what Mike's post says is:

While it’s easy to look around Silicon Valley and see very few (non Asian because they don’t count!) minorities and then conclude “you’re a bunch of racists,” I don’t think that’s productive. What I do think is productive is to get more minorities, and women, and everyone, focusing on math and science and computers in school, as early as possible.

Once they’re here they are welcomed with open arms.

The top ten, or so, reasons I’m a racist

Unless their ideas suck. And even if they do suck a little, at TechCrunch we’d write about it anyway to give exposure to these entrepreneurs. That’s another source of endless criticism.

Or the coverage wasn’t good enough.

Or that putting people on stage who didn’t strictly deserve it is racist because it makes people think that they’re only on stage because of their race.

But either way, unless we cover more minorities, we’re racist.

First, the link in the block quote above is to my last piece. And the implication (though I admit it is murky) is that my post accuses him of racism because he doesn't cover enough African-American entrepreneurs.

The part that is *very* clear is his representation that my piece accuses him of not sufficiently covering African-American entrepreneurs.

The best defense of this comes from Natrius on Hacker News who said:

Hank said nothing of the sort. He said there is no proof that Arrington goes out of his way to cover black founders as he had claimed. Hank didn't say Arrington should go out of his way, nor did he call Arrington a racist. Hank just said that Arrington's claim was incorrect, and from where I'm sitting, he's right.

And just to back up what Natrius said, here's a big hunk of the text from my last post where I specifically say the *opposite* of how Arrington characterizes my post:

Either way, Mike was within his rights to decide what he would or would not cover, or how he would cover it, and at what depth. He does not owe any person of color or female entrepreneur or anyone else anything. But to, after the fact, say that he bent over backwards to cover African American entrepreneurs is laughable.

Does this make Mike a bad guy? No. I presume in actuality, he wasn't even involved in the editorial process. So I won't blame him for the uncharacteristic lack of depth of demo day coverage. But I sure as hell am not going to let him claim credit for somehow being some kind of bend-over-backwards-to-cover-African-American-entrepreneurs kind of guy. Let's get real.

So to conclude, no one is accusing Arrington of being a racist. But it's clear he is (or at least his writing reflects him to be) incredibly insensitive to issues of race and privilege. No one imagines him sitting around spewing racial epithets or purposefully discriminating, or even thinking bad racial thoughts, but that is not a very high bar.

Mike, it would be great if you'd put an end to this pity party and join us in real discussion as you suggest you would like to. Most of us engaged in this debate are pretty reasonable people and if you really do want to "do something" as you suggest, now is a great time to work on it with us. And yes I've heard you want to work with will.i.am on the issue, so you can bring him too.

Arrington, Race, and Silicon Valley

2011-10-28T08:15:00.001-04:00

I spent this summer in Silicon Valley as part of the NewMe Accelerator. NewMe is the first Accelerator program focused on African-Americans. I had an absolute blast in California connecting to incredible mentors like Mitch Kapor, Ben Horowitz, Vivek Wadhwa, and, in my own way, both absorbing insight from, and, as the old man of the crew, mentoring the other members of the program.

As I have over the years spent much time in Silicon Valley, starting from my days writing Mac software in the 90s, it was great to rekindle some old relationships. Participating in the program moved my company, Kloudco, forward in some amazing ways, in large part from the brilliant insight of the other participants. There’s much more I want to say about my NewMe summer, but that will have to wait for a later blog post.

Our experiences in NewMe were captured in a new documentary that is part of Soledad O’Brien’s Black In America series on CNN. The documentary will air November 13th and on Wednesday, I and a few hundred other people saw an advance screening. You can check out some clips here.

One of the most striking things about the evening was the aftermath reaction to some of the comments that Mike Arrington, founder and former editor of TechCrunch, made on camera.

A Twitter fight erupted between Arrington and others such as Vivek Wadhwa, who is also in the documentary. In the calm of the day after, I want to share my thoughts.

Arrington Says: Silicon Valley Is a meritocracy

Mike said a few very clear things about his view of the state of diversity in Silicon Valley.

its true that there are very few African-Americans in Silicon Valley
despite this, Silicon Valley is a pure meritocracy
you become successful because you have a “big brain”

First, let me say, I think Mike truly believes everything that he has said about the tech world being a meritocracy. Lots of people believe that.

But I do not believe Silicon valley is a meritocracy. I would more properly say that tech *markets* are a meritocracy. There are very few businesses where a single individual in her bedroom can create a piece of software that can potentially touch millions of people without any additional capital. No matter how talented you are, if you want to open a hot new restaurant or a shoe factory, you need lots of money before you start. Not necessarily so with software.

Consumers and businesses, for the most part, don’t care what the ethnicity of their software or Internet service vendors are. Users want solutions. And so if an entrepreneur can get a great product completed cheaply, in many cases they can compete on totally even footing. Even if they ultimately need capital, explosive initial success knocks down all known barriers.

But the market *makers* operate in a world that is not particularly even-handed. The market makers are the folks that help new young companies and entrepreneurs by providing insight, mentoring, capital, and relationships. And this part of the tech world is driven by all the same types of biases that exist in the non-tech world. And it is *much* harder for even the most talented African Americans in the tech world to gain access to influential, insightful, connected mentors, let alone investors.

People, for the most part, want to work with people that are “like them” or that fit a pattern that appeals to them. There is an actual term for this among tech investors called “pattern matching”. It's the idea that, without objective facts, one can decide whether someone is likely to be successful based on indirect criteria. In other words, when they see a particular pattern of “personhood” they are excited.

And these patterns are discussed openly in the tech industry around issues like age. Since it is only moderately politically incorrect to suggest that younger entrepreneurs are “better”, it is done all the time. The best example of this might be Mike Moritz from Sequoia Capital, perhaps the most influential of all venture funds, admitting on a TechCrunch Disrupt stage that they have a strong bias towards very young entrepreneurs.

But if you believe that age is the only criteria that VCs use for pattern matching I wanna smoke some of what you’ve got.

To be clear, I am not saying any VC says at a partner meeting, “you know I really like this company’s product but did you notice he’s a negro?”

Never happens.

But I firmly believe market makers, both investors and the people who help you get ready to approach them, seek out entrepreneurs who appeal to them on some less than objective, visceral level, who feel “comfortable” to them. They don’t *need* to actively filter out undesirable profiles. They just focus on what *does* appeal to them. They focus on the “patterns” they find appealing and I am confident that not only is age a part of many investors' ideal patterns, but so are perhaps un-recognized criteria like race, gender, cultural affinity, etc. And on some level this should not be shocking as it reflects socialization that all of us must work hard and consciously not to act on.

Is this (racist/sexist/agist/_____ist)? Well in this context, using incendiary labels is only likely to make people more defensive. The bigger question is, is it a problem? Absolutely.

Is it possible to overcome these additional barriers? I have. But it is only by a sheer persistence and focus that, few other people, white, black, or otherwise, have. While I would never suggest that I am smarter than anyone else, my Arnold-Schwartenzegger-in-Terminator like determination has made my successes possible. Yes, I have definitely had help and support, but compared to some, not so much.

In fact some people get far more support than others. For example, I’m not going to name any names, but when a top tier VC writes a five million dollar check to a 19 year-old with a barely-beyond-napkin-stage *idea*, no customers and a fragile technology because they “present well” then clearly something else is at work. I am not saying that this exact scenario is common, but it does happen. And since everything is on a spectrum and I can guarantee there are no African-American, or for that matter Latino or female entrepreneurs that contribute such insane data points to that spectrum, it is troubling.

So the bottom line is, if the level of determination that I have was required from everyone on some kind of moderately equal basis, it would indeed be a level playing field — a meritocracy. But it's not.

Arrington says: I went out of my way to cover African-American entrepreneurs at TechCrunch

The other striking comment Mike made in the documentary was that he went out of its way to make sure African-American’s got covered in TechCrunch.

Bull.

And I say that with all due respect, because, again, I suspect he believes that it's true. But I just don’t buy it.

The NewMe organizers tried repeatedly to reach TechCrunch regarding covering the NewMe demo day. They never got a response. While this was going on, Mike was discussing, with CNN producers, being interviewed by Soledad O’Brien for the documentary. At some point, Mike agreed to do the documentary, and after he had shot his interview, told the NewMe organizers (after being approached at a party) that he would be sure to send someone to demo day. Before that time there had been no acknowledgement from TechCrunch that NewMe even existed. No emails responded to, nothing. Mike had only responded to CNN.

TechCrunch writer Alexia Tsotsis did ultimately show up to the demo day. Her article was complimentary about the idea of NewMe and she said, via Twitter, that it was the best run demo day she had seen. But she only wrote a sentence or two about each startup. She didn’t ask anyone for a live demo, or present anything of substance about any of the companies. In essence, she focused on the form of the demo day and the purpose, but not the companies.

Now, if this was the standard for how TechCrunch covers demo days, that would be fine. But I read TechCrunch voraciously, and I don’t believe I have ever seen such thin coverage of any demo day that did get covered. YCombinator has always gotten a full story on each company, as has (I believe) TechStars. Of course there are many accelerators and demo days, and I can’t say that TechCrunch covers every one in depth. But it was striking that they didn't do a substantive piece on even *one* company given that they did cover the event. (Note: several months later they did cover a company, but not in the demo day context as usually happens).

So my point is this. Though Mike was being interviewed by CNN about race in Silicon Valley in the context of the NewMe accelerator, he did not deem it appropriate to make sure his NewMe coverage was at least roughly on par with other accelerator demo day coverage on TechCrunch. Awareness that he was going to be on national television talking about fairness and balance and meritocracy and race in the Valley did not sway him. Perhaps he didn’t want to be seen as giving favor to NewMe since he was going to be in the documentary. Perhaps.

Either way, Mike was within his rights to decide what he would or would not cover, or how he would cover it, and at what depth. He does not owe any person of color or female entrepreneur or anyone else anything. But to, after the fact, say that he bent over backwards to cover African American entrepreneurs is laughable.

Does this make Mike a bad guy? No. I presume in actuality, he wasn't even involved in the editorial process. So I won't blame him for the uncharacteristic lack of depth of demo day coverage. But I sure as hell am not going to let him claim credit for somehow being some kind of bend-over-backwards-to-cover-African-American-entrepreneurs kind of guy. Let's get real.

The economy: programming is the problem, and the solution

2011-09-23T21:31:00.002-04:00

Tomorrow I am going to be on a panel called the Newark Leadership Roundtable, talking about the solution to the dire economic situation in inner cities in general, and in Newark in particular.

I have lots of thoughts and I am not sure with such a large and distinguished panel that I will have an opportunity to say everything I think, so I figured, both to prepare for the panel and to air my ideas out more fully I'd blog them here.

The situation in inner cities is a complex one and there are a variety of factors that drive the current context. But the truth is I don’t think any of the issues that are driving the inhospitable economic environment are local. Inner cities in general, and Newark in particular may have specific local problems but the primary issues are national if not global.

Broadly speaking, the problem, and the solution to the problem is technology. Back in 2009 I wrote a piece called “The problem with the economy: you aren’t needed anymore”. The basic thesis of that piece was that technology is reducing the number of people required to provide our planets most basic of needs. Be it energy production, food, transportation, retailing, communications, or a myriad of other industries, software is, at an astonishing pace, allowing us to operate at greater and greater levels of efficiency. In this context greater efficiency means employing fewer people.

This means that fewer and fewer people can control larger and larger chunks of the economy. This is driving a massive consolidation of economic power and wealth.

My piece ran in Business Insider where it was roundly criticized the editor-in-chief Henry Blodget, and many other commenters. But since that time others have written similar pieces. For example Douglass Rushkoff wrote a piece in cnn.com called "Are Jobs Obsolete". And Mark Andreessen, famed investor, and one of the creators of the first commercial web browser, recently wrote an article that appeared in the Wall Street Journal called “Why Software is Eating The World”, where he explains that essentially every business category is being transformed into a software company.

The implications of this are profound because when software does things, then people are not doing those things. As we become a planet where all of us are not needed to feed and clothe and shelter each other, we become a planet where wealth is much less evenly distributed. We become a planet where certain people have critical contributions to make and some just don’t. And so the question becomes how do we create a politically palatable framework for caring for those people who are just not needed.

This is an important question because it is essentially where we are now. There is massive unemployment in america, well north of 20%, and much higher in minority communities. The unemployed are not, for the most part, dying of starvation. They are just living very poor unhealthy unhappy lives. The employed majority’s survival is not dependent on the unemployed minority. Many employed folk consider the unemployed lazy and in circumstances of their own making.

The policical right considers this economic underclass useless and politically expendable. And that may be good cynical 2012 politics. But it will not be good politics as the number of unemployed grows and wealth continues to consolidate. Technology guarantees that the trend will continue. In fact technology *demands* that the trend continues. It will likely not be a straight line curve, we will have ups and downs in employment and GDP. But the bottom line is existing economic theory does not apply to a world that has infinite efficiency, and that is where we are headed.

So this is the big picture problem. It is not a local problem. It is not a problem for Newark, or Paterson, or Bedford Stuyvesant, or Harlem. It is a problem for the planet. We can’t do much to change the trajectory of the planet, but we must understand the trends if we are going to survive them.

The first thing to understand is that it is critical that a broader spectrum of society become creators of technology instead of consumers of it. My wife is a professor at Montclair State University where she teaches teachers in the school of Education, and researches educational strategies and social justice issues. And she tells me that elementary school curriculums, to the extent that they include technology, are focused on how to integrate it into the learning experience. For example, a big part of the discussion is about how we use facebook, or blogs, or twitter in the classroom. This is all good, but its like teaching kids to be patients rather than doctors. We are teaching consumption rather than production.

The point is that in order to have any economic self sufficiency, poor disenfranchised communities, must create their own opportunities, their own businesses, their own centers of power. And a big part of this must come from learning how to create software because software already drives most economic activity and it will ultimately drive almost all of it.

And so my fundamental point is that all kids, but particularly inner city minority kids that may not have significant opportunities, can learn to create those opportunities by learning to program. Creating software in twenty years will be like reading is today. We must engage. There is no reason that third grade kids are not learning the basics of programming. More importantly, learning how to program effects learning and reasoning in every other academic pursuit. Our educational system today is primarily focused on how much knowledge we can stuff in a kid’s head when what kids really need is to learn how to think, how to reason, how to gather information and how to develop answers, arguments, and strategy.

I believe every single school day, starting in third grade and going through at least tenth grade, that every student should have a period dedicated to programming. It is critical that we start this young before the educational system succeeds in teaching our kids that they are dumb and can't learn which is what it does today. And I am confident that programming is as learnable at an early age as is basic math or reading.

I realize that given that there are so few teachers that know how to program that there would be insecurity in schools giving students skills that most of their teachers don’t have. I also realize there are no curriculums focused on this. But this is why we must start now. There is nothing more important. Because as economic power is consolidated amongst those that control the software, not following this course will cement the groups that are currently at the bottom of the socio-economic structure into that position permanently.

To be clear, teaching poor kids in inner cities to program will not fix the bigger picture problem, but it can at least more fairly and more broadly distributed the world's wealth.

Twitter eats its young

2011-02-19T07:36:00.004-05:00

Yesterday Twitter committed a hostile act against both its customers and one of its largest Twitter client developers, UberMedia. They cut off all of UberMedia's clients from the Twitter platform. Basically Twitter claims they did it because the UberMedia clients violated certain rules. These are rules that apparently UberMedia has been violating for almost a year, or perhaps longer.

What I find most interesting is trying to analyze what the hell Twitter was thinking. In an excellent blog post, Mark Suster lays out the basic fact that Twitter could have been perceived as the good guys by just putting out a 72 hour warning to both UberMedia, and users of UberMedia Twitter clients, that they were going to be cut off if things's weren't fixed. This would give UberMedia a chance to fix things, and it would give customers a chance not to wake up blindsided and Twitterless. The Twitter team is smart, so it's hard to imagine they didn't consider this as an option, which in an of itself gives me pause.

The apparent issues involved trademark, and a potential privacy leak around direct messages. I'm sure Twitter would argue that UberMedia was not responsive to previous requests or warnings and that they have a right, and in fact a responsibility, to defend their trademark and protect customer privacy.

But here's the thing. They have waited on these issues for a year, so no matter how egregious the failings of UberMedia are, a 72 hour warning wasn't going to hurt anyone.

In response to Suster's post, Dick Costlo, Twitter CEO tweeted "@msuster your post is a misinformed and contrary to what I'm certain you would counsel as an investor."

Costolo's response is, on its face, ridiculous. First, Suster was not misinformed because everything he commented on was in the public record except his opinions which are, of course, just opinions. Second, I can't imagine Fred Wilson or Bijan Sabet, both early Twitter investors and board members, suggesting to Dick Costolo that it would be in Twitter's interests to *surprise* millions of customers by turning off their Twitter client with no warning. I could be wrong because I am certainly not in their heads, but neither Fred nor Bijan strikes me as customer-hostile, take-no-prisoners, do-whatever-you-need-to-do regardless-of-how-it-impacts-customers type of people. I would be shocked to hear that this was done on behalf of the Twitter board of investors or that they would, if asked, have suggested this was a good idea.

What's obvious is that these actions are being taken because Twitter is threatened by UberMedia, and possibly pissed. UberMedia has been buying up Twitter clients like a hungry tourist binging at a Vegas buffet. They have just raised $17 million dollars, and its CEO Bill Gross essentially invented Internet advertising when he created Overture. So perhaps they *should* be threatened. Still once you operate as an ecosystem, you have a broader responsibility than just to maximize profit and kill all potential competition from your invited partners. Maximizing profit is obviously a critical goal, but screwing platform developers and customers for no ostensibly defensible reason in a clearly hostile way may not be good business (it's certainly not *decent* business), and it unquestionably gives people a reason to distrust you with what is now, like it or not, a shared asset.

In many ways this reminds me of this week's ridiculous Apple policy change requiring that Apple get 30% of platform subscription revenue of 3rd party developers. Twitter clearly felt they could get away with this and so they did. It was a very Machiavellian, and I think poorly thought out move. But this is not a decision I think they would have or could have made three years ago. Similarly, Apple would not have been trying to take 30% of subscription revenue when they launched their developer platform. They feel they can now because they think they have all the tigers by their tails.

Perhaps Dave McClure said it best when he commented (on twitter) "yo @dickc -- guess Twitter won't be following that Google "Don't Be Evil" thing, eh? ;). " The dickc there is Dick Costolo. And that does about sum it up. It is hard to be a platform vendor and not end up occasionally on the dark side. There are a lot of interests to balance. But *damn* does it have to be so knee jerk, and blatantly obvious. If not for the benefit of the platform, at least in the interest of not decimating heretofore well curated trust in you as the keeper of said platform.

Apple is actually asking for 100% of SaaS mobile revenue

2011-02-16T08:16:00.004-05:00

This whole app store thing has me worried. Not just about how Apple's policies affect content access, but regarding how they affect my business, the online software as a service (SaaS) business.

It appears to me that the new "give Apple 30% of revenue" policy will apply to software subscriptions just as well as it appears to content. Many companies that offer a web service provide a mobile client. They charge on the web, and customers can access their service via mobile if they wish. Its a free option.

But now it appears that that will not be acceptable to Apple. Because if you offer a subscription *anywhere* you have to offer it through in app purchasing as well. This seems crazy to me for a bunch of reasons.

As an example, it sounds to me like for SalesForce to continue to offer their service, they will need to give Apple 30% of their subscription revenue for all customers that want to access SalesForce via mobile. So if SalesForce charges $50 per seat per month (I don't really know what their prices are right now), Apple wants $15 per month. They want their 30% cut of *all* of the SalesForce revenue even though the mobile product only offers a fraction of the value that the customer is paying for.

Think about it this way. Would the customer really pay that $50 if all they got was the mobile product? Lets say that, on average, the mobile product is 30% of the value of the product. The other 70% is the web based product, access to customer support, backup services, etc. If this scenario is even roughly correct, then what Apple is doing is asking for 100% or near 100% (perhaps more or less) of the value of the mobile experience. This analysis applies to any multi-platform SaaS product.

Extracting 100% of the value of the mobile product is a total non-starter, and if Apple enforces their new policy against SaaS businesses, I think most of the multi-platform ones will leave because the economics just won't work. The thing is, because Apple has so capriciously changed their policy, affecting companies like Amazon, and companies like SalesForce that have made a significant investment iOS investment, one has to wonder whether building iPhone apps is safe even if they don't initially enforce this policy against them. Because they could. If you always have to worry whether the platform vendor is going to figure out a way to put you out of business or extort you, it just may not be worth the risk. Damn I feel like I am writing about the Gambinos!

I am particularly worried about this because my company is going to launch a new service in April and we are busily working on an iOS application and we can't really turn back now. The main reason we chose iPhone is because I have an iPhone and an iPad and because the iPad currently leads the tablet market. But I am now worried that we may have made the wrong choice and should have gone the Android way.

If this doesn't get cleared up or changed, there will still be plenty of games and and productivity tools for the iPhone, but new multi-platform SaaS application developers won't touch iOS with a 10 foot pole. And existing companies will pull their iPhone apps in droves.

Well one thing for sure, the Android team must be kvelling!

Removing features you thought you bought - Are there legal issues?

2011-02-15T10:19:00.001-05:00

One of the interesting things that is coming out of Apples new subscription program which I just wrote about here, is the idea that features that you thought you bought can go away. Because Apple is now requiring that subscription services pay Apple 30% of revenues, many of those services will be unable to operate on the iPhone platform.

The idea that features of web services could go away has always been a potential issue, thought it is rarely something that happens. But I am not aware of this ever happening with a physical piece of hardware you have purchased.

In Apple's recent announcement, they have made it clear that book services such as Amazon Kindle, and music subscription services such as Spotify, Rdio, Slacker and pandora, will, effectively, not be able to operate on the iPhone going forward, even for current users. So what that means is that you bought a device, potentially to use a specific set of features, and those features may actually *go away*.

This is a significant step in terms of how we think of our devices. This blending of software services with hardware, while it has many benefits, has some dangers that I had never considered. For civilians that don't spend all day thinking about such things, what are *they* to think when one of their favorite products or services ceases to work anymore. Particularly when they have another year and a half left on their AT&T contract.

The contract issue is interesting because the user is committed to the carrier, but neither the carrier nor Apple is committed to continue to offer the services that the customer believed they were getting when they bought the device. I wonder if there is not some substantive legal issue for people who believed their contracts guaranteed them access to certain services, only to be cut off. In a certain sense there are issues here which relate to the underlying concepts of net neutrality.

I am not sure what to make of all of this, but my sense is that there are some significant issues that the government may be interested in.

RIP Pandora, Kindle on the iPhone (and many other content services)

2011-02-15T09:35:00.003-05:00

Apple today announced that they now require subscription services to fork over 30% of revenue to them if they want to run on the iPhone.

This, I fear, for several types of content, is a bridge too far. The problem is pretty basic. Many types of content businesses dont have 30% of margin to play with. In books, Amazon doesn't have 30% margins to give to Apple and remain in business. In music, services like Pandora probably don't even have 5% of margin to play with. Given that Pandora has just released financials in preparation for going public, someone with a bit more time on their hands could go through their recent filing and figure out what the specific numbers are.

But Apple is not stupid, so they have certainly run the numbers. And so it seems that they have decided that they now have a platform that is so popular, that the services that can't afford to pay them 30% are not needed anymore. In fact, perhaps it might be more accurate to say, these services aren't *wanted* any more. Clearly Apple has iBooks, which they want to be the bookseller on the iPhone. And obviously, with the Lala purchase last year, and their primary position in the music ecosystem, Apple feels comfortable being the only music service on the platform.

On the other hand, services like News Corp.'s The Daily, and other newspapers and magazines *do* have margin to play with. And if Apple can help them to build their businesses (or save their businesses), the 30% cut might be well worth it.

The biggest issue I see here is Apple's total disregard for the consumers here, who have invested in the iOS platform *because* they believed they could use a broad array of content services like Amazon Kindle, Pandora, Slacker, etc. For those customers that have investments in non apple content, what are they to do? Would they have bought an iPhone or an iPad if they knew that Apple would attempt to become the single source for important categories of content. I know in my case there is no way I would have bought an iPhone if I couldn't use Kindle or Slacker.

Yes, Apple is trying to do right by its investors in its stock. But once again, Apple is showing total disregard for investors in its products.

Poor kids are dumb. There's nothing we can do about it

2011-01-18T07:22:00.001-05:00

I just came across this article by Joan Barkan in the left leaning Dissent Magazine, about reforms in our education system, and I was, quite frankly, appalled.

The article's core premise is that Bill and Melinda Gates and the Broad Foundation have a disproportionate impact on the education agenda. I don't really have much of an opinion on that. My wife is an education professor and researcher and so I do see the education research funding process up close. But while Bill and Melinda fund what I think are some good initiatives I would not suggest that I know enough to opine on the overall effectiveness or subjective "goodness" of their work.

But the core of the article seems to be that none of the reforms that they have tried to effect have worked. As I was reading the article, I expected her to lay out some strategies that she thought *would* work. She did not.

Her argument seems to be that education in America, relative to other countries, works fine for families in the top 25% of income, but then gets less good relative to other countries as we include the lower 75%. And she sees this as proof that in fact there is nothing wrong with our educational system.

here is the key quote:

To justify their campaign, ed reformers repeat, mantra-like, that U.S. students are trailing far behind their peers in other nations, that U.S. public schools are failing. The claims are specious. Two of the three major international tests—the Progress in International Reading Literacy Study and the Trends in International Math and Science Study—break down student scores according to the poverty rate in each school. The tests are given every five years. The most recent results (2006) showed the following: students in U.S. schools where the poverty rate was less than 10 percent ranked first in reading, first in science, and third in math. When the poverty rate was 10 percent to 25 percent, U.S. students still ranked first in reading and science. But as the poverty rate rose still higher, students ranked lower and lower. Twenty percent of all U.S. schools have poverty rates over 75 percent. The average ranking of American students reflects this. The problem is not public schools; it is poverty. And as dozens of studies have shown, the gap in cognitive, physical, and social development between children in poverty and middle-class children is set by age three.

Drilling students on sample questions for weeks before a state test will not improve their education. The truly excellent charter schools depend on foundation money and their prerogative to send low-performing students back to traditional public schools. They cannot be replicated to serve millions of low-income children. Yet the reform movement, led by Gates, Broad, and Walton, has convinced most Americans who have an opinion about education (including most liberals) that their agenda deserves support.

In other words, regarding public schools, there's nothing to see here. Move along. All this money is being wasted (or worse) and there's nothing that can be done. And in fact all of these education reforms are "messing up" what it appears she thinks are schools that are running as well as is possible, given their students.

I find her argument disgusting. It is a horrific defense of the status quo wrapped in a pandering argument designed to curry favor with teachers unions that hate the idea of reform in any way shape or form.

I am not here to argue that all of the reform ideas floating around are good ones. But the idea that no reform ideas or experiments or research can be good because these poor dumb kids are hopeless by age 3 is really a modern day perhaps more politically palatable form of eugenics.

That this piece appears in "Dissent" which frames itself as "a magazine of the left" on its about page is even more shocking. It is also, I think, a reflection of the fact that the historical alignment of the policial left with the poor and lower middle class has given way to, among other things, powerful union lobbies that have no alignment of interests with "the people", much less poor people of color that probably come out in smaller numbers at the polls and are, for the most part, unfortunately, politically invisible.

Goldman Sachs: is the fundamental concept flawed?

2011-01-06T10:59:00.000-05:00

Today in the New York Times, Simon Johnson writes a piece entitled "Why Are Taxpayers Subsidizing Facebook, and the Next Bubble?". The central thesis of the piece is whether it is appropriate for an institution like Goldman that is "too big to fail" and therefore effectively has a government guarantee that it will not default on its debt, be an equity investor in Facebook.

It is one thing to invest money on behalf of your clients. Thats the core of what an investment bank does. No problem there. But we heard today that the division of Goldman that does just that, passed on the $450 million dollar investment in Facebook. As a result, Goldman itself did the investment, putting the deal directly on the company's balance sheet.

Simon argues that this is problematic because the Goldman balance sheet is effectively guaranteed by the government, and because Goldman is an actual bank (as opposed to an investment bank) it gets to borrow from the fed freely at essentially no cost. So, in essence the government is backing what one must admit is a risky investment in a private internet company.

I fundamentally agree with Johnson's assessment, but it seems to me the problem is much bigger than this Facebook investment. One really interesting question is whether an investment bank can reasonably also be a bank holding company. Allowing a firm like Goldman to become a bank creates all sorts of problems and I suspect this is just the tip of the iceberg. Banks should operate in a fundamentally risk averse manner. Investment banks can't. It seems to me that not only is the current structure of Goldman a problem, but the *concept* of Goldman is a problem.

We cannot allow any institution to be both secured by the government, and in the business of taking large risks. Of course defining "large risk" is problematic in that no one would have thought mortgages could, in any context, be defined as high risk. But while there may be gray areas where valuable collateral turns out to be not so valuable, an investment in Facebook ain't gray. And obviously much of Goldman's other investment banking related endeavors couldn't be defined as low risk either.

My point here is that I think Goldman needs to be clearly free to fail. That means we need to regulate them in such a way that we can guarantee that failure *is* an acceptable option. Regulators suck (see Bernie Maddoff) but in this case the not so free market sucks even more.

What everyone isn't getting about Wikileaks

2010-12-03T14:00:00.002-05:00

The pro Wikileaks argument sounds an awful lot like the music piracy thing. "Information wants to be free." The corollary is, "We have a right to know everything our leaders are doing and saying."

That is total bullshit.

And what is so fascinating about this is that the same people who make these arguments (like the EFF) are the ones up in arms about our loss of privacy. The issues are two sides of the same coin.

All information does not want to be free. My private information deserves to remain private. And if I must share it with the government for some reason, that doesn't mean that I should lose my right to privacy. If I have an opinion, or a need, or a "situation" and I share it with an ambassador, or the state department, or my local congressman, that does not mean that I am OK with that information being broadcast over the Internet for all the world to see.

To be clear, I am sure that there are things that are in the Wikileaks trove that should be public. But there is a huge difference between providing curated information that the public has a right to have and a need to know (like the Pentagon Papers), and broadcasting absolutely everything with the ethos that if the government is involved then we as citizens all have a right to know.

The reality is that we are not the government. We (hopefully) elect our government, and they act on our behalf. One of the ideas behind government is that it should act on our behalf, but that it must also keep our individual confidences, and the confidences of other nation states with which we deal. This is a huge issue because to the extent that government cannot be trusted to keep anyone's confidence, a hugely important role of government no longer works.

For example, we cannot negotiate with other states if everything they say will immediately be broadcast. There will be no dialog and no opportunity for understanding. And because many of the governments on the other side of the table (think North Korea, China, etc.) do a much better job of keeping their secrets, they will just know that they cannot trust any discussion with the US to not go public. This has no effect other than to make us more isolated.

This is not just hypothetical. One of the things the Wikileaks trove reveals is how difficult it is to deal with a foreign government when you know nothing about the leader or that country. We have been reduced to guessing what we should do and what North Korean might do and why. To be clear this information void is not because of Wikileaks (in fact the knowledge of the void is because of Wikileaks), but the point is that lack of information creates very difficult problems. And because other states are much less likely to have such leaks the US will be the only state that can't be trusted to communicate with.

Further reducing the US information flow making us a less effective state operator is not a good thing. One of the things that is clear from Wikileaks is that while there are unquestionably a few bad acts, the vast majority of the leaks reflect a US government that is trying to keep the world from blowing up, employing, at times, extraordinary statesmanship. There is nothing shocking in the Wikileaks release which is why everyone is spending so much time talking about the context of the release and not the actual content. So far, the most interesting leaks are personality issues such as our views of Karzai, Putin and Berlusconi. These are leaks that do nothing to improve our relations with these countries, and they do not in any substantive way help our citizens understand the world in which we live.

The bottom line here is that privacy has its value, not just between members of the public, but between citizens and their governments, and between government officials. Everything does not need or deserve to be public. There is a line. And while I certainly would not elect myself to decide where that line is, it does exist. And to those that say such a line is unneeded or inappropriate, I call bullshit on that. Of course whether we can do anything about it is a different matter.

The age of creepy, and what to do about it.

2010-11-15T10:46:00.002-05:00

The Web is, all too often for my comfort, devolving into the creepy. This is a sense that has been "creeping" up on me for the last several months, but the word is becoming increasingly common amongst those who have a platform to comment on such things.

I was most recently struck by the issue this this morning when I re-tweeted a post by Dave Winer about how a new iPhone photo product called Path, had without his permission, read his address book so it could make recommendations about who he might want to connect with on Path.

To me, creepy.

Then I read Dare Obasanjo's piece today about location services where he describes the auto location broadcasting feature of the Loopt iPhone app as "creepy".

And of course there is Eric Schmidt describing Google's policy as "to get right up to the creepy line and not cross it." Others would suggest that Google has failed at discerning that line. Fast Company recently published a piece outlining many of Schmidt's recent pronouncements entitled The 7 Creepy Faux Pas of Eric Schmidt.

Of course, when we are talking about where the creepy line is, we probably should also mention Facebook.

But the thing is, in a world where more and more information is available about us, it seems inevitable that we are just at the beginning of defining what is and isn't creepy. My perception of where that line is, is, I suspect, different from a 20 year old college student, who may not see any line at all. Clearly many don't. On the other hand, most of the people complaining about this stuff are probably closer to my age than to that college kid.

Amongst the populace that matters, it seems to me that what I think of as creepy just isn't that creepy to enough people to matter. In fact, I think that those of us who are concerned about the "line" are destined to die off or become numbed to the phenomenon and accepting of it. Fighting the bit by bit erasure of the "creepy line" is, I think about akin to fighting music piracy. Whatever moral arguments we may make just don't matter when the tide is going the other direction. Refusenicks will be replaced by kids who would be totally comfortable being the lead in the Truman Show.

And so my answer to the question of "what to do about it" is, nothing. Get with the program. You will be assimilated.

Apple Blinks. Flash Tools Now Allowed

2010-09-09T09:08:00.003-04:00

As many of you know, I have been fairly agressive in complaining about Apple developer guidelines, and specifically section 3.3.1 of their developer agreement which prevents developers from using any interpreted code, or, essentially, any compiler not created by Apple.

Apple instituted the 3.3.1 clause in order to lock out Adobe from offering a Flash based development platform for the iPhone. Adobe was offering a development tool that allowed flash developers to compile their programs into iPhone compatible binaries. There was no good reason for Apple to block this tool other than fear and loathing of Adobe.

Apple has conceded. They have essentially rescinded all of the madness associated with 3.3.1. This is an amazing turn of events. It is exceedingly rare for Apple to capitulate, and it can only mean that pressure from the popularity of the far more open Android has taken its toll. It looks to me like the accelerating rate of Android app development and user adoption has apple concerned, as well they should be.

To be clear, I am a Mac, iPhone, and iPad user. I love their products. But I have hated many of their policies and business practices. This change does not lead me to believe that Apple has changed its philosophy, but I am ecstatic that competition from Android is forcing Apple to make the iPhone a more competitive, more open, and therefore better platform for users and developers.

Judge Says TechCrunch Case vs. JooJoo Tablet Likely Has Merit

2010-08-29T20:50:00.003-04:00

I Just read on Daring Fireball that the judge has denied TechCrunch's claim for preliminary injunction against their former tablet partners and makers of JooJoo, Fusion Garage.

Gruber claims that "Mike Arrington gets smacked around in the first round of his lawsuit over the JooJoo/CrunchPad. In short: TechCrunch didn’t get much in writing regarding their “partnership” with Fusion Garage to develop the product, and, well, they should have."

He then goes on too snarkily add, "Curiously, I’ve seen no coverage of this decision on TechCrunch."

Because I am admittedly fascinated with the case, I actually took the time to read the judge's decision. Perhaps Gruber's snark would have been appropriate if his "in short" was anywhere within several miles of the truth regarding the decision. Honestly, the whole JooJoo thing seems like such distant and irrelevant news in light of the iPad, and soon Android and Chrome OS tablets. But its fascinating to me that Gruber feels compelled to totally misrepresent what the judge said.

The real "in short" of the judge's decision is this. The judge said that the preliminary injunction request was denied, essentially because it was not specific enough about how much profit there would be in the device, if any at all. He also said that there was no evidence that if TechCrunch prevailed in the final case that the injunction was necessary to insure that TechCrunch could receive a recovery. There were additional and more detailed legal conclusions, but they were all narrow and only relevant to immediate injunctive relief. So TechCrunch's request that 100% of the revenue of the *sales* of the device was overreaching, and legally insufficient and was therefore denied.

But the judge, then went on to spend quite a few pages laying out that it was likely that TechCrunch was in a partnership/joint venture with defendant FusionGarage, and that it was likely that Fusion Garage had breached its fiduciary duty, contrary to their assertions. It seems *very* likely to me, from reading the decision, that TechCrunch will win the case.

Specifically, the judge said, "Accordingly, TechCrunch has made a credible showing that it may be able to establish the existence of a joint venture under which Fusion Garage owed it certain fiduciary duties. Such duties may have precluded Fusion Garage from proceeding to market with the joojoo without taking appropriate steps to dissolve the relationship and to compensate TechCrunch."

If there were any money to be made, it is likely that TechCrunch would be getting a nice piece of it. Alas, given the circumstances, a victory here would be pyrrhic. The JooJoo will never make money and so the whole thing is moot. What's not moot is that Gruber, for some strange reason felt compelled to summarize the judge's decision in a patently false manner. Re: Daring Fireball, caveat emptor.

Death of the Relational Database 2010

2010-08-26T06:47:00.000-04:00

Back in 2008 I wrote a piece called Death of the Relational Database. It was one of the most popular pieces I have written, and for many reasons the subject matter is one in which I have an ongoing interest. As a result I wrote current piece to organize my thoughts for a talk I have proposed for the South By South West Conference. If you’d like to hear me talk (or argue about this subject with me) you can vote for my talk on the SXSW Panel Picker and meet me in Austin. Voting ends on August 28th.

Overview

People have begun to realize the enormous gap between the relational database abstraction and the way people actually think about information. To be clear, I am not suggesting that relational databases will stop being used or that they are going to go away, but that developers are going to stop thinking of their data in relational database terms.

Most Data Models Are Simple. Storing Them Isn’t.

Everyone from non-technical users to sophisticated developers thinks about information in a pretty simple way. There are objects, and there are connections or relationships between objects. For example if you have two objects, a cup and a table, the relationship between them might be “sitting on”, indicating that the cup is sitting on the table.

What makes this model so sturdy is that we can continuously add new objects: tables, cups, chairs, floors, table cloths, etc. And we can add infinite relationships, such as sitting on, sitting under, covering, etc. Computer scientists, and now, thanks to Facebook, everybody else, refers to this structure as a graph.

And while the graph model is incredibly compelling, the problem is that when programmers write programs that access data in memory, they are able to write code that models data as a graph of objects and connections. But when programmers write programs that need to store persistent data on disk, they have generally needed to use the much less natural and more complex relational database.

Historically, the reasons for using a different model from how we naturally think about data when storing things on disk boils down to the fact that the algorithms for implementing relational databases have been very, very efficient. They have worked well, they can guarantee data integrity, and they have been the best we’ve had in terms of performance. These reasons, by the way, are also the reasons, the underlying relational database technology will be around for a long time.

We're Really Talking About The Death of the Relational Database Abstraction

So when I talk about the death of the relational database, what I am actually talking about is not the death of the technology, but the death of the *abstraction* as a way that programmers and people think about data. The reason this death is occurring is because what programmers really want is to think about data in the most natural way possible. In the last several years that has become more and more feasible. Just as all software today actually runs in machine language, we have moved away from that as a model for thinking about code. We have become more abstract and now think in terms of much higher level languages that are more effective for programming.

Similarly, relational databases will continue to sit inside and under many data stores. But we are starting to see abstractions that move data management closer to how we think about organizing data and further away from the relational storage model. Sometimes those more natural abstractions sit on top of relational databases, with things like Ruby On Rails Active Record. And sometimes these abstractions are implemented in entirely new engines. But the implications of having a more natural way of thinking about and organizing data are profound.

The Web Is Really a Giant Graph Database

As an example of how profound storage and data modeling abstractions are, consider that the World Wide Web itself is a form of a graph database. You have objects, in this case pages, that are connected by links. Of course all of the objects are of the same type, i.e. pages and all of the connections are of the same type, i.e. links. Nevertheless, the Web serves as the quintessential example of the power of thinking about data in a more natural way. Improving the database abstraction in the database world will have similarly profound implications. Just as the leverage available to us increased massively when we went from machine language to assembler, and from assembler to C and from C to C++ and then to Java, so too will abstractions in our data model provide similar leverage and acceleration of innovation.

My belief is that databases will, over time, become more like graphs. There are several early products and technologies that implement graph database technology, including the one my company is developing. This talk, however, is not about whether technology A or technology B is going to win. There will, no doubt, be a proliferation of technologies, and I suspect there may not be any clear winner for some time, if ever.

But data storage technology is starting to move closer to how people think about data instead of requiring people to model their data the way the computers do.

What About The Semantic Web?

Now some of you may think these ideas are familiar. In fact if you have looked at any of the technology underlying what is known as the Semantic Web, the idea of the graph for data storage is not new. Tim Berners-Lee and the W3C have been promoting the concept and developing the standards for more than ten years. But the truth is most of the technological standards that underlie the official Semantic Web specifications have not taken off. It is my view that they never will. What is the difference between what I am saying and what the Semantic Web is about?

The key difference is the ideas I am talking about are a way to look at data management that can and will take shape in many forms. They are not tied to any specific implementation details. On the other hand, the W3C’s Semantic Web is a set of very specific standards for how a Web scale graph of objects should be implemented. The problem is that people have seemed to like the ideas, but not the standards so much.

For example, Facebook introduced the concept of the social graph and more recently a broader graph of things in the world. But until recently, Facebook used none of the Semantic Web standards, and with its latest technology uses only a tiny sliver of it. The reason is that the Semantic Web standards are, to be honest, not very good at speaking to the needs of regular developers. As a result some of the fantastic ideas in the Semantic Web are dying under the weight of an obtuse implementation specification.

Other Models

Of course, as I see it, a pure graph is the best way to represent information, but there are other non-relational ways to think about data that are starting to take off as well. Key value stores and document databases like MongoDB and CouchDB are becoming incredibly popular. This is, in part, because they perform really well, but also because for many applications, they line up much better with the way people think about their data than relational databases do.

Better Abstractions Means Better Results

Perhaps the most important point here is that the process of improving our data abstractions and moving away from the relational model, a process that has already begun and will accelerate, is going to have profound effects on developer productivity, and most importantly on the types of applications that are possible. Just as every new class of programming languages has provided more leverage and more useful applications, the same types of explosive advances will come from better more natural data storage abstractions.

Coda

Again, if you’re coming to South By South West (or even if you're not) and you think the subject is interesting, you can vote for my panel here by August 28th.

I guess Hacker News doesn't like negative articles about Apple

2010-06-16T17:50:00.003-04:00

Amazing.

I wrote a piece a few hours ago and noticed that someone had posted it to Hacker News. I never know when one of my pieces will get posted there or Reddit or wherever, so it is always pleasant to see.

The piece was about my concern about law enforcement and its role in the Apple Gizmodo situation and now the AT&T Goatse situation.

The piece got at least 10 points, and made it to the front page... where it was then killed.

This is amazing to me. I am not really a part of the Hacker News community, but the fact that apparently they censor articles because they disagree with the content is incredible. I thought the idea was, if people don't like something then that is reflected in the voting. But I guess sometimes Mr. Graham (the HN proprietor), knows best what is safe for people to read.

Next thing you know I am sure I will be banned from *reading* Hacker News. I'm sure its coming.

Law enforcement has apparently become an arm of AT&T and Apple

2010-06-16T16:05:00.003-04:00

I feel like we are living in what I thought was the fictional world of the movie RoboCop. In that movie law enforcement is turned over to private company OCP with the cyborg RoboCop leading the way. We're not quite there yet, but these last few months have exposed a frightening ability of corporate America to get official law enforcement to do its bidding.

First a little background.

As everyone who reads this blog I am sure knows, several months ago, the Gadget blog Gizmodo purchased a then secret unreleased iPhone from someone who claimed to find it in a bar. Gizmodo then wrote a major article about the phone and then gave it back to Apple.

What has followed is an investigation by local police, kicked off with a search warrant executed on the home of Jason Chen editor of Gizmodo, looking for evidence to support a criminal case against him for "theft" of the already returned phone. They broke down his door, when he was not home, and took all of the computers out of his house.

Fast forward to last week.

On Wednesday, Gawker published an article about a major security flaw in iPads discovered by Goatse Security. Goatse did not publicize the security flaw until it had already been closed there was no opportunity for the flaw to be exploited. AT&T then blamed their security failure on the whistleblowers.

Unauthorized computer “hackers” maliciously exploited a function designed to make your iPad log-in process faster by pre-populating an AT&T authentication page with the email address you used to register your iPad for 3G service.

The fact that AT&T chooses to blame the people who found the problem and reported it is bad enough. It is pretty clear to everyone in the tech universe that what Goatse did was a service to the community and that blaming them is lame. But what follows is truly shocking.

Today the leader of the Goatse team, Andrew Auernheimer, had an FBI search warrant executed against him.

Are you getting the pattern now?

Unfortunately, it gets worse. In executing the search warrant, the FBI found drugs, and arrested Auernheimer on possession charges. Now I have no idea if Auerheimer is an overall good guy. And I am not a drug user and personally dont like the idea. But the idea that the FBI gets to ransack your home because you told on some huge corporation who couldn't give a hoot about your security unless they are publicly embarrassed about it is the ultimate example of no good deed going unpunished.

And while there will be lots of talk in the coming days about AT&T, security, and the Goatse situation, I want to focus on a larger issue.

Why is it so easy for these huge private companies to get law enforcement to do their bidding?

In the Apple case, I'd really like to know how easy it would be to get a "task force" to search for evidence that *MY* **RETURNED** phone was "stolen".

Ridiculous.

The truth is that if you are just some regular schmo, and you go to the police and tell them your kid is missing, they will tell you you have to wait 24 hours, no matter how egregious the situation. But in the case of a lost but quickly returned phone, they have no problem sending in a crew of cops to ransack someone's house.

We all know that in the case of Apple, their beef isn't that Gizmodo bought the phone. Their beef is that Gizmodo wrote an article about their secret phone. If Gizmodo had bought the phone and returned it without writing the article, do you think there would be an investigation?

But Apple doesn't have a legal right to secrecy. They have to achieve that secrecy through vigilance. If they fail, there is no legal remedy unless it is some form of breach of contract in connection with a non-disclosure. It is certainly in no case criminal. But what law enforcement is really doing here is creating a punishment for having exposed Apple's secret, because even if they don't ultimately pursue a case, the horror of being searched and investigated by the police is a powerful deterrent.

To suggest anything else is patently absurd.

In the case of AT&T of course we don't yet know all the facts. But if things play out the way they look, its more of the same. Law enforcement is punishing someone for exposing an embarrassing corporate secret. This should not be the role of law enforcement in our society.

We should all have equal access to the law, and certainly there is no way in hell that I could get the police to investiate someone who returned my missing property. Ever. Of course I know that in this country you get as much access to the law as you can afford, but the fact that Apple can induce a criminal investigation that no regular person or even regular corporation could is scary.

Similarly, as far as I know, there has never been a case of the FBI investigating someone for exposing a security exploit even if they did so before the exploit had a chance to get fixed, which it did in this case. This case represents a new danger to all of us if security researchers are now punished for exposing their discoveries.

The bottom line is that if we are not careful, we are at grave risk of our freedoms being eroded. The usual concern about such issues is that government is too big. That is not my worry because no matter what, government will, by necessity, be big. My concern is who controls it. Because if Apple's or AT&T's vote counts more than mine and yours, we have a problem.

Told ya so...BP RIP

2010-06-15T10:24:00.000-04:00

This morning, I was reading an article in the New York Times that triggered me to look back on what I was saying about the spill back in May, and where we are now.

Back then I refuted the then common wisdom that the spill would cost 2 or 3 billion dollars. I said I thought such estimates were "ridiculous" and that I though the damages were going to be in the tens of billions and that, "I think the damages that come from this have the potential to mortally wound BP."

Finally, everyone has caught up to the obvious. BP's share price has fallen by 50% since the crisis and I don't think its done falling by a longshot. That BP is in a fight for survival has gone from a fringe viewpoint to conventional wisdom.

From today's New York Times:

The ultimate cost of the disaster remains uncertain. Wall Street estimates have put the bill for BP at anywhere from $17 billion to $60 billion, including penalties, damages and cleanup costs for the Deepwater Horizon disaster.

And then later in the same article.

“The costs are going to be significant, yes, and potentially they are an existential threat to BP,” said Bruce Bullock, director of the Maguire Energy Institute at Southern Methodist University.

This is the kind of stuff I'd rather be wrong about.

Sometimes it is a conspiracy

2010-06-14T10:05:00.002-04:00

I am the guy that is generally annoyed by unsubstantiated conspiracy theories from the political left and the right about things that supposedly the other side did. Conspiracies are hard to pull off because they generally involve, well, people conspiring. That means talking. And big conspiracies require *more* people conspiring. And making sure people STFU is hard. There is always someone who can't keep their mouth shut. That is why big conspiracies have a hard time surviving, particularly in the Internet age.

But sometimes conspiracies get pulled off. At least for a while. That doesn't mean that big conspiracies can ultimately be kept secret forever, but sometimes they can be kept quiet long enough for the dirty deed to get done.

And this is what I fear has happened in South Carolina.

Last Tuesday Alvin Greene won the South Carolina democratic nomination for senate.

Mr. Greene is unemployed, but managed to come up with the $10,400 South Carolina filing fee.
Mr. Greene currently being prosecuted for a felony obscenity charge.
Mr. Greene did no campaigning. None.
Mr. Greene got no press coverage before winning. None.
Mr. Greene raised no money. None.
Mr. Greene had no campaign staff or workers. None.
Mr. Greene has a hard time completing sentences.
Mr. Greene is a black man in a state that, lets just say, isn't the most racially open minded state in the country.
Mr. Greene got 60% of the democratic primary vote, *whipping* the leading candidate, a well known former judge and state legislator.

If you think this adds up, I have some BP shares I'd like to sell you.

Apple fears the killer app

2010-06-11T08:23:00.001-04:00

Ok, so were finally getting down to it. On at least two fronts, Apple has now essentially thrown out its draconian rules on what developers can and cannot do on its platform, and replaced them with essentially, no rules. The new "rules" appear to be, "its OK to do what you want in your app if we say so. And we'll figure that out *after* you've fully invested in our platform." In other words, you serve at the pleasure of the queen.

Now the truth is for the vast majority of app developers this is totally fine. People developing the uninspiring apps that mainly make up the App Store have nothing to fear. But those creative few that want to do something interesting with a UI, or want to use hardware in a new way, or who want to use a more advanced code execution technique are at grave risk.

For now, I will leave it to others to debate the impact of this strategy. I want to explore something different.

What is motivating Apple?

Apple claims its goal with all these rules is to keep out bad applications. But if that is the case, they are failing miserably because a lot of apps in the App Store, perhaps the majority, are total crap. Their stated rationale is, I believe, baloney. In fact, not only do I think Apple couldn't care less about whether apps are crap, I think they *love* the crap, and that their goal is in fact to keep out the awesome.

I know this sounds over the top. Let me explain.

I think Apple is fearful of any truly ground breaking stuff coming from a third party. Apple does not really want even a handful of awe inspiring market moving 3rd party apps. They want hundreds of thousands of decent or even mediocre or crappy apps. The rationale for this thinking is actually pretty reasonable. Its great to be able to claim having more apps than any other platform. Numeric superiority is a huge marketing tool. No one cares if your tens of thousands of apps are mainly crap.

And yet, in the history of computing, we know that its not the number of apps that make a platform, but the existence of "killer apps." Whether it was VisiCalc, or Lotus 123, or PageMaker, or Microsoft Office, the truth is users don't really want hundreds of apps, they want one or a small number that are really meaningful.

The good news about killer apps for platform vendors is that they can drive the platform into the stratosphere. This is certainly great for early stage platforms like the Mac with PageMaker in the 80's, or even with Facebook and Farmville in the last several years. But as platforms mature, Killer apps from third party companies pose more risk.

I think Apple has come to the conclusion that any killer apps for the iPhone need to be from Apple, and that those that are not from Apple are hugely dangerous. And this well may be true. Because if some third party invents something that fundamentally changes what it means to own a mobile device, and that software is available on other devices, overnight Apple is in the position of being the supplicant.

If that killer app vendor decides to support Android more effectively than they support Apple, or if for some reason they decided to drop the iPhone, that one vendor could have a devastating effect on Apple's position in the marketplace. This is the position that Apple was in with Adobe in the 90's and Jobs has made it clear he is fearful of ever being in that position again. Others have discussed this but it is usually framed in the context of why Apple doesn't want Adobe on its platform. But I think the broader issue is they don't want *any* companies generating hundreds of millions of dollars through some new mobile technology which Apple doesn't control.

The greatest support for my thesis is that there are not yet any third party companies that have made a huge amount of money on the iPhone. Has there been success? Yes. Has there been enough to support a major exit, or to even put someone on that trajectory? No. And I think Apple will work hard, through whatever rule changes and market behavior is necessary to guarantee that that never happens, at least with any non-vertical technology.

So I guess the question here is what to make of all of this. And I think the answer is clear. If your goal is to build a major company, iOS is probably not the place to make that investment. Certainly it may be a great place for an entrepreneur who wants to build a lifestyle business, though even there risks abound. And for games I think the iPhone is fine platform since games are probably not strategic. But for anyone else, either entrepreneur or investor, who aspires to build a truly market moving mobile company, I strongly suggest you think different. Apple isn't the only game in town, and as I have written previously, I don't think it will even be the biggest game in town much longer.

Apple will be hugely profitable, but not dominant

2010-06-07T18:04:00.002-04:00

There is a popular meme that Apple will dominate media markets as well as the smart phone market. Certainly Apple is currently the largest digital music vendor and Apple has made incredible gains against RIM that would scare the crap out of me if I were them.

But the most interesting question to me is where all of this leads. And at the end of the day, I see Apple remaining the largest and most profitable vendor in these markets, but not the dominant technology. The reason for this is simple. One vendor cannot provide sufficiently diverse distribution channels, form factors, feature sets, and designs.

Currently Apple's biggest distribution challenge is the fact that iPhones are only avaliable on AT&T. But even when Apple eventually cuts a deal with Verizon, I think it will be at a distribution disadvantage. While Apple's phones today are in my estimation somewhat better than the best Android phones, markets don't collectively assess products that way. Imagine what the average consumer will do when going to a phone store and being confronted with ten phones that seem very similar, where one is an iPhone and the other nine are Androids. Odds are they will buy one of the Android models.

None of this means that Apple will be in any trouble. Apple will continue to be a wildly successful company because they will by far be the most profitable company in the mobile market. I don't think Apple is trying to maximize market share at all. That is a nice bonus, but what Apple wants is to be optimized for making the maximum amount of money possible.

By the way, in my example above where the consumer is presented with one iPhone and nine androids, I didnt include RIM. Why is that? Because its fairly clear that RIM cannot survive as a market leader. They are destined to become the next Palm. Not next week, or next month, but within 18 months their current 35% of the smart phone market will be below 20%. There is no magic in what RIM can do around delivering mail, but replicating the sexiness of the iPhone OS (just renamed iOS) and Android is not in their organizational DNA. They can't do it, and the gap between RIM and everyone else is going to become increasingly obvious. They may get bought, perhaps by a desperate Microsoft, or they may just whither, but there will be no competitive RIM OS, and so there will be no competitive RIM.

Yes, NYT, of course the Internet causes A.D.D.

2010-06-07T15:33:00.000-04:00

Its great when the New York Times catches up with something I wrote about a long time ago. Their article the other day, "Hooked on Gadgets, and Paying a Mental Price" is something I wrote about in May of 2008, "The Internet Causes A.D.D. (Attention Deficit Disorder)". I love being a little bit ahead of my time!