Nick Bradbury's shared items in Google Reader

Rex’s prayer for the busy geek

2009-11-07T20:58:42Z

Lord grant me the serenity to accept that there are some things I just can’t keep up with, the determination to keep up with the things I must keep up with, and the wisdom to find a good RSS feed from someone who keeps up with what I’d like to, but just don’t have the damn bandwidth to handle right now.

The Disintermediation Era

2009-11-06T03:19:48Z

It must suck to be the middle-man today. Everywhere they turn, it’s bad news. Democratization this. Circumventing that. There was a point not that long ago that the middle-man provided great value. The record companies brought music to the masses. The media created channels for the news to get through. The Blockbusters of the world housed thousands of movies for people to rent. Telephone companies laid the lines for us to connect with one another around the world.

But now these middle-men are our modern villains – using every desperate trick in the book to hold onto customers while we find creative ways to go around them, go straight to the source and sometimes just do it ourselves. There is a mass disintermediation going on and every company that occupies the mediator position is at risk. Now it’s the media, the labels and the distributors of what has become digital content, but I doubt this will be the last frontier of democratization. I’m sorry to say it, but they are bringing it on themselves.

Why?

They forgot that the point of being the middle-man is, well, to be the middle-man. They made it about themselves. The musicians and the audience were forgotten in the shuffle. The news and the readers stopped mattering. The fact that people just wanted to connect via phone ceased to be part of any business plan. Profits became bigger than the people they needed to connect.
Convenience, which was once the advantage of using a middle-man, turned to inconvenience. Frustration set in when the middle-men became gatekeepers and ‘deciders’. They became greedy. Power hungry. And they reminded everyone that they were going to play by the rules that they set. Period.
They misread the early signs of democratization as a threat rather than an opportunity. All could have been forgotten if they had just realized that their impetuous children were neither impetuous nor children, but instead were giving really great cues on how they wanted things to work.
A false sense of scarcity is not a scarcity at all. Creating a business model on charging premiums for something in abundance (or potentially readily available) is bound to crater.
They seriously underestimated the intelligence of the public. Maybe they believed their own lies or maybe they just thought we couldn’t think for ourselves. We may be lazy from time to time, but we aren’t stupid. And we don’t like being belittled.

And so the scare tactics have been amplified, the sob stories are rampant and the battle has turned to an all out war. People getting sued at $16,000 per song for illegal downloads and the Anti-Counterfeiting Trade Agreement (ACTA) are some of the obvious examples, but everytime I look at my phonebill and see the newly minted extra charges, I can’t help but feel like my phone company is punishing me. I don’t want to see journalists out of work. I don’t think it’s their doing that the media companies are in trouble. But the layoffs are breaking my heart.

And it’s only really beginning. We are at a turn. A shifting of an era. Entirely new business models were created during the Industrial era. These business models were created to help manage, distribute and promote to the masses. When everything was local, we did this through relationships. It was easy to manage on our own. But the internet allowed for inexpensive and simplified management, distribution and promotion for all. Farming these things out only makes sense if it truly brings value such as: convenience, money saving and peace of mind.

I can only imagine where it goes from here. What else is going to be disintermediated as we gain more tools of control and simplification? Banking? Law? Public services? We’ve pretty much lost the travel agencies. Authors are self-publishing and more tools are available for distribution. Amateur movies are cheaper and simpler to make and are getting more attention. People are finding ways to go direct to farmers for their food.

It’s nothing to be mourned, but it is something to be heeded. Eras come and go and change happens. I read somewhere that only 1 of the original Dow Jones companies still exists and I’ll bet it exists because it looks nothing like it did back when the Dow Jones was born. (That would be General Electric – thanks David Damore!)

There is new intermediation needed. It has to do with helping us cut through the noise and get to the signals and it needs to be individual-driven. Things like Project VRM should be at the top of everyone’s radar. Finding new business models to further democratize badly managed industries is also a good bet. Either way, I’m looking forward to the changes and I’m open and ready for them.

Our Amazon Advertising Experiment

2009-11-05T15:00:43Z

Do you remember when I discussed the crushing disappointment that is Google AdSense in Podcast 64? If Stack Overflow, a site that does a million pageviews a day, can’t make enough from AdSense to pay even one person half time — and let me tell you, that’s being overly generous based on the actual income it generated — how does anyone make a decent living with AdSense? Seriously, how? Exclusively talking about Mesothomelia and Asbestos, or what?

As a result, we dropped AdSense like a hot (or, rather, a particularly cold) potato. Instead, we turned to our pal Alex of The Daily WTF, and hooked into his curated ad network for software developers. We are firm believers in responsible (read: no flash, no animation) and restrained (read: limited to 3 ad slots, reduced ads for >200 rep) advertising. This has worked quite well for us so far. How well? On the order of fifty to a hundred times better than AdSense! I am not exaggerating. Those are actual numbers.

Even though Alex does a great job, we always have a lot of left over unsold ad space. And as the site has grown over the last 6 months, this gap has widened. So then the question becomes — if AdSense doesn’t work for us (and boy, does it ever not work for us) — then what can you do with that remnant ad space? I hate the word monetization with a passion, but surely something useful could be done here?

That’s when Portman Wills approached us. He’s not only an old school 4 digit Stack Overflow user and fellow programmer — he also has extensive experience in his previous gigs with advertising code.

Portman is currently busy building cool stuff like shuffletime (not to mention his hilarious parody sites woofer and feeling unlucky). But he was enthused about the opportunity to help out Stack Overflow — and maybe, just maybe, generate some ads that were actually (gasp!) useful and relevant to his fellow programmers at the same time.

Thus, Portman generously offered to build a custom ad-serving site for us, which we gladly hosted at rads.stackoverflow.com.

Rads has three main components:

A spider which uses the Amazon Product Advertising API to crawl the Amazon product catalog.
A website which renders an advertisement based on Stack Overflow tags.
Some analytics to determine which ads, books, and tags are most effective.

The spider was fed the top 5000 tags on Stack Overflow. For each tag, it preformed a keyword search on the “Computers & Internet” node, returning the top 10 books with five-star reviews, sorted by number of reviews.

You can read the full skinny in Portman’s summary. We had high hopes of building something that connected great programmers with quality programming books on Amazon. The ads looked nice, too:

Excellent plan, right? Smart. Clever, even!

Well, it was a complete and utter failure.

Despite our purported cleverness, it didn’t work. Not even a little. The Amazon ad experiment was a total failure by any metric I can think of. Clicks, revenue, goodwill, newton-pounds, cuils, you name it. It was literally a waste of everyone’s time. Even flipping burgers would have paid more.

But this failure was not for lack of trying. If a guy as skilled as Portman — who not only has a deep background in custom advertising, but is also a programmer capable of writing a solution tailored to our specific audience — can’t make this work, I had to regretfully conclude that nobody could make it work. It’s just not possible.

So we scrapped the whole thing, and we’re going in a different direction. More news on that soon.

But in the meantime, since we had our fancy-shmancy Amazon Affiliates account set up, we might as well put it to good use. Even way back in the original Stack Overflow beta, people were proposing that we convert any Amazon book links to Stack Overflow amazon affiliate book links. I was hesitant to do this at the time, but given our failure, I was licking my wounds. I was willing to give it a try. Particularly since the community seemed totally OK with the concept.

So, onward to plan B: we now auto-insert Stack Overflow affiliate info into any amazon book links posted on Stack Overflow. Oh yeah, and here’s the kicker. These silly little rewritten text links work 200%-300% better than our custom amazon book ads!

Go figure.

All I can say is, advertising is hard, let’s go shopping! And when it’s not hard, it’s borderline scammy, which is something we just don’t do at Stack Overflow.

At any rate, I’m glad Portman is here to ~~take the blame~~help. Apparently we can add advertising to the long, long list of things that we suck at. But we do plan to suck less every year!

America Only Seems Polarized

2009-11-05T12:00:00Z

Barack Obama held out hope of overcoming partisan divides, lowering the temperature, and bringing Americans together. How's that working out? Not well, it appears. One year after he was elected, Americans look more polarized than ever.

In a special House election in upstate New York, a Conservative Party candidate, backed by Sarah Palin, took on a moderate Republican whom his supporters called a "radical leftist," forced her to withdraw, and then lost to the Democrat. It's entirely possible that in the Senate, not a single Republican will vote for an administration-supported health insurance overhaul.

Sen. Lindsey Graham (R-S.C.), laments that "it makes news when Democrats and Republicans do something of substance together and that truly is a shame." From cable TV news channels, you get the impression of a country not so much politically divided as verging on civil war.

Here's a solution to that problem: Stop watching cable TV news channels and listening to politicians. Using them as a gauge of how divided we are is like using the National Hockey League to estimate the level of violence in America.

Most Americans aren't rabid liberals or fanatical conservatives. Gallup recently found that more people call themselves conservative than liberal or moderate. But other polls contradict it. According to a 2008 survey by the National Opinion Research Center, when you give them more options—extremely liberal, liberal, slightly liberal, moderate, slightly conservative, conservative, or extremely conservative—you find that the largest ideological group is moderates, with 37.3 percent compared to 34.5 percent for the three conservative groups combined.

Add up the moderates and those who are only slightly liberal or slightly conservative and those who don't know—those clustered in the middle of the road—and you've got about two-thirds of the citizenry. As political scientists Morris Fiorina of Stanford's Hoover Institution and Samuel Abrams of Harvard put it, "the American electorate in 2008 is much better described as centrist than polarized."

Moreover, they note in a forthcoming paper, the public is not getting more polarized. "In terms of their ideological orientations," they note, "the American electorate looks about the same as it did when Democrat Jimmy Carter defeated Republican Jerry Ford in the not very polarized 1976 election"—Carter being conservative by Democratic standards and Ford moderate by GOP standards of the day.

So why does everything feel so bitterly divided? One reason is that the elected officials of the two major parties have definitely gotten more ideologically uniform. A generation ago, we had liberal Republicans and conservative Democrats, two species that are nearly extinct. In 1965, half of House Republicans voted in favor of creating Medicare. No such mass crossover this time.

Among ordinary people who identify with one party or the other, however, there is far more diversity of views than among either party's leaders. Gun owners and evangelical Christians are supposed to be repelled by elitist liberal Democrats, but Fiorina and Abrams report that nearly 40 percent of gun owners voted for Obama, along with more than a quarter of white evangelical Protestants. Though Republicans are the anti-abortion party, one-third of Democrats are closer to the GOP position than to that of their own party.

Strictly ideological parties mean most people have little choice but to vote for ideologues. Faced with a liberal Democrat and a conservative Republican, write Abrams and Fiorina, voters "tend to vote for the candidate on their side of the spectrum, although they might well have preferred more moderate choices."

Another reason for the acidic climate is the rise of cable TV networks that thrive by taking ideological sides, day in and day out. Twenty years ago, they didn't exist. Today, watching Fox News, you get the impression that huge numbers of Americans regard Obama as a Stalinist. Switch on MSNBC, and you would assume that most people want Dick Cheney sent to Guantanamo.

You would be mistaken. Fox News averages just 2.6 million viewers on a typical weeknight, or less than 1 percent of Americans. MSNBC does even worse, with 831,000 per night. The three major network newscasts, which offer less overt bias, pull in a combined total of more than 20 million viewers each evening.

The average American citizen, contrary to myth, is neither very angry, nor very far to the left or the right, nor inclined to treat anyone with different opinions as a mortal enemy. In a cluttered media environment, the most extreme voices tend to attract so much attention that it's easy to forget something important: Most people aren't listening.

Goodbye Microsoft, the next chapter

2009-11-04T18:43:41Z

Microsoft announced more layoffs today, and I was one of them. This was a total surprise to me, and management offered no explanation. This is pretty standard procedure, mostly for legal reasons, but none the less left me with a cold feeling...but only for a minute or two.

Today I start thinking about the next chapter in my life. It is always exciting to look beyond your normal boundaries and think about new possibilities. Being totally consumed with my job and traveling every week has left no time to think about other opportunities. That changes today. I couldn’t be more excited about the future.

I will be blogging more often now, and that excites me. There are lots of topics that I have wanted to dive into but just haven’t had the time. I will be seeing more friends too. Again, I have been so busy traveling that I haven’t had time to connect with friends all over the world. It’s all good.

My email at Microsoft will go dark in a day or two, so my new contact is DonaldDodge@gmail.com or send me a note on Facebook or Twitter @DonDodge

Thanks to all the fine people at Microsoft. You know who you are. There are thousands of talented people there and I enjoyed working with all of them. I’m sure we will see each other again at conferences and industry events. Its a small world…and getting more connected every day. I’ll see you on the web.

Subscribe - To get an automatic feed of all future posts subscribe here, or to receive them via email go here and enter your email address in the box in the right column. You can also follow me on Twitter @dondodge

.

10.24.09: Internet Antichrist

2009-10-24T16:24:00Z

I started thinking a few days ago about how the digitization and networking of so much of what we hold dear has changed things. I see that in my lifetime I will witness the end of books, or most of them, physical copies of recorded music and probably physical newspapers too. Stuff that’s been around for a thousand years will be gone in my lifetime! Film based photography is pretty much a remnant, an art form, an artisanal craft used by fine artists and high-end fashion photographers. And writing letters to one another? On paper? And dropping them in the mailbox? When was the last time I wrote and mailed a physical letter? All those academic books filled with Auden’s or Jane Austen’s letters — it’s hard to imagine a collection of someone’s text messages, tweets and e-mails. I suspect that television as we know it will be gone soon as well. All right, film and recorded music have only been around a hundred or so years, but books! All of which led me back to wondering — how did this get started?

The Internet, the World Wide Web, as much of a boon as it has been, has left an awful lot of wreckage in its wake, beyond just the elimination of those formats we thought of as eternal and the industries that produced and delivered them. Interconnectivity has facilitated the loss of privacy of many of the world’s citizens. We’ve been liberated and captured at the same time. I sense that the loss of privacy — which to me seems inevitable — is part and parcel of the whole project. You can’t have efficient search algorithms, cloud computing and digitized everything and anything and expect to retain the anonymity of the past.

Security races to keep up, but I wonder if the dream of unlimited access and personal and corporate data security aren’t simply incompatible. Maybe we just can’t have them both. Maybe we need to throw up our hands and give in. Stop resisting and surrender. Live totally and completely in public. The world would truly be the village that McLuhan predicted — a small town where everyone does know your business. Maybe that would keep us honest, and push the realization that as custodians of the planet we really are all in this together.

This “creative destruction” began in the ’60s, as did many things that we now both love and regret, and it was initially a spinoff of a project funded by US military agencies. The military (along with the space agency) gave us Velcro and (I believe) cheap integrated circuits (i.e. gizmolandia), as well as the blowback that helped nurture the current mess in the Middle East, South America and Afghanistan. The Internet’s connection to the military, as much as I would love it to be a big secret conspiracy, seems a lot more benign than that. Mephistopheles came to Faust in the form of a poodle. After all…in some versions of the story, he cannot enter your house unbidden — you have to invite him in, like a vampire.

One man foresaw a global network before any such thing was close to being possible. J. C. R. Licklider (sounds like a character in a Coen bros movie!) envisioned, in a 1960 paper called Man-Computer Symbiosis, "A network of such [computers], connected to one another by wide-band communication lines…[which provided] the functions of present-day libraries together with anticipated advances in information storage and retrieval and [other] symbiotic functions." [Source]

In other words, he saw it all coming.

[Source]

Is this man the antichrist? Or merely a prophet?

In a weird coincidence, Licklider began his career studying psychoacoustics (more on that later), and wrote a paper called “Duplex Theory of Pitch Perception” in 1951 that forms the basis of contemporary concepts of how we perceive pitch, even though it sounds like it might be about two-story apartments with uneven floors. That the man who predicted a worldwide information exchange network was initially interested in how we perceive music is slightly uncanny.

More about Licklider from Wikipedia:

“His ideas foretold of graphical computing, point-and-click interfaces, digital libraries, e-commerce, online banking, and software that would exist on a network and migrate wherever it was needed. He has been called ‘computing's Johnny Appleseed’ for having planted the seeds of computing in the digital age.”

Now, it’s been pointed out that he didn’t actually invent any of this stuff — he merely “planted the seed.” But often it seems that putting out the idea that something might be possible encourages others to actually make it possible. In a way, to imagine is to create.

In the ’50s, Licklider “worked on a Cold War project known as Semi Automatic Ground Environment (better known by its [weirdly appropriate] acronym ‘SAGE’) which was designed to create a computer-aided air defense system. The SAGE system included computers that collected and presented data to a human operator, who then chose the appropriate response. In 1957 he…conducted the first public demonstration of time-sharing,” [Source] which is when multiple parties can share the use of a single large computer. And in 1958, he became president of the Acoustical Society of America.

“He played a similar role in conceiving of and funding early networking research, most notably the ARPANET [acknowledged to be the predecessor to the Internet]. His 1968 paper on The Computer as a Communication Device predicts the use of computer networks to support communities of common interest and collaboration without regard to location.” [Source]

“Without regard to location”— the phrase resonates for me. It implies disincorporation — an out-of-body experience. In this case, it’s data that has no fixed place, no physical manifestation. But I sense it’s happening to us, too.

I had thought that the Internet began with the linking of some military computers in the Pentagon (ARPANET) in 1969, and that this network was an experimental project to create a system which was specifically designed so that its data could survive a nuclear attack. It turns out my hunch was wrong, although the military were indeed involved in funding the research. ARPANET (which Licklider was involved with) did give birth to internet protocols — how computers “talk” to one another — sometime later in the 1970s, but it was not, it seems, all about securing secret data from the electromagnetic pulses associated with nuclear weapons.

Bob Taylor, the Pentagon official who was in charge of the Advanced Research Projects Agency Network (or ARPANET) program, insists that its purpose was not military, but scientific. Though we might take whatever the Pentagon says with a big grain of salt, he could be telling the truth. Larry Roberts, who was employed by Taylor to build the Network, states that ARPANET was never intended to link people or act as a communications and information facility. So, the evolution into the Internet was completely unintentional, though Licklider foresaw it. ARPANET was primarily about finding a more efficient way of time-sharing.

Those were the days when computers looked like this:

They were extremely expensive, and there weren’t a lot of them, so many people, like my friend C’s brother, made a good living managing access to them. Time-sharing was a big issue. If however, access could be accomplished remotely, through a network, then the efficiency of the time-sharing would be increased. Time-sharing via these networks was focused on making it possible for research organizations (and the military) to use the processing power of other institutions’ computers when they had laborious calculations to do, or when someone else's facility might do the job better.

Because this research (used to develop ARPANET) was government-funded, its use was restricted to the military and university research facilities — C’s brother couldn’t use it to create or enhance the commercial enterprise he had established to manage computer access, for example.

“During the 1980s, the connections expanded to more educational institutions, and even to a growing number of companies such as Digital Equipment Corporation and Hewlett-Packard, which were participating in research projects or providing services to those who were.” [Source]

We can see by the involvement of these companies that the line between non-commercial use and commercial and public access was already getting fuzzy.

“Several other branches of the U.S. government, the National Aeronautics and Space Agency (NASA), the National Science Foundation (NSF), and the Department of Energy (DOE) became heavily involved in Internet research and started development of a successor to ARPANET. In the mid 1980s, all three of these branches developed the first Wide Area Networks based on TCP/IP.

“In 1984 the NSF…supported departments without such sophisticated network connections, using automated dial-up mail exchange. [For those who don’t remember or are too young, one used to access the Internet and send e-mail by modems that would “dial-up” using regular phone lines…a web page in this era would take many minutes to load; these were NOT the good old days in that sense.] This grew into the NSFNet backbone, established in 1986, and was intended to connect and provide access to a number of supercomputing centers established by the NSF.

“In 1992, Congress allowed commercial activity on NSFNet with the Scientific and Advanced-Technology Act, permitting NSFNet to interconnect with commercial networks. University users were outraged at the idea of noneducational use of their networks. Eventually, it was the commercial Internet service providers who brought prices low enough that junior colleges and other schools could afford to participate in the new arenas of education and research […and soon the rest of us].

“By 1990, ARPANET had been overtaken and replaced by newer networking technologies and the project came to a close.” [Source]

The mother, seed or egg that gave birth to the Internet was gone, and the floodgates had opened.

By the mid-’90s, access became easy enough that the commercialization of the Internet proceeded rapidly. I wondered to myself if the military kept a parallel World Wide Web, inaccessible to civilians, since they were so involved in the early stages of its development. They do, or did — it was called MILNET.

[Source]

A quarter of the earth’s people now use the Internet and the World Wide Web. We don’t know how many use MILNET. Finland and France are about to make Internet access a right, like a legal right to a trial, free speech or health services (well, these rights exist in some countries). The Finns want everyone in their entire country to have broadband (5mb) in a few years. (FYI, 5mb allows streaming video like most of us can see now, 10mb would allow HD streaming video and 100mb, which the Finnish government proposes offering by 2015, would, well, increase not only ease of access to information, but interactivity on a level and with repercussions we can hardly imagine.)

In the Meantime

While these networks were evolving, there were simultaneously a number of innovations and technological breakthroughs that allowed for the digitization of all sorts of media — the stuff that would soon be flying around those same networks.

The technology that allowed sound information (and soon all other information) to be digitized was largely developed by the phone companies. Bell Labs, a research division of AT&T, wanted to find more efficient and reliable ways of transmitting phone conversations. Phone lines up until that time were all analog, and with that technology the only way to squeeze more calls through a line was by rolling off the high and low frequencies, and turning the resulting lo-fi sound into waves that could run in parallel without interfering with one another — like terrestrial radio transmissions. TV and radio communications had the same problems.

Bell Labs was huge, and they had branches in many states, most of which are closed now. They invented the transistor and the semiconductors that made the integrated circuits in our tiny devices possible, they developed the laser — the list goes on and on. Their scientists won a lot of Nobel prizes.

When Bell Labs figured out how to digitize sound — to, in effect, sample a sound wave and slice it into tiny bits in a way that was not prohibitively expensive and that still left the human voice recognizable — they applied it to long distance calls, switchers and all manner of phone technology, allowing more calls to be made simultaneously, especially considering the limitations imposed by underwater cables. Much of the research regarding what makes a sound understandable (like a voice, in AT&T’s case) involves applying lessons from the science of psychoacoustics — how the brain perceives sound in all its aspects. We’re back to Licklider!

Out of this combination of psychoacoustic and technical research emerged digital equipment that was used in, among other places, recording studios — where I saw this technology. In the ’70s, the Harmonizers and digital delays that appeared little by little were in effect primitive samplers — the samples were usually less than a second long. These were quickly followed by machines that could hold longer samples of greater resolution, and manipulate those “sounds” more freely (clumps of data more than sounds, technically). All sorts of weirdness resulted. Bell Labs was involved in manufacturing a sound processor called a vocoder that would preserve certain aspects of talking (or singing), like speech formants — the shape of the sound apart from its pitch. Using this machine one could transmit these aspects of the voice separate from the rest of the vocalization in ways that rendered them unintelligible. One use for this was a sort of cryptology for the voice — a garbling that could be “decoded” at the other end. These machines were also adapted for music production. Here is Kraftwerk’s vocoder, made especially for them:

[Source]

I once used a vocoder borrowed from Bernie Krause when Eno and I did the Bush of Ghosts record. It was beautifully made, but rather complicated and very expensive.

A Harmonizer cost thousands of dollars, a digital reverb set a studio back maybe 10K, and a full-fledged sampling device like a Fairlight or later the Synclavier cost much, much more. But soon the price of memory and processing dropped, and the technology became more affordable. Inexpensive Akai samplers became the backbone of music like hip hop and DJ mixes, and sampled or digitally derived drum sounds took the place of live drummers in many recordings. And we were off to the races, for better or worse. With the digitization of sound, digital recording and eventually the CD became possible — and not too long after that, the capacity and speed of home computers was sufficient to record, archive, and process music.

Some years ago I visited Bell Labs and was shown the famous anechoic (perfect, sound absorbent) chamber. This was where John Cage claimed that he could hear both his heart pounding and the high-pitched whine of his nervous system. His insight was that true silence doesn’t exist — even if we can block out everything else, we can’t stop hearing ourselves.

Here is one such chamber:

[Source]

They also showed me a processor that could squeeze what seemed to the ear to be CD-quality sound into a miniscule bandwidth. I’m not sure, but I believe encoding music as MP3s had at that date already been invented in Germany, so this compressing/encoding was not a big surprise — but like most people, I worried that something in the quality of the music might have been sacrificed in this rezzing down process. I was right, but MP3s have improved quite a bit since then, and now I listen to most of the music I own in that format. I believe what Bell Labs was working on is used for satellite radio — getting more hi-fi sound into smaller transmissions.

In 1988 I went with designer Tibor Kalman to visit a printing studio on Long Island. It had a machine that could digitize and then subtly manipulate images (we wanted to “improve” the image on a Talking Heads record cover). This machine was, like those early computers, incredibly expensive and rare — we had to go to it (it couldn’t be brought to the design studio), and we had to book time in advance. Sytex I think it was called. This was exciting, but its cost and rarity meant we didn’t think much about incorporating its talents into more projects at that time.

After a while, though, the price of scanning dropped, and manipulating scanned images using something called Photoshop became common. Who would buy a film camera these days? Who buys film for their old camera? There are some holdouts, and I have no doubt that there is a richness or at least some special qualities that have been lost, but, well, for most of us, the trade-off seems fair — and inevitable. Needless to say, as these images became digitized they could enter the river of networked data.

Photojournalism went digital a number of years ago. In the beginning, the photographers, realizing that their images would be reproduced in newspapers no larger than 8x10 (if that), didn’t need to shoot at the highest available resolution on their new digital cameras, allowing them to squeeze more images onto their data chips — and giving them fewer problems with storage and developing in the field. To compare these low-res images to video, it’d be like if movies past a certain date were all captured at the quality of YouTube files. While researching archival news footage at some point, I discovered that when it migrated to videotape from 16mm film, the quality went way down.

The confluence of digitized media and the capability of digital information to be shared, transmitted and stored anywhere in the world — this volatile, disembodied mixture that Licklider predicted and whose seed he planted — has, duh, had a huge effect on countless institutions. Many that deal with physical objects — newsstands, record stores, bookstores — will all go away, along with their support structures: trucks, warehouses and all the people that worked in those places. For many of us this is not all bad. The record stores like Sam Goody or Coconuts were never great experiences.

Maybe the first institution to disappear almost completely as a result of this process was the letter. Conventional mail still exists — I get bills, junk mail and announcements — but communication related to my work and between my friends and me is almost all by e-mail or text, as has been for a while.

Television, not a big part of my life for quite a number of years anyway, is bound to migrate online and become something very different.

It’s not so surprising to witness the end of many of the delivery systems for recorded music — vinyl, cassettes and CDs. Somehow those changed from one form to another so rapidly over the decades that to see them all go away isn’t that much of a shock. I don’t really miss them all that much, to be honest. But to imagine that I might live to see the end of print — books, newspapers and many magazines — is mind-boggling. Publishers and news organizations might argue that they are not like the music business, but the patterns are too similar to ignore, except by those who don’t want to see them. Print and books have remained more or less unchanged since Gutenberg, but all that seems about to become history.

I’m not advocating trying to stop this — it all seems inevitable, and the access to information and convenience will be unprecedented — although without newspapers as a Fourth Estate, a check and balance, democracy as we were taught it, will not be, um, the same. We can’t rely on bloggers to police the entire government. Danielle comments, however, that the death of physical newspapers isn’t the same as the death of journalism — if the NY Times can find a way to make money as with digital distribution, it will continue to provide a similar function in society. Whether that will be possible is still an open question — but digitization doesn’t necessarily equal death, at least not yet.

The End of Privacy

Now that the Internet and the World Wide Web have enabled data, content and information to be shuttled anywhere in the world — even around China, sometimes — it seems inevitable that the flow goes both ways, or actually in many ways. The ability to access the Internet is incredibly useful to us and we can’t imagine life without it, so we don’t seem too bothered that as a result of this interconnectedness, the National Security Administration, for one, has access to our web lives and loves — and we don’t seem all that nervous that cloud computing will eliminate any real sense of privacy (despite assurances), or about the massive amounts of information Google and other commercial enterprises have about us.

Danielle points out that many people are in fact very nervous about this — that privacy & the Internet is a huge topic of concern. Google data mining, the ownership and confidentiality of social networking data, security of financial data, etc. — these are all topics that are regularly reported on in the press and about which people have very strong feelings. However, the sense I get on the street is that most ordinary folks are happy (so far) to give up some personal security for all the convenience they’re getting.

Google’s batteries of server farms allow us to search, so, naturally the NSA can also search, dredge and process. I typed in someone’s name yesterday and found that for a small fee, I could see how much they paid for their house, who their neighbors are and what their credit rating is! I was flabbergasted. That’s me, a private citizen, who can know stuff I’d sort of rather not know, not some corporation or governmental agency.

Here’s an NSA data mining facility in Yakima, Washington. (A massive one is being built in Utah.)

[Source]

So far I’m not aware of malicious use of all that information, not on a large scale anyway — though identity thieves and guys sucking up US credit card numbers by the truckload in Ukraine are a start.

I recently read an article regarding the security of so-called “scrubbed” data. Netflix or some other company wanted to employ a third party to analyze some of their customers’ patterns of purchase — but as a precaution they removed (scrubbed) the customers’ names off the data. So theoretically, the people being analyzed were now abstract entities. However, out of curiosity they hired another company, to see if any of those unidentified customers could possibly be re-identified. It turned out they could. Not due to a fault of the scrubbing, or some security or software malfunction, but because other data and patterns of customer and citizen behavior were available online, and correlating these with the patterns of the anonymous customers led to conclude, beyond a reasonable doubt, the re-identification of many.

To me this means that, yes, information already flows both, or rather all, ways. Privacy and security, as much as we might strive for them, are phantoms that we chase but can never truly catch. As much as we love getting information, data, media and connections, so we ourselves become available as data. Social websites like MySpace, Facebook and Twitter seem to use these conflicting urges — the urge to reveal oneself to the world, in all one’s intimate details, and yet simultaneously maintain some kind of privacy. Good luck with that.

The end of privacy in parts of the world is near. It will be traumatic for some, and a comfort for others — for to relinquish one’s privacy is to become a part of the hive and the herd, and there is a certain reassurance there. How our corporate culture and its twin, the government, make use of this process and this massive change in society leads one to imagine something closer to a paranoid Phillip K. Dick scenario than a return to the nurturing tribe (or the Global Village) that it will be for some. I suspect it will be both — liberating and restrictive. Conflicting and opposite tendencies, operating simultaneously.

So, there it is. The free flow of information, and the ability to digitize all media as it enters the river, has a lot more repercussions than the end of books, newspapers and CDs — it portends a massive social and political shift. Licklider may have seen this coming as well, but he didn’t let on about it.

Why do we have an IMG element? [dive into mark]

2009-11-04T02:48:16Z

What every developer should know about time - Windward Wrocks

2009-11-03T17:35:51Z

Holiday Lighting Win

2009-11-03T15:00:00Z

Picture by: JackBrimstone Submitted by: JackBrimstone via Fail Uploader

Update On FeedDemon Creator, Nick Bradbury

2009-11-02T15:00:00Z

Many of you may know that in 2005, NewsGator acquired FeedDemon, an RSS reader for PCs that was then integrated into the entire NewsGator RSS ecosystem. When we acquired FeedDemon its creator, Nick Bradbury, officially became a NewsGator employee.

Since then, our enterprise business has grown faster than anticipated and NewsGator has started to focus increasingly on Social Sites for enterprise organizations. Because of that, we recently made the decision to shut down NewsGator online and have our RSS readers sync with Google Reader.

As a result of this change, Nick Bradbury will return to being an independent developer. However, FeedDemon is not going anywhere and it will remain a NewsGator product. In fact, Nick has lots of exciting enhancements planned for FeedDemon.

We'd like to thank Nick for his tireless work and significant impact to NewsGator, and we are appreciative of the incredible contributions he has made to NewsGator's RSS and social computing products.

You can read some reflections about Nick's time at NewsGator from our CTO and co-founder, Greg Reinacker here.

NewsGator gives full control of FeedDemon back to developer

2009-11-02T12:47:32Z

Talking to DC

2009-10-29T14:47:43Z

Warning. This is a rare nerdy technical post more for. It is about Healthcare XML standards.

I’ve was kindly asked to testify at a meeting in DC this week about standards at an hour when I’m normally not awake. But despite a deep aversion to not getting enough sleep, I was up and on the phone. What made me do such a thing? Well, the discussion was about what actually will work in terms of making health data liquid. What standards should be used for the integration of such data?

Somewhat to my surprise and usually to my pain, I’ve been involved in several successful standards. One was used to exchange data between databases and consumer applications like spreadsheets and Access. It was called ODBC and worked surprisingly well after some initial hiccups. Another was the standard for what today is called AJAX, namely building complex interactive web pages like gmail’s. Perhaps most importantly there was XML. These are the successes. There were also some failures. One that stands in my memory is one called OLE DB which was an attempt to supplant/replace ODBC. One that comes close to being a failure was/is the XML Schema specification. From all these efforts, there were a few lessons learned and it is these that I shared with DC this Thursday. What are they?

Keep the standard as simple and stupid as possible. The odds of failure are at least the square of the degrees of complexity of the standard. It may also be the square of the size of the committee writing the standard. Successful standards are generally simple and focused and easy to read. In the health care world, this means just focus first on that data which can be encoded unambiguously such as demographics, test results, medicines. Don’t focus on all types of health data for all types of health. Don’t focus on how to know if your partner should have access to what (see points 2,3, and 4 below).
The data being exchanged should be human readable and easy to understand. Standards are adopted by engineers building code to implement them. They can only build if they can easily understand the standard (see above) and easily test it. This is why, in the last 15 years, text standards like HTTP, HTML, XML, and so on have won. The developers can open any edit editor, look at the data being sent/received, and see if it looks right. When Tim Berners Lee first did this on the internet, most of the “serious” networking people out there thought using text for HTTP was crazy. But it worked incredibly well. Obviously this worked well for XML too. This has implications. It isn’t enough to just say XML. The average engineer (who has to implement these standards) should be able to eyeball the format and understand it. When you see XML grammars that only a computer can understand, they tend not to get widespread adoption. There are several so-called XML grammars that layer an abstract knowledge model on top of XML like RDF and in my experience, they are much harder to read/understand and they don’t get used much. In my opinion Hl7 suffers from this.
Standards work best when they are focused. Don’t build an 18 wheeler to drive a city block. Standards often fail because committees with very different complex goals come together without actual working implementations to sanity check both the complexity (see point 1 above) and the intelligibility (see point 2 above). Part of the genius of the web was that Tim Berners-Lee correctly separated the protocol (HTTP) from the stuff the browser should display (HTML). It is like separating an envelope from the letter inside. It is basic. And necessary. Standards which include levels or layers all jammed into one big thing tend to fail because the poor engineers have to understand everything when all they need to understand is one thing. So they boycott it. In health care, this means don’t include in one standard how to encode health data and how to decide who gets it and how to manage security. If all I, as an engineer, want is to put together a list of medicines about a patient and send that to someone who needs it, then that’s all I should have to do. The resulting XML should look like a list of medicines to the me. Then, if it doesn’t work, I can get on the phone with my opposite number and usually figure out in 5 minutes what’s wrong. Also I can usually author this in a day or two because I don’t have to read/learn/understand a spec like a telephone book. I don’t have to have to understand the “abstract data model”. The heart of the initial XML spec was tiny. Intentionally so. I heard someone say indignantly about the push to simplify Health IT standards that we should be “raising the bar on standards” not lowering them. This is like arguing that we should insist that kids learn to drive an airplane to walk to the next door neighbor’s house. All successful standards are as simple as possible, not as hard as possible.
Standards should have precise encodings. ODBC was precise about data types. Basic XML is a tiny standard except for the precise encodings about the characters of the text, Unicode. That is most of the spec, properly so, because it ensures that the encodings are precise. In health care this means that the standard should be precise about the encodings for medicines, test results, demographics, and conditions and make sure that the encodings can be used legally and without royalties by all parties. The government could play a role here by requiring NPI’s for all doctor related activities, SNOMED CT for all conditions, LOINC for all labs, and some encoding for all medicines (be it NDC, rxNorm, or FDB) and guaranteeing that use of these encodings is free for all use.
Always have real implementations that are actually being used as part of design of any standard. It is hard to know whether something actually works or can be engineered in a practical sense until you actually do it. ODBC for example was built by many of us actually building it as we went along. In the health care world, a lot of us have built and used CCR as we go, learning what works and what doesn’t very practically and that has made it a good easy to use standard for bundling health data. And the real implementations should be supportable by a single engineer in a few weeks.
Put in hysteresis for the unexpected. This is something that the net formats do particularly well. If there is something in HTTP that the receiver doesn’t understand it ignores it. It doesn’t break. If there is something in HTML that the browser doesn’t understand, it ignores it. It doesn’t break. See Postel’s law. Assume the unexpected. False precision is the graveyard of successful standards. XML Schema did very badly in this regard. Again, CCR does fairly well here.
Make the spec itself free, public on the web, and include lots of simple examples on the web site. Engineers are just humans. They learn best by example and if the standard adheres to the points above, then the examples will be clear and obvious. Usually you can tell if a standard is going to work if you go to a web site by the group and there is a clear definition and there are clear examples of the standard that anyone can understand. When you go to the HL7 site the generality and abstraction and complexity are totally daunting to the average joe. It certainly confuses me. And make no mistakes. Engineers are average joes with tight time deadlines. They are mostly not PhD’s.

Let’s be honest, a lot of standards are written for purposes other than promoting interoperability. Some exist to protect legacy advantages or to create an opportunity to profit from proprietary intellectual property. Others seem to take on a life of their own and seem to exist solely to justify the continued existence of the standards body itself or to create an opportunity for the authors to collect on juicy consultant fees explaining how the standard is meant to work to the poor saps who have to implement it. I think we can agree that, whatever they are, those are usually not good standards. Health data interoperability is far too important an issue to let fall victim to such an approach.

Attention lust and Olympic craziness

2009-11-01T10:22:00Z

For many organizations and individuals, attention is the most precious resource. The pursuit of attention for our ads, or our city or our careers dominates all else.

How else to explain the silly math that is used to justify Olympic hoopla? Can imagine how little patience people would have for the IOC and their internal politics if they didn't have a show that so many people wanted to watch?

Almost every city that has hosted an Olympics regrets it financially. The TV networks spend billions. The advertisers pay for it. The hoopla is vast and loud.

For what?

For the attention. It's the attention that gets cities to put up with the ridiculous system for choosing host cities and gets the TV networks to ship camera crews half way around the world. It's the attention that turns the Olympic committee into vigilant trademark and copyright police. It's easy to cut countries or companies willing to bankrupt themselves for pride or attention a little slack. After all, the Olympics is a magical event.

Except it's not. The same craving for attention happens every day in every organization in search of just one more pair of eyeballs. As marketers discover that more eyeballs does not equal better, the quixotic quest for attention will start to abate.

The formula is simple but depressing: marketers have been lousy at harvesting attention because there was just so much of it. So it was more like strip mining than careful, efficient use of a natural resource. Now that attention is harder to get, people are overpaying for it and the Olympics is just one example. The alternative is to create focused, intense networks that ignore the masses. For most marketers, that's exactly what we need.

Mediactive » The Only ‘Journalism’ Subsidy We Need is in Bandwidth

2009-11-01T02:49:32Z

Interview with me in Computerworld

2009-10-31T23:55:01Z

Seth Weintraub interviewed me for Computerworld Blogs. I talk about NetNewsWire, feeds in general, tablets, my own reading process, and so on.

Privacy

2009-10-30T13:15:01Z

It's hard to be a teenager and get away with anything these days. Parents can determine from the phone bill who the teens have texted and when. Parents can even read the teen's text messages if the phone is left unattended. Parents can see e-mail messages, check what web sites have been visited, and stalk their kids via Facebook.

In some cases the parents can already track their kids via GPS devices in cars and phones. You know that trend will increase.

Yes, teens have countermeasures and workarounds. But that's a lot of effort, and it's hard to hide all the electronic clues of, for example, an unapproved association. Even if you hide all of your own electronic footprints, you could still pop up on someone else's Facebook page.

This got me thinking about privacy issues in general. Most people reflexively believe privacy is a good thing, and a lack of privacy is a bad thing. But what if privacy creates more problems than it solves?

Let's say you have a secret carnal desire for broccoli. In our current world, where privacy is still somewhat attainable, you hide your dirty little broccoli secret. If anyone were to find out, you'd be ostracized and mocked. So you carry your little secret around like a bag of shame, sneaking trips to the grocery store to get a fix.

Now imagine a world where no one has any privacy and your inappropriate desire for broccoli becomes common knowledge. Suddenly all the other broccoli lovers know you are one of them. You start hanging out together, sharing your broccoli stories. You make new friends. You are understood. It's a relief in many ways.

In a world with no privacy, no one will seem like a freak because so many people will appear to be one type of deviant or another. In that world, the biggest losers would be the people who have totally uninteresting flaws and passions. They would seem boring.

Like it or not, that world is probably coming.

Vaccines

2009-10-29T23:46:09Z

An interesting link on Daring Fireball today has me thinking about vaccines.

I’m still living with the effects of the chicken pox I had in third grade.

There was no vaccine then. Every kid just got it. It swept through school, and nobody tried too hard to prevent the spread, because every kid would get it, and it was better to get it when you were young.

It was just a thing. We thought we were modern because it was just chicken pox — not polio or smallpox or one of those scarier diseases that had been conquered.

But now there is a vaccine, and I wish like crazy there had been a vaccine when I was a kid.

I was nearly hospitalized — my Mom tells me I was within an hour or so of having to go the hospital when I could finally sip a few drops of water without vomiting.

I remember vomiting so much that the vomiting itself didn’t even bother me any more. I started crying out of frustration. Just when I started to feel a little better, a little cooler, and hungry and thirsty, I’d try the smallest sip of water, and whatever was left in me to come up would come back up. It just went on and on.

For days? I don’t know. It seemed like weeks of nights. Trying to sleep. Itchy, exhausted, unable. The screaming heat inside that wouldn’t end. Then the stomach convulsions.

I was told I had chicken pox not just on my skin but inside me, too. Could that be true? Was that possible? I still don’t know — but I didn’t question it, because it sure felt like it.

Of course I had chicken pox on my head. So I had pink hair — pink with the Calamine Lotion my Mom applied. The pink was mortifying to a third-grade boy.

A friend of my Mom’s, a woman from down the street, came by one day. (She was cool: she drove a white Corvette. Here’s one.) I can still feel the embarassment of my pink hair. The spots all over my face I could deal with: the pink hair was devastating.

I was finally able to drink water, and then I moved up to jello and then on to chicken noodle soup. I was able to sit at the dinner table and watch the network news. (ABC news broadcast from WPVI in Philadelphia.)

I got better. After about three weeks of the chicken pox, I could return to school, even though still slightly scarred. Prepared to be teased.

I was the smartest kid in the grade. I won every spelling bee. I got 100% test scores. I always had.

And though I was prepared to be teased for the visible remains of my chicken pox, I was not prepared to get in trouble with my teacher for cheating.

Cheating? Me? People cheated off me, not the other way around.

But I was looking at someone else’s paper. Because I couldn’t see the chalkboard anymore and I couldn’t read the questions to copy them down.

I kept getting in trouble, and I kept getting teased, and I was angry and hot-headed a whole bunch of the time. I started getting in trouble for accidents, and for things other kids had done. (Which wasn’t that new, actually: I was in trouble at school, or about to be and dreading it, most of the time as a kid.)

It was a few weeks before news got to my parents and they took me for an eye exam.

Chicken pox had ruined my eyesight.

So I got glasses, and that worked. But my vision, once gotten a head-start down that path, kept getting worse for a few years.

Though it pretty much stabilized while I was in high school, it got bad enough that it’s dangerous for me to walk around my own house with my contact lenses out.

I can tell you that my staircase has 16 steps. (Easy number for a programmer to remember!)

To read with my contacts out, I have to hold the book so close that I have to close one eye.

I can look down at my feet and not see the cat.

My parents were both near-sighted — they both got glasses in the eighth grade. Their eyesight is better than mine. Odds are I would have gotten glasses around the eighth grade too, and had eyesight about like theirs. Not great, but not terrible.

I wish, to this day, that there had been a chicken pox vaccine.

I later got shingles when I was 20. I won’t be surprised to get it again, but I sure hope not. Shingles hurts.

So I was out about three weeks from school. I hated school anyway.

But we were doing a special unit on the history of native Americans, which I thought would be pretty cool.

I missed some arts and crafts things — creating a diorama, carving a miniature dugout canoe out of Ivory soap. I was just as glad to miss that stuff, as I liked reading and writing better.

I also missed out learning about the history of native Americans, though I did pick up some later.

Which brings me back to the subject of vaccines. And, you know, I thought I was going to, but I don’t really need to state the obvious.

Why I Continue to Use Google Reader « The SiliconANGLE

2009-10-30T00:58:21Z

“The Who: Rock Band” On the Way? Daltrey Drops a Hint

2009-10-29T15:48:30Z

Photo: Arnold/WireImage

The Who may be the next band to receive their own branded Rock Band video game. Singer Roger Daltrey let the following tidbit slip in an interview with MassLive: “The game, yeah, yeah, they’re going to be doing a Who one next year. There is one planned. [The idea] is fabulous. Anything that gets non-musical people interested in music is wonderful.” So far, only the Beatles have their own Rock Band title; Guitar Hero has games pegged to acts including Aerosmith and Metallica.

Check out video game avatars of rock’s biggest names.

A Rock Band spokeperson told video game blog Kotaku, “We’re working closely with the Who on what’s next, but don’t have anything new to announce at this time.”

Given the cinematic and conceptual nature of the Who and Pete Townshend, Rock Band seems like a better fit for the Rock and Roll Hall of Fame band than Guitar Hero. Whereas GH has created lifelike avatars and restored famed venues for their artist-based games like GH: Metallica or GH: Van Halen, MTV Games upped the ante with The Beatles: Rock Band, crafting entire dreamscapes to accompany the music and offering replicas of the Fab Four’s instruments, an approach that would work well with the Who’s concept albums like Tommy and Quadrophenia.

Check out photos of the Who and more classic shots from Jim Marshall’s new book.

Unlike the Beatles, however, the Who have already offered up a number of their songs for preexisting Rock Band games: RB featured “Won’t Get Fooled Again,” RB2 had “Pinball Wizard” and July 2008 saw the The Best of the Who bring a dozen of the band’s hits to Rock Band as downloadable content.

FeedDemon, NewsGator, and Mr. Bradbury

2009-10-29T15:00:31Z

Way back in 2005, NewsGator acquired FeedDemon. I vividly remember sitting down with Nick Bradbury and talking about our shared vision for the future of FeedDemon and the NewsGator platform…and many a night drinking a lot of beer and talking about RSS and what we now call social computing. Through it all, Nick remained laser-focused on both the future of FeedDemon, and his customers and how they would be brought forward.

Along the way, we added NetNewsWire to the family, as well as SmartFeed (renamed NewsGator Go! for Windows Mobile). All of it was part of building out the original vision for a core online content platform, and best-of-breed applications on nearly any device to consume content. All in all, I think we delivered on this vision – we built the platform, added the best applications on the most popular platforms, and made it all work together.

Since then, as you may have gathered, our enterprise business has grown faster than we anticipated (this is a good thing!). As the company has started to focus more and more on enterprise customers, we made the difficult decision to shut down NewsGator Online, and focus our online platform in on our commercial clients. As part of this transition, most of our client applications (FeedDemon, NetNewsWire, and NetNewsWire for iPhone) were re-released to sync with Google Reader as an online store, rather than NewsGator Online.

As part of that transition, Nick has gone “back into the wild” as an independent developer. FeedDemon remains a NewsGator product, but Nick is 100% focused on it, and he has complete control over the product direction and feature set. I like to think of this not as the end of anything, but rather the beginning of the next phase of FeedDemon’s life. Just as the initial deal with NewsGator opened up new opportunities for it, so does this new direction…and FeedDemon customers will see lots of exciting things coming up.

On a personal note, I’d like to extend a big “thank you” to Nick, on behalf of myself and everyone at NewsGator. Nick has been instrumental in forming our consumer product direction, always makes sure we’re taking care of our customers in the best way we can, and has provided a huge amount of input on our other products. He’s been a very influential person at the company, and we look forward to this continuing in the future!