Connections

We're going to have some exciting discussions at ODCA Forecast 2012!

noreply@blogger.com (Gordon Haff) — Wed, 30 May 2012 00:50:00 +0000

Manhattan is going to be cloud computing central from June 11 to 14.

Cloud Expo is something of the center of gravity, but there's plenty else going on. CloudCamp NYC is on the evening of June 12. It's free and they're always great events with lots of interaction. On Wednesday, Rishidot Research is organizing DeployCon, an Enterprise PaaS Summit. This new event cuts right to one of the hottest topics in cloud computing--bringing together the developer-friendliness of PaaS with the operational needs of enterprise IT.

I'm going to try to spend some time at all three of those events, but what I'm most focused on is the Open Data Center Alliance's Forecast 2012 conference on Tuesday. The ODCA is a consortium of major IT organizations including, notably, large end-users; it's no vendor smoozefest. The idea is to identify customer requirements and to influence industry innovation to address those requirements. One such example is virtual machine interoperability, as is demonstrated in this Red Hat video using our CloudForms open hybrid cloud management software.

The event has a great lineup. For my part, I'll be a panelist at a pair of Forecast panels: one on software innovation and one on regulation. I'm excited about these panels for two reasons.

The first is that the organizers are doing a bang-up job of doing their best to ensure that these panels don't embody all those things that make us dread panels. You know what I mean. By the time everyone is done clearing their throats and telling you how smart they are, there's only time left for all the panelists to give more or less the same long-winded answer to a couple of desultory questions. No panelist slides on these ones. And I can assure you that the panelists are now quite familiar with terms like "rapid fire" and "interactive" as we've gone through our prep calls. The sad thing is that panels often have great potential. That potential is just so rarely achieved. I'm hopeful for these ones.

The other is that the topics for the two panels on which I'll be sitting are really interesting to me. I'm not going to steal my thunder in advance here, but I wanted to share a few thoughts in advance.

Rapid Fire Panel: Cloud Regulation (2:35-3:20)

Moderator: Deborah Salons

Panelists:

Brett Smith, Deutsche Bank

José E. González, Chief Business Development Officer, Trapezoid Digital Security Services, LLC

Gordon Haff, Cloud Evangelist, Red Hat

Marvin Wheeler, Chairman and Secretary, Open Data Center Alliance

My thoughts:

There are a lot of aspects to regulation but, given that I am neither a lawyer nor a governmental affairs expert, my real interest here is where regulation and technology interest. From the perspective of cloud computing--and cloud computing within large organizations in particular--my concern lies with questions such as how appropriate policies can be embedded in applications and control mechanisms so that automated processes don't run afoul of regulatory regimes. This is an important question because automation means "Hands Off!" Start interjecting manual processes to deal with regulatory requirements and you can't realize the benefits of automation.

Cloud Software Innovation Panel (1:50-2:35)

Moderator: Richard Villars, Vice President, Information & Cloud IDC

Panelists:

Elad Yoran, Chairman and CEO, Vaultive, Inc.

Gordon Haff, Cloud Evangelist, Red Hat

Greg Brown, McAfee

John Engates, CTO, Rackspace

Reuven Cohen, Senior Vice President, Virtustream

My thoughts:

This should be an interesting discussion. On the one hand, some question whether open source matters in the cloud even while almost all of the top public clouds have open source as their foundation. Their argument revolves around the fact that availability of source code doesn't have the same meaning in a world where software services are often delivered over the network and are intimately tied to the data and compute infrastructures on which they run. However, I'd argue that open is both a bigger and a broader issue in cloud computing as I wrote in Why the Future of the Cloud is Open.

How does this relate to software innovation? It relates because, even if open source historically was often about cheaper substitutes for expensive proprietary software, it's now more and more about fostering innovation through communities, including user communities. Is open source the only mechanism through which innovation can happen? Of course not. But it's a powerful mechanism as are other aspects of openness such as open APIs.

Links for 05-25-2012

noreply@blogger.com (Gordon Haff) — Fri, 25 May 2012 14:44:00 +0000

Doc Searls Weblog · After Facebook fails - "But totally personalized advertising is icky and oxymoronic. And, after half a decade or more at the business of making maximally-personalized ads, the main result is what Michael calls “the desultory ticky-tacky kind that litters the right side of people’s Facebook profiles.”"
Kodak was never going to be the Kodak of digital photography | Challengers - CNET News - "It's also not as if some other company has emerged as the Kodak of digital. In the company's golden age, consumers used a Kodak camera to take pictures on Kodak film, which they then had processed by a Kodak lab on Kodak paper. Today, no single company has replicated that ecosystem in digital form. There are camera manufacturers, and memory-card manufacturers, and printer companies, and photo-sharing sites--and all of these businesses benefit from healthy competition among multiple major players, rather than the monopolistic position that Kodak once enjoyed."
Facebook IPO Post Mortem – Killer – but not for the reasons you think ! « blog maverick - Cuban hits some key points. It's even harder to monetize with ads in mobile and he (I believe correctly) notes that we're moving away from all-you-can-eat mobile data which will make the situation even worse.
Stealing the Shows - TIME - RT @poniewozik: Fox suing over Auto Hop ad-skipper. I have a (pre-suit) column on it this week: my life as a TV thief.
Secret memo reveals which telecoms store your data the longest | Ars Technica - Not sure I buy that only Verizon retains text message contents. The others must store it somewhere for some length of time in order to transmit it, e.g. to phones that aren't currently on.
The Facebook Fallacy - Technology Review - Some good discussion in comments here about limitations of gaining insights from even large qtys of data.
Java creator unhappy with Oracle trial outcome | ITworld - Seems to sum things up pretty well (though I'd leave out the word 'sadly'). -- "Based on testimony in the trial, and remarks from Gosling and others, it's clear that whether they meant to or not, Google very much irked people at Sun Microsystems when Google decided to bypass Java and go with a clean-room implementation of Java in the form of the Dalvik VM in Android. And "irked" is probably an understatement. But sadly, you can't really sue people for being jerks. And there's also the argument that Sun may have forced Google's hand by dual-licensing Java under the GPL and a proprietary license for commercial use. That was certainly Sun's prerogative, of course, but it doesn't completely jibe with their much-touted "Java is free" mantra."
Create Custom Skins for Laptops & Netbooks | GelaSkins - This is very cool. You can create laptop skins with your own photos.
Update: Mike Lynch leaves HP Autonomy - ComputerworldUK.com - RT @maslett: "the entire management team and 20 percent of all Autonomy's staff have left since the HP takeover"
Opposing the New York Public Library - The Daily Beast - RT @thedailybeast: A battle for New York Public Library's (@NYPL) soul prompts the question: Who's reading the books?

Links for 05-23-2012

noreply@blogger.com (Gordon Haff) — Wed, 23 May 2012 19:57:00 +0000

Jonathan Ive interview: Apple's design genius is British to the core - Telegraph
Skilling up for Cloud architecture & planning | The Cloud Evangelist
Sheryl Sandberg Is The Valley's 'It' Girl - Just Like Kim Polese Once Was - Forbes
OSI Board Elects New Officers | Open Source Initiative - RT @richsands: “@robilad: "The board is happy to announce that ... Simon Phipps is the new President." ” Congrats @w ...
The hype cycle of Vendor Briefing Requests « The IIAR Blog - RT @jpuppet: RT @cloudpundit Hype cycle of Vendor Briefing Requests <- hilarious .. (analysts get frustrated too ...
(Humor) The Rosetta Stone of IT Industry Analysts | ZDNet - RT @IIAR: (Humor) The Rosetta Stone of IT Industry Analysts | ZDNet
The Levelator® from The Conversations Network - Levelator did a nice job of fixing up a podcast I recently recorded with very uneven levels
Identity Management in Red Hat Enterprise Linux - YouTube - A new (short) video about identity management in Red Hat Enterprise Linux
The-Cloud-Computing-Debate-Cutting-Through-the-Cloud-Clutter - CA Technologies
30 Minute Video Limit in Digital Cameras May Be On Its Way Out - RT @petapixel: 30 minute video limit in digital cameras may be on its way out: << tariff related
Oracle Itanium Exhibits Chronological
How to Scan Film Negatives with a DSLR
What's next after GPL and Apache? - "While the newest open source projects such as OpenStack and OpenShift have chosen to use the Apache License, a non-copyleft or "permissive" open source license with good patent protections, I believe in time we will see the licensing trend for new open source projects targeted at commercial collaboration swing back to the center, away from either the GPL or Apache extremities. Until recently, none of the mainstream "center ground" open source licenses has been optimal. These are the Mozilla Public License (MPL -- and its many "vanity name" variants), the Eclipse license, the CDDL (used widely by projects influenced by Sun Microsystems), and of course, the LGPL."

Podcast: Kurt Milne discusses how organizations are building clouds

noreply@blogger.com (Gordon Haff) — Thu, 17 May 2012 19:17:00 +0000

Kurt Milne is the managing director of the IT Process Institute and has been surveying organizations about their cloud adoption. The most common strategy is to leverage existing resources where possible by taking an open, hybrid approach. Kurt is also co-author, along with Andi Mann and Jeanne Morain of Visible Ops: Private Cloud. Kurt discusses:

Strategies for cloud adoption
How open cloud approaches are proving most popular
How organizations are achieving agility through self-service
Owning vs. renting capacity
How IT decision makers need to look at cloud through a framework of cost, benefit, and risk
The need for a portfolio view

Listen to MP3 (0:14:44)
Listen to OGG (0:14:44)

[TRANSCRIPT]

Gordon Haff: You're listening to the Cloudy Chat Podcast with Gordon Haff.

Gordon: Hi, this is Gordon Haff, cloud evangelist with Red Hat and I'm out here in Silicon Valley at the Open Cloud Conference. I'm here with Kurt Milne, who is the managing director of the IT Process Institute. Hi, Kurt.

Kurt Milne: Hello. Glad to be here.

Gordon: Kurt, could you maybe tell us a little bit about yourself and the IT Process Institute?

Kurt: The ITPI is an independent research organization. We use empirical evidence‑based studies to try to identify who are the top performing IT organizations and what they do that's different from other folks. We do studies. We write white papers, prescriptive patterns. We also self‑publish the Visible Ops books. A lot of folks have seen the little yellow and black Visible Ops book. We've got Vis Ops Security, and also most recently is Visible Ops Private Cloud.

Gordon: Kurt, you've started doing some survey work around cloud adoption. Rather than just getting opinions about where things are going and coming up with some numbers, this study that you've been working on really tries to correlate what people are doing with the results they are seeing, which seems pretty interesting. I know it's still pretty early for the analysis of data and finalizing things, but I'm wondering if you could share some of the interesting things that you're finding.

Kurt: We fielded a survey through the folks at Cloud Camp and also at some other sources. We got responses from about 150 companies that have deployed private or private‑hybrid cloud past the proof‑of‑concept stage. As you mentioned, the goal is to try to look at what were people actually doing. What were some of the pre‑conditions before they started their cloud project? What were some of the key dependencies during their project? Then, what were the results that they achieved? Both positive results and project friction points. The idea is to try to figure out what are those combinations of factors that contribute to cloud project success.

Gordon: Maybe you could share to start off with, have you found any surprises so far?

Kurt: Well I think one of the surprises is that there are a lot of organizations, about 40 percent of the respondents, that suggested that their primary cloud strategy or goal is to build an open cloud or a private‑hybrid cloud that leverages their existing assets as much as possible. Another strategic option would be to build what I call a siloed cloud, which would be maybe a cloud that's carved off in the data center for dev and test environment, self‑serve resources for developers, that sort of thing. But the primary strategy was that open cloud, to try to leverage assets as much as possible, and then, tap externally acquired.

Gordon: That seems to fly a little bit about some of the conventional wisdom at least that we seem to hear out here in Silicon Valley. That the cloud is new and enterprise IT is old, and that the cloud is all about really starting afresh in a greenfield.

Kurt: Well, I think if you step back and look at it from the CFO or controller's perspective and say, "Why are we renting computing assets from a third party, when we've got underutilized assets internally?" I think that story of, "Let's do what we can with what we've got before we tap external resource pools" ‑‑ I think that still makes business sense.

Gordon: Great. Yeah. This isn't to say, of course, that people are not making use of Amazon, or they're not putting in new servers, but it really does seem to speak to this idea that we can't just afford to throw everything away and start with a clean sheet of paper. That would be really appealing as an enterprise architect, but it's probably not practical for a lot of enterprises.

Kurt: Well, there were about 10 percent of the folks that filled out the survey that we got data on that did indicate that that was their cloud strategy, that they're some kind of service provider, that they're building a cloud solution, and that they're not encumbered by existing legacy applications. There are folks out there that have that luxury of being able to start from scratch, and just look at their requirements and not as many constraints, but most organizations have constraints already in place.

Gordon: Among those enterprises that are building open clouds, open‑hybrid clouds, why are they doing it, and how are they achieving success?

Kurt: Well, the use cases we looked at that were most frequently deployed by the 40 percent of the companies we looked at that are in that open‑cloud category, the self‑service development and test environments, the self‑service resources ‑‑ so that's really helping achieve an agility state, where folks can tap and get self‑service, on‑demand access to things that maybe they had to go through operations and wait for it previously. There's a self‑service agility element there. But then they're also using the cloud for basic blocking and tackling operations, things like backup, high‑availability disaster recovery as well. Then, we're also seeing a lot of interest, based on the survey, in scale‑up and scale‑out capabilities that may be difficult in a more traditional, static IT environment.

Gordon: It's interesting, although it's very consistent with a lot of other data we've seen, that cost savings does not seem to be the primary driver here. Not many a CIO is going to tell you, "Oh, I can spend as much money as I want," but that doesn't really seem to be the driving force behind people adopting cloud.

Kurt: I agree with you. I think there are some efficiency gains, when we look at what were the results of your cloud effort ‑‑ more development efficiency, more run‑time or operations efficiency. I think there are improvements there, but it doesn't really suggest that companies are tapping Amazon and others just for the cost‑savings aspects. It's really the process efficiencies in combination with those agility factors that I think is driving the adoption.

Gordon: Let's talk about hybrid clouds. I think sometimes "hybrid" has been taken to mean this auto‑magical, super‑fast, dynamic shifting of workloads among clouds, which, frankly, I haven't seen happening very much. But I do see, still, a lot of interest in being able to move between clouds, even if it's done at an administrative level as opposed to an automatic, load‑balancing way.

Kurt: Yeah. Interestingly, I think in the last couple of years, as cloud has gotten a lot of attention, people have talked about this bursting concept, where you're going to have a workload internally. Then if you have an unexpected or even a planned usage spike that you'll be tapping external resources, but that was the lowest prevalence. When we asked what folks were using their cloud for, that had the lowest percentage of companies indicating that that's what they were using it for. Whereas starting in the cloud as a prototype environment, doing dev and test work and then, once the workload stabilized, bringing it back in‑house, had a much higher rate of response versus bursting out.

Gordon: That seems to be consistent with particularly what some companies, like Zynga, for example, are doing, this idea that you own the base and you rent the peak.

Kurt: I agree. I think that is the Zynga model. I think it becomes an owning versus a renting kind of decision, right? In some cases, it makes sense to rent resources, and then, in other cases, it makes sense to purchase it and utilize the resources. I think, in Zynga's case and a lot of the respondents in the survey, there are times when you rent resources, but if it becomes stable and predictable, then it makes sense to buy the assets needed to support those over time.

Gordon: The interesting point you're making is what you're saying is this becomes, essentially, a financial or a capital‑budgeting decision at that point, OPEX versus CAPEX. When a company decides to rent or lease or buy company cars, for example, they don't have to get a different kind of car, depending upon what their financial model looks like, and that hasn't always been the case in the IT industry.

Kurt: It's interesting you mention that because I know with Zynga, one of the driving factors of their decision to build a private cloud was that they wanted some capital assets that they could depreciate. I think there's always this talk in IT of operationalizing costs, converting from a capital‑asset cost to an ongoing operations cost. In their case, with no assets, a completely virtual company, if you will, they were actually looking at getting assets to depreciate as one of the drivers for building something, which I thought was opposite of what a lot of people talk about.

Gordon: The other thing you see in this idea of pay‑per‑use and all. People and companies are very big on pay‑per‑use, as long as it means they pay less than they would have otherwise. They're not so big on pay‑per‑use if it means paying more.

Kurt: I think the other aspect to that is I think companies want to pay‑per‑use if that cost is tied to revenue. If you've got a model where, adding an incremental customer, you have an incremental cost associated with delivering service for that customer, I think the financial folks like that kind of arrangement. In some cases, they're willing to pay more for service, if that service is tied to revenue that's scaling up and down versus maybe paying less for a fixed asset, where that cost isn't tied to revenue. I think that goes into the decision, as well.

Gordon: That's also a fairly traditional part of corporate finance. You're willing, at some level, to pay at least a small premium for being able to match up your revenue streams and your costs streams. Basically, this isn't any different from that. Any final points you'd like to make?

Kurt: Well, I think it's interesting. We're entering the podcast, here, talking about more traditional management methods, but I really do see a lot of these cloud concepts enable new dynamic IT capabilities. IT decision makers continue to be very pragmatic and need to look at all of these things in a framework of benefit, cost and risk. I think these new capabilities have different cost and risk factors, but there's no magic here. It all needs to be viewed from a pragmatic lens, in order to make decisions on what's the best path forward.

Gordon: That's a great point. I think it's really exciting, a lot of this stuff that's happening in cloud, new capabilities, all of this API‑based, modular computing, self‑organizing, what have you. I think it's all great, mind you. I'm certainly not suggesting otherwise. But we do, at some level, need to balance all the new application style enthusiasm, by the fact that, in most cases, we have run the business applications that we probably don't want to just abandon or say they're old legacy apps that we really can't do anything with and not concentrate on them any longer. Those are the apps that are running the business, and, in many cases, there are things that open‑hybrid cloud management can do to make them more flexible, as well.

Kurt: Yeah, I think it becomes a portfolio‑view approach that organizations need to look at their whole suite of applications, determine what the best fit is from an environmental standpoint. They can't throw out legacy apps. The apps are being used by folks in the business to do business functions. I think creating some framework of being able to evaluate what's the business objective? What are the constraints? What are the architectural options that make sense across a whole range of physical, virtual and cloud environments? I think it's going to be a mixed model. I don't think any one of those is going to prevail or any one of those totally go away.

Gordon: You go into quite a bit of methodology associated with that in your last book.

Kurt: That's right. The plug for the book, the "Visible Ops Private Cloud: From Virtualization to Private Cloud in Four Practical Steps," was based on interviews of about 30 IT organizations that had deployed some form of private cloud. We were really trying to capture some of the lessons learned. What do you know now that you wish you had known at the beginning of the project? What were really the key success factors? Then, put those into a step‑wise methodology that we think any organization should at least consider some of the factors there as they develop their cloud strategy.

Gordon: Great. Well, thanks very much, Kurt. Been good talking to you.

Kurt: All right. Thanks, Gordon. I enjoyed it.

Gordon: Bye everyone.

Links for 05-15-2012

noreply@blogger.com (Gordon Haff) — Tue, 15 May 2012 19:34:00 +0000

Making Choices in the Age of Information Overload - NYTimes.com - "The Internet is, among other things, a massive, chaotic marketplace. Too much information, it turns out, is a lot like no information. “If we researched every single purchase, we wouldn’t have time to make any purchases,” says Anna Kirmani, a marketing professor at the University of Maryland. “I have better things to do with my time.” Signaling can be a shorthand to identify whom you want to buy from. That’s why we may need it now more than ever."
The Fertile Unknown: Improv Theater and Complex Adaptive Systems
Death of a Saleman: A Heartbreaking Work of Staggering Mediocrity : The New Yorker - "Ten years later, at the Ethel Barrymore, I found myself squirming in my seat from boredom and exasperation, amazed at how much glaringly conventional stagecraft “Salesman” was able to pack into its two acts. The rising action, the dramatic irony, the laborious, grandstanding speeches (“Spite, spite, is the word of your undoing…When you’re rotting somewhere beside the railroad tracks, remember, and don’t you dare blame it on me”)—I kept wanting to exclaim, “It sounds like a play!”"
How Yahoo Killed Flickr and Lost the Internet - Quite the read, though IMO somewhat overstated.
Anatomy of an iTV rumor - Apple 2.0 - Fortune Tech - So many "column inches" in the online tech press is devoted to endlessly regurgitating these sorts of rumors. Cheap clicks.
VMware pros glean vSphere roadmap from CTO in VMUG videos - "“Those of you who know our products deeply know that they don’t fit as well together as they need to,” Herrod told the VMUG attendees. “Some have multiple databases. Some don’t look the same. Some install differently. And what I can’t stand is that currently Site Recovery Manager doesn’t work with vCloud Director.”"
Do I Build or OEM My Cloud Service? The Top 5 Considerations | BostInno
Bits or pieces?: On the death of great companies
More Americans Leave Passports at Home for Summer Vacation | BostInno - More domestic flying than international in summer Euro airfare this summer was really $$ when I looked
Expensify Trips: Track your itinerary from your expense report « Expensify Blog - Expensify is apparently trying to bridge to offering a Tripit-like service

Podcast: Complex adaptive systems and APIs with James Urquhart of enStratus

noreply@blogger.com (Gordon Haff) — Tue, 15 May 2012 15:35:00 +0000

Cloud computing requires a mindset that approaches system architecture as a much more distributed, heterogenerous, and even self-organizing entity than was the historic norm in IT. VP of Product Strategy and GigaOm blogger James Urquhart shares his thoughts on the topic as he discusses:

Complex adaptive systems
What high availability means in the cloud
The role of standards

Listen to MP3 (0:14:22)
Listen to OGG (0:14:22)

Transcript:

Gordon Haff: Hi everyone. This is Gordon Haff, Cloud Evangelist with Red Hat. I'm here at the Open Cloud Conference in the Bay Area. I'm sitting here with James Urquhart, who's the VP of product strategy for enStratus. Hi, James.

James Urquhart: How are you, Gordon? Good to see you.

Gordon: We've known each other for a while. You've had blogs in a number of places that I've also written on and currently on GigaOm, right?

James: Yeah. I'm a regular contributor to the GigaOm cloud section. I should be blogging more often than I do. You'll see me about every two to three weeks on GigaOm.

Gordon: I have the same problem, James, getting stuff written on a regular basis.

We've had some really interesting conversations about how the cloud is changing systems architecture. In fact, you had some really interesting thoughts about how to think about architectures with the cloud.
James: For a long time I've had an interest in the subject of complex adaptive systems. There's an entire science around a world in which there are many, many different individual agents that each have their own behavioral decision making process, whether that's DNA or whether it’s the economic space with buyers and sellers. And then, they interact in very arbitrary ways over a very large scale creating a system that ends up having its own emergent behavior as a system that comes with no central control of that behavior. That's just the way things work out as these agents work out.

If you look at cloud computing, what we're really beginning to do in a very large way is to step out of the silo world into much more of a heavily integrated world where the applications, the infrastructure, the services being delivered are all agents that are being very often decided by different people. A great example of that is I might have multiple agents as an enterprise running on Heroku, which in turn is running on Amazon Web Services.

And so, the behaviors of the systems are decided very independently. The subcomponents of the system are decided very independently. What you're beginning to see is that complex adaptive systems behavior slowly but surely starting to show up in IT in general, in computing in general, on the Internet in general. In part because cloud computing is an enabler of that.

What that means is, if you embrace and understand the complex systems piece of the puzzle, what you're really going to begin to see is a way to understand and to embrace the complexity of the system and to understand how to do your little pieces to make sure that your agents that you care the most about thrive and survive in that system.

I think that's really, to me, the critical shift in thinking. From trying to figure out how to build something that just works and will never break to building something that adapts to the environment and constantly is able to thrive within a changing environment.

Gordon: I think one way that's sort of an interesting way to think about that at a conference a couple weeks ago, someone got up, up and asked "Is there a way to get five nines reliability in the cloud?" And of course, you’re coming from among other things working in a large systems in the past...The traditional thinking there was that you had some sort of failover clustering capability among large Unix systems or among large mainframes whereas from a transactional perspective, stock transactions, whatnot, you got however many minutes of downtime a year equated to five nines.

And really, though, that's not the right question to ask in the cloud, is it?

James: No. In fact, there's a really, really interesting part of complex adaptive system science that's really just starting to come out now and be explored by academia in a large way. Now, there's actually a tradeoff between stability and resiliency. If you attempt to say "I want five nines by knowing exactly what my stack is and exactly how that stack works and that nothing is going to fail in that stack," or "If something fails, I know exactly how something else will come in and replace it. But I'm going to make sure that this thing is as stable as possible." The problem you have is there's a number of things that can come in from the environment that shift the ground underneath your designs so much that there's no way that your design can in fact adapt to that change and it will fail as a whole.

A resilient architecture is one much more where you say, "Look. The individual components each have to be able to not only survive the environment as it stands, but the individual components have to be designed in a way that as a horizontally scalable system, as a group of agents working together, that as changes happen in the environment that the system somehow keeps going. The subsystem somehow finds a way to at least meet a minimum set of capability that keeps the system going."

If you look at the way Amazon's designed, if you look at the way that Netflix is designed, this is exactly what they do. That front page of Amazon's not an application that’s made up of a whole bunch of pieces that are all designed to be stable. It's made up of a whole bunch of things where there's a whole bunch of failover and a whole bunch of different ways that data can be gathered.

So go to a cache. If the cache is gone, you go to the core data source. If that data source is gone, there's this other data source that will give you kind of a remotely good picture that you can adapt to. If that data source is gone, then you can say, "Well, I'm just not going to display that element of the page."
But the home page, that Amazon purchase page, is always there. When was the last time you went to Amazon and it was gone completely? That kind of resiliency...Right? Things fail all the time in Amazon, but that resiliency of the overall system gets you the appearance of five nines plus.
I think that that's the beginning shift of the mentality to say, "Rather than focusing on the component and making the component as stable as possible, focus on the relationships between components and how components work together and how can you build as much resiliency into the different relationships and the way things work together so that the system as a whole is in fact quite available, and quite resilient.”

Gordon: This sort of idea that we're just going to have these utterly standardized APIs that work together in lockstep and communicate with that way. That's really not the future. What we're really talking about architecting for this very heterogeneous environment where you need to sometimes translate from one thing to another, connect to things in a loosely coupled way.

James: I don't think standard APIs are the problem. I think the way to look at it though is, you're trying to find the patterns and you're trying to make sure that you can build to the patterns that work and to adapt and evolve those patterns over time. But I think there's a place for standard APIs. I think there's a place...Frankly, provisioning a server as an action, there's very little highly differentiated ways that you can provision a server. I think it's very fair to say that we're getting to a point for a Linux system working on an X86 environment, there can be a very, very standard way of doing that basic task.

But that's not the application. That's not the thing that solves a business problem up above.
I believe that there are standards in places that you can come to, but the idea that there's one stack that solves the problem is...And I don't think I've heard anybody really argue that it's all going to be this one big stack and everybody's going to move to this or they're non standard.

I think what you have to realize is that there are different components in the different stacks, that they give you different value. I think it's fine to talk about open standards for interfaces and then for formats, but I think when you go farther than that, when you try to say the stack is locked down, you have to do it this way with these sets of components. I think that that's the point that you break the model. I think that's when the market says "That's a broken model," and they do something different.

A great example, really quickly, about that is just when ITIL took off and companies started identifying ITIL and really, really being hip on it. DevOps shows up. Because ITIL was broken for some aspects of what the business wanted to do, DevOps is much more flexible in terms of the agility when you need agility. So, in fact, it is disruptive to what we thought was the commodity way of doing IT. It's always going to be that way.

Gordon: I think, really, if you look at the history of IT, you've got big monolithic approaches really have not done as well as more nimble, more modular approaches.

James: And that's exactly true. I think...There's a gentleman by the name of Simon Wardley who has great writing on this, where he talks about there are spectrums of business activities and there are times when you need to be highly, highly agile and there are times when you need to be locked down and to very closely control change and control adjustments. But what happens is you go through that cycle and get to the end of the cycle, to where things are little bit more like that. That enables a whole new set of innovation on top of that which, in the end, may trickle down and say, "Yep, we need to rethink the way that we're doing X."

It's being prepared and being able to understand that that constant churn is a fact of life. It's something you need to develop your processes with that concept in mind. And that the patterns and the toolsets and the infrastructure that we build out for the cloud is going to have to take that complex systems approach in mind, as well, and really begin to embrace this concept of focusing on the relationships between things more than focusing on the components themselves.

Gordon: And we do certainly seem to be shifting to an API driven world in a lot of ways. I guess a lot of people tend to think in terms of the Amazon APIs, and the Flickr APIs, and in many cases the more consumer oriented services. But more and more businesses, credit card processors, banks, what have you are starting to expose APIs, if not for general public use then the use of their partners.

James: Yeah, and I think what's really, really fascinating about that is why those APIs are being exposed. If it's of a surface that you give it some data and instruction and it returns something back to you that's of value. You're basically providing that service through an API instead of through human methods or whatever it might have been before. In other areas where you're saying, "Well, the API is really about how you consume another resource downstream." The problem that you have is the API isn't enough. And so, I think what you're going to see is giant success in terms of exposing business capability through APIs. I know of companies out there like a giant construction company that has this phenomenal API layer over all of their backend systems. They're writing mobile apps that will blow your mind at a rate that, in turn, would blow your mind as well because they just call to a standard REST kind of syntax and structure.

That stuff really works really, really well. But when it comes to saying, "Hey, I want to provision a service so here's my API and that's going to work all the time." That's not true today and it may never be fully true. It may be more true than it is today, but I think you have to understand that there's a lot more that has to be standardized than just APIs to get to that point. That's going to take more work and that's going to take more effort.

But projects like OpenStack, like CloudStack, like Eucalyptus, they have a great opportunity to, in fact, create mini ecosystems or even large ecosystems out there where that's more true than it would be for the cloud as a whole.

It excites me that the API story is taking off because for developers it's powerful. But I also...It's temporary when I say that. I'm sort of saying it's not enough to say APIs. We need common formats and additional common interfaces. That work still needs to be done.

Gordon: Yeah, you really need hybrid cloud management to take care of some of that really at a level that's below what developers really ought to be worrying about.

James: Yeah, and that's why...This is the reason why enStratus is focused on the application level of operations. We're about application operations in the cloud. How do you consume cloud services to deliver application capabilities? We're largely focused on infrastructures and service today, but that's obviously an evolving picture. I think when you look at the problem of saying, "I'm going to...My tools for running my application are in a cloud service," that's very limiting in terms of how you do things.

Having tools that say, "Let's step back and abstract how we want to operate our applications in general," and then apply that to the different clouds we might want to consume in the way that we operate. Make sure we're applying consistent governance. Make sure we're applying consistent automation to the approach. Make sure we're applying that in a very independent way so not only are you independent from the clouds that you can choose but also in terms of the tools that you apply to operate.

So with DevOps tools, you want to use Chef/Puppet. What management tools do you want to use? Monitoring tools, those kinds of things, do you want to use in the environment? That's really what the enterprise needs, is that ability to begin to abstract application operations and begin to incorporate the things that they need to in that way.

Looking at application operations as separate from infrastructure and services operations. The delivery of a cloud service to the end customer. So building your private cloud is not an application operation problem. It's a service operations problem. Consuming that private cloud is an application operations problem.

Gordon: Great. Thanks very much, James. Anything else to add?

James: No. Congratulations to Red Hat on their wonderful launch and with their OpenShift stuff. I'm very excited to see what's going on in that PaaS side of the market. I think that's a really exciting space to watch. And I'm very happy to have been here with you today and have a chance to talk to you.

Gordon: Great. Thanks, James.

Links for 05-11-2012

noreply@blogger.com (Gordon Haff) — Fri, 11 May 2012 17:43:00 +0000

Tabletop Studios | Modahaus Table Top Studio Lighting Kits
The Man Who Took on Amazon and Saved a Bookstore - Forbes - RT @mathewi: how the owner of the Harvard bookstore figured out a way to compete with Amazon: -- will have to check out
HP Is Still Full of Leaks - Businessweek - "Time heals all wounds. Except at Hewlett-Packard (HPQ), where old wounds fester with remarkable persistence and where there seems to be a deep bench of executives unfamiliar with the concept of a bygone."

PaaS Infographic

noreply@blogger.com (Gordon Haff) — Thu, 10 May 2012 15:35:00 +0000

Lots of growth forecast. I'd argue that the relatively slow pickup to date has been a function of the fact that many first-generation PaaS platforms have been specific to a single provider. More open approaches, such as that taken by Red Hat's OpenShift, are provider-independent--which greatly reduces the possibility of vendor lock-in.

Links for 05-10-2012

noreply@blogger.com (Gordon Haff) — Thu, 10 May 2012 14:01:00 +0000

Why "Freemium" Fails for Startups: 3 Business Lessons from the Band New Order - Forbes - "Freemium is fantastic when it works. Problem is, it doesn’t work all that often."
How Hewlett-Packard lost its way - Fortune Tech - "Your comments are being live-blogged," one employee told her defiantly. Whitman challenged the man. "You all have taken leaking to a new art form," she said. "It's a sign of an unhappy company. You wish HP ill." The tapping suddenly stopped, and as the room fell silent, the mobile devices were lowered.
The trouble with PandoDaily – Milo Yiannopoulos – The Kernel - "The first time I saw PandoDaily, I was convinced it was unfinished, or a spoof. Now it’s been running for a while, I’m aghast. Were one to crowdsource the ten most anachronistic WordPress theme features from the last five years and jam them together with horse glue, you would struggle to come up with something uglier or more difficult to navigate." << Amusing in a high school fighting cliques sort of way.
10 Salts to Know - an article from Food52
Sell Oracle And Buy This Software Stock - Seeking Alpha - RT @screnshaw: "Red Hat is the leader in open source and cloud computing" @seekingalpha
Moore's Law Over, Supercomputing "In Triage," Says Expert - Technology Review - "If Sterling is right, and he is one of the deans of high performance computing, it seems likely that we'll never reach the next milestone in supercomputing in silicon. Physicist Michio Kaku says Moore's Law will collapse in about ten years. Sterling agrees, and says it comes down to the basic physical restrictions of working with atoms."
The 2nd Tenet of PaaS: The P in PaaS Stands for Platform, Not Product | tentenet.net - RT @bryanwche: Why #OpenShift is the ultimate cloud application platform for enterprise developers #PaaS #redhat
Red Hat OpenShift's Technology Foundations - YouTube - Here's video of @matthicksj explaining how #redhat infrastructure products used to support @openshift PaaS
Red Hat | Implementing an open cloud architecture - Long day of webinars about implementing an open cloud. Stop by if you get a chance!
Red Hat Unveils Enterprise Platform-as-a-Service (PaaS) Roadmap and Strategy - MarketWatch - RT @joefern1: Red Hat Unveils Enterprise PaaS Strategy << Multiple PaaS operations models
OpenCloudConf - Future of Clouds - Gordon Haff - YouTube - Here's a video of my cloud trends presentation from #opencloudconf
Oracle gets a chance to rewrite software law | Business Tech - CNET News - RT @bobmcmillan: A great take on the Google-Oracle case by @stshank Well written, dude. << Yep. Nice.
Politico endorses post-first, check-later journalism - Erik Wemple - The Washington Post
Searching the Internet B.G. (Before Google) - Input Output
Cloud is a corporate strategy, not a tactical solution — Cloud Computing News - "OK, maybe there are more bells and whistles, but treating cloud like a technology solution purchase is the wrong approach. Take a holistic approach to how IT can and should participate in the business of doing whatever your company does, then build the operational model to support that. Seek alignment in your organization and in your technology choices so you’re prepared for a Cloud Operating model and Fluid IT. Welcome to the modern IT world."
Why Publishers Don't Like Apps - Technology Review - "And Technology Review? We sold 353 subscriptions through the iPad. We never discovered how to avoid the necessity of designing both landscape and portrait versions of the magazine for the app. We wasted $124,000 on outsourced software development. We fought amongst ourselves, and people left the company. There was untold expense of spirit. I hated every moment of our experiment with apps, because it tried to impose something closed, old, and printlike on something open, new, and digital."
An IT Forecast: Where Cloud, Mobile and Data Are Taking Us | SYS-CON MEDIA - "A thorough piece on CNET by Gordon Haff looks at the interconnectivity of cloud computing, mobility and Big Data, and sees these three forces as instrumental in shaping the future of IT. "Through the lens of next-generation IT, think of cloud computing as being about trends in computer architectures, how applications are loaded onto those systems and made to do useful work, how servers communicate with each other and with the outside world, and how administrators manage and provide access," Haff writes."

Links for 05-07-2012

noreply@blogger.com (Gordon Haff) — Mon, 07 May 2012 12:44:00 +0000

Can Amazon become the biggest platform peddler in the world? • The Register
The frequent fliers who flew too much - latimes.com
(403) http://www.yelp.com/biz/WAo42c41_0Y0tifXbwuo8w?pt=check_in&ref=twitter&v=4b - I checked in at Fuki Sushi (4119 El Camino Real) on #Yelp
On APIs and Copyright – tecosystems - RT @sogrady: new post | "On APIs and Copyright": << Nice piece. Will be quoting from it shortly :-)
2012 Best Food Blog Awards: The Winners! - Saveur.com
Does the Noncommercial Creative Commons license make sense? | The Pervasive Data Center - CNET News - .@jasonbrooks NC is longtime beef of mine w CC as well Though believe @sogrady disagrees.
Computational Journalism Server – The Way Forward » Borasky Research Journal - Nice vote for OpenShift
Recapping the OpenShift Origin Launch and What Happened to Express and Flex? | OpenShift by Red Hat - Good overview of available resources.
Where Do All Those BuzzFeed Cute Animal Pictures Come From? - Alexis Madrigal - Technology - The Atlantic - ""I would love if every image contained some secret metadata and a way to license that image," Peretti told me. "But the practical reality is that it is pretty challenging, particularly in the web culture of animals and the images that spread on Pinterest and Tumblr." " --ME: Even when you try to do the right thing, it's usually not straightforward to provide attribution.
Rough Type: Nicholas Carr's Blog: Pay up, Yochai Benkler - I mostly agree that the center of grab in the Web (etc.) has shifted more commercial.
563 - Pop by Lat and Pop by Long | Strange Maps | Big Think
Triggertrap Mobile | Triggertrap
Revenue could be fatal: 3 reasons your startup should consider waiting | FoundersVoice - Pretty much agree with the people puking all over this.
Facebook Urges Members to Add Organ Donor Status - NYTimes.com - RT @R_Thaler: Yay! “@timoreilly: Wow. Facebook adds "organ donor status" to profiles. This is Sunstein and Thaler's ...
The 10th Tenet of Cloud: The Linux of Cloud is…Linux | tentenet.net - RT @bryanwche: The 10th Tenet of Cloud: The Linux of Cloud is...Linux

Standard APIs: There's no substitute for open

noreply@blogger.com (Gordon Haff) — Thu, 03 May 2012 18:46:00 +0000

[As I touch on intellectual property issues herein, I'd like to remind everyone that this is my personal blog and should not be in any way taken as official Red Hat positions or statements, nor presented as such.]

Just because something is widely used doesn't make it a standard--de facto, de jure, or otherwise--in the sense that anyone can use and build implementations of that standard without restriction. Indeed, as we shall see, even standards that are "blessed" by powers-that-be are not always fully open in the ways that I have outlined previously.

Standardization has been around for a long time. The IEEE tells us that:

Based on relics found, standardization can be traced back to the ancient civilizations of Babylon and early Egypt. The earliest standards were the physical standards for weights and measures. As trade and commerce developed, written documents evolved that set mutually agreed upon standards for products and services, such as agriculture, ships, buildings and weapons. Initially, these standards were part of a single contract between supplier and purchaser. Later, the same standards came to be used across a range of transactions forming the basis for modern standardization.

A lot of this early standardization pretty much came down to custom. The convoluted history of why we drive on one side of the road in a given country is instructive. (Though each country's conventions are now enshrined in The Geneva Convention on Road Traffic.)

The history of the shipping container, as detailed in Marc Levinson's The Box: How the Shipping Container Made the World Smaller and the World Economy Bigger, offers another, fairly typical, historical example. Incompatible container sizes and corner fittings required different equipment to load and unload and otherwise inhibited the development of a complete logistics system. The standardization that happened around 1970 made possible the global shipping industry as we know it today--and all that implies. The evolution of standardized railroad gauges is similarly convoluted. The development of many early-on computer formats and protocols was similarly darwinian.

It's tempting to take this past as prologue and conclude that similar processes will continue to play out as we move to new styles of computing in which different forms of interoperability assume greater importance. For example, published application programming interfaces (API) are at the heart of how modular software communicates in a Web services-centric world. One set of APIs wins and evolves. Another set of APIs becomes a favorite of some particular language community. Still another doesn't gain much traction and eventually withers and dies. It sounds like a familiar pattern.

But there's an important different. In today's software world, it's impossible to ignore intellectual property (IP) matters whether copyright, patent, trademark, or something else. An API isn't a rail gauge--though perhaps someone today would try to patent that too.

As a result, tempting as it might be to adopt some API or other software construct because it's putatively a "de facto" standard, which is mostly a fancy way of saying that it's popular, that may not be such a good idea.

RedMonk's Stephen O'Grady offers some typically smart commentary on why:

it’s worth noting that many large entities are already behaving as if APIs are in fact copyrightable. The most obvious indication of this is Amazon. Most large vendors we have spoken with consider Amazon’s APIs a non-starter, given the legal uncertainties regarding the intellectual property involved. Vendors may in certain cases be willing to outsource that risk to a smaller third party – particularly one that’s explicitly licensed like a Eucalyptus [coverage]. But in general the low risk strategy for them has been to assume that Amazon would or could leverage their intellectual property rights – copyright or otherwise – around the APIs in question, and to avoid them as a result. Amazon, while having declined to assert itself directly on this basis, has also done nothing to discourage the perception that it has strict control of usage of its APIs. In doing so, it has effectively turned licensed access to the APIs into a negotiable asset, presumably an outcome that advocates of copyrightable APIs would like to see made common.

In fact, lack of openness can even extend to standards that have gained some degree of governmental or quasi-governmental approval--which is, after all, a political process. Last decade's fierce battle over Microsoft's submittal of its OOXML document format as a standard to ECMA and ISO is perhaps the most visible example. The details of this particular fight are complicated, but, in Kurt Cagle's words, "The central crux of the [then-]current debate is, and should be, whether Microsoft’s OOXML does in fact represent a standard that is conceivably implementable by anyone outside of Microsoft."

Issues of the conditions that should be satisfied in order for a vendor's preferred approach/format/etc. to become a "blessed" standard continue to reverberate. The latest round is about RAND (Reasonable-and-Non-Discriminatory) licensing and whether that can take the place of truly open implementations. It's essentially an attempt to slip proprietary approaches requiring a patent license into situations, such as government procurements, that require open standards.

But, as Simon Phipps, a Director of the Open Source Initiative and of the UK’s Open Rights Group puts it:

The presence of RAND terms at best chills developer enthusiasm and at worst inhibits engagement, as for example it did in the case of Sender ID at IETF. As Välimäki and Oksanen say, RAND policy allows patent holders to decide whether they want to discourage the use of open source. Leaving that capability in the hands of some (usually well-resourced) suppliers seems unwise.

At one level, the takeaway here might be "it's complicated." And it is. But another takeaway is pretty simple. You can dress up proprietary standards in various ways and with various terms. And such standards have a place in the IT ecosystem. But they're not open, whatever you call them.

Links for 04-30-2012

noreply@blogger.com (Gordon Haff) — Mon, 30 Apr 2012 22:38:00 +0000

Historic Photos From the NYC Municipal Archives - In Focus - The Atlantic
Food Fascist | Michael Ruhlman
P.S. A Column On Things: The death of photojournalism as we have known it
RAND: Not So Reasonable? - Simon Says... - "When RAND terms are required (be they FRAND, RAND-Z or plain RAND), they indicate a desire to hold in reserve the option of controlling the relationship with the end user, introducing different terms depending on the competitive and market context. This scope for additional restrictions and for variability dependent on the field of use is what makes any form of RAND a red flag to open source developers."
Welcome - The Data Journalism Handbook
What We Can Learn From Super-Slow Motion | VSL
The End of Gaming (Consoles) | Gaming Business Review - "Of all the segments I have ever worked in, the gaming one has the most ‘religious fervor’ – often mixed with a bizarre bigotry. Fanboys rage at each other and the suppliers try hard to play to this audience. Yes, many know there is a bigger audience out there, but it was not the big game makers that produced Farmville, or Angry Birds or even Draw Something. Is this because the big game makers are busy trying to be a film-studio (or even replace film studios)? Maybe."

Links for 04-27-2012

noreply@blogger.com (Gordon Haff) — Fri, 27 Apr 2012 17:19:00 +0000

John Grisham Explains Baseball
Who Made That Pie Chart? - NYTimes.com
Is GPL licensing in decline? | Open Source Software - InfoWorld
Midsize Insider: IT Trends: A Vast, Moving Cloud? - "Are Big Data, mobility, and the cloud converging to reshape the IT environment at a fundamental level? Red Hat's "cloud evangelist" says that they are. True, his job is to say things like that. But in fact, these IT trends are hard to miss: They dominate the tech press week after week. They form the backdrop for all the buzz about individual tech firms."
Daily Research News Online no. 15349 - Gartner Offers $AUD 20m for IDEAS - Missed this news. Gartner acquiring IDEAS (which had acquired DH Brown long time ago)
State of the Computer Book Market, part 4: The Languages - O'Reilly Radar - "Java continues to be the number one language from book units sold and dollars. There is some shuffling going on with the languages tough. JavaScript is very hot now, as is R. Likely a result of Android and big data driving folks to these languages. "
Five not-so-obvious reasons why Apple won't be Sony redux | ZDNet - RT @sjvn: RT @dbfarber: Five not-so-obvious reasons why Apple won't be Sony redux @ldignan << Makes sense.
The Google attack: How I attacked myself using Google Spreadsheets and I ramped up a $1000 bandwidth bill | A Computer Scientist in a Business School - RT @samj: The Google attack: How I attacked myself using Google Spreadsheets and I ramped up a $1000 bandwidth bill ...
Talent Shortage Looms Over Big Data - WSJ.com
Open Cloud Conf - RT @pmikeyp: The Open Cloud Conference at the Silicon Valley Cloud Center is next week - << I'll be on panel Tues AM
The Library of Utopia - Technology Review - "Google's ambitious book-scanning program is foundering in the courts. Now a Harvard-led group is launching its own sweeping effort to put our literary heritage online. Will the Ivy League succeed where Silicon Valley failed?"
High-stakes one-shot Prisoner's Dilemma on a British game show with an astounding strategy - Boing Boing
High stakes for open source in the commercial cloud | Open Source Software - InfoWorld

Links for 04-25-2012

noreply@blogger.com (Gordon Haff) — Wed, 25 Apr 2012 17:12:00 +0000

What Your Klout Score Really Means | Epicenter | Wired.com - Fun stuff in the comment thread to :-) Some well above-average snark.
Apple's Tim Cook on Windows 8: It's like mating a fridge with a toaster - Computerworld Blogs - The degree to which tablets and laptops will converge over time is an important question that doesn't have an obvious answer.
Secure virtualization for tactical environments – Military Embedded Systems
What If a Highway Ran Through the Infinite Corridor? - An interstate was once proposed to run through Cambridge. vis @MIT_alumni
Meet Nicira. Yes, people will call it the VMware of networking — Cloud Computing News
Sony's fall and Japan's hang-ups | Nanotech - The Circuits Blog - CNET News
Introducing the data journalism handbook | News | guardian.co.uk - "Edited by Liliana Bounegru, Jonathan Gray and Lucy Chambers, the book will be made freely available online under a CC BY-SA license so anyone can read and share it. Additionally a printed version and an e-book will be published by O'Reilly Media. If you want to be notified when the book is released, you can sign up on the website."

Pike Street Market

noreply@blogger.com (Gordon Haff) — Mon, 23 Apr 2012 21:03:00 +0000

Pike Street Market
Originally uploaded by ghaff

I had a few hours when I was in Seattle and dropped by Pike Street Market.

I'm interviewed about cloud at Cloud Fair

noreply@blogger.com (Gordon Haff) — Mon, 23 Apr 2012 20:49:00 +0000

Gordon talks about what Red Hat has been up to recently in the cloud including Open Stack and Open Shift. Interviewed by Dell's Director of Web Technology Barton George at Cloud Fair 2012 in Seattle, WA. April 18, 2012

Links for 04-20-2012

noreply@blogger.com (Gordon Haff) — Fri, 20 Apr 2012 17:31:00 +0000

Links for 04-19-2012

noreply@blogger.com (Gordon Haff) — Thu, 19 Apr 2012 16:15:00 +0000

HealthCheck Fedora - Where's the beef? - The H Open Source: News and Features
What Amazon's ebook strategy means - Charlie's Diary - "Firstly, it's not an accident that Bezos' start-up targeted the book trade. Bookselling in 1994 was a notoriously backward-looking, inefficient, and old-fashioned area of the retail sector. There are structural reasons for this. A bookshop that relies on walk-in customers needs to have a wide range of items in stock because books are not fungible; a copy of the King James Version Bible is not an acceptable substitute for "REAMDE" by Neal Stephenson or "Inside the Puzzle Palace: A History of the NSA" by James Bamford."
Why Open Source Is the Key to Cloud Innovation
Why e-books cost so much | Internet & Media - CNET News - "Here's something that tends to get lost in the debate over e-book prices: Paper doesn't cost very much."
Yes, Copyright's Sole Purpose Is To Benefit The Public | Techdirt
It’s official: IBM and Red Hat get with OpenStack — Cloud Computing News - RT @SteveTomasco: It’s official: IBM and Red Hat get with OpenStack #ibmcloud #openstack
SAS 70 / SSAE 16 Issues & How to Fix Them » Data Center Knowledge - "For data centers, the challenge associated with the use of SAS 70 and SSAE 16 is that both standards are focused on internal controls over financial reporting (ICFR) concerns. ICFR is crucial for corporations that must comply with Sarbanes-Oxley requirements. In most cases, however, ICFR is of limited concern for the services data centers provide for customers. With limited reporting options, data centers were somewhat stuck between a rock and a hard place."
Why You Can’t Get a Taxi - Magazine - The Atlantic - "But just because Uber is good for its passengers and drivers doesn’t mean that it’s good for everyone. Taxi drivers are a powerful political constituency in many cities. And as Robert McNamara noted drily, “Like any other business, taxi drivers think it would be great if no one could compete with them.” In some cities, including San Francisco and Washington, D.C., a regulatory backlash has hit the company hard."
IBM and the Eames Office: Minds of Modern Mathematics iPad app

Podcast: I talk open clouds with Chris Wells

noreply@blogger.com (Gordon Haff) — Thu, 12 Apr 2012 16:05:00 +0000

My colleague Chris Wells turns the tables on me and interviews me about the characteristics of an open cloud. These include:

is open source
has a viable, independent community
is based on open standards
gives you the freedom to use IP
is deployable on the infrastructure of your choice
is pluggable and extensible with an open API
enables portability of applications and data to other clouds

Listen to MP3 (0:09:09)
Listen to OGG (0:09:09)

[Transcript]

Gordon Haff: You're listening to the Cloudy Chat Podcast with Gordon Haff.

Chris Wells: Welcome, everyone. For today's podcast, we're going to do something a little bit different and turn around. My name is Chris Wells and I'm a Product Marketing Manager here at Red Hat. Today I'm actually going to interview Gordon Haff, our cloud evangelist.

Gordon: Hey. Thanks, Chris.

Chris: We'll turn the tables here a little bit. I'll ask you some of the questions. I understand that you've been doing a lot of work, and Red Hat in particular has been doing a lot of work, around open cloud. Could you just talk a little bit about what does Red Hat mean when it says "open cloud?"

Gordon: Sure. Well, the idea of an open cloud is really that you can build a cloud out of all your IT infrastructure and not just a part of it. Also, there are a lot of other characteristics that are very important to, really, all the customers we talk about ‑ the ability to move applications from one cloud to another, the ability to develop applications once and deploy them anywhere you want, the ability, really, to be in control of your own roadmap. Obviously, as an open source company, Red Hat places a lot of value on openness across a number of different dimensions. I have to say, we've actually been a little bit surprised maybe about how much this message has resonated with our cloud customers.

Chris: Well, when you talked a little bit about the open source piece there, is it simple enough to say that open cloud equals open source, or is there more to it than that?

Gordon: There's a lot more to it than that. Open source is clearly very important. I think a lot of the aspects of openness around clouds are kind of hard to imagine how you might get there without open source, but open source by itself does have a lot of benefits. It lets the users control their own implementation. It doesn't tie them to a particular vendor. It lets users collaborate with communities. If they want something that's a bit different, or maybe they and some other end users want something that's a bit different, they can go in that direction and don't have to convince some vendor to do it.

Obviously, part of that is viewing source code and being able to do their own development. Although that's very important, it doesn't stop there.

Chris: Where else does it go?

Gordon: Staying on the open source theme, one of the first things is that open source isn't just code and license. Not like, "OK, I can see the code. It's licensed under Apache. Everything's great. Don't need to worry about it any more." The community that's associated with that open source code is really important. Really, if it's just open source and it's still just a single company that's involved in it, that probably doesn't buy an end user an awful lot, because all the developers are still with that single company. Really realizing the collaborative potential for open source means that you have a vibrant community, and that involves things like governance. How do you contribute code? What are the processes for that kind of thing? Where does innovation come from?

Also related to that is open standards. Again, these things are all related to each other. Again, I'd probably argue truly open standards aren't possible outside of open source, because then they're always going to be tied to a single vendor in some way or another.

Standardization, in the sense of official standards, even isn't necessarily the critical thing here. These things take a long time to roll out, and cloud computing is such a new area, but the idea that you have standards, even if they're not fully standardized yet, is still very important.

Chris: That's very interesting. Now, you mentioned a little bit earlier about talking to different types of customers. Do you see as customers have more and more interest in going to cloud computing, are they more interested in the open cloud or open source type of approach than they might have been in a traditional data center?

Gordon: Yeah, I think so because cloud is really about spanning all this heterogeneous infrastructure, whether it's public clouds, whether it's different virtualization platforms, or whether it's physical servers even. I think a lot of people think cloud is just virtualization or public clouds, but actually, pretty much everyone we talk to says that they really see for a lot of workloads, that maybe 20 to 25 percent of the workloads in organizations, that those are really going to stay in physical servers for the foreseeable future. There's definitely an interest in moving those workloads to the cloud.

Chris: Now, there's a lot of vendors in the cloud space besides Red Hat. Red Hat's obviously taken this approach to really go down the open cloud paradigm that we talked about because that's pretty consistent with our heritage and our history around being open source. What do you see as challenges for other vendors that today aren't open?

Gordon: The big challenge is that users are demanding openness. And in fact, if you look at the cloud computing marketing literature out there, not to mention any names, but you see some very much closed vendors out there who have "Open" in huge type on their websites. It's usually because they're trying to frame themselves as being open in some narrow sense. Perhaps they've contributed their APIs to some standards organization or something. I think they're going to be challenged when you compare them with a company like Red Hat, for example, which has a long heritage in open source, knows how to work with communities, knows how to ‑ really understands the depth of openness that is required. I think they're going to be challenged to combat that effectively.

Chris: For our listeners out there in the audience, I think most of them would agree, because I think you and I have talked to a lot of customers that definitely want this openness. The question's going to be everyone is trying to say that they're open. If you're in the customer's shoes today, or our listener's shoes, what kind of questions can they ask to actually figure out is something truly open versus someone just saying the word "open?"

Gordon: I've gone through a few of these already, but let me go down the list of things that we've come up with that, as we talk to our customers, really resonate with them as mattering. I've mentioned open source, mentioned the community associated with that open source, mentioned open standards.

Another important aspect of openness is freedom to use IP. Now, we don’t have a lot of time to get into that here but, suffice it to say that, although modern open source licenses and open standards can mitigate certain aspects of IP issues—patents, copyrights, and so forth—freedom to use IP is a separate issue that users ought to be aware of.

Is deployable in the infrastructure of your choice. This speaks to in cloud [how] it really can't be just an extension of a particular virtualization platform, for example. It really needs to be independent of that other layer and deployable in public, choice of virtualization platforms and physical servers.

The ability to extend APIs, adding features, can't be under the control of a single implementation or vendor. That was one reason that something called the Deltacloud API that Red Hat uses is under the auspices of the Apache Software Foundation, which is a very well‑regarded, meritocracy‑based governance regime, so that kind of governs how people can contribute and extend that.

Finally, just the idea that you have portability to other clouds. You can't have a cloud that requires that you develop your software in a particular way that's tied to that particular cloud so that you have to port it if you want to move it somewhere else. Those are really the main things that we think about when we think of an open cloud, and that's really resonated with our customers.

Chris: Well, Gordon, this is some great information that you've shared with our audience today. I think you've given them some great takeaways of characteristics they should look for around choosing a vendor around open and cloud and some key questions to ask. Thank you very much.

Gordon: Thanks, Chris.

Links for 04-10-2012

noreply@blogger.com (Gordon Haff) — Tue, 10 Apr 2012 13:41:00 +0000

iTWire - Why we need the GPL more than ever - The trend towards more permissive licensing really upsets some folks.
Here is why Facebook bought Instagram — Tech News and Analysis - "Instagram has what Facebook craves – passionate community. People like Facebook. People use Facebook. People love Instagram. It is my single most-used app."
Red Hat Contributes More to OpenStack than Canonical Ubuntu - InternetNews. - "Canonical also does not show up in the list of top Linux kernel contributors either. While as a company Canonical and its charismatic leadership continue to proclaim how they are moving the open source community forward, the numbers tell of a different story. The numbers show that Canonical's culture of contribution is not in the same league as Red Hat's."
Who Wrote OpenStack Essex? A Deep Dive Into Contributions - RT @cloudpundit: Who Wrote OpenStack Essex? A Deep Dive Into Contributions by @jzb via @RWW
Twitter / @RyanProgramming: An animated Cloud Computin ...
If Oracle wins its Android suit, everyone loses | Open Source Software - InfoWorld - Good history of Java from a legal and competitive environment perspective.
The Open Source Implications of the CloudStack Announcement – tecosystems - "While everyone wants to predict outcomes on project and API futures, the fact is that it’s too early in most cases to project accurately. "
Practical Tips from Four Years of Worldwide Travel | Legal Nomads
Good is my copilot - Boston.com - "Not every drama or near tragedy is a teachable moment. At the risk of sounding too mellow about the whole incident, we should just sit back, admit stuff happens, and recognize that there was a backup plan: the copilot."
An Animated History of the MBTA [Map] | BostInno - Animated MBTA map through history
darktable 1.0 released | darktable
Questions Amazon Should Answer About Its Cloud Strategy - Forbes - RT @jasonbrooks: Questions Amazon Should Answer About Its Cloud Strategy - Forbes << Good questions.

Of open source licenses and open cloud APIs

noreply@blogger.com (Gordon Haff) — Mon, 09 Apr 2012 17:55:00 +0000

Last week was active in the cloud world with Citrix sending CloudStack to the Apache Software Foundation and pulling out of the OpenStack project. Of course, there's been much fevered commentary, some smart--and some not so much.

[To be even more explicit than usual, I work as a Cloud Evangelist at Red Hat, which has partnerships, competes with, and/or has other relationships with various companies and projects mentioned. The opinions I express here are mine alone, should not be taken as official Red Hat positions, and are in no way based on non-public information.]

The basic facts about Citrix' April 3 announcement are as follows. As stated in their press release, "Citrix CloudStack 3 will be released today under Apache License 2.0, and the CloudStack.org community will become part of the highly successful Apache Incubator program." CloudStack is an Infrastructure-as-a-Service cloud management product that came into Citrix by way of its acquisition of Cloud.com in 2011. Not stated in the press release, but widely reported, was that Citrix was pulling out of OpenStack, the open source project on which it had previously planned to focus under the codename Project Olympus.

Those are the basics. Now for some observations.

Without overly downplaying this announcement, it highlighted the unfortunate rush in the press, blogs, and twitter to crown winners even in the early stages of a technology trend. Suddenly, OpenStack--which lots of folks had widely promoted as having "won" the cloud race--was being talked about as yesterday's news. Lest we forget, a different platform had been being talked up as the inevitable winner the year before that. Analyst Steven O'Grady at RedMonk has a typically more nuanced view: "While everyone wants to predict outcomes on project and API futures, the fact is that it’s too early in most cases to project accurately."

I have my own views and am obviously a big believer in Red Hat's cloud projects and position. I'm not going to get into those here but just point out that we're in the very early stages of a long and complicated game. I'd also point out that cloud management isn't one size fits all; different products/projects such as Red Hat's CloudForms and OpenStack address different use cases.
Steven among others also observes that the shift of CloudStack into Apache, and the corresponding shift of the code from the GPL license to the more permissive Apache license represents an overall trend:

Faded to the point that permissive licenses are increasingly seen as a license of choice for maximizing participation and community size. It’s not true that copyleft licenses are unable to form large communities; Linux and MySQL are two of the largest open source communities in existence, and both assets are reciprocally licensed. But the case can be made that this will in future be perceived as anachronistic behavior.

I agree with Steven. I wrote about the topic in more detail in a CNET Blog Network post last year. In addition to encouraging participation, as Steven notes, I also speculate that the success of open source as a development and innovation model has made open source projects less leery of protecting their code from freeloaders as the terms of a copyleft license attempts to do. (A copyleft license basically means that if you make changes to the code and distribute those changes in the form of a binary, you need to distribute the source code with the changes also.)

Red Hat also uses Apache licensing for projects such as the Deltacloud API (which is also governed by the Apache Software Foundation and which recently graduated from Incubator status--where CloudStack is today--to a top level project) and Project Aeolus (one of the main upstream projects for Red Hat CloudForms hybrid cloud management).

Another point about last week's happenings worth a mention is the API discussion. Application Programming Interfaces are the mechanism that lets you communicate with virtualization platforms and cloud providers. Arguably, they don't get enough attention. For example, what incantations do you need to make in order to spin up a machine image on Amazon?

By way of background, the Amazon APIs--at least those for doing relatively lowest-common-denominator tasks that pretty much any IaaS cloud needs to do--have come to be regarded by some as de facto standards. Which is to say, not really standards but things that are omnipresent enough that they can effectively be regarded as such. Formats, such as the specifics of images that run on Amazon are separate but related issue; I won't touch on those further here even though they're at least equally important.

Well, maybe.

One of the key points in the Citrix announcement was that the "proposed Apache CloudStack project will make it easier for customers of all types to deliver cloud services on a platform that is open, powerful, flexible and 'Proven Amazon Compatible.'" In other words, build an AWS-compatible private cloud. We've seen this before with Eucalyptus, which had its own announcement about a supposedly expanded relationship with Amazon a couple of weeks back.

It turns out there's a bit of a wrinkle though. I first got a hint of it from a twitter post by Netflix cloud architect Adrian Cockcroft. Which led me to this post by Gartner's Lydia Leong in which she writes: "With this partnership, Eucalyptus has formally licensed the Amazon API. There’s been a lot of speculation on what this means." As far as anybody knows, Citrix does not have a corresponding license from Amazon.

Why does this matter?

It matters because open APIs are one of the key characteristics of an open cloud. And this should serve as something of a wakeup call. Perhaps, as Lydia suggests in another post:

I think it comes down to the following: If Amazon believes that they can innovate faster, drive lower costs, and deliver better service than all of their competitors that are using the same APIs (or, for that matter, enterprises who are using those same APIs), then it is to their advantage to encourage as many ways to “on-ramp” onto those APIs as possible, with the expectation that they will switch onto the superior Amazon platform over time.

However, the fact that these APIs can be licensed and that one or more vendors believes there to be business advantage to licensing those APIs should set off at least gentle alarms. At the least, it raises questions about what behaviors Amazon could at least potentially restrict in the absence of an explicit license.

Over at Forbes, Dan Woods asks:

Are there limits to the use of Amazon’s APIs?
How will community experience inform the evolution of Amazon’s APIs?
What is the process that will govern the evolution of the Amazon APIs?

These and others are good questions to ask. For history has shown time and time again that de facto open is not open. Times change and companies change.

Links for 04-05-2012

noreply@blogger.com (Gordon Haff) — Thu, 05 Apr 2012 19:29:00 +0000

Prices Are People: A Short History of Working and Spending Money - Derek Thompson - Business - The Atlantic
Before the Internet: The golden age of online services | ITworld
IBM CIO Discusses Big Blue's BYOD Strategy | PCWorld Business Center
Just how big are porn sites? | ExtremeTech - Staggering numbers.
Google Art Project
Angry Birds, Farmville and Other Hyperaddictive ‘Stupid Games’ - NYTimes.com - "And so a tradition was born: a tradition I am going to call (half descriptively, half out of revenge for all the hours I’ve lost to them) “stupid games.” In the nearly 30 years since Tetris’s invention — and especially over the last five, with the rise of smartphones — Tetris and its offspring (Angry Birds, Bejeweled, Fruit Ninja, etc.) have colonized our pockets and our brains and shifted the entire economic model of the video-game industry. Today we are living, for better and worse, in a world of stupid games."
Can Copyleft Clouds Find Contributors?
5 takeaways from the CloudStack-OpenStack dustup — Cloud Computing News - Agree with some. Disagree with some.

Links for 04-04-2012

noreply@blogger.com (Gordon Haff) — Wed, 04 Apr 2012 13:23:00 +0000

Ask not what OpenStack can do for you… « Seeing the fnords - "Over the last months I’ve seen more and more tweets and news articles using the formulation “OpenStack should”, as in “OpenStack should support Amazon APIs since it’s the de-facto standard”. I think there is a fundamental misconception there and I’d like to address it."
Eleven Tips for Successful Cloud Computing Adoption | Cloud Computing Journal - Nice list.
Wig Wam Bam. « The Loose Couple's Blog - "The deep-rooted politic, hidden agendas and the overall return have made little sense in terms of commercial opportunity and the de facto positioning of “it [feature] will be available in the next release” will not have sat well within the corridors of power. Add that to the recent “insert coin to continue” trend within OpenStack and the dreadful, garish “loophole” in the Apache License (sigh) that almost begs for “embrace and extend but do not return code” will have contributed significantly to the sounding of the death knell. How the latter plays out for Cloudstack will be interesting to observe too."
The Amazon-Eucalyptus partnership - "With this partnership, Eucalyptus has formally licensed the Amazon API. There’s been a lot of speculation on what this means. My understanding is the following:"
Freeform Community Research - "So if it’s not Generation Y, who is it that’s pushing to have their own devices connected to the corporate network? As can be seen in Figure 1, outside of the IT department itself, it’s the senior executives who are most insistent on using a personal device for work. "

Links for 04-03-2012

noreply@blogger.com (Gordon Haff) — Tue, 03 Apr 2012 14:44:00 +0000

Citrix Breaks Away From OpenStack | Forrester Blogs - "With Eucalyptus staging a revival, cloudstack breaking away and OnApp locking in service provider customers, the window for OpenStack to become the Linux of IaaS is beginning to close. "
Girls Around Me and the end of Internet innocence | Molly Rants - CNET News - "Thank you, I-Free, for ushering in a tipping point in the conversation about online privacy. You freaked us right the hell out by showing us exactly what's possible with information we're freely sharing online--and maybe it'll finally make us all stop."
Love Hotels and Unicode | ReignDesign
Citrix Gives Cloudstack to the Apache Software Foundation and Turns its Back on OpenStack | ServicesANGLE - Ref: Citrix, CloudStack, OpenStack, AWS, etc. << My, but the game is most certainly afoot.
Connections: Crowdsourcing prediction and other data fun with Oscar predictions - Over weekend crunched 25+ yrs of data from an Oscar contest. Interesting crowdsourced predictions results
25 Years of IBM’s OS/2: The Strange Days and Surprising Afterlife of a Legendary Operating System | Techland | TIME.com
(13) Take a drive through Boston circa 1964
Hazing, Dartmouth, and Animal House, Part II - Ricochet.com

Crowdsourcing and other data fun with Oscar predictions

noreply@blogger.com (Gordon Haff) — Tue, 03 Apr 2012 01:22:00 +0000

And now for something completely different.

By way of background, a classmate of mine from undergrad has been holding Oscar parties for over 25 years. As part of this Oscar party, he's also held a guess-the-winners contest. With between 50 and 100 contest entries annually for most of the period, that's a lot of ballots. And, being Steve, he's carefully saved and organized all that data.

Over the years, we've chatted about various aspects of the results, observed some patterns, and wondered about others. For example, has the widespread availability of Oscar predictions from all sorts of sources on the Internet changed the scores in this contest? (Maybe. I'll get to that.) After the party this year, I decided to look at the historical results a bit more systematically. Steve was kind enough to send me a spreadsheet with the lifetime results and follow up with some additional historical data.
I think you'll find the results interesting.

But, first, let's talk about the data set. The first annual contest was in 1987 and there have been 1,736 ballots over the years with an average of 67 annually; the number of ballots has always been in the double-digits. While the categories on the ballot and some of the scoring details have been tweaked over the years, the maximum score has always been 40 (different categories are worth different numbers of points). There's a cash pool, although that has been made optional in recent years. Votes are generally independent and secret although, of course, there's nothing to keep family members and others from cooperating on their ballots if they choose to do so.

The first thing I looked at was whether there were any trends in the overall results. The first figure shows the mean scores from each year graphed in a time series, as well as a three-year moving average of that data. I'll be mostly sticking to three-year moving averages from hereon out as it seems to do a nice job of smoothing data that is otherwise pretty spiky, making it hard to discern patterns. (Some Oscar years bring more upsets/surprises than others, causing scores to bounce around quite a bit.)

Is there a trend? There does seem to be a slight permanent uptick in the 2000s, which is right where you'd expect there to be an uptick if widespread availability of information on the Internet were a factor. That said, the effect is slight. And running the data through the exponential smoothing function in StatPlus didn't turn up a statistically significant trend for the time series as a whole. (Which represents the sum total of statistical analysis applied to any of this data.) As we'll get to, there are a couple other things that suggest something is a bit different in the 2000s relative to the 1990s, but it's neither a big nor indisputable effect.

Color me a bit surprised on this one. I knew there wasn't going to be a huge effect but I expected to see a clearer indication given how much (supposedly) informed commentary is now widely available on the Internet compared to flipping through your local newspaper or TV Guide in the mid-nineties.

We'll return to the topic of trends but, for now, let's turn to something that's far less ambiguous. And that's the consistent "skill" of Consensus. Who is Consensus? Well, Consensus is a virtual contest entrant who, each year, looks through all the ballots and, for each category, marks its ballot based on the most common choice made by each of the human contest entrants. If 40 people voted for The Artist, 20 for Hugo, and 10 for The Descendants for Best Picture, Consensus would put a virtual tick next to The Artist. (Midnight in Paris deserved to win but I digress.) And so forth for the other categories. Consensus then gets scored just like a human-created ballot.

As you can see, Consensus does not usually win. But it comes close.
And Consensus consistently beats the mean, which is to say the average total score for all the human-created ballots. Apparently, taking the consensus of individual predictions is more effective than averaging overall results. One reason is that Consensus tends to exclude the effect of individual picks that are, shall we say, "unlikely to win." Whereas ballots seemingly created using a dartboard still get counted in the mean and thereby drive it down. If you look at the histogram for 2006 results, you'll see there are a lot of ballots scattered all over. Consensus tends to minimize the effect of the low-end outliers.

But how good is Consensus as a prediction mechanism compared to more sophisticated alternatives?
We've already seen that it doesn't usually win. While true, this isn't a very interesting observation if we're trying to figure out the best way to make predictions. We can't know a given year's winner ahead of time.
But we can choose experts in various ways. Surely, they can beat a naive Consensus that includes the effects of ballots from small children and others who may get scores down in the single digits.

For the first expert panel, I picked those with the top five highest average scores from among entrants in at least four of the first five contests. I then took the average of those five during each year and penciled it in as the result of the expert panel. It would have been interesting to also see the Consensus of that panel but that would require reworking the original raw data from the ballots themselves. Because of how the process works, my guess that that this would be higher than the panel mean but probably not much higher.

For the second panel, I just took the people with the 25 highest scores from among those who had entered the contest for at least 20 years. This is a bit of a cheat in that, unlike the first panel, it's retrospective--that is, it requires you in 1987 to know who is going to have the best track record by the time 2012 rolls around. However, as it turns out, the two panels post almost exactly the same scores. So, there doesn't seem much point in overly fussing with the panel composition. Whatever I do, even if it involves some prescience, ends up at about the same place.

So now we have a couple of panels of proven experts. How did they do? Not bad.
But they didn't beat Consensus.

To be sure, the trend lines do seem to be getting closer over time. I suspect, apropos the earlier discussion about trends over time, we're seeing that carefully-considered predictions are increasingly informed by the general online wisdom. The result is that Consensus in the contest starts to closely parallel the wisdom of the Internet because that's the source so many people entering the contest use. And those people who do the best in the contest over time? They lean heavily on the same sources of information too. There's increasingly a sort of universal meta-consensus from which no one seriously trying to optimize their score can afford to stray too far.

It's hard to prove any of this though. (And the first few years of the contest are something of an outlier compared to most of the 1990s. While I can imagine various things, no particularly good theory comes to mind.)

Let me just throw out one last morsel of data. Even if we retrospectively pick the most successful contest entrants over time, Consensus still comes out on top. Against a Consensus average of 30.5 over the life of the contest, the best >20 year contestant scored 28.4. If we broaden the population to include those who have entered the contest for at least 5 years, one person scored a 31--but this over the last nine year period when Consensus averaged 33.

In short, it's impossible to even beat Consensus consistently by matching it against a person or persons who we know with the benefit of hindsight to be the very best at predicting the winners. We might improve results by taking the consensus of a subset of contestants with a proven track record. It's possible that experts coming up with answers cooperatively would improve results as well. But even the simplest and uncontrolled Consensus does darn well.

This presentation from Yahoo Research goes into a fair amount of depth about different approaches to crowdsourced predictions as this sort of technique is trendily called these days. It seems to be quite an effective technique for certain types of predictions. When Steve Meretzky, who provided me with this data, and I were in MIT's Lecture Series Committee, the group had a contest each term to guess attendance at our movies. (Despite the name, LSC was and is primarily a film group.) There too, the consensus prediction consistently scored well.

I'd be interested in better understanding when this technique works well and when it doesn't. Presumably, a critical mass of the pool making the prediction needs some insight into the question at hand, whether based on their own personal knowledge or by aggregating information from elsewhere. If everyone in the pool is just guessing randomly, the consensus of those results isn't going to magically add new information. And, of course, there are going to be many situations where data-driven decisions are going to beat human intuition, however it's aggregated.

But we do know that Consensus is extremely effective for at least certain types of prediction. Of which this is a good example.