Infrastructure 2.0

bloxNews Readers: The Network is Strategic to Virtualization

Fri, 13 Nov 2009 15:09:40 PST

A special thanks to bloxNews readers who participated in the recent reader survey on CIOs, virtualization and the network. We asked readers questions about: CIO knowledge regarding DNS; their primary drivers for virtualization; and virtualization’s impact on the network.

We weren’t surprised that 79% of CIOS appreciated the critical importance of DNS and that enterprises were now focused on flexibility as a driver for virtualization. The survey findings regarding the strategic role of the network for the evolution of virtualization was encouraging and consistent with a recent panel I attended on the evolving roles of network pros.

Consistent with recent Nemertes research survey respondents for the bloxNews survey cited flexibility as the leading driver for virtualization.

Chart: Primary Driver for Virtualization

Perhaps the most notable finding from the survey was regarding the impact of virtualization on the network. 53% of survey respondents said that “without dynamic networks it won’t be possible to realize the full benefits of virtualization, including live migration (VMotion) and certain cloud architectures. 24% thought that virtualization would make networks less important and 23% thought the network effects of virtualization would be insignificant.

This marks a significant shift from the days of early stage (VLAN-centric) virtualization, where the network was seen as irrelevant. Virtual networks became increasingly dense and harder to manage. See "Virtual Machine Density as the New Measure of IT Efficiency" by F5’s Lori MacVittie.

It certainly appears that virtualization is now “on the radar” for bloxNews readers.

Also according to the survey of bloxNews readers, 79% of CIOs do understand the critical importance of DNS for the availability, performance and security of key applications.

Note: this survey was conducted during late October and early November 2009 by about 150 respondents. A FLIP™ High Definition camcorder was offered as a drawing incentive. A senior network engineer from Sutter Health won the FLIP®.

The Real Time Infrastructure Ultimatum

Thu, 05 Nov 2009 16:20:46 PST

For months the infrastructure 2.0 blog has talked about the automation of IT from a network perspective, including the automation of the network itself. While few may question the need for network automation most businesses today still run their networks like they ran their “supply chains” decades ago, before the network.

This great irony is about to change. Here’s why:

As virtualization entered the data center it became an accidental standard bearer for network automation. The power of virtualization helped to drive a cultural (including x as a service) shift in expectations, just as Nicholas Carr was declaring war on traditional “old world” IT with the help of Google, Amazon and a host of other cloud (and not so cloud) players.

IT directors watched operations pros create VMs in seconds while network teams could take hours (or days) to simply move an existing server. One visionary IT exec told me that an outside firm had calculated that moving a server cost his company almost as much as buying a new one. This was a very large network, but nonetheless increasing complexity was driving increasing cost and a need for more efficient strategic orchestration across ever larger pools of endpoints.

Years ago networks were the drivers of massive change. Today they often symbolize a resistance to change.

This is no doubt part of the appeal of cloud computing, and why Carr’s promise of IT as a plug and play utility is so compelling. Enterprise IT has not kept up with innovations it has, ironically, helped to enable. And the foundation for many IT departments is that inflexible, manually configured, kludge network.

The Ultimatum

Those who bet on the continuation of manual labor, processes, scripts, checklists and spreadsheets (to manage their network) will lose as they continue to be saddled with rising network management costs, inflexibility and the mounting pressure of IT consumerization. Those who embrace automation will likely have a strategic advantage… if they automate the right processes.

We’re now watching packs of IT vendors form to deliver containers of integrated solutions, pitting former partners against one another, partly in an effort to standardize the silos. I think this marks a mere transitional phase between the static network and the dynamic network.

The first pack to break from the complex, static network model into infrastructure 2.0 will likely win, as it will be able to demonstrate substantial opex reduction with increased IT agility. The other camp will be force to discount their goods in order to make up for the OPEX slaughter their customers will face.

The opportunity for substantial reductions in opex with heightened flexibility is simply too compelling to be passed by, for users and vendors.

The API is the New CLI

Wed, 04 Nov 2009 03:39:34 PST

Infrastructure 2.0, from a purely developmental standpoint, is about APIs. It’s about offering up the functionality and capabilities of a wide variety of infrastructure – network, storage, and application network – to be externally controlled, integrated, and leveraged for whatever purpose a developer might dream up. It enables providers and enterprises alike to turn infrastructure functionality into services. Need compression? Caching? Routing? Load balancing? Via service-enabled management APIs these can become services, provisioned and released through the invocation of a service. When expanded to include the sharing of actionable data – performance statistics, status, availability of application services (context!) – this integration becomes the mechanism through which a dynamic infrastructure is created. One that reacts to events and conditions in the network, storage, application network, and application infrastructure in real-time.

But all of this functionality, the automation of functions, the codification of processes (orchestration) requires integration. Where previous generations of administrators evaluated the manageability of network and application network devices based on the CLI (command line interface) the next generation of administrators and the developers who will support integration efforts, will almost certainly look to APIs as a means to determine suitability of solutions within their architecture.

APIs are the new CLI

In the past network administrators would compare the CLI syntax and functionality of network and application network devices to Cisco’s IOS. IOS became the de facto standard for command line interfaces and even today you’ll find reviewers and discussions that mention how “IOS-like” any given CLI might be. But as infrastructure 2.0 and the need for dynamic infrastructures continues to drive administrators and developers toward APIs for integration and automation the CLI will wane in importance and the API will rise to take its place. That’s because the APIs provided by network and application network devices will be the primary interface through which the device is configured, controlled, and managed.

Luckily network and application network vendors learned from the trials and travails of enterprise software and the first implementations of these APIs have been primarily standards (web-services, XML) based. Service-enabled APIs means both administrators and developers can take advantage of the functionality and do so in whatever language or environment they are most comfortable. This flexibility is key to adapting to the myriad possible environments and architectures in which such devices may be deployed.

The danger in this shift toward APIs is that it is infinitely more difficult to replace systems that are integrated via an API or library – any programmatic-based integration, really – than it is to replace those for which the CLI is the primary administrative route. Administrators comfortable with the APIs of a Cisco router or switch will be less inclined, for example, to replace those core networking devices with a Juniper or other networking solution because of the inherent difficulty and time involved in learning – and using – a new API. This is true across the infrastructure spectrum; the APIs that allow complete control and management over BIG-IP (iControl) are very different from those available for Citrix Netscaler, or Cisco ACE or any of the other API-enabled application delivery platforms.

It is quite possible that whomever can win the “API wars” for Infrastructure 2.0 will become the new de facto standard for that particular “tier” (for lack of a better term) in the infrastructure architecture. Eventually one API will be preferred over the other – either due to saturation and usage or specific demand and it will give the vendor an edge that will not easily be dulled.

EAI and Adapters

But it’s not just network-facing IT that will help set the direction of APIs. Because part of the premise of infrastructure 2.0 and the APIs that are part of parcel of the standards and devices within its domain is integration there is a developer-focused component to the success of infrastructure APIs. While administrators are most likely to be closest to the APIs of network and application network devices, developers are most likely closest to the applications that drive orchestration and integration with business-focused systems.

One of the ways software application vendors knew they’d “made it” was the inclusion of adapters in EAI (enterprise application integration) systems for their product. ODBC drivers for databases, message queuing adapters for MQ and JMS, and more recently “salesforce.com” and other SaaS offerings. The inclusion of adapters for specific solutions in EAI and development environments is tantamount to declaring that solution a “win” for the enterprise. Thus it will be important to vendors of network and application networking solutions to court management and orchestration system vendors to include at distribution an “adapter” or samples, at a minimum, as the means to integrate and include their particular solution.

This seems counterintuitive, as most APIs are service-enabled and thus the bulk of the integration work is implicit in the API. But the ease with which those APIs are used and integrated by developers is paramount to successful adoption of infrastructure 2.0 APIs. The inclusion as an “adapter” provides the ease of use, often via a GUI, necessary to garner use and support from developers and business-focused orchestration analysts because of the inherent differences in the data plane. Mapping of objects from one device to another, from one system to another, is required and it is this core requirement that is fulfilled by middleware systems such as EAI and ESB (enterprise service bus) implementations. The easy integration with these middle-tier applications will be increasingly important as we move from operational policies based purely on technical metrics toward data centers driven by both technical and business metrics.

APIs are the new basis for standards

The first generation of the Internet used protocols and structural definitions to engender interoperability and even portability. Infrastructure 2.0 heralds the coming of a second generation of the Internet just as Web 2.0 signaled the beginning of the second generation of the Web. This next generation of infrastructure interoperability and portability will certainly be driven by protocols, but those protocols will include APIs and encompass a broader set of functions at higher layers of the network stack. Many of the ongoing efforts in the standards arena today are based not on structural definitions of data but on the APIs that will enable integration across the infrastructure and the Internet, a la “InterCloud.”

Both are necessary components to ensuring interoperability and portability, but until we see standardization of meta-data and component definitions, the emphasis will continue to be on the APIs.

Infrastructure 2.0 – A Virtual Analogy

Tue, 27 Oct 2009 22:37:10 PDT

Is OS virtualization an end in itself? Is it both necessary and sufficient for all things Cloud and IaaS? Is it the panacea IT Operations has been looking for? From where I see it, abstracting the OS is certainly a great start, but it’s actually only 50% of the goal.

To a degree, OS virtualization is the “shiny metal object” de jure in that it’s captivating everyone’s attention. It is of course very valuable, and is causing an important inflection point in datacenter operations and economics. But there is a less-visible, less sexy side to datacenter operations and economics that lies “below” the CPU in the stack – it’s the I/O, network, network devices and address space. And this represents the other 50% of the transition to more agile and efficient IT.

The value of OS virtualization is in its ability to abstract the OS so that higher-level services are possible – workload consolidation, portability, migration, failover, scaling, etc. But viewing this purely from an above-the-CPU, software-centric perspective is myopic. Lots of other things need manipulation in a production datacenter. For example, when a server (or service) gets moved, I/O and addressing need to change; security policy (and/or devices) need to follow the application; switch/router ports may change; load balancing and other IP devices need to be reconfigured. While OS virtualization simplifies application workload management, it certainly doesn’t address these network-centric and QoS-centric issues.

This whole idea was neatly encapsulated recently in a blog by VMware’s Mark Thiele: “When you can log into a console and use your mouse pointer to drag a server into a network or resource pool and have the appropriate network security and routing policies applied, you’ll be getting close to IT nirvana”

And that’s the first big takeaway for what a more dynamic infrastructure (“Infrastructure 2.0”) will bring: The same level of agility, control, security and efficiency to the network that OS virtualization brings to the workload.

Unfortunately the networking half of the dynamic IT story is still sadly lacking in maturity… as evidenced by the many static network diagrams I see pinned to walls, and by the many manually-administered IP address and DNS spreadsheets sitting in managers’ offices. This dynamic network infrastructure is what marketers call a “Latent Want”. It’s a need that’s unfulfilled, but also largely unrecognized.

How’d we get into this mess?

The statically-defined address/naming space and networking topologies arose mostly as a function of the evolution of the CPU itself, and how datacenter networking, storage and security components evolved around it. Briefly, server technology slowly became laden peripherals like I/O cards with static state such as addresses and WW names; once these servers were cemented in the data center, the network & its devices had to be similarly statically configured (See more about how the Industry Went Amiss).

Fortunately, there are a number of products just coming to market that are beginning to bring virtualization/abstraction to the I/O and networking world as well. Also with the advent of unified computing concepts, virtual I/O, and converged networking, some of these tight I/O and network bonds are just now being broken. In an excellent Illuminata summary of the burgeoning abstraction of the network, Gordon Haff observes how more dynamic infrastructure is also helping: “I/O virtualization brings these principles to the edge of the network. Its general goal is to eliminate the inflexible physical association between specific network interface controllers (NICs) and host bus adapters (HBAs) and specific servers.”

The next step will be to extend these dynamic principles from VMs and I/O, now to the network.

Where we need to focus attention

Wayne Gretzky once famously said he "skates to where the puck is going, not to where it is." We’ve seen where OS virtualization is taking us. But let’s now anticipate where IT network operations will go in the future.

Let’s begin again with an OS virtualization analogy: Take VMware’s DRS – which orchestrates the creation, scaling and migration of VMs dynamically as demand changes. It’s a great illustration of workload management adapting to demand and to utilization. Similarly, we’d expect infrastructure to have similar dynamic properties - I/O, network switching, balancing, security and even inter-datacenter connectivity which would need to have the same level of fluidity.

Think we’re there now? Think again. Here are some use-case examples that just don’t have generalizable solutions yet – (whether in the physical or virtual server world):

Local Server repurposing: A server farm sits behind a firewall; each server has a specific I/O configuration, and needs access to a load balancer to handle spikes in traffic. Problem: if a server in this group should fail – or should more servers need to be added – only servers in that physical cluster (which have been configured with specific I/O) can be swapped-in. No others have access to the firewall or load balancer.
Virtual server migration to a new datacenter: Say you have a VM on a specific VLAN behind a specific firewall. And you want to live-migrate that server to a remote datacenter. Good luck with that – the firewall probably won’t be available, nor may the addressing be available (or portable) and neither may be the VLAN.
Environment failover: Now, say you have a complete server environment whose topology includes both physical and virtual servers, switches, load balancers, firewalls and VLANs. Now say you need to re-create this environment elsewhere due to a disaster. Your best hope is a team that can identically re-configure this topology fast (or, you have a warm recovery datacenter just waiting in the wings). But today, your options are limited in being able to accomplish this in SW.

Just a reminder here: OS virtualization is not the answer to any of the use-cases above. Rather, what we ideally want to solve for is a dynamically-reconfigurable infrastructure – one where network components are able to be created and implemented on-demand. (This is not unlike Lori MacVittie’s recent observation of AWS’ dynamic load balancing and scaling, where in effect, load balancers can be defined and instantiated in software.)

The completed analogy: the next step for the data center

The punch-line here is that there needs to be analogous “2.0” functions embedded in the network/infrastructure to what we already are familiar with in the software realm. Take for example:

Infrastructure abstraction – allows for logical provisioning of I/O, networks, network devices, storage connectivity and network devices in software; analogous to the creation and placement of virtual workloads in the software space
Infrastructure consolidation – by defining I/O in software, and by using converged networking, this greatly simplifies utilization and configuration of the physical infrastructure; analogous to logical consolidation of VMs and their workloads
Dynamic networking – networks, multi-pathing and addressing that adapts to sizes and locations of workloads, as well as adapting to failures and bottlenecks. Roughly analogous to high availability and wide-area migration services that are delivered in virtual OS environments.
Logically-defined load balancing and security policies – where IP load balancing, firewalls, etc. can be invoked for any processor in any location, and where IP loads can be distributed locally (or globally) on-demand; roughly analogous to virtual scale-out services and grids.
Dynamic QoS management - allows for optimal use of network capital, and (hopefully) best infrastructure efficiency; this is analogous to dynamically managing CPU utilization in the software world.

Parting words of sobriety

While this idealized picture is only a future, there are certainly companies and products beginning to chip-away at the market. But point-products (non-systems solutions) are never the entire answer. Rather, it’s high-time for the industry to begin to think about an approach to address this space. Like most industry maturity models, I would expect to see something like the following evolve over the next few years (these things take time):

Point-products: That address specific issues e.g. I/O virtualization, converged network techs, software-based network mgmt appliances
Industry awareness: For example, developing what the “infrastructure 2.0” working group is proposing
Common communications: APIs and protocols to allow interoperation of the infrastructure components and their logical configuration
Standards-based innovation: e.g. the DMTF or similar standards organization take on this set of issues for broader adoption
Automation: A broader set of tools get developed to orchestrate the infrastructure similar to what we’re seeing in the VM space

And finally: Technology is only part of this story. But there is also the fact that any form of automation/abstraction will massively impact IT operations, and therefore will butt-up against organizational structure, jobs, roles and people. So the sooner we recognize both the benefits and organizational impacts, the sooner we’ll be prepared to gladly absorb the changes this approach to infrastructure management will cause.

Ken Oestreich is VP of Product Marketking with Egenera, a frequent blogger as Fountainhead, and can also be found quipping on Twitter.

Why Infrastructure-as-a-Service is going to "Stick."

Mon, 26 Oct 2009 19:26:53 PDT

The Network Infrastructure Business is about to undergo its own Fork Lift Upgrade.

The conventional enterprise marketing strategy is to introduce new technology and product features at a rate which is intended to serve as a catalyst for driving equipment upgrades. It's a strategy vendors have come to depend on for their primary means of revenue growth. Unanticipated changes in business priorities, however, always puts the strategy at risk and undermines even the best laid plans. Examples of recent shifts in network infrastructure resource priorities are:

Talking about the power of the network has been replaced by discussing the power consumption of the network.
Protecting the enterprise network from incoming traffic used to be the focus. Today, the focus is connectivity management (i.e. Network Access Control [NAC],) data asset protection and implementing risk management using Network Behavioral Analysis solutions (NBA.) In other words, the focus has flipped; it’s about looking at what's traversing and coming out of the enterprise network.
Activity in the Wi-Fi market is now more about deploying location based services and perimeter firewalls.

On-going shifts in network resource priorities, while significant, are evolutionary and somewhat predictable. A bigger challenge facing network infrstructure companies is managing the impact Infrastructure-as-a-Service (IaaS) and cloud-based virtual enterprises could have on the market. If the IaaS market experiences significant growth, the network infrastructure market and everything associated with managing a network will be eventually transformed into a service industry. No on-site data centers, no wired infrastructure to manage and the on-going security challenge for an enterprise to properly differentiate between desktop and mobile user access is essentially eliminated.

For companies selling into the enterprise market this could mean a dramatic change in the customer profile and that customer's business needs – since the customer is now the Iaas provider. If you're a network infrastructure vendor and wish to retain your current set of customers, then now is the time to start thinking about how IaaS can be integrated with your existing business and consider scaling back the enterprise channel sales plans.

The IaaS Market's Secret Sauce: Automated Network Provisioning

Before handing over the IT farm to a 3^rd party, if you're a CIO for an enterprise, an initial question to seek an answer to is getting a clearer picture of the IaaS benefits. The benefits derived from migrating to IaaS need be more than simply a reduction in real estate or lease expenditures and leveraging economies of scale within the service provider.

If it were the situation, then migrating to IaaS could potentially turn into nothing more than a cost transfer that provides a marginal benefit over the long run. So, when does it make sense to make the switch? The answer requires taking a closer look at the type of automated network provisioning an IaaS provider is using.

In particular, the technology driving advances in automated network service provisioning, over time, is going to become more important to an IaaS provider. Automated network service provisioning solutions ultimately will have the critical task of binding together all the services that make up a private cloud-based virtual enterprise. From the services associated with user authorization, usage accounting, and dynamic allocation of compute resources to configuring the data loss prevention (DLP) services, they will all be eventually controlled by an automated network service provisioning.

Without question, server virtualization is the foundation technology upon which the cloud-based computing business is built. Virtualization technology also receives the most fanfare when the topic of discussion is cloud based computing. Because of the critical role that automated network service provisioning has in improving operational efficiency it deserves time in the limelight, too.

There are, however, two fundamental business issues that both a prospective IaaS tenant and IaaS provider need to address regarding tapping into the revenue potential and cost benefits of an IaaS business:

How to overcome the uncertainty around the IaaS business model becoming nothing more than a cost transfer exercise for the CIO.
How to take the cost out of change management whether it is for the IaaS provider or the tenant.

The answer is to incorporate automated business rule-driven network provisioning into the IaaS solution.

Why Business Rule-Driven Network Provisioning is King.

Business rule-driven network service provisioning enables both IaaS tenants and IaaS providers to define change requests using business terms instead of defining network element specific tasks. This simplifies the change management process by eliminating the need to define network element specific provisioning tasks. The built-in intelligence of the network provisioning tackles the network element specific details, determines the specific configuration dependencies and then securely administrates the change request.

IaaS customers will be ultimately seeking three operational benefits from IaaS:

Equal or greater business agility when deploying new applications.
Simplified regulatory compliance management.
Being able to periodically optimize their enterprise workflows.

Business rule-driven automated network service provisioning can serve these needs in spades and reduces operational costs. For an IaaS provider and its customers to derive the full benefit from business rule-driven provisioning, the network provisioning software for an IaaS deployment, at a minimum, needs to:

Be purpose-built for multi-tenant deployments.
Include a web-based portal that enables tenants to use business rule-driven provision.

The latter is about ensuring that enterprises can continue to maintain a high degree of business agility within their IT infrastructure. The former is about the technology underpinnings of an IaaS provider’s solution and sheds light on their ability to scale the service. If the automated network provisioning software was not designed from the ground-up to be a multi-tenant platform, nor capable of supporting easily configured customer-specific portals for monitoring and provisioning, it's less likely to scale to meet the unique business needs of an IaaS provider.

As compared to the importance of server virtualization technology, which is considered “the” enabling technology for cloud computing, automated network service provisioning when business rule-driven: drives out variable costs, improves efficiency and ultimately can become the face of the business by virtue of its portal capabilities. These benefits, over time, will elevate the interest and market demand for automated network service provisioning solutions to the same level as virtualization technology has today.

If you're a CIO who is considering shifting IT services to a cloud-based virtual enterprise and want to add some thunder to the solution. Then, make sure the IaaS provider's tenant portal offers business rule-driven provisioning. Otherwise, your cloud-based virtual enterprise initiative will become grounded -- leaving you in a fog.

Let the Network Infrastructure Market Disruption Begin...

The upside of using business rule-driven provisioning is it eliminates, margin robbing, variable costs associated with change management for both the IaaS provider and its tenants. The operational benefits could, whenever a CIO does an IaaS cost benefit analysis, potentially give an IaaS provider a sustainable competitive edge by tipping the scales in favor of IaaS – further strengthening the IaaS value proposition value.

IaaS providers can also take advantage of their ability to offer new services to their customers who would otherwise have to wait until the “in-a-box” version is released. For example, the ability to offer enterprise risk management services to customers. This is why IaaS is going to stick with customers. The combination of benefits ranging from the cost advantage when using business rule-driven network provisioning, web portal convenience and the ability to offer new enterprise services to customers sooner makes the IaaS value proposition very compelling.

When CIO's begin allocating an increasing percentage of their budget to IaaS, it will change the how companies serving the network infrastructure market operate. These companies will need to restructure their sales channels, business model and marketing activities. Whereas the enterprise market product mantra today is “ease of use”, within the IaaS market it’s going to be about scalability, flexibility, accounting features and manageability. Because these are fundamental changes rather than incremental changes, significant growth within the IaaS market is going to be a catalyst for many companies in the network infrastructure market to fork lift upgrade their enterprise business plans and operations.

IT Innovation Requires Network Innovation

Thu, 22 Oct 2009 10:44:17 PDT

A recent article by Larry Dignan about how IT has fallen behind the Tech Curve laments how slow and cumbersome enterprise IT has become relative to consumerized technologies. Larry covered a session at the recent Gartner Symposium and was advised by Gartner analysts that IT pros want the world to proceed in an orderly fashion and are weighed down by the legacy of previous choices. That’s a fair statement.

Gartner’s solution, or at least that posed by analysts David Mitchell Smith and Tom Austin is for IT to simply let users buy their own gear:

“Now Gartner has been on this user-provided IT pitch for a while now-the research firm equates the company laptop to the company car in the 1970s-and the prediction hasn’t exactly become the norm. However, the move to let employees bring their own gear increasingly makes sense. Why? Employees are already bringing what they want to work anyway. Exhibit A: The iPhone. Exhibit B: Google. Exhibit C: Facebook. You get the idea.”

Larry Dignan – How Did IT Fall…, Tech Republic October 2009

This notion –to simply let users make their own consumerized choices as a cure to the “crotchety” world of IT- misses the core of the problem; that is, the lack of the necessary network automation, integration and real-time visibility. IT won’t keep up unless the culture of manual labor, scripts and spreadsheets evolves.

Today’s CIOs are ensnared by red tape and manual processes (compared to consumers or even CFOs) when it comes to real-time visibility and control. They will never be able to keep up with consumerization until they automate infrastructure and can view and modify status and policy in real-time. Consumers use product cycles to discard the status quo while CIOs have to live with layers of previous purchases and vendor interrelationships, not to mention custom scripts, overlays and pockets of expertise.

Virtualization brought flexibility to within the VLAN, but for the most part it is confined there thanks to the messy collision between static networks and dynamic systems and increasingly dense VLANs. Again, the problem is the out-of-date network.

In some environments it costs almost as much to move a server than it does to buy a new one. That was one of the drivers of virtualization-lite, immediate capex reduction by allowing systems teams to manage, move spin up, etc new servers in minutes. Yet virtualization only postponed the reckoning by enabling admins to manage more server pool within VLANs. The old world challenges still wait at the border of the VLAN, at the network.

That, I think, is one of the reasons that the VMware, Cisco, etc partnership is so powerful; it has the potential to create a clean slate (an infrastructure 2.0).

Rather than merely accepting the consumerization of enterprise IT or the “eventuality” of public clouds, why not focus much needed attention back at the network, which was once and should perhaps again be the driving force for IT and business innovation?

I am a senior director at Infoblox. You can follow my rants in real time at www.twitter.com/archimedius.

Amazon Elastic Load Balancing Only Simple On the Outside

Thu, 15 Oct 2009 04:01:47 PDT

Amazon’s ELB is an exciting mix of well-executed infrastructure 2.0 and the proper application of SOA, but it takes a lot of work to make anything infrastructure look that easy.

The notion of Elastic Load Balancing, as recently brought to public attention by Amazon’s offering of the capability, is nothing new. The basic concept is pure Infrastructure 2.0 and the functionality offered via the API has long been available on several application delivery controllers for many years. In fact, looking through the options for Amazon’s offering leaves me feeling a bit, oh, 1999. As if load balancing hasn’t evolved far beyond the very limited subset of capabilities exposed by Amazon’s API.

That said, that’s just the view from the outside.

Though Amazon’s ELB might be rudimentary in what it exposes to the public it is certainly anything but primitive in its use of SOA and as a prime example of the power of Infrastructure 2.0. In fact, with the exception of GoGrid’s integrated load balancing capabilities, provisioned and managed via a web-based interface, there aren’t many good, public examples of Infrastructure 2.0 in action. Not only has Amazon leveraged Infrastructure 2.0 concepts with its implementation but it has further taken advantage of SOA in the way it was meant to be used.

NOTE: What follows is just my personal analysis, I don’t have any especial knowledge about what really lies beneath Amazon’s external interfaces. The diagram is a visual interpretation of what I’ve deduced seems likely in terms of the interactions with ELB given my experience with application delivery and the information available from Amazon and should be read with that in mind.

WHAT DOES THAT MEAN?

When I say Amazon has utilized SOA in a way that it was meant to be used I mean that their ELB “API” isn’t just a collection of Web Services, or POWS, wrapped around some other API. It’s actually a well-thought out and designed set of interfaces that describe tasks associated with load balancing and not individual product calls. For example, if you take a look at the ELB WSDL you can see a set of operations that describe tasks, not management or configuration options, such as:

CreateLoadBalancer
DeleteLoadBalancer
RegisterInstancesWithLoadBalancer
DeregisterInstancesFromLoadBalancer

To understand why these are so significant and most certainly represent tasks and not individual operations you have to understand how a load balancer is typically configured, and how the individual configuration components fit together. Saying “DeleteLoadBalancer” is a lot easier than what really has to occur under the covers. Believe me, it’s not as easy as a single API call to any load balancing solution. There’s a lot of relationships inherent in a load balancing configuration between the virtual server/IP address and the (pools|farms|clusters) and individual nodes, a.k.a. instance in Amazon-speak. Yet if you take a look at the parameters required to “register instances” with the load balancer, you’ll see only a list of instance ids and a load balancer name. All must be configured, but the APIs make this process appear almost magical.

The terminology used here indicates (to me at least) an abstraction which means these operations are not communicating directly with a physical (or even virtual) device but rather are being sent to a management or orchestration system that in turn relays the appropriate API calls to the underlying load balancing infrastructure.

The abstraction here appears to be pure SOA and it is, if you don’t mind my saying, a beautiful thing. Amazon has abstracted the actual physical implementation of not only the management or orchestration system, but also decoupled (as is proper) the physical infrastructure implementation from the services being provided. There is a clear separation of service from implementation, which allows for Amazon to be using product X or Y, hardware or software, virtual or concrete, and even one or more vendor solutions at the same time without the service consumer being aware of what that implementation may be.

The current offering appears to be pure layer 4 load balancing which is a good place to start, but lacks the robustness of a full layer 7 capable solution and eventually Amazon will need to address some of the challenges associated with load balancing stateful applications for its customers; challenges that are typically addressed by the use of persistence, cookies, and URI rewriting type functionality. Some of this type of functionality appears built-in, but is not well-documented by Amazon.

For example, the forwarding of client-IP addresses is a common challenge with load-balanced applications, and is often solved by using the HTTP custom header: X-Forwarded-For. Ken Weiner addresses this is a blog post, indicating Amazon is indeed using common conventions to retain the client IP address and forward it to the instances being load balanced. It may be the case that more layer 7 specific functionality is exposed than it appears, but is simply not as well documented. If the underlying implementation is capable – and it appears to be given the way ELB addresses client IP address preservation - it is a pretty good bet that Amazon will be able to address other challenges with relative ease given the foundation they’ve already built.

That’s agility; that’s Infrastructure 2.0 and SOA. Can you tell I’m excited about this? I thought you might.

This gives Amazon some pretty powerful options as it could switch out physical implementations with relative ease, as it so desires/needs, with virtually (sorry) no interruption to consumer services. Coupling this nearly perfect application of SOA with Infrastructure 2.0 results in an agility that is often mentioned as a benefit but rarely actually seen in the wild.

THIS IS INFRASTRUCTURE 2.0 IN ACTION

This is a great example of the power of Infrastructure 2.0. Not only is the infrastructure automated and remotely configured by the consumer, but it is integrated with other Amazon services such as CloudWatch (monitoring/management) and Auto Scaling. The level of sophistication under the hood of this architecture is cleverly hidden by the simplicity and elegance of the overlying SOA-based control plane which encompasses all aspects of the infrastructure necessary to deliver the application and ensure availability.

Several people have been trying to figure out what, exactly, is providing the load balancing under the covers for Amazon. Is it a virtual appliance version of an existing application delivery controller? Is it a hardware implementation? Is it a proprietary, custom-built solution from Amazon’s own developers? The reality is that you could insert just about any Infrastructure 2.0 capable application delivery controller or load balancer into the “?” spot on the diagram above and achieve the same results as Amazon. Provided, of course, you were willing to put the same amount of effort into the design and integration as has obviously been put into ELB.

While it would certainly be interesting to know for sure, the answer to that question is overridden in my mind by a bigger one: what other capabilities does the physical implementation have and will they, too, surface in yet another service offering from Amazon? If the solution has other features and functionality, might they, too, be exposed over time in what will slowly become the Cloud Menu from which customers can build a robust infrastructure comprising more than just simple application delivery? Might it grow to provide security, acceleration, and other application delivery-related services, too?

If the underlying solution is Infrastructure 2.0 capable – and it certainly appears to be - then the feasibility of such service offerings is more likely than not.

Infrastructure 2.0 Is the Beginning of the Story, Not the End

Thu, 08 Oct 2009 04:54:35 PDT

The term “Infrastructure 2.0” seems to be as well understood as the term “cloud computing.” It means different things to different people, apparently, and depends heavily on the context and roles of those involved in the conversation. This shouldn’t be surprising; the term “Web 2.0” is also variable and often depends on the context of the conversation. The use of the versioning moniker is meant, in both cases however, to represent a fundamental shift in the way the technologies are leveraged by people. In the case of Web 2.0 it’s about the shift toward interactive, integrated web applications used to collaborate (share) data with people. In the case of Infrastructure 2.0, it’s about a shift toward interactive, integration infrastructure used to collaborate (share) data with infrastructure.

The two are surprisingly similar in evolution, in key components, in defining attributes, in intended effects. Web 2.0 evolved to address a specific set of key pain points observed and felt by users for years. Infrastructure 2.0 is the data center equivalent; it’s the evolution of infrastructure to address a set of key pain points observed and felt by IT for years. It’s about enabling infrastructure components – network, application network, endpoints – with the ability to share data and the intelligence to make decisions based on that data in the proper context. It’s the ability to adapt to change and manage the massive volume of information that each individual infrastructure component inherently collects but rarely shares.

The rapid growth and emergence of sites like FriendFeed and TwitterFeed that exist solely to augment the usability of other Web 2.0 sites is a perfect example of the way in which Web 2.0 allows users to shift the burden of managing the ebb and flow of data across sites in a more effective manner. The automation of sharing across web 2.0 sites is still primitive; it doesn’t really take into consideration context and the rules by which content is pushed to one site or another are primarily simple and event-based. “When X happens, push Y.” What happened is that applications recognized that they couldn't adapt fast enough to the rapid changes occurring and addressed that gap with technology. Strategic points of control began to emerge that allowed for automation of sharing and feedback across web sites and ultimately people.

This is where Infrastructure 2.0 begins as well. Simple, event-based integration between the various layers of network, application network, and endpoints. In some cases, such as IF-MAP and IP address management, this is accomplished in much the same way as Web 2.0: a third party “manager” that collects the information provided and fires off an “event” that then shares the information with anyone “subscribed” to it. But it needs to build on that momentum, and continue to evolve, toward what everyone wants to happen but is afraid to mention - as if we’re afraid we might jinx efforts to get there if talk about it, or that we’ll be branded a renegade – or worse - for even thinking it might happen.

INFRASTRUCTURE 2.0 IS THE BEGINNING, NOT THE END

What we are working toward, as we should be in Web 2.0 as well, is the ability of the infrastructure to meet specified business and operational goals automatically. We shouldn’t have to sit down with a slide ruler and determine the right combination of network speeds, current application loads, and the amount of RAM available to meet a business-defined SLA (service level agreement) or performance guarantee.

What we eventually want to end up with, what we’re truly working toward, is the ability to specific a single policy that says “Application A Response Time Must be Less than 5ms” and let the infrastructure figure out how to meet that goal. The problem is this decision can’t be made by any single component; it has to made in the context of all the components. A router can’t decide to route to network A just because the response time would be better because it may be the case that the application instance it ends up routing to is already bogged down so much that its response time alone would push the total response time over the agreed upon performance metrics. That’s what makes context so important to the decision making process; what makes it imperative that context be shared across the infrastructure and the whole infrastructure be capable of integration. So the “big picture” is able to be understood and acted upon in such a way as to bring about the desired result: an application response time under the specified business limits.

We’re closer to that than you might think, but the implementation is scattered around in the infrastructure in varying degrees of readiness to meet what is, I’ll admit, such a lofty goal. But if the infrastructure has the visibility into the various factors that affect performance and is integrated with the infrastructure responsible for managing each of those factors, and can make decisions based on that information, then we can get to the point where the burden of managing the disparate pieces of the grand data center puzzle is shifted off of people and onto technology.

Infrastructure 2.0 is just the beginning of the story. It’s not the goal, the real goal is a dynamic infrastructure. Infrastructure 2.0 is the way we’re going to make that happen. Eventually.

Cisco, Infoblox, VMware Webinar Now Viewable On Demand

Mon, 05 Oct 2009 17:17:52 PDT

You can watch the recently recorded 60+ minute Nemertes webinar without registering via the infrastructure 2.0 blog. Speakers: Cisco - Chris Hoff; VMware - Mark Thiele; Nemertes - Andreas Antonopoulos; Infoblox - Richard Kagan.

You can view the webinar here:

Infoblox Webinar | Virtualization and the Future of the Network.

James Urquhart: Infrastructure 2.0 Young Turk

Thu, 01 Oct 2009 14:02:47 PDT

James just joined the Infrastructure 2.0 Working Group but has been discussing infrastructure 2.0 issues as long as anyone. I’ve requested a picture, but until I get one I’ll substitute with an image from Wikipedia of prominent Young Turk Ahmet Riza.

Here is a collection of related articles/blogs by James, tracing back to 2006:

Arguing for service virtualization in place of server virtualization (October 2006)
http://blog.jamesurquhart.com/2006/10/service-virtualization-defined.html

Exploring enterprise computing as a complex adaptive system (April 2007)
http://blog.jamesurquhart.com/2007/04/complexity-and-data-center.html

Early thoughts on software ‘fluidity’ (December 2007)
http://blog.jamesurquhart.com/2007/12/how-fluid-is-your-software.html

Discussions about VirtSec and the network with Rich Miller and Greg Ness (December 2007)
http://blog.jamesurquhart.com/2007/12/software-fluidity-and-system-security.html

Follow the Law computing
http://blog.jamesurquhart.com/2008/06/follow-law-computing.html

His first clarion call for Infrastructure 2.0
http://blog.jamesurquhart.com/2008/11/in-cloud-computing-good-network-gives.html

The coming revolution that is workload mobility
http://news.cnet.com/8301-19413_3-10127654-240.html

The lost opportunity for telecoms to provide consolidated service
http://news.cnet.com/8301-19413_3-10310644-240.html

You can follow James at www.twitter.com/jamesurquhart. He has also contributed to the infrastructure 2.0 blog.

OK... here is his picture:

Are You Ready for "Just in Time" IT?

Wed, 30 Sep 2009 16:45:48 PDT

In a matter of decades we watched the data network eliminate “middlemen” and arcane practices that had been around for centuries, only to see these practices re-emerge at the core of the network. Understanding this irony is pivotal to understanding what is about to take place within the network and its effect on the evolution of IT.

World history is filled with examples of the triumph of mobility and economy, and the similar triumph of automation over manual labor. The IT industry is no different. The fortunes of network equipment vendors will ride on how well they address a critical shift in thinking that has been accelerated by the rise of virtualization.

When virtualization entered the data center it marked the beginning of the end of (manual) labor intensive IT tasks. By decoupling application from dedicated hardware virtualization enabled a cottage industry of solutions dedicated to automation and change management. It also reinforced a profound shift in thinking that was already underway.

Cisco’s Chris Hoff mentioned this on a recent webcast when referring to prescient Carnegie Mellon research from 2001:

The rise of VMware, Citrix and Microsoft’s virtualization solutions presented both a cultural metaphor and a technological capability in plain view of IT pros and line of business leaders: automation of systems saved money and made IT more responsive. Networks managed by scripts and spreadsheets will seem slower and costlier as virtualization spreads.

The network equipment industry was not ready to play; that is, until VMware began its brilliant alliance-building strategy with several of the leaders. Products were slow to emerge (because of product limitations and vendor myopias) but the strategic framework of the mesh IMHO was starting to be architected.

I think VMware’s leadership in strategic alliances with the likes of Cisco and others gives them a competitive advantage in reaching the mesh first and translating it into market power.

Recently VMware’s Mark Thiele articulated a bold new data center vision that really looked like a model for driving the cloud computing market skyward. It showed how applications could migrate from one location to another, regardless of distance and in pursuit of even temporary advantages/savings or in avoidance of disasters.

The data center mesh Thiele predicted promises incredible economies and efficiencies and the ability to respond quickly and opportunistically to localized developments. It produces flexibility at an unprecedented scale compared to static “Fort Knox” facilities.

Cisco’s Hoff appears on the same page as he explores the implications of the shift in thinking:

One’s natural inclination is then to ask: If this is so powerful then why aren’t most IT departments doing it? Hoff a few days later discusses the technical hurdles to true VMotion in The Emotion of VMotion:

Don’t get me wrong, I think VMotion is fantastic and the options it can ultimately delivery intensely useful, but we’re hamstrung by what is really the requirement to forklift — network design, network architecture and the laws of physics. In many cases we’re fascinated by VM Mobility, but a lot of that romanticization plays on emotion rather than utilization. – Chris Hoff- Rational Survivability Sep 2009

Between Thiele and Hoff you have a model for how cloud computing could work as a standalone business (with meshes of small data centers); and a crisp explanation for why the dynamic mesh hasn’t yet happened. You can also see the return of the network as a strategic enabler of IT’s evolution. As Cisco’s James Urquhart blogged late last year, the network is the final frontier for cloud computing.

Yes, it makes sense for network vendors to trumpet the network as being strategic to high profile enterprise IT initiatives. Yet VMware could have ignored these vendors all along and viewed them as potential competitors. Instead they embraced them. And I think they saw the power of the mesh before anyone else and embraced the vision.

Stateful VMotion (or portability) is the most elegant solution to addressing the declining cloud economics of density and complexity. For that reason alone it is likely that the tech hurdles will be addressed sooner rather than later.

Unlike standards-based efforts that can play out like an ongoing game of Prisoner's dilemma, the gold spike scenario means that whoever achieves greater range and mobility first will have a significant competitive advantage.

The pony express, for example, lasted two days after the completion of the transcontinental telegraph.

While some network equipment vendors watch, others will be replacing manual tasks and processes with policies and mouse clicks. History will repeat itself once again.

Thiele's Mobile Data Center is a Mesh

Tue, 29 Sep 2009 11:42:41 PDT

As Cisco, HP, Juniper, F5, Amazon, Microsoft, Google and others circle the data center field of battle, one strategic ridge of high ground is in network automation, or the ability to move (or adjust/provision) IT assets at the push of a button without the need for extensive manual intervention. That manual intervention accounts for a sizable portion of the costs, risks and compromises of today’s static networks.

Several of these companies have already made notable announcements in 2009, yet none have yet to let the virtualization genie out of the VLAN bottle completely. When they do, watch out for a new wave of IT innovation.

Last week I listened in to four experts talk about Virtualization and the Future of the Network. After Nemertes’ co-founder Andreas Antonopoulos shared recent findings on the shift in virtualization drivers (from capex savings to flexibility), VMware’s Mark Thiele shared his perspective on how far flexibility could go and the potentially powerful economics of an evolved network.

Thiele’s Vision

Instead of isolated data centers around the world Thiele illustrated a dynamic and resilient mesh of processing power capable of quickly adjusting to developments as they occur. That mesh could avoid disasters by simply shifting its applications and processing cycles from one location to another in real time. It could also shift for other reasons, including localized cost increases or brownouts or a sudden change in a cloud providers terms of service or service level agreement.

Thiele makes his point on slide 13 as a hypothetical tsunami prepares to strike the Southern California coastline, applications and servers move in advance of its wake to optimized locations:

Imagine the sheer amount of resources (people, time, expense, etc.) that it would take with today’s static network infrastructure to move servers from one side of a continent to another or even to change cloud providers. Then add a little risk, a few angry users and a mountain of tasks and procedures to ensure that networks stay secure and available.

That’s why infrastructure 2.0 is Virtualization's Golden Spike; it enables a new level of scale, find ability and flexibility driven by powerful economics.

Thiele has shown that the evolution of IT may very well depend on the evolution of the network. Those players with the experience and technology to deliver on the promise are likely to win, despite offshore competition from low end players or category crossers convinced that the tired “speeds and feeds” mantra will carry them into new markets.

The development of dynamic linkages between physical and virtual infrastructure will unleash incredible innovation and automation. The outcome is inevitable; it makes too much sense:

Disaster avoidance is one of several infrastructure 2.0 payoff scenarios, which include: 1) VMotion between data centers (for avoidance, recovery, economy, regulatory developments, etc); 2) changing cloud providers to adjust to short term service/price adjustments; 3) IT integration from M&A; 4) using disaster recovery assets to provide incremental capacity; and 5) implementing real disaster recovery in smaller data center environments.

The first network vendor to deliver on this promise gains a significant advantage: the ability to sell gear that dramatically shrinks IT opex and enables massive flexibility and scale for a fraction of the upfront expense. This vision is the equivalent of supply chain for IT and represents a new era of computing potentials enabled by an even more strategic network.

If you’re still scratching your head wondering how a case can be made for additional investments in network gear and innovation in a tough economic environment, check out one of Thiele’s other slides on data center costs. He shows how two Tier 2 data centers empowered by infrastructure 2.0 can be build for less than half the cost of a single “Fort Knox” type data center.

Yes, the network is back again thanks to the potentials of virtualization and the new steam locomotive of IT, the portable virtual machine. Market caps of the networking players are bound to fluctuate as one or another reaches higher ground and competitors face increasingly uphill battles.

While some network vendors may see problems with virtualization, those who see the opportunity and can gain the high ground might experience a new era of growth.

Mark Thiele has articulated the payoff from his vantage point of running data centers for a living… for the leader in virtualization. He is also a founder of www.datacenterpulse.org.

I am a senior director at Infoblox. You can follow my rants in real time at www.twitter.com/archimedius.

Infrastructure 2.0 Isn't Just For Cloud Computing

Tue, 29 Sep 2009 04:23:01 PDT

Operational efficiency in the cloud comes in part from automation and orchestration as well as from the outsourcing of management and maintenance of the hardware. While you can’t achieve the latter without cloud or hosting externally, you can realize a lot of the same efficiencies in a traditional architecture just by leveraging existing collaborative capabilities of infrastructure 2.0.

Glenn Gruber of Software Industry Insights in “Who’ll Be the First to Offer Cash for Infrastructure” (which is a great read in general) says:

And for those who are thinking about evaluating a private cloud strategy (although a cloud on your own infrastructure in my view is not a cloud), they may be able to get more value out of virtualization and other strategies, but they lose the operational efficiencies and instant scalability to match spikes in demand that the Cloud has to offer.

I won’t argue the private cloud comment. Really, I won’t because what’s important is the reminder that cloud isn’t the only way to realize operational efficiencies in data center architectures. This is important because not all applications are going to be able to move to “cloud” – whether internal or external. Some applications are just not suited to a shared environment for many reasons including aging technology and the cost to essentially “rip and replace” the systems. But improving the efficiency of delivering those applications is something we can do whether we’re “going cloud” or not.

It’s easy to forget amidst the hype (or maybe the air is just really thin up here in the clouds) that there are other ways of doing things that can be just as beneficial to achieving operational efficiency. It’s true that some operational efficiencies associated with cloud can’t be achieved without, well, a cloud; on-demand scalability is part and parcel of the definition of a cloud and without cloud that capability – and its operational benefits – simply can’t be realized. But there are many other efficiencies – particularly those associated with automation and orchestration and integration – that can be realized by just about any architecture.

Automation and orchestration aren’t just for cloud
The operational efficiencies that come from automating and orchestrating processes in the data center usually associated with cloud and virtualization can be applied to any architectural model, provided the infrastructure is capable of being automated. The reduction of errors associated with manual operations and the time-savings can provide real operational benefits in terms of time recovery and financial efficiency. Whether via scripting or a standards-based control plane API, automation of mundane operational tasks (take down a server for maintenance, bring it up afterwards) can reduce the burden on staff by offloading the tasks to automation systems.

Infrastructure 2.0 isn’t specific to cloud or virtualization; it’s an evolutionary step in infrastructure that provides the framework through which automation and orchestration can be achieved. It’s the foundation for a dynamic infrastructure, the definition of which does not preclude the use of its capabilities in architectures other than cloud.
Feedback loops that include applications make (smarter) real-time decisions possible
The integration of status, capacity, and performance information from applications with the rest of the infrastructure means security, performance, and availability focused infrastructure can make better decisions in real-time regarding how best to distribute requests across resources. If one application instance is nearing capacity or processing a particularly intense request, that information can be relayed to the infrastructure such that the next request to the application is directed to another more capable instance. The ability to incorporate feedback from applications and across the infrastructure means decisions made by individual components are made within the context of the entire data center, not just its immediate network environment.

This integration is really part of automation and orchestration, as it implies that if actionable data is shared with the infrastructure subsequently the infrastructure will be able to use that information to make decisions without human intervention. And because it’s making decisions based on current information rather than on a static configuration the entire data center executes more efficiently by leveraging the resources available to meet demand as well as business requirements.

This is one of the core tenets of Infrastructure 2.0: collaborative feedback that results in actionable data upon which intelligent systems can make smarter decisions.

Cloud computing models offer greater operational efficiencies because the physical tasks associated with management and maintenance are completely removed from the shoulders of IT in addition to the more flexible use of resources and the inherent benefits of automation and orchestration . While you won’t be able to skip the monthly maintenance on your servers if you aren’t going to a “public” cloud model, you can apply some of the architectural lessons learned from cloud to traditional architectures to realize many of the same operational benefits.

Infrastructure Integration: Metadata versus API

Fri, 25 Sep 2009 03:51:58 PDT

Infrastructure 2.0 requires collaboration. Collaboration requires the ability to communicate. The ability to communicate requires integration. But how that integration will happen may shape the future of infrastructure and network architecture.

There is a growing recognition of the basic problems associated with the rapid rate of change inherent in on-demand architectures (cloud) and the complexity that comes from virtualized data centers. Challenges such as IP address and application management, visibility, and last but not least, integration.

Yes, that most dreaded of all technology concepts has finally come to the network.

The answer to the growing challenges of managing rapid change is automation and orchestration, but in order to build such solutions there is required the ability to integrate infrastructure – both with other infrastructure solutions and with the management systems and platforms that will actual control the orchestration of the data center. The need for this infrastructure integration is rising in awareness. That’s a Good Thing. But questions remain regarding how that integration should be achieved; what form should it take?

While traditional EAI (enterprise application integration) technology originally took the form of API-based integration – that is, libraries that included functions that could be invoked to execute specific functions – in later years with the advent of SOA and Web Services metadata-based integration patterns became much more popular. Metadata-based integration reduced the cost to create, maintain, and support integration libraries for the vendors and insulated customers from changes and the nitty-gritty details.

But then Web 2.0 and social networking became all the rage and integration between those sites reverted to the traditional API-based method with a slight twist. Rather than rely upon completely proprietary data formats, i.e. created specifically for the application, they began to offer both JSON (JavaScript Object Notation) and XML formats to exchange data. While not completely interoperable – the data itself is not compatible across applications – the format is, at least across platforms, languages, and implementations.

Infrastructure 2.0 needs to look at what has – and hasn’t – worked in the application space, and learn from it to lay a solid but extensible foundation for the future of infrastructure integration.

DIFFERENT STROKES FOR DIFFERENT FOLKS

Infrastructure solutions today use a variety of mechanisms to collaborate. The primary purpose has been to allow third-party development of management applications for specific applications or platforms, though there has also been a smattering of enterprise usage for specific integration purposes with data center management systems. Infrastructure today has also generally accepted as a standard format XML, though whether that’s via a RESTful API or a SOAPy API is very much dependent on the vendor’s view of the world and what its typical users demand.

There are a lot of infrastructure solutions (and even more announced/coming after VMWorld this year) that are API-enabled. The thing is that just like Web 2.0 and social networking APIs no two APIs are the same. That means configuring a VLAN on a Cisco switch or an F5 BIG-IP or an HP ProCurve Switch is a completely different process. The API calls themselves, the data required, the process – each is unique to the product. This complicates application portability across clouds (or data centers) because the orchestration and automation that enables a dynamic infrastructure is implicitly tightly-coupled to the infrastructure. That’s okay for the cloud provider, because they’re probably – like you – standardizing on certain vendors so it isn’t going to be a problem for them. And the granularity offered by the various APIs provides them with the ability to build out automation and orchestration solutions that are tailored to their environment. The more that can be automated the more simplified the provisioning process, which in turn offers value to its customers and the ability to differentiate in the market.

But when you try to take an application that may require services from security, acceleration, load balancing, IPS, IDS, firewall, and other infrastructure support solutions from one environment to another, that’s where the differences in the API becomes apparent – and problematic. You can’t automate the migration via the API because the product in one environment may be different than the next, and therefore such a method would be useless. The answer is, of course, to somehow just share the configuration data, but today that is just as tightly coupled to products as APIs.

There’s no standard way to share the metadata – the configuration - that describes those requirements across vendor lines. When you request configuration data from product B it’s completely different from that of product A, and neither one can completely understand the other. So what’s needed is a standardized but extensible metadata format – and a way to share/consume that metadata across clouds and data centers. That’s the concept behind constructing a mechanism through which metastructure data can be published, shared, and consumed, anyway.

NOT MUTUALLY EXCLUSIVE METHODS OF INTEGRATION

When it comes down to it the use of metadata and APIs to integrate and collaborate in a dynamic infrastructure is not an either-or proposition. On the contrary, in fact, both will be critical to the success of infrastructure 2.0 to solve the challenges associated with implementing a truly dynamic infrastructure.

APIs will be necessary to specifically automate and orchestrate data center operational and business processes while metastructure hubs will be necessary for portability, upgrades, and reconfiguration efforts. While it certainly appears, at least at first glance, that metastructure hubs and the metadata integration approach would work well for both design (configuration, a.k.a. governance) and run time dynamism, metadata integration does not enforce any order of operations. Can’t and shouldn’t enforce any order of operations. Infrastructure interested in certain events or data subscribe to a topic or channel and receive (or pull, depending on the model) updates at varying rates. Processes generally require that certain tasks be complete ere the next one begins, and thus require more control. That control comes from an API and a management system capable of executing specific automations across the infrastructure in the specified order at the specified time under specified conditions.

Though it might appear at first that the there are two competing methods of integration to enable the dynamic infrastructure, nothing could be further from the truth. Both metadata integration and API-based integration will be required to build out a truly portable, dynamic infrastructure. And if we look at what’s happened in the web application space, we see that it, too, has compromised on a combination of metadata (standardizing on XML and JSON) and APIs to enable the cross-application sharing of data and functions that essentially today make up the “social networking web”.

Interestingly enough it seems to be working for Web 2.0. Hopefully we’ll see the same kind of success and adoption if we enable similar integration mechanisms for Infrastructure 2.0.

Nemertes- Virtualization Drivers Shifting from Capex to Flexibility

Thu, 24 Sep 2009 11:30:39 PDT

Andreas Antonopoulos kicked off our recent webinar on Virtualization and the Future of the Network with some interesting research findings. Based on your own perceptions and existing projects, this may surprise you:

Since the early days of virtualization there has been a shift. Agility is now a more influential driver than cost savings.

You can view the entire preso here. Of course you may want to watch the on demand webinar which will be posted in early October.