Nati Shalom's Blog

GigaSpaces Cloudify & VMware CloudFoundary the new PaaS Jailbreaker

2012-01-16T13:47:15+01:00

I was reading Krishnan Subramanian's post, Two Events That “Clouded” Our Thinking In 2011. The thing that caught my attention was Krishnan's comments on why PaaS is a superior alternative to DevOps:

The shift in the thinking about the enterprise cloud consumption also poured water into the “DevOps” concept advocated by vendors and pundits with their foot in the IaaS world. When organizations embrace PaaS instead of infrastructure services, we don’t need the DevOps marriage and the associated cultural change (believe me, this cultural change is giving sleepless nights to many IT managers and some consultants are even making money helping organizations realize this cultural change). With PaaS, organizations can keep the existing distinction between the Ops and Dev teams without worrying about the cultural change. In fact, with cloud computing, the role of the Ops is not going away but it stays in the background offering an interface which developers can manage themselves.

Krishnan represents one of the common attitudes and subjects of debate between two main paradigms for developing and managing applications on the cloud:

PaaS -- PaaS takes a developer, application-driven approach. A PaaS platform provides generic application containers to run your code. The PaaS platforms deals with all the operational aspects needed to run your code such as deployment, scaling, fail-over, etc.
DevOps -- DevOps takes a more operations-driven approach. With DevOps, you get tools to automate your operational environment through scripts and recipes, and keep full visability and control over the underlying infrastructure.

The Difference Between PaaS and DevOps

Both PaaS and DevOps aim toward the same goal -- reducing the complexity of managing and deploying applications on the cloud. But they take a fairly different approach to deliver on that promise.

Christoper Knee summarized it in his blog post DevOps and PaaS -- Friend or Foe? as the difference between Developers and SysAdmin:

Developers may ask: "if I have a self-service portal for deploying applications (aka PaaS), do I need SysAdmins at all?"
SysAdmins may ask: "isn't PaaS just a monstrous black box that prevents me from provisioning the specific services we need to deploy real-world apps?" ... The typical SysAdmin thinks that they can get to 75% of PaaS functionality with DevOps tools like Chef without giving up any systems architecture flexibility.

In one of my earlier post on the subject, Productivity vs. Control tradeoffs in PaaS, I tried to outline the main limitation of most of the current "blackbox PaaS" implementations:

I thought that Carlos Ble's post Goodbye Google App Engine (GAE) is a good example that illustrates why the initial perception behind GAE as a simple platform that provides extreme productivity can be completely wrong.

...developing on GAE introduced such design complexity that working around it pushes us 5 months behind schedule.

Part of the reason that brought Carlos through that experience IMO is that in the course of trying to make GAE extremely productive the owner made the platform too opinionated, to the point where you lose all the potential productivity gains by trying to adopt their model. In addition, with a platform like GAE you have very little freedom to leverage existing frameworks such as your own database, or messaging system, or any other third-party service that can in itself be a huge contributor to productivity.

Instead, you're completely dependent on the platform provider's stack and pace of development, and that in itself can work against agility and productivity in yet another dimension. In this specific example, Carlos couldn’t use a specific version of a Python library that would have made his productivity higher, and instead had to work around issues that were already solved elsewhere. This is a good example how the lack of flexibility leads to poorer productivity even in the case of simple applications.

Putting DevOps & PaaS togatehr

It looks like more people in the the industry have come to recognize that rather than looking at DevOps and PaaS as two competing paradigms, it might be best to combine the two, as Christoper Knee pointed out in his post:

What if you could get a PaaS that wasn't a black box, enabling developers to deploy apps easily while still giving SysAdmins the ability to provision any services they needed (a la Cloud Foundry)?

I also came to that realization myself, as I pointed out in my 2012 predictions:

In 2012, we’ll see many of the DevOps tools, such as Chef and Puppet, integrated into application platforms, making it easier to deploy complex applications onto the cloud. In the same way, we’re going to see more Application Platforms adopting the automation and recipe model from the DevOps world into the application platform. The latter have the potential to transform the opinionated PaaS offerings as we know them today, with Heroku and GAE leading that trend, into a more open PaaS offering that better fits the way users develop apps today, and provide more freedom to choose your own stack, cloud, and application blueprint.

What Makes a Cloudify and CloudFoundary a PaaS Jailbreaker?

The CIO Maginze article Cloud computing disrupts the vendor landscape defines a new class of PaaS platforms that I think represent the definition for a DevOps PaaS:

A PaaS that allows developers to use whatever tool they want to build their cloud applications and the platform tackles the deployment, scaling and management of these apps in the cloud data center.

VMware CloudFoundry is one of the more notable references in that category. Quoting Christoper Knee:

CloudFoundry runs anywhere, incuding on your laptop. CloudFoundry's service container concept is particularly strong, kind of an appliance on steroids.

These ideas were the founding concept behind Cloudify, i.e. putting DevPops & PaaS togather in a single framework. As with CloudFoundry, Cloudify enables you to break away from the "blackbox PaaS". However, even though CloudFoundry is significantly more open than most other PaaS alternatives, at the core it is still based on the "my way or the highway" approach (aka opininated architecture), which forces you to fit into a specific blueprint that is mandated by the platform. Cloudify, on other hand, pushes the envelope even further by adopting the concpet of recipes that was first introduced by DevOps frameworks such as Chef and Puppet. It introduces more application-driven recipes through the introduction of Domain Specific Language that extends upon the groovy language.

The Cloudify recipes give you the full power to plug in any application stack on any cloud (including a non-virtualized environment) and manage them in similar way to the way you would manage them in your own datacenter or machines. You can also call Chef and Puppet from within the recipes. All this, without hacking the framework itself. As with other similar DSLs, the Cloudify DSL was designed to express even the most complex application management tasks, such as <recovery from a data center failure> in a single line and avoid all the verbose script and API calls that are often involved when you work at the infrastructure level.

All this makes Cloudify an even more open alternative that fits a large variety of the current enterprise application stacks, including:

JEE applications

Big Data applications

Multi-tier applications

Native applications (C++,..)

.Net, Ruby, Python, PHP applications

Multi-site applications

Low-latency applications (that can't run on VMs)

It also makes Cloudify more open for special customization of the existing stack, like in the case of:

An application that needs a certain version of MySQL (not the one that comes with the framework)

An application that needs to run on Redhat (not Ubuntu). or even more interesting -- a case where there are mutiple applications, each needing a different OS served at the same time.

Cloudify also provides a more advanced level of control geared for mission-critical apps, including:

Monitoring the application stack and topology

Adding custom application metrics

Adding custom SLAs

It can also work on wide variety of cloud environments, including Microsoft Azure and non-virtual enviroments.

One of the great powers of the recipe is that it is a great collaboration tool. Once you develop a recipe, it is very easy to share it wtih different groups -- whether internal groups like developement, QA, and operations, where the recipie provides a programmatic definition of thier environment, or it can be shared between the product team and the pro-services team, where your sales and pro-services can easily install and update product versions in a consistent way, as well as reproduce customer scenarios and share them with the support team. Recipes are also a great tool for collaboration through a wider community network, where users can collaborate by sharing common recipes and best practices over the web.

Quick Introduction to Cloudify Recipes

Below is a short snippet that shows a simple application recipe, of a typical java-based web application based on JBoss as the web container and MySQL as a database. The application recipe describes the services that comprise the application, and thier dependencies. The details on how to run MySQL and JBoss is provided in a seperate recipe for each of the individual services. A more detailed description of how a service recipe would look like can be seen here.

application {
       name="simple app"

       service {
              name = "mysql-service"
      }
       service {

              name = "jboss-service"
              dependsOn = ["mysql-service"]
       }
}

To get a taste, you can try one of the available recepes such as Cassandra, MongoDB, Tomcat, JBoss, Solar, etc., just as a simple way to try out these products on your own desktop or on any of the supported clouds, without the hassle that is often involved in doing so and without even direct relation to Cloudify per-se.

References

Realtime Analytics for Big Data: A Facebook Case Study

2012-01-09T22:04:25+01:00

via www.youtube.com

2012 Cloud, PaaS, NoSQL Predictions

2011-12-13T08:47:45+01:00

2011 is coming to its end and now is a good time to start planning for 2012. I thought that a good start would be too look at my 2011 predictions and if my previous (and first) attempt to predict someting in that turbulent environment held any water...so, here is a quick recap of 2011.

Recap of 2011

Private vs. Public Cloud - As I noted in my recent post Public vs. Private Clouds I felt that during 2011 the debate around public vs. public cloud would become less interesting, as most of the industry has started to accept the fact that there is a need for both environments, and the important issue would become how to make them work well together. The most interesting development in that regard was Rackspace’s recent announcement about their plan to support OpenStack based private clouds, which shows that even public cloud providers have fully embraced this idea.

OpenStack is evolving from a movement into a viable reality - the momentum around OpenStack has gone through ups and downs throughout the year as happens with every new technology. However looking back, it appears that 2011 was a fairly successful year for OpenStack with its first public cloud available already in the market. Dell and HP have started to offer the OpenStack based cloud to their customers, as has Citrix. Rackspace announced their plan to provide official support including for those who want to build their own OpenStack environment…that's quite big considering the short timeframe from when the technology was first introduced…still there is a long way to go but the future looks promising - check out this survey in that regard.

PaaS adoption has been happening at a slower pace than expected, despite the fact that the trend remains consistent. For PaaS startups 2011 was a fairly significant year with the acquisition of Heroku by Salesforce. Amazon Redhat and VMware joined the PaaS arena; Amazon with Elastic Beanstalk, Redhat with their OpenShift initiative, VMware with CloudFoundry, adding to its previous acquisition of SpringSource vFabric. This was a fairly significant year for us at GigaSpaces as we launched a new product in this same domain that aims to completely change the way PaaS is being taught today (stay tuned…).

Google App Engine have no doubt been the disappointment of the year by literally killing GAE as we knew it (amongst many other things) with their new pricing model.

Big Data has gone real time. Facebook made a big announcement on how they moved their batch-oriented analytics system to real time analytics (See my previous posts on this subject here and here). Twitter announced a launch of a new Real Time Analytics dashboard; while both join Google and Yahoo who have already started to make this shift. Google has also been transforming their web analytics framework into real time. As I noted in my 2011 predictions, the entire debate around NoSQL and SQL didn't make sense, and indeed we’ve seen quite a few announcements both from Cassandra and Couchbase on their support for SQL-like query support.

In Memory Data Grids have also taken a similar approach where, with GigaSpaces, we’ve launched our JPA support, other Data Grid implementations such as Infinispan and Gemfire seems to be heading in that same direction each adding different levels of SQL support. The interesting development in this regard is that we were able to prove that you could actually mix and match Document/Schemaless APIs with SQL APIs and have the flexibility to choose the right language for the job (See online demo Same Data Any API).

All in all I think that I came fairly close - don't you think…?

Ok that gives me enough confidence to try the same thing for 2012.

2012 predictions

Cloud

iCloud everywhere - IMO the biggest shift in Cloud is the fact that it’s going to become pretty much invisible to many of the end users as new mobile devices, operating systems and applications start to be designed with cloud support in mind. Apple iCloud and DropBox mark the beginning of this trend. Using cloud for collaboration and synchronization is definitely a killer app for many of the consumer based apps. I expect that in 2012 we’re going to continue to see a big push of many SaaS-based offerings in that space toward rich client support that uses the cloud as a backend and leverages the power of the new generation of advanced mobile devices. The difference is that those clients won’t be just another frontend for the same web UI, but something that will run almost entirely on the mobile device and will use more generic cloud services for synchronization and collaboration. This will create the need for more generic cloud services such as database as a service and other middleware services that can interact directly from mobile applications.

Moving from Amazon-centric clouds to Cloud Mashups – In 2011 we started to see new kinds of clouds starting to pop up. Literally every hardware vendor (IBM, Dell, HP,..), telco (ATT, Verizon, KT), and software provider (Oracle, Microsoft) are either developing or already offer something in this space. Each one tries to maintain a unique position to compete with Amazon either through SLAs, locality, security, or being more open through the support of OpenStack. In 2012, this movement is going to become even stronger as many of the players that have been making the investment during 2011 will come out full speed ahead in 2012.

Microsoft finally gets it with Azure - Microsoft has been around for a while with Azure with somewhat marginal success mostly around its .NET user base, an approach that is too narrow a play when it comes to cloud. Their cloud strategy is coming into focus with the offering of a more ubiquitous cloud supporting technologies that were previously unheard of on a MSFT cloud platform - such as Java, PHP and it wouldn't be too far to assume that they will be supporting Linux applications in the cloud as well.

Cost-driven Application Management - One of the things that is still fairly hard to measure in the cloud is cost, and more specifically how each component of our application and architecture contribute to cost. This is specifically true during current market conditions which are going to put even more pressure on cost savings. Cost-driven application design patterns will start to emerge, and will become an integral part of any design for cloud applications just as scalability and performance are today. A new form of Cost Driven Application Management (CDA) will start to emerge to provide better insight on how our application behaves from a cost analysis perspective - Newvem is a new startup in that space that already launched their private beta.

Mission Critical Apps move into the Cloud - As the industry matures there is no reason why we should draw the line for cloud adoption at simple apps. The challenge will be mostly around performance, latency, and ensuring continuous availability. A new class of middleware and application platforms that are designed specifically for cloud environments will become more popular to help in that transition. On the other hand, Java and JEE specifically will finally become more cloud ready as I noted in an earlier post - Java and the Center Stage.

Network Gets into Cloud API Stack - While compute and storage have become virtualized to fit into the cloud, we haven't seen much advancement on the network layer. Many of the networking providers are now launching APIs to enable better control over the cloud network. Alcatel recently announced an interesting cloud proposition in this domain specifically targeting telcos. The idea is to use the network as a vehicle for making distributed data centers look like one big cloud, making it possible to better leverage existing assets and offer SLA driven compute resources based on latency, location etc. Other cloud providers are also starting to open their network APIs starting from the Load Balancer down to the core switch. This opens up a new set of opportunities for integrating these network APIs with the upper layer of the application stack.

More OpenStack Clouds - 2011 was the just the beginning of that trend, 2012 will see more public and private cloud providers offering support for OpenStack APIs with RackSpace, Dell, and HP already making public announcements in this area. The interesting question in this regard would be how Citrix will play out their CloudStack acquisition with its OpenStack strategy.

PaaS

DevOps and PaaS Converge into App DevOps PaaS - One of the topics that drew a lot of my personal interest last year was the DevOps movement. For odd reasons, most of that movement was driven by Ops and less by Devs. In 2012, we’ll see many of the DevOps tools such as Chef and Puppet integrated into application platforms making it easier to deploy complex applications onto the cloud. In the same way, we’re going to see more Application Platforms adopting the automation and recipe model from the DevOps world into the application platform. The latter have the potential to transform the opinionated PaaS offerings as we know them today, with Heroku and GAE leading that trend, into a more open PaaS offering that better fits into the way users develop apps today and giving more freedom to choose your own stack, cloud, and application blueprint.

Beyond Google App Engine, Heroku - Heroku established itself as a one of the early PaaS providers in the market and is now expanding their offering to Java. CloudFoundry, DotCloud and others are slightly different but still follow the same path. In 2012 you should expect more choices for completely different PaaS platforms starting with JEE PaaS offerings from Redhat, IBM, and Oracle, to private PaaS offerings which essentially are frameworks to build your own PaaS, DevOps PaaS offerings (see note above), as well as vertical PaaS for specific industries. Magento announced their plan to provide PaaS for eComm and it wouldn’t be crazy to assume that others will follow that same path.

BigData

Not only Hadoop Centric - During 2010 and to a lesser degree 2011 Big Data discussions were pretty much centered around Hadoop. NoSQL solutions such as Cassandra and Mongo are gaining fast adoption mainly due to the operational and development complexity that comes with Hadoop. That movement is going to continue at an even greater pace, as Hadoop gets fragmented between many vendors and frameworks such as EMC, MapR, Cloudera, Yahoo, IBM each claiming to own their own Hadoop distro. With new funding in the hands of many of the NoSQL startups I’d expect to see more complete solution stacks targeted at Big Data.

In Memory Data-Grid and NoSQL Become Integrated - During the early days of the NoSQL movement it wasn't clear how the two technologies fit together. As I noted in my previous post here, it actually makes more sense to integrate the two technologies in a context of real time analytics for Big Data or real time data processing for Big Data. Indeed during 2011 I started to see more case studies showing the use of the two technologies as with Facebook and Twitter. MemBase is also a good example for that approach with their announcment earlier this year about their integration of Memcached and CouchDB together into a single product. At GigaSpaces we added built-in integration for Cassandra and MongoDB, as noted here, and plan to invest more in that direction during 2012.

A New Class of Big Data Application Platforms will Address the Development and Operational Complexity of Big Data Applications - As Big Data application become more mainstream we start to hit the next level of complexity, development, and operational complexity. Clearly plugging NoSQL into your architecture may address your scalability requirements but at the same time it’s going to make your development and management experience more complex. Not because the products themselves are complex, but mostly it’s because it is less obvious how to build and design the application around these new technologies. As in previous years, the goal of application platforms is to ease that task by putting together an integrated stack that makes it easier to develop Big Data applications as I noted in my post on Big Data Application Platforms.

References

Making Cloud Portability a Practical Reality

2011-12-08T21:49:35+01:00

In one of my previous posts Five Misconceptions On Cloud Portability I argued that:

The term "cloud portability" is often considered a synonym for "Cloud API portability," which implies a series of misconceptions. If we break away from dogma, we can find that what we really looking for in cloud portability is Application portability between clouds which can be a vastly simpler requirement, as we can achieve application portability without settling on a common Cloud API. ..

The following presentation shows how I could use the ideas from this post and provide a practical cloud portability solution today using Cloudify and JClouds.

via www.youtube.com

Public vs Private clouds (Again!)- it's not about the cost

2011-11-22T18:50:26+01:00

I was reading Geva Perry's post, 10 Predictions about Cloud Computing ..

As always I enjoyed reading Geva's throughts and for the most part comletely agree with him. The thing that caught my attention is Geva's point on private vs. public cloud. Geva re-iterated why the economical benefit of public cloud would turn private cloud into a small niche. Quoting Geva:

"Internal clouds will be niche. In the long-run, Internal Clouds (clouds operated in a company's own data centers, aka "private clouds") don't make sense. The economies of scale, specialization (an aspect of economies of scale, really) and outsourcing benefits of public clouds are so overwhelming that it will not make sense for any one company to operate its own data centers."

Last week I had an interesting discussion with Yoav Abrahami from Wix which made me rethink this entire argument. Wix runs their own managed data center on one of the hosters. They were able to build a pretty agile environment, which enables them to update their software a few times a day and in this way continually grow their business in this model. If they want extra capacity it takes them a couple of hours to get the hardware that they need. They pay a monthly fee and they don't need to commit to three years in advance for thier resources. The example of Wix is interesting as it doesn't fall exactly into the public cloud definition, yet they don't carry large part of a cost associated with the operation of a data center which makes the economical argument somewhat less relevant.

This brings me to the first point:

What makes a cloud private or public?

For most people, public cloud is synonymous with Amazon or Rackspace and private cloud is everything else. Which basically means that if you don't pay by the hour and can spawn x1000 of machines by a call of an API your not considered public cloud - is that really the right place to draw the line?

I beg to differ - the main thing that brought us to cloud in the first palce is agility at a reasonble cost. Agility in that context is our abiilty to launch new products, new features, and meet a growing demand continuously. Previously, it was the IT who stood in the way of the business and Cloud (public & private) emerged as a significantly better way to run IT. Wix is one good example of what some would consider a private cloud, but at the same time they were able to serve the agility and cost requirements of thier business. The thought that the only way to meet our business agility and cost would be through public cloud is wrong and there are many examples like Wix that prove otherwise. This brings me to the next point...

Its not about cost but ownership, and privacy.

The point in my previous argument is that the cost of running your own managed data center today can be fairly low and even lower than running on some of the known public clouds because you can optimize for a specific use case and workload. BeaTune (a referece from my previous post) is another good example in this regard. They claim that since they switched to HQ8 they were able to cut their infrastructure costs in half. Here are the exact numbers:

Amazon: “two servers $380/month. Add roughly $220/month for backup snapshots and EBS for the (pretty large) database and your total yearly costs are $,7200”

EQ8: “two 24GB RAM, i7-920 Quad-Core, 2x1.5TB HDD (Software-RAID 1) servers for $3,530/year at a hosting company that you can actually call

Source: Goodbye Amazon EC2 See you later cloud

The explanation that I have for that, is that with public clouds you're limited in the type of compromises that you can make, and therefore carry a fairly large cost just to be able to live to the promise of meeting huge peak loads, a high degree of failures, and more. If you design for a specific work load you could come up with different structure that fits a potentially more relaxed set of assumptions that could fit your business better and can result in a lower cost and even better performance and SLA. The other part of the explanation is that there is more than one mean to gain from the economy of scale even in the case of private clouds such as leasing the resources of our infrastrcture and using hosted data centers. A good analogy to that is leasing versus renting a car. If we run a fleet we don't have to buy the cars of the entire fleet, instead, we could lease those cars from a third party provider which uses the economy of scale to manage the cost and risk through his entire customer base. By leasing the car we still maintain pretty much the same degree of control as we would if we would own the car ourselves, but we have a more economimcal model to do so.

This brings me to the main point of this post. The important difference between private and public cloud is less about the cost but more to do with ownership and flexibility. I'll use another transportation analogy to illustrate my point. The cost of using public transportation is significantly lower than owning our own private car. If I only use the cost argument I could easily argue that owning a private car doesn't make any economical sense, still most of us wouldn't give up our private car, not because it is less expensive, but because we want the flexibility, and to control when and where we want to go, and are willing to pay extra to accommodate these needs. Pubic cloud in this example is analogous to public transportation; the fact that you can share the same infrastructure with many users enables the sharing of costs with all the users, and thus the cost per user is significantly reduced, while at the same time it means that you agree to compromise on some sort of a least common denominator which is also an attribute of that cost benefit.

If we agree on this observation, then its clear that niether private nor public clouds would ever go away but instead we'll be moving away from the old way of running data centers into a more modern data center, a hybrid data ceneter that can serve the need for control and cost. In both cases, controlling the cooling, real estate, and electiricity doesn't makes any sense, and indeed all that will be outsourced completely if that's not already the case. The main thing that will make private cloud different is who controls where and how our data is maintained, where and how our services will run, what type of infrastrcture (compute, network, storage) we choose to run our business, and most importantly how much we're willing to share that environment with other users to reduce the cost. Even with that, it's hard to draw a clear line between private and public cloud, as it may very well be the case that what's right in a particular case is a mashup between the two. For example, imagine the case where we will keep our data in a private cloud but run the compute nodes on a public cloud, is that a private, public cloud...who cares?

Final note

Many use the term private cloud to point to the old (and pretty clueless) way of running a data center. I argued that private cloud, as the name suggests, represents a business need to maintain high degree of privacy and control of its data and resources by ensuring that they are not shared with others. The key is the sharing aspect. The resources may very well be hosted somwhere else but the key is that they remain private. These needs will never cease to exist in the same way that we have public and private transportation, or the option to lease and rent a car, and will continue to have. The old way of running data centers must change and is changing to meet the cost and agility demand of the business, and is adopting many of the features of public cloud. With that the cost margin differences between private and public clouds becomes less signficant and for some scenarios has proven to be even cheaper (assuming that you can design for a particular work load and use case) in a private cloud sceanrio. All this makes the cost argument even less significant a factor for choosing between the two models.

Public cloud providers would also need to adapt to meet the demand of private cloud. Rackspace recently announced a new offering in that space around OpenStack.

Rackspace's announcement is another sign that any player that wants to compete in the IaaS market needs to offer some way to have a private and public cloud. Any offering with only half that is going to be left wanting.

In addition to that, Rackspace is conneting their hosting business which some could view as another form of private cloud with their public cloud offering which will give thier customers an even greater degree of flexability. Amazon already provides for a while now the Virtual Private Cloud (VPC) service which enables them to offer resources from the public cloud as just another node in the customer's private cloud. Amazon VPC doesn't provide the full degree of privacy and isolation of their network and resources, but is definitely another form of connecting private with public cloud offerings, and an indication of how Amazon has adapted to this demand. There are still many questions to be answered and challenges ahead, namely how to combine the two models well both technically and echonomically, but the main point is that the entire debate around private and public cloud, who's going to win should become moot. It's now time to focus our energy on how we combine the two.

References

Powered by Qumana

Five Misconceptions on Cloud Portability

2011-11-15T18:30:07+01:00

Three years ago when we started working on the first generation of our PaaS offering, cloud portability seemed to be pretty much “mission impossible.” At the time, we made a conscious decision to focus only on Amazon for our first generation PaaS as practically it was the only cloud in town.

Now, there are many different public and private cloud offerings: platforms like GoGrid, VMWare, Citrix/Xen, and Cisco UCS, with recent additions being OpenStack, Cloud.com, and Microsoft Azure. Frameworks like JClouds have been developed to make cloud portability an easier goal to reach.

As a result, cloud portability is not only a possibility, but it's easily done with the right constraints in mind.

Having said that, there are still too many options from the standpoints of API standardization, portable virtual machines, abstraction frameworks, orchestration frameworks, etc. None of them fully address the challenge and therefore a solution has to combine these options into one cohesive unit. Finding the right combination of features is a pretty tricky challenge and involves lots of trial and error, as the chances that you’ll pick the wrong ones are still pretty high, as we experienced ourselves.

In this and the next few posts, I wanted to share some of our experiences when developing Cloudify , our unified cloud deployment/management tool. I will start by covering five common misconceptions people have.

Five Common Misconceptions on Cloud Portability

1. Cloud portability = Cloud API portability

This is perhaps the most important observation in this entire discussion. When people think of Cloud portability, the immediate thing that comes to mind is Cloud API portability - a standard API, or a common abstraction, that maps all the different API’s into one common façade, right? Well... wrong.

What you're really looking for is the ability to run your application on different cloud platforms without any code change. Having a common or standard API is one way to achieve that goal but not necessarily the most practical one, given the speed in which cloud APIs evolve. This brings me to the first observation: Application Portability != Cloud API portability. Let me explain:

Most applications don’t need to interact with the cloud-specific API from the application itself. Most of the cloud API deals with stuff that happens outside of the application code, like starting new virtual machines or services, or providing elasticity. In many cases, systems management tools are the primary specific aspect for a given cloud platform; most provide support for managed middleware like MySQL (RDMS), Tomcat, memcached, or even map/reduce (through Hadoop, for example.) The mechanisms for using these are standard and common, but the APIs for managing the services are not. (There are exceptions, such as SimpleDB and SQS, which do use proprietary and localized APIs. I believe that over time, you'll see less and less of this as there will be enough nonproprietary options to restrict their use.)

The problem of dealing with application portability between clouds is vastly different than the problem of dealing with API portability. API portability is easy; cloud API portability is not.

I’ll spend more time on how we can use this realization to simplify the task of achieving cloud portability in a follow-up post.

2. The main incentive for Cloud Portability is - Avoiding Vendor lock-in

Vendor lock-in is one of the major barriers to cloud adoption as indicated in a recent survey by 451, GigaOM and North Bridge Venture - Future of cloud survey shows significance of open source

Interoperability and vendor lock-in were ranked the next most significant challenges, both with 25% of response. Though we saw vendor lock-in fade a bit as a concern for customers two to three years ago, it has risen again as a major issue in cloud computing. We believe this is in part because of the early nature of cloud computing and a desire from users to avoid getting stuck with a cloud vendor, framework or technology.

Many (myself included) view cloud portability as a mean to address vendor lock-in as the main drive for Cloud portability. If we can run our application on any cloud without any code change, we can in fact avoid vendor lock-in.

This brings me to the second realization. Cloud portability is more about business agility than it is about vendor lock-in. Let me explain:

It is not very likely that you would switch entirely between one cloud provider to the other, at least not frequently. If you are going to change cloud providers, its going to be more of a one-off event than something that you would practice on a regular basis.

On the other hand, it is probably more likely that you would use cloud portability to choose the right cloud for the job. A common example would be to run your test on the cloud and production on your own private cloud. Another example would be to run your demos and trials on the “cheapest” cloud and production with a cloud provider that offers a higher service level. A more advanced scenario would be to use cloud portability for cloud bursting.

In other words, cloud portability is about business flexibility. It gives us more choices between cost, SLA, and security tradeoffs between the different cloud offerings and product lines. Portability between our local data center and the cloud would also help in making the transition to the cloud more smooth.

3. Cloud portability isn’t for startups

If you're a startup, when the discussion on cloud portability comes up the immediate reaction is often “hmm.. interesting but not for me.” Being part of a startup company myself, I think that I can relate to that reaction. As I mentioned before, when I faced the choice for dealing with cloud portability a few years back, cloud portability was one of the first items that I took off of my list of TODOs to meet our time-to-market goal, and settled for Amazon.

Recently, I came across different cases that shows that dealing with cloud portability is often forced on us even when we don’t plan for it.

The first case is with MixPanel, a web analytics company who switched from RackSpace to Amazon:

When MixPanel started up, cost mattered as they were running out of their own pockets. In this case, every dollar counted and therefore the right choice at the time was Slicehost. As their business grew, Mixpanel switched from Slicehost to Linode and later to Rackspace due to a Ycombinator deal. As their business grew, they switched to Amazon who happened to provide more features and greater flexibility .

First was Slicehost back when everything was on a single 256MB instance.. Second was Linode because it was cheaper (money mattered to me at that point). Lastly, we moved over to the Rackspace Cloud because they cut a deal with Ycombinator... Even with all the lock in we have with Rackspace (we have 50+ boxes)..it’s really not about the money but about the features and the product offering, here’s why we’re moving”

source: Amazon vs Rackspace

The second case is BeaTunes that switched from Amazon to EQ8 Hetzner

In the case of BeaTunes, the trigger for switching off of Amazon was an Elastic Block Store (EBS) crash. During that time, I had to rely on basic support that Amazon provided. Given that BeaTunes is a small startup, they had almost no leverage to escalate their issue fast enough for the business need and had to hope that their particular issue would hit enough users in the forum to attract Amazon's support staff's attention.

They ended up choosing EQ8. Being a smaller hoster than Amazon made EQ8 a better fit for BeaTunes, a small startup itself. Even a small startup can run mission critical software, and when there is a failure, it is even more important to get a timely response as your entire future may rely on this. Unlike big companies, you often don’t have the means to survive such failures easily. It also happened that by switching to EQ8 BeaTunes could choose better hardware configuration that fit their needs and at half the price of the original services!

Amazon: “two servers $380/month. Add roughly $220/month for backup snapshots and EBS for the (pretty large) database and your total yearly costs are $,7200”

EQ8: “two 24GB RAM, i7-920 Quad-Core, 2x1.5TB HDD (Software-RAID 1) servers for $3,530/year at a hosting company that you can actually call

Source: Goodbye Amazon EC2 See you later cloud

These are only two samples, representative of a much bigger trend in the industry at large.

The interesting thing in both MixPanel's and BeaTunes' histories is that their choice of cloud changed over time due to changes in their maturity as a company, as well as changes in the business requirements which involve cost, support level, flexibility and feature set, etc. At each point in time, the right cloud happened to be a different cloud.

Personally, I found that issue around supportability in the case of BeaTunes quite interesting as quite often we tend to go for the brand thinking that it’s the “safest bet” where perhaps the right choice would be to choose the one that fits our size (and stage).

The main point in both examples is that even during a relatively short period of time, the two startups found themselves switching from one cloud to another. In the case of MixPanel, this happened four times during their company lifetime.

The thought that you can avoid such a move between cloud platforms is likely to be too optimistic, even if you’re a startup. Given that there are more options to deal with cloud portability today, and the effort is not as a great as was the case few years back, I would encourage every startup that is expecting rapid growth to re-examine their deployments and plan for cloud portability rather than wait to be forced to make the switch when you are least prepared to do so.

4. Cloud portability = Compromising on the least common denominator

By definition, standards tend to lag behind implementations. Standards are therefore a compromise on the least common denominator. In a dynamic environment, such as the one we're experiencing with cloud today, compromising on the least common denominator is a choice which may come with great cost on “reinventing” things that are already available outside of the standard or common API.

If we think of application portability, we don’t need to compromise on the least common denominator as most of the interaction with the cloud API happens outside of our application code anyway, to handle things like provisioning, setup, installation, scaling, monitoring, etc.

There are orchestration tools such as Chef and Puppet that can provide higher levels of abstraction for automating those processes between clouds without relying on a common cloud API. Frameworks like JClouds provide common abstractions to the common set of APIs (such as Compute and Storage APIs) while still allowing the application to interact with the underlying cloud-specific APIs and thus minimizing the areas of differences between clouds that would require specific handling.

5. The effort for achieving cloud portability far exceed the value

Many would argue that cloud portability comes with a cost and rightly so. Indeed, there is no such thing as free lunch, and cloud portability is no exception.

However, the cost isn’t as great as many of us think, especially now that there are more tools and frameworks available.

The effort to achieve cloud portability is far less than it used to be, in most cases, making it a greater and more valuable priority (with less investment) than it used to be.

Final notes

The term "cloud portability" is often considered a synonym for "Cloud API portability," which implies a series of misconceptions.

If we break away from dogma, we can find that what we really looking for in cloud portability is Application portability between clouds which can be a vastly simpler requirement, as we can achieve application portability without settling on a common Cloud API.

As in the case of MixPanel and BeaTune, choosing the right cloud for the job may vary over time. The right cloud when we create a new startup can be different than when we grow, and if we're very successful, we may find out that managing our own cloud is the right choice.

If we focus only on what’s needed to ensure application portability between clouds, we may find that cloud portability can be easier than it seems at first glance. If done correctly, it can result in greater flexibility for our businesses:

Choose the right cloud for the Job
Reduce vendor lock-in
Enable advanced deployments such as a hybrid cloud, and cloudbursting

In the next post I’ll touch on what it takes to turn cloud portability into a practical reality.

References

Behind the Curtains of Cloudify, GigaSpaces' new product

2011-10-24T14:46:39+02:00

Announcing a new product is a major and exciting milestone in a company's lifecycle. Our last announcment was the launch of XAP a few years ago - the Xtreme Application Platform aimed to provide a complete end to end scalability solution through the entire application stack, meaning Data, Web, Messaging, everything.

Today is one of those exciting days, as we are announcing a new product, Cloudify.

As its name suggests, Cloudify is primarily targeted at making the processes of bringing new and existing application to the Cloud extremely simple, to the point that it would require no code or architecture changes to deploy on any distributed platform.

In this post, I would like to share some of the motivations that led us to come out with Cloudify and provide a bit of an insight into how Cloudify is being used already.

The motivation behind Cloudify

The first thing that led us to Cloudify was the need to reduce the barrier of entry for applications that want to leverage the cost effectiveness and agility of the cloud, but are not yet ready to make the investment in re-writing their existing application, or switch to a new platform for that purpose.

In many situations, it was apparent that reducing the time to market in which those applications can offer through the cloud was a much bigger concern than making the application completely elastic. There are also those who are launching new applications but are not yet ready to make the switch to the cloud.

These organizations are looking for ways in which they can "future proof" their applications so that when they are ready to make the switch to cloud-based deployment, they can migrate with a simple configuration change. ISVs in particular, need this flexibility so that they can continue to serve their existing deployments while at the same time providing new cloud-enabled product offerings without the overhead of maintaining two completely separate product lines for that purpose and without disrupting thier current business.

The second motivation was around openness, and reducing the lock-in concerns. Too many cloud solutions and platforms introduce a high degree of lock-in, starting from the infrastucture, through the API and specific language and stack that is forced upon users.

Big players such as Oracle and VMware are starting to offer their own data center infrastructure and tie this even further into their platforms, leaving their customers with an even greater lock-in risk. As Oracle and VMware expand their stacks, more and more of their ecosystem partners are finding themselves in direct competition with little option of fighting back.

As many data centers become cloud-enabled through private cloud initiatives, it becomes more apparent that the right way to leverage cloud infrastrcture is through some sort of hybrid cloud. A common use case for mission-critical applications would be to run testing on the public cloud, for example, and the production application on a private cloud. Another use case could be to use the private cloud for running steady workloads and burst to the public cloud during peak loads.

Today, however, the private and public clouds look fairly different. Moving an application from one cloud environment to the other is still a fairly complex process. That brings me to third motivation: the need to provide a consistent cloud story behind both the private and public cloud.

Cloudify to the rescue

The heart of Cloudify relies on two main parts: a recipe model, which is a Groovy based DSL (Domain Specific Language) specifically designed to provide cloud semantics to any application, and an orchestration engine that is responsible for interpeting this recipe and executing it.

With these two components, we are able to reduce the barrier of entry for Cloudifying an application to the point where all you need is a simple start and stop script! You can automate the deployment of your application on any cloud, include monitoring, and even add more advanced capabilities such as elasticity and continuous availability. More advanced monitoring and SLA management can be added easily at any stage through a set of plug-ins. Below is a snippet of this recipe:

Cloudify's orchestration works at a process level.

This means that the deployment, installation,and scaling can be supported in the same way no matter what the deployment stack is, whether it's based on Java, .Net, C++, Ruby, PHP, or anything else.

Further, Cloudify comes with pre-integrated recipes that support the likes of Tomcat, JBoss, MySQL, Spring, Cassandra, MongoDB, HSQL, ActiveMQ, and Solr as well as a set of pre-backed application blueprints for Web Applications and Big Data Analytics applications.

This makes Cloudify open to any application stack.

Plus, it actually makes many of the popular OSS stacks run better.The Groovy-based DSL and the planned integration with other automation engines such as Chef make it fairly easy to extend the platform and plug in your application or stack of choice. We actually invested a lot of effort in Cloudify to make the process of developing and testing recipes very simple, to the point where you can test an entire application stack on a local Cloudify-managed cloud.

The third point relates to DevOps.

Many of the existing automation and orchestration services have been designed primarly for operation teams. They are often not even accessible or available at the development environment.

With Cloudify, we realized that the right way to make an application ready for the cloud would be to empower the developers of those applications with tools that fit their development environment through Local-Cloud, a simple DSL, and a set of IDE integrations along with the Cloudify Application Management and Monitoring system, which was designed to fit into both the development and operational environment.

With these, we can provide a smooth transition from the development environment to the production cloud environment through a single operation. The same abstraction enables us to move between different cloud environments, whether they happen to be public or private clouds. This allows us to choose the right cloud for the job.

For example, we can run our testing in the cloud and the production application locally, or vice versa. We can choose a cloud resource based on cost and/or locality; for example, we may want to use a "low cost" cloud for demos, and for production deployment, a cloud that provides the best service levels. All this brings a consistent cloud story between local, private, and public clouds.

What does this means for XAP users?

As mentioned above, XAP provides an end to end scale-out middleware stack.

Cloudify was designed to manage XAP middleware services as a first class citizen within the Cloudify environment. That means that with XAP, users can now have get significantly better automation, orchestration, as well as management and monitoring services than with previous releases. Cloudify comes with specific support for running XAP middleware components, including DataGrid, Messaging, Processing, Map/Reduce, et al.

The combination of the Cloudify and XAP also opens the opportunity to integrate XAP with other middleware services such as Tomcat and JBoss on the web container side, and MongoDB, Cassandra on the database side, managing the entire stack through a consistent provisioning and management experience. It also completely automates the process of deploying complex applications such as Big Data applications that span multiple sites.

How can Cloudify users benefit from XAP?

One point that needs to be made is that Cloudify and XAP protect you from external services. For example, Amazon provides an excellent feature set: RDS, SQS, SimpleDB, Elastic Caching, and more, but using these features ties you to Amazon. With Cloudify and XAP, you get each of these feature sets, along with multi-site replication and Xtreme Transaction Processing, which protects you from being locked in to cloud-specific services.

Who is integrating with Cloudify

Even though Cloudify wasn't officially released until today, the interest behind the product was enormous and brought many new prospects and existing customers to try out earlier versions of the product. I picked a few of the intreesting use cases:

A large Telco ISV plans to use Cloudify to move many of their existing network applications and services into a virtualized environment.
A leading Bank that is already using XAP plans to use Cloudify to manage their entire application stack and build thier own private PaaS.
A large ISV plans to use Cloudify to build a fully managed Big Data stack that integrates with Cassandra.
A large system integrator is using our Integration platform Cloudify for Azure to bring Java applications into Azure.
A large provider in the financial sector plans to use Cloudify to move their entire portfolio of products that were built through aquisitions into a common platform that can provide consistent management across the stack and reduce the cost of managing their existing application, as well as reduce the time it takes them to launch new features and applications.

Final notes..

Today many organizations and ISVs wanting to leverage the value of the cloud are left with a fairly limited number of choices - they can either choose to use one of the public PaaS platforms and give up full control over the langague, data center, and stack that they can choose, or they can choose to build their own platform and in that case face the complexity, cost, and more importantly time that is often involved in building such a platform.

Many of those organizations need a platform that they can use to Cloudify their application and build thier own PaaS without the complexity involved, without giving up control at the same time.

ISVs and SaaS providers need such a platform to embed as part of their own product distribution or service.

According to recent Forrester research (Sizing The Cloud), the demand for such middleware platforms is going to grow substantially over the course of the next years.

By 2020, the middleware virtualization market will be the largest private cloud segment in terms of total revenues, at $9 billion.

“a new set of technologies that allow virtualization of large-scale, server-based enterprise applications are starting to become more important. Application-aware virtualization (AAV) will be the primary driver of future growth in this category”

Cloudify is positioned specificaly to fill-in that void.

In addition to all that, Cloudify will come with a feature-rich free edition. Unlike many of the other alternative offerings in this space, we designed the free edition to serve real enterprise application workloads and not just demo or development environments. (We'll share more in few weeks time as we complete this beta stage).

The current beta version already comes with support for Tomcat, JBoss, MySQL, Spring, Cassandra, MongoDB, HSQL, and ActiveMQ. It also comes with examples that show full application stack deployments, including:

Web Application deployment stacks including the traditional PetClinic and SpringTravel applications, demonstrating an elastic deployment of a common web application stack.
A Real Time Analytics for Big Data example, demonstrating how you can process large volumes of events in real time and store them in a NoSQL backend.

Trying out these examples is an extremely simple process. You don't need to download or install a full virtual machine image or integrate with a particular cloud. All you need is to download Cloudify, start the Cloudify local-cloud and execute a single command <install-application ..> and you're good to go.

The beta is available for download at gigaspaces.com/cloudify (the link also includes an online demo for those interested).

On a more personal note - with Cloudify, I feel that we're able to take a lot of the experience that we've gained over the years in dealing with large-scale mission-critical applications and come out with something that integrates all this into a simple product. I believe that the days in which you had to "outsource" your entire application to someone else's PaaS, or SaaS just to mitigate the complexity will be gone. Cloudify brings the power back to where its belongs - to the application owner!

I look forward to hearing your feedback, receiving your input, and getting the ball rolling...

References:

Java and Center Stage: Meet us at JavaOne 2011

2011-09-30T19:02:28+02:00

Next week, I’m going to be in JavaOne. Unlike last year, where I expressed lots of skepticism about the way Oracle drives the Java Community, it seems that this year things are starting to get back on course – not that the skepticism has vanished completely, but at least there is a stronger sentiment that things are starting to settle down and take a more positive course.

It seems that now Java is taking center stage in the Cloud world, with more application platforms such as SalesForce/Heroku, VMware, Redhat/OpenShift, Cloud.com, JClouds, and obviously GigaSpaces providing a rich set of offerings that are based on Java.

One of the most interesting things in my opinion is that the combination of Cloud and Java brings new opportunities that until recently were unheard of:

Heroku, which was very much Ruby-centric, added support for Java.
GigaSpaces recently announced our partnership with Microsoft to bring Java to Microsoft's Azure community.

What’s interesting in those two examples is that the Cloud forces us to break away from the language wall. It brings communities that were living in completely isolated silos much closer together.

This forces a very different way of thinking about Java - and about the next generation Java platform specifically.

The new platform should be:

Open for other languages (through integration points like Protobuff, Thrift, REST, and more, as well as the ability to run those languages)
Embrace NoSQL implementations as an integral part of the platform
Add support for dynamic data structures such as JSON, and schema-free document models
Bring Dev and Ops closer by providing better exposure of operational aspect as part of the development frameowork
Make cloud deployment a first class citizen

It seems that Java EE 7 is taking the right path in that direction. From Java EE 7 and Cloud Computing:

The Java EE platform architecture is taking into account the operational impact of the cloud, more specifically by defining new roles, such as a PaaS administrator.
The Java EE platform may also establish a set of constraints for PaaS-specific features, such as multi-tenancy, that deployments may have to obey. Applications may also be able to identify themselves as designed for cloud environments.
All resource manager-related APIs, such as JPA, JDBC and JMS, will be updated to enable multi-tenancy. The programming model may be refined as well by introducing connectionless versions of the major APIs.
Java EE will define a descriptor for application metadata to enable developers to describe certain characteristics of their applications that are essential for the purpose of running them in a PaaS environment. These may include being multitenancy-enabled, resources sharing, quality of service information, dependencies between applications.

What This Means

Java is regaining some of its luster from the last few years. It was being perceived as vaguely pedestrian and ordinary, and languages like Ruby and Python were being seen as more dynamic environments for rapid and scalable deployments.

Now we're seeing Java as the lingua franca of enterprise development. With so many environments being heterogeneous, a homogenous platform becomes a very desirable resource, and the JVM is able to run many languages in an integrated environment: Java, Ruby, Python, Scala, and Groovy, to name only a few, and many of these can run as compiled code and as scripts.

Even beyond the multiple programming language support of the JVM, you have very broad support for almost any number of remote procedure call mechanism you can think of. The old J2EE approach, where you defined your requirement and there was a specific way to fulfill it (which meant that every architecture was more or less nudged to the middle), is no longer a real limit.

If you can do it, you can do it in Java, without Java forcing you to sacrifice in the process.

This is a very powerful concept.

It puts your organization back under your control. This is desirable because you know your product and architecture better than a community process does. You know whether you need to support REST here, and SOAP there; you know you need to use NoSQL in this area and MySQL in that area.

A Remaining Challenge

Java EE 7 may be becoming aware of the new cloud-based environment, but it's still only one aspect of application deployment and design. What it does not address is the entire application environment.

This is where applications such as Cloudify come in. Cloudify treats every artifact involved with your deployment as a managed element.

Consider: when you deploy an enterprise application, you're not just deploying some operational code. That code has external requirements, like a database (or NoSQL warehouse); it might also rely on an Apache-based load balancer, or perhaps another independently-deployed artifact.

That's quite a burden on operations, because all of these are separately deployed and managed.

Cloudify, however, centralizes the management of each element your application is composed of, over the entire application's lifecycle, and provides a bridge to whatever environment you wish to use.

For example, you could have a MySQL database, used by a Java application hosted by a single Tomcat instance, fronted by an Apache httpd server, hosted on an internal Linux server. Cloudify can deploy and monitor each of those elements: the MySQL database, the schema and initial data; Tomcat; the Java application; Apache httpd, and the server upon which it all runs.

Now let's look at a larger deployment: a set of MySQL servers, a cluster of GigaSpaces XAP nodes, an instance of GlassFish hosting some web services, six .Net-based web services, and five httpd servers being load-balanced, deployed on a platform such as Microsoft Azure. From Cloudify's perspective, the deployment and management process is the same. You have more artifacts to deploy, and you're telling Cloudify to deploy on a different platform... that's it.

Cloudify can make sure each resource is running optimally, as well. If you can measure an attribute - such as CPU load, or disk space, or free memory - you can tell Cloudify to assert that some headroom exists, and what to do if a metric is exceeded.

For example, if you have a CPU running at 100% for an extended period of time, Cloudify can automatically deploy another instance of the component consuming the CPU, and configure load balancing so your application is more scalable, without human intervention.

Likewise, if you have four servers running a given process, all at 0% CPU load, you can reduce the number of containers with that component so that you're not overallocating resources, saving energy and money.

The key thought here:

You know how to solve your problem. We can help.

We'll be on the floor at JavaOne 2011, in booth #5002. We'll be happy to show you both XAP, our industry-leading application platform, and Cloudify, our deployment solution for the cloud-enabled world.

(This post was co-written with Joseph Ottinger, who unfortunately won't be able to make it to JavaOne this year. But he says that the best people in GigaSpaces will be there, so you're not missing out!)

Big Data Application Platform

2011-09-02T16:23:16+02:00

It's time to think of the architecture and application platforms surrounding "Big Data" databases. Big Data is often centered around new database technologies mostly from the emerging NoSQL world. The main challenge that these databases solve is how to handle massive amount of data at a reasonable cost and without poor performanc - distributed databases emerged to address this challenge and today we're seeing high adoption rate and quite impressive success stories such as the Netflix use of Cassandra/DataStax solution. All that indicate the speed in which this market evolves.

The need for a Big Data Application Platform

Application platforms provide a framework for making the development of applications simpler. They do this by carving out the generic parts of applications such as security, scalability, and reliability (which are attributes of a 'good' application) from the parts of the applications that are specific to our business domain.

Most of the existing application platforms such as Java EE and Ruby on Rails were designed to work with centralized relational databases in mind. Clearly, that model doesn’t fit well to the Big Data world simply because it wasn’t designed to deal with massive amount of data in first place. In addition to that, frameworks such as Hadoop are considered too complex as noted in VP/Research Director for Forrester Research Mike Gilpin, in his post, "Big Data" technology: getting hotter, but still too hard:

"Big Data" also matters to application developers - at least, to those who are building applications in domains where "Big Data" is relevant. These include smart grid, marketing automation, clinical care, fraud detection and avoidance, criminal justice systems, cyber-security, and intelligence.

One "big question" about "Big Data": What’s the right development model? Virtually everyone who comments on this issue points out that today’s models, such as those used with Hadoop, are too complex for most developers. It requires a special class of developers to understand how to break their problem down into the components necessary for treatment by a distributed architecture like Hadoop. For this model to take off, we need simpler models that are more accessible to a wider range of developers - while retaining all the power of these special platforms.

Other existing models for handling Big Data such as Data Warehouse don’t cut it either, as noted in Dan Woods' post on Forbes, Big Data Requires a Big, New Architecture:

...to take maximum advantage of big data, IT is going to have to press the re-start button on its architecture for acquiring and understanding information. IT will need to construct a new way of capturing, organizing and analyzing data, because big data stands no chance of being useful if people attempt to process it using the traditional mechanisms of business intelligence, such as a data warehouses and traditional data-analysis techniques.

To effectively write Big Data applications, we need an Application Platform that would put together the different patterns and tools that are used by pioneers in that space such as Google, Yahoo, and Facebook in one framework and make them simple enough so that any organization could make use of them without the need to go through huge investment.

Here's my personal view on how that platform could look like based on my experience covering the NoSQL space for a while now and through my experience with GigaSpaces.

Characteristics of Big Data Application Platform

As with any Application Platform, a Big Data application platform needs to support all the functionality expected from any application platform such as scalability, availability, security, etc.

Big Data Application platforms are unique in the sense that they need to be able handle massive amounts of data and therefore need to come with built-in support for things like Map/Reduce, Integration with external NoSQL databases, parallel processing, and data distribution services and on top of that, they should make the use of those new patterns simple from a development perspective.

Below is a more concrete list of the specific characteristics and features that define what Big Data Application Platform ought to be. I've tried to point to the specific Java EE equivalent API and how it would need be extended to support Big Data application.

Support Batch and Real Time analytics

Most of the existing application platforms were designed for handling of transactional web applications and have little support for business analytics applications. Hadoop has become the de facto standard for handling Batch processing; Real Time analytics, however, is done through other means outside of the hadoop framework, mostly through an event processing framework, as I already outlined in details during my previous post Real Time Analytics for Big Data: An Alternative Approach.

A Big Data Application Platform would need to make Big Data application development closer to mainstream development by providing a built-in stack that includes integration with Big Data databases from the NoSQL world, and Map/Reduce frameworks such as Hadoop and distributed processing, etc.

It also needs to extend the existing Transaction processing and Event Processing semantics that come with JavaEE for handling of Real Time analytics that fits into the Big Data world as outlined in the references below:

Making Big Data Application Closer to Mainstream development practices

Domain model and Data access API

Writing Big Data application is quite different than writing a typical CRUD application to a centralized relational database in many forms. The main difference is with the design of our data domain model, and the API and Query semantics that well use to access and process that data.

Mapping has proven to be a fairly effective approach to map the impedance mismatch between different data models and API’s. A good reference on that regard is the use of O/R mapping tools such as Hibernate for bridging similar impedance mismatch between Object and Relational models.

Abstraction frameworks such as Spring has also proven to be quite useful means to plug-in different data sources that doesn’t comply to common API through plug-in approach.

To make the development of Big Data Application closer to mainstream development Big Data application platform need to come with similar mapping and abstraction tools that would make the transition from the existing API’s significantly smoothers.

Today we already have various mapping and abstraction framework available.

For Big Data Mapping tools:

For batch processing were already seeing the increase adoption of frameworks such as Hive which provide an SQL like façade for handling complex batch processing with hadoop.

JPA provides more standard JEE abstraction that fits into Real Time Big Data application. Google App Engine uses Data Nucleus on top of Big Table, With GigaSpaces we came up with a high performance JPA abstraction for our In Memory Data Grid using Open JPA, and Redhat came up with Hibernate OGM (Object-Grid Mapping).

In addition to these, we're seeing that the existing mapping tools add extensions to support data partitioning semantics through annotations as in the case of EclipseLink and others.

For Big Data Spring Abstraction:

SpringSource came out with an interesting and even more high level abstraction known as Spring Data that makes it possible to map different data stores of all kinds into one common abstraction through annotation and plug-in approach. The abstraction is leaky currently, but shows progress in the space.

Business logic

In Java EE, business logic refers to the parts in our application that are responsible for processing the data. As with JavaEE the data often lives in a centralized database SessionBean was often mapped to a single instance per user session.

With BigData the common pattern for processing data is through Map/Reduce. Map/Reduce was designed to handle the processing of massive amount of data through moving the processing logic to the data and distributing the logic in parallel to all nodes. Having said that, developing parallel processing code is considered fairly complex.

A Big Data Application Platform would need to make Map/Reduce and parallel execution simple. One way of doing that is by mapping this semantics into existing programming models. One example would be to extend the current SessionBean model to support this sort of semantics (as with the GigaSpaces Remoting semantics) – this makes parallel processing look like a standard remote method invocation. Another way is to provide more native parallel execution semantics by extending existing semantics such as createNativeQuery in JPA, or the executor API in Java.

Interestingly enough, at the time of writing this post I came across Robin van Breukelen's post, "Distributed Fork Join," that shows how you can use the latest Java 7 fork/join API in similar context.

Support new semantics that fits the dynamic web era

Clearly, SQL is a great query language (or else it wouldn't have taken over for database access!) but SQL also has its limits. When we're writing web content, we often deal with dynamic data structures that continuously evolve. When the amount of this data gets massive, as with Big Data, it becomes almost impossible to manage those changes through a standard SQL schema evolution model.

To address this need, BigData platforms need to add support schemaless semantics as a first class citizen. That often means that the data mapping layer mentioned earlier would need to be extended to support document semantics. An example for that is MongoDB, CouchBase, Cassandra and the new GigaSpaces Document API (which, as it turns out, maps to MongoDB and Cassandra very easily.)

Provide built-in semantics for handling of the tradeoffs between consistency, availability, scalability rather than trying to force a least common denominator as with XA and JTA

JEE application platform forced a fairly strict consistency model through JTA, Distributed Transactions, and so forth. Unlike Java EE, Big Data Application platforms need to support more relaxed versions of those semantics that would enable flexibility between consistency, scalability, and performance. A good example on how that configuration may look like is covered in Cassandra/DataStax documentation and one of my previous post on the subject of CAP theory.

Provide built in support for In-Memory data store to gain best performance and latency

RAM-based devices provide the best performance/latency results. Big Data platforms need to provide seamless integration between RAM and Disk based devices where data that is written in RAM would be synched into Disk asynchronously. In addition to that, they need to provide common abstractions that enable users to use the same Data access API for both devices and thus make it easier to choose to the right tool for the job without changing the application code.

Good starting points on that regard are the JPA abstractions now being supported by In-Memory Data-Grids and NoSQL data stores as well as the Spring Data abstraction as noted above.

Provide built-in support for event driven data distribution using pub/sub model

The current Java EE frameworks provide limited support for event processing through message-driven beans and JMS. For Big Data applications, we need to add data awareness to the current MDB model that make it easy to route messages based on data affinity and content of the message. We also need to provide more fine grained semantics for triggering events based on data operation (delete, update,..) as well as content as with CEP (Complex Event Processing).

In addition to that we need to provide higher level abstractions that use the underlying pub/sub support to enable data synchronization and locality. A good example for that is the use of LocalCache and LocalView where LocalCache is useful for random access patterns by caching the data that was used most recently and LocalView which provides means to Cache a specific segment of data that is known in advance. Both LocalCache and LocalView use the underlying pub/sub support to ensure continuous synchronization of the data from the BigData pool by tracking changes on the Big Data pool and updating the local copy of the data implicitly.

Built in support for public/private cloud

Big Data applications tend to consume lots of compute and storage resources. There are a growing number of cases where the use of the cloud enables significantly better economics for running Big Data applications. To take advantage of those economics, Big Data Application Platforms need to come with built-in support for public/private clouds that will include seamless transition between the various cloud platforms through integration with frameworks such as JClouds. Cloud Bursting provides a hybrid model for using cloud resources as spare capacity to handle load. To effectively handle Cloud Bursting with Big Data well have to make the data available for both the public and private side of the cloud under reasonable latency – which often requires other services such as data replication.

Open & Consistent management and orchestration across the stack

A typical Big Data application stack includes multiple layers such as the database itself, the web tier, the processing tier, caching layer, the data synchronization and distribution layer, reporting tools, etc. One of the biggest challenge is that each of those layers come with different management, provisioning, monitoring and troubleshooting tools. Big Data applications tend to be fairly complex by their very nature; the lack of consistent management, monitoring and orchestration across the stack makes the maintenance and management of this sort of application significantly harder.

In most of the existing Java EE management layers, the management application assumed control of the entire stack. With Big Data applications, that assumption doesn’t apply. The stack can vary quite significantly between different application layers therefore the management layer of Big Data Application Platform needs to come with a more open management that could host different databases, web-containers etc., and provide consistent management and monitoring through this entire stack.

Final words

Java EE application servers played an important role in making the development of database-centric web application closer to mainstream. Other frameworks such as Spring and Ruby on Rails later emerged to increase the development productivity of those applications. Big Data Application Platforms have a similar purpose – they should provide the framework for making the development, maintenance and management of Big Data Applications simpler. In a way, you could think of Big Data Application platforms as a natural evolution of the current application platforms.

With the current shift of Java EE application platforms toward PaaS, we're going to see even stronger demand for running Big Data applications on cloud based environments due to the inherent economic and operational benefits. Compared to the current PaaS model, moving data to the cloud is more complex and would require more advance support for data replication across sites, cloud bursting etc.

The good news is that Big Data Application platforms are being implemented with these goals in mind, and you can already see migration yielding the benefits one should expect.

References

NoSQL Job Trends – August 2011 (Dzone)
Big Data" technology: getting hotter, but still too hard (Forrester)
Big Data Requires a Big, New Architecture (Forbes)

Real Time analytics for Big Data: Facebook's New Realtime Analytics Syste

Real Time Analytics for Big Data: An Alternative Approach

2011-07-14T17:26:26+02:00

Lately, we've been talking to various clients about realtime analytics, and with convenient timing Todd Hoff wrote up how Facebook's realtime analytics system was designed and implemented (See previous review on that regard here).

They had some assumptions in design that centered around the reliability of in-memory systems and database neutrality that affected what they did: for memory, that transactional memory was unreliable, and for the database, that HBase was the only targeted data store.

What if those assumptions are changed? We can see reliable transactional memory in the field, as a requirement for any in-memory data grid, and certainly there are more databases than HBase; given database and platform neutrality, and reliable transactional memory, how could you build a realtime analytics system?

Joseph Ottinger and I discussed this, and this is what we came up with.

A Summary of History

To understand what a new design might look like, it’s often useful to consider a previous design. This is a very short summary of Facebook’s realtime analytics system.

First, it’s based on a system of key/value pairs, where the key might be a URL and the value is a counter. Thus, there’s a requirement for atomic, transactional updates to a very simple piece of data. The difficulties come from scale, not from the focus of the system.

The process flow is fairly simple:

A user creates an event by performing some action on the website. This generates an AJAX request, sent to a service.
Scribe is used to write the events into logs, stored on HDFS.
PTail is used to consolidate the HDFS logs.
Puma takes the consolidated logs from PTail and stores them into HBase in groupings that represent roughly 1.5 seconds’ worth of events.
HBase serves as the long-term repository for analytics data.

There are some questions around how PTail and Puma serve as scaling agents, and some of the notes around their use are still limited in scale – for example, one of the concerns is that an in-memory hash table will fill up, which sounds like fairly serious limitation to have to keep in mind.

A Potential for Improvement

There are lots of areas in which you can see potential improvements, if the assumptions are changed. As a contrast to Facebook's working system:

We can simplify the design. If memory can be seen as transactional - and it can - we can use them without transforming them as they proceed along our analytics workflow. This makes our design and implementation much simpler to implement and test, and performance improves as well.
We can strengthen the design. With a polling semantic, such systems are brittle, relying on systems that pull data in order to generate realtime analytics data. We should be able to reduce the fragility of the system, even while making it faster.
We can strengthen the implementation. With batching subsystems, there are limits shouldn’t exist. For example, one concern in Facebook's implementation is the use of an in-memory hash table that stores intermediate data; the in-memory aspect isn’t a concern until you realize that the batch sizes are chosen partially to make sure that this hash table doesn’t overflow available space.
We can allow deployments to change databases based on their requirements. There's nothing wrong with HBase, but it's got specific characteristics that aren't appropriate for all enterprises. We can design a system which you’d be able to deploy on various and flexible platforms, and we can migrate the underlying long-term data store to a different database if needed.
We can consolidate the analytics system so that management is easier and unified. While there are system management standards like SNMP that allow management events to be presented in the same way no matter the source, having so many different pieces means that managing the system requires an encompassing understanding, which makes maintenance and scaling more difficult.

What we want to do, then, is create a general model for an application that can accomplish the same goals as Facebook’s realtime analytics system, while leveraging the capabilities that in-memory data grids offer where available, potentially offering improvement in the areas of scalability, manageability, latency, platform neutrality, and simplicity, all while increasing ease of data access.

That sounds like quite a tall order, but it’s doable.

The key is to remember that at heart, realtime analytics represent an events system. Facebook’s entire architecture is designed to funnel events through various channels, such that they can safely and sequentially manage event updates.

Therefore, they receive a massive set of events that “look like” marbles, which they line up in single file; they then sort the marbles by color, you might say, and for each color they create a bundle of sticks; the sticks are lit on fire, and when the heat goes up past a certain temperature, steam is generated, which turns a turbine.

It’s a real-life Rube Goldberg machine, which is admirable in that it works, but much of it is still unnecessary if the assumptions about memory ("unreliable") and database ("HBase is the only target that counts") are changed. Looking at the analogy from the previous paragraph, there’s no need to change a marble into anything. The marble is enough.

A Plan for Implementation

Our design for implementation is built around putting data and messaging together. A data grid is a perfect mechanism for this, as long as it provides some basic features: transactional operations, push and pull semantics, and data partitioning.

A data grid does provide those basic features, or else it's not really much of a data grid; it'd be more of a cache otherwise.

With a data grid, then, the events come in as individual messages. When the user chooses an operation on the web site, an asynchronous operation would write the event, just as Facebook does today. However, instead of filtering and batching the events into various forms, the events are dispatched to waiting processes that perform many transactional updates in parallel.

There’s a danger that those updates might be slower than the generated events, if each event is processed sequentially. That said, this isn’t as much a problem as one might think; if data partitioning is used, then event handlers can receive partitioned events, which localizes updates and speeds them up dramatically.

In fact, you can still use batching to process events as a group; since the events would be partitioned coming in, the batch process would still be updating local data very quickly, which would be faster than individual event processing, even while retaining simplicity.

With this design, there is no overflow condition, because a system that’s designed to scale in and out as most data grids are will repartition to maintain even usage. If a data grid can’t provide this feature intrinsically, of course some management will be necessary, but finding data grids with this feature isn’t very difficult.

One other advantage of data grids is in write-through support. With write-through, updates to the data grid are written asynchronously to a backend data store – which could be HBase (as used by Facebook), Cassandra, a relational database such as MySQL, or any other data medium you choose for long-term storage, should you need that.

The memory system and the database - the external data store - work together. The in-memory solution is ideal for the realtime aspects, the events that affect now. The external data storage solution is designed to handle long-term data, for which speed is not as much of an issue.

A Discussion of Strengths

The key concept here is that event handling is the lever that can move the realtime analytics mountain. By providing a simple, scalable publisher/subscriber model, you simplify design; by using a platform that supports data partitioning, transactional updates, and write through capabilities, you gain scalability.

The data grid’s flexible query API means that events can literally react when data is available.

For a call center, for example, you want to immediately identify signals that show that the caller should be handled differently; imagine an ecommerce site that was able to determine immediately if a user was losing interest, and thus could respond appropriately, before the customer moves on.

With external processes and a long funnel for data, immediate-response capabilities are very difficult to implement, not just because of latency but because the data transformations tend to homogenize the data, instead of allowing rich expressions and flexible event types.

The data grid also has much richer support in terms of client applications. Instead of applications going through an API that focuses on a specific phase of the data’s life (for example, an API focused on HBase), you can focus on a generic API that can capture events at any point in their lifecycle, and from anywhere. An external monitoring process, then, can have the same immediate, partition-aware access to data that the integrated message-handling system does; adding features and analysis is just a matter of connecting a client to the data grid.

Here we have a quick demo that shows much of this in motion. We have a market analysis application, deployed into GigaSpaces XAP via our new Cloud deployment too, Cloudify; it uses an event-driven system to display realtime data, with a write-through to Cassandra on the back-end. The design is very simple, and demonstrates the principles we've discussed here - and can scale up and down depending on demand.

Final words

As real-time-analytics gets into mainstream application design, it becomes important to draw a blueprint on how to build this sort of system without re-inventing the wheel.

Todd Hoff (HighScalability) and Alex Himel (Facebook) provided a fairly detailed description on their solution and even more importantly they even shared the rationales that made them do things in certain ways.

One main difference in assumptions that lead to the different implementation strategies are in reliable memory for event processing, and in the use of passive data storage.

Another difference is that we had to to think of the solution as an easily cloneable solution and therefore a lot of attention was put on the simplicity of the runtime, packaging and management of the solution.

Yet another difference is that we couldn’t decide on a specific database as there isn’t a "one size fits all" solution – for certain customers, SQL would still be preferred choice and the fact that we can buffer the write to the database gives them more headroom while still allowing them to scale on writes.

I hope that this would lead to constructive dialogue on the various tradeoffs which will serve the entire industry...

References

Video: Realtime Analytics for Big Data: A Facebook Case Study

Real Time analytics for Big Data: Facebook's New Realtime Analytics System

2011-07-08T14:49:34+02:00

Recently, I was reading Todd Hoff's write-up on FaceBook real time analytics system. As usual, Todd did an excellent job in summarizing this video from Engineering Manager at Facebook Alex Himel, Engineering Manager at Facebook.

In this first post, I’d like to summarize the case study, and consider some things that weren't mentioned in the summaries. This will lead to an architecture for building your own Realtime Time Analytics for Big-Data that might be easier to implement, using Facebook's experience as a starting point and guide as well as the experience gathered through a recent work with few of GigaSpaces customers. The second post provide a summary of that new approach as well as a pattern and a demo for building your own Real Time Analytics system..

The Business Drive for real time analytics: Time is money

The main drive for many of the innovations around realtime analytics has to do with competitiveness and cost, just as with most other advances.

For example, during the past few years financial organizations realized that inter-day risk analysis of their customers' portfolios translated to increased profit as they could react faster to profit and loss events.

The same applies to many of the online ecommerce and social sites. Knowing what your users are doing on your site in real time and matching what they do with more targeted information transforms into better conversion rate and better user satisfaction, which means more money in the end.

Todd provides similar reasoning to describe the motivation behind Facebook's new system:

Content producers can see what people like, which will enable content producers to generate more of what people like, which raises the content quality of the web, which gives users a better Facebook experience.

Why now?

Real time analytics goes mainstream

The massive transition to online and social applications makes it possible to track user patterns like never before. The correlation between the quality of data that providers track and their business success is closely related: for example, e-commerce customers want to know what their friends think about products or services, right in the middle of their shopping experience. If sites cannot keep up with their thousands of users in real-time, they can lose their customers to sites that can.

So while risk analytics in financial industry was still a fairly small niche of the analytics market the demand for real time analytics in Social, eCommrce and SaaS applications brought the demand for real time analytics to main-stream business under massive load.

No one has time for batch processing anymore.

Technology advancement

Newer infrastructures and technologies like tera-scale memory, nosql, parallel processing platforms, and cloud computing, provide new ways to process massive amount of data in shorter time and with lower cost. As most of the current analytics systems weren’t built to take advantage of these new technologies and capabilities, they haven't been able to adopt to real time requirements without massive changes.

Hadoop Map/Reduce doesn’t fit the real time business

One of the hottest trends in the analytics space is the use of Hadoop as almost the de-facto standard for many of the batch processing analytics application. While Hadoop (and Map/Reduce in general) does a pretty good job in processing massive logs and data through parallel batch processing, it wasn’t designed to serve the real time part of the business.

Strong evidence for that can be seen from the moves of those who were known as the “poster children” for Map/Reduce: Google and Yahoo both moved away from Map/Reduce. Google has moved to Google Percolator for its indexing service. Yahoo came-up with a new service S4 which was designed specifically for real time processing.

It is therefore not surprising that Facebook reached the same conclusion as it relate to Hadoop:

Hadoop/Hive..Not realtime. Many dependencies. Lots of points of failure. Complicated system. Not dependable enough to hit realtime goals.

Facebook's Real Time Analytics system

According Todd, Facebook evaluated a fairly long list of alternatives including as MySQL, an in-memory cache, Hadoop Hive/Map Reduce. I highly recommend reading the full details from Todd's post.

I tried to outline Facebook architecture based on Todd's summary in the diagram below:

Every user activity triggers an asynchronous event, through AJAX – this event is logged in a tail log using Scribe. Ptail is used to aggregate the different individual logs into a consolidated stream. The stream is batched in 1.5 sec groupings by Puma which stores the event batch into HBase. The real time logs are kept for a certain period of time and than get cleared from the system.

Obviously this description is a fairly simplistic view – the full details are provided in the original post.

Evaluating the Facebook Architecture

Facebook reasoning behind their technology evaluation seems acceptable for the most part, although there are some obvious concerns.

There were two things that caught my attention:

Memory Counters

(Facebook) felt in-memory counters, for reasons not explained, weren't as accurate as other approaches. Even a 1% failure rate would be unacceptable. Analytics drive money so the counters have to be highly accurate.

It sounds to me that the evaluation was based on memcached. By default, it's not highly available; a failure would result in loss of data. Obviously, that doesn’t apply to some other memory based solutions such as In Memory Data Grids (for example, GigaSpaces and Coherence both were designed for high resiliency.)

Cassandra vs HBase

The choice of HBase over Cassandra was also very interesting mainly since Cassandra was developed by another Facebook team to address write scalability. The choice had to do with the write rate differences between the two alternatives at the time of evaluation:

HBase seemed a better solution based on availability and the write rate. Write rate was the huge bottleneck being solved.

Eric Hauser posted a comment on this analysis which seem to indicate that this issue has been addressed with Cassandra:

When Facebook engineers started the project 6 months ago, Cassandra did not have distributed counters which is now committed in trunk. Twitter is making a large investment on Facebook for real-time analytics (see Rainbird). Write rate should be less of of bottleneck for Cassandra now that counter writes are spread out across multiple hosts. For HBase, every counter is still bound by the performance of single region server? A performance comparison of the two would be interesting

Eric's comment is indicative of how dynamic the NoSQL space is. I’d be interested in how different the technology selection would be now.

The Architecture

There are a few common principles that drive the architecture for this type of system:

Events logging needs to be extremely fast, to minimize the latency impact on the site.
Events need to be reliable, otherwise the entire system's accuracy is questionable and the data is devalued.
The real time data in the form of logs is kept for a certain period of time (x hours or x days)
Write scalability is key.
Post processing can happens in batches, the size of the batch depends on how *real-time* we need this data to be.
Writes to a backend database need to be done asynchronously.

Facebook seem to follow those same principles in their architecture while keeping the system at scale at a fairly impressive rate.

There are a few questions that are still open as I don’t have full visibility into their system:

Are Ptail and Puma centralized components? If so, don't they pose a potential bottleneck? Based on Todd's summary and Alex' presentation, it seems that the way Facebook scaled their system is by splitting the PTail stream by categories of events so each type can be handled by a different data center.
Puma batches event logs in memory before it writes them into Hbase – what happens if Puma fails before the batch is written?
The solution seemed to be limited to handling simple counters. This seem to be a fairly severe limitation, as many systems need to produce more complex relationships even during the real time part of the system, as also indicated as part of the future enhancements, as shown:

"(We need to know) how many people across a time window liked a URL. Easy to do in MapReduce, hard to do with a naive counter solution.”

In one point, it is noted that Facebook chose not to rely on memory for counters. However, throughout the description it seems that there is still strong reliance on keeping data within memory boundaries:

“(We) write extremely lean log lines. The more compact the log lines the more can be stored in memory..”

“(We) batch for 1.5 seconds on average. Would like to batch longer but they have so many URLs that they run out of memory when creating a hashtable”

Looking backward – wouldn’t it be better to store the data in-memory in the first place? Why add the extra architecture components, if you're able to make memory work for you? This is a crucial question, of course, because it focuses attention on memory availability. As mentioned, though, there are in-memory data grids that are designed for just this kind of situation.

It is noted that Puma writes it batches to Hbase sequentially:

“(We) wait for last flush to complete for starting new batch to avoid lock contention issues...”

What if this rate is lower than the actual write rate?

Realtime.Analytics.next

Interestingly enough, I was asked to draw a solution for similar challenge for a voice recording system: collect lots data from various sources and process them in real time. The good news is that it's doable, as shown by Facebook's success.

The better news is that it's actually fairly easy. We can add rich query capabilities, elastic scaling, database and platform neutrality, evolution of data, and more without making things unnecessarily difficult - it's not so much that this is the next generation of realtime analytics as much as map/reduce and the hbase approach used by facebook is the previous generation.

I'll describe how it can be done more simply in the next post.

References

Read/write scale without complete re-write

2011-06-03T19:06:41+02:00

Last week I was attending one of our Partner events in Stockholm where I presented the convergence of trends in the data scalability world – specifically the transition from NoSQL to NewSQL and the convergence of trends that brings the existing SQL and new NoSQL world much closer together as I noted in a previous post, "YeSQL: An Overview of the Various Query Semantics in the Post Only-SQL World."

During the event, Ronnie Bodinger, Head of IT at Avanza Bank AB, gave an excellent talk on how they turned their existing online banking application into a new site that was designed for read/write scaling.

Avanza's System Description:

Avanza Bank is a Swedish bank that makes it easy for investors to make equity transactions and fund switches. It runs the most trades on the Stockholm Stock Exchange.

It prides itself for providing advanced tools for its investors through its online banking system.

The current online system is a typical web site based on Java/JSP and Spring.

Scaling architecture of the existing site

Most of the interaction with the current site are read-mostly, the main scaling challenge was scaling on concurrent read operations. Read scaling was addressed through a side-cache architecture that is common with many of the existing LAMP + Memcached deployments where the first query hits the database and the following queries hit the cache.

The New System

The new site was designed to fit into the real-time and social era. This means that a lot of the traffic and activities are now being generated by user activities and not by just by the site owner. This activities need to be presented in real-time to the bank users.

Challenges

The changes to the new site lead to a significant change in the traffic and load behavior that drives a new class of scaling challenges:

Write scaling

In case where there are lots of updates involved the existing side-cache architecture leads to diminishing returns as the cache becomes obsolete pretty quickly and therefore synchronizing the cache only adds overhead.

Using Oracle RAC with a high end hardware platform didn’t prove itself either and yielded a fairly expensive solution that didn’t meet the scaling requirements.

Unlike “Green Field” applications, Avanza has an existing online application (a “Brown Field”) that serves its current customers. That brings the following list of additional challenges :

Existing Data Model

The entire data model of the application was designed for a relational model – changing the data model or moving it to a new NoSQL architecture as was considered would involve a huge change that could turn into a potentially years-long effort.

Legacy system

The online bank application consists of large set of legacy application and third party services. Re-writing the existing infrastructure is either impossible (due to the dependency on third party tools) or impractical.

Complex environment

As it often the case, a large portion of the legacy applications weren’t designed for scale, and weren’t built with a clear holistic architecture as they were built through layers over the years. This increases the complexity of scaling by quite a bit.

Existing Skillset

The existing development team already had fairly good knowledge of Java and Java EE. Changing the team and/or developing a completely new skillset is a huge barrier as the ramp-up time required to bring new developers up to speed with the complexity of the system can take years.

The solution: Read/write scale without complete re-write

It was clear that meeting the new scaling challenges would involve changes to the existing application – the main question was how to scope that change so that it wouldn’t require a complete re-write. The second goal was to build the change in a way that would reduce the TCO of the current system.

To achieve those two goals the following approach was used:

Minimize the change by clearly Identifying the scalability hotspots

The areas of the application that need intensive write access are often small parts of the overall system. The first step would therefore be to minimize the change to only the hot spots of the application and keep the rest of the application un-touched. In the case of Avanza, the hot spots were identified on certain tables used by the online web application. Most of the backend systems were still accessing the database for reporting, synchronization and batch processing and could therefore remain unchanged.

Keep the database as is

One of the key piece in the current design is the ability to address the read/write scalability outside of the database context (See the next bullet). This makes it possible to keep the existing database and the schema of the data unchanged. In that way, all the rest of the systems continue to work with the database as if nothing changed.

Put an In Memory Data Grid as a front end to the database

Scaling the application is done by front-ending the application with an In-Memory-Data-Grid (IMDG). The IMDG contains all the hot tables or rows of the original database. The online web application accesses the IMDG instead of the database. The IMDG is distributed in nature thus allowing scalability by distributing the load over a cluster of machines for both read and write operations.

Use write-behind to reduce the synchronization overhead

Updates from the IMDG to the underlying database is done asynchronously in batches through an internal queuing mechanism (redo-log).

Use O/R mapping to map the data back into its original format

In many cases to achieve best scalability, we need to partition the data. This often involves changes to the data schema. Changing the data schema could break the entire system including the areas that don’t suffer from the scalability bottleneck. To meet this impedance mismatch, we scope the data schema changes only to the IMDG. The data is mapped from the IMDG schema to the original schema through standard O/R mapping tools such as Hibernate or OpenJPA.

Use standard Java API and framework to leverage existing skillset

One of the challenges with many of the new NoSQL database alternatives is that they often force a complete re-architecture. This comes with the a fairly high cost of re-building the skillsets within the organization across the board for developing against new APIs as well as for maintaining capacity and sizing of those new databases.

IMDGs such as GigaSpaces expose standard APIs, such as JPA. In addition, they allow organizations to extend the use of the existing database while removing a large part of read/write load – both the use of standard APIs and existing database enables organizations to leverage their existing skillset and still meet their scalability requirements. It also enables them to take a smoother transition (through baby steps) into a completely new scale-out architecture by allowing a plug-in new scalable database at a later stage.

Use two parallel (old/new) sites to enable gradual transition

Switching all the customers into a new system at once is often a bad idea. A better approach would be to enable gradual transition of selected customers into the new sites. A common model to achieve that would be to run two parallel sites. The challenge in doing so is the synchronization between the two parallel sites. In the case of Avanza, they used the GigaSpaces Mirror service to synch all the changes from one site to the other and in that way keep the two sites up to date.

The diagram below provides a visual summary of this approach:

The TCO angle

The second goal in the project was to reduce the TCO of the current system.

This is achieved in the following way:

Use RAM for high performance access and disk for long term storage

As I noted in one recent post, a RAM based solution can be 10x – 100x cheaper than Disk based solution for high performance applications.

In addition to that, the price for RAM goes down continuously.

1GB can cost only 1.9$ a month.. , storing 1T bytes of data completely in RAM can fit into a single RAC with total cost of $1.8k per month.

The optimal solution would therefore be to use RAM to manage data that needs high speed write/read access and disk-based storage for the long-term data that is accessed less frequently.

Use commodity Database and HW – A single instance of an Oracle RAC deployment could reach to $500k. Putting a data-grid in front of the database enables us to remove the needs for many of the high-end features available in the Oracle RAC database. It also enables us to remove the need for high end hardware such as storage devices, infiniband network etc.

In this specific case, it was possible to move the data into MySQL and use commodity Dell machines to run the entire relational data system.

Final words

Many existing applications are built with layers on top of layers on top of layers of development. Relational databases often sit at the heart of those systems. Scaling these systems is therefore an extremely challenging task. This leads a lot of organizations to take the easy route, simply paying more for more high-end hardware and databases. We have reached to the point where this approach doesn’t cut it for many cases - and it's simply too expensive to maintain in the face of cheaper, more scalable alternatives.

In a world where the impact of software accretion is no longer tolerable, it's clear that a chnage is inevitable if we're to meet the new demands for scalability. The real question is how to make that change. The approach that was taken by Avanza Bank in their use of GigaSpaces is an excellent reference: it shows that they were not just able to meet the new scaling requirements fairly quickly through measurable and easy small changes, but they also reduced their cost of ownership significantly as noted recently by Ronnie Bodinger's statement:

“Throwing out Oracle's sluggish databases increases Avanza performance of their business-critical systems, while reducing license costs significantly."

..

"Today we have a large Oracle Cluster, which costs a lot. The new system goes in a fraction of it."

You can read the full article here (Note the article is in Swedish – use Google Translator to read the English version of the article..)

The recorded version of Ronnie presentation as it was captures through one of the audience camera is available here.

Incidentally, I'd like to offer a sincere word of thanks to So4It, who did an excellent job in working with Avanza and organizing the partner event.

GigaSpaces Citrix integration on top of OpenStack

2011-05-25T19:27:00+02:00

In my one of my previous posts (GigaSpaces OpenStack Explained) I made a reference to the joint work that we are doing with Citrix through the integration of our new PaaS API:

The specific integration with NetScaler (load balancer) and XenServer will be achieved through more open interfaces provided through the Citrix/Openstack contribution, which means that OpenStack users can use those intefaces to plug in any hypervisor or load balancer.

In this post I’d like to elaborate more specifically on the current and planned integration work.

GigaSpaces Citrix integration on top of OpenStack

The block diagram below describes the main layers that comprise the joint GigaSpaces/Citrix integration.

The OpenStack layers (marked in bold-green) enable making these integration points more open, as we will be using the OpenStack API for the Compute (NOVA) and Load-Balancer API instead of using the Citrix API directly.

OpenStack Compute (Nova)

The GigaSpaces integration with OpenStack Nova API is done through the previously announced JClouds provider contribution.

This integration enables you to run Citrix Xen VMs and manage the them through the Citrix Management console. Similarly, you can use this same integration to plug-in other VMs.

OpenStack Load Balancer

The OpenStack Load Balanacer API is a slight variation on the current Rackspace Load Balancer API. This integration follows the exact path as with the NOVA integration i.e. we will use the JClouds Load Balancer abstraction and plug-in an OpenStack load balancer provider as one of the providers' plug-ins.

This integration enables you to run the Citrix Netscaler load balancer through the OpenStack API, and thus enable you to leverage the performance and scaling benefit of Netscalaer while at the same time keeping an open interface to plug-in other load balancers.

Citrix certified version of OpenStack

OpenStack enables you to download the full source code of the project and build an Amazon like cloud infrastructure in your local environment. As with many other open-source offerings (Linux is a good example) – most enterprises don't have the skills or the time to go through the process of building their own cloud from the source. These organizations would need a pre-packaged version of OpenStack that comes with built-in support, production management tools etc.

The Citrix certified version of OpenStack is geared specifically for this purpose.

The solution will be comprised of two primary components: a Citrix-certified version of OpenStack and a cloud-optimized version of XenServer.

The product will be sold with Rackspace Cloud Builders, who will provide deployment services, training, and ongoing support for customer clouds.

Customers can get started building their clouds today with the Early Access Program. The program will provide access to the software (Citrix), a reference architecture and PowerEdge C server platforms (Dell), and services (RAX Cloud Builders) they need to begin building their cloud. However, as Open Cloud and OpenStack are about openness, customers can work with any group of providers/partners to build their cloud, and will be supported in the Early Access Program. The end result is open source technologies delivering a massively scalable cloud operating system.

Citrix OpenStack virtual appliance project

This video demonstrates the deployment and management of an OpenStack cloud, with the software packaged as a virtual appliance. This allows service providers to track, install, and upgrade their cloud as a single virtual machine image, avoiding the complexity of deploying directly from packages. The video shows a complete installation from bare metal to working cloud, using Citrix's packaged solution. It then goes on to show how the cloud can be scaled up to add new nodes in one click.

References

PaaS on OpenStack

2011-04-27T13:31:55+02:00

In my last post (GigaSpaces OpenStack Explained) I introduced our plan to add support for OpenStack in our platform:

One of the goals for our second-generation PaaS/SaaS enablement platform was to enable smooth migration between different cloud providers. We were able to achieve this goal through the use of our own abstraction (the Scaling Handler) and through the integration with the JClouds project that provides common abstraction to most of the existing cloud providers. With that, we can ensure that any application can be moved from the likes of Amazon to OpenStack or to an organization's own private cloud with zero changes to the application code or configuration.

The only change involves is setting the user/key of the specific cloud.

By adding support for OpenStack, we now enable users to safely move to an OpenStack-based cloud when they're ready and with little effort, yet they gain all the benefits that comes with it in terms of cost, openness etc.

Yesterday, I had a session during the OpenStack Summit where I tried to present a more general view on how we should be thinking about PaaS in the context of OpenStack.

The key takeaway :

The main goal of PaaS is to drive productivity into the process by which we can deliver new applications.

Most of the existing PaaS solutions take a fairly extreme approach with their abstraction of the underlying infrastructure and therefore fit a fairly small number of extremely simple applications and thus miss the real promise of PaaS.

Amazon's Elastic Beanstalk took a more bottom up approach giving us better set of tradeoffs between the abstraction and control which makes it more broadly applicable to a larger set of applications.

The fact that OpenStack is opensource allows us to think differently on the things we can do at the platform layer. We can create a tighter integration between the PaaS and IaaS layers and thus come up with better set of tradeoffs into the way we drive productivity without giving up control. Specifically that means that:

Anyone should be able to:
- Build their own PaaS in a snap
- Run on any cloud (public/private)
- –Gain multi-tenancy, elasticity… Without code changes.
Provide a significantly higher degree of control without adding substantial complexity over our:
- Language choice
- –Operating System
- –Middleware stack
Should come pre-integrated with a popular stack:
- Spring,Tomcat, DevOps, NoSQL, Hadoop...
- Designed to run the most demanding mission-critical apps

The slides below illustrate a very high-level overview of the problems as well as how OpenStack can help provide a solution, from the presentation at the Summit:

PaaS on OpenStack

View more presentations or Upload your own.

We’ve recorded a short demo that shows how all different pieces pieces actually work together in the context of OpenStack:

You should note that since the demo was mainly targeted at illustrating the OpenStack integration through JClouds/OpenStack provider, it doesn’t cover much of the feature set, such as Multi-tenancy, fine-grained monitoring, or fail-over, nor does it show deploying full-blown web-apps and big-data apps, etc.

The actual code for the JClouds/OpenStack provider should be available through the JClouds project shortly.

Call for action

Today, there is a lot of work and interesting innovation being done in the PaaS world by different providers. Unfortunately a lot of that work is done with very little collaboration. The OpenStack community can be a great environment to put all those great ideas into something meaningful and open.

I hope that our initial joint work with OpenStack, Citrix, JClouds, and GridDynamics can be a good start in that direction.

I’m trying to figure out what’s should be the right way to establish a more formalized Open-PaaS group as part of the OpenStack community. Any ideas/help would be greatly appreciated...

GigaSpaces OpenStack Explained

2011-04-11T13:08:08+02:00

One of the major concerns of many IT organizations is cloud vendor lock-in. This concern was expressed recently in "Banks fear cloud vendor lock-in," from IT Wire:

The onset of cloud computing gives vendors the chance to lock customers in to their infrastructure, using proprietary protocols to ensure they’re on the monthly billing cycle as long as possible.

The OpenStack project emerged with a mission to address this concern by creating a community-led open source project enabling any organization to create and offer cloud computing services running on standard hardware.

GigaSpaces joined the OpenStack project with the mission to enable any organization to build its own Platform-as-a-Service (PaaS), with its own choice of language and best-of-breed middleware stack.

In this post, I’ll try to provide more insight into our current and future plans around OpenStack, and more specifically our joint collaboration with the Citrix OpenCloud initiative.

GigaSpaces OpenStack Explained

What does GigaSpaces' OpenStack support mean?

One of the goals for our second-generation PaaS/SaaS enablement platform was to enable smooth migration between different cloud providers. We were able to achieve this goal through the use of our own abstraction (the Scaling Handler) and through the integration with the JClouds project that provides common abstraction to most of the existing cloud providers. With that, we can ensure that any application can be moved from the likes of Amazon to OpenStack or to an organization's own private cloud with zero changes to the application code or configuration.

The only change involves is setting the user/key of the specific cloud.

By adding support for OpenStack, we now enable users to safely move to an OpenStack-based cloud when they're ready and with little effort, yet they gain all the benefits that comes with it in terms of cost, openness etc.

Can I use the OpenStack integration outside of GigaSpaces' platform?

Yes. The OpenStack integration was developed in close collaboration with GridDyanmics and Adrian Cole from JClouds. We intend to make the integration with OpenStack available to the entire Java community through the open source JClouds project. In this way, any Java application can easily run on OpenStack.

What specific application stack is currently supported?

Today, there are a wide variety of tools and frameworks available throughout the ecosystems in Java, .Net, Ruby, and more. We feel that limiting the platform to a specific stack that we control -- as in the case of Google App Engine -- is too restrictive. Instead, we wanted to create a foundation that enables users of the platform to easily host any service or framework they choose on the platform. We also wanted to do this in a way that provides consistent behavior in terms of deployment, monitoring, elasticity, and scaling. Therefore, we developed a Universal Service Manager and Service Orchestration framework. The framework makes it easier users to plug in their own choice of services that comprise the application stack (Tomcat, MySQL, NoSQL through Cassandra, Hadoop, Ruby...).

In addition, we will provide a set of built-in recipes. At first we will be targeting Big Data stacks, which include integration with NoSQL and Hadoop, and also reporting tools to make the deployment of Big Data applications significantly simpler.

The integration with OpenStack will make it possible to bring these benefits to any application running on OpenStack.

How is this related to the previously announced Open Elastic Platform with Citrix?

The new platform builds on the collaboration announced in October 2010 by Citrix and GigaSpaces, highlighting the integration between Citrix® NetScaler®, Citrix® XenServer® -- both part of the Citrix OpenCloud solution -- and GigaSpaces eXtreme Application Platform (XAP).

In this new release we are adding the following new development:

Greater Openness

Openness at the IaaS layer -– The specific integration with NetScaler (load balancer) and XenServer would be done through more open interfaces provided through the Citrix/Openstack contribution, which means that OpenStack users can use those intefaces to plug in any hypervisor or load balancer.

Openness at the application layer –- The previous version of GigaSpaces XAP offered a limited set of application stack support features, mostly geared to services that we control and mostly in Java. With the new platform we offer significantly greater flexibility on various fronts:

- Open to any application stack and language: As I mentioned earlier, users can now easily host any application or service and build their stack of choice, while at the same time managing and controling their application in a consistent way in terms of the deployment, monitoring, elasticity, etc.
- Users can choose their container of choice: The new platform will include support for Tomcat in addition to our existing Jetty support.
- Users can choose standard APIs to scale their data: In most cases users looking for data scalability have had to rewrite their applications. This is still the case with most of the NoSQL and in-memory data grid solutions. Through standard JPA support in our platform, we can finally reduce that lock-in barrier.

Better Application Monitoring, Specifically Geared for PaaS

One of the challenges with many of the existing monitoring systems is that they were not geared for PaaS-based deployment. In this new offering we provde PaaS-driven monitoring that is tightly integrated with the platform, and which can interact with the platform to deal with failure or scaling events without any human intervention. In addition, it provides a holistic view that includes the application and infrastructure, as can be seen in these screenshots:

Furthermore, the new monitoring was designed to integrate with the existing set of data-center monitoring tools.

Better Performance and Scalability

By leveraging the Citrix OpenCloud solution, GigaSpaces will provide tighter integration between the platform and the infrastructure layers to up-level services offered through OpenStack clouds. In this way, we leverage the years of experience of both platforms in delivering high performance and low latency to mission-critical applications and remove the hassle involved in tuning the entire stack. We believe that through this joint work we can bring a lot of that experience into the OpenStack community.

We can also now offer more fine-grained multi-tenancy support –- by combining process and VM-based multi-tenancy. This makes it possible to achieve significantly higher density and utilization of existing resources, and reduces the amount of resources -- and therefore the cost -- associated with serving a particular application load.

What does this means for Enterprises, ISV/SaaS, and Cloud/IaaS providers?

Any user of this new offering will benefit from the greater openess and reduced lock-in concern. They wouldn’t be relying on single vendor for their future. This also means that they will have better control over their own stack and cost margins, and will therefore gain the ability to offer more competitive prices with other equivalent cloud providers. More specifically

Enterprises can use this offering to build their own Enterprise PaaS layer that is specifically geared for big-data analytics, e-commerce, and financial applications.

SaaS ISVs can use this offering to SaaS-enable their application and provide the same solution off- and on-premises.

Infrastructure/Cloud providers can use this offering to offer an Amazon-like services stack, including RDS, SimpleDB, SQS, Cloudwatch, and Elastic Beanstalk as a pre-packaged solution.

Final Words

I’m very excited about all of these new developments. OpenStack fills in the missing piece in the cloud –- as its name suggests, it unlocks the cloud by providing a truly open cloud stack and provides an essential substrate that drives innovation and collaboration -- which couldn’t be done before. It's interesting to compare the level of effort that we had to invest in the past when we did the first integration with Citrix to the one with OpenStack. Openstack enables us to work completely in parallel and make progress without too much coordination and still achieve substantial progress. Moreover, every delivery of our work can be relevant to wider spectrum of users. It also enabled us to join forces with other members of the community, such as Adrian Cole from JClouds and GridDynamics, simply because we share a similar goal to support the OpenStack open cloud mission.

This is only the beginning. I hope that with this new development we can continue to work toward greater openess at the application platform layer together with Citrix and the rest of the OpenStack community.

I’m going to give a talk on on the subject (PaaS on Openstack) during the upcoming OpenStack Design Summit on April 26^th, 2011, in Santa Clara, California, where I hope to share some of these thoughts and hopefully get more work on this happening within the community.

Availability

I will also be announcing the first GigaSpaces code contribution to OpenStack at the Design Summit. Note that the integration with OpenStack will be made available to the entire Java community through the open source JClouds project.

You can also get an early look at our upcoming 2nd-generation CEAP release by registering for the Early Access Program.

References:

Multi-tenancy: emulation or the real thing? - My Take

2011-04-08T16:49:26+02:00

Phil Wainewright wrote and interesting take on Zdnet (Multi-tenancy: emulation or the real thing?) where he outlines different multi-tenancy levels:

Hosting single-tenant applications on multi-tenant infrastructure
Redistributing application processes across virtualized machines
Injecting a user-specific column in the database driver

Phil also addressed two approaches to achieve multi-tenancy, to which he referred as emulation - multi-tenancy that is pre-backed into the infrastructure as in the case of GigaSpaces, or the "real thing," which stands for a custom build where multi-tenancy is handled at the application level as in the case of Salesforce.

In his final takeaway, Phil suggested that to get the most out of multi-tenancy, you have to deal with it at the application level.

The build vs buy dilemma

Phil's point reminded me of the classic build vs buy dilemma – at first sight you would think that a custom build that is tailored to your specific domain and application would always be better than a generic solution. But is it?

I addressed "build vs buy" in a previous post on the subject (The impact of cloud computing on build vs. buy behavior), using Fred Brooks's article No Silver Bullet. There are many lessons that Fred points out in his article that are worth reading, but here's a relevant point I'd like to highlight:

The hardest single part of building a software system is deciding precisely what to build.

Fred suggests that we tend to justify custom builds without properly realizing the associated costs. In many cases, what we tend to refer to as “specific requirements” aren’t that specific after all - or to be more accurate, the differences don’t justify the costs.

What to choose?

In the context of multitenancy, we should consider the following things:

Does the difference justify the cost?

How different would Salesforce multi-tenancy be compared to the multitenancy of any other SaaS or PaaS provider?

I would argue that most multitenancy providers are the same. Differences are minor; the goal is multitenancy, not a mix-and-match of potential features.

Complexity consideration

Dealing with data multi-tenancy at the application level can be error prone and can be significantly more complex than dealing with it at the infrastructure level, with serious effects. For example, I heard from one SaaS provider that a bug in their system led to one customer accidentally getting access to another customer's data. In addition to the complexity, there are things that just can’t be dealt with properly at the application level – for example, in the case of Salesforce, a single tenant is bound to a single database instance and can’t scale beyond that single database.

Time to Market consideration

When you launch a new SaaS or PaaS application, time to market often plays a critical role. Dealing with multi-tenancy at the application level requires lots of effort which can lead to a significant delay in time to market.

Joseph Ottinger points out that this effort can be addressed in lots of ways - perhaps by programming language choice, for example. But even these solutions carry costs, in terms of your team's capabilities or even in scalability. The language you choose, for example, might be able to help your time to market, but might cost you in performance, or even address how your architecture is built. These choices can be made well, but you should go in with eyes open to what choices you're making.

Cost consideration

When were dealing with multitenancy at the application level, we're often limited to cross-grain multi-tenancy as we don’t have full control over the underlying infrastructure. For example, in the case of data-multi-tenancy we can easily scale out and and scale down our data services based on the actual capacity that is being used – doing that at the application level would be much harder, maybe close to impossible. That means that we can achieve better density and thus control the number of resources and thus the cost at each given point in time by using platform multitenancy.

It’s not a mutually exclusive decision

In the context of multi-tenancy, there are always going to be things better dealt with at the infrastructure level such as data-multi-tenancy – and application specific multi-tenancy such as the user interface layout.

By delegating the generic part of the challenges of multi-tenancy to the infrastructure layer we can devote more time and effort at areas more specific to our business.

The full story in 15 min

Phil's article triggered a more in-depth discussion with Uri Cohen, GigaSpaces product manager, which later led to a recorded interview (15 Min on Multi-Tenancy) in which Uri address the following topics:

Multi-tenancy: emulation or the real thing? the classic build vs buy dilemma
What is multitenancy anyway?
Why is it such a popular topic?
Is it a feature or best practice?
What is GigaSpaces Multi-tenancy?

Enjoy..

References:

Scaling Social Ecommerce: Architecture Case study

2011-04-04T15:18:25+02:00

In February this year I had the opportunity to meet Tomer Gabel, an Application Engineer at SHC Israel, who gave an excellent talk at one of our customer road shows about Social E-Commerce and the scalability challenges associated with handling of social graphs. Tomer later joined me for a case study, published on InfoQ.

In this post I tried to summarize the main takeaway from the interview. Enjoy..

What is Social E-Commerce anyway?

According to wikipedia, Social Shopping is a mechanism for e-commerce where shoppers' friends become involved in the shopping experience. Social shopping attempts use technology to mimic the social interactions found in physical malls and stores.

The opportunity

A recent study showed that over 92 percent of executives from leading retailers are focusing their marketing efforts on Facebook and subsequent applications. Furthermore, over 71 percent of users have confirmed they are more likely to make a purchase after “liking” a brand they find online. (source)

I recently came across an interesting analysis on the Subscribers, Fans, & Followers: The Social Break-Up. The report analyzed social behavior, and more specifically why consumers end brand relationships. Even though the focus of this post is not about ending brand relationships, I thought that some of the statistics presented in this report are quite useful for quantifying where we are today with social e-commerce in terms of the market size and its potential.

Facebook statistics:

73% of U.S. online consumers have created a profile on Facebook.
65% of U.S. online consumers are currently active on Facebook.
42% of U.S. online consumers (64% of those on Facebook) are “FANS” (have “liked” a company on Facebook).

Twitter statistics:

17% of U.S. online consumers have created a Twitter account.
9% of U.S. online consumers are currently active on Twitter.
5% of U.S. online consumers (56% of those on Twitter) are FOLLOWERS (use Twitter and have “followed” at least one company).
71% of FOLLOWERS expect to receive marketing messages from companies through Twitter.

Clearly the adoption of social commence is quite staggering.

The focus of this post wasn’t to convince you should invest in social commerce but to give a context for one of the common scalability challenges associate with building social ecommerce platform - the "Social Graph."

The Social Graph challenge

The Social Graph is “the global mapping of everybody and how they're related.”

Processing social networks is not an easy proposition:

Massive amounts of branching data
No data locality
Very few assumptions can be made about the data

In other words, to meet the capacity scaling demand you can’t store the data in a centralized location. That forces us to distribute the data. On the other hand, to access (or query) the data, we can’t assume that the data for our query is located in a single node. That forces us to look for the data on multiple nodes.

Let’s take a simple scenario to get some sense of the complexity of the problem:

Example:

Imagine every Facebook user (500 million)
Imagine each person is only connected to 100 others (conservative estimate)

Query: How is user X connected with Y?

X has 100 friends
Each of them has 100 friends
10,001 nodes visited!
101 reads from the underlying storage system!

Clearly even if we can scale at the capacity level we can’t scale at the query level.

The crux of the problem:

High branch factor necessitates many loads to serve even a simple request
No data locality + high branch factor means very high random I/O
Traditional storage models (RDBMS, flat files etc.) are a poor fit

The Solution:

Use Memory as the main storage
- Random I/O access works much better on memory devices than on disk (See more details here)
Execute the code with the data - Using Real Time Map/Reduce
- To reduce the number of iterations required to execute a particular query we use the executor API. The executor API enables us to push the code to the data. By doing that we can execute fairly complex data processing on the data node at memory speed vs network speed.
De-normalize the data
- To reduce the amount of traversal access and network hops per query on the graph we need to copy elements of the graph into each node. For example the list of Friends and friends of friends (up to a certain degree) could be stored in each node and thus become available to any element of the graph without the need to consult with other nodes.

The operational perspective

Scaling Social Ecommerce involves not just the application architecture but also the operational aspect. This is of particular importance as ecommerce and social sites tend to evolve quite rapidly and therefore the time it takes to release a new feature from development to production is critical. This process is often referred to as Continuous Deployment or in our specific case it can be referred to also as "Continuous Scaling."

There are basically two factors that are important to achieve continuous scaling/deployment:

Automation – if the process of deployment and scaling involves lots of human intervention than the time it would take to release a new feature would get significantly higher. It is therefore important that the entire process can be fully automated. In some cases, you may want to have manual check points in the process but even in that case the expectation is that manual intervention would involve clicking on a <continue> button and nothing else. To achieve this level of automation we need to interact with the our application infrastructure through an API. This process of automation and integration between the development and operational environment is often referred to as DevOps. The GigaSpaces reference for dev-ops API is provided here.
Schema evolution –There are many cases in which we would want to add or change our existing data structure to our existing application as part of the upgrade process without brining the system down. This is considered one of the more complex challenges to deal with with many of the existing data bases. A document model is schema-less and therefore more suited for this sort of rapid data change.

The full story

Tomer did a a great job describing in more details the scalability challenges and the approach that they have taken in Sears to address those challenges as well as providing the specific insight on the performance figures that they achieved with their current system. The full interview recording is provided below:

References

When Big-IT meets Big-Web

2011-03-16T14:23:48+01:00

Last week I had the honor to join an interesting event held by Battery Venture in Stanford titled “BiG IT meets Big Web” which brought together people from the financial industry (JP Morgan, NYSE, BOFA, Barclays, for example) and the Web facing world (Twitter, Netflix, Nimbula, DataStax, Bing Mobile, and others.)

As someone who happened to cover those two worlds for a while through my work at GigaSpaces, this was a unique opportunity meet the two camps under the same roof for the first time.

The two opening presentations by Brad Spiers (SVP President at Bank of America) and Adrian Cockcroft (Cloud Architect at Netflix) were great examples of the different in challenges and culture between the worlds.

Brad talked about Risk Management and Monte Carlo simulations as the core of their IT and DevOps as automation that is being done through scripting by the operation guys to ensure consistent deployment of their apps.

Adrian talked about their cloud strategy: running their entire data center on the cloud, how their devs handle also operations, as well as how they achieved continuous deployment to reduce their release cycles.

Adrian Kunzle, MD., Head of Engineering & Architecture at JPMorgan Chase, was somewhat in the middle between the two camps and provided a good example on how Big IT can adopt some of the lessons from Big Web despite the unique challenges that the finance industry faces - i.e. being highly regulated with lots of legacy apps that are hard to change, etc.

I thought that the main lessons from these talks could provide an interesting view on how Big IT can adopt some of the ideas and lessons from the Big Web world.

My Take: Bringing Big IT and Big Web closer

The move to private cloud and a PaaS model could be a great opportunity to address two of the main challenges that are often thought of as conflicting: control and agility. A common Cloud/PaaS gives the IT better governance and control of the stack that runs their application as well as a way to measure and control their application's SLA in a consistent way. On the other hand, it enables better agility as the cycle to deploy new application and setup a new environment can be reduced significantly.

The list below summarizes the main points that makes it possible to bring the worlds closer together .

Monte Carlo simulations and risk management applications are not the only applications that a typical bank runs. There are still many applications that can take advantage of the agility that comes with the Cloud/PaaS model and they could serve as the early adopters within the bank.
Applications should be abstracted from the notion of public vs private cloud - i.e. they should be able to run in both environments in the same way. The choice of where to run should be determined based on business justification - costs, security, etc.
Continuous deployment can be done by integrating the development and QA systems directly into the PaaS. As many of the banks are highly regulated the actual rollout to production will still involve more control but the cycle for rolling out up to staging environment could be fully automated and the time to release a new feature could be shortened significantly.
Many of the Big IT organizations have consolidated their use of language to Java and .Net. Hence by supporting Java and .Net, it is possible to cover 90% of the total applications.

Final notes

At the end of the day, and hearing the different views, it was clear that the difference in challenges and culture between the two industries is still fairly large. In many cases, and for good reasons, Big IT is highly regulated, many of the applications are highly transactional, and don’t have the same level of flexibility in terms of consistency, latency and even the types of applications that they are running. Those differences have been the excuse for people in the two camps to dismiss each other's arguments and live in separate worlds in the way they operate and runs their applications.

We're reaching the point where it becomes clear that there is a direct link between the way you run your applications and the way you run your business.

The Big Web world had the luxury to start with a clean slate and quickly iterate and develop efficient models for running their applications (and their businesses). For different reasons (regulation, legacy, applications) Big IT couldn’t move at the same speed but there no reason why it couldn’t adopt those same models even if it wouldn’t apply to their entire IT organization.

On the other hand, Big IT learned how to deal with scaling under extreme latency and throughput and without compromising on consistency. Big Web could benefit from some of the patterns that address this sort of extreme requirements without giving up on consistency or latency as I already noted in An interesting note on Google Megastore & CAP.

Anyway, I came out of this day encouraged that the two worlds can actually converge despite the challenges and differences. This sort of meeting is definitely a good and important start in that direction.

In the next post I’ll cover the second part of that day – specifically the DevOps pannel.

References

GigaSpaces New Cloud Platform - Sneak Preview

2011-03-08T23:14:44+01:00

The presentation below provides a sneak preview into one of the cool features of our upcomming Cloud Enabled Application Platform - Uniersal Service Manager (USM).

The universal service manager enable to handle the deployment, elasticity, continues avliability, scalabiltiy of existing application on any cloud or local data center. In this specific presentation were demonstrating how we can deploy NoSQL Service - Cassandra, using the USM , scale-it, monitor it, handle failure scenario etc. All that without writing any line of code and without any need for prior knowlage of GigaSpaes!

All you need to do is point to your cassandra directory, write few shell scripts for handling the pre-post deployment and the rest is taken care of by the platform.

Were showing a live demo of that feature on Cloud Connect, so if your in the area stop by our CloudConnect booth (106).

For those interested in trying it out themselfs were going to make the first milestone avliable for public download by the end of this month.

You can also send an email to (pm at gigaspaces dot com) and register for updates on this new release!

GigaSpaces new cloud platfom - sneak preview I

View more presentations or Upload your own.

Productivity vs. Control tradeoffs in PaaS

2011-03-08T03:05:07+01:00

Gartner published recently an interesting paper: Productivity vs. Control: Cloud Application Platforms Must Split to Win. (The paper requires registration.)

The paper does a pretty good job covering the evolution that is taking place in the PaaS market toward a more open platform and compares between the two main categories: aPaaS (essentially a PaaS running as a service) and CEAP (Cloud Enabled Application Platform) which is the *P* out of PaaS that gives you the platform to build your own PaaS in private or public cloud.

According to Gartner the main split between the two categories is Productivity vs Control:

The cloud application platform markets are splitting to support two different constituencies: mainstream application developments that are focused on fast time to deployment, and advanced projects requiring the full control of the underlying cloud application platform attributes.

The paper also provides useful takeaways that follow that line of thought:

The aPaaS market is now dominated by high-productivity offerings targeting mainstream, often opportunistically oriented, rapid application deployments, at times implemented by "citizen developers" without formal IT department approval.
High-control offerings are emerging to support the most-advanced, systematically oriented requirements. These projects often look primarily for a CEAP, rather than an aPaaS, to enable maximum control over the technology environment.

My take:

While I was reading through the paper I felt that something continued to bother me with this definition, even though I tend to agree with the overall observation. If I follow the logic of this paper than I have to give away productivity to gain control, hmm… that’s a hard choice.

The issue seem to be with the way we define productivity. Let me explain.

Productivity – redefined

The term "productivity" is pretty wide. Even after a short research on the subject it became clear to me that even today were lacking an agreed-upon model for measuring Programming productivity as noted in the wikipedia definition:

A generally accepted working definition of programmer productivity needs to be established and agreed upon. Appropriate metrics need to established. Productivity needs to be viewed over the lifetime of code. Example: Programmer A writes code in a shorter interval than programmer B but programmer A's code is of lower quality and months later requires additional effort to match the quality of programmer B's code; in such a case, it is fair to claim that programmer B was actually more productive.

During my short research I came across an interesting study on the subject by Google. I thought that the study went a long way toward defining productivity. It does so by taking a a specific challenge (HPC in this specific case) and measured the actual progress students made on developing a certain task at different stages using various technologies. For example, it measured how much time was spent to develop a certain algorithm, the time it took to debug it, optimize it and get it working. One of the interesting outcomes of this study that seems to be generally applicable to many other technologies is that we often spend more time debugging and optimizing code than on the actual development. It was also found that the biggest bang for the buck is gained by shortening the transition between the various development cycles and less by reducing the time of each individual cycle. (I’ll get back to that later on in this post.)

Which platform is more productive?

On a question comparing the RoR and Garils platforms on stackoverflow (Is developer productivity higher on Ruby on Rails or Grails?) it became clear that answer to the question is always subjective as it depends on your existing skillset, legacy, framework etc. In other words, there is no clear winner and the answer can be different on a per case basis. This brings me to the point that its going be hard to claim that one platform is more productive than the other without taking the specific context in mind.

Also see Joseph Ottinger's post, "It's not about C++ and Java performance," which addresses a series of benchmarks people were trying to use to assert than Java was better than C++ and vice versa.

Defining Platform-Productivity

The definition that seem to resonate best for me was provided by Can Wolf on one of his comments on stackoverflow thread above:

Productivity is measured by units of features being delivered (not lines of code).

Can Wolf's comment in brackets is even more interesting. In many of the recent language debate centered around productivity we seem to be zeroing on the number of lines of code required to deliver a given functionality as the main measure for productivity. As I noted in the opening of this section reducing the number of lines of code covers only a narrow aspect of productivity - and not necessarily the best one.

Measuring the the time it takes to bring new feature from development into production is where I believe we should put the focus when we measure productivity as this is what really matters – by looking at that measure we may find that language choice has actually only a marginal contribution on our overall productivity in comparison to how well we can speed up the time it takes to go through the QA and Production cycles. We may also find that the right choice would actually be to combine different languages for different purposes.

In a case of a PaaS platform that means that a platform that makes it extremely easy to write your own hello-world application wouldn’t be considered productive if it fails to take you effectively through the entire debugging and optimization cycles into production.

Productivity vs. Control tradeoffs in PaaS

I started this post by quoting Gartner on the choices between Productivity and Control when looking into the two main categories in PaaS market.

I thought that Carlos Ble's post Goodbye Google App Engine (GAE) is a good example that illustrate why the initial perception behind GAE as simple platform that provides extreme productivity can be completely wrong.

..developing on GAE introduced such a design complexity that working around it pushes us 5 months behind schedule.

Part of the reason that brought Carlos through that experience IMO is that in the course of trying to make GAE extremely productive the owner made the platform too opinionated to the point where you lose all the potential productivity gains by trying to adopt their model. In addition to that, with a platform like GAE you have very little freedom to leverage existing frameworks such as your own database, or messaging system, or any other third party service that can by itself be a huge contributor to productivity.

Instead, you're completely dependent on the platform provider's stack and pace of development and that in itself can work against agility and productivity in yet another dimension. In this specific example Carlos couldn’t use a certain version of a Python library which would have made his productivity higher and had to work around issues that were already solved elsewhere. This is a good example how the lack of flexibility lead to poorer productivity even in the case of simple applications.

Amazon PaaS (BeanStalk) vs Google App Engine – Productivity + Control

It's interesting to compare Google App Engine with Amazon Beanstalk. Amazon provides full control over almost every piece of their platform. You could choose your own operating system and integrate with external database and other services of your choice, etc. At the same time, Amazon provide layers of abstraction on top of their infrastructure through services like RDS, SimpleDB, SQS, MapReduce, and Beanstalk. In addition, to that they have an network of ecosystem PaaS providers such as Heroku, GigaSpaces and others that run on top of their IaaS services and complement their offering.

Open productivity

Amazon's approach is to provide what I would call Open Productivity. It enables you to start with extreme productivity where you would give up some degree of control at the earlier stages. Unlike GAE you're not locked into any of those layers of abstraction and you always have the choice to go another level down and pick your own database of choice, OS framework, etc.

Amazon's approach is pretty much aligned with the approach that we took in GigaSpaces in 2009 when we developed our first generation PaaS, as I noted in this post Google App Engine plus Amazon AWS: Best of both worlds. In other words, you don’t always have to trade extreme productivity for control. If the platform architecture is layered correctly it is possible to get a good degree of productivity and control as in the case of Amazon and GigaSpaces. Choosing between the degrees of productivity and control becomes a decision of which layer of abstraction to start with and not necessarily which platform to choose from.

Doing it the other way around - i.e. providing closed productivity first - without a good degree of control could lead to high productivity at the beginning but end up with low productivity as the project evolves as noted in Carlos' experience. In addition to that it is often much harder to take a closed productivity platform and open its control levels at a later stage if the platform was not designed for it from the start.

The caveat though is that it often takes more time to start with an open platform and build the layer of abstraction on top. In my view, this leads to a better evolution as you grow with your users and follow their demand rather than the other way around.

Bottom line – It’s not about productivity

After working on this short analysis, I came to the conclusion that since productivity is a broad term it would be wrong to think that our choice is to trade productivity with control when we select aPaaS or CEAP platforms.

To put it simply the main difference between the current class of aPaaS and CEAP is that most aPaaS are targeted toward the long-tail of web applications where CEAP is targeted to the more high end part of the spectrum.

All of the platform would try to provide extreme productivity for the type of applications that they are targeting.

A better way to make a platform choice would be based on the target applications that each platform is aimed for. For example I would look at GAE for extremely small Java or Payhon applications (widgets) or prototype, Heroku for small Ruby based App, Force for an application that needs to integrate more tightly with the salesforce ecosystem.

I would look for Beanstalk if I’m already running a simple java application and don’t plan to switch off of Amazon anytime soon. I would choose CEAP such as GigaSpaces in cases where I don’t want to be bound to a particular cloud provider and want the flexibility to run the same application on variety of environments private or public with or even without virtualization, In Java or .Net. The later would fit into most enterprises and the more high end SaaS providers where the first would probably more into end users applications.

Extensibility & Flexibility

One of the attributes of control is extensibility and flexibility. Extensibility and flexibility is another factor worth considering when selecting a platform. As it stands today, the various platforms tend to be quite different in the level of choice that they offer in language, frameworks, and the ability to plug-in your service of choice or using only parts of the platform (database, messaging, ..). As noted earlier in the GAE example, flexibility can actually lead to greater productivity.

What’s next.. toward a 2ndG PaaS

If we follow the evolution of aPaaS and CEAP solutions, we can see a consistent shift toward more open platforms than the one that was available in the first generation PaaS. With the emergence of more open infrastructures in the form of OpenStack, as well as the evolution of enterprise data centers toward hybrid clouds, we can expect even higher degree of flexibility not just on Amazon but on any cloud (private or public). The other thing that comes up often in many of the recent DevOps discussions is the demand to enable similar level of productivity and automation that Amazon (or GigaSpaces for that matter) provides for its own built-in services to external services of my choice.

Gartner's paper does a petty good job covering that shift in the PaaS market but the use of productivity as the differentiating factors can be misleading as I outlined in this paper.

The good news is that they were expected to get much better selection of choices between the various tradeoffs around productivity and control. I will dive into the details of that in a followup post.

PaaS trend discussion @ Cloud Connect

In this post I tried to lay out my own analysis on this matter. I’m very interested in other people experience on this subject so If this topic is of interest to you and you happened to be in CloudConnect next week I’d be happy if you would drop me an email (natis at gigaspaces dot com) or simply meet me at our CloudConnect booth (106).

Nati Shalom's Blog

GigaSpaces Cloudify & VMware CloudFoundary the new PaaS Jailbreaker

The Difference Between PaaS and DevOps

Putting DevOps & PaaS togatehr

What Makes a Cloudify and CloudFoundary a PaaS Jailbreaker?

Quick Introduction to Cloudify Recipes

References

Realtime Analytics for Big Data: A Facebook Case Study

2012 Cloud, PaaS, NoSQL Predictions

Making Cloud Portability a Practical Reality

Public vs Private clouds (Again!)- it's not about the cost

Five Misconceptions on Cloud Portability

Five Common Misconceptions on Cloud Portability

1. Cloud portability = Cloud API portability

2. The main incentive for Cloud Portability is - Avoiding Vendor lock-in

3. Cloud portability isn’t for startups

4. Cloud portability = Compromising on the least common denominator

5. The effort for achieving cloud portability far exceed the value

Final notes

Behind the Curtains of Cloudify, GigaSpaces' new product

References:

Java and Center Stage: Meet us at JavaOne 2011

What This Means

A Remaining Challenge

You know how to solve your problem. We can help.

Big Data Application Platform

Characteristics of Big Data Application Platform

Support Batch and Real Time analytics

Making Big Data Application Closer to Mainstream development practices

Domain model and Data access API

For Big Data Mapping tools:

For Big Data Spring Abstraction:

Business logic

Support new semantics that fits the dynamic web era

Provide built-in semantics for handling of the tradeoffs between consistency, availability, scalability rather than trying to force a least common denominator as with XA and JTA

Provide built in support for In-Memory data store to gain best performance and latency

Provide built-in support for event driven data distribution using pub/sub model

Built in support for public/private cloud

Open & Consistent management and orchestration across the stack

Final words

References

Real Time Analytics for Big Data: An Alternative Approach

A Summary of History

A Potential for Improvement

A Plan for Implementation

A Discussion of Strengths

Final words

Real Time analytics for Big Data: Facebook's New Realtime Analytics System

Why now?

Hadoop Map/Reduce doesn’t fit the real time business

Facebook's Real Time Analytics system

Evaluating the Facebook Architecture

Memory Counters

Cassandra vs HBase

The Architecture

Realtime.Analytics.next

Interestingly enough, I was asked to draw a solution for similar challenge for a voice recording system: collect lots data from various sources and process them in real time. The good news is that it's doable, as shown by Facebook's success.

References

Read/write scale without complete re-write

Avanza's System Description:

Scaling architecture of the existing site

The New System

Challenges

The TCO angle

Final words

GigaSpaces Citrix integration on top of OpenStack

GigaSpaces Citrix integration on top of OpenStack

OpenStack Compute (Nova)

OpenStack Load Balancer

Citrix certified version of OpenStack

Citrix OpenStack virtual appliance project

References

PaaS on OpenStack

GigaSpaces OpenStack Explained

OpenStack virtual appliance project

PaaS on OpenStack

Call for action

GigaSpaces OpenStack Explained

GigaSpaces OpenStack Explained

Availability