Nati Shalom's Blog

Platform as a Service: The Next Generation Application Server?

2009-07-13T15:00:00+02:00

Platform as a Service is a term that can be fairly confusing for many people. Normally the term is associated with Google App Engine from Google and Force.com from Salesforce.com as the main references for this model. From a technical point of view, it is aimed to provide a similar type of value to the one that is currently provided by many of the application servers i.e. it provides a generic container that can host different applications and shield them from the details of the underlying operating system, network, database implementation. Unlike most of the existing application servers it was designed for massive scaling from day one. Another big difference is in the way it is being consumed. With PaaS you don’t need to install any software and go through all the hoops to setup a cluster environment etc. PaaS is provided as a hosted service that is pre-configured and installed. You get a production ready environment right at the start.

Key Characteristics of a Cloud/SaaS Enabled Application Platform

Last week I came across David Mitchel Smith presentation from Gartner. David provided a good definition to the main PaaS characteristics. A snippet from his presentation covering this area is given below:

What’s interesting is the great emphasize on Multitenancy support. The fact that the platform is going to be shared between multiple applications and potentially even different customers require various levels of tenancy support and isolation to ensure that even though were taking advantage of the fact that we can share resources between applications, each application needs to be able to use the platform as if it is running on its own dedicated resources. Another interesting point is the need for XTP support. Many would view XTP as a niche that is normally referenced in the high end part of the market, so it is fair to ask why would XTP fit into a general PaaS solution? XTP represents a model for supporting enterprise transaction processing applications in an extremely scalable environment. In our case, scalability is not necessarily driven from the demand of a particular application but from the fact that many applications are going to run on a shared environment. A PaaS targeted to enterprise applications would need to provide support for this level of scalable transaction processing support as a core service.

Can you run your existing business applications with GAE or Force.com?

No. Unfortunately as in anything in life reality can really spoil the party.

Force.com was designed to make it simple to run business applications that are database centric i.e. CRM, Reporting etc. It provides a rich set of high level services that make building such applications extremely simple. Google provides a more generic application platform and recently announced support for Java which is a big step towards reducing vendor lock-in concerns. Google seems to be geared for consumer based applications. Force.com offers a more high level platform that is based on its proprietary services. This means that in order to take advantage of their service you will need to go through a complete re-write.

Force.com is based on a database centric architecture. They also seem to be limited in scalability as they partition their database per application. This means that if your application needs to scale more than what a single database can provide, you can find yourself pretty much locked.

Google's recent support for Java makes their offering closer to standard JEE application servers, however their current support impose a lot of limitations due to their sandbox model. These limitations mean that at the end of the day GAE can be applicable only to a small set of relatively simple applications. Since there is no guarantee or control over the resources that you are going to receive from their underlying infrastructure, it is likely that the application performance will be unpredictable and will therefore be affected by other applications that are sharing the same hardware.

The fact that the platform as a service shields you from the details of the underlying infrastructure is what makes it simple and that is both the advantage and limitation. You can’t control the environment, you can’t choose your operating system, you can’t install your own set of services and you can’t control the performance characteristics of this platform.

This puts a huge adoption barrier for most current enterprises.

IaaS vs PaaS

Infrastructure as a Service provider provides a hosted service model that offers plain machine level access. This model is also known as Server as a Service. The fact that you get access to the bare metal allows you to run almost any application on this hosted environment. Unlike PaaS, IaaS gives you extreme flexibility. You can choose your own operating system, install any package that you want, you can setup your firewall, security etc. Amazon is known to be the leader in this space and also provides a set of services on top of their infrastructure such as SimpleDB, SQS and MapReduce. Having said all that, this flexibility comes at the cost of complexity. In many cases you will need to install your own software, configure it, tune it etc. before you could make it run effectively on the cloud. Many application developers don’t have the skill-set to do that. This exposes many operational challenges as many organizations are not geared to support this type of environment from their own IT. The high level services provided by Amazon are still proprietary and would require a complete re-write if you plan to use such services.

PaaS for Enterprise applications – doing it right

The ideal solution would be to combine the best of the two worlds i.e. the flexibility of IaaS and simplicity of PaaS, and here is how:

Build a Generic PaaS on top of AWS – To build a PaaS we don’t need to re-invent the wheel. Unlike Google and Force.com we don’t need to own the infrastructure, we can actually use Amazon infrastructure or even better build our PaaS such that it can be portable between Amazon IaaS and a VMware IaaS. By doing so, the PaaS can provide us the ability to deploy applications in a simple way just as in GAE but would still enable us to control the environment, install our own software and get the full flexibility that IaaS provides.
JEE as first class citizen – Many of the existing enterprise applications are built in JEE. Making JEE a first class citizen within our PaaS environment will enable you to leverage the existing skill-sets within those organizations. Similar to the standard model that has more than one implementation out there, reducing the lock-in factor significantly.
Pre-configured for extreme scalability – All the services provided through the PaaS will need to be implemented and pre-configured for extreme scaling and come with a production ready setup to enable dynamic scaling and fault-tolerance.

Next generation application Server?

Yes, In my opinion, PaaS represents the next wave in middleware technology. One that is targeted for virtualized enterprises, one that was designed for scale-out from the get go, one that fits the new way of delivering SaaS applications and one that can be extremely simple to use. The current PaaS players i.e Google and Salesforce and to a lesser degree Microsoft represent one type of player, those that own it all, i.e. the infrastructure and the platform. There is already a new emerging category of players in the PaaS market, Application PaaS players. Application PaaS players would specialize on delivering only the platform and not the hardware and network infrastructure. They act as the bridge that will enable portability between different infrastructure providers including the internal IT (a.k.a private-cloud). The application PaaS players will also be segmented in similar ways to the way application servers are segmented today, i.e there would be the one targeting the low-end consumer market in similar ways to Google App Engine, the ones that will be aimed toward the high end of the market and those who will be specialized in certain languages or development framework, i.e. Ruby/ Java/ .Net etc. In the PaaS type of world, those who would be able to provide a holistic solution that works smoothly across all the application tiers would have an advantage over those who are providing point solutions.

What about my existing applications ?

Most people would categories PaaS as a platform that is delivered through the internet. That statement would make the idea behind PaaS irrelevant for a large part of the existing enterprise applications, as a majority of them are not ready to run their application in a hosted service over the internet. Let’s examine that statement:

Many of the existing IT run farms of application servers in their internal IT. Those application servers are running in an internal data center that is not that different from any other hosted services, only that it is a specialized hosted service tailored for the needs of the specific organization. I would therefore argue that those organizations that are already running such application server farms would find it easier to evolve such server farms to PaaS model than to change their entire IT infrastructure into internal cloud. The reason is that those applications were already written to run in an application container model, therefore a large part of the transition work can be done within the container implementation and outside the application code. Targeting them first would therefore be potentially easier transition than trying to transform all your other applications into a virtualized environment.

I know that there are many people out there that would argue that internal PaaS is not a PaaS because it doesn’t answer the exact definition of PaaS. However in my view the similarities exceed the differences and the value to the enterprise would be almost identical to the one that I would receive by any of the internet based PaaS platform.

Will I lose control?

PaaS will provide the internal IT much better control over the applications that are running on their environment. You can have high visibility as to ways the application consume resources. In the same way, you can control fairly tightly the application security, scalability, resource management and fault-tolerance which can finally be managed in a consistent way across all the applications.

Final words

Platform as a Service represent the next generation application server IMO. GAE and Force.com are most known references in the market for that model. Both tries to offer the complete stack and that’s both their advantage and limitation. There already is a new category of Application PaaS providers that specialize on providing PaaS on top of existing infrastructure providers. There would be generic PaaS providers (similar to GAE) geared for the low end part of the market and those that are geared for the high end of the market. There would also be more vertical PaaS providers (Similar to Force.com) i.e. those that will provide PaaS for a certain segment of applications or segment of the market such as Online Gaming PaaS, Telco PaaS etc. A good example for such a service is Twilio. As I outlined in my recent post (Google App Engine plus Amazon AWS: Best of both worlds), this is not just a theory but a reality in the making. GigaSpaces' contribution to this new reality is our new cloud framework on Amazon EC2. Geva Perry provides an excellent overview of other good examples that follows that same line his post What's Really Exciting About Cloud Computing.

References:

No to SQL? Anti-database movement gains steam My Take

2009-07-09T14:33:55+02:00

Eric Lai published a provoking article on Computerworld magazine titled “No to SQL? Anti-database movement gains steam” where he pointed to many references in which different Internet-based companies chose an alternative approach to the traditional SQL database. The write-up was driven from the the inaugural get-together of the burgeoning NoSQL community who seem to represent a growing Anti-SQL database movement.

Quoting Jon Travis from this article:

Relational databases give you too much. They force you to twist your object data to fit a RDBMS [relational database management system],

The article points to specific examples that led different companies such as Google, Amazon, Facebook to choose an alternative approach. I outlined below what i found to be the main drivers behind that trend:

Demand for extremely large scale:

“BigTable, is used by local search engine Zvents Inc. to write 1 billion cells of data per day.”

Complexity and cost of setting up database clusters:

“PC clusters can be easily and cheaply expanded without the complexity and cost of ‘sharding,’ which involves cutting up databases into multiple tables to run on large clusters or grids.”

Compromising reliability for better performance:

“There are different scenarios where applications would be willing to compromise reliability for better performance. A good example for such a scenario is HTTP Session data. In such a scenario the data needs to be shared between various web servers but since the data is transient in nature (it goes away when the user logs off) there is no need to store it in persistent storage.”

Having said all that, it seems that many still agree that despite all the limitations of traditional database solution, SQL database are probably not going away:

“It's true that [NoSQL] aren't relevant right now to mainstream enterprises," Oskarsson said, "but that might change one to two years down the line.”

The current "one size fit it all" databases thinking was and is wrong

The article seem to point to an interesting trend where a growing number of application scenarios cannot be addressed with a traditional database approach. This realization is actually not that new. In 2007 I wrote a summary of Michael Stonebrake’s article, "One size fits all: A concept whose time has come and gone" on my blog: Putting the database where it belongs. The great thing is that It looks like this “old” news is spreading to the larger community. This can be explained by the continuous growth of data volumes, together with the growing need to process larger amounts of data in a shorter time. These two trends force many users to think of an alternative approach to the traditional database. The classic early adopters are those who hit the wall. It is very likely that as these alternative solutions mature, they will find their way into mainstream development as well.

Not your mom and dad’s database

The article seems to over glorify some of the alternatives that where mentioned while downplaying their limitations. A good example is Amazon SimpleDB. I wrote in the past a post about this, Amazon SimpleDB is not a Database, where I outlined some of the limitations of the Amazon SimpleDB solution. As you can see from these limitations, SimpleDB cannot and shouldn’t be positioned as a direct alternative to your existing database.

While I share many of the thoughts and enthusiasm of the anti-SQL movement, I would highly recommend taking very cautious steps toward any of the alternative solutions. It is very important that you make yourself familiar with their strengths and weaknesses, and avoid hitting their limitations at a point in time when you have very little room to maneuver. I’ve seen several cases where users developed their data model in a centralized model and expected that it will scale seamlessly once they switch to a partitioned topology. The fact that you can switch between centralized and partitioned topologies without changing your code doesn’t mean that your application will behave correctly and will scale as you expect.

This topic has actually been the center of a discussion in Architect Summit meeting we had last summer, which was hosted by eBay:

Abstractions and Partition Awareness
A horizontally-partitioned system typically provides an abstraction that makes the partitions appear as a single logical unit. eBay and Flickr, for example, both use a proxy layer to route requests by a key to the appropriate partition, and applications are unaware of the details of the partition scheme. There was near-universal agreement, however, that this abstraction cannot insulate the application from the reality that there partitioning and distribution is involved. The spectrum of failures within a network is entirely different from failures within a single machine. The application needs to be made aware of latency, distributed failures, etc., so that it has enough information to make the correct context-specific decision about what to do. The fact that the system is distributed leaks through the abstraction.

My recommendation would be that you design your data model to fit into a partitioned environment even if during the initial stage you’re still going to use a single centralized server. This will allow you to scale when you need to, without going to through a massive change.

What about in-memory alternatives?

An option that i found missing from this article and becomes fairly popular with many large websites is the use of an in-memory data store, In-Memory-Data-Grids as they are often called, such as Memcached, GigaSpaces, Coherence, eXtremeScale etc. With this model we front-end the database with an in-memory cluster which becomes the system of record and uses the SQL database as the background persistent store. For those looking to build social network graphs, real time events (as in Twitter), real time analytics, fraud detection, session management, etc., that is probably the more natural choice. Todd Hoff from highscalability.com wrote a very good article on this subject: Are Cloud Based Memory Architectures the Next Big Thing? I also wrote a detailed description how this approach works with GigaSpaces and MySQL: Scaling Out MySQL.

What about ACID transactions, consistency etc?

The traditional 2PC (two phase commit) model in which consistency is achieved through a central transaction coordination server is not going to fit with many of the distributed data management alternatives. In an earlier post, “Lessons from Pat Helland: Life Beyond Distributed Transactions,” Pat Helland suggested an alternative model to distribute transactions, a workflow model. Instead of executing a long transaction and blocking until everything is committed, break the operation into small individual steps where each step can fail or succeed individually. By breaking the transaction into small steps, it is easier to ensure that each step can be resolved within a single partition, thus avoiding all the network overhead associated with coordinating a transaction across multiple partitions. This has been one of the core concepts in designing scalable applications with Space Based Architecture (SBA). The Actor model that was introduced with new functional languages like Scala and Erlang is built into the SBA model, with the difference that in SBA, actors can share state and pass events by references, and thus avoid the overhead of copying the data with every transaction.

Shay Banon wrote a good description on how the Actor model works with SBA in this post:

“The above is a diagram of a simple simple polling container that wraps a service (Actor, the Order service in our example). The polling container takes (removes) events (Data), process them, and writes back Data to the collocated Space (Data Grid) it is running with.”

The need for real time data processing – a real life example

In the past weeks we had been involved with a prospect who is looking to add a social network to his eCommerce site. One of the requirement for this new service was the need to build a graph that includes friends of friends, and products in catalogs. The process for building that graph with an In-Memory-Data-Grid took 2-3 msec vs. tens of seconds with a traditional approach (note that part of the complexity in this case is that the query itself is ad-hoc and can’t be easily partitioned). Building that graph on-the-fly at these speeds just couldn't be done with traditional SQL database.

The reason that enabled us to get to this level of performance was:

We kept the data in-memory.
A large part of the complex query was pushed to where the data is (you can think of it as a modern alternative to stored procedure).
We used partitioning to spread the data and leverage the accumulated memory capacity of those memory instances.
We used both a scale-out and scale-up model to parallelize the query against all instances and take full advantage of multi-core as well as multi-machine power.
We reduced the number of network hops by pushing the the heavy data manipulation to where the data is and by returning only the accumulated result over the network.

Final words

In summary I would say:

SQL databases are not going away anytime soon.
The current "one size fit it all" databases thinking was and is wrong.
There is definitely a place for a more a more specialized data management solutions alongside traditional SQL databases.

The adoption of these new solutions would be very much determined by two main factors:

How well they integrate with the “existing SQL world.”
How easy it will be to develop for these new alternatives, and how smooth a transition a given solution can offer for the average developer.

This is an area of continuous innovation that has been keeping us fairly busy in the past few years, and will probably continue to keep us busy. I’ll leave the details on how we deal with these two challenges to a separate post.

References:

Google App Engine plus Amazon AWS: Best of both worlds

2009-06-25T14:14:55+02:00

George Lawton wrote a a good summary of my JavaOne talk in his article titled Google App Engine plus Amazon AWS: Best of both worlds

Google App Engine (GAE) is focused on making development easy, but limits your options. Amazon Web Services is focused on making development flexible, but complicates the development process. Real enterprise applications require both of these paradigms to achieve success… What we really want is the flexibility and performance of AWS and the simplicity and ease of use of GAE.

This is exactly what we had been working on for the past year, leading us to the launch of our new cloud platform. With this platform we leverage GigaSpaces XAP as the high performance scale-out application server and Amazon as the robust and flexible IaaS. Together they form an alternative Platform as a Service geared for enterprise grade applications. This allows the cloud environment to inherit the extreme performance, latency and scalability of the XAP platform, which in turn enables achieving your performance and scaling target with less machines, implying a lower cost.

Real-life case study: Primatics financial – Risk analysis as a service

Francis de la Cruz and Argyn Kuketayev from Primatics Financial joined me through the presentation. In their part of the session they described their experience in developing a SaaS application for Real Time analytics.

Kuketayev described how Primatics used this approach to create a new automatically scaling cloud version of an existing banking application. Primatics initially developed a mortgage securities application that allows banks to estimate the value of a basket of hundreds of thousands of loans. The value of these loans fluctuates as economic conditions change and some portion of home owners cannot afford to make payments on their loans. Banks normally only need to assess the value of these loans at the end of each month, making them an ideal candidate for cloud services like AWS.

From a scalability perspective the challenge is to be able to provide a highly multi-tenant application that need to serve many firms, many users in that same firm each running many jobs at the same time. Implementing such a model can be fairly complex as you will need to be able to manage the life cycle of each job and each user independently and in isolation from one another.

Trying to build such a service directly on Amazon is going to be fairly complex, as you can learn from George’s summary below:

Primatics wrote the first version of EVOLV:Risk as a hosted web application for a regional bank.. The application needed to be fault tolerant so that if one node crashed, they did not have to restart the application over again from the beginning. Kuketayev said that it is not just about the loss of four hours, but the office is trying to close out the month and needs to access data to end the monthly cycle so they can go home.

Using GigaSpaces' toolset they rewrote the entire application infrastructure in about four-months to run on top of AWS. Now they can kick off as many instances as required for different banking customers, and each instance runs significantly faster than before. Kuketayev said that it is important for banks that none of their applications run on the same infrastructure as another bank.

The diagram below shows the specific architecture that Primatics ended up using. Those that are familiar with Space Based Architecture would find it fairly straight forward:

The application is built out of a set of processing units. Each processing unit contains the compute agents in the form of a polling-container. The compute agents gets a a reference to a remote Data Grid that is shared by all processing units. Each agent gets the job injected to it by the polling container and gets a reference to the data it required to process the job. Once the job is completed, the result is stored back in the space. The results are flashed out back to a database through a mirror service.

In a case of a failure, other compute agents are able to continue from the exact point of failure and continue the job processing as if nothing happened. This is because the state of the job is kept safe in the data-grid and not in the agent’s memory.

Kuketayev from Primatics nicely summarized thye lesson he learned after going through the experience of trying to build it on his own vs. trying to use GigaSpaces:

Kuketayev said that one of the biggest lessons is that you need to have your infrastructure do the provisioning for you automatically, or otherwise you end up spending a lot of time just turning things on and off. He said they are now using configuration APIs to automate this process, whereas before they were using scripts. This allow for automatically throttling and failover recovery without human intervention.

Kuketayev advised "You need to make sure you use the right tools … You don't want to have to worry about provisioning and reliability. Make sure you have provisioning, failover, monitoring and SLA out of the box."

The full JavaOne presentation is available here:

Alternative to Google App Engine on Amazon

View more OpenOffice presentations from Nati Shalom.

Final words

Fr solution providers the size of Primatics, building a risk analysis application as a service couldn’t be possible without cloud computing. Cloud enabled them to offer their solution as a service without the need to go through major investment of building a data center to support it.

Primatics’ experience is not special. One of the benefits of building Software as a Service is that you have one shared environment for all your customers. At the same time, one of the challenges is that in a shared environment, failure becomes more public and will impact ALL your clients. If the system doesn’t scale well, you’re going to be hit twice as hard as in a standalone application.

Building a robust and scalable SaaS application can be fairly complex. A good cloud infrastructure will get you a first class data center, but it won’t solve your application requirements.What’s interesting with cloud computing is that it forces you to think about the cost and efficiency of your application more than ever before. In the Primatics example, running a simulation of 100 nodes for 3 hours is very likely to fail at some point. A failure during such a simulation will immediately cost you 300 hours, not to mention the fact that you might lose the simulation window for the day and the reputation challenge you’ll will be facing with your customers. In addition, putting the data in-memory and making the application run 3-5 times faster means that you would need 1/5 of the machine power, which saves 80% of the cost of running the application.

I believe that the challenges imposed by cloud computing force us to focus on what we do best and avoid investing in areas which are not core to our business. Because the pay-per-use model significantly lowers the cost barrier, going down the path of writing your own infrastructure, as many have tried to do before, will be much more expensive and risky then ever before.

References:

GigaSpaces Launches a New Version of its Cloud Computing Framework

2009-06-01T22:42:40+02:00

We have been working for a while on our new Cloud Computing Framework (CCF). We wanted to wait until we had some real customers and partners behind the platform in production before spreading the word on a larger scale. And now, I am very happy to report that we have reached this point sooner than expected and just in time for the JavaOne event.

The new version includes an enhanced integration with the Amazon billing system Devpay designed to enable users to use GigaSpaces as a service on a true pay per use model. All you need is an Amazon account and your done. Unlike other products, that are offered in similar fashion, we decided not to provide just a bundle of our software packed in an Amazon image (AMI) but instead provide a more tight integration with EC2 that will enable our users to use a single GigaSpaces product for all GigaSpaces versions as well as customize the installation to include their own software packages and application code on the fly. The new CCF provides a true end to end experience that will enable our users to deploy and manage their *entire* application from the load-balancer to the database in one single click. The latest release of CCF includes support for the new GigaSpaces 7.0 release. Users that are looking to try out the new XAP product can now do that for free without the need to download or install anything. This will enable you to try out any of the built-in examples, test the high availability features and run benchmarks in just one click. The new version includes lots of additional improvements to the user experience as well as new features and bug fixes. To view a detailed list of the new features click here.

Those that will be visiting JavaOne this week will be able to join the hands-on labs session and experience a full deployment of JEE application using CCF. You will also be able to gain insight from one of our customers, Primatics Financial on how they used CCF to build their Risk Management application as a service – See full details here

In this post I will try to address some of the most common questions that we often receive when introducing this new service:

Who is using GigaSpaces CCF and Why?

The new release is already used by different customers in the following areas:

ISV’s and Service providers using CCF for SaaS enablement:

Many ISV's are looking for ways to move their standalone applications into a SaaS based offering. In doing so they face a few challenges:

1. Scalability – now their application needs to serve not just the users of a certain customer at a certain location but all customers from all locations. This increases the scalability challenge on their product by an order of magnitude.

2. Continuous high availability – one of the challenges that is imposed by having a shared platform for all customers is that the impact of failure can be catastrophic as failure can affect all of your clients at the same time and becomes public fairly quickly.

3. Portability – There are cases where some customers would still require their application be installed in their local data center for security and latency purposes. Today, this often means you will need to develop two seperate branches of your product to serve these two scenarios. With GigaSpaces you can write your application once and deploy it in the cloud or in your customer data center without having to modify your application and without locking your application to a specific cloud provider.

4. Performance – In developing high performance applications such as Trading, Risk Analysis, or SIP server, keeping the latency at sub msec and throughput at a maximum becomes critical. This is where our XAP middleware has a proven record in running numerious production mission critical applications.

The following are a few public references that chose to use XAP:

- Primatics Financial is using GigaSpaces CCF to build a high performance risk management solution

- Orbyte is using GigaSpaces CCF for building state of the art high performance trading applications that can be offered as service.

Other customers include Online Gaming companies that were looking for a simple and cost effective manner to launch new online games and provide a gaming application platform that enables other gaming providers to host their applications easily in a cloud environment. See further details on our online gaming solution here.

Enterprise customers using CCF to improve time to market of new applications and for large scale testing

According to recent research conducted by Forrester, one of the main requirements driving cloud adoption by the enterprise market is that current IT can’t keep up with the business demands of their organization:

Capacity planning is too difficult – The current method of provisioning applications based on estimated peak loads has become either too costly or impossible. Most systems are non linearly scalable which means that knowing how many resources would be needed to meet certain goal is known only through trial and error.
Time to market – launching new applications and services often takes months and sometimes year.
The business wants a simple and cost effective way to prototype – not all applications make it to production but these initiatives still require the business to invest upfront in the purchase and configuraiton of infrastructure to support these prototypes.
Cost saving – There are sets of applications that have fluctuating demand on resources. Enabling these applications to “lease” a common set of resources only when under heavy demand can lead to huge cost saving and efficiency.

I discussed the enterprises challenges in my previous talks. The most recent one is the one I gave during the CloudSlam event: Practical Guide to Developing Enterprise Applications on the Cloud- Online Presentation

A good example of an enterprise customer who is already using our CCF in production is a Large Telco Service Provider that searched for ways to reduce the time to market for launching new scalable web applications. This customer was able to bring a project that previously took more than 6 months just to finish the development and testing to 4 weeks with only 90$ spent on pre-investment.

Large scale testing, performance benchmarks and demo as a service

Users that are not ready to use the cloud to run their production systems can still receive benefits from the cloud for testing and demonstration purposes. There are a few challenges that we tried to address with CCF to make this type of use simpler:

1. Making cloud testing environments identical to your local testing environment – To effectively test your application on the cloud and deploy a production version of your system locally, your application must be at a point where it behaves fairly the same in both environments.

2. Setting up a large scale environment of hundreds of nodes can be fairly complex

3. Trouble shooting failure in such an environment can be a fairly complex and lengthy process.

If your application is already running on our XAP product then porting it to the cloud and running it locally in your production environment will not require any change to your code or configuration. With CCF setting up a large scale environment of hundreds of nodes with the new CCF takes just a few minutes. We designed the CCF to run not only GigaSpaces but other external services which are essential for tests such as JMmeter, Tomcat servers etc. Users can easily configure their own initialization and command scripts when the machine is launched and define which software packages will be installed on their machines on the fly. We added the ability to capture the logs of multiple machines easily and monitor the system and middleware behavior at runtime using a zero installation environment all through the web.

GigaSpaces and our existing customers and partners have already been using CCF to run live demos and recently GigaSpaces started using CCF for our own internal large scale testing purposes. Using the cloud for demo purposes has been extremely useful to show cases that previously were hard to demonstrate such as fail-over and scalability in a real production environment. You can read more on how you can use CCF for running your own demo on the cloud in a similar way without going through all the challenges we had gone through in one of my recent posts: Demo as a Service . Similarly CCF can be ideal for running large scale Benchmarks.

Partner ecosystem -

We designed our CCF in away that would enable us to run not just GigaSpaces components but almost any application such as JEE or Spring based applications. This enables our Solution provider partners to use CCF to offer their specialized solutions and offerings for both GigaSpaces and their other software partners.

System Integrators are also looking for ways to gain a competitive advantage by offering turn key solutions at a significantly shorter time to market and lower initial investment. I tried to outline below some of the public references on these two categories:

Solution providers

Real time production monitoring: Dynatrace offers enhanced integration with CCF for enhanced production monitoring. Dynatrace users are able to monitor in real time how their application is performing and trace their application bottlenecks in real time – see details on this integration here.

Model Driven Development framework - New Technology/enterprise (NT/e) - GigaSystemBuilder helps Java developers prove applications quickly - in a few days - on a local grid or in the cloud, and then rapidly move on to production development, rather than waste time on finger-trouble or implementation details – see more information here as well as a live demo on their deployment manager solution

System integrator

In many cases customers are looking for experts that will enable them to gain the benefit of the cloud and bridge some of the skill-set gaps. AAR is one of our leading system integrators in that field who has already delivered several solutions on top of our GigaSpaces CCF for Enterprise customers. For them, using CCF was a tool to gain a competitive edge and provided their customers the ability to launch new applications in a matter of weeks as opposed to months and at a fraction of the setup (development environment, testing environment, staging environment) and hardware costs that are associated with setting up such an environment with traditional projects.

Here is a quote from one of our System Integrators in the UK that describes their experience in using CCF for the Telco Service provider that I mentioned earlier.

“We use GigaSpaces XAP and CCF to deploy a standard JEE web application on EC2. To address security concerns we kept the business logic outside of the cloud on the customer site. We exposed the business logic through a secured web services channel. We use XAP data grid for maintaining the session high availability and for reducing the latency associated with accessing the remote data center. We used the XAP integrated Jetty container for hosting our web application and through that gained the built-in self healing and auto-scaling provided through the CCF. In this way we were able to deal with potential failure or load without human intervention. From a management and monitoring perspective we were able to leverage the integrated ganglia monitoring and integrated visual representation of different metrics, like cpu/memory/network usage of the entire cluster. We used JMX to bind our custom Mbeans to the integrated ganglia UI. Overall GigaSpaces CCF gave us big boost on utilizing ec2”

I must admit that from our end this was probably the smoothest project we had been engaged with. The reason was probably related to the fact that unlike other projects we had full control over the environment. This enabled us to minimize potential miss configuration errors which is one of the major areas for complexity in setting up such a project. Another benefit was the support. When we faced an issue it was very easy to log-in directly to the machine and fix the problem on the spot without the need to go through a complex process of authorization from the IT Security and without the need to create a reproduction package and ship it over.

The customer was able to deploy a new business application quickly without putting all the initial investment upfront. In this way they gained the freedom to decide at any point if the application should continue, be canned or if they should move it entirely to their local IT.

Looking into the future

The future looks very promising as we continue to work on bringing new customers and businesses towards using CCF and the latest XAP 7.0. We are currently working on supporting the Amazon European datacenter and taking advantage of some of the new XAP 7.0 cluster administration API to enable users to provide thier own custom SLA deployment. At this point, I would like to ask for your specific feedback and wish list for our next major CCF release.

The Need for Speed - Saving Costs By Improving Application Efficiency

2009-06-01T00:55:00+02:00

In times like these, improving application performance isn't a major focus for most IT organizations. The common perception is that as long as you're meeting the bare minimum demanded by your users, you're okay - anything beyond that is a luxury you can’t afford. Well, I happen to think this perception is dead wrong: these days, you just can’t afford not to invest in high performance. The reason is simple: high performance == higher utilization.

I’ll explain what I mean. If you do something that makes an application run 10 times faster (this is a typical performance boost experienced by GigaSpaces users - and by the way, XAP 7.0 will be even faster), without changing your loads or service levels, then that application will consume 90% less resources. Or in other words, you can consolidate the servers running this application at a ratio of 10:1. The amazing thing is, this isn’t instead of the server consolidation you'll get from vendors like VMware - it comes on top of and in addition to it, because it helps you cram more virtual machines and more applications onto every piece of physical hardware.

A great example of this is an eBay subsidiary, Marktplaats, which has moved its application to XAP and is now expecting to reduce their data center from a few hundred servers to only a handful - the consolidation ratio is a whopping 18:1. Marktplaats says this reduction is largely a result of the huge performance boost they experienced, which was made possible by XAPs In-Memory Data Grid and parallel processing capabilities.

XAP also makes it possible for extreme performance to thrive in unexpected places - one example is an XTP trading platform which, thanks to GigaSpaces XAP, has become SaaS-enabled, a major differentiator for the platforms makers, Orbyte Solutions. Another is our recently-announced joint solution with Mule, the open source ESB, which proves that "high performance SOA" is not an oxymoron :)

Interesting talks, and free drinks in JavaOne

2009-05-25T20:46:55+02:00

Its been two years since I've last visited the JavaOne conference. This year is going to be particularly interesting as its going to be the first major Java event following the Oracle acquisition of Sun.

I will have a Technical Session on Tuesday titled: Alternative to Google Application Engine for Java™ Technology-Based Applications, where I'm going to outline the difference between the Google and Amazon approach for cloud computing and discuss how we can combine the best of the two approaches. Argyn Kuketayev and Francis de la Cruz from Primatics is going to join me and present their experience in deploying a risk management application as a service and provide some of the technical details on how they where able to scale-out their application on the cloud.

Daniel Templeton from Sun Microsystems will have a lab session: PetClinic in the Clouds: Scaling a Classic Enterprise Application In this Hands-on Lab, participants will take a popular Web application (the Spring PetClinic sample application) and modify it so that it can be deployed on the Amazon EC2 cloud computing infrastructure. They will be exposed to using the GigaSpaces platform as a service, in-memory data grid concepts, the OpenSpaces framework, cloud computing concepts, and persistence as a service using Sun's MySQL™ database technology.

We are also co-hosting an event Tuesday June 2 at 8PM with our partner Webtide with whom we've done a great integration for Jetty.

Among those who will attend the party will give a chance to win a free book Savvy Guide for cloud computing by Jim Liddle.

*Note: Space is going to be limited so if you want to ensure your place make sure to register on the online registration site that we set for this event.

The list below include all the sessions and labs that I hope to see – any recommendations on other interesting talks would be appreciated.

Session ID	Session Title	Session Type	Speakers and Company	Date/Time	Venue Room
TS-4605	Enterprise JavaBeans™ 3.1 (EJB™ 3.1) Technology Overview	Technical Session	Kenneth Saks, Sun Microsystems, Inc.; Marina Vatkina, Sun Microsystems, Inc.	Tuesday June 02 10:50 AM - 11:50 AM	Hall E 134
TS-4308	Architecting Robust Applications for Amazon EC2 Chris Richardson,	Technical Session	Chris Richardson Consulting	Tuesday June 02 12:10 PM - 1:10 PM	Esplanade 307-310
TS-4390	Castle in the Clouds: SaaS Enabling JavaServer™ Faces Applications	Technical Session	Lucas Jellema, AMIS	Tuesday June 02 12:10 PM - 1:10 PM	Esplanade 302
TS-3817	Google App Engine: Java™Technology in the Cloud	Technical Session	Toby Reyelts, Google; Max Ross, Google; Don Schwarz, Google	Tuesday June 02 3:20 PM - 4:20 PM	Hall E 135
TS-5454	Alternative to Google Application Engine for Java™ Technology-Based Applications	Technical Session	Nati Shalom, GigaSpaces Argyn Kuketayev Francis de la Cruz Primatics	Tuesday June 02 4:40 PM - 5:40 PM	Esplanade 302
LAB-5564BYOL	PetClinic in the Clouds: Scaling a Classic Enterprise Application	Hands On Lab	Michal Bachorik, Sun Microsystems, Inc.; Shay Hassidim, GigaSpaces; Daniel Templeton, Sun Microsystems, Inc. Wednesday June 03	Wednesday 1:35 PM - 3:15 PM	Hall E 132
TS-5214	Java™ Persistence API 2.0: What's New ?	Technical Session	Linda DeMichiel, Sun Microsystems, Inc.; Anil Gaur, Sun Microsystems, Inc.	Wednesday June 03 2:50 PM - 3:50 PM	Hall E 134
BOF-1304	Meet The App Engine (Java™) Team	BOF	Kevin Gibbs, Google; Toby Reyelts, Google; Max Ross, Google; Don Schwarz, Google	Wednesday June 03 7:45 PM - 8:35 PM	Hall E 135
PAN-5366	Cloud Computing: Show Me the Money	Panel Session	Jeff Barr, Amazon.com; Jeff Collins, Intuit; Chris Fry, Salesforce; Simon Guest, Microsoft; Gregor Hohpe, Google, Inc.; Raghavan Srinivas, Self; Lew Tucker, Sun Microsystems, Inc.	Thursday June 04 9:30 AM - 10:30 AM	Gateway 102-103
BOF-5392	Grails Integration Strategies	BOF	Dave Klein, Contegix	Thursday June 04 6:30 PM - 7:20 PM	Esplanade 307-310
TS-5307	Building Next-Generation Web Applications with the Spring 3.0 Web Stack	Technical Session	Keith Donald, SpringSource; Jeremy Grelle, SpringSource	Friday June 05 12:10 PM - 1:10 PM	Esplanade 307-310
LAB-5960	Storing Data in the Cloud	Hands On Lab	Craig Hubbard, Sun Microsystems, Inc.; Chris Kutler, Sun Microsystems, Inc.; Craig McClanahan, Sun Microsystems, Inc.	Thursday June 04 9:30 AM - 11:10 AM	Hall E 130-131

Tip* - If you want to find your own sessions I would highly recommend using the JavaOne search tool.

Seeyu next week!

GigaSpaces based solution makes it to the finalist of Cisco Developer Contest

2009-05-19T11:35:27+02:00

I was very pleased to read an email from Leonardo, who was the winner of the OpenSpaces Developer Challenge (a worldwide programming contest using the Gigaspaces application server which was held last year), saying that he is now a finalist in the Cisco developer contest. Here's a bit about him and the application he submitted:

About Leonardo

Leonardo worked for several ISPs in various roles as network administrator and java programmer for IT consulting firms, and finally as software architect in high-performance Java EE based projects. He is passionate about parallel programming, distributed computing and more recently semantic web and its applications on software engineering.

Leonardo was the winner of the OpenSpaces Developer Challenge. He enjoys reading about various technologies in the field of computer science. When he is not developing code, he prefers to spend time with family and friends, walk in the park, or watch a movie.

About the application

Resource Management Platform is a proposal to develop an event based platform that leverages AXP, Services Gateway Initiative (OSGI), Jini and JavaSpaces technologies to enable deployment of IP Multimedia Subsystem (IMS) applications based on Session Initiation Protocol (SIP); more specifically, the Call Section Control Function (CSCF) components. It will have admission control mechanisms to manage Call processing.

This solution improves infrastructure manageability for large scale IMS applications. Such a platform will potentially be useful to enable deployment of high-performance, network-based SaaS (Software as a Service) or Cloud Computing solutions at the network edge by leveraging AXP.

You can find the full details about his project here.

Leonardo's project is interesting, because it shows how you can use Space Based Architecture (SBA) for implementing a scalable Telco application and offer it as SaaS application on the cloud.

Interestingly enough, I got another email the week before from Amin Abbaspour, who presented another case study illustrating how you can build a scalable SMS service using SBA, as shown in this diagram:

What the two projects have in common, from an architecture perspective, is that they both represent a highly scalable Event Driven design. The unique thing about Event Driven applications is that they require a combination of messaging, data and service interaction that needs to be tightly orchestrated to meet high performance/low-latency requirements without compromising on consistency, ordering (FIFO) and reliability. This combination of requirements represent one of the hardest challenges in building scalable architectures. Trying to meet this type of challenge in the traditional way by integrating messaging system for event delivery , database or simple caching (like Memcached or TC) for data and a traditional application server for business logic is going to lead to fairly complex architecture. Trying to reach linear scalability and keeping the latency low with so many moving parts is close to impossible. This is what makes SBA such a good fit. The main difference about SBA is that it recognizes there is strong dependency between messaging, data and business logic. The key is to have one shared clustering, high availability and scalability for all three components of the architecture. This makes it possible to reduce the number of moving parts and network hops associated with each business transaction, thereby increasing reliability.

On a personal level, I was very pleased to see that the software we are developing is helping people like Leonardo and Amin to build their own carrier and put themselves in a unique spot in highly competitive market.

Good luck Leonardo and Amin!

References

Practical Guide to Developing Enterprise Applications on the Cloud- Online Presentation

2009-05-18T16:57:48+02:00

According to a recent survey, available skill sets is one of the leading decision factors for organizations in determining which application platform to use, while scalability and availability are next. This reveals one of the main obstacles for bringing enterprise applications to the cloud: How do you take something as disruptive as cloud computing, and bring it to an enterprise environment, without forcing a complete re-write?

In my recent talk at the CloudSlam (online) Conference, I tried to summarize the challenges people face when trying to deploy enterprise applications on the cloud. Try to imagine a scenario where you have an existing JEE system, and your system is under load. At this point you would like to add more machines to accommodate that load. What will happen to your application in this case? Will your application be able to take advantage of the additional capacity? How do you know how many machines should be added to meet the load in the first place?

The answers I usually hear to these questions are fairly consistent. If you’re lucky, you’ll only require configuration changes to be at a point where you can utilize the extra resources. Knowing how many machines to add in order to meet a certain load is yet another challenge. Going through the normal capacity planning process could take weeks with the way systems are currently running.

At the same time, if you’re going to deploy our application in a static manner (reserved instances, static IP, …), it would probably make it simpler to deploy your existing application on the cloud, but it probably won’t make much economical sense. So the question I was trying to answer is how we get from the static application deployment most of us are using today, to a point where we can get end-to-end scaling and elasticity of our application from the load-balancer to the database without going through a complete re-write.

In this presentation, I tried to suggest a set of solutions to overcome the various challenges I mentioned earlier and how to apply them in a gradual manner to avoid huge initial investments and high risk. There were also some interesting questions toward the end, centered specifically around one of the hardest problems – getting the data pieces sorted out in a such a dynamic environment. Those who missed this presentation can now view it online:

Note: The voice quality isn’t that great, plus the desktop sharing on Webex didn’t give a clear indication when my desktop was actually shared, so the beginning of the talk is missing the first two or three slides. If you have questions or want to get a copy of the presentation, just shoot me an email.

During the upcoming JavaOne conference there will a hands-on lab session by Daniel Templeton from Sun Microsystems (PetClinic in the Clouds: Scaling a Classic Enterprise Application) where users can go through the steps and deploy a Pet Clinic application on the cloud.

In addition to that lab, I'm going to have a Technical Session (Alternative to Google Application Engine for Java™ Technology-Based Applications).

For those that are not going to join the JavaOne event and want to get some hands-on experience on how the steps that i outlined in the presentation can be implemented in real application, I’d recommend looking Pet Clinic demo that is available on Amazon cloud using our cloud framework.

I’m happy to say that the number of customers that are already using this approach through our Enterprise Cloud middleware platform is growing quickly and were seeing more and more application that would traditionally considered as not "cloud compatible", being deployed on the cloud. Below are few of the public references that were mentioned just recently on Jim Liddle's blog:

Twitter as a scalability case study its the architecture, stupid!

2009-05-02T00:13:12+02:00

In May of last year I wrote my first write-up on Twitter scalability, titled Twitter as a scalability case study. Back then, Twitter was still struggling with its Ruby implementation and claimed to be 10000 Percent Faster. During the past few weeks I came across the an interview entitled Twitter jilts Ruby for Scala which described some of the new development that led Twitter to switch some of its Ruby backend development to Scala. I had follow-on discussions with few colleagues ever since WRT to Scala and functional languages in general which led me to write this piece.

I must admit that whenever a scalability discussion becomes a language-choice discussion, I get irritated. It's enough to read a comment such as the one below to see that there is something is missing from the puzzle:

"Today, Payne said, most of the hip dev set codes in Ruby or PHP or Python because they're perceived as "agile" languages.. but also because the deverati grew bored with languages like Java and C++"

After I read the following note, things became a bit clearer:

"By mid-2008, one of these Ruby message queues completely crashed and developers needed two and a half hours to shove the dropped Tweets back through the system. When your game is micro-blogging, that's a lifetime. Then, in his spare time, one developer ported the code to Scala. According to Payne, the Scala queue could process the same message backlog in 20 seconds."

Later, I read Bill Venner’s excellent interview, Twitter on Scala, which i found quite insightful. But after reading the following response on the use of the Actor model in Twitter I felt even more confused:

"..we found that actors weren’t necessarily the ideal concurrency model for all parts of that system. Some parts of the concurrency model of that system are still actor based. For example, it uses a memcache library that Robey wrote, which is actor based. But for other parts we’ve just gone back to a traditional Java threading model. The engineer working on that, John Kalucki, just found it was a little bit easier to test, a bit more predictable. The nice thing was, it took minutes to switch code that was actor based over to something thread based."

Now to be clear, I have nothing against Scala or Ruby or any of the dynamic languages – quite the contrary. I believe that productivity, being agile, and enjoying writing the code are important factors when we make our language choice. At the same time we should remember that these are only some of the factors; other factors which should influence our decision are available tooling, existing skill-set, maturity, performance, etc. I also don’t care that much how Twitter makes their technology choices. What gets me worried is comments such as this one:

“the switch should stand as a lesson to cutting-edge coders everywhere”

When this type of statements gets to the press and gets backed with intelligent stories that justifies these choices, it is clear to me that many "naive" individuals will start following the same choices without understanding the full picture and history that led to the decision, thinking that they can base their decision on “proven success”. Looking at the history could evil a fairly different picture.

What I found missing in the entire discussion both here in Twitter case and in many of the other web2.0 scalabilty architecture stories is something that will indicate that there is some consistent methodology behind the architectural choices. For example, most of the limitations that were mentioned about Ruby’s threading model, as well as its memory utilization issues, were well known before Twitter chose Ruby as their core language. Why did it take such a long time to realize that you can’t come up with scalable architecture without addressing those limitations? I could argue the exact same thing about Digg and others.

What i would recommend is to look at Brian Zimmer’s summary on Scalability Worst Practices, in particular the “Golden Hammer” point:

"The Golden Hammer refers to the old adage: if all you have is a hammer, everything looks like a nail. Many developers fall prey to the idea of using only one technology – the cost of this is having to build and maintain an infrastructure in the chosen technology which may be readily available in another technology which is more suited to the specific problem area's functionality or abstractions. Forcing a particular technology to work in ways it was not intended is sometimes counter-productive."

The lesson in our specific Twitter case is that language choice should be used as a means for implementing a given architecture and not the other way around. We may find that implementing various parts of our architecture require different technologies and languages, and that’s perfectly fine, as long it follows that order. In Twitter’s case, choosing Ruby for the front-end and Scala and Java for the heavy stuff sounds reasonable, only that I would expect them to get to this realization much sooner.

Final words

In this blog I pointed my comments specifically at Twitter, however Twitter is only an example. Many Web 2.0 sites developed their scalability architecture in a similar trial-and-error approach, which, as we can learn by their histories, tends to be very painful and costly. And yet, building a scalable Web 2.0 application such as Twitter shouldn’t be rocket science. I’m certain that if you follow the right design principles and learn from other proven scalability patterns, you can avoid a large part of the painful “trial and error” experience before coming up with the right solution. Making the right build vs buy decisions can make a lot of difference. In one of my recent posts, Designing a Scalable Twitter, I tried to suggest a methodology for coming up with scalable Twitter architecture. Like anything in life, I’m sure that there is more then one possible solution and more then one language that could be used to implement our solution. In many cases you will find that even though language and platforms could vary between different implementations, the architecture principles remain pretty much the same.

To sum that up, I would say that scalability is first and foremost an architecture choice; different languages can make the implementation of a certain architecture simpler, but language by itself doesn’t make any application more scalable. Different parts of an application may require different set of languages, and choosing the right langrage should also take into consideration existing skillset, maturity, existing development tools and best practices – and obviously, the application’s performance and scalability characteristics.

Designing a Scalable Twitter

2009-04-14T21:45:20+02:00

Guy Nirpaz, Uri Cohen and Shay Banon came up with an interesting exercise as part of the recent partner training that took place at the GigaSpaces office. In this exercise, the students were asked to come up with a scalable design for Twitter, using Space-Based Architecture.

There are some interesting scalability lessons from this exercise, which are applicable to anyone looking to implement new-style real-time web applications such as the ones used for social networking.

In this post I'll try to summarize the main patterns to put into place and considerations to make when designing such a scalable architecture.

Background:

For those of you who are not yet familiar with the service, Twitter is sort of a SMS-service meets discussion board. You can post short messages (up to 140 characters) that can be shared with a group of subscribers that are referred to as "followers". The main difference between twitter and other messaging applications is that both SMS and Instant Messaging (IM) applications were designed primarily for one-on-one communications whereis Twitter was designed primarily for broadcast communications (publish/subscribe, or pub/sub). Another aspect that is special about Twitter is that by default anyone can follow anyone else. In other words, it was designed for open communications, not private, as were IM and SMS.

What are Twitter's scalability challenges?

1. Sending a tweet (a message on Twitter is known as a 'tweet') -– The challenge is how to handle an ever-growing volume of tweets and re-tweets and responses that can lead to a viral "message storm"

2. Reading tweets – The challenge is how to handle a large number of concurrent users that continually “listen” for tweets from users (or topics) they follow.

Designing A Scalable Twitter

Choosing the right scalability patterns

Almost every challenge in software architecture has its roots in one of the existing patterns. So the simplest course is to start by looking for those patterns, and choosing the right patterns to scale the application. Looking at many other scalable architectures, we'll begin with a partitioning pattern as the core design principle. By partitioning our Twitter-like application we'll spread the load across a cluster of servers and scale by simply adding more servers (i.e., partitions). Another important architectural observation about Twitter is that it doesn’t fit into the classic database-centric design that most web applications do. On the flip side, it doesn’t fit well with a messaging-centric design (pub/sub) either. It is a combination of the two.

A pattern that is suitable for this type of collaborative messaging is known as a blackboard pattern. In our design, we will use those two design patterns -- partitioning and blackboard -- as the foundation for our scalable Twitter application. With the foundation in place, let’s list the requirements and examine how these patterns can be used to scale the app.

Scalability Requirements

We'll assume a relatively extreme scaling requirement:

Tweet Volume: 10 billion tweets per day
Tweet Storage: 100 Gigabytes per day (with 10:1 compression)

Additional assumptions:

Tweets are limited to 140 characters
Tweets are immutable, i.e., there are no updates, only inserts
Twitter limits client applications to 70 requests per hour

Now that we have the foundational patterns and clear requirements, we can design the architecture. We'll start first with the blackboard system.

Using an In-Memory Data Grid (IMDG) as a Blackboard System

The are several approaches to building a blackboard system. To maximize performance and scalability, we'll store the data in memory, thus avoiding disk I/O, which is often the main cause for contention. For years, Java has provided a model for designing blackboard systems known as JavaSpaces. More recently, distributed caching has become popular and can provide similar capabilities to those of JavaSpaces. Let's examine two popular distributed caching approaches for our blackboard system:

Simple read-mostly caching using memcached
Read/write caching, also known as an In-Memory Data Grid (IMDG)

Choosing between memcached and an IMDG

Memcached enables us to to store the data (tweets) in a distributed memory set and read it in a scalable fashion. Having said that, be aware that memcached is not transactionally-safe and is not designed for reliability (i.e., it doesn’t support fail-over and high availability). That means that if we use memcached or something similar, we will have to use a database as the back-end. Every tweet posted will have to be written to both memcached and the database in a synchronous fashion to ensure that no tweet will be lost. This approach may be good enough for scaling read access, however, for writes and updates it offers limited scalability.

Unlike memcached, which was designed for simple read-mostly caching, In-Memory Data Grids are designed for handling a read/write scenario, and can therefore act as the system-of-record for both write and read operations. We can still use a database for long-term persistence, but because the IMDG maintains its reliability purely in memory, we can write and update the database asynchronously and avoid hitting the database bottleneck.

Todd Hoff, author of highscalability.com wrote an interesting summary that covers the different products in this space in a recent post: Are Cloud Based Memory Architectures the Next Big Thing?

Todd provide a clear explanation of how an IMDG works (using GigaSpaces):

A POJO (Plain Old Java Object) is written through a proxy using a hash-based data routing mechanism to be stored in a partition on a Processing Unit. Attributes of the object are used as a key. This is straightforward hash based partitioning like you would use with memcached.

You are operating through GigaSpace's framework/container so they can automatically handle things like messaging, sending change events, replication, failover, master-worker pattern, map-reduce, transactions, parallel processing, parallel query processing, and write-behind to databases.

Scaling is accomplished by dividing your objects into more partitions and assigning the partitions to Processing Unit instances which run on nodes-- a scale-out strategy. Objects are kept in RAM and the objects contain both state and behavior. A Service Grid component supports the dynamic creation and termination of Processing Units.

Back to our Twitter app: Given the scalability requirements, we will need to scale both reads and writes, and therefore, an IMDG is a more suitable approach to implementing the blackboard system.

Now let’s examine how the use of an IMDG as the blackboard system enables us to scale both sending and reading tweets. Let's start by designing the partitioned cluster.

Designing a partition architecture

One of the main considerations in designing a partition cluster of any kind is determining the partition key, such as a Customer ID in a CRM application or a Trade ID in a trading application. At first glance, it sounds like a trivial decision, but choosing the right partitioning key requires a deep understanding of the application usage patterns and data model. In the case of Twitter, we could choose to partition the application by the data-type, the user, the tweet itself or the followers. Our first goal is selecting a key that will that will be granular enough to enable scaling the application just by adding more partitions, while making sure that we don't end up with a key that is too fine-grained -- making it sub-optimal for querying purposes.

If we use the timestamp key, for example, our application will be optimized for “inserts” (writes), however, even a simple query such “retrieve the tweets of a certain user” will force us to execute an aggregated query against all partitions. Alternatively, if we partition the data based on user-id, we'll be able to easily spread the load from different users across partitions. Retrieving the tweets of a certain user is going to be resolved in one call to a single partition. We may encounter a problem if a single user generates a significant higher load than average, however, in the case of Twitter, we can assume that this is not very likely. Partitioning by user-id is a good compromise.

Data capacity analysis

With such extreme requirements it is clear that storing all tweets in memory is going to require huge memory capacity. Very quickly this will become economically prohibitive, so we need to devise a scheme in which the IMDG acts as a buffer for most of the load on the system, and then offloads the data and queries to an underlying persistent storage. In our Twitter example, it is fair to assume that most real-time queries (those that require fast access to the data) will be resolved in data from the last hour or 24 hours. Queries that require older data will need to hit the database for the initial call. However, subsequent access to fetch new updates should be resolved purely in-memory.

Using this approach, we'll need about 10 servers, each holding 10GB of data in memory to accommodate 24 hours of activity. If we also want to back up the data in memory, we will need double the amount of servers.

Choosing the right eviction policy

It's reasonable to assume that recent data is accessed most and older data is rarely used. To ensure that we get the maximum hit ratio on our memory front-end, let's choose a time-based eviction policy, which always holds the most recent updates in memory. When we will reach our memory capacity limit the oldest data will automatically get evicted from memory. The actual window of time in which we will be able to keep in memory is obviously dependent on the size of the cluster. With an IMDG implementation all tweets are stored in a persistent storage, which means that when tweets are evicted they are not deleted from the system.

Scaling tweet writes:

If we select user-id as the partitioning key, each user tweet will be sent to a specific partition. Multiple users may be routed to the same partition. Usually the algorithm to determine which partition fits a certain user is something like:

routing-key.hashCode() % #of partitions

In GigaSpaces, this is done by marking the routing attribute of our tweet class with an @SpaceRouting annotation.

The web front-end application will call space.write( new Tweet(..),..) to send the tweets. This way there is nothing in our web client code that exposes the fact that the underlying implementation interacts with a cluster of partitions (spaces in GigaSpaces). Those details are abstracted within the space proxy. When the write method is called on the space proxy it parses the field that matches @SpaceRouting from our Tweet() object and uses this field value to calculate the partition it belongs to. It then uses that value to route the Tweet(..) object to the appropriate partition.

With this approach, the web application can be written in a very simple way and can interact with the entire cluster as if it was a single server.

The data from the memory partitions gets stored asynchronously into a persistent storage. The persistent storage could be a database, but it could also be other things, such as an index search engine based on Compass/Lucene.

Scaling tweet reads:

To those familiar with messaging system, at first glance Twitter looks like a classic publish subscribe application. A closer look, however, reveals that any attempt to implement Twitter with something like a JMS message queue is going to fail in achieving a scalable system. This is especially true if you consider that the system needs to maintain a durable queue for each user. That could easily lead to a scenario in which each tweet is published to thousands of subscribers and every re-tweet can potentially lead to a "message storm".

As I discuss above, the right way to think about this type of application is as a blackboard pattern, just as a blackboard (or these days, a whiteboard) is used by a group of people (followers, in the case of Twitter) to share information and collaborate. When someone writes something on the board, everyone sees it and can choose to react. Unlike messaging (take email for example), we don’t need to send separate messages to each subscriber. Instead everyone is looking at the same board. Everything is also copied from the board to paper. When the board runs out of space, we erase it. And we can always page through the paper copy to access the board history.

In Twitter, this means that each follower that follows a group of people is basically polling for messages posted by those users from the last time he read them. To make things more tangible we can express this type of query with the following SQL syntax:

SELECT * FROM Post WHERE UserID=<id> AND PostedOn > <from date>.

The <from date> will normally be the last few minutes, if we're constantly looking for new messages.

But there's a caveat. Remember that we partitioned the application by user-id? This means that each user's tweets are stored in a separate partition. How can we read all users' posts? If we poll for each user individually, we will end up with a lot of network calls. The simplest approach would be to execute one call that looks for ALL the users we're following and look for updates (new tweets) from those users. The pattern we'll use to perform such this task is mapreduce. One way to do that with GigaSpaces is through the distributed task API:

The distributed task API is a modern version of the stored procedure. The following snippet shows what such a call would look like:

AsyncFuture<Long> future = gigaSpace.execute(new GetTweetsUpdates());
long result = future.get(); // result will be the number of primary spaces

The GetTweetsUpdates() class contains code that will be injected in each partition and will enable us to look for updates from the users we follow in a single call. Because the call runs in-process, and because the data is stored in-memory, executing such a task is extremely fast compared with the equivalent with database and stored procedure operations. Execution is aggregated to the caller implicitly. The caller can use a reducer to aggregate the results into a single result object.

Scaling the web front-end

Nothing really new here. We'll use a classic web front-end, which is comprised of a load-balancer and a cluster of web servers that act as a front end to our IMDG instances. The web application will use a single cluster-aware IMDG proxy to send new tweet posts. The IMDG proxy will be responsible for mapping the tweet with the actual partition that hosting the tweet. That logic is kept completely out of the application code. This allows us to keep our web tier clean and simple.

Keeping the web layer stateless to avoid session stickiness

One common pattern for keeping the web tier scalable is to use a Shared-Nothing Architecture, which basically means that the web tier will be stateless. This requires keeping the user session state external to the web-tier. As previously demonstrated, the IMDG can be used as high-performance, scalable data store for maintaining shared session state information. This allows us to avoid session stickiness and to scale the web tier without being locked in to a specific server throughout the entire session, in case the server is over-loaded.

For more information on how to scale the web tier, as well as other important capabilities such as self-healing and auto-scaling, see the following tutorial: Scaling Your Web Application.

Making it simple and cost-effective using cloud computing

Twitter is yet another example for a situation in which system load is highly variable and the difference between average load and peak load can be quite significant. In such cases, provisioning our system can be fairly hard and costly. This is where cloud computing and SLA-driven deployments can help us scale on demand and pay only for what we use.

Once we figured out a way to partition the application, it's going to be much simpler to package the application into self-sufficient units (referred to in GigaSpaces as processing-units) and scale the application simply by adding or removing these units on demand. You can learn more about this here

Final words

Scaling a real-time web application such as Twitter or Facebook introduces unique challenges that are are quite different from those of a "classic" database-centric application. The most profound difference is the fact that unlike with traditional sites, Twitter is a heavy read/write application, and not read-mostly. This seemingly minor difference can break most existing models for web application scalability. Using a combination of memcached + MySQL is not going to cut it for this type of application.

The good news is that with the right patterns and set of tools, building a scalable architecture that meets such challenges isn’t that difficult. There are already plenty of success stories that demonstrate that, such as the following example from highscalability.com: Handle 1 Billion Events Per Day Using a Memory Grid

The proposed architecture is by no means perfect and can be further optimized to meet even better performance and latency, but that will come at the cost of simplicity. I believe that the proposed architecture should get you pretty far as-is. Avoid going through more advanced optimizations until the point they are an absolute must.

References

Challanges for Developing Enterprise Application on the Cloud

2009-04-14T21:44:22+02:00

In the past few weeks I found myself involved in various discussions centered around the challenges that enterprises face today when they want to deploy their application on the cloud.

These discussions were very timely as they gave me some interesting ideas for my talk next week Practical Guide for Developing Enterprise Application on the Cloud, taking place on Monday, April 20 at 11:00 am Eastern Daylight Time (GMT -04:00, New York), at the CloudSlam online conference.

I thought that some of those discussions are worth sharing. In this post I'll try to summarize the main highlights from those discussions.

I’ll start by pointing to the following discussion thread on the cloud mailing list: Challenges faced by developers and architects when moving to the cloud:

Kent Langley responded by listing the following challenges in his response:

Dealing with a lack of persistence in some cases Dealing with distributed programming models (prob. one of the most important ones imho) Having to think about the whole stack. Not just the code.

Caching considerations

Messaging

Using Memory Data Grids

Understanding configuration management tools that might be involved Working more closely with the operations group in some case

Kevin Apte added the following comment:

There are many challenges- There is no out-of-the-box infrastructure for
hosting the typical J2EE and SOA Stack in the cloud. There is no Weblogic,
WebSphere, ALBPM, Message Bus like Tibco available in the cloud.

A development team could certainly move all of this into the cloud, but the
configuration, licensing issues etc. are all something the team would have
to solve on its own. This is far too bleeding edge for many people.

Robert Hankel added other challenges

.. The problem of adding additional resources dynamically (e.g. more WebCache instances, or WebLogic servers) requires sophisticated distributed system management infrastructure where the entity being managed is no longer a physical or virtual box, but rather an array of boxes acting collectively as a single system..

Grig Gheorghiu posted an interesting write-up Experiences deploying a large-scale infrastructure in Amazon EC2, where he provide a very insightful summary of his lessons with deploying a large scale application on EC2.

Below are the main takeaways from Grig's summary, which I found relevant for this discussion:

1) Deploy multiple Web servers
2) Deploy multiple load balancers
3) Deploy several database servers.
4) Another way of dealing with databases is to not use them

Challenges summary?

It's easy to see that there is a common theme behind all those comments. Taking existing enterprise applications to the cloud can be very difficult simply because a) most of today's enterprise applications were built using frameworks and technologies not yet supported as first class citizen by cloud providers and b) most of those applications were not designed to take advantage of the cloud's elasticity.

Rather then pointing to my direct response to each of those challenges i thought that it would be better to provide a short summary of the main possible solutions that came through this discussion.

Does it have to be that difficult?

No. Below are two main approaches to those challenges.

- Packaging static images

The simplest approach would obviously be to package your local IT environment into images that could be easily ported to the cloud in the exact same way they run in your local IT environment; right? Well, yes, you can package anything in an image bundle and host your virtual machines in a reserved mode with fixed IP configuration. However, being able to technically do that doesn't mean that it makes sense. I would question what's the difference between this environment and any other hosting environment, and what do you expect to get by moving to such a hosted environment vs. running it in your local IT environment.

If you are going to try deploying your existing IT application on the cloud using static images, then most likely you'll end up "porting" not just the application but also the problems you were facing in your local IT environment; i.e. your application will be over-provisioned based on the peak load and you’ll end up with poorly utilized environment.

- Fully elastic application

The main driver for moving to cloud based environment in the first place was to be able to grow as you need and pay for what you use.

The question is whether you can deploy your application without changes to the application while at the same time leverage the elasticity that cloud brings.

Sounds impossible? Well, a good example that does just that is Storage. With storage you can take your existing application, run it with your local (static) disk, and then plug in a network storage and run the same application on that network device, without changing the application. In that world, instead of taking your existing local disk and virutalize it, you are taking the application and plug it to another device that has visualization built-in.

We can use the same approach as with storage; i.e. move your existing application code and run on top of a different underlying implementation that will enable you to capture the elasticity of the cloud without forcing you to re-write your entire application. If you're running in a JEE environment, it should be fairly easy. If your application has strong "ties" to back-end systems, you can use the hybrid model where your application front-end is running on the cloud, while being connected to the back-end system, using secured communication channel.

Is it really that simple?

Yes. My experience with the integration work that we had done between our Cloud Computing Framework (CCF) and EC2 showed me that getting a production-ready JEE application, including a load-balancer, self healing, auto scaling, security, database and even data grid capabilities plugged in, is actually much simpler than with any other environment I'm aware of. This is due to the built-in automation, predefined images, and the fact that I don't need to download and setup anything to get the entire system up and running. In fact, it's so simple that we decided to built our entire Demo as a Service framework around it; a framework that is used quite successfully and constantly, with customers, prospects, and now with partners.

Are there any production references?

Yes. Read Jim Liddle's blog post and see a good example of an Enterprise JEE deployment that is already running on EC2, in production, on top of our new Cloud Computing Framework. In his post, Jim describes how this Telco operator were able to address some of the common challanges that were mentioned above such as security,flexability, cost, development complexity and lock-in as well as high-avliability and scalability in a relatively simple manner .

What next?

In my discussion in the CloudSlam event I’ll try to provide a more detailed practical guide that demonstrating how you can take a step by step approach for porting existing JEE application to the cloud. I hope to get lots of questions and feedback during the discussion so that I can share those with you in one of my follow-up posts.

Is Cloud Computing a revolution or an evolution?

2009-03-26T03:53:00+01:00

Over the past few months I occasionally got into heated debate with different colleagues regarding whether cloud computing is a revolution or an evolution. Will it change our world, or is it going to be more of the same thing with different flavors?

Well, I must admit that I didn’t made up my mind yet. One of the ways to answer this question is by looking at similar trends in other industries.

I recently came across an interesting post by Clay Shirky, Newspapers and Thinking the Unthinkable, where he discuss the impact of the internet on the newspaper industry and specifically the idea behind paying for news in similar fashion that people pay for music. I brought some of his thoughts below where he describes human behavior when a disruptive idea is brought to a well established industry:

Revolutions create a curious inversion of perception. In ordinary times, people who do no more than describe the world around them are seen as pragmatists, while those who imagine fabulous alternative futures are viewed as radicals. The last couple of decades haven’t been ordinary, however. Inside the papers, the pragmatists were the ones simply looking out the window and noticing that the real world was increasingly resembling the unthinkable scenario. These people were treated as if they were barking mad. Meanwhile the people spinning visions of popular walled gardens and enthusiastic micropayment adoption, visions unsupported by reality, were regarded not as charlatans but saviors. […]

When reality is labeled unthinkable, it creates a kind of sickness in an industry. Leadership becomes faith-based, while employees who have the temerity to suggest that what seems to be happening is in fact happening are herded into Innovation Departments, where they can be ignored en masse. This shunting aside of the realists in favor of the fabulists has different effects on different industries at different times. One of the effects on the newspapers is that many of their most passionate defenders are unable, even now, to plan for a world in which the industry they knew is visibly going away. […]

With the old economics destroyed, organizational forms perfected for industrial production have to be replaced with structures optimized for digital data. […]

That is what real revolutions are like. The old stuff gets broken faster than the new stuff is put in its place. The importance of any given experiment isn’t apparent at the moment it appears; big changes stall, small changes spread. Even the revolutionaries can’t predict what will happen. Agreements on all sides that core institutions must be protected are rendered meaningless by the very people doing the agreeing. (Luther and the Church both insisted, for years, that whatever else happened, no one was talking about a schism.) Ancient social bargains, once disrupted, can neither be mended nor quickly replaced, since any such bargain takes decades to solidify. […]

And so it is today. When someone demands to know how we are going to replace newspapers, they are really demanding to be told that we are not living through a revolution. They are demanding to be told that old systems won’t break before new systems are in place. They are demanding to be told that ancient social bargains aren’t in peril, that core institutions will be spared, that new methods of spreading information will improve previous practice rather than upending it. They are demanding to be lied to.

There are fewer and fewer people who can convincingly tell such a lie.

Now try to replace the term newspaper industry with “IT industry” and see what you get.

What are your thoughts on this matter, does cloud computing represent a revolution or an evolution?

Demo as a Service

2009-03-23T22:56:33+01:00

Many of you out there who tried to run a demo online in front of a live audience are probably very familiar with the feeling when the demo is not working. Bill Gates trying to show Windows Media Center on Conan O’Brian is a good example of that :)

Having gone through this experience so many times, I know that getting a demo running successfully is not a trivial task at all.

There are many challenges involved in getting a demo right:

Getting the demo inline with latest and greatest – every new version can break your demo. Since in most cases the demo is not going through the normal QA and testing procedures, the chance that your demo might break because of new changes get fairly high.
Disseminating the demo among all the peers in the organization is also a challenge – if your demo is a little bit complex and requires installation of a database, a web server, etc., most likely you’ll end up with each sales rep having his own version of the demo, if at all. This is simply due to the fact that it takes time to get all those pieces to work and every update becomes a real project.
Governing the quality of the demo – once the demo is on the hands of those sales rep you have absolutely no control over what they are running. Many will try to add their own tweaks, and without noticing end up with a broken demo or a demo that is not geared to highlight the right set of features.
Demonstrating non-functional behavior such as scalability and availability on your own laptop is not that convincing. For example with GigaSpaces, which is a middleware product, scalability, performance and high availability are our main features. You can imagine how “impressed” I would be if you would showed me this type of feature on a single laptop.
Now imagine what can happen in the ideal scenario, where everything worked out just fine, and the customer asked for a copy of the demo – ouch! This is where you’re going to try to find some really good excuses, because chances are that by giving the customer your demo she is going to be exposed to details you really didn’t want her to be exposed to at this stage.
If you do get the chance to try and install the demo on the customer’s machines you might succumb to the “it’s not working on my machine” syndrome, which is often caused by environmental issues that are not so easy to spot.
Last but not least is the collaboration challenge – in most cases demos are one of those tasks where we don’t have dedicated resources allocated to maintain and develop them. This is a classic scenario where collaboration can help to leverage the power of all the users and enable everyone to contribute to the demo in an effective manner. But this requires an initial effort to set such a collaborative environment, which the majority of us don’t spend when we set a demo. At the same time, there are only small number of users who would be willing to go through this investment themselves and contribute to such an environment.

The solution: “Demo as a Service”

The idea behind Demo as a Service is basically to apply the same principles of SaaS model to our demo.

This means that the demo would be fully accessible through the web (even if it’s a desktop application), there would be no installation required. It will be completely multi-tenant, meaning that each user can run on her own isolated environment even though she would share the same infrastructure with other users.

In this section I can refer to GigaSpaces as an example of a company that faced the obstacles I mentioned above and needed such a demo environment. I’ll share some of the principles behind the approach we eventually took to address these challenges. Assuming that others would be interested in creating a similar demo environment, we designed the system in a generic way that can be easily used by any user, even those that are not using our product, to run their own Demo as a Service without going through the same level of investment.

Here are the Demo as a Service platform design principles:

Using cloud computing – as you can imagine, many of the things that I'll describe wouldn’t be possible if it wasn’t for cloud computing. Cloud computing enabled us to create a full production environment per demo in matter of two-three minutes, through a browser, from anywhere in the world. All you needed is an account set up and Internet access.
No installation – only Internet access is required. I must admit that achieving this goal was a bit tricky at first, seeing as our product is not demonstrable through a web UI, and even if it was, there is not much you can see without running an application on top of it. So we had to create an environment that will enable you to deploy a demo application and watch what happens behind the scenes while the application was running. This included the ability to watch the cluster as it grows or shrink, crash machines and see how the application reacts to the event, etc. To achieve these goals, we set up the environment in such away that users can actually log into the servers running the application. We also needed to show the system view – CPU utilization, memory, etc. – of those specific machines. This meant that we needed to create system monitoring per demo and wire it on the fly so that it will monitor only the machines running that specific demo.
Demo on demand – in our case we needed to support the option of creating a full demo environment that includes a load balancer, web containers, a Data Grid and a database, all in one click. Seeing as this happens while we’re facing users, we couldn’t afford any manual operations, and had to ensure that the entire setup would take less than five minutes. At the same time, when the demo is finished, we wanted to be able to clean the machines completely in one click. This had to be done in a way that wouldn’t conflict with other potential users running other demos at the same time.
Fail proof – getting a reliable Internet connection is not always possible, and quite often the connection gets broken just a few minutes before the demo starts. We wanted to be able to recover easily from this type of failure. For this purpose, the web application we use to launch the demo is kept completely stateless. In addition, we keep the entire application in the cloud server repository, so we don’t need to ship any code from the desktop to the server. This enabled us to run the demo even under a low-bandwidth network. In case of connection failure, the deployment is not affected, and once the connection is re-established we are able to get the current state of the system as if nothing happened.
Demos should be easily shared – one of the problems with demos, or any stand-alone application, is that it’s very hard to share the environment the application is running in, and therefore we found ourselves shipping different versions of the application when we wanted to collaborate or troubleshoot the application. One of the benefits of cloud computing is that I can easily share the resources I'm using over the Internet. This can also be useful if I want to interactively work with another user on the same machine, or even troubleshoot what she is doing. All I need to do is log into the same machine and see what’s going on. There’s no need to ship any code. I can even fix issues immediately as I discover them, without needing to send long emails and package my patch for hours.
Simple upgrade – we wanted to be able to launch a new version of our software, or the demo, very easily. We also wanted to enable users to use their current version and control when they do the switch to the new version. To achieve this, we used a shared repository model. The shared repository always keeps the latest update as well as the pervious version. The application only needs to indicate which version it wants to use, via an XML configuration file, and that’s it. In this case, we avoid the need to disseminate copies of our software with all our people or even force them to change their demos at the same time. All they need to do is to point the demo to the right version on S3, and when the demo runs it picks up that version and uses it.
Governing the quality of the demo – now that the demo is running on a shared environment, we can ensure that everyone is using the same copy of the demo. Since we control not only the code but also the environment the demo is running in, we can eliminate any chances to the environment or human errors, which often happens when you ship a copy of the demo. If there is a fix that needs to be applied, it is applied immediately and everyone gets it.
Support desktop (fat-client) applications – there are cases in which the application needs to run a desktop application for its UI. In our case, part of our management tools needs to be accessed that way. For that purpose, each machine that is running can start the desktop application on the server and choose whether to launch only the application screen or the entire desktop. This is done without installing any software and without any manual setup.
Simulate a real production environment – in order to demonstrate things like failover and scaling in a convincing way, we needed to use real machines and a real data center. This is another area where cloud computing came in pretty handy. We could easily instantiate real machines and allocate specific machines per demo and per user. This is extremely powerful, because it gives users the ability to experience the full product in a matter of minutes, without needing to download it or install it.
Support multiple users at the same time (multi-tenancy) – multi-tenancy is a core principle in SaaS. It basically means that users can run on a shared environment while still being completely independent and isolated from each other. In our specific case, this means that:
- All users share the same demo management portal.
- The same user can run multiple application in complete isolation.
- Different users can run at the same time in complete isolation.
- Each application can be managed independently.

To achieve that goal, we had to design the management console to enable such isolation. In our case, we could use the same Amazon account to serve multiple distinct users, in other words there is no direct mapping between an Amazon user and our system user. The same user can run multiple application or different versions of the same application at the same time, and manage them separately.

Can I try it live now?

Yes. The built-in demos are available on www.gigaspaces.com/mycloud – to run the demos you don’t need to have an Amazon account, you can use a free trial user, which basically means that you’ll be running on our behalf. Note that this will only give you access to the built-in demos not your own application. A description of the available demos and how to run them is provided here.

Can I use it to deploy my own (non-GigaSpaces) demo?

Yes, you can use tihs same environment to deploy Tomcat or any other application fairly easily. Note that this option will require that you log in using your Amazon account.

An example on how to deploy your own Tomcat application is available here.

Note that this mode only provides the ability to launch new custom machines but doesn’t include all the elasticity, self-healing functionality that you would gain by deploying your web application on GigaSpaces XAP.

What’s the difference between a demo environment and a production environment?

One of the interesting thing is that there is not much difference. The difference would most likely be configuration – for example, security setup, or the size of the machines, which might be different in a demo because you would likely use low-class machines. Other than that, it’s pretty much the same environment. The beauty of that, IMO, is that you can move from demo to development to production without changing the environment.

Can I use this environment to run a POC and benchmark?

Yes. Since you get full access to the machines, you can keep them running as long as you wish, customize them and install your own software as well. This is actually the recommended way to run a POC, as it is more cost effective. You don’t spend time allocating the machines, setting up the environment and tuning the system, things that often take weeks. It is also easier to share the environment between two separate parties, e.g. the end-user and the vendor, without going through all the security processes required when you attempt to get access to the customer’s on-premise machines. In this way, you don’t need to ship code around when something is not working properly. I can log into the machine and see the exact behavior you’re seeing, and fix things immediately.

Final words

Demo as a Service is a very powerful model for overcoming many of the challenges involved in running demos and POCs. Cloud computing provides totally new capabilities, which allowed us at GigaSpaces to create a full demo environment for our product, at the click of a button. Having said that, based on our experience, setting up a full demo environment can take a while and would require investment. The mycloud environment was designed to get you the benefit of Demo as a Service while saving you the effort required to implement one. It is still in its early stages but is already being used extensively by our sales team and our partners. I’m eager to hear more feedback from users and suggestions on how to can make it better.

Leadership is not management

2009-03-05T21:44:43+01:00

In one of my recent flights (and i had too many of them …) to Melbourne Australia i pulled a book which caught my attention just few minutes before the plane took off.

Its one of those moments where you see something and it takes you few seconds to decide that “that’s the one”.

I’m referring to Seth Godin book Tribes.

Tribes is one of those inspiring books that makes you think during and long after you read it.

Here are some of Seth wisdom that i picked from the book

Leadership is not management:

Management is about manipulating resources to get a known job done,..leadership is about creating a change that you believe in.

Movement have leaders and movement makes things happen. Leaders have followers Managers have employees. Managers makes widgets. Leaders make change.

…

Managers manage by using authority the factory gives them. You listen to your manager or you loose your Job. A manager can’t make change because that’s not his Job. His Job is to complete tasks assigned to him by someone else in the factory.

Leaders on the other hand don’t care that much for organizational structure or the official blessing of whatever the factory they work for. They use passion and ideas to lead people, as appose to using threats and Burecreacrucy to manage them. Leaders must become aware of how the organization they works, because this awareness allows them to use it. Leaders doesn’t always start at the top but it always manage to effect the folks at the top.

How do you manage leaders?

Given that leaders can appear anywhere in an organization, it seems to me that the job of senior management is to find them and support them. Leaders have tribes of their own, and someone needs to lead those tribes. Which leads to the idea of positive deviance.
As a general rule, managers don't like deviants. By definition, deviance from established standards is a failure for a manager working to deliver on spec. So, most of the time, most managers work hard to stamp out deviance (and the deviants who create it). Managers stamp out deviants. That's what they do.

Leaders understand a different calculus. Leaders understand that change is not only omnipresent, but the key to success.
And it turns out that employees who are committed to change and engaged in making things happen are happier and more productive. Putting these two facts together, it's not hard to come to the conclusion that you desperately need more leaders, more deviants-more agents of change, not fewer

Change

Change isn't made by asking permission. Change is made by asking forgiveness, later.

Food for thoughts…

Its time for auto scaling avoid peak load provisioning for web applications

2009-03-04T13:21:20+01:00

Many web applications, including eBanking, Trading, eCommerce and Online Gaming, face large, fluctuating loads. Some of the load is predictable, for example, if you launch a new campaign you expect to see certain growth around that campaign; some of the load is seasonal, like in Christmas for most eCommerce sites or quarterly-based in other cases. But today, with social networks and the speed at which we can disseminate information around the globe, we’re seeing more and more unpredictable load. This effect was known in the financial industry as the “perfect storm,” which basically meant that a chain of events in the market led to a storm of events that brought many systems down. In social networks, this type of viral behavior is also referred to as the Digg effect. The Digg effect means that a site can generate a single news item that will generate more traffic than was ever anticipated to be handled by the site, in a fraction of a second. The barrier for generating such events has been significantly reduced, due to all the new tools available today such as twitter, Digg and Facebook, a trend is appearing across many industries where over time, online systems need to face ever-growing, unpredictable peak load events.

Peak load behavior in various industries

The effect of peak-load provisioning

Over-provisioning

A traditional way of dealing with loads is to provision the system for peak load. You try to estimate what’s going to be the peak load on the system, then double the number to have enough spare in case you were wrong, and buy the amount of hardware that will be able to handle this most extreme scenario. Interestingly enough, that number determines not just the amount of servers you’ll in the production environment, but also the staging and to a lesser degree the testing and development environments. It also determines the number of software licenses, etc. Now, when this number is low – say 2,3 servers – that’s not ideal but still manageable. But what happens when you need 10 servers or 100 servers? Also, what will the organization’s data center will look like if each application provisions its own spare capacity? Its really simple to imagine, right? On average, the data center will be extremely over-provisioned, which means the system will be extremely under-utilized most of the time, which means a huge waste of money.

Under-provisioning

The other side of being over-provisioned is being under-provisioned. Being under-provisioned means that you have too little resources to handle the traffic that the system is experiencing. Well, that’s just a way to put this in nice words. Remember the perfect storm?

Public failure due to under-provisioning

I've collected only a few examples that show what under provisioning can lead to – failure! The fact that the system is now online means that failure is also online, and quickly becomes a news item, resulting in a PR nightmare for your company.

“Right-sizing” – an oxymoron?

By right-sizing, I mean provisioning the system to fit its needs exactly. There's a small caveat here – how do you know the right size? With the viral behavior of traffic these days, trying to predict the load is almost like trying to predict the weather – you can probably have a good guess for next month, but your guess is going to be less accurate the longer the period you’re trying to estimate. So the question is, can we do right-sizing?

Right-sizing under fluctuating load

A basic assumption is that you can’t do right-sizing for a long period of time – even a year. You need to estimate the load for short window of time, and then be able to up-scale or down-scale the system system easily when you see there is a need for it. I think there are primarily two ways we can achieve this type of scaling – flexible scaling and auto scaling:

Flexible scaling – this means that scaling up or down will be a matter of plugging a machine into the network and wouldn’t require re-architecture, changes to our code or even configuration changes. The scaling operation would require a manual process for adding a machine, but wouldn’t require complex development and tuning and, can be done in matter of minutes once the machine is plugged into the network.
Auto-scaling – means that you can monitor the behavior of the system, and based on these metrics, decide in real time whether it needs to grow or shrink. To enable true auto-scaling, we need to have a pool of available resources that we can pull or release in an on-demand basis. This is basically cloud computing – a cloud (whether a public cloud, like Amazon EC2, or a private one running on-premise) provides the flexibility to add these machines on demand and pay for only for what we consume.

The missing piece – middleware virtualization

If you take pretty much any existing application, add another machine into the network, and watch what happens, you wouldn’t be surprised that nothing much would happen at all.

Non-virtualized middleware

Today’s applications aren’t able to dynamically take advantage of new computing resources that become available. That’s also true for cloud based environments like EC2 – the fact that you can now add machines easily is nice, but it doesn’t mean that the application can do anything with it. What is missing is a layer that helps the application take advantage of these new resources dynamically, as they are being added to the system. This is where middleware virtualization comes to the rescue.

Virtualized middleware

Like in storage virtualization, where we can add physical disks easily without touching the application that is utilizing them, we can do the same for the web, business logic, messaging and data layers of our applications. The idea is very similar to storage: we decouple the application from the physical resources and then use a client-side proxy that maps between the application API calls and the appropriate resource that will be used to serve that call. On the web layer, you can use a load-balancer to get to this level of virtualization; on the messaging, business logic and data layers, you can use In-Memory-Data-Grids for the same purpose.

Where to start?

By now, you’re probably nodding your head with agreement, but at the same time you’re probably scratching your head thinking, “how the hell do I get there without re-writing my entire application?” Indeed, everything I’ve been saying is pure common sense, yet for years the industry has struggled with this challenge and hasn’t found a way to apply it effectively. So what’s different now?

The step-by-step approach

IMO, the key to successfully achieving flexible scaling or auto-scaling is by finding the right steps that will get you there with minimum risks. The first steps need to show value with very little effort. Once you get into a better comfort level, you can think of taking more serious steps.

If you take a simple web application and break it down to its components (web, business-logic, data) you can easily see that scaling the data layer would probably be harder then scaling the web layer. In this post I'll focus on the steps required to scale the web layer.

How to add auto-scaling to your existing web tier

Auto-scaling on the web tier means the following:

Dynamically provisioning the web container
Dynamically discovering the web containers and adding them to the load balancer

In a stateless application, these two should do the trick. However, in a more stateful application, you’ll also need to address the way session data is handled when adding or removing machines. For this, you need to be able to address high availability of session data and and data affinity between the load balancer and the web container.

1. Making the session reliable – to make session information reliable, you can back them in an In-Memory Data Grid, which is kept outside the web container. This way, if the web container changes location due to a scaling event, the session information is kept in that shared memory, and a new server can load it on-demand. To avoid the performance overhead of using remote memory, you can use a local cache that is loaded on-demand. For session information, this is fairly useful, because during a particular session it is very likely that you’ll need to read the same user data over and over again. The combination of local and remote Data Grid allows you to get to the great performance of local memory, while still keeping the memory reliable, and without having to overload each server will all the session data.

2. Maintaining data affinity between the load balancer and the web container: In many cases data affinity is done through “sticky sessions”. This basically means that the load balancer “remembers” the first server the user interacted with, and then during the entire session routes all requests from that particular user to the same server. This method is often based on identifying the server based on its IP – but in a dynamically-scalable environment, this assumption no longer holds. The same user session can be on one particular host at one point in time, and then on a different host in another point in time. To overcome this issue, you need to use a routing scheme that isn’t based on host names, but rather on logical IDs. This means that each web container needs to have a logical ID, and the load-balancer can then route the requests based on this ID. The ID will be consistent regardless of where this web container is running, so this solves the problem.

Further optimization – avoiding session stickiness

Imagine a scenario where a user is now associated with a particular server during a session, and that server becomes overloaded. This is where session stickiness becomes a scalability limitation. To avoid this limitation, you can keep the session data outside the web container on a shared In-Memory Data Grid. This means the same user session can be served at any point in time by any of the web containers – which gives you better scalability. The downside of this approach is performance, because you wouldn’t be exploiting the local memory of each web container. You’d also need to apply more stringent locking on the data, because in this scenario a concurrent update might happen under the same user session.

Try it now on the Amazon cloud

All of the above looks nice on paper – we all know how it works, right? :)

But does it really work? We’ve all gone through the experience of reading something we like, and then going and getting it and being disappointed. And in this case, “trying it out” can be fairly complex, because you need to download the software, read the manual, get several machines deployed and configure the environment, and only then we you really give it a try. If you’re like me, you’ll probably give up right there and go through these steps only if you don’t have a choice. This is another area where cloud-computing can be really handy – getting a production-ready test-drive environment.

We’ve just launched a new service on www.gigaspaces.com/mycloud where we give you access to Amazon EC2, with GigaSpaces XAP running as part of the environment, allowing you to experience how this model works without needing to download any software, setup machines, or configure the clustering and load balancer. You can get a fully production-ready environment with all the capabilities I outlined above above in matter of minutes.

If you want to deploy your existing web application on the cloud, check out the deployment instructions in our Cloud Computing Framework documentation. This process is quite straightforward but nevertheless is still not as easy as it could be. If you want to try out the auto-scaling capabilities really quickly, using a demo application, follow the quick steps below:

1. Visit the mycloud page and get your free access code – this will give you access to the cloud.

2. Select the Trader-Stock-Demo – when you click the demo, the system will start provisioning the web application on the cloud. This will include a machine for managing the cloud, a machine for the load-balancer and machines for the web and Data Grid containers. After approximately 3 minutes, the application will be deployed completely. At this point you should see the “running” link on the load balancer machine. Click on this link to open your web client application.

3. Test auto-scaling – click multiple times on the “running” link to open more clients. This increases the load (requests/sec) on the system. As soon as the requests/sec grow beyond a certain threshold, you’ll see new machines being provisioned. After approximately 2 minutes, the machine will be running and a new web container will be auto-deployed into that machine. This new web container will be linked automatically with the load balancer, and the load balancer in turn will spread the load to this new machine. This will reduce the load on each of the existing servers.

4. Test self-healing – you can now kill one of the machines and see how your web client is behaving. You’ll see that even though the machine was killed, the client was hardly effected and the system scaled itself down automatically.

What’s going on behind the scenes?

All this may seem like magic to you. If you want to access the machines and watch the web containers, the Data Grid instances and the machines, as well as real-time statistics, click the Manage button. This opens up a management console that is running on the cloud, and can be accessed through the web. With this tool you can view all the components of our systems. You can even query the data using an SQL browser and view the data as it enters the system. In addition, you can choose to add more services, and even relocate services from one machine to another just by dragging and dropping them in the UI.

The next steps

As you can see, the combination of middleware-virtualization and cloud computing can bring together the promise of right-sizing through auto-scaling. There are many things that can be done to make this entire process even simpler, not just for the demo application but for your own custom application.

At this point I'd really like to hear your feedback – in particular, I’d like to hear if you see areas that could be improved or areas that are missing in this approach.

To send your feedback you can either post a comment on this post or send an email to pm at gigaspaces dot com.

Additional References

Writing your own real-time search engine and Adwords service

2009-02-23T23:00:51+01:00

There are various ways to implement a search engine. The typical way uses a crawler technique. The crawler continuously scans the information, collects changes and indexes them. The indexing service is a centerpiece in the architecture and enables quick matching of keywords into search results.

In some cases this technique is not applicable. For example, imagine the case of eCommerce applications. When you submit a bid on an auction, you expect to see it immediately popping up in the search results.This calls for a rather different architecture then the one typically used for Internet applications. The key is how fast we can put new data into the index server. Sounds simple, right? Well, most index servers are highly optimized for fast read, however they tend to be quite heavy on write operations.

This is where it makes sense to put the index server in-memory. Having an in-memory index enables both fast writes and fast searches. There is one small caveat, however – memory is limited in capacity and is not considered reliable, i.e. if the memory fails the data stored in that memory is gone. This is where In-Memory Data Grids come to the rescue. An In-Memory Data Grid (IMDG) addresses the capacity and reliability of memory. Capacity is addressed by breaking the data into multiple partitions; reliability is achieved by having at least one copy of each partition available in another memory instance. This is exactly what Shay Banon did in his Compass project, where goes into additional detail about how this model works.

Quoting Banon from his excellent post: Collocated Indexing and Distributed Search with GigaSpaces

"This type of integration takes collocation of indexing and searching to a new level. Indexing and Search operations are performed in a collocated manner in memory making them extremely fast. Scalability is easily handled by adding more partitions, and high availability is provided by adding backups to each partition"

Me and Shay had various discussions about that model. I personally see great potential in this type of offering. With the move of many of the batch analytics application into real-time-analytics, real-time search will become more common for many applications.

Using the search engine as an advertising channel - Adwords

Once you got your own search running, you’ve created a potential commerce area without even noticing. Different suppliers can compete on specific keywords and on the real-estate (location on the page) they'll get on the search result (first, second, up, down..). Unlike TV commercial that sell ads on specific timeslot, Adwords provides an advertising channel per search click! One common way to take advantage of Adwords is to use Google Adsense which enables you to put Google ads in your site and get rewarded for the clicks that your site is generating. Having said that, the limitation of this model is that it relies on on the fact that you’re using Google as your search engine. If you are looking to use real-time search that is tailored to your site, this is not going to work. So if you’ve already gotten to the conclusion that you need your own custom search engine, you might as well implement your own Adwords to drive more money out of it.

Adwords is a very interesting model – it’s basically a sophisticated biding system. The algorithm involves matching between the supplier’s phrase and the phrase in the search query. The matching first looks at those bidders with sufficient budget on their account, it then looks at the track record of each supplier and rates the supplier based on the number of click-throughs that they got on their ads. For those who were selected, there is another algorithm the determines the order and specific location in which the ads appear on the screen. One of the most trivial one is exact match of keywords, i.e. ads with exact match appear higher up in the list.

The challenge with implementing your own Adwords is that this matching process needs to happen in real time during the search. So how can this be done?

If we consider Shay’s comment above, it should be fairly clear. If we already have the index in-memory, and search can be executed within the index server itself, why can't we use the same idea for matching Adwords?

Well, the idea behind Space Based Architecture really came from these type of bidding scenarios. We realized that executing the matching algorithm collocated with the data is going to speed up the matching time significantly and improve response time. Below, I tried to sketch a simple diagram that illustrates how this model would work.

The indexed data will be stored in one set of partitioned cluster and Adwords data will be stored in another partition.

The search services will be running collocated with the index service as Banon suggested. In similar way, the Adwords matching services will be running collocated with the Adwords data.

A search request gets to the search portal. The search portal executes the search query as well as the Adwords matching in parallel.

It uses a map/reduce execution, meaning it is able to aggregate all matching results from each partition.

To enable this type of operation you can use either the Service Virtualization Framework in case you'd like a strongly typed query service, or the executors framework in case you’re looking for ad-hoc tasks execution.

You can use Futures as the return value in order to simplify the process of executing the search and Adwords matching in parallel. Future will enable you to fork those two tasks and then collect the results using that future handle.

If you would like to process all the results as they they arrive, you can use AsyncResultFilter for that purpose (a typical use case will be to print the result on your page while still waiting for the rest of the result to arrive). For more information and options on using parallel execution to speed up the search and matching process, refer to the executors framework.

A real-life scenario

One of the recent real-life scenarios that I came across recently is Rednano. Rednano is a local portal in Singapore that provides localized search capabilities. Interestingly enough, they built their portal on Spring and Hibernate. They also looked for a way to build their new add-on services such as Adwords in a scalable manner, and GigaSpacs was pretty much the closest solution they could get to meet their needs.

According to Patrick Ng, CTO from Rednano, they where able to integrate GigaSpaces in their existing Spring architecture in matter of two days! Patrick provides an interesting insight on their selection process in an online presentation during a Cloud event in Singapore.

Rednano - The Decision For Selecting Partner - Technology

View more presentations from gigaspaces. (tags: rednano technology)

During the writing of these lines, I came across some interesting news: Rednano wins award for breaking new ground. Quoting from the news item:

Local search and directory engine rednano.sg has won a prestigious international award given to companies with effective online search technologies.

The search engine, a collaboration between Singapore Press Holdings and Schibsted ASA, won the Digital Market Award at the Fastforward'09 business and technology conference in Las Vegas earlier this week.

The annual award is given to companies which constantly 'create new and value-added services for its users' in the fast-moving technology world.

Congratulation to Rednano on their success!

Have we learned the lesson from the recent economic meltdown?

2009-02-15T20:03:45+01:00

I got an interesting cartoon from a very close friend, which I felt was worth sharing because it very simply explains what's wrong with the current way of thinking in many areas of the industry. Instead of encouraging criticism and accepting it in a constructive way we often fight back, arguing that "you must be stupid" not to see how great we are. When we finally realize we were wrong, we'll blame you for causing us to lose many jobs, and if that's not enough, when we're finally out of business we'll ask you to subsidize us so we can survive all the mistakes we've made! Sounds familiar?

Obama attacks Wall Street's executive really makes me wonder.. are we really going to learn the lessons from this crisis?

Oh in case you wonder this cartoon was drawn on the 2/16/1992 :)

Simple and cost effective Enterprise-SOA with GigaSpaces and Mule

2009-02-10T07:57:07+01:00

MuleSource is a rather popular open source ESB implementation. What I particularly liked about it when I first saw it is how simple it is to glue different services that are communicating in different protocols in a very simple way even if they don't comply with the same standard API.

Another thing that I like about MuleSource is its light weight and simple architecture. Unlike many other ESB implementations, Mule does not bind itself to the use of web services. In fact you can use Mule for orchestrating the flow between SAP and Siebel in the same way that you can orchestrate two POJO services collocated under the same process. This approach has made the integration with GigaSpaces possible as it has enabled us to introduce many of our innovative ideas and bring in the value of Space Based Architecture into the Enterprise SOA world. It is also a good fit with the type of customers we are engaged with in where latency and performance play an important role. The interesting thing is that thanks to the Mule connector architecture your application code does not need to “know” anything about GigaSpaces. All it "sees" is better performance and reliability. For GigaSpaces users the integration with Mule helps to connect GigaSpaces applications to the external world quite easily and leverage many of the existing sets of connectors available with Mule to plug-in those other protocols without writing additional code. We already have a few customers who chose to use GigaSpaces and Mule together and are quite happy with the results. An example of some of the feedback being received can be seen in the following post GigaSpaces and Mule= Pure Sex.

GigaSpaces and Mule ESB integration overview:

Basically there are a few components to the integration:

GigaSpaces connector – The GigaSpaces connector enables GigaSpaces to plug-in as an high performance transport.
Mule server clustering – This is where GigaSpaces clustering is leveraged to enable in-memory fault tolerant solution for the Mule server itself.

The GigaSpaces Mule Connector:

The diagram below shows how services that are running within Mule can use the GigaSpaces Connector as a transport layer.

As can be seen in this diagram the services are unaware of the GigaSpaces implementation. These details are abstracted by the Mule server. The interaction is done declaratively outside of the services code. When a service receives an event it processes it and the result of that event is sent back to Mule which in return calls the GigaSpaces Connector to store that state. Other service uses the same connector to wait for that state change and once that happen that triggers the appropriate method on that service implementation and so forth.

Mule server clustering

The diagram below illustrates how Mule integrates with GigaSpaces in-memory cluster to ensure high availability and reliability.

In general all the Mule state (SEDA Queues in particular) as well as the application state is stored in the GigaSpaces in-memory cluster. That state is replicated synchronously to a backup instance. When the primary node fails the back--up node takes over automatically and continues to process the events from the exact point where they failed. This enables continuous high availability of the application and seamless fail-over without enforcing the user to take explicit action during that failure event and without the need to handle complex re-tries to ensure the reliability and consistency of the application after a failure event.

Announcement:

Given the positive experience both us (GigaSpaces) and the MuleSource team have had, we decided to formalize this integration. . This Wednesday (Feb 11 @ 9 AM PDT/ 12 PM EDT/ 6 PM CET) we are hosting a joint webinar in which we will unveil the details behind this new development. In this webinar Uri Cohen, GigaSpaces Product Manager, and Ken Yagen, Sr. Director of Engineering at MuleSource, will discuss the details behind this integration and use a live demonstration to illustrate how it works. This will also be a good opportunity to ask the experts more questions about this joint offering. To join this webinar make sure that you register prior to the event here

See you at the webinar!

What is the cloud? An end-user view.

2009-02-03T14:19:11+01:00

There is a continuous debate surrounding THE definition of cloud computing.
Is it just another name for Virtualization? Is it a “re-packaging” of Grid Computing? Or is it just the familiar hosting services we know and love with an additional (and more sophisticated) layer of automation?
To be honest – I think it will be very hard to finalize this debate conclusively by just sticking to high-level terminologies. As technical folks we often tend to dive into technical debates and in the course of doing so, we loose sight of the reasons that brought up this debate in the first place. The discussion about the definition of cloud computing is no exception.

Instead of continuing the endless debate, I thought that taking an end-user view would be much simpler to agree on. Here is a snippet from a response that I gave on one of the recent threads on the cloud mailing list: Is "Internal Cloud" an oxymoron? that tries to address this question:

Based on my experience there are two main driving forces for Cloud-Computing:

1. On demand computing i.e. the ability to get a resource when I need it in matters of minutes.
2. Pay-per use i.e. the ability to pay only for what I use.

The rest is implementation detail. From an end user perspective it doesn't matter that much if I can get the above value from an internal or external cloud.

In my view what we’ll see in the near future is a hybrid-cloud i.e. local-IT + Internet Cloud. A typical scenario would be to run testing on
public cloud and production internally. There are many other scenarios that justify this type of hybrid model.

In the long run, the only real question about internal clouds is whether local-IT would be able to compete with the cost and level of service of the public cloud. In my mind, this is where economies of scale play a critical role (See James Liddle’s recent blog: The Economics of Cloud ). i.e. the bigger the data-center is, the higher the chance that the cost and level of service will be better.
It may very well be that over time, the public clouds will become the only option. Just like electric energy. In fact just like electric energy, there will be those who will “generate” their own power, but will develop it as a hybrid. Sometime they pull resources from the public “utility”, while at other times they use their own “utility”.

This thinking was later enhanced by Miha Ahronovitz in his response on another thread here.

Understanding the cloud-stack by analogy

Another way that I found useful when I need to explain the cloud-stack is by analogy to the existing application stack:

The above diagram is a very simplified view of the world. It explains (without too many words) the analogy between today's world, which consists of dedicated resources per application, to the cloud world, which consists of a virtualized pool of resources. As can be seen in this diagram, Applications morphs into Software-as-a-Service, Middleware morphs into Platform-as-a-Service and Operating-System, Hardware and Network morphs into Infrastructure-as-a-Service.

Saving cost using Application/Middleware virtualization

2009-01-22T10:46:19+01:00

Earlier this week, I gave a joint webinar with James Liddle, where we outlined
practical guidelines for saving costs using middleware and application-level Virtualization:

Saving the cost of peak/static provisioning using on-demand scaling
Saving the downtime cost
Saving costs through outsourcing part of our application and operations to the cloud
Saving costs using application level optimization (doing more with less)
Saving costs using platform consolidation to reduce the number of software components as well as utilize OpenSource and more commodity Software packages

Towards the end (Slide 20), Jim Liddle presented real life case studies from the iPhone launch in the UK and how some of hose principles have been applied to enable a successful launch in the UK.

Additionally, Jim went through some of the motivations and case studies that led different Telco, Online Gaming and Start-up companies to utilize our cloud, offering to gain better cost effectiveness.

For those who did not have the chance to participate in the webinar, we uploaded a recorded version of that presentation for you to view.

Savig cost using application level virtualization

View more presentations or upload your own. (tags: computingvirtualization cloud)

I would like to point out a few specific items from this presentation:

Beyond Server Side Consolidation (Slide 5)

Server-Side-Consolidation (SSC) played an important role in bringing the concept of virtualization to mass adoption. It forced organization to be able to map their application into concrete packages and look at machines as a logical entity rather then just a physical entity. Server-Side-Consolidation also brought a relatively simple model for cost saving: Instead of running applications on dedicated HW you can consolidate them into one machine. By doing so, you can reduce the number of servers and save the hardware and the operational costs associated with that.

Having said that, SSC is only one *very narrow* aspect of virtualization that unfortunately became too coupled with Virtualization.

The next step is obviously to move from SSC to application and middleware level Virtualization. Application Virtualization refers to the opposite scenario. Instead of putting multiple applications on the same hardware, we are taking a given application and spreading it on a pool of machines. This holds significant potential for making applications more efficient. For example, just think of the saving potential gained by moving applications from static peak-load provisioning to on-demand provisioning. In addition to that, we can utilize commodity hardware resources to get the power of high end machines.

Practical steps ( Slide 19)

One of the main concerns that most people have WRT to application level virtualization is the effort associated with applying those principles.

The table below captures the value vs effort that we can receive with each part of our middleware and application level virtualization.

As you can see, there are steps that require non or very little changes to our application:
For example, taking an existing web application that requires 10 machines to meet certain peak load, however on average it needs only 3 machines. Normally we would statically provision 10 machines to meet the peak load which means that on average those machines would be poorly utilized. . Instead, we can provision our web container on-demand and use Pay-Per-Use model to pay only what is consumed. This will enable us to save roughly 7 machines (10 - 3 ) and wouldn't require any changes to our code. (You can learn how you can do that with GigaSpaces here)

If we have a computational business logic or even a rendering application (a good example of that is Slideshare or Yutube). Those type of applications tend to deal with fluctuating loads. So if on average we consume 10 machines and for peak we need up to 90 machines - you will be able to save ~80 machines! by provisioning the computational/rendering machines on-demand. (See here on how to use GigaSpaces for MapReduce computation and here to learn on how to use the Actor model)

On the messaging and data side, we can reduce the amount of machines and the cost of scaling by partioning our messaging or data thus enabling linear scaling of those layers. In addition to that utilizing memory resources instead of files provide a significant boosst in performance. The combination of the two enables us to utilize commodity software and hardware resources instead of high-end resources - for example we don't need to rely on high-end databases, we can simply use MySQL.

Final Words

In this presentation we tried to gather most of the knowledge based on our past experience. Most of these lessons are generic and not necessarily specific to GigaSpaces.

I also tried to provide some practical implementation guidelines (Slide 19):

Avoid radical change, enabling a gradual process
Choose an architecture supporting linear scalability
Minimize vendor lock-in
- Enable application portability and freedom of choice of:
- - Cloud provider, Web container, Programming language, Database
- Minimize API lock in:
- - Use of standards
  - API Abstractions – when standards are not available
Future proof your application
- Don’t make decisions today, but be ready to make one without major effort
- Avoid long-term commitment – choose the right licensing model

Real life case studies:

•

I would like to encourage you again to listen to the case studies (Slide 20) and learn how others apply some of those principles in their local-IT and Cloud.

Additional references:

New quick-start guide shows how you can easily scale your on-demand scaling to your EXISTING web application.
New whitepaper: Survival Guide for IT During the Economic Meltdown

Nati Shalom's Blog

Platform as a Service: The Next Generation Application Server?

Key Characteristics of a Cloud/SaaS Enabled Application Platform

Can you run your existing business applications with GAE or Force.com?

IaaS vs PaaS

PaaS for Enterprise applications – doing it right

Next generation application Server?

What about my existing applications ?

Will I lose control?

Final words

References:

No to SQL? Anti-database movement gains steam My Take

The current "one size fit it all" databases thinking was and is wrong

Not your mom and dad’s database

What about in-memory alternatives?

What about ACID transactions, consistency etc?

The need for real time data processing – a real life example

Final words

References:

Google App Engine plus Amazon AWS: Best of both worlds

Real-life case study: Primatics financial – Risk analysis as a service

Final words

References:

GigaSpaces Launches a New Version of its Cloud Computing Framework

Who is using GigaSpaces CCF and Why?

ISV’s and Service providers using CCF for SaaS enablement:

Enterprise customers using CCF to improve time to market of new applications and for large scale testing

Large scale testing, performance benchmarks and demo as a service

Partner ecosystem -

Looking into the future

The Need for Speed - Saving Costs By Improving Application Efficiency

Interesting talks, and free drinks in JavaOne

GigaSpaces based solution makes it to the finalist of Cisco Developer Contest

References

Practical Guide to Developing Enterprise Applications on the Cloud- Online Presentation

Twitter as a scalability case study its the architecture, stupid!

Final words

Designing a Scalable Twitter

Background:

What are Twitter's scalability challenges?

Designing A Scalable Twitter

Choosing the right scalability patterns

Scalability Requirements

Using an In-Memory Data Grid (IMDG) as a Blackboard System

Choosing between memcached and an IMDG

Designing a partition architecture

Data capacity analysis

Choosing the right eviction policy

Scaling tweet writes:

Scaling tweet reads:

Scaling the web front-end

Keeping the web layer stateless to avoid session stickiness

Making it simple and cost-effective using cloud computing

Final words

References

Twitter getting bigger - Scaling becomes more of an issue

Memory-Based Architectures and the Cloud

Handle 1 Billion Events Per Day Using a Memory Grid

Are Cloud-Based Memory Architectures the Next Big Thing?

Scaling Your Web Application

Persistence on the Cloud – How and Why?

Challanges for Developing Enterprise Application on the Cloud

Challenges summary?

Does it have to be that difficult?

- Packaging static images

- Fully elastic application

Is it really that simple?

Are there any production references?

What next?

Is Cloud Computing a revolution or an evolution?

Demo as a Service

The solution: “Demo as a Service”

Can I try it live now?

Can I use it to deploy my own (non-GigaSpaces) demo?

What’s the difference between a demo environment and a production environment?

Can I use this environment to run a POC and benchmark?

Final words

Leadership is not management

Leadership is not management:

How do you manage leaders?