Architecture + Strategy

Building Highly Scalable Java Applications on Windows Azure (JavaOne 2010)

David C. Chou — Thu, 30 Sep 2010 10:06:15 GMT

JavaOne has always been one of my favorite technology conferences, and this year I had the privilege to present a session there. Given my background in Java, previous employment at Sun Microsystems, and the work I’m currently doing with Windows Azure at Microsoft, it’s only natural to try to piece them together and find more ways to use them. Well, honestly, this also gives me an excuse to attend the conference, plus the co-located Oracle OpenWorld, along with 41,000 other attendees. ;)

Building Highly Scalable Java Applications on Windows Azure - JavaOne S313978

View on Docs.com http://docs.com/8FAZ.

A related article published on InfoQ may also provide some context around this presentation. http://www.infoq.com/news/2010/09/java-on-azure-theory-vs-reality. Plus my earlier post on getting Jetty to work in Azure - http://blogs.msdn.com/b/dachou/archive/2010/03/21/run-java-with-jetty-in-windows-azure.aspx, which goes into a bit more technical detail on how a Java application can be deployed and run in Windows Azure.

Java in Windows Azure

So at the time of this writing, deploying and running Java in Windows Azure is conceptually analogous to launching a JVM and run a Java app from files stored on a USB flash drive (or files extracted from a zip/tar file without any installation procedures). This is primarily because Windows Azure isn’t a simple server/VM hosting environment. The Windows Azure cloud fabric provides a lot of automation and abstraction so that we don’t have to deal with server OS administration and management. For example, developers only have to upload application assets including code, data, content, policies, configuration files and service models, etc.; while the Windows Azure manages the underlying infrastructure:

application containers and services, distributed storage systems
service lifecycle, data replication and synchronization
server operating system, patching, monitoring, management
physical infrastructure, virtualization, networking
security
“fabric controller” (automated, distributed service management system)

The benefit of this cloud fabric environment is that developers don’t have to spend time and effort managing the server infrastructure; they can focus on the application instead. However, the higher abstraction level also means we are interacting with sandboxes and containers, and there are constraints and limitations compared to the on-premise model where the server OS itself (or middleware and app server stack we install separately) is considered the platform. Some of these constraints and limitations include:

dynamic networking – requires interaction with the fabric to figure out the networking environment available to a running application. And as documented, at this moment, the NIO stack in Java is not supported because of its use of loopback addresses
no OS-level access – cannot install software packages
non-persistent local file system – have to persist files elsewhere, including log files and temporary and generated files

These constraints impact Java applications because the JVM is a container itself and needs this higher level of control, whereas .NET apps can leverage the automation enabled in the container. Good news is, the Windows Azure team is working hard to deliver many enhancements to help with these issues, and interestingly, in both directions in terms of adding more higher-level abstractions as well as providing more lower-level control.

Architecting for High Scale

So at some point we will be able to deploy full Java EE application servers and enable clustering and stateful architectures, but for really large scale applications (at the level of Facebook ad Twitter, for example), the current recommendation is to leverage shared-nothing and stateless architectures. This is largely because, in cloud environments like Azure, the vertical scaling ceiling for physical commodity servers is not very high, and adding more nodes to a cluster architecture means we don’t get to leverage the automated management capabilities built into the cloud fabric. Plus the need to design for system failures (service resiliency) as opposed to assuming a fully-redundant hardware infrastructure as we typically do with large on-premise server environments.

(Pictures courtesy of LEGO)

The top-level recommendation for building a large-scale application in commodity server-based clouds is to apply more distributed computing best practices, because we’re operating in an environment with more smaller servers, as opposed to fewer bigger servers. The last part of my JavaOne presentation goes into some of those considerations. Basically - small pieces, loosely coupled. It’s not like the traditional server-side development where we’d try to get everything accomplished within the same process/memory space, per user request. Applications can scale much better if we defer (async) and/or parallelize as much work as possible; very similar to Twitter’s current architecture. So we could end up having many front-end Web roles just receiving HTTP requests, persist some data somewhere, fire off event(s) into the queue, and return a response. Then another layer of Worker roles can pick up the messages from the queue and do the rest of the work in an event-driven manner. This model works great in the cloud because we can scale the front-end Web roles independently of the back-end Worker roles, plus not having to worry about physical capacity.

In this model, applications need to be architected with these fundamental principles:

Small pieces, loosely coupled
Distributed computing best practices
- asynchronous processes (event-driven design)
- parallelization
- idempotent operations (handle duplicity)
- de-normalized, partitioned data (sharding)
- shared nothing architecture
- optimistic concurrency
- fault-tolerance by redundancy and replication
- etc.

Thus traditionally monolithic, sequential, and synchronous processes can be broken down to smaller, independent/autonomous, and loosely coupled components/services. As a result of the smaller footprint of processes and loosely-coupled interactions, the overall architecture will observe better system-level resource utilization (easier to handle more smaller and faster units of work), improved throughput and perceived response time, superior resiliency and fault tolerance; leading to higher scalability and availability.

Lastly, even though this conversation advocates a different way of architecting Java applications to support high scalability and availability, the same fundamental principles apply to .NET applications as well.

Run Java with GlassFish in Windows Azure

David C. Chou — Tue, 18 Jan 2011 07:58:58 GMT

At PDC10 (Microsoft’s Professional Developers Conference 2010), Microsoft has again provided affirmation of support for Java in Windows Azure. “We're making Java a first-class citizen with Windows Azure, the choice of the underlying framework, the choice of the development tool.”, said Bob Muglia (President of Server and Tools at Microsoft), during his keynote presentation (transcript). Then during PDC Vijay Rajagopalan delivered a session (Open in the Cloud: Windows Azure and Java) which provided more details on the state of many deliverables, including:

Vijay also talked about, during his presentation, a successful deployment of Fujitsu’s Interstage application server (a Java EE 6 app server based on GlassFish) in Windows Azure. Plus a whole slew of base platform improvements announced via the Windows Azure team blog, which helped to address many of the limitations we observed last year, such as not being able to use NIO as described in my earlier work with Jetty and Windows Azure.

Lots of great news, and I was finally able to sit down and try some things hands-on, with the latest release of the Windows Azure SDK (version 1.3; November 2010) that included many of the announced improvements.

Java NIO now works!

First off, one major limitation identified previously was that because of the networking sandbox model in Windows Azure (for security reasons) also blocked the loopback adapter which NIO needed. At PDC this was discussed, and the fact that Fujitsu Interstage app server worked (which used GlassFish which used NIO) proved this works. And fortunately, there isn’t anything additional we need to do to “enable” NIO; it just works now. I tried my simple Jetty Azure project by changing it back to using the org.eclipse.jetty.server.nio.SelectChannelConnector, deployed into Windows Azure, and it ran!

Also worth noting was that the startup time in Windows Azure was significantly improved. My Jetty Azure project took just a few minutes to become accessible on the Internet (it had taken more than 20 minutes at one point in time).

Mario Kosmiskas also posted an excellent article which showed Jetty and NIO working in Windows Azure (and many great tips which I leveraged for the work below).

Deploying GlassFish

Since Fujitsu Interstage (based on GlassFish) already works in Azure, GlassFish itself should work as well. So I thought I’d give it a try and learn from the exercise. First I tried to build on the Jetty work, but started running into issues with needing Visual Studio and the Azure tools to copy/move/package large amounts of files and nested folders when everything is placed inside of the project structure – GlassFish itself has 5000+ files and 1100+ folders (the resulting super-long file path names for a few files caused the issue). This became a good reason to try out the approach of loading application assets into role instances from Blob storage service, instead of packaging everything together inside of the same Visual Studio project (as is the best practice for ASP.NET Web Role projects).

This technique was inspired by Steve Marx’s article last year (role instance using Blob service), and realized using the work from Mario Kosmiskas’ article (PowerShell magic), I was able to have an instance of GlassFish Server Open Source Edition 3.1 (build 37; latest at the time of this writing) deployed and running in Windows Azure, in a matter of minutes. Below is a screenshot of the GlassFish Administration Console running on a cloudapp.net subdomain (in Azure).

To do this, basically just follow the detailed steps in Mario Kosmiskas’ article. I will highlight the differences here, plus a few bits that weren’t mentioned in the article. Easiest way is to just reuse his Visual Studio project (MinimalJavaWorkerRole.zip). I started from scratch so that I could use GlassFish for various names.

1. Create a new Cloud project, and add a Worker Role (I named mine GlassFishService). The project will open with a base project structure.

2. Copy and paste in the files (from Mario’s project, under the ‘JettyWorkerRole’ folder, and pay attention to his comments on Visual Studio’s default UTF-8 enocding):

lib\ICSharpCode.SharpZipLib.dll
Launch.ps1
Run.cmd

Paste them into Worker Role. For me, the resulting view in Solution Explorer is below:

3. Open ServiceDefinition.cscfg, and add the Startup and Endpoints information. The resulting file should look something like this:

<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="GlassFishAzure" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
  <WorkerRole name="GlassFishService">
    <Imports>
      <Import moduleName="Diagnostics" />
    </Imports>
    <Startup>
      <Task commandLine="Run.cmd" executionContext="limited" taskType="background" />
    </Startup>
    <Endpoints>
      <InputEndpoint name="Http_Listener_1" protocol="tcp" port="8080" localPort="8080" />
      <InputEndpoint name="Http_Listener_2" protocol="tcp" port="8181" localPort="8181" />
      <InputEndpoint name="Http_Listener_3" protocol="tcp" port="4848" localPort="4848" />
      <InputEndpoint name="JMX_Connector_Port" protocol="tcp" port="8686" localPort="8686" />
      <InputEndpoint name="Remote_Debug_Port" protocol="tcp" port="9009" localPort="9009" />
    </Endpoints>
  </WorkerRole>
</ServiceDefinition>

As you can see, the Startup element instructs Windows Azure to execute Run.cmd as a startup task. And the InputEndpoint elements are used to specify ports that GlassFish server needs to listen to for external connections.

4. Open Launch.ps1 and make a few edits. I kept the existing functions unchanged. The parts that changed are shown below (and highlighted):

$connection_string = 'DefaultEndpointsProtocol=http;AccountName=dachou1;AccountKey=<your acct key>

# JRE
$jre = 'jre-1.6.0_23.zip'
download_from_storage 'java' $jre $connection_string (Get-Location).Path
unzip ((Get-Location).Path + "\" + $jre) (Get-Location).Path

# GlassFish
$glassfish = 'glassfish-3.1-b37.zip'
download_from_storage 'apps' $glassfish $connection_string (Get-Location).Path
unzip ((Get-Location).Path + "\" + $glassfish) (Get-Location).Path

# Launch Java and GlassFish
.\jre\bin\java `-jar .\glassfish3\glassfish\modules\admin-cli.jar start-domain --verbose

Essentially, update the script with:

Account name and access key from your Windows Azure Storage Blob service (where you upload the JRE and GlassFish zip files into)
Actual file names you’re using
The appropriate command that references eventual file locations to launch the application (or call another script)

The script above extracted both zip files into the same location (at the time of this writing, in Windows Azure, it is the local storage at E:\approot). So your commands need to reflect the appropriate file and directory structure based on how you put together the zip files.

5. Upload the zip files into Windows Azure Storage Blob service. I followed the same conventions of placing them into ‘apps’ and ‘java’ containers. For GlassFish, I used the glassfish-3.1-b37.zip downloaded directly from glassfish.java.net. For Java Runtime (JRE) I zipped the ‘jre’ directory under the SDK I installed. To do the upload, many existing tools can help you do this from a user’s perspective. I used Neudesic’s Azure Storage Explorer. As a result, Server Explorer in Visual Studio showed this view in Windows Azure Storage (and the containers would show the uploaded files):

That is it! Now we can upload this project into Windows Azure and have it deployed. Just publish it and let Visual Studio and Windows Azure do the rest:

Once completed, you could go to port 8080 on the Website URL to access GlassFish, which should show something like this:

If you’d like to use the default port 80 for HTTP, just go back to ServiceDefinition.cscfg and update the one line into:

GlassFish itself will still listen on port 8080, but externally the Windows Azure cloud environment will receive user requests on port 80, and route them to the VM GlassFish is running in via port 8080.

Granted, this is a very simplified scenario, and it only demonstrates that the GlassFish server can run in Windows Azure. There is still a lot of work that are needed to enable functionalities Java applications expect from a server, such as systems administration and management components, integration points with other systems, logging, relational database and resource pooling, inter-node communication, etc.

In addition, some environmental differences such as Windows Azure being a stateless environment, non-persistent local file system, etc. also need to be mitigated. These differences make Windows Azure a little different from existing Windows Server environments, thus there are different things we can do with Windows Azure instead of using it simply as an outsourced hosting environment. My earlier post based on the JavaOne presentation goes into a bit more detail around how highly scalable applications can be architected differently in Windows Azure.

Java deployment options for Windows Azure

With Windows Azure SDK 1.3, we now have a few approaches we can pursue to deploy Java applications in Windows Azure. A high-level overview based on my interpretations:

Worker Role using Visual Studio deployment package – This is essentially the approach outlined in my earlier work with Jetty and Windows Azure. With this approach, all of the files and assets (JRE, Jetty distributable, applications, etc.) are included in the Visual Studio project, then uploaded into Windows Azure in a single deployment package. To kick-off the Java process we can write bootstrapping code in the Worker Role’s entry point class on the C# side, launch scripts, or leverage SDK 1.3’s new Startup Tasks feature to launch scripts and/or executables.

Worker Role using Windows Azure Storage Blob service – This is the approach outlined in Mario Kosmiskas’ article. With this approach, all of the files and assets (JRE, Jetty distributable, applications, etc.) are uploaded and managed in the Windows Azure Storage Blob service separately, independent of the Visual Studio project that defines the roles and services configurations. To kick-off the Java process we could again leverage SDK 1.3’s new Startup Tasks feature to launch scripts and/or executables. Technically we can invoke the script from the role entry class in C# too, which is what the initial version of the Tomcat Accelerator does, but Java teams may prefer an existing hook to call the script.

Windows Azure VM Role – The new VM Role feature could be leveraged, so that we can build a Windows Server-based image with all of the application files and assets installed and configured, and upload that image into Windows Azure for deployment. But note that while this may be perceived as the approach that most closely aligns to how Java applications are deployed today (by directly working with the server OS and file system), in Windows Azure this means trading off the benefits of automated OS management and other cloud fabric features.

And perhaps, with the new Remote Desktop feature in Windows Azure, we can probably manually install and configure Java application assets. But doing so sort of treats Windows Azure as a simple hosting facility (which it isn’t) and defeats the purpose of all the fault-tolerance, automated provisioning and management capabilities in Windows Azure. In addition, for larger deployments (many VM’s) it would become increasingly tedious and error-prone if each VM needs to be set up manually.

In my opinion, the Worker Role using Windows Azure Storage Blob service approach is ideally suited for Java applications, for a couple of reasons:

All of the development and application testing work can still be accomplished in the tools you’re already using. Visual Studio and dealing with Windows Azure SDK and tools are only needed from a deployment perspective – deploying the script that launches the Java process
Managing individual zip files (or other archive formats) and at any granularity level, instead of needing to put everything into the same deployment package when using Visual Studio for everything
Loading application assets from the Windows Azure Storage Blob service also provides more flexibility for versioning, reuse (of components), and managing smaller units of changes. We can have a whole set of scripts that load different combinations of assets depending on version and/or intended functionality

Perhaps, at some point we could just upload scripts into Blob service and the only thing is to tell the Windows Azure management portal to run, as the Startup Task, which scripts for which roles and instances from the assets we store in the Blob service. But there are still things we could do with Visual Studio – setting up IntelliTrace, facilitating certificates for Remote Desktop, etc. However, treating the Blob service as a distributed storage system, automating the transfer of specific components and assets from the storage to each role instance, and launching the Java process to run the application, could be a very interesting way for Java teams to run applications in Windows Azure platform.

Lastly, this approach is not limited to Java applications. In fact, as long as any software components that can be loaded from Blob storage, then installed in an automated manner driven by scripts (and we may even be able to use another layer of script interpreters such as Cygwin), they can use this approach for deployment in Windows Azure. By managing libraries, files, and assets in Blob storage, and maintaining a set of scripts, a team can operate and manage multiple (and multiple versions of) applications in Windows Azure, with comparatively higher flexibility and agility than managing individual virtual machine images (especially for smaller units of changes).

Designing for Cloud-Optimized Architecture

David C. Chou — Mon, 24 Jan 2011 02:49:00 GMT

I wanted to take the opportunity and talk about the cloud-optimized architecture, the implementation model instead of the popular perceptions around leveraging cloud computing as a deployment model. This is because, while cloud platforms like Windows Azure can run a variety of workloads, including many legacy/existing on-premises software and application migration scenarios that can run on Windows Server; I think Windows Azure’s platform-as-a-service model offers a few additional distinct technical advantages when we design an architecture that is optimized (or targeted) for the cloud platform.

Cloud platforms differ from hosting providers

First off, the major cloud platforms (regardless how we classify them as IaaS or PaaS) at the time of this writing, impose certain limitations or constraints in the environment, which makes them different from existing on-premises server environments (saving the public/private cloud debate to another time), and different from outsourced hosting managed service providers. Just to cite a few (according to my own understanding, at the time of this writing):

Amazon Web Services
- EC2 instances are inherently stateless; that is, their local storage is non-persistent and non-durable
- Little or no control over infrastructure that are used beneath the EC2 instances (of course, the benefit is we don’t have to be concerned with them)
- Requires systems administrators to configure and maintain OS environments for applications
Google App Engine
- Non-VM/OS instance-aware platform abstraction which further simplifies code deployment and scale, though some technical constraints (or requirements) as well. For example,
- Stateless application model
- Requires data de-normalization (although Hosted SQL will mitigate some concerns in this area)
- If the application can't load into memory within 1 second, it might not load and return 500 error codes (from Carlos Ble)
- No request can take more than 30 seconds to run, otherwise it is stopped
- Read-only file system access
Windows Azure Platform
- Windows Azure instances are also inherently stateless – round-robin load balancer and non-persistent local storage
- Also due to the need to abstract infrastructure complexities, little or no control for the underlying infrastructure is offered to applications
- SQL Azure has individual DB sizing constraints due to its 3-replica synchronization architecture

Again, just based on my understanding, and really not trying to paint a “who’s better or worse” comparative perspective. The point is, these so-called “differences” exist because of many architectural and technical decisions and trade-offs to provide the abstractions from the underlying infrastructure. For example, the list above is representative of most common infrastructure approaches of using homogeneous, commodity hardware, and achieve performance through scale-out of the cloud environment (there’s another camp of vendors that are advocating big-machine and scale-up architectures that are more similar to existing on-premises workloads). Also, the list above may seem unfair to Google App Engine, but on the flip side of those constraints, App Engine is an environment that forces us to adopt distributed computing best practices, develop more efficient applications, have them operate in a highly abstracted cloud and can benefit from automatic scalability, without having to be concerned at all with the underlying infrastructure. Most importantly, the intention is to highlight that there are a few common themes across the list above – stateless application model, abstraction from infrastructure, etc.

Furthermore, if we take a cloud computing perspective, instead of trying to apply the traditional on-premises architecture principles, then these are not really “limitations”, but more like “requirements” for the new cloud computing development paradigm. That is, if we approach cloud computing not from a how to run or deploy a 3rd party/open-source/packaged or custom-written software perspective, but from a how to develop against the cloud platform perspective, then we may find more feasible and effective uses of cloud platforms than traditional software migration scenarios.

Windows Azure as an “application platform”

Fundamentally, this is about looking at Windows Azure as a cloud platform in its entirety; not just a hosting environment for Windows Server workloads (which works too, but the focus of this article is on cloud-optimized architecture side of things). In fact, Windows Azure got its name because it is something a little different than Windows Server (at the time of this writing). And that technically, even though the Windows Azure guest VM OS is still Windows Server 2008 R2 Enterprise today, the application environment isn’t exactly the same as having your own Windows Server instances (even with the new VM Role option). And it is more about leveraging the entire Windows Azure platform, as opposed to building solely on top of the Windows Server platform.

For example, below is my own interpretation of the platform capabilities baked into Windows Azure platform, which includes SQL Azure and Windows Azure AppFabric also as first-class citizens of the Windows Azure platform; not just Windows Azure.

I prefer using this view because I think there is value to looking at Windows Azure platform holistically. And instead of thinking first about its compute (or hosting) capabilities in Windows Azure (where most people tend to focus on), it’s actually more effective/feasible to think first from a data and storage perspective. As ultimately, code and applications mostly follow data and storage.

For one thing, the data and storage features in Windows Azure platform are also a little different from having our own on-premises SQL Server or file storage systems (whether distributed or local to Windows Server file systems). The Windows Azure Storage services (Table, Blob, Queue, Drive, CDN, etc.) are highly distributed applications themselves that provide a near-infinitely-scalable storage that works transparently across an entire data center. Applications just use the storage services, without needing to worry about their technical implementation and up-keeping. For example, for traditional outsourced hosting providers that don’t yet have their own distributed application storage systems, we’d still have to figure out how to implement and deploy a highly scalable and reliable storage system when deploying our software. But of course, the Windows Azure Storage services require us to use new programming interfaces and models (REST-based API’s primarily), and thus the difference with existing on-premises Windows Server environments.

SQL Azure, similarly, is not just a plethora of hosted SQL Server instances dedicated to customers/applications. SQL Azure is actually a multi-tenant environment where each SQL Server instance can be shared among multiple databases/clients, and for reliability and data integrity purposes, each database has 3 replicas on different nodes and has an intricate data replication strategy implemented. The Inside SQL Azure article is a very interesting read for anyone who wants to dig into more details in this area.

Besides, in most cases, a piece of software that runs in the cloud needs to interact with data (SQL or no-SQL) and/or storage in some manner. And because data and storage options in Windows Azure platform are a little different than their seeming counterparts in on-premises architectures, applications often require some changes as well (in addition to the differences in Windows Azure alone). However, if we look at these differences simply as requirements (what we have) in the cloud environment, instead of constraints/limits (what we don’t have) compared to on-premises environments, then it will take us down the path to build cloud-optimized applications, even though it might rule out a few application scenarios as well. And the benefit is that, by leveraging the platform components as they are, we don’t have to invest in the engineering efforts to architect and build and deploy highly reliable and scalable data management and storage systems (e.g., build and maintain your own implementations of Cassandra, MongoDB, CouchDB, MySQL, memcarche, etc.) to support applications; we can just use them as native services in the platform.

The platform approach allows us to focus our efforts on designing and developing the application to meet business requirements and improve user experience, by abstracting away the technical infrastructure for data and storage services (and many other interesting ones in AppFabric such as Service Bus and Access Control), and system-level administration and management requirements. Plus, this approach aligns better with the primary benefits of cloud computing – agility and simplified development (less cost as a result).

Smaller pieces, loosely coupled

Building for the cloud platform means designing for cloud-optimized architectures. And because the cloud platforms are a little different from traditional on-premises server platforms, this results in a new developmental paradigm. I previously touched on this topic with my presentation at JavaOne 2010, then later on at Cloud Computing Expo 2010 Santa Clara; just adding some more thoughts here. To clarify, this approach is more relevant to the current class of “public cloud” platform providers such as ones identified earlier in this article, as they all employ the use of heterogeneous and commodity servers, and with one of the goals being to greatly simplify and automate deployment, scaling, and management tasks.

Fundamentally, cloud-optimized architecture is one that favors smaller and loosely coupled components in a highly distributed systems environment, more than the traditional monolithic, accomplish-more-within-the-same-memory-or-process-or-transaction-space application approach. This is not just because, from a cost perspective, running 1000 hours worth of processing in one VM is relatively the same as running one hour each in 1000 VM’s in cloud platforms (although the cost differential is far greater between 1 server and 1000 servers in an on-premises environment). But also, with a similar cost, that one unit of work can be accomplished in approximately one hour (in parallel), as opposed to ~1000 hours (sequentially). In addition, the resulting “smaller pieces, loosely coupled” architecture can scale more effectively and seamlessly than a traditional scale-up architecture (and usually costs less too). Thus, there are some distinct benefits we can gain, by architecting a solution for the cloud (lots of small units of work running on thousands of servers), as opposed to trying to do the same thing we do in on-premises environments (fewer larger transactions running on a few large servers in HA configurations).

I like using the LEGO analogy below. From this perspective, the “small pieces, loosely coupled” fundamental design principle is sort of like building LEGO sets. To build bigger sets (from a scaling perspective), with LEGO we’d simply use more of the same pieces, as opposed to trying to use bigger pieces. And of course, the same pieces can allow us to scale down the solution as well (and not having to glue LEGO pieces together means they’re loosely coupled).

But this architecture also has some distinct impacts to the way we develop applications. For example, a set of distributed computing best practices emerge:

asynchronous processes (event-driven design)
parallelization
idempotent operations (handle duplicity)
de-normalized, partitioned data (sharding)
shared nothing architecture
fault-tolerance by redundancy and replication
etc.

Asynchronous, event-driven design – This approach advocates off-loading as much work from user requests as possible. For example, many applications just simply incur the work to validate/store the incoming data and record it as an occurrence of an event and return immediately. In essence it’s about divvying up the work that makes up one unit of work in a traditional monolithic architecture, as much as possible, so that each component only accomplishes what is minimally and logically required. Rest of the end-to-end business tasks and processes can then be off-loaded to other threads, which in cloud platforms, can be distributed processes that run on other servers. This results in a more even distribution of load and better utilization of system resources (plus improved perceived performance from a user’s perspective), thus enabling simpler scale-out scenarios as additional processing nodes and instances can be simply added to (or removed from) the overall architecture without any complicated management overhead. This is nothing new, of course; many applications that leverage Web-oriented architectures (WOA), such as Facebook, Twitter, etc., have applied this pattern for a long time in practice. Lastly, of course, this also aligns well to the common stateless “requirement” in the current class of cloud platforms.

Parallelization – Once the architecture is running in smaller and loosely coupled pieces, we can leverage parallelization of processes to further improve the performance and throughput of the resulting system architecture. Again, this wasn’t so prevalent in traditional on-premises environments because creating 1000 additional threads on the same physical server doesn’t get us that much more performance boost when it is already bearing a lot of traffic (even on really big machines). But in cloud platforms, this can mean running the processes in 1000 additional servers, and for some processes this would result in very significant differences. Google’s Web search infrastructure is a great example of this pattern; it is publicized that each search query gets parallelized to the degree of ~500 distributed processes, then the individual results get pieced together by the search rank algorithms and presented to the user. But of course, this also aligns to the de-normalized data “requirement” in the current class of cloud platforms, as well as SQL Azure’s implementation that resulted in some sizing constraints and the consequent best practice of partitioning databases, because parallelized processes can map to database shards and try not to significantly increase the concurrency levels on individual databases, which can still degrade overall performance.

Idempotent operations – Now that we can run in a distributed but stateless environment, we need to make sure that same process that gets routed to multiple servers don’t result in multiple logical transactions or business state changes. There are processes that could and prefer duplicate transactions, such as ad clicks; but there are also processes that don’t want multiple requests be handled as duplicates. But the stateless (and round-robin load-balancing in Windows Azure) nature of cloud platforms requires us to put more thoughts into scenarios such as when a user manages to send multiple submits from a shopping cart, as these requests would get routed to different servers (as opposed to stateful architectures where they’d get routed back to the same server with sticky sessions) and each server wouldn’t know about the existence of the process on the other server(s). There is no easy way around this, as the application ultimately needs to know how to handle conflicts due to concurrency. Most common approach is to implement some sort of transaction ID that uniquely identifies the unit of work (as opposed to simply relying on user context), then choose between last-writer or first-writer wins, or optimistic locking (though any form of locking would start to reduce the effectiveness of the overall architecture).

De-normalized, partitioned data (sharding) – Many people perceive the sizing constraints in SQL Azure (currently at 50GB – also note it’s the DB size and not the actual file size which may contain other related content) as a major limitation in Windows Azure platform. However, if a project’s data can be de-normalized to a certain degree, and partitioned/sharded out, then it may fit well into SQL Azure and benefit from the simplicity, scalability, and reliability of the service. The resulting “smaller” databases actually can promote the use of parallelized processes, perform better (load more distributed than centralized), and improve overall reliability of the architecture (one DB failing is only a part of the overall architecture, for example).

Shared nothing architecture – This means a distributed computing architecture in which each node is independent and self-sufficient, and there is no single point of contention across the system. With data sharding and maintained in many distributed nodes, the application itself can and should be developed using shared-nothing principles. But of course, many applications need access to shared resources. It is then a matter of deciding whether a particular resource needs to be shared for read or write access, and different strategies can be implemented on top of a shared nothing architecture to facilitate them, but mostly as exceptions to the overall architecture.

Fault-tolerance by redundancy and replication – This is also “design for failures” as referred to many cloud computing experts. Because of the use of commodity servers in these cloud platform environments, system failures are a common thing (hardware failures occur almost constantly in massive data centers) and we need to make sure we design the application to withstand system failures. Similar to thoughts around idempotency above, designing for failures basically means allowing requests to be processed again; “try-again” essentially.

Lastly, each of the topic areas above is worthy of an individual article and detailed analysis; and lots of content are available on the Web that provide a lot more insight. The point here is, each of the principles above actually has some relationship with, and dependency on, the others. It is the combination of these principles that contribute to an effective distributed computing, and cloud-optimized architecture.

Azure Capacity Assessment

David C. Chou — Wed, 02 Feb 2011 17:11:38 GMT

Hanu Kommalapati, a very esteemed colleague and co-author on our book SOA with .NET and Windows Azure, provided some guidance on how to estimate and assess capacity requirements in Windows Azure. Read more about it on Hanu’s blog at http://blogs.msdn.com/b/hanuk/archive/2011/02/01/windows-azure-capacity-assessment.aspx.

Applied Architecture Patterns on the Microsoft Platform

David C. Chou — Fri, 04 Feb 2011 02:26:35 GMT

Just saw a friend, Richard Seroter the “Architect Extraordinaire”, publish a new book from Packt – Applied Architecture Patterns on the Microsoft Platform; his second book after SOA Patterns with BizTalk Server 2009. Even though technical books usually don’t make it onto NY Times Best Sellers and that they usually don’t help the authors retire from their full-time jobs, putting one together and making it through the publishing process is still a very significant effort. Thus I just want to say, congrats Richard! :)

The new book covers different architectural perspectives, and provides a nice overview on the latest shipping technologies on the Microsoft enterprise platform including WCF/WF 4.0, Windows Server AppFabric (which is one of the most interesting things on the Microsoft platform today IMO, including the Windows Azure AppFabric), BizTalk Server, SQL Server and StreamInsight, and Windows Azure platform. It then goes into more detail around various types of application and data integration patterns (and at different levels; not just messaging patterns), and discusses the trade-offs and best practices for the multiple solutions that can be used to apply each pattern, and scenarios.

It’s not another one of those “what’s possible with cloud computing” books. The authors took a pragmatic approach to identify and describe today’s real-world architectural issues and patterns, from simple workflows, the requisite pub-sub, content-based routing, message broadcasting, etc., to complex event processing, master data synchronization, handling large data and burst Web traffic; and provided architectural considerations (including on-premises and cloud-based models) and options on how these commonly encountered patterns can be implemented with the components of the Microsoft enterprise platform.

Kudos to the authoring team!

The Windows Azure Handbook, Volume 1: Strategy & Planning

David C. Chou — Sat, 05 Mar 2011 00:23:17 GMT

I was happy to see that a colleague and friend, David Pallman (GM at Neudesic and one of the few Windows Azure MVP’s in the world), successfully published the first volume of his The Windows Azure Handbook series:

Planning & Strategy
Architecture
Development
Management

I think that is a very nice organization/separation of different considerations and intended audiences. And the ‘sharding’ of content into four separate volumes is really not intended as a ploy to sell more with what could be squeezed into one book. :) The first volume itself has 300 pages! And David only spent the minimally required time on the requisite “what is” cloud computing and Windows Azure platform topics (just the two initial chapters, as a matter of fact). He then immediately dived into more detailed discussions around the consumption-based billing model and its impacts, and many aspects that contribute into a pragmatic strategy for cloud computing adoption.

Having worked with Windows Azure platform since its inception, and successfully delivering multiple real-world customer projects, David definitely has a ton of lessons, best practices, and insights to share. Really looking forward to the rest of the series.

Kudos to David!

Unplug from your day job. Be inspired at MIX

David C. Chou — Mon, 07 Mar 2011 19:43:51 GMT

One of my favorite annual conferences is back again this year in Las Vegas. Yes, despite the fabulous location and the plethora of distractions it provides. ;)

Join the conversation at MIX – see the latest tools and technologies and draw inspiration from a professional community of your peers and experts. More About MIX.

PREVIEW THE NEXT WEB
MIX is where you’ll learn about the future of web, from the diversity of devices and interaction models on the front-end, to the tools and technologies that power the user experience, to the services that make it all possible. There’s no better place to hear about the future of Silverlight, Internet Explorer, Windows Phone, ASP.NET, and technologies like HTML5 and CSS3.

GET REAL ANSWERS FROM THE EXPERTS
MIX isn’t just about getting a first look at the latest technologies and trends – it’s an opportunity for you to have your questions answered by industry and Microsoft experts.

NETWORK WITH THE COMMUNITY
MIX is a gathering of developers, designers, UX experts and business professionals creating the most innovative and profitable consumer sites on the web. The opportunities for networking are limitless…this is Las Vegas, after all.

MIX is a unique opportunity to engage with Microsoft and industry professionals in a two-way conversation about the future of web - from the diversity of devices and interaction models on the front-end, to the tools and technologies that power the user experience, to the services that make it all possible. MIX is for professionals who design and build cutting-edge websites.
live.visitmix.com

Cloud-optimized architecture and Advanced Telemetry

David C. Chou — Tue, 15 Mar 2011 02:37:28 GMT

One of the projects I had the privilege of working with this past year, is the Windows Azure platform implementation at Advanced Telemetry. Advanced Telemetry offers an extensible, remote, energy-monitoring-and-control software framework suitable for a number of use case scenarios. One of their current product offerings is EcoView™, a smart energy and resource management system for both residential and small commercial applications. Cloud-based and entirely Web accessible, EcoView enables customers to view, manage, and reduce their resource consumption (and thus utility bills and carbon footprint), all in real-time via the intelligent on-site control panel and remotely via the Internet.

Much more than Internet-enabled thermostats and device end-points, “a tremendous amount of work has gone into the core platform, internally known as the TAF (Telemetry Application Framework) over the past 7 years” (as Tom Naylor, CEO/CTO of Advanced Telemetry wrote on his blog), which makes up the server-side middleware system implementation, and provides the intelligence to the network of control panels (with EcoView being one of the applications), and an interesting potential third-party application model.

The focus of the Windows Azure platform implementation, was moving the previously hosted server-based architecture into the cloud. Advanced Telemetry completed the migration in 2010, and the Telemetry Application Framework is now running in Windows Azure Platform. Tom shared some insight from the experience in his blog post “Launching Into the Cloud”. And of course, this effort was also highlighted as a Microsoft case study on multiple occasions:

Energy Monitoring Firm Saves Money, Scales Business with Hosted Computing Platform
Startup Uses Cloud Computing to Change Business Model, Becomes Instantly Profitable
SQL Azure team blog’s Interview with Tom Naylor, CTO of Advanced Telemetry
Windows Azure AppFabric team blog’s How Advanced Telemetry became instantly profitable after migrating to the Windows Azure Platform
Channel 9 Video - Building on Azure: Advanced Telemetry

The Move to the Cloud

As pointed out by the first case study, the initial motivation to adopt cloud computing was driven by the need to reduce operational costs of maintaining an IT infrastructure, while being able to scale the business forward.

“We see the Windows Azure platform as an alternative to both managing and supporting collocated servers and having support personnel on our side dedicated to making sure the system is always up and the application is always running,” says Tom Naylor. “Windows Azure solves all those things for us effectively with the redundancy and fault tolerance we need. Because cost is based on usage, we’ll also be able to much more accurately assess our service fees. For the first time, we’ll be able to tell exactly how much it costs to service a particular site.”

For instance, in the Channel 9 video, Tom mentioned that replicating the co-located architecture from Rackspace to Windows Azure platform resulted in approximately 75% cost reduction on a monthly basis in addition to other benefits. One of the major ‘other’ benefits is agility, which arguably is much more valuable than the cost reduction normally associated with cloud computing benefits. In fact, as the second case study pointed out, in addition to breaking ties to an IT infrastructure, Windows Azure platform become a change enabler that supported to shift to a completely different business model for Advanced Telemetry (from a direct market approach to that of an original equipment manufacturer (OEM) model). The move to Windows Azure platform provided the much needed scalability (of the technical infrastructure), flexibility (to adapt to additional vertical market scenarios), and manageability (maintaining the level of administrative efforts while growing the business operations). The general benefits cited in the case study were:

Opens New Markets with OEM Business Model
Reduces Operational Costs
Gains New Revenue Stream
Improves Customer Service

Cloud-Optimized Architecture

However, this is not just another simple story of migrating software from one data center to another data center. Tom Naylor understood well the principles of cloud computing, and saw the value in optimizing the implementation for the cloud platform instead of just using it as a hosting environment for the same thing from somewhere else. I discussed this in more detail in a previous post Designing for Cloud-Optimized Architecture. Basically, it is about leveraging cloud computing as a way of computing and as a new development paradigm. Sure, conventional hosting scenarios do work in cloud computing, but there is more value and benefits to gain if an application is designed and optimized specifically to operate in the cloud, and built using unique features from the underlying cloud platform.

In addition to the design principles around “small pieces, loosely coupled” fundamental concept I discussed previously, another aspect of the cloud-optimized approach is to think about storage first, as opposed to thinking about compute. This is because, in cloud platforms like Windows Azure platform, we can build applications using the cloud-based storage services such as Windows Azure Blob Storage and Windows Azure Table Storage, which are horizontally scalable distributed storage systems that can store petabytes and petabytes of data and content without requiring us to implement and manage the infrastructure. This is in fact, one of the significant differences between cloud platforms and traditional outsourced hosting providers.

In the Channel 9 video interview, Tom Naylor said “what really drove us to it, honestly, was storage”. He mentioned that the Telemetry Application Platform currently handles about 200,000 messages per hour, each containing up to 10 individual point updates (which roughly equates to 500 updates per second). While this level of traffic volume isn’t comparable to the top websites in the world, it still poses significant issues for a startup company to store and access the data effectively. In fact, the data required the Advanced Telemetry team to cull the data periodically in order to maintain a relatively workable size for the operational data.

“We simply broke down the functional components, interfaces and services and began replicating them while taking full advantage of the new technologies available in Azure such as table storage, BLOB storage, queues, service bus and worker roles. This turned out to be a very liberating experience and although we had already identified the basic design and architecture as part of the previous migration plan, we ended up making some key changes once unencumbered from the constraints inherent in the transitional strategy. The net result is that in approximately 6 weeks, with only 2 team members dedicated to it (yours truly included), we ended up fully replicating our existing system as a 100% Azure application. We were still able to reuse a large percentage of our existing code base and ended up keeping many of the database-driven functions encapsulated in stored procedures and triggers by leveraging SQL Azure.” Tom Naylor described the approach on his blog.

The application architecture employed many cloud-optimized designs, such as:

Hybrid relational and noSQL data storage – SQL Azure for data that is inherently relational, and Windows Azure Table Storage for historical data and events, etc.
Event-driven design – Web roles receiving messages act as event capture layer, but asynchronously off-loads processing to Worker roles

Lessons Learned

In the real world, things rarely go completely as anticipated/planned. And it was the case for this real-world implementation as well. :) Tom Naylor was very candid about some of the challenges he encountered:

Early adopter challenges and learning new technologies – Windows Azure Table and Blob Storage, and Windows Azure AppFabric Service Bus are new technologies and have very different constructs and interaction methods
“The way you insert and access the data is fairly unique compared to traditional relational data access”, said Tom, such as the use of “row keys, combined row keys in table storage and using those in queries”
Transactions - initial design was very asynchronous; store in Windows Azure Blob storage and put in Windows Azure Queue, but that resulted in a lot of transactions and significant costs based on the per-transaction charge model for Windows Azure Queue. Had to leverage Windows Azure AppFabric Service Bus to reduce that impact

The end result is a an application that is horizontally scalable, allowing Advanced Telemetry to elastically scale up or down the deployments of individual layers according to capacity needs, as different application layers are nicely decoupled from each other, and the application is decoupled from horizontally scalable storage. Moreover, the cloud-optimized architecture supports both multi-tenant and single-tenant deployment models, enabling Advanced Telemetry to support customers who have higher data isolation requirements.

Rise of the Cloud Ecosystems

David C. Chou — Wed, 16 Mar 2011 07:24:58 GMT

I had the opportunity to participate at this year’s CloudConnect conference, with my session on Building Highly Scalable Applications on Windows Azure, which is mostly an update from my earlier presentations at JavaOne and Cloud Computing Expo. I was pleased to learn that the cloud-optimized design leveraging distributed computing best practices approach, aligned well to similar talks by well-known cloud experts from Amazon, Google, etc. A more detailed discussion on this topic can be found in my earlier post - Designing for Cloud-Optimized Architecture.

One of the major takeaways I had from the conference, was the focus on ‘cloud platforms’ (or platform-as-a-service generally) messaging, further reinforcing the platform view of cloud computing, as opposed to the infrastructure-level perspectives, or mixed views around the popular SaaS, PaaS, and IaaS service delivery models.

It started with Werner Vogels in his keynote presentation. Werner said, “it is all about the cloud ecosystem”, that “everything are cloud services; everything as a cloud service”, and “not constrained by any model”. And “let a thousand platforms bloom”, where “ecosystems grow as big as possible”. This implies that the popular models (e.g., SaaS, PaaS, IaaS) are irrelevant, because everything are simply services that we can consume, and these services can span the entire spectrum of IT capabilities as we know, and potentially more.

It is interesting to see how the platform messaging evolved over the past few years. For instance, Werner Vogels used to refer to Amazon Web Services as “Infrastructure as a Service”. However, I think Werner Vogels has always advocated the platform view, similar to Gartner’s notion of “application infrastructure as a service”, instead of the overused IaaS (infrastructure-as-a-service) service delivery model (as popularized by NIST’s definition of cloud computing). Perhaps, it was also because many people started incorrectly referring to Amazon Web Services as IaaS and not seeing the platform view, that Werner chose to clarify that models are irrelevant and “it is all about the cloud ecosystem”.

I also belong to the camp that advocates the platform view, and the further ecosystem view of cloud computing. The platform and ecosystem views of cloud computing represent a new paradigm, and promote a new way of computing. Though I think the SaaS, PaaS, and IaaS classifications (or service delivery models) still have some uses too. They are particularly relevant when trying to understand the general differences and trade-offs between the service delivery models (as defined by NIST), from a layers and levels of abstractions perspective.

Perhaps, what we shouldn’t do, is to try to fit cloud service providers into these categories/models. As often, a particular service provider may have offerings in multiple models, have offerings that don’t fit well in these models, or it’d be over-simplifying to refer to cloud platforms, like Amazon Web Services, Google App Engine, Windows Azure platform, etc., strictly in this platform-as-a-service definition. These platforms have a lot more to offer than simply a higher-level abstraction compute service.

Cloud Platforms

As Werner Vogels said, “cloud is actually a very large collection of services”, cloud platforms aren’t just a place to deploy and execute workloads. Cloud platforms provide the necessary capabilities, delivered as distinct services, that applications can leverage to accomplish specific tasks.

Amazon Web Services has always been a cloud platform; today it is a collection of services that provide capabilities for compute (EC2, EMR), content delivery (CloudFront), database (SimpleDB, RDS), deployment and management (Elastic Beanstalk, CloudFormation), e-commerce (FWS), messaging (SQS, SNS, SES), monitoring (CloudWatch), networking (VPC, ELB), payments and billing (FPS, DevPay), storage (S3, EBS), support, etc. It is not just a hosting environment for virtual machines (which the popular IaaS model is more aligned with). In fact Amazon Web Services released S3 into production (March 2006) before EC2 (limited public beta in August 2006, removed beta label in October 2008).

Similarly, Microsoft has been using the platform-as-a-service messaging when describing Windows Azure platform, but it is also about the collection of various capabilities (delivered as services). For example, below is a visual representation of the application platform capabilities in Windows Azure platform that I have been using since 2009 (though the list of capabilities grew over that period):

And below shows how those capabilities are delivered as services and organized in Windows Azure platform.

This is important because the platform view is one of the major differentiators of cloud platforms when compared to the more conventional outsourced hosting providers. When building applications using hosting providers (or strictly infrastructure-as-a-service offerings), we have to incur the engineering efforts to design, implement, and maintain our own solutions for storage, data management, security, caching, etc. In cloud platforms such as Amazon Web Services, Google App Engine, and Windows Azure platform, these capabilities are baked into the platform and available as services that are readily accessible. Applications just need to use them, without having to be concerned with any of their underpinnings and operations.

An analogy can be drawn from high-level differences between getting food products from Costco to put together a semi-homemade meal, versus getting raw ingredients from a supermarket and prepare, cook, and finish a fully-homemade meal. :) The semi-homemade model offers higher agility (less time and efforts required to put together a meal and to scale it for bigger parties) and economy of scale in Costco’s case, while the fully-homemade model offers more control.

Another distinction between cloud platforms and typical IaaS offerings, is that cloud platforms are more of a way of computing – a new/different paradigm; whereas IaaS offerings are better aligned towards hosting scenarios. This doesn’t mean that there are no overlaps between the two models, or that one is necessarily better than the other. They are just different models; with some overlaps, but ideally suited for different use cases. For cloud platforms, ideal use cases are aligned to net-new, or greenfield development projects that are cloud-optimized. Again, hosting scenarios also work on cloud platforms, but cloud-optimized applications stand to gain more benefits from cloud platforms.

Cloud Ecosystems

The cloud ecosystem view takes the cloud platform view one step further, and includes partners and third parties that enable their services to participate in an ecosystem. The collective set of capabilities from multiple organizations and potentially services spanning multiple platforms and cloud environments together form an ecosystem that feeds and builds upon each other (in composite, federated, application models), and generating best practices and reusable processes, communities, etc. This can also be viewed as a natural evolution of platform paradigms, when drawing inference from other models where the iterations typically evolved from technology maturity, critical mass in adoption, and then building ecosystems. The platform with the largest and most diverse ecosystem, gets to ride the paradigm shift and enjoy a dominant position for that particular generation.

The Web platform stack model I discussed back in 2007 is one way of looking at the ecosystem model (apologies for the rich color scheme; I was going through a coloring phase at the time).

In essence, a cloud ecosystem itself will likely have many layers of abstractions as well; one building on top of another. One future trend in cloud computing may very well be the continued climb into higher levels of abstraction, as differences and complexities at one level often represent development opportunities (e.g., for specializations, consolidations/aggregations, derivations, etc.) at a higher level.

Ultimately, cloud platforms enable the dynamic environments that support the construction of ecosystems. This is one aspect inherent in cloud platforms, but not as much for lower-level IaaS environments. And as the ecosystems grow in size and diversity, the network effect (as discussed briefly back in 2008) will contribute to increasingly intelligent and interactive environments, and generate, collectively, tremendous value.

Internet Service Bus and Windows Azure AppFabric

David C. Chou — Thu, 24 Mar 2011 08:39:45 GMT

Microsoft’s AppFabric, part of a set of ”application infrastructure” (or middleware) technologies, is (IMO) one of the most interesting areas on the Microsoft platform today, and where a lot of innovations are occurring. There are a few technologies that don’t really have equivalents elsewhere, at the moment, such as the Windows Azure AppFabric Service Bus. However, its uniqueness is also often a source of confusion for people to understand what it is and how it can be used.

Internet Service Bus

Oh no, yet another jargon in computing! And just what do we mean by “Internet Service Bus” (ISB)? Is it hosted Enterprise Service Bus (ESB) in the cloud, B2B SaaS, or Integration-as-a-Service? And haven’t we seen aspects of this before in the forms of integration service providers, EDI value-added networks (VANs), and even electronic bulletin board systems (BBS’s)?

At first look, the term “Internet Service Bus” almost immediately draws an equal sign to ‘Enterprise Service Bus’ (ESB), and aspects of other current solutions, such as ones mentioned above, targeted at addressing issues in systems and application integration for enterprises. Indeed, Internet Service Bus does draw relevant patterns and models applied in distributed communication, service-oriented integration, and traditional systems integration, etc. in its approaches to solving these issues on the Internet. But that is also where the similarities end. We think Internet Service Bus is something technically different, because the Internet presents a unique problem domain that has distinctive issues and concerns that most internally focused enterprise SOA solutions do not. An ISB architecture should account for some very key tenets, such as:

Heterogeneity – This refers to the set of technologies implemented at all levels, such as networking, infrastructure, security, application, data, functional and logical context. The internet is infinitely more diverse than any one organization’s own architecture. And not just current technologies based on different platforms, there is also the factor of versioning as varying implementations over time can remain in the collective environment longer than typical lifecycles in one organization.
Ambiguity – the loosely-coupled, cross-organizational, geo-political, and unpredictable nature of the Internet means we have very little control over things beyond our own organization’s boundaries. This is in stark contrast to enterprise environments which provide higher-levels of control.
Scale – the Internet’s scale is unmatched. And this doesn’t just mean data size and processing capacity. No, scale is also a factor in reachability, context, semantics, roles, and models.
Diverse usage models – the Internet is used by everyone, most fundamentally by consumers and businesses, by humans and machines, and by opportunistic and systematic developments. These usage models have enormously different requirements, both functional and non-functional, and they influence how Internet technologies are developed.

For example, it is easy to think that integrating systems and services on the Internet is a simple matter of applying service-oriented design principles and connecting Web services consumers and providers. While that may be the case for most publicly accessible services, things get complicated very quickly once we start layering on different network topologies (such as through firewalls, NAT, DHCP and so on), security and access control (such as across separate identity domains), and service messaging and interaction patterns (such as multi-cast, peer-to-peer, RPC, tunneling). These are issues that are much more apparent on the Internet than within enterprise and internal SOA environments.

On the other hand, enterprise and internal SOA environments don’t need to be as concerned with these issues because they can benefit from a more tightly controlled and managed infrastructure environment. Plus integration in the enterprise and internal SOA environments tend to be at a higher-level, and deal more with semantics, contexts, and logical representation of information and services, etc., organized around business entities and processes. This doesn’t mean that these higher-level issues don’t exist on the Internet; they’re just comparatively more “vertical” in nature.

In addition, there’s the factor of scale, in terms of the increasing adoption of service compositions that cross on-premise and public cloud boundaries (such as when participating in a cloud ecosystem). Indeed, today we can already facilitate external communication to/from our enterprise and internal SOA environments, but to do so requires configuring static openings on external firewalls, deploying applications and data appropriate for the perimeter, applying proper change management processes, delegate to middleware solutions such as B2B gateways, etc. As we move towards a more inter-connected model when extending an internal SOA beyond enterprise boundaries, these changes will become progressively more difficult to manage collectively.

An analogy can be drawn from our mobile phones using cellular networks, and how its proliferation changed the way we communicate with each other today. Most of us take for granted that a myriad of complex technologies (e.g., cell sites, switches, networks, radio frequencies and channels, movement and handover, etc.) is used to facilitate our voice conversations, SMS, MMS, and packet-switching to Internet, etc. We can effortlessly connect to any person, regardless of that person’s location, type of phone, cellular service and network, etc. The problem domain and considerations and solution approaches for cellular services, are very different from even the current unified communications solutions for enterprises. The point is, as we move forward with cloud computing, organizations will inevitably need to integrate assets and services deployed in multiple locations (on-premises, multiple public clouds, hybrid clouds, etc.). To do so, it will be much more effective to leverage SOA techniques at a higher-level and building on a seamless communication/connectivity “fabric”, than the current class of transport (HTTPS) or network-based (VPN) integration solutions.

Thus, the problem domain for Internet Service Bus is more “horizontal” in nature, as the need is vastly broader in scope than current enterprise architecture solution approaches. And from this perspective Internet Service Bus fundamentally represents a cloud fabric that facilitates communication between software components using the Internet, and provides an abstraction from complexities in networking, platform, implementation, and security.

Opportunistic and Systematic Development

It is also worthwhile to discuss opportunistic and systematic development (as examined in The Internet Service Bus by Don Ferguson, Dennis Pilarinos, and John Shewchuk in the October 2007 edition of The Architecture Journal), and how they influence technology directions and Internet Service Bus.

Systematic development, in a nutshell, is the world we work in as professional developers and architects. The focus and efforts are centered on structured and methodical development processes, to build requirements-driven, well-designed, and high-quality systems. Opportunistic development, on the other hand, represents casual or ad-hoc projects, and end-user programming, etc.

This is interesting because the majority of development efforts in our work environments, such as in enterprises and internal SOA environments, are aligned towards systematic development. But the Internet advocates both systematic and opportunistic developments, and increasingly more so as influenced by Web 2.0 trends. Like “The Internet Service Bus” article suggests, today a lot of what we do manually across multiple sites and services, can be considered a form of opportunistic application; if we were to implement it into a workflow or cloud-based service.

And that is the basis for service compositions. But to enable that type of opportunistic services composition (which can eventually be compositing layers of compositions), technologies and tools have to be evolved into a considerably simpler and abstracted form such as model-driven programming. But most importantly, composite application development should not have to deal with complexities in connectivity and security.

And thus this is one of the reasons why Internet Service Bus is targeted at a more horizontal, and infrastructure-level set of concerns. It is a necessary step in building towards a true service-oriented environment, and cultivating an ecosystem of composite services and applications that can simplify opportunistic development efforts.

ESB and ISB

So far we discussed the fundamental difference between Internet Service Bus (ISB) and current enterprise integration technologies. But perhaps it’s worthwhile to discuss in more detail, how exactly it is different from Enterprise Service Bus (ESB). Typically, an ESB should provide these functions (core plus extended/supplemental):

Dynamic routing
Dynamic transformation
Message validation
Message-oriented middleware
Protocol and security mediation
Service orchestration
Rules engine
Service level agreement (SLA) support
Lifecycle management
Policy-driven security
Registry
Repository
Application adapters
Fault management
Monitoring

In other words, an ESB helps with bridging differences in syntactic and contextual semantics, technical implementations and platforms, and providing many centralized management capabilities for enterprise SOA environments. However, as we mentioned earlier, an ISB targets concerns at a lower, communications infrastructure level. Consequently, it should provide a different set of capabilities:

Connectivity fabric – helps set up raw links across boundaries and network topologies, such as NAT and firewall traversal, mobile and intermittently connected receivers, etc.
Messaging infrastructure – provides comprehensive support for application messaging patterns across connections, such as bi-directional/peer-to-peer communication, message buffers, etc.
Naming and discovery – a service registry that provides stable URI’s with a structured naming system, and supports publishing and discovering service end point references
Security and access control – provides a centralized management facility for claims-based access control and identity federation, and mapping to fine-grained permissions system for services

A figure of an Internet Service Bus architecture is shown below.

And this is not just simply a subset of ESB capabilities; ISB has a couple of fundamental differences:

Works through any network topology; whereas ESB communications require well-defined network topologies (works on top)
Contextual transparency; whereas ESB has more to do with enforcing context
Services federation model (community-centric); whereas ESB aligns more towards a services centralization model (hub-centric)

So what about some of the missing capabilities that are a part of ESB, such as transformation, message validation, protocol mediation, complex orchestration, and rules engine, etc.? For ISB, these capabilities should not be built into the ISB itself. Rather, leverage the seamless connectivity fabric to add your own implementation, or use one that is already published (can either be cloud-based services deployed in Windows Azure platform, or on-premises from your own internal SOA environment). The point is, in the ISB’s services federation model, application-level capabilities can simply be additional services projected/published onto the ISB, then leveraged via service-oriented compositions. Thus ISB just provides the infrastructure layer; the application-level capabilities are part of the services ecosystem driven by the collective community.

On the other hand, for true ESB-level capabilities, ESB SaaS or Integration-as-a-Service providers may be more viable options, while ISB may still be used for the underlying connectivity layer for seamless communication over the Internet. Thus ISB and ESB are actually pretty complementary technologies. ISB provides the seamless Internet-scoped communication foundation that supports cross-organizational and federated cloud (private, public, community, federated ESB, etc.) models, while ESB solutions in various forms provide the higher-level information and process management capabilities.

Windows Azure AppFabric Service Bus

The Service Bus provides secure messaging and connectivity capabilities that enable building distributed and disconnected applications in the cloud, as well hybrid application across both on-premise and the cloud. It enables using various communication and messaging protocols and patterns, and saves the need for the developer to worry about delivery assurance, reliable messaging and scale.

In a nutshell, the Windows Azure AppFabric Service Bus is intended as an Internet Service Bus (ISB) solution, while BizTalk continues to serve as the ESB solution, though the cloud-based version of BizTalk may be implemented in a different form. And Windows Azure AppFabric Service Bus will play an important role in enabling application-level (as opposed to network-level) integration scenarios, to support building hybrid cloud implementations and federated applications participating in a cloud ecosystem.

Event - XAPFest LA 2011 Saturday, June 4th, 2011 | 9AM – 10PM

David C. Chou — Thu, 12 May 2011 00:48:20 GMT

XAPFest LA 2011

Saturday, June 4th, 2011 | 9AM – 10PM

Loews Santa Monica

1700 Ocean Avenue, Santa Monica, CA, 90401 | (310) 458-6700

*Registration is required

about XAPFest

XAPFest is a Windows Phone 7 Hackathon for developers of every background and skill level. iOS and Android developers, .NET veterans, and curious programmers who want to try their hand at programming for Windows Phone 7 are all invited! It is a great opportunity to get to learn new technologies, meet members of the SoCal tech community, code, and have fun!

XAPFest consists of 1 full day of hacking – breakfast, lunch, dinner, plus snacks and drinks will be provided. This is a BYOL (Bring Your Own Laptop) Event. Please remember to install the free WP7 tools and SDK before the event begins, and if you don’t have a copy of Windows Vista or Windows 7 (required to install the WP7 tools), you can request a Windows 7 license. Loaner Windows Phone 7 units will be available to teams and individuals for the purposes of testing apps.

the prizes

Best App - $2000 (sponsored by Infragistics) & Windows Phone 7 device

Best Original App - $500 AMEX gift card & Windows Phone 7 device

Best Port – $500 AMEX gift card & Windows Phone 7 device

User’s Choice - XBOX 360 with Kinect & Windows Phone 7 device

rules for apps to be eligible

Applications must NOT already be in the Windows Phone 7 marketplace
Applications CAN be worked on before the event but must not be in the marketplace prior to the event
Teams CAN submit multiple applications and all CAN be submitted for the $2000 Grand Prize
Teams are limited to 3 or less members

event schedule

June 4, 2011	XAPFest LA 2011
9:00am	Doors open
9:30am	Welcome & Announcements
9:45am	Coding begins
12:00pm	Lunch
6:00pm	Coding ends
6:30pm	Reception and Dinner; judges begin reviewing apps
9:00pm	Judging is finished, Top 10 Windows Phone 7 Apps Presentation & Awards
10:00pm	Winners are announced

registration

Click here to see the full XAPFest event schedule. Registration is required

BUILD | 2011.09.13-16 | Anaheim, CA

David C. Chou — Thu, 02 Jun 2011 17:52:38 GMT

BUILD is the event that shows you how to take advantage of the future of Windows. Get insight on creating touch-centric user experiences, fast, fluid, and dynamic applications that leverage the power and flexibility of the core of Windows, used by more than a billion people around the world. Learn how to create powerful new apps while retaining the ability to use your existing apps. See how web-connected and web-powered apps built using HTML5 and JavaScript have full access to the power of the PC. Explore how the full power of hardware-accelerated Internet Explorer 10 transforms your experiences with the web. See how the UI was designed to work seamlessly with a diversity of devices and form factors. BUILD is the first place to dive deep into the future of Windows.

If you are a contemporary developer, who thrives on the newest and coolest, who loves the freedom of the web and the power of all devices from mobile to desktop, you need to join us to help BUILD the future. Our approach means no compromises—you get to use whatever kind of device you prefer to run the apps you love.

In 1995, Windows changed the PC. BUILD will show you that Windows 8 changes everything.

Connect with us

Crowdsourcing Discussion at Caltech | June 11, 2011

David C. Chou — Wed, 08 Jun 2011 21:39:59 GMT

Crowdsourcing, as a method of leveraging the massive pool of online users as resources, has become a viable set of techniques, tools, and marketplaces for organizations and entrepreneurs to drive innovation and generate value. It is applied effectively in a variety of explicit/implicit, and systematic/opportunistic models: collective intelligence (Wikipedia, Yelp, and Twitter analytics); collaborative filtering (Amazon’s product and Netflix’s movie recommendation engines); social tagging (del.icio.us, StumbleUpon, and Digg); social collaboration (Amazon’s Mechanical Turk); and crowdfunding (disaster relief, political campaigns, micropatronage, startup and non-profit funding, etc.)

An entrepreneur can utilize crowdsourcing tools for funding and monetization, task execution, and market analysis; or implement crowdsourcing as a technique for data cleansing and filtering, derived intelligence, etc. However, leveraging crowdsourcing also requires an entrepreneur to navigate a complex landscape of questions. How is it different from outsourcing? Is it truly cost-efficient? What motivates individual contributions? How to grow and sustain an active community? How to ensure quality or service level? What are the legal and political implications? Join us for a program which explores such questions, and the use of crowdsourcing to match specific needs to an available community.

Come interact with the local community, and an esteemed group of speakers which includes Peter Coffee (VP, Head of Platform Research at
Salesforce.com), Dana Mauriello (Founder and President at ProFounder), Michael Peshkam (Founder and CEO at IamINC), Nestor Portillo (Worldwide Director, Community & Online Support at Microsoft), Arvind Puri (VP, Interactive at Green Dot), and Alon Shwartz (Co-Founder & CTO at Docstoc.com).

June 11 9am-11am. Visit the website for more details and registration information - http://www.entforum.caltech.edu/

Free HTML5 Training | August 19, 2011 | Los Angeles

David C. Chou — Tue, 26 Jul 2011 21:20:19 GMT

As developers, you keep hearing a lot about HTML5, but many don’t know what it actually means or is truly capable of. If you want to learn about it, your choices are limited; you can either pick up a book or attend an expensive conference that often only scratches the surface. The HTML5 Web Camp is an opportunity to connect with designers and developers and show you what’s possible, and how you can start using it today. HTML5 WebCamp is a completely free event, and will start with demos and presentations, and will end with hands on lab sessions, that will allow you to walk through materials and get your questions answered! Space is limited.

Date: August 19, 2011; 8:30am – 5:30am

Here is a list of topic will be covered at the Los Angeles WebCamp:

Introduction to HTML 5 and IE9
Introduction to CSS3
Overview of HTML5 Graphics with Canvas & SVG
Adding Video, and Sound with HTML5
Practical HTML5: Using It Today
Hands on Labs

Windows Azure DevCamps Silicon Valley | October 28-29, 2011

David C. Chou — Thu, 06 Oct 2011 07:57:07 GMT

Event Location

Mt. View, CA
Date Oct. 28-29, 2011
Time 8:00AM - 6:00PM Location
Silicon Valley Center
1065 La Avenida St Building 1
Mt. View, CA 94043
REGISTER NOW >>
Events run from
8:00 AM - 6:00 PM

Featured Presenter

James Conard
James Conard is the Senior Director of the Windows Azure Evangelism team at Microsoft. His team is responsible for helping developers build applications in Windows Azure by providing key development resources including toolkits, samples, and training kits and by engaging with the community through conferences, training events and Developer Camps.
We'll also see presentations from: Bruno Terkaly , Wade Wegner, Roger Doherty, Nick Harris and Bruno Nowak

Come join us for 2 days of cloud computing!

Developer Camps (DevCamps for short) are free, fun, no-fluff events for developers, by developers. You learn from experts in a low-key, interactive way and then get hands-on time to apply what you've learned.

What am I going to learn at the Windows Azure Developer Camp?
At the Azure DevCamps, you'll learn what's new in developing cloud solutions using Windows Azure. Windows Azure is an internet-scale cloud computing and services platform hosted in Microsoft data centers. Windows Azure provides an operating system and a set of developer services used to build cloud-based solutions. The Azure DevCamp is a great place to get started with Windows Azure development or to learn what's new with the latest Windows Azure features.

Come for one day, come for both. Either way, you'll learn a ton. Here's what we'll cover.
Agenda

Day 1

• Getting Started with Windows Azure

• Using Windows Azure Storage

• Understanding SQL Azure

• Securing, Connecting, and Scaling Windows Azure Solutions

• Windows Azure Application Scenarios

• Launching Your Windows Azure App

Day 2

On Day 2, you'll have the opportunity to get hands on developing with Windows Azure. If you're new to Windows Azure, we have step-by-step labs that you can go through to get started right away. If you're already familiar with Windows Azure, you'll have the option to do build an application using the new Windows Azure features and show it off to the other attendees for the chance to win prizes. Either way, Windows Azure experts will be on hand to help.

More Windows Azure DevCamps in December 2011

David C. Chou — Tue, 25 Oct 2011 01:01:11 GMT

Event Locations

12/2 - Santa Monica, CA

12/13 - Redmond, WA

12/15- Phoenix, AZ
Events run from
9:00 AM - 4:30 PM

Register
To register or to view
the full agenda, select
a location above.

Come join us for 1 day of cloud computing!
Developer Camps (DevCamps for short) are free, fun, no-fluff events for developers, by developers. You learn from experts in a low-key, interactive way and then get hands-on time to apply what you've learned.
What am I going to learn at the Windows Azure
Developer Camp?

At the Azure DevCamps, you’ll learn what’s new in developing cloud solutions using Windows Azure. Windows Azure is an internet-scale cloud computing and services platform hosted in Microsoft data centers. Windows Azure provides an operating system and a set of developer services used to build cloud-based solutions. The Azure DevCamp is a great place to get started with Windows Azure development or to learn what’s new with the latest Windows Azure features.

Agenda

8:00	Arrival and Registration
9:00	Welcome
9:15	Getting Started with Windows Azure
10:00	Session 2 - Using Windows Azure Storage
11:00	Break
11:15	Session 3 – Understanding SQL Azure
12:15	Lunch
1:15	Session 4 – Securing, Connecting, and Scaling
2:15	Break
2:30	Session 5 – Azure Application Scenarios
3:30	Session 6 - Launching Your Azure App

TechNet Events Presents: The Pathway to the Private Cloud

David C. Chou — Wed, 26 Oct 2011 18:31:47 GMT

Get Ready for Tomorrow, Today - Hyper-V Virtualization for the Cloud

Virtualization is one of the critical elements of networks operations of all kinds. Virtualization is a key to cloud operations. Joins us as we discuss the key components of virtualization that provide the operational foundation for both Public and Private Cloud.

Private Cloud 201: Microsoft Private Cloud Tools and Technologies

So you have heard the private cloud story from a 101 level and you want to know more. Join us as we discuss Private cloud in greater detail with a focus on the tools and technologies that make Private Cloud such an appetizing It and business opportunity for business both large and small.

Ask More Questions! Attend an Afternoon IT Camp!

These are informal event where the true agenda is up to you. That’s right, we will have a list of topics (such as ways to get on Windows Server 2008 R2) plus your write-in topic(s) and we will put it to a vote. The ones that get the most votes will be the ones that get discussed. This will be very interactive and works best with participation from everyone. We look forward to having you join us for this new informal event.

For more information or to register,
visit > www.technetevents.com
OR CALL 1-877-MSEVENT

Location	TechNet	IT Camp
Bellevue	November 14	November 14
Los Angeles	November 14	November 14
Denver	November 15	November 15
Tempe	November 17	November 17
Mountain View	November 18	November 18
Portland	November 18	November 18
Lehi	December 6	December 6

TechNet Events Presents: The Pathway to the Private Cloud
8:30 PM – 12:30 PM

IT Camp

1:00 PM – 5:00 PM

Seating is limited,
so register today.

Windows Phone Night Parties Los Angeles – Nov 30 & Dec 1, 2011

David C. Chou — Wed, 26 Oct 2011 19:30:21 GMT

Join us for one of our exclusive events across the U.S. and get a first look at the hottest new Windows Phones such as the HTC Titan, HTC Radar, Samsung Focus S and Samsung Focus Flash. You know Windows Phone is packed with killer social integration, a wide variety of games and apps, and a ton of other great features, now come see it all on cool new hardware.

See where the night takes you when you challenge your friends to a Fruit Ninja battle, or chill out in one of our device lounges as you sip cocktails and sample good eats from top local chefs. Plus, catch live performances from today’s best indie bands like Matt & Kim or Young the Giant. These events are FREE, brought to you by Windows Phone, the phone that puts people first. Tickets are very limited, so be sure to reserve yours today on our Windows Phone Facebook page.

Los Angeles

November 30 & December 1

Featuring: MuteMath

Windows Phone 7 at Hacker Dojo Quadrathlon

David C. Chou — Tue, 08 Nov 2011 02:13:56 GMT

30 to Launch: A Windows Phone App and Your Moment in the Spotlight!

From Nov 14-Dec 14 2011, we are throwing down a challenge and an opportunity to HackerDojo members. Join us over these 4 weeks to build a Windows Phone app! Your app might get an opportunity for a moment in the spotlight! Plus, you could win a Windows Phone- yours to keep after you submit your app to the Windows Phone Marketplace!

November 14, 7pm–9:30pm

Session 1: Windows Phone 7.5: Overview for Developers. Building Applications with Visual Studio® and Silverlight®
Speakers: Matt Harrington, Kevin Ashley
Event info: 30 to Launch - Windows Phone - Coding Camp - Week 1

To get started, do the following pre-requisites:
a) Complete the Windows Phone Training & Set up your machines
b) Download Windows Phone Tools SDK
c) Register at Windows Phone Marketplace*
On site, we will have sessions (and pizza!), and support:
Loaner Windows Phones are available on site at the Hardware Lab for you to play with.
We’ll also give the first 50 participants who are onsite and register on the Marketplace a token for the $99 fee *

November 22, 7pm–9:30pm

Session 2: Windows Phone Live Tiles and Push Notifications. Building Applications in the Cloud with Windows Azure™
Speaker: Kevin Ashley
Event info: 30 to Launch - Windows Phone - Coding Camp - Week 2

We will make available Windows Phones loaners for specific “augmented reality” projects.

November 29, 7pm-9:30pm

Session 3: Monetizing a Windows Phone 7.5 Application. Fun and Games Development
Speaker: Kenny Spade
Event info: 30 to Launch - Windows Phone - Coding Camp - Week 3

Labs: Keep Working on App Development. In-lab session this week. We will make available Windows Phones loaners onsite.

December 6, 7pm-9:30pm

Go Live! Publish App to Windows Phone Marketplace! We will have a session during the week to help!
Speaker: Kevin Ashley
Event info: 30 to Launch - Windows Phone - Coding Camp - Week 4

We will be cheering you on!
PLUS:

Up to 50 participants with apps in development **will get a Windows Phone device**! Plus we will give you a Marketplace token if you didn’t apply for it in Week 1 – subject to remaining supplies*).
Showcase Opportunity! Subject to meeting the 30 to Launch showcase app criteria, your app becomes eligible for a Showcase! Details and Terms and Conditions coming soon!

Lastly, some details:
* $99 registration fee applies. We will provide up to a maximum of 50 tokens good for a free registration to the Windows Phone marketplace, to the first 50 onsite that register on the marketplace. Windows Phone Marketplace terms and conditions apply to your use of the Marketplace.
**Not available to Government or Public Sector Employees. Microsoft employees are also not eligible for this offer. Limited to one phone device per person. Participants remain responsible for acquiring a calling plan and for all related charges and taxes in connection with the Windows Phone.
***We will be giving away one Windows Phone Device each to a maximum of 50 winners based on a random drawing on the last session on Dec 6, 2011. Detailed terms and conditions are listed here

Campaign Participation Eligibility:
- Participants must be US residents 18+ or older - Microsoft employees are not eligible for the marketplace rebate or device
- Not available for Government or Public Sector Employees

What's new in Windows Phone 7.5

Improved Live Tiles. The Live Tiles on Start dish up more (and quicker) updates on your apps, and they're also a whole lot of fun—the Pictures Tile animates with a slideshow of your favorite snapshots and Group Tiles flash your friends' latest updates.

Threads. Start with a text, and finish in Facebook or Messenger chat (or vice versa). The whole conversation stays in one thread, and all it takes to switch back and forth is a tap. See how

App Connect. Get the app you need—sometimes before you even know you need it. Search the web, and alongside the usual results you'll see apps you can use—like a movie app when you're looking up showtimes. You'll also see photo apps in the Pictures Hub and music apps in Music + Videos.

Visual voicemail. See your voicemails in a list, then listen to them on your phone.

And many, many, many more… find out here.

NuCon 2012: Technology Leadership Conference

David C. Chou — Sat, 14 Jan 2012 07:40:00 GMT

Thursday, February 16, 2012 from 8:30 AM to 5:00 PM (PT)

Hyatt Regency Irvine
1700 Jamboree Boulevard
Irvine, CA 92614

http://www.neudesic.com/nucon/

NuCon 2012 is designed to give business leaders valuable insight into current technology trends. The one-day conference offers a variety of business and technical presentations led by noted experts on relevant topics that will help you build an IT strategy for the future of your company. You'll have the opportunity to mingle with peers and experts, demo the latest technology, play Xbox Kinect games, and even win some cool prizes.

NuCon 2012 offers 12 unique sessions delivered by noted experts and geared toward Technical and Business Decision Makers, Developers, IT Professionals and Business Strategy Planners. The sessions are conveniently organized in three general categories to make it easy for members of your organization to select the topics most relevant to their role. A General Admission ticket allows attendees to customize the conference to fit their specific interests.

Microsoft DevCamps for Phone, Cloud and Web

David C. Chou — Wed, 14 Mar 2012 07:18:21 GMT

ATTEND A CAMP, BUILD AN APP
Developer Camps (DevCamps for short) are free, fun, no-fluff events for developers, by developers. You learn from experts in a low-key, interactive way and then get hands-on time to apply what you've learned. Check out the four different DevCamps currently being offered and register today!

CLOUD CAMP
Join us to learn about the exciting new Windows Azure developer platform! Windows Azure will open up a host of new application and multi-screen development opportunities, and this briefing will give you a jumpstart in understanding how to take advantage of them.

HTML5 Web Camp
As developers, you keep hearing a lot about HTML5, but many don't know what it actually means or is truly capable of. If you want to learn about it, your choices are limited. The HTML5 Web Camp is an opportunity to connect with designers and developers and show you what's possible, and how you can start using it today. Space is limited.

PHONE CAMP
Take your apps to the next level at the Windows Phone Dev Camp. We'll be covering how to bring the cloud to your app by consuming external APIs, add UX polish with Expression Blend, reach new markets through globalization and localization, and keep your app running at peak performance through advanced debugging and testing. In the afternoon, we'll break into labs to reinforce what we covered, and offer a chance to present your application for a chance to win in our application competition.

STUDENT PREP NIGHT
If you're a student, interested in attending DevCamp (Phone Camp, particularly), we'd love to have you join us for a night of preparation and pizza! We want to be sure you have all the tools you need to be successful with the professional developer community.

DATES and LOCATIONS

March 30, 2012 | Los Angeles | University of Southern California

DevCamps

Cloud

HTML5 Web

Phone

Student Prep Night (March 29)

April 20, 2012 | Irvine | UC Irvine Extension

DevCamps

Cloud

HTML5 Web

Phone

Student Prep Night (April 19)

April 27, 2012 | Redmond | Microsoft Conference Center

DevCamps

Cloud

HTML5 Web

Phone

May 18, 2012 | Denver | University of Colorado at Denver

DevCamps

Cloud

HTML5 Web

Phone

Student Prep Night (May 17)

May 25, 2012 | Phoenix | Desert Willow Conference Center

DevCamps

Cloud

HTML5 Web

Phone

Reimagining App Development: Introducing Windows 8 | 4.4.12 | Microsoft Silicon Valley

David C. Chou — Thu, 15 Mar 2012 04:25:33 GMT

We are inviting the Silicon Valley startup community to a full-day, knockout, deep dive event taking place on our Silicon Valley Campus. Developers and designers alike will take the stage to show you how to take advantage of this new platform opportunity. Several startups will demo their brand new apps and talk about their early experiences developing for Windows 8, while the Microsoft team will provide attendees the opportunity to play with the platform and the tools needed to get started. Sessions will cover designing for the new Metro UI, building for the platform, and monetizing your apps.

Agenda:

Time	Topic
12:00 PM	Registration
1:00 PM	Welcome & Kick-Off Dan’l Lewin, Corporate Vice President, Strategic and Emerging Business Development
1:15 PM	Windows 8 Introduction, Keynote, Demo, & Metro UI App Design
2:15 PM	Break \| Windows 8 Hack Stations
2:30 PM	Windows 8 Showcase Panel
3:15 PM	Windows 8 Marketplace Opportunity Robert Youngjohns, President, Microsoft North America
4:00 PM	Break \| Windows 8 Hack Stations
4:15 PM	Build & Reimagine your App on Windows 8
5:00 PM	VC & Entrepreneur Panel
5:45 PM	Closing Remarks & Next Steps
6:00 PM	Reception & Windows 8 Hack Stations

RSVP

April 4, 2012

Registration: 12:00 PM

Event: 1:00 PM-7:00 PM

Microsoft Silicon Valley

1065 La Avenida, Bldg 1

Mountain View, CA 94043

650.693.4000

Register today!

http://introducingwindows8.eventbrite.com/

Windows 8 Developer Event, Club Nokia Los Angeles, April 23 2012

David C. Chou — Tue, 17 Apr 2012 19:21:22 GMT

Windows reimagined.

Learn everything you need to start building Metro-style apps for Windows today at our free, full-day Windows Developer Event.

We'll show you how to use Visual Studio to code fast, fluid, immersive and beautiful Metro-style applications in HTML5/JavaScript, XAML/C# and C/C++. Your investments in these languages carry forward, making Windows a no-compromise platform for developers.

Whatever language you choose, your app gets deep integration with the Windows shell, including notifications, live tiles, deep links, and contracts with other apps. And now you can build once and support all Windows customers, no matter what type of PC they have—from tablets to laptops to convertibles to desktops.

Seating is limited and registration is not guaranteed. Secure your spot today!

Prerequisites

Please bring a notebook computer and your personal identification.
Be sure to download the Windows 8 Consumer Preview.
And, don't forget to download Visual Studio 11 Express for Windows 8 and seize the future of development.

Notes - This free event is brought to you by Microsoft. However, you are responsible for booking and funding your own travel and accommodations. Please note that there is limited space available for this event, so be sure to register early.

Los Angeles, CA
Date April 23, 2012
Time 9:00AM – 5:00PM
Location
Club Nokia
800 West Olympic Blvd

Get Your App Early Access to Windows 8 Store

David C. Chou — Wed, 09 May 2012 06:00:36 GMT

As you likely know Windows 8 is coming in the near future. You can download, use it, and even develop apps for it today. At this point in time the Windows Store is not open for everyone to deploy, but we are looking for the first wave of great applications which highlight the power of Metro and Windows 8, especially those developers that want to get to market first and build the awareness and brand for their applications.

In order to submit your application today you need a token which is something I can help you get.

What do you need to do to get a token? Here are the key steps:

Create a great Windows 8 Metro-style application or game and get it ready
Let me know about it by contacting me via this blog
I’ll help you register so you can get your application through our Application Accelerator Labs where the app will get reviewed to confirm it is done and conforms to the Metro guidelines and certification requirements.

This is a great opportunity to not only be first to market with your app, but also to get feedback from a Microsoft Services Engineer to make your app great. If you are serious about creating an application this is a chance that you probably don’t want to pass up.

In addition, our team is holding a series of events and office hours to help you – we want to make sure you have what you need to be successful. You can come learn more about how to build apps for Windows 8 or show up and build your app with one of our evangelists or others in your community available to help you if you need it. You can find more information about our events and availability at…

30 days to Launch Windows 8 Apps @ RocketSpace SF

David C. Chou — Wed, 23 May 2012 04:17:29 GMT

Come join us at RocketSpace in San Francisco every Monday night throughout the month of June to build great Windows 8 apps. Throughout the course of this four-part series, guests will have a chance to learn from Microsoft experts about the world’s largest opportunity for app developers.

We’ll also have great food and drink every Monday, and you’ll also have the opportunity to win great prizes! (See Official Rules here)

https://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032514381&Culture=en-US

RocketSpace

181 Fremont Street

San Francisco, CA 94105

Mondays, June 4, 11, 18, 25 from 6pm-9pm

Series Schedule:

June 4 - Week 1:
· Business Opportunity for Windows 8
· Designing Apps with Metro Principles and the Windows Personality

June 11 - Week 2:
· How Metro style apps drive end user impact
· Building Metro style apps with HTML and JavaScript
· Creating Windows 8 Metro Style User Interface (UI)

June 18 - Week 3:

· Bring Your App to Life with Live Tiles and Push Notifications
· Integrating with key Windows 8 features

June 25 - Week 4:
· Applying the finishing touches
· The Business Opportunity Continued: Monetizing your app
· The Windows Store