Haishi's Blog

A Message to My Readers: Visual Studio Live! is heading back to Redmond

2013-05-21T23:00:00.001-07:00

Dear readers:

I have never ever posted a message like this but I’ll make an exception this time. Visual Studio Live! is heading back to Microsoft campus on August 19-23 in Redmond, WA. Surrounded by your fellow developers, Visual Studio Live! provides you with immediately usable education that will keep you relevant in the workforce. Learn how you can Code like a Rock star at Visual Studio Live! Redmond — bring the issues that keep you up at night and prepare to leave this event with the answers, guidance and training you need. Register now: http://bit.ly/UGRD6_Reg

SPECIAL OFFER: As a thank you to the readers of my blog, I can extend $500 savings on the 5-day package. Register here: http://bit.ly/UGRD6_Reg and use code UGRD6.

Thank you for reading!

Best regards,

Haishi

Walkthrough: File Sharing Between Your Local Machine And Your Virtual Machines on Windows Azure

2013-05-07T17:52:00.001-07:00

With Windows Azure Virtual Network Point-to-Site Connectivity, it’s amazingly easy for cloud developers to connect their development machines to their Windows Azure virtual machines running on Windows Azure Virtual Network. In this walkthrough I’ll walk you through the steps of setting up a file share between your development machine and a virtual machine running on Windows Azure.

Log on to Windows Azure Management Portal.
On command bar, click on NEW button, then select NETWORKS->VIRTUAL NETWORK->CUSTOM CREATE menu.
On Virtual Network Details page, enter network NAME as pointtosite, create or select a AFFINITY GROUP, then click next arrow.
On DNS Servers and VPN Connectivity page, check Configure Point-to-Site VPN, then click next arrow:
On Point-to-Site Connectivity page, click next arrow.
On Virtual Network Address Spaces page, click on add gateway subnet button, and then click on check icon to complete network creation.
After the virtual network has been created, open its DASHBOARD page, and the click CREATE GATEWAY icon to create the dynamic routing gateway.
Add a new Windows Server 2012 virtual machine to the virtual network. Note that when you specify user credential, make sure to use the same user id and password of your local account. This is because the virtual machine on Windows Azure is not under your domain controller, and we are using the same user credential on both local and virtual machines to allow file sharing.
As the virtual machine is cooking, let’s create two self-signed certificates: one for root, and another for client identification. During the following steps we’ll need to upload the root certificate to Windows Azure so Windows Azure can validate the client machine using the certificate chain.
Launch Developer Command Prompt for VS2012 as administrator. Change current folder to a folder where you want to keep the generated certificates. Here I’ll use folder c:\books.
Use command
```
makecert -sky exchange -r -n "CN=MyFakeRoot" -pe -a sha1 -len 2048 -ss My
```
to create root certificate.
Use command
```
makecert -n "CN=MyLaptop" -pe -sky exchange -m 96 -ss My -in "MyFakeRoot" -is my -a sha1
```
to create client certificate.
Launch certmgr. Export the root certificate as a MyFakeRoot.cer file (without private key).
[Optional] if you are configuring VPN for another client, you’d need to install the client certificate on target machine.
Go back to Windows Azure Management Portal. On DASHBOARD page of the virtual network, click on link Upload client certificate to upload the root certificate. Note at this point the gateway should have been created.
After certificate has been uploaded, you can download and install VPN client from the DASHBOARD page (AMD64 Client link for 64-bit machines, x86 Client link for 32-bit machines).
After VPN client has been installed, you can see the VPN connection on your Windows network connection list. Click on the network to connect. When prompted by the VPN client, click Connect button to continue.
[Optional] Now you can use ipconfig/all to verify if VPN connection has been successfully established (look for PPP adapter).
Log on to the virtual machine, create a new Share folder under c:\. Share the folder with the user you specified when you created the virtual machine.
On Management Portal, record the virtual machines private IP on its DASHBOARD page. In my case the IP is 10.0.1.4.
Now you can use Explorer on your local machine and access the shared folder by \\10.0.1.4\Share.

Online Contents on Windows Azure SDK 2.0 (Being Updated NOW!)

2013-04-30T11:08:00.001-07:00

Announcements and Updates

MSDN

Channel 9

blog.haishibai.com

Gaurav Mantri’s Personal Blog

http://gauravmantri.com/2013/04/30/introducing-windows-azure-sdk-2-0-for-net/

Service Bus Contents (Thanks to Abhishek for the list)

Release notes: http://t.co/c5aLW86qNj
Clemens’s video: http://t.co/CpDjbzSy1z
Auth docs update: Service Bus Authentication
Blog on Task based APIs: http://blogs.msdn.com/b/windowsazure/archive/2013/04/11/task-based-apis-for-service-bus.aspx
Samples

SAS: http://code.msdn.microsoft.com/Shared-Access-Signature-0a88adf8
OnMessage: http://t.co/YWcSqszHPE
Browse: http://t.co/ezf6i5uRo5

Windows Azure SDK 2.0 Features (4) – Even More Things You May be Interested

2013-04-30T11:03:00.001-07:00

This is the third part of the blog series of introducing some of the exciting new features of Windows Azure SDK 2.0. In this series I’ll take you on a tour of new Windows Azure SDK 2.0 features and experience them firsthand. You can find the SDK 2.0 announcement here. You can acquire SDK 2.0 through Web PI, or you can download it Windows Azure .NET Developer Center.

Previous parts:

Windows Azure SDK 2.0 Features (1) – A Even Better Server Explorer

Windows Azure SDK 2.0 Features (2) – A Even Better Web Sites Experience

Windows Azure SDK 2.0 Features (3) – Even Bigger Machines

PowerShell Improvements

While it is packaged separately, PowerShell cmdlets have gone through some changes in the past few months as well. You can find full list of changes here. Here are some of the new features worth mentioning (many thanks to Guang Yang for providing the list):

Moved to PowerShell 3.0.
Service bus namespace management cmdlets.
Cloud services scaffolding improvements: new role cmdlets and role template support.
Azure Store cmdlets. Create add-on right from PowerShell console.
Storage Blob CRUD cmdlets.
Website diagnostic log streaming.
PS remoting.
High memory VM images.
IaaS support improvements.

Storage Client 2.0

Now New Projects uses Storage Client 2.0. You can find out what’s new in Storage Client 2.0 here.

Windows Azure Active Directory

In case if you have missed, Windows Azure Active Directory has gone in GA as well. See the announcement here and here.

ASP.NET 2012.2 and Web Tools 2012.2

NOTE: This is NOT part of SDK 2.0 release, however worth mentioning here as it’s also include tones of improvements in terms of features and tooling. Here are couple of quick links:

http://weblogs.asp.net/scottgu/archive/2013/02/18/announcing-release-of-asp-net-and-web-tools-2012-2-update.aspx

http://sedodream.com/2013/02/19/VideosOnWebPublishUpdatesInASPNET20122.aspx

Service Bus New Features

I’ve written a 4-part series on Service Bus new features while the features were in preview. Now you can use these features in production:

New features in Service Bus Preview Library (January 2013) - 1: Message Pump

New features in Service Bus Preview Library (January 2013) – 2: Auto-expiration

New features in Service Bus Preview Library (January 2013) – 3: Queue/Subscription Shared Access Authorization

New features in Service Bus Preview Library (January 2013) – Epilogue: Message Browse

Windows Azure SDK 2.0 Features (3) – Even Bigger Machines

2013-04-30T10:59:00.001-07:00

Previous parts:

Windows Azure SDK 2.0 Features (1) – A Even Better Server Explorer

Windows Azure SDK 2.0 Features (2) – A Even Better Web Sites Experience

High Memory Machines

With Windows Azure IaaS went GA (see announcements on MSDN and Scott Gu's blog), developers and ISVs can now take advantages of more VM image templates, larger VM sizes and reduced VM prices. And because now IaaS is in GA status, their VMs running on Windows Azure are formally assured with one of the industry’s highest monthly SLAs.

Windows IaaS now offer two types of high memory machines, A6 (28GB/4 core) and A7(56GB/8 core). A machine with more cores and higher memory enables you to scale-up your application to improve system performance and to increase system throughput. More importantly, it enables you to run your existing applications against bigger loads instead of having to change your application architecture for scaling out. Of course, scaling out is definitely the preferred way of scaling on Windows Azure (and any other cloud platforms). However, changing application architecture is not an easy task, if even possible at all (for instance when you use 3rd party applications that are designed for single machines). Now with the high memory machines, not only you can deploy your existing applications that require high memory directly on IaaS, you can also design your Cloud Services to take advantage of these machines.

Sample Scenario

In this sample scenario, we’ll create a Cloud Service that allows users to navigate through large data sets. The data sets I chose to use in this case are Sea Surface Temperature, Salinity and Density data sets from NASA. The data sets consist of 10,800 1920x1080 .tiff images, which occupy about 7.3G of disk space. The idea is to provide a web site that allows user to navigate through the data sets and switch freely among the three images series. To improve loading speed, I’m using a co-located cache cluster to cache image frames. Obviously the bigger the cache, the more pictures I’ll be able to preload into the cache to provide a better user experience.

I’ve pre-loaded all images to my Windows Azure storage account. When the web role starts, it will start to pre-load as many images as possible. As a user requests for a images, the system will check the cache first, if the image is not in cache, it queues a new loading request.

In future versions, more analytical features will be added, such as looking for sudden changes in image series, comparing different images, etc.

I picked this scenario out of tens of candidates, many of which are compute-intensive scenarios. However, I picked this scenario at the end because it uses every aspects of the powerful machine offering – more cores, abundant memory, and higher network bandwidth allocations. We often hear people saying “that’s a Big Data problem” when they see a large amount of data. However, in my opinion, Big Data is not a problem, it’s a system. It’s a system to gain BI from various of data sources and analysis. A Big Data system include 1) data discovery; 2) data collection; 3) data transforms; 4) data storage; 5) analysis; 6) result presentation and sharing. Running a complex simulation only focuses on the fifth part. On the other hand, this scenario shows how to collect, store, transform, and share large amount of data.

Implementation

The project is a Cloud Service with a ASP.NET MVC 4 Web Role (using Internet Application template). I enabled co-located cache with 50% of memory allocated for the cache cluster with cache entities living forever. Images are served to clients via a API controller. The following is the gist of this part of code:

DataCache cache = new DataCache();
var index = string.Format("{0}-{1:0000}", series, frame);
HttpResponseMessage response = new HttpResponseMessage();         
var data = cache.Get(index);   
if (data != null)
{
    response.Content = new StreamContent(new MemoryStream((byte[])data));
    response.Content.Headers.ContentType = new MediaTypeHeaderValue("image/tiff");
}
else
{
    //queue another request to get the image from blog storage and then put it in cache.
}

Result

I’ve published an early version of the app on one of the A7 machines (http://oceandatasample.cloudapp.net/). The publish process is exactly as before. The only thing I did was to change my service definition to use a large VM:

The application largely works. You can drag the slider to navigate through 3600 frames of HD images of each data series, and you can switch among data series at any time by clicking on corresponding water drops at the top of the screen. The play buttons don’t work quite right yet.

Windows Azure SDK 2.0 Features (2) – A Even Better Web Sites Experience

2013-04-30T10:56:00.001-07:00

Previous parts:

Windows Azure SDK 2.0 Features (1) – A Even Better Server Explorer

Window Azure Web Sites has been one of the most popular services that new Windows Azure users have been enjoying. You can create a new web site in seconds and your web site is up and running in no time on one of the most advanced data centers of the world – for free! You can learn more about Windows Azure Web Sites on www.windowsazure.com. Ever since the beginning the service has been well known for its simplicity, speed, and friendly developer experience. Now, with Windows Azure SDK 2.0 provide you even better experiences by integrating Web Sites into Visual Studio Explorer, and providing live log data streaming right into Visual Studio.

Server Explorer Support

Yes! Now you can manage your Web Sites from Visual Studio Server Explorer as well!

Right-click on any web site to bring up its contextual menu:

From the menu you can perform common operations such as browsing to the web site, open management portal to manage the site, to stop/start the site, etc.

That’s something cool, but I won’t call it amazing. However, here’s something that will knock your socks off – right-click one of your Web Sites and select View Stream Logs in Output Window. Then, launch your Web Site and perform some operations. You can see your application tracing information in the output window – live! And you can also download the log files by clicking on download button (indicated by the arrow in the following screenshot). In my case the tracing information is in Chinese because I’ve been working on a couple of localization scenarios - but you get the idea!

More Diagnostics Support & Others

The log streaming feature is very cool indeed, but that’s not all the diagnostics support the team has put into this release. In this part of the article I’ll walk through you a small web site to experience other diagnostic features as well as some other improvements in this version of SDK.

In Visual Studio 2012, create a new ASP.NET MVC4 Web Project (using Internet Application template).
Before we publish the site, let’s modify the HomeController to trace each of the actions. For instance, add this line to Index() method (you need to resolve namespace for Trace ):
```
Trace.TraceInformation("Landed on home page");
```
In solution explorer, right-click on the web site project and select “Publish…”.
Click “Import…” to import publish profile. Hmm… now there pops a Import Publish Profile dialog, which allows you either to import a Web Site publish profile as before, or to import the profile from an existing site:
After you’ve imported the profile, follow the wizard to finish publishing.
Now refresh the Server Explorer, you should see your web site listed. Right-click on the site and select View Streaming Logs in Output Window from the popup menu.
Navigate to different pages, and you’ll see your traced information appended to output window. The following shows in session I had with my test site. You can see the log messages, as well as inactivity reports before I stopped the streaming.
Right-click on the site and click View Settings in popup menu.
In the Configuration view, you can configure most of your web site settings here, and you can click on Full Web Site Settings to open Windows Azure Management Portal and directly land on the setting page of the site. You can see you can easily turn on and off web server logging, detailed error messages, as well as failed request tracing here. Our application log is current set to save to local file system with log level set to Information. You can also save your logs into your Windows Azure storage account. We’ll see how to do it next.
Go to Logs tab. Click on Configure Logging link. This brings you to the management portal.
Change APPLICATION LOGGING (STORAGE) to ON (I also changed LOGGING LEVEL to Information in this case). If you see this option disabled, that means you don’t have a storage account linked to this site yet. Go to LINKED RESOURCES tab and link to a storage account of your choice.
Click manage connection button. Click on Synchronize primary key button and storage key is synced over – isn’t that sweet?
Save your changes.
Navigate through pages. And refresh the Logs tab to see latest trace entries:

Windows Azure SDK 2.0 Features (1) – A Even Better Server Explorer

2013-04-30T10:53:00.001-07:00

This is the first part of the blog series of introducing some of the exciting new features of Windows Azure SDK 2.0. In this series I’ll take you on a tour of new Windows Azure SDK 2.0 features and experience them firsthand. You can find the SDK 2.0 announcement here. You can acquire SDK 2.0 through Web PI, or you can download it Windows Azure .NET Developer Center.

Server Explorer Improvements

In the past releases, Windows Azure SDK has been steadily supplementing Visual Studio Server Explorer with more and more productivity features. These features put the ability of provisioning and managing cloud resources at the fingertips of developers. Developers don’t have to leave Visual Studio if they found they needed to get some cloud resources provisioned, or to get connection information to a resource. And they can use data manipulation capabilities to quickly test their code while programming. Here are some new features that got added with Windows Azure SDK 2.0.

Table Storage Tooling Improvements

Now you can directly create new storage Tables within Server Explorer:
Now Table explorer provides complete CRUD support of table entities. The following screenshot shows the Add Entity window, which is quite self-explanatory:
There’s even a nice Query Builder that allows you to build up table queries without needing to understand Data Service Queries:

Cloud Service Diagnostics

Throughout the SDK versions Cloud Service diagnostics has been becoming easier and easier. Now with the improved Server Explorer, you can view diagnostics data and modify diagnostics settings direct within Visual Studio. If you had used diagnostics.wadcfg, or had tried to configure performance counters in code, you’d be delighted to find out the rich editing features provided by diagnostics settings dialog.

View diagnostics data in a combined view in Visual Studio. Now you can request diagnostics logs at any time from Visual Studio and view them in an intuitive, combined view:
In the combined summary view, you can view Windows Azure application logs, Event logs, Windows Azure infrastructure logs, as well as contents in Windows Azure log directories at the same time. This view allows you to view all log sources at the same time so that you can easily discover correlations of these entries from different sources. This is definitely a very powerful tool to enable more efficient troubleshooting processes. The following screenshots shows the testing diagnostics data one of my test applications generates:
You can remotely change diagnostics settings from Visual Studio using diagnostics configuration dialog. For instance, you can change data transfer schedules and filters, and you can modify what performance counters to be collected.

Cloud + Devices Scenarios (1): Say a Picture

2013-03-23T06:09:00.001-07:00

Say a Picture is a sample Windows Azure Cloud Service that allows remote users to dictate a picture together over connected Kinect sensors. Kinect sensors enable users to interact with the system in a casual living room setting, with multiple participants work together either locally or remotely. The solution combines device’s capability of data collection and cloud’s ability of coordinating work over the Internet to create an enjoyable experience for users. In this post I’ll walk through the creative process of this project so that you can build similar (and cooler) applications like this. I’ll also provide a link to download the complete source code at the end of this post.

What you need

Microsoft Kinect for Windows
Get Kinect SDK. I recommend to get both SDK as well as the Toolkit from the page.
Microsoft Speech SDK.

System Architecture

Our system is comprised of a client and a server. The client talks to the Kinect sensor and uses speech recognition to convert spoken English to short commands, such as “add a horse” and “make it red”. Then, the client sends parsed short commands to the server. The server takes in commands sent from client and converts them into drawings on a HTML 5 canvas. The server is designed to support multiple concurrent sessions. Clients use the same session id share the same canvas so that multiple remote users can join the same painting session to work on the same painting. The following diagram depicts the overall system architecture:

Client Implementation

Client is implemented as a Windows Console application using C#, Kinect SDK as well as Speech SDK. The key of client implementation is the grammar tree that is used by speech recognition engine. The grammar tree is an XML document. The following is a picture representation. As you can see, this grammar is quite simple with a very limited vocabulary. In the diagram, #object represents one of recognized objects – trees, mountains, horses, and birds, etc.. #color is one of supported colors such as red, green, and blue. Using an explicit grammar tree not only allows command interpretation much easier (comparing to interpreting natural language), but also allows users to chat casually in between the commands, making it a even better experience.

Speech recognition using Speech SDK is rather simple, once you create a SpeechRecognitionEngine instance, you can load the grammar tree and then simply wait for SpeechRecognized event, in which you can get recognized text as well as a confidence value that indicates how confident the engine is about the result:

mSpeechEngine = new SpeechRecognitionEngine(info.Id);
using (var memoryStream = new MemoryStream(Encoding.ASCII.GetBytes(Properties.Resources.SpeechGrammar)))
{
  var g = new Grammar(memoryStream);
  mSpeechEngine.LoadGrammar(g);
}
mSpeechEngine.SpeechRecognized +=mSpeechEngine_SpeechRecognized;
...

once a short sentence that matches defined grammar rules is detected, the client sends the string to server via an API call.

Server Implementation

Server is implemented as a Windows Azure Cloud Service with a single ASP.NET Web Role. The Web Role uses SignalR to broadcast states to all clients within the same session. And it uses HTML 5 + jQuery as the frontend. Because command interpreter is implemented as a browser-side JavaScript, the server-side code is extremely simple – it gets commands that are passed in by clients, and then broadcasts commands to all clients within the same session:

[HttpGet]
public void SendCommand(string group, string text)
{
  var context = GlobalHost.ConnectionManager.GetHubContext<CommandHub>();
  context.Clients.Group(group).command(new { Command = text });
}

Command Interpreter

The command interpreter is not that complex in this case, because we are not dealing complex constructs such as sub clauses. We can simply go through the command from left to right and handle the keywords as we go. A recommended way of creating this interpreter is to create a separate function for each of the keywords. Because we can do linear scan of words in this case, I used a simple array. However if your grammar is more complex, you may want to use operator/operand stacks. The interpreter also maintains a reference to latest referenced object as “it”. For instance, once you added a bird to the canvas, you can use command “move it left” to move the bird because “it” automatically references to the last object you interact with. The following code snippet shows the implementation of the two top-level functions. handlCommand() handles an incoming command by looping through each words and calling handleWord() on each one. Then, handleWord() method calls into corresponding functions based on the keyword it encounters.

function handleCommand(commandText) {
  commandWords = commandText.split(' ');
  var i = 0;
  while (i < commandWords.length) {
     i = handleWord(i);
  }
}
function handleWord(index) {
  switch (commandWords[index]) {
    case 'add':
      return handleAddCommand(index + 1);
    case 'move':
       return handleMoveCommand(index + 1);
  ...
      default:
        return index + 1;
  }
}

Note the functions don’t return any errors. They simply skip the sentences they don’t understand and wait for users to try again. A very nice command implemented by this interpreter is the “put it on …” command. For instance, in the following scenario, you have a plant that you want to put on the desk. Instead of moving it around step by step, you can directly say “put it on desk” to make the plant land perfectly on desk. Note in this version of the code I added some hardcoded adjustments to make this scenario work perfectly. A more generic implementation requires a more sophisticated object description/interaction system.

Application Scenarios

There are many possible applications scenarios of using Kinect Speech Recognition + Cloud Service. Here is a short list of some of the scenarios I can immediately think of:

Collaborative painting as a casual, multiplayer game.
Interactive interior design system that allows a designer to work with her clients remotely. And they can work together on the room layout by simply talking about it.
Collaborative software/hardware/system design, where team members can talk about a possible design and see it painted immediately on the screen.
Automatic picture/animation generation as you tell a story. This could be a nice touch to bedtime stories.

And with more sophisticated codes, you can do things such as running a physical simulation by simply describe it; playing chess by speaking out your next moves; playing multiplayer strategy game by dictating your troops…. the possibilities are endless.

What’s next

This initial version has only limited functionality. As the goal is to allow users to interact with the system naturally, there are quite some improvements can be made to make user experiences even better. One of enhancements I’m planning is to allow the system to learn about new concepts easily so that the system can be extended to accommodate more and more scenarios. Another area of improvements is to allow rich, natural interactions among objects. The “put x on y” command is a preview of such interactions.

You can get complete source code here.

A Tour of Windows Azure Active Directory Features in Windows Azure Management Portal – Logging in and Enabling Multi-factor Authentication

2013-03-04T01:12:00.001-08:00

With new release of Windows Azure Management Portal (Scott Gu’s announcement), it’s now possible for you to manage your Windows Azure AD objects in the management portal itself (team announcement). This post provides guided tour of these exciting new features. First, we’ll have a brief review what Windows Azure Active Directory is. Then, we’ll introduce how to get a Windows Azure subscription that is linked to your Windows Azure AD tenant. And then, we’ll go through the basics, starting with correct steps to authenticate in order to see these new features. And at last, we’ll go through steps of setting up multi-factor authentication for managing your Windows Azure subscriptions as well as using it in your own ASP.NET applications.

What is Windows Azure Active Directory

Windows Azure Active Directory is a REST-based service made by Microsoft that provides multi-tenant directory service for the cloud. You can either project users from your on-premises Active Directory to your Windows Azure AD tenant, or start with a fresh Windows Azure AD tenant and keep the entire directory on the cloud. Windows Azure Active Directory is designed for SaaS application developers to have a low cost (actually, it’s completely free!), scalable, and cloud-ready directory service that they can easily leverage in their applications. It provides WebSSO capabilities with ws-Federation and SAML 2.0. It also provides a REST-based Graph API that you can use to query directory objects. There are many contents available from Microsoft about Windows Azure Active Directory. I did a survey a while ago. I recommend you to read through the listed resources in the survey if you want to learn more about Windows Azure Active Directory.

A Subscription with a Tenant

At the time when this post is written, you need to create a new Windows Azure subscription using your Windows Azure AD tenant credential. If you are an Office365 user, you already have a Windows Azure AD tenant. If you want to sign up for a free testing tenant, you can follow this link. Once you’ve provisioned your administrator for your Windows Azure AD tenant, you can use that account to log in to www.windowsazure.com to get a new Windows Azure subscription associated with the tenant.

[Updated April 12 – 2013] With Windows Azure Active Directory hitting GA, now you can create and manage directory tenant directly using subscriptions based on a Microsoft account. For more information please see this blog.

Basics

Go to https://manage.windowsazure.com.
Instead of typing in your user id and password in the log in form, click on Office 365 users: sign in with your organizational account link to the left of the screen:
Log in with your Windows Azure Active Directory tenant account:
Once you log in, click on ACTIVE DIRECTORY link in the left pane. You’ll see two tabs on the page: one is ENTERPRISE DIRECTORY, where you manage your tenant objects; the other is ACCESS CONTROL NAMESPACES, where you manage your ACS namespaces:
Click on the tenant name. You’ll enter the page where you can mange your users, domains, as well as directory sync with on-premises directories:
Now, with USERS tab selected, let’s click on CREATE button on command bar to create a new user.
Fill out the form. At this point you can choose to assign the user to one of two pre-defined roles, User or Global Administrator. Only Global Administrators can manage your Windows Azure AD tenants. Here we’ll choose User role. Click next arrow to continue.
The new user will be assigned with a temporary password, which has to be changed when the user first logs in. Click on Create button to display the temporary password:
Here you can choose to copy the password to clipboard and share the password manually, or to directly send the password to up to 5 mail addresses for the new user. Click check button to complete the process.
To allow this new user to log in to management portal, we need to add him as a co-administrator of our subscription. To do this, click on SETTINGS in the left pane (1), then make sure ADMINISTRATORS tab is selected (2), and finally click on ADD button in command bar (3).
Enter the email address of this new user, select the subscription(s) you want to grant access to, and then click on check icon to complete the process.
Now you can log out and log in using the new user account. You’ll have to change your password when you log in for the first time:
Note because now you are logged in as a User, you can’t manage your Windows Azure AD tenant using this account. When you click on ACTIVE DIRECTORY in the left pane, you only see your ACS namespaces but not tenant objects.
Now, log back in as an administrator. Go back to ACTIVE DIRECTORY, open your AD tenant, and click on the user name to bring up his profile. Here you can promote the user to administrator (1), or block user from accessing the portal (2), or rest user’s password (3).
Just for fun, let’s block the account, save changes, and log out. Then, when you try to log in using the account, your access is denied:

2-Factored Authentication – the # key is your friend!

Many of our customers have requested the ability of enabling 2-factored authentication when users try to access the management portal. Now this has been enabled and the configuration is really easy.

The 2-factored authentication only applies to Global Administrators. So first let’s promote the user we just created to administrator. The following screenshot shows that I’ve changed user role to Global Administrator (1), entered an alternative email address as requested (2), checked Require Multi-factor Authentication (3), and I’ve also kindly unblocked user from logging in (4). Once all these are done, click SAVE (5)to save the changes.
Log out and try to log in using the account. You’ll see something different this time, telling you the user has not configured for multi-factor authentication yet. Click on Set it up now to continue.
You’ll see the page where you are asked to enter a mobile phone number. Enter your phone number (make sure your phone is accessible to you) and click on Save.
A computer will call you and ask you to press # key on your phone. Press the # key to finish verification. Once verification successes, click Close button to complete this step:
Now the log in process continues. A computer will call you again to ask you to press # key. Answering the call and pressing the # key allow you to log in to the management portal. From now on, whenever you try to log in using this user account, you’ll need to answer the phone and press the # key as the second factor authentication before you are granted access to the portal. That’s pretty cool, isn’t it?

A code sample

At last, let’s have a simple code sample to leverage the multi-factor authentication in your own web applications. It’s amazingly easy:

If you haven’t done so, download and install Microsoft ASP.NET Tools for Windows Azure Active Directory – Visual Studio 2012 (preview). Note, before installing ASP.NET Tools for Windows Azure AD you need to install ASP.NET and Web Tools 2012.2 Update first. For more details please read here.
Launch Visual Studio, create a new ASP.NET MVC 4 Internet application.
Click on PROJECT –> Enable Windows Azure Authentication … menu.
You’ll be asked to enter your Windows Azure AD tenant. Enter your Windows Azure AD domain, and click Enable:
A log in window shows up. Log in as an administrator of your tenant.
And it’s done! Click Close to close the dialog box.
Press F5 to launch application.
Log in using a user that you’ve already configured multi-factor authentication. You’ll get the same 2-factored authentication experience using the combination of your credential and mobile phone.

Hope this helps. Enjoy!

Use GIMP to remove solid background of an image

2013-02-27T15:18:00.001-08:00

Okay, this is not a typical topic I’d write a blog for. But I think this post could be useful to the “accidental graphic designers” like myself to get a very common task done quickly and with high quality – to remove solid background of an image.

When you embedded an image, you usually want to remove image background so that when your website design changes the image will still look perfect. The problem is that very often you get images that are anti-aliased against white background. And when you remove the background using common tools such as color selection tool, there are always some annoying leftover pixels around the image. The image may look fine in front of a light background, but turns ugly when you switch to a darker background:

Here I’m going to introduce a simple method using GIMP. Unlike other methods you can find on Internet, this method doesn’t need you to do any fine-tuned selection or editing. Within a couple of steps you’ll get an image that is perfected anti-aliased against alpha channel.

Open the picture you want to work on in GIMP.
In Layers window, right-click the layer and select Add Alpha Channel.
Duplicate the layer.
Hide the lower layer, then select the top layer.
Use color selection tool to select background:
Use Select->Grow menu to grow the selection by 1-2 pixels.
Press Delete key to delete the background. This will remove some image borders. Don’t worry, we’ll recover them later.
Now hide the top layer, show and select the bottom layer.
Press Ctrl+Shit+A to unselect everything.
With bottom layer selected, select Colors –> Color to Alpha… menu.
Select the color you want to change to alpha (in this case white), and click OK to apply the filter:
The above step removes all white color from the image. All you need to do is to turn on the top layer again. Then you get a perfect picture!
Optionally, use Image –> Autocrop image to crop the image.
Export to a png file.

That’s it! Hope this helps!

A Love Story: Devices and Cloud – Perfect Together

2013-02-27T12:46:00.001-08:00

My new article has just been published on Cloud Computing Journal:

A Love Story: Devices and Cloud – Perfect Together

We’ve heard about Google Glasses. And we’ve heard about Apple iWatch. And yes, we’ve heard about people embedding cellphones into shoes [1]. Once again, the term “wearable computing” is becoming a hot topic in news and literatures. Some have even announced that the age of wearable technology is here [2]. The world is surely changing fast. Aren’t we still debating who makes the best smartphones and the best tablets? What’s going on here?
The foreseeable booming of wearable computing is just another chapter of the unstoppable fusion process between cyber space and reality. Eventually, machine computing will become ubiquitous, and the boundary between human bodies and devices will fade away. This is a vision shared by many scientists, innovators, entrepreneurs and Hollywood writers. It’s becoming a reality faster than we might have expected. Devices will no longer be just peripheral augmentations, but be inextricably intertwined with human lives.

Hope this would be an interesting read for you!

Learn Windows Azure Cloud Services from Beginning: a Mini Online Course

2013-02-21T19:00:00.001-08:00

I just finished publishing 7 episodes of a new video series, Windows Azure Cloud Services. This series is a mini online course that teaches you Windows Azure Cloud Services from beginning. We'll start our cloud journey by setting up development environment, and then continue to explore some fundamental concepts of Windows Azure Cloud Services. The series builds a solid foundation for you to create highly-available, scalable applications and services using Windows Azure's rich PaaS environment, and to deliver great SaaS solutions to customers anywhere around the world.

There are still more episodes to come, but the first 7 episodes cover all the basic concepts, tools, and procedures you need to know to get started. Here’s the list of episodes:

Hope you enjoy the series. If you have any feedbacks, please leave comments on the posts, or send me a message to @HaishiBai2010.

Windows Azure Active Directory (current) resource list

2013-02-16T13:50:00.001-08:00

This post is a survey of currently available Windows Azure Active Directory resources. I’ve listed links to various concepts, tools, and scenarios. Many of the links, unsurprisingly, point to Vittorio's blog. I hope this post can provide a little help for you to sort out WA AD resources.

WA AD is under active development and things will probably change. I intend to keep this post refreshed once a while, with preview features clearly marked. If you’ve spot a mistake or out-of-date info, or have anything you want to add to this list, please let me know!

Last updated: February 19, 2013

Concepts

Introduction
http://www.windowsazure.com/en-us/home/features/identity/
MSDN
http://msdn.microsoft.com/en-us/library/windowsazure/jj673460.aspx
Windows Azure Active Directory is not a FP. If anything, it is an IP; or rather, it defines an IPs space.
http://blogs.msdn.com/b/vbertocci/archive/2013/02/08/multitenant-sts-and-token-validation.aspx
- Tokens from Windows Azure AD are all signed with the same certificate for every tenant

Tooling for Common Tasks

To provision a new Windows Azure Active Directory Tenant

If you are an Office365 customer, you already have one.
Use this link to set up one tenant for free as part of our developer preview program. [PREVIEW!]

Scenarios

How to provision a Windows Azure Active Directory Tenant as an Identity Provider in an ACS Namespace

http://blogs.msdn.com/b/vbertocci/archive/2012/11/07/provisioning-a-directory-tenant-as-an-identity-provider-in-an-acs-namespace.aspx

Integrating Multi-Tenant Cloud Applications with Windows Azure Active Directory

http://www.windowsazure.com/en-us/develop/net/tutorials/multitenant-apps-for-active-directory/ [Preview]

To use Windows Azure Active Directory as a trusted IP of a Windows Azure Web Site

Use Windows Azure Authentication.
Use Identity and Access Tools to configure RP. Beware of cookie protection method constrains:
http://blogs.msdn.com/b/vbertocci/archive/2013/01/28/running-wif-based-apps-in-windows-azure-web-sites.aspx

To use Windows Azure Active Directory as a trusted IP of a Windows Azure Cloud Service

Use Identity and Access Tools to configure RP. Create service principal on Windows Azure Active Directory tenant.

How to get role and group membership claims for users signing in via Windows Azure Active Directory

http://blogs.msdn.com/b/vbertocci/archive/2013/01/22/group-amp-role-claims-use-the-graph-api-to-get-back-isinrole-and-authorize-in-windows-azure-ad-apps.aspx

Delegation without ActAs token

Both frontend and backend belongs to the same owner/realm.
Bearer tokens are allowed.
The risk of token being stolen across layers is understood and managed.

http://blogs.msdn.com/b/vbertocci/archive/2013/01/09/using-the-jwt-handler-for-implementing-poor-man-s-delegation-actas.aspx

Windows Store app accessing Web PI

http://blogs.msdn.com/b/vbertocci/archive/2013/02/18/use-the-jwt-handler-to-write-a-authorization-server-in-3-lines-of-code-see-it-in-action-with-a-windows-store-app-client-amp-web-api-pr.aspx

Claims issued by a Windows Azure Active Directory Tenant

This is not a complete list. It contains only claims that are known to be provided

http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name (UPN of the user – “user@domain”, see Vittorio's blog)
http://schemas.xmlsoap.org/claims/FirstName

Common URLs

Active Directory tenant federation metadata
https://accounts.accescontrol.windows.net/[tenant (???.onmicrosoft.com)]/FederationMetadata/2007-06/FederationMetadata.xml
ws-Federation endpoint
https://accounts.accesscontrol.windows.net/[tenant id]/v2/wsfederation (you can get tenant id from the metadata above - entityID)

New features in Service Bus Preview Library (January 2013) – Epilogue: Message Browse

2013-02-04T16:10:00.001-08:00

[This post series is based on preview features that are subject to change]

This is an epilogue of my 3-part blog series on the recently released Service Bus preview features.

part 1: Message Pump

part 2: Auto-expiration

part 3: Shared Access Authorization

Why an epilogue?

I’ve missed another feature in the preview library – message browse, which allows you to iterate through the messages without blocking any receivers. But, before I get to that, I have to clarify one thing regarding what I said in the Message Pump post. When you call Client.Receive() method, you can specify a timeout of any duration to wait for a message to arrive. This is not a long-polling, but it does allow you to specify a timeout period before the method returns. So, the price comparison in the original post isn’t that relevant after all. However, the gist of that post is never about price reduction, but about how Message Pump enables event-driven programming model, which is really natural to client applications, especially rich clients. With that said, let’s move on to the main topic: Message Browse.

Message Browse

Message browse can be useful in scenarios where you need to provide monitoring or tracing capabilities without affecting existing clients. For example, you can examine top 10 messages from dead-lettered queue. Or, you can provide live monitoring of pending jobs (such as if a job related to a specific customer has appeared on the queue). Here I’ll just present a very simple Console application as an example:

static void Main(string[] args)
{
    var conString = "[SB Connection String]";
    var queueName = "workqueue";
    QueueClient sender = QueueClient.CreateFromConnectionString(conString, queueName);
    for (int i = 0; i < 10; i++)
        sender.Send(new BrokeredMessage(string.Format("Message {0}", i+1)));
    QueueClient receiver = QueueClient.CreateFromConnectionString(conString, queueName, 
        ReceiveMode.ReceiveAndDelete);

    BrokeredMessage message = null;
    do
    {
        message = receiver.Peek();
        if (message != null)
            Console.WriteLine("Message found: " + message.GetBody<string>());
    } while (message != null);

    Console.ReadLine();

    message = null;

    do
    {
        message = receiver.Receive();
        if (message != null)
            Console.WriteLine("Message received: " + message.GetBody<string>());
    } while (message != null);

    Console.ReadLine();
}

The above code sends 10 messages to a queue, browses the messages by using the Peek() method (highlighted yellow), and then retrieves the messages using the Receive() method (highlighted green). Because the Peek() method doesn’t remove any messages from the queue, Recevie() calls are not affected afterwards and all the messages can be received.

Send us your feedbacks!

I hope you’ve learned some useful information from this blog series. And I know the production team is eager to hear from you. Are the features useful in your scenarios? What are we missing? Where do we need to improve? The whole idea of the preview library is to get your feedbacks so that we can make the final product better. So, if you have any feedbacks, either positive or negative, please don’t hesitate to send them my way, and I’ll make sure they are heard by the team. You can either leave comments to this blog, or send tweets to @HaishiBai2010. Thank you for reading!

New features in Service Bus Preview Library (January 2013) – 3: Queue/Subscription Shared Access Authorization

2013-01-29T19:44:00.001-08:00

This is the last part of my 3-part blog series on the recently released Service Bus preview features.

[This post series is based on preview features that are subject to change]

part 1: Message Pump

part 2: Auto-expiration

Queue/Subscription Shared Access Authorization

Service Bus uses Windows Azure AD Access Control (a.k.a. Access Control Service or ACS) for authentication. And it uses claims generated by ACS for authorizations. For each Service Bus namespace, there’s a corresponding ACS namespace of the same name (suffixed with “-sb”). In this ACS namespace, there’s a “owner” server identity, for which ACS issues three net.windows.servicebus.action claims with values Manage, Listen, and Send. These claims are in turn used by Service Bus to authorize user for different operations. Because “owner” comes with all three claims, he is entitled to perform any operations that are allowed by Service Bus. I wrote a blog article talking about why it’s a good practice to create additional server identities with minimum access rights - these additional identities can only perform a limited set of operations that are designated to them.

However, there are still two problems: first, by default the security scope of these claims are global to the namespace. For example, once a user is granted Listen access, he can listen to any queues and subscriptions. If you want granular access control at entity level, you’ll need to create additional Relying Parties with entity paths as realms, and a “longest prefix match” policy is applied when authorization is performed (see this doc). Second, identity management is cumbersome as you have to go through ACS and manage service identities. With Service Bus preview features, you’ll be able to create authorization rules by users or clients. And because you can apply multiple authorization rules, you can manage end user access rights separately even if they share the same queue or subscription. This design gives you great flexibility in managing access rights.

Now let’s walk through a sample to see how that works. In this scenario I’ll create two queues for three users: Tom, Jack, and an administrator. Tom has send/listen access to the first queue. Jack has send/listen access to the second queue. In addition, he can also receive messages from the first queue. The administrator can manage both queues. Their access rights are summarized in the following table:

User	Queue 1	Queue 2
Tom	Listen/Send	No access
Jack	Listen only	Listen/Send
Admin	Manage	Manage

Create a new Windows Console application.
Install the preview NuGet package:
```
install-package ServiceBus.Preview
```

Implement the Main() method:

static void Main(string[] args)
{
  var queuePath1 = "sasqueue1";
  var queuePath2 = "sasqueue2";

  NamespaceManager nm = new NamespaceManager(
    ServiceBusEnvironment.CreateServiceUri("https", "[Your SB namespace]", string.Empty),
    TokenProvider.CreateSharedSecretTokenProvider("owner", "[Your secret key]"));

  QueueDescription desc1 = new QueueDescription(queuePath1);
  QueueDescription desc2 = new QueueDescription(queuePath2);

  desc1.Authorization.Add(new SharedAccessAuthorizationRule("ForTom", "pass@word1", new AccessRights[] { AccessRights.Listen, AccessRights.Send }));
            
  desc2.Authorization.Add(new SharedAccessAuthorizationRule("ForJack", "pass@word2", new AccessRights[] { AccessRights.Listen, AccessRights.Send }));
  desc2.Authorization.Add(new SharedAccessAuthorizationRule("ForJack", "pass@word2", new AccessRights[] { AccessRights.Listen }));
            
  desc1.Authorization.Add(new SharedAccessAuthorizationRule("ForAdmin", "pass@word3", new AccessRights[] { AccessRights.Manage }));
  desc2.Authorization.Add(new SharedAccessAuthorizationRule("ForAdmin", "pass@word3", new AccessRights[] { AccessRights.Manage }));

  nm.CreateQueue(desc1);
  nm.CreateQueue(desc2);
}

Okay, that’s not the most exciting code. But it should be fairly easy to understand. For each queue I’m attaching multiple SharedAccessAuthorizationRule, in which I can specify the shared key, as well as assigned rights associated with the rule. And I have implemented desired security policy with few lines of code. These entity-level keys have several advantages (comparing to the two problems stated above): first, they provide fine-granular user access control. Second, they are easier to manage. And third, when they are compromised, the damage can be constrained within the scope of affected entity or even affected user only.

This concludes my 3-part blog series on Service Bus preview features. Many thanks to Abhishek Lal for help and guidance on the way, and to the friends on Twitter who helped to get the words out. Thank you for reading!

New features in Service Bus Preview Library (January 2013) – 2: Auto-expiration

2013-01-27T00:56:00.001-08:00

This is the second part of my blog series on the recently released Service Bus preview features.

[This post series is based on preview features that are subject to change]

part 1: Message Pump

Auto-Expiration

There’s a new property, AutoDeleteOnIdle, added to Service Bus entity descriptions such as QueueDescription, TopicDescription, and SubscriptionDescription. When this property is set to a TimeSpan (from 1 minute to TimeSpan.MaxValue), the corresponding entity will be automatically deleted after being inactive during the given time period. AutoDeleteOnIdle attribute can make entity managements a lot easier in many cases. For instance, a chat server won’t need to monitor chat room activities to clean up topics and subscriptions it may have created for a chat room – the entities clean themselves up when the chat room becomes inactive. Now let’s walk through a sample to see how this property works.

Create a new Windows Console application.
Install the preview NuGet package:
```
install-package ServiceBus.Preview
```

Implement the Main() method:

static void Main(string[] args)
{
  var queuePath = "autodeletequeue";
            
  NamespaceManager nm = new NamespaceManager(
    ServiceBusEnvironment.CreateServiceUri("https", "[Your SB namespace]", string.Empty),
    TokenProvider.CreateSharedSecretTokenProvider("owner", "[Your secret key]"));
           
  QueueDescription queue = new QueueDescription(queuePath);
  queue.AutoDeleteOnIdle = TimeSpan.FromMinutes(1);
            
  Console.WriteLine("Creating queue...");
  nm.CreateQueue(queue);
  Console.WriteLine("Queue created!");
  
  DateTime creation = DateTime.Now;
  while (nm.QueueExists(queuePath))
  {
    Console.WriteLine("Queue exists after: " + (DateTime.Now - creation).TotalSeconds.ToString("#.#") + " seconds");
    Thread.Sleep(1000);
  }
            
  Console.WriteLine("Queue removed!");
            
  Console.ReadLine();
}

The code is very straightforward – it creates a queue with the AutoDeleteOnIdle property set to 1 minute. And it waits till the queue disappears. The following is a screenshot of the program when I ran it in a coffee shop (whose Internet isn’t that great):

As you can see the queue disappears in about 2 minutes. I tried the program several times and the queue went away in about 1.5 to 3 minutes (though the AutoDeleteOnIdel is set to 1 minute). This should not be a concern as in this case we are leaving the system to automatically recycle the entities for us. The entities being removed a bit later isn’t unacceptable. On the other hand, we need to have a better understanding of what “Idle” means. Does it count as “idle” if there are no messages passing through the pipeline, or there has to be absolutely no activities? It turned out, “Idle” means no attached clients are trying to send or receive messages. Let’s modify the Main() method a little:

...
QueueClient client = QueueClient.CreateFromConnectionString("[Connection String]", queuePath);

DateTime creation = DateTime.Now;
while (nm.QueueExists(queuePath))
{
  client.Receive(TimeSpan.FromSeconds(3));
  Console.WriteLine("Queue exists after: " + (DateTime.Now - creation).TotalSeconds.ToString("#.#") + " seconds");
  Thread.Sleep(1000);
}
...

The Receive() calls make the queue active, though they don’t get any messages back. Now consider the following code:

...
QueueClient client = QueueClient.CreateFromConnectionString("[Connection String]", queuePath);
client.OnMessage((m) => {}, 1);
            
DateTime creation = DateTime.Now;
while (nm.QueueExists(queuePath))
{
  Console.Write("Queue exists after: " + (DateTime.Now - creation).TotalSeconds.ToString("#.#") + " seconds");
  Thread.Sleep(1000);
}
...

In this version, I’m not calling Receive() method anymore. However, because I subscribed to OnMessage event, the queue is kept alive. If you want to release the queue in this case, you’ll need to close the attached client(s) by calling Close() method on QueueClient class.

And this concludes the second part of this series. I hope this helps. Thank you reading!

New features in Service Bus Preview Library (January 2013) - 1: Message Pump

2013-01-25T15:31:00.001-08:00

Recently Microsoft has announced the new Windows Azure Service Bus Push Notification Hubs. And many samples and videos have been posted on the new feature. To support Notification Hubs, a new Service Bus previews features library (Microsoft.ServiceBus.Preview.dll) has been released to NuGet gallery. In this series of post I’ll drill down to several other cool new features and important enhancements contained in this library.

[This post series is based on preview features that are subject to change]

Message Pump

Up until now, if you want to receive messages from a Windows Azure Service Bus queue or topic/subscription, you need to periodically poll the queue or the subscription asking for new messages. The following code should look quite familiar:

while (!IsStopped)
{
    ...
    BrokeredMessage receivedMessage = null;
    receivedMessage = Client.Receive();

     if (receivedMessage != null)
    {
        ...
        receivedMessage.Complete();
    }
    ...    
    Thread.Sleep(10000);
}

Actually, the above code is a simplified version of auto-generated code when you used “Worker Role with Service Queue” template to add a new Worker Role. This pattern works well in this case because you do need a loop or other blocking wait in your Run() method to keep your role instances running. However, there are a couple of problems with this pattern. First, the Thread.Sleep() calls cause unnecessary delays in the system – the above code can only respond to at most one message every ten seconds. This kind of throughputs is unacceptable to many systems. Of course we can reduce the sleep interval, let’s say to get it down to 1 second. This makes the system more responsive, but it creases number of service calls by 10 times. Polling at a 1 second interval generates 86,400 billable messages (60 * 60 * 24) per day, even if most of them are NULL messages. That doesn’t cost much – at the price of $0.01 per 10,000 billable messages it translates to 8.64 cents per day. However that IS a lot of service calls. Second, in some applications, especially client applications, event-driving programming model is often preferred. Service Bus preview features changes all these. Underneath it uses long-polling so that you don’t occur service transactions as often. And you get immediate feedbacks when a new message shows up in the pipeline. For instance, let’s say if default long-polling timeout is 1 minute, the number of billable messages reduces to 1,440 (60 * 24) per day. That’s quite a improvement in terms of reducing number of service calls. In addition, the preview library supports event-driven model instead of polling - you can simply wait for OnMessage events.

The following is a walkthrough of using the preview library. The walkthrough uses a simple WPF application that allows you to send and receive messages.

Create a new WFP application.
Install the preview NuGet package:
```
install-package ServiceBus.Preview
```
Get a minimum UI in place:
```
<Window x:Class="EventPumpWPF.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="Message-Driven Messaging" Height="350" Width="525" FontSize="18">
        <StackPanel Grid.Column="0">
            <TextBox x:Name="messageText" />
            <Button x:Name="sendMessage" Click="sendMessage_Click_1">Send</Button>
        <ListBox x:Name="messageList"/>
    </StackPanel>
</Window>
```

Modify the code-behind:

public partial class MainWindow : Window
    {
        const string conString = "[SB connection string]";
        const string queueName = "workqueue";
        QueueClient mSender, mReceiver; 
        
        public MainWindow()
        {
            InitializeComponent();
            mSender = QueueClient.CreateFromConnectionString(conString, queueName);
            mReceiver = QueueClient.CreateFromConnectionString(conString, queueName, ReceiveMode.ReceiveAndDelete);

            mSender.OnMessage((m) =>
            {
                messageList.Dispatcher.Invoke(() =>
                    {
                        messageList.Items.Add(m.GetBody<string>());
                    });
            }, 1);
        }

        private void sendMessage_Click_1(object sender, RoutedEventArgs e)
        {
            mSender.Send(new BrokeredMessage(messageText.Text));
            messageText.Text = "";
        }
    }

And that’s all! The only line that is new is highlighted – very simple and very straightforward.

If you’ve observed closely, you might notice there’s a second parameter (highlight in green) to OnMessage() method. This method controls how many concurrent calls to the callback (the first parameter) can occur. To illustrate the effect of this parameter, let’s modify the code a little.

First, we add a randomizer to MainWindow class:
```
Random rand = new Random();
```
And we’ll update or message handler to add a random sleep. This is to simulate fluctuations in processing time:
```
mSender.OnMessage((m) =>
{
    Thread.Sleep(rand.Next(1000, 3000));
    messageList.Dispatcher.Invoke(() =>
    {
        messageList.Items.Add(m.GetBody<string>());
    });
}, 1);
```
Finally, we change the sending code to send 10 messages instead of 1:
```
for (int i = 0; i < 10; i++)
{
    mSender.Send(new BrokeredMessage(messageText.Text + i.ToString()));
}
```
Now launch the program and send a message “m”, which morphs into ten messages. The code takes a while to execute because of the random sleeps and there’s only a single entry is allowed to the callback. But because the single-entrance limit, you eventually get all messages back in-order.
Now modify the second parameter (highlighted in green) to 10. Run the app again. Now the code takes shorter time to execute because the callback can be invoked multiple times at the same time. But the message display may be out-of-order:

There you go. A very cool addition to Service Bus provided by Service Bus preview features. The feature is very useful when you want to use event-driven programming. I hope this helps. Thank you for reading!

Dealing with long-running jobs

2012-12-21T18:04:00.001-08:00

Many systems need to deal with long-running jobs. Long-running jobs are slow, and they often take up lots of system resources. In addition, long-running jobs raise additional requirements such as progress report/monitoring, cancellation, failover, and prioritization, etc. For such systems, it’s very important to use a scalable, efficient and reliable architecture to ensure system performance. This blog article explores some of the common patterns of such a system and provides some guidance on implementing a Request-Acknowledge-Push pattern using Windows Azure Cloud Services.

[Now this article has a corresponding Channel 9 episode. Check it out at: http://channel9.msdn.com/Series/Cloud-Patterns/Episode-1-Long-running-tasks-Request-Acknowledge-Push-pattern]

Patterns

There are several patterns that can be used by clients to submit long-running jobs to server. However the idea behind these patterns is the same, which is to avoid temporal coupling between the client and the actual executors.

Request-Acknowledge

This is probably the simplest pattern, in which a client sends a job to a server, and the server acknowledges the submission. Then, the client and the server move on with their own businesses. If we map this pattern to a Windows Azure Cloud Service, we get the following diagram. As a matter of fact, if you create a new Cloud Service using a Web Role and a Worker Role with Service Bus Queue template, this is (almost) exactly what you get – the only missing part you need to add is the web role submitting a job to the Service Bus queue.

Let’s imagine a user submits an order to a e-commerce site. From the perspective of the end user, the process (or the workflow) of ordering an item completes when the order has been successfully submitted. Although the new order triggers a long process on the service side, the user doesn’t have to wait for that process to complete. With the expectation that an order often takes days (or even longer) to process, the user will leave with the expectation that his/her order will be eventually processed at a later time. Note that in this case the Web Role made the promise based on its faith in Service Bus, which guarantees delivery of the job when there’s an appropriate recipient asking for it. This recipient can be a Worker Role in the same Cloud Service, or it can be any legitimate Service Bus queue client running at anywhere – on cloud or on-premises, it doesn’t matter.

Request-Acknowledge-Poll

In many cases a client does care the progress of a submitted job and wants to get immediate feedbacks on job progress – how many times have you been impatiently watching the file upload/download bars? However, with HTTP protocol, the server sending acknowledgement completes the HTTP request. To get progress updates, the client will need to send additional requests to poll for updates. To achieve this we need two additional pieces – one additional service for clients to query for updates, and a job status repository so that the service can query for job statuses. The following diagram should be self-explanatory – the Web Role provides an additional method for clients to query for job status updates, and an extra repository is added to save job statuses. Note that because submitting a job and polling for progress are two separate HTTP requests, you’ll need to provide some mechanism, such as a GUID job id, to correlate polling requests with corresponding submissions. This id, sometimes referred as a (job) token, is usually generated on the server and is provided to the client when server sends the job submission acknowledgement.

Request-Acknowledge-Push

Can we eliminate the need for an extra database? Wouldn’t it be nice if we can directly push the job statues back to the client? We can use Request-Acknowledge-Push pattern for this purpose. This pattern is a variation of Request-Acknowledge-Callback, where the job executor (the Worker Role) proactively calls into a callback service, such as a SMTP service, to report job statuses. However, as it’s hard for a web client to host a callback service, I adapted the pattern to allow job executors to push job statuses to an arbitrary recipient, including the client. Particularly, the following diagram shows such an implementation using SignalR to push job statuses back to the web client. In this implementation, the Web Role hosts a SignalR hub, which the Worker Role connects to (via a hub proxy) and reports job updates. And then, the hub pushes the updates to attached client(s).

Implementation of the Request-Acknowledge-Push Pattern

In this part I’ll briefly go through an implementation of Request-Acknowledge-Push pattern using SignalR. Instead of a complete walkthrough, I’ll just mention some key pieces of the code. If you are interested in getting the complete code, please contact me and I’ll put the code somewhere.

The first thing we need is a type that represents a job:

public class Job
{
  public string ClientId { get; set; }
  public string JobId { get; set; }
  public string Content { get; set; }
}

In above definition I embedded the client id into the Job type itself. This might raise some eyebrows as it’s debatable if the originating client logically belongs to the Job type. I chose to do this because in a push scenario, we need to be able to associate a job with originating client so that we know which client to send updates to. We can do this using a separate lookup, of course. But that requires some extra work as we need to make sure the reliability and availability of this lookup mechanism – that’s too much trouble.

The Worker Role creates a SignalR proxy and sends progress updates back to the Web Role, as depicted in above diagram:

HubConnection connection = new HubConnection("http://127.0.0.1:81");
IHubProxy proxy = connection.CreateHubProxy("ProgressNotifier");
connection.Start().Wait();

...

while (!IsStopped)
{
   BrokeredMessage receivedMessage = null;
   receivedMessage = Client.Receive();

  if (receivedMessage != null)
  {
     Job job = receivedMessage.GetBody<Job>();
     ...
     proxy.Invoke("ProgressUpdated", job.ClientId, job.JobId, 35);
     ...
     proxy.Invoke("ProgressUpdated", job.ClientId, job.JobId, 100);
     receivedMessage.Complete();
  }
...

The Web Role provides implementation of the SignalR hub (note how we identify which client to talk to using client id embedded in Job type):

public class ProgressNotifier : Hub
{
   public void ProgressUpdated(string clientId, string jobId, int progress)
   {
      Clients.Client(clientId).progressUpdated(jobId, progress);
   }
}

And the method to submit a Job:

[HttpGet]
public string ReqAck(string clientId)
{
   ...
   string jobId = Guid.NewGuid().ToString();
   client.Send(new BrokeredMessage(new Job
            {
                ClientId = clientId,
                JobId = jobId,
                Content = "some workload"
            }));
    return jobId;
}

On client side, SignalR connection Id is recorded when a connection is established:

var clientId;

$(function () {
        hub = $.signalR.progressNotifier;
        hub.on("progressUpdated", function (jobId, progress) {
           ...
        });
        $.signalR.hub.start().done(function () {
            clientId = $.connection.hub.id;
        });
        ...
});

And it’s sent to the server when the client submits a job:

var viewModel = {
  reqAck: function () {
    $.getJSON('/api/Job/ReqAck?type=a&clientId=' + clientId, function (json) {
      ...
    });
  },
  ...

Dealing with Cancelation

It’s often a good idea to support job cancellation for long-running jobs. Some systems support only explicit cancelation, which means user has to explicitly cancel a job. Some other systems automatically cancel a job when the client that has submitted the job disconnects. There’s also a third type of cancellation where a user re-submits a job (possibly with modifications) that causes previously submitted jobs cancelled. When cancellation happens, the job to be cancelled can be any of the following states: waiting in queue, being executed, and completed or faulted. There’s nothing much we can do for the last case – the job has been completed before we can cancel it. Depends on your system requirements, you can allow user to undo a job, which is just another job to be pumped into the system.

Stop list

When a job is waiting in the queue, we can simply remove it from the queue. However, most queue services only allow sequential access of items, which means you’ll have to scan the queue to find and remove the job. This scanning process competes with job executors who also poll the same queue, hence it may briefly block all the executors as it locks the queue items for examination. Moreover, removing a job from waiting queue doesn’t handle the case that the job is already being processed. You’ll also need to locate and inform the executor who’s running the job to stop processing.

A better method is to create a separate stop list, which holds indexes to the jobs to be cancelled. To cancel a job, you simply add the job id into this list. The executors will check to see if a job is in this list before running it. The executors are also free to check this list whenever a job can be safely cancelled. So the stop list handles both waiting jobs and active jobs. Obviously, the stop list has to be scalable and available to all running executors, and it should be cleaned once a job is successfully cancelled.

Disconnected Client

With ASP.Net, you can use System.Web.HttpResponse.IsClientConnected to check if the client who initiated the request is still connected. Note this method only works when you use IIS instead of development web server. If you are using SignalR, you can also respond to OnDisconnected event on your hub and add the job originated from the client into the stop list.

Other Concerns

To avoid this post getting too long, here I’ll just briefly mention some of other concerns you’ll need to address – they are also important, I’m simply running out of time here. If you are interested in any of the following topics, please comment and I’ll try to expand the post later.

Dealing with Failures

There are three main types of failures: executor failing to process a job, timeout, and system-wide failure. Workflow Foundation provides many powerful features such as failover, recovery from system failure, workflow cancellation, etc. If the demand of fault tolerance is high, you may consider using Workflow Foundation to take advantage of these features.

Dealing with Duplicated Requests

There are two strategies of handling duplicated requests. One is to discard duplicated requests, the other is to process only the newest requests. Choosing which strategy depends on your system requirements. Windows Azure Service Bus supports detection of duplicated messages. You may want to leverage the feature when dealing with duplicated requests.

Dealing with Priorities

It’s very common that jobs come with associated priorities. Intuitively, if the waiting queue is a priority queue, handling job priority is relatively easy – when an executor is free to get a new job it will get a job with highest priority. At the time when this post is written, there’s no built-in support of priority queue in Windows Azure Service Bus yet, but there are several implementations from the community. An alternative way is to have multiple queues, each associated with different priority levels such as “high”, “medium”, and “low”. A free executor will always try to get job from the queue with highest priority before it moves on to the next queue.

Dealing with Heavy Loads

In above patterns we use separate Worker Roles as job executors. Because you can scale different roles independently, you can easily have multiple Worker Roles to act as competing consumers of the job queue to load balance the jobs, or to handle unexpected spikes in workloads. A work role places a temporary lock on the queued item it works on using PeekLock, and it removes the item from the queue by marking the item Complete (or Abandon in case of errors). Read this windowsazure.com article for more details.

Depends on the loads of your site, you may also consider leveraging ASP.NET MVC4 asynchronous methods to achieve high concurrency using a single Web Role. One common problem for ASP.NET to support high concurrency is thread starvation. With asynchronous methods, you can achieve much higher concurrency because the thread pool is not exhausted as quickly as when synchronous methods are used. Read reference material #4 at the end of this post for more details.

Last but not least, many long-running processes are I/O-bound. I/O operations may become a bottleneck of the system when workload increases. And when you use an additional repository for job statuses, the repository itself may become a bottleneck as well. Caching is often an effective way to reduce I/O operations. You should consider caching when possible.

Summary

In summary:

Consider Request-Acknowledge-Push pattern using SignalR
Consider ASP.NET asynchronous methods for higher throughputs
Consider Stop List for job cancelation
Consider Workflow Foundation if you need advanced features such as failover

References & extended readings

Some concepts, patterns, code snippets, and diagrams in this article are derived from the following resources:

Service Design Patterns: Fundamental Design Solutions for SOAP/WSDL and RESTful Web Services, Robert Daigeneau et. al., 2011
BUILD 2012 Day 2 Keynote demonstrations
windowazure.com: Implementing Reliable Message Receive Loops
asp.net: Using Asynchronous Methods in ASP.NET MVC 4
Enterprise Integration Patterns: Designing, Building and Deploying Messaging Solutions, Gregor Hohpe et. al., 2003

Complete walkthrough: Setting up a ADFS 2.0 Server on Windows Azure IaaS and Configuring it as an Identity Provider in Windows Azure ACS

2012-11-20T20:51:00.001-08:00

I’ve done this several times but never got a chance to document all the steps. Since now I’ve got another opportunity to do this yet again, I’ll document all the steps necessary, starting from scratch, to configure an ADFS 2.0 server on Windows Azure IaaS, and to configure it as an Identity Provider in Windows Azure ACS. Then we’ll use ACS to protect our web application that uses role-based security. Although I’m stripping down the steps to the bare minimum and cutting some corners , this will still be a long post, so please bear with me. Hopeful this could be your one-stop reference if you have such a task at hand. Here we go:

Create a Virtual Network

Since we are starting from scratch, we’ll start from a Virtual Network. Then, on this Virtual Network, we’ll set up our all-in-one AD forest with a single server what is both a Domain Controller (DC) and an ADFS 2.0 server. The reason we put this server on a Virtual Network is to ensure the server get a local static IP (or an ever-lasting dynamic) address.

Log on to Windows Azure Management Portal.
Add a new Virtual Network. In the following sample, I named my Virtual Network haishivn, with address space 192.168.0.0 – 192.168.15.255.
Click CREATE A VIRTUAL NETWORK link to create the network.

Provision the Virtual Machine

In Windows Azure Management Portal, select NEW->COMPUTE->VIRTUAL MACHINE->FROM GALLERY.
Select Windows Server 2012, October 2012, and then click Next arrow.
On next screen, enter the name of your virtual machine, enter administrator passport, pick the size you want to use, and then click Next arrow.
On next screen, enter a DNS name for your server, and make sure to put your server on the Virtual Network you just created.
(Note in the following screenshot I’m using haishidc, while in later steps I’m using haishidc2 as I messed up something in the first run and had to start over. So, please consider haishidc and haishidc2 the same)
On last screen, leave availability set empty for now. Complete the wizard.
Once the virtual machine is created, click on the name of the machine and select ENDPOINT tab.
Click ADD ENDPOINT icon on the bottom toolbar.
In ADD ENDPOINT dialog, click next arrow to continue.
Add port 443 as shown in the following screenshot:
Similarly, add port 80.

Set up The Virtual Machine

Once the virtual machine is provisioned, we are ready to set up our cozy Active Directory with one member, which will be the Domain Controller as well as the ADFS 2.0 Server.

RDP into the virtual machine you just created. Wait patiently till Server Manager shows up.
Go to Local Server tab, scroll down to the ROLES AND FEATURES section, then click TASKS->Add Roles and Features.
In Add Roles and Features Wizard, click Next to continue.
On next screen, keep Role-based or feature-based installation checked, click Next to continue.
On Server selection screen, accept default settings and click Next.
On Server Roles screen, check Active Directory Domain Service. This will pop up a dialog prompting to enable required features. Click Add Features to continue.
Check Active Directory Federation Services. Again, click Add Features in the pop-up to add required features.
Click Next all the way till the end of the wizard workflow, accepting all default settings.
Click Install to continue. Once installation completes, click Close to close the wizard.

Configure AD and Domain controller

Now you’ll see a warning icon reminding you some additional configurations are needed
Click on the icon and click on the second item, which is Promote this server to a domain controller.
In Active Directory Domain Services Configuration Wizard, select Add a new forest, enter cloudapp.net as Root domain name, and then click Next to continue. Wait, what? How come our root domain name is cloudapp.net? Actually it doesn’t matter that much – you can call it microsoft.com if you want. However, using cloudapp.net saves us a little trouble when we try to use self-issued certificates in later steps. From the perspective of outside word, the ADFS server who issues the token will be [your virtual machine name].cloudapp.net. In the following steps, we’ll use IIS to generate a self-issued cert that matches with this DNS name. The goal of this post is to get the infrastructure up and running with minimum efforts. Proper configuration and usage of various certificates deserves another post by itself.
On next screen, provide a DSRM password. Uncheck Domain Name System (DNS) server as we don’t need this capability in our scenario (this is an all-in-one forest anyway). Click Next to continue.
Keep clicking Next till Install button is enabled. Then click Install.
The machine reboots and you loose your RDP connection.

(optional: grab a soda and some snacks. This will take a while)

Create Some Test Accounts

Before we move forward, let’s create a couple of user groups and a couple of test accounts.

Launch Active Directory Users and Computers (Window + Q, then search for “users”).
Right-click on Users node, then select New->Group:
In New Object window, enter Manager as group name, and change Group scope to Domain local:
Follow the same step, create a Staff group.
Right-click on Users node, then select New->User to create a new user:
Set up a password for the user, then finish the wizard. On a test environment, you can disallow password change and make the the password never expire to simplify password management:
Double-click on the user name, and add the user to Manager group:
Create another user, and add the user to Staff group.

Configure SSL Certificate

Now it’s the fun part – to configure ADFS.

Reconnect to the Virtual Machine.
Launch Internet Information Services Manager (Window + Q, then search for “iis”).
Select the server node, and then double-click Server Certificates icon in the center pane.
In the right pane, click on Create Self-Signed Certificate… link. Give a friendly name to the cert, for example haishidc2.cloudapp.net. Click OK. If you open the cert, you can see the cert is issued to [your virtual machine name].cloudapp.net. This is the reason why we used cloudapp.net domain name.

Configure ADFS Server

Go back to Server Manager. Click on the warning icon and select Run the AD FS Management snap-in.
Click on AD FS Federation Server Configuration Wizard link in the center pane.
In AD FS Federation Server Configuration Wizard, leave Create a new Federation Service checked, click Next to continue.
On next screen, keep New federation server farm checked, click Next to continue.
On next screen, You’ll see our self-issued certificate is automatically chosen. Click Next to continue.
On next screen, setup Administrator as the service account. Click Next.
Click Next to complete the wizard.

Provision ACS namespace

If you haven’t done so, you can follow these steps to provision a Windows Azure ACS namespace:

Log on to Windows Azure Management Portal.
At upper-right corner, click on your user name, and then click Previous portal:
This redirects to the old Silverlight portal. Click on Service Bus, Access Control & Caching in the left pane:
Click New icon in the top tool bar to create a new ACS namespace:
Enter a unique ACS namespace name, and click Create Namespace:
Once the namespace is activated. Click on Access Control Service in the top tool bar to manage the ACS namespace.
Click on Application integration link in the left pane. Then copy the URL to ws-Federation metadata. You’ll need the URL in next section.

Configure Trust Relationship with ACS – ADFS Configuration

Now it’s the fun part! Let’s configure ADFS as a trusted Identity Provider of your ACS namespace. The trust relationship is mutual, which means it needs to be configured on both ADFS side and ACS side. From ADFS side, we’ll configure ACS as a trusted relying party. And from ACS side, we’ll configure ADFS as a trusted identity provider. Let’s start with ADFS configuration.

Back in AD FS Management snap-in, click on Required: Add a trusted relying party in the center pane.
In Add Relying Party Trust Wizard, click Start to continue.
Paste in the ACS ws-Federation metadata URL you got from your ACS namespace (see above steps), and click Next to continue:
Keep clicking Next, then finally Close to complete the wizard.
This brings up the claim rules window. Close it for now.
Back in the main window, click on Trust Relationships->Claims Provider Trust node. You’ll see Active Directory listed in the center pane. Right-click and select Edit Claim Rules…
In the Edit Claim Rules for Active Directory dialog, click Add Rule… button.
Select Send Group Membership as a Claim template. Click Next.
On next screen, set the rule name as Role claim. Pick the Manager group using the Browse… button. Pick Role as output claim type. And set claim value to be Manager. Then click Finish. What we are doing here is to generate a Role claim with value Manager for all users in the Manager group in our AD.
Add another rule, and this time select Send LDAP Attribute as Claims template.
Set rule name as Name claim. Pick Active Directory as attribute store, and set up the rule to map Given-Name attribute to Name claim:
Back in the main window, click on Trust Relationships->Relying Party Trusts node. You’ll see your ACS namespace listed in the center pane. Right-click on it and select Edit Claim Rules…
Add a new rule using Pass Through or Filter an Incoming Claim template.
Pass through all Role claims:
Similarly, add another pass-through rule for Name claim.

Now our ADFS server is configured to trust our ACS namespace, and it will issue a Name claim and a Role claim for authenticated users.

Configure Trust Relationship with ACS – ACS Configuration

Now let’s configure the second part of the trust relationship.

Download your ADFS metadata from https://[your virtual machine name].cloudapp.net/FederationMetadata/2007-06/FederationMetadata.xml (hint: use Chrome or Firefox to make download easier – ignore the certificate warnings).
In ACS management portal, click on Identity providers link in the left pane. Then click on Add link:
On next screen, select WS-Federation identity provider, then click Next:
On the next screen, enter a display name for the identity provider. Browse to the metadata file you’ve downloaded in step 1. Finally, set up a login link text, and click Save to finish.

Test with a web site

Now everything is up and running, let’s put them into work.

Launch Visual Studio 2012.
Create a new Web application using ASP.NET MVC 4 Web Application – Internet Application template.
Open _LoginPartial.cshtml and remove the following part. We don’t have required claims (name identifier and provider) to make this part work. For simplicity let’s just remove it
```
 @using (Html.BeginForm("LogOff", "Account", FormMethod.Post, new { id = "logoutForm" })) {
     @Html.AntiForgeryToken()
     <a href="javascript:document.getElementById('logoutForm').submit()">Log off</a>
}
```
Right-click on the project and select Access and Identity… menu. Not seeing the menu? Probably you don’t have the awesome extension installed yet. You can download it here.
In the Identity and Access wizard, select Use the Windows Azure Access Control Service option. If you haven’t used the wizard before, you’ll need to enter your ACS namespace name and management key. You can also click on the (Change…) link to switch to a different namespace if needed:
Where to get the ACS namespace management key, you ask? In ACS namespace management portal, click on the Management service link in the left pane, then click on ManagementClient link in the center pane:
On next screen, click on Symmetric Key link, and finally in the next screen, copy the Key field.
Where were we… right, the Identity and Access wizard. Once you’ve entered your ACS information correctly, you’ll see the list of trusted identity providers populated. Select identity provider(s) you want to use. In this case we’ll select the single ADFS provider:
Click OK to complete the wizard – that was easy, wasn’t it?
Now launch the application by pressing F5.
You’ll see a certificate warning – that’s a good sign! This means the redirection to ADFS is working and the browser is complaining about our self-issued cert. Click Continue to this website to continue.
You’ll be asked to log on to your domain. Type in your credential to log in (make sure you are using the right domain name). You can also use [user]@[domain] format, for example joe@cloudapp.net:
And it works just as designed:

Role-based security

That was exciting, wasn’t it? Now let’s have more fun. In this part we’ll restrict access to the About() method of home controller to Manager role only.

Open HomeController.cs and decorate the About() method:
```
[Authorize(Roles="Manager")]
public ActionResult About()
{
    ViewBag.Message = "Your app description page.";
    return View();
}
```
Launch the app, log in as joe, who’s a Manager. Click on About link. Everything is good.
Now restart the app, log in as a Staff user – access denied! A-xcellent.

Summary

There you go! An Active Directory, a Domain Controller, an ADFS server, all on Windows Azure IaaS. And the ADFS server is configured as a trusted identity provider to our ACS namespace, which in turn provides claim-based authentication to our web application who uses role-based security! That’s really fun!

Bonus Item

Still reading? Thank you! Now you deserve a little bonus. Remember the log on dialog in above test? That’s not very nice looking. Follow this link to learn a little trick to bring up a login form instead.

Walkthrough: Using New Relic to monitor your Windows Azure Cloud Services

2012-11-19T14:18:00.001-08:00

With the release of Windows Azure Store, more and more powerful SaaS solutions can be easily incorporated into Windows Azure’s ecosystem, providing tremendous opportunities for service providers to utilize these new capabilities in their services and applications. In BUILD 2012 Azure keynote we demonstrated how you can use New Relic to instrument and monitor your Cloud Services. In this post I’ll walk you through the configuration steps behind the science that made the demo possible.

Setting up the Cloud Service

In this part we’ll create a brand-new Cloud Service that contains some pages and a couple of Web API methods. Of course you can start with you own service, but if you are trying this for the first time, using a dummy service is probably a better idea.

Create a new Cloud Service with a ASP.Net MVC 4 Web Role (using Internet Application template).
Add a new Web API controller to the Web Role using API controller with empty read/write actions.

Provisioning New Relic service

In this part we’ll provision a new New Relic account using Windows Azure Management Portal.

Log on to Windows Azure Management Portal.
Use NEW->Store to launch Windows Azure Store wizard.
Select New Relic, then click right arrow to continue
Select a package you want to use (you can start with the free package). Pick a name for your subscription, and then click right arrow to continue
After reviewing summary, click Purchase to complete the wizard
After the service is provisioned, you can get your API KEY and APPLICATION KEY by click on CONNECTION INFO icon.

Configuring New Relic Agent

To allow New Relic to collect performance metrics, you’ll need to deploy a New Relic agent along with your service. Fortunately this is very easy using New Relic NuGet package.

Add NewRelicWindowsAzure (New Relic x64 for Windows Azure) NuGet package to the Web Role.
The install wizard will ask for your license key. Paste in the license key you get in step 6 of above section
Then the wizard will ask you for an application name. This name will be used in your New Relic portal to visually identify your application.

After the wizard completes you can see a newrelic.cmd file, which is registered as a startup task that installs NewRelicAgent_x64_{version}.msi to your host machine.

Give it a try!

That’s all you need to do to get started! Now publish your Cloud Service to Windows Azure, click around to generate some requests, and wait for a couple of minutes for your application to show up in New Relic portal. To access the New Relic portal from Windows Azure Management Portal, click on MANAGE icon of your New Relic subscription:

Adding Browser Tracing

New Relic agent reports server-side metrics such as app server response time and throughputs etc. To measure client-side metrics such as client perceived response time, you’ll need to enable browser tracing, which will inject some JavaScript snippets to your web page to report performance data back to New Relic. You can use New Relic API methods, NewRelic.Api.Agent.NewRelic.GetBrowserTimingHeader() and NewRelic.Api.Agent.NewRelic.GetBrowserTimingFooter() to generate these snippets. For example, in my ASP.NET MVC 4 _layout.cshtml page, I added the following calls:

<!DOCTYPE html>
<html lang="en">
    @Html.Raw(NewRelic.Api.Agent.NewRelic.GetBrowserTimingHeader())
    ...
    @Html.Raw(NewRelic.Api.Agent.NewRelic.GetBrowserTimingFooter())
</html>

Because the layout page is shared among all pages, I can trace into all page requests in this way.

Trace Web API

The above method for browser tracing only works for web pages that use _layout.cshtml page. What about API controllers, or WCF services that don’t have a frontend? You can define custom instrumentations easily with New Relic as well. The following is an example to trace Get() method of my API controller:

Add a new file, named CustomInstrumentation.xml, to the root folder of your Web Role project. In the following file, I’m instructing New Relic instrumentation to trace Get method of my APIController defined in my web project
```
<?xml version="1.0" encoding="utf-8" ?>
<extension xmlns="urn:newrelic-extension">
  <instrumentation>
    <tracerFactory>
      <match assemblyName="NewRelicSample.Web" className="NewRelicSample.Web.Controllers.APIController">
        <exactMethodMatcher methodName="Get" />
      </match>
    </tracerFactory>
  </instrumentation>
</extension>
```
Uncomment the following line from newrelic.cmd, and redeploy your service
```
copy /y CustomInstrumentation.xml %NR_HOME%\extensions >> d:\nr.log
```

You can also directly call into New Relic API from any classes, such as a WCF service implementation, to trance performance data, as shown in the following example:

public async Task<HttpResponseMessage> Post()
{
    NewRelic.Api.Agent.NewRelic.SetTransactionName("API", "Video Creation");

    Stopwatch watch = Stopwatch.StartNew();
     … doing stuff …
     watch.Stop();

    NewRelic.Api.Agent.NewRelic.RecordResponseTimeMetric("Video Creation", watch.ElapsedMilliseconds);
     return Request.CreateResponse(HttpStatusCode.Created, video);
}

Screenshots

Here are some screenshots highlighting just a few interesting features of New Relic.

Map view gives you a visual presentation of external service dependencies. In the following sample you can see my service depends on Twitter, Windows Azure Table Storage, as well as Windows Azure ACS. You can drill into each service to get more details as well.

Geographic view gives you an intuitive representation of client perceived response times across the states or around the globe.

Detailed transaction view allows you to drill down all the way to call stacks and even SQL statements so you can easily identify bottlenecks across different application layers.

Have a large system? Trace your Key Transactions in a separate view to keep a close eye on key components of your system.

Summary

In this walkthrough we went through all the steps necessary to utilize New Relic to monitor your Cloud Service performance from both server side and client side. Ready to get started? Follow this link to receive your free Windows Azure trail subscription with free New Relic service offering!

Play fireworks together! Process of building a SignalR sample

2012-11-06T16:47:00.001-08:00

In my BUILD 2012 session on Cloud Services, I mentioned a fireworks sample but didn’t get time to show it. It is an end-to-end sample showcasing how to use SignalR to build massive interactive web applications. As promised, I’m sharing the source, as well as some of “Performance by Design” thinking that I went through building the sample. I hope you can pick up some practical SignalR tricks and tips here as well as some hints on designing an application with performance considerations. I have a version deployed at http://firework.cloudapp.net. This is a two-instance deployment with a Service Bus backplane, so it should hold up for hundreds of concurrent fireworks. The application works on your PCs, Macs (with HTML 5 enabled browsers) as well as mobile devices such as Surface, WP, iPhone and iPad. I haven’t tested on Android devices, though. Note that because the code does intense graphics, your mobile devices may have a hard time keep up with rendering, especially when there are lots of fireworks. My Windows Phone 7 could handle about 500 concurrent fireworks.

The Application

The idea is simple – to allow many users to play virtual fireworks together over the Internet. It’s a single page web application written in JavaScript and HTML 5. The application supports two firework types – simple and complex, but it can be extended by providing additional subclasses of Firework class. In addition, you can pick from several different colors (include a random multi-color option) as well as how trails of sparks are rendered.

The Code

Complete code of the application can be found at GitHub. Here are some brief descriptions that may help you to understand the code. There are comments inline with the code as well.

Everything interesting is in the default Home view (Views/Home/Index.cshtml).
Firework is the base class that implement basic behaviors of a firework. It defines a series of Sparks, which in turn contain a list of Trail points.
FireworkPatternFactory is a class that helps to reduce amount of calculations. See the last section for more details.
SimpleFirework and ComplexFirework are two subclasses of Firework. If you want to implement additional firework types, you’ll need to initialize the Sparks in your constructor and implement updateSparks() to update the sparks. Trails are automatically maintained by the base class.
Rendering routine is the updateCanvas() method, which is set on a timer with 50ms intervals.

SignalR Tricks & Tips

This post is not an introduction to SignalR. Instead, I’ll provide a couple of quick tricks & tips that you can use when implementing your own SignalR application on Windows Azure.

Enable Service Bus backplane

SignalR Service Bus backplane allows you to scale SignalR hubs to multiple instances. You’ll need to enable the backplane if your Cloud Service will have multiple instances.

You’ll need to add a reference to Microsoft ASP.NET SignalR ServiceBus Libraries NuGet package.

Initialize the backplane in Global.asax.cs

private string topicPathPrefix = "firework";

protected void Application_Start()
{
    ...
    GlobalHost.DependencyResolver.UseServiceBus(
                    ConfigurationManager.AppSettings["Microsoft.ServiceBus.ConnectionString"], 
                          partitionCount: 5, 
                          nodeCount: getRoleInstanceCount(), 
                          nodeId: getRoleInstanceNumber(), topicPrefix: topicPathPrefix);
}

private int getRoleInstanceCount()
{
    return RoleEnvironment.CurrentRoleInstance.Role.Instances.Count;
}
private int getRoleInstanceNumber()
{            
    var roleInstanceId = RoleEnvironment.CurrentRoleInstance.Id;
    var li1 = roleInstanceId.LastIndexOf(".");
    var li2 = roleInstanceId.LastIndexOf("_");
    var roleInstanceNo = roleInstanceId.Substring(Math.Max(li1, li2) + 1);
    return Int32.Parse(roleInstanceNo);
}

Note the nodeCount should be consistent with number of your role instances, and nodeId should be unique within the cluster.

Enable WebSockets

At the time when this post is written, WebSockets is not enabled by default on osFamily 3. However you can easily do this via a startup script:

%SystemRoot%\system32\dism.exe /online /enable-feature /featurename:IIS-WebSockets

You don’t have to do this – SignalR will fall back to other means of establishing connections. But WebSockets is preferred. In addition, you need to set runtime context to elevated to make this work. The operation may takes a long time. I’d suggest to make it a background task so that your deployments can go through faster (and continue to work even if the command fails).

Service Bus Connection String

At the time when this post is written, you need to make sure your service bus connection string contains token provider type:

<add key="Microsoft.ServiceBus.ConnectionString" value="Endpoint=sb://[your namespace].servicebus.

         windows.net/;SharedSecretIssuer=owner;SharedSecretValue=[your secret key];provider=SharedSecret" />

Performance by Design

This is a very “busy” application – on one hand it needs to sync its states with hundreds of other clients; and on the other hand it needs to render thousands of elements on the screen at a very fast pace. To ensure performance, during the designing phase I paid extra attention to these two elements: Reduce amount of information that needs to be synced; Reduce amount of calculation. Here are processes of some design decisions I made along the way. I hope this can provide some ideas to help you to design your first interactive web application.

What to sync

iteration 1

When you render a beautiful firework, you need to know the positions of every single spark as well as its trail. If you want to directly sync all the information the amount of traffic will be considerable – each complex firework contains 48 sparks with 240 trail points – that’s near 300 coordinates, along with color of each spark, to be synced. Now multiply that by 100 users, at every 50 milliseconds. That’s craziness. We need a much simpler representation of a firework. And here’s the abstraction I use – a firework is described by a base position (where user has clicked), a base color, a type (simple or complex), a trail type (dot, line, or none), and a phase. Note I’m cheating a little here – for multi-color fireworks I’m not going to sync color of each spark. No one would care about (or event notice) they have different colors - knowing when and how to cheat is important ;). The phase is an interesting concept – basically when a firework is ignited it’s at phase 0, and it goes through all the phases till it burns out. The program is designed in the way that you can paint a firework of any phase at any time – you don’t have to start with phase 0. This allows me to sync firework states at any time.

iteration 2

That’s much better, but can we do even better? Obviously we can compress the data, such as encoding color, type, and trail to a single byte. But there will still be a lot of data to sync. Eventually, I decided to sync the actions instead of sync the states. Generally speaking, when you sync among distributed systems, syncing the states is much more reliable than syncing the actions, because with a single action missing, duplicated, or out of place, the whole synchronization is messed up. However, for this particular application, missing some of the actions – and there’s only one “add” action – is not that terrible. So some clients may miss a firework or two -- not a big deal. And it’s also not critical to make sure all pixels on the screens are in sync either. Hence syncing the actions is a very reasonable choice in this case, and it dramatically reduces the amount of data to be transferred. All clients maintain their states separately, and they try to perform the same actions by best attempts. For a firework application that’s acceptable.

Reduce calculation

trajectory

When fireworks explode we need to calculate trajectories of the sparks. In order to avoid repetitive SIN/COS calculations, I pre-calculated points of an expanding circle and saved all the points in memory. Then, when I need to plot a spark (or its trail) on screen, I simply lookup corresponding values by phases. Acceleration of gravity is kept to a separate pre-calculated array so I can adjust and apply gravity effects independently from the perfect explosion circle.

trail

Drawing a perfect trail of expanding circle (the spark) takes some non-trivial calculations, which is not the price I want to pay. Instead of calculating perfect polygons to compose the trail, I chose to use approximate bands that connect centers of the trail points. This is a much simpler calculation and the result is very reasonable. In the following diagram, the red area shows approximation bands, while the yellow area shows perfect trail polygons. You can see the algorithm is biased vertically so when the sparks fall the trails look more spectacular (again, knowing when and how to cheat is important).

What’s left

A better, stretchable UI.
Explosion effect for complex firework isn’t that nice. Because of the 3D perspective in real life, the center should not be hollow but filled with sparks that are coming toward you. A pre-calculated 3D projection is needed, as well as the spark coordinates to be extended to 3 dimensions. It’s not a terribly hard change but needs a little 3D geometry.

Summary

Here you go, a SingalR firework program that I coded up in about 3 days. I hope you can get some useful tips setting up the Service Bus backplane. If you had some fun playing with the sample that’s even better! And if you are inspired to build much, much better interactive web applications, my goal here is achieved :).

Recipes for Multi-tenant Cloud Service Design – Recipe 2: Centralized SSL Certificate Management

2012-10-08T18:41:00.001-07:00

This is the second part of my blog series on multi-tenant Cloud service design. Each of the posts has two parts – discussions on Multi-tenancy and recipes to address specific challenges. When I first planned for the series the discussion parts were merely around particular problems, but somehow they grew into full-sized beasts by themselves. So, if you want to get to the recipes directly, you can safely do so. And if you just want to read higher-level discussions, you can skip the recipe parts without getting disconnected from post to post. On the other hand, if you want to get both the big picture as well as practical tactics to make the picture reality along the way, read through all texts :). Here’s the first part:

Recipe 1: Throttling by Tenants Using Multi-site Cloud Services and IIS 8 CPU Throttling

More about Multi-tenancy

In the previous article I listed some of the challenges an ISV has to face when designing for multi-tenant Cloud Services. Before we jump into any specifics again, I’d like to share some thoughts on different strategies of getting customers to the cloud. Many ISVs realize the benefits of IaaS and want to migrate their customers to the low cost, highly reliable hosting environment. One intuitive strategy to do so is to have dedicated resource(s) for each customer. This design is also referred as single-tenant, multi-instance architecture. At the surface this strategy has some very tempting benefits, among which code reuse is probably the dominating one. Because customers are still isolated to dedicated resources, the existing software should work as-it-is, shouldn’t it? Well, the bad news is, it’s rarely the case. As I mentioned in the last post, moving to the cloud is a major shift in business model and should not been taken lightly. Business aspects aside, such strategy is not immune to many of the technical challenges I mentioned in previous post such as authentication, authorization and performance. Managing a multi-instance environment can be challenging as well. Because the instances are independent from each other, chances are software updates and database patches need to be applied individually. This increased complexity makes service management and upgrades difficult. In addition, isolated resources make re-purposing of resources harder, causing unnecessary waste of capacities. For example, an ISV have to anticipate and allocate sufficient disk space for each customer. On one hand, the unused disk space is wasted (or could have been at least utilized better – remember resources are not free). On the other hand, when the customer exceeds allocated resources, addition resources have to be provisioned. Again this increases management complexity.

What becomes more interesting is when centralized features, such as administrative features and BI capabilities, are developed. Some annoying problems may jump out on you here and here. For instance, how a new feature reliably identifies individual tenants when the tenants don’t have such identifiers built in themselves? This sounds like a very small problem and there seems to be a very easy solution – a global lookup table. Technically a lookup table would have worked fine, but throughout my previous projects I’ve observed such lookup tables going out of sync pretty easily. One major reason is that the operation teams who stand up new tenants consider maintaining this global table as an additional burden – this is something they didn’t have to do and doing so doesn’t contribute or affect their deliverables. Of course in an ideal world everybody will do the right thing, but in reality the new features are often victims of such neglects and behave miserably - they try to query a system that don’t exist, they miss new systems that are coming online, and so on and so forth. Another similar problem is to open up effective communication channels to existing systems while owner of existing systems are not willing to cooperate. Such resistance is actually not that surprising - after all the initial motivation of using multi-instance architecture is to reuse existing code! The point is, again, going to multi-tenancy is a business model shift and need company-wide commitments. Many of such projects failed not merely because of technical reasons. As the discussion here is already going out of the technical scope, I shall stop here.

The multi-instance architecture is not all bad. In addition to reuse of existing implementations, it provides very good tenant isolation across all system layers; It allows uneven topologies – meaning it allows different customers to run different versions of software on different hardware. I know I just mentioned such things as downsides, but in some particular situations they are actually desirable. For example, I used to have a customer who made a major update to the software and wanted its customers to pay to upgrade. However at the same time it would like existing customers to keep using the older versions as long as they renew maintenance contracts. In another example, an ISV would like to allocated a small number of servers that were dedicated to student users while keep other paid users on the main cluster.

Enough said on multi-instance architecture for now. The other architecture choice is actually also obvious – to refactor the single-tenant system into a multi-tenant system. There’s no way to sugarcoat it – the refactoring will be a big project. And without doubts some system are harder to be refactored than others. Here are several characteristics that I’ve observed make such refactoring harder:

Stateful services. It’s not unusual to see long-living components who assume they have global knowledge of the system, or have global control over the system. Such components are often named as “managers”, “dispatchers”, “schedulers” and such. These components are dangerous because they often maintain heavy states, and they often assume they live throughout the lifetime of the service. In a multi-tenant environment, especially a cloud-hosted multi-tenant environment, no components live forever. For example, an server instance may be brought down by the hosting environment for maintenance at certain intervals (such as quarterly). Will these components be ready to persist and rehydrate their states? What happens when this happens? Those are the exact hard questions that lead people to stateless services. The greater danger, however, is the temptation to promote such components to the global level. And you might be surprised by how often that happens! Prompting such components creates single points of failures (SPOF), and failures of such components are often catastrophic.
Resources hogs. In the previous post we examined how to guard one type of resource hogs by throttling CPU usage. There are several other resources at play, such as storage, memory, and bandwidth. Because resource pressures build up much quicker on a multi-tenant system than on a single-tenant system, excessive resource consumption can easily create bottlenecks here and there. And if such problems are not corrected early, you may find yourself in a very unpleasant position of redesigning in very late phases of the project.
Tightly-coupled components. The single-tenant to multi-tenant migration is mostly a long process for any sizable projects. Chances are you’ll need to swap out old parts piece by piece. Tightly-coupled components make such swapping very hard – I guess I don’t need to say more on that.

Recipe 2: Centralized SSL management

Problem

When an ISV needs to host a large number of secured sites that use SSL certificates, certificate management may become problematic in terms of increasing site density, ensuring manageability, as well as improving performance. If you have a large number of SSL certificates on an IIS server, you may run into serious performance issues when you handle HTTPS traffics. This is because ALL certificates are loaded in memory when HTTPS traffics come in. This not only consumes memory, but also slows down the response time. In addition, it’s often desirable to remove certificate management tasks from developers’ responsibilities. For Cloud Services running on Windows Azure, certificate management is mostly separated from service management. However, there are three small catches:

The certificate thumbprints are included in .cscfg file. Although this file can be uploaded and modified without redeploying the service, changes in this file may cause the service to be redeployed anyway. For instance, when you renew a certificate with a new thumbprint, IIS configurations on the instance virtual machines have to be updated, causing service interruptions. This is a big problem when you have many certificates to renew.
There could be a disconnection between the system administrator and the developers – the administrator may have updated the .cscfg file while the developers are still building/deploying services with older .cscfg file, causing more fictions.
Cloud Service .csdef file doesn’t allow you to specify multiple certificates for HTTPS endpoints on the same port. The workarounds include using different ports, which is usually very undesirable, or use wildcard certificate or a SAN certificate.

What I’d like to achieve here are the following:

Complete separation of concern. The developers should not care about what certificates are used and when they are changed or renewed. I don’t want any references to certificate thumbprints anywhere in the code, including .cscfg file.
I want to enable IIS 8 centralized certificate management so certificate management can be greatly simplified.
I can use other certificates other than wildcard certs and SAN certs to secure my web sites.

Solution

In a nutshell, the idea of centralized SSL certificate management is simple – instead of installing certificates on each individual machines, certificates are kept on a network share as (.pfx) files. When IIS is configured to use centralized certificate management, it will look up in this location to find correctly certificate to resolve binding by matching host headers with certificate file names. When a system administrator needs to renew a certificate, all he needs to do is to xcopy the renewed certificate over the old one and the job is done. In addition to significant improvement in manageability, centralized SSL certificates also provide a very noticeable performance boost especially when you have many certificates. When IIS is configured to use SSL certificates, ALL certificates are loaded into memory when it handles a HTTPS request. The CPU and memory cost is ignorable when you have only a few certificates, but it becomes huge when you have hundreds and thousands of certificates to load. With centralized SSL certificates, certificates are loaded on-demand, which greatly reduces the processing time and memory consumption.

Sample walkthrough

As usual, the following is a walkthrough of creating the sample scenario from scratch. I do assume you are familiar with Windows Azure Cloud Service, IIS manager and ASP.NET MVC in general, so some of the steps won’t be as details as for beginners.

Prerequisites

Visual Studio 2012
Windows 8 with IIS 8.0
Windows Azure SDK

Step 1: Enable secured connection for a Cloud Service with multiple sites

Follow Step 1 of recipe 1 to create the baseline application. You can skip step 3 to 5 because you won’t need the CPUThrottleController in this case. You can also directly use the result solution of previous exercise as well.
Make a wildcard self-issued cert for testing:
```
makecert.exe -r -pe -n "CN=*.haishibai.com" -b 01/01/2010 -e 01/01/2050 -eku 1.3.6.1.5.5.7.3.1 -ss my -sr localMachine -sky exchange -sp "Microsoft RSA SChannel Cryptographic Provider" -sy 12
```
The above command installs a wildcard certificate to Local Computer personal store.
Open the certificate in mmc, export the certificate to a .pfx file. Name the file as _.haishibai.com.pfx. When you deploy the Cloud Service to Azure, this is the certificate you, or your system administrator, needs to upload.
Copy the thumbprint of the certificate. This is the thumbprint (with spaces removed) you need to enter to .cscfg file.
Import the .pfx file to Trusted Root Certification Authorities of Local Computer store. This is to avoid certificate warnings during our tests.
Go back to Visual Studio. Double-click the Web Role in the Cloud Service project to bring up property page.
Go to Certificates tab. Click on Add Certificate link.
Set the name of the certificate to WildcardCert, and pick the certificate from Local Computer store:
Go to Endpoints tab. Add an endpoint with https protocol. Pick the SSL certificate we just added:
Make sure all sites in the .csdef file refer to the new endpoint:
```
...
<Sites>
  <Site name="gold" physicalDirectory="..\..\..\IIS8Features.Web">
    <Bindings>
      <Binding name="Endpoint1" endpointName="Endpoint1" hostHeader="gold.haishibai.com" />
      <Binding name="Https" endpointName="HttpsIn" hostHeader="gold.haishibai.com" />
    </Bindings>
  </Site>
  <Site name="silver" physicalDirectory="..\..\..\IIS8Features.Web">
    <Bindings>
      <Binding name="Endpoint1" endpointName="Endpoint1" hostHeader="silver.haishibai.com" />
      <Binding name="Https" endpointName="HttpsIn" hostHeader="silver.haishibai.com" />
  </Bindings>
  </Site>
</Sites>
...
```
Press F5 to run the application. Ignore the errors and certificate warnings you see in the browser. Type in https://gold.haishibai.com:444 or https://silver.haishibai.com:444 in address bar. The site should be loaded correctly without certificate warnings. Note on your machine you may have to use a different port – see recipe 1 part 1 for details on how to get the correct port number.

There’s nothing new so far – this is what you typically do when you enable SSL for multi-site Cloud Service. Note that there’s no reference to actual certificate in the .csdef file. However, you can see the thumbprint in .cscfg file:

<Role name="IIS8Features.Web">
  ...
  <Certificates>
    <Certificate name="WildcardCert" thumbprint="55B5D291B7B56BDD4C6C38CE93CA3E3DCD49E7BD" thumbprintAlgorithm="sha1" />
  </Certificates>
  ...
</Role>

Step 2: Enable centralized SSL certificate management

Create a new network share, and copy the _.haishibai.com.pfx file into the folder. This shared folder will be the centralized certificate store.
Make sure Centralized SSL Certificate Support has been enabled. If not, enable the feature from Control Panel –> Turn Windows Features On or Off:
Open IIS manager. Select the server node. You’ll see the new Centralized Certificates icon. Double-click the icon to open the feature:
In action pane, click Edit Feature Settings…
In Edit Centralized Certificates Settings dialog, enter credential to the network share. Note that in current version, all certificates under the folder have to have the same private key password:
Click OK to update the settings.
Now we can remove references to certificates in our code. First, remove <Certificates> element from both .cscfg and .csdef.
Then, remove the certificate attribute from <InputEndpoint> element as well:
```
<Endpoints>
  <InputEndpoint name="Endpoint1" protocol="http" port="80" />
  <InputEndpoint name="HttpsIn" protocol="https" port="443" certificate="WildcardCert" />
</Endpoints>
```
Press F5 to launch the application.
Open IIS manager. Select the gold site. Then click Bindings… link in action pane.
In Site Bindings dialog, select the https binding, and then click Edit… button.
In Edit Site Binding dialog, check Use Centralized Certificate Store, then click OK.
Repeat step 10 to 12 for silver.haishibai.com.
Now navigate to https://gold.haishibai.com or https://silver.haishibai.com. The sites should load without certificate warnings. IIS looks under the shared folder for matching certificate. Because wildcard certificate is supported, IIS picks up _.haishibai.com (we use _ instead of * because * is an invalid character for file names) in this case. Of course, now you are free to use any number of domain specific certificates as well – simply copy the certs to the shared folder following the naming convention, for example gold.haishibai.com.pfx for gold.haishibai.com domain and so on.

Some additional notes

At this point I don’t have everything automated. The major problem I’m facing is to auto-configure the centralized certificate store. This configuration is (rightfully) not saved in any configuration files, so digging that up is kind of hard. However, the following are some of the other pieces of puzzle that may become useful when you try to automate the process:

Use WebPI command line tool (v4 preview) in a startup task to install required IIS components. For example, to enable Centralized SSL Certificate Support, use command:
```
webpicmdline /Install /Products:CertProvider
```
Here are some of the other IIS components you can install by WebPI (to get a complete list, use webpicmdline /List /ListOption:Available):

ID Title

AppWarmUp IIS: Application Initialization

IPSecurity IIS: IP and Domain Restrictions

RequestMonitor IIS: Request Monitor
I *think*you should be able to use appcmd to change default site settings to enable centralized SSL certificate for newly created sites. However I haven’t tried it myself.

Recipe 2 Summary

In this post we enabled IIS8 Centralized SSL Certificate Support and configured our web site to use HTTPS endpoints without having any references to any certificates in our .csdef or .cscfg files.

Cross-role Events using Windows Azure Caching (Preview)

2012-10-05T16:48:00.001-07:00

Problem

There are several different options for Web Roles and Worker Roles to communicate with each other within the boundary of a Cloud Service:

Use Service Bus Queues or Windows Azure Queue Storage (async communication, not constrained by service boundary)
Use Internal Endpoints to directly dispatch workloads (direct communication, within service boundary)
Use standard cross-machine communications such as WCF (depends on WCF configuration)
Use a common storage such as Windows Azure Table Storage (shared-storage integration pattern, not constrained by service boundary)

Other than the WCF option, which is more complex comparing to other options, none of above options provides nice support for event-driven pattern. It would be nice if we can implement some simple cross-role eventing mechanism so that we can raise events across role boundaries.

Solution

Windows Azure Caching (Preview) provides some built-in notification capabilities. You can subscribe to notifications at cluster, region, as well as item level. Because the cache cluster is accessible to all configured roles, we can easily tap into this notification mechanism and gain some simple eventing capabilities without much effort. Of course, to implement a robust event system requires much more than what’s presented here. This post is merely to provide some foods for thoughts.

The first thing is to enable notification on your cache cluster. This is a simple configuration change:

Then, create a DataCache client with change-polling interval reduced. By default the interval is 5 minutes. In the following code I set the interval to 5 seconds instead. Note that in the code I’m looking for a specific cache named “role_events”. I think it’s a good idea to have named cache used for events separate from other caches - but of course you can use default cache if you want to.

DataCacheFactoryConfiguration configuration = new DataCacheFactoryConfiguration();
configuration.NotificationProperties = new DataCacheNotificationProperties(1000, new TimeSpan(0, 0, 5));
DataCacheFactory factory = new DataCacheFactory(configuration);
DataCache cache = factory.GetCache("role_events");

Once we have the cache client, we can subscribe to different cache notifications. In the following code I register for cluster-level add and replace events – depends on your design, you may want to use different events.

if (cache != null)
{
  cache.AddCacheLevelCallback(DataCacheOperations.AddItem | DataCacheOperations.ReplaceItem,
    (namedCache, regionName, key, version, cacheOperation, nd) =>
    {
          //CALL CALLBACKS
     });
}

Now the callbacks will be called whenever an item is added or updated in “role_events” cache. To make things even a little nicer, I wrote a simple wrapper to enable .Net style events. Again, this is an over-simplified sample. Don’t use it in your production code .

public class EventBus
{
    public static event EventHandler<BrokeredEventArgs> EventReceived;
    private static DataCache mCache;
    private EventBus() { }
        
    static EventBus()
    {
        DataCacheFactoryConfiguration configuration = new DataCacheFactoryConfiguration();
        configuration.NotificationProperties = new DataCacheNotificationProperties(1000, new TimeSpan(0, 0, 5));
        DataCacheFactory factory = new DataCacheFactory(configuration);
        DataCache cache = factory.GetCache("role_events");
        if (mCache != null)
        {
            mCache.AddCacheLevelCallback(DataCacheOperations.AddItem | DataCacheOperations.ReplaceItem,
            (namedCache, regionName, key, version, cacheOperation, nd) =>
            {
                if (EventReceived != null)
                    EventReceived(mCache, new BrokeredEventArgs(key, mCache.Get(key)));
            });
        }
    }
}

Sending event is easy – simply add or update an item in the cache. Note that here I’m not using any custom classes. It’s all standard caching operation. On the other hand, it doesn’t hurt to add a Send() method on EventBus class to provide some additional abstraction. Note that on the sender’s side we don’t need aggressive notification polling as it is on the receiver’s side. Keep that in mind when you implement such as method.

DataCache cache = new DataCache();
cache.put("JobCreated", job);

Receiving events is easy as well:

EventBus.EventReceived += (o, e) =>
  {
     var name = e.EventName;
     var payload = e.Payload;
  };

And BrokeredEventArgs is a simple class with a name and a payload:

public class BrokeredEventArgs:EventArgs
{
  public string EventName { get; private set; }
  public object Payload { get; private set; }
  public BrokeredEventArgs(string name, object payload)
  {
    EventName = name;
    Payload = payload;
  }
}

That’s it! One last thing I want to mention is that, for simplicity (yeah, right), above code doesn’t prepare for any transient errors. However in reality you need to prepare for such errors especially when the caching cluster is hosted on a dedicated role. In this case the cluster role instances and other web/worker role instances are initialized independently. Chances are when you make your first caching call the cluster is not ready yet. Although DataCache has some built-in retries when it connects to a cluster, you’d better still prepare for such errors.

Recipes for Multi-tenant Cloud Service Design – Recipe 1: Throttling by Tenants Using Multi-site Cloud Services and IIS 8 CPU Throttling

2012-10-03T17:22:00.001-07:00

About Multi-tenancy

One of the major mind shifts an ISV needs to go through when it migrates on-premises systems to cloud-based systems is the shift from multi-instance (or single-tenant) architecture to multi-tenant architecture. Instead of deploying separate service instances for individual customers, an ISV can serve all its customers from the cloud. During this migration, however, some problems, which may not be so critical in a multi-instance system, will surface and cause troubles if the ISV doesn’t plan ahead. So, migrating an existing service to cloud is not just a simple matter of switching host environments. There are several new challenges the ISV has to face:

Tenant isolation. An ISV needs to ensure customers are isolated from each other – one customer should not gain access to other customers’ data, either accidentally or intentionally; workflows from different customers should not interfere with each other; provisioning or deprovisioning a customer should have zero impact on other customers; customer-level configuration changes should have no global impacts… In other words, a customer should feel (and be assured) that the service is logically dedicated to him, regardless the fact that it’s hosted on a shared infrastructure.
Problem containment. A multi-tenant system comes with many benefits such as dramatic reduction in TCO, agility in reacting to market changes, and high-availability based on robust, fault-tolerance hosting environment such as Windows Azure. However, it also brings additional risks to an ISV. For example, a successful DoC attack on a on-premises system brings down only one customer, while the same attack may cause a system-wide failure across all customers. That’s a much higher risk to deal with. An ISV needs ways to closely monitor system health and to constraint the impacts to the smallest scope when problems do occur.
Security and compliance. This is a broad problem that contains several aspects across a wide spectrum. First, authentication and authorization may become challenging for systems that have been relying on Active Directory. How do you leverage Active Directory on the cloud? Second, many industries and countries have strict compliance requirements such as where the data can be stored and who can execute a particular workflow. How does an ISV satisfy the compliance requirements while it runs the service on Cloud? Third, on a hosted environment, services clients communicate with the services over the public Internet. This requires precautions against attacks such as phishing and eavesdropping, etc. Fourth, on a multi-instance (or a single-tenant) system, the system contains a single security boundary, while in a multi-tenant system, the system contains multiple security boundaries isolating different customers. An ISV has to make sure such finer security boundaries are reinforced to guard against attacks such as spoofing and impersonation, etc.
Data migration and transformation. Is the ISV’s database schema designed to hold data for multiple customers? Should the ISV consider using separate database for different customers? What are the needs to migrate existing customer data? Will the database layer scale? Should the data be synced among different sources? All these questions require serious consideration and careful planning.
Transient errors. Service calls fail. Maybe the connection is dropped. Maybe the service is down. Maybe the server is throttling your requests. Maybe your requests timed out due to heavy workloads… Transient errors are the type of errors that are temporary and automatically resolved – if you try the same operation again later the call is likely to succeed. However, if an ISV’s code is not prepared for transient errors and takes granted that certain calls will complete, it will probably be surprised by strange behaviors that are hard to reproduce and very difficult to fix. And I can almost guarantee you when that happens, it ALWAYS happens at the worst possible time.
Performance, performance, performance. I can’t emphasize enough on the importance of performance because now the ISV’s services and clients are communicating over a long wire. Is added latency acceptable? How to reduce it? In addition, transferring large amount of data on corporate network might work fine, but is it still feasible to transfer the same amount of data over the Internet? Can we keep the same level of caching per customer without exhausting server resources? How to maintain performance level when number of requests spikes? How to handle long-running third-party service calls? How to identify and resolve performance bottlenecks across application layers? Remember, in the world of Internet, a slow service equals to a broken service – and you can quote me on that.
BI opportunities. It’s much easier for a multi-tenancy system to gain a holistic view of all customers because the system is directly connected with all customer activities. Very few ISVs can resist (nor should they) the tremendous business value such insights can bring them. And many more innovative scenarios become possible when deeper BI are acquired. However, BI doesn’t come for free. It’s by itself a sizable project. What I’ve observed is that without proper planning, some initial data-mining capabilities are sprinkled here and there and people LOVE them. And more and more demands come in (often from “the top”) as the project goes along, until the system starts to suffer from lacking of proper BI architecture. In one particular case, because the OLAP queries were locking up transactional data, they had to be pulled out from the system in a very late stage so that the service was not interrupted. And I can tell you the situation wasn’t pretty.
Multi-tenancy administration. Managing a multi-tenant system introduces new requirements on administrative capabilities. The existing, single-tenant management API and UI are very unlikely to suffice for the requirements of a multi-tenant environment. Here’s a non-exhaustive list of such requirements I’ve seen in my previous projects – (dynamic) customer provisioning/deprovisioning, centralized user management, centralized system monitoring and billing. Among these requirements, centralized user management, service usage monitoring and billing are particularly interesting – we’ll go into more details on those in later articles.

As you can see, there are LOTS to consider when implementing a multi-tenant system. For many ISVs, this is a big leap – it’s a leap from being a software vendor to a service vendor. What I’ve listed about are mostly on the technical side. There are also many changes on the business side as this is essentially a different business model. I shall not discuss more on the business and marketing side as I’m not an expert, but an ISV should be aware what it’s taking on when embarking such a journey. Said that, I don’t want you to get discouraged. Instead, you should be excited as MANY people have worked hard to provides all kinds of guidance, services and tools to help building multi-tenant systems. And many companies have implemented cloud-based, multi-tenant systems with great success. As a world leader of services, Microsoft is no exception. If you looked closely to the new features provided by Windows Server 2012, IIS 8, and Windows Azure, you can see multi-tenancy written everywhere. There are good chemical reactions happening between Windows Azure and Windows Servers – they leverage each other and pull each other forward. The interaction creates not only a better and better cloud platform, but also stronger, more scalable on-premises servers.

For the rest of the series, I’ll go through several recipes that provide potential solutions to many of above problems and challenges. Each recipe is presented in a problem-solution format, with tags on problems indicating to which areas the problems are related. I’ll be using features and capabilities of Windows Server 2012, IIS 8, Windows Azure, Windows Azure SDK, as well as .Net 4.5. However, you should be able to try out most of the samples with a Windows 8 + IIS 8 + Visual Studio 2012 + Windows Azure SDK machine. Now let’s jump in!

Recipe 1: Throttling by Tenants Using Multi-site Cloud Services and IIS 8 CPU Throttling

Problem [Tenant isolation][Problem containment][Performance]

It’s very common for an ISV to provide different sites for its SaaS subscribers. The approach creates not only a sense of ownership, but also security boundaries and management boundaries to manage the customers more effectively. Because customers from all sites compete for the same pool of precious resources the ISV can provide – CPU, storage, bandwidth, etc., the ISV has to make sure excessive workload from one customer has minimum impacts on other customers. In addition, the ISV may have signed different SLAs with different customers, it has to ensure the prime subscribers granted with sufficient resources to keep the SLA promises.

In summary, our requirements are:

Be able to throttle users based on their service subscriptions.
Heavy workloads should not jeopardize the overall system performance.

Solution

With IIS 8 on Windows Server 2012, IIS application pools are isolated into sand-boxes. The sand-boxes provide not only the security boundaries, but also the resource management boundaries. With sand-boxed application pools, an ISV can truly limit how much CPU each tenant can use (read more about sand-boxing on www.iis.net). In this sample solution, I’ll assume the ISV has two levels of subscriptions: gold and silver. To ensure SLA, the ISV allows gold subscribers to consume more CPU than the silver subscribers can ever request. In addition, the CPU throttling also makes sure the system is not saturated by unexpected surge of workloads. The solution is simple - I’ll use Web Role multi-site capability to create two separate sites, one for each level of subscription. And then, I’ll configure the corresponding application pools according to predefined throttling policy.

Sample walkthrough

The following is a walkthrough of creating the sample scenario from scratch. I do assume you are familiar with Windows Azure Cloud Service, IIS manager and ASP.NET MVC in general, so some of the steps won’t be as details as for beginners.

Prerequisites

Visual Studio 2012
Windows 8 with IIS 8.0
Windows Azure SDK

Step 1: Create a simple multi-site Web Role with a heavy workload simulator.

Create a new Cloud Service with a ASP.NET MVC4 Web Role.
In Solution Explorer, right-click on the Cloud Service project and select Properties…. Then, in Web tab, change Local Development Server to Use IIS Web Server.
Add a new CPUThrottleController Controller with an empty Index view.
CPUThrottleController contains two methods:
```
public ActionResult Index()
{
  return View();
}
public ActionResult HeavyLoad()
{
  int count = 4;
  for (int i = 0; i < count; i++)
  {
    ThreadPool.QueueUserWorkItem((obj) =>
    {
      Random rand = new Random();
      while (true) { rand.Next(); }
    });
  }
  return new HttpStatusCodeResult(200);
}
```
The Index() method returns the Index view, which we’ll implement in a moment. The HeavyLoad() method simulates a CPU intensive work by creating a tight loop. Because my test machine has four logical processors, I used count 4 to make sure all of them are saturated. You can adjust this value according to your machine’s configuration.
Modify the Index view for CPUThrottleController and add a Kill CPU link:
```
@Html.ActionLink("Kill CPU", "HeavyLoad");
```
Now define the sites in Cloud Service project’s .csdef file:
```
...
<Runtime executionContext="elevated"/>
<Sites>
  <Site name="gold" physicalDirectory="..\..\..\IIS8Features.Web">
    <Bindings>
      <Binding name="Endpoint1" endpointName="Endpoint1"  hostHeader="gold.haishibai.com"/>
    </Bindings>
  </Site>
  <Site name="silver" physicalDirectory="..\..\..\IIS8Features.Web">
    <Bindings>
       <Binding name="Endpoint1" endpointName="Endpoint1"  hostHeader="silver.haishibai.com"/>
     </Bindings>
  </Site>
</Sites>
...
```
Note: you’ll need to update physicalDirectory values to reflect your own solution folder structure and Web Role project name. Also, I included instruction to run RoleEntry with elevated privilege, because we’ll need to reconfigure application pools, which needs administrative accesses. You can also update the hostHeader values to use the values of your choice.
Edit your hosts file under %sysemroot%\System32\Drivers\etc folder to include two lookup entries for above two host names:
```
127.255.0.0 gold.haishibai.com
127.255.0.0 silver.haishibai.com
```

Step 2: Test the application without throttling

Press F5 to start the application. You’ll see a browser with a 400 error – that’s expected.
Launch another browser and go to either http://gold.haishibai.com:82 or http://silver.haishibai.com:82. Note that on your system you may get a different port number. The way to get this port number is to open IIS manager and examine the binding of deployed sites:
Start the Task Manager. Arrange the browser window and Task Manager window side-by-side:
Now in the browser address, add /CPUThrottle then press enter. Then, click on the Kill CPU link on the page. Watch CPUs go nuts:
Close BOTH browser window. Visual studio will clear up the web sites and corresponding application pools it created for you. And your CPU usage should drop back to normal.

Step 3: Enable Throttling

Add a reference to Microsoft.Web.Administration.dll to your Web Role project (browse to %systemroot%\System32\inetsrv folder to find the assembly).
Modify the WebRole class to reconfigure application pools to enable CPU throttling:
```
public class WebRole : RoleEntryPoint
{
    public override bool OnStart()
    {
        CustomerProfile[] profiles = 
        { new CustomerProfile{Name ="gold", CPUThrottle = 40000},
            new CustomerProfile{Name ="silver", CPUThrottle = 30000}};
        using (ServerManager serverManager = new ServerManager())
        {
            var applicationPools = serverManager.ApplicationPools;
            foreach (var profile in profiles)
            {
                var appPoolName = serverManager.Sites[RoleEnvironment.CurrentRoleInstance.Id + "_" + profile.Name].Applications.First().ApplicationPoolName;
                var appPool = applicationPools[appPoolName];
                appPool.Cpu.Limit = profile.CPUThrottle;
                appPool.Cpu.Action = ProcessorAction.Throttle;
            }
            serverManager.CommitChanges();
        }
        return base.OnStart();
    }
    private struct CustomerProfile
    {
        public string Name;
        public int CPUThrottle;
    }
}
```
In above implementation, the OnStart() method finds the application pool each site is using, and then applies CPU throttling to each pool according to CustomerProfile settings. CPUThrottle is the value of 1/1000 of CPU percentage (so, 10 percent is 10,000). This sets the upper limit of CPU power the application pool is allowed to use. You can see for “gold” users we are allowing up to 40%, while for silver users we are allowing up to 30%. ProcessorAction.Throttle specifies when the throttling limit is hit, user requests should be throttled. There are other options such as KillW3wp (similar to what you get in IIS 7) and ThrottleUnderLoad. You can read more about these options on www.iis.net.
That’s all we have to do! Now launch the application again and try the Kill CPU link. You can see CPU consumption is well under control. If you want, you can launch another browser and navigate to the other site and do the same Kill CPU operation. Because collectively the applications are allowed to use up to 70% percent of CPU, the CPUs are not saturated in either case:
.

Some additional notes

You can also configure the throttling settings via AppCmd. For example, this command limits CPU consumption of DefaultAppPool to 30%:
```
%systemroot%\system32\inetsrv\appcmd set apppool DefaultAppPool /cpu.limit:30000 /cpu.action:Throttle
```
If you want to use appcmd in a startup task to configure the application pools, be aware that by the time the startup task runs, the application pools have not been created. As a workaround, you can use appcmd to change the default application pool settings so that all application pools created hereafter will be affected. However it’s not possible to set up different settings for different application pools:
```
%systemroot%\system32\inetsrv\appcmd set config -section:system.applicationHost/applicationPools /applicationPoolDefaults.cpu.limit:90000 /commit:apphost
```
After the sites are deployed, you can also change the setting in IIS manager UI – simply select the application pool you want to configure and bring up Advanced Settings:

Part 1 Summary

In this post we created a multi-site Web Role for two levels of subscriptions – silver and gold. And then we configured corresponding application pools to ensure gold users have abundant CPU resources to use regardless the loads on the other group.

Claim-based access control for PHP running on IIS

2012-09-18T23:06:00.001-07:00

The goal of this post is to introduce an easy way to enable claim-based access control on PHP applications running on an IIS server using WIF. Please note the specific requirement here – I want to utilize WIF as much as possible while keeping PHP side as simple as possible. The solution presented here is not necessarily the best choice when you want to enable claim-based access control on PHP applications. Probably a third-party PHP library or extension can serve the purpose better. On the other hand, if you are interested in reusing WIF FAM and SAM to protect a FastCGI application in general, this post may give you some ideas. In addition, the solution has an extra feature - It allows “SSO” between PHP pages and ASP.NET pages. Users only need to authenticate once and then will be able to access both types of pages freely with required claims available to them.

Because this is an advanced topic, I’ll assume you are familiar with the following concepts, Windows features and software:

Control panel (manage Windows features)
Internet Information Service Manager
Windows Azure ACS
WIF components such as FAM and SAM

However I won’t assume you have much of PHP knowledge - the PHP codes we use are very basic and this is intentional. One key benefit of claim-based architecture is to allow application developers to focus on application logics instead of dealing with security details. In this example, the PHP code deals NOTHING related to security protocols or token processing. Although the final code is not quite satisfying, the spirit of claim-based architecture is kept true.

I’ll omit most of screenshots to make the post shorter because there are lots of steps here. However I’ll provide more details on key steps when necessary.

Enable PHP on IIS

First thing first, we need to install PHP on IIS. There are many articles on Internet talking about this topic. And the steps below are indeed abstracted from several of them.

Before continue, you need to make sure you have IIS 7.0 installed with CGI feature enabled. This can be done via Turn Windows features on or off dialog.
Download PHP from http://www.php.net/downloads.php. In this post I chose to use the PHP 5.4.7 VC9 x86 Non Thread Safe version. Unpack all files to a local folder, for example C:\PHP.
Open Internet Information Services Manager.
In Features View at server level, double-click on Handler Mappings icon.
In Actions pane, click Add Module Mapping…
Enter the following settings:

Field Value

Request path *.php

Module FastCgiModule

Executable C:\PHP\PHP-CGI.EXE

Name PHP
Click on Request Restrictions button.
On Mapping tab, change mapping option to File or folder.
OK all the way back.
Copy C:\PHP\php.ini-development to C:\PHP\php.ini.
Find this line in php.ini:
```
;fastcgi.impersonate = 1
```
remove the semi-colon to uncomment the setting. Save changes.
Back in Internet Information Service Manager, add a new Application to Default Web Site. Enter PHP as Alias, and pick a physical path, for example C:\PHP\Test. Click OK to create the application.
Create a index.php under C:\PHP\Test with a single line:
```
<?php echo 'Hello PHP'; ?>
```
Save the file.
Navigate to http://localhost/PHP/index.php. You should see the Hello World page if everything works out.

Enable FAM and SAM

Now we’ve have a PHP FastCGI module in our pipeline, let’s inject WIF modules on top of that so FAM can intercept incoming traffic and engage the ws-Federation dance as configured. Because this interception happens before the traffic reaches the FastCGI handler, it really doesn’t matter if the FastCGI module is for PHP or something else. It will work the same way.

Add a web.config file under C:\PHP\Test folder. The following is an example of such file:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <configSections>
    <section name="system.identityModel" type="System.IdentityModel.Configuration.SystemIdentityModelSection, System.IdentityModel, 
             Version=4.0.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089"/>
    <section name="system.identityModel.services" type="System.IdentityModel.Services.Configuration.SystemIdentityModelServicesSection, 
             System.IdentityModel.Services, Version=4.0.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089"/>
  </configSections>
  <system.webServer>
    <validation validateIntegratedModeConfiguration="false"/>
    <modules runAllManagedModulesForAllRequests="true">
      <add name="WSFederationAuthenticationModule" type="System.IdentityModel.Services.WSFederationAuthenticationModule, 
           System.IdentityModel.Services, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" preCondition="managedHandler"/>
      <add name="SessionAuthenticationModule" type="System.IdentityModel.Services.SessionAuthenticationModule, System.IdentityModel.Services, 
           Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" preCondition="managedHandler"/>
    </modules>
  </system.webServer>
  <system.web>
    <httpRuntime requestValidationMode="2.0"/>
    <pages validateRequest="false"/>
    <authorization>
      <deny users="?"/>
    </authorization>
    <authentication mode="None"/>
    <compilation debug="true" targetFramework="4.5"/>
  </system.web>
  <system.identityModel>
    <identityConfiguration>
      <audienceUris>
        <add value="http://localhost/PHP/"/>
      </audienceUris>
      <issuerNameRegistry type="System.IdentityModel.Tokens.ConfigurationBasedIssuerNameRegistry, System.IdentityModel, 
                        Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
        <trustedIssuers>
          <add thumbprint="{cert thumbprint}" name="https://{ACS namespace}.accesscontrol.windows.net/"/>
        </trustedIssuers>
      </issuerNameRegistry>
      <certificateValidation certificateValidationMode="None"/>
    </identityConfiguration>
  </system.identityModel>
  <system.identityModel.services>
    <federationConfiguration>
      <cookieHandler requireSsl="false"/>
      <wsFederation passiveRedirectEnabled="true" issuer="https://{ACS namespace}.accesscontrol.windows.net/v2/wsfederation" 
              realm="http://localhost/PHP/" reply="http://localhost/PHP/index.php" requireHttps="false"/>
    </federationConfiguration>
  </system.identityModel.services>
</configuration>

Note you’ll need to update highlighted portions to match with your configurations.

Configure ACS

Now our Relying Party is ready. We need to configure ACS to trust our RP. There’s nothing special here – just regular ACS configuration.

Log on to Windows Azure Management Portal. Don’t have an account yet? Sign up for 90 day trail here.
Navigate to the ACS namespace you want to manage.

Add a new Relying Party:

Name	PHP
Realm	http://localhost/PHP/
Return URL	http://localhost/PHP/index.php
Error URL (optional)	<leave empty>
Token format	SMAL 2.0
Token encryption policy	None
Token lifetime (secs)	600
Identity providers	<identity provider(s) of your choice>
Rule groups	<create new rule group, or reuse existing groups. Note when you sue a new group, you’ll need to go to the group and generate default rules>

Save changes

Test

Now launch browser and navigate to http://localhost/PHP/index.php again. You’ll be redirected to IP login page, and you’ll gain access to index.php after you log in.

Passing the Claims

So far so good! But we are far from finish yet. The biggest problem we are having now is that the PHP code doesn’t have access to the claims. It’s true that FedAuth cookies are available in request headers and I can simply ask PHP code to deal with them. However this will require the PHP code to understand security token handling, which is not as “claim-based” as I wanted. I want to do something nicer – I’ll return the claims to PHP instead. Now how do I pass the claims? I can use session variables. However I need to remember that I’m talking about sharing session states between PHP and ASP.NET here. One possible solution is to use a common storage such as a database to share session data, and communicate session id between ASP.NET and PHP via a cookie so they can look up session data independently. In this example I’ll simply pass all claims via cookies. I don’t think DB access code is that exciting after all – the key is to extract the claims and pass them “somehow”.

To do this, we need to customize SAM so that we can put the claims into cookies.

Create a new .Net class library targeting at .Net Framework 4.5.
Add references to System.IdentityModel.dll and System.IdentityModel.Services.dll.
Override SetPrincipalFromSessionToken of SessionAuthenticationModule class. This method is where WIF fill up IClaimsPrincipal for .Net thread. It seems like a natural place to make the claims available to PHP as well. Note the method calls into the base method first so that if there are ASP.NET pages in the same application they’d also work as normal:
```
namespace PHPSample
{
  public class MySessionAuthenticationModule : SessionAuthenticationModule
  {
    protected override void SetPrincipalFromSessionToken(System.IdentityModel.Tokens.SessionSecurityToken sessionSecurityToken)
    {
      base.SetPrincipalFromSessionToken(sessionSecurityToken);
      HttpContext.Current.Response.Cookies.Clear();
      foreach (var claim in sessionSecurityToken.ClaimsPrincipal.Claims)
        HttpContext.Current.Response.Cookies.Add(new HttpCookie(claim.Type, claim.Value));
    }
  }
}
```
Compile the assembly. Make a bin folder under C:\PHP\Test and copy the assembly under the bin folder.
Update web.config file to use the custom module:
```
<add name="SessionAuthenticationModule" type="PHPSample.MySessionAuthenticationModule, PHPSample" preCondition="managedHandler"/>
```
To read the cookies on PHP side:
```
<?php 
  echo 'Hello PHP!'; 
  $cookie=$_COOKIE;
  foreach ($cookie as $key=>$val)
    echo "<br>$key--> $val";
?>
```
Launch the application. This time you should see claims displayed on PHP page:

Of course there are much more can be done here – such as passing claims via sessions instead of cookies, wrapping up the claims to simulate similar behavior of IClaimsPrincipal, and implementing sign out, etc. But these are out of the scope of this post. I wish you enjoyed reading the long post. And now, I’ll go to watch some recorded Bing Bang Theory episodes. Good night.

ID	Title
AppWarmUp	IIS: Application Initialization
IPSecurity	IIS: IP and Domain Restrictions
RequestMonitor	IIS: Request Monitor

Field	Value
Request path	*.php
Module	FastCgiModule
Executable	C:\PHP\PHP-CGI.EXE
Name	PHP

Haishi's Blog

A Message to My Readers: Visual Studio Live! is heading back to Redmond

SPECIAL OFFER: As a thank you to the readers of my blog, I can extend $500 savings on the 5-day package. Register here: http://bit.ly/UGRD6_Reg and use code UGRD6.

Walkthrough: File Sharing Between Your Local Machine And Your Virtual Machines on Windows Azure

Online Contents on Windows Azure SDK 2.0 (Being Updated NOW!)

Announcements and Updates

MSDN

Channel 9

blog.haishibai.com

SDK 2.0

Service Bus

Gaurav Mantri’s Personal Blog

Service Bus Contents (Thanks to Abhishek for the list)

Windows Azure SDK 2.0 Features (4) – Even More Things You May be Interested

PowerShell Improvements

Storage Client 2.0

Windows Azure Active Directory

ASP.NET 2012.2 and Web Tools 2012.2

Service Bus New Features

Windows Azure SDK 2.0 Features (3) – Even Bigger Machines

High Memory Machines

Sample Scenario

Implementation

Result

Windows Azure SDK 2.0 Features (2) – A Even Better Web Sites Experience

Server Explorer Support

More Diagnostics Support & Others

Windows Azure SDK 2.0 Features (1) – A Even Better Server Explorer

Server Explorer Improvements

Table Storage Tooling Improvements

Cloud Service Diagnostics

Cloud + Devices Scenarios (1): Say a Picture

What you need

System Architecture

Client Implementation

Server Implementation

Command Interpreter

Application Scenarios

What’s next

A Tour of Windows Azure Active Directory Features in Windows Azure Management Portal – Logging in and Enabling Multi-factor Authentication

What is Windows Azure Active Directory

A Subscription with a Tenant

Basics

2-Factored Authentication – the # key is your friend!

A code sample

Use GIMP to remove solid background of an image

A Love Story: Devices and Cloud – Perfect Together

Learn Windows Azure Cloud Services from Beginning: a Mini Online Course

Windows Azure Active Directory (current) resource list

Concepts

Tooling for Common Tasks

To provision a new Windows Azure Active Directory Tenant

To manage a Windows Azure Active Directory Tenant

To provision a service principal

To validate token by signer and issuer pair

To configure applications that use WIF

To query Windows Azure Active Directory

To handle JWT token

To configure a web application to use Windows Azure AD

Scenarios

How to provision a Windows Azure Active Directory Tenant as an Identity Provider in an ACS Namespace

Integrating Multi-Tenant Cloud Applications with Windows Azure Active Directory

To use Windows Azure Active Directory as a trusted IP of a Windows Azure Web Site

To use Windows Azure Active Directory as a trusted IP of a Windows Azure Cloud Service

How to get role and group membership claims for users signing in via Windows Azure Active Directory

Delegation without ActAs token

Windows Store app accessing Web PI

Claims issued by a Windows Azure Active Directory Tenant

Common URLs

New features in Service Bus Preview Library (January 2013) – Epilogue: Message Browse

Why an epilogue?

Message Browse

Send us your feedbacks!

New features in Service Bus Preview Library (January 2013) – 3: Queue/Subscription Shared Access Authorization

Queue/Subscription Shared Access Authorization

New features in Service Bus Preview Library (January 2013) – 2: Auto-expiration

Auto-Expiration

New features in Service Bus Preview Library (January 2013) - 1: Message Pump

Message Pump

Dealing with long-running jobs