Another IIS Blog

Managing Windows EC2 Instances remotely with Powershell

thomad — Thu, 12 Aug 2010 23:54:24 GMT

For a long time now I'm trying to find out how to remotely manage an EC2 machine with Powershell.It finally worked. Here is how:

Start a new Windows Server 2008 EC2 instance. Keep the machine name, public DNS name and password for your EC2 machine handy. You'll need it later.
Configure your Security Groups (which are firewall rules in essence) to allow connections to port 5985.
Connect to it and install PowerShell 2.0 for Windows Server 2008. Reboot.
After the reboot I pretty much followed this walkthrough. Here the short version:
a) Run Enable-PSRemoting –force on your EC2 machine; elevated of course.
b) On the client, assuming it's Win7 runthe following Powershell commands:
   Start-Service WinRM
   Set-ItemProperty –Path HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System –Name LocalAccountTokenFilterPolicy –Value 1 –Type DWord
   Set-Item WSMan:\localhost\Client\TrustedHosts –Value <Public DNS Name of your EC2 machine> -Force -Concatenate
You are ready to go. Connect from your client to EC2 with the New-PSSession cmdlet, for example:
New-PSSession -credential <machine name of your EC2 machine>\Administrator -computername <Public DNS Name of your EC2 machine>
To execute commands on your EC2 machine I used the Enter-PSSession cmdlet, for example:
Enter-PSSession <id returned by New-PSSession>
After this you'll have a command-line on your EC2 machine.

Troubleshooting Helper Script for Microsoft Web Matrix

thomad — Tue, 06 Jul 2010 20:55:24 GMT

Well, this didn't work too well. I had to replace the previous post with this one because the script had some issues. Let's start again:

Web Matrix is released!

Microsoft Web Matrix is in beta now. As with all beta software there is a chance you run into issues you can't explain and can't figure out yourself. Time to go to the Web Matrix forums to ask for help. But sometimes a description of the problem is not enough. With the following executable you can attach a ZIP archive with all Web Matrix related log files to your problem description. With this information an expert has a much better chance to find the root cause for your problem. All you have to do is to run the executable. After you ran it you will find a ZIP archive with the name WebMatrixKaputtLogs.ZIP in the current directory.

Here is the the download URL. Unpack the ZIP and copy WebMatrixKaputt.exe to a directory of your choice.

What WebMatrixKaputt.EXE Does

1) Gathering system information via the systeminfo.exe tool. The output is saved to a file with the name systeminfo.txt in the temp directory.

2) A ZIP file with all the log and configuration information produced by Web Matrix are added to the ZIP archive:

The systeminfo.txt file generated in the previous step
The Web Platform Installer logs (files underneath the %userprofile%\appdata\local\microsoft\web platform installer\logs directory). ATTENTION: these logs might contain the passwords you used to install Web Platform components.
Log files and trace files written by IIS Express (files underneath %userprofile%\documents\IISExpress8\config, %userprofile%\documents\IISExpress8\logfiles and %userprofile%\documents\IISExpress8\TraceLogfiles)
Log files written by HTTP.SYS, the underlying HTTP kernel-mode driver used by IIS Express (files underneath the %windir%\system32\logfiles\HTTPErr directory)
Log files written during IIS Express process startup (%temp%\iisexpress8 directory)

Hope you never have to use it though :)

My Web Site is so slow… And I don't know what to do about it!

thomad — Tue, 08 Jun 2010 04:39:51 GMT

TechEd Northamerica started today. It's huge - more than 11,000 attendees from what I hear. One of my talks, titled "My Web Site is so slow…and I don't know what to do about it", happened this afternoon (here the PPTX).

I had so much information plus eight demos that I ended exactly 6 seconds before they cut me off. I promised the audience to post all the resources and tools I used to come up with this talk on my blog. So I'm sitting here in the Sheraton Canal Street lobby, enjoy free WIFI, a pint of beer assembling all the information I collected on web performance over the last year.

Before I start listing Best Practices for specific topics I need to call out the three books I found the most useful:

Books

1) The two Steve Souders books "High Performance Web Sites" and "Even Faster Web Sites" are probably a must-have for everybody concerned about web performance.

Unfortunately Souders is thin when it comes to IIS; ASP.NET isn't mentioned once. Reason enough to give this TechEd talk.

2) Only recently I discovered "Ultra-Fast ASP.NET" by Richard Kiessig. The title wasn't too appealing but I browsed through it anyway. And "Eureka" - this is a great book. As a Program Manager in IIS for almost 10 years I'm used to authors who know how to write but have no idea what they are writing about. Not so with Richard Kiessig's book. It's very thorough, everything is well-reasoned and explained. I learned quite a few things about IIS and ASP.NET that I wasn't aware of.

General Web Performance Tools

Well, the obvious ones:

Fiddler - Eric Lawrence's Web Debugging Proxy is tremendously to understand what's going on between a web browser and the web server

YSlow - for a quick look on how a particular web-site is doing with regards to performance.

PageSpeed - Google's answer to YSlow.

WCAT - WCAT is the tool you want if you want your web server to break a sweat. And if you are like I am and too lazy to write WCAT scripts you probably want the WCAT Extension for Fiddler. With it you can select a few Fiddler sessions and the WCAT Extension for Fiddler will create and run the WCAT script for you - all automatically.

Reference web sites

Yahoo's exceptional performance team

Steve Souders site and Blog Feed.

There are probably more, but I'm at Marguerites now, its getting late and there is still a lot to write.

Improving Web Server Performance

I structured the talk in five parts. First I explained why performance is important, then I went through some terminology and discussed the areas where web performance can be improved. Then I went through every performance area and discussed Best Practices. Here are the three areas:

Speeding up Server Execution
Optimizing Network Transfer
Frontend Optimizations

Best Practices

Here are the Best Practices discussed. As already said it in the talk: web performance is a huge area and one could probably talk for days about Performance Best Practices. I picked the ones I thought have the most impact for my TechEd attendees.

Server Execution Best Practices

The Server Execution section was about doing more, faster with fewer resources. This is the area where IIS really shines. No other general purpose web server is capable of sustaining the throughput IIS/ASP.Net are able to handle. I think however that by focusing too much on Server Execution we neglected the other areas, i.e. network transfer and frontend optimizations. But be it as it is - here are the Server Execution Best Practices I discussed:

Best Practice	Links/Resources/Comments
Run Windows Server 2008 (R2) on new hardware	I didn't find an article that explicitly discusses this. We ran into this issue however when Windows Server 2008 was in Beta. The Windows Server 2008 bits are optimized to take advantage of the features of new CPU architectures. You probably will see that Windows Server 2008 doesn't perform well on old hardware compared to Windows Server 2003.
IIS: Put existing default document on top of the Default Document list	The list of IIS default documents increased over the years (iisstart.htm, default.htm, default.asp, index.htm, index.php etc.). Problem is that IIS has to go through the list of default documents and look in the file system until it finds one that actually exists. You can increase the execution speed by stripping the list of default documents or putting the one you are using on top of the list.
IIS: Use IIS7 or ASP.NET output caching	There is a good tutorial on the ASP.NET web-site on how to take advantage of Output Caching. Ultra-Fast ASP.NET also has a chapter on it. IIS7 Output Caching is discussed in this Learn.IIS.Net article.
IIS: Use authentication and SSL only if necessary	Couldn't find something linkable but the problem is obvious: authentication can introduce a round-trip to the backend, e.g. Active Directory and encryption is CPU extensive.
IIS: Turn off unnecessary features like Request Tracing	Request and ETW Tracing is pretty lean and invaluable for troubleshooting problems. But if you are under a lot of load you should probably turn these features off.
PHP: Use FastCGI and not CGI	The IIS team is committed to make PHP run well on Windows. Providing a first class FastCGI implementation was the first step which made this a reality. The Web Platform Installer makes installing PHP on IIS a one-click experience. Everything else on http://php.iis.net
PHP: Enable Wincache	if you run PHP without an output cache you will quickly run into performance problems. Wincache is an Open Source output cache for PHP developed by the IIS team. The simplest way to install it: Web Platform Installer
PHP: Set FastCGI MaxInstances to 0	FastCGI spawns multiple PHP processes for each web application. If you are in a hosted environment this might soon become an issue. Here is a great blog post on how MaxInstances can mitigate this problem.
ASP.NET: turn of debug="true" in web.config	Increased memory footprint and unoptimized code are only two of the problems you'll see if the debug flag is set to true. ScottGu blogged about this a while ago.
ASP.NET: Read through the ASP.NET Performance Coding Architecture Guides	Great book and also free.
ASP.NET: Read about the Large Object Heap	When I talked with the ASP.NET developers about this TechEd presentation they urged me to talk about the Large Object Heap (LOH). Every object larger than 85k goes into the LOH. The problem with the LOH is that the .NET Framework has a hard time to Garbage Collect LOH objects frequently enough in certain web scenarios, e.g. if you allocate a large object for every incoming HTTP request. Read more about the LOH here
ASP.Net: Use ASPNET_COMPILER to optimize for the first impression	ASP.NET applications can take a long time to start up. Due to human nature the first impression counts however. That's why the Web Platform Installer runs ASPNET_COMPILER at the end of an install of an ASP.NET application. ASPNET_COMPILER does a batch compile of all ASP.NET files and resources. This speeds up startup, especially on slow disks! More about ASPNET_COMPILER here.
ASP.NET: Review IIS idle timeout and proactive recycling settings	Given the considerable startup time of ASP.NET applications you might want to look at the IIS recycling and idle timeout settings. IIS recycles Application Pools proactively every 29 hours. It also times out Application Pools when the site is inactive. A proactive recycle might be initiated right when you have the most traffic on your site and an idle timeout might happen just because you removed your server from the web farm for a few minutes. If you can afford it you might want to turn off idle timeout and find some better recycling settings for your ASP.NET application.
ASP.NET: review Thread Pool settings	there is a setting in ASP.NET that might not work well for high throughput sites. It's the maxConcurrentRequestsPerCPU setting. It's set to 12 by default, i.e. not more than 12 concurrent requests per CPU will be executed. Usually this setting works well. We've seen instances however where a higher setting is necessary. Thomas has a great blog post on the thread settings in IIS7.

That's it from the Server Execution side. Now on to Network throughput.

Optimizing Network Transfer Best Practices

Best Practice	Links/Resources/Comments
Use Content Distribution Networks	Steve Souders book has a whole chapter about it. The only thing to add is that Microsoft hosts JQuery on a CDN for you - free!
Use compression	There are several great articles on compression on several blogs: Scott Forsyth's: IIS7 Compression. Good? Bad? How Much? Kanwal's "Changes to compression in IIS7" and the IIS compression configuration reference: httpCompression urlCompression
Minify Javascript and CSS	Enough said in Steve Souders book. Minification tools which work well on Windows: Microsoft Ajax Minifier 4.0 JSMin by Douglas Crockford
Use Doloto for splitting the initial Javascript payload	Very well explained on the Doloto web site. No reason for me to rehash.
Do not scale images in HTML	Steve Souders first book
Optimize Images	Steve Souders: Even Faster Web Sites
ASP.NET: Disable ViewState	Before you remove ViewState you should understand it. Here a good start. Once you are sure you don't need it - EnableViewState="false" is your friend.
Use the Cache-Control header to help the browser cache	IIS allows you to set the Cache-Control header via the UI. I wrote a blog entry on how to do it a couple of weeks ago. Here is also a command-line script if using the UI is too cumbersome: appcmd set config "Default Web Site/scripts" /section:staticContent /clientCache.cacheControlMode:UseMaxAge /clientCache.cacheControlMaxAge:"365.00:00:00" In ASP.NET you can do it programatically via the Response.Cache object or you can do it declaratively via the OutputCache Location="Client" directive.
Avoid Redirects. If you can't use permanent redirects (301's) instead of temporary redirects (302's).	A redirect introduces another network round-trip. Avoid it. Best described here or in Steve Souders first book.
Combine Javascript and CSS	Same as row above.
Strip unnecessary response headers	In IIS it's easy to remove the X-Powered-By header via the UI. The ETag header is not as easy unfortunately. "Ultra-Fast ASP.NET" has an example how to remove the ETag in managed code. I have code how to remove the ETag in native code for all responses. I will post it later however. It's getting really late :)
Reduce DNS Lookups	DNS lookups introduce network roundtrips because the name has to be resolved and for that the browser needs to talk to a DNS server. Sometimes it's a good idea to use another one or two DNS names however. For static content for example. Static content doesn't need all the other stuff you might need for dynamic content, e.g. cookies. Steve Souders has a whole chapter about this topic.
Use authentication/SSL only when necessary	Do you really need authentication or SSL for images or scripts? Are they really worth protecting? Authentication can introduce additional round-trips because sometimes needs a challenge response authentication dance is needed to authenticate successfully.
Use the IIS7 bandwidth throttling module if your bandwidth is misused by somebody	Great example in "Ultra-Fast ASP.NET". You can throttle the bandwidth of a response based on the user, system load or any other criteria. Probably worth considering if you don't have unlimited bandwidth, especially if you have media content. Remember: On average only 20% of a video is usually watched by a user before he moves on to something else. If you already downloaded the full movie in this time you wasted a whole lot of bandwidth.
Review Cookies and their sizes	Cookies can get pretty big. Consider using the Cookie path feature if you can't reduce the size
ASP.NET: Keep Master Pages lean	Master Pages are part of every response. Don't bloat them.
Careful with ETags on IIS 6.0	KB articles 922733 and 922703 explain why.
Use lowercase to reference your resources	Windows is not case-sensitive and it doesn't matter if you use lowercase, uppercase or a mix of both. The problem is the browser however. Because other OSs are case-sensitive the browser has to be case-sensitive as well. This means that it will cache multiple versions of a resource if you use different cases. Another issue: resources compress better if lowercase is used. Why? Because most letters are lowercase. Using a smaller set of characters will produce more dictionary matches and compression is more effective. I read this is in "Ultra-Fast ASP.NET". Need to try this though!

Frontend Optimizations

Ok, people are leaving the bar. Need to wrap this up. Let's go:

Best Practice	Links/Resources/Comments
Scripts should go to the bottom of the page	Downloads of additional resources are blocked if a Javascript download is discovered. If Javascript is at the top of the page rendering of the page is probably not as fast as it could be. Responsiveness of the web application suffers. Here is the Steve Souders example page
Stylesheets on top	Rendering can only be done once the styles are downloaded. Steve Souders example page
Avoid CSS Expressions	Same here: Steve Souders example page

And now good night from New Orleans!

Why unknown file types are not served by IIS

thomad — Fri, 14 May 2010 05:05:11 GMT

Quick one today.

I was just trying to publish some videos that I shot with my Google Nexus One. The media format of the videos is .3gp. When I tried to watch the video on my local IIS 7.5 machine I got a detailed 404.3 error message:

I would have seen only a simple "404 - File Not Found" error if I would have made the request from a remote machine.

The Problem

IIS doesn't serve unknown file extensions. It doesn't do that because IIS handler mapping is usually done based on the extension of the request. If a handler can't be matched it falls through to the static file handler which would serve the request as text. This would be a security problem because files might be returned as text that should be executed as script (e.g. ASP, ASP.NET or PHP files) but not be returned as text. Here is a real-world example:

Let's assume you uninstalled PHP or ASP.NET but you still have some ASP.NET files or PHP files in your IIS directories.

Problem #1: PHP or ASP.NET files contain your code (i.e. your intellectual property). You don't want to reveal your superior coding skills to others, do you?
Problem #2: Often times scripts or code contains passwords, fore example to a database.

It wouldn't be a good idea for the static file handler to serve these files as text. And that's why the "don't serve unknown file extensions" feature exists.

Now let's get back to my problem. In order to serve .3gp files I simply added a Mime Type entry to the static file handler section that tells it about the file extension and it's Mime Type. Here is the command:

%windir%\system32\inetsrv\appcmd set config /section:staticContent /+"[fileExtension='.3gp',mimeType='video/3gpp']"

And while we are at it. You might run into the same problem with MP4 files. Here is the appcmd for it:

%windir%\system32\inetsrv\appcmd set config /section:staticContent /+"[fileExtension='.mp4',mimeType='video/mpeg']"

More Details

Here are some additional links in case you need more details on how to add Mime Types in IIS 6.0 or IIS 7.x.

IIS 6.0: http://support.microsoft.com/kb/326965

IIS 7.0 and 7.5: http://technet.microsoft.com/en-us/library/cc725608(WS.10).aspx

Technorati Tags: 3GP,404.3,404,IIS7,Mime Types,Mime

WCAT Fiddler Extension for Web Server Performance Tests

thomad — Wed, 12 May 2010 06:14:06 GMT

For web server performance testing the Microsoft Web Capacity Analysis Tool (WCAT) is the tool of choice of the IIS team as well as the Windows Performance Team. Simple tests that hammer IIS with thousands of requests to the same HTTP page to very sophisticated web hosting and benchmark scenarios - there is nothing that WCAT can't do. Even on ApacheCon I heard some hard-core Apache guys talking about using WCAT for their performance tests.

WCAT is extremely versatile and can simulate thousands of concurrent users making requests to a single web site or multiple web sites. The WCAT engine uses a simple script to define the set of HTTP requests to be played back to the web server. Extensibility is provided through plug-in DLLs and a simple API. WCAT is the best tool to hammer your web server with requests and make it really sweat.

Writing a WCAT script is not rocket science. However, writing a WCAT script to replay client requests is a lot of work because all hyperlinks and other resources have to be added manually. Lot's of typing… So I thought I write a Fiddler extension that does the work (download here). To use it is simple. Selecting a set of requests in the Fiddler UI and clicking the "Run WCAT script for selected sessions" in the context menu will do the trick. The WCAT Fiddler extension will then generate a WCAT script from the selected sessions and run the selected requests as a WCAT transaction against the configured server. Once the test is over it will show the results using the built-in reporting page.

WCAT Fiddler Extension Details

After you installed the WCAT Fiddler Extension you get two additional menu items in Fiddler. The first one is the "Configure WCAT Run" menu item in the "Tools" menu.

In the configuration dialog below you can configure the following:

Server Name
This might look a little weird given that the server name could be inferred from the host name used in the selected sessions. WCAT is a server performance tool though and for this reason it will run the selected sessions only against the server that is specified in this dialog. The WCAT Fiddler extension uses localhost as the default. So even if sessions get selected that Internet Explorer requested from a remote web server, e.g. www.microsoft.com, WCAT will establish a connection with the WCAT-specified server and run the www.microsoft.com requests against it (i.e. the host header will say www.microsoft.com but the server that receives the request is the local machine. WCAT will also not use the Internet Explorer HTTP interfaces to make its requests. It's using its own infrastructure. WCAT requests won't show up in Fiddler nor can a proxy server be used with WCAT.
Virtual Clients
WCAT has a controller/client architecture that allows multiple physical machines (clients) to team up and send requests against a single web server. For the WCAT Fiddler Extension this feature is not used. The WCAT Configuration dialog allows the configuration of virtual clients however. Virtual Clients are the number of concurrent threads that the WCAT Client will use to send request against the web server. One thing to remember: the default configuration uses the local machine as the WCAT client as well as the WCAT server. Both need resources (CPU/memory) and performance tests will definitely be skewed by running client and server on the same machine.
Duration
The duration of the test. Default is 30 seconds.
Warm-up time
The time WCAT gives the web server to warm-up its caches. After the warm-up time will release the throttle and send everything it has against the web server.

Run WCAT script for selected sessions

The second menu item is available in Fiddler's context menu.

The following happens if the menu "Run WCAT script for selected sessions" is clicked:

The code checks if WCAT is installed. It show message if not and returns.
It iterates through all selected sessions and writes a WCAT script file to the temp directory (scripts can be found via "dir %temp%\*.wcat).
It starts a WCAT controller and a WCAT client which executes the WCAT script using the parameters configured in the WCAT Configuration dialog.
After the test completes the results are displayed in Internet Explorer (results file can be found via "dir %temp%\*.wcat.xml").

Results

There is a lot of interesting data in the WCAT results page. The two most important are shown in the summary. Requests/sec is pretty obvious. "Transactions per Second" is not hard to understand either. The generated WCAT script puts the selected Fiddler sessions in a so-called transaction. WCAT executes all requests of a given transaction before it starts a new one. So "Transactions per Second" is the number of these selected Fiddler sessions that were handled per second. These two results give a rough ballpark figure how fast your web server runs.

The results have to be taken with a grain of salt. There are many things to look at before these numbers will reflect the maximum of what your web server can handle. For example:

the WCAT client will consume CPU cycles if you run the WCAT client on the same machine as your web server
the network might be maxed out and not be able to send more data
the virtual clients you configured might not be able to generate enough traffic for the web server

I will follow up with another post discussing how to properly test web server performance.

Known Issues:

To show WCAT Reports in Internet Explorer WCAT's reports.XSL File is needed. Unfortunately it needs to run scripts and shows the following dialog:

The dialog sometimes ends up in the background and IE appears to hang.
SOLUTION: Just TAB through your windows and click "OK" once you find the dialog.
Currently only GET requests are supported by the WCAT Fiddler Extension.
I'm pretty new to writing installers. The MSI installer puts the WCAT Fiddler Extension in the c:\Users\<username>\My Documents\fiddler2\scripts directory. No messages, no dialogs. It just does it.

Careful when using the Cache-Control Header UI in IIS 7.x!

thomad — Thu, 06 May 2010 06:19:00 GMT

I was just reading Eric Lawrence's blog on Cache headers. In it Eric describes that it's necessary to use sensible values for the Max-Age headers because IE and several browsers don't like values greater than the size of a signed 32-Bit Integer, i.e. 2,147,483,647. Bigger values result in the content not being cached at all. Using the max 32-Bit integer value (2,147,483,647) is more than enough though. It caches the content for more than 68 years (max-age is using seconds so you divide 2,147,483,647 by 60 seconds by 60 minutes by 24 hours by 365 days which gets us to 68 years).

As an IIS guy I asked myself if the IIS User Interface does the right thing when it comes to this header. Unfortunately it doesn't.

Cache-Control User Interface

The Cache-Control User Interface is somewhat hidden behind the "HTTP Response Headers" icon.

Clicking the icon shows existing response headers in the center. It offers the "Actions" menu on the right side. By clicking "Set Common Headers" the resulting dialog page will allow the configuration of the Cache-Control headers.

The Cache-Control dialog allows the configuration in seconds, minutes, hours or days. The value is stored as a timespan value in the IIS configuration system. In a perfect world the IIS User Interface would only allow values up to the magic 2 million (max 32-Bit signed integer) so that all browsers understand the max-age value. The world is not perfect though :(. The underlying base data type for a timespan value is a 64-Bit integer representing ticks (one tick is hundred nanoseconds, i.e. a millisecond has 10,000 ticks). The Cache-Control dialog allows values up to the max value of the timespan data type. The max is 922,337,203,685 seconds, i.e. roughly 15 billion minutes or 256 million hours or 10 million days or roughly 29247 years. Values far bigger than what web browsers understand in a Cache-Control header!

What the IIS run-time does with large timespan values

When the underlying IIS run-time reads the max-age timespan value it converts it to an unsigned 32-Bit integer - no questions asked. This doesn't work too well for two reasons:

1) Large timespan's simply don't fit into a 32-Bit integer and we end up doing a modulo operation. Using the max value for seconds (922,337,203,685) for example will set the max-age header value to 3,214,202,341 which is 922,337,203,685 mod 4,294,967,295.

2) Due to the issues Eric described in his blog post the IIS run-time should not set the header if the value is larger than a 32-Bit signed integer.

We'll fix this issue in future IIS version - but until then Eric's advise holds true: use sensible values for your cache headers!

Running Wordpress on MariaDB on Windows

thomad — Sun, 25 Apr 2010 13:18:00 GMT

You might have read that the MySQL codebase was forked. The MySQL founder, Monty Widenius, is - how should we say it - not quite happy with the direction MySQL is going after the Oracle acquisition. So Monty and his team forked MySQL and called his branch MariaDB after his youngest daughter.

I wanted to check out MariaDB, running Wordpress with MariaDB on Windows. Not a problem. All I had to do was to trick Web Platform Installer into thinking that MySQL is already installed.

Here step-by-step on how you do it:

MariaDB Installation

MySQL and MariaDB can probably run side-by-side. That would make the installation and configuration more complex however and that's why I'd recommend you simply uninstall MySQL. You can do this by going to the Control Panel and select "Uninstall Programs". Pick MySQL and uninstall it.
Setup of MariaDB is pretty easy although no Windows Installer (.MSI) is available yet. For my install I downloaded the MariaDB version 5.1.39 bits. When you read this you might want to check www.askmonty.org for newer version. Copy the contents of the zip file into a dedicated directory, e.g. c:\mariadb.
Start a command shell (elevate to Administrator on Windows 7 or Vista) and run "mysqld.exe --install". mysqld.exe can be found in the bin directory, e.g. c:\mariadb\bin. This will install MariaDB as a Windows System Service.
Start the MariaDB service by executing the command: net start mysql
MariaDB is now running.
There is no root password for MariaDB yet and that's why I would recommend you configure one. Later on Web Platform Installer will ask you for the root password you configured here!
Start mysqladmin with the following arguments:
mysqladmin.exe -u root password <your supersecure password>
Like mysqld.exe mysqladmin.exe can be found in the bin directory. Time to get Wordpress installed.

Tricking Web Platform Installer

Web Platform Installer (WebPI) does an extensive dependency check on startup to figure out what products are already installed and which products might still need to install. Because Wordpress depends on MySQL we have to trick WebPI so that it thinks MySQL is aleady installed. If you peak into the WebPI RSS feed you see that it checks for a MySQL install by checking for a reg key
HKEY_LOCAL_MACHINE\SOFTWARE\MySQL AB\MySQL Server 5.1

On 64-Bit versions of the OS it also checks for a potential 32-Bit install of MySql against the reg key
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\MySQL AB\MySQL Server 5.1
Let's simply set the registry key with the following command: reg add "HKLM\SOFTWARE\MySQL AB\MySQL Server 5.1". Now WebPI will be convinced that MySQL is already installed.

Installing Wordpress

Installing Wordpress is easy. Just start WebPI and click Wordpress. For a little more handholding refer to Richard Marr's blog.

Simple, wasn't it?

Technorati Tags: Wordpress,MariaDB,Web Platform Installer

Setting up SSL made easy…

thomad — Fri, 16 Apr 2010 07:42:00 GMT

This blog post is about a small tool I just finished. It makes configuring and using SSL much easier. In the first part of the blog post I quickly review the problems and difficulties you run into when setting up SSL. Then I introduce SELFSSL7 and finally walk through several usage scenarios.

Setting up SSL - what a pain!

I'm sometimes amazed how hard seemingly simple things still are. Take SSL as an example. Shouldn't setting up SSL be easier than what's shown in the walkthrough on how to set up SSL in IIS7. Here are the typical steps you have to go through before you can access your SSL site.

Steps to set up SSL

Generate a certificate request
Send the request to a Certificate Authority
Wait until the certificate is returned to you
Install the certificate and finally
you need to add a SSL binding in IIS

Once you've completed all of these steps you are able to use the magic "https" prefix to browse to your web-site. There are all kinds of things that can go wrong during these steps not to talk about the time it takes until you finally have SSL working. It's certainly a royal pain if you want to do that on your local dev box.

Self-signed certificates

IIS7 made it much easier by allowing you to create a self-signed certificate and use this certificate for your SSL site. But there are still issues, for example TRUST.

As soon as you are using self-signed certificates Internet Explorer will complain that it doesn't trust the certificate:

Firefox will do the same:

Establishing trust between your browser and your web server

When a web browser like Firefox or Internet Explorer receive a SSL certificate from the server it usually checks the certificate for the following three things:

Is the certificate still valid or is it already expired?
Is the common name of the certificate the same as the user entered in the browsers address bar?
Does the browser trust the certificate or the issuer of the certificate?

Checking for #1 and #2 are pretty straightforward. For #1 the browser simply compares the current date with the expiration date of the certificate. The browser complains if the current date is already past the expiration date of the certificate. Checking #2 is easy, too. Compare the address entered in the address bar with the "common name" property in the certificate. If the name doesn't match the browser will complain. Checking for trust is a bit trickier. All browsers have a list of baked-in certificates (Trusted Root Certificates). The browser will complain if the SSL certificate IIS sends you is not a descendant from one of these Trusted Root Certificates. Here is the list of Trusted Root Certificate Authorities" that Internet Explorer uses ("Tools" - "Internet Options" - "Content" - "Certificates" - select the "Trusted Root Certification Authorities" tab).

The problem with IIS7 self-signed certificates

IIS7 introduced a new feature which allows you to create a self-signed certificate that can be used for SSL. Problem is that it is a bit limited in what you can do with it.

The common name is always the machine name of your IIS server.
The certificate is only valid for one week.
The certificate is not added to the "Trusted Root Certificate Authorities" of any browser. Not even on your local machine. You need to export the certificate to a file and import it back in so that it can be added to the "Trusted Root Certificate Authorities".

Here comes SelfSSL7!

SelfSSL.exe was a tool that we shipped as part of the IIS6 Resource Kit. By simply executing it you got the full monty:

a self-signed certificate with a configurable common name and a configurable expiration date
an IIS SSL binding
and SelfSSL also added the self-signed certificate to the "Trusted Root Certificate Authorities" list so that #3 didn't get in your way

SelfSSL still works if you have "IIS6 Management Compatibility". It was time however to rewrite this tool for IIS7 and to add some features to it.

What SelfSSL7 offers:

SelfSSL7 creates self-signed certificates with

one or more configurable common name
a configurable expiration date
a configurable key size

SelfSSL7 also allows you to add a SSL bindings in IIS. Configurable are

the site name SSL is to be configured on
the IP address
the port

When the /T parameter is specified the self-signed certificate is added to the "Trusted Root Certificate Authorities" of the current user which will make the certificate trusted and Internet Explorer won't complain anymore.

And best of all: SelfSSL7 has intelligent defaults, i.e. you only have to enter SELFSSL7.EXE and it will do the work for you to set up a fully functioning SSL for your local IIS7 machine.

SelfSSL7 can also export the self-signed certificate to a password protected PFX file. This allows you to import the certificate on another machine which then can also trust the self-signed certificate.

Examples and Usage

Let's start with the command-line help by entering "SELFSSL7.EXE /?":
Configure SSL on the local machine and make the SSL certificate trusted for the local user:

Defaults are used to configure the IIS binding and make the certificate trusted in the user's root certificate authorities store. With this you should be able to enter https://<machinename> without getting a certificate warning.
Configure new SSL certificate that also works for https://localhost. Overwrite IIS binding that we configured in 2)

I use
/Q to overwrite existing IIS SSL bindings
/T to add the certificate also to the user's certificate store so the SSL certificate is trusted by IE
/I to add an IIS binding
/S specifies the site we want to use to add the binding
/N specifies two common names: IIS7BRICK is my machine name and LOCALHOST is the local loopback adapter name
You should be able to browse to https://localhost and https://<machinename> without getting security warnings from Internet Explorer. If you click the the little lock at the right side of Internet Explorer's address bar

you can find the properties of the certificate we just created, e.g. the validity dates, the common names and the key size (1024 Bit).
Let's now create a certificate with different properties using a stronger key size (/K 2048) and a longer validity date (/V 1000 = 1000 days).

If you look at the certificate properties you can see the differences:
As a last step lets also export the certificate we are creating. This can be useful if you want to avoid the security warnings also on other machines. The only thing you have to do is to import the exported certificate into your other machines "Current User"-"Trusted Root Certification Authorities" store. You do this by simply double-clicking the PFX file you get after this step.

Summary

With SelfSSL7.exe it becomes extremely simple to set up a SSL site. There are several command-line options that allow you to customize the SSL experience. However, due to the smart defaults you are able to run SELFSSL7.EXE without any parameters and get a fully functioning SSL site.

Microsoft Web Platform on Amazon EC2

thomad — Mon, 12 Apr 2010 21:35:00 GMT

One thing we often hear from Web developers, especially those who offer Web site design and consulting services, is the need to get a Web site up quickly, which runs fast and gives them total control of the server for maximum flexibility and scalability. Today's post is about the four Microsoft Web Platform Images (AMIs) we offer on the Amazon EC2 platform. These four images allow you to start a blog, stream media or run a web application within minutes - live on the Internet!

Amazon EC2

Amazon's EC2 has taken an alternative approach to Web hosting. As a Web developer you've probably heard of "cloud computing". It is an alternative take on Web hosting that puts you in control of the servers you spin up and you only pay for what you use. If you've ever bought a book or something else from Amazon, you can use that same account to buy a server and try it out for a few hours, total price: a few bucks, no long running contract or monthly fees.

Microsoft Web Platform

The Microsoft Web Platform includes the Web server, frameworks, database and tools needed to build and run Web sites on Windows. With the release of Web Platform Installer, it is now easy to get all the components of the Web Platform installed, as well as some of the world's most popular ASP.NET and PHP applications as well as media streaming.

EC2 meets the Microsoft Web Platform

EC2 and the Microsoft Web Platform come together in a really slick way. The four AMIs below give you the latest Microsoft Web Platform stack, including Visual Studio 2010 which was just released today. Each image comes with a complete walkthrough.

Full Development Server Image

The "Full Development Server Image" features the latest development stack for the Microsoft Web Platform and since yesterday it includes Visual Studio 2010. It is set up to allow you the publishing of your web application from your local development machine directly to your EC2 instance of this image in the cloud. The image runs Windows Server 2008. It includes the following Microsoft Web Platform features and products:

Here is a link to the walkthrough. It shows you how to set up an EC2 account, how to publish an ASP.NET 4.0 web application to your EC2 instance.

Media Server Image

Smooth Streaming, an IIS Media Services extension, enables adaptive streaming of media to Silverlight and other clients over HTTP. Smooth Streaming provides a high-quality viewing experience that scales massively on content distribution networks, making true HD 1080p media experiences a reality.

Smooth Streaming is the productized version of technology first used by Microsoft to deliver on-demand video of the 2008 Summer Olympics for NBCOlympics.com. By dynamically monitoring both local bandwidth and video rendering performance, Smooth Streaming optimizes playback of content by switching video quality in real-time.

The image comes with media content and is able to stream video as soon as you boot up your EC2 instance. It is very easy to replace the built-in media content with your own video content - all shown in the accompanying walkthrough.

DotNetNuke Image

DotNetNuke is the leading Web Content Management System (CMS) and Application Development Platform for Microsoft .NET. This AMI has DotNetNuke 5.2.2 pre-installed. All you have to do is to finish the last few installation steps, all described in detail in the following walkthrough. The image contains Internet Information Services 7.0, Microsoft SQL Server Express 2008 with SP1, and .NET Framework 3.51 SP1 on Windows Server 2008.

Wordpress Image

WordPress is one of the most popular blogging platforms available. The Wordpress AMI is pre-configured with Microsoft Internet Information Services 7.0, MySQL Server 5.1 Community Edition, PHP 5.2.13, PHP WinCache 1.0 and Wordpress 2.9.2 - all hosted on Windows Server 2008. The only thing you have to do is to finish the Wordpress installation by entering a few questions (usernames, passsword etc.) and within a few minutes you have a state-of-the-art blogging platform live on the Internet. The walkthrough guides you through all the details.

Questions or comments? Send them to ec2@microsoft.com.

A New Site - Quick!

thomad — Sat, 10 Apr 2010 07:08:00 GMT

For testing purposes I sometimes need a new IIS site, often times running in its own AppPool. And I usually need it fast and without much programming. I hate ports and so I normally put my site name in the hosts file. Locally this works great!

Because I'm an old kind-a guy I still use batch files with some pixie dust from appcmd. I thought somebody might find this script useful, too:

@ECHO OFF
REM ===============================================================================================
REM CREATE A NEW SITE IN ITS OWN APPPOOL. 
REM ARGUMENTS: SITENAME 
REM
REM STEPS:
REM 1) SETUP AND ARGUMENTS CHECKS
REM 2) CREATE APPPOOL (SAME NAME AS SITE)
REM 3) CREATE DIRECTORY c:\inetpub\<sitename>
REM 4) CREATE DEFAULT.ASPX FILE IN c:\inetpub\<sitename>
REM 5) CREATE SITE WITH PHYSICALPATH c:\inetpub\<sitename>
REM 6) ASSIGN SITE ROOT APP TO APPLICATIONPOOL <sitename>
REM 7) ADD SITENAME TO HOSTS FILE RESOLVING TO 127.0.0.1
REM 8) LAUNCH IE WITH SITE 
REM ===============================================================================================


REM GOTO USAGE IF NO COMMAND-LINE ARGUMENTS
if "%1" == "" GOTO USAGE





REM CREATE SOME LOCAL VARIABLES
SETLOCAL
SET NEWDIR="%SYSTEMDRIVE%\inetpub\%1"
SET APPCMD=%WINDIR%\system32\inetsrv\appcmd.exe





@ECHO CREATE THE APPPOOL
%APPCMD% ADD APPPOOL /NAME:"%1"



@ECHO CREATE THE CONTENT DIRECTORY
MD %NEWDIR%



@ECHO CREATE DEFAULT.ASPX FILE IN CONTENT DIRECTORY
ECHO ^<%%=DateTime.Now%%^> > "%NEWDIR%\default.aspx"



@ECHO CREATE NEW SITE 
%APPCMD% ADD SITE /NAME:"%1" /BINDINGS:"http://%1:80" /PHYSICALPATH:"%NEWDIR%"



@ECHO ASSIGN TO APPLICATION POOL
%APPCMD% SET APP "%1/" /APPLICATIONPOOL:"%1"



@ECHO ADD SITENAME TO HOSTS FILE
ECHO 127.0.0.1  %1 >> %WINDIR%\system32\drivers\etc\hosts



@ECHO LAUNCH INTERNET EXPLORER WITH THE NEW SITE
START IEXPLORE http://%1/

REM USAGE MESSAGE IF NO COMMAND-LINE ARGUMENT WAS GIVEN

GOTO EXIT
:USAGE
@ECHO USAGE: %0 ^<name of new site^>
@ECHO     Example: "%0 MyNewSite"
:EXIT

How to run a CGI program under IIS 7.0 or IIS 7.5

thomad — Mon, 05 Apr 2010 06:32:03 GMT

Looking around I didn't find a good documentation on how to get good old CGI's running on IIS 7 or 7.5. Here is a quick walkthrough:

1. Let's write a quick CGI:
Take the following code and save it as simplecgi.cs in the directory c:\inetpub\wwwroot\cgi

using System;
using System.Collections;

class SimpleCGI
{
    static void Main(string[] args)
    {
        Console.WriteLine("\r\n\r\n");
        Console.WriteLine("<h1>Environment Variables</h1>");
        foreach (DictionaryEntry var in Environment.GetEnvironmentVariables())
            Console.WriteLine("<hr><b>{0}</b>: {1}", var.Key, var.Value);
    }
}

2. Change into the C:\inetpub\wwwroot\cgi directory and compile the source by using the following command-line:


%windir%\Microsoft.NET\Framework\v2.0.50727\csc.exe SimpleCGI.cs

You will have simplecgi.exe in the cgi directory. You can execute it on the command-line to see its output.

3. For security reasons every CGI has to be registered in the ISAPI/CGI Restriction list. To do that you have to open INETMGR, click the machine node (name of your machine) and find the ISAPI/CGI Restriction List menu icon.

Select the item and add the following entries in the dialog box:

Alternatively you can use the command-line:


%windir%\system32\inetsrv\appcmd set config -section:isapiCgiRestriction 
/+[path='c:\inetpub\wwwroot\cgi\simplecgi.exe',allowed='true',description='SimpleCGI']

4. The next step is to create a virtual directory that allows CGIs to execute. Right click the "Default Web Site" in INETMGR and select "Add Virtual Directory". Add the following entries:

Here is the command-line which does the same:

appcmd add vdir /app.name:"Default Web Site/" /path:/cgi 
/physicalPath:"c:\inetpub\wwwroot\cgi"

5. One last step: we still don't allow "Execute" access in this directory. For this you have to go to the Handler Mappings menu of the CGI virtual directory (make sure you select the CGI virtual directory on the left hand side!).

Go to the "Edit Feature Permissions" link in the Actions Menu on the right hand side, open it and check "Execute" and click "OK".

via command-line:

appcmd set config "Default Web Site/cgi" /section:handlers 
-accessPolicy:"Read,Script,Execute"

Now you are ready to go. Type "http://localhost/cgi/simplecgi.exe and you should see the following output:

Wasn't to bad after all? Was it?

Measuring incoming and outgoing bandwidth per Application Pool

thomad — Wed, 31 Mar 2010 18:00:21 GMT

This blog post is about working in a big company where the left hand often times doesn't know what the right hand does.

These days we are talking quite a bit about how to best measure resource consumption (disk, memory, CPU, requests per second, incoming, outgoing bandwidth) of individual Application Pools. As part of this exercise we looked at IIS and HTTP.SYS performance counters and found a pretty embarrassing issue when it comes to measuring bandwidth.

You might know that HTTP.SYS does a lot of content caching for IIS. The benefit is that some requests can be directly returned from kernel mode without ever having to go to user mode where IIS lives. This can speed up response time quite drastically. IIS has a per Web-Site performance counter that measures bandwidth ("Web Service", "Bytes Received/sec" and Web Service", "Bytes Sent/sec"). The problem is that this only measures the bandwidth for requests that make it to user mode. All requests that get cached by HTTP.SYS in kernel-mode will not be measured.

The good thing is that HTTP.SYS has the same performance counters ("HTTP Service Url Groups", "BytesReceivedRate" and "HTTP Service UrlGroups", "BytesSentRate"). Great!

Here comes the issue though: the counters are per-Url Group and not per Web-Site or per Application Pool and there is no obvious way to figure out which Web Site or Application Pool is in which Url Group (what Url Groups are is explained here). HTTP.SYS provides no APIs to get to this information. The only way to make the connection between a Url Group and an Application Pool is to parse the output of the "NETSH.EXE HTTP SHOW SERVICESTATE" command which shows a snapshot of the HTTP.SYS service configuration. By parsing out the UrlGroups and the associated Application Pool you can now write a program that measures the incoming and outgoing bandwidth on a per Application Pool basis.

Last night I wrote a PowerShell script that does exactly that. Here roughly the steps the script follows:

Execute the netsh command and store the output in variable $file_lines
Loop through each line and find the string "URL group ID: "
If the string is found we look for the associated AppPool which is shown after the string "Request queue name: "
Ignore non-IIS Url Groups ("Request queue is unnamed.")
Add URL Group/AppPool combo to a hash table which has the Application Pool as a key and an array of Url Groups as its value.
Create a perf counter for BytesSentRate, BytesReceivedRate and AllRequests.
Loop through the Url Groups for each Application Pool and add the values together to get the total value.
Write values to console.

$URLGROUP_STRING = "URL group ID: ";
$APPPOOL_NAME_STRING = "Request queue name: ";
$NON_IIS_URLGROUP = "Request queue is unnamed.";
$UrlPoolHashTable = @{};


$file_lines = &"c:\windows\system32\netsh.exe" http show servicestate

for($i = 0; $i -lt $file_lines.Length; $i++)
{
    $res = $file_lines[$i].IndexOf("$URLGROUP_STRING");    
    if ($res -ge 0)
    {
        $urlgroup = $file_lines[$i].SubString($res + $URLGROUP_STRING.Length);        
        while (++$i-lt $file_lines.Length)
        {        
            $res = $file_lines[$i].IndexOf("$APPPOOL_NAME_STRING");
            if ($res -ge 0)
            {
                $appPool = $file_lines[$i].`
                                SubString($res + $APPPOOL_NAME_STRING.Length);
                if ($false -eq $appPool.Contains("$NON_IIS_URLGROUP"))
                {
                    if ($UrlPoolHashTable.ContainsKey($appPool))
                    {
                        $UrlPoolHashTable[$appPool] += "$urlgroup";                    
                    }
                    else
                    {
                        $UrlPoolHashTable[$appPool] = , "$urlgroup";
                    }                    
                }
                break;
            }
        }
    }    
}


foreach ($appPoolkey in $UrlPoolHashTable.Keys)
{
    $bwOut = new-object System.Diagnostics.PerformanceCounter(
                            "Http Service Url Groups","BytesSentRate");
    $bwIn  = new-object System.Diagnostics.PerformanceCounter(
                            "Http Service Url Groups","BytesReceivedRate");
    $req  = new-object System.Diagnostics.PerformanceCounter(
                            "Http Service Url Groups","AllRequests");

    $inTotal = 0;
    $outTotal = 0;
    $reqTotal = 0;
    foreach ($urlgroup in $UrlPoolHashTable[$appPoolkey])
    {
        $bwOut.instanceName = $urlgroup;
        $bwIn.instanceName = $urlgroup;
        $req.instanceName = $urlgroup;
        $outTotal += $bwOut.RawValue;
        $inTotal += $bwIn.RawValue;
        $reqTotal += $req.RawValue;        
    }
    "Application Pool: $appPoolKey";
    "             Bandwidth out: " + $outTotal + " bytes";
    "             Bandwidth in:  " + $inTotal + " bytes";
    "             Request Total: " + $reqTotal;
        
    
}

P.S: The counter gets reset when the Url Group is removed, e.g. when the W3SVC service is stopped. Also: this probably only works with the english version of Windows. I'm assuming that localized Windows versions will have different strings to search for.

Put the brakes on your Application Pools: CPU Rate Limits in Windows Server 2008

thomad — Tue, 16 Feb 2010 00:52:59 GMT

Are you using IIS Application Pools to host your customer's web sites? The more sites you host on a single machine the more likely it is one of these sites misbehaves and monopolizes system resources like the CPU. It's so easy for a programmer to do - forget to call the MoveNext() function in a database script and you spin the CPU at 100% until the underlying script engine times out.

This can mean of course that other sites on your machine might not get enough CPU cycles and starve. All this because somebody doesn't play nice and eats all the pie. CPU rate limits on Windows Server 2008 and Windows Server 2008 R2 allow you to cut the pie more equally.

CPU rate limits can limit a particular user account to consume only a configurable percentage of each physical CPU, e.g. 10%. This becomes very handy if the new IIS Application Pool Identities feature in IIS 7.5 (Windows Server 2008 R2) or Windows Sever 2008 SP2 is used.

The following technet article which describes the feature in detail.

I wrote a PowerShell script iif you want to try it yourself:

           param ( [Parameter(Mandatory=$true)][string] $account, [Parameter(Mandatory=$true)][int] $rate) 
$objAccount = New-Object System.Security.Principal.NTAccount($account) 
$strSID = $objAccount.Translate([System.Security.Principal.SecurityIdentifier]).Value 
if ($strSID -eq $null) 
{ 
 "Account $account cannot be translated into a SID"; 
 return; 
} 
"SID for $account : `t$strSID"; 
$quota_hive = "HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\Quota System\$strSID" 

$out = New-Item -path "$quota_hive" -force 
$out = New-ItemProperty -path "$quota_hive" -name CpuRateLimit -value $rate 
"CPU Rate for account $account set to $rate %"

          

And here is the same as command-line tool if you don't want to use a PowerShell script.

Now here comes the drawback. The CPU Rate Limit feature has a bug. The kernel is holding on to a handle to the quota object and never lets go of it. This means once you set the CPU rate limit to a particular percentage you can't change this percentage without rebooting the machine. We are working on fixing it - not sure when we'll have a fix though.

IIS7 and Failover Clustering

thomad — Tue, 27 Oct 2009 07:00:00 GMT

We recently published an article on how to enable failover clustering for IIS7 and Microsoft Cluster Service (MSCS). MSCS provides failover and increased availability of applications by failing over to a second machine that is on stand-by or by just taking the sick machine out of the network so that no requests are routed to it anymore.

There is a fundamental problem with configuring failover clustering for IIS however: IIS in itself is not an application; IIS is an application platform and it's hard to come up with a clear yay or nay when to fail over. We don't know enough to decide if an application is terminally sick and can't recover. Is it when a web application returns 500 errors, when the application is overloaded and TCP connections can't be established anymore, when its Application Pool crashes repeatedly? What if more than one application is hosted on the IIS machine? Should IIS only fail over when W3SVC is terminated? Should IIS fail over if one of the web applications is dead, when 50% of the apps are dead? 90%?

There is no good way to make these decisions without involving the people managing the IIS machine and the web application developers that run code on the IIS box. And that's why we decided to publish sample code that somebody can take and adjust for their specific needs.

The sample code in the article is a script that can be configured as a Generic Script Resource in the MSCS service. The script monitors the run-time status of the IIS7 "Default Web Site" and IIS7 "DefaultAppPool" and will trigger a fail over if one of them is stopped. It is trivial to change the Site or Application Pool name in the script and it should be easy for a developer familiar with VBScript to add some custom functionality to this script.

Happy fail over!

Kerberos Authentication Issues

thomad — Sat, 24 Oct 2009 05:30:58 GMT

We ran into some problems with Kerberos authentication lately and this forced me to unearth the knowledge I once had about Kerberos.

The Issue:
Some Microsoft employees ran into 400 errors when accessing certain Intranet web pages. A 400 error is pretty much a bad request. Kerberos authentication was the cause for this problem.

The Details:
The web sites in question returned a "400 error - request too long" for content protected with IIS Windows Integrated Authentication. It turns out that people who faced this problem had an Authorization header that was larger than 16kB and that HTTP.SYS (the kernel-mode driver underneath IIS) has a request limit of 16kB, i.e. no request can be larger than 16kB by default.

The problem and the fix - increasing the maximum header size and the maximum request size of HTTP.SYS - is described in this KB article. So why would the Kerberos Authorization get larger than 16kB? Turns out that the Kerberos ticket which is encoded in the Authorization header contains all group memberships of the user who wants to authenticate against the web server. The more groups a user is a member of the larger the Authorization header gets. The magic number of group memberships seems to be around 300.

So here are a couple of ways to investigate the problem if you think you run into similar issues:

1) The whoami tool can tell you your group memberships:

whoami /groups

2) Here is a piece of ASP.NET code that tells you how big the authorization header is and the group memberships of the authenticating user (just save the file as kerb_check.aspx in your wwwroot directory and turn Windows Integrated authentication on and anonymous authentication off):

<%@Language ="C#"%>
<%  
    int authHeaderSize = 0; 
    string username = Request.ServerVariables["AUTH_USER"]; 
    if (username == "")
    {
        Response.Write("<br/>Authentication is not enabled");    
    }
    else
    {
        authHeaderSize = Request.Headers["Authorization"].Length;
        Response.Write("<h2>Size of " + 
                        username + 
                        "'s Authorization header: <br/><b>" + 
                        authHeaderSize + 
                        "</b> bytes. <p/></h2>");
        Response.Write("<h2><br/><b>Group Memberships for user " + username + ":</b></h2>");
        int count=0;
        System.Security.Principal.WindowsIdentity winId = 
                (System.Security.Principal.WindowsIdentity)HttpContext.Current.User.Identity;
        foreach (System.Security.Principal.IdentityReference ir in winId.Groups)
        {
            try
            {
                Response.Write("<br/>" + 
                               ((System.Security.Principal.NTAccount)ir.Translate
                               (typeof(System.Security.Principal.NTAccount))).Value);   
                count++;   
            }
            catch (Exception inner)
            {
                Response.Write("<br/> --- cannot resolve group ---");
            }
        }
        Response.Write("<br/><br/><b>You are member of " + count + " groups.</b>"); 
    }   
 %>

Session-based Authentication
Another problem I discovered while looking at the issue: when Kerberos authentication is used IIS7 and IIS 7.5 force the re-authentication of every request. This is unfortunate because it doesn't scale well. Just imagine a single html page with 50 images. If your Kerberos header is 10kB Internet Explorer sends 510kB (51 requests x10kB authorization header) only for authentication purposes.

When NTLM is used on the other hand, session-based authentication is enabled. If session-based authentication is enabled authentication only has to be done per TCP/IP session. This scales much better because Internet Explorer usually creates only 2 to 6 TCP/IP connections to a web server. HTTP requests are then done within these TCP/IP sessions, i.e. the connections are reused to make multiple requests. So if session-based authentication is enabled instead of sending 510kb authentication headers only 20-60kb authentication headers will be sent by IE.

I'm still investigating why session-based authentication is not used for Kerberos - I hope there is a good reason for it :). There is a configuration property in IIS7.x that allows you to switch Kerberos to session-based authentication. It is the AuthPersistNonNTLM property. If you want to turn on server-wide session-based authentication for Kerberos here is your command:


%windir%\system32\inetsrv\appcmd.exe set config -section:windowsAuthentication /authPersistNonNTLM:"True" /commit:apphost

That's it for today.

Thomas