The Way I See It

IIS 7 Tip # 11 You can restrict SSL to only the Control Channel on FTPS

vijaysk — Tue, 07 Jul 2009 23:04:23 GMT

The File Transfer Protocol (FTP) has been around even before I was born :) It is a good protocol for transferring files across the network but has one major flaw. If you use authentication it sends the password in a Base64 encoded format over the wire. If someone intercepts this he/she can easily decode the password. So if someone intercepts the traffic for your FTP site that allows a user to write files he/she can get access to an username and password that has write permissions on the server.

FTPS (FTP-Secure) address this flaw. FTPS allows you to use TLS or SSL cryptography to encrypt the data that is being transferred.

With Windows 2008 the FTP service was rewritten to support FTPS and shipped as an extension for IIS 7.0. Earlier versions of IIS only support FTP.

With FTPS you no longer have to worry about someone intercepting your password because of the encryption. But encryption comes with a performance overhead. (Consider situations where users frequently transfer files in 100s of MBs to the server all that has to go through en/decryption)

The best thing about the IIS FTPS extension is that you can control the encryption levels.

The FTP (even FTPS) protocol uses two channels(basically two ports) to communicate with the server. The Control Channel is used to transfer the commands (it also carries the credentials) and the Data Channel to transfer the files.

In situations where protecting just the password is enough and you are not concerned about the content being transferred you can remove the encryption on the Data Channel. This will lower the CPU cycles burnt for en/decrypting the large files that are transferred.

Even the Control Channel can be tweaked to just use encryption for credentials. But commands sometimes contain folder locations so its better to keep it encrypted.

Disappearing SSL certificates from IIS 7.0 manager

vijaysk — Fri, 22 May 2009 21:37:33 GMT

“I install a SSL server certificate using the ‘Complete Certificate Request’ wizard in IIS manager and when I refresh the view the certificate disappears. “

I have heard that a couple of times and every time I used to go “What ?” Until someone showed it to me.

If you are one of those who are wondering about this read on.

The Server Certificates module in IIS manager displays a list of certificates from the Local Machine SSL store.

But it only lists the certificate if

1. The certificate has a private key

2. The certificate is meant for Server Authentication

And this is where the disappearing act occurs.

The IIS Manager enumerates all the extensions of the certificate and checks if OID 2.5.29.37 (Extended Key Usage) exists. If it does the certificate Enhanced Key Usage section must contain 1.3.6.1.5.5.7.3.1 (Server Authentication).

In the repro’ I was shown the user had actually downloaded the intermediate certificate and used that .cer file to complete the certificate request. In this case the wizard will go thro’ all the steps but when you refresh the view the certificate will not be listed.

IIS 7 Tip # 10 You can generate machine keys from the IIS manager

vijaysk — Wed, 13 May 2009 22:10:27 GMT

The machineKey element of the ASP.NET web.config specifies the algorithm and keys that ASP.NET will use for encryption. By default the validationKey and the decryptionKey keys are set to AutoGenerate which means the runtime will generate a random key for use. This works fine for applications that are deployed on a single server. When you use webfarms a client request can land on any one of the servers in the webfarm. Hence you will have to hardcode the validationKey and the decryptionKey on all your servers in the farm with a manually generated key.

There are a lot of articles that describe how to use RNGCryptoServiceProvider to generate a random key. There are also a lot of online tools that generate random keys for you. But I would suggest writing your own script because any one who has access to these keys can do evil things like tamper your forms authentication cookie or viewstate.

With IIS 7 you no longer have to do this manually. The IIS 7.0 manager has a built in feature that you can use to generate these keys.

It uses RNGCryptoServiceProvider internally to create a random key. The value is stored locally in the web.config of that application something like

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <system.web>
        <machineKey decryptionKey="F6722806843145965513817CEBDECBB1F94808E4A6C0B2F2,IsolateApps" validationKey="C551753B0325187D1759B4FB055B44F7C5077B016C02AF674E8DE69351B69FEFD045A267308AA2DAB81B69919402D7886A6E986473EEEC9556A9003357F5ED45,IsolateApps" />
    </system.web>
</configuration>

You can copy it and paste it in the web.config file of all the servers in the webfarm.

Setting SMTP RelayIpList from a script.

vijaysk — Thu, 07 May 2009 22:58:00 GMT

SMTP service running on IIS 6.0 provides you options to restrict IP addresses of computers that can relay mail messages through this server.

The value(list of IP addresses) is stored in the RelayIpList metabase property as an octet string. I was recently contacted by a server administrator to figure out if there is an automatic script that can add IP addresses to this list. He wanted to add around 200 IP address and grant them access.

I have written a script that you can use in such scenarios. It reads IP addresses from a text file and populates them into the RelayIpList.

USAGE : cscript ImportRelayList.vbs

PREREQUISITE : This script needs ip.txt in the same folder.

Store your IP addresses in ip.txt FORMAT: Each line should be IP,MASK

I have attached the script to this blog post. Hope it helps.

IIS 7 Tip # 9 Set Application Pool Defaults…

vijaysk — Tue, 05 May 2009 22:57:25 GMT

When ever you create a new application pool IIS only asks you for four things 1. Name, 2. .NET Framework version, 3. Managed pipeline mode and 4. Weather to start the application pool immediately

What about the remaining advanced settings like the identity that application pool has to run under? The application pool inherits the rest of the settings.

If you want to change the default settings that IIS uses to create a new application pool you can do it under the Set Application Pool Defaults… section

For example if you change the Managed Pipeline Mode to Classic in the defaults and add a new application pool the mode dropdown will be set to Classic.

Can the validity period of the Self-Signed Certificate that IIS 7.0 manager creates be changed ?

vijaysk — Fri, 01 May 2009 12:31:00 GMT

IIS 7.0 has a nice feature of creating self-signed certificates … very handy for creating test certificates.

I was recently asked if there is a way to modify the validity period of the certificate it creates.

Unfortunately No!

The module that creates the certificate uses a simple logic of incrementing the year by 1. Something like

SYSTEMTIME systemtime = new SYSTEMTIME();

systemtime.wYear = (short) (DateTime.Now.Date.Year + 1);

systemtime.wMonth = (short) DateTime.Now.Date.Month;

systemtime.wDay = (short) DateTime.Now.Date.Day;

It’s hardcoded and hence the validity of the certificate will always be 1 year from the current date.

Also the name of the certificate will always be the FQDN.

In case you want to create certificates with a shorter validity period or different names there are are a lot of tools like SelfSSL that you can use to generates self-signed certificates.

IIS 7 Tip # 8 You can create Self-Signed SSL certificates from the IIS manager

vijaysk — Wed, 29 Apr 2009 22:53:00 GMT

IIS 7.0 manager has a nice feature of generating self-signed certificates that you can use for running tests.

At the global level select Server Certificates and in the Actions Pane on the right you get the option to Create Self-Signed Certificate…

The wizard that pops up just asks for a friendly name for the certificate.

Note this is just a friendly name that you can use to identify the certificate. It is not the Common Name / Subject.

Once you hit OK IIS manager will create a Self-Signed certificate that has the Common Name as the FQDN of the machine and is valid for a year.

Again the friendly name that you specify will not change the Common Name / Subject of the certificate meaning the Issued To: field will always be the FQDN of the machine.

Apart from creating the certificate IIS Manager also adds the certificate to the Trusted Root Certification Authorities on the machine.

IIS 7 Tip # 7 You can use the application pool identity for the anonymous authentication credentials

vijaysk — Fri, 03 Apr 2009 14:49:00 GMT

When a client accesses a web site on anonymous authentication IIS uses a pre configured account to access the corresponding files on disk. In IIS 5.0 / 6.0 we used a local account called the IUSR_machinename for anonymous authentication. With IIS 7.0 we moved to a generic built in account called IUSR which is now machine dependent.

But in IIS 7.0 you also have the option to use the application pool identity as the anonymous user identity.

In IIS 6.0 if you wanted to use the application pool identity for anonymous access you would have to configure it manually under the Authentication Settings. This would mean the username and password would be saved in multiple locations. Whenever you had to change the password you would have to reset it in the application pool settings and then in the authentication settings.

With IIS 7.0 you just have to configure the user identity in the application pool settings. And in the properties of the Anonymous Authentication module select the Application pool identity option.

Getting better stack traces in Process Monitor / Process Explorer

vijaysk — Thu, 02 Apr 2009 05:57:00 GMT

Process Monitor and Process Explorer are great tools for troubleshooting issues on Windows machines. Process Explorer can be used to investigate a running process from handles to dlls loaded. Process Monitor is my favourate and it can be used to monitor file system / registry activity on a machine. It logs all access to the file system / registry by all processes on the machine (can be filtered).

Process Monitor also shows you the call stack of the thread that lead to the file system / registry access.

The call stack in the above image is not very helpful as it is only showing the offset addresses(under Location). Not a lot of people realize that in both Process Monitor and Process Explorer you can configure a symbol server. You can point to the public Microsoft Symbol Server at http://msdl.microsoft.com/download/symbols and Process Monitor / Process Explorer will download the necessary symbol files and show you a better call stack with all the function names instead of the address offsets.

But to enable Process Monitor / Process Explorer to talk to the Microsoft Symbol Server you need to install WinDbg (Microsoft Debugging Tools For Windows) on the machine. You need this because the dbghelp.dll has to upgraded to enable it to connect to a symbol server.

Once you install WinDbg in Process Monitor go to Options > Configure Symbols and configure the dbghelp.dll and the symbol server path.

So here I have configured the dbghelp.dll path to point to the location where my windbg is installed. The Symbols path is pointing to the Microsoft Symbol Server … It specifies c:\symcache as the location where it can cache the symbol files it downloads.

Now if you go back into Process Monitor / Process Explorer and check the call stack it will look something like this.

Now you get proper function names as per the public microsoft symbols. In the symcache folders you will see all the symbols that got downloaded.

Now this is not limited to just Microsoft symbols. If you have symbols created for your application components you can include those as well and get the function names in the call stack.

IIS 7 Tip # 6 You can reset the TCP connection when “Service Unavailable” happens

vijaysk — Wed, 01 Apr 2009 08:36:00 GMT

Instead of returning the “Service Unavailable” message to the client you can terminate the TCP connection. This can be helpful in environments where you have a Load Balancer which ’understands’ a TCP reset instead of “Service Unavailable”. In IIS 7.0 you can set it via the IIS manager UI in the Advanced Properties of the Application Pool.

This option is available in IIS 6.0 as well. But you need to set it using the adsutil.vbs script (or edit the metabase directly)

cscript adsutil.vbs set w3svc/apppools/loadbalancercapabilities 1

Caution while xcopying IIS 7.0 config files

vijaysk — Fri, 13 Mar 2009 23:11:00 GMT

Metabase.xml is the central store where IIS 6.0 stores most of its configuration information. Its a plain text file and stores all the information in a simple XML format. The XML format naturally raised a notion of being able to XCOPY the config file to another server and transferring the settings with it. But if you copy over a metabase.xml file from another server your IIS admin service will no longer start. This happens because the metabase.xml file contains ACLs that control access to any metabase key. Stored under the AdminACL tag these keys are encoded based on the machinekeys of the server. When you move the metabase.xml to another server the keys can no longer be decoded and hence your IIS Admin service will not be able to start.

With IIS 7.0 we moved to a new XML based configuration store that is modeled after ASP.NET. It is no longer centralized into a single file. The hierarchical store starts with the applicationHost.config file and can be distributed among web.config files under your application.

This move also enables the long lasting idea of xcopy-deployment. You can now have all the settings in a web.config along with your application content and move it around.

Another change that was made is that the local accounts/groups that IIS 6.0 used (IUSR_MACHINENAME / IIS_WPG) were replaced by built-in accounts (IUSR / IIS_IUSRS). The built-in accounts have the same SID across Windows 2008 servers and are not machine specific.

So technically you now have an IIS configuration store that is virtually machine independent and you can just copy your applicationHost.config from one server to another and IIS will pick up the settings and just work.

But there is a catch. Try this.

On an IIS 7.0 server change the application pool identity (for the DefaultAppPool) to a custom domain identity. (Advanced Settings > Application Pool Identity > Custom Account)

Then move the applicationHost.config to a different IIS 7.0 server.

When you try to run a website using the DefaultAppPool you will find that the Application will get disabled with the following error in the event log.

Application pool TestApplicationPool has been disabled. Windows Process Activation Service (WAS) did not create a worker process to serve the application pool because the application pool identity is invalid.

So lets try to change the application pool identity to another domain account or reset the password for that account.

You type in the username and password and hit OK and you will get the following error message

There was an error while performing this operation.

Details: Bad Data. (Exception from HRESULT: 0x80090005)

Any username / password will not work. (You can however set the identity of the application pool to one of the built in accounts.)

Wondering what’s going on ? Initially when you set the application pool identity to a domain account IIS has to keep a local copy of the username and password. So it stores a copy in its applicationHost.config and since it is not advisable to keep the password in clear text format it goes ahead an encrypts it. You will see something like this in the config file.

For the encryption it uses machine specific keys in the iisConfiguration and iisWasKey containers. When the applicationHost.config is moved to a different server IIS can no longer decrypt the settings.

To get this working you can export and import the keys from the original server.

Export using the following commands

aspnet_regiis -px "iisConfigurationKey" "D:\iisConfigurationKey.xml" -pri
aspnet_regiis -px "iisWasKey" "D:\iisWasKey.xml" –pri

And for the import use

aspnet_regiis -pi "iisConfigurationKey" "D:\iisConfigurationKey.xml"
aspnet_regiis -pi "iisWasKey" "D:\iisWasKey.xml"

So whenever you are trying to xcopy-deploy your application on multiple servers you need to check if there are any encrypted sections and if you do ensure you port the iisConfigurationKey and the iisWasKey as well.

Also I would recommend using the Web Deployment Tool ( MSDeploy ) which makes deployment a lot easier. You can create a package (settings and content) of the whole server / specific application and use it to deploy. But the tool is in BETA still.

IIS 7 Tip # 5 Run a command when Rapid Fail Protection is triggered.

vijaysk — Fri, 13 Mar 2009 01:26:00 GMT

Rapid-Fail Protection disables application pools if they crash multiple times within in a specified time period. This prevents the failing application pool from getting into a continuous loop of crashing and restarting. This protects other application pools running on the server as repeated failures can consume lot of system resources. When rapid-fail protection kicks in it stops the application pool that is repeatedly crashing and your clients will start getting a 503 – Service Unavailable error. An administrator will have to manually enable the application pool again.

You also have to option to configure an executable to run when ever rapid-fail protection is triggered. For example below I have configured the application pool to restart the IIS service using iisreset.exe … the /rebootonerror will reboot the whole server if iisreset.exe for some reason fails to restart the services.

<applicationPools>
       <add name="DefaultAppPool" autoStart="true">
         <failure autoShutdownExe="c:\windows\system32\iisreset.exe" autoShutdownParams="/rebootonerror" />
       </add>
</applicationPools>

This option is also available on IIS 6.0 but it is not exposed via the IIS manager. You can set is as follows

cscript adsutil.vbs set W3SVC/AppPools/DefaultAppPool/autoshutdownapppoolexe “c:\windows\system32\iisreset.exe”

IIS 7 Tip # 4 Application Pool Recycling Events has an UI

vijaysk — Mon, 09 Mar 2009 23:11:00 GMT

Tucked away in the IIS manager is an UI screen to configure the Events that get logged when an application pool is recycled.

When you right click on the application pool and choose Recycling you get to the screen where you configure the Recycling Conditions. When you click ‘Next’ you will get a screen where you can configure the events that are logged when recycling happens.

You can also set these settings from the Advanced Settings of an application pool

On IIS 6.0 you had to configure this using the adsutil.vbs script (or directly edit the metabase.xml) ref: http://support.microsoft.com/kb/332088

Also the Event Viewer on Windows 2008 has been improved and you can now configure it to send emails whenever an event is triggered.

As a web administrator you might want to keep a tab on the Runtime recycling events. For example the Unhealthy ISAPI recycling usually indicates a web application ‘hang’ because of which it is being recycled. From the IIS Manager you can turn on the Runtime recycling so that those events are logged in the event viewer. Then from the Event Viewer you can attach a task to those events and configure it to send out an email. So every time there is an application pool recycle you get automatically notified via email.

IIS 7 Tip # 3 You can now load the user profile of the application pool identity

vijaysk — Sun, 08 Mar 2009 07:41:00 GMT

IIS 6.0 does not load the user profile of the application pool identity. But with IIS 7.0 you now have a choice to load the profile if needed. This feature is disabled by default on Windows 2008.

<applicationPools>
    <add name="DefaultAppPool">
        <processModel identityType="NetworkService" loadUserProfile="true" idleTimeout="00:05:00" />
    </add>
</applicationPools>

If you change this to True the profile of your Application Pool is loaded and is available for your application.

You can use this to isolate your applications even further. For example when this option is set to False ( the profile is not loaded ) your application will use the c:\windows\temp folder as its temporary directory. If you have other application pools even they will use the same c:\windows\temp folder. If you set the option to load the user profile the temporary directory will be now change to use the profile’s temporary folder C:\Users\apppooluserid\AppData\Local\Temp.

If the profile is loaded you also have access to all the custom environment variables for that user.

Here’s a question what do you think will the temporary folder be when the Application Pool identity is set to Network Service and Load User Profile is set to True ?

It is not C:\Users\NetworkService\AppData\Local\Temp. but C:\Windows\ServiceProfiles\NetworkService\AppData\Local\Temp

IIS 7 Tip #2 You can now run 32 bit and 64 bit applications on the same server

vijaysk — Fri, 06 Mar 2009 20:01:00 GMT

On a 64 bit Windows 2003 machine IIS 6.0 could either be run in 32 bit mode or 64 bit mode. You toggled the Enable32bitAppOnWin64 metabase key and all the worker process would run in a particular bitness mode.

With IIS 7.0 the Enable32bitAppOnWin64 key has been moved to the Application Pool level. So you can now set the bitness of a particular Application Pool.

appcmd set apppool /apppool.name:DefaultAppPool /enable32bitapponwin64:true

This will make only the DefaultAppPool to run in 32 bit mode. You can have other application pools running simultaneously in 64 bit mode.

You can also list the apppools based on bitness using the appcmd command. To list all the application pools runningin 64bit mode use the following command

appcmd list apppools /enable32bitapponwin64:false

Now since application pools can be run in different bitness mode we also need to ensure that the dlls/modules that get loaded in the process are of the correct bitness. To do this you can configure a preCondition. For example this global module entry specifies that it should only be loaded if the bitness of the application pool is 32 bit.

You can also set the application bool bitness via the IIS manager