Brian Desmond - Exchange

Cisco ACE Sample Configuration for Exchange 2010

Cisco’s ACE appliances and modules are something that I see constantly at customers. Unfortunately, Cisco’s application specific documentation is rather lacking. There’s a rather simplistic sample config at http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/App_Networking/Exchange_VSphere_UCS_NetApp.html#wp345264, but, that doesn’t really have everything you need. The folks at F5 on the other hand have extremely well documented application guides for their hardware and that’s one of the reasons I usually send customers to F5 first. The contents of this post are essentially the same for ACE appliances and ACE modules.

In any case, the rest of this post is a working sample configuration for the topology below. A thread on a discussion alias prompted me to clean this up and post it.

In our sample configuration, we’ll configure the load balancer in a one-armed configuration with Source NAT (SNAT). Clients will access services via the mail.contoso.com and autodiscover.contoso.com URLs. In addition, clients who go to a non SSL URL or go to the root instead of /owa will be redirected.

Before we get started, a few pre-requisites:

You should have a working context on your ACE appliance/module with resources allocated.
Your CAS servers should have static ports configured as described here.
Your SSL certificate and the necessary intermediate and root certificates should be installed on the load balancer.
A VIP will need to be allocated. We’ll use 192.168.100.200 in this sample
If you’re expecting more than ~65K client connections, you’ll need to use more than one IP in the SNAT pool. This example assumes one is enough. We’ll use 192.168.100.199 in this sample.

The entire config is posted below, but, first, I wanted to speak to a few specific elements and why they’re here:

crypto chaingroup DigiCert cert DigiCertRootCA cert DigiCertIntermediateCA cert mail_contoso_com	This creates a valid SSL certificate chain
serverfarm host Exchange-CAS-RPC predictor leastconns probe TCP135 probe TCP60000 probe TCP60001 fail-on-all rserver CAS01 inservice rserver CAS02 inservice	Here we’re validating the availability of the RPC ports on the CAS server. The “fail-on-all” keyword is key here. This makes sure that if any of the three services are unavailable, the server is removed from the farm
parameter-map type http Exchange-OWA case-insensitive persistence-rebalance set header-maxparse-length 16384 set content-maxparse-length 8192	There are three things going on here: We’re making URL matching case insensitive persistence-rebalance makes sure that each HTTP request is evaluated individually, and that sticky cookies are inserted only on the first request The max parse lengths define how far into a request the ACE looks for cookies, headers, and so forth. Without this, you’ll likely see issues with partially loaded images and random session timeouts. It’s possible that you’ll need to extend the numbers here.
sticky ip-netmask 255.255.255.255 address source Exchange-CAS-RPC timeout 7200 replicate sticky serverfarm Exchange-CAS-RPC	Here we define source IP stickiness. This is used for the RPC services. Warning: If you have source NAT upstream (such as in a network merger scenario) where many clients are appearing to come from the same IP, this will cause load balancing to not be even.
sticky http-cookie Exchange-Sticky Exchange-CAS-HTTPS-Cookie cookie insert browser-expire replicate sticky serverfarm Exchange-CAS-HTTPS	Here we create a cookie valid for the life of the browser session to ensure that OWA and ECP sessions go to the same CAS server.
action-list type modify http Exchange-CAS-HTTP header insert request X-Forwarded-For header-value "%is"	This enables us to insert the client’s original IP as an additional HTTP header so that IIS logs can be correlated to a specific client.
policy-map type loadbalance first-match Exchange-CAS-HTTPS match OWA http url /owa.* sticky-serverfarm Exchange-CAS-HTTPS-Cookie action Exchange-CAS-HTTP ssl-proxy client Exchange-CAS match ECP http url /ecp.* sticky-serverfarm Exchange-CAS-HTTPS-Cookie action Exchange-CAS-HTTP ssl-proxy client Exchange-CAS match EWS http url /ews.* sticky-serverfarm Exchange-CAS-HTTPS-Cookie action Exchange-CAS-HTTP ssl-proxy client Exchange-CAS match ActiveSync http url /Microsoft-Server-ActiveSync.* sticky-serverfarm Exchange-CAS-HTTPS-AuthZHeader action Exchange-CAS-HTTP ssl-proxy client Exchange-CAS match OutlookAnywhere http header User-Agent header-value "MSRPC" sticky-serverfarm Exchange-CAS-HTTPS-AuthZHeader action Exchange-CAS-HTTP ssl-proxy client Exchange-CAS class Exchange-CAS-HTTPS-RootRequest serverfarm Exchange-OWA-REDIRECT class class-default sticky-serverfarm Exchange-CAS-HTTPS-SourceIP action Exchange-CAS-HTTP ssl-proxy client Exchange-CAS	This policy map enables SSL decryption, inspection of the URL, and application of the correct load balancing action and stickiness.

crypto chaingroup DigiCert
  cert DigiCertRootCA
  cert DigiCertIntermediateCA
  cert mail_contoso_com

access-list all line 10 extended permit ip any any 
access-list all line 20 extended permit icmp any any 

probe https Exchange-OWA
  interval 30
  ssl version all
  request method get url GET /owa/auth/logon.aspx
  expect status 400 404
probe tcp TCP135
  description RPC Endpoint Mapper
  port 135
  interval 30
  connection term forced
probe tcp TCP60000
  description RPC Client Access
  port 60000
  interval 30
  connection term forced
probe tcp TCP60001
  description Address Book Service
  port 60001
  interval 30
  connection term forced

rserver host CAS01
  ip address 192.168.100.101
  inservice
rserver host CAS02
  ip address 192.168.100.102
  inservice
rserver redirect OWA-SSL-REDIRECT
  webhost-redirection https://mail.contoso.com/owa 302
  inservice

serverfarm host Exchange-CAS-HTTPS
  predictor leastconns
  probe Exchange-OWA
  rserver CAS01 443
    inservice
  rserver CAS02 443
    inservice
serverfarm host Exchange-CAS-RPC
  predictor leastconns
  probe TCP135
  probe TCP60000
  probe TCP60001
  fail-on-all
  rserver CAS01
    inservice
  rserver CAS02
    inservice
serverfarm redirect Exchange-OWA-REDIRECT
  rserver OWA-SSL-REDIRECT
    inservice

parameter-map type http Exchange-OWA
  case-insensitive
  persistence-rebalance
  set header-maxparse-length 16384
  set content-maxparse-length 8192
parameter-map type ssl SSL_PARAMS
  cipher RSA_WITH_RC4_128_MD5
  cipher RSA_WITH_RC4_128_SHA
  cipher RSA_WITH_3DES_EDE_CBC_SHA

sticky ip-netmask 255.255.255.255 address source Exchange-CAS-RPC
  timeout 7200
  replicate sticky
  serverfarm Exchange-CAS-RPC
sticky http-cookie Exchange-Sticky Exchange-CAS-HTTPS-Cookie
  cookie insert browser-expire
  replicate sticky
  serverfarm Exchange-CAS-HTTPS
sticky http-header Authorization Exchange-CAS-HTTPS-AuthZHeader
  timeout 7200
  replicate sticky
  serverfarm Exchange-CAS-HTTPS
sticky ip-netmask 255.255.255.255 address source Exchange-CAS-HTTPS-SourceIP
  timeout 7200
  replicate sticky
  serverfarm Exchange-CAS-HTTPS

action-list type modify http Exchange-CAS-HTTP
  header insert request X-Forwarded-For header-value "%is"

ssl-proxy service Exchange-CAS
  key mail_contoso_com
  cert mail_contoso_com
  chaingroup DigiCert
  ssl advanced-options SSL_PARAMS

class-map match-any Exchange-CAS-HTTPS
  2 match virtual-address 192.168.100.200 tcp eq https
class-map type http loadbalance match-any Exchange-CAS-HTTPS-RootRequest
  2 match http url /
class-map match-any Exchange-CAS-RPC
  2 match virtual-address 192.168.100.200 tcp eq 60001
  3 match virtual-address 192.168.100.200 tcp eq 60000
  4 match virtual-address 192.168.100.200 tcp eq 135
class-map match-any Exchange-OWA-REDIRECT
  2 match virtual-address 192.168.100.200 tcp eq www
class-map type management match-any mgmt-cm
  2 match protocol https any
  3 match protocol snmp any
  4 match protocol ssh any
  5 match protocol icmp any

policy-map type management first-match mgmt-pm
  class mgmt-cm
    permit

policy-map type loadbalance first-match Exchange-CAS-HTTPS
  match OWA http url /owa.*
    sticky-serverfarm Exchange-CAS-HTTPS-Cookie
    action Exchange-CAS-HTTP
    ssl-proxy client Exchange-CAS
  match ECP http url /ecp.*
    sticky-serverfarm Exchange-CAS-HTTPS-Cookie
    action Exchange-CAS-HTTP
    ssl-proxy client Exchange-CAS
  match EWS http url /ews.*
    sticky-serverfarm Exchange-CAS-HTTPS-Cookie
    action Exchange-CAS-HTTP
    ssl-proxy client Exchange-CAS
  match ActiveSync http url /Microsoft-Server-ActiveSync.*
    sticky-serverfarm Exchange-CAS-HTTPS-AuthZHeader
    action Exchange-CAS-HTTP
    ssl-proxy client Exchange-CAS
  match OutlookAnywhere http header User-Agent header-value "MSRPC"
    sticky-serverfarm Exchange-CAS-HTTPS-AuthZHeader
    action Exchange-CAS-HTTP
    ssl-proxy client Exchange-CAS
  class Exchange-CAS-HTTPS-RootRequest
    serverfarm Exchange-OWA-REDIRECT
  class class-default
    sticky-serverfarm Exchange-CAS-HTTPS-SourceIP
    action Exchange-CAS-HTTP
    ssl-proxy client Exchange-CAS
policy-map type loadbalance first-match Exchange-CAS-RPC
  class class-default
    sticky-serverfarm Exchange-CAS-RPC
policy-map type loadbalance http first-match Exchange-OWA-REDIRECT
  class class-default
    serverfarm Exchange-OWA-REDIRECT

policy-map multi-match vlan100
  class Exchange-OWA-REDIRECT
    loadbalance vip inservice
    loadbalance policy Exchange-OWA-REDIRECT
  class Exchange-CAS-RPC
    loadbalance vip inservice
    loadbalance policy Exchange-CAS-RPC
    loadbalance vip icmp-reply active
    nat dynamic 1 vlan 1000
  class Exchange-CAS-HTTPS
    loadbalance vip inservice
    loadbalance policy Exchange-CAS-HTTPS
    loadbalance vip icmp-reply active
    nat dynamic 1 vlan 1000
    appl-parameter http advanced-options Exchange-OWA
    ssl-proxy server Exchange-CAS

interface vlan 100
  ip address 192.168.100.10 255.255.252.0
  access-group input all
  nat-pool 1 192.168.100.199 192.168.100.199 netmask 255.255.255.255 pat
  service-policy input mgmt-pm
  service-policy input vlan100
  no shutdown

ip route 0.0.0.0 0.0.0.0 192.168.100.1

Setting Static Ports for Exchange Client Access

If you are deploying Exchange in an environment with load balancers or firewalls which aren’t able to handle dynamic RPC port ranges, chances are you’ll be defining static ports for the RPC Client Access Service and the Address Book Service on each CAS server. If you’re using Public Folders, you’ll want a third static port on the Mailbox servers hosting Public Folders. I typically use these ports for this:

RPC Client Access Service – TCP 60,000
Address Book Service – TCP 60,001
RPC Client Access (Public Folders) – TCP 60,002

For the first two, I’ve included a script below which makes quick work of setting the ports. Just run it on the CAS server to make the required changes.

param([int32]$MAPIPort = 60000, [int32]$AddressBookPort = 60001, [bool]$RestartServices = $true)
# ==============================================================================================
# NAME: Configure Exchange Static Ports
# 
# AUTHOR: Brian Desmond, brian@briandesmond.com
# DATE  : 4/9/2012
# 
# COMMENT: 
# 
# ==============================================================================================

Set-PSDebug -Strict:$true

function CheckProcessElevation()
{
    $identity  = [System.Security.Principal.WindowsIdentity]::GetCurrent()
    $principal = New-Object System.Security.Principal.WindowsPrincipal($identity)

    return $principal.IsInRole([System.Security.Principal.WindowsBuiltInRole]::Administrator)
}

function CreateRegistryKeyIfNecessary([string]$path)
{
    if (Test-Path -Path $path)
    {
        return
    }
    else
    {
        [void](New-Item -Path $path)
    }
}

Function Test-RegistryValue($regkey, $name) 
{
    Get-ItemProperty $regkey $name -ErrorAction SilentlyContinue | Out-Null  
    $?
}

function CreateOrUpdateRegistryValue([string]$path, [string]$valueName, [Microsoft.Win32.RegistryValueKind]$valueType, $value)
{
    if ((Test-Path -Path $path) -ne $true)
    {
        CreateRegistryKeyIfNecessary $path
    }
    
    if ((Test-RegistryValue $path $valueName) -eq $false)
    {
        [void](New-ItemProperty -Path $path -Name $valueName -PropertyType $valueType -Value $value)
    }
    else
    {
        [void](Set-ItemProperty -Path $path -Name $valueName -Value $value)
    }
}

if ((CheckProcessElevation) -eq $false)
{
    Write-Warning "Script must be run from an elevated prompt. Exiting..."
    exit 1
}

$domtParamsPath = "HKLM:\System\CurrentControlSet\Services\MSExchangeAB\Parameters"
$momtParamsPath = "HKLM:\System\CurrentControlSet\Services\MSExchangeRPC\ParametersSystem"

Write-Host "Setting Address Book Service Port to $($AddressBookPort)"
CreateOrUpdateRegistryValue $domtParamsPath "RpcTcpPort" "String" $AddressBookPort.ToString()
Write-Host "Setting RPC Client Access Port to $($MAPIPort)"
CreateOrUpdateRegistryValue $momtParamsPath "TCP/IP Port" "DWord" $MAPIPort

if ($RestartServices)
{
    Write-Host "Restarting Services..."
    Restart-Service -Name "MSExchangeAB" -Confirm:$false
    Restart-Service -Name "MSExchangeRPC" -Confirm:$false
}
Write-Host "Complete." -ForegroundColor Green

If you’re looking to restrict the port used for Public Folder access, you’ll need to do this in addition to the script above. The registry setting you want is below:

Key: “HKLM\System\CurrentControlSet\Services\MSExchangeRPC\ParametersSystem”

Value Name: "TCP/IP Port”

Value Type: REG_DWORD

Value Data: “60002” (decimal)

Add Office 365 Exchange Online to your PowerShell Profile

The Exchange Online component of Office365 exposes a variant of the Exchange Management Shell that you’d normally use if you were managing an on-premises Exchange 2010 organization. Connecting to it requires a few steps which are documented here. I’ve been pasting in the three commands one at a time now for months and it’s gotten rather annoying. A bit of research reveals that you can add custom PowerShell code that is available anytime you launch a shell by modifying your PowerShell profile. You can read more about the various profiles you can modify here, but, I decided to simply modify the one specific to my user account. To do this, open a new PowerShell window and run this command:

notepad $PROFILE

If you haven’t done this before, notepad will prompt you to create a new file. Plug this code in the resultant file:

function Connect-ExchangeOnline
{
  $LiveCred = Get-Credential
  $global:Session365 = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.outlook.com/powershell/ -Credential $LiveCred -Authentication Basic -AllowRedirection
  
  Import-PSSession $global:Session365
}

function Disconnect-ExchangeOnline
{
  Remove-PSSession $global:Session365
}

Save and restart PowerShell, and you’ll be able to run Connect-ExchangeOnline to connect to Exchange Online in one easy step.

Configuring the Dell/Quest Free/Busy Connector for Lotus Notes and Exchange - Part 1

The goal of this post is to introduce the Quest Free/Busy (F/B) connector that comes with Coexistence Manager for Notes (CMN), discuss how it works, and discuss the interface with Exchange. In this post we’ll also configure the Quest Web Services and Domino Free Busy Connector Service. Future posts in this series will discuss configuring the remaining components of the CMN F/B Connector. First, let’s take a look at the sample environment we’ll be using for this discussion:

There are three components of the F/B Connector which you’ll need to deploy:

Domino Free/Busy Service – This component is responsible for accepting F/B requests from Exchange users, retrieving, and processing the data from Domino, and returning it to Exchange. This component also includes two web services which run inside IIS:
- Autodiscover implementation
- Exchange Web Services (EWS) implementation
Exchange Free/Busy Service – This component is responsible for accepting F/B requests from Lotus Notes (via QCALCON) for Exchange users, retrieving and processing the data, and returning it to QCALCON.
QCALCON Task – This is a Domino server tasks from Quest which handles requests for Exchange user F/B information. These requests are sent to the Quest Exchange Free/Busy Service.

Quest recommends that you separate the first two components on to two separate servers for performance reasons. They don’t make any data readily available as to when this is necessary, so you’ll need to make a judgment call and do some testing in the lab as to whether or not this is necessary. In a large environment, it’s possible to scale some of the components out behind a load balancer as well. In addition, Quest also recommends physical hardware in lieu of Virtual Machines, although my personal opinion is that given proper resource allocation, this guidance is stuck somewhere in the era of the Notes UI design.

The way Quest integrates Notes F/B data with Exchange is clever, and to understand it, you’ll need to have a bit of background on how Outlook clients (and others) get F/B info. Prior to Exchange 2007, Exchange stored F/B information in Public Folders, and Outlook clients knew where to go in the Public Folder store to find the data. With the desire to move away from Public Folders, this information became available via Exchange Web Services (EWS), also sometimes called the Availability Service (AS). This is a SOAP based web service hosted on the CAS server and accessible via HTTPS. Outlook 2007 and newer knows how to access this endpoint as does Outlook for Mac and various other EWS clients. Exchange 2007 also introduced the ability to provide a means for cross-organization F/B info without any complex public folder replication. The way this works is you define an “availability address space” in Exchange which tells Exchange for a given subdomain, send those F/B requests over to a different AS endpoint. If you’re not familiar, take a look at this post before reading further. This functionality is what Quest leverages.

What Quest has done is re-implement the Autodiscover service as well as the Availability Service such that Exchange thinks it’s talking to another Exchange organization, when in fact it’s actually talking to the Quest Domino F/B server. In Exchange, we define an availability address space for lotus.contoso.com which will resolve (via Autodiscover) to Q-LNFBC01.

Pre Requisites

With the background information out of the way, let’s go ahead and start setting this up. You’re going to need a number of resources:

One to two servers to install the Quest components. For this example, I’ll use the two pictured above.
A Lotus Notes server to install the QCALCON task.
Administrative access to Lotus Notes
Exchange Organization Management level access to Exchange
A standard Domino user ID file and password with mail file.
A standard Exchange mailbox enabled user and password

Install PowerGUI on Q-EXFBC01 and Q-LNFBC01, and then Free/Busy coexistence components. The installers are quite self explanatory, so I won’t walk through those. Be sure to only install “Web Server Components” and “Lotus Notes Components” on the Domino F/B server and the “Exchange Components” on the Exchange F/B server.

Note: It’s possible to separate the Web Server components on to a separate server or set of load balanced servers if you have sufficient demand. For this walk through, I will assume they’re installed on Q-LNFBC01, though.

Configuring the Web Services and Domino Free Busy Connector Service

Once the installations are complete, we’ll use PowerGUI to configure the web services and Domino Free/Busy service.

Launch PowerGUI on Q-LNFBC01.
Browse to PowerGUI\CMN Free/Busy Connector Management\Configure Web Services and Lotus Notes Components in the Navigation Tree.
Select Configuration Wizard from the task pane on the right.
Configure an SMTP domain of lotus.contoso.com. Select autodiscover.lotus.contoso.com.

Note: You must create a DNS A record for autodiscover.lotus.contoso.com which points to the IP of Q-LNFBC01.

Configure the following values for Domino:
- Domino Server Name: LN-ADM01/CONTOSO.
- Domino ID File Path: (browse to the ID file)
- Domino Password: (password to the ID file)

Complete the wizard.
Start the “Quest CMN Domino Free/Busy Connector Service” service.

Note: The domain specified in Step 4 needs to match the Internet address field in your Notes person documents. Chances are your Notes environment is configured to accept mail for *@lotus.contoso.com for mail routing coexistence, but your users have Internet addresses in the form of *@contoso.com. To work around this in CMN, you need to add an SMTP Domain Mapping. To do this, you’ll need to work in PowerShell directly:

Launch the Free Busy Connector Management Shell (Start>All Programs>Quest Software>Quest Coexistence Manager for Notes>Free Busy Connector).
Run this command:

Set-CmnDominoFreeBusyConfig -SmtpDomainMappings "lotus.contoso.com=contoso.com"

Restart the Quest CMN Domino Free/Busy Connector Service.

Next, you’ll need to obtain an SSL certificate for the web services. While it’s possible to use a self signed certificate, your troubleshooting overhead will be substantially minimized if you obtain a proper trusted certificate. To obtain a certificate, you’ll need to generate a Certificate Signing Request (CSR).

Launch Internet Information Services (IIS) Manager (start>run>inetmgr).
Select Q-LNFBC01 in the COnnections tree.
Double click Server Certificates in the center pane.
Select Create Certificate Request in the task pane on the right.
Enter a Common name of autodiscover.lotus.contoso.com and populate the request of the screen as appropriate.
Upload the resultant CSR to your Certificate Authority (I recommend DigiCert if you don’t have a preference).

Once you receive the certificate back from your CA, return to the Server Certificates view in IIS Manager.

Select Complete Certificate Request in the task pane on the right.
Browse to the file you received from your CA and complete the wizard.
Browse to Q-LNFBC01\Sites\Default Web Site in the Connections pane.
Select Bindings in the task pane at right.
Click Add.
Select type https.
Find your SSL certificate in the SSL certificate drop-down.

In the next post in this series, we’ll configure the Exchange Fee/Busy Connector and the Domino QCALCON server task.

Configuring the Dell/Quest Free/Busy Connector for Lotus Notes and Exchange - Part 2

In Part 1 we took at look at the architecture of the Quest Free/Busy (F/B) Connector in Coexistence Manager for Notes (CMN) as well as how Exchange interfaces with it. We also configured the F/B Connector web services and the Domino Free Busy Connector Service. In this post, we’ll configure the Exchange Free Busy Connector Service, the Domino QCALCON task, and the Exchange organization. As a reminder, here’s a copy of our sample environment that will be referenced:

Configuring the Exchange Free Busy Connector

Create a standard mailbox enabled user (e.g. svc_xch_cmn_fb) in Exchange. This will be used by CMN to query Exchange F/B information.
Launch PowerGUI on Q-EXFBC01.
Browse to PowerGUI\CMN Free/Busy Connector Management\Configure Domino Server and Exchange Components.
Launch the Configuration Wizard from the task pane.
- Preferred Exchange server: mail.contoso.com.

Note: This is generally the URL which end users would access your CAS servers with internally such as for OWA. While the PowerGUI UI suggests you should enter a specific CAS server, it is better to enter a load balanced URL or at least the friendly URL listed on your SSL certificate. In this example we will use mail.contoso.com but you should substitute the value most appropriate for your environment.

Domain\Username: CONTOSO\svc_xch_cmn_fb.
Password: (service account password).

Complete the wizard.
Start the Quest CMN Exchange Free/Busy Connector Service service.

Configuring the Exchange Organization

Configuring your Exchange organization is perhaps the easiest part of this project. You’ll simply need to configure an Availability Address Space for lotus.contoso.com. If you’re not familiar with this, take a moment to review this post.

Launch the Exchange Management Shell (EMS).
Run the following PowerShell command:

Add-AvailabilityAddressSpace -ForestName "lotus.contoso.com" -AccessMethod OrgWideFB -UseServiceAccount:$true

Configuring the QCALCON Task

The final server component you’ll need to configure is the QCALCON task. Tasks are background processes of sort in Domino at least to the extent I understand them. You can configure them to either run at startup or at a specific time. For this step, install PowerGUI and the CMN Domino Server Components on LN-ADM01. The task and its configuration files are installed directly in the Domino server folder (e.g. c:\lotus\domino).

Configuring the QCALCON task is quite straight-forward.

Launch PowerGUI.
Browse to PowerGUI\CMN Free/Busy Connector Management\Configure Domino Server and Exchange Components\Advanced\Domino Server Task (QCALCON)
If prompted to browse for the Config File, you’ll find it under your Domino installation folder (e.g. c:\lotus\domino\qcalcon.exe.config).
Select Set Foreign Domain Name in the task pane. Enter mail.box when prompted.
Select Set Exchange Free/Busy Connector host name in the task pane. Enter Q-EXFBC01.
Open your notes.ini file (likely in c:\lotus\domino\notes.ini), and verify that the ServerTask= line includes qcalcon. If it doesn’t, add it to the end of the list.

Note: You may need to provide a fully qualified hostname in step 4.

Once you’ve configured QCALCON, you can start the task.

Launch the Lotus Domino Console (or connect remotely via Domino Administrator)
Run “load qcalcon”. You should see output like this if it’s succesful:

06/26/2011 03:08:55.25 PM [08F4:0005-060C] SchMsgQHandles_New> Opening queues for LWPSCHEDGATEWAY
06/26/2011 03:08:55.25 PM [08F4:0005-060C] SchMsgQHandles_New> InputQ: 121D0h, error = 0h: No error
06/26/2011 03:08:55.25 PM [08F4:0005-060C] SchMsgQHandles_New> OutputQ: 122A8h, error = 0h: No error
06/26/2011 03:08:55.25 PM [08F4:0004-00F0] SchMsgQHandles_New> Opening queues for MAIL.BOX
06/26/2011 03:08:55.25 PM [08F4:0004-00F0] SchMsgQHandles_New> InputQ: 12380h, error = 0h: No error
06/26/2011 03:08:55.25 PM [08F4:0004-00F0] SchMsgQHandles_New> OutputQ: 12450h, error = 0h: No error
06/26/2011 03:08:55 PM QCalCon Server: Starting
06/26/2011 03:08:55 PM QCalCon Server: Version 1.0.3.10
06/26/2011 03:08:55 PM QCalCon Server: Creating queue for mail.box
06/26/2011 03:08:55 PM QCalCon Server: Creating queue for LWPSCHEDGATEWAY
06/26/2011 03:08:55 PM QCalCon Server: Started

Note: If you need to change any of the values in the QCalCon config file, you’ll need to restart the task. To do this, you can issue a “tell qcalcon quit” command in the Lotus Domino Console.

In the next post in this series, we’ll look at configuring the proper documents in the Notes configuration as well as configuring test users in Exchange.

Configuring the Dell/Quest Free/Busy Connector for Lotus Notes and Exchange - Part 3

In Part 1 we took at look at the architecture of the Quest Free/Busy (F/B) Connector in Coexistence Manager for Notes (CMN) as well as how Exchange interfaces with it. We also configured the F/B Connector web services and the Domino Free Busy Connector Service. In Part 2, we configured the Exchange Free Busy Connector Service, the Domino QCALCON task, and the Exchange organization. In this post, we’ll complete the configuration by configuring Lotus Notes as well as building a test user in Exchange and Lotus Notes to validate the configuration. At the end of this post you should have working Free/Busy coexistence between Exchange and Notes.

As a reminder, here’s a copy of our sample environment that will be referenced:

Configuring the Lotus Notes Domain

All of the configuration tasks in this section will be performed in the Domino Administrator tool.

Browse to Configuration > Messaging > Domains.
Select Add Domain on the toolbar.
Populate the Basics tab with the following data:

Domain type: Foreign Domain
Foreign domain name: Exchange

Populate the Mail Information tab with the following data:

Gateway server name: LN-ADM01/CONTOSO.
Gateway mail file name: mail.box.

Populate the Calendar Information tab with the following data:

Calendar system name: LN-ADM01/CONTOSO.
Calendar system: mail.box.

Click Save & Close.

At this point, Domino should begin routing Exchange calendar requests to the QCALCON task and over to Exchange. To test this, we’ll need to configure a couple test users.

Testing the Configuration

We’ll create two test users for this exercise. User George Washington will have mail on Lotus Notes, while user Abraham Lincoln will have mail on Exchange. First, let’s configure Exchange.

Create a Contact in Exchange for George Washington. Specify an External Email Address (targetAddress) of gwashington@lotus.contoso.com.
Create a Mailbox in Exchange for Abraham Lincoln. Ensure the primary email address for Abraham Lincoln is alincoln@contoso.com.

In step 1, we’ve created an object which will be used for routing mail as well as for ensuring that the availability service redirects free/busy information to the Quest components. Next, let’s add these users to Lotus Notes.

Switch to the People & Groups tab in Domino Administrator.
Browse to Domino Directories > CONTOSO’s Directory > People.
Click Add Person.
Populate the Basics tab as shown below:

Populate the Miscellaneous tab as shown:

Click Save & Close.

Next, we need to register George Washington with Notes.

In People & Groups, expand People on the right under Tools, and click Register….
Populate the Basics tab as shown. Be sure to select LN-SRV01 by clicking Registration Server….

Check Advanced in the lower left
Switch to the Address tab.
Populate the Address tab as shown:

Click Register.
Press F9 to refresh the People view.

What we’ve done is created a Person document for Abraham Lincoln which forward to Exchange and specifies the Foreign Domain created earlier for calendaring. We’ve also created a full fledged mailbox enabled user for George Washington in Lotus Notes.

Testing

Populate Abraham Lincoln’s calendar with a few appointments in Exchange. In Lotus Notes, populate George Washington’s calendar with a few appointments as well.

Open George Washington’s person document from the People view in the NAB.
Click Open Mail File… on the toolbar.
Click Mail and then Switch to Calendar in the upper left of George Washington’s mail file to open the calendar.
Create a few appointments.

In Exchange, invite George Washington to a meeting using Outlook. Verify the free/busy information is displayed (e.g. no hash marks). In Lotus Notes, invite Abraham Lincoln to a meeting. verify that the Find Available Times tab shows free/busy information. Be advised that in both cases it may take a few moments for data to become available.

Troubleshooting

There are a number of places you can look to for troubleshooting information depending on where you think the problem lies. All of the Quest components log useful information to a file:

QCALCON (Domino to Exchange lookups) – c:\lotus\domino\qcalcon.exe.log.
Exchange Free/Busy Connector Service (Domino to Exchange lookups) – C:\Program Files (x86)\Quest Software\Quest Coexistence Manager for Notes\Free Busy Connector\ExchangeFreeBusyService.exe.log.
Domino Free/Busy Connector Service (Exchange to Domino lookups) – C:\Program Files (x86)\Quest Software\Quest Coexistence Manager for Notes\Free Busy Connector\DominoFreeBusyService.exe.log.

One additional useful tip surrounding the Quest components deals with caching. Both components keep requests cached in memory for 5 minutes by default. When you're troubleshooting, this can be problematic as you must restart the service each time you change a user in Notes or Exchange to see the effect. You can go in the Advanced configuration of either service in PowerGUI and tweak the cache lifetimes to "0" to disable caching. Just be sure to return the cache to the default values once you put the environment in production.

The Outlook availability service logging is also quite useful. You can enable that by going to Options>Advanced>Other>Enable Troubleshooting Logging. Restart Outlook and create a new meeting request. You’ll find logging information in %temp%\olkas which will include the exact errors and XML returned by the Quest components.

Cross-Forest Availability with Exchange 2007 and Exchange 2010

Prior to Exchange 2007, Exchange stored F/B information in Public Folders, and Outlook clients knew where to go in the Public Folder store to find the data. With the desire to move away from Public Folders, this information became available via Exchange Web Services (EWS), also sometimes called the Availability Service (AS). This is a SOAP based web service hosted on the CAS server and accessible via HTTPS. Outlook 2007 and newer knows how to access this endpoint as does Outlook for Mac and various other EWS clients. Exchange 2007 also introduced the ability to provide a means for cross-organization F/B info without any complex public folder replication. The way this works is you define an “availability address space” in Exchange which tells Exchange for a given subdomain, send those F/B requests over to a different AS endpoint. This is a very common scenario particularly with mergers and acquisitions. Let’s consider one such scenario and see how to set this up (as well as how it works).

In an effort to become Santa’s sole source cargo supplier, your employer, Wing Tip Toys (wingtiptoys.com), acquires the Fabrikam Coal Company (fabrikam.com). You manage the Exchange 2010 organization for Wing Tip Toys, and Fabrikam Coal runs Exchange 2007. Later in the merger process, you’ll consolidate Fabrikam into your organization, but, as soon as the merger closes, you’ll need to make it possible for Wing Tip Toys users to view F/B information for Fabrikam users. To do this, you need to configure an Availability Address Space in your Exchange organization for fabrikam.com. You can do this using these PowerShell commands:

# These credentials are a standard mailbox enabled user in the Fabrikam organization
$credentials = Get-Credential

Add-AvailabilityAddressSpace -ForestName "fabrikam.com" -AccessMethod OrgWideFB -Credentials $credentials

This tells Exchange to route F/B requests for *@fabrikam.com to a CAS in the fabrikam.com organization. In order to find Fabrikam’s CAS servers, your CAS servers will use Autodiscover. One important thing to note is that your CAS will ONLY use this route we’ve defined if the Fabrikam user we’re trying to retrieve F/B info for has a Contact or Mail Enabled User (MEU) in the Wing Tip Toys Active Directory environment. That contact or MEU must have a targetAddress which ends with @fabrikam.com.

Note: targetAddress is the name of the attribute in Active Directory. The *-MailContact and *-MailUser cmdlets as well as the Exchange Management Console refer to this attribute as the ExternalEmailAddress.

Here’s a quick (simplified) diagram of what happens:

WTT user looks up F/B info for john@fabrikam.com.
WTT CAS searches Active Directory for a contact or MEU with a targetAddress of john@fabrikam.com.
Active Directory returns a match.
WTT CAS performs an Autodiscover search for Fabrikam.com (this includes all of the usual Autodiscover mechanisms).
Fabrikam returns Autodiscover results.
WTT contacts Fabrikam’s availability service (authenticating with the credentials provided earlier) and asks for information pertaining to john@fabrikam.com.
Fabrikam’s AS returns information to WTT’s CAS.
The WTT CAS returns the information to the user.

Warning: This behavior is different for Outlook 2003 clients. Outlook 2003 clients have no knowledge of the Availability Service and as such they only obtain F/B info from Public Folders. Exchange 2010 SP1 introduced a change in behavior whereby F/B requests to Public Folders for users which are in a different forest are intercepted. The mailbox server intercepts these Public Folder requests and performs steps 4 – 7, contacting the remote CAS directly. If you’ve made firewall rules on the basis that only CAS servers perform cross-forest availability lookups, you’ll need to adjust those rules accordingly.

When testing cross-forest F/B lookups, you may need some extra logging to sort things out. The best place to collect this data without engaging PSS is actually in the Outlook client. You can enable this logging by opening Outlook’s Options (either via the Tools menu in Outlook 2007 or backstage in Outlook 2010), and then going to Advanced. Check the “Enable troubleshooting logging” box and restart Outlook. When you make future Free/Busy requests, you’ll find those logged under %temp%\olkas.

Getting Started with a Lotus Notes to Exchange Mail Migration

I thought it would be useful to give a quick overview of the coexistence and migration components of a Lotus Notes to Exchange migration as well as a couple links I’ve come across that are pretty useful. I’ll put some info on configuring the various coexistence pieces in a separate series of posts later. First, two excellent resources if you’re just getting started with Notes:

Here’s my three part series on configuring the Free/Busy Connector:

My assumption if you’re reading this is that you’re familiar with Exchange but not with Lotus Notes. It’s very helpful to have skilled and knowledgeable Lotus administrators at your disposal. With that in mind, I’ll do what I can to help you navigate the components of Lotus Notes that are going to be relevant. The first thing you’re going to need is a copy of the Domino Administrator and Lotus Notes client on your machine. This often comes as a single package – just be sure to select “single user mode” during the install if you’re prompted. First, though, the Domino Administrator end user experience, in case you aren’t yet familiar:

At some point you’re going to need to do something that’s only available in the Domino server’s command line interface. If you’re running Domino on Windows, you can Remote Desktop to the server and launch the Lotus Domino Console. If you’re running Domino on something else, or you don’t have Remote Desktop access, the console pictured below is also available inside the Domino Administrator program. To get to the console via Domino Administrator, switch to the Server tab and then the Status tab inside there, and finally, select Server Console. Pictured below is the Lotus Domino Console application available on Windows:

The Global Address List (GAL) equivalent in Notes is the Notes Address Book, usually called the “NAB”. It’s possible to have more than one of these, but, for simplicity we’ll assume you only have one NAB. The NAB is usually stored inside a database called names.nsf. Inside the NAB are person documents for each of your users. The person document is the rough equivalent of the Active Directory user object. Users with a proper mailbox in Notes will have the “mail file” field populated in their person document. Mail files are what Notes calls mailboxes. Everyone gets a separate file on the file system. You’ll be spending a good amount of time in the NAB, so hop over to the People & Groups tab in the Domino Administrator program and take a look. You can double click in any field to edit it.

If you’d like to create a new user and mailbox for them, switch over to the Configuration tab and then drill down to Registration> Person on the right. Fill in the form and check Advanced in the bottom left. On the Mail tab, pick where you want to create their mail file using the Mail Server button. On the ID Info tab, check the In File” button and browse to a folder to store the Notes ID you’ll need to access this user’s mail file. Finally, click Register to make things happen. If you go back to the NAB, you should see your new user.

If nothing else, the contextual help in the document editors can be amusing:

The actual migration of data and the coexistence period are two problems you’ll need to tackle. There are a couple companies that make tools in this space – Quest and BinaryTree. I’m familiar with the Quest offerings in this space and I’ve used them successfully at a number of customers, so, that’s what I’ll focus on going forward. That said, BinaryTree is a reputable ISV and you should certainly do your homework. On the Quest side, there are two products which can be purchased individually:

Notes Migrator for Exchange (NME)
Coexistence Manager for Notes (CMN)

NME is the product which is primarily responsible for the actual data migration. It can do some limited directory sync task, but, it’s really geared towards migrating data. CMN on the other hand has three components:

Free/Busy Connector
Mail Connector
Directory Connector

The Free/Busy (F/B) connector enables Lotus Notes users to view calendar information for Exchange users and vice-versa. This component is the most complex to setup, but also quite likely the most important. The Mail Connector serves as an SMTP gateway between Notes and Exchange. The service adjusts the contents of messages so that they work and display correctly in the opposing clients. It also can handle some of the nuances of Lotus Notes email such as Doc Links and Active Mail. Finally, the Directory connector will ensure that users or contacts in Exchange and person documents in Notes are in sync and the address books for end users are functionally identical regardless of which system they’re on.

Error When Removing Exchange 2000 or 2003 Server

There are a litany of issues that will cause Exchange setup to fail when you try to uninstall an Exchange server. It’s really tempting to just skip the uninstall and go delete the server entry from AD using something like ADSI Edit, but, you’re always better off just fixing the problem. Michael B. Smith has a good review of these, however I thought I’d touch on one more, specifically the error "One or more users currently use a mailbox store on this server". This can be really frustrating, especially when you’ve checked each mailbox store for mailboxes and done a search using AD Users and Computers for mailboxes on that server and turned up nothing. Exchange setup is doing a search of AD for users who have a msExchHomeServerName value which is equal to the legacyExchangeDN of the server you’re trying to remove.

There’s a handy KB article, 924170, which suggests a mechanism to find the problem users, but, my experience tonight is that the directions don’t really actually work. The good news this is really easy to straighten out with adfind and possible admod. What you want to do is two-fold. First, find the legacyExchangeDN of the server you’re trying to remove. You can do this with adfind:

adfind -config -f "cn=SOME-SERVER" legacyExchangeDN

You should get a result something like this:

/o=Brian Lab/ou=First Administrative Group/cn=Configuration/cn=Servers/cn=BRIAN-XMB01

What you’ll want to do now is find all the users who reference this value in their msExchHomeServerName attribute:

adfind -gcb -f "msExchHomeServerName=/o=Brian Lab/ou=First Administrative Group/cn=Configuration/cn=Servers/cn=BRIAN-XMB01"

If there’s only a few results, the easiest solution is to just find each user using AD Users and Computers on a machine with Exchange System Manager installed, right click the user, select Exchange Tasks, and then use the Remove Exchange Attributes option.

If there’s a lot of users, you’ll probably want a more automated solution. You can use adfind with admod to clear the msExchHomeServerName attribute on a lot of users at once.

Warning: Be careful with this – there’s no undo function! You can use the –upto switch in lieu of –unsafe to process a limited number of objects.

adfind -gcb -f "msExchHomeServerName=/o=Brian Lab/ou=First Administrative Group/cn=Configuration/cn=Servers/cn=BRIAN-XMB01" -adcsv | admod -unsafe msExchHomeServerName:-:

Cleanup User Objects with Invalid MailNicknames

Exchange 2000 and Exchange 2003 have no problem with users (or groups and contacts) which have a space in their mailNickname attribute. Unfortunately if you try to work with one of these users using Exchange 2007 or Exchange 2010, the PowerShell cmdlets will throw a validation error similar to the following:

Property expression "John Doe" isn't valid. Valid values are: Strings formed with characters from A to Z (uppercase or lowercase), digits from 0 to 9, !, #, $, %, &, ', *, +, -, /, =, ?, ^, _, `, {, |, } or ~. One or more periods may be embedded in an alias, but each period should be preceded and followed by at least one of the other characters. Unicode characters from U+00A1 to U+00FF are also valid in an alias, but they will be mapped to a best-fit US-ASCII string in the e-mail address, which is generated from such an alias.
+ CategoryInfo : NotSpecified: (brianlab.local...Doe, John:ADObjectId) [Update-Recipient], DataValidationException
+ FullyQualifiedErrorId : 385167D4,Microsoft.Exchange.Management.RecipientTasks.UpdateRecipient

Cleaning this up is more or less a pre-requisite to a migration. I put a quick VBScript together which will do this. The script below handles users but you can easily update the LDAP filter to grab groups or contacts as well. You’ll need to update the search base and DC at the top of the script.

Warning: This makes modifications to live recipients and if you’re running Exchange 2000 or Exchange 2003, this will trigger the RUS. I’d highly suggest doing something like an LDIF backup of the users who will be affected in case anything goes wrong.

'==========================================================================
' NAME: Cleanup mailNickname's with spaces
'
' AUTHOR: Brian Desmond, brian@briandesmond.com
' DATE  : 11/6/2010
'
' COMMENT: 
'
'==========================================================================

Option Explicit

Const SEARCH_BASE = "DC=brianlab,DC=local"
Const AD_DC = "BRIAN-RTDC01"

Dim cnxn
Set cnxn = WScript.CreateObject("ADODB.Connection")
cnxn.Provider = "ADsDSOObject"
cnxn.Open "Active Directory Provider"

Dim cmd
Set cmd = WScript.CreateObject("ADODB.Command")
cmd.ActiveConnection = cnxn

cmd.CommandText = "<LDAP://" AD_DC & "/" & SEARCH_BASE & ">;(&(objectCategory=person)(objectClass=user)(mailNickname=*\20*));distinguishedName,mailNickname;subtree"
cmd.Properties("Page Size") = 100
cmd.Properties("Timeout") = 30
cmd.Properties("Cache Results") = False

WScript.Echo cmd.CommandText

Dim rs
Set rs = cmd.Execute

Dim nickName
Dim dn

While Not rs.eof 
    nickName = rs.Fields("mailNickname").Value
    dn = rs.Fields("distinguishedName").Value
    
    WScript.Echo "Fixing " & nickName & "(" & dn & ")"
    
    SetAttribute dn, "mailNickname", Replace(nickName, " ", "")
    
    rs.MoveNext
Wend 

rs.close
cnxn.Close

Set rs = Nothing
Set cmd = Nothing
Set cnxn = Nothing 

Sub SetAttribute(objectDn, name, value)
    Dim obj
    Set obj = GetObject("LDAP://" & objectDn)
    
    obj.Put name, value
    obj.SetInfo
    
    Set obj = Nothing 
End Sub

Exposing Organization Level Message Tracking in Exchange 2010 OWA

If your end users are using Exchange Server 2010 OWA, one of the things they can now do is track their messages from the Exchange Control Panel and see largely the same data an administrator sees. What isn’t enabled by default is the ability to delegate this on an organization wide basis via the Exchange Control Panel. There is a duplicate of this interface which lets a user search the entire organization’s message tracking logs. In order to access this, you need to do a couple of things:

Create an Active Directory security group to delegate the rights to in RBAC
Create a new Management Role Assignment for the Message Tracking role
Create a new Management Role Assignment for the View-Only Recipients role

I went ahead and created a group called “Exchange Message Tracking Access” and added my account to it. Next, I ran the following PowerShell command:

New-ManagementRoleAssignment -Role "Message Tracking" -SecurityGroup "Exchange Message Tracking Access"
New-ManagementRoleAssignment -Role "View-Only Recipients" -SecurityGroup "Exchange Message Tracking Access"

Once you login to Outlook Web App with a user who is in the Exchange Message Tracking Access group and go to the Options (Exchange Control Panel [ECP]) section, you’ll see the option of managing “My Organization” in the upper left hand corner:

If you select “My Organization”, you’ll get a UI like this:

The key thing to nice here is the user friendly UI with the help bubbles. You can delegate this to service desk or other front line support personnel so they can handle end user questions as to the delivery status of their message. Each result provides a details view which shows most of the information available to administrators using Message Tracking cmdlets except in a friendly and understandable interface.

How to Create an ActiveSync Device Report

Exchange logs quite a bit of info about ActiveSync device partnerships and you can use this to create reports about the utilization of mobility features in your organization. Getting this data requires a couple of intermediate steps before you can export it to a CSV for processing in something like Excel (or another script). The PowerShell script below will export all of the ActiveSync device relationships in your organization. Keep in mind that this will include old relationships which are no longer active. Depending on how large your organization is and the number of device relationships out there, it may take a little while for the script to run.

Note: If you have a mixed version organization (e.g. Exchange 2007 and Exchange 2010), you’ll need to run the script twice. Once in the Exchange 2007 Management Shell and once in the Exchange 2010 Management Shell. The cmdlets used here are not backwards (or forward compatible). I’ve provided two versions of the script - one for Exchange 2007 and one for Exchange 2010.

Exchange 2007 Version

$devices = @()
$mailboxes = Get-CASMailbox -ResultSize:Unlimited | Where-Object {$_.HasActiveSyncDevicePartnership -eq $true -and $_.ExchangeVersion.ExchangeBuild -ilike "8*"}

foreach ($m in $mailboxes) 
{
	$devices += Get-ActiveSyncDeviceStatistics -Mailbox $m.Identity
}

$devices | Export-Csv DeviceStats.csv

Exchange 2010 Version

$devices = @()
$mailboxes = Get-CASMailbox -ResultSize:Unlimited | Where-Object {$_.HasActiveSyncDevicePartnership -eq $true -and $_.ExchangeVersion.ExchangeBuild -ilike "14*"}

foreach ($m in $mailboxes) 
{
	$devices += Get-ActiveSyncDeviceStatistics -Mailbox $m.Identity
}

$devices | Export-Csv DeviceStats.csv

You can open the exported CSV in Excel from here and generate reports based on that. There is quite a bit of information in the report including some personally identifiable information (PII) for the devices so keep that in mind before redistributing the raw data file.

BlackBerry Enterprise Server and Exchange Server 2010 Throttling Policies

One of the new features in Exchange Server 2010 is the concept of Client Throttling Policies. In summary, Client Throttling Policies are designed to limit the amount of system resources a given user can consume and in turn impact performance for other Exchange users. Out of the box there is a default throttling policy (use the Get-ThrottlingPolicy cmdlet) applied to all users:

RunspaceId                     : ba3cdf92-fc9f-4a70-a912-2cf225e6d573
IsDefault                      : True
EASMaxConcurrency              : 10
EASPercentTimeInAD             :
EASPercentTimeInCAS            :
EASPercentTimeInMailboxRPC     :
EWSMaxConcurrency              : 10
EWSPercentTimeInAD             :
EWSPercentTimeInCAS            :
EWSPercentTimeInMailboxRPC     :
EWSMaxSubscriptions            :
EWSFastSearchTimeoutInSeconds : 60
EWSFindCountLimit              :
IMAPMaxConcurrency             :
IMAPPercentTimeInAD            :
IMAPPercentTimeInCAS           :
IMAPPercentTimeInMailboxRPC    :
OWAMaxConcurrency              : 5
OWAPercentTimeInAD             :
OWAPercentTimeInCAS            :
OWAPercentTimeInMailboxRPC     :
POPMaxConcurrency              : 20
POPPercentTimeInAD             :
POPPercentTimeInCAS            :
POPPercentTimeInMailboxRPC     :
PowerShellMaxConcurrency       : 18
PowerShellMaxCmdlets           :
PowerShellMaxCmdletsTimePeriod :
ExchangeMaxCmdlets             :
PowerShellMaxCmdletQueueDepth :
RCAMaxConcurrency              : 20
RCAPercentTimeInAD             :
RCAPercentTimeInCAS            :
RCAPercentTimeInMailboxRPC     :
MessageRateLimit               :
RecipientRateLimit             :
ForwardeeLimit                 :
CPUStartPercent                : 75
AdminDisplayName               :
ExchangeVersion                : 0.10 (14.0.100.0)
Name                           : DefaultThrottlingPolicy_f017f530-3edf-4c59-9955-d94bb7892fb0
DistinguishedName              : CN=DefaultThrottlingPolicy_f017f530-3edf-4c59-9955-d94bb7892fb0,CN=Global Settings,CN=
                                 GreenOrg,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=green,DC=briandesmond,D
                                 C=net
Identity                       : DefaultThrottlingPolicy_f017f530-3edf-4c59-9955-d94bb7892fb0
Guid                           : af1aeaac-4d88-43da-92df-24c0924d4ad8
ObjectCategory                 : green.briandesmond.net/Configuration/Schema/ms-Exch-Throttling-Policy
ObjectClass                    : {top, msExchGenericPolicy, msExchThrottlingPolicy}
WhenChanged                    : 10/10/2009 5:44:29 PM
WhenCreated                    : 10/10/2009 5:44:11 PM
WhenChangedUTC                 : 10/10/2009 10:44:29 PM
WhenCreatedUTC                 : 10/10/2009 10:44:11 PM
OrganizationId                 :
OriginatingServer              : BRIAN-GRDC02.green.briandesmond.net
IsValid                        : True

As you can see, most of the valuesa re null, however each of the services has a *MaxCurrency property. These define the maximum number of connections a given user can have to that service. For example, the EASMaxConcurrency value limits a given user to a maximum of ten Exchange ActiveSync connections. Each of the values is documented in the Set-ThrottlingPolicy cmdlet documentation. Of particular note to this discussion is the RCAMaxConcurrency value which defines the maximum number of concurrent connections a given user can have to the RPC Client Access service. The RPC Client Access service is new in Exchange 2010 and it handles all MAPI connections to Exchange.

BlackBerry Enterprise Server (BES) uses a single service account to proxy all of the connections to Exchange on behalf of BlackBerry users. The side effect of this is that it’s quite likely that BES will need to have more than twenty (default limit) connections open to Exchange at a given time. If you review the documentation from RIM, they recommend setting the RCAMaxConcurrency value to null (equivalent to unlimited) for all users. This is really not a great idea at all.

Instead, what you can do is define a new Client Throttling Policy without an RCAMaxConcurrency value and apply it directly to the BES service account. The PowerShell script below does just that. The script assumes that your BES service account is called “BESAdmin”. If it isn’t modify the script accordingly.

New-ThrottlingPolicy "BES Throttling Policy" -RCAMaxConcurrency:$null
Set-Mailbox besadmin -ThrottlingPolicy "BES Throttling Policy"

You can easily confirm that the new policy is applied to the BESAdmin account by inspecting the properties of BESAdmin mailbox:

Get-Mailbox besadmin | fl Name,ThrottlingPolicy

You should see results similar to the following:

Name : BES Admin
ThrottlingPolicy : BES Throttling Policy

Redirecting OWA URLs in Exchange 2010

One of the things I’ve been doing for as long as I can remember is redirecting requests that don’t go to https://owa.customer.com/owa (or /exchange) to the correct URL. So, if someone goes to http://owa.customer.com or https://owa.customer.com, they get redirected to the correct (secure) URL. Historically I’ve always done this with two components:

A custom website listening on Port 80 on each CAS server
A default.aspx file in the root of the Default Web Site redirecting to /owa

This approach no longer works with Exchange 2010 CAS because the PowerShell virtual directory actually operates over Port 80 (authentication is Kerberized). If you try and tinker with this, you’ll start getting errors from Remote PowerShell like this:

VERBOSE: Connecting to cas01.customer.com
[cas01.customer.com] The WinRM service cannot process the request because the request needs to be sent to a different machine. Use the redirect information to send the request to a new machine. Redirect location reported: https://owa.customer.com/owa/PowerShell. To automatically connect to the redirected URI, verify "MaximumConnectionRedirectionCount" property of session preference variable "PSSessionOption" and use "AllowRedirection" parameter on the cmdlet.
+ CategoryInfo : OpenError: (System.Manageme....RemoteRunspace:RemoteRunspace) [], PSRemotingTransportRedirectException
+ FullyQualifiedErrorId : PSSessionOpenFailed

In order to work around this, you need to use the HTTP Redirection feature in IIS (the default.aspx trick mentioned above should work too), as well as remove the requirement for SSL at the top level Default Web Site object. You have to be careful doing this because when you set settings on the web site, IIS will push them down to any virtual directory below which does not explicitly set that setting itself. To setup the redirect, select the Default Web Site in IIS Manager, and open the HTTP Redirect option under IIS. Complete it like this:

Warning: It’s very important that you check the checkboxes exactly as shown in the screenshot above!

Once this step is complete, you need to remove the enforced redirect from each of the virtual directories under the Default Web Site. To do this, select each virtual directory individually, and then open the HTTP Redirect property and uncheck the “Redirect requests to this destination” checkbox. You’ll need to do this on the following virtual directories:

aspnet_client
Autodiscover
ecp
EWS
Microsoft-Server-ActiveSync
OAB
PowerShell
Rpc

Note: The Exchange, Exchweb, and Public virtual directories should redirect to /owa.

If at this point you simply browse to http://cas01.customer.com, you’ll get an HTTP 403.4 error. This is because SSL is required at the top-level website. In order to get the redirect working, we need to disable SSL for the toplevel website while leaving it enabled for the relevant child virtual directories.

Select the Default Web Site and open the SSL Settings properties. Uncheck the Require SSL checkbox as shown below:

Like the redirection settings, this change will be inherited down the tree for any virtual directory which does not explicitly set the setting independently. Ensure that SSL is required for the following virtual directories:

Autodiscover
ecp
EWS
Microsoft-Server-ActiveSync
OAB
owa
Rpc

Warning: If you require SSL for the PowerShell virtual directory, you will render Remote PowerShell inoperable!

Once you’ve configured the redirection and SSL settings, open a command prompt and run iisreset. At this point you should be able to browse to http://localhost on the CAS server and get redirected to https://owa.customer.com/owa. These steps were tested on Windows Server 2008 R2. While they should be similar under Windows Server 2008, they may not be identical.

Update-StorageGroupCopy fails with ESE Error

If you get the following error when you try to seed a CCR/SCR target node, you may have a Windows Firewall problem.

WARNING: Replication for storage group 'MyCluster\SG01' is still suspended. If needed, you can use the Resume-StorageGroupCopy cmdlet in the Exchange Management Shell to resume replication.
Update-StorageGroupCopy: Seeding failed : Database seeding error: Error returned from an ESE function call (0xc7ff1004), error code (0x0).

If you take a network trace while this cmdlet is running, you'll find numerous unanswered TCP SYNs from the source node to the target node. Looking in TCPView from Sysinternals, you'll find the PowerShell.exe process listening on the port these SYNs are sent to. If you're using the Windows Firewall, you'll need create a rule that permits PowerShell.exe to listen.

I initially assumed these were RPCs so I created a Dynamic RPC rule for PowerShell.exe, but, this didn't work. Thus I just fell back to a rule which permitted TCP connections on any port to PowerShell.exe. I scoped the rule to the subnets which had my CCR and SCR targets.

Resuming a Failed Exchange 2007 Setup

If you've ever had Exchange 2007 setup fail partially, you've probably noticed that some of the components (maybe even the one it failed during) show up as installed, but, if you run the ExBPA (Best Practices Analyzer), it will probably complain that setup failed. With most applications, you can open the Add/Remove Programs (or Programs and Features in WS2008) Control Panel and begin a repair. Unfortunately Exchange does not offer a repair option.

The trick with Exchange is to fix it with the exsetup utility. If, for example, you need to repair a Mailbox server role installation, go to the Exchange installation folder and run "exsetup /role:Mailbox". The utility will resume setup where it left off and clean things up.

Exchange 2007 Protocol Logging – IMAP & POP

I had a question come across my desk the other day about how to enable protocol logging in Exchange 2007 for IMAP and POP. Protocol logging for these protocols generates comma delimited logs similar to HTTP logs (except for IMAP). In Exchange 2003 you enabled these with a couple of registry tweaks. These don't however work in Exchange 2007. Exchange 2007 instead uses a couple of config files on the file system.

To enable the logging for IMAP, you would do this:

Browse to C:\Program Files\Microsoft\Exchange Server\ClientAccess\PopImap
Open Microsoft.Exchange.Imap4.exe.config with text editor (e.g. Notepad) and scroll to the bottom
Modify ProtocolLog from false to true.
Modify LogPath as necessary
Restart the MsExchangeImap4 service

For POP, simply edit Microsoft.Exchange.Pop3.exe.config instead and restart the MsExcahngePop3 service.

You can use a number of tools to parse the logs, although one tool I would recommend is LogParser from Microsoft.

Script to Dump Exchange Mailbox Info to Spreadsheet (CSV)

I found another useful script I wrote last year on my hard drive this evening. It's pasted in below. This script will dump quite a bit of useful information about each mailbox on a particular server or set of servers to a CSV file which you can in turn import into Excel and create a spreadsheet from. I typically would import data into a SQL Server table using DTS (Data Transformation Services) if I needed to do alot of computation or data mining. Excel gets very slow when doing tasks that really require an index over a lot of data.

Note: The script uses WMI to get this information so Exchange 2003 is required. Only Exchange View Only level permissions are required in Active Directory, however you will likely need local Administrator privleges on each Exchange server. I don't have an Exchange 2003 server readily available to test and I was running as an Exchange Full Admin when I originally wrote this script.

There are a few properties which I did not export as I did not need them at the time. The specific meaning of each property available is available on MSDN. Adding these properties to the script should be self explanatory (especially given a very similar script at the bottom of the MSDN article).

There are two things you must edit in order for this script to function within your organization:

Line 28:
"Const TOTAL_SERVERS = 3"

You should put the total number of servers you plan to inventory in TOTAL_SERVERS.

Lines 36 - 37:
strComputer(0) = "xmb01"
strComputer(1) = "xmb02"
strComputer(2) = "xmb03"

You should create or remove additional lines for each server name in the strComputer() array. Note that the array starts with index 0. The script has been tested with twelve servers and sixty thousand mailboxes.

Here is the code for the script. Use this at your own risk, it's not my fault if anything happens.

'==========================================================================
' NAME   : Exchange Mailbox Stats Dumper
' AUTHOR : Brian Desmond, brian@briandesmond.com
' DATE   : 12/28/2005
' COMMENT: This script requires Exchange 2003. It will dump information
'			about each mailbox on the mailbox servers specified
'
'	Version		Date		Author			Note
'	-----------------------------------------------------------------
'	1.0			28Nov05		Brian Desmond	Initial Version
'	1.1			03Sep06		Brian Desmond	
'	1.2			13Dec08		Brian Desmond	Fixed array sizing bug,
'											Added error handling note
'											Added TODOs
'											Moved configurable items up
'==========================================================================
Option Explicit

' Note this script currently uses On Error Resume Next
' this isn't best practice - in reality this should be tightly 
' wrapped around the WMI connection logic in the loop rather
' than up here.
On Error Resume Next

' TODO: Configure this
' This is the total number of servers which you
' will specify for inventory
Const TOTAL_SERVERS = 3

Dim strComputer()
ReDim strComputer(TOTAL_SERVERS)

' TODO: Populate this array
' Enter each server name below as an entry in the array
' starting with zero
strComputer(0) = "xmb01"
strComputer(1) = "xmb02"
strComputer(2) = "xmb03"

'==========================================================================

Dim objWMIService
Dim colItems

Dim fso
Set fso = CreateObject("Scripting.FileSystemObject")

Dim fil
Set fil = fso.CreateTextFile("mailboxes.txt")

Dim objItem
Dim line
Dim i

' Write a header row to the CSV
fil.WriteLine """Server"",""Storage Group"",""Mail Store"",""Mailbox GUID"",""Display Name"",""LegacyDN"",""Size"",""Item Count"",""Associated Content Count"",""Deleted Message Size"",""Date Absent"",""Storage Limit Level"""

For i = 0 To TOTAL_SERVERS - 1
	Set objWMIService = GetObject("winmgmts:" _
	    & "{impersonationLevel=impersonate}!\\" & strComputer(i) & _
	        "\ROOT\MicrosoftExchangeV2")
	
	Set colItems = objWMIService.ExecQuery _
	    ("Select * from Exchange_Mailbox")
	
	For Each objItem in colItems
		line = """" & objItem.ServerName & """"
		line = line & ","
		line = line & """" & objItem.StorageGroupName & """"
		line = line & ","
		line = line & """" & objItem.StoreName & """"
		line = line & ","
		line = line & """" & objItem.MailboxGUID & """"
		line = line & ","
		line = line & """" & objItem.MailboxDisplayName & """"    
		line = line & ","
		line = line & """" & objItem.LegacyDN & """"
		line = line & ","
		line = line & """" & objItem.Size & """"
		line = line & ","
		line = line & """" & objItem.TotalItems & """"
		line = line & ","
	    line = line & """" & objItem.AssocContentCount & """"
	    line = line & ","
	    line = line & """" & objItem.DeletedMessageSizeExtended & """"
	    line = line & ","
	    line = line & """" & objItem.DateDiscoveredAbsentInDS & """"
	    line = line & ","
	    line = line & """" & objItem.StorageLimitInfo & """"
	    
	    fil.WriteLine line 
	    'WScript.Echo line 
	Next
Next

fil.Close
Set fso = Nothing
Set objWMIService = Nothing

Dedicated Exchange Sites in Active Directory

A comment I received on a previous post on sites and subnets in Active Directory was "what benefit(s) does a dedicated Exchange site provide?". There's a couple things to consider here with the advent of Exchange 2007. The first is the great degree of dependency Exchange has on Active Directory data for everything it does. The second, applicable to Exchange 2007 deployments is that Exchange now uses the Active Directory site topology to route email. I'm not familiar enough with this scenario yet to speak to it, but I will speak to the need for fast and reliable global catalog access for Exchange servers.

With Exchange 2000 and 2003, the vast majority of the configuration data for Exchange is stored in Active Directory in the Exchange services container in the configuration partition. All of the configuration data for recipients is stored on top of the objects representing them in Active directory – users for mailboxes, contacts, distribution groups, public folders, etc. More specifically when Exchange needs access to this data, it goes to a global catalog server to get it as a global catalog holds the relevant data for every single recipient in the forest. The configuration partition information can be read from any DC in a forest due to the replication scope of the partition. In a busy Exchange deployment, Exchange places an disproportionate load on the global catalogs it uses as compared to clients. Every single message that has to be routed requires a look up against a global catalog, distribution list expansions, address book builds, etc.

When the Active Directory traffic generated by Exchange isn't segmented off of that generated from end users, workstations, and other applications there tends to be a performance hit seen by both parties. If the shared domain controllers are too busy with Exchange, complaints about logon times can be seen. Conversely, if the shared domain controllers are too busy with logon traffic and servicing other applications, Exchange will see a hit must notably in message routing. These scenarios of course apply to larger Exchange deployments – a small operation is unlikely to run into this.

The solution to this problem is to deploy a dedicated Active Directory site for Exchange. If you have multiple datacenters with highly concentrated Exchange deployments in them, the solution might actually be multiple Exchange sites enterprise-wide. Place a couple of global catalogs in these sites which are geared towards very high read performance – 64 bit deployments with as much memory as possible to hold a large amount (or all of) the DIT in memory will see a huge performance gain.

This article from TechNet explains a handful of performance counters which should be monitored to see if Active Directory may be the root of Exchange performance issues.

This article from Microsoft IT Showcase explains exactly how to go about configuring a dedicated Exchange site. Note that Microsoft IT deployed a series of /32 subnet objects to create their Exchange site. It is also possible to provision dedicated Exchange subnet(s) in your datacenter(s) and associate those with an Exchange site. I've taken both paths and generally it tends to depend more on whether the network guys at the shop are willing to light up a dedicated site or not.

How to Import/Export Exchange 2003 Sender Filters

Susan Bradley asked me this evening how to go about import/exporting the list of filtered sender domains in Exchange 2003. This will do a merge rather than a replace on import. There are two steps, import and export.

Export

Open a command prompt, and run this command:

ldifde -m -f senderfilter.ldf -r "(objectCategory=msExchSMTPTurfList)" -d "cn=configuration,dc=yourdomain,dc=local" -l msExchTurfListNames

You'll need to replace dc=yourdomain,dc=local with the dn of your domain nc head. Here's the formula for building that in case you're not sure:

Add dc= to the beginning and then replace every period (“.“) with ,dc=. So, a couple examples:

sbs1.house.briandesmond.com becomes dc=sbs1,dc=house,dc=briandesmond,dc=com
bigtire.local becomes dc=bigtire,dc=local
smalltire.bigwheel.com becomes dc=smalltire,dc=bigwheel,dc=com

This will export your blocked domains to a text file called senderfilter.ldf. You can just open it in notepad (notepad senderfilter.ldf from the command prompt) and view it.

Import

To import the file, you need to change dc=yourdomain,dc=local to reference the target domain in the ldf file first. The import command is:

ldifde -i -f senderfilter.ldf

If you're going to be importing this list frequently, there's a command to save you editing the ldf file everytime. Open the ldf file and replace the reference to the source domain (dc=yourdomain,dc=local) to dc=x. Then, use this import command instead:

ldifde -i -f senderfilter.ldf -c dc=x dc=targetdomain,dc=local

This will replace the occurrences of dc=x with dc=targetdomain,dc=local

Exchange System Manager Public Folder Errors

I was working on a customer's Exchange server today when I got presented with an error in Exchange System Manager (ESM) while trying to do some work on their public folder hierarchy. I was first getting this error:

---------------------------
Exchange System Manager
---------------------------

The operation failed because of an HTTP error 501 (Not implemented). Verify that the ExAdmin virtual root exists on the destination server.

ID no: c1030af7 Exchange System Manager

---------------------------
OK
---------------------------

I traced this down to someone having applied IP address level restrictions to the various Exchange virtual directories (including exadmin and public) which prohibited me from connecting with ESM to work on the public folder hierarchy. I removed these and then I got a new error:

---------------------------
Exchange System Manager
---------------------------

The operation failed due to an invalid format in the HTTP request. Verify that the host header is correct for the virtual server.

ID no: c1030af0
Exchange System Manager

---------------------------
OK
---------------------------

I figured out that the Default Web Site which had all the Exchange virtual directories in it was bound to a specific IP address on this server (which had multiple IPs) and that was not the address that ESM was attempting to connect to. Changing this to all unassigned took care of the issue.

After the fact I found this useful link on other errors ESM might throw while managing public folders: http://www.microsoft.com/technet/prodtechnol/exchange/2003/insider/publicfolders.mspx