One of the challenges that came to light during my involvement with this project was that the vCAC design depended upon the use of vCAC agents for installing software packages in virtual machine workloads.  In effect it was assumed TCP/IP connectivity was available between the management infrastructure and consumer virtual machines – not common practice in a multi-tenant environment.  Without going in to specific details of the project, what was needed was a mechanism to install software in consumer virtual machines without compromising security between the provider and the consumer(s).  A number of options were considered, including:

• Multiple options around the concept of secured networking (using tools such as Private VLANs, vShield Edge, vShield App etc…)

• Leveraging the VIX API

I was personally an advocate of the second option as it involved not TCP/IP connectivity between consumer’s and/or the provider infrastructure.  Furthermore it would work for virtual machines running on isolated networks too.  During the various design discussions that ensued I produced a quick example script to prove the concept of connecting to vCloud Director, identifying a specific vApp and its constituent virtual machines, and then locating the same virtual machines in vSphere and installing the required software.  The following video and script is an updated version of that initial test, which I enhanced with the ability to cope with multiple virtual machines, Powered Off virtual machines and packaged as an advanced function.

function Install-WinPkgInvCloudVM {
<#
.SYNOPSIS
    Install a software package in vCloud Director managed virtual machine.
.DESCRIPTION
    Identify virtual machine(s) within a vCloud Director vApp and install a package in the guest operating system
.PARAMETER CIServer
    The hostname of a vCloud Director Server managing the vApp(s).
.PARAMETER CIUser
    The username of a valid user to connect to the vCloud Director Server managing the vApp(s).
.PARAMETER CIPassword
    The password for the user CIUser to connect to the vCloud Director Server managing the vApp(s).
.PARAMETER VApp
    The name or object or collection of objects of a vApp in which to search for virtual machines.
.PARAMETER Package
    The full path to a software package to be installed in the windows guest.
.PARAMETER Command
    The command required to install the software package.
.PARAMETER VIUser
    The username of a valid user to connect to the vCenter Server managing ESXi Server hosts on which to search for volumes.
.PARAMETER VIPassword
    The password for the user VIUser to connect to the vCenter Server managing ESXi Server hosts on which to search for volumes.
.PARAMETER GuestUser
    The username of a valid user to connect to the vCenter Server managing ESXi Server hosts on which to search for volumes.
.PARAMETER GuestPassword
    The password for the user VIUser to connect to the vCenter Server managing ESXi Server hosts on which to search for volumes.
.EXAMPLE
    PS C:\ Install-WinPkgInvCloudVM -CIServer "mycloud.myurl.com" -CIUser "MyvCDUser" -CIPassword "MyvCDPassword" -CIVApp "MyVApp" `
    -Org "MyOrg" -SrcPath "C:\Users\myuser\mypackage.msi" -DstPath "C:\Users\myuser\" -Command "C:\Users\myuser\mypackage.msi /S" `
    -VIUser "MyVCUser" -VIPassword "MyVCPassword" -GuestUser "MyGuestUser" -GuestPassword "MyGuestPassword"
#>
[CmdletBinding()]
param(
    [parameter(Mandatory=$true,
        HelpMessage="Supply the hostname of the vCloud Director Server managing the vApp(s).",
        ValueFromPipeline=$false)]
    [String]
    $CIServer
    ,
    [parameter(Mandatory=$true,
        HelpMessage="Supply a valid username for the supplied CIServer.",
        ValueFromPipeline=$false)]
    [String]
    $CIUser
    ,
    [parameter(Mandatory=$true,
        HelpMessage="Supply a valid password for the supplied CIServer.",
        ValueFromPipeline=$false)]
    [String]
    $CIPassword
    ,
    [parameter(Mandatory=$true,
        HelpMessage="Supply a vApp name.",
        ValueFromPipeline=$true,
        ValueFromPipelineByPropertyName=$true)]
    [String[]]
    $CIVapp
    ,
    [parameter(Mandatory=$true,
        HelpMessage="Supply an Org name to narrow search",
        ValueFromPipeline=$false,
        ValueFromPipelineByPropertyName=$true)]
    [String]
    $Org
    ,
    [parameter(Position=0,
        Mandatory=$false,
        HelpMessage="Supply a OrgVdc name to narrow search.",
        ValueFromPipeline=$false,
        ValueFromPipelineByPropertyName=$true)]
    [String]
    $OrgVdc
    ,
    [parameter(Mandatory=$false,
        HelpMessage="Supply a the full path to a local software package to be copied to the vApp.",
        ValueFromPipeline=$false)]
    [String]
    $SrcPath
    ,
    [parameter(Mandatory=$false,
        HelpMessage="Supply a the full path to a destination folder for the software package.",
        ValueFromPipeline=$false)]
    [String]
    $DstPath
    ,
    [parameter(Mandatory=$false,
        HelpMessage="Supply a command to start the installation of the software package.",
        ValueFromPipeline=$false)]
    [String]
    $Command
    ,
    [parameter(Mandatory=$true,
        HelpMessage="Supply a valid username for the supplied VIServer.",
        ValueFromPipeline=$false)]
    [String]
    $VIUser
    ,
    [parameter(Mandatory=$true,
        HelpMessage="Supply a valid password for the supplied VIServer.",
        ValueFromPipeline=$false)]
    [String]
    $VIPassword
    ,
    [parameter(Position=3,
        Mandatory=$false,
        HelpMessage="Supply a user for guest operating system.",
        ValueFromPipeline=$false)]
        [ValidateNotNull()]
    $GuestUser
    ,
    [parameter(Position=4,
        Mandatory=$false,
        HelpMessage="Supply a password for the guest operating system.",
        ValueFromPipeline=$false)]
    [ValidateNotNull()]
    $GuestPassword
    ,
    [parameter(Mandatory=$false,
        HelpMessage="Specify if installation should continue in supported VMs, when other un-supported guest operating systems are detected.",
        ValueFromPipeline=$false)]
    [Switch]
    $NonSupportedGuestOveride = $false
)
Begin {
    # Define supported virtual machines guest operating systems for this function
    $SuppOSFullNames = "Microsoft Windows XP Professional (32-bit)",
    "Microsoft Windows Server 2003 (64-bit)",
    "Microsoft Windows Server 2003 (32-bit)",
    "Microsoft Windows 7 (64-bit)",
    "Microsoft Windows Server 2008 R2 (64-bit)",
    "Microsoft Windows Server 2008 (64-bit)",
    "Microsoft Windows Server 2008 (32-bit)"
}
Process {
    # Connect to the vCloud Director Server
    Write-Host Connecting to vCloud Director Server $CIServer
    Connect-CIServer -Server $CIServer -User $CIUser -Password $CIPassword | Out-Null

    # Locate the vApp within vCloud Director (assumes no duplicate names can existing within a given Organisation)
    Write-Host "Searching for the vApp" $CIVapp
    if ($OrgVdc){
        $vCDvApp = Get-CIVApp -Name $CIVApp -Org $Org -OrgVdc $OrgVdc
    }else{
        $vCDvApp = Get-CIVApp -Name $CIVApp -Org $Org
    }

    if (!$vCDvApp){
        Write-Host -ForegroundColor Red "The vApp" $CIVApp "could not be found"
    }

    # Identify the virtual machine(s) located within the vApp
    Write-Host Identifying vCloud virtual machines within vApp $CIVapp
    $vCDVms = Get-CIVM -VApp $vCDvApp
    if (!$vCDVms){
        Write-Host -ForegroundColor Yellow "vApp" $CIVapp "does not contain any virtual machines"
        Write-Host -ForegroundColor Yellow "Package will not be copied and installed - skipping vApp"
    }
    Write-Host $CIVapp "contains" ($vCDVms | Measure-Object).Count "virtual machines:"
    $vCDVms | Select Name, Status, GuestOSFullName, Href | Format-Table -AutoSize | Out-String

    # Identify the vCenter Server, MoRef and other useful properties of the virtual machine(s) located within the vApp
    Write-Host "Retrieving vSphere properties for vCloud virtual machines:
    $vSphereMaps = @()
    $vCDVms | foreach {
        # Retrieve CIVm ExtensionData
        $vCloudExtData = $_.ExtensionData.VCloudExtension

        # Start building a custom object containing useful vSphere VM information
        $vSphereMap = New-Object System.Object
        $vSphereMap | Add-Member -MemberType NoteProperty -Name VmName -Value $_.Name
        $vSphereMap | Add-Member -MemberType NoteProperty -Name Status -Value $_.Status
        $vSphereMap | Add-Member -MemberType NoteProperty -Name Href -Value $_.Href
        $vSphereMap | Add-Member -MemberType NoteProperty -Name vCenterServer -Value
        $vCloudExtData.SyncRoot[0].Any.SyncRoot[0].VmVimObjectRef.VimServerRef.name
        $vSphereMap | Add-Member -MemberType NoteProperty -Name VmMoRef -Value
        $vCloudExtData.SyncRoot[0].Any.SyncRoot[0].VmVimObjectRef.MoRef
        Write-Host vCloud virtual machine $vSphereMap.VmName is managed by vCenter Server $vSphereMap.vCenterServer and has the MoRef
        $vSphereMap.VmMoRef

        # Check if the identified Guest OS is supported for this function and flag if not supported
        $SuppCount = 0
        $GuestOsSupported = "Yes"
        foreach ($SuppOSFullName in $SuppOSFullNames) {
            if ($_.GuestOsFullName -eq $SuppOSFullName) {
                $SuppCount ++
            }
        }
        if ($SuppCount -lt 1) {
            Write-Host -ForegroundColor Red "Unsupported guest operating system detected in virtual machine" $_.Name
            Write-Host -ForegroundColor Red "OSFullName must be one of:" ($SuppOSFullNames | Format-Table | Out-String)
            $GuestOsSupported = "No"
            if (!$NonSupportedGuestOveride){
                Write-Host -ForegroundColor Red "unction aborted. Use the 'NonSupportedGuestOveride' switch in order to overide and install packages in any supported virtual machines"
                exit 1
            }
        }

        # Add VM Guest OS information to the custom object
        $vSphereMap | Add-Member -MemberType NoteProperty -Name GuestOsFullName -Value $_.GuestOsFullName # May add multiple OS support later.
        $vSphereMap | Add-Member -MemberType NoteProperty -Name GuestOsSupported -Value $GuestOsSupported # May add multiple OS support later.

        # Add custom object to an array
        $vSphereMaps += $vSphereMap
    }

    # Summarise the mapping information rerieved
    Write-Host vCloud Director Server to vCenter Server mapping information is as follows
    $vSphereMaps | Format-Table @{Expression={$_.VmName};Label="Name"},Href, Status,@{Expression={$_.vCenterServer};Label="vCenter Server"},@{Expression={$_.VmMoRef};Label="MoRef"},@{Expression={$_.GuestOsFullName};Label="Guest OS"},@{Expression={$_.GuestOsSupported};Label="Supported"}  | Out-String

    # Disconnect from the vCloud Director Server
    Write-Host Disconnecting from vCloud Director Server $CIServer
    Disconnect-CIServer -Server $CIServer -Force -Confirm:$false

    # Connect to the first identified vCenter Server (assume all VMs in a vApp are managed by a common vCenter Server)
    $VIServer = $vSphereMaps[0].vCenterServer
    Write-Host Connecting to vCenter Server $VIServer
    Connect-VIServer -Server $VIServer -User $VIUser -Password $VIPassword | Out-Null

    # Loop through each discovered VM in the vApp
    $vSphereMaps | foreach {
        Write-Host "Processing virtual machine" $_.VmName

        # Verify the VM has a supported Guest OS
        if (($_.GuestOsSupported -eq "Yes") -and ($_.Status -eq "PoweredOn")){
            Write-Host "Virtual machine Guest OS is supported for package copy and install"

            # Identify the vSphere VM
            Write-Host Locating vSphere virtual machine object using MoRef $_.VmMoRef
            $VIVm = Get-VM -id ("VirtualMachine-" + $_.VmMoRef)
            Write-Host "Located vSphere virtual machine" $VIVm.Name "in $VIServer "inventory"

            # Copy install package to the virtual machine
            Write-Host Copying install package to vSphere virtual machine $VIVm.Name
            Write-Host "Transfering local package" $SrcPath "to virtual machine destination" $DstPath
            Copy-VMGuestFile -VM $VIVm -GuestUser $GuestUser -GuestPassword $GuestPassword -Source $SrcPath -Destination $DstPath -LocalToGuest -Force

            # Execute install package in the virtual machine
            $FileName = $SrcPath.Split("\")[(($SrcPath.Split("\")).Length -1)]
            Write-Host Executing package $DstPath within guest operating system of vSphere virtual machine $VIVm.Name
            $Result = Invoke-VMScript -VM $VIVm -GuestUser $GuestUser -GuestPassword $GuestPassword -ScriptText $Command
            $ExitCode = $Result.ExitCode
            Write-Host vSphere virtual machine $VIVm.Name returned exit code $ExitCode
        }elseif ($_.Status -eq "PoweredOff"){
            Write-Host -ForegroundColor Yellow "Virtual machine is powered off"
            Write-Host -ForegroundColor Yellow "Package cannot be copied and installed - skipping virtual machine" $_.VmName
        }else{
            Write-Host -ForegroundColor Yellow "Virtual machine Guest OS not supported for package copy and install"
            Write-Host -ForegroundColor Yellow "Package will not be copied and installed - skipping virtual machine" $_.VmName
        }
    }

    # Disonnect from the identified vCenter Server
    Write-Host Disconnecting from vCenter Server $VIServer
    Disconnect-VIServer -Server $VIServer -Force -Confirm:$false
}
End{}
}

When I get some more free time, I might take a look at adding some enhancements, such as support for both Windows and Linux based virtual machine guest operating systems and/or enhanced support for handling virtual machines in different power states.  Feel free to leave comments and provide some suggestions.

4503 – Architecting a UK Government-assured Multi-site vCloud

Abstract:  Resiliency is a key aspect of any infrastructure—it is especially important in infrastructure-as-a-service solutions, such as a vCloud assured for use by the UK Government and Public Sector. This session will include a tour of a real world solution deployed by Skyscape. Throughout the session there will be detailed guidance on how the solution achieves high-availability and disaster recovery capabilities using VCE hardware with EMC VNX and VPLEX, whilst achieving accreditation by the UK Government up to Restricted level and compliance with ISO27001 standards.

Presenters:  Aidan Dalgleish and Richard Stinton

4667 – Monitoring your vCloud Infrastructure with vCenter Operations

Abstract:  In this session we will cover the requirements needed to monitor your vCloud Infrastructure. Multiple real world scenarios will be demonstrated with recommendations on best practice and potential pitfalls, how to integrate vCenter Operations with vCloud Director and vFabric Hyperic.

Presenters:  David Hill and Aidan Dalgleish

4804 – Addressing your DR IP ails without the need for ale

Abstract:  Recovery of virtual machines in a disaster recovery situation is always complicated, and without the luxury of stretched physical layer 2 networks, the inflexibility of IP addressing can be a significant challenge. We find ourselves jumping through hoops to accommodate the location specific requirements of IP addressing and routing. This session will walk you through how to setup VMware Infrastructure IP addressing in order to simplify DR failover by removing the requirement to reconfigure your virtual machines to match the new address space. During the session we will cover the infrastructure requirements of this including but not limited to vShield Manager, vShield Edges, VXLAN and network overlay technologies. At the conclusion of the session you will be able to articulate the benefits of overlay networks and how they offer the potential to help simplify disaster recovery processes for your infrastructure.

Presenters:  Kamau Wanguhu and Aidan Dalgleish

This paper is intended to compliment the VMware vCloud Director Infrastructure Resiliency Case Study paper published previously by Duncan Epping and Chris Colotti.  The original paper provides a great explanation of the concepts and principles behind how vCloud Director disaster recovery solutions can be implemented, whilst this new paper describes how the solution can be automated with VMware vSphere PowerCLI.

The content of this paper is derived from work carried out at Oxford University, which was built upon the principles detailed in the original paper but complimented with some automation courtesy of vSphere PowerCLI.  Additional information is available in the VMworld sessions INF-BCO2155 “vCloud DR for Oxford University Computing Services – Real World Example” and INF-VSP2164 “Automation of vCloud Director Disaster Recovery”.

I would like to thank Alan for his support in getting this paper published and Gary Blake for helping build the Oxford University vCloud DR solution.  Last but not I need to thank Adrian Parks and Jon Hutchings from Oxford University Computing Services, for working with us on this project and trusting us with their vCloud.

Resiliency is a key aspect of any infrastructure—it is even more important in infrastructure-as-a-service solutions. This session will include a tour of a real world vCloud DR solution deployed at Oxford University Computing Services (OUCS). Throughout the session there will be detailed guidance on how both the management and resource clusters were designed, deployed and automated. Furthermore the details of how SRM in conjunction with VMware vSphereTM PowerCLI (PowerCLI) was used to automate the end-to-end recovery of a vCloud Director–based infrastructure. This session will offer a perfect complimentary follow on from the whitepaper regarding the high-level process produced by Duncan Epping and Chris Colotti and the Automation session by Alan Renouf and Aidan Dalgleish. The paper combined with the Automation session will describe the process and the automation principles whereas this session will describe a complete real world implementation.

Presentation Materials

The presentation materials for this session are available from the VMworld 2012 Virtual Pavilion.

Feedback

Thank you to those of you that provided feedback. In the most part it seems like people enjoyed the sessions and found the content useful.  I wanted to take the opportunity to thank people for their feedback and make it clear I have read all of the comments provided and will take this in to consideration for future versions of this presentation or anything similar in nature.

Resiliency is a key aspect of any infrastructure—it is even more important in infrastructure-as-a-service solutions. This session will include guidance, examples and demonstrations of the use of VMware vSphere^TM PowerCLI (PowerCLI) to automate the recovery of a vCloud Director–based infrastructure. In particular the session will focus on the automation of the recovery steps for vCloud Director managed vApp workloads, that cannot be recovered with vSphere Site Recovery Manager (SRM). This session will offer a perfect complimentary follow on from the whitepaper regarding the high-level process released by Duncan Epping and Chris Colotti. The paper discussed the process and this session will describe the automation.

Presentation Materials

Whilst the presentation materials for this session should have already been released, I’m conscious that this is in the form of a .pdf and hence does not include the recorded demonstration videos.  Furthermore I recognise that the PowerCLI code included was in the form of an image and not text that can easily be leveraged.  In light of this I wanted to take opportunity to release these materials in a format that can be consumed more easily.

The original presentation, including the VMworld recording is available from the VMworld 2012 Virtual Pavilion.

Within the presentation we covered a selection of key functions used to automate the recovery of vCloud Director, including:

• Mount-VmfsVolumes
• Disable-VMwareHA
• Exit-MaintenanceMode
• Register-VirtualMachines
• Start-vShieldEdge
• Find-ProviderVdc
• Find-OrganizationVdc
• Find-VApps
• Set-OptionVaules
• PowerOn-VApp

In conjunction with the PowerCLI code for each of the above function I’ve added some brief explanation around the purpose of each function.

Mount-VmfsVolumes

Following successful use of your storage management utility to break replication and make volumes read/write (if required by storage platform), virtual machines will appear ‘Inactive’ in the vCenter Server inventory. To rectify this, replicated VMFS volumes need to be force mounted by the recovery hosts.

function Mount-VmfsVolumes ($ClusterName, $EsxUser, $EsxPassword){
    $Cluster = Get-Cluster -Name $ClusterName
    $AvailVMHosts = $Cluster | Get-VMHost | where {($_.ConnectionState -eq "Maintenance")}
    $AvailVMHosts | Get-VMHostStorage -RescanAllHba | Out-Null
    foreach ($AvailVMHost in $AvailVMHosts){
        Connect-VIServer -Server $AvailVMHost.Name -User $EsxUser -Password $EsxPassword | Out-Null
        $VMHost = Get-VMHost -Server $AvailVMHost.Name
        $HstSys = Get-View $VMHost.id
        $HstStSys = Get-view $VMHost.StorageInfo
        $HstDsSys = Get-View $HstSys.ConfigManager.DatastoreSystem
        $UnresVols = $HstDsSys.QueryUnresolvedVmfsVolumes()
        foreach ($UnresVol in $UnresVols){
            $DevicePaths = New-Object System.Collections.ArrayList
            $Extents = $UnresVol.Extent
            $Extents | Foreach {
                $DevicePath = $_.DevicePath
                $DevicePaths += $DevicePath
            }
            $VmfsResSpec = New-Object Vmware.Vim.HostUnresolvedVmfsResolutionSpec[](1)
            $VmfsResSpec[0] = New-Object Vmware.Vim.HostUnresolvedVmfsResolutionSpec
            $VmfsResSpec[0].ExtentDevicePath = $DevicePaths
            $VmfsResSpec[0].UuidResolution = “forceMount”
            $HstStSys.ResolveMultipleUnresolvedVmfsVolumes($VmfsResSpec) | Out-Null
        }
        Disconnect-VIServer -Server $AvailVMHost.Name -Confirm:$false | Out-Null
    }
}

Disable-VMwareHA

Following the successful mounting of replicated VMFS volumes, there is a requirement to take the recovery ESXi Server hosts out of maintenance mode for the vCloud Director resource cluster. If there is an intention to use the vCloud Director API to initiate vApp PowerOn operations (in order to honour any defined power sequence) then we must first disable VMware HA in order to prevent virtual machines being powered on automatically.

function Disable-VMwareHA ($ClusterName){
    $Cluster = Get-Cluster -Name $ClusterName
    $Cluster | Set-Cluster –HARestartPriority Disabled –Confirm:$false | Out-Null
}

Exit-MaintenanceMode

Having made the decision regarding the use of VMware HA the recovery ESXi Server hosts need to be removed from Maintenance Mode.

function Exit-MaintenceMode ($ClusterName){
    $Cluster = Get-Cluster -Name $ClusterName
    $Cluster | Get-VMHost -State Maintenance | Set-VMHost –State Connected  | Out-Null
}

Register-VirtualMachines

In the event that VMware HA is reconfigured to prevent virtual machines being powered on automatically, the associated consequence is that the virtual machines will remain ‘Inactive’ and require registering.

function Register-VirtualMachines ($ClusterName, $EsxUser, $EsxPassword){
    $Cluster = Get-Cluster -Name $ClusterName
    $AvailVMHosts = $Cluster | Get-VMHost | where {$_.ConnectionState -eq "Connected"}
    $InActVms = $Cluster | Get-VM | where {$_.ExtensionData.OverallStatus -eq "gray"}
    $InActVmObjs = @()
    $InActVms | Foreach {
        $InActVmObj = New-Object -TypeName PSOBject -Property @{
            VmPathName = $_.ExtensionData.Config.Files.VmPathName
            VmName = $_.Name
            VmResourcePool = $_.ResourcePool
            VmMoRef = $_.Id
        }
        $InActVmObjs += $InActVmObj
     }
     foreach ($InActVmObj in $InActVmObjs){
         $RandVMHostIndex = Get-Random -Minimum 0 -Maximum ($AvailVMHosts | Measure-Object).Count
         $InActVmObj | Add-Member -MemberType NoteProperty -Name VmHostIndex -Value $RandVMHostIndex
     }
     $VMHostIndex = 0
     foreach ($AvailVMHost in $AvailVMHosts){
         Connect-VIServer -Server $AvailVMHost.Name -User $EsxUser -Password $EsxPassword | Out-Null
         $HstResPools = Get-ResourcePool -Server $AvailVMHost.Name
         $HstResPoolsCount = ($HstResPools | Measure-Object).Count
         $HstVmFolder = Get-Folder –Name vm -Server $AvailVMHost.Name
         $HstVmFolderRef = Get-View $HstVmFolder.Id
         $HstInActVmObjs = $InActVmObjs | where {$_.VmHostIndex -eq $VMHostIndex}
         foreach ($HstInActVmObj in $HstInActVmObjs){
             $VmPath = $HstInActVmObj.VmPathName
             $VmName = $HstInActVmObj.VmName
             $ResourcePool = $HstInActVmObj.VmResourcePool
             if ($HstResPoolsCount -gt 1){
                 $VmResPool = $HstResPools | where {$_.Name -eq $ResourcePool}
             }else{
                 $VmResPool = Get-ResourcePool -Server $AvailVMHost.Name
             }
             $VmResPoolRef = (Get-View $VmResPool.id).MoRef
             $HstVmFolderRef.RegisterVM($VmPath,$VmName,$false,$VmResPoolRef,$null)
         }
         $VMHostIndex ++
         Disconnect-VIServer -Server $AvailVMHost.Name -Confirm:$false
     }
}

Start-vShieldEdge

Even if a decision has been taken to use the vCloud Director API to power on virtual machines, this will not address vShield Edge appliances deployed on organization external routed networks. It is logical that the relatively low number of vShield Edge appliances supporting organization networks be started earlier, in order to support any connectivity to vApp external dependencies.

function Start-vShieldEdge ($ClusterName){
    $Cluster = Get-Cluster -Name $ClusterName
    $Cluster | Get-ResourcePool –Name “System vDC*” | Get-VM -Name "vse-*" | Start-VM | Out-Null
}

Find-ProviderVdc

Automation of this activity can range from being relatively simple to more complex, depending on the approach taken. This could be achieved manually in the form of a simple ‘mapping table’, however this section will highlight how this can be achieved dynamically.

function Find-ProviderVdc ($ClusterName, $VcdServer, $VcdUser, $VcdPassword){
    $Cluster = Get-Cluster -Name $ClusterName
    $AvailVMHosts = $Cluster | Get-VMHost | where {$_.ConnectionState -eq "Connected"}
    $RefVMHost = $AvailVMHosts | Select -First 1
    Connect-CIServer -Server $VcdServer -User $VcdUser -Password $VcdPassword | Out-Null
    $Result = Search-Cloud -QueryType Host -Name $RefVMHost.Name
    $RefVMHostHref = (Get-CIView -SearchResult $Result).Href
    $Pvdcs = Get-ProviderVdc
    foreach ($Pvdc in $Pvdcs){
        foreach ($VMHost in $Pvdc.ExtensionData.HostReferences.HostReference){
            if ($VMHost.Href -eq $RefVMHostHref){
                $MatchedPvdc = $Pvdc
                Write-Host "Match found in Provider vDC" $MatchedPvdc.Name
            }
        }
    }
    Disconnect-CIServer -Server $VcdServer -Confirm:$false
}

Find-OrganisationVdc

In order to power on vApps we first need to understand which organization virtual datacenters are present and later which vApps they contain. Having previously determined the provider virtual datacenter the process for determining associated organization virtual datacenters is relatively simple.

function Find-OrganisationVdc ($VcdServer, $VcdUser, $VcdPassword, $ProviderVdc){
    Connect-CIServer -Server $VcdServer -User $VcdUser -Password $VcdPassword | Out-Null
    $Ovdcs = Get-OrgVdc -ProviderVdc $ProviderVdc
    Disconnect-CIServer -Server $VcdServer -Confirm:$false
}

Find-vApps

Having previously determined the organization virtual datacenters the process for determining associated vApps is relatively simple.

function Find-VApps ($VcdServer, $VcdUser, $VcdPassword, $OrganisationVdc){
    Connect-CIServer -Server $VcdServer -User $VcdUser -Password $VcdPassword | Out-Null
    $vApps = Get-CIVApp –OrgVdc $OrganisationVdc
    Disconnect-CIServer -Server $VcdServer -Confirm:$false
}

Set-OptionValues

When created virtual machines are assigned a UUID, derived from the physical ESXi Server host UUID and the path to the virtual machine configuration file. Although the process to mount VMFS volumes does not alter the path to the configuration file, the virtual machine will have been recovered on a different ESXi Server hosts. The result being that the constituent virtual machines of a vApp may be detected as being moved or copied.

function Set-OptionValues ($ClusterName, $Key, $Value){
    $Cluster = Get-Cluster -Name $ClusterName
    $VMs = $Cluster | Get-VM
    Foreach ($VM in $VMs){
        $VirtualMachineConfigSpec = new-object VMware.Vim.VirtualMachineConfigSpec
        $VirtualMachineConfigSpec.ExtraConfig += new-object VMware.Vim.OptionValue
        $VirtualMachineConfigSpec.ExtraConfig[0].key = $Key
        $VirtualMachineConfigSpec.ExtraConfig[0].value = $Value
        ($VM.ExtensionData).ReconfigVM_Task($VirtualMachineConfigSpec)
     }
}

PowerOn-vApp

Lastly, having identified the vApps for a given organization virtual datacenter, it is relatively simple to initiate a power on operation.

function PowerOn-VApp ($OrganisationVdc, $VcdServer, $VcdUser, $VcdPassword){
    Connect-CIServer -Server $VcdServer -User $VcdUser -Password $VcdPassword | Out-Null
    $vApps = Get-CIVApp –OrgVdc $OrganisationVdc | where {$_.Status -ne "PoweredOn"}
    Foreach ($vApp in $vApps){
        ($vApp.Extensiondata).PowerOn() | Out-Null
    }
}

Feedback

Thank you to those of you that provided feedback. In the most part it seems like people enjoyed the sessions and found the content useful.  I wanted to take the opportunity to thank people for their feedback and make it clear I have read all of the comments provided and will take this in to consideration for future versions of this presentation or anything similar in nature.

In recent months I’ve been involved in a project with Gary Blake to implement one of, if not the, first vCloud Director Disaster Recovery (DR) solution.  The design is based upon the solution described in the VMware vCloud Blogs written by Chris Colotti and Duncan Epping.  In order to provide a slick automated solution, I was keen to leverage the power of Windows PowerShell (PowerShell) and VMware vSphere PowerCLI (PowerCLI).  Unsatisfied with the thought of manually running some basic scripts, I wanted VMware Site Recovery Manager (SRM) to drive the majority, if not all, of the recovery process.  Throughout this project a number of challenges arose in executing PowerCLI functions in order to achieve the desired result, so I thought I would share some details of the final approach used.

The Challenge

In order to use SRM to drive the automated recovery of vCloud Director Resource Clusters, it was decided to implement a series of additional Recovery Plan Steps.  These would execute PowerCLI functions that accept a series of parameters, including some of those available from the SRM Environment Variables (the Site Recovery Manager Administration Guide contains further details).  If you were to search the internet on this subject you would undoubtedly find a large number of resources that suggest the solution is to; a) Call a PowerShell script using of a batch or ‘helper’ script, or b) Call a PowerShell script directly.  Whilst both approaches can fulfil the requirement, I wanted more!  I’ll explain why…

Using a batch script to call a PowerShell script works perfectly well but my issue with this approach was that SRM receives the exit code from the batch script and not the PowerShell script.  I.e. if the batch script successfully runs but the PowerShell script experiences an error, SRM will still interpret the exit code of 0 from the batch script and hence interpret the Recovery Plan Step as successfully completed – not the slick solution I had in mind.

Calling a PowerShell script directly addresses the issue of SRM receiving the relevant exit code, since if an error is triggered by the PowerShell script it will be picked up by SRM and the Recovery Plan Step will be marked as failed.  Problem Solved?  Nearly, but I still wanted something more!  Whilst this ultimately works, what I really wanted was the ability to call specific custom functions and in the process supply SRM Environment Variable as parameters.

The Solution

The end solution implemented was to provide the capability for SRM, through the use of additional Recovery Plan Steps, to call bespoke PowerShell Advanced Functions directly.  In order to achieve this some specific configuration steps are required.  We’ll cover these configurations steps in two stages; a) Calling PowerCLI Cmdlets from SRM, and b) Calling Advanced Functions from SRM.

Calling PowerCLI Cmdlets from SRM

The process for calling a PowerShell Cmdlet directly from SRM is relatively straight forward and effectively the same as that for calling a PowerShell script.  The specifics arise when there is a requirement to call a PowerCLI Cmdlet.  More specifically we need to ensure that the PowerCLI Cmdlets, that are distributed as a PowerShell Snapin, are loaded.  Assuming PowerShell and PowerCLI are already installed, the following steps provide an explanation of how this can be achieved:

1. Create a dedicated user as a service account for SRM.  Configure the SRM service to run as this user.
2. Log on to the SRM Server, using the previously created service account
3. Open the 32bit PowerShell: Start > Accessories > Windows PowerShell > Windows PowerShell (x86)
4. Set the Execution Policy by typing: Set-ExecutionPolicy RemoteSigned
5. Find the path to the PowerShell Profile by typing: $profile.  Validate it exists by typing: Test-Path $profile
6. If the result was ‘False’, create a new profile by typing: New-Item -path $profile -type file –force
7. Open the profile file by typing: notepad $profile
8. Configure the profile to load vSphere Cmdlets by adding: Add-PSSnapin VMware.VimAutomation.Core
9. Repeat step 8 for additional Cmdlets (e.g. vCloud Director) by adding: Add-PSSnapin <Snapin Name>

Upon completion of these configuration steps it will be possible to use an SRM Recovery Plan Step to execute any PowerCLI Cmdlets by selecting ‘Command on SRM Server’ and using the format C:\Windows\SysWOW64\WindowsPowerShell\v1.0\Powershell.exe <Cmdlet> <Params>.  This is shown in the example below:

Calling Advanced Functions from SRM

Having worked through the guidance provided in the previous ‘Calling PowerCLI Cmdlets from SRM’ section, you should have the capability to call PowerShell or PowerCLI Cmdlets (and Cmdlets for any other Snapins added) directly from an SRM Recovery Plan Step.  What you won’t yet be able to do is call any custom Advanced Functions you have created.  The following steps provide an explanation regarding how this can be achieved:

1. Take the <filename>.ps1 file containing the functions and create a module by renaming it <filename>.psm1
2. Navigate to the directory C:\Windows\SysWOW64\WindowsPowerShell\v1.0\Modules
   
3. Create a directory called <filename>, then copy the PowerShell module fileto the directory
4. Open the 32bit PowerShell: Start > Accessories > Windows PowerShell > Windows PowerShell (x86)
5. Type Get-Module –ListAvailableand verify that your new module is present
6. Open the profile file by typing: notepad $profile
7. Configure the profile file to load the new module by adding: Import-Module <filename>

Upon completion of these configuration steps it will be possible to use an SRM Recovery Plan Step to execute PowerCLI functions.  Furthermore, assuming you’ve written them correctly (a subject for another post I think…), then you will also receive the relevant exit codes to indicate if the SRM Recovery Plan Step should be marked as successful or failed.  In order to execute a custom Advanced Function simply select ‘Command on SRM Server’ and use the format C:\Windows\SysWOW64\WindowsPowerShell\v1.0\Powershell.exe <Function> <Params>. This is shown in the example below:

Implementation Tips

In addition to the implementation steps covered so far there are a few additional pointers worth mentioning that will save you some frustration.  In particular I’m referring to the use of single vs double quotes and how to use the SRM Environment Variables as parameters.

Firstly, if you are having issues with calling any function the best course of action (apologies if I’m ‘teaching you to suck eggs’) is to first run your function on the command line and then transpose it in to the SRM Recovery Plan Step and add the Environment Variables etc…  If you are still having issue then alter your problematic function such that it writes any supplied parameters to a log file, such that you confirm if they are being interpreted correctly. You’ll notice in the above example that various parameters, such as path names, are passed in double quotes and in the most part this works fine, however on some occasions you may find that the value is not passed correctly and some ‘tweaking’ is required.  As a general rule I would recommend the following approach:

• In the first instance do not use any quotes – You’ll soon recognise if this is an issue as the PowerShell function will throw an error with respect to the parameters.  I.e. The value of the supplied argument to a given parameter is actually being interpreted as a parameter, which is undefined.
• In cases where a given argument is causing an error (path names being a good example) then surround the argument in double quotes.
• Finally if you function is still generating an error or the value of the given argument is being interpreted incorrectly try using single quotes.  This generally seemed to rectify issues where a given argument contained a space or special character.

Lastly, in the event you are attempting to pass SRM Environment Variables, you may find that the environment variable is interpreted literally.  In these cases the solution, as can been seen in the above screen shot for the value %VMware_VC_Host%, is to use the Get-Content Cmdlet.  For example replace the normal SRM Environment Variable with (Get-Content env:VMware_VC_Host). Credit due to Gary Blake for figuring this one out.

Conclusion

With the application of some simple configuration steps and some well considered scripting it is possible to use SRM to execute PowerShell advanced functions, that could be used to perform all manner of custom tasks for those more complex disaster recovery solutions.  For instance, in the second example above an advanced function is being called to check the status of a defined service within a Microsoft Windows guest operating system and report back a success or fail (exit 0 or 1).  Now what would be really nice would be the ability for SRM to perform a defined action based on the exit code…