And that’s not all, the Onyx program will translate the captured SOAP traffic into PowerShell code. This allows you to see which vSphere APIs are used and how parameter objects for these methods are constructed.

The new Onyx 2.0 has a lot of new features and several improvements for older features.

• The most eye-catching new feature is that Onyx 2.0 now also can, besides PowerShell code, produce SOAP, C# and JavaScript code. This should make life of the SDK developers and vOrchestrator users a lot easier!

• Onyx 2.0 supports vSphere 4.1! In other words all new APIs, that were introduced with vSphere 4.1, are captured and shown correctly in the produced output.

• A simple, but very user-friendly, feature is that Onyx 2.0 allows you start the vSphere client or PowerCLI session directly from the Onyx Gui.

• The vSphere client makes a lot of obvious API calls which are probably not important for the action you’re trying to capture. A good example is the continous use of PropertyCollector calls to refresh the values on the vSphere client screens. If you don’t want to obfuscate the resulting script with these calls, it’s now quite easy to suppress them. From the Onyx Gui you open the Settings menu and specify which methods should be ignored.

Using Onyx is quite simple!

Unfortunately the instructions on the VMware LABS Onyx page still seems to contain the Onyx 1.0 instructions.And although they are still valid, the usage of Onyx became a lot easier.

When you start the program you are presented with the main screen.

Top-left in the toolbar you find the Connect button that allows you to set up the connection to the vCenter or ESX(i) server for which you want to capture the actions.

There is also a History option (the list box) that allows you to choose, by default, from your last 10 connections.

The next window will ask you to which vCenter or ESX(i) server you want to connect.

With a new connection, you also have the option to automatically start the vSphere client or the PowerCLI client. Just enter your credentials and Onyx will take care of starting the client and setting up the connection.

And now the fun begins!

To start translating all your actions in the client into PowerShell code you hit the Start button in the toolbar.

Let’s have a look at a simple example. We’ll add 1 vCPU and a NIC to a guest in the vSphere client.

We immediately see the PowerShell code the Onyx program generated. Small remark, the colours in the output were done by me to better show the different parts of the generated code !

With the help of the SDK Reference it is now quite easy to find back what each part of the generated code is doing.

The action we did in the vSphere client obviously uses the ReconfigVM_Task method which requires one parameter, a VirtualMachineConfigSpec. In this VirtualMachineConfigSpec there is the numCPUs property that permits the caller to define the number of vCPUs. And additionally there is the VirtualDeviceConfigSpec object that takes care of the virtual HW, in this case the new NIC.

Notice how the object type that is entered in the device property, depends on the type of HW that was chosen. In this case the new NIC was specified as a vmxnet3 type card.

Also note that the generated code uses the actual MoRef to get the VirtualMachine object. This is not considered good coding practice. So if you are going to use the generated code, make sure you fetch the guest based on it’s name. Something like this:

...

$_this = Get-VM -Name "PC1" | Get-View

$_this.ReconfigVM_Task($spec)

One of the exciting new features in Onyx 2.0 is its ability to generate different types of output.

There is the raw SOAP.

And there is C#, which SDK developers will surely appreciate.

And there is JavaScript that can be used with vOrchestrator.

What is also a definite improvement compared to Onyx 1.0, is the fact most of the settings are now accessible via the Settings button in the toolbar.

There are two settings that I particularly like.

One is the option were you can easily specify which of the methods shall be ignored by Onyx. In other words which of the methods you don’t want to see in the generated output.

The other one is the obfuscation feature. This allows you to suppress sensitive information from the generated output and/or the log.

The new Onyx 2.0 definitely is a more user-friendly and robust application than it’s predecessor. But while using the product there are some features that seem to be missing or that would make the product even better.
What follows is a “personal” list of notes I took while test driving the product.

• “Start a new Connection” has a “Launch Client when connected” option, which is great! But the Username and Password have to be entered each time. It would have been handy if there was something like the “Use Windows session credentials” option like the vSphere client has.

• The Connect button in the Onyx toolbar has a “History” option where you can chose any of the previous connections you made in the past. Unfortunately the “History” doesn’t keep track of the client with which you connected, neither does it allow you to specify a client to connect.

• The Connection history is kept in the Onyx.settings file, which is in fact an XML file. When you want to remove one of these old connections, you will have to edit the Onyx.settings file. It would have been handy to have this option available under the Settings option.

• The “Launch Client when connected” option is a great new feature. It would have been useful if it allowed the user, in the Settings perhaps, to add additional clients. I’m thinking of the PowerGui, vEcoShell, PowerShellPlus… users.
• The raw SOAP output only shows the request packets, not the reply packets. From the log it is obvious that this information is available. Perhaps it could be a useful feature to allow for the selection of the reply packet in the raw SOAP output.

Don’t let it be misunderstood, Onyx 2.0 is a product that you need when you take automation through PowerCLI serious. A product that you should know and use in your vSphere environment.

• It will help you understand what goes on under the hood.

• It will make it easier to find which methods and objects to use for those functions that are not yet covered by a PowerCLI cmdlet.

Download Onyx 2.0 and start playing!

This post will show you:

• how to set up the SDK for use from PowerShell
• some usage examples from PowerShell.

The SDK package doesn’t contain a DLL with a manifest so we can’t use the LoadFile method to load the DLL into memory. But the ThinApp SDK has foreseen this.

First make sure you have the ThinAppSDK.dll and ThinAppSDKSrvr.exe files together in a directory.

Then register the DLL with the regsvr32 command. On a Windows client that has UAC enabled make sure you run regsvr32 from an elevated command prompt (run as administrator) !

Thanks to Greg Geldorp, the VMware Engineer that owns the ThinApp SDK, we know how the DLL gets loaded on a 64-bit client.

I quote from his reply in this thread.

“ThinAppSDK.dll is the DLL that implements the ThinApp SDK objects like ThinApp.Management. It is a 32-bit DLL, if you create a ThinApp.Management object from a 32-bit process Windows will load ThinAppSDK.dll into that process (if the DLL was registered correctly with regsvr32). However, it’s not possible to load a 32-bit DLL into a 64-bit process. So if you try to create a ThinApp.Management object from a 64-bit process, ThinAppSDK.dll cannot be loaded directly into that process.
That’s where ThinAppSDKSrvr.exe comes in. Windows will detect this situation and will automatically launch ThinAppSDKSrvr.exe. Since ThinAppSDKSrvr.exe is a 32-bit application, it can load ThinAppSDK.dll just fine. Windows will then “forward” ThinApp SDK calls (e.g. the GetThinAppType method call) from the 64-bit process to ThinAppSDKSrvr.exe, which will in turn forward it to ThinAppSDK.dll. The result will be passed back from ThinAppSDK.dll to ThinAppSDKSrvr.exe via Windows to the 64-bit application.”

You can always verify, for example with MSInfo32, that the DLL has been loaded.

What follows was tested on Windows XP Pro Sp3 32-bit and on Windows 7 64-bit.

All the following examples are based on a ThinApped version I made of XmlPad3, my favorite ‘free’ XML editor.

Let’s start with something simple.

The following lines will take an EXE produced by the ThinApp build process and determine the ThinApp type of file.

$sampleApp = "C:\ThinApp\XmlPad3\bin\XmlPad.exe"

$thinMgmt = New-Object -ComObject ThinApp.Management
$thinMgmt.GetThinAppType($sampleApp)

The result of the this simple script is not world-shocking but at least it show that we can interact with the ThinApp package.

From the VMware ThinApp SDK API Reference Guide, that is part of the SDK, we learn that this file is the main data container.

Let’s take another entry point of the package. In this case the cmd.exe entry point.

And we learn that this is a shortcut, or entry point, but not a container.

Time to move on to a somewhat more advanced script.

With the SDK there are 2 samples and one of these is a script that lists subfolders and files for a specific path in the package.

This script is a loose emulation of the original.

$sampleApp = "C:\ThinApp\XmlPad3\bin\XmlPad.exe"
$target = "%ProgramFilesDir%\WMHelp Software\WMHelp XmlPad"

$thinMgmt = New-Object -ComObject ThinApp.Management
$package = $thinMgmt.OpenPackage($sampleApp)
$vfs = $package.GetVFileSystemObject($null)
$subFolderCount = 0
$fileCount = 0
$totalFileSize = 0
if($vfs.FolderExists($target)){
    $folder = $vfs.GetFolder($target)
    Write-Host "Directory of" $sampleApp "|" $target
    $subFolderCount = $folder.SubFolders.Count
    $folder.SubFolders | %{
        Write-Host $_.DateLastModified "<DIR>" $_.Size $_.Name
    }
    $fileCount = $folder.Files.Count
    $folder.Files | %{
        $totalFileSize += $_.Size
        Write-Host $_.DateLastModified "     " $_.Size $_.Name
    }
}
elseif($vfs.FileExists($target)){
    $file = $vfs.GetFile($target)
    Write-Host "Directory of" $sampleApp "|" $file.Path
    $fileCOunt = 1
    $totalFileSize += $file.Size
    Write-Host $file.DateLastModified "     " $file.Size $file.Name
}
else{
    Write-Host "Directory of" $sampleApp
    Write-Host "File not found"
}
Write-Host $fileCount "File(s)" $totalFileSize "bytes"
Write-Host $subFolderCount "Dir(s)"

A run of this script produces something like this:

The ThinApp SDK offers even for the PowerShell adept easy access to his ThinApp packages.  Another small step towards the complete automation of your ‘virtual’ environment.

In PowerCLI 4.1 some objects now have, besides the ParentId property, the actual parent object itself. This will take quite some complexity out of several scripts. This is definitely a good improvement.

The 2nd novelty in that post were nested properties. Some SDK object properties are now mapped directly in the PowerCLI object. For example the VMHost object now holds the StorageInfo and NetworkInfo properties besides some others. And what is even better, there is hardly any performance impact since the values will only be loaded the first time you access the properties (similar to the ExtensionData property).

The selection of the SDK properties that are available this way were made by the PowerCLI Dev Team and are probably based on the most commonly used SDK object properties. But what if you want another property of the SDK object in your script ?

You can of course access all the properties of the SDK object through the ExtensionData property. But that comes with a performance penalty and will be visible when you run your script against a  larger vSphere environment.

Luckily PowerCLI 4.1 also introduced the New-VIProperty cmdlet.

With that cmdlet it’s quite simple to map your selection of SDK object properties into the PowerCLI object.

Suppose you want to see/check the ESX(i) host’s hardware in a script. Then you can just add this line

New-VIProperty -Name 'Hardware' -ObjectType 'VMHost' `
	-ValueFromExtensionProperty 'Hardware' -Force

And bingo, each object returned by Get-VMHost will now have a Hardware property.

Another example is the yellow folder to which a VirtualMachine belongs. In the PowerCLI object for a VirtualMachine you have a Folder property, but that points to the blue folder in which a guest is located.

Again the New-VIProperty comes to the rescue.

New-VIProperty -Name 'YellowFolderName' -ObjectType 'VirtualMachine' -Value {
	param($vm)

	$entity = Get-View $vm.ExtensionData.ResourcePool
	while($entity.GetType().Name -ne 'Folder'){
		$entity = Get-View $entity.Parent
	}
	$entity.Name
} -Force -BasedOnExtensionProperty 'ResourcePool'

Heck, you can even use a recursive function. Like this:

New-VIProperty -Name 'YellowFolderName' -ObjectType 'VirtualMachine' -Value {
	param($vm)

	function Get-FirstFolder{
		param($entityMoRef)

		$entity = Get-View $entityMoRef
		while($entity.GetType().Name -ne 'Folder'){
			$entity = Get-FirstFolder $entity.Parent
		}
		$entity
	}

	(Get-FirstFolder $vm.ExtensionData.ResourcePool).Name
} -Force

Don’t know how you feel, but I’m convinced of the power of the New-VIProperty cmdlet

With the power of the New-VIProperty cmdlet I feel that these extra properties, that PowerCLI 4.1 introduces, should have been optional or even left out by default. I know there is hardly any performance impact involved, but on the other hand they don’t make it easier for a PowerCLI newbie to grasp what is there.

What do you think ?

At about the same time Alan got the once-in-a-lifetime oppurtunity to join EMC as a vSpecialist.

Now you don’t become a vSpecialist for free! Alan has been spending a fair amount of time in training, coming up to speed with VCE (VMware/Cisco/EMC) and traveling all over the globe. The little time he was at home, went of course in the first place to his loved ones.

As a consequence, our chapter-writing schedule failed misserably.
So we decided to get some extra help for the book in order to keep our deadline.

And now, for the proverbial “silver lining”, have a look who agreed to help us out:

Glenn Sizemore	Jonathan Medd	Arnim van Lieshout
Winner of the Scripting Games 2010 vExpert	PowerShell MVP	vExpert

In other words, the book will now be written by “4 vExperts and a MVP” !

Can’t wait to read it

Unfortunately this new feature is not available in PowerCLI 4.1. That means you can’t set this up in your configuration scripts through a PowerCLI cmdlet. In most such cases you can fall back on one of the SDK APIs to bypass this lack of a cmdlet. But unfortunately the new “managers”, of which HostActiveDirectoryAuthentication is one, are not available in the VMware.Vim assembly either.

Rob raised this in a recent PowerCLI Community thread. Yasen, one of the PowerCLI Dev Team members, provided a bypass. To make this bypass a bit more accessible, I decided to roll it up in a PowerShell function.

The Active Directory Authentication feature has been discussed at length in several vSphere 4.1 blog posts.

A great video, that explains what it is and how you can use it, was made by David Davis. See his VIDEO: New vSphere 4.1 Windows Active Directory Authentication post.

#requires -version 2

function Get-VMHostAuthentication{
<#
.SYNOPSIS
	The function retrieves the authentication services from an ESX(i) host
.DESCRIPTION
	This function retrieves the configured authentication services from one
	or more ESX(i) hosts.
.NOTES
	Author: Luc Dekens
.PARAMETER VMHost
	Specify the ESX(i) host
.EXAMPLE
	PS> Get-VMHost | Get-VMHostAuthentication
.EXAMPLE
	PS> Get-VMHostAuthentication -VMHost (Get-VMHost)
#>
	param(
	[parameter(ValueFromPipeline = $true,Position=1,Mandatory = $true)]
	[VMware.VimAutomation.ViCore.Impl.V1.Inventory.VMHostImpl[]]$VMHost)

	process{
		if(!$VMHost){$VMHost = $_}
		foreach($esx in $VMHost){
			$filter = New-Object VMware.Vim.PropertyFilterSpec -Property @{
				ObjectSet = New-Object VMware.Vim.ObjectSpec -Property @{
					Obj = $esx.ExtensionData.ConfigManager.AuthenticationManager
				}
				PropSet = New-Object VMware.Vim.PropertySpec -Property @{
					Type = "HostAuthenticationManager"
					All = $true
				}
			}
			$collector = Get-View $esx.ExtensionData.Client.ServiceContent.PropertyCollector
			$content = $collector.RetrieveProperties($filter)
			$stores = $content | Select -First 1 | %{$_.PropSet} | where {$_.Name -eq "info"}
			foreach($authConfig in $stores.Val.AuthConfig){
				$row = New-Object PSObject
				$row | Add-Member -MemberType NoteProperty -Name Name -Value $null
				$row | Add-Member -MemberType NoteProperty -Name Type -Value $null
				$row | Add-Member -MemberType NoteProperty -Name Enabled -Value $null
				$row | Add-Member -MemberType NoteProperty -Name Domain -Value $null
				$row | Add-Member -MemberType NoteProperty -Name Membership -Value $null
				$row | Add-Member -MemberType NoteProperty -Name Trust -Value $null
				$row.Name = $esx.Name
				$row.Enabled = $authConfig.Enabled
				switch($authConfig.GetType().Name){
					'HostLocalAuthenticationInfo'{
						$row.Type = 'Local authentication'
					}
					'HostActiveDirectoryInfo'{
						$row.Type = 'Active Directory'
						$row.Domain = $authConfig.JoinedDomain
						$row.Membership = $authConfig.DomainMembershipStatus
						$row.Trust = $authConfig.TrustedDomain
					}
				}
				$row
			}
		}
	}
}

function Set-VMHostADDomain{
<#
.SYNOPSIS
	The function adds or removes an ESX(i) 4.1 server to/from an Active Directory domain
.DESCRIPTION
	By adding an ESX(i) 4.1 host to an AD domain, you can use AD authentication for console
	access, SSH and permissions on the host's child objects
.NOTES
	Author: Luc Dekens
.PARAMETER VMHost
	Specify the ESX(i) host
.PARAMETER Domain
	The name of the Active Directory domain in FQDN notation
.PARAMETER User
	An Active Directory account that administrative authority to add hosts to AD
.PARAMETER Password
	The password for the AD account specified in -User
.PARAMETER Credential
	The credentials for an AD account with administrative authority to add hosts to AD
.PARAMETER ADJoin
	A switch indicating if the host shall be added ($true) or removed ($false) from AD
.PARAMETER RemovePermission
	Will remove ($true) all AD permissions that still exist on the ESX(i) host and
	it's children
.EXAMPLE
	PS> Get-VMHost | Set-VMHostADDomain -ADJoin:$true -Domain $domain -User $user -Password $pswd
.EXAMPLE
	PS> Set-VMHostADDomain -VMHost (Get-VMHost) -ADJoin:$true -Credential $cred
.EXAMPLE
	PS> Set-VMHostADDomain -VMHost (Get-VMHost) -ADJoin:$false -RemovePermission:$true
#>
	param(
	[parameter(ValueFromPipeline = $true,Position=1,Mandatory = $true)]
	[VMware.VimAutomation.ViCore.Impl.V1.Inventory.VMHostImpl[]]$VMHost,
	[string]$Domain,
	[string]$User,
	[string]$Password,
	[System.Management.Automation.PSCredential]$Credential,
	[switch]$ADJoin,
	[switch]$RemovePermission = $false
	)

	process{
		if(!$VMHost){$VMHost = $_}
		foreach($esx in $VMHost){
			$filter = New-Object VMware.Vim.PropertyFilterSpec -Property @{
				ObjectSet = New-Object VMware.Vim.ObjectSpec -Property @{
					Obj = $esx.ExtensionData.ConfigManager.AuthenticationManager
				}
				PropSet = New-Object VMware.Vim.PropertySpec -Property @{
					Type = "HostAuthenticationManager"
					All = $true
				}
			}
			$collector = Get-View $esx.ExtensionData.Client.ServiceContent.PropertyCollector
			$content = $collector.RetrieveProperties($filter)
			$stores = $content | Select -First 1 | %{$_.PropSet} | where {$_.Name -eq "supportedStore"}
			$result = $stores.Val | where {$_.Type -eq "HostActiveDirectoryAuthentication"}
			$hostADAuth = [VMware.Vim.VIConvert]::ToVim41($result)

			if($ADJoin){
				if($Credential){
					$User = $Credential.GetNetworkCredential().UserName
					$Password = $Credential.GetNetworkCredential().Password
				}
				$taskMoRef = $esx.ExtensionData.Client.VimService.JoinDomain_Task($hostADAuth,$Domain,$User,$Password)
			}
			else{
				$taskMoRef = $esx.ExtensionData.Client.VimService.LeaveCurrentDomain_Task($hostADAuth,$RemovePermission)
			}
			$esx.ExtensionData.WaitForTask([VMware.Vim.VIConvert]::ToVim($taskMoRef))
		}
	}
}

Line 1: This function uses PowerShell v2 features and will not run with PowerShell v1

Line 3-63: The Get-VMHostAuthentication function

Line 38-60: Return information for each of the configured authentication methods.

Line 65-138: The Set-VMHostADDomain function

Line 24,108: Handles use of the function in a pipeline or with a -VMHost parameter

Line 26-34,110-118: Creates the PropertyFilterSpec object through the use of the hash table property list.

Line 119-123: The ‘bypass’ from Yasen. It uses the PropertyCollector to  ultimately get at the MoRef of the HostActiveDirectoryAuthentication manager.

Line 123: The MoRef is converted to a VIM 4.1 MoRef with the ToVim41 method of the builtin VIConvert class.

Line 125: Depending on the state of the ADJoin switch, the host is ‘joined’ or ‘removed’ from the AD domain.

Line 126-129: If the AD account is passed through a PSCredential object, the clear text account and password are extracted.

Line 135: The script waits for the completion of the called method.

To display the authentication services for an ESX(i) server that has no AD authentication configured.

Get-VMHost $esxName | Get-VMHostAuthetication

This returns the following information.

To add a specific ESX(i) server to an Active Directory domain you can do

Set-VMHostADDomain -VMhost (Get-VMHost $esxName) -ADJoin:$true -Domain 'test.local' -User 'Administrator' -Password 'MyPassword'

or

Get-VMHost $esxName | Set-VMHostADDomain -ADJoin:$true -Domain 'test.local' -User 'Administrator' -Password 'MyPassword'

The Get-VMHostAuthentication function now returns.

You can also enable AD authentication on a series of ESX(i) hosts.

Set-VMHostADDomain -VMhost (Get-VMHost) -ADJoin:$true -Domain 'test.local' -Credential (Get-Credential)

or like this

Get-VMHost | Set-VMHostADDomain -ADJoin:$true -Domain 'test.local' -Credential (Get-Credential)

To remove a host, you have to set the -ADJoin switch to $false.
Like this

Get-VMHost | Set-VMHostADDomain -ADJoin:$false

The previous call assumes that there were no permissions with AD principals left on the ESX(i) server or any of it’s children.
If there were, you would get an error message.

You can let the ‘unjoin’ remove any remaining permissions for you.
For that you use the -RemovePermission parameter.

Get-VMHost | Set-VMHostADDomain -ADJoin:$false -RemovePermission:$true

The New-VIProperty cmdlet allows you to add a new properties to any PowerCLI object (InventoryItem, HostSystem, VirtualMachine…). And it’s up to you to define what the value of this new property will !

The Remove-VIProperty cmdlet allows you to remove such a new property.

These new cmdlets, combined with the new Extension property, will change several of the well established ways that you use in your PowerCLI scripts. And besides adding a lot of flexibility, the extension property will make your scripts considerably faster (if used wisely).

In short, it’s a new property you add to any of the PowerCLI objects. You select the name of the property and also what will be the value of the new property.

An example will perhaps make it clearer.

Let’s take the PowerCLI Datastore object. It’s returned by the Get-Datastore cmdlet and has the following properties.

Notice the FreeSpaceMB and CapacityMB properties, which are both giving the values in MB.

You are now tasked to create a report containing the name of the datastore and the free space in GB.

Till now you had to do something like this:

And if you wanted this in a nicer format, you could do:

But with the New-VIProperty cmdlet this can be done differently now.

First you define the new property. You specify a Name for the property, the PowerCLI object on which you want to add the new property and how the value of the new property shall be obtained/calculated.

And from now on you can use the new FreeSpaceGB property for example in a Select-Object cmdlet like this.

No more need to remember the tedious expression from before. You compose your New-VIProperty cmdlet and the new property will be there each time you retrieve a Datastore object.

Note that an extension property defined this way, will only exist for as long as you have your PowerCLI environment running.  If you restart your PowerCLI environment, the new property will not be there anymore.

If you want to have some of these new properties available each time you start your PowerCLI environment, you could for example place the New-VIProperty statements in your profile.

In the previous example we populated the new property with a recalculated value of an existing property of the PowerCLI object. But there are other ways of setting the value of a new property.

In the sample script I used the $args[0] variable to refer to the PowerCLI object itself. You can also define the current PowerCLI object with the param statement.

New-VIProperty -Name FreeSpaceGB -ObjectType Datastore -Value {
	"{0:f1}" -f ($Args[0].FreeSpaceMB/1KB)
}

is equivalent to

New-VIProperty -Name FreeSpaceGB -ObjectType Datastore -Value {
	param($ds)

	"{0:f1}" -f ($ds.FreeSpaceMB/1KB)
}

Suppose we are asked to provide per datastore how many guests are stored on that datastore. As always with PowerShell/PowerCLI there are many ways of doing this, but in this case we are going for the New-VIProperty cmdlet.

In the API Datastore object there is a property, called VM, that holds a MoRef to each guests stored on that datastore. If we count the number of entries in this property, we will have the number of guests.

In the latest PowerCLI build the complete API object is accessible under the ExtensionData propery. That will allow us to do something like this.

New-VIProperty -Name VMcount -ObjectType Datastore -Value {
	param($ds)

	$ds.ExtensionData.Vm.Count
}

Great, we now have access to the complete API object. No more need to do a Get-View everytime we need the API object.
But the tradeoff is that the mapping of the API object takes resources.

Your scripts will run longer when the API object has to be fetched and mapped each time a PowerCLI object is fetched.
Luckily the PowerCLI Dev Team came up with a great solution!
The ExtensionData property will only map to the API object when you access the property or any of the subproperties.
And to make it even more optimal, the Dev Team introduced the -BasedOnExtensionProperty parameter.

This parameter allows you to specify which specific properties of the API object you are going to use in the -Value script block.

That way access can even be more optimised. In our sample this would become.

New-VIProperty -Name VMcount -ObjectType Datastore -Value {
	param($ds)

	$ds.ExtensionData.Vm.Count
} -BasedOnExtensionProperty 'vm'

When you now run the Get-Datastore cmdlet against a fair number of datastores, you will notice that the execution time will have improved dramatically.

Quite often the actions you have to take for the -Value parameter are simple one-to-one mappings or any of the following aggregation functions: SUM, AVERAGE or AVG, MIN, MAX, UNIQUE, COUNT.

With the -ValueFromExtensionProperty parameter you can use a simple mapping or one of the aggregation functions without setting up a complete script block.
Our example would become

New-VIProperty -Name VMcount -ObjectType Datastore -ValueFromExtensionProperty 'COUNT vm'

This concludes my short review of the New-VIProperty and Remove-VIProperty cmdlets.

Play around with these new cmdlets and feel free to post some of the amazing use cases you will no doubt come up with.

Exactly such a question was raised by Alasdair in his thread new-datastore on ESX4i Installable local disk in the PowerCLI Community recently.

When you create a new datastore via the vSphere client and you select a LUN or local disk that has already partitions that are in use, you get two options.

You can select to take the complete LUN/local disk and this will remove the partitions that were already there.

Or you can select the option to take the “free space” partition.

The following functions will allow you to find the “free space” partitions and to create a VMFS datastore on such a partition.

function Get-ScsiFreePartition{
	param (
	[parameter(ValueFromPipeline = $true,Position=1)]
	[ValidateNotNullOrEmpty()]
	[VMware.VimAutomation.Client20.VMHostImpl]
	$VMHost
	)

	process{
		$esx = $VMHost | Get-View
		$storMgr = Get-View $esx.ConfigManager.DatastoreSystem
		$storSys = Get-View $esx.ConfigManager.StorageSystem

		$lunExt = $storMgr.QueryAvailableDisksForVmfs($null)
		foreach($lun in $lunExt){
			$info = $storMgr.QueryVmfsDatastoreCreateOptions($lun.DevicePath)
			foreach($dsOpt in $info){
				$info2 = $storSys.ComputeDiskPartitionInfo($lun.DevicePath,$dsOpt.Info.Layout)
				$info2.Layout.Partition | where {$_.Type -eq "vmfs"} | %{
					New-Object PSObject -Property @{
						DeviceName = $lun.DeviceName
						DeviceSizeMB = $lun.Capacity.block * $lun.Capacity.BlockSize / 1MB
						Partition = $_.Partition
						PartitionSizeMB = ($_.End.block - $_.Start.Block) * $_.Start.BlockSize / 1MB
						FullDisk = &{if($_.Partition -eq 1){$true}else{$false}}
					}
				}
			}
		}
	}
}

function New-PartitionDatastore{
	param (
	[parameter(ValueFromPipeline = $true,Position=1)]
	[ValidateNotNullOrEmpty()]
	[VMware.VimAutomation.Client20.VMHostImpl]
	$VMHost,
	[parameter(Position=2)]
	[ValidateNotNullOrEmpty()]
	[PSObject]
	$Partition,
	[parameter(Position=3)]
	[ValidateNotNullOrEmpty()]
	[String]
	$Name
	)

	process{
		$esx = $VMHost | Get-View
		$storMgr = Get-View $esx.ConfigManager.DatastoreSystem

		$lunExt = $storMgr.QueryAvailableDisksForVmfs($null)
		$device = $lunExt | where {$_.DeviceName -eq $Partition.DeviceName}
		$dsOpt = $storMgr.QueryVmfsDatastoreCreateOptions($Partition.DeviceName) | where {$_.Info.VmfsExtent.Partition -eq $Partition.Partition}

		$spec = $dsOpt.Spec
		$spec.Vmfs.VolumeName = $Name
		$spec.Extent += $spec.Vmfs.Extent
		$dsMoRef = $storMgr.CreateVmfsDatastore($spec)
		Get-Datastore (Get-View $dsMoRef).Name
	}
}

Line 14,53: The QueryAvailableDisksForVmfs method will return all disks that can be used to create VMFS datastores.

Line 16,55: The QueryVmfsDatastoreCreateOptions method will return all possible options to create (or extend) a datastore on the specific disk.

Line 18: The ComputeDiskPartitionInfo method will determine the disk partition layout for the specified disk layout.

Line 20-25: Creates and populates the objects with the available partition configurations.

Line 57-59: Populate the VmfsDatastoreCreateSpec object.

Line 60: Create the new VMFS datastore with the CreateVmfsDatastore method.

Line 61: Return a VMware.VimAutomation.Client20.DatastoreImpl object to make the function consistent with the New-Datastore cmdlet.

With the Get-ScsiFreePartition you can find the “free space” partitions on LUNs or local disks connected to your ESX(i) server.

$esxName = "esx4i.test.local"
$esxImpl = Get-VMHost -Name $esxName
$esxImpl | Get-ScsiFreePartition | ft -AutoSize

This will produce output similar to this.

All returned partitions that are partition one mean the complete LUN or local disk. And they will hence destroy the existing data. For your convenience the function adds the FullDisk property which indicates this fact again.

If we want to use the “free space” partition, we have to do some filtering. Something like this for example

$esxName = "esx4i.test.local"
$esxImpl = Get-VMHost -Name $esxName
$partition = $esxImpl | Get-ScsiFreePartition | where {!$_.FullDisk} | Select -First 1
$esxImpl | New-PartitionDatastore -Partition $partition -Name "MyDS"

will create a new VMFS datastore on the first “free space” partition and the datastore will be named MyDS.

The properties of the datastore show that it is located on partition 3, the previous “free space” partition.

Notice that the New-PartitionDatastore returns the newly created datastore, similar to what the New-Datastore cmdlet does.

Since vSphere doesn’t maintain any historical data with the guests themselves, we have to fall back on the Tasks and Events to create such a report. The basic algorithm to query the tasks, and their related events, is already published in Events – Part 3 : Auditing VM device changes. But to get a historical record of the servers where your guests have been hosted requires a bit more logic in the script.

There are 2 distinct parts in the script.

1. Get all the vMotion tasks. A vMotion can be user- or DRS-initiated. These produce different tasks.
2. Compile the final report by only looking at the selected guests

$hours = 24 # Number of hours back
$start = (Get-Date).AddHours(-$hours)
$tasknumber = 999 # Windowsize for task collector
$eventnumber = 100 # Windowsize for event collector
$tgtTaskDescriptions = "VirtualMachine.migrate","Drm.ExecuteVMotionLRO"
$migrations = @()
$report = @()

# Get the guest for which we want the report
$vmHash = @{}
Get-Datacenter -Name "MyDC" | Get-VM | %{
	$vmHash[$_.Name] = $_.Host
}

# Retrieve the vMotion tasks and the corresponding events
$taskMgr = Get-View TaskManager
$eventMgr = Get-View eventManager

$tFilter = New-Object VMware.Vim.TaskFilterSpec
$tFilter.Time = New-Object VMware.Vim.TaskFilterSpecByTime
$tFilter.Time.beginTime = $start
$tFilter.Time.timeType = "startedTime"

$tCollector = Get-View ($taskMgr.CreateCollectorForTasks($tFilter))

$dummy = $tCollector.RewindCollector
$tasks = $tCollector.ReadNextTasks($tasknumber)

while($tasks){
	$tasks | where {$tgtTaskDescriptions -contains $_.DescriptionId} | % {
		$task = $_
		$eFilter = New-Object VMware.Vim.EventFilterSpec
		$eFilter.eventChainId = $task.EventChainId

		$eCollector = Get-View ($eventMgr.CreateCollectorForEvents($eFilter))
		$events = $eCollector.ReadNextEvents($eventnumber)
		while($events){
			$events | % {
				$event = $_
				switch($event.GetType().Name){
					"VmBeingHotMigratedEvent" {
						$migrations += New-Object PSObject -Property @{
							VMname = $task.EntityName
							Source = $event.Host.Name
							Destination = $event.DestHost.Name
							Start = $task.StartTime
							Finish = $task.CompleteTime
							Result = $task.State
							User = $task.Reason.UserName
							DRS = &{if($task.DescriptionId -like "Drm.*"){$true}else{$false}}
						}
					}
					Default {}
				}
			}
			$events = $eCollector.ReadNextEvents($eventnumber)
		}
		$ecollection = $eCollector.ReadNextEvents($eventnumber)
# By default 32 event collectors are allowed. Destroy this event collector.
		$eCollector.DestroyCollector()
	}
	$tasks = $tCollector.ReadNextTasks($tasknumber)
}

# By default 32 task collectors are allowed. Destroy this task collector.
$tCollector.DestroyCollector()

# Handle the guests that have been vMotioned
$grouped = $migrations | Group-Object -Property VMname
$grouped | Sort-Object -Property Count -Descending | where{$vmHash.ContainsKey($_.Name)} | %{
	$i = 1
	$row = New-Object PSObject
	Add-Member -InputObject $row -Name VM -Value $_.Name -MemberType NoteProperty
	$_.Group | Sort-Object -Property Finish | %{
# The original location of the guest
		if($i -eq 1){
			Add-Member -InputObject $row -Name ("Time" + $i) -Value $start -MemberType NoteProperty
			Add-Member -InputObject $row -Name ("Host" + $i) -Value $_.Source -MemberType NoteProperty
			$i++
		}
# All the vMotion destinations
		Add-Member -InputObject $row -Name ("Time" + $i) -Value $_.Finish -MemberType NoteProperty
		Add-Member -InputObject $row -Name ("Host" + $i) -Value $_.Destination -MemberType NoteProperty
		Add-Member -InputObject $row -Name ("DRS" + $i) -Value $_.DRS -MemberType NoteProperty
		Add-Member -InputObject $row -Name ("User" + $i) -Value $_.User -MemberType NoteProperty
		$i++
	}
	$report += $row
	$vmHash.Remove($_.Name)
}

# Add remaining guests to report
$vmHash.GetEnumerator() | %{
	$row = New-Object PSObject
	Add-Member -InputObject $row -Name VM -Value $_.Name -MemberType NoteProperty
	Add-Member -InputObject $row -Name Time1 -Value $start -MemberType NoteProperty
	Add-Member -InputObject $row -Name Host1 -Value $_.Value -MemberType NoteProperty
	Add-Member -InputObject $row -Name DRS1 -Value $false -MemberType NoteProperty
	$report += $row
}

$report | Export-Csv "C:\vMotion-history.csv" -NoTypeInformation -UseCulture

Line 1-2: Specify how far back you want to report on through the $start variable

Line 5: The vMotion tasks can be triggered by a user or by DRS.

Line 10-13: We collect all the guests on which we want to report. This can be changed at will, just make sure the hash table $vmHash is populated.

Line 15-66: This is the same logic as was used in Events – Part 3 : Auditing VM device changes

Line 69-70: We group the discovered vMotions per guest and then we sort the groups descending on the Count property. This is required because we will later use the Export-Csv cmdlet and if the array holds rows of varying length, the longest row should be first.

Line 76-80: For the vMotioned guest we first make an entry with the original location at time $start.

Line 82-86: For each vMotion we add the same properties Time, Host, DRS and User, but each time with an incremented suffix number.

Line 89: A guest that was vMotioned is removed from hash table to avoid duplicate entries later one.

Line 93-100: The guest that were not vMotioned during the reporting interval are added to the report.

The script produces a CSV file that contains quite a bit of information.

A guest that was not vMotioned during the reporting interval will only have a Time1 and Host1 column.

Guests that were vMotioned will have additional Timen and Hostn columns. Notice the DRSn column which indicates if a vMotion was triggered by DRS or by a user.  The Usern column will display the username if it was a user-invoked vMotion.

After some deep-diving in the VMware.VumAutomation cmdlets, I discovered that this question could probably not be answered with the VMware.VumAutomation cmdlets  . On top of that, the current build seems to have a problem with Get-Baseline cmdlet when you use the -Inherit parameter.

So, as always, into the SDK

Luckily, the HostPatchManager holds the answer. Via the QueryHostPatch_Task method you have access to all the installed bulletins.

Update June 25th 2010: after comments from Suresh and James I investigated what goes wrong when you run the script against an ESXi server. It turned out that the QueryHostPatch_Task method doesn’t return the bulletins for ESXi hosts.

And this seems to be consistent with the behaviour of the esxupdate query command.

I’m not sure if this is to be considered a bug or if this is intentional !

The script has been updated to display a warning when run against an ESXi server.

Update June 30th 2010: After further investigation it looks as if the problem only exist for ESXi 4 and ESXi 4u1 servers.  Until further conclusive test I have adapted the script.

$report = @()
Get-View -ViewType HostSystem -Filter @{"Name"="esx4i.test.local"} | %{
	$esx = $_
	if($esx.Config.Product.ProductLineId -ne "embeddedesx" -or ($esx.Config.Product.ProductLineId -eq "embeddedesx" -and $esx.Config.Product.Build -ge 261974)){
		$patchMgr = Get-View $esx.ConfigManager.PatchManager
# 	$spec = New-Object VMware.Vim.HostPatchManagerPatchManagerOperationSpec
# 	$spec.cmdOption = "-a"
# 	$taskMoRef = $patchMgr.QueryHostPatch_Task($spec)
		$taskMoRef = $patchMgr.QueryHostPatch_Task($null)
		$task = Get-View $taskMoRef
		while("running","queued" -contains $task.Info.State){
			sleep 2
			$task.UpdateViewData("Info")
		}
		$result = [xml]$task.Info.Result.XmlResult
		$result."esxupdate-response".bulletin | %{
			$row = "" | Select ESX,"Bulletin ID",Installed,Summary
			$row.ESX = $esx.Name
			$row."Bulletin ID" = $_.id
			$row.Installed = [datetime]$_.installDate
			$row.Summary = $_.summary
			$report += $row
		}
	}
	else{
		Write-Host -fo red "Host: " -NoNewline
		Write-Host -fo white $esx.Name -NoNewline
		Write-Host -fo red " OS: " -NoNewline
		Write-Host -fo white $esx.Config.Product.FullName -NoNewline
		Write-Host -fo gray " does not support the QueryHostPatch method"
	}
}
$report | Sort-Object -Property ESX,Installed | ft -AutoSize

Line 4: Check if the host runs an ESXi version.

Line 9: For our initial purpose we don’t need any of the properties in the HostPatchManagerPatchManagerOperationSpec object. Note that you could pass esxupdate options. If you add for example the option -a, the method will return all bulletins, not just the ones that are applicable. You will need to uncomment lines 6-8 and comment out line 9.

Line 15: The result is returned as an XML object.

Line 16: All returned bulletins are a node under “esxupdate-response”.bulletin. Note that we need the quotes around the first path element, otherwise the dash would pose problems.

The script, as it is written above, dumps the results to the console. But you can just as well send the results to a CSV or XLS (see my Beyond Export-Csv: Export-Xls post) file.

Note that the script can run against all or a selection of your ESX servers in 1 run. You can use the -Filter parameter on the Get-View cmdlet or you can use a Where-Object cmdlet to select specific ESX hosts.

If one of the hosts you selected runs an ESXi version, the script will display a message on the console.

A nice addition to your daily reports for your ESX servers.

With the recent introduction of the vCenter Update Manager PowerCLI snapin, it is now fairly easy to automate all aspects of patch and update management in your vSphere environment. And with the help of the extended events produced by Update Manager, you can easily produce reports. This post shows one way of doing this.

Again we take the 2nd script from the  Events, Dear Boy, Events – Part 2 post and we produce a spreadsheet with all the events known in our vSphere environment.

As you can see the events coming from Update Manager can be recognised by the value in the Description field. All event types for Update Manager start with the text com.vmware.vcIntegrity in this field.

This allows us to easily filter out all the events for the Update Manager report.

Unfortunately there is, afaik, no documentation available on the exact contents of these events. So if you want to use them, there will be a bit of reverse-engineering and guess-work involved. But with what you can see in the Task&Events tab in the vSphere client, you have a fairly good idea of what is available in these events.

Btw there are 124 different events the Update Manager can produce ! So there is a lot more information out there.

$events = Get-VIEvent -Start (Get-Date).AddDays(-1) | `
	where {$_.EventTypeId -like "com.vmware.vcIntegrity*"} | `
	Sort-Object -Property CreatedTime
$events | %{
	New-Object PSObject -Property @{
		Date = $_.CreatedTime
		User = $_.UserName
		Host = $_.Host.Name
		Message = $_.FullFormattedMessage
	}
} | Export-Csv "C:\Update-Manager-report.csv" -NoTypeInformation -UseCulture

Line 2: The filter only lets Update Manager events through

Line 3: If you want the output in chronological order you better sort the events on their creation time.

Line 5: When using the New-Object cmdlet with the -Property hash table, there is no way you can determine the order in which the properties will appear in the objects. This is a known “feature” and there are some open questions to change or influence this behaviour.

All in all this is a rather simple and straightforward script, but it will allow you to add the Update Manager activities to your daily vSphere reporting.

The produced spreadsheet looks something like this. Note that the order of the columns will vary.

The Update Manager events allow for more advanced reports.

You could for example maintain a list that records at which precise date and time a specific patch was applied to which entity. Could be quite useful when problem solving.

Let me know if there is any interest in such a script and I’ll cook something together