When version 1.0.0 of this open-sourced (!) product was released, I had expected much more buzz on social media from VMware PowerCLI users. 

This appliance does in fact bring an answer to a wish that many PowerShell/PowerCLI users have had for years: a “Scripting Host“!

This Hitchhikers Guide to SRS 1.0.0 post will show how I build my own customised SRS appliance, and how I use it to run PowerShell/PowerCLI scripts.

Introduction

When you visit the Script Runtime Service for vSphere (SRS) repository, you’ll notice that this open-sourced project comes with extensive documentation.

Always a great characteristic for an open-sourced project!

The installation as a VM  (from the downloadable OVF files) or in a Kubernetes cluster is well documented.

The available documentation in the SRS repo contains instructions on how to build, and customise, your own SRS appliance under the Build and Run page.

This blog post documents how I did exactly that.

I wanted to have a fully automated, self-documenting, and repeatable method, think CI, to roll out my own SRS station.

1. Roll out the Builder station and install the prerequisites.
2. Start the Build of the SRS Appliance on the Builder station
3. Retrieve the OVF/OVA files, created from the SRS Appliance, that came out of the build
4. Use the OVF/OVA to deploy your SRS station

Creating the SRS Builder

The Prerequisites

To create an SRS OVA/OVF yourself you need a “builder” station. And on that station there need to be a number of packages installed. 

I included the installation of these packages in the “build” I describe in the next section.

The buildappliance.sh script, which you will need to run on the builder station, will use the packages to setup and configure an SRS appliance (VMware Photon based), and as the last step will generate an OVA file from the appliance.

That OVA file can then be used to deploy your SRS VM.

This is also the place where you can install extra applications and extra PowerShell modules that will be included in your DIY SRS OVA.

Create the Builder station

Preparation

I used an Ubuntu 18.04 LTS station as the Builder station. As the download of the ISO is also automated, I use the Daily Build.

#region Get the Ubuntu Bionic Cloud Image OVA Daily Build

$uri = 'https://cloud-images.ubuntu.com/bionic/current/'
$ovaName = 'bionic-server-cloudimg-amd64.ova'
$repository = 'D:\Repository\Linux\Ubuntu\'

$currentOVA = Get-ChildItem -Path "$repository\$ovaName" -ErrorAction SilentlyContinue
if (-not $currentOVA -or $currentOVA.LastWriteTime.Date -lt $now.Date) {
  $sWeb = @{
    Uri = "$uri$ovaName"
    OutFile = "$repository$ovaName"
    Verbose = $false
  }
  Invoke-WebRequest @sWeb
  $currentOVA = Get-ChildItem -Path "$repository\$ovaName"
}
#endregion

I also check if the DNS A and PTR records for the Builder station are present in DNS. The Builder station I used has an FQDN of srsbuilder.local.lab with an IP address of 192.168.10.88.

$hostName = 'srsbuilder'
$domain = 'local.lab'
$ipAddress = '192.168.10.88'

$if = Get-Netadapter
$sDns = @{
   AddressFamily = 'IPv4'
   InterfaceIndex = $if.ifIndex
}
$dns = Get-DnsClientServerAddress @sDns
$dnsServer = $dns.ServerAddresses | Get-Random
$ip = $ipAddress.Split('.')
$reverseZone = "$($ip[2]).$($ip[1]).$($ip[0]).in-addr.arpa"

try{
  Write-Verbose (Get-LogText -Text "Check DNS A record for $hostname.$domain with $ipAddress")
  $sQDns = @{
    Name = "$hostName.$domain"
    Server = $dnsServer
    ErrorAction = 'Stop'
    Verbose = $false
  }
  Resolve-DnsName @sQDns  | Out-Null
  Write-Verbose (Get-LogText -Text "DNS A record exists for $hostname.$domain with $ipAddress")
}
catch{
  Write-Verbose (Get-LogText -Text "Create A record for $hostname.$domain with $ipAddress")
  $sADns = @{
    Name = $hostName
    ZoneName = $domain
    IPv4Address = $ipAddress
    A = $true
    CreatePtr = $true
    ComputerName = $dnsServer
  }
  Add-DnsServerResourceRecord @sADns | Out-Null
}
try {
  Write-Verbose (Get-LogText -Text "Check DNS PTR record for $hostname.$domain with $ipAddress")
  $sQdns = @{
    Name = $ipAddress
    Type = 'PTR'
    Server = $dnsServer
    Verbose = $false
    ErrorAction = 'Stop'
  }
  Resolve-DnsName @sQdns | Out-Null
  Write-Verbose (Get-LogText -Text "DNS PTR record exists for $hostname.$domain with $ipAddress")
}
catch {
  Write-Verbose (Get-LogText -Text "Create PTR record for $hostname.$domain with $ipAddress")
  $sADns = @{
    Name = "$($ip[3])"
    PtrDomainName = "$hostname.$domain"
    ZoneName = $reverseZone
    ComputerName = $dnsServer
  }
  Add-DnsServerResourceRecordPtr @sADns | Out-Null
}

Deploy the Builder

For the rollout of this Builder station, I used the Cloud-Init method I described in Cloud-Init – Part 2 – Advanced Ubuntu. I had to make some small changes to the Install-CloudInitVM function from that post, so I was able to specify the memory size and the disk size. The default sizes were insufficient to run the SRS Appliance build.

The updated function now has MemoryGB and DiskGB parameters.

I also had to add a snippet to handle the issue with the $PSScriptRoot parameter when running the code from the Visual Studio Code editor.

function Install-CloudInitVM {
  <#
.SYNOPSIS
  Deploy a VM from an OVA file and use cloud-init for the configuration
  .DESCRIPTION
  This function will deploy an OVA file.
  The function transfer the user-data to the cloud-init process on the VM with
  one of the OVF properties.
.NOTES
  Author:  Luc Dekens
  Version:
  1.0 05/12/19  Initial release
.PARAMETER OvaFile
  Specifies the path to the OVA file
.PARAMETER VmName
  The displayname of the VM
.PARAMETER ClusterName
  The cluster onto which the VM shall be deployed
.PARAMETER DsName
  The datastore on which the VM shall be deployed
.PARAMETER PgName
  The portgroupname to which the VM shall be connected
.PARAMETER CloudConfig
  The path to the YAML file containing the user-data
.PARAMETER Credential
  The credentials for a user in the VM's guest OS
.EXAMPLE
  $sCloudInitVM = @{
    OvaFile = '.\bionic-server-cloudimg-amd64.ova'
    VmName = $vmName
    ClusterName = $clusterName
    DsName = $dsName
    PgName = $pgName
    CloudConfig = '.\user-data.yaml'
    Credential = $cred
  }
  Install-CloudInitVM @sCloudInitVM
#>

  [cmdletbinding()]
  param(
    [string]$OvaFile,
    [string]$VmName,
    [string]$ClusterName,
    [string]$DsName,
    [string]$PgName,
    [string]$CloudConfig,
    [PSCredential[]]$Credential,
    [int]$MemoryGB,
    [int]$DiskGB
  )

#region Bypass for known issue in VSC ()
# See https://github.com/PowerShell/vscode-powershell/issues/633

  if ($psISE) {
    $dir = Split-Path -Path $psISE.CurrentFile.FullPath
  }
  else {
    if ($profile -match "VScode") {
      $dir = Split-Path $psEditor.GetEditorContext().CurrentFile.Path
    }
    else {
      $dir = $PSScriptRoot
    }
  }
#endregion

  $waitJob = (Get-Command -Name "$dir\Wait-Job.ps1").ScriptBlock
  $userData = Get-Content -Path "$dir\$CloudConfig" -Raw

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting deployment of $vmName"

  $start = Get-Date

  $vm = Get-VM -Name $vmName -ErrorAction SilentlyContinue
  if ($vm) {
    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Cleaning up"
    if ($vm.PowerState -eq 'PoweredOn') {
      Stop-VM -VM $vm -Confirm:$false | Out-Null
    }
    Remove-VM -VM $vm -DeletePermanently -Confirm:$false
  }

  $ovfProp = Get-OvfConfiguration -Ovf $ovaFile
  $ovfProp.Common.user_data.Value = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($userData))
  $ovfProp.NetworkMapping.VM_Network.Value = $pgName

  $sApp = @{
    Source = $ovaFile
    Name = $vmName
    Datastore = Get-Datastore -Name $dsName
    DiskStorageFormat = 'Thin'
    VMHost = Get-Cluster -Name $clusterName | Get-VMHost | Get-Random
    OvfConfiguration = $ovfProp
  }
  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Importing OVA"
  $vm = Import-VApp @sApp

  if($MemoryGB){
    $vm = Set-VM -VM $vm -MemoryGB $MemoryGB -Confirm:$false
  }
  if($DiskGB){
    $hd = Get-HardDisk -VM $vm | Set-HardDisk -CapacityGB $DiskGB -Confirm:$false
  }

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting the VM"
  Start-VM -VM $vm -Confirm:$false -RunAsync | Out-Null

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Waiting for cloud-init to finish"

  $User = $Credential.GetNetworkCredential().UserName
  $Password = $Credential.GetNetworkCredential().Password

  $sJob = @{
    Name = 'WaitForCloudInit'
    ScriptBlock = $waitJob
    ArgumentList = $vm.Name, $User, $Password, $global:DefaultVIServer.Name, $global:DefaultVIServer.SessionId
  }
  Start-Job @sJob | Receive-Job -Wait

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Deployment complete"

  Write-Verbose "nDeployment took $([math]::Round((New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds,0)) seconds"
}

The call to the Install-CloudInitVM functions is rather straightforward.

$vmName = 'SRSBuilder'

# Get credential for logging on to the guest
# Replace with your secrets manager application when
# running this on PSv6 or higher

$viCred = Get-VICredentialStoreItem -Host $vmName
$secPassword = ConvertTo-SecureString -String $viCred.Password -AsPlainText -Force
$cred = [Management.Automation.PSCredential]::new($viCred.User, $secPassword)

$sCloudInitVM = @{
  OvaFile = 'D:\Repository\Linux\Ubuntu\bionic-server-cloudimg-amd64.ova'
  VmName = $vmName
  ClusterName = 'cluster'
  DsName = 'vsanDatastore'
  PgName = 'vdPg1'
  CloudConfig = 'user-data-srsbuilder.yaml'
  Credential = $cred
  MemoryGB = 4
  DiskGB = 25
  Verbose = $true
}
Install-CloudInitVM @sCloudInitVM

Notice how I used the VICredentialStore to store and retrieve the credentials for the Builder station. When you run this from a PowerShell version greater than 5.1, you will have to replace that part with calls to the secrets manager of your choice.

The real strength of using this Cloud-Init method is that one can use a YAML file to specify how the target station, the Builder station, in this case, needs to be configured. And also specify which packages shall be installed on the target station as part of the deployment.

The following extract of my YAML file shows the packages that are required. It also includes pulling down the SRS appliance.

# SRS 1.0
# A) dotnet sdk
# 1) add the Microsoft package signing key
- wget https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
# 2) add the package repository
- dpkg -i packages-microsoft-prod.deb
# 3) donet sdk
- apt-get update
- apt-get install -y apt-transport-https
- apt-get update
- apt-get install -y dotnet-sdk-3.1
# B) docker
- curl -fsSL https://get.docker.com -o get-docker.sh
- sh get-docker.sh
# C) PowerShell
- apt-get install -y wget apt-transport-https software-properties-common
- wget -q https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb
- dpkg -i packages-microsoft-prod.deb
- apt-get update
- add-apt-repository universe
- apt-get install -y powershell
# D) VMware PowerCLI
- pwsh -C '& {Install-Module -Name VMware.PowerCLI -Scope AllUsers -Force -Confirm:$false -AllowClobber}'
# E) OvfTool
- /root/ovftool-ftp.sh
- chmod 744 /root/VMware-ovftool-4.4.1-16812187-lin.x86_64.bundle
- /root/VMware-ovftool-4.4.1-16812187-lin.x86_64.bundle --eulas-agreed --required --console
# F) packer
- curl -fsSL https://apt.releases.hashicorp.com/gpg | apt-key add -
- apt-get update
- apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
- apt-get update
- apt-get install packer
# X) Additional modules
- pwsh -C '& {Install-Module -Name ImportExcel -Scope AllUsers -Force -Confirm:$false -AllowClobber}'
- pwsh -C '& {Install-Module -Name Posh-Ssh -Scope AllUsers -Force -Confirm:$false -AllowClobber}'
# Y) Clone SRS repo
- mkdir /root/SRS
- git clone https://github.com/vmware/script-runtime-service-for-vsphere /root/SRS
# End SRS 1.0

Verify the Builder

Once the Builder station is deployed, it is useful to check that everything the creation of the SRS Appliance needs, is present.

A word of warning, the following code includes all versions as hard-coded. This is not ideal, and especially for products that have a daily build, this will require updating the code each time. Since this was, at the time of writing this post, not my top priority, I postponed looking for a more portable solution to a later date.

#region Helper functions
function Get-LogText {
  param([string]$Text)

  $dt = (Get-Date).ToString('yyyyMMdd HH:mm:ss.fff')
  $app = Split-Path -Path $MyInvocation.ScriptName -Leaf
  "$dt - $app - $Text"
}
#endregion

#region Preamble

$vmName = 'SrsBuilder'

$now = Get-Date

$viCredObj = Get-VICredentialStoreItem -Host $vmName
$sObj = @{
  TypeName = 'PSCredential'
  ArgumentList = $viCredObj.User,(ConvertTo-SecureString -String $viCredObj.Password -AsPlainText -Force)
}
$cred = New-Object @sObj

$vm = Get-VM -Name $vmName
#endregion

#region dotnet SDK
$check_dotnet = @'
dotnet --version
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_dotnet
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if($result.ScriptOutput.Trim("n") -ne '3.1.404'){
  Write-Host (Get-LoGText -Text "Dotnet SDK installation failure") -ForegroundColor red
}
else{
  Write-Host (Get-LogText -Text "Dotnet SDK $($result.ScriptOutput.Trim("n")) installation OK") -ForegroundColor green
}
#endregion

#region Docker
$check_docker = @'
docker --version
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_docker
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if ($result.ScriptOutput.Trim("n") -ne 'Docker version 20.10.1, build 831ebea') {
  Write-Host (Get-LoGText -Text "Docker installation failure") -ForegroundColor red
} else {
  Write-Host (Get-LogText -Text "$($result.ScriptOutput.Trim("n")) installation OK") -ForegroundColor green
}
#endregion

#region PowerShell v7
$check_PSv7 = @'
pwsh -C '& {$PSVersionTable.PSVersion.ToString()}'
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_PSv7
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if ($result.ScriptOutput.Trim("n") -ne '7.1.0') {
  Write-Host (Get-LoGText -Text "PowerShell installation failure") -ForegroundColor red
} else {
  Write-Host (Get-LogText -Text "PowerShell $($result.ScriptOutput.Trim("n")) installation OK") -ForegroundColor green
}
#endregion

#region VMware PowerCLI
$check_PowerCLI = @'
pwsh -C '& {(Get-Module -Name VMware.PowerCLI -ListAvailable).Version.ToString()}'
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_PowerCLI
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if ($result.ScriptOutput.Trim("n") -ne '12.1.0.17009493') {
  Write-Host (Get-LoGText -Text "VMware PowerCLI installation failure") -ForegroundColor red
} else {
  Write-Host (Get-LogText -Text "VMware PowerCLI $($result.ScriptOutput.Trim("n")) installation OK") -ForegroundColor green
}
#endregion

#region VMware ovftool
$check_ovftool = @'
ovftool --version
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_ovftool
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if ($result.ScriptOutput.Trim("n") -ne 'VMware ovftool 4.4.1 (build-16812187)') {
  Write-Host (Get-LoGText -Text "VMware OVFTool installation failure") -ForegroundColor red
} else {
  Write-Host (Get-LogText -Text "$($result.ScriptOutput.Trim("n")) installation OK") -ForegroundColor green
}
#endregion

#region Packer
$check_packer = @'
packer --version
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_packer
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if ($result.ScriptOutput.Trim("n") -ne '1.6.6') {
  Write-Host (Get-LoGText -Text "Packer installation failure") -ForegroundColor red
} else {
  Write-Host (Get-LogText -Text "Packer $($result.ScriptOutput.Trim("n")) installation OK") -ForegroundColor green
}
#endregion

#region Additional

#region ImportExcel
$check_ImportExcel = @'
pwsh -C '& {(Get-Module -Name ImportExcel -ListAvailable).Version.ToString()}'
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_ImportExcel
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if ($result.ScriptOutput.Trim("n") -ne '7.1.1') {
  Write-Host (Get-LoGText -Text "ImportExcel installation failure") -ForegroundColor red
} else {
  Write-Host (Get-LogText -Text "ImportExcel $($result.ScriptOutput.Trim("n")) installation OK") -ForegroundColor green
}
#endregion

#region Posh-Ssh
$check_PoshSSH = @'
pwsh -C '& {(Get-Module -Name Posh-SSH -ListAvailable).Version.ToString()}'
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_PoshSSH
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if ($result.ScriptOutput.Trim("n") -ne '2.3.0') {
  Write-Host (Get-LoGText -Text "Posh-SSH installation failure") -ForegroundColor red
} else {
  Write-Host (Get-LogText -Text "Posh-SSH $($result.ScriptOutput.Trim("`n")) installation OK") -ForegroundColor green
}
#endregion

#endregion

Run the Build

When the Builder station is deployed, and we verified that all prerequisites are in place, we can start the deployment of the SRS Applicance. From which, if all goes well, the OVA file will be generated.

During my experimenting with building the SRS Appliance, I stumbled on two what I suspect are issues.

As a bypass I build the SRS Appliance for now with Photon V3 Rev 2.

I did not succeed in building the SRS Appliance with Photon V3 Rev 3 image. I opened an Issue for that, and I’m curious to know if the issue is a Photon issue or something in my environment.

Another issue, which seems to be a real, known issue, is that you apparently can not use a Distributed Switch Portgroup to build the SRS Appliance. The underlying issue seems to stem from the Packer application, more specifically the VMware-Iso builder. This builder uses VNC to ‘talk’ with the ESXi node on which the SRS Appliance is created.

There are two issues here:

• VNC is not included anymore in ESXi since 6.7
• With the ESXi API you can not interact with a VDS

The VNC issue is resolved by changing some Packer settings. But for the VDS issue, there is no solution besides switching to the vSphere-Iso builder in the Packer step, so I had to build the SRS Appliance on a VSS Portgroup.

I packaged the preparation steps and the start of the SRS Appliance build in a function, named Invoke-SRSBuild.

function Invoke-SRSBuild {
  [CmdletBinding()]
  param(
    [Parameter(Mandatory)]
    [String]$BuilderName,
    [Parameter(Mandatory)]
    [PSCredential]$Credential,
    [Switch]$PrepOnly,
    [Switch]$BuildOnly,
    [Parameter(Mandatory)]
    [ValidateSet('V3Rev2', 'V3Rev3')]
    [String]$PhotonVersion,
    [Parameter(Mandatory)]
    [String]$Portgroup,
    [Switch]$PackerLog
  )

  #region Helper functions
  function Get-LogText {
    param([string]$Text)

    $dt = (Get-Date).ToString('yyyyMMdd HH:mm:ss.fff')
    $app = Split-Path -Path $MyInvocation.ScriptName -Leaf
    "$dt - $app - $Text"
  }

  function Remove-SRSFolder {
    Param(
      [Parameter(Mandatory = $true)]
      [VMware.VimAutomation.ViCore.Types.V1.DatastoreManagement.StorageResource]$Datastore
    )

    $dsBrowser = Get-View -Id $Datastore.ExtensionData.Browser
    $spec = New-Object -TypeName VMware.Vim.HostDatastoreBrowserSearchSpec
    $spec.MatchPattern = 'SRS_Appliance'
    $spec.Query = New-Object -TypeName Vmware.Vim.FolderFileQuery
    $spec.Details = New-Object -TypeName VMware.Vim.FileQueryFlags
    $result = $dsBrowser.SearchDatastore("[$($Datastore.Name)]", $spec)

    if ($result.File.Count -gt 0 -and $result.File[0]) {
      $dc = Get-VMHost -Datastore $Datastore | Get-Datacenter
      $fileMgr = Get-View FileManager
      $fileMgr.DeleteDatastoreFile("[$($Datastore.Name)] $($result.File[0].Path)", $dc.ExtensionData.MoRef)
    }
  }
  #endregion

  #region Bypass for known issue in VSC ()
  # See https://github.com/PowerShell/vscode-powershell/issues/633

  if ($psISE) {
    $dir = Split-Path -Path $psISE.CurrentFile.FullPath
  } else {
    if ($profile -match "VScode") {
      $dir = Split-Path $psEditor.GetEditorContext().CurrentFile.Path
    } else {
      $dir = $PSScriptRoot
    }
  }
  #endregion

  #region Preamble

  $vm = Get-VM -Name $BuilderName
  #endregion

  if (-not $BuildOnly.IsPresent) {

    #region SSH service
    $ssh = Get-VMHostService -VMHost $vm.VMHost | Where-Object { $_.Label -eq 'SSH' }
    if ($ssh.Running) {
      Write-Verbose (Get-LogText -Text "SSH is running on $($vm.VMHost.Name)")
    } else {
      Write-Verbose (Get-LogText -Text "Starting SSH on $($vm.VMHost.Name)")
      $ssh = Start-VMHostService -HostService $ssh -Confirm:$false
      if (-not $ssh.Running) {
        Write-Verbose (Get-LogText -Text "Could not start SSH on $($vm.VMHost.Name)")
        throw "SSH service not started on $($vm.VMHost.Name)"
      }
    }
    #endregion

    #region GuestIPHack
    $esxcli = Get-EsxCli -VMHost $vm.VMHost -V2
    $sOpt = @{
      option = '/Net/GuestIPHack'
    }
    $opt = $esxcli.system.settings.advanced.list.Invoke($sOpt)
    if ($opt.IntValue -eq 1) {
      Write-Verbose (Get-LogText -Text "GuestIPHack setting is correct on $($vm.VMHost.Name)")
    } else {
      Write-Verbose (Get-LogText -Text "GuestIPHack setting is not correct on $($vm.VMHost.Name)")
      $sOpt.Add('intvalue', [long]1)
      if ($esxcli.system.settings.advanced.set.Invoke($sOpt)) {
        Write-Verbose (Get-LogText -Text "GuestIPHack setting corrected on $($vm.VMHost.Name)")
      } else {
        Write-Verbose (Get-LogText -Text "Could not change GuestIPHack setting on $($vm.VMHost.Name)")
        throw "Could net set GuestIPHack"
      }
    }
    #endregion

    #region Make sure folder doesn't exist
    $ds = Get-Datastore -RelatedObject $vm
    Remove-SRSFolder($ds)
    #endregion

    #region Update photon-build.json
    # Suspected issue when using a vdPortgroup

    $jsonFile = New-TemporaryFile
    $photonBuilder = @{
      builder_host = $vm.VMHost.Name
      builder_host_username = $viCredObj.User
      builder_host_password = $viCredObj.Password
      builder_host_datastore = (Get-Datastore -RelatedObject $vm).Name
      builder_host_portgroup = $Portgroup
    }
    $photonBuilder | ConvertTo-Json | Set-Content -Path $jsonFile
    $sCopy = @{
      VM = $vm
      GuestCredential = $Credential
      LocalToGuest = $true
      Source = $jsonFile
      Destination = '/root/SRS/appliance/photon-builder.json'
      Force = $true
      Confirm = $false
    }
    Copy-VMGuestFile @sCopy
    Remove-Item -Path $jsonFile -Confirm:$false
    #endregion

    #region Update photon-version.json
    $photonTab = Get-Content -Path "$dir\Photon-version.json" | ConvertFrom-Json
    $photonSelected = $photonTab.Photon | Where-Object { $_.Label -eq $PhotonVersion }
    $jsonFile = New-TemporaryFile
    $sCopy = @{
      VM = $vm
      GuestCredential = $Credential
      GuestToLocal = $true
      Source = '/root/SRS/appliance/photon-version.json'
      Destination = $jsonFile
      Force = $true
      Confirm = $false
    }
    Copy-VMGuestFile @sCopy
    $photonBuilder = (Get-Content -Path $jsonFile | ConvertFrom-Json)[0]
    if ($photonBuilder.iso_url -ne $photonSelected.Uri -or $photonBuilder.iso_checksum -ne $photonSelected.CheckSum) {
      $photonBuilder.iso_url = $photonSelected.Uri
      $photonBuilder.iso_checksum = $photonSelected.CheckSum
      @($photonBuilder) | ConvertTo-Json | Set-Content -Path $jsonFile
      $sCopy = @{
        VM = $vm
        GuestCredential = $Credential
        LocalToGuest = $true
        Source = $jsonFile
        Destination = '/root/SRS/appliance/photon-version.json'
        Force = $true
        Confirm = $false
      }
      Copy-VMGuestFile @sCopy
      Remove-Item -Path $jsonFile -Confirm:$false

    }
    #endregion

    #region Handle VNC for ESXi 6.7 and later
    if ([Version]$vm.VMHost.Version -ge [Version]'6.7.0') {
      $updateJSON = 'sed' +
      ' -i.bak -e ''/vnc_disable_password/ i \      "vnc_over_websocket": true,''' +
      ' -e ''/vnc_disable_password/ i \      "insecure_connection": true,''' +
      ' -e ''/vnc_disable_password/d'' ~/SRS/appliance/photon.json'

      $sInvoke = @{
        VM = $vm
        GuestCredential = $Credential
        ScriptText = $updateJSON
        ScriptType = 'bash'
      }
      $result = Invoke-VMScript @sInvoke
    }
    #endregion

  }

  if (-not $PrepOnly.IsPresent) {

    #region Get PowerCLI folder path
    $findDir = 'pwsh -C ''& {Split-Path -Path (Split-Path -Path (Get-Module -Name VMware.PowerCLI -ListAvailable).ModuleBase)}'''
    $sInvoke = @{
      VM = $vm
      GuestCredential = $Credential
      ScriptText = $findDir
      ScriptType = 'bash'
    }
    $pcliPath = (Invoke-VMScript @sInvoke).ScriptOutput
    #endregion

    #region Start build
    $sInvoke = @{
      VM = $vm
      GuestCredential = $Credential
      ScriptText = "/root/SRS/build.sh $pcliPath"
      ScriptType = 'bash'
    }
    if ($PackerLog.IsPresent) {
      $packerLogName = 'packer.log'
      $sInvoke.ScriptText = "export PACKER_LOG=1;export PACKER_LOG_PATH='$($packerLogName)';$($sInvoke.ScriptText)"
    }
    $buildResult = Invoke-VMScript @sInvoke
    $buildResult.ScriptOutput > "$dir\Buildlog.txt"
    #endregion

    #region Rettrieve Packer log
    if($PackerLog.IsPresent){
      $sCopy = @{
        VM = $vm
        GuestCredential = $Credential
        GuestToLocal = $true
        Source = "/root/SRS/appliance/$($packerLogName)"
        Destination = "$dir\$($packerLogName)"
        Force = $true
        Confirm = $false
      }
      Copy-VMGuestFile @sCopy
    }
    #endregion

    #region Clean up
    # Packer leaves an orphaned entry in the VCSA behind

    $endpointVM = Get-VM -Name 'SRS_Appliance' -ErrorAction SilentlyContinue
    if($endpointVM){
      while($endpointVM.PowerState -ne 'PoweredOff'){
        sleep2
        $endpointVM = Get-VM -Name 'SRS_Appliance' -ErrorAction SilentlyContinue
      }
      Remove-VM -VM $endpointVM   -DeletePermanently -ErrorAction SilentlyContinue -Confirm:$false
    }
    #endregion
  }
}

The call to the Invoke-SRSBuilder function is again straight-forward.

$vmName = 'SRSBuilder'

#region Credential
# Get the credentials for the SRS Builder station
# Alternative method required when using PSv6 or later

$viCredObj = Get-VICredentialStoreItem -Host $vmName
$sObj = @{
  TypeName = 'PSCredential'
  ArgumentList = $viCredObj.User, (ConvertTo-SecureString -String $viCredObj.Password -AsPlainText -Force)
}
$cred = New-Object @sObj
#endregion

$sBuild = @{
  BuilderName = $vmName
  Credential = $cred
  PhotonVersion = 'V3Rev2'
  Portgroup = 'PG1'
  PackerLog = $true
}
Invoke-SRSBuild @sBuild

The OVA

When the call to the build.sh script completes successfully, there will be OVA/OVF files created on the Builder station.

Since the Builder station is, in my setup, a temporary station, I need to download those files to a more permanent location. The following snippet uses the Get-ScpFile cmdlet from the Posh-Ssh module to do that.

#requires -Modules Posh-Ssh

#region Preamble

$vmName = 'SrsBuilder'

$now = Get-Date

$viCredObj = Get-VICredentialStoreItem -Host $vmName
$sObj = @{
  TypeName = 'PSCredential'
  ArgumentList = $viCredObj.User, (ConvertTo-SecureString -String $viCredObj.Password -AsPlainText -Force)
}
$cred = New-Object @sObj

$vm = Get-VM -Name $vmName

$fqdn = (Resolve-DnsName -Name $vm.ExtensionData.Guest.IpAddress).NameHost
#endregion

#region Download OVA
$sSCP = @{
  ComputerName = $fqdn
  Credential = $cred
  RemoteFile = '/root/SRS/appliance/output-vmware-iso/SRS_Appliance_1.0.0.ova'
  LocalFile = 'D:\OVA\SRS\V1.0.0\Rev2-VSS\SRS_Appliance_1.0.0.ova'
  AcceptKey = $true
}
Get-SCPFile @sSCP
#endregion

Deploy SRS

Once we have the OVA available, it is a piece of cake to deploy our SRS VM with the Get-OvfConfiguration and Import-VApp cmdlets.

Note that deploying the SRS OVA to a Distributed Switch Portgroup is perfectly possible. The VDS issue mentioned earlier only comes into play when Packer with the VMware-Iso builder is involved. 

$vmName = 'srs1'
$clusterName = 'cluster'
$numCPU = 2
$memoryGB = 4
$dsName = 'vsanDatastore'
$harddiskGB = 2
$ovaPath = 'D:\OVA\SRS\V1.0.0\Rev2-VSS\SRS_Appliance_1.0.0.ova'
$networkName = 'vdPg1'

$cluster = Get-Cluster -Name $clusterName
$esx = Get-VMHost -Location $cluster | Get-Random
$ds = Get-Datastore -Name $dsName
$ovfProp = Get-OvfConfiguration -Ovf $ovaPath
$ovfProp.Common.guestinfo.hostname.Value = $vmName
$ovfProp.Common.guestinfo.ipaddress.Value = '192.168.10.43'
$ovfProp.Common.guestinfo.netmask.Value = '24'
$ovfProp.Common.guestinfo.gateway.Value = '192.168.10.1'
$ovfProp.Common.guestinfo.root_password.Value = 'Welcome2020!'
$ovfProp.Common.guestinfo.dns.Value = '192.168.10.2'
$ovfProp.Common.guestinfo.domain.Value = 'local.lab'
$ovfProp.Common.srs.vcaddress.Value = 'vcsa7.local.lab'
$ovfProp.Common.srs.vcpassword.Value = 'Welcome2020!'
$ovfProp.Common.srs.vcuser.Value = 'administrator@vsphere.local'
$ovfProp.Common.srs.vcthumbprint.Value = '5bf3246fde90fd3b7ab84144f50105114c2826e1'
$ovfProp.NetworkMapping.PG1.Value = $networkName

$vm = Import-VApp -Name $vmName -Source $ovaPath -VMHost $esx -OvfConfiguration $ovfProp -Datastore $ds
Start-VM -VM $vm -Confirm:$false

Your first test to see if the deployment went ok, is to try and access the swagger page on your SRS VM. The URI is https://<your-SRS-FQDN>/swagger. When all goes well, you will see a page like this.

Using SRS

Now we can start using the SRS server to run our scripts. The way to do that is quite simple. The complete process is described in the Run Scripts section in the SRS repository.

A Simple Sample

My test installation of SRS is done on a VM that is named SRS1.

A basic REST API call to verify that everything is working, is to call the api/about method.

function Invoke-SrsMethod {
  [cmdletbinding()]
  param(
    [Parameter(Mandatory)]
    [String]$FQDN,
    [Parameter(Mandatory)]
    [String]$API,
    [Parameter(Mandatory)]
    [String]$Method
  )

  $sWeb = @{
    Uri = "https://$($FQDN)/$($API)"
    Method = $Method
  }

  try {
    $result = Invoke-WebRequest @sWeb
  }
  catch [System.Net.WebException] {
    switch ($error[0].Exception.Status) {
      ([System.Net.WebExceptionStatus]::TrustFailure) {
        if ($PSVersionTable.PSVersion.Major -lt 6) {
          if (-not ([System.Management.Automation.PSTypeName]"TrustAllCertsPolicy").Type) {
            Add-Type -TypeDefinition @"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
    public bool CheckValidationResult(
        ServicePoint srvPoint, X509Certificate certificate,
        WebRequest request, int certificateProblem)
    {
        return true;
    }
}
"@
          }
          if ([System.Net.ServicePointManager]::CertificatePolicy.ToString() -ne "TrustAllCertsPolicy") {
            [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
          }
        } else {
          $sWeb.Add('SkipCertificateCheck', $true)
        }
        $result = Invoke-WebRequest @sWeb
      }
      Default {
        Write-Error "Unhandled WebException $($error[0].Exception.Status)"
      }
    }
  }
  catch {
    Write-Error "Unhandled exception code $($error[0].Exception.gettype().Name)"
  }

  switch ($result.StatusCode) {
    200 {
      $result
    }
    Default {
      Write-Error "Unhandled StatusCode $($result.StatusCode)"
    }
  }
}

$vmName = 'srs1'

$vm = Get-VM -Name $vmName
$fqdn = (Resolve-DnsName -Name $vm.ExtensionData.Guest.IpAddress).NameHost

$about = Invoke-SrsMethod -FQDN $fqdn -API 'api/about' -Method 'Get'
$about.Content | ConvertFrom-Json

You notice that I created a function Invoke-SRSMethod instead of just calling the Invoke-WebRequest cmdlet directly. The reason for creating that function is that I needed to be able to bypass invalid certificates.

With PSv6 that has become easy, since the Invoke-WebRequest cmdlet now has the SkipCertificateCheck switch. 

But I wanted to have a function that would also work in a PSV5.1 environment, hence the function and the logic in there.

When all goes well, that snippet should return the following.

A PowerCLI example

The full sequence for running code on the SRS station is perfectly explained in the Run Scripts section on the SRS repository.

A full, scripted example, excluding the earlier Invoke-SRSMethod function, could look like this.

#region Preamble
$vmName = 'srs1'

$vm = Get-VM -Name $vmName
$fqdn = (Resolve-DnsName -Name $vm.ExtensionData.Guest.IpAddress).NameHost

# Get credential for logging on to the guest OS
$viCred = Get-VICredentialStoreItem -Host $global:DefaultVIServer.Name
$secPassword = ConvertTo-SecureString -String $viCred.Password -AsPlainText -Force
$cred = [Management.Automation.PSCredential]::new($viCred.User, $secPassword)

$headers = @{
  "accept" = "application/json"
  "content-type" = "application/json"
}
#endregion

#region Login
$sLogon = @{
  FQDN = $fqdn
  API = '/api/auth/login'
  Method = 'Post'
  Credential = $cred
  Headers = $headers
}
$logon = Invoke-SrsMethod @sLogon
$headers.Add('X-SRS-API-KEY', $logon.Headers['X-SRS-API-KEY'])
#endregion

#region Create Runspace
$sRSCreate = @{
  FQDN = $fqdn
  API = '/api/runspaces'
  Method = 'Post'
  Credential = $cred
  Headers = $headers
  Body = @{
    name = 'MyRS'
    run_vc_connection_script = $true
  }
}
$createRS = Invoke-SrsMethod @sRSCreate
$rs = $createRS.Content | ConvertFrom-Json
#endregion

#region Wait till RS is ready
while ($rs.state -eq 'Creating') {
  $sRSGet = @{
    FQDN = $fqdn
    API = "/api/runspaces/$($rs.Id)"
    Method = 'Get'
    Headers = $headers
  }
  $getRS = Invoke-SrsMethod @sRSGet
  $rs = $getRS.Content | ConvertFrom-Json
}
#endregion

#region Run Script
$code = {
  Get-VM
}
$sSCRCreate = @{
  FQDN = $fqdn
  API = '/api/script-executions'
  Method = 'Post'
  Headers = $headers
  Body = @{
    runspace_id = $rs.id
    name = 'MyScript'
    script = $code.ToString()
    script_parameters = @()
  }
}
$createSCR = Invoke-SrsMethod @sSCRCreate
$scr = $createSCR.Content | ConvertFrom-Json
#endregion

#region Wait for Script to end
while($scr.state -eq 'running'){
$sSCRCreate = @{
  FQDN = $fqdn
  API = "/api/script-executions/$($scr.id)"
  Method = 'Get'
  Headers = $headers
}
$getSCR = Invoke-SrsMethod @sSCRCreate
$scr = $getSCR.Content | ConvertFrom-Json
}
#endregion

#region Retrieve Script output
$sSCROut = @{
  FQDN = $fqdn
  API = "/api/script-executions/$($scr.id)/output"
  Method = 'Get'
  Headers = $headers
}
$outSCR = Invoke-SrsMethod @sSCROut
$outSCR.Content | ConvertFrom-Json
#endregion

#region Retrieve Script streams
'information', 'error', 'warning', 'debug', 'verbose' |
ForEach-Object -Process {
  $sSCRStr = @{
    FQDN = $fqdn
    API = "/api/script-executions/$($scr.id)/streams/$($_)"
    Method = 'Get'
    Headers = $headers
  }
  $getStream = Invoke-SrsMethod @sScrStr
  if($getStream.Content -ne '[]'){
    Write-Host "--- $_ ---" -ForegroundColor Green
    $getStream.Content | ConvertFrom-Json | Out-Default
    Write-Host "------------" -ForegroundColor Green
  }
}
#endregion

#region Remove Runspace
$sRSDel = @{
  FQDN = $fqdn
  API = "/api/runspaces/$($rs.id)"
  Method = 'Delete'
  Headers = $headers
}
$rsDel = Invoke-SrsMethod @sRSDel
$rsDel.Content | ConvertFrom-Json
#endregion

#region Logout
$sLogout = @{
  FQDN = $fqdn
  API = "/api/auth/logout"
  Method = 'Post'
  Headers = $headers
}
$srsLogout = Invoke-SrsMethod @sLogout
#endregion

A Function

That turned out to be a whole lot of code to just run a simple Get-VM.

But since we will be using most of the time the same logic, we can easily turn that into a function. That will make submitting code to run on the SRS a lot simpler.

And while we are at it, let’s include an option to run the code from an existing .ps1 file.

Function Invoke-SRSScript {
  [cmdletbinding()]
  param(
    [Parameter(Mandatory = $true)]
    [String]$SRSHost,
    [Parameter(ParameterSetName = 'ScriptBlock', Mandatory = $true)]
    [scriptblock]$Code,
    [Parameter(ParameterSetName = 'ScriptFile', Mandatory = $true)]
    [String]$Path,
    [Parameter(Mandatory = $true)]
    [PSCredential]$VCCredential
  )

  #region Helper functions
  function Invoke-SrsMethod {
    [cmdletbinding()]
    param(
      [Parameter(Mandatory)]
      [String]$FQDN,
      [Parameter(Mandatory)]
      [String]$API,
      [Parameter(Mandatory)]
      [String]$Method,
      [PSCredential]$Credential,
      [PSObject]$Headers,
      [PSObject]$Body
    )

    $sWeb = @{
      Uri = "https://$($FQDN)$($API)"
      Method = $Method
    }
    if ($Credential) {
      $sWeb.Add('Credential', $Credential)
    }
    if ($Headers) {
      $sWeb.Add('Headers', $Headers)
    }
    if ($Body) {
      $sWeb.Add('Body', ($Body | ConvertTo-Json))
    }

    try {
      $result = Invoke-WebRequest @sWeb
    } catch [System.Net.WebException] {
      switch ($error[0].Exception.Status) {
        ([System.Net.WebExceptionStatus]::TrustFailure) {
          if ($PSVersionTable.PSVersion.Major -lt 7) {
            if (-not ([System.Management.Automation.PSTypeName]"TrustAllCertsPolicy").Type) {
              Add-Type -TypeDefinition @"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
    public bool CheckValidationResult(
        ServicePoint srvPoint, X509Certificate certificate,
        WebRequest request, int certificateProblem)
    {
        return true;
    }
}
"@
            }
            if ([System.Net.ServicePointManager]::CertificatePolicy.ToString() -ne "TrustAllCertsPolicy") {
              [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
            }
          } else {
            $sWeb.Add('SkipCertificateCheck', $true)
          }
          $result = Invoke-WebRequest @sWeb
        }
        Default {
          Write-Error "Unhandled WebException $($error[0].Exception.Status)"
        }
      }
    } catch {
      Write-Error "Unhandled exception code $($error[0].Exception.gettype().Name)"
      $error[0].Exception | Format-Custom
    }

    switch ($result.StatusCode) {
      200 {
        $result
      }
      202 {
        $result
      }
      401 {
        Write-Error "Unauthorized $($result.StatusCode)"
      }
      500 {
        Write-Error "Server error $($result.StatusCode)"
      }
      Default {
        Write-Error "Unhandled StatusCode $($result.StatusCode)"
      }
    }
  }
  #endregion

  #region Preamble

  $vm = Get-VM -Name $SRSHost
  $fqdn = (Resolve-DnsName -Name $vm.ExtensionData.Guest.IpAddress).NameHost

  switch ($PSCmdlet.ParameterSetName){
    'ScriptBlock' {
      $strCode = $Code.ToString()
    }
    'ScriptFile' {
      $strCode = Get-Content -Path $Path | Out-String
    }
  }

  $headers = @{
    "accept" = "application/json"
    "content-type" = "application/json"
  }
  #endregion

  #region Login
  $sLogon = @{
    FQDN = $fqdn
    API = '/api/auth/login'
    Method = 'Post'
    Credential = $VCCredential
    Headers = $headers
  }
  $logon = Invoke-SrsMethod @sLogon
  $headers.Add('X-SRS-API-KEY', $logon.Headers['X-SRS-API-KEY'])
  #endregion

  #region Create Runspace
  $sRSCreate = @{
    FQDN = $fqdn
    API = '/api/runspaces'
    Method = 'Post'
    Credential = $cred
    Headers = $headers
    Body = @{
      name = 'MyRS'
      run_vc_connection_script = $true
    }
  }
  $createRS = Invoke-SrsMethod @sRSCreate
  $rs = $createRS.Content | ConvertFrom-Json
  #endregion

  #region Wait till RS is ready
  while ($rs.state -eq 'Creating') {
    $sRSGet = @{
      FQDN = $fqdn
      API = "/api/runspaces/$($rs.Id)"
      Method = 'Get'
      Headers = $headers
    }
    $getRS = Invoke-SrsMethod @sRSGet
    $rs = $getRS.Content | ConvertFrom-Json
  }
  #endregion

  #region Run Script
  $sSCRCreate = @{
    FQDN = $fqdn
    API = '/api/script-executions'
    Method = 'Post'
    Headers = $headers
    Body = @{
      runspace_id = $rs.id
      name = 'MyScript'
      script = $strCode
      script_parameters = @()
    }
  }
  $createSCR = Invoke-SrsMethod @sSCRCreate
  $scr = $createSCR.Content | ConvertFrom-Json
  #endregion

  #region Wait for Script to end
  while ($scr.state -eq 'running') {
    $sSCRCreate = @{
      FQDN = $fqdn
      API = "/api/script-executions/$($scr.id)"
      Method = 'Get'
      Headers = $headers
    }
    $getSCR = Invoke-SrsMethod @sSCRCreate
    $scr = $getSCR.Content | ConvertFrom-Json
  }
  #endregion

  #region Retrieve Script output
  $sSCROut = @{
    FQDN = $fqdn
    API = "/api/script-executions/$($scr.id)/output"
    Method = 'Get'
    Headers = $headers
  }
  $outSCR = Invoke-SrsMethod @sSCROut
  $outSCR.Content | ConvertFrom-Json
  #endregion

  #region Retrieve Script streams
  $streams = 'information','error','warning','debug','verbose'

  $streams | ForEach-Object -Process {
    $sSCRStr = @{
      FQDN = $fqdn
      API = "/api/script-executions/$($scr.id)/streams/$($_)"
      Method = 'Get'
      Headers = $headers
    }
    $getStream = Invoke-SrsMethod @sScrStr
    if ($getStream.Content -ne '[]') {
      $out = ($getStream.Content | ConvertFrom-Json).message | Out-String
      switch($_){
        'information' {
          Write-Information -MessageData $out
        }
        'error' {
          Write-Error -Message $out
        }
        'warning' {
          Write-Warning -Message $out
        }
        'debug' {
          Write-Debug -Message $out
        }
        'verbose' {
          Write-Verbose -Message $out
        }
      }
    }
  }
  #endregion

  #region Remove Runspace
  $sRSDel = @{
    FQDN = $fqdn
    API = "/api/runspaces/$($rs.id)"
    Method = 'Delete'
    Headers = $headers
  }
  $rsDel = Invoke-SrsMethod @sRSDel
  $rsDel.Content | ConvertFrom-Json
  #endregion

  #region Logout
  $sLogout = @{
    FQDN = $fqdn
    API = "/api/auth/logout"
    Method = 'Post'
    Headers = $headers
  }
  $srsLogout = Invoke-SrsMethod @sLogout
  #endregion

}

This Invoke-SRSScript function can be used in two variations (parametersets). With the Code parameter, you pass a ScriptBlock.

$sScript = @{
  SRSHost = 'srs1'
  Code = { Get-VM }
  VCCredential = $cred
}
Invoke-SRSScript @sScript

With the Path parameter, you point to a .ps1 file.

$sScript = @{
  SRSHost = 'srs1'
  Path = 'D:\Git\SRS-Explore\Use\Sample.ps1'
  VCCredential = $cred
}

Some Administration

The SRS comes with a number of preset values, see the Initial Configuration documentation. In some situations, you might want to change one or more of these settings.

The obvious solution is to rebuild your SRS OVA with the desired settings, and re-deploy your SRS VM. 

But you can also change these settings on the fly on an existing and running SRS. The following snippet is just an example where I change the script-runtime-service setting from the default 10 minutes to 20 minutes.

$vmName = 'srs1'

# Get the SRS credentials

$viCred = Get-VICredentialStoreItem -Host $vmName
$secPassword = ConvertTo-SecureString -String $viCred.Password -AsPlainText -Force
$cred = [Management.Automation.PSCredential]::new($viCred.User, $secPassword)

$vm = Get-VM -Name $vmName

#region Change SRS setting

$code = @'
kubectl get cm service-settings -n script-runtime-service -o yaml | sed -e 's/\("MaxRunspaceIdleTimeMinutes": \).*/\120,/' | kubectl apply -f -
'@

$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptType = 'bash'
  ScriptText = $code
}
Invoke-VMScript @sInvoke
#endregion

#region Check new SRS settings

$code = @'
kubectl get cm service-settings -n script-runtime-service -o yaml
'@

$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptType = 'bash'
  ScriptText = $code
}
Invoke-VMScript @sInvoke

#endregion

Note that such a change will not be applied immediately, it might take some time (we are talking minutes).

Epilogue

This concludes, for now, my somewhat lengthy Hitchhikers Guide to SRS 1.0.0, which resulted from my playing/testing/exploring the Script Runtime Service for vSphere (SRS) 1.0.0.

Is this something to should have gotten more attention when it was released?

Definitely!

I will surely be doing some more experimenting with the new and shining tool.

Enjoy!

Like often, and one of the VMware PowerCLI features I absolutely like, when a cmdlet is missing a feature, you can fall back on the API to solve the issue.

Introduction

Like so often, the reason I had a closer look at this issue was due to a question in PowerCLI Community on VMTN.

My first step in such cases is always to have a look at the vSphere Web Services API.

But unfortunately, the ReconfigVM_Task method documentation didn’t make me any wiser. There is apparently no object, property, or method to mount an ISO from a Content Library onto a VM.

In the next step I had a look at the vSphere REST API Reference. But in there I could also not find anything specific about Content Library ISOs and VM CD drives.

When all else fails, look at the code!

And with Code Capture we have just that option!

The first thing I noticed while looking at the generated code, there are no ‘special’ object, properties, or method used. Just a plain ReconfigVM_Task call with an edit action on the CD drive device.

The only special feature was the value in the FileName property.

That value definitely pointed to the ISO file in the Content Library. At first I tried to compose that value by hand/script. But it turned out that some parts of the value were not obtainable via normal ways.

I found out then when I use the vimdatastore provider, the returned FullDatastorePath contained exactly the value I neededd.

The Code

function Set-CDDriveCLIso {

    <#
    .SYNOPSIS
      Mount an ISO from a Content Library
      .DESCRIPTION
      This function will mount an ISO located on a Content Library
      on a CDDrive of a VM
    .NOTES
      Author:  Luc Dekens
      Version:
      1.0 24/12/20  Initial release
    .PARAMETER VM
      Specifies the virtual machines on whose guest operating systems
      you want to run the script.
    .PARAMETER CDDrive
      Specifies the CDDrive on which the ISO will be mounted
    .PARAMETER ContentLibrary
      Specifies the Content Library on which the ISO is located.
    .PARAMETER ContentLibraryIso
      Specifies the ISO item on the Content Library
    .EXAMPLE
      $cl = Get-ContentLibrary -Name MyCL
      $iso = Get-ContentLibraryItem -ContentLibrary $cl -Name MyISO
      Get-VM -Name 'MyVM' | Get-CDDrive -Name 'CD/DVD drive 0' |
      Set-CDDriveCLIso -ContentLibraryIso $iso -Confirm:$false
    .EXAMPLE
      $cd = Get-VM -Name MyVM | Get-CDDrive
      Set-CDDriveCLIso -CDDrive $cd -ContentLibraryIso $iso -Confirm:$false
    #>
    
    [cmdletbinding(SupportsShouldProcess = $true, ConfirmImpact = 'High')]
    param(
        [parameter(Mandatory = $true, ValueFromPipeline = $true)]
        [VMware.VimAutomation.ViCore.Types.V1.VirtualDevice.CDDrive]$CDDrive,
        [parameter(Mandatory = $true)]
        [VMware.VimAutomation.ViCore.Types.V1.ContentLibrary.ContentLibraryItem]$ContentLibraryISO
    )
    
    $target = "VM:$($CDDrive.Parent.Name) CD:$($CDDrive.Name)"
    $action = "Mount ISO $($ContentLibraryISO.Name) from $($ContentLibraryISO.ContentLibrary.Name)"
        
    if ($PSCmdlet.ShouldProcess($target, $action)) {
        $driveName = -join ((65..90) | 
                Get-Random -Count 3 | 
                ForEach-Object -Process { [char]$_ })
        $filter = "$($ContentLibraryISO.Name)*.iso" 
        
        $ds = Get-Datastore -Name $ContentLibraryISO.ContentLibrary.Datastore
        
        New-PSDrive -Name $driveName -PSProvider VimDatastore -Root '\' -Location $ds | Out-Null
        $clPath = Get-ChildItem -Path "$($driveName):" -Filter "$($ContentLibraryIso.Id)" -Recurse |
            Select-Object -ExpandProperty FolderPath
        $isoPath = Get-ChildItem -Path "$($driveName):\$($clPath.Split(' ')[1])" -Filter $filter -Recurse | 
            Select-Object -ExpandProperty DatastoreFullPath
        Remove-PSDrive -Name $driveName -Confirm:$false | Out-Null
            
        $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
            
        $change = New-Object VMware.Vim.VirtualDeviceConfigSpec
        $change.Operation = [Vmware.vim.VirtualDeviceConfigSpecOperation]::edit
            
        $dev = $cd.ExtensionData
        $dev.Backing = New-Object VMware.Vim.VirtualCdromIsoBackingInfo
        $dev.Backing.FileName = $isoPath
            
        $change.Device += $dev
        $change.Device.Connectable.Connected = $true
            
        $spec.DeviceChange = $change
            
        $vm.ExtensionData.ReconfigVM($spec)
    
        Get-CDDrive -Id $CDDrive.Id
    }
}

Annotations

Line 34-37: The parameters that the function requires are objects returned by the Get-CDDrive and Get-ContentLibraryItem cmdlets. The function does not implement OBN support.

Line 40-41: This provides meaningful content for the message returned when the WhatIf switch is used.

Line 44-46: To avoid conflicts with PSDrives the user might already have defined, the function generates a 3-letter, random string that will be used as the drivename.

Line 52-53: To avoid a situation where the same ISO file might be uploaded to two different Content Libraries, the function first finds the path to the folder that corresponds with the requested Content Library.

Line 54-55: This search finds the actual path to the desired ISO file.

Line 60-72: A ‘regular’ call to the ReconfigVM_Task method.

Line 74: The function returns the CDDrive object, just like the regular Set-CDDrive cmdlet does. But since there is currently not yet support for ISO files from a Content Library, the filepath looks like a regular path to an ISO file. The Web Client does show that this is an ISO from a Content Library.

Sample Run

Using the function is rather straightforward.

$vmName = 'photonps'
$cdName = 'CD/DVD drive 1'
$cLibName = 'ConLib1'
$cLibItemName = 'photon-minimal-3.0-a0f216d'
    
$vm = Get-VM -Name $vmName
    
$cl = Get-ContentLibrary -Name $cLibName
$iso = Get-ContentLibraryItem -Name $cLibItemName -ContentLibrary $cl
    
Get-CDDrive -VM $vm -Name $cdName |
Set-CDDriveCLISO -ContentLibraryISO $iso -Confirm:$false

The Web Client shows that an ISO from a Content Library is loaded.

Enjoy!

This post will be about my first experiences with the PowerShell Crescendo module, which was just released in Preview 1.

Now, why would I care to write a post about a preview of a module?

The reason is quite simple. This module is, in my opinion, covering something that was missing in PowerShell. Namely the possibility to use native commands, and present the results as objects.

The blog post by Jason Helmick I linked to earlier explains this all in more detail.

What follows are my first, admittedly simple, trials using the Crescendo module. If you don’t want to copy code from this post, I created the VMW-Crescendo repository that contains all the files.

An Example

The first example I tackled is the DOS command tasklist, which displays a list of the running processes. 

Not to complicate things at the start, I decided to only implement the module for running tasklist on the local computer.

And needless to say, since this is a DOS command, it will not run on other platforms besides Windows.

The idea behind the Crescendo module is that you write in JSON, how the cmdlet(s) you will create use the native command. This not only includes specifying which parameters are used but also how the returned output will be converted to an object.

Since PowerShell contains cmdlets to convert to and from JSON, I decided to try and write those definitions in PowerShell. And then convert that code to JSON format, which I could then feed to the Crescendo cmdlet.

The Code

#Requires -Version 7.0

#region An example

$taskConfig = @{
  '$schema' = './Microsoft.PowerShell.Crescendo.Schema.json'
  'Verb' = 'Get'
  'Noun' = 'TaskList'
  'OriginalName' = 'tasklist'
  'OriginalCommandElements' = @("/V")
  'OutputHandlers' = @(
        @{
            'ParameterSetName' = 'Default'
            'Handler' = {
              $labels = $args[0][1]
              $index = 0
              $fieldInfo = $args[0][2].Split(' ') | ForEach-Object -Process {
                @{
                  start = $index
                  length = $_.Length
                }
                $index += $_.Length + 1
              }
              $args[0] | Select-Object -skip 3 -PipelineVariable row | ForEach-Object -Process {
                $obj = [ordered]@{}
                $fieldInfo | ForEach-Object -Process {
                  $obj.Add($labels.SubString($_.start,$_.length).Trim(' '),$row.SubString($_.start,$_.length).Trim(' '))
                }
                New-Object -TypeName PSObject -Property $obj
              }
            }
        }
  )
}

$taskConfig | ConvertTo-Json | Set-Content -Path "$PSScriptRoot\tasklist.json"

Export-CrescendoModule -ConfigurationFile "$PSScriptRoot\tasklist.json" -ModuleName "$PSScriptRoot\TaskList.psm1" -Force -Confirm:$false

#endregion

Annotations

Line 1: Although the blog post mentions that a module generated with Crescendo can be used on PowerShell 5.1 and 7, I noticed that since the generated code uses the ternary operator, this not really the case. Hence the requirement to run this on PowerShell v7.

Line 5-35: As I mentioned earlier, I created the definition of the cmdlets to be created in PowerShell.

Line 10: By default, the tasklist command is started with the /V (for verbose) parameter.

Line 11-33: The OutputHandler is where you define how the output of the native command will be converted to an object. I did have to use some ‘trickery’ to avoid having to use hard-coded offset. Luckily the one can use the underlining to determine the start and length of each column.

Line 37: The Powershell object is converted to JSON code, and saved to a file.

Line 39: The Export-CrescendoModule cmdlet takes the JSON file and generates the module.

Sample Run

The generated modules, at least in this Preview 1, only generate a .psm1 file. You can use the Import-Module pointing to that .psm1 file.

The module contains the Get-TaskList cmdlet, that requires no further parameters and returns a series of objects, containing the results.

Since the Get-TaskList cmdlet returns objects, we can now easily use other PowerShell cmdlets to further handle the returned data. For example:

Get-TaskList | Export-Csv -Path .\TaskList.Csv -NoTypeInformation -UseCulture

The resulting CSV file contains all the data.

VMware Tools Debug Levels

The second example I concocted, uses one of the native commands that come with the installation of the VMware Tools. The VMwareToolboxCmd.exe has the option to get and set the debug levels on the VMware Tools services. 

Note that you need to run this command in Administrator mode.

The Code

#Requires -Version 7.0

#region VMware Tools debug levels

$getDebugConfig = @{
  '$schema' = './Microsoft.PowerShell.Crescendo.Schema.json'
  'Verb' = 'Get'
  'Noun' = 'ToolsDebugSetting'
  'OriginalName' = "$($env:ProgramFiles)\VMware\VMware Tools\VMwareToolboxCmd.exe"
  'Description' = 'Retrieve the debug levels for VMware Tools'
  'OriginalCommandElements' = 'logging','level','get'
  'Parameters' = @(
    @{
      'Name' = 'Svc'
      'OriginalName' = 'vmsvc'
      'ParameterType' = 'switch'
      'Description' = 'Primary VMware Tools service'
    },
    @{
      'Name' = 'Vss'
      'OriginalName' = 'vmvss'
      'ParameterType' = 'switch'
      'Description' = 'Volume Shadow Copy service'
    },
    @{
      'Name' = 'Resolution'
      'OriginalName' = 'vmresset'
      'ParameterType' = 'switch'
      'Description' = 'Resolution Set Tool'
    },
    @{
      'Name' = 'Tray'
      'OriginalName' = 'vmtray'
      'ParameterType' = 'switch'
      'Description' = 'The Tray icon in Windows'
    },
    @{
      'Name' = 'User'
      'OriginalName' = 'vmusr'
      'ParameterType' = 'switch'
      'Description' = 'The VMware Tools user service'
    },
    @{
      'Name' = 'Toolbox'
      'OriginalName' = 'toolbox'
      'ParameterType' = 'switch'
      'Description' = 'Toolbox UI'
    },
    @{
      'Name' = 'ToolBoxCmd'
      'OriginalName' = 'toolboxcmd'
      'ParameterType' = 'switch'
      'Description' = 'Toolbox command line'
    }
  )
  'OutputHandlers' = @(
        @{
            'ParameterSetName' = 'Default'
            'Handler' = {
              $service,$level = $args[0].Split('=')
              New-Object -TypeName PSObject -Property @{
                "$($service -replace '.level','')" = $level.Trim(' ')
              }
            }
        }
  )
}
$getDebugConfig | ConvertTo-Json | Set-Content -Path "$PSScriptRoot\getdebug.json"

$setDebugConfig = @{
  '$schema' = './Microsoft.PowerShell.Crescendo.Schema.json'
  'Verb' = 'Set'
  'Noun' = 'ToolsDebugSetting'
  'OriginalName' = "$($env:ProgramFiles)\VMware\VMware Tools\VMwareToolboxCmd.exe"
  'Description' = 'Retrieve the debug levels for VMware Tools'
  'OriginalCommandElements' = 'logging', 'level', 'set'
  'Parameters' = @(
    @{
      'Name' = 'Svc'
      'OriginalName' = 'vmsvc'
      'ParameterType' = 'switch'
      'Description' = 'Primary VMware Tools service'
    },
    @{
      'Name' = 'Vss'
      'OriginalName' = 'vmvss'
      'ParameterType' = 'switch'
      'Description' = 'Volume Shadow Copy service'
    },
    @{
      'Name' = 'Resolution'
      'OriginalName' = 'vmresset'
      'ParameterType' = 'switch'
      'Description' = 'Resolution Set Tool'
    },
    @{
      'Name' = 'Tray'
      'OriginalName' = 'vmtray'
      'ParameterType' = 'switch'
      'Description' = 'The Tray icon in Windows'
    },
    @{
      'Name' = 'User'
      'OriginalName' = 'vmusr'
      'ParameterType' = 'switch'
      'Description' = 'The VMware Tools user service'
    },
    @{
      'Name' = 'Toolbox'
      'OriginalName' = 'toolbox'
      'ParameterType' = 'switch'
      'Description' = 'Toolbox UI'
    },
    @{
      'Name' = 'ToolBoxCmd'
      'OriginalName' = 'toolboxcmd'
      'ParameterType' = 'switch'
      'Description' = 'Toolbox command line'
    },
    @{
      'Name' = 'Level'
      'OriginalName' = ''
      'ParameterType' = 'string'
      'Mandatory' = $true
      'Description' = 'New debug level'
    }
    )
    }
$setDebugConfig | ConvertTo-Json | Set-Content -Path "$PSScriptRoot\setdebug.json"

$sCrecendo = @{
  ConfigurationFile = "$($PSScriptRoot)\getdebug.json", "$($PSScriptRoot)\setdebug.json"
  ModuleName = "$($PSScriptRoot)\ToolsDebug.psm1"
  Force = $true
  Confirm = $false
  Verbose = $true
}
Export-CrescendoModule @sCrecendo
#endregion

Annotations

Line 1: Although the blog post mentions that a module generated with Crescendo can be used on PowerShell 5.1 and 7, I noticed that since the generated code uses the ternary operator, this not really the case. Hence the requirement to run this on PowerShell v7.

Line 5-66: This part is the definition for the Get-ToolsDebugSettings cmdlet.

Line 11: With the OriginalCommandElements property we define which parameters shall be always added to the native command.

Line 12-55: Each possible service is defined as a Switch parameter. Each entry also has the cmdlet parameter name (Name) and how it shall be translated to a native command parameter (OriginalName).

Line 59-63: The Output Handler takes the line that is returned and converts it into an object.

Line 68: The PowerShell object is converted into JSON and save to a file.

Line 70-128: This part is the definition for the Set-ToolsDebugSettings cmdlet.

Line 120-126: The Set cmdlet will have an additional parameter (Level) that indicates which new debug level shall be set for the selected service. Note that the debuglevel is added as-is, no parameter on the native command.

Sample Run

The cmdlets from the generated module need to be called from an elevated prompt (same as the native command).

#Requires -Version 7.0

Import-Module "$PSScriptRoot\ToolsDebug.psm1" -Force

Get-ToolsDebugSetting -Vss
Set-ToolsDebugSetting -Vss -Level debug
Get-ToolsDebugSetting -Vss

And just as with the native command we can query and set the debug levels.

In Conclusion

Is this new Crescendo module something that you should be using immediately?

Definitely not, this is a module, based on an interesting concept, that only just entered Preview 1.

But it is surely a concept that is interesting.
I’ll be watching where this goes, and what can be done with it.

Enjoy!

Introduction

In Post 4 of the cloud-init series, we ran our PowerShell/PowerCLI scripts natively in the Guest OS of the instance.

That might have an impact in some situations. When the first script installs for example the VMware PowerCLI modules, these modules will stay installed in the Guest OS. Consider this as a kind of PowerShell tattooing.

Another task that might be somewhat of a challenge is testing your PowerShell scripts against multiple environments, different module versions, different OS versions… Again, when we make changes to the Guest OS, these changes might be hard to remove.

Besides deploying a new ‘cattle’ station for each possible combination, we can also run our scripts in Docker containers. Once the script is done, we throw away the container instance. And we are back to a pristine OS.

And as it turns out, containers also offer some new features that are harder, or even impossible, to implement when running locally in the Guest OS.

Since we will be using several docker commands in the following sections, it will be useful to have the Docker Reference documentation nearby.

Install Docker

Ubuntu

The Ubuntu OVAs we have been installing in the previous posts in this series, do not contain the docker package. We can easily include that in our YAML file under the runcmd section.

#cloud-config
hostname: ubuntubioniccontainer
fqdn: ubuntubioniccontainer.local.lab
write_files:
- path: /etc/netplan/50-cloud-init.yaml
  content: |
    network:
     version: 2
     ethernets:
      ens192:
       addresses: [192.168.10.84/24]
       gateway4: 192.168.10.1
       dhcp6: false
       nameservers:
         addresses:
           - 192.168.10.2
           - 192.168.10.3
         search:
           - local.lab
       dhcp4: false
       optional: true
- path: /etc/sysctl.d/60-disable-ipv6.conf
  owner: root
  content: |
    net.ipv6.conf.all.disable_ipv6=1
    net.ipv6.conf.default.disable_ipv6=1
runcmd:
- netplan --debug apply
- sysctl -w net.ipv6.conf.all.disable_ipv6=1
- sysctl -w net.ipv6.conf.default.disable_ipv6=1
- apt -y autoremove
- apt-get update
- apt-get remove docker docker-engine docker.io
- apt install docker.io -y
- systemctl enable docker
- systemctl start docker
timezone: Europe/Brussels
system_info:
  default_user:
    name: default-user
    lock_passwd: false
    sudo: ["ALL=(ALL) NOPASSWD:ALL"]
disable_root: false
ssh_pwauth: yes
users:
  - default
  - name: luc
    gecos: LucD
    lock_passwd: false
    groups: sudo, users, admin
    shell: /bin/bash
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
chpasswd:
  list: |
    default-user:$6$2csGx3AAX0RPRMPB$85sEEZ76z0V93M9pvXGtTFrwlMOaV2VjRHB6IKPRrwc.4qEqcs2EnDzn8oUlcVVb9Biq.Pg7qQchLgNLuetkT0
    luc:$6$.4Kyvju2v5iR$bvZHOntkH9EqgGdMc7fKZKbB132LYHVIFgxN7Ho.mlMGloUayOaKijR6MvLp/ncf4SynTigjaOKGk11EF9gru/
    root:$6$m1TXY60tUAViCYb1$/w.CqV50.M0Nlt6mx2726.bxWZ7/TCqWjOOw4S3h1H8AN.SPWQnKoASXth2E3euBXwebU4Q7dpoGU1SiQontT1
  expire: false
package_upgrade: true
package_reboot_if_required: true

Annotations

Line 33-36: Install the docker package, start it and make sure it runs after a reboot.

Line 33: If there should already be a docker package installed, this line removes that package. If no docker package is installed, the line does nothing.

Photon

The Photon OVA we use comes with docker installed. No need for any additional commands in the YAML file.

Image microsoft/powershell

In the following sections we will run a number of scripts in the microsoft/powershell container on Ubuntu and Photon. Only when there is a difference between both platforms, will the text mention the platforms.

A Simple Run

Let’s try running a simple PowerShell script in a container on the station.

$vmName = 'ubuntubioniccontainer'
$code = @'
sudo docker run --rm microsoft/powershell:latest pwsh -C 'whoami;Get-Process;pwd'
'@

$credItem = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credItem.User, (ConvertTo-SecureString $credItem.Password -AsPlainText -Force))

$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'bash'
    ScriptText = $code
    NoIPinCert = $true
    Verbose = $true
}
Invoke-VMScriptPlus @sInvoke

Annotations

Line 3: The docker command we sent to the station. The details of the different parts in this line are explained in the following table.

The output will be something like this.

Notice how docker will search for and install the image (since it isn’t present). That is the output in the yellow box. The actual output of the commands we sent to pwsh, are in the green box.

The second time we run the same script, we will only see the actual output of the commands. And that’s because the image is already present.

Modules

Time to start diving into PowerCLI. For that we need to install the VMware PowerCLI modules of course.

$vmName = 'ubuntubioniccontainer'

$script = @'
$ProgressPreference = "SilentlyContinue"
Install-Module -Name VMware.PowerCLI -AllowClobber -Force | Out-Null
Get-Module -Name VMware* -ListAvailable
'@

$docker = @'
sudo docker run --rm microsoft/powershell:latest pwsh -C '$($script.Replace("r",'').Split("n") -join ';')'
'@

$credItem = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credItem.User, (ConvertTo-SecureString $credItem.Password -AsPlainText -Force))

$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'bash'
    ScriptText = $ExecutionContext.InvokeCommand.ExpandString($docker)
    NoIPinCert = $true
    Verbose = $true
}
Invoke-VMScriptPlus @sInvoke

Annotations

Line 3-7: We use a here-string for the script we want to execute under pwsh in the container.

Line 9-11: And another here-string for the docker command.

Line 10: The script is running on a Windows box, hence the <CR><LF> to <LF>. The second manipulation is to convert the array of lines to a single line, where the lines are separated with a semi-column. Note that the actual conversion does not happen here.

Line 20: The script uses the ExpandString method to perform the text manipulations we described in the comments on Line 10. Also note that we don’t have to escape the dollar-signs in the $script part, since the variable substitution will not go that deep.

The result (don’t mind the funny characters for now).

ScreenSpace

When we use all the defaults in the container, we seem to end up with a simple screen that has the default 80 x 25 dimensions. For better results, we can introduce bigger screens inside the container.

$vmName = 'ubuntubioniccontainer'

$script = @'
reset -w
$ProgressPreference = "SilentlyContinue"
Install-Module -Name VMware.PowerCLI -AllowClobber -Force | Out-Null
Get-Module -Name VMware* -ListAvailable
'@
$docker = @'
sudo docker run --rm --tty \
    -e TERM=xterm -e LINES=25 -e COLUMNS=135 \
    microsoft/powershell:latest \
    pwsh -C '$($script.Replace("`r",'').Split("`n") -join ';')'
'@

$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))

$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'bash'
    ScriptText = $ExecutionContext.InvokeCommand.ExpandString($docker)
    NoIPinCert = $true
    Verbose = $true
}
Invoke-VMScriptPlus @sInvoke

Annotations

Line 4: The reset -w command is a fix for an issue that Docker seems to have when a pseudo-terminal (the –tty parameter) is used. With the reset -w command, the terminal will reread the content of setupterm and adjust the lines and columns accordingly.

Line 10: The –tty parameter allocates a pseudo-terminal to the container.

Line 11: The -e parameter is used to set a number of environment variables for the container. In this case, specify that the container shall use an xterm with dimensions 25 x 135.

The changed screen is visible in the output (compare it with the width of the previous output).

Volumes

With the method we just showed, we will have to incur the overhead of installing the VMware PowerCLI modules on each run, because we throw away the container after we used it.

But there is a solution for this, by using docker volumes. They are kept inside the guest OS of the instance and can be attached to each container we run.

$vmName = 'ubuntubioniccontainer'

$script = @'
reset -w
\`$ProgressPreference = "SilentlyContinue"
if(-not (Get-Module -Name VMware.PowerCLI -ListAvailable)){
    Install-Module -Name VMware.PowerCLI -AllowClobber -Force | Out-Null
}
\`$WarningPreference = 'SilentlyContinue'
\`$sConf = @{
    InvalidCertificateAction = 'Ignore'
    ParticipateInCeip = \`$false
    DisplayDeprecationWarnings = \`$false
    Scope = 'User'
    Confirm = \`$false
}
Set-PowerCLIConfiguration @sConf | Out-Null
Connect-VIServer -Server $vcsaName -User $vcsaUser -Password $vcsaPswd | Out-Null
Get-VMHost
Disconnect-VIServer -Server $vcsaName -Confirm:\`$false
'@

$docker = @'
sudo docker volume create PCLIModules > /dev/null
cat > ~/script.ps1 << EOF
$($script.Split("r"))
EOF
sudo docker run --rm \
    --tty \
    -v /PCLIModules:/root/.local/share/powershell/Modules \
    -v ~:/Scripts \
    -e TERM=xterm -e LINES=25 -e COLUMNS=135 \
    microsoft/powershell:latest \
    pwsh -F /Scripts/script.ps1
rm ~/script.ps1
'@

$vcsaName = 'vcsa.local.lab'
$credVCSA = Get-VICredentialStoreItem -Host $vcsaName
$vcsaUser = $credVCSA.User
$vcsaPswd = $credVCSA.Password

$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))

#$credItem = Get-VICredentialStoreItem -Host $vmName
#$cred = New-Object System.Management.Automation.PSCredential ($credItem.User, (ConvertTo-SecureString $credItem.Password -AsPlainText -Force))
$cred = New-Object System.Management.Automation.PSCredential ($vmUser, (ConvertTo-SecureString $vmPswd -AsPlainText -Force))

$dockerExpanded = $ExecutionContext.InvokeCommand.ExpandString($docker)
$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'bash'
    ScriptText = $ExecutionContext.InvokeCommand.ExpandString($dockerExpanded)
    NoIPinCert = $true
}
Invoke-VMScriptPlus @sInvoke

Line 5: We have some 'Inception' going on here, hence the '' sequence before the dollar sign. In line 50 we substitute the variables in the bash script itself. That way we can provide the script as an inline file in the bash script. But since bash also interprets that back-tick, we have to escape it. The back-tick itself is to avoid variable substitution that is done in line 55.

Line 24: We create a docker volume named PCLIModules. If that volume already exists, the bash script will just continue.

Line 25-27: Instead of having the PowerShell/PowerCLI script in the Command parameter of the pwsh command, we use an inline file. This effectively creates a .ps1 file in the home directory of the user in the guest OS.

Line 30: The Volume parameter on the docker run command maps the docker volume PCLIModules to the folder /root/.local/share/powershell/Modules in the container. With an Install-Module, the modules will be installed in this /root/.local/share/powershell/Modules , and thus be stored on the docker volume PCLIModules. This volume can be reused on subsequent runs of the container, eliminating the need to install the VMware PowerCLI modules on each run.

Line 31: On the docker run command we map the user's home directory (~) in the guest OS to a folder, named Scripts in the container.

Line 34: The pwsh command that runs in the container uses the File parameter to run the .ps1 file created in lines 25-27.

Line 35: The .ps1 file that was created before calling the docker run command, is now removed.

Line 55: The second substitution round. In this case we substitute primarily all non-escaped (back-tick) dollar sign in the PowerShell script (at this point in the script present as an inline file in the bash script).

The result when we run the script.

Notice how the ScriptText property contains the bash script as it is presented to the Guest OS of the instance. In there the dollar signs are escaped (with a back-slash) to avoid substitution by bash.

ANSI codes and Existing Scripts

In this section, we will tackle two subjects.

• Eliminate those 'funny' characters in the output
• How to run my existing script, similar to what we did in Part 4 - Running Scripts.

As it turns out, these 'funny characters' are ANSI escape codes. In the container, we are using a pseudo-terminal, and as a result, the OS in the container sends those ANSI escape codes to position the cursor, change the text colour, but also those '...' (three dots) when PowerShell has to cut short the value of a property.

For now, the only way if found to eliminate those ANSI escape codes, is to find and replace them.

For using your existing scripts in this setup, we adopt the same strategy as we did in Part 4 - Running Scripts.

Our existing script, in Sample-PCLI-Script3.ps1, looks like this.

Get-VMHost |
    Select Name, NumCpu, Version |
    ConvertTo-Html

And the script to run all this looks like this

$vmName = 'ubuntubioniccontainer'

$prefix = @'
reset -w
$ProgressPreference = "SilentlyContinue"
if(-not (Get-Module -Name VMware.PowerCLI -ListAvailable)){
    Install-Module -Name VMware.PowerCLI -AllowClobber -Force | Out-Null
}
$WarningPreference = 'SilentlyContinue'
$sConf = @{
    InvalidCertificateAction = 'Ignore'
    ParticipateInCeip = $false
    DisplayDeprecationWarnings = $false
    Scope = 'User'
    Confirm = $false
}
Set-PowerCLIConfiguration @sConf | Out-Null
Connect-VIServer -Server $vcsaName -User $vcsaUser -Password $vcsaPswd | Out-Null
'@
$script = Get-Content -Path .\Sample-PCLI-Script3.ps1
$suffix = @'
Disconnect-VIServer -Server $vcsaName -Confirm:$false
'@

$docker = @'
sudo docker volume create PCLIModules > /dev/null
cat > ~/script.ps1 << 'EOF'
$($prefix.Split("r"))
$($script -join "n")
$($suffix.Split("r"))
EOF
sudo docker run --rm \
    --tty \
    -v /PCLIModules:/root/.local/share/powershell/Modules \
    -v ~:/Scripts \
    -e TERM=xterm -e LINES=25 -e COLUMNS=135 \
    microsoft/powershell:latest \
    pwsh -F /Scripts/script.ps1 > ~/result.txt
cat ~/result.txt
rm ~/result.txt
rm ~/script.ps1
'@

$vcsaName = 'vcsa.local.lab'
$credVCSA = Get-VICredentialStoreItem -Host $vcsaName
$vcsaUser = $credVCSA.User
$vcsaPswd = $credVCSA.Password

$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))

$dockerExpanded = $ExecutionContext.InvokeCommand.ExpandString($docker)
$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'bash'
    ScriptText = $ExecutionContext.InvokeCommand.ExpandString($dockerExpanded)
    NoIPinCert = $true
    Verbose = $true
}
$result = Invoke-VMScriptPlus @sInvoke

# Remove ANSI Escape Sequences
($result.ScriptOutput.Split("n") |
    Foreach-Object -Process {
        ($_ -replace "\xe2\x80\xa6","…") -replace "\x1b\[\?[1-9]h|\x1b\[[1-9]n|\x1b=|\x1b\]0;|\x07|\x1b\]0;", ''
    }) -join "n"

Annotations

Line 3, 20, 21: We split the code into three parts. A fixed prefix, the code of our existing script and a suffix.

Line 27-31: We use the same concept of an inline file as we did in the previous section. The only difference is that we now have three parts instead of one.

Line 64-67: The ANSI escape sequences are removed from the output. The RegEx expression does not cover all possible ANSI escape sequences, but it did the job for all the ones I encountered. On the other hand, the RegEx expression can easily be expanded.

The result, which is an HTML file, looks like this.

The part where we filter away the ANSI code sequence is also used to translate some well-known characters. Remember the funny characters in the module list?

With the first -replace operator in the previous code, we can change this back to the character we are used to. The hex sequence 0xE2, 0x80, 0xA6 that get back is, in fact, the code for the horizontal ellipsis, the 3 horizontal dots PowerShell uses to indicate a cut off value.

There are most probably many other characters or character combinations that do not translate well, but at least we have a method to correct this.

InFile & OutFile

Using inline files and the stdout stream for input and results works in most cases, but it might make things needlessly complex in others. To avoid that, we can make use of the InFile and OutFile parameters on the Invoke-VMScriptPlus function. The parameters allows us to pass one or more files along with the script, and retrieve one or more files back from the environment where the script runs.

If we use InFile and OutFile together with containers, there is some Inception going on. The following schematic might make this a bit more visible.

As you can see, the InFile files are stored in the guest OS of the instance. And then through the Volume parameter on the docker command, passed along into the container.

The same goes for the OutFile files but in the reverse sense.

We are going to execute an existing script. Note how to make a small change in the path of the resulting file. We have to use the path to the Scripts folder, which will be mapped through the Volume parameter.

Get-VMHost |
Select Name, NumCpu, Version |
ConvertTo-Html | Out-File -FilePath \Scripts\report.html

Write-Host "Script completed"

And this is the script that will run that script in a container.

$vmName = 'ubuntubioniccontainer'
$oldScript = '.\Sample-PCLI-Script.ps1'
$tempScript = 'script.ps1'

$script = @'
reset -w
$ProgressPreference = "SilentlyContinue"
if(-not (Get-Module -Name VMware.PowerCLI -ListAvailable)){
    Install-Module -Name VMware.PowerCLI -AllowClobber -Force | Out-Null
}
$WarningPreference = 'SilentlyContinue'
$sConf = @{
    InvalidCertificateAction = 'Ignore'
    ParticipateInCeip = $false
    DisplayDeprecationWarnings = $false
    Scope = 'User'
    Confirm = $false
}
Set-PowerCLIConfiguration @sConf | Out-Null
Connect-VIServer -Server $vcsaName -User $vcsaUser -Password $vcsaPswd | Out-Null

####### My Script ######
$(Get-Content -Path $oldScript -Raw)
########################

Disconnect-VIServer -Server $vcsaName -Confirm:$false
'@

$docker = @'
sudo docker volume create PCLIModules > /dev/null
sudo docker run --rm \
    --tty \
    -v /PCLIModules:/root/.local/share/powershell/Modules \
    -v $(pwd):/Scripts \
    -e TERM=xterm -e LINES=25 -e COLUMNS=135 \
    microsoft/powershell:latest \
    pwsh -F /Scripts/$($tempScript)
'@

$vcsaName = 'vcsa.local.lab'
$credVCSA = Get-VICredentialStoreItem -Host $vcsaName
$vcsaUser = $credVCSA.User
$vcsaPswd = $credVCSA.Password

$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))

$tempScript = 'script.ps1'
$ExecutionContext.InvokeCommand.ExpandString($script) | Out-File -LiteralPath $tempScript
$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'bash'
    ScriptText = $ExecutionContext.InvokeCommand.ExpandString($docker)
    NoIPinCert = $true
    InFile = $tempScript
    OutFile = 'report.html'
}
Invoke-VMScriptPlus @sInvoke

Remove-Item -LiteralPath $tempScript -Confirm:$false

Invoke-Item -Path .\report.html

Annotations

Line 2: The name of our script we want to execute. Note that we had to change the Path of the resulting file. This to allow the mapping through the volume parameter on the docker run command.

Line 3: A filename for the script we are going to run in the container. Ultimately this could be replaced by a randomly generated filename.

Line 5-27: The full script we are sending to the container. It is comprised of the Prefix, our original script and the Suffix. The Prefix and Suffix are the same as we have shown in previous sections. The only difference is that all three parts now come together in one here-string.

Line 23: We substitute this line with our original script later on with the call to ExpandString.

Line 29-38: The actual bash script, with the docker commands, that we are running inside the Guest OS of the instance.

Line 49: We expand the $script variable and save the resulting text to a file.

Line 54: Another call to ExpandString. This time to substitute the variables in the $docker variable.

Line 56: The script file we send along to the Guest OS.

Line 57: The output file we retrieve from the Guest OS.

The script's output looks like this. Notice that it only contains the result from the Write-Host line in our original script.

And the result, report.html, is displayed in a web browser through the Invoke-Item cmdlet.

Image vmware/powerclicore

Shortly after the new tdnf package for PowerShell was made available, a new version of the vmware/powerclicore docker image was created.

The difference with the microsoft/powershell image is the underlying OS, Photon v3R2 instead of Ubuntu 18.04.

Let's start with a simple script, similar to the one we used earlier.

$code = @'
docker run --rm microsoft/powershell:latest pwsh -C 'whoami;Get-Process;pwd'
'@

$credItem = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credItem.User, (ConvertTo-SecureString $credItem.Password -AsPlainText -Force))

$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'bash'
    ScriptText = $code
    NoIPinCert = $true
    GuestOSType = 'linux'
}
Invoke-VMScriptPlus @sInvoke | select -ExpandProperty ScriptOutput

With the microsoft/powershell image this produces the following. We see the result of all three commands.

On the vmware/powerclicore image, the result is a bit different. The whoami output is there, but the output from the other two commands is missing.

As it turns out, there is an issue when passing commands to pwsh via the Command parameter. When using the File parameter everything seems to work well.

One difference to note, compared with the micorosft/powershell image, the vmware/powerclicore:latest image, and the Photon OS in that image, have minimal terminal types present. As a result, you will need to set the TERM environment variable to linux instead of xterm.

A typical call would look something like this.

docker volume create PCLIModules > /dev/null
docker run --rm \
    --tty \
    -v /PCLIModules:/root/.local/share/powershell/Modules \
    -v `$(pwd):/Scripts \
    -e TERM=linux -e LINES=25 -e COLUMNS=135 \
    vmware/powerclicore:latest \
    pwsh -F /Scripts/script.ps1

Image mcr.microsoft.com/powershell:preview

Note that there is also an image for the latest PS v7 Preview available. If you want to test/develop your scripts in this PS v7 Preview, this is a simple and easy method to do it.

All previous methods discussed in this post work against this Preview.

Enjoy!

p

Introduction

The purpose of this post is quite straightforward. How can we use the ‘cattle‘ stations we created to run our PowerShell/PowerCLI scripts?

Since all the stations we deployed with cloud-init come with the VMware Tools installed, we can use the GuestOperations, and all cmdlets and functions based on these API, to communicate with the Guest OS.

In practice, this means that the Invoke-VMScript cmdlet should be used. But unfortunately, the current implementation of that cmdlet has some issues with multi-line bash scripts (and a few other problems).

To avoid those issues I have written my Invoke-VMScriptPlus function. Besides fixing the above-mentioned issues, it also adds some extra features that come in handy, as you will notice later on in this post.

Local PowerShell Script

Ubuntu

We start simple. In this post, we will show how you can run your PowerShell scripts locally inside the guest OS of the stations we deployed with cloud-init.

Note that in the examples I use a VICredentialStoreItem to fetch the credentials for the target guest OS. As you might know by now, the VICredentialStoreItem cmdlets are only supported on a Windows platform (since they use the DPAPI methods).

Any which way you get those credentials will of course work. But I consider that not to be part of the cloud-init series of posts.

Back to the task at hand, let’s use a simple example to start.

$vmName = 'ubuntubionicps'
$code = @'
$PSVersionTable
'@

$credItem = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credItem.User, (ConvertTo-SecureString $credItem.Password -AsPlainText -Force))

$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'powershellv6'
    ScriptText = $code
}
Invoke-VMScriptPlus @sInvoke

Nothing too complicated at first sight. We sent a PowerShell script to an Ubuntu box. But we do seem to get an error.

That doesn’t really give us a lot of useful information on what is going on. My first step in analysing such non-obvious errors is to add some verbosity.

$vmName = 'ubuntubionicps'
$code = @'
$PSVersionTable
'@

$credItem = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credItem.User, (ConvertTo-SecureString $credItem.Password -AsPlainText -Force))

$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'powershellv6'
    ScriptText = $code
    Verbose = $true
}
Invoke-VMScriptPlus @sInvoke

And now we get some more clues. Btw, I only listed part of the verbose output that helps in analysing the issue.

The error comes immediately after the PUT where we copy the script text to a file in the guest OS. Since SSL/TLS secure channel is mentioned, let’s have a look at the certificate that is used on that ESXi node.

And there is our answer. The certificate used on the ESXi node in question only contains a SAN for the FQDN, not the IP address.

Since this is something I watched happen at several sites, I added the NoIPinCert switch to the Invoke-VMScriptPlus function. When that switch is set, the function will take the IP address and replace it by the FQDN. This, of course, assumes that DNS resolution is working!

$vmName = 'ubuntubionicps'
$code = @'
$PSVersionTable
'@

$credItem = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credItem.User, (ConvertTo-SecureString $credItem.Password -AsPlainText -Force))

$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'powershellv6'
    ScriptText = $code
    NoIPinCert = $true
}
Invoke-VMScriptPlus @sInvoke

And now we get the result we were expecting.

Don’t mind the properties beside ScriptOutput and the funny characters in ScriptOutput for now. We’ll get to that later.

Photon

And for good measure, let’s run the same on a Photon station.

$vmName = 'photonps'
$code = @'
$PSVersionTable
'@

$credItem = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credItem.User, (ConvertTo-SecureString $credItem.Password -AsPlainText -Force))

$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'powershellv6'
    ScriptText = $code
    NoIPinCert = $true
}
Invoke-VMScriptPlus @sInvoke

Confirmed, also a success with a Photon guest OS.

Local PowerCLI Script

So we can run PowerShell script, but what about VMware PowerCLI? I want to be able to run my PowerCLI script on such a station.

Since we need to have the modules installed, before we can actually use any of the PowerCLI cmdlets, let’s include that in our test script.

$vmName = 'ubuntubionicps'

$code = @'
$ProgressPreference = "SilentlyContinue"
if(-not (Get-Module -Name 'VMware.PowerCLI' -ListAvailable)){
    Install-Module -Name VMware.PowerCLI -Scope CurrentUser -AllowClobber -Force | Out-Null
}
$WarningPreference = 'SilentlyContinue'
$sConf = @{
    InvalidCertificateAction = 'Ignore'
    ParticipateInCeip = $false
    DisplayDeprecationWarnings = $false
    Scope = 'User'
    Confirm = $false
}
Set-PowerCLIConfiguration @sConf | Out-Null
Connect-VIServer -Server $vcsaName -User $user -Password $pswd | Out-Null
Get-VMHost | Select Name,Version,PowerState,NumCpu,MemoryTotalGB | ConvertTo-Csv -NotypeInformation -UseCulture
Disconnect-VIServer -Server $vcsaName -Confirm:$false
'@

$vcsaName = 'vcsa.local.lab'
$credVCSA = Get-VICredentialStoreItem -Host $vcsaName
$user = $credVCSA.User
$pswd = $credVCSA.Password

$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))

$sInvoke = @{
    VM = Get-VM -Name $vmName
    GuestCredential = $cred
    ScriptType = 'powershellv6'
    ScriptText = $ExecutionContext.InvokeCommand.ExpandString($code)
    NoIPinCert = $true
}
$result = Invoke-VMScriptPlus @sInvoke
$result.ScriptOutput | ConvertFrom-Csv -UseCulture | Format-Table -AutoSize

Annotations

Line 3-20: The script we are executing inside the target guest OS.

Line 4: Since we are using variable substitution on the script text later on (see line 34), we need to escape the dollar signs in the variables we do NOT want to be substituted. This is done through a backtick before the dollar sign.

Line 4: We mute the progress information. This to avoid unnecessary output during the installation of the PowerCLI modules.

Line 5-7: A simple test to check if the PowerCLI modules are installed or not. If they aren't, the script will install them. Note that since we are running our scripts locally in the guest OS, once the modules are installed they will stay installed for any future script runs. If you want to start fresh for each script, you will have to deploy a new instance. The 'cattle' principle!

Line 9-16: We want our script to be as 'silent' as possible, that is why we change some PowerCLI configuration settings. Note that disabling the CEIP participation in this example script is a personal choice and you can leave it on without any impact on your script(s).

Line 18: This sample code will output results in CSV format (ConvertTo-Csv) to the console. This will make it easy for the calling script to handle the returned data.

Line 34: Instead of hard-coding specific values in the script we send to the target station, we use variable substitution to fill in the values at the moment we send the script to the target station. This is done through the ExpandString method.

Line 38: This show how easy it is to receive the result when we have the data converted into a well-known format (CSV in this case).

The script results in the following. Instead of Format-Table we could have used any other cmdlet that sends the result to a file, or as the body of an email...

The same script runs in the same way on Ubuntu and Photon. And in both cases produces the same result.

Your Existing Scripts

A question I heard several times when I presented the concept of 'cattle' stations at conferences, was "What will happen with my existing scripts?".

As it turns out, you will not have to do a lot. In most cases, you will able to just take your script and with a bit of PowerShell magic, you can use it in this setup.

Let's take this simple script as an example. And yes, this is exactly the same code as ran before. But we're demonstrating the concept.

We save the following lines as Sample-PCLI-Script.ps1, and it is representative for all your existing scripts.

Get-VMHost | Select Name, Version, PowerState, NumCpu, MemoryTotalGB |
ConvertTo-Csv -NotypeInformation -UseCulture

We now need to use a slightly different script to submit this to our target station.

$vmName = 'ubuntubionicps'

$prefix = @'
$ProgressPreference = "SilentlyContinue"
if(-not (Get-Module -Name 'VMware.PowerCLI' -ListAvailable)){
    Install-Module -Name VMware.PowerCLI -Scope CurrentUser -AllowClobber -Force | Out-Null
}
$WarningPreference = 'SilentlyContinue'
$sConf = @{
    InvalidCertificateAction = 'Ignore'
    ParticipateInCeip = $false
    DisplayDeprecationWarnings = $false
    Scope = 'User'
    Confirm = $false
}
Set-PowerCLIConfiguration @sConf | Out-Null
Connect-VIServer -Server $vcsaName -User $user -Password $pswd | Out-Null
'@

$code = Get-Content -Path .\Sample-PCLI-Script.ps1

$suffix = @'
Disconnect-VIServer -Server $vcsaName -Confirm:$false
'@

$vcsaName = 'vcsa.local.lab'
$credVCSA = Get-VICredentialStoreItem -Host $vcsaName
$user = $credVCSA.User
$pswd = $credVCSA.Password

$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))

$sInvoke = @{
    VM = Get-VM -Name $vmName
    GuestCredential = $cred
    ScriptType = 'powershellv6'
    ScriptText = ($ExecutionContext.InvokeCommand.ExpandString($prefix) + "n$coden" + $ExecutionContext.InvokeCommand.ExpandString($suffix))
    NoIPinCert = $true
}
$result = Invoke-VMScriptPlus @sInvoke
$result.ScriptOutput
$result.ScriptOutput | ConvertFrom-Csv -UseCulture | Format-Table -AutoSize

Annotations

Line 3-18: Under the $prefix variable we store the 'fixed' code that prepares the environment. Just as the same part in the previous example.

Line 20: This is where we read our existing script into a variable.

Line 22-24: The $suffix variable contains the 'fixed' closing part of the code.

Line 38: We compose the ScriptText from the three variables $preifx, $code and $suffix. Note that for the $code variable we don't need to do any variable substitution.

Will this work for all your existing scripts?

Probably not, you will need to verify if there are for example any Connect- and/or DisConnect-VIServer lines in your script.

Infile and Outfile

In the previous examples, we always send the result back over the standard output. This is not a feasible solution in all cases. What for example when your script produces multiples outputs? And what when you need to read data as input for your script?

For all those reasons I added the InFile and OutFile parameters to Invoke-VMScriptPlus. For more details see my Invoke-VMScriptPLus V3 post.

The following demonstrates, in a simple way, how you can use those InFile and OutFile parameters to run your scripts on a 'cattle' station.

This is our original script, saved as Sample-PCLI-Script4.ps1.

$esxName = Import-Csv -Path .\esxnames.csv | Select -ExpandProperty Name
Get-VMHost -Name $esxName | Select Name, NumCpu, Version |
Export-Csv -Path .\report.csv -UseCulture -NoTypeInformation

Write-Host "Script completed"

The esxnames.csv file contains the following.

Our calling script. Notice the use of the InFile and OutFile parameters on lines 40 and 41.

$vmName = 'ubuntubionicps'

$prefix = @'
$ProgressPreference = "SilentlyContinue"
if(-not (Get-Module -Name 'VMware.PowerCLI' -ListAvailable)){
    Install-Module -Name VMware.PowerCLI -Scope CurrentUser -AllowClobber -Force | Out-Null
}
$WarningPreference = 'SilentlyContinue'
$sConf = @{
    InvalidCertificateAction = 'Ignore'
    ParticipateInCeip = $false
    DisplayDeprecationWarnings = $false
    Scope = 'User'
    Confirm = $false
}
Set-PowerCLIConfiguration @sConf | Out-Null
Connect-VIServer -Server $vcsaName -User $user -Password $pswd | Out-Null
'@

$code = Get-Content -Path .\Sample-PCLI-Script4.ps1

$suffix = @'
Disconnect-VIServer -Server $vcsaName -Confirm:$false
'@

$vcsaName = 'vcsa.local.lab'
$credVCSA = Get-VICredentialStoreItem -Host $vcsaName
$user = $credVCSA.User
$pswd = $credVCSA.Password

$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))

$sInvoke = @{
    VM = Get-VM -Name $vmName
    GuestCredential = $cred
    ScriptType = 'powershellv6'
    ScriptText = ($ExecutionContext.InvokeCommand.ExpandString($prefix) + "n$($code -join "n")n" + $ExecutionContext.InvokeCommand.ExpandString($suffix))
    GuestOSType = 'Linux'
    InFile = '.\esxnames.csv'
    OutFile = 'report.csv'
    NoIPinCert = $true
}
$result = Invoke-VMScriptPlus @sInvoke
$result.ScriptOutput

The output from our script is now just a message we wrote in the original script.

But the result can now be found in the file report.csv.

Conclusion

I hope the previous sections have shown that it doesn't require rocket science to run your scripts on instances deployed with the help of cloud-init.

Will the above methods always work for your scripts?

Probably not, but it shouldn't be too difficult to make them work.

I'm always interested to hear about cases that at first can't be run on 'cattle' stations. If you think you have such a case, let me know through the comments.

Enjoy!

The main reason to use Photon OS is that it is open-sourced, it has a small footprint and it is optimised for VMware vSphere.

Some Background

Does this mean that Photon OS is the ideal guest OS for our ‘cattle‘ stations where we want to run our PowerShell scripts?

Not really, the major flaw with Photon OS, imho, is that the available packages are limited and rather infrequently maintained.

In fact, until recently the available PowerShell package was an outdated, not supported anymore version. But the worst part of the deal, it didn’t even allow to perform an Install-Module. Which, again imho, is a basic concept in PowerShell. And which didn’t allow me to install, for example, VMware PowerCLI on a Photon station.

But then recently this happened!

The available PowerShell v6 TDNF package was upgraded to a currently supported version, and more importantly, the Import-Module cmdlet  worked! 

Allowing us to install VMware PowerCLI on a Photon instance.

Is this the end of all the PowerShell woes on Photon OS?

Not really! The PowerShell version in this TDNF package will also run out of support in a not so far future and a PowerShell TDNF package upgrade is (still) not automated and somewhat of a black art.

But for now, let’s be happy with what we have and ‘cloud-init’ our Photon ‘cattle’ station!

Issues and some Solutions

No UserData Property

While the Ubuntu cloud image, as we saw in Part 1 of the series, has the option to pass the user-data to cloud-init via an OVF property, this feature is unfortunately not available in the Photon OVA.

There are some solutions available.

• Roll your own OVA image
• Use the NoCloud option, which means creating an ISO containing the user-data and attaching this ISO to the instance before cloud-init runs.

Since I prefer to use the ‘standard’ OVA, I opt for creating such a seed ISO.

The Root Password

This is somewhat of a chicken-and-the-egg dilemma.

In the Photon OVA image, the root account is set to force a password change on first use. But unfortunately, this also disallows setting the root password via the user-data. See Issue#931 for more details.

Again, there are two options available.

• Roll your own OVA image
• Use a two-stage process, wherein the first stage we use William Lam‘s Set-VMKeystrokes function. Then in the second stage, we attach the seed ISO and start the instance. And thus kicking off the cloud-init process.

And again, since I prefer to use the ‘standard’ OVA, I opt for the Set-VMKeystrokes solution.

Install-CloudInitVM Revisited

function Install-CloudInitVM
{
  <#
.SYNOPSIS
  Deploy a VM from an OVA file and use cloud-init for the configuration
  .DESCRIPTION
  This function will deploy an OVA file.
  The function transfer the user-data to the cloud-init process on the VM with
  one of the OVF properties.
.NOTES
  Author:  Luc Dekens
  Version:
  1.0 05/12/19  Initial release
  1.1 07/12/19  Added Photon deployment
                Added seed ISO option
.PARAMETER OvaFile
  Specifies the path to the OVA file
.PARAMETER VmName
  The displayname of the VM
.PARAMETER ClusterName
  The cluster onto which the VM shall be deployed
.PARAMETER DsName
  The datastore on which the VM shall be deployed
.PARAMETER PgName
  The portgroupname to which the VM shall be connected
.PARAMETER CloudConfig
  The path to the YAML file containing the user-data
.PARAMETER CloudConfigISO
  The ISO seed file, containing the meta-data and user-data
.PARAMETER Credential
  The credentials for a user in the VM's guest OS
.PARAMETER Photon
  Switch to indicate a Photon is deployed. This will add an
  additional step to reset the root password before starting
  the actual cloud-init based deployment
.EXAMPLE
  $sCloudInitVM = @{
    OvaFile = '.\bionic-server-cloudimg-amd64.ova'
    VmName = $vmName
    ClusterName = $clusterName
    DsName = $dsName
    PgName = $pgName
    CloudConfig = '.\user-data.yaml'
    Credential = $cred
  }
  Install-CloudInitVM @sCloudInitVM
.EXAMPLE
  $sCloudInitVM = @{
    OvaFile = '.\photon-hw13_uefi-3.0-9355405.ova'
    VmName = $vmName
    ClusterName = $clusterName
    DsName = $dsName
    PgName = $pgName
    CloudConfigISO = '.\seed.iso'
    Credential = $cred
    Photon = $true
  }
  Install-CloudInitVM @sCloudInitVM
#>

  [cmdletbinding()]
  param(
    [string]$OvaFile,
    [string]$VmName,
    [string]$ClusterName,
    [string]$DsName,
    [string]$PgName,
    [Parameter(ParameterSetName = 'YAML')]
    [string]$CloudConfig,
    [Parameter(ParameterSetName = 'SeedISO')]
    [string]$CloudConfigISO,
    [PSCredential[]]$Credential,
    [Switch]$Photon
  )

  $waitJob = (Get-Command -Name .\Wait-Job.ps1 ).ScriptBlock
  if ($PSCmdlet.ParameterSetName -eq 'YAML')
  {
    $userData = Get-Content -Path $CloudConfig -Raw
  }

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting deployment of $vmName"
  if ($Photon)
  {
    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - This is a Photon deployment"
  }

  $start = Get-Date

  $vm = Get-VM -Name $vmName -ErrorAction SilentlyContinue -Verbose:$false
  if ($vm)
  {
    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Cleaning up"
    if ($vm.PowerState -eq 'PoweredOn')
    {
      Stop-VM -VM $vm -Confirm:$false -Verbose:$false | Out-Null
    }
    Remove-VM -VM $vm -DeletePermanently -Confirm:$false -Verbose:$false
  }

  $ovfProp = Get-OvfConfiguration -Ovf $ovaFile -Verbose:$false
  if (-not $Photon)
  {
    $ovfProp.Common.user_data.Value = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($userData))
    $ovfProp.NetworkMapping.VM_Network.Value = $pgName
  }
  else
  {
    $ovfProp.NetworkMapping.None.Value = $pgName
  }
  $sApp = @{
    Source = $ovaFile
    Name = $vmName
    Datastore = Get-Datastore -Name $dsName -Verbose:$false
    DiskStorageFormat = 'Thin'
    VMHost = Get-Cluster -Name $clusterName -Verbose:$false | Get-VMHost  -Verbose:$false | Get-Random
    OvfConfiguration = $ovfProp
    Verbose = $false
  }
  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Importing OVA"
  $vm = Import-VApp @sApp

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting the VM"
  Start-VM -VM $vm -Confirm:$false -RunAsync -Verbose:$false | Out-Null

  if ($Photon)
  {
    while (-not $vm.ExtensionData.Guest.GuestOperationsReady)
    {
      Start-Sleep 2
      $vm.ExtensionData.UpdateViewData('Guest')
    }

    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Resetting password"

    $PswdIsReset = $false
    while (-not $PswdIsReset)
    {
      Set-VMKeystrokes -VMName $VM.Name -StringInput 'root' -ReturnCarriage $true | Out-Null
      Start-Sleep 1
      Set-VMKeystrokes -VMName $VM.Name -StringInput 'changeme' -ReturnCarriage $true | Out-Null
      Start-Sleep 1
      Set-VMKeystrokes -VMName $VM.Name -StringInput 'changeme' -ReturnCarriage $true | Out-Null
      Start-Sleep 1
      Set-VMKeystrokes -VMName $VM.Name -StringInput 'Welcome2019!' -ReturnCarriage $true | Out-Null
      Start-Sleep 1
      Set-VMKeystrokes -VMName $VM.Name -StringInput 'Welcome2019!' -ReturnCarriage $true | Out-Null
      Start-Sleep 1
      Set-VMKeystrokes -VMName $VM.Name -StringInput "exit" -ReturnCarriage $true | Out-Null

      Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Testing new password"

      $sInvoke = @{
        VM = $vm
        ScriptType = 'bash'
        ScriptText = 'whoami'
        GuestUser = $Credential.GetNetworkCredential().UserName
        GuestPassword = $Credential.GetNetworkCredential().Password
        Verbose = $false
      }
      try
      {
        Invoke-VMScript @sInvoke -ErrorAction Stop | Out-Null
        $PswdIsReset = $true
      }
      catch [VMware.VimAutomation.ViCore.Types.V1.ErrorHandling.InvalidGuestLogin]
      {
        Start-Sleep -Seconds 1
      }
      catch
      {
        Write-Error "Exception $($error[0].Exception.GetType().FullName)"
        return
      }
    }

    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Shutdown VM"

    Shutdown-VMGuest -VM $vm -Confirm:$false -Verbose:$false | Out-Null
    while ($vm.ExtensionData.Runtime.PowerState -ne 'poweredOff')
    {
      Start-Sleep 2
      $vm.ExtensionData.UpdateViewData('Runtime.PowerState')
    }

    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Attaching seed ISO"

    $isoName = Split-Path -Path $CloudConfigISO -Leaf
    $isoFile = "DS:\$($vm.ExtensionData.Summary.Config.VmPathName.Split(' ')[1].Split('/')[0])\$($isoName)"
    Get-PSDrive -Name DS -ErrorAction SilentlyContinue -Verbose:$false | Remove-PSDrive -Confirm:$false -Verbose:$false
    New-PSDrive -Location (Get-Datastore -VM $vm) -Name DS -PSProvider VimDatastore -Root '\' -Verbose:$false | Out-Null
    Get-Item -Path $CloudConfigISO |
    Copy-DatastoreItem -Destination $isoFile -Force -Confirm:$false -Verbose:$false
    $isoDSPath = $vm.ExtensionData.Summary.Config.VmPathName.Replace("$vmName.vmx", $isoName)
    Get-CDDrive -VM $vm -Verbose:$false | Set-CDDrive -IsoPath $isoDSPath -StartConnected:$true -Confirm:$false -Verbose:$false | Out-Null

    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting the VM"
    Start-VM -VM $vm -Confirm:$false -Verbose:$false | Out-Null
  }

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Waiting for cloud-init to finish"

  $User = $Credential.GetNetworkCredential().UserName
  $Password = $Credential.GetNetworkCredential().Password

  $sJob = @{
    Name = 'WaitForCloudInit'
    ScriptBlock = $waitJob
    ArgumentList = $vm.Name, $User, $Password, $global:DefaultVIServer.Name, $global:DefaultVIServer.SessionId
  }
  Start-Job @sJob | Receive-Job -Wait

  if ($Photon)
  {
    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Restart OS"
    $vmGuest = Stop-VMGuest -VM $vm -Confirm:$false -Verbose:$false
    while ($vmGuest.VM.ExtensionData.Runtime.PowerState -ne 'poweredOff')
    {
      Start-Sleep 2
      $vmGuest.VM.ExtensionData.UpdateViewData('Runtime.PowerState')
    }
    $vm = Start-VM -VM $vmName -Confirm:$false -Verbose:$false
    while ($vm.PowerState -ne 'PoweredOn')
    {
      Start-Sleep 2
      $vm = Get-VM -Name $VmName -Verbose:$false
    }
    while (-not $vm.ExtensionData.Guest.GuestOperationsReady)
    {
      Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Waiting for GuestOperations ready"
      Start-Sleep 2
      $vm.ExtensionData.UpdateViewData('Guest')
    }
    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Eject CDROM"
    $sInvoke = @{
      VM = $vm
      ScriptType = 'bash'
      ScriptText = 'eject cdrom'
      GuestUser = $Credential.GetNetworkCredential().UserName
      GuestPassword = $Credential.GetNetworkCredential().Password
      Verbose = $false
    }
    Invoke-VMScript @sInvoke | Out-Null
    Get-CDDrive -VM $vm -Verbose:$false | Set-CDDrive -NoMedia -Confirm:$false -Verbose:$false | Out-Null
  }

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Deployment complete"
  Write-Verbose "`nDeployment took $([math]::Round((New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds,0)) seconds"
}

Annotations

The v1.0 version of this function has already been annotated in Part 1 of this series. Those annotations will not be repeated here.

Line 68-71: To distinguish between the way the user-data is passed, the function now has two mutually exclusive parameters, CloudConfig and CloudConfigIso.

Line 73: Since Photon requires some special actions, the function now has a switch, obviously named Photon, that specifies if the cloud-init deployment is intended for a Photon instance or not.

Line 77-80: When the user-data is provided as a YAML file, the function reads the file.

Line 83-86: A small reminder in the verbose output that the function will handle this as a Photon deployment.

Line 102-110: The Ubuntu and the Photon OVA files have different OVF properties. This If-Then-Else handles the differences.

Line 126-199: This part of the function handles most of the peculiarities of a Photon deployment.

Line 136-175: This part handles the reset of the root password. As mentioned earlier, it uses William’s Set-VMKeystrokes function. I experienced intermittent race conditions in sending the keystrokes. Hence the loop with the retries. As a test, to check if the root password was reset, the command ‘whoami‘ is executed trough Invoke-VMScript on the instance.

Line 179-184: Before attaching the seed ISO, the Photon OS is shutdown gracefully. In a While-loop the script test until the instance is actually powered off.

Line 188-195: The seed ISO is copied to the VM’s folder, and then attached.

Line 198: The VM is started with the seed ISO attached. As a result, cloud-init will now use the YAML files it finds on the CD drive as datasource.

Line 216-221: Since the cloud-init implementation on Photon is not configured to use the power-state module, the test we used on the Ubuntu instance, can not be used for Photon. As a bypass, the function stops/starts the guest OS on the instance.

Line 235-244: The function uses the stop/start sequence to disconnect the seed ISO. To avoid having a question on the VM, the function first runs the ‘eject cdrom‘ command inside the instance.

The Seed ISO

As mentioned earlier, I prefer to use a seed ISO file to solve the issue of not having an OVF property for the user-data.

To create this seed ISO file, we need a Linux station that has the ‘genisoimage‘ command available. In the following sample script, I use an Ubuntu station just for that purpose.

$workVM = 'UbuntuPS'
$isoName = 'genisoimage-ps.iso'
$isoVolId = 'cidata'
$isoFiles = @{
    'meta-data' = 'meta-data.yaml'
    'user-data' = 'user-data.yaml'
}

$code1 = @'
sudo apt-get install genisoimage -y > null
genisoimage -output $isoName -volid $isoVolId -joliet -rock $($isoFiles.Keys -join ' ')
'@

$workCred = Get-VICredentialStoreItem -Host $workVM

$isoFiles.GetEnumerator() | ForEach-Object -Process {
    Copy-Item -Path $_.Value -Destination $_.Name
}

$sInvoke = @{
    VM = $workVM
    ScriptType = 'Bash'
    ScriptText = $ExecutionContext.InvokeCommand.ExpandString($code1)
    GuestUser = $workCred.User
    GuestPassword = ConvertTo-SecureString -String $workCred.Password -AsPlainText -Force
    GuestOSType = 'Linux'
    Verbose = $true
    NoIPinCert = $true
    InFile = $isoFiles.Keys
    OutFile = $isoName
}
Invoke-VMScriptPlus @sInvoke

Remove-Item -Path @($isoFiles.Keys) -Confirm:$false
Move-Item -Path $isoName -Destination "..\$isoName" -Force -Confirm:$false

Annotations

Line 4-7, 16-18: The script converts (copies) the provided YAML files to standard named YAML files. This allows the script to be used with different versions of the data and only needs a change in the $isoFiles hash table.

Line 32: This script uses the latest version of my Invoke-VMScriptPlus function. The reason is twofold: the bash script is multi-line and the script used the InFile/OutFile parameters to copy the YAML files to the Linux box and the ISO file back to the calling station.

User-Data

The user-data that is used for a Photon based instance is, for now, rather simple and straightforward.

Note the installation of the tdnf PowerShell package (as mentioned earlier in this post) in the runcmd section.

#cloud-config
fqdn: PhotonPS.local.lab
timezone: Europe/Brussels
hostname: PhotonPS
write_files:
    - path: /etc/systemd/network/10-static.network
      permissions: 0644
      content: |
        [Match]
        Name=eth0

        [Network]
        Address=192.168.10.81/24
        Gateway=192.168.10.1
        DNS=192.168.10.2 192.168.10.3
        DHCP=no
        Domains=local.lab
        NTP=192.168.10.2 192.168.10.3
        LinkLocalAddressing=no
        LLDP=true
    - path: /etc/systemd/network/99-dhcp-en.network
      permissions: 0644
      content: |
        [Match]
        Name=e*

        [Network]
        DHCP=no
    - path: /etc/gnutls/default-priorities
      content: |
        SYSTEM=NONE:!VERS-SSL3.0:!VERS-TLS1.0:+VERS-TLS1.1:+VERS-TLS1.2:+AES-128-CBC:+RSA:+SHA1:+COMP-NULL
runcmd:
- tdnf distro-sync
- tdnf update -y
- tdnf install bindutils -y
- tdnf install powershell -y
- tdnf install gnuTLS -y
- touch /etc/gnutls/default-priorities
package_upgrade: true
package_reboot_if_required: true

Meta-data

If you plan on deploying multiple instance, just make sure that the instance-id has a unique value.

instance-id: iid-local01
local-hostname: photonps

Sample Run

For a Photon instance the function is called like this.

$vmName = 'PhotonPS'

# Get credential for logging on to the guest OS
$viCred = Get-VICredentialStoreItem -Host $vmName
$secPassword = ConvertTo-SecureString -String $viCred.Password -AsPlainText -Force
$cred = [Management.Automation.PSCredential]::new($viCred.User, $secPassword)

$sCloudInitVM = @{
  OvaFile = 'D:\Repository\VMware\Photon\photon-hw13_uefi-3.0-9355405.ova'
  VmName = $vmName
  ClusterName = 'cluster'
  DsName = 'vsanDatastore'
  PgName = 'vdPg1'
  CloudConfigISO = '..\genisoimage-ps.iso'
  Photon = $true
  Credential = $cred
  Verbose = $true
}
Install-CloudInitVM @sCloudInitVM

The result of this, since we used the Verbose switch, looks like this.

Again, don’t give too much importance to the total execution time. This was a run in my lab environment, which has limited resources.

On a side-note, notice how the PowerCLI cmdlets used in the function all show this, imho, annoying ‘Finished execution‘ message.
I would love to be able to suppress these verbose messages from the PowerCLI cmdlets. That is why I launched PowerCLI Idea #225 “Get rid of the over-eager verbosity on PowerCLI cmdlets“.

Enjoy!

For the installation of software on the instance we use the user-data YAML file. In the following sections, we discuss some of the more obvious candidates you want to have on your station. Remember that the basic idea of the series is to have a “cattle” station to test/run (and optionally develop) your scripts.

Scripting Packages

PowerShell

Let’s start by installing PowerShell on the instance. This comes down to adding a couple of instructions to the YAML file we used in Part 1.

#cloud-config
hostname: ubuntubionicps
fqdn: ubuntubionicps.local.lab
write_files:
- path: /etc/netplan/50-cloud-init.yaml
  content: |
    network:
     version: 2
     ethernets:
      ens192:
       addresses: [192.168.10.82/24]
       gateway4: 192.168.10.1
       dhcp6: false
       nameservers:
         addresses:
           - 192.168.10.2
           - 192.168.10.3
         search:
           - local.lab
       dhcp4: false
       optional: true
- path: /etc/sysctl.d/60-disable-ipv6.conf
  owner: root
  content: |
    net.ipv6.conf.all.disable_ipv6=1
    net.ipv6.conf.default.disable_ipv6=1
runcmd:
- netplan --debug apply
- sysctl -w net.ipv6.conf.all.disable_ipv6=1
- sysctl -w net.ipv6.conf.default.disable_ipv6=1
- apt-get -y update
- add-apt-repository universe
# PowerShell
- wget https://raw.githubusercontent.com/PowerShell/PowerShell/master/tools/install-powershell.sh
- chmod 755 install-powershell.sh
# PowerShell v6
- ./install-powershell.sh
# PowerShell v7
- ./install-powershell.sh -preview
- apt-get -y clean
- apt-get -y autoremove --purge
timezone: Europe/Brussels
system_info:
  default_user:
    name: default-user
    lock_passwd: false
    sudo: ["ALL=(ALL) NOPASSWD:ALL"]
disable_root: false
ssh_pwauth: yes
users:
  - default
  - name: luc
    gecos: LucD
    lock_passwd: false
    groups: sudo, users, admin
    shell: /bin/bash
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
chpasswd:
  list: |
    default-user:$6$aPp//e2ueP$ETEXcAhAyQuJ4qNCbqxmmSYGZbg2wFwpP/YITvoXdgxwZBnf32drePKi2OIn5fLqtH5pHO03yRdPXK3ToLG6b0
    luc:$6$kW1WwJ2K$M6415du1BZd.qt92SvR6X.RuyDhEZmgR4hz4NcKH9XHn2850Vc6zHpubXM6uUeqMUaJQ740ogROB74gfBEhn9.
    root:$6$Js9CVr06$br9qf0VxuBsdY7Vtg/0pk9jLlycYBDLVsvbKwLDleCK7dSDheOxWaFOWdjkiqSPRrWG./N8V5RgCVwugZGnTc1
  expire: false
package_upgrade: true
package_reboot_if_required: true
power_state:
  delay: now
  mode: reboot
  message: Rebooting the OS
  condition: if [ -e /var/run/reboot-required ]; then exit 0; else exit 1; fi

Annotations

Line 33-39: The instructions to install PowerShell are added to the runcmd section.

Line 33: You can use comments in your YAML file. I highly recommend doing that. It will help others, and probably yourself later on.

Line 34-35: The method uses the install-powershell.sh bash script to install the latest PowerShell v6 and the latest PowerShell v7 Preview. I personally prefer this method vs the apt-get method, since it is globally usable on multiple Linux distributions. The install-powershell.sh script determines on which distro it is running, and launches the appropriate command(s).

Result

Git & repo cloning

For your scripts, you, of course, use a repository. That means you will need to configure git, and a way to clone repositories to your instance.

The following extract of the YAML file only shows the lines in the runcmd section.

runcmd:
- netplan --debug apply
- sysctl -w net.ipv6.conf.all.disable_ipv6=1
- sysctl -w net.ipv6.conf.default.disable_ipv6=1
- apt-get -y update
- add-apt-repository universe
# PowerShell
- wget https://raw.githubusercontent.com/PowerShell/PowerShell/master/tools/install-powershell.sh
- chmod 755 install-powershell.sh
# PowerShell v6
- ./install-powershell.sh
# PowerShell v7
- ./install-powershell.sh -preview
# Git
- git config --global user.name "LucD"
- git config --global user.email "lucd@lucd.info"
- mkdir /home/luc/vCheck-vSphere
- chown luc:luc /home/luc/vCheck-vSphere
- git clone https://github.com/alanrenouf/vCheck-vSphere /home/luc/vCheck-vSphere
- apt-get -y clean
- apt-get -y autoremove --purge

Annotations

Line 14-16: Since git is by default installed in the Ubuntu 18.04 LTS Cloud Image OVA file, we only need to configure some settings. In this we update some configuration settings, primarily to be able to correct sign off commits.

Line 17-19: As part of the deployment process through cloud-init, you can clone specific repositories to the instance. In these lines I used Alan Renouf‘s vCheck-vSphere repository as an example.

Result

Going GUI

There might be occasions when you need to provide a temporary station with a GUI. That can also be done with cloud-init’s user-data YAML file.

The Desktop

The choice of which desktop and which desktop manager is completely up to you. Just be aware that some of these might require more resources to be assigned to the instance.

The following sample is intended for an Ubuntu 18.04 LTS instance. Again this extract only shows the runcmd section of the YAML file.

# Install GUI
- apt-get install -y tasksel
- tasksel install lubuntu-core
# ==> know issue
- apt-get --purge remove -y light-locker
- service lightdm start
# Install RDP
- apt-get install -y apt-transport-https
- apt-get install -y xrdp
- systemctl enable xrdp

Annotations

Line 1-10: In this example, I use lubuntu as the desktop. Primarily because it is fast and resource-friendly.

Line 2: We use tasksel to install the desktop, so we have to make sure that it is available.

Line 5: There is a known issue with the light-locker package. Until a final fix is available, the easiest solution is to just uninstall it. If you absolutely need a screen-saver, install one of the alternatives. Like for example xscreensaver.

Line 6: Next we start the display manager lightdm.

Line 7-10: We need a way to connect to the station we are deploying. In this example, I opted for RDP as my protocol.

Line 8: The apt-transport-https package is a pre-requisite.

Line 10: Configure the xrdp service to automatically start at the boot.

Visual Studio Code

Now that we have a GUI, we can use several GUI based applications. For working with PowerShell, the Visual Studio Code editor with the PowerShell extension is an obvious choice.

# Install Visual Studio Code
- add-apt-repository "deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main"
- apt-get update
- apt-get install -y code
- mkdir /home/luc/.vscode
- chown luc:luc /home/luc/.vscode
- code --install-extension ms-vscode.powershell --user-data-dir /home/luc/.vscode/ --extensions-dir /home/luc/.vscode/extensions/
- code --install-extension eamodio.gitlens --user-data-dir /home/luc/.vscode --extensions-dir /home/luc/.vscode/extensions/

Annotations

Line 2-4: The standard installation procedure for Code on a Linux platform.

Line 5-8: This installs the extensions for PowerShell and GitLens. Remember that the cloud-init stages run under the root account, if you need to install packages for a user, make sure to change the owner.

Result

Conclusion

We have now set the stage to actually start using these instances. In one of the next posts in this cloud-init series, we will show how to run scripts on these stations.

If you have suggestions or questions about specific additions on these instances, feel free to use the comments.

Enjoy!

In this first part, we will introduce cloud-init and how you can use it from your PowerShell/PowerCLI scripts. Since the Ubuntu distribution is very popular, on-premises and in the cloud, this introduction will focus on that distro to demonstrate the concept. In the following parts, we will tackle Photon, containers and how to run your scripts on these stations.

Before diving into the technical stuff, first a reminder why we should treat our servers as cattle.

A list with some of the major arguments (at least for me).

• No maintenance required when using the latest distributions. Meaning no long-running updates and security patches before usage.
• No “less than 5% used” machines anymore in your environment.
• No risk of system tattooing from any previous usage.
• An easy way to test new versions of your scripts, the OS, PowerShell and PowerCLI.

Ingredients

To use cloud-init, the requirements, in the cases we will handle here, are minimal.

• An OVA image that is configured to use cloud-init
  • Several Linux distributions nowadays have such a ‘cloud’ image: Ubuntu, Photon…
• A datasource, which is the configuration data provided by the user to the cloud-init process, and which defines how the resulting station will be configured.
• A cloud, in it’s broadest sense, to run the VM. From the cloud-init documentation “… all major public cloud providers, provisioning systems for private cloud infrastructure, and bare-metal installations.”
• A script to trigger and control it all.

Canonical, the company behind Ubuntu, maintains a document, named Cloud Instance Initialisation with cloud-init, that has a schematic that captures the cloud-init process perfectly.

If we annotate and update this schema for the usage we have in mind, it becomes like this.

The OVA File

Since our target cloud platform in this series is a vSphere environment, we will use OVA files as the source for the VM(s) we are going to deploy.

The Ubuntu OVA image that we will be using in this post is for Ubuntu Server 18.04 LTS, aka Bionic Beaver.

This OVA image allows using the OVFProperties as the datasource. Note that the user-data we pass in this way needs to be Base64 encoded.

The User-data

As mentioned in the previous section, in this Ubuntu image we will use the OVFProperties as the datasource for our user-data.

The content of the user-data is provided to the script as a YAML file. The syntax for such a file and the available is defined in the cloud-init Documentation.

The following is the sample YAML file we will use in the rest of this post.

#cloud-config
hostname: ubuntubionic
fqdn: ubuntubionic.local.lab
write_files:
- path: /etc/netplan/50-cloud-init.yaml
  content: |
    network:
     version: 2
     ethernets:
      ens192:
       addresses: [192.168.10.79/24]
       gateway4: 192.168.10.1
       dhcp6: false
       nameservers:
         addresses:
           - 192.168.10.2
           - 192.168.10.3
         search:
           - local.lab
       dhcp4: false
       optional: true
- path: /etc/sysctl.d/60-disable-ipv6.conf
  owner: root
  content: |
    net.ipv6.conf.all.disable_ipv6=1
    net.ipv6.conf.default.disable_ipv6=1
runcmd:
- netplan --debug apply
- sysctl -w net.ipv6.conf.all.disable_ipv6=1
- sysctl -w net.ipv6.conf.default.disable_ipv6=1
- apt-get -y update
- add-apt-repository universe
- apt-get -y clean
- apt-get -y autoremove --purge
timezone: Europe/Brussels
system_info:
  default_user:
    name: default-user
    lock_passwd: false
    sudo: ["ALL=(ALL) NOPASSWD:ALL"]
disable_root: false
ssh_pwauth: yes
users:
  - default
  - name: luc
    gecos: LucD
    lock_passwd: false
    groups: sudo, users, admin
    shell: /bin/bash
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
chpasswd:
  list: |
    default-user:$6$aPp//e2ueP$ETEXcAhAyQuJ4qNCbqxmmSYGZbg2wFwpP/YITvoXdgxwZBnf32drePKi2OIn5fLqtH5pHO03yRdPXK3ToLG6b0
    luc:$6$kW1WwJ2K$M6415du1BZd.qt92SvR6X.RuyDhEZmgR4hz4NcKH9XHn2850Vc6zHpubXM6uUeqMUaJQ740ogROB74gfBEhn9.
    root:$6$Js9CVr06$br9qf0VxuBsdY7Vtg/0pk9jLlycYBDLVsvbKwLDleCK7dSDheOxWaFOWdjkiqSPRrWG./N8V5RgCVwugZGnTc1
  expire: false
package_upgrade: true
package_reboot_if_required: true
power_state:
  delay: now
  mode: reboot
  message: Rebooting the OS
  condition: if [ -e /var/run/reboot-required ]; then exit 0; else exit 1; fi

Annotations

Line 1: the user-data file always has to start with the line ‘# cloud-config’

Line 2-3: These lines follow the regular YAML syntax of key-value entries. The lines defines the hostname and the FQDN that the instance will get.

Line 4-26: The write-files section instructs cloud-init to create files, with the specified properties and content on the instance. In this example file the files are used to define a static IPv4 address and to disable IPv6. Which files need to be used is dependent on the OS that runs in the VM. Note that this example will use netplan for the network configuration.

Line 27-34: The runcmd section defines commands that will run on the instance during the first boot of the guest OS. In this example the commands are used to activate the network (it uses netplan), set some variables and set and update the application repositories.

Line 35: This line defines which timezone needs to be configured.

Line 36-40: An Ubuntu image comes configured with a default user. In the system_info section, the settings for this default user are specified.

Line 43-50: The users section allows to define additional users on the instance. Note that the default user needs to come first in this list.

Line 51-56: The chpasswd allows you to change and/or set the passwords for the users. The required format and how these entries are created will be discussed in one of the following sections.

Line 57-58: These lines request an upgrade of the packages present on the instance. And perform a reboot when one of the package upgrades would require that.

Line 59-63: The power-state section instructs cloud-init in which state the instance shall be left after cloud-init completes. In this example, the system is rebooted on condition that a specific file exists. This file is created by cloud-init when it has decided that a reboot of the instance is required.

Password encoding

In user-data, there are several places where a password can be provided. The value needs to be the hash of the password, not the password itself. With the help of an existing Linux box (UbuntuWork in this case), I use the following script to create such SHA-512 encoded password hashes. The script uses the mkpasswd command to generate the hash.

Automation rules!

function Get-PasswordHash
{
    param(
        [String]$VMName,
        [String]$GuestUser,
        [String]$GuestPassword,
        [String]$Password
    )

    $sInvoke = @{
        VM = $VMName
        ScriptType = 'bash'
        ScriptText = "mkpasswd -m SHA-512 $Password"
        GuestUser = $GuestUser
        GuestPassword = $GuestPassword
    }
    (Invoke-VMScript @sInvoke).ScriptOutput.Trim("n")
}

$sHash = @{
    VM = 'UbuntuWork'
    GuestUser = 'root'
    GuestPassword = 'VMware1!'
    Password = 'VMware1!'
}
Get-PasswordHash @sHash

This returns something like this.

A word of warning, such hashed password are not super-safe. A decent password cracker with sufficient compute power and a good wordlist can most probably crack this in less than one minute.

The Script

function Install-CloudInitVM
{
  <#
.SYNOPSIS
  Deploy a VM from an OVA file and use cloud-init for the configuration
  .DESCRIPTION
  This function will deploy an OVA file.
  The function transfer the user-data to the cloud-init process on the VM with
  one of the OVF properties.
.NOTES
  Author:  Luc Dekens
  Version:
  1.0 05/12/19  Initial release
.PARAMETER OvaFile
  Specifies the path to the OVA file
.PARAMETER VmName
  The displayname of the VM
.PARAMETER ClusterName
  The cluster onto which the VM shall be deployed
.PARAMETER DsName
  The datastore on which the VM shall be deployed
.PARAMETER PgName
  The portgroupname to which the VM shall be connected
.PARAMETER CloudConfig
  The path to the YAML file containing the user-data
.PARAMETER Credential
  The credentials for a user in the VM's guest OS
.EXAMPLE
  $sCloudInitVM = @{
    OvaFile = '.\bionic-server-cloudimg-amd64.ova'
    VmName = $vmName
    ClusterName = $clusterName
    DsName = $dsName
    PgName = $pgName
    CloudConfig = '.\user-data.yaml'
    Credential = $cred
  }
  Install-CloudInitVM @sCloudInitVM
  #>

  [cmdletbinding()]
  param(
    [string]$OvaFile,
    [string]$VmName,
    [string]$ClusterName,
    [string]$DsName,
    [string]$PgName,
    [string]$CloudConfig,
    [PSCredential[]]$Credential
  )

  $waitJob = (Get-Command -Name .\Wait-Job.ps1).ScriptBlock
  $userData = Get-Content -Path $CloudConfig -Raw

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting deployment of $vmName"

  $start = Get-Date

  $vm = Get-VM -Name $vmName -ErrorAction SilentlyContinue
  if ($vm)
  {
    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Cleaning up"
    if ($vm.PowerState -eq 'PoweredOn')
    {
      Stop-VM -VM $vm -Confirm:$false | Out-Null
    }
    Remove-VM -VM $vm -DeletePermanently -Confirm:$false
  }

  $ovfProp = Get-OvfConfiguration -Ovf $ovaFile
  $ovfProp.Common.user_data.Value = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($userData))
  $ovfProp.NetworkMapping.VM_Network.Value = $pgName

  $sApp = @{
    Source = $ovaFile
    Name = $vmName
    Datastore = Get-Datastore -Name $dsName
    DiskStorageFormat = 'Thin'
    VMHost = Get-Cluster -Name $clusterName | Get-VMHost | Get-Random
    OvfConfiguration = $ovfProp
  }
  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Importing OVA"
  $vm = Import-VApp @sApp

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting the VM"
  Start-VM -VM $vm -Confirm:$false -RunAsync | Out-Null

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Waiting for cloud-init to finish"

  $User = $Credential.GetNetworkCredential().UserName
  $Password = $Credential.GetNetworkCredential().Password

  $sJob = @{
    Name = 'WaitForCloudInit'
    ScriptBlock = $waitJob
    ArgumentList = $vm.Name, $User, $Password, $global:DefaultVIServer.Name, $global:DefaultVIServer.SessionId
  }
  Start-Job @sJob | Receive-Job -Wait

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Deployment complete"

  Write-Verbose "nDeployment took $([math]::Round((New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds,0)) seconds"
}

Annotations

Line 52: To find out when cloud-init has finished it's run, the function uses a background job. This background job is stored in a separate .ps1 file. More on the content and purpose of that 'wait' script later.

Line 53: The user-data is read from the YAML file. It is important to use the Raw switch, otherwise, we would lose the line separators.

Line 60-68: If a VM with the same Displayname is already present, it will be stopped (when powered on) and removed.

Line 70-71: The user-data is assigned to the Common.user_data property of the OVF properties. The user-data needs to be converted to base64.

Line 72: The Ubuntu OVA allows to pass the Portgroup to which the VM shall be connected.

Line 74-83: The VM is installed from the OVA with the Import-VApp cmdlet.

Line 86: The VM is powered on. During the boot process, the first phase of cloud-init will run.

Line 90-98: The function uses a background job to check if the cloud-init process has completed. More on that later.

The Wait job

I decided to store the WaitJob script in a separate .ps1 file. Primarily because I can easily reuse it this way and it makes the code of scripts that use shorter.

The Wait Job is started as a background job. This allows the calling code to simply use the Wait-Job cmdlet.

[CmdletBinding()]
param(
    [string]$vmName,
    [string]$user,
    [string]$pswd,
    [string]$vcsaName,
    [string]$SessionId
)

$sleepTime = 5
Connect-VIServer -Server $vcsaName -Session $SessionId | Out-Null
$notFinished = $true
while ($notFinished)
{
    Try
    {
        $vm = Get-VM -Name $vmName -ErrorAction Stop
        while ($vm.PowerState -ne 'PoweredOn' -and -not $vm.ExtensionData.Guest.GuestOperationsReady)
        {
            Start-Sleep -Seconds $sleepTime
        }
        $fileExist = $false
        while (-not $fileExist)
        {
            $sInvoke = @{
                VM = $vm
                ScriptType = 'bash'
                ScriptText = '[ -f /var/lib/cloud/instance/boot-finished ] && echo "File exist"'
                GuestUser = $user
                GuestPassword = $pswd
                ErrorAction = 'Stop'
            }
            try
            {
                $result = Invoke-VMScript @sInvoke
                $fileExist = [Boolean]$result.ScriptOutput
            }
            catch
            {
                Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') Exception:tId: $($error[0].Exception.ErrorId)  Category: $($error[0].Exception.ErrorCategory)"
                Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff')ttLine: $($error[0].InvocationInfo.Line)"
            }
        }
        $sInvoke = @{
            VM = $vm
            ScriptType = 'bash'
            ScriptText = 'cat /var/lib/cloud/instance/boot-finished'
            GuestUser = $user
            GuestPassword = $pswd
        }
        $result = Invoke-VMScript @sInvoke
        $result.ScriptOutput
        $notFinished = $false
    }
    catch
    {
        Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') Exception:tId: $($error[0].Exception.ErrorId)  Category: $($error[0].Exception.ErrorCategory)"
        Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff')ttLine: $($error[0].InvocationInfo.Line)"
    }
    Start-Sleep -Seconds $sleepTime
}

Annotations

Line 7 + 11: In a background job we do not inherit any open connections to a vSphere Server. A script can use the SessionId of an open connection in the calling script, to open a connection, without having to provide credentials.

Line 18: Before using the call to Invoke-VMScript, the Wait Job makes sure the VM and the VMware Tools in there, are ready to receive such a call.

Line 28: Inside the guest OS, the function uses the simple bash expression to test for the presence of a file. The file /var/lib/cloud/instance/boot-finished is created by cloud-init when the process completes.

Line 44-52: The Wait Job function returns the content of the file to the caller.

Sample Run

With the above code, we have everything in place to deploy a VM from an Ubuntu OVA, with the guest OS configuration is done through cloud-init, based on a YAML file.

A typical call could look something like this.

$vmName = 'UbuntuBionic'

# Get credential for logging on to the guest OS

$viCred = Get-VICredentialStoreItem -Host $vmName
$secPassword = ConvertTo-SecureString -String $viCred.Password -AsPlainText -Force
$cred = [Management.Automation.PSCredential]::new($viCred.User, $secPassword)

$sCloudInitVM = @{
  OvaFile = '.\bionic-server-cloudimg-amd64.ova'
  VmName = $vmName
  ClusterName = 'cluster'
  DsName = 'vsanDatastore'
  PgName = 'vdPg1'
  CloudConfig = '.\user-data-bionic.yaml'
  Credential = $cred
  Verbose = $true
}
Install-CloudInitVM @sCloudInitVM

With the Verbose switch set to $true, the output of this call would look like this.

I highlighted with red boxes the verbose messages from the function. The text in the green box is the content of the /var/lib/cloud/instance/boot-finished file.

Do not look too much at the timings. This run happened in my lab environment, which has limited resources. The main reason for showing this is to demonstrate how easy such a cloud-init based deployment can be incorporated in your pipelines.

This concludes Part 1 in the cloud-init series. It provides some functions and scripts to deploy a VM, starting from an OVA file, with the configuration of the guest OS done through cloud-init.

In the upcoming parts in this series, I will show some more advanced Ubuntu deployments, show how cloud-init can be used with Photon and show how you can use these deployed VMs as your 'cattle'.

Enjoy!

The original Invoke-VMScriptPlus post, and the addition of PS Core support, described in the Invoke-VMScriptPlus v2 post, keep being some of my most read posts. Time for another update.

In this v3 version, I introduce some new features to the function.

• PSv6 and PSv7 support
• Use files (input and output) from within your scripts
• improved sudo support

Update July 2nd 2021

• Fixed incorrect variable NameHost
• Added tests to detect IP or FQDN in URI
• Updated test to check if type TrustAllCertsPolicy exists or not

Update April 15th 2020

• Added SkipCertificateCheck switch

Update January 16th 2020

• Bug fix which occured when connected to an ESXi node

Update November 18th 2019

• Added NoIPinCert switch

The Code

class MyOBN:System.Management.Automation.ArgumentTransformationAttribute {
    [ValidateSet(
      'Cluster', 'Datacenter', 'Datastore', 'DatastoreCluster', 'Folder',
      'VirtualMachine', 'VirtualSwitch', 'VMHost', 'VIServer'
    )]
    [String]$Type
    MyOBN([string]$Type) {
      $this.Type = $Type
    }
    [object] Transform([System.Management.Automation.EngineIntrinsics]$engineIntrinsics, [object]$inputData) {
      if ($inputData -is [string]) {
        if (-NOT [string]::IsNullOrWhiteSpace( $inputData )) {
          $cmdParam = "-$(if($this.Type -eq 'VIServer'){'Server'}else{'Name'}) $($inputData)"
          $sCmd = @{
            Command = "Get-$($this.Type.Replace('VirtualMachine','VM')) $($cmdParam)"
          }
          return (Invoke-Expression @sCmd)
        }
      } elseif ($inputData.GetType().Name -match "$($this.Type)Impl") {
        return $inputData
      } elseif ($inputData.GetType().Name -eq 'Object[]') {
        return ($inputData | ForEach-Object {
            if ($_ -is [String]) {
              return (Invoke-Expression -Command "Get-$($this.Type.Replace('VirtualMachine','VM')) -Name $_")
            } elseif ($_.GetType().Name -match "$($this.Type)Impl") {
              $_
            }
          })
      }
      throw [System.IO.FileNotFoundException]::New()
    }
  }
  function Invoke-VMScriptPlus {
    <#
  .SYNOPSIS
  Runs a script in a Linux guest OS.
  The script can use the SheBang to indicate which interpreter to use.
  .DESCRIPTION
  This function will launch a script in a Linux guest OS.
  The script supports the SheBang line for a limited set of interpreters.
  .NOTES
  Author:  Luc Dekens
  Version:
  1.0 14/09/17  Initial release
  1.1 14/10/17  Support bash here-document
  2.0 01/08/18  Support Windows guest OS, bat & powershell
  2.1 03/08/18  PowerShell she-bang for Linux
  2.2 17/08/18  Added ScriptEnvironment
  2.3 11/03/19  Resolve IP to FQDN to support certificate for ESXi node
  2.4 22/04/19  Switch to provide password inline to 'sudo' lines
  2.5 07/06/19  Switch WaitForToolsVersionChange to wait for a version change
  3.0 17/11/19  Added powershellv7 support, added InFile & OutFile
  3.1 18/11/19  Added switch NoIPinCert
  3.2 15/04/20  Added switch SkipCertificateCheck
  3.3 02/07/21  Fixed issue with NameHost, added test for IP or FQDN, changed  type test TrustAllCertsPolicy
  .PARAMETER VM
  Specifies the virtual machines on whose guest operating systems
  you want to run the script.
  .PARAMETER GuestUser
  Specifies the user name you want to use for authenticating with the
  virtual machine guest OS.
  .PARAMETER GuestPassword
  Specifies the password you want to use for authenticating with the
  virtual machine guest OS.
  .PARAMETER GuestCredential
  Specifies a PSCredential object containing the credentials you want
  to use for authenticating with the virtual machine guest OS.
  .PARAMETER ScriptText
  Provides the text of the script you want to run. You can also pass
  to this parameter a string variable containing the path to the script.
  Note that the function will add a SheBang line, based on the ScriptType,
  if none is provided in the script text.
  .PARAMETER ScriptType
  The supported Linux interpreters.
  Currently these are bash,perl,python3,nodejs,php,lua,powershell,powershellv6,powershellv7
  .PARAMETER ScriptEnvironment
  A string array with environment variables.
  These environment variables are available to the script from ScriptText
  .PARAMETER GuestOSType
  Indicates which type of guest OS the VM is using.
  The parameter accepts Windows or Linux. This parameter is a fallback for
  when the function cannot determine which OS Family the Guest OS
  belongs to
  .PARAMETER CRLF
  Switch to indicate of the NL that is returned by Linux, shall be
  converted to a CRLF
  .PARAMETER Sudo
  Switch to convert all 'sudo' lines to an inline password 'sudo' line.
  Only taken into account when the GuestOSType is 'Linux'
  .PARAMETER KeepFiles
  Switch to indicate that the temporary files, the script and the output files,
  shall not be deleted.
  Only to be used for debugging purposes.
  .PARAMETER InFile
  One or more files that will be copied to the guest OS.
  These files will be copied to the directory from where the script will run
  and can be used from within the script.
  .PARAMETER InFile
  One or more files that will be copied from the guest OS after the script has ran.
  These files will be copied from the directory from where the script runs.
  .PARAMETER Server
  Specifies the vCenter Server systems on which you want to run the
  cmdlet. If no value is passed to this parameter, the command runs
  on the default servers. For more information about default servers,
  see the description of Connect-VIServer.
  .PARAMETER WaitForToolsVersionChange
  When the invoked code changes the version of the VMware Tools, this switch
  tells the function to wait till this version change is visible in the script
  .PARAMETER NoIPinCert
  When certificates are used that do not contain the IP address of the ESXi node
  as a Subject Alternative Name (SAN), this switch tells the function to convert
  the IP address in all URI used for file transfers, to a FQDN.
  .PARAMETER NoIPinCert
  When certificates are used that do not contain the IP address of the ESXi node
  as a Subject Alternative Name (SAN), this switch tells the function to convert
  the IP address in all URI used for file transfers, to a FQDN.
  .PARAMETER SkipCertificateCheck
  When a non-trusted certificate is used on the ESXi node that hosts the targetted
  VM, the transfer of files to and from the VM's Guest OS will fail.
  This switch tells the function to ignore invalid certificates on the ESXi node.
  .EXAMPLE
  $pScript = @'
  #!/usr/bin/env perl
  use strict;
  use warnings;
  print "Hello world\n";
  '@
  $sCode = @{
  VM = $VM
  GuestCredential = $cred
  ScriptType = 'perl'
  ScriptText = $pScript
  }
  Invoke-VMScriptPlus @sCode
  .EXAMPLE
  $pScript = @'
  print("Happy 10th Birthday PowerCLI!")
  '@
  $sCode = @{
  VM = $VM
  GuestCredential = $cred
  ScriptType = 'python3'
  ScriptText = $pScript
  }
  Invoke-VMScriptPlus @sCode
  .EXAMPLE
  $pScript = @'
  Get-Content -Path .\MyInput.txt | Set-Content -Path .\MyOutput.txt
  '@
  $sCode = @{
  VM = $VM
  GuestCredential = $cred
  ScriptType = 'powershellv7'
  ScriptText = $pScript
  InFile = 'C:\Test\MyInput.txt'
  OutFile = 'C:\Report\MyOutput.txt'
  }
  Invoke-VMScriptPlus @sCode
  #>
    [cmdletbinding()]
    param(
      [parameter(Mandatory = $true, ValueFromPipeline = $true)]
      [MyOBN('VirtualMachine')]
      [VMware.VimAutomation.ViCore.Types.V1.Inventory.VirtualMachine[]]$VM,
      [Parameter(Mandatory = $true, ParameterSetName = 'TextScript')]
      [Parameter(Mandatory = $true, ParameterSetName = 'TextExe')]
      [String]$GuestUser,
      [Parameter(Mandatory = $true, ParameterSetName = 'TextScript')]
      [Parameter(Mandatory = $true, ParameterSetName = 'TextExe')]
      [SecureString]$GuestPassword,
      [Parameter(Mandatory = $true, ParameterSetName = 'CredScript')]
      [Parameter(Mandatory = $true, ParameterSetName = 'CredExe')]
      [PSCredential[]]$GuestCredential,
      [Parameter(Mandatory = $true, ParameterSetName = 'TextScript')]
      [Parameter(Mandatory = $true, ParameterSetName = 'CredScript')]
      [String]$ScriptText,
      [Parameter(Mandatory = $true, ParameterSetName = 'TextScript')]
      [Parameter(Mandatory = $true, ParameterSetName = 'CredScript')]
      [ValidateSet('bash', 'perl', 'python3', 'nodejs', 'php', 'lua', 'powershell',
        'powershellv6', 'powershellv7', 'bat', 'exe')]
      [String]$ScriptType,
      [Parameter(Mandatory = $true, ParameterSetName = 'TextExe')]
      [Parameter(Mandatory = $true, ParameterSetName = 'CredExe')]
      [string]$ExeName,
      [String[]]$ScriptEnvironment,
      [ValidateSet('Windows', 'Linux')]
      [String]$GuestOSType,
      [Switch]$CRLF,
      [Switch]$Sudo,
      [Switch]$KeepFiles,
      [MyOBN('VIServer')]
      [VMware.VimAutomation.ViCore.Types.V1.VIServer]$Server = $global:DefaultVIServer,
      [Switch]$WaitForToolsVersionChange,
      [String[]]$InFile,
      [String[]]$OutFile,
      [Switch]$NoIPinCert,
      [Switch]$SkipCertificateCheck
    )
    Begin {
      #region Helper functions
      function Send-GuestFile {
        [cmdletbinding()]
        param(
          [Parameter(Mandatory = $true)]
          [String]$File,
          [Parameter(Mandatory = $true, ParameterSetName = 'File')]
          [String]$Source,
          [Parameter(Mandatory = $true, ParameterSetName = 'Data')]
          [String]$Data
        )
        if ($PSCmdlet.ParameterSetName -eq 'File') {
          $Data = Get-Content -Path $Source -Raw
        }
        $attr = New-Object VMware.Vim.GuestFileAttributes
        $clobber = $true
        $fileInfo = $gFileMgr.InitiateFileTransferToGuest($moref, $auth, $File, $attr, $Data.Length, $clobber)
        if ($Server.ProductLine -eq 'embeddedEsx') {
          $fileInfo = $fileInfo.Replace('*', ([System.Uri]$server.ServiceUri).Host)
        }
        if ($NoIPinCert.IsPresent) {
          $ip = $fileInfo.split('/')[2].Split(':')[0]
          if ($ip -as [IPAddress]) {
            $hostName = Resolve-DnsName -Name $ip | Select-Object -ExpandProperty NameHost
            $fileInfo = $fileInfo.replace($ip, $hostName)
          }
        }
        $sWeb = @{
          Uri = $fileInfo
          Method = 'Put'
          Body = $Data
        }
        if ($SkipCertificateCheck.IsPresent) {
          if ($PSVersionTable.PSVersion.Major -lt 6) {
            [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
          } else {
            $sWeb.Add('SkipCertificateCheck', $true)
          }
        }
        Write-Verbose -Message "Copying $($PSCmdlet.ParameterSetName) to $File"
        $copyResult = Invoke-WebRequest @sWeb
        if ($copyResult.StatusCode -ne 200) {
          Throw "ScripText copy failed!rStatus $($copyResult.StatusCode)r$(($copyResult.Content | ForEach-Object{[char]$_}) -join '')"
        }
      }
      function Receive-GuestFile {
        [cmdletbinding()]
        param(
          [String]$Source,
          [String]$File
        )
        $fileInfo = $gFileMgr.InitiateFileTransferFromGuest($moref, $auth, $Source)
        if ($Server.ProductLine -eq 'embeddedEsx') {
          $fileInfo.Url = $fileInfo.Url.Replace('*', ([System.Uri]$server.ServiceUri).Host)
        }
        if ($NoIPinCert.IsPresent) {
          $ip = $fileInfo.Url.split('/')[2].Split(':')[0]
          if ($ip -as [IPAddress]) {
            hostName = Resolve-DnsName -Name $ip | Select-Object -ExpandProperty NameHost
            $fileInfo.Url = $fileInfo.Url.replace($ip, $hostName)
          }
        }
        $sWeb = @{
          Uri = $fileInfo.Url
          Method = 'Get'
        }
        if ($SkipCertificateCheck.IsPresent) {
          if ($PSVersionTable.PSVersion.Major -lt 6) {
            [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
          } else {
            $sWeb.Add('SkipCertificateCheck', $true)
          }
        }
        $fileContent = Invoke-WebRequest @sWeb
        if ($fileContent.StatusCode -ne 200) {
          Throw "Retrieve of script output failed!rStatus $($fileContent.Status)r$(($fileContent.Content | ForEach-Object{[char]$_}) -join '')"
        }
        if ($File) {
          $fileContent.Content | Set-Content -Path $File -Encoding byte -Confirm:$false
        } else {
          $fileContent.Content
        }
      }
      #endregion
      #region Set up guest operations
      $si = Get-View ServiceInstance -Server $Server
      $guestMgr = Get-View -Id $si.Content.GuestOperationsManager
      $gFileMgr = Get-View -Id $guestMgr.FileManager
      $gProcMgr = Get-View -Id $guestMgr.ProcessManager
      #endregion
      #region Set up shebang table
      $shebangTab = @{
        'bash' = '#!/usr/bin/env bash'
        'perl' = '#!/usr/bin/env perl'
        'python3' = '#!/usr/bin/env python3'
        'nodejs' = '#!/usr/bin/env nodejs'
        'php' = '#!/usr/bin/env php'
        'lua' = '#!/usr/bin/env lua'
        'powershellv6' = '#!/usr/bin/env pwsh'
        'powershellv7' = '#!/usr/bin/env pwsh-preview'
      }
      #endregion
      #region Handle SkipCertificateCheck (if used)
      if ($SkipCertificateCheck.IsPresent -and
        $PSVersionTable.PSVersion.Major -lt 6 -and
        -not ('TrustAllCertsPolicy' -as [Type])) {
        Add-Type @'
    using System.Net;
    using System.Security.Cryptography.X509Certificates;
    public class TrustAllCertsPolicy : ICertificatePolicy {
    public bool CheckValidationResult(
        ServicePoint srvPoint, X509Certificate certificate,
        WebRequest request, int certificateProblem) {
            return true;
        }
    }
'@
      }
      #endregion
    }
    Process {
      foreach ($vmInstance in $VM) {
        #region Test conditions for running script in guest OS
        if ($vmInstance.PowerState -ne 'PoweredOn') {
          Write-Error "VM $($vmInstance.Name) is not powered on"
          continue
        }
        $vmInstance.ExtensionData.UpdateViewData('Guest')
        if ($vmInstance.ExtensionData.Guest.ToolsRunningStatus -ne 'guestToolsRunning') {
          Write-Error "VMware Tools are not running on VM $($vmInstance.Name)"
          continue
        }
        if (-not $vmInstance.ExtensionData.Guest.GuestOperationsReady) {
          Write-Error "VM $($vmInstance.Name) is not ready to use Guest Operations"
          continue
        }
        $moref = $vmInstance.ExtensionData.MoRef
        #endregion
        #region Create Authentication Object (User + Password)
        if ('CredScript', 'CredExe' -contains $PSCmdlet.ParameterSetName) {
          $GuestUser = $GuestCredential.GetNetworkCredential().username
          $plainGuestPassword = $GuestCredential.GetNetworkCredential().password
        }
        if ('TextScript', 'TextExe' -contains $PSCmdlet.ParameterSetName) {
          $bStr = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($GuestPassword)
          $plainGuestPassword = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto($bStr)
        }
        $auth = New-Object VMware.Vim.NamePasswordAuthentication
        $auth.InteractiveSession = $false
        $auth.Username = $GuestUser
        $auth.Password = $plainGuestPassword
        #endregion
        #region Determine GuestOSType
        if (-not $GuestOSType) {
          Write-Verbose "No GuestOSType value provided. Trying to determine now."
          switch -Regex ($vmInstance.Guest.OSFullName) {
            'Windows' {
              Write-Verbose "It's a Windows guest OS"
              $GuestOSType = 'Windows'
              if (-not $ExeName -and 'bat', 'powershell', 'powershellv6', 'powershellv7' -notcontains $ScriptType) {
                Write-Verbose "Invalid scripttype provided"
                Write-Error "For a Windows guest OS the ScriptType can be Bat, PowerShell, PowerShellv6 or PowerShellv7"
                continue
              }
            }
            'Linux' {
              Write-Verbose "It's a Linux guest OS"
              $GuestOSType = 'Linux'
              if (-not $ExeName -and 'bat', 'powershell' -contains $ScriptType) {
                Write-Verbose "Invalid scripttype provided"
                Write-Error "For a Linux guest OS the ScriptType cannot be Bat"
                continue
              }
            }
            Default {
              Write-Verbose "Can't determine guest OS type."
              Write-Error "Unable to determine the guest OS type on VM $($vmInstance.Name)"
              Write-Error "Try using the GuestOSType parameter."
              continue
            }
          }
        }
        if ($GuestOSType -eq 'Linux') {
          Write-Verbose "Seems to be a Linux guest OS"
          # Test if code contains a SheBang, otherwise add it
          $targetCode = $shebangTab[$ScriptType]
          if ($ScriptText -notmatch "^$($targetCode)") {
            Write-Verbose "Add SheBang $targetCode"
            $ScriptText = "$($targetCode)nr$($ScriptText)"
          }
          # Take care of the 'sudo' switch
          if ($Sudo) {
            Write-Verbose "Setting up sudo usage"
            $ScriptText = ($ScriptText | ForEach-Object -Process {
                $_ -replace 'sudo', "echo $plainGuestPassword | sudo -S"
              })
          }
        }
        #endregion
        #region Create a temp directory
        $tempFolder = $gFileMgr.CreateTemporaryDirectoryInGuest($moref, $auth, "$($env:USERNAME)_$($PID)", $null, $null)
        Write-Verbose "Created temp folder in guest OS $tempFolder"
        #endregion
        #region Create temp file for script
        $suffix = ''
        if ('bat', 'exe' -contains $ScriptType) {
          $suffix = ".cmd"
        }
        if ('powershell', 'powershellv6', 'powershellv7' -contains $ScriptType) {
          $suffix = ".ps1"
        }
        Try {
          $tempFile = $gFileMgr.CreateTemporaryFileInGuest($moref, $auth, "$($env:USERNAME)_$($PID)", $suffix, $tempFolder)
          Write-Verbose "Created temp script file in guest OS $tempFile"
        } Catch {
          Write-Verbose "Encountered a problem creating the script file in the guest OS"
          Throw "$error[0].Exception.Message"
        }
        #endregion
        #region Create temp file for output
        Try {
          $tempOutput = $gFileMgr.CreateTemporaryFileInGuest($moref, $auth, "$($env:USERNAME)_$($PID)_output", $null, $tempFolder)
          Write-Verbose "Created temp output file in guest OS $tempOutput"
        } Catch {
          Write-Verbose "Encountered a problem creating the output file in the guest OS"
          Throw "$error[0].Exception.Message"
        }
        #endregion
        #region Copy script to temp file
        if ($ExeName) {
          Send-GuestFile -Data $ExeName -File $tempFile
          Write-Verbose "Copied ExeName to temp script file"
        } else {
          if ($GuestOSType -eq 'Linux') {
            $ScriptText = $ScriptText.Split("r") -join ''
          }
          Send-GuestFile -Data $ScriptText -File $tempFile
          Write-Verbose "Copied scripttext to temp script file"
        }
        #endregion
        #region Get current environment variables
        $SystemEnvironment = $gProcMgr.ReadEnvironmentVariableInGuest($moref, $auth, $null)
        #endregion
        #region Copy InFiles to to guest OS
        if ($InFile) {
          $InFile | ForEach-Object -Process {
            $destinationFilePath = "$tempFolder/$(Split-Path -Path $_ -Leaf)"
            Write-Verbose "Upload InFile $_"
            Send-GuestFile -Source $_ -File $destinationFilePath
          }
        }
        #endregion
        #region Run script
        if ($WaitForToolsVersionChange) {
          $toolsVersion = $vmInstance.ExtensionData.Guest.ToolsVersion
        }
        switch ($GuestOSType) {
          'Linux' {
            # Make temp file executable
            $spec = New-Object VMware.Vim.GuestProgramSpec
            $spec.Arguments = "751 $tempFile"
            $spec.ProgramPath = '/bin/chmod'
            Try {
              $procId = $gProcMgr.StartProgramInGuest($moref, $auth, $spec)
              Write-Verbose "Make script file executable"
            } Catch {
              Write-Verbose "Encountered a problem making the script file executable in the guest OS"
              Throw "$error[0].Exception.Message"
            }
            # Run temp file
            $spec = New-Object VMware.Vim.GuestProgramSpec
            if ($ScriptEnvironment) {
              $spec.EnvVariables = $SystemEnvironment + $ScriptEnvironment
            }
            $spec.Arguments = " > $($tempOutput) 2>&1"
            $spec.ProgramPath = "$($tempFile)"
            $spec.WorkingDirectory = $tempFolder
            Try {
              $procId = $gProcMgr.StartProgramInGuest($moref, $auth, $spec)
              Write-Verbose "Run script with '$($tempFile) > $($tempOutput)'"
            } Catch {
              Write-Verbose "Encountered a problem running the script file in the guest OS"
              Throw "$error[0].Exception.Message"
            }
          }
          'Windows' {
            # Run temp file
            $spec = New-Object VMware.Vim.GuestProgramSpec
            $spec.WorkingDirectory = $tempFolder
            if ($ScriptEnvironment) {
              $spec.EnvVariables = $SystemEnvironment + $ScriptEnvironment
            }
            if ($ExeName) {
              $spec.ProgramPath = "cmd.exe"
              $spec.Arguments = " /s /c start """" ""$ExeName"""
            } else {
              switch ($ScriptType) {
                'PowerShell' {
                  $spec.Arguments = " /C powershell -NonInteractive -File $($tempFile) > $($tempOutput)"
                  $spec.ProgramPath = "cmd.exe"
                }
                { 'PowerShellv6', 'PowerShellv7' -contains $_ } {
                  $psCmd = 'pwsh.exe'
                  if ($ScriptType -eq 'PowerShellv7') {
                    $psCmd = 'pwsh-preview.exe'
                  }
                  $spec.Arguments = " /C ""$psCmd"" -NonInteractive -File $($tempFile) > $($tempOutput)"
                  $spec.ProgramPath = "cmd.exe"
                }
                'Bat' {
                  $spec.Arguments = " /s /c cmd > $($tempOutput) 2>&1 /s /c $($tempFile)"
                  $spec.ProgramPath = "cmd.exe"
                }
              }
            }
            Try {
              $procId = $gProcMgr.StartProgramInGuest($moref, $auth, $spec)
              Write-Verbose "Run script with '$($spec.ProgramPath) $($spec.Arguments)'"
            } Catch {
              Write-Verbose "Encountered a problem running the script file in the guest OS"
              Throw "$error[0].Exception.Message"
            }
          }
        }
        if ($WaitForToolsVersionChange) {
          Write-Verbose "Waiting for VMware Tools version to change"
          while ($toolsVersion -eq $vmInstance.ExtensionData.Guest.ToolsVersion) {
            Start-Sleep -Seconds 1
            $vmInstance.ExtensionData.UpdateViewData('Guest')
          }
          Write-Verbose "VMware Tools version changed from $toolsVersion to $($vmInstance.ExtensionData.Guest.ToolsVersion)"
        }
        #endregion
        #region Wait for script to finish
        Try {
          $pInfo = $gProcMgr.ListProcessesInGuest($moref, $auth, @($procId))
          Write-Verbose "Wait for process to end"
          while ($pInfo -and $null -eq $pInfo.EndTime) {
            Start-Sleep 1
            $pInfo = $gProcMgr.ListProcessesInGuest($moref, $auth, @($procId))
          }
        } Catch {
          Write-Verbose "Encountered a problem waiting for the script to end in the guest OS"
          Throw "$error[0].Exception.Message"
        }
        #endregion
        #region Retrieve output from script
        Write-Verbose "Get output from $tempOutput"
        $scriptOutput = Receive-GuestFile -Source $tempOutput
        #endregion
        #region Copy OutFiles from guest OS
        if ($OutFile) {
          $OutFile | ForEach-Object -Process {
            $sourceFilePath = "$tempFolder/$_"
            Write-Verbose "Download OutFile $_"
            Receive-GuestFile -Source $sourceFilePath -File $_
          }
        }
        #endregion
        #region Clean up
        # Remove temporary folder
        if (-not $KeepFiles) {
          $gFileMgr.DeleteDirectoryInGuest($moref, $auth, $tempFolder, $true)
          Write-Verbose "Removed folder $tempFolder"
        }
        #endregion
        #region Package result in object
        New-Object PSObject -Property @{
          VM = $vmInstance
          ScriptOutput = & {
            $out = ($scriptOutput | ForEach-Object { [char]$_ }) -join ''
            if ($CRLF) {
              $out.Replace("n", "n`r")
            } else {
              $out
            }
          }
          Pid = $procId
          PidOwner = $pInfo.Owner
          Start = $pInfo.StartTime
          Finish = $pInfo.EndTime
          ExitCode = $pInfo.ExitCode
          ScriptType = $ScriptType
          ScriptSize = $ScriptText.Length
          ScriptText = $ScriptText
          OutFiles = $OutFile
          GuestOS = $GuestOSType
        }
        #endregion
      }
    }
  }

Annotations

The following annotations will only document the additions to the v2 version of the function. Refer to the blog posts mentioned in the introduction to see the annotations of the previous versions of the function.

Line 191-192: Addition/change of the script types. Now includes powershellv6 and powershellv7.

Line 201: A switch that allows the caller to ask for sudo support. In practice the function will pipe, via an echo command, the guest credential’s password to each sudo command in the script.

Line 206-207: New parameters InFile and OutFile. They permit files to be copied to/from the script environment.

Line 208: The NoIPforCert switch

Line 209: The SkipCertificateCheck switch

Line 214-262: Internal helper function to send files from the caller’s environment to the guest OS.

Line 248-255,286-293: The SkipCertificateCheck switch calls Invoke-WebRequest with the option to not check the certificate. In pre-V6 that is done through the CertificatePolicy class, in PS v6 and higher, the SkipCertificateCheck switch on Invoke-WebRequest is used.

Line 264-307: Internal helper function to receive files from the guest OS into the caller’s environment.

Line 237-242,276-281: If the NoIPforCert switch is $true, the IP address in the URI returned by the ESXi node (where the VM is running) will be converted to the FQDN of the ESXi node.

Line 248-255,286-293: If the SkipCertificateCheck switch is used, the following Invoke-WebRequest is called with the setting to check checking certificates.

Line 325-326: New she-bang entries for PowerShell v6 and v7. Note that at the time this post was published, that v7 was still in preview.

Line 331-345: When the SkipCertificateCheck is used and the PS version is pre-V6 and the type hasn’t been declared yet, the TrustAllCertsPolicy class is defined.

Line 365-369: Additional ‘ready’ test. If this property is not $true, the VMware Tools inside the guest OS are not ready to accept any GuestOperations related calls.

Line 400: For any Windows guest OS, the function currently only accepts BAT, PS, PSv6 and PSv7 scripts.

Line 411: For any Linux (this includes MacOS) guest OS, the function accepts all scripttypes, except BAT and PowerShell (meaning PS pre-v6).

Line 438-444: sudo support. Each line of the user’s script containg the sudo command, will be prefixed with an echo command, which will provide the password to the sudo prompt.

Line 465: In this version of the function, all files will be stored in a temporary directory in the guest OS.

Line 510-517: The function allows the caller to copy one or more files, from the caller’s environment to the temporary folder in the guest OS. Since these files will be in the same folder as the actual script, the script can reference these files.

Line 581-595: The function supports for a Windows guest OS, the use of PSv5.*, PSV6 and PSv7.

Line 651-658: The function allows the caller to copy one or more files, from the folder, where the script ran in the guest OS, to the caller’s environment. This allows an alternative method to send data back to caller besides the stdin channel.

Line 663-667: When the KeepFiles switch is not used, the function will remove the temporary directory in the guest OS, where all the script related files are stored.

Usage

The following section will show some examples of how to use the new features introduced in the v3 of the Invoke-VMScriptPlus function.

Sudo

One of security measure on many Linux systems is that you need to use sudo to run commands with elevated privileges.

This is a typical example of such a case.

$vmName = 'ubuntuVM'
$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))


$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptText = 'apt install gawk'
    ScriptType = 'bash'
}
Invoke-VMScriptPlus @sInvoke

The OS returns the following error.

Ok, let’s place sudo in front of that.

$vmName = 'ubuntuVM'
$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))


$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptText = 'sudo apt install gawk'
    ScriptType = 'bash'
}
Invoke-VMScriptPlus @sInvoke

But that just makes the next issue obvious. How to answer to a prompt in a script. The method to solve this quite simple. Echo the password and then pipe it to the sudo command.

But this not always a practical solution, and probably not very safe either. To avoid having to handle this in the code you sent to the guest OS, the Invoke-VMScriptPlus function has the Sudo switch.

$vmName = 'ubuntuVM'
$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))


$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptText = 'sudo apt install gawk'
    ScriptType = 'bash'
    Sudo = $true
}
Invoke-VMScriptPlus @sInvoke

It will extract the password from the GuestCredential parameter value, and insert the echo with the password on each line in your code that starts with sudo.

PowerShell v7

At the moment, of this writing, PowerShell v7 is still a preview.
Since you can install PSv6 and PSv7 side-by-side, you can run your scripts in either version.

This is a handy way to test your scripts for readiness for PSv7.

With the ScriptType parameter, you easily define against which PowerShell version your script should run. First we use powershellv6.

$vmName = 'ubuntuVM'
$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))


$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptText = '$PSVersionTable'
    ScriptType = 'powershellv6'
}
Invoke-VMScriptPlus @sInvoke

This results in.

And then the same, but with powershellv7.

$vmName = 'ubuntuVM'
$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))


$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptText = '$PSVersionTable'
    ScriptType = 'powershellv7'
}
Invoke-VMScriptPlus @sInvoke

And now we get.

Infile/OutFile

The Invoke-VMScriptPlus function captures the output of your script on the stdin stream. But sometimes you would like to produce multiple outputs, or output in a specific format.

The same goes for your input to your script. You can pass input along to your script, for example with a here-string, but this at least requires extra steps to run your script.

For that reason, this v3 version of Invoke-VMScriptPlus, added two new parameters, Infile and OutFile.

Each of these parameters allow you to specify one or more files in your local environment that will be passed and/or retrieved to the environment where your script runs in the guest OS of the target VM.

A somewhat contrived example on what this allows you to do.

$vmName = 'ubuntuVM'

$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))

$code = @'
Get-Content -Path .\in.txt |
ForEach-Object -Process {
   $_ | Set-Content -Path ".\$_.txt"
}
'@

# Create input file
1..3 | Set-Content -Path .\in.txt
Get-ChildItem -Path . -Filter *.txt

$sInvoke = @{
   VM = $vmName
   GuestCredential = $cred
   ScriptText = $code
   ScriptType = 'powershellv7'
   InFile = 'in.txt'
   OutFile = '1.txt', '2.txt', '3.txt'
}
Invoke-VMScriptPlus @sInvoke

Get-ChildItem -Path . -Filter *.txt

In short, the script creates an input file, then runs a script on the target VM, passing along the input file.
The script on the target VM creates a number of files, which are returned to the caller when the script completes.

This is the output from the above code.

This is a handy feature when your script requires an existing file as input and produces multiple output files.

NoIPinCert

When the function needs to retrieve files from the guest OS, the ESXi node on which the VM is running, provides a URI. This URI normally contains the IP address of the ESXi node.

This can cause an issue when using certificates, and not bypassing the certificate check. The reason is sometimes that the IP address of the ESXi node is not included as a Subject Alternate Address (SAN) in the certificate. This is visible through the following error message. Note that the following output also shows some verbose output to demonstrate when this is happening, and also to show that the error occurs when doing a GET with an URI that contains the IP address.

To avoid this error, the Invoke-VMScriptPlus function, by default, translates this IP address into the FQDN of the ESXi node. The following verbose output shows the adapted URI for the GET

In some environments these automatic conversion of the IP address to the FQDN might cause an issue. One reason for this to happen might be that the ESXi node is added with it’s IP address. Another might be that the DNS query to get the FQDN of the ESXi node fails.
Especially for those occasion, the function now has this NoIPforCert switch. This switch, when set, instructs the function to NOT translate the IP address in the URI to the FQDN.

A sample call that uses this switch can look like this.

$sInvoke = @{
    VM = $vmName
    ScriptType = 'bash'
    ScriptText = 'whoami'
    GuestCredential = $cred
    Verbose = $true
    NoIPinCert = $true
}
Invoke-VMScriptPlus @sInvoke

Enjoy!

The following is a write up of a part of session HBI1729BU ,that was presented at VMworld US 2019.

The code shown in this post is also available in the PowerCLI Community Repository.

Concept

The idea behind this snippet came when I had checked what was in $global:DefaultVIServers for the n-th time in a short PowerCLI session.

And it’s probably not only that specific variable that must be all too familiar to most of us. Any entry in the following list of variables and cmdlets must have been executed by most of us multiple times.

$Global:DefaultVIServer
$Global:DefaultVIServers
$PSVersionTable
Get-Location
Get-Module -Name VMware.PowerCLI

It would be an important timesaver, when the information returned by those instructions, would be permanently in view. At your fingertips, so to say.

The use of the PowerShell profile(s) and the Prompt function, immediately came to mind.

Not to overload your PS prompt, I decided to show part of the information in the console’s Window Title bar. That can be done by assigning a value to $host.ui.RawUI.WindowTitle. 

Implementation

The Code

#
# Provide environment information in the PS Console
#
# History:
# 1.0 - August 4th 2019 - LucD
#       Initial version (for session HBI1729BU VMworld US 2019)
#
# 1) PS prompt
# - current (local) time
# - execution time of the previous command
# - shortened PWD
# 2) Window title
# - User/Admin
# - PS-32/54-Edition-Version
# - PCLI version
# - git repo/branch
# - VC/ESXi:defaultServer-User [# connections]

function prompt
{
    # Current time
    $date = (Get-Date).ToString('HH:mm:ss')
    Write-Host -Object '[' -NoNewline
    Write-Host -Object $date -ForegroundColor Cyan -BackgroundColor DarkBlue -NoNewline
    Write-Host -Object ']' -NoNewline

    # Execution time previous command
    $history = Get-History -ErrorAction Ignore -Count 1
    if ($history)
    {
        $time = ([DateTime](New-TimeSpan -Start $history.StartExecutionTime -End $history.EndExecutionTime).Ticks).ToString('HH:mm:ss.ffff')
        Write-host -Object '[' -NoNewLine
        Write-Host -Object "$time" -ForegroundColor Yellow -BackgroundColor DarkBlue -NoNewline
        Write-host -Object '] ' -NoNewLine
    }

    # Shorted PWD
    $path = $pwd.Path.Split('\')
    if ($path.Count -gt 3)
    {
        $path = $path[0], '..', $path[-2], $path[-1]
    }
    Write-Host -Object "$($path -join '\')" -NoNewline

    # Prompt function needs to return something,
    # otherwise the default 'PS>' will be added
    "> "

    # Refresh the window's title
    Set-Title
}

function Set-Title
{
    # Running as Administrator or a regular user
    $userInfo = [Security.Principal.WindowsIdentity]::GetCurrent()
    if ((New-Object Security.Principal.WindowsPrincipal $userInfo).IsInRole([Security.Principal.WindowsBuiltinRole]::Administrator))
    {
        $role = 'Admin'
    }
    else
    {
        $role = 'User'
    }

    # Usertype user@hostname
    $user = "$role $($userInfo.Name)@$($env:computername)"

    # PowerShell environment/PS version
    $bits = 32
    if ([Environment]::Is64BitProcess)
    {
        $bits = 64
    }
    $ps = " - PS-$($bits): $PSEdition/$($PSVersionTable.PSVersion.ToString())"

    # PowerCLI version (derived from module VMware.PowerCLI)
    $pcliModule = Get-Module -Name VMware.PowerCLI -ListAvailable |
    Sort-Object -Property Version -Descending |
    Select-Object -First 1
    $pcli = " - PCLI: $(if($pcliModule){$pcliModule.Version.ToString()}else{'na'})"

    # If git is present and if in a git controlled folder, display repositoryname/current_branch
    $gitStr = ''
    if ((Get-Command -Name 'git' -CommandType Application -ErrorAction SilentlyContinue).Count -gt 0)
    {
        $gitTopLevel = git rev-parse --show-toplevel 2> $null
        if ($gitTopLevel.Length -ne 0)
        {
            $gitRepo = Split-Path -Path $gitTopLevel -Leaf
            $gitBranch = (git branch | Where-Object { $_ -match "\*" }).Trimstart('* ')
            $gitStr = " - git: $gitRepo/$gitBranch"
        }
    }

    # If there is an open vSphere Server connection
    # display [VC|ESXi] last_connected_server-connected_user [number of open server connections]
    if ($global:defaultviserver)
    {
        $vcObj = (Get-Variable -Scope global -Name 'DefaultVIServer').Value
        if ($vcObj.ProductLine -eq 'vpx')
        {
            $vcSrv = 'VC'
        }
        else
        {
            $vcSrv = 'ESXi'
        }
        $vc = " - $($vcSrv): $($vcObj.Name)-$($vcObj.User) [$($global:DefaultVIServers.Count)]"
    }

    # Update the Window's title
    $host.ui.RawUI.WindowTitle = "$user$ps$pcli$vc$gitStr"
}

# Set title after starting session
Set-Title

Annotations

Line 19-51: Replacement function for the PS prompt.

Line 22-25: Current time in the format ‘HH:mm:ss’. Note that ‘HH’ asks for a 24-hour value.

Line 28-35: The execution time of the previous command, in the format ‘HH:mm:ss:ffff’.

Line 28: The execution time is based on the content of PS History. With Get-History we obtain the start time of the command, and with New-TimeSpan we obtain the duration for running that command.

Line 38-43: These lines take the path to the current folder, and shorten that path to only show the start of the path and the last two directories. When the path is shortened, it is indicated by two dots.

Line 53-114: Function to populate the Windows Title bar

Line 56-64: These lines determine if the PS session is started “As Administrator” or as a regular user.

Line 67: The user and station information.

Line 70-75: The PS edition (Desktop or Core) and the PS version.

Line 78-81: If installed, the version of VMware PowerCLI. Note that this looks at the VMware.PowerCLI module to obtain that version number.

Line 84-94: If the git command is present on the station, these lines will check if the current directory is inside a git repository. It will then display the name of the git repository and the current branch.

Line 98-110: These lines check if there is an open connection to a vSphere Server. It will show if the connected vSphere Server is a vCenter or an ESXi node.

Line 109: If multiple connections to a vSphere Servers are open, this will display the number of these connections. The vSphere Server that is shown is the last connected one. This corresponds with what is kept in the $global:DefaultVIServer variable.

Line 117: Initial call to the Set-Title function. This will populate the Windows Title bar when the PS console is opened.

Installation

You can just copy and rename the code to one of the profile files. To make that process easier, I also provide the following installation script.

[cmdletbinding(SupportsShouldProcess = $true, ConfirmImpact = 'High')]
param(
    #        [Parameter(Mandatory = $true)]
    [ValidateSet('CurrentUserCurrentHost', 'CurrentUserAllHosts',
        'AllUsersCurrentHost', 'AllUsersAllHosts')]
    [string]$Scope,
    [switch]$NoClobber,
    [switch]$Backup,
    [string]$NewProfile = '.\NewProfile.ps1'
)

if ($PSCmdlet.ShouldProcess("$($Profile.$Scope)", "Create $Scope profile"))
{
    $profilePath = $Profile."$Scope"
    Write-Verbose -Message "Target is $profilePath"
    $createProfile = $true
    if (Test-Path -Path $profilePath)
    {
        Write-Verbose -Message "Target exists"
        if ($NoClobber)
        {
            Write-Verbose -Message "Cannot overwrite target due to NoClobber"
            $createProfile = $false
        }
        elseif ($Backup)
        {
            Write-Verbose -Message "Create a backup as $profilePath.bak"
            Copy-Item -Path $profilePath -Destination "$profilePath.bak" -Confirm:$false -Force
        }
        elseif (-not $NoClobber)
        {
            Write-Verbose -Message "Target will be overwritten"
        }
        else
        {
            Write-Verbose -Message "Use -NoClobber:$false or -Backup"
        }
    }
    if ($createProfile)
    {
        if (-not $NewProfile)
        {
            $script:MyInvocation.MyCommand | select *
            $folder = Split-Path -Parent -Path $script:MyInvocation.MyCommand.Path
            $folder = Get-Location
            $NewProfile = "$folder\NewProfile.ps1"
        }
        Write-Verbose -Message "New profile expected at $NewProfile"
        if (Test-Path -Path $NewProfile)
        {
            Write-Verbose -Message "Copy $NewProfile to $profilePath"
            Copy-Item -Path $NewProfile -Destination $profilePath -Confirm:$false
        }
        else
        {
            Write-Warning -Message "Could not find the new profile file!"
            Write-Warning -Message "Use the NewProfile parameter or store a NewProfile.ps1 file in folder $folder."
        }
    }
}

Save the above code to a .ps1 file, let’s say install-profile.ps1. Also copy the profile code to a .ps1 file, let’s say newprofile.ps1.

You can now install this new profile with a simple call.

.\install-profile.ps1 -NewProfile .\NewProfile.ps1 -Scope CurrentUserCurrentHost -Backup 

The installation script has a number of parameters:

• NewProfile : the file that contains the profile you want to install
• Scope : for which Scope you want to install the new profile.
• Backup : backup the current profile before overwriting it.
• NoClobber : avoids overwriting an existing profile file.

Usage

When the new profile is installed, and when you restart your console session, you will see the new profile in action.

The following screenshot shows the features of the new profile.

And the following short video shows the installation process and the features the profile offers.

Enjoy!