LucD notes

vNIC transmit and receive rates

LucD — Sat, 21 Apr 2012 18:59:32 +0000

The VMTN PowerCLI Community is a constant source of inspiration for blog posts. User roswellevent raised, in the thread Any way to get virtual machine Nic card usage?, the question if it was possible to get the transmit and receive rate for each vNIC in virtual machines.

Since I’m interested in all things statistics in vSphere I decided to tackle the question. Finding the metrics to use for this kind of report is not too difficult. Under the PerformanceManager Network section we find the metrics net.received.average and net.transmitted.average. And, provided your Statistics Level is set to 3 for the timeframe you want to report on, the metrics capture statistics on the device level.

Great, exactly what we need! A quick check in the Performance tab in the vSphere Client showed an additional problem to solve. The instance didn’t mention Network Adapter 1 but a number.

Background

It turns out that this number is the internal Key used for the vNICs. Could have been a bit more user-friendly in the vSphere Client

The solution is quite simple, the script fetches all the vNICs of a VM and then stores the Key together with the human-understandable name in a hash table. This hash table allows the script to replace the Instance value by the Name of the vNIC.

The script

function Get-VmNicStat{
<#
.SYNOPSIS  Retrieve network rates per vNIC per VM
.DESCRIPTION The function will calculate the average
  transmit receive rate, over a select time interval,
  for each vNIC on the VMs selected.
.NOTES  Author:  Luc Dekens
.PARAMETER VM
  The Virtual Machine(s)
  This is a required parameter.
.PARAMETER Start
  A DateTime value that specifies the start of the timeframe
  you want to use.
  This is a required parameter.
.PARAMETER Finish
  A DateTime value that specifies the end of the timeframe
  you want to use.
  The default is the current time.
.EXAMPLE
  PS> Get-VM | Get-VmNicStat -Start $start
#>

  param(
  [CmdletBinding()]
  [parameter(Mandatory = $true, ValueFromPipeline = $true)]
  [VMware.VimAutomation.ViCore.Impl.V1.Inventory.VirtualMachineImpl[]]$VM,
  [parameter(Mandatory = $true)]
  [System.DateTime]$Start,
  [System.DateTime]$Finish = (Get-Date)
  )

  begin{
    $metrics = "net.received.average","net.transmitted.average"
  }

  process{
    $stats = Get-Stat -Entity $VM -Stat $metrics -Start $Start

    foreach($vm in ($stats | Group-Object -Property {$_.Entity.Name})){
      $nicTab = @{}
      $vm.Group[0].Entity.NetworkAdapters | %{
        $key = [string]$_.ExtensionData.Key
        $value = $_.Name
        $nicTab.Add($key,$value)
      }

      $vm.Group | where {$nicTab.ContainsKey($_.Instance)} |
      Group-Object -Property Instance | %{
        New-Object PSObject -Property @{
          VM = $vm.Name
          Start = $Start
          Finish = $Finish
          NIC = $nicTab[$_.Group[0].Instance]
          "Avg Received (KBps)" = [Math]::Round(($_.Group |
            where {$_.MetricId -eq "net.received.average"} |
            Measure-Object -Property Value -Average).Average,1)
          "Avg Transmitted (KBps)" = [Math]::Round(($_.Group |
            where {$_.MetricId -eq "net.transmitted.average"} |
            Measure-Object -Property Value -Average).Average,1)
        }
      }
    }
  }
}

Annotations

Line 33: The metrics the functions uses.

Line 37: Retrieve the statistical data for the virtual machines over the selected timeframe.

Line 39: The function uses the Group-Object cmdlet to split the returned statistical by VM.

Line 40-45: Look at all the vNICs connected to the VM and create a hash table, linking the Key with the Name of the vNIC.

Line 47: The Where-clause will filter out the statistical data for the vNICs of the VM

Line 48: Use the Group-Object to to split the statistical data per vNIC

Line 49-60: Create an object per vNIC per VM

Line 53: Use the hash table to fetch the “Network Adapter x” name of the vNIC

Line 54-59: Calculate the average transmit and recieve values over the selected timeframe.

Sample usage

You can use the function like this

$vms = Get-VM -Name vm*
Get-VmNicStat -VM $vms -Start (Get-Date).AddHours(-1)

And you can also use it in a pipeline, like this

Get-VM -Name vm* |
Get-VmNicStat -Start (Get-Date).AddHours(-1) |
Select VM,Start,Finish,NIC,"Avg Received (KBps)","Avg Transmitted (KBps)" |
Format-Table -AutoSize

The resulting output looks like this

Enjoy!

Test if the datastore can be unmounted

LucD — Sun, 15 Apr 2012 21:29:48 +0000

Lately I have been playing around with the new Storage related features in vSphere 5. One of the novelties is that you can now unmount a VMFS datastore and detach a SCSI LUN through the API.
To be able to unmount a datastore, some conditions have to be met. In the vSphere Client you get an informative popup that tells what is prohibiting the datastore unmount. If not all conditions are met, you can not continue with the unmount.

Nice feature, but what for those of us that want to automate this ?

Update April 23th 2012: Use the RetrieveDasAdvancedRuntimeInfo method to find the actual datastores that are used for the heartbeat.

What to look for ?

From the popup, it is quite obvious what specific feature can prohibit the unmount of a datastore.

But if you look at Cormac’s post called What could be writing to a VMFS when no Virtual Machines are running?, there are a couple of other settings that might influence the ability to unmount a datastore.

In Cormac’s post you’ll find that a Distributed Virtual Switch configuration file or the presence of a scratch partition might also cause IO that would prevent a datastore unmount.

So I decided to expand the function with a check for those 2 additional causes.

The script

function Get-DatastoreUnmountStatus{
<#
.SYNOPSIS  Check if a datastore can be unmounted.
.DESCRIPTION The function checks a number of prerequisites
  that need to be met to be able to unmount a datastore.
.NOTES  Author:  Luc Dekens
.PARAMETER Datastore
  The datastore for which you want to chekc the conditions.
  You can pass the name of the datastore or the Datastore
  object returned by Get-Datastore
.EXAMPLE
  PS> Get-DatastoreUnmountStatus -Datastore DS1
.EXAMPLE
  PS> Get-Datastore | Get-DatastoreUnmountStatus
#>
  param(
  [CmdletBinding()]
  [Parameter(Mandatory=$true,ValueFromPipeline=$true)]
  [PSObject[]]$Datastore
  )

  process{
    foreach($ds in $Datastore){
      if($ds.GetType().Name -eq "string"){
        $ds = Get-Datastore -Name $ds
      }
      $parent = Get-View $ds.ExtensionData.Parent
      New-Object PSObject -Property @{
        Datastore = $ds.Name
        # No Virtual machines
        NoVM = $ds.ExtensionData.VM.Count -eq 0
        # Not in a Datastore Cluster
        NoDastoreClusterMember = $parent -isnot [VMware.Vim.StoragePod]
        # Not managed by sDRS
        NosDRS = &{
          if($parent -is [VMware.Vim.StoragePod]){
            !$parent.PodStorageDrsEntry.StorageDrsConfig.PodConfig.Enabled
          }
          else {$true}
        }
        # SIOC disabled
        NoSIOC = !$ds.StorageIOControlEnabled
        # No HA heartbeat
        NoHAheartbeat = &{
          $hbDatastores = @()
          $cls = Get-View -ViewType ClusterComputeResource -Property Host |
            where{$_.Host -contains $ds.ExtensionData.Host[0].Key}
          $cls | %{
            ($_.RetrieveDasAdvancedRuntimeInfo()).HeartbeatDatastoreInfo | %{
              $hbDatastores += $_.Datastore
            }
          }
          $hbDatastores -notcontains $ds.ExtensionData.MoRef
        }
        # No vdSW file
        NovdSwFile = &{
          New-PSDrive -Location $ds -Name ds -PSProvider VimDatastore -Root '\' | Out-Null
          $result = Get-ChildItem -Path ds:\ -Recurse |
          where {$_.Name -match '.dvsData'}
          Remove-PSDrive -Name ds -Confirm:$false
          if($result){$false}else{$true}
        }
        # No scratch partition
        NoScratchPartition = &{
          $result = $true
          $ds.ExtensionData.Host | %{Get-View $_.Key} | %{
            $diagSys = Get-View $_.ConfigManager.DiagnosticSystem
            $dsDisks = $ds.ExtensionData.Info.Vmfs.Extent | %{$_.DiskName}
            if($dsDisk -contains $diagSys.ActivePartition.Id.DiskName){
              $result = $false
            }
          }
          $result
        }
      }
    }
  }
}

Annotations

Line 24-26: A simple Object By Name implementation. It allows to pass the datastores by name or by object to the function.

Line 27: The parent of the datastore is used in a couple of tests further on in the script

Line 31: Test if there are any VMs or Templates present on the datastore

Line 33: Test if the datastore belongs to a Storage Cluster

Line 35-40: Test if the datastore is managed by Storage DRS. The Enabled property in the StorageDrsPodConfigInfo object is used as the indicator.

Line 42: Test if SIOC is enabled for the datastore

Line 44-54: Test if the datastore is one of the Heartbeat Datastores. The test first gets all the clusters known in vCenter and then filters out those clusters that have hosts on which the datastore is visible.

Line 56-62: Test to see if the is a Distributed Virtual Switch configuration file present on the datastore.

Line 64-74: Test if the datastore is used as a scratch partition.

Sample usage

The function can be used in a stand-alone or in a pipeline construct.

Get-DatastoreUnmountStatus -Datastore DS1

The pipeline construct could look like this

Get-Datastore DS* | Get-DatastoreUnmountStatus

The function produces 1 object per datastore. Each of the properties in the object represents 1 test that the function executes. Something like this.

The output shows us that we will probably not be able to unmount datastore DS1, because:

the datastore is part of a Datastore Cluster
the datastore is managed by Storage DRS
there are VMs (or Templates) on the datastore
the datastore is used by the Datastore Heartbeat
the datastore has SIOC enabled

The functions to solve this and allow an unmount of the datastore will follow in some future posts.

Enjoy !

Automate your VMTN search

LucD — Wed, 29 Feb 2012 21:26:54 +0000

Recently I had the pleasure of doing a guest post, called Finding your way in the PowerCLI Community, on the PowerCLI blog. The subject of the post was how to find community threads, that might hold an answer to your question.

Now this wouldn’t be a PowerShell/PowerCLI blog, if I didn’t try to automate the procedure. And with a serious amount of RegEx involved, I was able to create some working code. Here it is, my Find-VMTNPowerCLI function.

Warning: pure PowerShell, no PowerCLI content !

The script

function Find-VMTNPowerCLI{
<#
.SYNOPSIS  Search the VMTN PowerCLI related communities
.DESCRIPTION The function automates the functionality that
  is available through the Search page
.NOTES  Author:  Luc Dekens
.PARAMETER Question
  The question as you would enter it on the VMTN Search page
.PARAMETER Start
  The start date of the period in which you are looking for
  threads that fitt your search.
  The default is December 24th 2007.
.PARAMETER Finish
  The start date of the period in which you are looking for
  threads that fitt your search.
  The default is now.
.PARAMETER ReplyStart
  The start date of the period in which you are looking for
  replies that fitt your search.
  The default is December 24th 2007.
.PARAMETER ReplyFinish
  The start date of the period in which you are looking for
  replies that fitt your search.
  The default is now.
.PARAMETER Author
  Filter the returned threads to those started by one of the
  authors.
.PARAMETER ReplyAuthor
  Filter the returned threads to those where the reply came
  from one of the authors.
.PARAMETER PowerCLI
  Switch, search the PowerCLI Community.
  The default for this switch is $true.
.PARAMETER Onyx
  Switch, search the Onyx Community.
  The default for this switch is $false.
.PARAMETER UpdateManager
  Switch, search the UpdateManager Community.
  The default for this switch is $false.
.PARAMETER PowerShellers
  Switch, search the PowerShellers group.
  The default for this switch is $false.
.PARAMETER AutomationTools
  Switch, search the AutomationTools COmmunity.
  The default for this switch is $false.
.PARAMETER Descending
  Switch, when true the returned threads will be
  in a descending Reply Date order.
  The default for this switch is $true.
.EXAMPLE
  PS> Find-VMTNPowerCLI -Question 'svmotion thin'
.EXAMPLE
  PS> Find-VMTNPowerCLI -Question '5.0.1' -PowerShellers
.EXAMPLE
  PS> Find-VMTNPowerCLI -Question 'regex get-view' `
  >> -Start (Get-Date).AddDays(-14)
#>

  param(
  [CmdletBinding()]
  [parameter(Mandatory = $true)]
  [string]$Question,
  [datetime]$Start = (New-Object DateTime -ArgumentList 2007,12,24),
  [datetime]$Finish = (Get-Date),
  [datetime]$ReplyStart = $Start,
  [datetime]$ReplyFinish = $Finish,
  [string[]]$Author,
  [string[]]$ReplyAuthor,
  [switch]$PowerCLI = $true,
  [switch]$Onyx,
  [switch]$UpdateManager,
  [switch]$PowerShellers,
  [switch]$AutomationTools,
  [switch]$Descending = $true
  )

  function Get-VMTNSearch{
    param(
    [string]$url,
    [switch]$Group
    )

    $pst = [TimeZoneInfo]::FindSystemTimeZoneById("Pacific Standard Time")
    $t0 = Get-Date
    $t1 = [TimeZoneInfo]::ConvertTime($t0,$pst)
    $tdiff = $t0 - $t1

    $regBodyText = [regex]'(?s)\[DocumentBodyStart.*?-->(?.*)