VirtuallyCrazy

Using a Dynamic Alarm to Rescan Unavailable Datastores

noreply@blogger.com (Jason Willey) — Fri, 20 May 2016 00:55:00 +0000

Hi All,

So in our particular retail environment we have had an ongoing issue with Iscsi datastore availability in the case of a power outage. We also have the same problem on in store deployments when the onsite technicians power on the components in the wrong order (ie power on the hosts before the iscsi array). The result of this problem was of course escalations for retail stores being down for a significant period of time following a power outage.

Of course I googled the heck out of this problem - and the only solution I have found was to change the boot delay on the ESXi hosts to exceed the array boot time. That was not a great solution for us - because it would delay starts on planned reboots without a corresponding power outage.

One day I was working on another reactive alarm and I got that light bulb over my head. I thought "I wonder if there is an alarm type for datastore connectivity to hosts that I could use to trigger an action?". It turns out there is a condition that can be monitored at the datastore level to alarm for this - it is called Datastore State to All Hosts. In the screenshots below - I created a custom rule with this condition. For a review on setting up Powercli based alarms - refer to my blog post here.

Now that we have the alarm setup - let's get into the Powercli code behind it. In our particular implementation - we only have to worry about 3 nodes in our clusters, so my script is focused on that, the methodology would also work with a larger cluster and a foreach-object loop - but I am just going to use my already written script to show how I solved my particular problem, but the logic will scale.

# In response to a datastore alarm, remove all snapshots on the datastore.
# This is just for demonstration purposes.
# Find more info at http://blogs.vmware.com/vipowershell
$basePath = "C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\scripts"
$ProgressPreference = "SilentlyContinue"
$env:APPDATA = "c:\Documents and Settings\All Users\Application Data"

Add-PSSnapin VMware.Vimautomation.Core -ea SilentlyContinue
$WarningPreference = "SilentlyContinue"
Connect-VIServer localhost -User "nobody\someone" -Password "xxxxxxxxxxxxxxxxxxxxxx" | Out-Null
$email = "jason.willey@gmail.com"
$datastoreId = "Datastore" + $env:VMWARE_ALARM_TARGET_ID
$datastore = Get-Datastore $datastoreId
$impactedhosts = $datastore |get-vmhost
$cluster = $impactedhosts |select -First 1 |get-cluster 
# host one check
$Hostone = $impactedhosts |where {$_.name -match "x1"}  
$Hostoneconnected = $Hostone.ConnectionState
$hostonestate = get-vmhost $hostone |get-datastore $datastore
$hostonestate = $hostonestate.Accessible
if ($Hostoneconnected -match "connected") {
$hostonestate = get-vmhost $hostone |get-datastore $datastore
$global:hostonestate = $hostonestate.Accessible }
#host two check
$Hosttwo = $impactedhosts |where {$_.name -match "x2"}  
$Hosttwoconnected = $Hosttwo.ConnectionState
if ($Hosttwoconnected -match "connected") {
$hosttwostate = get-vmhost $hosttwo |get-datastore $datastore
$global:hosttwostate = $hosttwostate.Accessible }
# Host Three check
$Hostthree = $impactedhosts |where {$_.name -match "x3"}  
$Hostthreeconnected = $Hostthree.ConnectionState
if ($Hostthreeconnected -match "connected") {
$hostthreestate = get-vmhost $hostthree |get-datastore $datastore
$global:hostthreestate = $hostthreestate.Accessible}
#  starting rescan loop
$global:counter = 0 
while ($global:hostonestate -match "False" -or $globalhosttwostate -match "False" -or $global:hostthreestate -match "False" -and $counter -lt 10 {
if ($global:Hostonestate -match "False") {Get-VMHostStorage -VMHost $Hostone -RescanAllHba -RescanVmfs |Out-Null
 $global:hostonestate = get-vmhost $hostone |get-datastore $datastore;
 $global:hostonestate = $$global:hostonestate.Accessible  }
if ($global:Hosttwostate -match "False") {Get-VMHostStorage -VMHost $Hosttwo -RescanAllHba -RescanVmfs |Out-Null
 $global:hosttwostate = get-vmhost $hosttwo |get-datastore $datastore;
 $global:hosttwostate = $global:hosttwostate.Accessible  }
if ($global:Hostthreestate -match "False") {Get-VMHostStorage -VMHost $Hostthree -RescanAllHba -RescanVmfs |Out-Null
 $$global:hostthreestate = get-vmhost $hostthree |get-datastore $datastore;
 $global:hostthreestate = $global:hostthreestate.Accessible  } 
sleep 120
$global:counter = $global:counter + 1
if $counter -eq 9 { $global:hosterror = "$datastore has been rescanned every 2 minutes for 20 minutes and is not available on all nodes of $cluster"

}
if ($globbal:hosterror -ne $null) {Send-MailMessage -To $email -Subject "Unable to rescan ISCSI storage in cluster $cluster" -Bodyashtml -Body "$global:hosterror"  -SmtpServer "m.nobody.com" -From "VMware@nobody.com"}

So I will skip all the logic for interpreting the alarm received as that is detailed in another post here.

First things first - I translate the actual datastore ID presented by the alert to a more script friendly datastore name and get the hosts that are supposed to be connected to this datastore. I also get the cluster name for use later on when we send an email alert in the case we were unable to successfully rescan.

$datastoreId = "Datastore" + $env:VMWARE_ALARM_TARGET_ID
$datastore = Get-Datastore $datastoreId
$impactedhosts = $datastore |get-vmhost
$cluster = $impactedhosts |select -First 1 |get-cluster

From there - I now verify that each host is actually connected to vcenter, and verify if the datastore is viewed as available from the host side. I then repeat this logic for each possible host.

$Hostone = $impactedhosts |where {$_.name -match "x1"}  
$Hostoneconnected = $Hostone.ConnectionState
$hostonestate = get-vmhost $hostone |get-datastore $datastore
$hostonestate = $hostonestate.Accessible
if ($Hostoneconnected -match "connected") {
$hostonestate = get-vmhost $hostone |get-datastore $datastore
$global:hostonestate = $hostonestate.Accessible }

Now I set up the rescanning loop. In the case of our environment we have chosen to rescan every two minutes for twenty minutes and then send an email alert to second level support. I used $global: variables here so they will survive the loop. So after the rescan, it checks the status of the datastore again, and continues running the loop until all the datastores have returned, or the loop counter reaches 10. At the end of each pass of the loop there is a 120 second sleep to give us our two minutes between rescans. One the last iteration of the loop - it writes an error message that after 20 minutes of 2 minute rescans - the cluster still has a datastore availability problem.

$global:counter = 0 
while ($global:hostonestate -match "False" -or $globalhosttwostate -match "False" -or $global:hostthreestate -match "False" -and $counter -lt 10 {
if ($global:Hostonestate -match "False") {Get-VMHostStorage -VMHost $Hostone -RescanAllHba -RescanVmfs |Out-Null
 $global:hostonestate = get-vmhost $hostone |get-datastore $datastore;
 $global:hostonestate = $$global:hostonestate.Accessible  }
if ($global:Hosttwostate -match "False") {Get-VMHostStorage -VMHost $Hosttwo -RescanAllHba -RescanVmfs |Out-Null
 $global:hosttwostate = get-vmhost $hosttwo |get-datastore $datastore;
 $global:hosttwostate = $global:hosttwostate.Accessible  }
if ($global:Hostthreestate -match "False") {Get-VMHostStorage -VMHost $Hostthree -RescanAllHba -RescanVmfs |Out-Null
 $$global:hostthreestate = get-vmhost $hostthree |get-datastore $datastore;
 $global:hostthreestate = $global:hostthreestate.Accessible  } 
sleep 120
$global:counter = $global:counter + 1
if $counter -eq 9 { $global:hosterror = "$datastore has been rescanned every 2 minutes for 20 minutes and is not available on all nodes of $cluster"
}

The last step once the loop has been terminated is to check if the error message has been populated - send an email out to the second level support team letting them know that automatic rescanning has not been successful.

if ($globbal:hosterror -ne $null) {Send-MailMessage -To $email -Subject "Unable to rescan ISCSI storage in cluster $cluster" -Bodyashtml -Body "$global:hosterror"  -SmtpServer "m.nobody.com" -From "VMware@nobody.com"}

I am hoping that you find this useful, as I have heard this is a very common problem, especially with Iscsi implementations. Leave me a comment if you have any questions or suggestions, or just leave a comment if you had a similar problem and solved it in a different way.

Powercli VM hardening script

noreply@blogger.com (Jason Willey) — Thu, 05 May 2016 05:10:00 +0000

Someone on the Linkedin Powercli Forum (a great group) asked if anyone had a VM hardening script. I was working on one based on the output of our VROPs implementation. This may not contain all of the settings available in the hardening guide, but it did take care of most of the ones that VROPS was alerting on.

One important caveat : the vm needs to be shut down when you run this script, as all the advanced settings are locked while the VM is running.

Param(
  [Parameter(Mandatory=$True,Position=1)]
  [string]$targetvm
)
$vm = Get-VM $targetvm
$vm  |New-AdvancedSetting -name 'log.keepOld' -Value 10 -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.ghi.launchmenu.change' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.device.edit.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.hgfsServerSet.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.toolsautoInstall.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.unity.push.update.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.disk.Wiper.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.ghi.protocolhandler.info.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'RemoteDisplay.maxConnection' -Value 2 -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.vmxDnDVersionGet.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.bios.bbs.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.unity.taskbar.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.diskShrink.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.unity.windowContents.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.unityInterlockOperation.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.ghi.trayicon.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.vixMessage.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.ghi.autologin.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.device.connectable.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.monitor.control.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.memSchedFakeSampleStats.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'log.rotateSize' -Value 1024000 -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.unityActive.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.getCreds.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.ghi.shellAction.disable' -Value false -confirm:$false
$vm  |New-AdvancedSetting -name 'isolation.tools.trashFolderState.disable' -Value false -confirm:$false

so I wrote this with the parameter -targetvm as a parameter. Then I can call it on any subset of machines I choose such as Get-folder dev |get-vm |foreach-object {./vmsecurityupdate $_.name}

Most of the parameters above were recommended against "default build" VMs, so it is likely if you you ran the VROPs VM hardening alert you might see the same reccomendations. You may want more settings.. or maybe less depending many business factors. The easy way to plan your settings is to do a get-advancedsetting vmname |select * and find out what setting are important to you or your organization. My long term goal is to get this script into our build automation so every VM we push out would have an improved security posture.

I hope this helps out.

Creating a reactive alarm for ESXi ROBO cluster or as I call it "DRS-light"

noreply@blogger.com (Jason Willey) — Sat, 23 Apr 2016 01:08:00 +0000

I have decided to start blogging again after a long time away. Eight years to be precise.

The first item I want to share is a solution for a problem that occured in my company's Retail implementation that uses VMware ROBO licensing. For anyone that is familiar with ROBO - you know that there is one major exclusion that makes cluster management challenging - and that is the lack of DRS. In any large scale implementation without DRS I think it is fairly common to end up with unbalanced workloads after maintenance or outages. This problem is compounded when you have a large support team of varying skill levels.

How I chose to address this problem was to create an alarm based on Host Memory Usage, as Host memory is generally the first problem that occurs in our particular environment. This methodology would also work for CPU constraints, but that is not a problem we had.

My starting point was the excellent article from the Powercli blog on using Powercli scripts in actions.

One thing to note : the syntax suggested in the article for the actual alarm action did not work for me - so I used the following path to execute the Powershell code with the alarm :

in case that type is tool small to read - I used the following to call Powershell
(instead of the batch file method suggested in the original article) :
c:\windows\system32\WindowsPowerShell\v1.0\powershell.exe

Here is the full code of the script I am using - I will detail out the the purpose section by section (starting at line 1 as the rest is covered in the original blogpost by VMware pretty pretty well:

# In response to a host alarm, add a VM to that host.
# This is just for demonstration purposes.
# Find more info at http://blogs.vmware.com/vipowershell
$basePath = "C:\PS_SCRIPTS\Alarms"
$ProgressPreference = "SilentlyContinue"
$env:APPDATA = "c:\Documents and Settings\All Users\Application Data"

## Import the admin credential.
#. "$basePath\credentialManagement.ps1"
#$credential = Import-PSCredential "$basePath\systemCredentials.enc.xml"

# Log in.
Add-PSSnapin VMware.Vimautomation.Core -ea SilentlyContinue 
$WarningPreference = "SilentlyContinue"
Connect-VIServer localhost -User "somedomain/someuser" -Password "xxxxxxx"
$hostId = "HostSystem-" + $env:VMWARE_ALARM_TARGET_ID 
$vmhost = Get-VMHost -Id $hostId 
$cluster = Get-VMHost $vmhost |Get-Cluster 
$vm = get-vmhost $vmhost |get-vm |where {$_.name -ne "z*"}|sort-object memorygb -descending |select -first 1 
$destination = get-cluster $cluster |get-vmhost |where {$_.name -ne $vmhost} |sort-object MemoryUsageGB |select -first 1 
$destinationfree = ($destination.MemoryTotalGB - $destination.MemoryusageGB) 
$vmhostfree = ($vmhost.MemoryTotalGB - $vmhost.MemoryUsageGB) 
$vmsize = ($vm.memoryGB) 
$difference = ($destinationfree - $vmhostfree)
if (($difference - $vmsize) -gt 0) { Move-VM -VM $vm -Destination $destination -Confirm:$false} 
$count = $cluster |get-vmhost
$count = $count.count
else {Send-MailMessage -To "somepeople@somecompany.com" -Subject "Unable to Balance Retail Cluster $cluster - $date" -Body "Unable to balance Cluster $cluster due to insufficient resources available. There are currently $count Host(s) available in the Cluster."  -SmtpServer "m.somecompany.com" -From "vmware_team@somecompany.com" }

Now I will break down what the various lines of the script are doing :

$hostId = "HostSystem-" + $env:VMWARE_ALARM_TARGET_ID 
$vmhost = Get-VMHost -Id $hostId 
$cluster = Get-VMHost $vmhost |Get-Cluster

These three lines are taking the Alarm value ($env:VMWARE_ALARM_TARGET_ID) and converting it to a more usable form for Powercli commands. In this case the alarm returns the hostId number, but without the prefix to query it using get-vmhost. So the first operation I do is add the HostSystem- to the id to make it easier to query. Then I retrieve the object for the vmhost and the cluster for future actions :

$vm = get-vmhost $vmhost |get-vm |where {$_.name -ne "z*"}|sort-object memorygb -descending |select -first 1

The next objective is to get the list of VMs on the host that is having memory pressure and sort it based on the most memory usage. The line where {$_.name -ne "z*"} is not necessary in most environments, but in our case we have a guest that starts with z that is considered "immobile" :

$destination = get-cluster $cluster |get-vmhost |where {$_.name -ne $vmhost} |sort-object MemoryUsageGB |select -first 1

This next lines then find the host in the cluster with the most memory free and stores that variable for a later calculation :

$destinationfree = ($destination.MemoryTotalGB - $destination.MemoryusageGB) 
$vmhostfree = ($vmhost.MemoryTotalGB - $vmhost.MemoryUsageGB)
$vmsize = ($vm.memoryGB)

Then the current Host free and VM guest memory usage, and difference between the source and destination host are stored as variables for calculations :

$difference = ($destinationfree - $vmhostfree)
if (($difference - $vmsize) -gt 0) { Move-VM -VM $vm -Destination $destination -Confirm:$false}

Now all the collected variables are used to calculate if you should move the vm to another host. The If statement is used to make sure that you will not move a VM to another host with less available memory than the original host:

$count = $cluster |get-vmhost
$count = $count.count

Now we gather some data in case we were unable to move a vm (such as if there is no other node with more free RAM) :

else {Send-MailMessage -To "somepeople@somecompany.com" -Subject "Unable to Balance Retail Cluster $cluster - $date" -Body "Unable to balance Cluster $cluster due to insufficient resources available. There are currently $count Host(s) available in the Cluster."  -SmtpServer "m.somecompany.com" -From "vmware_team@somecompany.com" }

The last step is now to send an email to the support team with some context around the error message so they can investigate. This is to cover in case a cluster is missing nodes.

So that is my solution for balancing memory in a DRS-less ROBO environment

If you have any thoughts or suggestions - please comment.

VMware EVC - incompatible ?

noreply@blogger.com (Jason Willey) — Fri, 24 Oct 2008 03:47:00 +0000

When installing some brand spanking new DL585 Quad Core systems I was unable to initially enable EVC for the new servers. Both new servers were showing as having incompatible hardware.

The answer was the No Execute Page Protection being disabled by default. I was also surprised to find that the AMD Virtualization support was also disabled by default. I also saw some posting in the VMware forum that other newer HP models have the same default setting. Once the settings were adjusted, EVC was no longer reporting incompatible hardware.

VMware 3.5u2 Hardware Monitoring

noreply@blogger.com (Jason Willey) — Sat, 27 Sep 2008 00:33:00 +0000

In the ESX 3.5u2 release notes there is a brief blurb about the display of system health information:

"Display of System Health Information – More system health information is displayed in the VI Client for both ESX Server 3.5 and VMware ESX Server 3i."

I don't think that blurb did this feature justice.

Expanding some of the nodes shows a surprising level of detail.

I think VMware definitely undersold this feature in the release notes.

VMware Upgrade Manager - Using Baselines to Check Patch Compliance and Perform Remediation

noreply@blogger.com (Jason Willey) — Fri, 26 Sep 2008 01:39:00 +0000

Patching servers or guests using VUM is based on application of baselines. You can apply baselines at any level in the hierarchy of Hosts & Clusters for Host based updates or at any level of the hierarchy of Virtual Machines and Templates for Guest based updates. Baselines can either be static or dynamic, and can be manipulated via the baseline tab of VUM. In the screen shot below it shows the default baselines created by VUM, which are all dynamically updated baselines.

If you choose to create a new baseline you can customize it for Host or Guest Updates, whether the baseline is Fixed or Dynamic. After selecting these criteria, this is where creating a baseline gets interesting. You can select products, update severity, language, date range criteria, and which vendor updates your baseline will include. Currently vendors include VMware, Microsoft, Apple, Mozilla, and Adobe. That is a much larger list than I expected. You can also choose specific updates to include or exclude on subsequent screens.

You can also check patch details from the subsequent inclusion/exclusion screen (or from the update repository tab). I chose this update in particular to display, because I was surprised to find in among the included updates.

Now that we have created baselines or used the default baselines, now it is time to attach them. If you want to applying a baseline to an ESX host, you would apply them in the Hosts and Clusters view of the inventory. If you are choosing to apply guest based baselines then you would apply them in the Virtual Machines and Templates. You can apply them at any container level within the hierarchy. I chose to deploy our host based baselines at the datacenter level.

when you attach the baseline you will be asked to select which applicable baseline you wish to apply to the selected container. Once the baseline is applied you will get a view that shows the number of hosts that are compliant with the baseline, the number that are not compliant, and the unknown servers (which have not yet been scanned)

Now from each host (or guest) you will now have additional options in the context menu, Scan for Updates and Remediate.

before you scan for updates, make sure that your update repository has already been populated. When I scanned without a populated repository, I received and error stating that the scan could not be completed. You can scan from any level in the hierarchy that has an applied baseline. For the purposes of this blog, I scanned at an individual host level. Once the scan has been completed, you will now see an updated screen showing you the updates compliance status based on your scan.

Now it's time to remediate. clicking remediate will bring up a wizard, asking you to select the baselines that you will be applying. and allows you to select include or exclude updates contained within the baselines.

Now you are asked for when you will apply the updates, and you can have an opportunity to adjust the default failure options.

Normally I wouldn't publish a confirmation screen, but there is one specific thing I would like to point out. This is the only place in this wizard where you can see the scope of the update remediation. Since this tool will try to force your servers into maintenance mode, including rebooting servers that are unable to enter maintenance mode (if you have selected that option), it is good to confirm that the scope is limited to what you have intended to patch.

You will now see a task for remediation on the selected hosts, and after it is completed, you will be able to see an event for each patch applied as well as events for entering maintenance mode, and rebooting the server.

In my opinion the time spent configuring VMware Update Manager is time well spent, as it will save much more time in the long run. It's very nice to see VMware follow in the footsteps of Microsoft SUS and WSUS in making a free patch management tool, as opposed to going in the direction of "paid updates" model.

Configuring VMware Update Manager

noreply@blogger.com (Jason Willey) — Wed, 24 Sep 2008 01:36:00 +0000

As mentioned yesterday, in this post I will discuss how I have configured VMware Update Manager, and even show some of the options that I chose not to configure.

First configuration option that I set, and possibly the most important, depending on your network design was the proxy configuration. Instead of redacting the server names, I will be replacing them with fake names. One thing to take note of - VUM does not support proxy auto-detection. If you have a proxy, you will need to configure it here.

Once you have configured your proxy, you can go to the update downloads to configure what will be downloaded and when.

If you click on the Edit Update Downloads object, it will present you with a wizard for configuring which updates you want , and when it will retrieve them. In our environment we already have a very good solution for patching all of our servers, so we will be only downloading ESX host updates.

At this point you can set your schedule, you can setup one time updates, at server startup, hourly, daily, weekly, and monthly. At this time it appears that you can only have one update job per implementation of VUM, so plan accordingly.

Now you can enter email addresses for people that you would like to be notified of new updates.

You can also configure options around guest and host updates. You have the option to snapshot the virtual machines before applying the updates, allowing an easy fallback in the case of an virtual machine having issues from a patch update. You can also choose to keep that snapshot for up to 100 hours, or to not automatically delete it.

In the ESX Host Settings, VUM reminds you that host patches are only installed when an ESX server is in maintenance mode and prompts you for how to react to not being able to place the server in maintenance mode. It can Fail the Task (writing an event), Retry the job, Power Off and Retry, and Suspend and Retry. All the retry options include the ability to limit the number of retries and insert a delay between retries.

One last configuration screen is Port Settings. If you had any need to change the ports that VUM is using to communicate, here is where it would be done. I wouldn't recommend changing them unless you have a specific security requirement.

Now that you have VUM configured, you can check the Events tab for information on update downloads, scanning and updates to VUM configuration

One other support tool that is included on the server where you installed VUM is the Generate Update Manager log bundle tool. This tool creates a zip file on your desktop with all the logs and configuration files for VMware Update Manager that you can either examine or forward on to your VMware support. By default this icon is in the root of VMware folder under Program Files.

In my next post I will go over the VMware Update Manager repository and the information available there, as well as creating, editing, and using baselines.

Installing VMware Update Manager

noreply@blogger.com (Jason Willey) — Tue, 23 Sep 2008 01:53:00 +0000

So I was installing VMware Update Manager on my existing Virtual Center server, so I thought I would share the installation experience. Before installing, make sure to check out VMware's Release Notes. It also can't hurt to read through the Administrator's Guide

I used the zip file for Vmware-VIMSetup2.5.0-104263.

First you run VMware-Update-Manager.exe from the \updatemanager directory to start the installation.

After thoroughly reading the legal notice and accepting it, you are prompted to select your installation folder and your folder for downloaded patches.

The directory for downloading patches needs to have at least 18GB of space, or you will be forced to choose another location.

Now you will be prompted for a Virtual Center login.

Now you will be asked for which type of database you will use. You can use either locally installed SQL Express, SQL (via ODBC), or Oracle.

We chose SQL server, so at this point you are prompted to create a DSN.

confirm or change the default ports and server name. You can also setup a proxy in this stage if you need one.

If you have the VI toolkit for PowerShell installed you get a bonus of the Update Manager toolkit being installed as well.

Now you can install the plugin via the Virtual Center client's Plugin Manager to install VMware Update Manager Extension version 1.0u2

After selecting your language preference you are presented with another installer.

Thoroughly read the legal notice and accept it.

The install will now continue and exit normally.

To enable the plugin, you need to check the Enabled checkbox on the Plugin Manager's installed plugins tab.

Now you will see a button for Update Manager in the toolbar.

In my next post I will go into the configuration of VMware Update Manager, such as setting update frequency, and controlling what updates are downloaded.

In case you need to change the location where your updates are being stored. Gabe over at Gabe's Virtual World has an article on how to change the location of the updates : http://www.gabesvirtualworld.com/?p=28

Coming Attractions - PowerShell for XenServer

noreply@blogger.com (Jason Willey) — Wed, 17 Sep 2008 11:50:00 +0000

Citrix continues their announcements this week with the announcement of their very own PowerShell SnapIn for managing XenServer environments.

"Now that XenServer and XenCenter 5.0 have been released, we turn our attention to other things. One of those new projects is a PowerShell SnapIn for XenServer, which I'm pleased to announce today.

The new SnapIn is with private beta testers now, and will be available through this site very soon. "

It seems most of the Citrix announcements are getting lost in all the great announcements from VMworld. Unfortunately I couldn't make it to VMworld this year, but luckily fellow virtualization bloggers like Rich Bramley at VM/ETC and Scott Lowe at his self-titled blog are keeping the rest of us up to date on the latest happenings.

One "off beat" story coming out of VMworld is about a reported guerilla marketing campaign , that either is either a surprisingly aggressive attack focused on VMware at their own conference, or a great big hoax. Personally my bet is on big hoax, but only Microsoft and the individual that registered the website know for sure at this point in time.

XenServer 5 - Initial Impressions

noreply@blogger.com (Jason Willey) — Mon, 15 Sep 2008 23:24:00 +0000

You have to love VMworld, it seems to bring out the best in the virtualization industry. Last week was Microsoft's big announcement, and today Citrix had a big announcement of their own. Unfortunately I couldn't make it to VMworld this year, but that did afford me an opportunity to take a look at XenServer's new release. I will be using XenServer 5.0 Express for the purposes of my testing.

I started with the "check for upgrades" I praised in a previous blog, expecting to be informed that there was a new version. Using XenCenter 4.1 it did not see any new updates. Since it was just released today, I can cut them a break on that.

When I downloaded the XenCenter upgrade, I noticed the filename contained "RC4". That is not enough to worry me, yet.

My first step was updating the client, as I have learned from VMware.. always update the management tools first. On launching the new version of XenCenter I received a dialog letting me know that checking for updates could now be scheduled. A very nice addition to an already convenient feature.

After upgrading XenCenter to version 5.0.0 the upgrade tool was even more confused about available versions. Not being able to recognize 5.0. Hopefully this will be taken care of shortly.

The server upgrade process is much like the initial installation process. When performing the upgrade the installer creates a backup of the current installation on the backup partition, which you can restore with the host-restore function. a very nice safety net. Dmitry Kushak expands on this process on the Citrix forum

At this point - my installation ceases to function at this point, informing me that a base installation could not be found when using the cdrom installation source. Another user indicated that he downloaded the ISO again, and did not have the same issue. I tried that but it didn't help. So I copied the CD up to my webserver, and attempted to do use the http install method instead of the cd based installation. This installation worked. Very odd that it worked, considering I used the same media that wouldn't boot for the webserver source.

On reboot the new bootloader and xsconsole feature are very apparent. Now after a XenServer splash screen it boots to the xsconsole menu.

This is a very nice interface that allows you to perform administrative tasks without memorizing commands. For example - start a VM.

Interact with your storage repositories

Configure and test your network connections:

Overall it looks like XenServer is making some moves to be more competitive with VMware, as their HA solution is starting to look quite a bit like VMware's version. I also see in the release notes some new guest support and a fair amount of new hardware support, such as 8GB HBAs and 10G NICs. For a more complete list of features download the release notes or check out Peter Levine's blog entry

One new guest note stood out for me :

"Windows Server 2008 32-bit and 64-bit support, with WHQL signed para-virtual drivers and initial enlightenment
optimizations."

I think that is the first time I have seen the use of enlightenment optimizations outside of Microsoft's Hyper-V materials. A search on Windows paravirtualization turns up an interesting article about the importance of installing the guest tools.

All in all, the update experience was pretty good, outside of the CDROM installation media issue. Now the download experience, that was not not pleasant. I would expect Citrix to want to keep track of who downloaded their software, as it could lead to future sales. I also don't mind filling out a short survey... once. During the process of writing this article, I had to fill out the survey 6 times. It wasn't just for downloading XenServer or XenCenter itself, it was also for downloading the documentation. If you closed the browser - you have to fill out the survey again. Even if you login to your MyCitrix account, you still have to fill out the survey again. There's a fine line between collecting valuable marketing information and driving off customers - and I would hate to see Citrix on the wrong side of that line.

Update:

Jonathan Thorpe of Citrix jumped on the problem with the CDROM installation method (I guess I'm not the only person not at VMworld today) and pointed me toward a workaround that will allow me to install from CD media. it is simply to drop to a shell during the installation (press F2) and type "lsmod ide-generic". It appears that this module needed to be loaded for the installer to recognize my particular combination of Motherboard and IDE DVD drive. you can read about it here: http://forums.citrix.com/thread.jspa?messageID=1341120&

Thank Jonathan!

Microsoft Shakes Up the Virtualization World Yet Again

noreply@blogger.com (Jason Willey) — Tue, 09 Sep 2008 02:32:00 +0000

Today was a big day full of announcements for Microsoft, as most of these announcements have been covered in detail by other bloggers, I am going to link to their stories and give my own analysis and commentary instead of asking everyone to read the same press releases.

First up - Microsoft Hyper-V Server 2008

One thing I have heard consistently when discussing Hyper-V is that "You still have to buy an operating system license. So ESX3i and XenServer Express are still cheaper. "

Pretty soon that will not be true anymore, as Microsoft announced that not only will their new bare metal, designed from the ground up hypervisor be released with 30 days, it will be free, which happens to be my favorite price.

Virtualization.info has an excellent article on the Virtual Server 2008 announcement today. Release : Microsoft Hyper-V 1.0

Next up : System Center Virtual Machine Manager 2008

This will be released within 30 days. One thing I found surprising about this announcement is that they will support management of VMware VI3, they will not be supporting XenServer. I would not have expected this, considering how close Citrix and Microsoft are these days.

Lastly on the server virtualization side was Live Migration.

Microsoft seems to have realized that the lack of Live Migration was holding them back from competing directly with VMware, especially when it comes to Enterprise class environments. the good news is that it will be released with Windows 2008 R2. The bad news..it will likely not be in our hands until 2010.

I think this is a great announcement for all the consumers of virtualization technology, as VMware, Microsoft, and Citrix will all be pushing to achieve the be the leader in the virtualization market of the future. To use a boxing metaphor, VMware is still the undisputed champion today, but they will need to fight hard to stay there.

Read Yellowbricks.com's scoop on the subject : FYI: Microsoft’s live migration

There were many other announcements including :

Microsoft creating a virtualization lab for partners to validate their solutions.
Hyper-V compatibility for the Server 2008 logo program.
Sun plans to offer a Solaris Operating System as a certified guest under Hyper-V, and Windows 2008 to be certified under Sun xVM.
NetApp announced Hyper-V optimized solutions.

Definitely a big day for press releases.

A Hands On Review of SearchMyVM

noreply@blogger.com (Jason Willey) — Sat, 06 Sep 2008 00:17:00 +0000

In my last post I discussed the OVF import process, so after some hands on experience with the SearchMyVM appliance, I'm ready to give my impressions.

Let me apologize in advance for where I have edited the screenshots. I have removed anywhere that it referenced our server names or IP addresses.

Starting with initial bootup you are presented with a text based menu to setup your network configuration

After confirming your IP information, your DNS, and an NTP server you are directed to connect to the IP address of the server where you are presented with a "google-like" screen that presents you with some sample queries.

First you will need to point to your Virtual Center. You will need to use the FQDN for your server, and provide a user id that has the ability to read objects.

So now it SearchMyVM will index your VMware Environment. In our environment it only took about 15 minutes until all the information showed up. You may receive incomplete query responses until the index is built.

So If you need a function that is not on the common list there is a visual query editor that allows you to build simple or complex queries from a series of drop down lists. They fully explain all objects, including the type of parameter

Additionally if you would like to read additional information about each parameter you can access the detailed help files from the main interface screen.

The results returned are static, you can't click on them or export them from within the interface (although there are some PowerShell techniques for exporting web data)

In summation, here's my impressions of the appliance.

Key strengths :

Extremely easy configuration. You can have this appliance up and running within 15 minutes.
Support for the OVF format. This cuts down the provisioning time dramatically.
Fast Indexing - Our indexing appeared to cover 500 guests and 25 hosts within 15 minutes of connecting to virtual center. Although I was only searching for base objects like virtual machines and hosts at that time, other objects like VMware Tools status may have taken longer.
Very well written help files and examples - I don't see any written documentation available on their website yet, but the help files contained within the appliance are more than enough to get started.

Thing I would like to see in future releases:

The ability to export query results to remote shares. Being able to view query results is good, but being able to *use* query results would make this product a must have.
The ability to drill down into search results. Search results are currently static.
Plain English queries. Currently the query format is very structured, it would be nice to be able to use commands like "show vms with attached cdrom".
Support for SCVMM and XenCenter.

Vkernel has setup up a section of their forum for discussing the product at http://www.vkernel.com/forum/viewtopic.php?t=120 . If you would like to contribute, you will need to register with their forum. I will be watching future releases of the appliance as it appears to have potential, especially for large environments that lack PowerShell scripting talent.

Importing OVF File From the Internet in VirtualCenter

noreply@blogger.com (Jason Willey) — Thu, 04 Sep 2008 18:46:00 +0000

I'm sure many of you have seen the announcement of the SearchMyVM from Vkernel. I was intrigued by the product, so I signed up to download it. They mentioned a direct link to the OVF file, so I poked around in Virtual Center and noticed the Virtual Appliance tab. I had read about this function, but had no reason to actually use it in our VMware environment at prior to now. Vkernel gave me a reason, so I thought I would share the experience.

From this menu you can enter a url to directly download a OVF file from. In this example I used http://downloads.vkernel.com/downloads/vkernel/ovf/VK_Search_1.0_Beta/VK_Search_1.0_Beta.ovf

Before continuing you have they show you how big the download will be, and how much disk space it will consume.

Once you have selected the name for the appliance, where you will store the files, and what network you will be using Virtual Center will grab the OVF file and import it

Once this process is complete you have a virtual machine configured with the manufacturer recommended specifications with almost no effort.

I'm really impressed with this process. Hopefully application vendors will embrace this format as the new way to deploy test machines. If this is embraced widely in the consumer level virtualization products, this will be a boon for IT people studying for certification exams at home as well.

I also wanted to mention a new member of the virtualization blogging family : Mark Bradley. Not only is Mark a good friend of mine, and an expert consultant in the UK with a great technical mind especially when it comes to scripting. I'm expecting some great stuff once he gets underway with the VI PowerShell Toolkit for VMware. His site is http://www.ridethevirt.blogspot.com/

While You are Learning PowerShell...

noreply@blogger.com (Jason Willey) — Mon, 01 Sep 2008 21:33:00 +0000

I think everyone sees the potential of PowerShell for managing Windows 2008, as well as for managing VMware, but learning PowerShell is not an overnight task for most of us. So while we are all pondering the scripts posted in the VMware PowerShell scripting contest, and feverishly reading up on PowerShell - I thought I would post one of my favorite "old school" VMware utility sites.

www.run-virtual.com

a colleague in a Redhat class had told me about this site when we were talking about implementing trunking, and how many changes that would take in our fairly large environment. He suggested I check out the ITQ VLan and Portgroup Manager. It was definitely a time saver.

We also tried some of the other utilities on his site, and found them to be useful. The gentleman that runs the site is actually a technology evangelist for VMware in the EMEA region.

Some of these tools could definitely fill the gap while we are all becoming VMware PowerShell masters.

Applying XenServer Updates

noreply@blogger.com (Jason Willey) — Mon, 01 Sep 2008 02:02:00 +0000

I've been playing with the XenServer 4.1 product on a low end spare server I have laying around. To it's credit XenServer is the only one the big three hypervisors that will run on this Pentium D 820 server, as Hyper-V requires DEP, 64bit and processor side virtualization, and ESX3i requires specific approved hardware to function. Granted, without the processor virtualization assistance, XenServer is limited to deploying Linux guests only, it's still enough to be able to play with the interface and deploy some basic machines.

While not a complex and deep as VMware Update Manager, I am impressed with Citrix's simple "just the servers" approach to patching. While we are deploying VMware Update Manager to handle the ESX hosts, I can't see us ever using the guest patching capabilities, nor could I see any large organization doing so. In my opinion , most large IT shops won't want to maintain a separate update process for virtual machines and a separate update process for physical machines, despite the ability to roll back changes and other benefits of VMware Update Manager.

One of the best features of Citrix's product is their "Check For Updates" launcher. It gives you a clear view of what updates you already have, and which updates you should apply. Along with the download links are links to the patch notes. While it is much more rudimentary than VMware Update Manager, I appreciate the simplicity.

The Citrix patching utilities obviously don't compare with VMware Update Manager, but they certainly have the advantage over the VMware patching website. :

I found the deployment of the updates once they were downloaded to be incredibly simple, and the updater to be intelligent when it came to evaluating the server receiving the patches. It first attempted to evaluate moving the guest to another host, which was impeded by the fact I only have one server. Once it realized there were no migration options, it suspended the VM while applying the update and rebooting. After communication was re-established with XenCenter, the guest was resumed automatically. End to end, a pretty simple and intelligent process.

I hope VMware and Citrix (as well as Microsoft) are learning from each other, and improving their products based on their competitors strengths and weaknesses. I would like to see VMware implement a "simple" host updater into a future version of VirtualCenter. I will also be watching to see if XenCenter implements some of the more advanced features of VMware Update Manager directly into XenCenter.

PowerShell for VMware

noreply@blogger.com (Jason Willey) — Thu, 28 Aug 2008 08:44:00 +0000

Rich over at VM/ETC just posted a great bunch of links to various PowerShell scripts that he has run across in his Internet travels, as well as some links on learning the PowerShell scripting language.

http://vmetc.com/2008/08/27/powershell-scripting-examples-for-vmware-virtual-infrastructure/

One other link I had planned on posting as well was one from the Planet VMware blog : Managing VMware with PowerShell Webinar

I expect that the VMware Developer Center's PowerShell scripting contest should yield many interesting new VMware PowerShell scripts once it's finished.

Free Microsoft Training - How Cool is That ?

noreply@blogger.com (Jason Willey) — Mon, 25 Aug 2008 21:30:00 +0000

Have you been waiting for an introduction to the some of the newer Microsoft technologies? Wait no longer, Microsoft Learning has something prepared for you, assuming you have a few hours to spare.

Microsoft is branding them as clinics, but I think the name doesn't do them justice.    They are interactive and engaging combinations of video presentations, demos, interactive content, and quizzes.   Now I wouldn't recommend them as study guides for the MCITP exams, but they are definitely not "fifty thousand foot view" marketing slides either.   Most of them appear to have a 90 day subscription, so if you don't have 2 whole hours to spend, you can chip away at it a little at a time.

Clinic 5935: Introducing Hyper-V in Windows Server 2008
Clinic 5938: Introducing Terminal Services Presentation Virtualization in Windows Server®2008
Clinic 5936: Introducing Security and Policy Management in Windows Server® 2008
Clinic 5939: Introducing Server Management in Windows Server®2008
Clinic 5937: Introducing Branch Office Management in Windows Server®2008

To review all the free clinics available from Microsoft click the link below.   There's a large variety of content available at the best possible price.   I suspect some of the material is only free temporarily, so don't wait too long to give it a try.
Microsoft Learning - Free Content

They also appear to be releasing a large amount of content for a $39 usd per two hours of content price point as well.   I plan on giving a couple a try and assessing whether they are still a great value at $39.

VMware Joins the SVVP

noreply@blogger.com (Jason Willey) — Thu, 21 Aug 2008 06:44:00 +0000

In a letter from the CEO of VMware today it was announced that VMware is in the Server Virtualization Validation Program. I'm fairly sure they weren't on the list yesterday when I checked. This officially ends any ambiguity of whether you will be fully supported using VMware with Microsoft products.

VMware in the SVVP is a great win for them, but I don't think it dulls Microsoft's new advantage of having the only virtualization platform that is guaranteed not to have to be reproduced on physical hardware.

You can read VMware's statement here :
The Console : VMware is in the SVVP

One Small Press Release for Microsoft, One Giant Leap for Virtualization

noreply@blogger.com (Jason Willey) — Wed, 20 Aug 2008 05:36:00 +0000

Microsoft made a major annoucnement today regarding the licensing and support of their products in the virtual environment, and even threw a curveball in one of the linked kb article.

First up, the shift in licensing policy. previously Microsoft licensing was tied to server named instances, with a single move permitted every 90 days. This announcement seems to be suggesting a move toward capacity on demand. The other part of the announcement was a brief explanation of the much longer new Microsoft whitepaper on licensing on virtual hardware, found here: Microsoft Server Products in Virtual Environments I didn't find anything too surprising in the document.

The other half of the announcement was an explanation of what the SVVP is, and how it is going to help you when you need to call Microsoft for support. The SVVP is the Server Virtualization Validation Program. Similar to the move by Citrix to expands the ranks of it's application ISV partners, Microsoft has created a program to enhance cooperation among TSanet members with a focus on virtualization. The cost of entry is not too bad, as they are requiring a TSanet membership (this appears to be $1500 annually), and $250 per virtualized configuration. It will not be a logo program, but rather a list of certified vendors and applications maintained by Microsoft. I think this will help reduce finger pointing between application vendors that are members, but it is not a seismic shift in policy, as Microsoft still reserves the right to require a virtual instance to be reproduced outside of the 3rd party virtualization environment. Some industry heavyweights have already joined the program, including the expected such as Citrix and Cisco, and at least one surprise, Sun Microsystems.

Now for the curveball. It was very subtle, hidden in the kb article associated with the press release. I don't know if you noticed that when I mentioned reproducing a problem, I didn't say "on physical hardware". Microsoft has announced that you now have the choice of reproducing the problem on physical hardware *or* in Hyper-V. Now, that is a brilliant move by Microsoft, that may lead to beachheads of Hyper-V in otherwise pure VMware or XenServer shops.

Citrix Throws Down the Gauntlet

noreply@blogger.com (Jason Willey) — Tue, 19 Aug 2008 07:25:00 +0000

How many times have you heard "I don't know if my application works in Citrix?", or XenServer, VMware, Windows 2003...and so on.

Usually getting on the approved list involves heavy investment, and a good relationship between the companies involved in the agreements. Definitely not something your smaller or more specialized application vendors can afford, or in some case have any interest in. This usually leads to testing in our corporate environments. I have even had application vendors go as far as request documentation on how we installed their product, so they could market their product as having a Citrix install procedure.

Citrix has thrown down the gauntlet with a new approach, which will get all of us out of the business of providing functionality testing for vendor applications in our environments. Citrix is now offering a free partnership level, which provides access to Citrix testing labs for 3-7 days.

I applaud this move, and I would love to see other vendors adopt this approach.

Read more about this announcement at :
http://community.citrix.com/pages/viewpage.action?pageId=29851667

That's Why They Call It the Bleeding Edge...

noreply@blogger.com (Jason Willey) — Tue, 12 Aug 2008 18:35:00 +0000

As almost anyone running VMware must already know - VMware had a major bug in ESX 3.5 Update 2. http://kb2.vmware.com/kb/1006716.html I find it surprising that anyone is already running this update in production, as it was just released on 7/25.

My company was not affected, because we have a "forum vetting" policy for all new non security related updates. It's an unwritten rule that we always wait at least a month before progressing any major update, and we thoroughly check the company forums and non company sources for experiences that might be similar to what we will experience. This policy has paid dividends on the rare occasion when a major release has suffered issues. Using this method, we were able to avoid major issues with Windows 2003 SP1 and the some of the HP service packs and firmware in the past. By no means am I advocating using the internet as the sole source for testing, but it certainly is valuable as an additional check.

I have to give credit to VMware despite the circumstances today, as they are on top of making people aware of the issue. We received a call from our assigned SE this morning alerting us to the issue. Some issues are unavoidable, I'm just glad VMware handles them in the correct way.

Virtualization Certification Trifecta

noreply@blogger.com (Jason Willey) — Fri, 08 Aug 2008 21:21:00 +0000

For a long time, there was only one player in the Virtualization certification market - VMware with their VCP. I was waiting for them to broaden their certification beyond the initial certification, so I was please to see the announcement of the VCDX program.

VMware is no longer the only one in the virtualization certification race, as Citrix has released their Xen CCA, and Microsoft is preparing a virtualization certification as well.

So how do those certification programs compare in difficulty and return on investment?

Let's start with the first program out of the gate - the VCP.

Difficulty: I found this certification to not be incredibly difficult for people that have spent a good amount of time with VMware, and have had access to the official training materials. This certification has some "paper certification" protection in the sense that to be a VCP you must have completed some form of official training. I'm not sure I completely agree with this requirement, as I have seen some passionate fans of the product being held back by the lack of training budget.
Return on Investment: Very high in my opinion, but I think that large company HR departments aren't looking for the VCP yet , it seems to be more the consulting companies specializing in Virtualization that know to ask. A quick search of dice.com found 96 of the 1547 VMware jobs in the US looking for a VCP. Just for a point of reference - a search for MCSE found 1801 results. Here's an interesting article on the value of the VMware certification: http://servervirtualization.blogs.techtarget.com/2007/10/12/the-value-of-the-vcp-vmware-certified-professional/

Now the VMware VCDX - of course the tests are not fully released so this is all speculation based on the announcements that were made.

Difficulty: I think this will be the king of Virtualization certifications for the near future. I think that for the rest of 2008 and probably well into 2009 there will be less than 1000 of them. I think the most difficult part for the majority of people will be the Submission of a successful VMware Infrastructure design plan.
Return on Investment : I think this will be very high for high end consulting jobs, and elite enterprise customers - I don't see it being a big draw for the average job. Oddly enough - there are already 4 jobs from dice.com looking for the certification. The VMware Enterprise Administration exam is just being released this month.

Citrix CCA in XenServer - They appear to have recently released a new revision of this exam - A02 that is focused on the XenServer Enterprise Edition 4.1 product. http://www.citrixtraining.com/courses/course_list.cfm

Difficulty : While I have not taken this exam (yet), from the material available it appears to be very similar to the VCP in difficulty level, without the requirement to take the training class that tends to restrict some people.
Return on Investment : Hard to say. It appears to be a niche skill right now, and most of the people that are actually looking for Xen on dice.com are actually VMware consulting companies. I found no jobs looking for Xen CCAs It is possible that companies are training up internally rather than searching for external talent.

Microsoft's 70-652 Technical Specialist : Windows Server Virtualization, Configuring. The test is planned to be released this August. http://www.virtualization.info/2008/07/microsoft-releases-hyper-v-10-exam.html (link courtesy of Mark Bradley)

Difficulty: Based purely on the published exam objectives, I would place the difficulty somewhere between VCP/Xen CCA and VCDX. I think anyone with a background in Windows clustering will tend to have a advantage on this exam.
Return on Investment : Time will tell. There are already 16 jobs on dice.com looking for Hyper-V and the Microsoft marketing machine is only getting started. I think the strength of Microsoft name and marketing arm will drive the value of this certification very quickly.

I would definitely like to go for the trifecta of Virtualization Certification, with a VCP, XenServer CCA, and a MS TS in Hyper-V. Having a little product diversity never hurt the resume.

I would be interested to hear other people's opinion on the value of the various certifications, so feel free to leave a comment, I would be especially interested to hear from my new readers in the UK and Australia as to how you see the value of the various certifications in your local job markets.

Learning from Idiots...ITidiots that is.

noreply@blogger.com (Jason Willey) — Tue, 05 Aug 2008 01:22:00 +0000

A good friend of mine, Mark Bradley introduced me to a trio of video podcasters from the UK who are collectively referred to as the ITidiots.

Their HD feed is located at :
http://feeds.feedburner.com/ItIdiotsHD

As you can see to the lower right of my blog, I have added an RSS feed of their latest episodes in HD. I find them very entertaining and informative on a wide range of subjects, including certification. Anyone who is still chasing their 2003 MCSE should check through their back catalog of episodes - especially on DHCP, DNS and their 5 part review of the dreaded Microsoft 70-290 exam. They may not go into tremendous depth, but it definitely a enough to get your feet wet in a subject you may not have previously known. Mark has mentioned that the forums are very interactive, and they gather some of their subjects from user input.

In case you prefer the regular definition episodes - here is there feed for that:
http://feeds.feedburner.com/itidiots.xml

They also have a show specifically about the Mac - but that is against my beliefs, so you'll have to go to their website at http://www.itidiots.com/ for more information on that subject.

Think you're ready for a Microsoft Cluster Assessment? - Part 2

noreply@blogger.com (Jason Willey) — Mon, 04 Aug 2008 22:45:00 +0000

In part one, I discussed some of the technical things I learned from our latest Microsoft Cluster Assessment (http://download.microsoft.com/download/9/4/a/94adbabc-1e53-4a8e-abb5-34d3e31240e5/CSRAP.pdf). In the second part I am going to discuss some of the less tangible aspects of the assessment.
So Microsoft breaks it the operational aspects into three major categories

Supportability and Best Practices
Recovery Best Practices
Nodes, Cluster and Site Recovery Procedures
The bottom line of all these items: Documentation.

One of the things Microsoft stressed was a cluster build document for each type of cluster. In general we had always built clusters like IKEA furniture - i.e.

Assemble component A (the operating system)
Assemble component B (Windows Clustering ),
Assemble component C (SQL )
Combine them into one big happy set of clustered servers.

Microsoft recommends having end to end documentation for each type of cluster you build, rather than a modular documentation set.

There is also a heavy focus on Disaster Recovery, if you don't have any DR or BCM plans, prepare to see quite a bit of red on your final assessment.

Think you're ready for a Microsoft Cluster Assessment? - Part 1

noreply@blogger.com (Jason Willey) — Thu, 31 Jul 2008 00:57:00 +0000

If you've never had a Microsoft Cluster Assessment, I would encourage you to give it a try. They are standard practice in our organization, as in the past we have had more issues with clusters than any other servers that we deploy. The reports that we receive at the end are great data for improving the next set of clusters that are deployed. While I' not going to get into details about our particular assessment, I think that some of the things I learned in this assessment are safe to share. This is not meant to be a comprehensive checklist - Microsoft already did a great job of that, this is more the common things that sometimes get missed.

make use of the Microsoft Cluster Resource Center. There is a tremendous amount of well organized material on this site that will help avoid both operational issues and technical issues. http://www.microsoft.com/windowsserver2003/technologies/clustering/resources.mspx#ENH
The servers need to be identical prior to clustering the servers. I'm a big fan of unattended build processes, but not every IT organization uses unattended builds. Keep in mind the more manual interaction you have with a build, the less likely they are to identical. Servers shouldn't be like snowflakes.
In addition to being current on all current MS hotfixes, apply any applicable updates from this knowledge base article: http://support.microsoft.com/kb/935640 That article applies to SP2, fixes for SP1 and base install are linked at the bottom of that document.
When installing hotfixes pay close attention to failures prior to continuing with hotfix deployment. If you apply a hotfix that supersedes an existing hotfix you will be unable to return to install the original, and the servers will appear to have inconsistent patches, despite having the same end result.
Check your network configuration: http://support.microsoft.com/kb/258750
Make use of the ClusPrep tool. If you need to check an x64 server, you will need to host the file on a 32bit server. My understanding is it is back ported from the 2008 clusprep tool, and therefore there will not be any further updates to the tool. http://www.microsoft.com/downloads/details.aspx?FamilyID=bf9eb3a7-fb91-4691-9c16-553604265c31&DisplayLang=en

In part two, I'll discuss some of the common operational readiness steps that are crucial to successful clustering.