Microsoft may position Hyper-V Live Migration as good enough, but a side-by-side comparison clearly reveals that an imitation is never as good as the original.  VMware vMotion continues to improve and vSphere 5 enjoys many new enhancements, including support for multiple 10GbE interfaces to increase bandwidth for migration traffic — complementing the well-proven ability to migrate up to 8 VMs at a time.

Hyper-V Live Migration, introduced two years ago after much delay, will evidently remain stagnant for quite some time to come.  In fact, the shipping version of Hyper-V can still only accommodate a single migration at a time — whether a source or a destination.  This leads to the somewhat disingenuous claim that Hyper-V supports up to 8 concurrent Live Migrations per [16-node] cluster!

Principled Technologies conducted migration testing in two different scenarios, providing objectivity to correct the inaccurate claims of parity between the platforms. In the first scenario, one host in a cluster is running 10 VMs and is put into maintenance mode.  Comparing elapsed times to evacuate each hypervisor quantifies migration speed without conjecture.  In the second scenario, a single busy VM is moved from one host to another.  The elapsed time is considered, but more important is the performance impact to the application undergoing migration.

Take a look at the results and see for yourself:

Host Evacuation: Entering Maintenance Mode

Under real-world conditions, VMware vMotion is over five times faster than Live Migration.  When Patch Tuesday rolls around and it’s time to update Windows-based hypervisors, be sure to build plenty of margin into the maintenance window to allow for host evacuations.  What’s amazing about this victory is that although vSphere 5 now supports multiple 10GbE NICs for vMotion traffic, this scenario used just a single interface on each host.

Tier-One Application: Degradation Caused by Migration

A busy database VM is expected to deliver continuously; minimally impacting application performance during migration is a crucial requirement when demanding SLAs must be met.  VMware vMotion cranks out 63 percent more orders than Hyper-V during a 4-minute migration window. In fact, vSphere needed just half a minute to move a very busy 4-vCPU SQL Server VM with 16GB RAM.  The charts below make it clear: vSphere empowers IT architects to virtualize with confidence.

Reliability: The Prime Directive

Having the best possible performance is not the only thing that matters — live migration would be of limited use if it caused workloads become unstable or crash.  Therefore, imagine the surprise to learn that Hyper-V VMs fail in a reproducible, albeit unpredictable, way with the dreaded Blue Screen of Death (BSOD) after undergoing Live Migration.  I’m sure this will be eventually resolved through some future Patch Tuesday, but that’s not what consumers of your IT infrastructure want to hear.

Hyper-V: “Missing Something” or “Good Enough”?

Last, but not least, an enthusiastic Hyper-V advocate and long-time VCritical reader once provided this revealing insight:

Thanks to Principled Technologies, we now lay this Microsoft conjecture to rest.

VMware vSphere: The Best Performance and Reliability

The success of a cloud deployment depends entirely on the foundational infrastructure.  Not only does VMware vSphere provide the best performance density per host, it also offers the best operational management capabilities for your cloud.  Don’t fall for the “good enough” trap!

Be sure to download the full report from Principled Technologies.

Related posts:

1. How to see if Quick Migration is right for your workload
2. Let them eat cake!
3. Hyper-V Console Disconnects During Live Migration
4. VMware vSphere 5 outperforms Hyper-V by nearly 20%

There were many clever entries, but in the end “Hard Hat Area” won out.  Thanks to all who participated.

Related posts:

1. Cloud Power Caption Contest [Win VMware Workstation 8!]
2. vComeback Contest Winners! [Microsoft Cloud Power]
3. vComeback Contest: Microsoft Cloud Power Billboard
4. Idle RHEV Hypervisors save power?

As it turns out, this Microsoft Cloud wasn’t zoned for optimal utilization — it ended up in a construction site.  Sort of like unsuspecting consumers of the Office 365 service, which is so prominently featured on the sign.  Or, as they say at Azure: Temporary “DNS issue” folks. Nothing to see here.

Cloud Power: Under Construction. Hard Hat area. (Winner: Josh Atwell)

Submit Your Caption for a Chance to Win!

Think of your most clever caption to the photo above and post it below as a comment for a chance to win a copy of VMware Workstation 8 ($199 value).  Hurry, contest ends October 11.

Here are a few examples to get you started:

• This cloud looked a lot taller in the brochure.
• Are they building an entrance?  Or an exit?
• Traffic fines doubled… and so have my operational costs!

Google vs. Microsoft

Want to know the real irony here?  This obstruction is a pedestrian footbridge that will be frequented by Google employees on their commute to one of many buildings in the area.  You can’t make this stuff up.

View Larger Map

Rules: Winner selection is subjective.  If similar entries are received, the first submitter will have priority.  Entry must be received by October 11, 2011 11:59 p.m. PDT.  Prize not claimed within five days of notification will be forfeited.  Contestants must be legal residents of earth.  Not sponsored or endorsed by any third-party.

Related posts:

1. Announcing the Cloud Power Caption Contest Winner
2. vComeback Contest: Microsoft Cloud Power Billboard
3. vComeback Contest Winners! [Microsoft Cloud Power]
4. Unboxing the Cloud

• VMware vSphere 5 delivered 18.9% higher aggregate performance than Hyper-V
• Performance among individual VMs was much more consistent with VMware than with Microsoft
• Overall VMware ESXi performance actually improved with higher density, while  Hyper-V declined with just 25% oversubscription

A chart tells the story very nicely:

Let’s take a closer look at these three victories.

Higher Performance

Using virtual machines running the latest Microsoft Windows Server 2008 R2 SP1 and SQL Server 2008 R2, VMware ESXi 5 churned out 18.9% more orders per minute with the DVD Store 2 benchmark, proving once again that superior vSphere technology enables more work to be done on less physical equipment.  Higher density virtualization translates into  fewer servers, network ports, power, space, cooling, and software licenses — reducing costs for your cloud infrastructure.

Fairer Resource Management

With the shift to cloud-based, multi-tenant computing environments, it’s more important than ever to be able to guarantee resources to infrastructure consumers.  The advanced resource management capabilities of VMware ESXi continue to shine — delivering 39.2% better standard deviation among virtual machines.  The erratic performance of Hyper-V workloads translates into an inconsistent experience and missed SLAs in densely-virtualized environments.

Better Scalability

Thanks in part to the full range of advanced memory management technologies in vSphere, VMware ESXi performance actually increased by 11.2% when more VMs were added to a host.  Hyper-V, with apparently sub-optimal scheduling capabilities — and lack of memory page sharing, compression, and hypervisor swap — did not fare as well; performance dropped by 3.3% in a very reasonable oversubscription scenario.

VMware vSphere: The best cloud infrastructure platform

While Microsoft claims that Dynamic Memory, introduced recently in Service Pack 1, has closed the gap between Hyper-V and vSphere, these independent test results tell a very different story.  Although that new feature does allow Hyper-V to power on more VMs than before, infrastructure architects will need to consider the performance impact on workloads in higher-density private clouds built from the unproven and less mature Windows-based hypervisor.

Please download the full report (or summary) and see why VMware vSphere is the best cloud infrastructure platform.

Related posts:

1. The Truth About Hyper-V Memory Overcommit
2. VMware vSphere 4.1 – the best virtualization platform yet
3. VMware vMotion: Over 5 Times Faster Than Hyper-V Live Migration
4. SearchServerMisinformation.com

VMware vSphere 5 License Reporting

The new vSphere 5 licensing model is accompanied by a new licensing report that allows customers to query vRAM utilization.  Although this report is accessed through the standard vSphere Client, it is based on the next-generation Flex framework and served up via the vSphere Web Client — a server normally used by end-user web browsers.  The report looks like this:

VCVA License Reporting Bug

Conveniently, the VCVA comes with the Web Client already installed.  Unfortunately, a small workaround is needed before the license reporting feature can be used on the VCVA, which shows the following error by default:

While the actual bug is in the vSphere Client, the workaround is to re-register the Web Client from the command line on the VCVA.  Keep in mind the default username/password for the VCVA are root/vmware, shown below — please change yours.  To fix the reporting issue, log in via SSH and run the following commands, which will prompt interactively for confirmation:

# /usr/lib/vmware-vsphere-client/scripts/admin-cmd.sh unregister https://`hostname`:9443/vsphere-client localhost root vmware
# /usr/lib/vmware-vsphere-client/scripts/admin-cmd.sh register https://`hostname`:9443/vsphere-client localhost root vmware

Here is the process in action:

After re-registering the vSphere Web Client, the license report will work as expected.

Related posts:

1. VMware vCenter CapacityIQ 1.0.1 for vSphere
2. Wanna see a million ESX licenses?
3. vSphere issues? Ask VMware.
4. Installing vCenter on Linux Technical Preview

Bradley Whitt – Gleacher & Company, Inc.

I was curious if you’ve seen, notice any reaction from your customers and prospects to the VMware pricing changes on VSphere? And curious whether you’re seeing more Xen downloads, that type of thing?

David Henshall

We have. We’ve seen an uptick in downloads as well as direct inquiries coming in based upon what they did with their licensing. And we’re not headlong into the server virtualization space. And obviously, we’re doing well with that platform across desktop and networking and now in the cloud platform. But it’s not a focused go-to-market for us. So I’d say probably Microsoft with Hyper-V and System Center has probably seen a much bigger uptick as a result. But we have seen greater interest in XenServer.

Admittedly, Citrix is in a tough spot — they have to choose the path of least resistance, which in their case means de-emphasizing and cutting products and solutions that compete with their biggest partner.  Given this revealing perspective and the open source community shifting to KVM, it’s hard to see a future for XenServer.

Transcript from Seeking Alpha.

Related posts:

1. vSphere Smokes Competitors in InfoWorld Shoot-Out
2. VMware vSphere 4.1 – the best virtualization platform yet
3. Hyper-V deployments suddenly drop to zero
4. Don’t Sysprep Me, Bro!

Next question: Does Microsoft plan to do anything similar to the vTax?

NO, we have no intention of imposing:

• A VM Memory vTax
• A VM Core vTax
• A VM Replication vTax

This is great for vSphere customers, who often license Windows Server Datacenter Edition — per CPU socket — to run an unlimited number of VMs on an ESXi host.  As CPU cores and DIMM capacities continue to increase and Windows pricing remains constant, customers everywhere stand to save considerably on Microsoft licensing by consolidating Windows workloads onto fewer and fewer sockets.

It’s unusual to see a long-term price commitment like this from a strategic* enterprise software provider — especially one that, not long ago, introduced a new premium license for SQL Server weighing in at $55,000 per socket and clearly intended to maximize revenues from virtualized databases.

*Inside joke.  Did you get it?

Related posts:

1. Use Coreinfo to view VM core and socket count
2. Nice writeup on Windows Server licensing
3. Wanna see a million ESX licenses?
4. When does your “free” Hyper-V Server cost $1304?

Now with VMware ESXi 5, it is possible to perform an automated installation to USB storage.  Simply specify usb-storage as part of the install command in an automated install script.  For example:

install --firstdisk=usb-storage --overwritevmfs

For more flexibility in a single script, multiple disk types can be specified; they will be tried in order.  Below is an example install command from a script for HP servers.  If USB flash is not installed, the on-board SAS disks will be used.  If neither are present — such as when installing virtual ESXi systems — generic local storage is the last resort.

install --firstdisk=usb-storage,hpsa,local --overwritevmfs

Perhaps not one of the major new feature of VMware vSphere 5, but handy.

Boot Hyper-V From Flash?

You might not know this, but Microsoft Hyper-V has had the ability to boot from flash as well — announced two years ago, but yet to be acknowledged in an actual production deployment.  Of course, this bleeding-edge configuration option is only supported if installed by OEMs, not end-users.  Anyone considering Hyper-V for virtualization is highly encouraged to give it a try; it’s a great opportunity to learn a lot about Windows registry manipulation and command line tools such as imagex and diskpart.

But seriously, VMware vSphere 5 will be here soon, and you’re going to love it.

Related posts:

1. Create ESXi 4 USB flash drives with Workstation
2. VMware ESXi 4: SD Flash in BL460c G6
3. The VMware ESXi 4 64MB Hypervisor Challenge
4. A very flashy hypervisor: Hyper-V Server R2

William Lam is the de-facto authority on fully-automated ESXi deployments, and others have provided recipes for configuring VMware ESXi 4.1 installation media for interactive PXE installation.

The good news is that with VMware ESXi 5 it is no longer necessary to create the image.tgz file to accommodate a manual PXE installation, and another handy enhancement to boot.cfg makes it slightly cleaner to organize multiple installation sources on a PXE/TFTP server.

Take a look at the following procedures to learn more:

Prepare the VMware ESXi 5 Installation Media

On your TFTP server, simply extract the contents of the installation ISO into a new directory.

# mkdir esxi50
# mount -o loop VMware-VMvisor-Installer-5.0.0-nnnnnn.x86_64.iso tmp/
# rsync -a tmp/ esxi50/
# chmod +w esxi50/*

That’s it.  No need to create any additional files with ESXi 5.

Modify the ESXi boot.cfg Configuration File

In order to reflect the fact that the installation source is not in the root of the TFTP server, two tweaks are needed.

1. Remove all of the slashes (/) from the boot.cfg file so relative paths will be used (vi :%s#/##g)
2. Add a “prefix” directive to the boot.cfg file to specify the proper subdirectory, from the perspective of the TFTP root

End result will look something like this:

This is much cleaner that repeating the full path dozens of times throughout the file.

Add a PXE Menu Entry

Typically the PXE menu is pxelinux.cfg/default; add the following section and you’ll be set:

LABEL esxi50
 KERNEL /esxi50/mboot.c32
 APPEND -c /esxi50/boot.cfg
 MENU LABEL ESXi-5.0.0

Ready for interactive installation

Now you are ready to boot your physical or virtual machines for an interactive installation of VMware ESXi 5.  It’s very easy to run ESXi 5 inside a VM, now with 64-bit nested guest capabilities!

Related posts:

1. Peaceful Coexistence: WDS and Linux PXE Servers
2. VMware ESXi and Hyper-V installation comparison
3. Create ESXi 4 USB flash drives with Workstation
4. VMware ESXi 4: SD Flash in BL460c G6

VMware vSphere 5 makes it easier than ever to virtualize hypervisor hosts. With new capabilities to run nested 64-bit guests and take snapshots of virtual ESXi VMs, the sky is the limit for your cloud infrastructure development lab. Heck, you can even run Hyper-V on top of vSphere 5 — not that you’d want to.

Physical Host Setup

The physical host running VMware ESXi 5 requires just a few configuration changes; here is a guide:

• Install VMware ESXi 5 on a physical host and configure networking, storage, and other aspects as needed
• Configure a vSwitch and/or Port Group to have Promiscuous Mode enabled
• Create a second Port Group named “Trunk” with VLAN ID All (4095) if you want to use VLANs on virtual hypervisors
• Log in to Tech Support Mode (iLO or ssh) and make the following tweak to enable nested 64-bit guests

  echo 'vhv.allow = "TRUE"' >> /etc/vmware/config

Virtual VMware ESXi Machine (vESXi) Creation

For various reasons, it’s not feasible to clone virtual ESXi VMs. As an alternative, create a fully-configured shell VM to use as a template — it can be cloned before ESXi is installed.

Create a new VM with the following guidance:

• Guest OS: Linux / Red Hat Enterprise Linux 5 (64-bit)
• 2 virtual sockets, 2+ GB RAM
• 4 NICs — connect NIC 1 to the management network and the rest to the “Trunk” network:
• Thin provisioned virtual disks work fine
• Finish creating the VM, then edit the following settings
  • Options/General Options: change Guest Operating System to Other – VMware ESXi 5.x
  • CPU/MMU Virtualization: Use Intel VT … EPT… ( bottom radio button)
• Don’t power this VM on — keep it to act as a template
• Clone and install VMware ESXi via ISO image or PXE boot
• Add to vCenter and configure virtual ESXi hosts for action

Nested 64-bit Guests

With the release of VMware vSphere 5, nested guests can be 64-bit operating systems. Just be sure to make the change to /etc/vmware/config on the physical host as indicated above.

Nested guests can be migrated with vMotion between virtual or physical VMware ESXi hosts; this requires a vMotion network and shared storage.

Nested Hyper-V Virtual Machines

It is possible to run other hypervisors as vSphere virtual machines, and even power on nested VMs. Here you can see Hyper-V running a CentOS virtual machine — all on VMware ESXi. Talk about disrupting the space-time continuum!

A couple of extra tweaks are needed to enable this, and performance is not great. Nevertheless, an amazing feat of engineering from VMware!

Do the following to enable Hyper-V on VMware ESXi:

• Add hypervisor.cpuid.v0 = FALSE to the VM configuration

• Add —-:—-:—-:—-:—-:—-:–h-:—- to the CPU mask for Level 1 ECX (Intel)

For another take, check out William Lam’s post on this topic.

Parting Thoughts

Given the right hardware, it is possible to create a fully-functional VMware test lab that is completely virtual. Go ahead and experiment with the Distributed Virtual Switch, vShield, vCloud Director, and everything else without deploying a ton of servers and storage.

How are you taking advantage of a virtual vSphere environment?

Related posts:

1. VMware ESX 4 can even virtualize itself
2. New VMware ESXi Management Kit
3. Taking snapshots of VMware ESX 4 running in a VM
4. PowerShell Prevents Datastore Emergencies