My name is Nils Nieuwejaar. I’ve been a member of the engineering team at Virtual Computer since a little before we opened our doors two years ago. In this forum, I’ll be writing about a variety of technical issues, covering virtualization in general and NxTop in particular.

Our primary focus at Virtual Computer is on solving a host of management problems for our customers. While we make extensive use of virtualization to solve these problems, we don’t usually think of ourselves as a virtualization company in the traditional sense. More to the point, we don’t necessarily expect our customers to have a deep familiarity with virtualization technology.

My goal here is not to make a virtualization expert out of anybody, but simply to give our users enough information to understand how the different components in the full NxTop system work. This knowledge will help them get the best performance out of the system, and help them diagnose any problems that may arise in their environments. I’ll be covering topics that future customers ask the sales team about, that current customers ask the support team about, and that the support team asks the engineering team about.

Fully Virtualized I/O

When discussing NxTop with customers, they frequently have questions about how I/O works in virtualized systems like NxTop. The answer to this simple question turns out to be somewhat complicated.

I’ll start by talking fully virtualized I/O, which may be the simplest case to understand and the most complex to implement. When a typical Windows application wants to read data from disk, it makes a system call into the operating system, and the operating system in turn makes a call into the device driver controlling that disk.

When running on bare metal (i.e., on a real machine instead of a virtual machine), the disk driver builds up a command structure describing the I/O operation it wants to perform (read/write, sector ID, size, etc.), and then writes a “start I/O” command to a special address in the computer’s memory. The disk hardware is notified when that special address is written to. The disk controller reads the command structure from the computer’s memory, triggers the hardware to carry out the operation it describes, and notifies the OS when the operation is complete. The OS then returns from the system call, and the application proceeds.

The illustration above shows the different components you will find on a typical bare metal system. There are applications running on top of an operating system, and the operating system interacts directly with physical hardware.

The picture below shows a typical simple virtual platform:

Again we have applications running on an operating system, but in this case the operating system is running on top of a hypervisor instead of bare metal. A hypervisor is similar to an operating system, but instead of hosting applications like an operating system, it actually hosts operating systems. The hypervisor manages core resources like CPU and memory, and passes guest I/O requests to a virtual hardware platform. The virtual hardware that receives the I/O requests is nothing more than a piece of software owned and operated by a host operating system. The fundamental trick of virtualization is to make the guest operating system believe that it is interacting with real hardware instead of a piece of software.

(side note: whether the hypervisor and virtual hardware should be shown inside, beside, below, or on top of the host operating system varies from system to system, and gets into the distinction between ‘Type 1′ and ‘Type 2′ hypervisors, More on that in a different post.)

In a fully virtualized system, the disk driver believes that it is controlling a real hard drive. So to read a block of data from disk, it will build up exactly the same control structure described above, and it will write the ’start I/O’ command to exactly the same memory address. In this system however, it is the hypervisor that notices that that address has been written instead of a real disk controller. The hypervisor notifies the virtual hardware, which then reads the control structure out of memory, decodes it, and carries out the described operation.

In most cases, the guest’s disk is actually just a file in the host operating system’s file system. When the guest OS wants to read block 100 from its disk, the virtualization layer instead reads block 100 from the file. (side note: This is actually a significant oversimplification. In practice, a guest’s disks are stored in one or more files, each of which has a somewhat complex internal structure. More on this in another post.)

After the software layer reads the data from disk, it copies the data into the guest operating system’s memory, and sends it exactly the same ‘work completed’ signal that a real disk controller would. The guest operating system continues on its merry way, unaware that the disk request was satisfied by virtual hardware rather than physical.

This sounds relatively simple, and indeed it is when reading a single disk block. However, the IDE interface includes dozens of commands, errors, and status variables. It includes programmed I/O and DMA, supports hard drives and CDROMs, and so on. The guest operating system takes a high-level file operation from an application, translates it into very detailed, low-level IDE commands, and then the virtual hardware has to decipher those low-level IDE commands and transform them into file operations on its virtual hard disk. Doing all of this work for every disk operation can be time consuming, which results in poor disk performance for applications running in the guest operating system.

Paravirtualized I/O

To help avoid the performance problems that come with operating on emulated hardware, we use a different I/O model for performance-sensitive devices such as disk and network. This model is referred to as paravirtualization as opposed to the full virtualization we’ve already discussed.

In paravirtualized I/O, the device driver running in the guest operating system understands that it is running on virtualized hardware. Instead of attempting to talk to a physical disk, a paravirtualized (PV) disk driver in the guest will communcate directly with a partner device driver running in the host OS. As illustrated below, the PV driver bypasses the virtual hardware model, avoiding all of the expensive encoding and decoding steps.

Just a couple of notes on terminology: since there is no disk device per se here – just two cooperating drivers, we typically don’t talk about ‘PV devices’. Instead we mostly talk about PV drivers. If you have ever used VMware’s Tools, VirtualBox’s Guest Additions, or Microsoft’s Hyper-V Integration Services in a virtual machine, you have been using some type of PV drivers.

Each driver actually has two parts: a front end and a back end. The frontend driver runs within the guest. It plugs into the guest operating system’s driver stack in essentially the same way a physical driver would, so the rest of the operating system interacts with it just like a physical device. The backend driver runs in the host operating system. It receives I/O requests from the frontend driver, and executes them. These I/O requests arrive from the guest fully formed in a commonly agreed upon format. There is no expensive decoding/translating process as with emulated devices.

In addition to communicating at a higher level, PV drivers generally offer additional opportunities for improved performance. We can change buffer sizes, queue depths, algorithms, or features at any time. Since the I/O model is simpler, it is easier to identify and fix any performance or correctness problems. For a single point of comparison: in NxTop Engine, basic IDE emulation takes 4 times as much code as the backend disk driver, and runs at a fraction of the speed.

Since the front end and back drivers are generally written in conjunction with one another, there is no ambiguity about the expected behavior. When writing software that emulates a particular hardware device, the documentation of the device’s behavior may be unavailable or incomplete. This may cause the implementor of the virtual hardware to be reluctant to support the highest performing and most complicated mechanisms, instead forcing the device driver to fallback to older, simpler, and better documented mechanisms.

The most significant drawback of the PV I/O model is that you need new drivers for each operating system you want to use as a guest. The devices provided by a fully virtualized system tend to be common enough that drivers will be available for nearly every OS. For PV I/O, you will always need to write new drivers. The NxTop platform includes PV drivers for disk, network, mouse, and USB for Windows XP, Vista, and Windows 7, which covers nearly all of our customers’ needs.

The screenshot below shows a picture of the Windows 7 Device Manager, with the NxTop PV drivers installed on the NxTop Engine.

You may have noticed that prior to this I never mentioned whether I was describing the NxTop Engine (i.e., the client) or NxTop Center (i.e., the server). In fact, this discussion applies equally well to both components of the NxTop system. On the server, we make use of Microsoft’s Hyper-V Integration Services when the IT admin is managing and publishing a NxTop image. On the client, we make use of our own PV drivers when the end user is running the published image.

Finally, I should mention that there is another interesting type of I/O in a virtual environment: passthrough. We don’t currently make use of this in NxTop engine, but I’ll talk more about it in future posts.

NxTop is compatible with, but does not require, Intel vPro and VT-d.

Several of the other client hypervisor products in works are being centered on Intel vPro.  This makes sense on one level, since vPro is at its core a management and control point that is independent of the operating system. A client hypervisor is a very logical extension of that. The rub is that there are many corporate PCs with years of life remaining in them that are not vPro enabled.  There are also many enterprises ordering large volumes of PCs who do not want to pay a premium for vPro-enabled PCs for all classes of users.  When you are dealing with hundreds of thousands of PCs, any incremental cost per unit adds up very quickly, so this is a real consideration in today’s budget conscious times.

One of the major stumbling blocks of server-based desktop virtualization is that while it offers significant management and security benefits, it can generally only be deployed for a subset of an organization’s users.  Limiting client hypervisor compatibility to the highest end of the corporate PC market and not providing backwards compatibility with existing business class PCs would impose the same limitations on the adoption of client-side virtualization.  For obvious reasons, we did not want to see that happen.

The primary aspect of vPro that is relevant to client virtualization is Intel Virtualization Technology for Directed I/O (VT-d).  VT-d extends the base Intel virtualization extensions for the x86 architecture that exist in most business class machines today (VT-x) to include an input/output memory management unit (IOMMU).  The IOMMU makes it possible to securely assign physical hardware components directly to specific virtual machines.  This has many practical uses (particularly in overcoming performance challenges in areas such as graphics), but it has some major downside in that it requires hardware-specific drivers in each virtual machine and makes supporting the full array of graphics cards quite challenging.  Stay tuned for more on this in a future post.

With NxTop, we have achieved a very high level of performance without reliance on IOMMU.  This enables NxTop Engine to run on any platform with VT-x.  So when we go into an enterprise where they are currently buying the latest Dell E-series PCs but they have a bunch of older D630s (usually with a mix of Intel and NVIDIA graphics chips), it’s never a problem to get started.  We are not talking to the client about a utopian management model in the future when all of their current PCs are in the graveyard.  We are saying, “Hey, let’s gets started—TODAY.”

With this as a backdrop, I do not in any way want to leave the impression that we don’t see value in both vPro and VT-d/IOMMU functionality.  Continuing innovation from the processor manufacturers will only expand the set of management and performance features we can offer as part of NxTop, and we are embracing this innovation with open arms.  However, we don’t think client virtualization can take off without support for a wide range of PC platforms both new and existing.

As I look back in the rearview mirror at two-plus years of talking with IT professionals and industry analysts about client hypervisor technology, hardware compatibility is by far the topic that has generated the most questions. It isn’t all that surprising, since it is in fact one of the biggest challenges that comes with bringing bare-metal hypervisor technology from the datacenter to the PC. There is critical functionality on end-user PCs that server hypervisors never needed to deal with, such as high performance graphics, wireless, power management, USB peripherals, laptop lid closure events, etc.

If I look outward at the rest of the industry, I see two reactions to this challenge. The reaction among other desktop virtualization startups was to punt on the hypervisor. Implementing a bare-metal client hypervisor is hard, especially for a startup with a finite set of resources. It is much easier to focus efforts elsewhere and in full hand-waving mode say, “Type-1 client hypervisors will eventually be a ubiquitous commodity.” As it happens, this is probably true. However, it is not the case today, it definitely won’t be the case in 2010, and who can say with 100 percent certainty when (or if) it will ever be the case? I’d probably bet a hundred bucks on it but certainly not millions in venture capital. For startups, there is certainly risk in spreading yourself too thin, but there is greater risk in not being in control of your own destiny. There is further risk in not being in control of the end-user experience, since that will ultimately make or break the success of client-side virtualization more than any other factor.

The incumbent desktop virtualization players followed a different approach. They are tackling the client hypervisor but zeroing in on a very small hardware compatibility list. They already have a captive audience (and revenue stream) with their server virtualization and server-hosted desktop virtualization products, so the bar for them is simply to show enough forward progress with client hypervisors to freeze their customers. This worked for a while in 2009, but we saw the freeze start to thaw when the major virtualization players laid a collective client hypervisor goose egg at VMworld 2009. VMware shops that had told us “looks great, but we’re gonna wait to see CVP” came calling again in September. We are seeing the thaw turn into a full melt now that the “late 2009″ client hypervisor target the big guys communicated has come and gone. This won’t last forever, but we are certainly not going to let the window of opportunity we have in early 2010 pass without capitalizing.

I won’t claim that we have achieved universal hardware compatibility with every PC on the planet, but I do believe that through a combination of a superior client hypervisor architecture, hard work, and close collaboration with key PC manufacturers, we are the undisputed leader in client hypervisor hardware compatibility. In my next post, I will provide a detailed explanation of why this is the case.

In the time since the 1.2 release, we have had our heads down and focused on taking the NxTop Engine experience to greater heights. It hasn’t been glamorous stuff. For the most part, it has been benchmark, optimize, and repeat in a number of performance and user experience areas. There are literally folks on our engineering team that have been so deep into Xen and the Windows stack that I am not sure they will ever be the same.

The result of these efforts is a great set of year end updates to NxTop that will allow our customers to accelerate their rollout plans as we head into the new year. The coming year is predicted to be a big one for client hypervisor technology, and the fact that NxTop has the maturity and performance to run on real end-user desktops while competing offerings amble towards beta is very gratifying to us.

But don’t worry, we aren’t resting on our laurels. Work is already well underway on our next major release of NxTop, and based on what I have seen so far I think it will really turn some heads.

(And, yes, that photo in the bottom center is us being asked to move the smart car!)

Hope everyone’s enjoying the show — if you haven’t stopped by for a NxTop demo yet, you can find us at booth #1940.

The “Get Smart” program will feature a number of activities at VMworld where attendees can learn about NxTop’s unique PC life cycle configuration management capabilities, as well as interact with some of our key partners who help bring it all together.  The more you interact with us and our partners, the more chances you will have to win the car.  Keep reading the blog and come by and see us at Booth #1940 next week to get all of the details, including some “extra credit” opportunities for all of you star pupils out there.

Travel budget blues keeping you away from VMworld this year?  There are still plenty of opportunities to get involved.  In conjunction with the “Get Smart” program, we have launched a new online community site that includes a very nifty total cost of ownership (TCO) calculator.  The tool is highly configurable, so if you don’t like our cost assumptions, simply plug in your own.  Think our overall methodology is flawed?  Stop by the forums and say what’s on your mind.  While you are at it, sign up for one of our upcoming webinars on “The New Economics of PC Management,” which will provide another chance to see NxTop in action, along with an in-depth review of our TCO methodology.  Online forum contributions and webinar attendance will earn VMworld attendees additional chances to win the car and provide those playing along at home with a chance to win.

Stay tuned more more contest details as VMworld gets under way.  See you in San Francisco!

Fortunately, there are some great efforts afoot to raise awareness of all of the amazing innovation going on in the Boston area.  Scott Kirsner’s Innovation Economy blog (recently relocated to Boston.com) has become required reading for Boston area startups, and the folks over at Xconomy and Mass High Tech also do a fantastic job highlighting the innovative companies on the rise in Massachusetts.  Additionally, while we often catch flack for letting Facebook fly the coop from Harvard to Palo Alto, we have actually figured out how to use social media here on the east coast. 

Case in point is Mass Innovation Nights (MIN), an organization that has emerged as the confluence of Massachusetts high tech innovators, the social networking world, the mainstream media, and the marketplace.  The organization holds monthly events at the Charles River Museum of Industry and Innovation in Waltham (You can’t find a more appropriate location than that!) where the hottest young companies is Massachusetts show off their latest innovations.

This month, MIN will be conducting its first “virtual” innovation night on Wednesday, August 12.  The event will kick-off at 6:30 p.m. on Twitter using the hashtag #MIN5.  (If you are new to Twitter, the MIN site has some good tips and tools for following hashtags.)  From there, the event will evolve into a phone-based “speed dating” style format with the help of some neat technology from past MIN participant BlitzTime.

One of the three rules for the virtual event is to play nice, but we are far too competitive for that at Virtual Computer.  So, we are putting out the challenge to the Boston startup community.  Do you think you have a more innovative product than us?  If so, the stage is set.  Let’s see what you’ve got!  We’ll be in on the virtual event, as well as jockeying for a spot in a future live Mass Innovation Night.  So, you better bring your biggest innovation, because we’ll be bringing NxTop.

Walking around BriForum, one really gets the sense that there is a perfect storm of industry events brewing that is really going to cause desktop virtualization to take off in the next 12 months.  For example, all signs are that Microsoft “got it right” with Windows 7, and that its release will be a catalyst for many organizations to look for new and innovative ways to deploy and manage their desktops.  We think we have one for them.  If you happen to be a BriForum and would like to connect to see a live demo of NxTop, feel free to grab me by the shirt, ping me on Twitter (@dlane), or use the form we have available on our web site.

Fortunately for me, my time spent contemplating the future of desktop virtualization with really sharp people will not end with my departure from BriForum on Thursday.  I am excited to be teaming up with Rachel Chalmers of The 451 Group next week on the webinar, “Five Ways Virtualization Is Changing Your PC.”  It will be held next Wednesday (July 29) at  11:30 a.m. ET.  We will explore a number of the ways in which client-side desktop virtualization will transform the PC for both IT admins and end-users, including:

• The ability to run multiple operating systems on a single device and easily move from a personal to a corporate environment
• Easy migration from Windows XP to Windows Vista or soon Windows 7
• Enhanced data protection, backup and security
• Improved PC management for mobile and remote workers
• Remote desktop access to your documents, settings and self-installed applications

Visit our registration page to learn more or to register.

At the Massachusetts Innovation and Technology Exchange (MITX) 6^th Annual Technology Awards this past Tuesday, Virtual Computer was awarded the PriceWaterhouseCoopers Promise Award in recognition of our efforts with NxTop.  If you are in the New England area, I encourage you to check out MITX.  It is a great confluence of people and organizations in the technology space focused on understanding “what’s next.”  We’re grateful to both MITX and PWC for the recognition.

On the following night, the Massachusetts Network Communications Council (MassNetComms, for short) held its 2009 Innovators Summit, bringing together some of the hottest companies in New England for a series of panels that quickly turned competitive as the format shifted to tightly time boxed product pitches.  Fortunately, the combination of a great product and the persuasive powers of our VP of Engineering, Peter Marconi, garnered Virtual Computer “Best Product Concept” honors. 

At the end of the day, our success is measured by what we deliver to customers and not by the size of our trophy case.  At the same time, it has been a grueling couple of months around the Virtual Computer offices putting the finishing touches on our latest product release, so it was great to see our team get some much deserved recognition.