Virtualization Eh

There are two forms of hardware assist that vSphere can leverage if your server platform supports them.  The vSphere monitor will determine if they are available and use them appropriately.  They can make a significant performance difference so it’s important to understand them.

First, is CPU virtualization assist known as AMD-V and Intel VT-x that replace Binary Translation (aka BT) which is CPU virtualization done completely in software.  As you can imagine using these hardware instructions greatly increases performance.  These have been around for quite sometime and you’d really have to dig for old server class hardware that doesn’t have them.

Second, is memory assist known as AMD RVI (Rapid Virtualization Indexing) and Intel EPT (Extended Page Tables) which replaces shadow page tables managed completely in software (sometimes referred to as SWmmu).  These additional functions allow the memory scheduler to use hardware to manage the mapping between physical host memory and virtual guest memory.  Again these functions increase performance and reduce virtualization overhead.  Its benefit can be felt the most with memory intensive workloads like Tier 1 applications such as databases, messaging, XenApp, etc.

To reduce virtualization overhead and maximize performance you want to ensure your server has these functions and that they are ‘enabled’ in the BIOS (I see many cases where people have forgot to enable them or have toggled them off by accident).

So how can I check if I have these assists available and that my virtual machines are leveraging them?

First check what has been set within the virtual machine configuration.  Here they can be switched manually.  Automatic is recommended and will allow the monitor to make its best attempt.

Second, search for “MONITOR MODE” in the vmware.log file of the running virtual machine.  These are the lines you’re interested in:

vmx| MONITOR MODE: allowed modes                   : BT32 HV HWMMU
vmx| MONITOR MODE: user requested modes        : BT32 HV HWMMU
vmx| MONITOR MODE: guestOS preferred modes   : HWMMU HV BT32
vmx| MONITOR MODE: filtered list                         : HWMMU HV BT32
vmx| HV Settings: virtual exec = ‘hardware’; virtual mmu = ‘hardware’

Decoded:

• BT32 or BT – Binary Translation which is full software virtualization of CPU and Memory
• HV – Hardware Virtualization (AMD-V, Intel VT-x) which is CPU assist
• HWMMU – Hardware Memory Management Unit (AMD RVI, Intel EPT) which is Memory assist in addition to CPU assist (preferred option)
• The order of modes is also important

In the example above, the server supports (ie: allowed modes) all of the modes above validating that the server does have both assists.  In the settings of the VM you have the option of selecting the monitor mode manually (ie: user requested modes).  Each operating system also has its own preference (ie: guestOS preferred modes) that is tied back to the operating system selection you make when creating the VM.

After all these options are taken into consideration (ie: filtered list) the most performant choice is made and documented on the next line.  In this case, virtual exec (cpu assist) = hardware and memory mmu (memory assist) = hardware.  It is still relatively common today not to have Intel EPT and therefore memory mmu = software.

More Information:

• Software and Hardware Techniques for x86 Virtualization
• Virtual Machine Monitor Execution Modes in VMware vSphere 4.0

Also remember that a ‘Restart’ of the guest is not enough to change the monitor mode – you must ‘Power Off’ the virtual machine when making a monitor mode change.