Originally hitting the blogosphere via Scott Drummonds here, and presented as a future technology by Kit Colbert and Fei Guo at VMworld 2009, this cool new performance feature is for those over-committing memory and walking the thin line between paging – evil – and virtual happiness.
We should all know that when we over-commit memory on an ESX host, that is to say allocate more virtual memory than we have physical memory to back it, the risk becomes the memory scheduler will eventually need to page some memory out to disk – more commonly known as swapping. Memory access times for those swapped pages go from microseconds to milliseconds when retrieved from disk – this is bad.
But doesn’t the balloon driver save us?
Yes, but only for awhile. The balloon driver is another mechanism to manage memory pressure before pages must be swapped to disk. It is better for the guest OS to decide what should be swapped out because it has visibility to which memory pages are more frequently used. It can then ‘intelligently’ swap something else that will have less of an impact on the guest. This process occurs because the balloon driver increases memory pressure and the memory freed by this process can then be used elsewhere. But eventually there is nothing else to intelligently swap and the ESX host will stop using the balloon drivers and start to randomly swap memory pages to disk. This ‘unintelligent’ process will cause performance issues.
So how does Memory Compression help?
This mechanism is one more tool that fits between Ballooning and Swapping. Before memory pages are swapped to disk as a last resort, they are first compressed (think zipped) and stored in a special new memory cache, in an attempt to keep them off disk a little longer at the expense of some processor cycles. To decompress a page from cache takes approximately 20 microseconds as compared milliseconds if it has to go to disk for it. So in times of high memory pressure, this mechanism transparently begins compressing pages so as to maximize performance as long as possible. Eventually though if the memory commitment is too large, or not transient, paging to disk will still occur.
Memory Compression is a great new feature but doesn’t remove an administrators responsibility to properly size a host and its guests. Think of it as a safety measure, not a license to blindly over-commit.
