ESX Server beats Hyper-V in vConsolidate Performance Benchmark Numbers by up to 125%

 

The Internet is great. There is a wealth of information up there, and with a little bit of work, you can find the best nuggets of information. You can even find performance benchmarking information comparing VMware ESX Server and Hyper-V. Stick with me here as I try to explain:

Principled Technologies is a company whose mission is: To provide the best possible technology assessment services to our clients. Whether you’re a vendor of technology products—hardware, software, services, or Web sites—or a user of those products, we can help assess the product’s performance, quality, market readiness, and other key characteristics.

They have done a LOT of work benchmarking different products, software, and hardware, and their reports are thorough. Recently, they have published two papers that have some interesting correlations. In August, 2008, Principled Technologies (PT) released a paper commissioned by IBM that was designed to compare IBM and HP servers. To perform this comparison, they used Intel's vConsolidate Benchmarking tool with VMware ESX Server 3.5 U1 as the virtualization backend virtualization platform. In September, 2008, PT released a paper commissioned by Intel to show the differences between their X7460 and X7350 processors. They also used vConsolidate to benchmark these as well. One interesting note, they used Microsoft Windows Server 2008 with Hyper-V as the backend virtualization platform.

Intel's vConsolidate

Intel's vConsolidate is a 3rd party developed benchmarking tool, similar to VMmark, that can be used compare two separate platforms. They do this by creating a group of 5 VMs that consist of a consolidate stack unit or CSU. With the speeds and capacity of modern servers, one CSU will not max out the system resources. Therefore, you stack multiple CSUs on the system until the CPU processing approaches 100% utilization. Once there, you can aggregate and normalize the data to come out with a score. These VMs consist of the following:

	Application	OS
WebBench (Web)	IIS	Windows 32-Bit
LoadSim (mail)	Exchange	Windows 32-Bit
SysBench (database)	SQL Server	Windows 64-Bit
SPECjbb2005 (Java)	BEA JVM	Windows 64-Bit
Idle	NA	Windows 32-Bit

Results

What was interesting about these two papers is that both of them used a VERY similar machine in their test which was a quad socket, quad core Intel Xeon processor X7350 based server. Both servers had 64GB RAM associated with them, and used vConsolidate Benchmarking to gather the results. The only major difference between these systems is that one used Hyper-V and the other used VMware ESX Server. PT also published the raw data numbers for all of these tests, and some interesting results came out. Here is a table with the average of the data from the outputs of the optimum CSU test results.

5 CSUs	VMware ESX	Microsoft Hyper-V	%gain over Hyper-V
Database	136.028	60.526	125%
Java	17296.6875	15268.49	13%
Mail	20.104	15.368	31%
Web	469.344	384.764	22%

VMware ESX Server beats Microsoft Hyper-V by up to 125%

First of all, neither of these tests were commissioned by VMware or Microsoft, which means we couldn't influence them in the least. Also, this comparison stressed the entire system by testing all the resources on the system by using a mixed workload of real world enterprise applications instead of simply one type of application. If you look at the results, you can see that VMware outperforms Hyper-V with an average win of 48%. It also shows that VMware ESX Server is the superior choice for high-performing environments.

So, what does this really mean?

• This comparison proves that VMware ESX server is NOT a commodity!
  • Even though it is free, it provides virtualization capabilities far superior than other virtualization platforms.
• Don't be afraid to run those Tier 1 applications within a VMware VM.
  • They will run very well, and you will see tremendous ROI by running on VMware Infrastructure 3.
• Check out other benchmarking and performance papers at VMware and our Performance Team's Blog, VROOM!, the VMware performance blog, for up to date information on what they are doing.

Full Disclosure

Now, these two papers show VERY similar systems, but they weren't identical. One difference is that the VMware ESX Server was an IBM x3850 M2 server which had 32 x 2GB DIMMS for its 64GB RAM whereas the Hyper-V Server was an Intel Whitebox server with 16 x 4GB DIMMs for its 64GB RAM. The IBM/VMware box used vConsolidate v1, whereas the Intel/Hyper-V system used vConsolidate v2. This use of different versions of the vConsolidate benchmark could introduce the possibility that the CSUs drove different loads on the system. Observation of the differences of vConsolidate aggregate scores between v1 and v2 seem to confirm this. However, our VMware performance team contacted the Intel vConsolidate team and learned that the raw scores reported are comparable between vConsolidate versions one and two. In other words, the RAW data shown in the table above for the Database, Java, Mail, and Web workloads are comparable.

Lastly, your mileage may vary, but I am sure you would see the same results if you set this up.

ESXi and the undocumented Tech Support Mode

KB Article 1003677 talks about this thing called Tech Support Mode. This is a command-line interface that can be used with VMware ESXi Hosts to assist with the troubleshooting process.

For information on the Architecture of VMware ESXi, please see the Architecture of VMware ESXi Whitepaper which goes into great detail explaining how ESXi is designed and how it differs from classic ESX Server.

When you use Tech Support Mode, you have access to a command line shell that allows you to do things like restart services, but this mode should ONLY be used while working with VMware Technical Support to troubleshoot an issue.

To use Tech Support Mode:

1. Log in to your ESXi host at the console.
2. Press Alt+F1 to switch to the console window.
3. Enter unsupported to start the Tech Support Mode login process. Note that no text will appear on the console window.
4. Enter the password for the root user. Tech Support Mode is now active.
5. Complete tasks in Tech Support Mode.
6. Enter the command exit to exit Tech Support Mode.
7. Press Alt+F2 to return the server to DCUI mode.

After logging in to Tech Support Mode, you will see the following where you can run the commands:

 

Some people equate Tech Support Mode with the Service Console of classic ESX Server. There are some that think ESXi relies on Tech Support Mode for Management and that ESXi is no different than classic ESX Server. This is not true. This "busybox" interface of ESXi is NOT necessary for ESXi to function. In fact you can easily disable Tech Support mode, and ESXi will run just fine.

To disable Tech Support Mode:

1. Connect VMware Infrastructure Client (VI Client) to an ESXi host or a VirtualCenter Server.
2. Browse to a host in the inventory list.
3. Click the Configuration tab.
4. Click the advanced Settings link.
5. Click "VMkernel" in the left-hand side pane.
6. In the list of parameters, deselect VMkernel.Boot.techSupportMode.
7. Restart the ESXi host. Before restarting the host, you should shut down virtual machines on that host or migrate them to another host using VMotion or cold migration.

With Tech Support Mode disabled, you can press Alt+F1, and switch to the console window, but when you type in unsupported, you will NOT be able to enter the special Tech Support Mode (it has been disabled and is not running) as shown by the following:

Since you can run ESXi with Tech Support Mode disabled, you can see that ESXi doesn’t rely on this, and it is nothing like the Console OS of classic VMware ESX 3.

Keeping your Disks Aligned

Update: I just wanted to add that Andrew Bull helped me with this, so thank you Andrew for the help.

I think that VMs run well. They are speedy and responsive, but sometimes you just want to eek out the best in performance.

Everyone knows that when you create a VMFS Volume through VirtualCenter (or is it vCenter now), the VMFS partition is properly aligned.

But what about the actual Virtual Disk? If you create a VMDK on that properly aligned VMFS, it is created as aligned, but the partition inside the VM won't necessarily be aligned. With Windows OSs, if you simply run through the base install, you don't get aligned disks. So what you need to do is create the partition on the disk before you install the OS on that disk. Now because I can use templates, if I create a VM with properly aligned disks, I can then use this VM as a template and create many VMs off of this template knowing that all of these VMs will be properly aligned.

Here is what I have done when creating and setting up the alignment.

First of all, I created a VM with Windows 2003 and updated it to the latest service packs and hotfixes. I call this VM adminvm, why, why not?

Now, let's say I want to create a new VM called vmlab-dns01. First thing I would do is create the VM the same way I would any Virtual Machine. After the VM is created, I would edit the settings of my adminvm and Add the Virtual Disk of vmlab-dns01 to this Virtual Machine by performing the following steps:

1) Edit Settings of adminvm

2) Add New Hard Disk

3) Choose to use Existing Disk

4) Browse to the [VMFS] directory_of_vm and choose the Virtual Disk

5) Click through taking the defaults and Press OK to exit when you get to this screen

6) Boot the adminvm, login, and open a command prompt and run the following commands

diskpart <- launches the disk partitioning environment

list disk <- shows which disks diskpart sees

select disk 1 <- disk 1 is the disk I added to mine, but yours could be different

create partition primary align=64 <- creates a primary partition aligned on the 64KB Offset

list disk

exit

7) Shutdown the Virtual Machine

8) Remove the Virtual Disk from the configuration of adminvm

Now I can boot vmlab-dns01 and start installing my operating environment knowing that I have a properly aligned Virtual Disk. Here is what that looks like when installing my Windows Server 2008 DCE Core VM:

Summary

To create the aligned partition, the diskpart commands I used were the following:

diskpart

list disk

select disk 1

create partition primary align=64

exit

I hope this helps you in your VM best practices

What's the deal with SVVP

Update (10/16/2008): As the actual details as to what OSs and Versions of VMware ESX Server are currently supported and to what extent, please monitor the VMware page on Microsoft's SVVP from Windows Server Catalog as anything in this blog may be outdated at time of reading as we update our Certifications.

 

Recently VMware became the first Hypervisor to be certified under SVVP and was listed along with Cisco as certified platforms. Since then, other Hypervisor vendors have released their certifications, and there have been questions that have come up as to what the certification means. If you look at VMware's Page up on the Windows Server Catalog today, you see the following:

This shows that we have certified ESX Server version 3.5 Update 2 to run x86 based Windows Server 2008 (and below) on Opteron Platforms, and that the Virtual Machine memory size (not the ESX Server itself but the Virtual Machine) is certified to run up to 4GB. This doesn't mean other configurations won't work, it does mean, however, that this is what we certified through the Microsoft Certification Process.

This has generated some confusion with our customers, partners, and others out there as to why we have these limits where Citrix and Novell have different limits and certified platforms. Our Microsoft Alliance team has posted the following to the VMware field and has allowed me to post it on my blog trying to clarify the current situation and what our customers, partners, and others should expect to see.

Microsoft now supports their server products on VMware.  This eliminates a significant barrier to adoption for many prospective customers, and makes all our customers more confident about deploying on VMware.

You can see the list of certified hypervisors here.  Products supported by Microsoft are here.  The official support policy from Microsoft is here.

Since this policy change was announced, several questions have come up from VMware’s partners and customers:

1.  What about Active Directory?  Its not on the list.

   a.  Active Directory is a Server Role provided in Windows Server since Server 2000.  This is similar to file services, print services, or other roles that are included in the operating system.  All of these roles are supported under SVVP.  Active Directory is fully supported under SVVP.

2.  Why isn’t VMware certified with 64-bit versions of Windows?

   a.  Each SVVP run is a multi-day process.  The tools provided by the qualification kit have not always worked as expected.  We have worked through the learning curve, and are now doing SVVP runs continuously.  We chose to certify the 32-bit version of the OS first in order to cover the large installed base of virtual machines that haven’t yet upgraded.  64-bit versions are in the queue and will begin to appear in the next few weeks.  VMware ESX and ESXi products will both be supported.

3.  Why isn’t ESXi certified?

  a.  As a new product, ESXi will be certified after ESX certification is complete.  Our priorities have been placed on the larger number of users running ESX today.

4.  What do the memory limits mean in the certification?

  a.  Certification is completed for a specific “maximum” virtual machine.  The numbers shown are the biggest configuration that VMware has submitted for certification.  These configurations will expand in the fourth quarter.

5.  Why aren’t we certified with bigger memory limits in the virtual machine?

  a. Our initial test runs were conservative to ensure that we fully understood the behavior of the qualification kit.  We’re upgrading the servers soon to provide support for larger VMs.  We plan to deliver certification for all our products at the maximum supportable memory limit with ESX 3.5 update 3.  We will be re-certifying update 2 variants to increase the memory limit well beyond 4 GB early in the fourth quarter.

6.   How long will it take for new product releases to be fully certified?

  a.  Our plan is to ensure that all SVVP certifications are complete within 60 days of product GA.  We hope to significantly out-perform on this commitment if possible.

7.  What does SVVP mean for hardware certification and the VMware HCL?

  a.  SVVP isn’t related to VMware’s HCL.  Microsoft’s SVVP program demonstrates that a hypervisor runs Windows just like physical hardware, and is a requirement for Microsoft to offer support for Windows and other server products like they do on logoed hardware.  VMware’s HCL remains the standard for determining what hardware is supported with VMware products.

8.  I’ve still got questions about Microsoft support with VMware.

  a.  Ask your VMware representative or channel partner.  VMware is happy to help you to understand the details and implications of this program, and how it helps you to deploy Microsoft products on VMware.

I hope that this clears up any confusion, and I want you to know that we are working to get our limits updated to the maximum supported limits by the current product (see response 5).

VMworld 2008, all I can say is - WOW!

Good Morning! I am trying something new and blogging from my email. I honestly don't know if it will work, but I am giving it a shot. (It worked the second try when I verified the address!)

If you do one thing at VMworld, you have to go to the Second Day Keynote. Every year we always show the coolest things at the second day keynote and this year was no exception. I have been to all of the VMworlds, and I have to admit, I didn't think we could top Mendel's demonstration of FT last year. Steve Herrod blew it away!

I don't know when it will be posted, but keep monitoring http://www.vmworld.com and watch the keynote when you get a chance.

Steve did a fantastic job showing where VMware is taking our customers, and really showing how you will benefit from what VMware will deliver.

The atmosphere was charged, the demos were fantastic, and I am so excited to be here at VMware.

What it proved to me is that we have the foundation, we have the resources, we have the vision, and we will DELIVER!

I am sorry for all the ra-ra talk, but this last keynote just got me psyched!

Sent via BlackBerry by AT&T

High Availability for VCMS? It's Clustered!

Recently VMware Technical Support Posted KB Article 1006803 which shows a supported method for clustering VirtualCenter Management Server 2.5 using Microsoft Clustering Service. It isn't the only way supported, and VMware Support will work with you if you are trying a clustering method that is a little different, but this is one documented method.

A detailed description and reference, along with screenshots and steps, is provided on our new VI:OPS site (which just went live) in DOC-1111.

We have had a detailed document on how to cluster VCMS since VirtualCenter Management Server 2.0.1 Patch 2, and now we have a documented use case that works and is updated for VirtualCenter Management Server 2.5.

Now, you can have VirtualCenter up and running, and if something happens to the box, you can fail the cluster over to the other system running MSCS. If VCMS isn't clustered you don't lose anything if VCMS fails (except possibly tasks in progress), but now you have more redundancy and it comes back more quickly.

One of the nice things here about Clustering VCMS is that you can run it all inside of Virtual Machines. This means that you can have a backup copy of your VCMS server without having to buy any new physical servers to place it on (run it on your existing infrastructure)!

Now, I haven't played with it much, but I just got a lab with some servers and shared storage.  I am getting excited about the opportunity to give this a try. With VMworld coming up, I may not get to it until after, but I will get to this and report back what happens.

Until then, check it out, and check out the new VI:OPS site.

Now, is a Great Time to be Keeping Time

There have been issues in the past where a guest OS's time would drift away from the current time. I have seen instances with ESX Server 2.5 where the clock in a VM would be a day behind or ahead if I let it run over the weekend. For most applications this is bad, but for applications that rely on the ability to keep synched with other applications (whether physical or virtual) this is unusable. Back in the 2.5 days there were ways to address this that helped a lot, but didn't solve the problem entirely  like adding the clock=pit boot option to the guest's kernel command line in the /etc/lilo.conf (See KB Article 1420). These fixes also meant that there had to be some changes to the VM and folks didn't like to change the VM kernel. One of the major reasons for this time drift is that the default Linux kernel timer frequency is 1000 Hz which means that the Linux kernel wanted a timer interrupt 1000 times a second. When you have SMP VMs it doesn't just double, but there is a quadratic increase (2 CPUs = 6000 Hz, 4 CPUs = 20,000 Hz).

 

Making the changes in the aforementioned KB Article 1420 would help with the time drift, but it didn't fix it for all VMs and there could still be some drifting. It also appeared that paravirtualized Xen VM guests didn't have this problem. The reason for this is that when you create the paravirtualized VM, it automatically sets the kernel timer frequency within the VM to 250 Hz, so in a VMware VM, which we used the default kernel timer interrupt of 1000 Hz because one normally wouldn't want to recompile the kernel in the VM to adjust the timer, in Xen you got the benefits of less timer interrupts because they give you no choice and change the kernel timer frequency automatically.

 

With ESX Server 3.0.x, we made some improvements for the VM, and the drift for a lot of workloads went away, but it could still rear its ugly head. With ESX Server 3.5 things got a lot better. Not only that, Linux kernels changed the number of Hz they default with, or they have added kernel boot parameters (divider=10) to allow a person to decrease the number of interrupts necessary for a VM.

 

VMware released a very comprehensive document titled, "Timekeeping in VMware Virtual Machines" (http://www.vmware.com/pdf/vmware_timekeeping.pdf). It goes into what the problem is, why it occurs, and how to get your VMs keeping track of time reliably.

 

So like the title of the blog says, now truly is a great time to be keeping time.

VMware Single CPU Licensing Policy

VMware announced this clarification to licensing servers with a single processor on the first of April, and I thought it was great clarification.  I know that in the past, when customers have asked to run VMware ESX Server on a server with a single CPU installed (1 or more cores), we have, on a case by case basis, allowed customers to split the licenses.  This happened for a lot of customers that had branch offices, or retail outlets where it made sense to run single processor servers.

Now, we have publicly clarified this via our EULA.  To do this, you need to have your VMware ESX Server managed by a license server, and you still have to buy licenses in increments of two, but I think this is a great step forward.

Demystifying Microsoft Licensing Policies for Virtual OSs

So, I get this question all of the time.  How do I make sure that I am appropriately licensed to run Microsoft OSs in my VMware VMs.  Can't I just have the number of licenses purchased for all the VMs I am going to run, and then it doesn't matter where I place them?  Also, how do I account for VMotion?

 

Well, you can't just buy all the licenses necessary for the VMs that you have because you have to physically assign the licenses to a physical entity.  Plus, there are different types of licenses (Standard, Enterprise, Data Center Edition) and depending on which one you buy will allow you to run 1, 4, or unlimited virtual machines on that box.

 

Now I am not a Microsoft Employee, and anything you read here is my opinion and shouldn't be used as legal documentation.  You should always consult your Microsoft Sales Representative, but I feel this is accurate when considering simply the Microsoft OS licenses, it doesn't cover the Applications at all.

 

Microsoft has posted some Windows Server Virtualization Calculators that you can use to help determine what licensing you need, and I have posted a You!Tube that tries to demystify this whole thing.  Check out my Hey Dad - Microsoft Virtual Machine Licensing Policies video.

 

Let me know what you think.

One thing to try if DPM does not work in ESX Server 3.5

I was experimenting with the new DPM (Distributed Power Management) feature in VMware Infrastructure 3 the other day, and was having some problems.

I had 4 servers in my cluster (DRS (Distributed Resource Scheduling) and HA were enabled, but DPM was NOT). The systems were:
2 Dual Socket Quad Core Intel
2 Quad Socket Quad Core Intel

They were all attached to the same iSCSI storage. I had one (1) VM created and running and it was able to VMotion between ALL machines. HA is setup so that it can handle 1 system failure. I manually had three of the servers enter Standby Mode and then Powered them back on through the VirtualCenter MUI. When I tried the fourth server, it failed to enter Standby. I enabled DPM in Automatic Mode, and nothing happened. I thought it must be something like it needed some time before it kicked off so I went to lunch. After coming back, all physical boxes were still powered on.

I left it overnight. The machines were still powered on. To get DPM working, I performed the following:

• I disabled DPM
• I, again, did the "Enter Standby -> Poweron" operation for ALL servers
• This time it was successful for ALL servers
• Lastly, I then re-enabled DPM

YEAH! At this point, once I enabled DPM, it sent first one server to standby, and then moved the 5 VMs off of the second server and set that one to standby.

Thank you to the DPM developers for all of their assistance. Also, an interesting tidbit from one of the developers:

...if DPM thinks some machines in the cluster can't come out of standby but others can, it can still work. It just will consider only machines that can come out of standby as candidates to evacuate and power down...

I hope this helps.

Welcome to the Dog House

Hello,

I have just started to blog, so excuse me if my etiquette and style is a little raw. I work for VMware, Inc., and have been with them for over 8 years working as a Systems Engineer on the West and as we have grown, the Northwest. Now, I am working as a Specialist Systems Engineer assisting our Sales teams with Industry Analysis. With that being said, the postings on this site are my own and do not represent VMware’s positions, strategies or opinions. (disclaimer)

A few of my favorite things are my family, baseball, technology and my dogs. Dante is a Flat Coat Retriever and is the BEST DOG EVER. Spot, my little Rat Terrier, is a little more of a challenge, but I love him anyways.


Really, I wanted to start blogging about my views and experiences with Virtualization and if possible give out some good and relevant information.

Please, bear with me, as this is going to be fun.