Vmotion in Vsphere

I have just read something which had me thinking, should you increase the amount of Vmotions on a Vsphere host, from the current limitation of two per host. This can be easily achieved by editing the vpdx.cfg file on your virtual centre server as in this post http://www.boche.net/blog/?p=806. I tend to think it's there for a reason and in my experience I would advise against it.

Ok so whats my thinking behind this?  Well if you examine the actual Vmotion process there are some caveats in there that could cause performance issues if you there is a lot of I/O (I am thinking Database and Application servers here)

The following below is the Vmotion Process in depth as I understand it

1: Command sent to VC to check prerequisites 

2: Provision new VM on target host

3: Precopy memory from source to target, with ongoing memory changes logged in a memory bitmap

4: Quiesce VM on the source host and copy memory bitmap to target host

5: Start VM on target host

6: "Demand page" the source VM when applications attempt to read/write modified memory

7: Note - The new VM comes up before ALL the memory is copied over. We can do this, because once the initial copy has taken place, subsequent changes to the app touch only a fraction of all the memory pages.

8: "Background page" the source VM until all memory has been successfully copied

9: Delete VM from source host

I think the important issue here is point 5, the Virtual machine starts on the new Vsphere host BEFORE the memory map is copied over.

I have seen issues in a production environment whereby you Vmotion from a host running at high Memory utilisation and with a guest with that has a lot of current I/O, it will become very slow and unresponsive. In essence it seems that when the VM is migrated a memory delta file will be created on the source host. This will then be copied to the target host, and if there is a shortage of physical memory on the source host a disk based swap file will be used which is considerably slower and hence possible performance issues.

So to sum up the more concurrent Vmotions you run the more risk you have of problems.