Now that we can gather diagnostic info, I think I know why our P2Vs kept failing last week. Another one just died right in front of my eyes. I think either the Ethernet or NFS server at this site occasionally "blips" offline when it gets busy and that messes up P2V migrations. The RHEV export domain is an NFS share offered by an old Storagetek NAS, connected over a 10/100 Ethernet. All of us would prefer a gb Ethernet but this is what we have available. First, an extract from /var/log/messages showing the start and end times for virt-p2v-server, about 3 1/2 hours. [root at Fedora16-64P2V log]# tail /var/log/messages -c 1000 16-64P2V systemd-logind[704]: Removed session 16. Nov 22 19:04:46 Fedora16-64P2V systemd-logind[704]: New session 17 of user root. Nov 22 19:04:48 Fedora16-64P2V virt-v2v[1733]: p2v-server started. Nov 22 20:01:01 Fedora16-64P2V systemd-logind[704]: New session 18 of user root. Nov 22 20:01:02 Fedora16-64P2V systemd-logind[704]: Removed session 18. Nov 22 21:01:01 Fedora16-64P2V systemd-logind[704]: New session 19 of user root. Nov 22 21:01:01 Fedora16-64P2V systemd-logind[704]: Removed session 19. Nov 22 22:01:02 Fedora16-64P2V systemd-logind[704]: New session 20 of user root. Nov 22 22:01:02 Fedora16-64P2V systemd-logind[704]: Removed session 20. Nov 22 22:33:44 Fedora16-64P2V virt-v2v[1733]: FATAL: Error receiving data: Nov 22 22:33:44 Fedora16-64P2V systemd-logind[704]: Removed session 17. Nov 22 22:33:44 Fedora16-64P2V virt-v2v[1733]: WARNING: Error messages were written to /var/log/virt-p2v-server.1322010288.log. Nov 22 22:33:44 Fedora16-64P2V virt-v2v[1733]: p2v-server exited. [root at Fedora16-64P2V log]# Next, the output from the trace log file from virt-p2v-server: [root at Fedora16-64P2V log]# more virt-p2v-server.1322010288.log virt-v2v: Error receiving data: [root at Fedora16-64P2V log]# Before it blew up, I was watching network usage of my RHEV host with the Fedora migration server VM and it consistently pegged above 90%+. I've never seen an Ethernet that saturated before. The iftop utility on my Fedora guest migration server showed data consistently flying at 69-90+ Mb per second. - Greg Scott
Trying that same P2V again, this time with everything connected to a brand new HP Procurve 2910al-48G. All NICs on my hosts now show up at 1000 Gb now and this P2V still failed, this time after running 5 hours. The source server is an aging HP Proliant ML110 running Windows 2003 (I think). The conversion server is a 64 bit Fedora VM running in a RHEV environment. The RHEV Export domain lives in an older Storagetek NAS. For the next troubleshooting step, we'll brew up another NFS server from somewhere and eliminate the Storagetek. Here is an extract from /var/log/messages. [root at Fedora16-64P2V log]# grep p2v-server messages . . (Older unrelated messages removed) . . Nov 23 14:55:59 Fedora16-64P2V virt-v2v[3452]: p2v-server started. Nov 23 20:04:29 Fedora16-64P2V virt-v2v[3452]: WARNING: Error messages were written to /var/log/virt-p2v-server.1322081759.log. Nov 23 20:04:29 Fedora16-64P2V virt-v2v[3452]: p2v-server exited. [root at Fedora16-64P2V log]# And here is the trace log file from virt-p2v-server: [root at Fedora16-64P2V log]# more virt-p2v-server.1322081759.log virt-v2v: Error receiving data: [root at Fedora16-64P2V log]# - Greg
I did one with ntbackup and restore a few months ago and ran into problems with the virtual cpu not matching the virtual bios. That one was in a libvirt environment and that VM will retire soon. But with the current project, these VMs will be around a while. Sent from my Droid Charge on Verizon 4GLTE Matthew Booth <mbooth at redhat.com> wrote: On 12/02/2011 03:12 AM, Greg Scott wrote:> Hey Matt - > > With a new release of virt-v2v a while away, what if we were to try a > P2V by hand? I'm thinking of maybe provisioning a Windows RHEV VM, > doing an ntbackup of the source physical machine and doing a bare metal > restore on the VM. I've done it that way before with mixed results. > > We have that hack I put together to load the viostor block driver stuff > on the source machine so the target VM will boot. Are there any HAL > kinds of things or other hardware/firmware issues we need to also be > thinking about?Windows mostly fixes itself. If it boots and you don't see any missing drivers in Device Manager, you're probably good to go. Matt -- Matthew Booth, RHCA, RHCSS Red Hat Engineering, Virtualisation Team GPG ID: D33C3490 GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490
> Trying a P2V again, this time using a physical Fedora system for > the NFS RHEV Export domain instead of the Storagetek NFS server. > Everything else is the same. Same source server, same conversion > server, same v2v and p2v versions for now. The only variable > different is the RHEV Export NFS server.This time, the P2V finished at 10:36 PM. Start time was 8:38 PM. Total size was 74 GB over roughly 2 hours of elapsed time. I just started a RHEV import and that should finish a couple of hours from now. - Greg