Xen Users, I''m relatively new to Xen, but I''ve been able to build a Xen host cluster using Ubuntu 11.10 AMD64, DRBD, LVM (for the domain disks) and the Ubuntu Xen package, 4.1.1-2ubuntu4. It works for the most part, but there''s a problem with networking on vm migration that I''ve been struggling with. Using xen-tools I can bootstrap an Ubuntu 11.04 vm (haven''t gotten an 11.10 vm to boot properly yet), and can start it on either host with bridged networking functioning properly. Both a regular migrate and a live migrate to the other host seem to work properly , but once the migrate is complete I can no longer ping the vm from the outside and cannot send anything from the vm either. This problem happens regardless of which host you start the vm on, and networking does not return when the vm is migrated back to the original host. In fact, if you migrate/live migrate the vm to the same host it''s running on, this problem also occurs. When pinging from the outside, pings continue through the live migration process, but stop the instant the vm is running only on the new host. This issue does not go away with time, nor with trying to ping something from within the vm. The vm''s network interface is created when the vm is migrated, and using brctl that interface does appear to be attached to the bridge properly. On the vm, neither the IP, nor the mac address appears to have changed. Both are declared explicitly in the vm''s config file. Also, o n the vm both the transmitted and received packets shown with ifconfig increment, but when watching ifconfig on the host the vm has been migrated to, the received packet count is 0 on the vif for the vm. I''ve read a lot about arp related issues with xen live migrate, but don''t believe this is the same problem. I have the arp notify and accept options set to 1 in sysctl on both the hosts and the vm. I can see network traffic using tshark on the vm, including arp requests from the machine trying to ping the vm from the outside, and responses with the correct mac address. I''ve been looking through the xend and xen-hotplug logs, but haven''t noticed anything that immediately stuck out to me as related to this yet. Have also been reading the xen bug lists etc to see if I''m running up against a bug of sorts but haven''t found anything there yet either. P lease let me know what logs, additional information etc I can provide to help figure this out asap. I''m sure it''s probably something simple I''m just missing. Thanks. Any help you all can provide is greatly appreciated. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Seems like doing a ''xm network-detach'' and ''xm network-attach'' with the appropriate options brings the network back for the vm after the migration. Not sure if that helps point to the root cause or a proper fix though. As always, any help anyone can provide is greatly appreciated. Thanks ----- Original Message ----- From: "Dave Nuzzo" <dnuzzo@pangeaequity.com> To: xen-users@lists.xensource.com Sent: Tuesday, October 25, 2011 9:33:40 AM Subject: [Xen-users] Xen Live Migration Networking Problem Xen Users, I''m relatively new to Xen, but I''ve been able to build a Xen host cluster using Ubuntu 11.10 AMD64, DRBD, LVM (for the domain disks) and the Ubuntu Xen package, 4.1.1-2ubuntu4. It works for the most part, but there''s a problem with networking on vm migration that I''ve been struggling with. Using xen-tools I can bootstrap an Ubuntu 11.04 vm (haven''t gotten an 11.10 vm to boot properly yet), and can start it on either host with bridged networking functioning properly. Both a regular migrate and a live migrate to the other host seem to work properly, but once the migrate is complete I can no longer ping the vm from the outside and cannot send anything from the vm either. This problem happens regardless of which host you start the vm on, and networking does not return when the vm is migrated back to the original host. In fact, if you migrate/live migrate the vm to the same host it''s running on, this problem also occurs. When pinging from the outside, pings continue through the live migration process, but stop the instant the vm is running only on the new host. This issue does not go away with time, nor with trying to ping something from within the vm. The vm''s network interface is created when the vm is migrated, and using brctl that interface does appear to be attached to the bridge properly. On the vm, neither the IP, nor the mac address appears to have changed. Both are declared explicitly in the vm''s config file. Also, on the vm both the transmitted and received packets shown with ifconfig increment, but when watching ifconfig on the host the vm has been migrated to, the received packet count is 0 on the vif for the vm. I''ve read a lot about arp related issues with xen live migrate, but don''t believe this is the same problem. I have the arp notify and accept options set to 1 in sysctl on both the hosts and the vm. I can see network traffic using tshark on the vm, including arp requests from the machine trying to ping the vm from the outside, and responses with the correct mac address. I''ve been looking through the xend and xen-hotplug logs, but haven''t noticed anything that immediately stuck out to me as related to this yet. Have also been reading the xen bug lists etc to see if I''m running up against a bug of sorts but haven''t found anything there yet either. Please let me know what logs, additional information etc I can provide to help figure this out asap. I''m sure it''s probably something simple I''m just missing. Thanks. Any help you all can provide is greatly appreciated. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I''ve tried creating the bridges manually instead of letting xend create them and this problem appears to still happen. Any ideas on this at all? As always, thanks and any help anyone can provide is greatly appreciated. ----- Original Message ----- From: "Dave Nuzzo" <dnuzzo@pangeaequity.com> To: xen-users@lists.xensource.com Sent: Tuesday, October 25, 2011 9:33:40 AM Subject: [Xen-users] Xen Live Migration Networking Problem Xen Users, I''m relatively new to Xen, but I''ve been able to build a Xen host cluster using Ubuntu 11.10 AMD64, DRBD, LVM (for the domain disks) and the Ubuntu Xen package, 4.1.1-2ubuntu4. It works for the most part, but there''s a problem with networking on vm migration that I''ve been struggling with. Using xen-tools I can bootstrap an Ubuntu 11.04 vm (haven''t gotten an 11.10 vm to boot properly yet), and can start it on either host with bridged networking functioning properly. Both a regular migrate and a live migrate to the other host seem to work properly, but once the migrate is complete I can no longer ping the vm from the outside and cannot send anything from the vm either. This problem happens regardless of which host you start the vm on, and networking does not return when the vm is migrated back to the original host. In fact, if you migrate/live migrate the vm to the same host it''s running on, this problem also occurs. When pinging from the outside, pings continue through the live migration process, but stop the instant the vm is running only on the new host. This issue does not go away with time, nor with trying to ping something from within the vm. The vm''s network interface is created when the vm is migrated, and using brctl that interface does appear to be attached to the bridge properly. On the vm, neither the IP, nor the mac address appears to have changed. Both are declared explicitly in the vm''s config file. Also, on the vm both the transmitted and received packets shown with ifconfig increment, but when watching ifconfig on the host the vm has been migrated to, the received packet count is 0 on the vif for the vm. I''ve read a lot about arp related issues with xen live migrate, but don''t believe this is the same problem. I have the arp notify and accept options set to 1 in sysctl on both the hosts and the vm. I can see network traffic using tshark on the vm, including arp requests from the machine trying to ping the vm from the outside, and responses with the correct mac address. I''ve been looking through the xend and xen-hotplug logs, but haven''t noticed anything that immediately stuck out to me as related to this yet. Have also been reading the xen bug lists etc to see if I''m running up against a bug of sorts but haven''t found anything there yet either. Please let me know what logs, additional information etc I can provide to help figure this out asap. I''m sure it''s probably something simple I''m just missing. Thanks. Any help you all can provide is greatly appreciated. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users