Trying to PXE boot a domU with xen-qemu-dm on Debian Squeeze. VIF is bridged to a dedicated physical interface. boot = ''n'' in the domU configuration. The gPXE DHCP client in Qemu would always time out and the VM would halt without booting. On watching the physical interface, I couldn''t see any DHCP traffic at all. After hours of tcpdump-ing first the physical then the virtual interfaces, realised that the bridge was taking time to come up. gPXE was sending out DHCP requests before the bridge came up, so they never reached the wire. The DHCP requests were visible in the VIF traffic, but not in the ETH traffic. Solutions: 1. Press Ctrl-B to reach the gPXE command line. Wait for 30-odd seconds for the bridge to come up. Type in: autoboot . You have to get to a domU VNC session fast in order to be able to do this. 2. Insert a 40-second delay into /etc/xen/scripts/vif-common.sh (use a sleep after the iptables commands). Unfortunately, this delays startup of all bridged domUs by that long. Is there a better way, e.g. by somehow forcing gPXE to sleep for some time before it tries the DHCP? In any case, now our automated Debian install is working for bridged domU VMs. Hope this is useful to others with the same problem. Regards, -- Raj -- Raj Mathur || raju@kandalaya.org || GPG: http://otheronepercent.blogspot.com || http://kandalaya.org || CC68 It is the mind that moves || http://schizoid.in || D17F
Hello. El 11/08/12 22:48, Raj Mathur (राज माथुर) escribió:> Is there a better way, e.g. by somehow forcing gPXE to sleep for some > time before it tries the DHCP? > > In any case, now our automated Debian install is working for bridged > domU VMs. Hope this is useful to others with the same problem.By default, at least in Debian, when you attach a network interface (vif, in this case) to a bridge, there is delay before the packets are forwarded to the newly attached interface. The delay time is configurable via interfaces file or brctl utility. There was a thread about this 2012-04-03 08:42. I quote the relevant part: > Other thing you should have in mind if you use bridged network, > is gPXE's timeout vs bridge forward delay. In short, by default > gPXE get desperate of not getting DHCP answer before the bridge > become operation, and fails to boot. Can be fixed manually with > "brctl setfd $BRIDGENAME 5" or in your network configuration file. > > I don't use network-bridge script, my Debian's > /etc/network/interfaces looks like this (note that Dom0 has no > IP on that interface in my setup): > iface xenbr22 inet manual > bridge_ports eth1.22 > bridge_stp off > bridge_fd 5 -- Alexandre Kouznetsov _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
On Monday 13 Aug 2012, Alexandre Kouznetsov wrote:> Hello. > > El 11/08/12 22:48, Raj Mathur (राज माथुर) escribió: > > Is there a better way, e.g. by somehow forcing gPXE to sleep for > > some time before it tries the DHCP? > > > > In any case, now our automated Debian install is working for > > bridged domU VMs. Hope this is useful to others with the same > > problem. > > By default, at least in Debian, when you attach a network interface > (vif, in this case) to a bridge, there is delay before the packets > are forwarded to the newly attached interface. The delay time is > configurable via interfaces file or brctl utility.Thanks for that information. That is exactly the problem we were facing. In our case, the delay was no trouble except when we needed to PXE boot to install. After install, we boot the VM with the normal kernel= and root= parameters, and bridge forward delays aren't noticeable. Two further questions arise: 1. Presumably the bridge forward delay is there for a purpose. Can reducing that value result in other problems? 2. Is it possible to get the current value of the forward delay from a bridge? If we can do that, we can script a reduced delay when PXE booting and installing a VM, then go back to the default delay after the VM install is over. Regards, -- Raj -- Raj Mathur || raju@kandalaya.org || GPG: http://otheronepercent.blogspot.com || http://kandalaya.org || CC68 It is the mind that moves || http://schizoid.in || D17F _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
On Monday 13 Aug 2012, Raj Mathur (राज माथुर) wrote:> On Monday 13 Aug 2012, Alexandre Kouznetsov wrote: > > Hello. > > > > El 11/08/12 22:48, Raj Mathur (राज माथुर) escribió: > > > Is there a better way, e.g. by somehow forcing gPXE to sleep for > > > some time before it tries the DHCP? > > > > > > In any case, now our automated Debian install is working for > > > bridged domU VMs. Hope this is useful to others with the same > > > problem. > > > > By default, at least in Debian, when you attach a network interface > > (vif, in this case) to a bridge, there is delay before the packets > > are forwarded to the newly attached interface. The delay time is > > configurable via interfaces file or brctl utility. > > Thanks for that information. That is exactly the problem we were > facing. > > In our case, the delay was no trouble except when we needed to PXE > boot to install. After install, we boot the VM with the normal > kernel= and root= parameters, and bridge forward delays aren't > noticeable. > > Two further questions arise: > > 1. Presumably the bridge forward delay is there for a purpose. Can > reducing that value result in other problems? > > 2. Is it possible to get the current value of the forward delay from > a bridge? If we can do that, we can script a reduced delay when PXE > booting and installing a VM, then go back to the default delay after > the VM install is over.To answer my own questions, a detailed explanation is available at: http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge#FAQ Specifically, see: http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge#Does_DHCP_work_over.2Fthrough_a_bridge.3F While it doesn't tell you how to get the current value of the Forwarding Delay in Linux, it does mention that the default is 30 seconds, and gives the pros and cons of changing it. Regards, -- Raj -- Raj Mathur || raju@kandalaya.org || GPG: http://otheronepercent.blogspot.com || http://kandalaya.org || CC68 It is the mind that moves || http://schizoid.in || D17F _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Hello. El 13/08/12 11:49, Raj Mathur (राज माथुर) escribió:> 1. Presumably the bridge forward delay is there for a purpose. Can > reducing that value result in other problems?Check the thread I have referenced (2012-04-03 08:42 -0500), this question has been raised there. Generally speaking: _should_ be no problem. At least I have not encountered any (yet), and nobody has raised his hand to tell he had some. It has mainly to do with STP, which becomes relevant only when there is a ethernet loop risk. In any case, I disable STP on my VM bridges and pay specific attention not to put more than one physical interface in the same bridge, hope that eliminates possibility of looping. In a attempt to prevent surprises, I do not reduce the delay to 0, it is set to something closer to the defaults instead. 5 seems to be just small enough to allow gPXE to do it's job.> 2. Is it possible to get the current value of the forward delay from a > bridge? If we can do that, we can script a reduced delay when PXE > booting and installing a VM, then go back to the default delay after the > VM install is over.Not sure if it's a good idea. 15 (the usual default) seems to be as good as 5. Personally, I prefer to leave this kind of things alone: set once, make it persistent, leave it static unless there is a good reason to change it dynamically. brctl seems not to offer a _nice_ interface to get the forward_delay value. If you wish to parse a multi-colon output, try "brctl showstp <bridge>". Also, you could check under /sys/devices/virtual/net/*. Greetings. -- Alexandre Kouznetsov _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Hello, On Monday 13 August 2012 19:18:44 Raj Mathur (राज माथुर) wrote:> http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge#Doe >s_DHCP_work_over.2Fthrough_a_bridge.3F > > While it doesn''t tell you how to get the current value of the Forwarding > Delay in Linux, it does mention that the default is 30 seconds, and > gives the pros and cons of changing it.brctl showstp "$bridge" | grep ''forward delay'' "forward delay" is the locally configured value, while "bridge forward delay" is the globally enforced value provided via STP, that is the "forwarding delay" configured on the root bridge is propagated to all the other bridges, which then use that value to overwrite any previously configured local value. So if your host is a leaf bridge, make sure to disable STP as well. Sincerely Philipp -- Philipp Hahn Open Source Software Engineer hahn@univention.de Univention GmbH be open. fon: +49 421 22 232- 0 Mary-Somerville-Str.1 D-28359 Bremen fax: +49 421 22 232-99 http://www.univention.de/ _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users