Hello, i''ve set up a domU (called ''gwdomu'') for acting as a router (NAT + dhcp + firewall services) for a private LAN of machines. i''ve installed dnsmasq package on this domU 1) with as physical machine client in the LAN, it works : the PC gets an IP address from dnsmasq / gwdomu, and can go outside via ''gwdomu'' 2) with a Xen virtual machine client (i called it ''clientdomu''), it doesn''t work. ''gwdomu'' received several DHCPDiscover requests from ''clientdomu'' and send DHCPOFFER to the client mac address but never received DHCPREQUEST / DHCPACK from the client notes : 1) i''m using VLAN 2) i''m using Debian Lenny + Xen 3.2-1 for Dom0 and DomU 3) In the ''clientdomu'' /etc/network/interfaces i''ve added the option post-up ethtool -K eth0 tx off but with no effect... i also put tx off in dom0 ethX interface with no more effect. trace : domu''s config file # vlan115 vif = [ ''mac=00:16:3E:7F:BF:11,bridge=xenbr115'' ] dhcp = "on" clientdomu# cat /etc/net/interfaces auto eth0 iface eth0 inet dhcp post-up ethtool -K eth0 tx off clientdomu# ifconfig eth0 Link encap:Ethernet HWaddr 00:16:3e:7f:bf:11 clientdomu# dhclient eth0 Internet Systems Consortium DHCP Client V3.1.1 Copyright 2004-2008 Internet Systems Consortium. All rights reserved. For info, please visit http://www.isc.org/sw/dhcp/ Listening on LPF/eth0/00:16:3e:7f:bf:11 Sending on LPF/eth0/00:16:3e:7f:bf:11 Sending on Socket/fallback DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 6 DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 8 DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 10 DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 19 DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 18 5 bad udp checksums in 5 packets No DHCPOFFERS received. No working leases in persistent database - sleeping. ---- gwdomu with dnsmasq activated (tail syslog) with a DHCP domU: Jan 30 00:03:41 gwnat dnsmasq[27902]: DHCPOFFER(eth0) 192.168.115.107 00:16:3e:7f:bf:11 Jan 30 00:03:46 gwnat dnsmasq[27902]: DHCPDISCOVER(eth0) 00:16:3e:7f:bf:11 Jan 30 00:03:46 gwnat dnsmasq[27902]: DHCPOFFER(eth0) 192.168.115.107 00:16:3e:7f:bf:11 Jan 30 00:04:29 gwnat dnsmasq[27902]: DHCPDISCOVER(eth0) 00:16:3e:7f:bf:11 Jan 30 00:04:29 gwnat dnsmasq[27902]: DHCPOFFER(eth0) 192.168.115.107 00:16:3e:7f:bf:11 Jan 30 00:04:30 gwnat dnsmasq[27902]: DHCPDISCOVER(eth0) 00:16:3e:7f:bf:11 Jan 30 00:04:30 gwnat dnsmasq[27902]: DHCPOFFER(eth0) 192.168.115.107 00:16:3e:7f:bf:11 Jan 30 00:04:38 gwnat dnsmasq[27902]: DHCPDISCOVER(eth0) 00:16:3e:7f:bf:11 Jan 30 00:04:38 gwnat dnsmasq[27902]: DHCPOFFER(eth0) 192.168.115.107 00:16:3e:7f:bf:11 gwdomu with dnsmasq activated (tail syslog) with a DHCP physical PC: Jan 29 13:01:41 gwnat dnsmasq[16634]: DHCPDISCOVER(eth0) 00:04:75:d1:37:49 Jan 29 13:01:41 gwnat dnsmasq[16634]: DHCPOFFER(eth0) 192.168.115.104 00:04:75:d1:37:49 Jan 29 13:01:41 gwnat dnsmasq[16634]: DHCPREQUEST(eth0) 192.168.115.104 00:04:75:d1:37:49 Jan 29 13:01:41 gwnat dnsmasq[16634]: DHCPACK(eth0) 192.168.115.104 00:04:75:d1:37:49 kodos any idea ? thank you for your help Arnaud _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sat, Jan 30, 2010 at 6:40 PM, Arnaud JAYET <ajayet@free.fr> wrote:> 2) i''m using Debian Lenny + Xen 3.2-1 for Dom0 and DomUstart with that. The usual debugging method is to try deactivating checksum offload on all interfaces (domU''s ethx, dom0''s vifx.y, dom0''s ethx), try without vlan, or change to newer kernel. But the thing is the default kernel and Xen package on debian stable is simply old and unmaintained (I assume you use the default package, last updated 24 Mar 2009?), and is known to have some bugs, so sometimes it''s a lot easier to simply jump to using a newer Xen and kernel version. Personally I use RHEL5 + default 2.6.18 kernel-xen + (on some servers) updated Xen 3.4.x RPM from Gitco, works great. But changing distros might be too much effort :D So what I suggest is you do is try see if debian has newer xen kernel available (from unstable?) and use it for dom0 and domU. If they don''t have it, try using latest 2.6.31 kernel + patch from http://code.google.com/p/gentoo-xen-kernel/downloads/list. -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Fajar A. Nugraha wrote :> On Sat, Jan 30, 2010 at 6:40 PM, Arnaud JAYET <ajayet@free.fr> wrote: >> 2) i''m using Debian Lenny + Xen 3.2-1 for Dom0 and DomU > > start with that. > > The usual debugging method is to try deactivating checksum offload on > all interfaces (domU''s ethx, dom0''s vifx.y, dom0''s ethx), try without > vlan, or change to newer kernel. But the thing is the default kernel > and Xen package on debian stable is simply old and unmaintained (I > assume you use the default package, last updated 24 Mar 2009?), and is > known to have some bugs, so sometimes it''s a lot easier to simply jump > to using a newer Xen and kernel version. > > Personally I use RHEL5 + default 2.6.18 kernel-xen + (on some servers) > updated Xen 3.4.x RPM from Gitco, works great. But changing distros > might be too much effort :D So what I suggest is you do is try see if > debian has newer xen kernel available (from unstable?) and use it for > dom0 and domU. If they don''t have it, try using latest 2.6.31 kernel + > patch from http://code.google.com/p/gentoo-xen-kernel/downloads/list. >You''re maybe right, Xen 3.2 is not the last version but i don''t want to change my Debian Stable for RHEL, sorry ;-) Well, i have solved the problem by disabling tx checksum Dom0''s vifx.y of the domU. let''s say my domU has ID no. 39, after doing ''ethtool -k vif39.0 tx off'' in dom0 it works ; when i launch dhclient on the domU, i get immediatly an IP address (tx is also disabled in domU by adding the line ''post-up ethtool -k eth0 tx off'' in /etc/network/interfaces I wonder now how i can automate disabling tx offload on vifx.y interfaces created when a domU is started ; manually by xm create command or at dom0 boot time with hard links in /etc/xen/auto to the config file ??? # xm list Name ID Mem VCPUs State Time(s) Domain-0 0 256 4 r----- 91.9 gwnat.dev 1 128 1 -b---- 11.4 client.dev 3 256 1 r----- 1.4 # brctl show bridge name bridge id STP enabled interfaces xenbr114 8000.0030482d925d no eth1.114 xenbr115 8000.0030482d925d no eth1.115 vif1.0 vif3.0 # ethtool -k xenbr115 Offload parameters for xenbr115: Cannot get device rx csum settings: Operation not supported Cannot get device flags: Operation not supported rx-checksumming: off ==> tx-checksumming: off <==scatter-gather: off tcp segmentation offload: off udp fragmentation offload: off generic segmentation offload: off large receive offload: off DomU vif interfaces have tx-checksumming on by default : # ethtool -k vif1.0 Offload parameters for vif1.0: Cannot get device rx csum settings: Operation not supported Cannot get device flags: Operation not supported rx-checksumming: off ===> tx-checksumming: on <===scatter-gather: on tcp segmentation offload: on udp fragmentation offload: off generic segmentation offload: off large receive offload: off # ethtool -k vif3.0 Offload parameters for vif3.0: Cannot get device rx csum settings: Operation not supported Cannot get device flags: Operation not supported rx-checksumming: off ===> tx-checksumming: on <===scatter-gather: on tcp segmentation offload: on udp fragmentation offload: off generic segmentation offload: off large receive offload: off ------------ My dom0 /etc/network/interfaces file eth0 = dom0 management (not used for xen bridges) eth1 = vlans for domUs (here 2 tests VLAN 114 and 115) dot1q tagged # Dom0 management interface allow-hotplug eth0 iface eth0 inet static address 192.168.101.37 netmask 255.255.255.0 network 192.168.101.0 broadcast 192.168.101.255 gateway 192.168.101.254 ### VLAN TEST 114 auto eth1.114 iface eth1.114 inet manual auto xenbr114 iface xenbr114 inet manual netmask 255.255.255.0 network 192.168.114.255 bridge-ports eth1.114 ### VLAN TEST 115 auto eth1.115 iface eth1.115 inet manual auto xenbr115 iface xenbr115 inet manual netmask 255.255.255.0 network 192.168.115.255 bridge-ports eth1.115 -- Arnaud _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sat, Jan 30, 2010 at 11:56 PM, Arnaud JAYET <ajayet@free.fr> wrote:> Well, i have solved the problem by disabling tx checksum Dom0''s vifx.y of > the domU.Ah, so the usual fix works :D> I wonder now how i can automate disabling tx offload on vifx.y interfaces > created when a domU is started ; manually by xm create command or at dom0 > boot time with hard links in /etc/xen/auto to the config file ???You could hack /etc/xen/scripts/vif-bridge, preferably be creating a new file and use it on domU config file (per domU) or set it to be default on xend-config.sxp (global). In my case, I hacked it to automatically restart snmp every time a new interface is activated. You might still want to take a look at newer kernels, which probably has the fix without the need to disable tx checksum. -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Fajar A. Nugraha a écrit : > You could hack /etc/xen/scripts/vif-bridge, preferably be creating a> new file and use it on domU config file (per domU) or set it to be > default on xend-config.sxp (global). In my case, I hacked it to > automatically restart snmp every time a new interface is activated.Interesting. But how can you specify a specific network or vif script inside a domU .cfg config file ? I don''t see any explanation about this in Xen user documentation. To automate domU tx checksum disabling i do the following hack : in /etc/xen/scripts/xen-network-common.sh, i added the ehttool line at end of add_to_bridge() function : # Usage: add_to_bridge bridge dev add_to_bridge () { local bridge=$1 local dev=$2 # Don''t add $dev to $bridge if it''s already on a bridge. if [ -e "/sys/class/net/${bridge}/brif/${dev}" ]; then ip link set ${dev} up || true return fi brctl addif ${bridge} ${dev} ip link set ${dev} up ### disabling tx checksum for vif x.y, DHCP trouble ethtool -K ${dev} tx off } When creating the domU, it''s vif X.Y interface has now tx checksum off and my dhcp''s request problem inside domU disappears. my DomU has now its dynamic IP address at boot. I also noticed that the ehttool line in /etc/network/interfaces inside the domU is useless. Without this line, domU''s eth0 tx checksum is on but DHCP request works. domU (ID 17) boot : ... ... Listening on LPF/eth0/00:16:3e:7f:bf:11 Sending on LPF/eth0/00:16:3e:7f:bf:11 Sending on Socket/fallback DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 4 DHCPOFFER from 192.168.115.1 DHCPREQUEST on eth0 to 255.255.255.255 port 67 DHCPACK from 192.168.115.1 bound to 192.168.115.107 -- renewal in 19160 seconds. done. ... ... domU# ethtool -k eth0 Offload parameters for eth0: Cannot get device rx csum settings: Operation not supported Cannot get device flags: Operation not supported rx-checksumming: off ==> tx-checksumming: on scatter-gather: on tcp segmentation offload: on udp fragmentation offload: off generic segmentation offload: off large receive offload: off domO# ethtool -k vif17.0 Offload parameters for vif17.0: Cannot get device rx csum settings: Operation not supported Cannot get device flags: Operation not supported rx-checksumming: off ==> tx-checksumming: off scatter-gather: on tcp segmentation offload: on udp fragmentation offload: off generic segmentation offload: off large receive offload: off ---- A more general question above my specific DHCP problem. Is it a good idea to always disable tx (and rx ?) interface checksum for DomUs (inside domU and inside dom0 vif interfaces) ? I red in the Xen mailing list some performance troubles due to domU tx checksums. In recent Xen version (like 3.4) does the tx/tx checksum are disabled by default in domO vif X.Y interfaces and domUs ? Thank you for your help Fajar. I hope this thread could help Debian Lenny users. Arnaud _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sun, Jan 31, 2010 at 8:48 PM, Arnaud JAYET <ajayet@free.fr> wrote:> Interesting. But how can you specify a specific network or vif script inside > a domU .cfg config file ? > I don''t see any explanation about this in Xen user documentation.run "xm create --help_config | grep vif" You''ll see all possible parameters for vif. Here''s what I use on one domU: vif = [ ''mac=00:16:3E:F3:AE:C0, bridge=br102, script=vif-bridge-snmp, vifname=rad-eth0'', ]> A more general question above my specific DHCP problem. Is it a good idea to > always disable tx (and rx ?) interface checksum for DomUs (inside domU and > inside dom0 vif interfaces) ?TCP checksum offload is supposed to make network faster, so AFAIK it''s a good idea to leave them on. Which is why I suggested you try newer kernel to see if the problem is fixed there so you can run it with the default settings.> In recent Xen version (like 3.4) does the tx/tx checksum are disabled by > default in domO vif X.Y interfaces and domUs ?AFAIK It has always been enabled by default. Like I said earlier, it worked great on RHEL5 :D -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users