Aleksander Adamowski
2006-Mar-17 18:46 UTC
[Fedora-xen] 2.6.15-1.2054_FC5xen0 - no network for guest domains
Hi!
I''m using xenguest-install.py to install a guest domain on x86_64.
The anaconda installer start up fine, but when I configure the network
for the guest domain (be it DHCP or static IP), after trying to contact
installation FTP server, I discover that the guest domain has no network
connectivity at all - it cannot resolve the FTP server''s hostname, or
connect to it directly by its IP address; I cannot ping from other hosts
or from dom0 to the guest domain.
The physical network controller is a Broadcom Corporation NetXtreme
BCM5704 Gigabit Ethernet using the tg3 driver.
The virtual network devices are created by Xen on domain 0:
# ifconfig
eth0 Link encap:Ethernet HWaddr 00:15:60:0B:ED:88
inet addr:192.168.254.208 Bcast:192.168.254.255
Mask:255.255.255.0
inet6 addr: fe80::215:60ff:fe0b:ed88/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:475078 errors:0 dropped:0 overruns:0 frame:0
TX packets:601247 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:99094177 (94.5 MiB) TX bytes:322269097 (307.3 MiB)
Interrupt:17
eth0:0 Link encap:Ethernet HWaddr 00:15:60:0B:ED:88
inet addr:192.168.254.4 Bcast:192.168.254.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:17
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:80137 errors:0 dropped:0 overruns:0 frame:0
TX packets:80137 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:288852189 (275.4 MiB) TX bytes:288852189 (275.4 MiB)
vif1.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:99 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:6 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:4158 (4.0 KiB) TX bytes:0 (0.0 b)
xenbr0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:99 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2772 (2.7 KiB) TX bytes:468 (468.0 b)
I suspect that this may have something to do with x86_64 architecture
and the tg3 driver. Any ideas?
--
Best Regards,
Aleksander Adamowski
GG#: 274614
ICQ UIN: 19780575
http://olo.ab.altkom.pl
Stephen C. Tweedie
2006-Mar-20 21:15 UTC
Re: [Fedora-xen] 2.6.15-1.2054_FC5xen0 - no network for guest domains
Hi, On Fri, 2006-03-17 at 19:46 +0100, Aleksander Adamowski wrote:> I''m using xenguest-install.py to install a guest domain on x86_64. > > The anaconda installer start up fine, but when I configure the network > for the guest domain (be it DHCP or static IP), after trying to contact > installation FTP server, I discover that the guest domain has no network > connectivity at all - it cannot resolve the FTP server''s hostname, or > connect to it directly by its IP address; I cannot ping from other hosts > or from dom0 to the guest domain.Works fine for me on x86_64, so it''s not arch-specific. Hmm --- do you have a strong firewall enabled on the dom0 that might be getting in the way?> The physical network controller is a Broadcom Corporation NetXtreme > BCM5704 Gigabit Ethernet using the tg3 driver. > > The virtual network devices are created by Xen on domain 0:This does not look at *all* like a normal xen networking config:> # ifconfig > eth0 Link encap:Ethernet HWaddr 00:15:60:0B:ED:88 > Interrupt:17this implies that eth0 is a physical device;> eth0:0 Link encap:Ethernet HWaddr 00:15:60:0B:ED:88 > inet addr:192.168.254.4 Bcast:192.168.254.255 Mask:255.255.255.0 > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > Interrupt:17with an alias;> vif1.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FFand then there''s a vif1.0, which I assume is the one belonging to the domU guest''s back-end driver in dom0. And finally you''ve got> xenbr0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FFthe bridge to bind them all together. With xen, normally you have no physical eth0: eth0 is a virtual loopback net device, the other end of which is bound to vif0.0; the original physical eth0 is renamed to peth0; and vif0.0 and peth0 are bridged together on xenbr0. So once I start a domU guest in such an environment, bridging shows: # brctl show bridge name bridge id STP enabled interfaces xenbr0 8000.feffffffffff no peth0 vif0.0 vif1.0 Your environment has no virtual eth0, no renamed physical peth0 and no loopback vif0.0; it doesn''t look like any of the dom0 bits that xen tries to set up have been initialised. What sort of network config did you have set up before starting xend? What does "brctl show" show? --Stephen
Aleksander Adamowski
2006-Mar-30 09:43 UTC
Re: [Fedora-xen] 2.6.15-1.2054_FC5xen0 - no network for guest domains
Stephen C. Tweedie wrote:>This does not look at *all* like a normal xen networking config: > > >># ifconfig >>eth0 Link encap:Ethernet HWaddr 00:15:60:0B:ED:88 >> Interrupt:17 >> >> >this implies that eth0 is a physical device; > > >>eth0:0 Link encap:Ethernet HWaddr 00:15:60:0B:ED:88 >> inet addr:192.168.254.4 Bcast:192.168.254.255 Mask:255.255.255.0 >> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >> Interrupt:17 >> >> >with an alias; > >You were right, it seems that the eth0:0 alias had confused the network startup scripts and they didn''t set up the bridging and virtual interfaces correctly. In my case "brctl show" has been showing a "Function not implemented" error in the interfaces column. I got rid of the unnecessary (not anymore) alias from ifcfg-eth0:0 and assigned it''s IP address to ifcfg-eth0. Restarting the network afterwards using the /etc/init.d/network script didn''t solve the problem, but rebooting the system did - all virtual inferfaces and bridging started working properly. Here''s my current state: [root@domzero2 ~]# ifconfig eth0 Link encap:Ethernet HWaddr 00:15:60:0B:ED:88 inet addr:192.168.254.4 Bcast:192.168.254.255 Mask:255.255.255.0 inet6 addr: fe80::215:60ff:fe0b:ed88/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:3953 errors:0 dropped:0 overruns:0 frame:0 TX packets:3482 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:470410 (459.3 KiB) TX bytes:446958 (436.4 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:209 errors:0 dropped:0 overruns:0 frame:0 TX packets:209 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:32078 (31.3 KiB) TX bytes:32078 (31.3 KiB) peth0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MULTICAST MTU:1500 Metric:1 RX packets:3989 errors:0 dropped:0 overruns:0 frame:0 TX packets:3485 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:488345 (476.8 KiB) TX bytes:463068 (452.2 KiB) Interrupt:17 vif0.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:3489 errors:0 dropped:0 overruns:0 frame:0 TX packets:3961 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:448316 (437.8 KiB) TX bytes:471006 (459.9 KiB) xenbr0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:701 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:38658 (37.7 KiB) TX bytes:468 (468.0 b) [root@domzero2 ~]# brctl show bridge name bridge id STP enabled interfaces xenbr0 8000.feffffffffff no peth0 vif0.0 [root@domzero2 ~]# netstat -nr Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 192.168.254.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 0.0.0.0 192.168.254.1 0.0.0.0 UG 0 0 0 eth0 [root@domzero2 ~]# dmesg | egrep ''(eth|vif|xenbr)'' eth0: Tigon3 [partno(349321-001) rev 2100 PHY(5704)] (PCIX:133MHz:64-bit) 10/100/1000BaseT Ethernet 00:15:60:0b:ed:88 eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[1] Split[0] WireSpeed[1] TSOcap[0] eth0: dma_rwctrl[769f4000] eth1: Tigon3 [partno(349321-001) rev 2100 PHY(5704)] (PCIX:133MHz:64-bit) 10/100/1000BaseT Ethernet 00:15:60:0b:ed:87 eth1: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1] eth1: dma_rwctrl[769f4000] tg3: eth0: Link is up at 100 Mbps, full duplex. tg3: eth0: Flow control is off for TX and off for RX. eth0: no IPv6 routers present device vif0.0 entered promiscuous mode xenbr0: port 1(vif0.0) entering learning state xenbr0: topology change detected, propagating xenbr0: port 1(vif0.0) entering forwarding state ADDRCONF(NETDEV_UP): peth0: link is not ready tg3: peth0: Link is up at 100 Mbps, full duplex. tg3: peth0: Flow control is off for TX and off for RX. ADDRCONF(NETDEV_CHANGE): peth0: link becomes ready device peth0 entered promiscuous mode xenbr0: port 2(peth0) entering learning state xenbr0: topology change detected, propagating xenbr0: port 2(peth0) entering forwarding state xenbr0: no IPv6 routers present vif0.0: no IPv6 routers present peth0: no IPv6 routers present eth0: no IPv6 routers present>With xen, normally you have no physical eth0: eth0 is a virtual loopback >net device, the other end of which is bound to vif0.0; the original >physical eth0 is renamed to peth0; and vif0.0 and peth0 are bridged >together on xenbr0. So once I start a domU guest in such an >environment, bridging shows: > ># brctl show >bridge name bridge id STP enabled interfaces >xenbr0 8000.feffffffffff no peth0 > vif0.0 > vif1.0 > >Your environment has no virtual eth0, no renamed physical peth0 and no >loopback vif0.0; it doesn''t look like any of the dom0 bits that xen >tries to set up have been initialised. What sort of network config did >you have set up before starting xend? What does "brctl show" show? > >Thanks for the hint, it has directed me on the right tracks! BTW. Would it be possible to make the Xen network stuff support migrating to Xen from a setup with aliases on physical interfaces, like in my case? I predict that this scenario will be quite typical of migrations to Xen: imagine one has a server with multiple aliases on a physical interface (because there are several services that depend on specifiv IP addresses - as it was in my case). Such a setup is of course a typical candidate for migrating to a virtualized server setup with Xen - I think there will be more people with setups like mine, wanting to switch to Xen as flawlessly as possible. Not very nice that interface aliases currently aren''t handled gracefully - it would be good to make it work for next RHEL... What''s your opinion? -- Best Regards, Aleksander Adamowski GG#: 274614 ICQ UIN: 19780575 http://olo.ab.altkom.pl
Stephen C. Tweedie
2006-Mar-31 21:00 UTC
Re: [Fedora-xen] 2.6.15-1.2054_FC5xen0 - no network for guest domains
Hi, On Thu, 2006-03-30 at 11:43 +0200, Aleksander Adamowski wrote:> You were right, it seems that the eth0:0 alias had confused the network > startup scripts and they didn''t set up the bridging and virtual > interfaces correctly.OK, could you open a bugzilla for this please? We probably want to fix this at some point, even though you''ve been able to work around it for now. Thanks, Stephen
Aleksander Adamowski
2006-Apr-03 16:01 UTC
Re: [Fedora-xen] 2.6.15-1.2054_FC5xen0 - no network for guest domains
Stephen C. Tweedie wrote:>OK, could you open a bugzilla for this please? We probably want to fix >this at some point, even though you''ve been able to work around it for >now. > >Got it: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=187776 -- Best Regards, Aleksander Adamowski GG#: 274614 ICQ UIN: 19780575 http://olo.ab.altkom.pl