Motiejus Jakštys
2012-Mar-16 12:58 UTC
no network on domU in network-bridge configuration on Xen-4.0
I created a very standard Xen-4.0 domU with network-bridge configuration.
However, no packets get out from the virtual machine to the network which
bridge (peth0) is attached to. From guest I can ping the hypervisor, but
can''t
ping anything outside it.
IP addresses:
- `x.x.x.121` -- hypervisor buddha
- `x.x.x.162` -- virutal machine, xen6
What is happening? It is a very standard network configuration. With another
hypervisor exactly same configuration works (same OS, same versions, same
config), but with other box it doesn''t.
Symptoms look similar to [this][1], however, my network configuration seems to
look fine. Any ideas?
xen6:~$ ping -c 1 x.x.x.121
PING x.x.x.121 (x.x.x.121) 56(84) bytes of data.
64 bytes from x.x.x.121: icmp_req=1 ttl=64 time=0.093 ms
--- x.x.x.121 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.093/0.093/0.093/0.000 ms
buddha$ sed -n ''/^[^#].*\(\(network\)\|\(vif\)\)/p''
xend-config.sxp
(network-script network-bridge)
(vif-script vif-bridge)
buddha# ifconfig -a
eth0 Link encap:Ethernet HWaddr 00:30:48:5a:05:fa
inet addr:x.x.x.121 Bcast:x.x.x.255 Mask:255.255.255.0
inet6 addr: fe80::230:48ff:fe5a:5fa/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6082 errors:0 dropped:0 overruns:0 frame:0
TX packets:1138 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:573467 (560.0 KiB) TX bytes:230756 (225.3 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:26 errors:0 dropped:0 overruns:0 frame:0
TX packets:26 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2428 (2.3 KiB) TX bytes:2428 (2.3 KiB)
peth0 Link encap:Ethernet HWaddr 00:30:48:5a:05:fa
inet6 addr: fe80::230:48ff:fe5a:5fa/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:6218 errors:0 dropped:0 overruns:0 frame:0
TX packets:1141 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:693978 (677.7 KiB) TX bytes:235320 (229.8 KiB)
Interrupt:26
vif1.0 Link encap:Ethernet HWaddr fe:ff:ff:ff:ff:ff
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:3 errors:0 dropped:0 overruns:0 frame:0
TX packets:4369 errors:0 dropped:43 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:84 (84.0 B) TX bytes:487332 (475.9 KiB)
buddha# xm network-list xen6
Idx BE MAC Addr. handle state evt-ch tx-/rx-ring-ref BE-path
0 0 00:16:3E:F3:0F:D9 0 4 15 769 /768
/local/domain/0/backend/vif/1/0
buddha# brctl show
bridge name bridge id STP enabled interfaces
eth0 8000.0030485a05fa no peth0
xen6# ifconfig -a
eth0 Link encap:Ethernet HWaddr 00:16:3e:f3:0f:d9
inet addr:x.x.x.162 Bcast:x.x.x.255 Mask:255.255.255.0
inet6 addr: fe80::216:3eff:fef3:fd9/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6870 errors:0 dropped:0 overruns:0 frame:0
TX packets:209 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:628821 (614.0 KiB) TX bytes:31636 (30.8 KiB)
Interrupt:17
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1 errors:0 dropped:0 overruns:0 frame:0
TX packets:1 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:80 (80.0 B) TX bytes:80 (80.0 B)
xen6:~$ ping -c 1 x.x.x.121
PING x.x.x.121 (x.x.x.121) 56(84) bytes of data.
64 bytes from x.x.x.121: icmp_req=1 ttl=64 time=0.081 ms
--- x.x.x.121 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.081/0.081/0.081/0.000 ms
Here is the ethernet controller:
02:05.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit
Ethernet (rev 10)
Subsystem: Super Micro Computer Inc Device 1648
Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 26
Memory at fc9f0000 (64-bit, non-prefetchable) [size=64K]
Expansion ROM at <ignored> [disabled]
Capabilities: [40] PCI-X non-bridge device
Capabilities: [48] Power Management version 2
Capabilities: [50] Vital Product Data
Capabilities: [58] MSI: Enable- Count=1/8 Maskable- 64bit+
Kernel driver in use: tg3
I read somewhere (can''t recall where) that IPMI creates issues with
networking.
So I disabled IPMI.
buddha# uname -a
Linux buddha 2.6.32-5-xen-amd64 #1 SMP Mon Jan 16 20:48:30 UTC 2012 x86_64
GNU/Linux
buddha# lsb_release -a
Distributor ID: Debian
Description: Debian GNU/Linux 6.0.4 (squeeze)
Release: 6.0.4
Codename: squeeze
I asked this on [unix.stackexchange.com][2] as well.
Thanks.
[1]: http://lists.fedoraproject.org/pipermail/xen/2006-March/000249.html
[2]: http://unix.stackexchange.com/questions/34333
Ian Campbell
2012-Mar-16 14:22 UTC
Re: no network on domU in network-bridge configuration on Xen-4.0
On Fri, 2012-03-16 at 12:58 +0000, Motiejus Jakštys wrote:> buddha# ifconfig -a[...]> vif1.0 Link encap:Ethernet HWaddr fe:ff:ff:ff:ff:ff > inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link > UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 > RX packets:3 errors:0 dropped:0 overruns:0 frame:0 > TX packets:4369 errors:0 dropped:43 overruns:0 carrier:0 > collisions:0 txqueuelen:32 > RX bytes:84 (84.0 B) TX bytes:487332 (475.9 KiB)[...]> buddha# brctl show > bridge name bridge id STP enabled interfaces > eth0 8000.0030485a05fa no peth0vif1.0 does not appear to be on the bridge. This would usually indicate some sort of problem with your hotplug scripts. I presume that you can ping from dom0 (budha) to an external address? i.e. the hosts external connectivity is ok. Ian. _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Motiejus Jakštys
2012-Mar-16 15:26 UTC
Re: no network on domU in network-bridge configuration on Xen-4.0
On Fri, Mar 16, 2012 at 14:22, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Fri, 2012-03-16 at 12:58 +0000, Motiejus Jakštys wrote: > >> buddha# ifconfig -a > [...] >> vif1.0 Link encap:Ethernet HWaddr fe:ff:ff:ff:ff:ff >> inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link >> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 >> RX packets:3 errors:0 dropped:0 overruns:0 frame:0 >> TX packets:4369 errors:0 dropped:43 overruns:0 carrier:0 >> collisions:0 txqueuelen:32 >> RX bytes:84 (84.0 B) TX bytes:487332 (475.9 KiB) > [...] >> buddha# brctl show >> bridge name bridge id STP enabled interfaces >> eth0 8000.0030485a05fa no peth0 > > vif1.0 does not appear to be on the bridge. This would usually indicate > some sort of problem with your hotplug scripts.I cut the output at the wrong place :/ buddha# brctl show bridge name bridge id STP enabled interfaces eth0 8000.0030485a05fa no peth0 vif1.0> I presume that you can ping from dom0 (budha) to an external address? > i.e. the hosts external connectivity is ok.Yes, it's fine. -- Motiejus Jakštys _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Pandu Poluan
2012-Mar-16 16:30 UTC
Re: no network on domU in network-bridge configuration on Xen-4.0
On Mar 16, 2012 10:31 PM, "Motiejus Jakštys" <desired.mta@gmail.com> wrote:> > On Fri, Mar 16, 2012 at 14:22, Ian Campbell <Ian.Campbell@citrix.com>wrote:> > On Fri, 2012-03-16 at 12:58 +0000, Motiejus Jakštys wrote: > > > >> buddha# ifconfig -a > > [...] > >> vif1.0 Link encap:Ethernet HWaddr fe:ff:ff:ff:ff:ff > >> inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link > >> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 > >> RX packets:3 errors:0 dropped:0 overruns:0 frame:0 > >> TX packets:4369 errors:0 dropped:43 overruns:0 carrier:0 > >> collisions:0 txqueuelen:32 > >> RX bytes:84 (84.0 B) TX bytes:487332 (475.9 KiB) > > [...] > >> buddha# brctl show > >> bridge name bridge id STP enabled interfaces > >> eth0 8000.0030485a05fa no peth0 > > > > vif1.0 does not appear to be on the bridge. This would usually indicate > > some sort of problem with your hotplug scripts. > > I cut the output at the wrong place :/ > > buddha# brctl show > bridge name bridge id STP enabled interfaces > eth0 8000.0030485a05fa no peth0 > vif1.0 > > > > I presume that you can ping from dom0 (budha) to an external address? > > i.e. the hosts external connectivity is ok. > > Yes, it''s fine. >Could it be iptables/ebtables misconfiguration on dom0? Rgds, _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Motiejus Jakštys
2012-Mar-18 15:26 UTC
Re: no network on domU in network-bridge configuration on Xen-4.0
On Fri, Mar 16, 2012 at 03:26:09PM +0000, Motiejus Jakštys wrote:> On Fri, Mar 16, 2012 at 14:22, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > On Fri, 2012-03-16 at 12:58 +0000, Motiejus Jakštys wrote:I managed to narrow down the problem. The symptoms are exactly like here: http://www.digipedia.pl/usenet/thread/18711/28726/ Once after yet another reboot hypervisor did not have internet access, but guest did (!). The proposed workaround in the link is for BCM5708 (driver bnx2), however, my card is BCM5707 (driver tg3). And b57udiag (broadcom B5707 management service) does not have such an option to "disable management". From related things it has only IPMI. I disabled IPMI, but no help. Maybe other ideas how should I tune the NIC config? Documentation[1]. I also use the latest driver from Broadcom site (3.122g, whereas linux-2.6.32 version is 3.116). # ethtool -i peth0 driver: tg3 version: 3.122g firmware-version: 5704-v3.36 bus-info: 0000:02:05.0 Next step would be to upgrade the firmware of the NIC itself, but I couldn't find the image anywhere.. Any hints? Seems that problem is not really Xen specific, rather Broadcom one. Anywhere else I should ring the bells? [1]: ftp://ftp.supermicro.com/CDR-APLUS_1.12_for_A+_platform/Broadcom/Build8.2.6/DOS/UserDiag/B57UDIAG.pdf Thanks, Motiejus Jakštys _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Fajar A. Nugraha
2012-Mar-19 02:22 UTC
Re: no network on domU in network-bridge configuration on Xen-4.0
2012/3/18 Motiejus Jakštys <desired.mta@gmail.com>:> On Fri, Mar 16, 2012 at 03:26:09PM +0000, Motiejus Jakštys wrote: >> On Fri, Mar 16, 2012 at 14:22, Ian Campbell <Ian.Campbell@citrix.com> wrote: >> > On Fri, 2012-03-16 at 12:58 +0000, Motiejus Jakštys wrote: > > I managed to narrow down the problem. The symptoms are exactly like here: > http://www.digipedia.pl/usenet/thread/18711/28726/ > > Once after yet another reboot hypervisor did not have internet access, but > guest did (!). > > The proposed workaround in the link is for BCM5708 (driver bnx2), however, my > card is BCM5707 (driver tg3). > > And b57udiag (broadcom B5707 management service) does not have such an option > to "disable management". From related things it has only IPMI. I disabled IPMI, > but no help. Maybe other ideas how should I tune the NIC config? > Documentation[1]. > > I also use the latest driver from Broadcom site (3.122g, whereas linux-2.6.32 > version is 3.116). > > # ethtool -i peth0 > driver: tg3 > version: 3.122g > firmware-version: 5704-v3.36 > bus-info: 0000:02:05.0 > > Next step would be to upgrade the firmware of the NIC itself, but I couldn''t > find the image anywhere.. Any hints?Usually the vendor site (e.g. HP, IBM) has it.> > Seems that problem is not really Xen specific, rather Broadcom one. Anywhere > else I should ring the bells? > > [1]: ftp://ftp.supermicro.com/CDR-APLUS_1.12_for_A+_platform/Broadcom/Build8.2.6/DOS/UserDiag/B57UDIAG.pdfAsk supermicro for updated firmware, perhaps? You might also want to try: - booting native kernel, and use qemu/KVM with bridge setup (to make sure that the problem is NOT xen-related), possibly with manually-created bridge - test latest vanilla kernel (which should work fine as dom0 kernel), just in case it''s fixed there already - if it IS xen-specific problem, try latest 4.1. -- Fajar
Motiejus Jakštys
2012-Mar-19 16:40 UTC
Re: [SOLVED] no network on domU in network-bridge configuration on Xen-4.0
On Mon, Mar 19, 2012 at 09:22:07AM +0700, Fajar A. Nugraha wrote:> 2012/3/18 Motiejus Jakštys <desired.mta@gmail.com>: > > On Fri, Mar 16, 2012 at 03:26:09PM +0000, Motiejus Jakštys wrote: > >> On Fri, Mar 16, 2012 at 14:22, Ian Campbell <Ian.Campbell@citrix.com> wrote: > >> > On Fri, 2012-03-16 at 12:58 +0000, Motiejus Jakštys wrote: > > > > I managed to narrow down the problem. The symptoms are exactly like here: > > http://www.digipedia.pl/usenet/thread/18711/28726/What a silly error. Mac filtering was enabled on the switch... :) Thanks all for your answers & time. -- Motiejus Jakštys _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Ian Campbell
2012-Mar-19 16:56 UTC
Re: [SOLVED] no network on domU in network-bridge configuration on Xen-4.0
On Mon, 2012-03-19 at 16:40 +0000, Motiejus Jakštys wrote:> On Mon, Mar 19, 2012 at 09:22:07AM +0700, Fajar A. Nugraha wrote: > > 2012/3/18 Motiejus Jakštys <desired.mta@gmail.com>: > > > On Fri, Mar 16, 2012 at 03:26:09PM +0000, Motiejus Jakštys wrote: > > >> On Fri, Mar 16, 2012 at 14:22, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > >> > On Fri, 2012-03-16 at 12:58 +0000, Motiejus Jakštys wrote: > > > > > > I managed to narrow down the problem. The symptoms are exactly like here: > > > http://www.digipedia.pl/usenet/thread/18711/28726/ > > What a silly error. Mac filtering was enabled on the switch... :)That would never have occurred to me. Glad it's fixed now, thanks for letting us know. Ian. _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users