I have been working on getting networking setup with Xen on SLES 11 sp2 for over
a week now and I am at a loss as to how to proceed. I have read every bit of
documentation I could find (most of which is incorrect in one or more ways),
have googled and read everything I could find about the subject, and nothing
works. I''m sure I must be missing something stupid, but for the life
of me, I can''t see what it is. So I am hoping that if I explain what
I''m trying to do here, someone will hit me upside the head with a clue
by four and I can finally put this problem to bed.
Here is the situation:
All Dom0 hosts are in one subnet and are connected to a switch with ports
configured for trunking. (I will use 172.16.211.0/24 in the following
examples.)
The DomU hosts that will be built on these Dom0 servers will be in one of three
different networks (depending on their purpose). My examples will use the
following networks:
Dev: 172.16.213.0/24 VLAN: 1
Prod: 172.16.217.0/24 VLAN: 217
Train: 172.16.219.0/24 VLAN: 219
Admin: 172.16.211.0/24 VLAN: 211
So here is what we would like to have:
On the Dom0, we assign an IP address in the Dom0 network, ie. 172.16.211.100 and
the default gateway for the system is 172.16.211.2.
We use vlan tagging to segregate traffic for the Dev, Prod, Admin and Train
networks.
There are bridge devices created for each VLAN that get exported to the
DomU''s.
So for example:
Dom0:
Physical device: em1 is assigned IP address 172.16.211.100/24
There are VLANs created using vconfig as follows: vconfig add em1 213 (and so
on for all 5 vlans)
There are bridge devices associated with each VLAN such that the brctl show
command output looks like this:
# brctl show
bridge name bridge id STP enabled interfaces
br211 8000.047d7b16089b yes vlan211
br213 8000.047d7b16089b yes vlan1
br217 8000.047d7b16089b yes vif6.0
vlan217
br219 8000.047d7b16089c yes vif6.1
vlan219
and at the DomU level, vif6.0 shows as eth0 and vif6.1 shows as eth1.
In this case, the DomU has an ip address of 172.16.217.11 assigned to eth0.
My problem is this:
From Dom0, I can ping my gateway 172.16.211.2 and since the switch knows about
all of the VLANS, I can also ping 172.16.213.2, 172.16.217.2, etc. so
basically, I can see the world, however, if I do an arping -I em1 to any of the
network gateways except for my default gateway, I get no response.
From DomU, I can ping nothing. Not the gateway of 172.16.217.2 or anything
else.
If from Dom0, I run a tcpdump -i br217, and from DomU, starting trying to ping
my gateway, I see the following:
15:41:35.453693 arp who-has 172.16.217.2 tell 172.16.217.11
15:41:36.469757 arp who-has 172.16.217.2 tell 172.16.217.11
15:41:37.469629 arp who-has 172.16.217.2 tell 172.16.217.11
15:41:38.469678 arp who-has 172.16.217.2 tell 172.16.217.11
15:41:39.485728 arp who-has 172.16.217.2 tell 172.16.217.11
15:41:40.485439 arp who-has 172.16.217.2 tell 172.16.217.11
If there is any more information I can provide that will help someone help me,
just ask.
Tom S.
_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
Alexandre Kouznetsov
2012-Jun-14 19:14 UTC
Re: Problem with network in DomU with SLES 11 SP2
El 13/06/12 21:57, Schenk, Thomas escribió:> So for example: > > Dom0: > > Physical device: em1 is assigned IP address 172.16.211.100/24 > > There are VLANs created using vconfig as follows: vconfig add em1 213 > (and so on for all 5 vlans)Isn''t 172.16.211.100/24 supposed to belong to VLAN211? You just binded it to vlan213. I do not work with SLES, but in my case, if Dom0 shall have a IP on a bridge, I assign the IP to the bridge itself, not to the physical interface which is member of the bridge. Let me illustrate: # brctl show xenbr01 8000.0026b9874553 no eth0 [...] # ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:26:b9:87:45:53 inet6 addr: fe80::226:b9ff:fe87:4553/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 [...] # ifconfig xenbr01 xenbr01 Link encap:Ethernet HWaddr 00:26:b9:87:45:53 inet addr:10.13.1.11 Bcast:10.13.1.255 Mask:255.255.255.0 inet6 addr: fe80::226:b9ff:fe87:4553/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 [...]> # brctl show > bridge name bridge id STP enabled interfaces > br211 8000.047d7b16089b yes vlan211 > [...]Please show the configuration of vlan211 and vlan217. cat /proc/net/vlan/config> and at the DomU level, vif6.0 shows as eth0 and vif6.1 shows as eth1.Please show the Xen configuration file for DomU. Particularly the "vif=" line.> In this case, the DomU has an ip address of 172.16.217.11 assigned to eth0.Please show the output of "ifconfig eth0" from DomU and your DomU''s routing table.> My problem is this: > > From Dom0, I can ping my gateway 172.16.211.2 and since the switch > knows about all of the VLANS, I can also ping 172.16.213.2, > 172.16.217.2, etc. so basically, I can see the world, however, if I do > an arping -I em1 to any of the network gateways except for my default > gateway, I get no response. > > From DomU, I can ping nothing. Not the gateway of 172.16.217.2 or > anything else. > > If from Dom0, I run a tcpdump -i br217, and from DomU, starting trying > to ping my gateway, I see the following: > > 15:41:35.453693 arp who-has 172.16.217.2 tell 172.16.217.11As I understand, 172.16.217.2 is reachable on VLAN217, but does not responds properly to ARP requiests. Maybe it''s reproducible from Dom0, so we can isolate it as router''s issue? You could configure you Dom0 to have a valid IP on VLAN217 and run arping, but I wish to suggest you to simplify your test field first, see below. Who is 172.16.*.2, a Layer 3 switch? The setup description is little bit confusing, maybe the real problem has something so do with that. Let me suggest you to simplify your test setup. If it works, you can add complexity until you get what you need, test on each step, so if you encounter a problem it can be isolated. Consider doing the following: Disable/shutdown temporarily all your network bridges and VMs. On Dom0, set up br211, make physical interface vlan211 (that''s how they call it in SLES?) to be member of it. Assign a valid IP address to br211 on Dom0. Check with ping and traceroute, if you router''s different IP''s are reachable. On Dom0, set up br217, make physical interface vlan217 to be member of it. Do not assign IP from 172.16.217.0/24 to Dom0 yet. Config your test DomU to have 1 network interface within br217, assign a 172.16.217.0/24 IP to that interface within DomU. Test if 172.16.217.2 is reachable from DomU. If success, add another bridge to the setup, and so on. If not, we can torubleshoot in detail (personally I don''t trust L3 switches for routing). -- Alexandre Kouznetsov