I''m very close to getting bonding + vlans + bridging to work.
The only part that''s not working is ARP.
Why i have no idea? But if put in an arp entry for the default route
manually with ''arp -s 192.168.129.1 00:00:0C:07:0C:C9''
everything
works great. (Everything that goes though the default route anyway).
Now if i run tcpdump on any of the interfaces involved in the bridge,
eth0, eth1, bond0, vlan151 or vlan151-br, vif0,0, or veth0 (i don''t
rename any interfaces) then it works fine, I get arp replays. But if i
use ''ip'' to turn on promisc any of the interfaces it
doesn''t work,
just like it doesn''t work without promisc off.
The script is a little diffrent then the last one i posted, I''ll atach
it at the end.
Basically I''m combining eth0 and eth1 into bond0, add vlans to bond0
named "vlanXXX" make a bridge named vlanXXX-br and add vlanXXX to it,
and then domU''s are started with bridge=vlanXXX-br in there xen config
file to decide what vlan they should be part of.
So what is tcpdump doing to make arp work???
Eli
--- script ---
#!/bin/sh
# read in config for dom0
. /etc/xen/network/dom0
modprobe bonding
modprobe 8021q
modprobe bridge
ip link set bond0 down arp off
ip link set bond0 addr fe:ff:ff:ff:ff:ff
ip addr flush bond0
ip link set bond0 up
ifenslave bond0 eth0 eth1
vconfig set_name_type VLAN_PLUS_VID_NO_PAD
# vlans are stored in /etc/xen/network/vlans one number per line
# it''ll create a vlanXXX-br bridge for eath vlan
for vlan in `cat /etc/xen/network/vlans` ; do
vconfig add bond0 ${vlan}
ip link set vlan${vlan} up
brctl addbr vlan${vlan}-br
brctl stp vlan${vlan}-br off
brctl setfd vlan${vlan}-br off
#ip link set vlan${vlan} down arp off
brctl addif vlan${vlan}-br vlan${vlan}
ip link set vlan${vlan}-br up
done
# setup dom0''s ip addr on one of the vlan bridgs
# as defined in /etc/xen/network/dom0
brctl addif vlan${dom0_VLAN}-br vif0.0
ip link set veth0 addr 00:0d:60:4e:48:88 arp on
ip link set vif0.0 up
ip link set veth0 up
ethtool -K veth0 tx off
ip addr add ${dom0_IPADDR} broadcast ${dom0_BOADCAST} dev veth0
ip route add default via ${dom0_GATEWAY} dev veth0
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Sébastien CRAMATTE
2006-Aug-31 18:36 UTC
Re: [Xen-users] bonding + vlans ALMOST working, NO ARP
I''ve seen that xen open suse package contains somes patch for bonding I don''t know if this is just a patch for bridge script or for netdev driver too ... I can''t able to open it with alien + rpm under my debian sarge ... maybe you or someone can ? this is the link : http://download.opensuse.org/distribution/SL-OSS-factory/inst-source/suse/src/xen-3.0.3_11134-4.src.rpm Regards Eli Criffield a écrit :> I''m very close to getting bonding + vlans + bridging to work. > > The only part that''s not working is ARP. > > Why i have no idea? But if put in an arp entry for the default route > manually with ''arp -s 192.168.129.1 00:00:0C:07:0C:C9'' everything > works great. (Everything that goes though the default route anyway). > > Now if i run tcpdump on any of the interfaces involved in the bridge, > eth0, eth1, bond0, vlan151 or vlan151-br, vif0,0, or veth0 (i don''t > rename any interfaces) then it works fine, I get arp replays. But if i > use ''ip'' to turn on promisc any of the interfaces it doesn''t work, > just like it doesn''t work without promisc off. > > The script is a little diffrent then the last one i posted, I''ll atach > it at the end. > > Basically I''m combining eth0 and eth1 into bond0, add vlans to bond0 > named "vlanXXX" make a bridge named vlanXXX-br and add vlanXXX to it, > and then domU''s are started with bridge=vlanXXX-br in there xen config > file to decide what vlan they should be part of. > > So what is tcpdump doing to make arp work??? > > Eli > > --- script --- > #!/bin/sh > # read in config for dom0 > . /etc/xen/network/dom0 > > modprobe bonding > modprobe 8021q > modprobe bridge > > ip link set bond0 down arp off > ip link set bond0 addr fe:ff:ff:ff:ff:ff > ip addr flush bond0 > ip link set bond0 up > ifenslave bond0 eth0 eth1 > > vconfig set_name_type VLAN_PLUS_VID_NO_PAD > > # vlans are stored in /etc/xen/network/vlans one number per line > # it''ll create a vlanXXX-br bridge for eath vlan > > for vlan in `cat /etc/xen/network/vlans` ; do > vconfig add bond0 ${vlan} > ip link set vlan${vlan} up > brctl addbr vlan${vlan}-br > brctl stp vlan${vlan}-br off > brctl setfd vlan${vlan}-br off > #ip link set vlan${vlan} down arp off > brctl addif vlan${vlan}-br vlan${vlan} > ip link set vlan${vlan}-br up > done > > # setup dom0''s ip addr on one of the vlan bridgs > # as defined in /etc/xen/network/dom0 > brctl addif vlan${dom0_VLAN}-br vif0.0 > > ip link set veth0 addr 00:0d:60:4e:48:88 arp on > ip link set vif0.0 up > ip link set veth0 up > ethtool -K veth0 tx off > > ip addr add ${dom0_IPADDR} broadcast ${dom0_BOADCAST} dev veth0 > ip route add default via ${dom0_GATEWAY} dev veth0 > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Tomasz Chmielewski
2006-Aug-31 18:54 UTC
Re: [Xen-users] bonding + vlans ALMOST working, NO ARP
Sébastien CRAMATTE wrote:> I''ve seen that xen open suse package contains somes patch for bonding > I don''t know if this is just a patch for bridge script or for netdev > driver too ... > > I can''t able to open it with alien + rpm under my debian sarge ... > maybe you or someone can ? > > this is the link : > http://download.opensuse.org/distribution/SL-OSS-factory/inst-source/suse/src/xen-3.0.3_11134-4.src.rpmThere you go: http://www2.wpkg.org/xen-3.0.3_11134-4.src.tar.bz2 -- Tomasz Chmielewski Software deployment with Samba http://wpkg.org _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Eli Criffield
2006-Aug-31 19:34 UTC
Re: [Xen-users] bonding + vlans ALMOST working, NO ARP
I used rpm2cpio xen-3.0.3_11134-4.src.rpm |cpio -ivmd There is a bonding and bridge patch but it just changes the network-bridge script. Eli On 8/31/06, Tomasz Chmielewski <mangoo@wpkg.org> wrote:> Sébastien CRAMATTE wrote: > > I''ve seen that xen open suse package contains somes patch for bonding > > I don''t know if this is just a patch for bridge script or for netdev > > driver too ... > > > > I can''t able to open it with alien + rpm under my debian sarge ... > > maybe you or someone can ? > > > > this is the link : > > http://download.opensuse.org/distribution/SL-OSS-factory/inst-source/suse/src/xen-3.0.3_11134-4.src.rpm > > There you go: > > http://www2.wpkg.org/xen-3.0.3_11134-4.src.tar.bz2 > > > -- > Tomasz Chmielewski > > Software deployment with Samba > http://wpkg.org > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Tomasz Chmielewski
2006-Sep-01 14:33 UTC
Re: [Xen-users] bonding + vlans ALMOST working, NO ARP
Eli Criffield wrote:> I''m very close to getting bonding + vlans + bridging to work. > > The only part that''s not working is ARP. > > Why i have no idea? But if put in an arp entry for the default route > manually with ''arp -s 192.168.129.1 00:00:0C:07:0C:C9'' everything > works great. (Everything that goes though the default route anyway). > > Now if i run tcpdump on any of the interfaces involved in the bridge, > eth0, eth1, bond0, vlan151 or vlan151-br, vif0,0, or veth0 (i don''t > rename any interfaces) then it works fine, I get arp replays. But if i > use ''ip'' to turn on promisc any of the interfaces it doesn''t work, > just like it doesn''t work without promisc off. > > The script is a little diffrent then the last one i posted, I''ll atach > it at the end. > > Basically I''m combining eth0 and eth1 into bond0, add vlans to bond0 > named "vlanXXX" make a bridge named vlanXXX-br and add vlanXXX to it, > and then domU''s are started with bridge=vlanXXX-br in there xen config > file to decide what vlan they should be part of. > > So what is tcpdump doing to make arp work???Doesn''t it look like this issue: http://wiki.xensource.com/xenwiki/XenFaq#head-2e2a099d5575bb46de6d52786f093c13599af333 3.3. Why does my new domain receive no network traffic until after it initiates an outgoing connection? This is an issue that occurs under the following circumstances: * You do not specify the domain''s MAC address, causing a random MAC address to be selected at domain creation time * The upstream router has a local ARP cache When a domain is destroyed, the host system''s ARP cache is purged of addresses for the defunct virtual NIC. If the domain is recreated it is likely to be allocated a different random MAC address. This is no problem for the host machine, but the external switch/router still knows the "old" MAC address. The switch/router requires an outbound packet to the outside world to implicitly eradicate the old ARP-cache entry. This is not really Xen''s fault at all, just a property of the implementation of ARP resolution. -- Tomasz Chmielewski http://wpkg.org _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Eli Criffield
2006-Sep-01 15:19 UTC
Re: [Xen-users] bonding + vlans ALMOST working, NO ARP
Nea all mac''s are set static so the switchs and routers with arp cache don''t have to update anything. Eli On 9/1/06, Tomasz Chmielewski <mangoo@wpkg.org> wrote:> Eli Criffield wrote: > > I''m very close to getting bonding + vlans + bridging to work. > > > > The only part that''s not working is ARP. > > > > Why i have no idea? But if put in an arp entry for the default route > > manually with ''arp -s 192.168.129.1 00:00:0C:07:0C:C9'' everything > > works great. (Everything that goes though the default route anyway). > > > > Now if i run tcpdump on any of the interfaces involved in the bridge, > > eth0, eth1, bond0, vlan151 or vlan151-br, vif0,0, or veth0 (i don''t > > rename any interfaces) then it works fine, I get arp replays. But if i > > use ''ip'' to turn on promisc any of the interfaces it doesn''t work, > > just like it doesn''t work without promisc off. > > > > The script is a little diffrent then the last one i posted, I''ll atach > > it at the end. > > > > Basically I''m combining eth0 and eth1 into bond0, add vlans to bond0 > > named "vlanXXX" make a bridge named vlanXXX-br and add vlanXXX to it, > > and then domU''s are started with bridge=vlanXXX-br in there xen config > > file to decide what vlan they should be part of. > > > > So what is tcpdump doing to make arp work??? > > Doesn''t it look like this issue: > > http://wiki.xensource.com/xenwiki/XenFaq#head-2e2a099d5575bb46de6d52786f093c13599af333 > > 3.3. Why does my new domain receive no network traffic until after it > initiates an outgoing connection? > > This is an issue that occurs under the following circumstances: > > * You do not specify the domain''s MAC address, causing a random MAC > address to be selected at domain creation time > * The upstream router has a local ARP cache > > When a domain is destroyed, the host system''s ARP cache is purged of > addresses for the defunct virtual NIC. If the domain is recreated it is > likely to be allocated a different random MAC address. This is no > problem for the host machine, but the external switch/router still knows > the "old" MAC address. The switch/router requires an outbound packet to > the outside world to implicitly eradicate the old ARP-cache entry. This > is not really Xen''s fault at all, just a property of the implementation > of ARP resolution. > > > -- > Tomasz Chmielewski > http://wpkg.org >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Eli Criffield
2006-Sep-01 17:46 UTC
Re: [Xen-users] bonding + vlans ALMOST working, NO ARP
I opened a bug for this issue http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=753 On 9/1/06, Eli Criffield <elicriffield@gmail.com> wrote:> Nea all mac''s are set static so the switchs and routers with arp cache > don''t have to update anything. > > Eli > > On 9/1/06, Tomasz Chmielewski <mangoo@wpkg.org> wrote: > > Eli Criffield wrote: > > > I''m very close to getting bonding + vlans + bridging to work. > > > > > > The only part that''s not working is ARP. > > > > > > Why i have no idea? But if put in an arp entry for the default route > > > manually with ''arp -s 192.168.129.1 00:00:0C:07:0C:C9'' everything > > > works great. (Everything that goes though the default route anyway). > > > > > > Now if i run tcpdump on any of the interfaces involved in the bridge, > > > eth0, eth1, bond0, vlan151 or vlan151-br, vif0,0, or veth0 (i don''t > > > rename any interfaces) then it works fine, I get arp replays. But if i > > > use ''ip'' to turn on promisc any of the interfaces it doesn''t work, > > > just like it doesn''t work without promisc off. > > > > > > The script is a little diffrent then the last one i posted, I''ll atach > > > it at the end. > > > > > > Basically I''m combining eth0 and eth1 into bond0, add vlans to bond0 > > > named "vlanXXX" make a bridge named vlanXXX-br and add vlanXXX to it, > > > and then domU''s are started with bridge=vlanXXX-br in there xen config > > > file to decide what vlan they should be part of. > > > > > > So what is tcpdump doing to make arp work??? > > > > Doesn''t it look like this issue: > > > > http://wiki.xensource.com/xenwiki/XenFaq#head-2e2a099d5575bb46de6d52786f093c13599af333 > > > > 3.3. Why does my new domain receive no network traffic until after it > > initiates an outgoing connection? > > > > This is an issue that occurs under the following circumstances: > > > > * You do not specify the domain''s MAC address, causing a random MAC > > address to be selected at domain creation time > > * The upstream router has a local ARP cache > > > > When a domain is destroyed, the host system''s ARP cache is purged of > > addresses for the defunct virtual NIC. If the domain is recreated it is > > likely to be allocated a different random MAC address. This is no > > problem for the host machine, but the external switch/router still knows > > the "old" MAC address. The switch/router requires an outbound packet to > > the outside world to implicitly eradicate the old ARP-cache entry. This > > is not really Xen''s fault at all, just a property of the implementation > > of ARP resolution. > > > > > > -- > > Tomasz Chmielewski > > http://wpkg.org > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Tomasz Chmielewski
2006-Sep-04 11:21 UTC
Re: [Xen-users] bonding + vlans ALMOST working, NO ARP
Eli Criffield wrote:> Nea all mac''s are set static so the switchs and routers with arp cache > don''t have to update anything.Hi, Did you solve it? I noticed the same happens for me, with bonding enabled, no VLANs. I need to add host MAC addresses via "arp -s <IP> <MAC>", otherwise, I can''t reach them from guest domains. -- Tomasz Chmielewski http://wpkg.org _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Eli Criffield
2006-Sep-04 18:51 UTC
Re: [Xen-users] bonding + vlans ALMOST working, NO ARP
Nope haven''t solved it. You should add your findings to the bug at http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=753 Maybe there needs to be a patch to fix it. Eli On 9/4/06, Tomasz Chmielewski <mangoo@wpkg.org> wrote:> Eli Criffield wrote: > > Nea all mac''s are set static so the switchs and routers with arp cache > > don''t have to update anything. > > Hi, > > Did you solve it? > > I noticed the same happens for me, with bonding enabled, no VLANs. > > I need to add host MAC addresses via "arp -s <IP> <MAC>", otherwise, I > can''t reach them from guest domains. > > > -- > Tomasz Chmielewski > http://wpkg.org >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users