Hi, Apologies: I recall this being asked (and answered) before, but I cannot find the thread. I have two NICs. I would like to setup three bridges: xen-br0 - eth0 xen-br1 - eth1 xen-br2 - virtual only for domU interconnect Where would be a suitable location to implement this so the startup scripts pick this configuration? Furthermore, do MAC addresses need to be explicitly declared in the domU configuration files? The example files state that random MAC addresses will be used, but simply stating nics=2 does not seem to work. Thanks Gil _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
wow, i was just going to post about this, i can''t find it either ... i think it''s in xend-config.sxp: under (network-script network bridge=xen-br0 netdev=eth0) (network-script network bridge=xen-br1 netdev=eth3) but this isn''t actually setting up two bridges for me ... and i think you don''t need to declare specific MAC''s, as long as the bridge/vif is set up correctly. ( I am specifying MAC''s for DHCP reservation) any help for the both of us? maybe we''ll figure it out together andrew On Thu, 28 Apr 2005, Gil Freund wrote:> Hi, > > Apologies: I recall this being asked (and answered) before, but I > cannot find the thread. > > I have two NICs. I would like to setup three bridges: > > xen-br0 - eth0 > xen-br1 - eth1 > xen-br2 - virtual only for domU interconnect > > Where would be a suitable location to implement this so the startup > scripts pick this configuration? > > Furthermore, do MAC addresses need to be explicitly declared in the > domU configuration files? The example files state that random MAC > addresses will be used, but simply stating nics=2 does not seem to > work. > > Thanks > > > Gil > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 4/28/05, andrew mathes <amathes@backnine.org> wrote:> wow, i was just going to post about this, i can''t find it either ... > > i think it''s in xend-config.sxp: under > (network-script network bridge=xen-br0 netdev=eth0) > (network-script network bridge=xen-br1 netdev=eth3) > > but this isn''t actually setting up two bridges for me ...Nor here. I can set up other bridges manual (see below)> > and i think you don''t need to declare specific MAC''s, as long as the > bridge/vif is set up correctly. ( I am specifying MAC''s for DHCP > reservation) > > any help for the both of us? maybe we''ll figure it out togetherOK, I will start with manual settings: circus:/etc/xen/scripts# ./network start bridge=xen-br1 netdev=eth1 antispoof=yes network start bridge=xen-br1 netdev=eth1 antispoof=yes iptables: No chain/target/match by that name in the domU I can see two NIC, but DHCP does not work. DomU domains with static IPs work. Now I have a bridge (xen-br1) using the mac address of eth1, however, when I do ./network stop bridge=xen-br1 netdev=eth1 antispoof=yes xen-br1 us still up. Is this normal?> andrew > > On Thu, 28 Apr 2005, Gil Freund wrote: > > > Hi, > > > > Apologies: I recall this being asked (and answered) before, but I > > cannot find the thread. > > > > I have two NICs. I would like to setup three bridges: > > > > xen-br0 - eth0 > > xen-br1 - eth1 > > xen-br2 - virtual only for domU interconnect > > > > Where would be a suitable location to implement this so the startup > > scripts pick this configuration? > > > > Furthermore, do MAC addresses need to be explicitly declared in the > > domU configuration files? The example files state that random MAC > > addresses will be used, but simply stating nics=2 does not seem to > > work. > > > > Thanks > > > > > > Gil > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
ok, manually i get the same results as you, can''t figure out how to kill the new bridge without rebooting the box. also, it seems the new bridge doesn''t pick up an ip address for me either, or bcast, netmask ... seems like it should? xen-br0 has all that information... anyone have advice for us newbs? On Thu, 28 Apr 2005, Gil Freund wrote:> On 4/28/05, andrew mathes <amathes@backnine.org> wrote: >> wow, i was just going to post about this, i can''t find it either ... >> >> i think it''s in xend-config.sxp: under >> (network-script network bridge=xen-br0 netdev=eth0) >> (network-script network bridge=xen-br1 netdev=eth3) >> >> but this isn''t actually setting up two bridges for me ... > > Nor here. I can set up other bridges manual (see below) > >> >> and i think you don''t need to declare specific MAC''s, as long as the >> bridge/vif is set up correctly. ( I am specifying MAC''s for DHCP >> reservation) >> >> any help for the both of us? maybe we''ll figure it out together > > OK, I will start with manual settings: > > circus:/etc/xen/scripts# ./network start bridge=xen-br1 netdev=eth1 > antispoof=yes > network start bridge=xen-br1 netdev=eth1 antispoof=yes > iptables: No chain/target/match by that name > > in the domU I can see two NIC, but DHCP does not work. DomU domains > with static IPs work. > > Now I have a bridge (xen-br1) using the mac address of eth1, however, when I do > ./network stop bridge=xen-br1 netdev=eth1 antispoof=yes > xen-br1 us still up. Is this normal? > > >> andrew >> >> On Thu, 28 Apr 2005, Gil Freund wrote: >> >>> Hi, >>> >>> Apologies: I recall this being asked (and answered) before, but I >>> cannot find the thread. >>> >>> I have two NICs. I would like to setup three bridges: >>> >>> xen-br0 - eth0 >>> xen-br1 - eth1 >>> xen-br2 - virtual only for domU interconnect >>> >>> Where would be a suitable location to implement this so the startup >>> scripts pick this configuration? >>> >>> Furthermore, do MAC addresses need to be explicitly declared in the >>> domU configuration files? The example files state that random MAC >>> addresses will be used, but simply stating nics=2 does not seem to >>> work. >>> >>> Thanks >>> >>> >>> Gil >>> >>> _______________________________________________ >>> Xen-users mailing list >>> Xen-users@lists.xensource.com >>> http://lists.xensource.com/xen-users >>> >> > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
ok, things seem to be ok on my side now, i just put the manual start in my auto/ script directory, and seems to start the bridge correctly. However, the secondary vif for every VM seems to be tied to the first bridge, no matter how i bring it up (manually:./vif-bridge up domain=VM1 vif=vif1.1 bridge=xen-br1 mac=AA:00:00:00:00:12) the hwAddr of the resulting vif is ALWAYS the same as the first bridge (xen-br0) and i can''t seem to get it to tie to the second bridge ... making networking fairly useless on that second nic thanks for any help- On Thu, 28 Apr 2005, Gil Freund wrote:> On 4/28/05, andrew mathes <amathes@backnine.org> wrote: >> wow, i was just going to post about this, i can''t find it either ... >> >> i think it''s in xend-config.sxp: under >> (network-script network bridge=xen-br0 netdev=eth0) >> (network-script network bridge=xen-br1 netdev=eth3) >> >> but this isn''t actually setting up two bridges for me ... > > Nor here. I can set up other bridges manual (see below) > >> >> and i think you don''t need to declare specific MAC''s, as long as the >> bridge/vif is set up correctly. ( I am specifying MAC''s for DHCP >> reservation) >> >> any help for the both of us? maybe we''ll figure it out together > > OK, I will start with manual settings: > > circus:/etc/xen/scripts# ./network start bridge=xen-br1 netdev=eth1 > antispoof=yes > network start bridge=xen-br1 netdev=eth1 antispoof=yes > iptables: No chain/target/match by that name > > in the domU I can see two NIC, but DHCP does not work. DomU domains > with static IPs work. > > Now I have a bridge (xen-br1) using the mac address of eth1, however, when I do > ./network stop bridge=xen-br1 netdev=eth1 antispoof=yes > xen-br1 us still up. Is this normal? > > >> andrew >> >> On Thu, 28 Apr 2005, Gil Freund wrote: >> >>> Hi, >>> >>> Apologies: I recall this being asked (and answered) before, but I >>> cannot find the thread. >>> >>> I have two NICs. I would like to setup three bridges: >>> >>> xen-br0 - eth0 >>> xen-br1 - eth1 >>> xen-br2 - virtual only for domU interconnect >>> >>> Where would be a suitable location to implement this so the startup >>> scripts pick this configuration? >>> >>> Furthermore, do MAC addresses need to be explicitly declared in the >>> domU configuration files? The example files state that random MAC >>> addresses will be used, but simply stating nics=2 does not seem to >>> work. >>> >>> Thanks >>> >>> >>> Gil >>> >>> _______________________________________________ >>> Xen-users mailing list >>> Xen-users@lists.xensource.com >>> http://lists.xensource.com/xen-users >>> >> > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 4/29/05, andrew mathes <amathes@backnine.org> wrote:> ok, things seem to be ok on my side now, i just put the manual start in my > auto/ script directory, and seems to start the bridge correctly. However, > the secondary vif for every VM seems to be tied to the first bridge, no > matter how i bring it up (manually:./vif-bridge up domain=VM1 vif=vif1.1 > bridge=xen-br1 mac=AA:00:00:00:00:12)Questions: 1. Did you modify the distribution scripts or add a new one? 2. How did you configure the NIC''s in the DomU config file?> > the hwAddr of the resulting vif is ALWAYS the same as the first bridge > (xen-br0) and i can''t seem to get it to tie to the second bridge ... > making networking fairly useless on that second nic > > thanks for any help- > > On Thu, 28 Apr 2005, Gil Freund wrote: > > > On 4/28/05, andrew mathes <amathes@backnine.org> wrote: > >> wow, i was just going to post about this, i can''t find it either ... > >> > >> i think it''s in xend-config.sxp: under > >> (network-script network bridge=xen-br0 netdev=eth0) > >> (network-script network bridge=xen-br1 netdev=eth3) > >> > >> but this isn''t actually setting up two bridges for me ... > > > > Nor here. I can set up other bridges manual (see below) > > > >> > >> and i think you don''t need to declare specific MAC''s, as long as the > >> bridge/vif is set up correctly. ( I am specifying MAC''s for DHCP > >> reservation) > >> > >> any help for the both of us? maybe we''ll figure it out together > > > > OK, I will start with manual settings: > > > > circus:/etc/xen/scripts# ./network start bridge=xen-br1 netdev=eth1 > > antispoof=yes > > network start bridge=xen-br1 netdev=eth1 antispoof=yes > > iptables: No chain/target/match by that name > > > > in the domU I can see two NIC, but DHCP does not work. DomU domains > > with static IPs work. > > > > Now I have a bridge (xen-br1) using the mac address of eth1, however, when I do > > ./network stop bridge=xen-br1 netdev=eth1 antispoof=yes > > xen-br1 us still up. Is this normal? > > > > > >> andrew > >> > >> On Thu, 28 Apr 2005, Gil Freund wrote: > >> > >>> Hi, > >>> > >>> Apologies: I recall this being asked (and answered) before, but I > >>> cannot find the thread. > >>> > >>> I have two NICs. I would like to setup three bridges: > >>> > >>> xen-br0 - eth0 > >>> xen-br1 - eth1 > >>> xen-br2 - virtual only for domU interconnect > >>> > >>> Where would be a suitable location to implement this so the startup > >>> scripts pick this configuration? > >>> > >>> Furthermore, do MAC addresses need to be explicitly declared in the > >>> domU configuration files? The example files state that random MAC > >>> addresses will be used, but simply stating nics=2 does not seem to > >>> work. > >>> > >>> Thanks > >>> > >>> > >>> Gil > >>> > >>> _______________________________________________ > >>> Xen-users mailing list > >>> Xen-users@lists.xensource.com > >>> http://lists.xensource.com/xen-users > >>> > >> > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
i added a new script to auto:
import os
os.system(''./etc/xen/scripts/network start bridge=xen-br1
netdev=eth3'')
os.system(''xm create -f /etc/xen/xm.conf vmid=1'')
os.system(''xm create -f /etc/xen/xm.conf vmid=2'')
os.system(''xm create -f /etc/xen/xm.conf vmid=3'')
os.system(''xm create -f /etc/xen/xm.conf vmid=4'')
my vif in the xm.conf file is:
vif = [ ''mac=aa:00:00:1a:%d:11, bridge=xen-br0'' % (vmid),
''mac=aa:00:00:1a:%d:12, bridge=xen-br1'' % (vmid)]
(i''ve tried them as two seperate lines as well ... ie. vif=[ and vif=[)
are you up and going now?
On Fri, 29 Apr 2005, Gil Freund wrote:
> On 4/29/05, andrew mathes <amathes@backnine.org> wrote:
>> ok, things seem to be ok on my side now, i just put the manual start in
my
>> auto/ script directory, and seems to start the bridge correctly.
However,
>> the secondary vif for every VM seems to be tied to the first bridge, no
>> matter how i bring it up (manually:./vif-bridge up domain=VM1
vif=vif1.1
>> bridge=xen-br1 mac=AA:00:00:00:00:12)
>
> Questions:
> 1. Did you modify the distribution scripts or add a new one?
> 2. How did you configure the NIC''s in the DomU config file?
>
>>
>> the hwAddr of the resulting vif is ALWAYS the same as the first bridge
>> (xen-br0) and i can''t seem to get it to tie to the second
bridge ...
>> making networking fairly useless on that second nic
>>
>> thanks for any help-
>>
>> On Thu, 28 Apr 2005, Gil Freund wrote:
>>
>>> On 4/28/05, andrew mathes <amathes@backnine.org> wrote:
>>>> wow, i was just going to post about this, i can''t find
it either ...
>>>>
>>>> i think it''s in xend-config.sxp: under
>>>> (network-script network bridge=xen-br0 netdev=eth0)
>>>> (network-script network bridge=xen-br1 netdev=eth3)
>>>>
>>>> but this isn''t actually setting up two bridges for me
...
>>>
>>> Nor here. I can set up other bridges manual (see below)
>>>
>>>>
>>>> and i think you don''t need to declare specific
MAC''s, as long as the
>>>> bridge/vif is set up correctly. ( I am specifying
MAC''s for DHCP
>>>> reservation)
>>>>
>>>> any help for the both of us? maybe we''ll figure it out
together
>>>
>>> OK, I will start with manual settings:
>>>
>>> circus:/etc/xen/scripts# ./network start bridge=xen-br1 netdev=eth1
>>> antispoof=yes
>>> network start bridge=xen-br1 netdev=eth1 antispoof=yes
>>> iptables: No chain/target/match by that name
>>>
>>> in the domU I can see two NIC, but DHCP does not work. DomU domains
>>> with static IPs work.
>>>
>>> Now I have a bridge (xen-br1) using the mac address of eth1,
however, when I do
>>> ./network stop bridge=xen-br1 netdev=eth1 antispoof=yes
>>> xen-br1 us still up. Is this normal?
>>>
>>>
>>>> andrew
>>>>
>>>> On Thu, 28 Apr 2005, Gil Freund wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Apologies: I recall this being asked (and answered) before,
but I
>>>>> cannot find the thread.
>>>>>
>>>>> I have two NICs. I would like to setup three bridges:
>>>>>
>>>>> xen-br0 - eth0
>>>>> xen-br1 - eth1
>>>>> xen-br2 - virtual only for domU interconnect
>>>>>
>>>>> Where would be a suitable location to implement this so the
startup
>>>>> scripts pick this configuration?
>>>>>
>>>>> Furthermore, do MAC addresses need to be explicitly
declared in the
>>>>> domU configuration files? The example files state that
random MAC
>>>>> addresses will be used, but simply stating nics=2 does not
seem to
>>>>> work.
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>> Gil
>>>>>
>>>>> _______________________________________________
>>>>> Xen-users mailing list
>>>>> Xen-users@lists.xensource.com
>>>>> http://lists.xensource.com/xen-users
>>>>>
>>>>
>>>
>>> _______________________________________________
>>> Xen-users mailing list
>>> Xen-users@lists.xensource.com
>>> http://lists.xensource.com/xen-users
>>>
>>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
On 4/30/05, andrew mathes <amathes@backnine.org> wrote:> i added a new script to auto:OK, you are way ahead of me. I use separate conf files in /etc/xen/auto as those a re different machine. I think I am failing to comprehend the script / conf relation in this directory. What I am looking to achieve is the to bring the bridges up on xend start and have them available for whichever domain requires them.> > import os > > os.system(''./etc/xen/scripts/network start bridge=xen-br1 netdev=eth3'') > > os.system(''xm create -f /etc/xen/xm.conf vmid=1'') > os.system(''xm create -f /etc/xen/xm.conf vmid=2'') > os.system(''xm create -f /etc/xen/xm.conf vmid=3'') > os.system(''xm create -f /etc/xen/xm.conf vmid=4'') > > my vif in the xm.conf file is: > > vif = [ ''mac=aa:00:00:1a:%d:11, bridge=xen-br0'' % (vmid), > ''mac=aa:00:00:1a:%d:12, bridge=xen-br1'' % (vmid)] > > (i''ve tried them as two seperate lines as well ... ie. vif=[ and vif=[) > > are you up and going now? > > On Fri, 29 Apr 2005, Gil Freund wrote: > > > On 4/29/05, andrew mathes <amathes@backnine.org> wrote: > >> ok, things seem to be ok on my side now, i just put the manual start in my > >> auto/ script directory, and seems to start the bridge correctly. However, > >> the secondary vif for every VM seems to be tied to the first bridge, no > >> matter how i bring it up (manually:./vif-bridge up domain=VM1 vif=vif1.1 > >> bridge=xen-br1 mac=AA:00:00:00:00:12) > > > > Questions: > > 1. Did you modify the distribution scripts or add a new one? > > 2. How did you configure the NIC''s in the DomU config file? > > > >> > >> the hwAddr of the resulting vif is ALWAYS the same as the first bridge > >> (xen-br0) and i can''t seem to get it to tie to the second bridge ... > >> making networking fairly useless on that second nic > >> > >> thanks for any help- > >> > >> On Thu, 28 Apr 2005, Gil Freund wrote: > >> > >>> On 4/28/05, andrew mathes <amathes@backnine.org> wrote: > >>>> wow, i was just going to post about this, i can''t find it either ... > >>>> > >>>> i think it''s in xend-config.sxp: under > >>>> (network-script network bridge=xen-br0 netdev=eth0) > >>>> (network-script network bridge=xen-br1 netdev=eth3) > >>>> > >>>> but this isn''t actually setting up two bridges for me ... > >>> > >>> Nor here. I can set up other bridges manual (see below) > >>> > >>>> > >>>> and i think you don''t need to declare specific MAC''s, as long as the > >>>> bridge/vif is set up correctly. ( I am specifying MAC''s for DHCP > >>>> reservation) > >>>> > >>>> any help for the both of us? maybe we''ll figure it out together > >>> > >>> OK, I will start with manual settings: > >>> > >>> circus:/etc/xen/scripts# ./network start bridge=xen-br1 netdev=eth1 > >>> antispoof=yes > >>> network start bridge=xen-br1 netdev=eth1 antispoof=yes > >>> iptables: No chain/target/match by that name > >>> > >>> in the domU I can see two NIC, but DHCP does not work. DomU domains > >>> with static IPs work. > >>> > >>> Now I have a bridge (xen-br1) using the mac address of eth1, however, when I do > >>> ./network stop bridge=xen-br1 netdev=eth1 antispoof=yes > >>> xen-br1 us still up. Is this normal? > >>> > >>> > >>>> andrew > >>>> > >>>> On Thu, 28 Apr 2005, Gil Freund wrote: > >>>> > >>>>> Hi, > >>>>> > >>>>> Apologies: I recall this being asked (and answered) before, but I > >>>>> cannot find the thread. > >>>>> > >>>>> I have two NICs. I would like to setup three bridges: > >>>>> > >>>>> xen-br0 - eth0 > >>>>> xen-br1 - eth1 > >>>>> xen-br2 - virtual only for domU interconnect > >>>>> > >>>>> Where would be a suitable location to implement this so the startup > >>>>> scripts pick this configuration? > >>>>> > >>>>> Furthermore, do MAC addresses need to be explicitly declared in the > >>>>> domU configuration files? The example files state that random MAC > >>>>> addresses will be used, but simply stating nics=2 does not seem to > >>>>> work. > >>>>> > >>>>> Thanks > >>>>> > >>>>> > >>>>> Gil > >>>>> > >>>>> _______________________________________________ > >>>>> Xen-users mailing list > >>>>> Xen-users@lists.xensource.com > >>>>> http://lists.xensource.com/xen-users > >>>>> > >>>> > >>> > >>> _______________________________________________ > >>> Xen-users mailing list > >>> Xen-users@lists.xensource.com > >>> http://lists.xensource.com/xen-users > >>> > >> > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
i''m confused what the "mac" paramater on vif''s in the startup configuration file for a domain does. i have this in my configuration file: vif = [ ''mac=aa:00:00:1a:%d:11'' % (vmid), ''bridge=xen-br0''] vif = [ ''mac=aa:00:00:1a:%d:12'' % (vmid), ''bridge=xen-br1''] and while my REAL question is why this doesn''t bind the second vif to bridge1, my first concern is that what''s happening is the second line''s MAC is being given to eth0 on my VM, and eth1 is getting something fairly random looking. is this mac the mac of the bridge being binded to, the mac of the new vif in the domain? And if it''s the mac of the if in the domain, why would these two lines not assign those two macs to eth0/1 respectively? I''m sure someone out there has bound multiple ethernet cards to different interfaces ... seems like a very common server setup, any enlightenment from xen guru''s out there? mostly curious about the "vif" paramater in your domain startup configuration as well as if you used the default "vif-bridge" script. thanks a lot Andrew Mathes _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Andrew! andrew mathes wrote:> i''m confused what the "mac" paramater on vif''s in the startup > configuration file for a domain does. > > i have this in my configuration file: > vif = [ ''mac=aa:00:00:1a:%d:11'' % (vmid), ''bridge=xen-br0''] > vif = [ ''mac=aa:00:00:1a:%d:12'' % (vmid), ''bridge=xen-br1''] > > and while my REAL question is why this doesn''t bind the second vif to > bridge1,The first time I used xen, I thought all this would happen automagically. Butwhen I figured out (like you) that this is not true, so I started to configure everything outside of xen''s startup scripts in advance of creating domains: 1. Create all the bridge devices that are mentioned in xenU configuration files; since xen-br0 is already there, in your case only xen-br1 has to be created: brctl addbr xen-br1 brctl stp xen-br1 off brctl setfd xen-br1 0 ip link set xen-br1 up I disable spanning tree since normally it causes more problems than it solves and it unnecessarily delays the initial forwarding. When the domUs boot, their eth0, eth1 etc. are added to the bridges as specified by your xen config file and get the MAC addresses as specified there with the mac=... parameter. 2. If you want to do networking between xen0 and xenU, configure the bridge device with an IP address, i. e. ifconfig xen-br1 10.1.2.3 netmask 255.255.255.0 broadcast 10.2.3.255 the bridge xen-br1 has the MAC address fe:ff:ff:ff:ff:ff. This is ok and poses no problem, because this is a valid (local scope individual) MAC address. 3. If you want to do external networking, you have to add one of dom0''s physical networking devices to the bridge, i. e. brctl addif xen-br1 eth1 The bridge will get the MAC address of the physical NIC eth1. And as above, dom0 communicates via xen-br1, not via eth1, so leave eth1 unconfigured. You can also use VLAN devices instead of physical NICs here, they are equivalent to physical NICs in this context (at least as long /proc/sys/net/bridge/bridge-nf-filter-vlan-tagged is 1). 4. Since VLAN devs carry their own MAC address (by default the same as their physical base device), you could add a VLAN to a bridge to change the bridges MAC address, i.e. if you would like to give xen-br1 the MAC address 00:00:01:02:03, you could use ip link set eth0 up vconfig add eth0 815 ip link set addr eth0.815 00:00:00:01:02:03 brctl addif xen-br1 eth0.815 Of course you could also change the MAC of a physical NIC instead of using a VLAN device but I prefer using VLANs.> my first concern is that what''s happening is the second line''s > MAC is being given to eth0 on my VM, and eth1 is getting something > fairly random looking.My xenU eth''s MAC addresses are exactly what I specify in the configuration files. I have no idea where this random MAC stuff that you experience comes from.> is this mac the mac of the bridge being binded to, the mac of the new > vif in the domain?The MAC of xenU''s eth0, eth1 etc. The MAC of vifX.Y in dom0 always shows fe:ff:ff:ff:ff.> And if it''s the mac of the if in the domain, why > would these two lines not assign those two macs to eth0/1 respectively?That''s what they do at least for me. Although I don''t use %d etc., I don''t think that this makes any difference (I use scripts to generate all xen configuration files, virtual disks, startup configs inside those virtual disks, the bridges etc.).> I''m sure someone out there has bound multiple ethernet cards to > different interfaces ... seems like a very common server setup, any > enlightenment from xen guru''s out there?I''m not a guru, but maybe the above can help a bit.> mostly curious about the "vif" paramater in your domain startup > configuration as well as if you used the default "vif-bridge" script.Sorry, I''m doing everything outside the xen autoconfiguration magic, so no vif-bridge script. Mark. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sun, May 01, 2005 at 09:45:22PM -0700, andrew mathes wrote:> i have this in my configuration file: > vif = [ ''mac=aa:00:00:1a:%d:11'' % (vmid), ''bridge=xen-br0''] > vif = [ ''mac=aa:00:00:1a:%d:12'' % (vmid), ''bridge=xen-br1'']the configuration files are actually python scripts, so the 2nd line there is replacing the information from the first. This is what you actually need: vif = [ ''mac=aa:00:00:1a:%d:11, bridge=xen-br0'' % (vmid), ''mac=aa:00:00:1a:%d:12, bridge=xen-br1'' % (vmid)] J -- Jody Belka knew (at) pimb (dot) org _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
everyone has been very helpful so far, thank you all, hopefully i will become knowledgeable enough to help out others. However ... if I use the below, it DOES set up two vif''s with the correct addresses, however, they can''t reach/be reached by the internet. (Only eth3 (of dom0)->xen-br1 is external available, so it seems to me they aren''t binding to br1, but to br0. My reason for thinking this is 1) the no external availability and 2) in the dom0 ifconfig the vif.1 has the same MAC as br0, leading me to believe it''s bound to that MAC. Mark wrote: "The MAC of xenU''s eth0, eth1 etc. The MAC of vifX.Y in dom0 always shows fe:ff:ff:ff:ff." is this true? It seems vifX.1 (bound to br1) should have MAC of br1, vifX.0 bound to br0 should have MAC of br0. Maybe this is where I''m going wrong. Everything worked fine when i was using eth0/br0 as my external interface ... so i feel like the bridge binding is my only problem ... ugh, sorry I''m having so many problems ... On Mon, 2 May 2005, Jody Belka wrote:> On Sun, May 01, 2005 at 09:45:22PM -0700, andrew mathes wrote: >> i have this in my configuration file: >> vif = [ ''mac=aa:00:00:1a:%d:11'' % (vmid), ''bridge=xen-br0''] >> vif = [ ''mac=aa:00:00:1a:%d:12'' % (vmid), ''bridge=xen-br1''] > > the configuration files are actually python scripts, so the > 2nd line there is replacing the information from the first. This is > what you actually need: > > vif = [ ''mac=aa:00:00:1a:%d:11, bridge=xen-br0'' % (vmid), > ''mac=aa:00:00:1a:%d:12, bridge=xen-br1'' % (vmid)] > > > J > > -- > Jody Belka > knew (at) pimb (dot) org > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
ok, so, ... brctl show shows this:
xen-br0 8000.001143fd7101 no eth0
vif10.0
vif11.0
vif12.0
vif9.0
xen-br1 8000.001143fd7102 no eth3
vif10.1
vif11.1
vif12.1
vif9.1
so obviously the interfaces are bound to the correct bridges, i just
can''t
reach any of the interfaces through the second bridge (eth0 isn''t
plugged
in ...)
I know eth0 worked when it was plugged in ... so maybe there''s a
different
issue here? i know eth3 works, i ssh in to it and it can get out ...
don''t know where to go from here ...
any suggestions?
On Mon, 2 May 2005, andrew mathes wrote:
> everyone has been very helpful so far, thank you all, hopefully i will
become
> knowledgeable enough to help out others.
>
> However ... if I use the below, it DOES set up two vif''s with the
correct
> addresses, however, they can''t reach/be reached by the internet.
(Only eth3
> (of dom0)->xen-br1 is external available, so it seems to me they
aren''t
> binding to br1, but to br0.
>
> My reason for thinking this is 1) the no external availability and 2) in
the
> dom0 ifconfig the vif.1 has the same MAC as br0, leading me to believe
it''s
> bound to that MAC.
>
> Mark wrote:
> "The MAC of xenU''s eth0, eth1 etc. The MAC of vifX.Y in dom0
always shows
> fe:ff:ff:ff:ff."
>
> is this true? It seems vifX.1 (bound to br1) should have MAC of br1,
vifX.0
> bound to br0 should have MAC of br0. Maybe this is where I''m
going wrong.
>
> Everything worked fine when i was using eth0/br0 as my external interface
...
> so i feel like the bridge binding is my only problem ...
>
> ugh, sorry I''m having so many problems ...
>
> On Mon, 2 May 2005, Jody Belka wrote:
>
>> On Sun, May 01, 2005 at 09:45:22PM -0700, andrew mathes wrote:
>> > i have this in my configuration file:
>> > vif = [ ''mac=aa:00:00:1a:%d:11'' % (vmid),
''bridge=xen-br0'']
>> > vif = [ ''mac=aa:00:00:1a:%d:12'' % (vmid),
''bridge=xen-br1'']
>>
>> the configuration files are actually python scripts, so the
>> 2nd line there is replacing the information from the first. This is
>> what you actually need:
>>
>> vif = [ ''mac=aa:00:00:1a:%d:11, bridge=xen-br0'' %
(vmid),
>> ''mac=aa:00:00:1a:%d:12, bridge=xen-br1'' %
(vmid)]
>>
>>
>> J
>>
>> --
>> Jody Belka
>> knew (at) pimb (dot) org
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>
>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Hi Andrew! andrew mathes wrote:> ok, so, ... brctl show shows this: > > xen-br0 8000.001143fd7101 no eth0 > vif10.0 > vif11.0 > vif12.0 > vif9.0 > xen-br1 8000.001143fd7102 no eth3 > vif10.1 > vif11.1 > vif12.1 > vif9.1 > > so obviously the interfaces are bound to the correct bridges, i just > can''t reach any of the interfaces through the second bridge (eth0 isn''t > plugged in ...)This looks good. So since it worked for me, check if - all interfaces are up including (the unconfigured) eth3 in domain0 - eth1 of domain10 (domain11, domain12) and the external host connected to eth3 of domain0 are configured with the same IP subnet - make shure that spoof protection makes no problems, so better disable it by setting /proc/sys/net/conf/*/rp_filter to 0 - the bridge is in forwarding state? With dmesg you should see something like xen-br1: port 1(vif10.1) entering learning state xen-br1: topology change detected, propagating xen-br1: port 1(vif10.1) entering forwarding state and so on for the other vifs For debugging you should use something simpler than ssh. I always use "ping" together with "tcpdump". I. e. in domain0 use "tcpdump -eni eth3 icmp or arp") to check how far the address resolution and the ping packets go and if answers are coming in. Try this on all hosts. Sometimes a broadcast ping, i. e. "ping -b 10.11.12.255" if your network is 10.11.12.0/24, or "ping -I eth0 -b 255.255.255.255" if eth0 is on the subet you wish to check, is more robust in the presence of misconfigured IP addresses. Mark. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
hm, tcpdump from external -> eth3 of dom0 looks fine tcpdump from exteran -> eth1 of domU = no reply tcpdump from eth3 -> ext = fine tcpdump from eth3->domU ... the ping says it''s ok, but tcpdump shows nothing likewise domU->eth3 ... says it''s pinging, but tcpdump shows nothing so is what''s happening here is both sides pinging the bridge, but nothing is going through? i have this in dmesg: device vif1.0 entered promiscuous mode xen-br0: port 1(vif1.0) entering learning state xen-br0: topology change detected, propagating xen-br0: port 1(vif1.0) entering forwarding state device vif1.1 entered promiscuous mode xen-br1: port 2(vif1.1) entering learning state xen-br1: topology change detected, propagating xen-br1: port 2(vif1.1) entering forwarding state so that looks good ... i''m not supposed to make the domU gateway the same as the domO nic or anything am i? perhaps that dump information will clarify the problem ... i hope =) On Mon, 2 May 2005, Mark Doll wrote:> Hi Andrew! > > andrew mathes wrote: >> ok, so, ... brctl show shows this: >> >> xen-br0 8000.001143fd7101 no eth0 >> vif10.0 >> vif11.0 >> vif12.0 >> vif9.0 >> xen-br1 8000.001143fd7102 no eth3 >> vif10.1 >> vif11.1 >> vif12.1 >> vif9.1 >> >> so obviously the interfaces are bound to the correct bridges, i just >> can''t reach any of the interfaces through the second bridge (eth0 isn''t >> plugged in ...) > > This looks good. So since it worked for me, check if > > - all interfaces are up including (the unconfigured) eth3 in domain0 > - eth1 of domain10 (domain11, domain12) and the external host connected > to eth3 of domain0 are configured with the same IP subnet > - make shure that spoof protection makes no problems, so better disable > it by setting /proc/sys/net/conf/*/rp_filter to 0 > - the bridge is in forwarding state? With dmesg you should see something > like > > xen-br1: port 1(vif10.1) entering learning state > xen-br1: topology change detected, propagating > xen-br1: port 1(vif10.1) entering forwarding state > > and so on for the other vifs > > For debugging you should use something simpler than ssh. I always use > "ping" together with "tcpdump". I. e. in domain0 use "tcpdump -eni eth3 > icmp or arp") to check how far the address resolution and the ping > packets go and if answers are coming in. Try this on all hosts. > Sometimes a broadcast ping, i. e. "ping -b 10.11.12.255" if your network > is 10.11.12.0/24, or "ping -I eth0 -b 255.255.255.255" if eth0 is on the > subet you wish to check, is more robust in the presence of misconfigured > IP addresses. > > Mark. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
andrew mathes wrote:> hm, tcpdump from external -> eth3 of dom0 looks fine > > tcpdump from exteran -> eth1 of domU = no reply > > tcpdump from eth3 -> ext = fine > > tcpdump from eth3->domU ... the ping says it''s ok, but tcpdump shows > nothing > > likewise domU->eth3 ... says it''s pinging, but tcpdump shows nothing > > so is what''s happening here is both sides pinging the bridge, but > nothing is going through? i have this in dmesg: > > device vif1.0 entered promiscuous mode > xen-br0: port 1(vif1.0) entering learning state > xen-br0: topology change detected, propagating > xen-br0: port 1(vif1.0) entering forwarding state > device vif1.1 entered promiscuous mode > xen-br1: port 2(vif1.1) entering learning state > xen-br1: topology change detected, propagating > xen-br1: port 2(vif1.1) entering forwarding state > > so that looks good ... > > i''m not supposed to make the domU gateway the same as the domO nic or > anything am i? >Sorry to come so late to this thread - but did you share your routes (ip route show, or netstat -rn) output, also your arp -a, ifconfig -a? If you have two real interfaces on the same subnet, you may need to set arp_filter on. thanks, Nivedita _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Andrew! andrew mathes wrote:> hm, tcpdump from external -> eth3 of dom0 looks fine > > tcpdump from exteran -> eth1 of domU = no reply > > tcpdump from eth3 -> ext = fine > > tcpdump from eth3->domU ... the ping says it''s ok, but tcpdump shows > nothing > > likewise domU->eth3 ... says it''s pinging, but tcpdump shows nothing >Run tcpump on xen-br1 instead of eth3 and you should see the ICMP packets.> so is what''s happening here is both sides pinging the bridge, but > nothing is going through? i have this in dmesg: > > device vif1.0 entered promiscuous mode > xen-br0: port 1(vif1.0) entering learning state > xen-br0: topology change detected, propagating > xen-br0: port 1(vif1.0) entering forwarding state > device vif1.1 entered promiscuous mode > xen-br1: port 2(vif1.1) entering learning state > xen-br1: topology change detected, propagating > xen-br1: port 2(vif1.1) entering forwarding state > > so that looks good ...ACK.> i''m not supposed to make the domU gateway the same as the domO nic or > anything am i?dom0 isn''t a gateway (forwarding is disabled). Bridging runs entirely on layer 2 (Ethernet), not layer 3 (IP).> > perhaps that dump information will clarify the problem ... > > i hope =)Not really. I have ne idea where the mistake in your config is. :-( Mark. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
all i can think of now is it''s a routing issue- here is my route -n on dom0: 10.255.0.0 10.254.1.254 255.255.255.0 UG 0 0 0 eth0 10.254.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 10.1.1.0 0.0.0.0 255.255.255.0 U 0 0 0 xen-br1 10.254.0.0 10.254.1.254 255.255.0.0 UG 0 0 0 eth0 0.0.0.0 10.1.1.254 0.0.0.0 UG 0 0 0 xen-br1 and from domU: Destination Gateway Genmask Flags Metric Ref Use Iface 10.255.0.129 10.254.1.254 255.255.255.255 UGH 0 0 0 eth0 10.255.0.0 10.254.1.254 255.255.255.0 UG 0 0 0 eth0 10.254.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 10.1.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1 10.254.0.0 10.254.1.254 255.255.0.0 UG 0 0 0 eth0 0.0.0.0 10.1.1.254 0.0.0.0 UG 0 0 0 eth1 so, it looks to me like both things are routing correctly ... but I can''t get anywhere through the 10.1.1. subnet ... (eth1/xen-br1 (tied to dom0 eth3 which is 10.1.1.53)) arp -a (dom0): sudo arp -a ? (10.1.1.243) at 00:11:43:FD:71:8C [ether] on xen-br1 ? (10.254.1.254) at <incomplete> on eth0 no arp -a output on domU ifconfig output looks good, as well as brctl, everything looks bound correctly, although i''m still confused why vifX.1 while bound to xen-br1 doesn''t get it''s mac address ... seems odd to me well, that''s all the output i can do, and if there''s nothing wrong in these routes ... i''m stumped, one network card for me i guess =/ On Mon, 2 May 2005, Nivedita Singhvi wrote:> andrew mathes wrote: > >> hm, tcpdump from external -> eth3 of dom0 looks fine >> >> tcpdump from exteran -> eth1 of domU = no reply >> >> tcpdump from eth3 -> ext = fine >> >> tcpdump from eth3->domU ... the ping says it''s ok, but tcpdump shows >> nothing >> >> likewise domU->eth3 ... says it''s pinging, but tcpdump shows nothing >> >> so is what''s happening here is both sides pinging the bridge, but nothing >> is going through? i have this in dmesg: >> >> device vif1.0 entered promiscuous mode >> xen-br0: port 1(vif1.0) entering learning state >> xen-br0: topology change detected, propagating >> xen-br0: port 1(vif1.0) entering forwarding state >> device vif1.1 entered promiscuous mode >> xen-br1: port 2(vif1.1) entering learning state >> xen-br1: topology change detected, propagating >> xen-br1: port 2(vif1.1) entering forwarding state >> >> so that looks good ... >> >> i''m not supposed to make the domU gateway the same as the domO nic or >> anything am i? >> > > Sorry to come so late to this thread - but did you > share your routes (ip route show, or netstat -rn) output, > also your arp -a, ifconfig -a? > > If you have two real interfaces on the same subnet, > you may need to set arp_filter on. > > thanks, > Nivedita > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
andrew mathes schrieb:> all i can think of now is it''s a routing issue- here is my route -n on > dom0: > > 10.1.1.0 0.0.0.0 255.255.255.0 U 0 0 0 > xen-br1 > 0.0.0.0 10.1.1.254 0.0.0.0 UG 0 0 0 > xen-br1 > > and from domU: > > Destination Gateway Genmask Flags Metric Ref Use > Iface > 10.1.1.0 0.0.0.0 255.255.255.0 U 0 0 0 > eth1 > 0.0.0.0 10.1.1.254 0.0.0.0 UG 0 0 0 > eth1OK, network 10.1.1.0/24 with router 10.1.1.254 via eth3 (xen-br3). Both dom0 and domU are simple hosts and don''t route. Fine.> ifconfig output looks good, as well as brctl, everything looks bound > correctly, although i''m still confused why vifX.1 while bound to xen-br1 > doesn''t get it''s mac address ... seems odd to meMy vifX.Y always have fe:ff:ff:ff:ff:ff. Do you mean, while vifX.1 does not get the configured MAC your vifX.0 does?> well, that''s all the output i can do, and if there''s nothing wrong in > these routes ... i''m stumped, one network card for me i guess =/One last idea. I don''t remember: Have you tried xen-br0 on eth3 and it worked? If not, are your eth0 and eth3 NICs different hardware models/use different device drivers? Maybe it''s a NIC driver or hardware problem. Maybe setting the NIC in promicious mode might help "ip l s eth3 promisc on" or "ifconfig eth3 promisc". Maybe the driver is incompatible with the bridging code. Mark. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Might be worth checking all the MAC addresses (the VIFS at both "ends" (the dom0 vif devices and the domU eth devices). Might as well check the "eth" devices in dom0. Can you post those MAC addresses, plus your domain configs where they are set. If you get clashes in the MAC addresses, all sorts of weird behaviour can happen! Cheers, Mark On Wednesday 04 May 2005 09:49, Mark Doll wrote:> andrew mathes schrieb: > > all i can think of now is it''s a routing issue- here is my route -n on > > dom0: > > > > 10.1.1.0 0.0.0.0 255.255.255.0 U 0 0 0 > > xen-br1 > > 0.0.0.0 10.1.1.254 0.0.0.0 UG 0 0 0 > > xen-br1 > > > > and from domU: > > > > Destination Gateway Genmask Flags Metric Ref Use > > Iface > > 10.1.1.0 0.0.0.0 255.255.255.0 U 0 0 0 > > eth1 > > 0.0.0.0 10.1.1.254 0.0.0.0 UG 0 0 0 > > eth1 > > OK, network 10.1.1.0/24 with router 10.1.1.254 via eth3 (xen-br3). Both > dom0 and domU are simple hosts and don''t route. Fine. > > > ifconfig output looks good, as well as brctl, everything looks bound > > correctly, although i''m still confused why vifX.1 while bound to xen-br1 > > doesn''t get it''s mac address ... seems odd to me > > My vifX.Y always have fe:ff:ff:ff:ff:ff. Do you mean, while vifX.1 does > not get the configured MAC your vifX.0 does? > > > well, that''s all the output i can do, and if there''s nothing wrong in > > these routes ... i''m stumped, one network card for me i guess =/ > > One last idea. I don''t remember: Have you tried xen-br0 on eth3 and it > worked? If not, are your eth0 and eth3 NICs different hardware > models/use different device drivers? Maybe it''s a NIC driver or hardware > problem. Maybe setting the NIC in promicious mode might help "ip l s > eth3 promisc on" or "ifconfig eth3 promisc". Maybe the driver is > incompatible with the bridging code. > > Mark. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users