I have the following networking setup: - eth0 and eth1 corresponding to the physical ports on the host NIC - bond0 which enslaves eth0 + eth1 - br0 which attaches bond0 - xenbr0 - xenbr5 all of which are internal only networks (10.0.0.0; 10.1.1.0; etc) Br0 is used both for the host to access the outside world as well as DomUs acting as gateway machines for each xenbr network. For example the DomU named Gateway0 will have eth0 attached to br0 giving it connectivity to the Internet. Eth1 on Gateway0 will connect to an internal only-network, xenbr0, upon which will reside a bunch of other DomUs. The latter communicate with the outside world according to the NAT firewall rules running on Gateway0. Gateway1 is attached to both br0 (outside world) and xenbr1 (private network of DomUs) and so on. Basically then, any DomUs which have a network interface possessing an external IP will be on br0 and xenbr(x), all other DomUs will be on xenbr(x) only. All of this was created on Fedora 17 with xen4.1. I recently conducted a new OS install, upgrading to Fedora 19 and xen4.2. I restored the VMs and everything was working just fine as before. The problem I have is as follows: if I create a new VM which is attached to br0 it won''t connect with the outside world. It will show eth0 as being up, as having an external IP address and, it''s able to ping the physical gateway for the subnet upon which the host sits but that''s as far as it can go. I''ve tried guest creation using virt-install as well as simply putting together a .cfg file and pointing to a xen kernel. I use either Debian 7.1 or Centos6.4 for my DomUs. If I create a DomU attached to any of the internal-only xenbr(x) networks, each of which is moderated by a gateway, it''s able to communicate with the outside world via NAT. Also, note, the 5 existing ''gateway'' DomUs, all of which are attached to br0, can still communicate directly with the Internet via their external facing network interfaces. Given I''ve moved from 4.1 (libxm) to 4.2 (libxl), I''m wondering is there something about the guest creation process under libxl which changes how the DoumU interfaces are created? I''ve tried pinging / checking network conf files / recreating guests with new MAC addresses etc. but I can''t seem to get any clue as to what''s happening. Could someone perhaps point me in a diagnostically productive direction?
Alexandre Kouznetsov
2013-Oct-04 19:51 UTC
Re: networking issues - internet facing guest interfaces
El 04/10/13 14:33, Adam Gold escribió:> I have the following networking setup: > - eth0 and eth1 corresponding to the physical ports on the host NIC > - bond0 which enslaves eth0 + eth1 > - br0 which attaches bond0 > - xenbr0 - xenbr5 all of which are internal only networks (10.0.0.0; > 10.1.1.0; etc) > > Br0 is used both for the host to access the outside world as well as DomUs > acting as gateway machines for each xenbr network. For example the DomU > named Gateway0 will have eth0 attached to br0 giving it connectivity to the > Internet. Eth1 on Gateway0 will connect to an internal only-network, > xenbr0, upon which will reside a bunch of other DomUs. The latter > communicate with the outside world according to the NAT firewall rules > running on Gateway0. Gateway1 is attached to both br0 (outside world) and > xenbr1 (private network of DomUs) and so on. Basically then, any DomUs > which have a network interface possessing an external IP will be on br0 and > xenbr(x), all other DomUs will be on xenbr(x) only.Seems to me like a nice and clean setup.> The problem I have is as follows: if I create a new VM which is attached to > br0 it won''t connect with the outside world. It will show eth0 as being up, > as having an external IP address and, it''s able to ping the physical gateway > for the subnet upon which the host sits but that''s as far as it can go. > I''ve tried guest creation using virt-install as well as simply putting > together a .cfg file and pointing to a xen kernel. I use either Debian 7.1 > or Centos6.4 for my DomUs. If I create a DomU attached to any of the > internal-only xenbr(x) networks, each of which is moderated by a gateway, > it''s able to communicate with the outside world via NAT. > > Also, note, the 5 existing ''gateway'' DomUs, all of which are attached to > br0, can still communicate directly with the Internet via their external > facing network interfaces. Given I''ve moved from 4.1 (libxm) to 4.2 > (libxl), I''m wondering is there something about the guest creation process > under libxl which changes how the DoumU interfaces are created? > > I''ve tried pinging / checking network conf files / recreating guests with > new MAC addresses etc. but I can''t seem to get any clue as to what''s > happening. Could someone perhaps point me in a diagnostically productive > direction?Double check your configuration: - Xen''s config files for DomUs, the compare the "vif" directives (sometimes a character by character check is needed to find a typo). - Correct mapping of vifX.Y within ethernet bridges (brctl show). Other things worth checking are: - Monitor your br0 traffic on dom0 with tcpdump, compare a outgoing connection attempt coming form a working domU and the one coming form a failing one. Consider using -e switch to see MAC addresses. Make sure the packets are still sent out, see if there is any problem receiving coming back packets. The problem might be outside of your Dom0 box. - Check your Internet gateway/firewall configuration, or a L3 switch if you use one at the other end of the bond. - Make sure you are using the right IP configuration (IP, netmask, gateway). Try using a know working IP on a faulty DomU (unassigning it temporary form the previous host). Hope it helps. -- Alexandre Kouznetsov