Hello, xen-users!
I am fiddling with the Xen 2.0 and Xen 3.0 configuration in order to do
a combined installation of a firewall system and a webserver on the same
machine. Now I use a recent Xen 3.0 on top (or below, depending on how
you think of it) a Debian Sarge 3.1r1. The idea is to have the
following:
- Dom0 connected with a single external IP routing everything to Dom1
- Dom1 with a firewall system and two virtual network cards
- Dom2 the webserver behind the firewall with a single virtual network
card
I tried to use direct access but abandoned the idea because the system
is colocated and has only one IP address. This brings me to my problem.
I read the various networking threads a couple of times (including the
ideal(istic) firewall thread). Somehow I cannot completely wrap my mind
around Xen''s networking concepts. I think I can work with two bridges
and internal local networks where the Dom0 will do SNAT for outbound
packets. The setup looks a bit like this (work in progress, just a quick
sketch):
http://web.luchs.at/information/media/xen_setup.png
After starting the two domains and manually setting up the second bridge
I get something like this on a test system:
samuel:~# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 64 1 r----- 143.2
astaro 1 120 1 ------ 2838.8
webserver 2 48 1 -b---- 34.5
samuel:~# brctl show
bridge name bridge id STP enabled interfaces
xenbr0 8000.feffffffffff no peth0
vif0.0
vif1.0
xenbr1 8000.b67150095f2d no dummy0
vif1.1
vif2.0
samuel:~#
vif1.0 and vif1.1 belong to the firewall, vif2.0 belongs to the
webserver. I gathered from the threads that the interface names change
when I restart the domains. What is the best practice to pin down
interface names?
How can I create xenbr1 automatically after Dom0 comes up? In the above
listing dummy0 is out of place because right after the boot process,
there are no domains running and therefore xenbr1 cannot be created with
the vif interfaces. Do I need a dummy interface for every bridge that is
used to connect domains?
Another thing I noticed is that I have a lot of network devices that are
unused. The system has 27 net devices, vif is numbered up to vif0.7 and
I have veth devices up to veth7. Why is that? Xen 2.0 didn''t create so
many devices. What are they used for? Is this a kind of device pool?
I hope my questions are not redundant. I spent days wading through the
docs and the mailing list archives, I may have missed something due to
growing confusion.
Best regards,
Lynx.
--
"From the delicate strands,
between minds we weave our mesh:
a blanket to warm the soul."
--- Lady Deirdre Skye (SMAC) ---
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
On 12/20/05, René Pfeiffer <lynx@pentex.at> wrote:> Hello, xen-users! > > I am fiddling with the Xen 2.0 and Xen 3.0 configuration in order to do > a combined installation of a firewall system and a webserver on the same > machine. Now I use a recent Xen 3.0 on top (or below, depending on how > you think of it) a Debian Sarge 3.1r1. The idea is to have the > following: > > - Dom0 connected with a single external IP routing everything to Dom1 > - Dom1 with a firewall system and two virtual network cards > - Dom2 the webserver behind the firewall with a single virtual network > card > > I tried to use direct access but abandoned the idea because the system > is colocated and has only one IP address. This brings me to my problem. > I read the various networking threads a couple of times (including the > ideal(istic) firewall thread). Somehow I cannot completely wrap my mind > around Xen''s networking concepts. I think I can work with two bridges > and internal local networks where the Dom0 will do SNAT for outbound > packets. The setup looks a bit like this (work in progress, just a quick > sketch): > > http://web.luchs.at/information/media/xen_setup.png > > After starting the two domains and manually setting up the second bridge > I get something like this on a test system: > > samuel:~# xm list > Name ID Mem(MiB) VCPUs State Time(s) > Domain-0 0 64 1 r----- 143.2 > astaro 1 120 1 ------ 2838.8 > webserver 2 48 1 -b---- 34.5 > samuel:~# brctl show > bridge name bridge id STP enabled interfaces > xenbr0 8000.feffffffffff no peth0 > vif0.0 > vif1.0 > xenbr1 8000.b67150095f2d no dummy0 > vif1.1 > vif2.0 > samuel:~# > > vif1.0 and vif1.1 belong to the firewall, vif2.0 belongs to the > webserver. I gathered from the threads that the interface names change > when I restart the domains. What is the best practice to pin down > interface names? > > How can I create xenbr1 automatically after Dom0 comes up? In the above > listing dummy0 is out of place because right after the boot process, > there are no domains running and therefore xenbr1 cannot be created with > the vif interfaces. Do I need a dummy interface for every bridge that is > used to connect domains? > > Another thing I noticed is that I have a lot of network devices that are > unused. The system has 27 net devices, vif is numbered up to vif0.7 and > I have veth devices up to veth7. Why is that? Xen 2.0 didn''t create so > many devices. What are they used for? Is this a kind of device pool? > > Best regards, > Lynx.Hi Lynx, With Xen 3.0 you cannot pin down vif names. Besides, there is no need for that if you are using the scripts that come with Xen to start the network and bind the vifs to the bridges. If I may try to help you, there are some scripts and considerations. In dom0, create two bridges. I will call them switches, because that is what they really are like. ----------begin of script---------- #!/bin/bash # creating switch 1 brctl addbr sw1 brctl stp sw1 off brctl setfd sw1 0 # creating switch 2 brctl addbr sw2 brctl stp sw2 off brctl setfd sw2 0 # creating tun/tap device tunctl -t tap0 # adding tun/tap device to bridge brctl addif sw2 tap0 # configuring tap0 device ip address add 192.168.2.2/30 dev tap0 ----------end of script---------- The above script should be exec in every boot, so you should put it into rc.local or other boot script. Also, you must install uml-utilities, where you can have the tunctl executable. Next, you program the binding to the dom interfaces. Files from /etc/xen: /etc/xen/dom1: nics=3 vif = [ ''mac=aa:00:00:01:01:00,bridge=xenbr0'', ''mac=aa:00:00:01:01:01,bridge=sw1'', ''mac=aa:00:00:01:01:02,bridge=sw2'' ] /etc/xen/dom2: nics=1 vif = [ ''mac=aa:00:00:01:02:01,bridge=sw1'' ] Now boot both domains and configure their interfaces (for RH based distros, files are in /etc/sysconfig/network-scripts, and are called ifcfg-ethX, where X is the interface number - see what is usage for you distro): dom1/eth0: your real IP address to internet/cidr dom1/eth1: 192.168.1.1/30 dom1/eth2: 192.168.2.1/30 dom2/eth0: 192.168.1.2/30 After that, you will see that dom0 does not have any real interface with an IP address. That is the deal. Look at the attached image, that is what you have after the configuration above. You will see it is exactly what you would have if you use real machines :) So, the base machine is behind the firewall, you connect to the firewall and from there you connect to the base machine. Just like a firewall with DMZ and LAN :) About dummy interfaces, I have never used them. Nor with xen2, neither with xen3. And about creating bridges, the Xen network script creates just one bridge, xenbr0. It is not prepared for creating other bridges, that is why I use a script to create all bridges/switches but xenbr0. Hope that can help you. -- Bye, Fernando Maior LPIC/1(31908) LinuxCounter(391325) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hello! After some sleep and not poking around in the configs some questions were answered. On Dec 20, 2005 at 0928 -0200, Fernando Maior appeared and said:> > With Xen 3.0 you cannot pin down vif names. Besides, there is no need for > that if you are using the scripts that come with Xen to start the network > and bind the vifs to the bridges.That''s something I understood yesterday. The vifs are bound to the bridges given in the domain''s config file. Also my kernel wasn''t ready to handle multiple dummy devices (the dummy driver has to be a module to have this feature). In addition to that I thought that I can use the Xen network scripts for the creation of all devices necessary, but now I believe it is better to use extra scripts to handle everything besides Xen''s xenbr0 and its virtual interfaces.> If I may try to help you, there are some scripts and considerations. > > In dom0, create two bridges. I will call them switches, because that is > what they really are like. [...]I will give this a shot and see how it works. Many thanks! Best, Lynx. -- "From the delicate strands, between minds we weave our mesh: a blanket to warm the soul." --- Lady Deirdre Skye (SMAC) --- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Tue, 2005-12-20 at 13:46 +0100, René Pfeiffer wrote:> Hello! > > After some sleep and not poking around in the configs some questions > were answered. > > On Dec 20, 2005 at 0928 -0200, Fernando Maior appeared and said: > > > > With Xen 3.0 you cannot pin down vif names. Besides, there is no need for > > that if you are using the scripts that come with Xen to start the network > > and bind the vifs to the bridges. > > That''s something I understood yesterday. The vifs are bound to the > bridges given in the domain''s config file. Also my kernel wasn''t ready > to handle multiple dummy devices (the dummy driver has to be a module to > have this feature). In addition to that I thought that I can use the Xen > network scripts for the creation of all devices necessary, but now I > believe it is better to use extra scripts to handle everything besides > Xen''s xenbr0 and its virtual interfaces. > > > If I may try to help you, there are some scripts and considerations. > > > > In dom0, create two bridges. I will call them switches, because that is > > what they really are like. [...] > > I will give this a shot and see how it works. Many thanks! > > Best, > Lynx. >Fernando made a really important point that I hope didn''t slip by. Your original e-mail described binding an external IP address to Dom0. I would recommend never doing such a thing. If someone compromises dom0, they have everything. We use Xen virtualized VPN gateways in the open source ISCS network security management project (http://iscs.sourceforge.net) with a slightly simply set up than Fernando''s. But the idea is still the same, We heavily shield dom0 with no IP addresses bound to the public interface and pass all external traffic through the firewall as you proposed. By the way, we also prefer to have a separate VPN gateway through which we administer the Xen server with its various VPN gateways for our clients. This way, if there is a problem in the domU that can be addresses from the dom0, we have not cut ourselves off from remote administration. Hope this helps - John -- John A. Sullivan III Open Source Development Corporation +1 207-985-7880 jsullivan@opensourcedevel.com Financially sustainable open source development http://www.opensourcedevel.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Dec 20, 2005 at 1130 -0500, John A. Sullivan III appeared and said:> > Fernando made a really important point that I hope didn''t slip by. Your > original e-mail described binding an external IP address to Dom0. I > would recommend never doing such a thing. If someone compromises dom0, > they have everything.Yes, I didn''t miss that point.> [...] > We heavily shield dom0 with no IP addresses bound to the public > interface and pass all external traffic through the firewall as you > proposed.That''s what I have in mind. The problem with the setup is the fact that the server is "heavily colocated", so we probably have to assign Dom0 an external IP address for system administration. I proposed to my colleagues to use a second IP address for the firewall and make the access to Dom0 VPN-only in addition to limiting packets from selected networks only. Thanks for your insights! Best, Lynx. -- "From the delicate strands, between minds we weave our mesh: a blanket to warm the soul." --- Lady Deirdre Skye (SMAC) --- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Tue, 2005-12-20 at 23:32 +0100, René Pfeiffer wrote:> On Dec 20, 2005 at 1130 -0500, John A. Sullivan III appeared and said: > > > > Fernando made a really important point that I hope didn''t slip by. Your > > original e-mail described binding an external IP address to Dom0. I > > would recommend never doing such a thing. If someone compromises dom0, > > they have everything. > > Yes, I didn''t miss that point. > > > [...] > > We heavily shield dom0 with no IP addresses bound to the public > > interface and pass all external traffic through the firewall as you > > proposed. > > That''s what I have in mind. The problem with the setup is the fact that > the server is "heavily colocated", so we probably have to assign Dom0 an > external IP address for system administration. I proposed to my > colleagues to use a second IP address for the firewall and make the > access to Dom0 VPN-only in addition to limiting packets from selected > networks only. > > Thanks for your insights! > > Best, > Lynx. >Why do you need a second IP address (unless I missed something). To eliminate the need to publicly expose the dom0 even in colocation scenarios, we typically assign dom0 a private address only and access it via VPN. Thus one only needs a public IP address for the VPN gateway - John -- John A. Sullivan III Open Source Development Corporation +1 207-985-7880 jsullivan@opensourcedevel.com Financially sustainable open source development http://www.opensourcedevel.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Dec 20, 2005 at 1806 -0500, John A. Sullivan III appeared and said:> [...] > Why do you need a second IP address (unless I missed something).The second IP address is not necessary. I just thought about having alternative access to the server by using Dom0 or one of the DomUs.> To eliminate the need to publicly expose the dom0 even in colocation > scenarios, we typically assign dom0 a private address only and access > it via VPN. Thus one only needs a public IP address for the VPN > gateway -This means that in your setting the access happens through a VPN gateway running in one of the DomUs, right? Best, Lynx. -- "From the delicate strands, between minds we weave our mesh: a blanket to warm the soul." --- Lady Deirdre Skye (SMAC) --- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Wed, 2005-12-21 at 10:06 +0100, René Pfeiffer wrote:> On Dec 20, 2005 at 1806 -0500, John A. Sullivan III appeared and said: > > [...] > > Why do you need a second IP address (unless I missed something). > > The second IP address is not necessary. I just thought about having > alternative access to the server by using Dom0 or one of the DomUs. > > > To eliminate the need to publicly expose the dom0 even in colocation > > scenarios, we typically assign dom0 a private address only and access > > it via VPN. Thus one only needs a public IP address for the VPN > > gateway - > > This means that in your setting the access happens through a VPN gateway > running in one of the DomUs, right?Yes, if I understand you correctly. When two IPs are available, we prefer to have a separate physical gateway but that does not appear to be an option for you so, yes, a little bit of the tail wagging the dog! Hope this helps - John> > Best, > Lynx. >-- John A. Sullivan III Open Source Development Corporation +1 207-985-7880 jsullivan@opensourcedevel.com Financially sustainable open source development http://www.opensourcedevel.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users