Richard Ta-Min
2004-Jul-17 00:08 UTC
[Xen-devel] Communication between Domain0 and Domain1
Hello,
I am using ''xen unstable / xenolinux 2.4.26'' and I am having
some problems
to get the domain0 and domain1 to communicate with each other. The python
scripts xc_dom_create.py is not available on the xen unstable release. If
I try to run the copy xc_dom_create.py that I have from xen 1.2 I get an
error on the line "xc = Xc.new()". So this is what I did to start a
new
domain1 on xen unstable. From within domain0 I ran the following:
1.) run ''xend start''
2.) run the script xen_nat_enable that I got from xen1.2
3.) I ran the following command from within a bash script
xm create -c vmid=1 \
name=domain1 \
kernel=/boot/vmlinuz-2.4.26-xenU \
memory=64 \
disk=phy:hda9,hda9,w \
ipaddr=169.254.1.1 \
root=/dev/hda9 ro \
ip=169.254.1.1 \
gateway=169.254.1.0 \
netmask=255.255.0.0 \
hostname=host_dom1
Domain1 is loaded and I can successfully login to domain1
through the console. Note: leaving ''mingetty tty1'' in the
inittab file of domain1 is fine. I read somewhere that I should
disable that.
However, I cannot ssh into domain1 from domain0.
==========================================================[root@domain0 /]# ssh
169.254.1.1
ssh: connect to host 169.254.1.1 port 22: No route to host
==========================================================
Below I have copied & paste the network configuration for domain0 and
domain1. I am sorry if I include a lot of junk in the email.
Here is the network configuration of domain0
==================================================================[root@domain0
/]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0E:A6:6B:70:CC
inet addr:128.100.241.161 Bcast:128.100.241.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4228 errors:0 dropped:0 overruns:0 frame:0
TX packets:400 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:493895 (482.3 Kb) TX bytes:60050 (58.6 Kb)
Interrupt:22 Memory:feafc000-0
eth0:xen Link encap:Ethernet HWaddr 00:0E:A6:6B:70:CC
inet addr:169.254.1.0 Bcast:169.254.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:22 Memory:feafc000-0
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:19836 errors:0 dropped:0 overruns:0 frame:0
TX packets:19836 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1357481 (1.2 Mb) TX bytes:1357481 (1.2 Mb)
vif1.0 Link encap:Ethernet HWaddr AA:00:01:75:09:CB
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
[root@domain0 /]# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
128.100.241.0 * 255.255.255.0 U 0 0 0 eth0
169.254.0.0 * 255.255.0.0 U 0 0 0 eth0
169.254.0.0 * 255.255.0.0 U 0 0 0 eth0
127.0.0.0 * 255.0.0.0 U 0 0 0 lo
default bcit-ec 0.0.0.0 UG 0 0 0 eth0
==================================================================
Here is the network configuration of domain1
==================================================================[root@domain1
/]# ifconfig
eth0 Link encap:Ethernet HWaddr AA:00:00:75:09:CB
inet addr:169.254.1.1 Bcast:169.254.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
[root@domain1 /]# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
169.254.0.0 * 255.255.0.0 U 0 0 0 eth0
127.0.0.0 * 255.0.0.0 U 0 0 0 lo
default 169.254.1.0 0.0.0.0 UG 0 0 0 eth0
===================================================================
Also, it seems that the way to control XEN on the xen unstable release is
a bit different from xen1.2. The python scripts to use are a bit
different. There a few scripts that I do not know what they do, example
xend, netfix and xfrd. That is probably because it is my 1st experience
with Python. Are there any documentation ?
Thanks
Richard
-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
> Hello, > > I am using ''xen unstable / xenolinux 2.4.26'' and I am having some problems > to get the domain0 and domain1 to communicate with each other. The python > scripts xc_dom_create.py is not available on the xen unstable release. If > I try to run the copy xc_dom_create.py that I have from xen 1.2 I get an > error on the line "xc = Xc.new()". So this is what I did to start a new > domain1 on xen unstable. From within domain0 I ran the following:Don''t mix the tools! The ''xm'' command replaces all the old xc_* tools.> 1.) run ''xend start'' > > 2.) run the script xen_nat_enable that I got from xen1.2I''m not sure this script is still useful. We tend to use bridging rather than routing by default, but I believe bridge-nf still enables you to NAT onto an outgoing interface. I haven''t tried this, though.> 3.) I ran the following command from within a bash script > xm create -c vmid=1 \ > name=domain1 \ > kernel=/boot/vmlinuz-2.4.26-xenU \ > memory=64 \ > disk=phy:hda9,hda9,w \ > ipaddr=169.254.1.1 \ > root=/dev/hda9 ro \ > ip=169.254.1.1 \ > gateway=169.254.1.0 \ > netmask=255.255.0.0 \ > hostname=host_dom1 > > Domain1 is loaded and I can successfully login to domain1 > through the console. Note: leaving ''mingetty tty1'' in the > inittab file of domain1 is fine. I read somewhere that I should > disable that.tty1 works fine now, as you''ve observed.> Here is the network configuration of domain0 > ==================================================================> [root@domain0 /]# ifconfig > eth0 Link encap:Ethernet HWaddr 00:0E:A6:6B:70:CC > inet addr:128.100.241.161 Bcast:128.100.241.255 Mask:255.255.255.0 > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:4228 errors:0 dropped:0 overruns:0 frame:0 > TX packets:400 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:493895 (482.3 Kb) TX bytes:60050 (58.6 Kb) > Interrupt:22 Memory:feafc000-0 > > eth0:xen Link encap:Ethernet HWaddr 00:0E:A6:6B:70:CC > inet addr:169.254.1.0 Bcast:169.254.255.255 Mask:255.255.0.0 > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) > Interrupt:22 Memory:feafc000-0 > > lo Link encap:Local Loopback > inet addr:127.0.0.1 Mask:255.0.0.0 > UP LOOPBACK RUNNING MTU:16436 Metric:1 > RX packets:19836 errors:0 dropped:0 overruns:0 frame:0 > TX packets:19836 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:1357481 (1.2 Mb) TX bytes:1357481 (1.2 Mb) > > vif1.0 Link encap:Ethernet HWaddr AA:00:01:75:09:CB > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)When you ran ''xend start'' a bridge device should have been created. Have you got the bridge utils installed (/sbin/brctl) ?> Also, it seems that the way to control XEN on the xen unstable release is > a bit different from xen1.2. The python scripts to use are a bit > different. There a few scripts that I do not know what they do, example > xend, netfix and xfrd. That is probably because it is my 1st experience > with Python. Are there any documentation ?''xm'' is the main user tool -- netfix and xfrd are internal rather than user tools. We do desperately need to update the documentation -- sorry. Ian ------------------------------------------------------- This SF.Net email is sponsored by BEA Weblogic Workshop FREE Java Enterprise J2EE developer tools! Get your free copy of BEA WebLogic Workshop 8.1 today. http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Derek Glidden
2004-Jul-18 18:43 UTC
Re: [Xen-devel] Communication between Domain0 and Domain1
On Jul 16, 2004, at 8:08 PM, Richard Ta-Min wrote:> Hello, > > I am using ''xen unstable / xenolinux 2.4.26'' and I am having some > problems > to get the domain0 and domain1 to communicate with each other. The > pythonHi Richard, In addition to what Ian mentioned, I''ve found it''s personally best to assign static IP to your host OSs. It removes any question of what IP they may be getting assigned which removes one variable from any networking/connectivity debugging. The way the networking works currently (-unstable) is just like a typical router/firewall type setup: (internet) <-> [fw eth0] <-routing-> [fw eth1] <-> [host eth0] is equivalent to how the networking is set up with Xen between the dom0 host and the VMs: (internet) <-> [dom0 eth0] <-routing-> [dom0 vif] <-> [VM eth0] If you''re comfortable with using bridging, then when you create the bridge interface and bind the dom0 eth0 and vif to it, you''re essentially telling the dom0 OS to copy any packets it recieves on the eth0 physical device to also go to the dom0 vif, which will pass that traffic to the eth0 of the virtual host. It''s bypassing any explicit routing decisions on dom0 and just saying "pass all traffic directly to the vif as if it were plugged into the external interface." The dom0 host should be able to access the VM host via whatever IP address you''ve assigned as well since it should "see" that IP directly just as anything else on the network will. (I haven''t had the best luck with Linux bridging though. Occasionally the bridge just "goes away" and mysteriously stops passing traffic to the bridge interfaces...) An alternative (and the way I prefer because of the bridge issues) is to skip the bridging and just use your dom0 host as a firewall. Assign the dom0 vif an IP like 192.168.1.1, then assign the VM eth0 an IP like 192.168.1.2 and point to 192.168.1.1 as the default route for the VM. Create an ethernet alias on the dom0 host for whatever external IP you wish to assign to the VM, and use iptables to NAT traffic to/from that interface. Everyone else on the network will see the VM as the external IP you''ve assigned, but the dom0 host itself will use the 192.168.1.2 address to access the VM. This is the way I''ve set up my Xen dev box and it''s working great for me. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- "I think that''s what they mean by | "nickels a day can feed a child." | http://www.eff.org/ I thought, "How can food be so | http://www.anti-dmca.org/ cheap over there?" It''s not, they |-------------------------- just eat the nickels." -- Peter Nguyen ------------------------------------------------------- This SF.Net email is sponsored by BEA Weblogic Workshop FREE Java Enterprise J2EE developer tools! Get your free copy of BEA WebLogic Workshop 8.1 today. http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> An alternative (and the way I prefer because of the bridge issues) is > to skip the bridging and just use your dom0 host as a firewall. Assign > the dom0 vif an IP like 192.168.1.1, then assign the VM eth0 an IP like > 192.168.1.2 and point to 192.168.1.1 as the default route for the VM. > Create an ethernet alias on the dom0 host for whatever external IP you > wish to assign to the VM, and use iptables to NAT traffic to/from that > interface. Everyone else on the network will see the VM as the > external IP you''ve assigned, but the dom0 host itself will use the > 192.168.1.2 address to access the VM. This is the way I''ve set up my > Xen dev box and it''s working great for me.I haven''t had any problems with bridging, but I agree that the L3 routing solution may be better under some circumstances. It''s a slight pain that the vifx.y interface in dom0 needs to be given it''s own an IP address, as it won''t accept being operated in ''pointopoint'' mode. I suspect there''s some ioctl we could add to the backend driver that would enable this. Anyone know about this? It would be good to have a ''vif-router'' script to use as an alternative to ''vif-bridge'' for users wanting to operate a routed configuration. If you''ve got something suitable we could check in to the repo that would be great. I guess a modified ''network'' script would be required too. Ian ------------------------------------------------------- This SF.Net email is sponsored by BEA Weblogic Workshop FREE Java Enterprise J2EE developer tools! Get your free copy of BEA WebLogic Workshop 8.1 today. http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Derek Glidden
2004-Jul-18 22:29 UTC
Re: [Xen-devel] Communication between Domain0 and Domain1
On Jul 18, 2004, at 3:09 PM, Ian Pratt wrote:> I haven''t had any problems with bridging, but I agree that the L3 > routing solution may be better under some circumstances.I haven''t had great luck with bridging in linux period, not just with Xen. Fortunately I''ve rarely needed it. In any case, the reason I''m personally using VMs is to strictly control what is allowed in and out of each particular VM and to be able to control through firewalling anyway, and doing some VM-based solution is a heck of a lot cheaper than buying a dozen physical pieces of hardware and putting them all on a DMZ behind a dedicated firewall, especially if all one of those VMs may be doing is DNS. It''s a little more load on the box overall to have your dom0 doing the packet filtering, but if your boxes were overloaded anyway, you probably wouldn''t be doing VMs. :)> It would be good to have a ''vif-router'' script to use as an > alternative to ''vif-bridge'' for users wanting to operate a routed > configuration. If you''ve got something suitable we could check in > to the repo that would be great. I guess a modified ''network'' > script would be required too.If I can get the VMs stabilized, I''ll work on that next since right now I''ve just got everything in script I wrote that "brute-force" ups a bunch of aliases and adds a bunch of NAT rules that I''m running manually. I haven''t looked real close at the bridge config/script so I don''t know if it handles downing a VM gracefully; iptables isn''t very good at dynamically removing rules. You have to know what order they went in to be able to remove it in the order it was created. i.e. you can create a rule by saying "from source IP such and destination IP such, do thusly" but you can''t remove it with the same terminology, you have to say "remove rule number twelve." So bringing up a VIP and assigning an eth0 alias and creating a NAT rule is pretty easy, but there''s no graceful way to handle removing the NAT rule if you want to down the VM/VIP. The way we''ve been dealing with this issue where I work, using UML, is to have the VM "up" and "down" scripts modify a set of iptables rules to either include or exclude the config for a particular VM, and then require that the rules be reloaded after up or down a VM which will re/create any necessary aliases and reload all the iptables rules. It''s not as elegant, but it does work. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- "We all enter this world in the | Support Electronic Freedom same way: naked; screaming; soaked | http://www.eff.org/ in blood. But if you live your | http://www.anti-dmca.org/ life right, that kind of thing |--------------------------- doesn''t have to stop there." -- Dana Gould ------------------------------------------------------- This SF.Net email is sponsored by BEA Weblogic Workshop FREE Java Enterprise J2EE developer tools! Get your free copy of BEA WebLogic Workshop 8.1 today. http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> > On Jul 18, 2004, at 3:09 PM, Ian Pratt wrote: > > > I haven''t had any problems with bridging, but I agree that the L3 > > routing solution may be better under some circumstances. > > I haven''t had great luck with bridging in linux period, not just with > Xen. Fortunately I''ve rarely needed it. > > In any case, the reason I''m personally using VMs is to strictly control > what is allowed in and out of each particular VM and to be able to > control through firewalling anyway, and doing some VM-based solution is > a heck of a lot cheaper than buying a dozen physical pieces of hardwareWith the bridge-nf patch that we build into dom0 by default its possible to do all the normal iptables firewalling with a bridge setup.> > It would be good to have a ''vif-router'' script to use as an > > alternative to ''vif-bridge'' for users wanting to operate a routed > > configuration. If you''ve got something suitable we could check in > > to the repo that would be great. I guess a modified ''network'' > > script would be required too. > > If I can get the VMs stabilized, I''ll work on that next since right now > I''ve just got everything in script I wrote that "brute-force" ups a > bunch of aliases and adds a bunch of NAT rules that I''m running > manually. > > I haven''t looked real close at the bridge config/script so I don''t know > if it handles downing a VM gracefully; iptables isn''t very good at > dynamically removing rules. You have to know what order they went in > to be able to remove it in the order it was created. i.e. you can > create a rule by saying "from source IP such and destination IP such, > do thusly" but you can''t remove it with the same terminology, you have > to say "remove rule number twelve." So bringing up a VIP and assigning > an eth0 alias and creating a NAT rule is pretty easy, but there''s no > graceful way to handle removing the NAT rule if you want to down the > VM/VIP.Yep, iptables isn''t so smart. I wander if its possible to do something by having rules for a particular domain on a single chain, and then jsut delete the whole chain when a VM dies? Ian ------------------------------------------------------- This SF.Net email is sponsored by BEA Weblogic Workshop FREE Java Enterprise J2EE developer tools! Get your free copy of BEA WebLogic Workshop 8.1 today. http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Chris Andrews
2004-Jul-19 10:13 UTC
Re: [Xen-devel] Communication between Domain0 and Domain1
>>I haven''t looked real close at the bridge config/script so I don''t know >>if it handles downing a VM gracefully; iptables isn''t very good at >>dynamically removing rules. You have to know what order they went in >>to be able to remove it in the order it was created. i.e. you can >>create a rule by saying "from source IP such and destination IP such, >>do thusly" but you can''t remove it with the same terminology, you have >>to say "remove rule number twelve." So bringing up a VIP and assigning >>an eth0 alias and creating a NAT rule is pretty easy, but there''s no >>graceful way to handle removing the NAT rule if you want to down the >>VM/VIP.I''m not sure that''s the case. If you''ve added a rule with -A, specifying the syntax, you can remove it by specifying -D and the same syntax. It''ll remove one rule that exactly matches the syntax you specify to -D. I often use this to drop a LOG rule in temporarily: # iptables -A INPUT -j LOG ... stuff happens ... # iptables -D INPUT -j LOG This is in addition to the -D <rule number> behaviour, which is indeed a real pain to use. Cheers, Chris. ------------------------------------------------------- This SF.Net email is sponsored by BEA Weblogic Workshop FREE Java Enterprise J2EE developer tools! Get your free copy of BEA WebLogic Workshop 8.1 today. http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel