Hi, I have a server I basically inherited that has been performing well for several years but is now not working. So I have ssh access from offsite. When I log in I am in dom0. This is wrong; the unit was set up so that dom0 would have no direct internet connection quite intentionally as a security feature. There is one hosted domain, Centos 5.8 the same as dom0. Up until yesterday I got the hosted domain when I logged in. At the moment I can ssh to dom0 but when logged in can''t ping out. So that''s a problem. I can use xm console hostdomain to see the VM, which appears to be fine on its own but which is disconnected. I don''t know enough about xen networking to know where to look, but it seems reasonable that being unable to ping out would be indicative of something, I just don''t know what. Any suggestions welcome, will even RTFM if someone will point me to it. Dave -- The problem with being cynical is you can''t keep up! -- anon. philosopher
Hello. El 03/04/13 15:38, Dave Stevens escribió:> I have a server I basically inherited that has been performing well for > several years but is now not working. So I have ssh access from offsite. > When I log in I am in dom0. This is wrong; the unit was set up so that > dom0 would have no direct internet connection quite intentionally as a > security feature. There is one hosted domain, Centos 5.8 the same as > dom0. Up until yesterday I got the hosted domain when I logged in. > > At the moment I can ssh to dom0 but when logged in can''t ping out. So > that''s a problem. > > I can use xm console hostdomain to see the VM, which appears to be fine > on its own but which is disconnected. > > I don''t know enough about xen networking to know where to look, but it > seems reasonable that being unable to ping out would be indicative of > something, I just don''t know what. Any suggestions welcome, will even > RTFM if someone will point me to it.In order to make a reliable fix, you will need to get a better idea how it is supposed to work. Check this reference: http://wiki.xen.org/wiki/Xen_Networking While this, please provide some output of the current status of your system, with the output of this commands: xm list brctl show ifconfig iptables -L -v -- Alexandre Kouznetsov
Quoting Alexandre Kouznetsov <alk@ondore.com>:> Hello. > > El 03/04/13 15:38, Dave Stevens escribió: >> I have a server I basically inherited that has been performing well for >> several years but is now not working. So I have ssh access from offsite. >> When I log in I am in dom0. This is wrong; the unit was set up so that >> dom0 would have no direct internet connection quite intentionally as a >> security feature. There is one hosted domain, Centos 5.8 the same as >> dom0. Up until yesterday I got the hosted domain when I logged in. >> >> At the moment I can ssh to dom0 but when logged in can''t ping out. So >> that''s a problem. >> >> I can use xm console hostdomain to see the VM, which appears to be fine >> on its own but which is disconnected. >> >> I don''t know enough about xen networking to know where to look, but it >> seems reasonable that being unable to ping out would be indicative of >> something, I just don''t know what. Any suggestions welcome, will even >> RTFM if someone will point me to it. > > In order to make a reliable fix, you will need to get a better idea > how it is supposed to work. Check this reference: > http://wiki.xen.org/wiki/Xen_NetworkingThanks for the reference, it seems well written and makes sense.> > While this, please provide some output of the current status of your > system, with the output of this commands: > xm list > brctl show > ifconfig > iptables -L -v >ok, here in order: [root@skeena ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 9879 8 r----- 870.8 bulkley 1 2048 4 -b---- 3283.0 [root@skeena ~]# [root@skeena ~]# brctl show bridge name bridge id STP enabled interfaces virbr0 8000.000000000000 yes xenbr0 8000.feffffffffff no vif1.0 vif0.0 peth0 xenbr1 8000.feffffffffff no vif1.1 vif0.1 peth1 [root@skeena ~]# [root@skeena ~]# ifconfig eth1 Link encap:Ethernet HWaddr 00:30:48:CE:84:7F inet addr:10.10.254.240 Bcast:10.10.254.255 Mask:255.255.255.0 inet6 addr: fe80::230:48ff:fece:847f/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:81102 errors:0 dropped:0 overruns:0 frame:0 TX packets:34828 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:10772813 (10.2 MiB) TX bytes:2721383 (2.5 MiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:3086 errors:0 dropped:0 overruns:0 frame:0 TX packets:3086 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:4816900 (4.5 MiB) TX bytes:4816900 (4.5 MiB) peth0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST NOARP MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) Interrupt:250 Base address:0xc000 peth1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:42346 errors:0 dropped:0 overruns:0 frame:0 TX packets:73652 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:9145360 (8.7 MiB) TX bytes:5504883 (5.2 MiB) Interrupt:251 Base address:0xe000 vif0.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:13 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:2562 (2.5 KiB) vif0.1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:34830 errors:0 dropped:0 overruns:0 frame:0 TX packets:81102 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:2721787 (2.5 MiB) TX bytes:10772813 (10.2 MiB) vif1.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:2 errors:0 dropped:0 overruns:0 frame:0 TX packets:12 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:56 (56.0 b) TX bytes:2520 (2.4 KiB) vif1.1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:38756 errors:0 dropped:0 overruns:0 frame:0 TX packets:21268 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:1085168 (1.0 MiB) TX bytes:7679247 (7.3 MiB) virbr0 Link encap:Ethernet HWaddr 00:00:00:00:00:00 inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:14 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:3077 (3.0 KiB) xenbr0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:13 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:2380 (2.3 KiB) TX bytes:0 (0.0 b) xenbr1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:60059 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:8474417 (8.0 MiB) TX bytes:0 (0.0 b) [root@skeena ~]# [root@skeena ~]# iptables -L -v Chain INPUT (policy ACCEPT 19688 packets, 3488K bytes) pkts bytes target prot opt in out source destination 0 0 ACCEPT udp -- virbr0 any anywhere anywhere udp dpt:domain 0 0 ACCEPT tcp -- virbr0 any anywhere anywhere tcp dpt:domain 0 0 ACCEPT udp -- virbr0 any anywhere anywhere udp dpt:bootps 0 0 ACCEPT tcp -- virbr0 any anywhere anywhere tcp dpt:bootps Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 ACCEPT all -- any virbr0 anywhere 192.168.122.0/24 state RELATED,ESTABLISHED 0 0 ACCEPT all -- virbr0 any 192.168.122.0/24 anywhere 0 0 ACCEPT all -- virbr0 virbr0 anywhere anywhere 0 0 REJECT all -- any virbr0 anywhere anywhere reject-with icmp-port-unreachable 0 0 REJECT all -- virbr0 any anywhere anywhere reject-with icmp-port-unreachable 0 0 ACCEPT all -- any any anywhere anywhere PHYSDEV match --physdev-in vif1.1 0 0 ACCEPT all -- any any anywhere anywhere PHYSDEV match --physdev-in vif1.0 Chain OUTPUT (policy ACCEPT 33423 packets, 4609K bytes) pkts bytes target prot opt in out source destination [root@skeena ~]#> -- > Alexandre Kouznetsov > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users >-- The problem with being cynical is you can''t keep up! -- anon. philosopher
Hello. El 04/04/13 10:13, Dave Stevens escribió:> [root@skeena ~]# xm list > Name ID Mem(MiB) VCPUs State Time(s) > Domain-0 0 9879 8 r----- 870.8 > bulkley 1 2048 4 -b---- 3283.0 > [root@skeena ~]# > > [root@skeena ~]# brctl show > bridge name bridge id STP enabled interfaces > virbr0 8000.000000000000 yes > xenbr0 8000.feffffffffff no vif1.0 > vif0.0 > peth0 > xenbr1 8000.feffffffffff no vif1.1 > vif0.1 > peth1This tells us what Domains have network interfaces in what bridges. It seems like there is a mix of configurations. Looks fine, for a bridged networking (not NATted, see below), except there might be a problem with peth0 and peth1.> [root@skeena ~]# ifconfig > eth1 Link encap:Ethernet HWaddr 00:30:48:CE:84:7F > inet addr:10.10.254.240 Bcast:10.10.254.255 Mask:255.255.255.0 > inet6 addr: fe80::230:48ff:fece:847f/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:81102 errors:0 dropped:0 overruns:0 frame:0 > TX packets:34828 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:10772813 (10.2 MiB) TX bytes:2721383 (2.5 MiB) > > lo Link encap:Local Loopback > inet addr:127.0.0.1 Mask:255.0.0.0 > inet6 addr: ::1/128 Scope:Host > UP LOOPBACK RUNNING MTU:16436 Metric:1 > RX packets:3086 errors:0 dropped:0 overruns:0 frame:0 > TX packets:3086 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:4816900 (4.5 MiB) TX bytes:4816900 (4.5 MiB) > > peth0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF > UP BROADCAST NOARP MTU:1500 Metric:1 > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) > Interrupt:250 Base address:0xc000 > > peth1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF > inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link > UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 > RX packets:42346 errors:0 dropped:0 overruns:0 frame:0 > TX packets:73652 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:9145360 (8.7 MiB) TX bytes:5504883 (5.2 MiB) > Interrupt:251 Base address:0xe000This is weired. The common way to give network to DomUs is to define a bridge, put the physical interface into it permanently, and plug DomU''s interfaces there ion demand (you probably found that in the Wiki). The very old network-bridge xen script was defining the bridges as following: save IP configuration from eth0, rename eth0 to peth0 (aka "physical eth0"), create bridge called "eth0" (now that the name is free), set IP configuration it took from the physical eth0 on the new bridge. That way, before xend was brought up you had a plain eth0 to your network. After xend startup you had the same eth0 to your network. This proved to be little bit confusing and the configuration scripts produced unstable results, so late Xen documentation suggests to set up bridges using you platform means, and just tell Xen what bridge to use for what DomU''s NIC. Check this oldier wiki http://wiki.xensource.com/xenwiki/XenNetworking. It probably reflect better your scenario (and imho has nicer pictures) What I see here are peth0 and peth1, they are supposed to be physical interfaces with a valid MAC, and be of the same nature. Actually, they have HWaddr FE:FF:FF:FF:FF:FF and one of them have a inet6 addr. eth1 seems to be a plain physical interface (I bet it''s the one you use to connect to your server) while a bridge called "eth1" is expected. eth0 is simply missing. Please tell me, how many physical network cards does your server have? How are they supposed to be used? Was any physical NIC added or removed recently? Maybe some NIC just died? The change of NIC number could confuse the network-script that badly. Did I told it was unstable? It would be of great reference if you attach your DomU .cfg (under /etc/xen) file and /etc/xen/xend-config.sxp (I took the path form a Debian, it may be different in CentOS). Please check what version of this components do you have installed: Xen hypervisor Linux kernel xenutils (the name of package may be different in CentOS) xentools (the name of package may be different in CentOS)> [root@skeena ~]# iptables -L -v > [...] > 0 0 ACCEPT all -- any virbr0 anywhere > 192.168.122.0/24 state RELATED,ESTABLISHED > 0 0 ACCEPT all -- virbr0 any 192.168.122.0/24 > anywhere > 0 0 ACCEPT all -- virbr0 virbr0 anywhere > anywhere > 0 0 REJECT all -- any virbr0 anywhere > anywhere reject-with icmp-port-unreachable > 0 0 REJECT all -- virbr0 any anywhere > anywhere reject-with icmp-port-unreachableThis must be the rules to restrict traffic within a bridge, dedicated to internal communication between Dom0 and DomU''s. Unused, apparently. The rest of ipfilter configuration is permissive, so it shall not be the issue. As the general solution to your problem, in case you just want to "make it work back" and not doing any new implementation (Dom0 OS and Xe upgrade would be probably cleaner if reinstalled), I would suggest to re-do the networking configuration. Probably your DomU was configured with NATted networking, when it broke Dom0 kept it'' sexternal IP address. The bridged networking is simpler and easer to debug. Greetings. -- Alexandre Kouznetsov
On Thu, 2013-04-04 at 12:16 -0600, Alexandre Kouznetsov wrote:> This is weired.It looks fairly normal for the way that RedHat/CentOS does it. What is odd is that there is no ifconfig output for eth0. --Greg
Quoting Alexandre Kouznetsov <alk@ondore.com>:> Hello. > > El 04/04/13 10:13, Dave Stevens escribió: >> [root@skeena ~]# xm list >> Name ID Mem(MiB) VCPUs State Time(s) >> Domain-0 0 9879 8 r----- 870.8 >> bulkley 1 2048 4 -b---- 3283.0 >> [root@skeena ~]# >> >> [root@skeena ~]# brctl show >> bridge name bridge id STP enabled interfaces >> virbr0 8000.000000000000 yes >> xenbr0 8000.feffffffffff no vif1.0 >> vif0.0 >> peth0 >> xenbr1 8000.feffffffffff no vif1.1 >> vif0.1 >> peth1 > > This tells us what Domains have network interfaces in what bridges. > It seems like there is a mix of configurations. Looks fine, for a > bridged networking (not NATted, see below), except there might be a > problem with peth0 and peth1. > >> [root@skeena ~]# ifconfig >> eth1 Link encap:Ethernet HWaddr 00:30:48:CE:84:7F >> inet addr:10.10.254.240 Bcast:10.10.254.255 Mask:255.255.255.0 >> inet6 addr: fe80::230:48ff:fece:847f/64 Scope:Link >> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >> RX packets:81102 errors:0 dropped:0 overruns:0 frame:0 >> TX packets:34828 errors:0 dropped:0 overruns:0 carrier:0 >> collisions:0 txqueuelen:0 >> RX bytes:10772813 (10.2 MiB) TX bytes:2721383 (2.5 MiB) >> >> lo Link encap:Local Loopback >> inet addr:127.0.0.1 Mask:255.0.0.0 >> inet6 addr: ::1/128 Scope:Host >> UP LOOPBACK RUNNING MTU:16436 Metric:1 >> RX packets:3086 errors:0 dropped:0 overruns:0 frame:0 >> TX packets:3086 errors:0 dropped:0 overruns:0 carrier:0 >> collisions:0 txqueuelen:0 >> RX bytes:4816900 (4.5 MiB) TX bytes:4816900 (4.5 MiB) >> >> peth0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF >> UP BROADCAST NOARP MTU:1500 Metric:1 >> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 >> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 >> collisions:0 txqueuelen:1000 >> RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) >> Interrupt:250 Base address:0xc000 >> >> peth1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF >> inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link >> UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 >> RX packets:42346 errors:0 dropped:0 overruns:0 frame:0 >> TX packets:73652 errors:0 dropped:0 overruns:0 carrier:0 >> collisions:0 txqueuelen:1000 >> RX bytes:9145360 (8.7 MiB) TX bytes:5504883 (5.2 MiB) >> Interrupt:251 Base address:0xe000 > This is weired. > > The common way to give network to DomUs is to define a bridge, put > the physical interface into it permanently, and plug DomU''s > interfaces there ion demand (you probably found that in the Wiki). > The very old network-bridge xen script was defining the bridges as following: > save IP configuration from eth0, > rename eth0 to peth0 (aka "physical eth0"), > create bridge called "eth0" (now that the name is free), > set IP configuration it took from the physical eth0 on the new bridge. > > That way, before xend was brought up you had a plain eth0 to your > network. After xend startup you had the same eth0 to your network. > This proved to be little bit confusing and the configuration scripts > produced unstable results, so late Xen documentation suggests to set > up bridges using you platform means, and just tell Xen what bridge > to use for what DomU''s NIC. > > Check this oldier wiki > http://wiki.xensource.com/xenwiki/XenNetworking. It probably reflect > better your scenario (and imho has nicer pictures) > > What I see here are peth0 and peth1, they are supposed to be > physical interfaces with a valid MAC, and be of the same nature. > Actually, they have HWaddr FE:FF:FF:FF:FF:FF and one of them have a > inet6 addr. eth1 seems to be a plain physical interface (I bet it''s > the one you use to connect to your server) while a bridge called > "eth1" is expected. eth0 is simply missing. > > > Please tell me, how many physical network cards does your server > have? How are they supposed to be used? Was any physical NIC added > or removed recently? Maybe some NIC just died? The change of NIC > number could confuse the network-script that badly. Did I told it > was unstable?well the MB is a Supermicro 2P with 2 nics on it, eth0 and 1. The incoming line goes into a Cisco router and that sends packets to eth1 on the MB. The nic hasn''t changed recently, still using the usual one. There is no add-in nic.> > It would be of great reference if you attach your DomU .cfg (under > /etc/xen) file and /etc/xen/xend-config.sxp (I took the path form a > Debian, it may be different in CentOS).ok: [root@skeena xen]# cat xend-config.sxp # -*- sh -*- # # Xend configuration file. # # This example configuration is appropriate for an installation that # utilizes a bridged network configuration. Access to xend via http # is disabled. # Commented out entries show the default for that entry, unless otherwise # specified. #(logfile /var/log/xen/xend.log) #(loglevel DEBUG) #(xend-http-server no) (xend-unix-server yes) #(xend-tcp-xmlrpc-server no) #(xend-unix-xmlrpc-server yes) #(xend-relocation-server no) # The relocation server should be kept desactivated unless using a trusted # network, the domain virtual memory will be exchanged in raw form without # encryption of the communication. See also xend-relocation-hosts-allow option (xend-unix-path /var/lib/xend/xend-socket) # Port xend should use for the HTTP interface, if xend-http-server is set. #(xend-port 8000) # Port xend should use for the relocation interface, if xend-relocation-server # is set. #(xend-relocation-port 8002) # Address xend should listen on for HTTP connections, if xend-http-server is # set. # Specifying ''localhost'' prevents remote connections. # Specifying the empty string '''' (the default) allows all connections. #(xend-address '''') #(xend-address localhost) # Address xend should listen on for relocation-socket connections, if # xend-relocation-server is set. # Meaning and default as for xend-address above. #(xend-relocation-address '''') # The hosts allowed to talk to the relocation port. If this is empty (the # default), then all connections are allowed (assuming that the connection # arrives on a port and interface on which we are listening; see # xend-relocation-port and xend-relocation-address above). Otherwise, this # should be a space-separated sequence of regular expressions. Any host with # a fully-qualified domain name or an IP address that matches one of these # regular expressions will be accepted. # # For example: # (xend-relocation-hosts-allow ''^localhost$ ^.*\.example\.org$'') # #(xend-relocation-hosts-allow '''') (xend-relocation-hosts-allow ''^localhost$ ^localhost\\.localdomain$'') # The limit (in kilobytes) on the size of the console buffer #(console-limit 1024) ## # To bridge network traffic, like this: # # dom0: fake eth0 -> vif0.0 -+ # | # bridge -> real eth0 -> the network # | # domU: fake eth0 -> vifN.0 -+ # # use # # (network-script network-bridge) # # Your default ethernet device is used as the outgoing interface, by default. # To use a different one (e.g. eth1) use # # (network-script ''network-bridge netdev=eth1'') # # The bridge is named xenbr0, by default. To rename the bridge, use # # (network-script ''network-bridge bridge=<name>'') # # It is possible to use the network-bridge script in more complicated # scenarios, such as having two outgoing interfaces, with two bridges, and # two fake interfaces per guest domain. To do things like this, write # yourself a wrapper script, and call network-bridge from it, as appropriate. # (network-script network-bridge-multi) # The script used to control virtual interfaces. This can be overridden on a # per-vif basis when creating a domain or a configuring a new vif. The # vif-bridge script is designed for use with the network-bridge script, or # similar configurations. # # If you have overridden the bridge name using # (network-script ''network-bridge bridge=<name>'') then you may wish to do the # same here. The bridge name can also be set when creating a domain or # configuring a new vif, but a value specified here would act as a default. # # If you are using only one bridge, the vif-bridge script will discover that, # so there is no need to specify it explicitly. # (vif-script vif-bridge) ## Use the following if network traffic is routed, as an alternative to the # settings for bridged networking given above. #(network-script network-route) #(vif-script vif-route) ## Use the following if network traffic is routed with NAT, as an alternative # to the settings for bridged networking given above. #(network-script network-nat) #(vif-script vif-nat) # Dom0 will balloon out when needed to free memory for domU. # dom0-min-mem is the lowest memory level (in MB) dom0 will get down to. # If dom0-min-mem=0, dom0 will never balloon out. (dom0-min-mem 256) # In SMP system, dom0 will use dom0-cpus # of CPUS # If dom0-cpus = 0, dom0 will take all cpus available (dom0-cpus 0) # Whether to enable core-dumps when domains crash. #(enable-dump no) # The tool used for initiating virtual TPM migration #(external-migration-tool '''') # The interface for VNC servers to listen on. Defaults # to 127.0.0.1 To restore old ''listen everywhere'' behaviour # set this to 0.0.0.0 #(vnc-listen ''127.0.0.1'') # The default password for VNC console on HVM domain. # Empty string is no authentication. (vncpasswd '''') # The default keymap to use for the VM''s virtual keyboard # when not specified in VM''s configuration (keymap ''en-us'') # The VNC server can be told to negotiate a TLS session # to encryption all traffic, and provide x509 cert to # clients enalbing them to verify server identity. The # GTK-VNC widget, virt-viewer, virt-manager and VeNCrypt # all support the VNC extension for TLS used in QEMU. The # TightVNC/RealVNC/UltraVNC clients do not. # # To enable this create x509 certificates / keys in the # directory /etc/xen/vnc # # ca-cert.pem - The CA certificate # server-cert.pem - The Server certificate signed by the CA # server-key.pem - The server private key # # and then uncomment this next line # (vnc-tls 1) # # The certificate dir can be pointed elsewhere.. # # (vnc-x509-cert-dir /etc/xen/vnc) # # The server can be told to request & validate an x509 # certificate from the client. Only clients with a cert # signed by the trusted CA will be able to connect. This # is more secure the password auth alone. Passwd auth can # used at the same time if desired. To enable client cert # checking uncomment this: # # (vnc-x509-verify 1) # Allow probing of disk image file format. This is insecure! It lets # a malicious domU read any file in dom0. Applies only to fully # virtual domUs. Required for using formats other than raw. #(enable-image-format-probing no) # Number of seconds xend will wait for device creation #(device-create-timeout 100) # Strict checking when doing PCI passthrough; enabled by default #(pci-dev-assign-strict-check yes) [root@skeena xen]# -- ends ---> > Please check what version of this components do you have installed: > Xen hypervisor[root@skeena ~]# xm info host : skeena.bvserver.ca release : 2.6.18-308.20.1.el5xen version : #1 SMP Tue Nov 13 11:03:56 EST 2012 machine : x86_64 nr_cpus : 8 nr_nodes : 1 sockets_per_node : 2 cores_per_socket : 4 threads_per_core : 1 cpu_mhz : 2311 hw_caps : 178bfbff:efd3fbff:00000000:00000110:00802009:00000000:000037ff total_memory : 16383 free_memory : 4123 node_to_cpu : node0:0-7 xen_major : 3 xen_minor : 1 xen_extra : .2-308.20.1.el5 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : unavailable cc_compiler : gcc version 4.1.2 20080704 (Red Hat 4.1.2-52) cc_compile_by : mockbuild cc_compile_domain : centos.org cc_compile_date : Tue Nov 13 10:06:48 EST 2012 xend_config_format : 2 [root@skeena ~]#> Linux kernel[root@skeena ~]# uname -a Linux skeena.bvserver.ca 2.6.18-308.20.1.el5xen #1 SMP Tue Nov 13 11:03:56 EST 2012 x86_64 x86_64 x86_64 GNU/Linux [root@skeena ~]#> xenutils (the name of package may be different in CentOS)> xentools (the name of package may be different in CentOS)can''t find an applicable name for the toolsets Dave> >> [root@skeena ~]# iptables -L -v >> [...] >> 0 0 ACCEPT all -- any virbr0 anywhere >> 192.168.122.0/24 state RELATED,ESTABLISHED >> 0 0 ACCEPT all -- virbr0 any 192.168.122.0/24 >> anywhere >> 0 0 ACCEPT all -- virbr0 virbr0 anywhere >> anywhere >> 0 0 REJECT all -- any virbr0 anywhere >> anywhere reject-with icmp-port-unreachable >> 0 0 REJECT all -- virbr0 any anywhere >> anywhere reject-with icmp-port-unreachable > > This must be the rules to restrict traffic within a bridge, > dedicated to internal communication between Dom0 and DomU''s. Unused, > apparently. The rest of ipfilter configuration is permissive, so it > shall not be the issue. > > > As the general solution to your problem, in case you just want to > "make it work back" and not doing any new implementation (Dom0 OS > and Xe upgrade would be probably cleaner if reinstalled), I would > suggest to re-do the networking configuration.I''m agreeable, just not sure how> Probably your DomU was configured with NATted networking, when it > broke Dom0 kept it''s external IP address. The bridged networking is > simpler and easier to debug. > > Greetings.Thank you.> > -- > Alexandre Kouznetsov > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users >-- The problem with being cynical is you can''t keep up! -- anon. philosopher
Quoting Dave Stevens <geek@uniserve.com>:> Quoting Alexandre Kouznetsov <alk@ondore.com>:I forgot to post the domU config: [root@skeena xen]# cat bulkley name = "bulkley" uuid = "27180e16-68fd-f419-d0d9-abebc2de4a98" maxmem = 2048 memory = 2048 vcpus = 4 bootloader = "/usr/bin/pygrub" on_poweroff = "destroy" on_reboot = "restart" on_crash = "restart" vfb = [ "type=vnc,vncunused=1,keymap=en-us" ] disk = [ "phy:/dev/VolGroup00/bulkley,xvda,w" ] vif = [ "mac=00:16:36:35:12:ee,bridge=xenbr0,script=vif-bridge","mac=00:16:3e:52:61:ce,bridge=xenbr1,script=vif-bridge" ] -- The problem with being cynical is you can''t keep up! -- anon. philosopher
Quoting Dave Stevens <geek@uniserve.com>:> Quoting Alexandre Kouznetsov <alk@ondore.com>:I forgot to post the domU config: [root@skeena xen]# cat bulkley name = "bulkley" uuid = "27180e16-68fd-f419-d0d9-abebc2de4a98" maxmem = 2048 memory = 2048 vcpus = 4 bootloader = "/usr/bin/pygrub" on_poweroff = "destroy" on_reboot = "restart" on_crash = "restart" vfb = [ "type=vnc,vncunused=1,keymap=en-us" ] disk = [ "phy:/dev/VolGroup00/bulkley,xvda,w" ] vif = [ "mac=00:16:36:35:12:ee,bridge=xenbr0,script=vif-bridge","mac=00:16:3e:52:61:ce,bridge=xenbr1,script=vif-bridge" ] -- The problem with being cynical is you can''t keep up! -- anon. philosopher
Hello. El 04/04/13 13:26, Greg Woods escribió:> On Thu, 2013-04-04 at 12:16 -0600, Alexandre Kouznetsov wrote: > >> This is weired. > > It looks fairly normal for the way that RedHat/CentOS does it. What is > odd is that there is no ifconfig output for eth0.Please post an example, it would be great. I expected pethX to keep the original interface''s MAC. Beside the lost eth0 (btw, just realized that it might be just down), since there is peth1 and eth1, I expected eth1 to be a bridge, not a plain interface. -- Alexandre Kouznetsov
Hello. El 04/04/13 13:52, Dave Stevens escribió:>> Please tell me, how many physical network cards does your server have? >> How are they supposed to be used? Was any physical NIC added or >> removed recently? Maybe some NIC just died? The change of NIC number >> could confuse the network-script that badly. Did I told it was unstable? > > well the MB is a Supermicro 2P with 2 nics on it, eth0 and 1. The > incoming line goes into a Cisco router and that sends packets to eth1 on > the MB. The nic hasn''t changed recently, still using the usual one. > There is no add-in nic.Great. Make sure, with "ifconfig -a" that eth0 is at least present (although inactive). If it''s not in the output, it''s probably dead and you will need an add-on card, and consider a soon change of the MB.> [root@skeena xen]# cat bulkley > name = "bulkley" > uuid = "27180e16-68fd-f419-d0d9-abebc2de4a98" > maxmem = 2048 > memory = 2048 > vcpus = 4 > bootloader = "/usr/bin/pygrub" > on_poweroff = "destroy" > on_reboot = "restart" > on_crash = "restart" > vfb = [ "type=vnc,vncunused=1,keymap=en-us" ] > disk = [ "phy:/dev/VolGroup00/bulkley,xvda,w" ] > vif = [ > "mac=00:16:36:35:12:ee,bridge=xenbr0,script=vif-bridge", > "mac=00:16:3e:52:61:ce,bridge=xenbr1,script=vif-bridge" > ]Please tell me, how is the network [supposed to be] configured within DomU? It has 2 network interfaces, are they both used and have IP address? Those IPs, are they public or private?> [root@skeena xen]# cat xend-config.sxp > [...] > # It is possible to use the network-bridge script in more complicated > # scenarios, such as having two outgoing interfaces, with two bridges, and > # two fake interfaces per guest domain. To do things like this, write > # yourself a wrapper script, and call network-bridge from it, as > appropriate. > # > (network-script network-bridge-multi)This one is the tricky one. You can look for /etc/xen/scripts/network-bridge-multi Surely it''s the script that is making all the mess with eth and peth.>> As the general solution to your problem, in case you just want to >> "make it work back" and not doing any new implementation (Dom0 OS and >> Xe upgrade would be probably cleaner if reinstalled), I would suggest >> to re-do the networking configuration. > > I''m agreeable, just not sure howSet our bulkley DomU to not to startup automatically Comment out "network-bridge-multi" line in xend-config.sxp. I would remove the mention of "script=vif-bridge" in bulkley config file (just leave mac= and bridge=, let the defaults do the rest), but maybe it''s unnecessary . Reboot. Make sure you have got your eth1 and eth0 plain and clean ("ifconfig -a"), no peth or weired bridges. Build two ethernet bridges, xenbr0 containing eth0 and xenbr1 containing eth1. Check CentOS documentation about that, not my strength. I found this quick reference, maybe it will help: http://wiki.xen.org/wiki/Network_Configuration_Examples_%28Xen_4.1%2B%29#Red_Hat-style_bridge_configuration_.28e.g._RHEL.2C_Fedora.2C_CentOS.29 http://www.centos.org/docs/5/html/Virtualization-en-US/ch-virt-bridge-errors.html Make sure the correct bridges configuration is set up on boot automatically. I would leave the physical interfaces eth0 and eth1 without IP configuration, and set IP directly on the bridge (or at least it''s the Debian used way, not sure if CentOS documentation suggests it differently). When I have a Xen machine with a DomU with public IP, my Dom0 does not have any IP address on the bridge that contains the public physical interface. "brctl show" should display something like this: xenbr0 8000.003048ce847e no eth0 xenbr1 8000.003048ce847f no eth1 And ifconfig should show eth0, eth1, xenbr0 and xenbr1. Not sure if virbr0 will be there, but should be problem in either case. After this, your server will be ready to start the bulkley DomU with networking in bridged mode. Note that your DomU will need to have configured the right IP''s on it''s interfaces. I''m still not sure about NAT ghost in your setup. Greetings. -- Alexandre Kouznetsov
On Thu, 2013-04-04 at 16:26 -0600, Alexandre Kouznetsov wrote:> Hello. > > El 04/04/13 13:26, Greg Woods escribió: > > On Thu, 2013-04-04 at 12:16 -0600, Alexandre Kouznetsov wrote: > > > >> This is weired. > > > > It looks fairly normal for the way that RedHat/CentOS does it. What is > > odd is that there is no ifconfig output for eth0. > Please post an example, it would be great. > > I expected pethX to keep the original interface's MAC.It doesn't on my systems. I'll admit that I do not fully understand how all this works, but here is partial ifconfig output from a working CentOS 5.9 system: eth2 Link encap:Ethernet HWaddr 00:1B:21:A1:CA:B0 inet addr:192.168.203.2 Bcast:192.168.203.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:40680 errors:0 dropped:0 overruns:0 frame:0 TX packets:40048 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:10464680 (9.9 MiB) TX bytes:10335739 (9.8 MiB) peth2 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:40298 errors:0 dropped:0 overruns:0 frame:0 TX packets:40189 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:9853371 (9.3 MiB) TX bytes:10403372 (9.9 MiB) Memory:fa7e0000-fa800000 xenbr2 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:78922 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:19374293 (18.4 MiB) TX bytes:0 (0.0 b) From "brctl show": xenbr2 8000.feffffffffff no vif5.1 vif4.1 vif3.1 vif2.1 vif1.1 vif0.2 peth2 Then all the /etc/xen config files just reference xenbr2. All of the xenbrX and pethX devices have this same pseudo-MAC address. It is not clear to me how they are connected to the underlying ethX device. All this said, if "ifconfig eth0" shows no output, while this device is supposed to be used for bridged networking in Xen, that's a problem. If it were me, I'd be looking at "dmesg" output to figure out why the "eth0" device isn't being set up at boot time. Fix what you know is broken first. --Greg _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Hello. Great reference. El 05/04/13 09:28, Greg Woods escribió:> On Thu, 2013-04-04 at 16:26 -0600, Alexandre Kouznetsov wrote: >> I expected pethX to keep the original interface's MAC. > > It doesn't on my systems. I'll admit that I do not fully understand how > all this works, but here is partial ifconfig output from a working > CentOS 5.9 system: > > eth2 Link encap:Ethernet HWaddr 00:1B:21:A1:CA:B0 > inet addr:192.168.203.2 Bcast:192.168.203.255 > Mask:255.255.255.0 > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:40680 errors:0 dropped:0 overruns:0 frame:0 > TX packets:40048 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:10464680 (9.9 MiB) TX bytes:10335739 (9.8 MiB) > > peth2 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF > UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 > RX packets:40298 errors:0 dropped:0 overruns:0 frame:0 > TX packets:40189 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:9853371 (9.3 MiB) TX bytes:10403372 (9.9 MiB) > Memory:fa7e0000-fa800000 > > xenbr2 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF > UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 > RX packets:78922 errors:0 dropped:0 overruns:0 frame:0 > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:19374293 (18.4 MiB) TX bytes:0 (0.0 b) > > From "brctl show": > > xenbr2 8000.feffffffffff no vif5.1 > vif4.1 > vif3.1 > vif2.1 > vif1.1 > vif0.2 > peth2 > > Then all the /etc/xen config files just reference xenbr2.> > All of the xenbrX and pethX devices have this same pseudo-MAC address. > It is not clear to me how they are connected to the underlying ethX > device.It seems like, Dom0's IP connectivity is not done directly via the actual device. Instead it creates a vif0.2 and maps it as eth2 to Dom0, as it would do with any other Dom. Seems much cleaner than the setup I described before, as "old ways", when a eth was renamed to peth and a bridge called eth was created.> All this said, if "ifconfig eth0" shows no output, while this device is > supposed to be used for bridged networking in Xen, that's a problem. If > it were me, I'd be looking at "dmesg" output to figure out why the > "eth0" device isn't being set up at boot time. Fix what you know is > broken first.I's more clear to me now. In Dave's case, "ifconfig eth0" within Dom0 should show a eth0 in inactive state. That should be normal, if Dom0 where not supposed to have a IP interface for that network. What is still weired, is that he said his eth1 goes to a Cisco router, while this configuration would make more sense if eth0 where the public interface. Hey, Dave, didn't you misswired your Supermicro networking connections? -- Alexandre Kouznetsov _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Alexandre Kouznetsov
2013-Apr-05 16:05 UTC
Re: newbie in trouble with CentOS Xen - new reference
Hello. With the reference kindly provided by Greg Woods, it''s more clear to me how your setup is supposed to work. Maybe a less invasive fix may be applied. Please double check, what eth corresponds to what physical port on your Supermicro MB, and which one of them is the public (the one that goes to Cisco router?). Is it really eth1? Also, verify the networking config on DomU, it shall have eth0 and eth1 as well, check which of them is supposed to be the public network and what IP does it have configured. Maybe it''s just misswired. If that is the case, there is no need to re-configure your networking, after all. Greetings. -- Alexandre Kouznetsov
On Fri, 2013-04-05 at 10:05 -0600, Alexandre Kouznetsov wrote:> It seems like, Dom0''s IP connectivity is not done directly via the > actual device. Instead it creates a vif0.2 and maps it as eth2 to Dom0, > as it would do with any other Dom. > Seems much cleaner than the setup I described before, as "old ways", > when a eth was renamed to peth and a bridge called eth was created.We''re getting a bit away from the original topic, but this is still a useful discussion. This explains a lot. We''ve got the ethX devices which are the physical device, same as any non-Xen system, and the xenbrX devices, which are the actual bridges. The pethX devices don''t seem to serve any real purpose. I would hazard a guess that the pethX devices aren''t really needed at all, and are just there for historical compatibility. --Greg
El 05/04/13 10:13, Greg Woods escribió:> On Fri, 2013-04-05 at 10:05 -0600, Alexandre Kouznetsov wrote: > >> It seems like, Dom0''s IP connectivity is not done directly via the >> actual device. Instead it creates a vif0.2 and maps it as eth2 to Dom0, >> as it would do with any other Dom. >> Seems much cleaner than the setup I described before, as "old ways", >> when a eth was renamed to peth and a bridge called eth was created. > > We''re getting a bit away from the original topic,The original topic is still discussed on another branch of this thread (:> We''ve got the ethX devices > which are the physical device, same as any non-Xen system,No. While Dom0 is booted, it''s virtualized. See all that vif0.X, they are interfaces exported to a Domain with ID 0.> and the > xenbrX devices, which are the actual bridges.Right, they communicate the backend vif''s with the physical network.> The pethX devices don''t > seem to serve any real purpose. I would hazard a guess that the pethX > devices aren''t really needed at all, and are just there for historical > compatibility.Yes they do serve a purpose. ethX devices you see in your Dom0 (note, this seems to be CentOS specific Xen stuff) are not real network interfaces. They are virtualized, just like any other network interface in any other DomU. They are backed by a corresponding vif0.X device. Just as in case of DomU, vif0.X devices are bridged together with a physical interface within Dom0. That is where pethX comes into play, it links the bridge to the physical network. Also, it''s used to interact with the physical card, for MTU, error correction settings, WOL, firmware, etc. So the topology is like this: ----------- Dom0 eth0 ----------- | Xen PV VIF | ----------- vif0.0 | xenbr0 | peth0 ----------- | physical port Greetings. -- Alexandre Kouznetsov
Quoting Alexandre Kouznetsov <alk@ondore.com>:> El 05/04/13 10:13, Greg Woods escribió: >> On Fri, 2013-04-05 at 10:05 -0600, Alexandre Kouznetsov wrote: >> >>> It seems like, Dom0''s IP connectivity is not done directly via the >>> actual device. Instead it creates a vif0.2 and maps it as eth2 to Dom0, >>> as it would do with any other Dom. >>> Seems much cleaner than the setup I described before, as "old ways", >>> when a eth was renamed to peth and a bridge called eth was created. >> >> We''re getting a bit away from the original topic, > The original topic is still discussed on another branch of this thread (: > >> We''ve got the ethX devices >> which are the physical device, same as any non-Xen system, > No. While Dom0 is booted, it''s virtualized. See all that vif0.X, > they are interfaces exported to a Domain with ID 0. > >> and the >> xenbrX devices, which are the actual bridges. > Right, they communicate the backend vif''s with the physical network. > >> The pethX devices don''t >> seem to serve any real purpose. I would hazard a guess that the pethX >> devices aren''t really needed at all, and are just there for historical >> compatibility. > > Yes they do serve a purpose. ethX devices you see in your Dom0 > (note, this seems to be CentOS specific Xen stuff) are not real > network interfaces. They are virtualized, just like any other > network interface in any other DomU. They are backed by a > corresponding vif0.X device. Just as in case of DomU, vif0.X devices > are bridged together with a physical interface within Dom0. That is > where pethX comes into play, it links the bridge to the physical > network. Also, it''s used to interact with the physical card, for > MTU, error correction settings, WOL, firmware, etc. > > So the topology is like this: > > ----------- > Dom0 eth0 > ----------- > | > Xen PV VIF > | > ----------- > vif0.0 > | > xenbr0 > | > peth0 > ----------- > | > physical port > > Greetings. > > -- > Alexandre Kouznetsov >Alexandre, I don'' know if I can send an attachment to this list, but I''ll see if one comes through, also pasted in below. A diagram of the original "supposed to be like this" setup. Dave LAN0 LAN1 | | +-----+-----------------------------------------------------+-----+ | | | | | +---+-------------------------+ +-------------------------+---+ | | | | | | | | | | | peth0 xenbr0 | | xenbr1 peth1 | | | | | | | | | | vif0.0 vif1.0 vif1.1 | | vif2.0 vif2.1 vif0.1 | | | | | | | | | | | | | | | +---+------------+-------+----+ +----+-------+------------+---+ | | | | | | | | | | | +------+-------+-----------+-------+------+ | | | | | | | | | | | | | | | +----+-------+----+ +----+-------+----+ | | | | | | | | | | | | | | | | | | eth0 | | eth0 eth1 | | eth0 eth1 | | eth1 | | | | | | | | | | | | | | | | +-+-+ | | +-+-+ +-+-+ | | +-+-+ +-+-+ | | +-+-+ | | | | | | | | | | | | | | | | | | | | | | www ssh | | www ssh ftp pop | | www ssh ftp pop | | ftp pop | | | | | | | | | | Domain0 | | Domain1 | | Domain2 | | Domain0 | +-----------+ +-----------------+ +-----------------+ +-----------+> > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users >-- The problem with being cynical is you can''t keep up! -- anon. philosopher _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Hello. El 05/04/13 10:58, Dave Stevens escribió:> A diagram of the original "supposed to be like this" setupYes, after the help of Greg''s reference, the configuration you sent "Thu, 04 Apr 2013 09:13:42 -0700" in more understandable and looks much better. The only relevant difference is that eth0 is not UP in Dom0 (surely it''s still present, but you better make sure with "ifconfig -a"), and that you diagram describes DomU''s with two interfaces in the same bridge, while your setup surely implies a DomU with two interfaces in different bridges. You tell me if this differences are relevant for what you intend to do. If not, double check you wiring and IP addressing, it should work. Greetings. -- Alexandre Kouznetsov
On Fri, 2013-04-05 at 09:58 -0700, Dave Stevens wrote:> Quoting Alexandre Kouznetsov <alk@ondore.com>:Thanks for the diagrams, guys. This makes it a lot clearer to me. --Greg
Quoting Alexandre Kouznetsov <alk@ondore.com>:> Hello. > > El 05/04/13 10:58, Dave Stevens escribió: >> A diagram of the original "supposed to be like this" setup > Yes, after the help of Greg''s reference, the configuration you sent > "Thu, 04 Apr 2013 09:13:42 -0700" in more understandable and looks > much better. > > The only relevant difference is that eth0 is not UP in Dom0 (surely > it''s still present, but you better make sure with "ifconfig -a"), > and that you diagram describes DomU''s with two interfaces in the > same bridge, while your setup surely implies a DomU with two > interfaces in different bridges. > > You tell me if this differences are relevant for what you intend to > do. If not, double check you wiring and IP addressing, it should work. > > Greetings.Greetings yourself. I started apache in dom0 and from outside wa getting the default apache page, have now replaced it with something slightly more meaningful. So I think the xenbr1 connection to vif4.1 must not be working properly in sending packets to domU''s eth1 in dom0: peth1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:118842 errors:0 dropped:0 overruns:0 frame:0 TX packets:181148 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:24736385 (23.5 MiB) TX bytes:17074473 (16.2 MiB) Interrupt:251 Base address:0xe000 so lots of traffic there and then: xenbr1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:143222 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:21810836 (20.8 MiB) TX bytes:0 (0.0 b) so why no TX bytes? and vif4.1 is: vif4.1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:40649 errors:0 dropped:0 overruns:0 frame:0 TX packets:23529 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:1138388 (1.0 MiB) TX bytes:8233545 (7.8 MiB) and domU''s eth1 is: eth1 Link encap:Ethernet HWaddr 00:16:3E:52:61:CE inet addr:204.174.35.215 Bcast:204.174.35.255 Mask:255.255.255.0 inet6 addr: fe80::216:3eff:fe52:61ce/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:23474 errors:0 dropped:0 overruns:0 frame:0 TX packets:40601 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:8214830 (7.8 MiB) TX bytes:1705458 (1.6 MiB) domU is lower thandom0 because I rebooted the VM Does any of this suggest anything to you? Dave> > -- > Alexandre Kouznetsov > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users >-- The problem with being cynical is you can''t keep up! -- anon. philosopher
El 05/04/13 20:02, Dave Stevens escribió:> I started apache in dom0 and from outside wa getting the default apache > page, have now replaced it with something slightly more meaningful.While testing low level connectivity (layer 2 and 3), a ping (aka ICMP echo) is usually enough. A high level service, such as Apache, would also work, but may introduce false positives and false negatives if not used properly.> So I > think the xenbr1 connection to vif4.1 must not be working properly in > sending packets to domU''s eth1 > > in dom0: > > peth1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF > inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link > UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 > RX packets:118842 errors:0 dropped:0 overruns:0 frame:0 > TX packets:181148 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:24736385 (23.5 MiB) TX bytes:17074473 (16.2 MiB) > Interrupt:251 Base address:0xe000 > so lots of traffic there and then: > > xenbr1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF > UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 > RX packets:143222 errors:0 dropped:0 overruns:0 frame:0 > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:21810836 (20.8 MiB) TX bytes:0 (0.0 b) > > so why no TX bytes?That is not normal. There is always feedback traffic, even if the outgoing packets are routed to other interface.> and vif4.1 is: > vif4.1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF > inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link > UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 > RX packets:40649 errors:0 dropped:0 overruns:0 frame:0 > TX packets:23529 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:500 > RX bytes:1138388 (1.0 MiB) TX bytes:8233545 (7.8 MiB) > > and domU''s eth1 is: > > eth1 Link encap:Ethernet HWaddr 00:16:3E:52:61:CE > inet addr:204.174.35.215 Bcast:204.174.35.255 > Mask:255.255.255.0 > inet6 addr: fe80::216:3eff:fe52:61ce/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:23474 errors:0 dropped:0 overruns:0 frame:0 > TX packets:40601 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:8214830 (7.8 MiB) TX bytes:1705458 (1.6 MiB) > > domU is lower thandom0 because I rebooted the VM > > Does any of this suggest anything to you?I guess, peth1 is still within the bridge xenbr1. In a case like this, I would get a tcpdump and take a look on vif4.1, xenbr1 and peth1. Consider using -e key, to log MAC address of the devices. If you ping your DomU from outside, in "normal working" case, you should see a echo-request on peth1, xenbr1 and vif4.1, followed by a echo-replay. The request should have the MAC address of your router (or host within the same broadcast domain) as source, and the MAC address of your DomU (as defined in it''s config and shown by ifconfig) as destination. The replay should have the same addresses inverted. If you don''t see echo-request on peth1, I would troubleshoot the switch. If you don''t see echo-request on xenbr1 or vif4.1, something is wrong with how they are participating in the bridge (try add and remove with brctl? see dmesg errors). If you don''t see echo-request on eth1 in DomU, while they are visible on vif4.1, it must be Xen''s fault somehow. If you see echo-request passing through but not echo-replay, or wrong MAC in it, commonly it has to do with routing errors. Greetings. -- Alexandre Kouznetsov
Quoting Alexandre Kouznetsov <alk@ondore.com>:> El 05/04/13 20:02, Dave Stevens escribió: >> I started apache in dom0 and from outside wa getting the default apache >> page, have now replaced it with something slightly more meaningful. > While testing low level connectivity (layer 2 and 3), a ping (aka > ICMP echo) is usually enough. A high level service, such as Apache, > would also work, but may introduce false positives and false > negatives if not used properly. > > >> So I >> think the xenbr1 connection to vif4.1 must not be working properly in >> sending packets to domU''s eth1 >> >> in dom0: >> >> peth1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF >> inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link >> UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 >> RX packets:118842 errors:0 dropped:0 overruns:0 frame:0 >> TX packets:181148 errors:0 dropped:0 overruns:0 carrier:0 >> collisions:0 txqueuelen:1000 >> RX bytes:24736385 (23.5 MiB) TX bytes:17074473 (16.2 MiB) >> Interrupt:251 Base address:0xe000 >> so lots of traffic there and then: >> >> xenbr1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF >> UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 >> RX packets:143222 errors:0 dropped:0 overruns:0 frame:0 >> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 >> collisions:0 txqueuelen:0 >> RX bytes:21810836 (20.8 MiB) TX bytes:0 (0.0 b) >> >> so why no TX bytes? > That is not normal. There is always feedback traffic, even if the > outgoing packets are routed to other interface. > >> and vif4.1 is: >> vif4.1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF >> inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link >> UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 >> RX packets:40649 errors:0 dropped:0 overruns:0 frame:0 >> TX packets:23529 errors:0 dropped:0 overruns:0 carrier:0 >> collisions:0 txqueuelen:500 >> RX bytes:1138388 (1.0 MiB) TX bytes:8233545 (7.8 MiB) >> >> and domU''s eth1 is: >> >> eth1 Link encap:Ethernet HWaddr 00:16:3E:52:61:CE >> inet addr:204.174.35.215 Bcast:204.174.35.255 >> Mask:255.255.255.0 >> inet6 addr: fe80::216:3eff:fe52:61ce/64 Scope:Link >> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >> RX packets:23474 errors:0 dropped:0 overruns:0 frame:0 >> TX packets:40601 errors:0 dropped:0 overruns:0 carrier:0 >> collisions:0 txqueuelen:1000 >> RX bytes:8214830 (7.8 MiB) TX bytes:1705458 (1.6 MiB) >> >> domU is lower thandom0 because I rebooted the VM >> >> Does any of this suggest anything to you? > I guess, peth1 is still within the bridge xenbr1. > > In a case like this, I would get a tcpdump and take a look on > vif4.1, xenbr1 and peth1. > > Consider using -e key, to log MAC address of the devices. > If you ping your DomU from outside, in "normal working" case, you > should see a echo-request on peth1, xenbr1 and vif4.1, followed by a > echo-replay. > The request should have the MAC address of your router (or host > within the same broadcast domain) as source, and the MAC address of > your DomU (as defined in it''s config and shown by ifconfig) as > destination. The replay should have the same addresses inverted. > If you don''t see echo-request on peth1, I would troubleshoot the switch. > If you don''t see echo-request on xenbr1 or vif4.1, something is > wrong with how they are participating in the bridge (try add and > remove with brctl? see dmesg errors). > If you don''t see echo-request on eth1 in DomU, while they are > visible on vif4.1, it must be Xen''s fault somehow. > > If you see echo-request passing through but not echo-replay, or > wrong MAC in it, commonly it has to do with routing errors. > > Greetings. > > -- > Alexandre Kouznetsov > >OK, thanks for this, I''ll do some more work and let you know. Dave> _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users >-- The problem with being cynical is you can''t keep up! -- anon. philosopher