Hi all, I am having trouble getting a RAID card (more specifically the 3Ware 9650SE-4LPML) working on Xen (64-bit). When using a vanilla (non-Xen) kernel, the card is successfully detected by the 3w_9xxx driver. When booting a custom built xen0 kernel, including the 3w_9xxx driver, the driver is loaded (I see the driver identification in dmesg), but doesn''t seem to find any device to attach to. I.e. no additional output from the driver. I have also attempted to bundle the driver in a initrd-package, but with the same result; I see the driver being loaded but no other output from it. I took a quick look at the source code for the driver, but didn''t see any options for enabling debugging messages. Are there some know issues with 64-bit Xen and the 3w_9xxx driver? Regards, Martin Adolfsson _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Yves-Gaël Chény
2008-Apr-22 08:49 UTC
Re: [Xen-users] Problems w/ 3Ware 9650 and 64-bit Xen
Hi, with a 3w_9xxx card, I have had a lot of problem using the last release of the 2.6.18-8 xen kernel, 3.1.3 and 3.2 So i use now only the 3.1.0 and no problem at all. regards yves Martin Adolfsson a écrit :> Hi all, > > I am having trouble getting a RAID card (more specifically the 3Ware > 9650SE-4LPML) working on Xen (64-bit). > > When using a vanilla (non-Xen) kernel, the card is successfully detected > by the 3w_9xxx driver. > > When booting a custom built xen0 kernel, including the 3w_9xxx driver, > the driver is loaded (I see the driver identification in dmesg), but > doesn''t seem to find any device to attach to. I.e. no additional output > from the driver. > > I have also attempted to bundle the driver in a initrd-package, but with > the same result; I see the driver being loaded but no other output > from it. > > I took a quick look at the source code for the driver, but didn''t see > any options for enabling debugging messages. > > Are there some know issues with 64-bit Xen and the 3w_9xxx driver? > > Regards, > Martin Adolfsson > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Tue, 2008-04-22 at 10:49 +0200, Martin Adolfsson wrote:> Hi all, > > I am having trouble getting a RAID card (more specifically the 3Ware > 9650SE-4LPML) working on Xen (64-bit). > > When using a vanilla (non-Xen) kernel, the card is successfully detected > by the 3w_9xxx driver. > > When booting a custom built xen0 kernel, including the 3w_9xxx driver, > the driver is loaded (I see the driver identification in dmesg), but > doesn''t seem to find any device to attach to. I.e. no additional output > from the driver. > > I have also attempted to bundle the driver in a initrd-package, but with > the same result; I see the driver being loaded but no other output from it. > > I took a quick look at the source code for the driver, but didn''t see > any options for enabling debugging messages. > > Are there some know issues with 64-bit Xen and the 3w_9xxx driver? > > Regards, > Martin Adolfsson > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-usersI have a 9650SE-8LP on the current Centos 5.1 x86_64 Xen kernel operating with no problems. Admittedly it isn''t very busy at the moment, but it is detected no problem. The Centos kernel is 2.6.18-53.1.14.el5xen. Dmesg output: 3ware 9000 Storage Controller device driver for Linux v2.26.02.008. ACPI: PCI Interrupt 0000:0b:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:0b:00.0 to 64 scsi0 : 3ware 9000 Storage Controller 3w-9xxx: scsi0: Found a 3ware 9000 Storage Controller at 0xbab00000, IRQ: 16. 3w-9xxx: scsi0: Firmware FE9X 3.08.00.016, BIOS BE9X 3.08.00.004, Ports: 8. Vendor: AMCC Model: 9650SE-8LP DISK Rev: 3.08 Type: Direct-Access ANSI SCSI revision: 05 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
We had the same problems with a 3ware 9650 8LP, patching the kernel and some more tryes gives no positive result, so we user the 2.6.20-server XEN kernel from ubuto for Debian Etch (but 32-bit), with 64 it could work too.... Martin Adolfsson schrieb:> Hi all, > > I am having trouble getting a RAID card (more specifically the 3Ware > 9650SE-4LPML) working on Xen (64-bit). > > When using a vanilla (non-Xen) kernel, the card is successfully detected > by the 3w_9xxx driver. > > When booting a custom built xen0 kernel, including the 3w_9xxx driver, > the driver is loaded (I see the driver identification in dmesg), but > doesn''t seem to find any device to attach to. I.e. no additional output > from the driver. > > I have also attempted to bundle the driver in a initrd-package, but with > the same result; I see the driver being loaded but no other output > from it. > > I took a quick look at the source code for the driver, but didn''t see > any options for enabling debugging messages. > > Are there some know issues with 64-bit Xen and the 3w_9xxx driver? > > Regards, > Martin Adolfsson > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On April 22, 2008 01:49 am Martin Adolfsson wrote:> I am having trouble getting a RAID card (more specifically the 3Ware > 9650SE-4LPML) working on Xen (64-bit).I fought with this setup for over three weeks before giving up on Xen (at least on Debian/Ubuntu) on newer hardware. The driver for this card was added to the mainline kernel in 2.6.20, hence the standard Xen kernel 2.6.18 won''t work. I tried running Debian Lenny (with the 2.6.22-xen kernel from Ubuntu), Debian Sid, Ubuntu Hardy, and a bunch of different variations on the above. The only semi-stable configuration I achieved was Ubuntu Hardy with the 2.6.24 dom0 kernel, and a forced install of the Gutsy 2.6.22-xen kernel for the domU. But that was only semi-stable. Running lots of HVMs and PVs at the same time could lock the server. And depending on which version of the libc6 package was installed, you could get lots of "bus error" messages and truncated libraries. Plus, getting networking to work on eth3-eth6 (and not on eth0-eth2) in Xen was a nightmare, and made even worse if you tried to create a bond0 interface of eth3-eth6 and use that for the Xen bridge (never managed to get that to work with Xen 3.2). After three weeks, I gave up and installed Debian Lenny, upgraded to the 2.6.24 kernel, install KVM, and haven''t looked at Xen since. Had a working VM setup in less than a day, with a 4-port bond0 as the physical device for the bridge. My experience has shown (at least for Debian hosts): - if your hardware is supported by 2.6.18, then you can run Xen 3.0 nicely, 3.1 nicely, and 3.2 nicely (once you get around that stupid XenStore crap) - if your hardware requires a newer kernel, and supports hardware virtualisation, you''re better off with KVM. It''s a lot simpler to manage (a couple of shell scripts and the standard Linux admin tools), you can use the standard Debian tools to set up your bonded interfaces and bridges, and you always have a VNC terminal to access to the console of the VMs. With the development of paravirtual drivers for net/block I/O, it''ll really give Xen a run. Our hardware: Tyan h2000M motherboard 2x AMD Opteron 2200-series CPUs (dual-core, 2 GHz) 8 GB DDR2-800 SDRAM 3Ware 9650SE-12ML PCIe RAID controllers 12x 500 GB SATA-II HDs in an 11-disk RAID6 w/Hot Spare Intel Pro/1000 quad-port PCIe NIC Chenbro 5U rackmount case 1350 watt PSU with 4-redundant connectors/power modules -- Freddie Cash fjwcash@gmail.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Martin Adolfsson
2008-Apr-25 05:01 UTC
Re: [Xen-users] Problems w/ 3Ware 9650 and 64-bit Xen
Thank you for your reply. MS wrote:> We had the same problems with a 3ware 9650 8LP, > patching the kernel and some more tryes gives no positive result,Ok - you are referring to the 3Ware 9xxx driver for 2.6.18? (Downloaded from 3ware.com separately) I did try that, but it resulted in a kernel crash during the boot sequence.> so we user the 2.6.20-server XEN kernel from ubuto for Debian Etch (but > 32-bit), > with 64 it could work too....Interesting, is that the unstable version of Xen? I was under the impressions that the latest Xen-based kernel was 2.6.18, or is there somewhere I can fetch a stable release of Xen based on 2.6.20? Thanks in advance. //Martin _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Martin Adolfsson
2008-Apr-25 05:16 UTC
Re: [Xen-users] Problems w/ 3Ware 9650 and 64-bit Xen
Hi, Freddie Cash wrote:> On April 22, 2008 01:49 am Martin Adolfsson wrote: >> I am having trouble getting a RAID card (more specifically the 3Ware >> 9650SE-4LPML) working on Xen (64-bit). > > The driver for this card was added to the mainline kernel in 2.6.20, hence > the standard Xen kernel 2.6.18 won''t work.Ah, I missed that earlier. Now, however, I''ve downloaded the driver from 3ware.com, for kernel 2.6.18, but that seems to result in a kernel crash during the detection phase. :-\> I tried running Debian Lenny (with the 2.6.22-xen kernel from Ubuntu), > Debian Sid, Ubuntu Hardy, and a bunch of different variations on the > above. > > The only semi-stable configuration I achieved was Ubuntu Hardy with the > 2.6.24 dom0 kernel, and a forced install of the Gutsy 2.6.22-xen kernel > for the domU. But that was only semi-stable. Running lots of HVMs and > PVs at the same time could lock the server.That doesn''t sound too promising at all. :-\ Did you try using the 3Ware supplied 2.6.18 driver?> And depending on which version of the libc6 package was installed, you > could get lots of "bus error" messages and truncated libraries. Plus,Nasty.> getting networking to work on eth3-eth6 (and not on eth0-eth2) in Xen was > a nightmare, and made even worse if you tried to create a bond0 interface > of eth3-eth6 and use that for the Xen bridge (never managed to get that > to work with Xen 3.2).Yes, I use a fairly complex setup with a firewall in a domU, multiple interfaces and a bridged/routed setup. Basically, I threw out all the bundled xen-networking scripts and wrote new ones from scratch.> After three weeks, I gave up and installed Debian Lenny, upgraded to the > 2.6.24 kernel, install KVM, and haven''t looked at Xen since. Had a > working VM setup in less than a day, with a 4-port bond0 as the physical > device for the bridge.OK - I''ll take that into consideration, but given the time frames of this project, I probably have to stick with Xen one way or another. :) //Martin _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Martin Adolfsson
2008-Apr-25 05:21 UTC
Re: [Xen-users] Problems w/ 3Ware 9650 and 64-bit Xen
Hi, Mike Brady wrote:> I have a 9650SE-8LP on the current Centos 5.1 x86_64 Xen kernel > operating with no problems. Admittedly it isn''t very busy at the > moment, but it is detected no problem. > > The Centos kernel is 2.6.18-53.1.14.el5xen. > > Dmesg output: > > 3ware 9000 Storage Controller device driver for Linux v2.26.02.008. > ACPI: PCI Interrupt 0000:0b:00.0[A] -> GSI 16 (level, low) -> IRQ 16 > PCI: Setting latency timer of device 0000:0b:00.0 to 64 > scsi0 : 3ware 9000 Storage Controller > 3w-9xxx: scsi0: Found a 3ware 9000 Storage Controller at 0xbab00000, > IRQ: 16. > 3w-9xxx: scsi0: Firmware FE9X 3.08.00.016, BIOS BE9X 3.08.00.004, Ports: > 8. > Vendor: AMCC Model: 9650SE-8LP DISK Rev: 3.08 > Type: Direct-Access ANSI SCSI revision: 05Interesting. The driver version stated by your output is "v2.26.02.008". The version reported by my driver downloaded from 3ware.com is "2.26.06.003-2.6.18". (My drivers history states that 9650SE support was added in 2.26.06.001.) Do you know if that driver is the one bundled with the standard kernel, or one that has been applied afterwards by centos? //Martin _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
There is an bug in the 3Ware driver for the Kernel 2.6.18. You can patch it out, but it brings some other problems. Xen + 3Ware 96xx and newer are a problem with xen, we have a few xen machines with this problem, so we use the 2.6.20 Ubutu kernel. A better choice for xen are the Adaptec controllers 3xxx and 5xxx. Martin Adolfsson schrieb:> Thank you for your reply. > > MS wrote: >> We had the same problems with a 3ware 9650 8LP, >> patching the kernel and some more tryes gives no positive result, > > Ok - you are referring to the 3Ware 9xxx driver for 2.6.18? > (Downloaded from 3ware.com separately) I did try that, but it resulted > in a kernel crash during the boot sequence. > >> so we user the 2.6.20-server XEN kernel from ubuto for Debian Etch >> (but 32-bit), >> with 64 it could work too.... > > Interesting, is that the unstable version of Xen? I was under the > impressions that the latest Xen-based kernel was 2.6.18, or is there > somewhere I can fetch a stable release of Xen based on 2.6.20? > > Thanks in advance. > > //Martin > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Fri, 2008-04-25 at 07:21 +0200, Martin Adolfsson wrote:> Hi, > > Mike Brady wrote: > > I have a 9650SE-8LP on the current Centos 5.1 x86_64 Xen kernel > > operating with no problems. Admittedly it isn''t very busy at the > > moment, but it is detected no problem. > > > > The Centos kernel is 2.6.18-53.1.14.el5xen. > > > > Dmesg output: > > > > 3ware 9000 Storage Controller device driver for Linux v2.26.02.008. > > ACPI: PCI Interrupt 0000:0b:00.0[A] -> GSI 16 (level, low) -> IRQ 16 > > PCI: Setting latency timer of device 0000:0b:00.0 to 64 > > scsi0 : 3ware 9000 Storage Controller > > 3w-9xxx: scsi0: Found a 3ware 9000 Storage Controller at 0xbab00000, > > IRQ: 16. > > 3w-9xxx: scsi0: Firmware FE9X 3.08.00.016, BIOS BE9X 3.08.00.004, Ports: > > 8. > > Vendor: AMCC Model: 9650SE-8LP DISK Rev: 3.08 > > Type: Direct-Access ANSI SCSI revision: 05 > > Interesting. The driver version stated by your output is "v2.26.02.008". > The version reported by my driver downloaded from 3ware.com is > "2.26.06.003-2.6.18". (My drivers history states that 9650SE support was > added in 2.26.06.001.) > > Do you know if that driver is the one bundled with the standard kernel, > or one that has been applied afterwards by centos? > > //Martin >Sorry I don''t know the source of the driver other than it is whatever comes with Centos 5.1. I have only had the box a few weeks. The first 9650 I got was dead. It wouldn''t even come up to the BIOS. I upgraded the motherboard and 9650 BIOS before I did anything and the replacement has just worked with Centos 5.1, so I haven''t had cause to look into it any further. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On April 24, 2008 10:16 pm Martin Adolfsson wrote:> Freddie Cash wrote: > > On April 22, 2008 01:49 am Martin Adolfsson wrote: > > I tried running Debian Lenny (with the 2.6.22-xen kernel from > > Ubuntu), Debian Sid, Ubuntu Hardy, and a bunch of different > > variations on the above. > > > > The only semi-stable configuration I achieved was Ubuntu Hardy with > > the 2.6.24 dom0 kernel, and a forced install of the Gutsy 2.6.22-xen > > kernel for the domU. But that was only semi-stable. Running lots of > > HVMs and PVs at the same time could lock the server. > > That doesn''t sound too promising at all. :-\ Did you try using the > 3Ware supplied 2.6.18 driver?No, never tried that. Have since moved the box to plain 64-bit Debian Lenny using KVM to run (currently) 6 HVMs, with 4 more in various stages of testing (all using kvmbr0 bridge device on top of a bond0 made up of 4 gigabit NICs).> > And depending on which version of the libc6 package was installed, > > you could get lots of "bus error" messages and truncated libraries. > > Plus, > > Nasty.Yeah, the Ubuntu launchpad and forums for hardy are full of libc6 issues regarding bus errors and truncated libraries, with no real solution as yet (that I saw).> > getting networking to work on eth3-eth6 (and not on eth0-eth2) in Xen > > was a nightmare, and made even worse if you tried to create a bond0 > > interface of eth3-eth6 and use that for the Xen bridge (never managed > > to get that to work with Xen 3.2).> Yes, I use a fairly complex setup with a firewall in a domU, multiple > interfaces and a bridged/routed setup. Basically, I threw out all the > bundled xen-networking scripts and wrote new ones from scratch.What''s annoying is I was able to do the same using the supplied networking scripts in Xen 3.0.3 in Debian Etch. Something changed between that and Xen 3.2, but I was never able to figure it out. Tried a couple of times to figure out all the ip, brctl, and ifconfig commands that were going on in the background, but gave up in frustration (oh, if only a distro had the balls to unify networking commands like the BSDs do, such that ifconfig is the only command you need to do all this).> > After three weeks, I gave up and installed Debian Lenny, upgraded to > > the 2.6.24 kernel, install KVM, and haven''t looked at Xen since. Had > > a working VM setup in less than a day, with a 4-port bond0 as the > > physical device for the bridge. > > OK - I''ll take that into consideration, but given the time frames of > this project, I probably have to stick with Xen one way or another. :)Good luck. Hope you get this one figured out. -- Freddie Cash fjwcash@gmail.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
My domU cannot communicate with anything other than the dom0. The dom0
has no problems with any kind of communications.
I have seen some posts that imply that xenbr0 should have an IP address,
however, I cannot verify that.
What works:
Ping Dom0->DomU
Ping DomU->Dom0
SSH, etc Dom0->DomU
SSH, etc DomU->Dom0
Anything Dom0->(internet, intranet)
What doesn''t Work:
Anything DomU->(internet, intranet, including gateway)
Some Information:
Dom0 IP: 10.0.0.12
DomU IP: 10.0.0.13
Gateway IP: 10.0.0.1
Below are some config and status snapshots:
[root@xen ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:1D:09:29:80:41
inet addr:10.0.0.12 Bcast:10.0.0.255 Mask:255.255.255.0
inet6 addr: fe80::21d:9ff:fe29:8041/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:694 errors:0 dropped:0 overruns:0 frame:0
TX packets:86 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:65309 (63.7 KiB) TX bytes:19087 (18.6 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:2211 errors:0 dropped:0 overruns:0 frame:0
TX packets:2211 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2625224 (2.5 MiB) TX bytes:2625224 (2.5 MiB)
peth0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:774 errors:0 dropped:0 overruns:0 frame:0
TX packets:206 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:73258 (71.5 KiB) TX bytes:26937 (26.3 KiB)
Interrupt:16 Memory:f8000000-f8012100
vif0.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:98 errors:0 dropped:0 overruns:0 frame:0
TX packets:700 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:21671 (21.1 KiB) TX bytes:65669 (64.1 KiB)
vif1.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:129 errors:0 dropped:0 overruns:0 frame:0
TX packets:532 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:3612 (3.5 KiB) TX bytes:45994 (44.9 KiB)
xenbr0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:650 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:52872 (51.6 KiB) TX bytes:0 (0.0 b)
[root@xen ~]# brctl show
bridge name bridge id STP enabled interfaces
xenbr0 8000.feffffffffff no vif1.0
peth0
vif0.0
[root@xen ~]# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere PHYSDEV
match --physdev-in vif1.0
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
[root@xen ~]# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
10.0.0.0 * 255.255.255.0 U 0 0 0
eth0
169.254.0.0 * 255.255.0.0 U 0 0 0
eth0
default 10.0.0.1 0.0.0.0 UG 0 0 0
eth0
from everything I know, this looks like it should work.
Any ideas?
Thanks,
Stuart
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Stuart Rench
2008-Apr-25 20:12 UTC
RE: [Xen-users] UPATED xenbr0 doesn''t have an IP (should it?!)
So now I am convinced that something in iptables and nat has gone
awry...but I am EXTREMELY weak on IPTABLES...
If I were to flush all dom0 iptables to start from scratch, what is a
bare minimum to allow for the following basic network architecture?
Gateway - 10.0.0.1
XenServer - 10.0.0.12
Virtual Server - 10.0.0.13
Anyone else on my network - 10.0.0.x
Thanks in advance!
/Stuart
-----Original Message-----
From: Stuart Rench
Sent: Friday, April 25, 2008 2:06 PM
To: Stuart Rench
Subject: RE: [Xen-users] xenbr0 doesn''t have an IP (should it?!)
A little more info...
I may have isolated the root problem, but don''t know the cause...
A 3rd party machine tries to arping and sees this in a tcpdump:
13:22:06.939437 arp who-has 10.0.0.13 tell 10.0.0.11
13:22:07.939442 arp who-has 10.0.0.13 tell 10.0.0.11
13:22:08.939446 arp who-has 10.0.0.13 tell 10.0.0.11
never any answers....10.0.0.12 (the host) who knows where 10.0.0.13 is
sees this in a tcpdump:
13:21:21.917162 arp who-has 10.0.0.13 tell 10.0.0.11
13:21:21.917201 arp reply 10.0.0.13 is-at 00:16:3e:5a:e7:02 (oui
Unknown)
13:21:22.917172 arp who-has 10.0.0.13 tell 10.0.0.11
13:21:22.917209 arp reply 10.0.0.13 is-at 00:16:3e:5a:e7:02 (oui
Unknown)
13:21:23.917169 arp who-has 10.0.0.13 tell 10.0.0.11
13:21:23.917203 arp reply 10.0.0.13 is-at 00:16:3e:5a:e7:02 (oui
Unknown)
but if i arping 10.0.0.12 from 10.0.0.11, i see the response in
10.0.0.12:
13:22:31.683323 arp reply 10.0.0.12 is-at 00:1d:09:29:80:41 (oui
Unknown)
so this leads me to believe that the host is routing (or blocking) the
fwding of arp....i turned iptables off and i don''t see any change.
overview, 10.0.0.13 is my virtual host
10.0.0.12 is the Xen virtualization server
10.0.0.11 is a real server on the network....
-----Original Message-----
From: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Stuart Rench
Sent: Friday, April 25, 2008 12:35 PM
To: xen-users@lists.xensource.com
Subject: [Xen-users] xenbr0 doesn''t have an IP (should it?!)
My domU cannot communicate with anything other than the dom0. The dom0
has no problems with any kind of communications.
I have seen some posts that imply that xenbr0 should have an IP address,
however, I cannot verify that.
What works:
Ping Dom0->DomU
Ping DomU->Dom0
SSH, etc Dom0->DomU
SSH, etc DomU->Dom0
Anything Dom0->(internet, intranet)
What doesn''t Work:
Anything DomU->(internet, intranet, including gateway)
Some Information:
Dom0 IP: 10.0.0.12
DomU IP: 10.0.0.13
Gateway IP: 10.0.0.1
Below are some config and status snapshots:
[root@xen ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:1D:09:29:80:41
inet addr:10.0.0.12 Bcast:10.0.0.255 Mask:255.255.255.0
inet6 addr: fe80::21d:9ff:fe29:8041/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:694 errors:0 dropped:0 overruns:0 frame:0
TX packets:86 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:65309 (63.7 KiB) TX bytes:19087 (18.6 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:2211 errors:0 dropped:0 overruns:0 frame:0
TX packets:2211 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2625224 (2.5 MiB) TX bytes:2625224 (2.5 MiB)
peth0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:774 errors:0 dropped:0 overruns:0 frame:0
TX packets:206 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:73258 (71.5 KiB) TX bytes:26937 (26.3 KiB)
Interrupt:16 Memory:f8000000-f8012100
vif0.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:98 errors:0 dropped:0 overruns:0 frame:0
TX packets:700 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:21671 (21.1 KiB) TX bytes:65669 (64.1 KiB)
vif1.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:129 errors:0 dropped:0 overruns:0 frame:0
TX packets:532 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:3612 (3.5 KiB) TX bytes:45994 (44.9 KiB)
xenbr0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:650 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:52872 (51.6 KiB) TX bytes:0 (0.0 b)
[root@xen ~]# brctl show
bridge name bridge id STP enabled interfaces
xenbr0 8000.feffffffffff no vif1.0
peth0
vif0.0
[root@xen ~]# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere PHYSDEV
match --physdev-in vif1.0
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
[root@xen ~]# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
10.0.0.0 * 255.255.255.0 U 0 0 0
eth0
169.254.0.0 * 255.255.0.0 U 0 0 0
eth0
default 10.0.0.1 0.0.0.0 UG 0 0 0
eth0
from everything I know, this looks like it should work.
Any ideas?
Thanks,
Stuart
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
John Haxby
2008-Apr-28 09:02 UTC
Re: [Xen-users] UPATED xenbr0 doesn''t have an IP (should it?!)
Stuart Rench wrote:> So now I am convinced that something in iptables and nat has gone > awry...but I am EXTREMELY weak on IPTABLES... > > If I were to flush all dom0 iptables to start from scratch, what is a > bare minimum to allow for the following basic network architecture? > > Gateway - 10.0.0.1 > XenServer - 10.0.0.12 > Virtual Server - 10.0.0.13 > > Anyone else on my network - 10.0.0.x >The main thing that affects traffic to and from domU in dom0 is the FORWARD chain in the filter table: if you flush this (iptables -F FORWARD) then the usual default policy is ACCEPT which means that traffic can be forwarded. The default rule that permits traffic from some source vifX.0 phydev is only needed when the table''s policy is not ACCEPT or when there is some other rule in the FORWARD chain that rejects traffic. You might find "iptables -I FORWARD 1 -j LOG" useful, although, be warned, this can generate a _lot_ of messages that will wind up in /var/log/messages, but you will be able to see what traffic iptables is seeing on that chain. It''s also possible that you have rules in some other table that are causing you trouble; running iptables-save will show you all the rules in all the chains in all the tables. You may have something odd in the nat table that is giving you grief. jch _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users