I have a hardware here with 9 NICs (8 of them on-board) and I want to set up 8 NICs as real network interfaces, one for each xen domain. The 9th NIC should be Domain-0 which is working well. Currently I have only set up one domain which is running in bridged mode at the moment. I am currently trying to set up a second domain which uses eth2 but I didn''t succeed. I want to get the system run with 8 domains in which every guest is using one NIC. Is there some documentation or FAQ about how to set up this configuration? Cheers, Frank _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
are you using PCI passthru? If you are passing each device directly to the DomU, then it will just be standard linux networking. Being onboard though, I suspect you are dealing with some sort of realtek switch fabric that shows up as a single PCI device (roboswitch type) and you would not be able to pass that thru. Can you do an lspci and show us how those devices present themselves? Frank Luedke wrote:> I have a hardware here with 9 NICs (8 of them on-board) > and I want to set up 8 NICs as real network interfaces, > one for each xen domain. The 9th NIC should be Domain-0 > which is working well. Currently I have only set up > one domain which is running in bridged mode at the moment. > > I am currently trying to set up a second domain which > uses eth2 but I didn''t succeed. I want to get the > system run with 8 domains in which every guest is > using one NIC. > > Is there some documentation or FAQ about how to set up > this configuration? > > Cheers, > Frank > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Jason Clark wrote:> are you using PCI passthru? If you are passing each device directly to > the DomU, then it will just be standard linux networking. Being > onboard though, I suspect you are dealing with some sort of realtek > switch fabric that shows up as a single PCI device (roboswitch type) > and you would not be able to pass that thru. Can you do an lspci and > show us how those devices present themselves? > > Frank Luedke wrote: >> I have a hardware here with 9 NICs (8 of them on-board) >> and I want to set up 8 NICs as real network interfaces, >> one for each xen domain. The 9th NIC should be Domain-0 >> which is working well. Currently I have only set up >> one domain which is running in bridged mode at the moment. >> >> I am currently trying to set up a second domain which >> uses eth2 but I didn''t succeed. I want to get the >> system run with 8 domains in which every guest is >> using one NIC. >> >> Is there some documentation or FAQ about how to set up >> this configuration? >> >> Cheers, >> Frank >> >> _______________________________________________ >> Xen-users mailing list >> Xen-users@lists.xensource.com >> http://lists.xensource.com/xen-users >> >> >> > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Close :) I have the realtek chipset for Dom0 and I have Marvell Yukon for the 8 DomU. [...] 01:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E Gigabit Ethernet Controller (rev 19) 02:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E Gigabit Ethernet Controller (rev 19) 03:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E Gigabit Ethernet Controller (rev 19) 04:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E Gigabit Ethernet Controller (rev 19) 05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) 05:05.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 Gigabit Ethernet Controller (rev 13) 05:06.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 Gigabit Ethernet Controller (rev 13) 05:07.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 Gigabit Ethernet Controller (rev 13) 05:08.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 Gigabit Ethernet Controller (rev 13) [...] So it should work when I set up my networking as described in the Manual, 5.3.1 ? .fl Jason Clark wrote:> are you using PCI passthru? If you are passing each device directly to > the DomU, then it will just be standard linux networking. Being onboard > though, I suspect you are dealing with some sort of realtek switch > fabric that shows up as a single PCI device (roboswitch type) and you > would not be able to pass that thru. Can you do an lspci and show us > how those devices present themselves? > > Frank Luedke wrote: >> I have a hardware here with 9 NICs (8 of them on-board) >> and I want to set up 8 NICs as real network interfaces, >> one for each xen domain. The 9th NIC should be Domain-0 >> which is working well. Currently I have only set up >> one domain which is running in bridged mode at the moment. >> >> I am currently trying to set up a second domain which >> uses eth2 but I didn''t succeed. I want to get the >> system run with 8 domains in which every guest is >> using one NIC. >> >> Is there some documentation or FAQ about how to set up >> this configuration?_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I think those first 4 ports should pass thru, but the last ones Im not sure about because those appear to be a sub device and I haven''t had much luck passing just the sub piece. You should be able to pass 1:00.0 or 2:00.0 etc and it should show up as ethX in your DomU at which point it is 100% regular linux networking. Frank Luedke wrote:> Close :) I have the realtek chipset for Dom0 and I have > Marvell Yukon for the 8 DomU. > > [...] > 01:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E > Gigabit Ethernet Controller (rev 19) > 02:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E > Gigabit Ethernet Controller (rev 19) > 03:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E > Gigabit Ethernet Controller (rev 19) > 04:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E > Gigabit Ethernet Controller (rev 19) > 05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. > RTL-8139/8139C/8139C+ (rev 10) > 05:05.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 > Gigabit Ethernet Controller (rev 13) > 05:06.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 > Gigabit Ethernet Controller (rev 13) > 05:07.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 > Gigabit Ethernet Controller (rev 13) > 05:08.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 > Gigabit Ethernet Controller (rev 13) > [...] > > So it should work when I set up my networking as described > in the Manual, 5.3.1 ? > > .fl > > > Jason Clark wrote: > >> are you using PCI passthru? If you are passing each device directly to >> the DomU, then it will just be standard linux networking. Being onboard >> though, I suspect you are dealing with some sort of realtek switch >> fabric that shows up as a single PCI device (roboswitch type) and you >> would not be able to pass that thru. Can you do an lspci and show us >> how those devices present themselves? >> >> Frank Luedke wrote: >> >>> I have a hardware here with 9 NICs (8 of them on-board) >>> and I want to set up 8 NICs as real network interfaces, >>> one for each xen domain. The 9th NIC should be Domain-0 >>> which is working well. Currently I have only set up >>> one domain which is running in bridged mode at the moment. >>> >>> I am currently trying to set up a second domain which >>> uses eth2 but I didn''t succeed. I want to get the >>> system run with 8 domains in which every guest is >>> using one NIC. >>> >>> Is there some documentation or FAQ about how to set up >>> this configuration? >>> > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I have tested with 02:00.0 and 05:06.0 but I didn''t succeed. My config contains pci = [ ''02:00.0'' ] (or 05:06.0), but not even the link of my NIC is coming up. I did a a unbind and bind as described in 5.3.1.3 in the manual. The ID is showing up in pcislots: # [root@localhost pciback]# cat /sys/bus/pci/drivers/pciback/slots 0000:05:06.0 I am running a Fedora Core 5 here. I have edited the xenguest-install to include "pci =..." in the auto-generated config. The installer is also always creating a random MAC address - maybe this is the problem? (I will take a look at this later). Any other hints for me? Thanks Frank Jason wrote:> I think those first 4 ports should pass thru, but the last ones Im not > sure about because those appear to be a sub device and I haven''t had > much luck passing just the sub piece. You should be able to pass > 1:00.0 or 2:00.0 etc and it should show up as ethX in your DomU at > which point it is 100% regular linux networking. > > > Frank Luedke wrote: >> Close :) I have the realtek chipset for Dom0 and I have >> Marvell Yukon for the 8 DomU. >> >> [...] >> 01:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E >> Gigabit Ethernet Controller (rev 19) >> 02:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E >> Gigabit Ethernet Controller (rev 19) >> 03:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E >> Gigabit Ethernet Controller (rev 19) >> 04:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E >> Gigabit Ethernet Controller (rev 19) >> 05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. >> RTL-8139/8139C/8139C+ (rev 10) >> 05:05.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 >> Gigabit Ethernet Controller (rev 13) >> 05:06.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 >> Gigabit Ethernet Controller (rev 13) >> 05:07.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 >> Gigabit Ethernet Controller (rev 13) >> 05:08.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 >> Gigabit Ethernet Controller (rev 13) >> [...] >> >> So it should work when I set up my networking as described >> in the Manual, 5.3.1 ? >> >> .fl_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Nope, this is not working at all :( I tried an unbind using grub and an unbind via echo and pciback at runtime but the installation of my virtuals does not start (xenguest-installer.py). I always get to the point where I configure the network interface in the installer, but that''s it. Maybe the FC5 Installer is not compatible with pci binding? I am currently setting up a dummy system via bridged network. I will try to reconfigure this to use a real NIC after installation. At least I will get some more logfiles from the virtual machine :) .fl Frank Luedke wrote:> I have tested with 02:00.0 and 05:06.0 but I didn''t succeed. > My config contains pci = [ ''02:00.0'' ] (or 05:06.0), but not > even the link of my NIC is coming up. > > I did a a unbind and bind as described in 5.3.1.3 in the manual. > The ID is showing up in pcislots: > > # [root@localhost pciback]# cat /sys/bus/pci/drivers/pciback/slots > 0000:05:06.0 > > I am running a Fedora Core 5 here. I have edited the xenguest-install > to include "pci =..." in the auto-generated config. The installer is > also always creating a random MAC address - maybe this is the problem? > (I will take a look at this later). > > Any other hints for me? > > Thanks > Frank > > > Jason wrote: >> I think those first 4 ports should pass thru, but the last ones Im not >> sure about because those appear to be a sub device and I haven''t had >> much luck passing just the sub piece. You should be able to pass >> 1:00.0 or 2:00.0 etc and it should show up as ethX in your DomU at >> which point it is 100% regular linux networking. >> >> >> Frank Luedke wrote: >>> Close :) I have the realtek chipset for Dom0 and I have >>> Marvell Yukon for the 8 DomU. >>> >>> [...] >>> 01:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E >>> Gigabit Ethernet Controller (rev 19) >>> 02:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E >>> Gigabit Ethernet Controller (rev 19) >>> 03:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E >>> Gigabit Ethernet Controller (rev 19) >>> 04:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E >>> Gigabit Ethernet Controller (rev 19) >>> 05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. >>> RTL-8139/8139C/8139C+ (rev 10) >>> 05:05.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 >>> Gigabit Ethernet Controller (rev 13) >>> 05:06.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 >>> Gigabit Ethernet Controller (rev 13) >>> 05:07.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 >>> Gigabit Ethernet Controller (rev 13) >>> 05:08.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 >>> Gigabit Ethernet Controller (rev 13) >>> [...] >>> >>> So it should work when I set up my networking as described >>> in the Manual, 5.3.1 ? >>> >>> .fl > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
try exporting every PCI id associated to that card to a single DOMU and see if that works. Frank Luedke wrote:> I have tested with 02:00.0 and 05:06.0 but I didn''t succeed. > My config contains pci = [ ''02:00.0'' ] (or 05:06.0), but not > even the link of my NIC is coming up. > > I did a a unbind and bind as described in 5.3.1.3 in the manual. > The ID is showing up in pcislots: > > # [root@localhost pciback]# cat /sys/bus/pci/drivers/pciback/slots > 0000:05:06.0 > > I am running a Fedora Core 5 here. I have edited the xenguest-install > to include "pci =..." in the auto-generated config. The installer is > also always creating a random MAC address - maybe this is the problem? > (I will take a look at this later). > > Any other hints for me? > > Thanks > Frank > > > Jason wrote: > >> I think those first 4 ports should pass thru, but the last ones Im not >> sure about because those appear to be a sub device and I haven''t had >> much luck passing just the sub piece. You should be able to pass >> 1:00.0 or 2:00.0 etc and it should show up as ethX in your DomU at >> which point it is 100% regular linux networking. >> >> >> Frank Luedke wrote: >> >>> Close :) I have the realtek chipset for Dom0 and I have >>> Marvell Yukon for the 8 DomU. >>> >>> [...] >>> 01:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E >>> Gigabit Ethernet Controller (rev 19) >>> 02:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E >>> Gigabit Ethernet Controller (rev 19) >>> 03:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E >>> Gigabit Ethernet Controller (rev 19) >>> 04:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E >>> Gigabit Ethernet Controller (rev 19) >>> 05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. >>> RTL-8139/8139C/8139C+ (rev 10) >>> 05:05.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 >>> Gigabit Ethernet Controller (rev 13) >>> 05:06.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 >>> Gigabit Ethernet Controller (rev 13) >>> 05:07.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 >>> Gigabit Ethernet Controller (rev 13) >>> 05:08.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 >>> Gigabit Ethernet Controller (rev 13) >>> [...] >>> >>> So it should work when I set up my networking as described >>> in the Manual, 5.3.1 ? >>> >>> .fl >>> > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users