Hi, I''m trying to pass through 3 NICs to a domU, and i have some problems with it I have hid the devices from the dom0 with the following kernel parameter: xen-pciback.hide=(03:01.0)(06:01.0)(06:02.0)(06:1e.0) This does hide the devices from the dom0, and they show up as assignable devices # xl pci-list-assignable-devices 0000:03:01.0 0000:06:01.0 0000:06:02.0 0000:06:1e.0 If at this moment i have a guest up and running, i can use the xl pci-attach command to add the devices to the guest, and that way everything works fine, lspci lists the network cards, and i can use them - they work as they supposed to. However, when i try to add the devices via the domU config file (pci = [ ''03:01.0'', ''06:01.0'', ''06:02.0'' ]), only the first device show up in the guest (the order here doesn''t matter, always the first device shows up). Reviewing the guest''s dmesg, i can''t see anything about the other devices, they don''t show up in lspci, but they are no longer assignable according to the dom0. I have to detach and reattach them with xl to be able to use them in the guest. What am i doing wrong? Regards, Gabor Gudiczius p.s.: i''m using gentoo with gentoo-sources 3.3.8 for both the dom0 and the domU, pci backend and frontend drivers are compiled into the kernel
On Mon, 2012-08-06 at 19:44 +0100, Gabor Hudiczius wrote:> Hi, > > I''m trying to pass through 3 NICs to a domU, and i have some problems > with it > I have hid the devices from the dom0 with the following kernel > parameter: xen-pciback.hide=(03:01.0)(06:01.0)(06:02.0)(06:1e.0) > This does hide the devices from the dom0, and they show up as assignable > devices > # xl pci-list-assignable-devices > 0000:03:01.0 > 0000:06:01.0 > 0000:06:02.0 > 0000:06:1e.0 > > If at this moment i have a guest up and running, i can use the xl > pci-attach command to add the devices to the guest, and that way > everything works fine, lspci lists the network cards, and i can use them > - they work as they supposed to. > However, when i try to add the devices via the domU config file (pci = [ > ''03:01.0'', ''06:01.0'', ''06:02.0'' ]), only the first device show up in the > guest (the order here doesn''t matter, always the first device shows up). > Reviewing the guest''s dmesg, i can''t see anything about the other > devices, they don''t show up in lspci, but they are no longer assignable > according to the dom0. I have to detach and reattach them with xl to be > able to use them in the guest. > > What am i doing wrong? > > Regards, > Gabor Gudiczius > > p.s.: i''m using gentoo with gentoo-sources 3.3.8 for both the dom0 and > the domU, pci backend and frontend drivers are compiled into the kernelWhich version of xen though?
Hi,> Which version of xen though?Xen 4.1.1 with xen-tools 4.1.1 (latest stable versions in gentoo portage) regards, Gabor Hudiczius
On Tue, 2012-08-07 at 09:11 +0100, Gabor Hudiczius wrote:> Hi, > > > > Which version of xen though? > > Xen 4.1.1 with xen-tools 4.1.1 (latest stable versions in gentoo portage)xl in 4.1 was really more of a preview, and in particular I don''t think the PCI passthrough stuff was fully baked. I''d suggest that for 4.1 you continue to use xm for now. It would be good if you could confirm that 4.2-rc1 works for you (or not) though, since it should all be good there. Ian.
Hi, thanks, it works perfectly with the latest 4.2 (4.2.0-rc2-pre) Regards, Gabor Hudiczius On 12.08.07. 10:24, Ian Campbell wrote:> On Tue, 2012-08-07 at 09:11 +0100, Gabor Hudiczius wrote: >> Hi, >> >> >>> Which version of xen though? >> >> Xen 4.1.1 with xen-tools 4.1.1 (latest stable versions in gentoo portage) > > xl in 4.1 was really more of a preview, and in particular I don''t think > the PCI passthrough stuff was fully baked. > > I''d suggest that for 4.1 you continue to use xm for now. > > It would be good if you could confirm that 4.2-rc1 works for you (or > not) though, since it should all be good there. > > Ian. >
On Wed, 2012-08-08 at 11:17 +0100, Gabor Hudiczius wrote:> thanks, it works perfectly with the latest 4.2 (4.2.0-rc2-pre)Good! Thanks for letting us know. Ian.
> thanks, it works perfectly with the latest 4.2 (4.2.0-rc2-pre)Hi, well, it''s not quite perfect... When i try to set a maxmem value different from the memory value, the guest doesn''t start Other guests work fine, and this one starts too, when i delete either the pci or the maxmem settings, or when i set the maxmem value equal to the memory value. I attach the guest config (rita), the xl log (xl-rita.log), and the output of both a successful (rita-start-success) and a failed (rita-start-failure) create attempt (xl -vvv create rita). Both xm create and xm mem-set works. Regards, Gabor Hudiczius _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
On Fri, 2012-08-10 at 18:03 +0100, Gabor Hudiczius wrote:> > thanks, it works perfectly with the latest 4.2 (4.2.0-rc2-pre) > > Hi, > > well, it''s not quite perfect... > > When i try to set a maxmem value different from the memory value, the > guest doesn''t startI think this is probably/likely a kernel issue (so I''ve CC''d Konrad) Setting "host_e820=1" in the guest cfg might help, but I''ve a feeling it happens automagically if you specify "pci = " so perhaps not. You might need swiotlb=force too, Konrad can probably best advise. I''d be useful to see the actual kernel logs from both the success and failure case. I set "XENCONSOLED_TRACE=guest" in /etc/{sysconfig,default}/xencommon (which one you have depends on distro) and mkdir /var/log/xen/console, after restarting xenconsoled you should get a log file for each guest.> Other guests work fine, and this one starts too, when i delete either > the pci or the maxmem settings, or when i set the maxmem value equal to > the memory value. > > I attach the guest config (rita), the xl log (xl-rita.log), and the > output of both a successful (rita-start-success) and a failed > (rita-start-failure) create attempt (xl -vvv create rita). > > Both xm create and xm mem-set works. > > Regards, > Gabor Hudiczius
Hi,
> I think this is probably/likely a kernel issue (so I''ve
CC''d Konrad)
> Setting "host_e820=1" in the guest cfg might help, but
I''ve a feeling
> it happens automagically if you specify "pci = " so perhaps not.
>
> You might need swiotlb=force too, Konrad can probably best advise.
unfortunately neither of those options helped
> I''d be useful to see the actual kernel logs from both the success
and
> failure case. I set "XENCONSOLED_TRACE=guest" in
> /etc/{sysconfig,default}/xencommon (which one you have depends on
> distro) and mkdir /var/log/xen/console, after restarting xenconsoled
> you should get a log file for each guest.
i''m on gentoo, and i don''t have that configuration file
i have to edit /etc/conf.d/xenconsoled, and add "--log=guest
--log-dir=/var/log/xen/console/" to the XENCONSOLED_OPTS variable
logs attached (i hope, i didn''t mix them up):
successful start with pci passthrough enabled and maxmem equal to memory:
rita-start-success-passthrough.txt
xl-rita.log-success-passthrough.txt
guest-rita.log-success-passthrough.txt
successful start with maxmem different from memory and pci passthrough
disabled:
rita-start-success-maxmem.txt
xl-rita.log-success-maxmem.txt
guest-rita.log-success-maxmem.txt
failed starts with pci passthrough enabled and maxmem different from
memory - the console log were empty (0 Bytes) in these cases:
without host_e820=1 or swiotlb=force:
rita-start-failure-passthrough-maxmem.txt
xl-rita.log-failure-passthrough-maxmem.txt
with swiotlb=force:
rita-start-failure-passthrough-maxmem-swiotlb.txt
xl-rita.log-failure-passthrough-maxmem-swiotlb.txt
with host_e820=1:
rita-start-failure-passthrough-maxmem-hoste820.txt
xl-rita.log-failure-passthrough-maxmem-hoste820.txt
with swiotlb=force and host_e820=1:
rita-start-failure-passthrough-maxmem-hoste820-swiotlb.txt
xl-rita.log-failure-passthrough-maxmem-hoste820-swiotlb.txt
Regards,
Gabor Hudiczius
_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
On Tue, Aug 14, 2012 at 11:27:45AM +0100, Ian Campbell wrote:> On Fri, 2012-08-10 at 18:03 +0100, Gabor Hudiczius wrote: > > > thanks, it works perfectly with the latest 4.2 (4.2.0-rc2-pre) > > > > Hi, > > > > well, it''s not quite perfect... > > > > When i try to set a maxmem value different from the memory value, the > > guest doesn''t start > > I think this is probably/likely a kernel issue (so I''ve CC''d Konrad) > Setting "host_e820=1" in the guest cfg might help, but I''ve a feeling itIt is called ''e820_host=1''> happens automagically if you specify "pci = " so perhaps not.No. You still need to provide e820_host=1 and pci=xxx stanze for that to work.> > You might need swiotlb=force too, Konrad can probably best advise.iommu=soft.> > I''d be useful to see the actual kernel logs from both the success and > failure case. I set "XENCONSOLED_TRACE=guest" > in /etc/{sysconfig,default}/xencommon (which one you have depends on > distro) and mkdir /var/log/xen/console, after restarting xenconsoled you > should get a log file for each guest. > > > > Other guests work fine, and this one starts too, when i delete either > > the pci or the maxmem settings, or when i set the maxmem value equal to > > the memory value.I have to confess that I don''t recall testing maxmem/memory without e820_host option as well, the e820_host=1 is the option to make it work. Without the e820_host the E820 that is put in the guest is not very PCI friendly and you are probably populating the memory that the PCI card is using.> > > > I attach the guest config (rita), the xl log (xl-rita.log), and the > > output of both a successful (rita-start-success) and a failed > > (rita-start-failure) create attempt (xl -vvv create rita). > > > > Both xm create and xm mem-set works.I should mention that ''e820_host'' option is only available in ''xl''. I did write a patch for xm (see attached) but since xend is deprecated did not think to post it.> > > > Regards, > > Gabor Hudiczius >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Hi!> It is called ''e820_host=1''i had the same result (logs attached)> iommu=soft.i already had that in the config file (also attached)> I should mention that ''e820_host'' option is only available in ''xl''. > I did write a patch for xm (see attached) but since xend is deprecated > did not think to post it.i''m using xl, and xen 4.2.0-rc3-pre i had some problems with 4.1.1 (which did not attach the devices when i was using xl) Regards, Gabor Hudiczius _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users