I think this is a bug in the xl toolchain. lspci -s 0000:12:0.* 12:00.0 Display controller: ATI Technologies Inc Device 671d 12:00.1 Audio device: ATI Technologies Inc Device aa80 sudo xm pci-list-assignable-devices 0000:13:00.0 0000:13:00.1 0000:12:00.0 0000:12:00.1 So there is a multi-function device (actually one ASIC of a AMD Radeon 6990 card). It has two function addresses 0 = Display controller, 1 = Audio Device. However when running xl create where the line for pci is of the form pci=[''12:00.*''] Where * means pass through all devices, you get an error as below. Parsing config file xenwin7-1.cfg xc: info: VIRTUAL MEMORY ARRANGEMENT: Loader: 0000000000100000->0000000000179830 TOTAL: 0000000000000000->000000007f800000 ENTRY ADDRESS: 0000000000100000 xc: info: PHYSICAL MEMORY ALLOCATION: 4KB PAGES: 0x0000000000000200 2MB PAGES: 0x00000000000003fb 1GB PAGES: 0x0000000000000000 libxl: error: libxl_pci.c:790:libxl__device_pci_add PCI device 0:12:0.70 is not assignable There is no function 70 on that device. In relation to that, the BDF documentation (http://wiki.xensource.com/xenwiki/BDFNotation) seems to imply that you can specify the extended BDF notation on your kernel boot line in grub. i.e. xen-pciback.hide=(0000:12:00.*) does not work. You''ll get an error in the kernel log. Actually, if you look at the code for pci_stub.c in the linux kernel (at least for Jeremy branch 2.6.32.48), extended BDF is not supported. Matt
On Fri, 2012-02-17 at 22:56 +0000, Matthew Hook wrote:> I think this is a bug in the xl toolchain.I agree. Which version of Xen are you running? There have been various fixes to the stuff in xen-unstable but from your error message (which doesn''t appear in the unstable code base AFAICT) I guess you are running 4.1. Are you able to try unstable?> lspci -s 0000:12:0.* > 12:00.0 Display controller: ATI Technologies Inc Device 671d > 12:00.1 Audio device: ATI Technologies Inc Device aa80 > > sudo xm pci-list-assignable-devicesNB: it is not recommended to mix and match xm and xl.> 0000:13:00.0 > 0000:13:00.1 > 0000:12:00.0 > 0000:12:00.1 > > So there is a multi-function device (actually one ASIC of a AMD Radeon > 6990 card). > It has two function addresses 0 = Display controller, 1 = Audio Device. > > However when running xl create where the line for pci is of the form > pci=[''12:00.*''] > Where * means pass through all devices, you get an error as below. > > Parsing config file xenwin7-1.cfg > xc: info: VIRTUAL MEMORY ARRANGEMENT: > Loader: 0000000000100000->0000000000179830 > TOTAL: 0000000000000000->000000007f800000 > ENTRY ADDRESS: 0000000000100000 > xc: info: PHYSICAL MEMORY ALLOCATION: > 4KB PAGES: 0x0000000000000200 > 2MB PAGES: 0x00000000000003fb > 1GB PAGES: 0x0000000000000000 > libxl: error: libxl_pci.c:790:libxl__device_pci_add PCI device > 0:12:0.70 is not assignable > > There is no function 70 on that device.The * turns into a "LIBXL_PCI_FUNC_ALL (~0U)" in the internal represenation, I bet this is some sort unhelpful and confusing of formatting snafu related to that.> In relation to that, the BDF documentation > (http://wiki.xensource.com/xenwiki/BDFNotation) seems to imply that > you can specify > the extended BDF notation on your kernel boot line in grub. i.e. > xen-pciback.hide=(0000:12:00.*) does not work. You''ll get an error in > the kernel log.That is the old wiki, the new page seems to be http://wiki.xen.org/wiki/Bus:Device.Function_(BDF)_Notation -- does this still give you that impression? Please feel free to update to clarify.> Actually, if you look at the code for pci_stub.c in the linux kernel > (at least for Jeremy branch 2.6.32.48), extended BDF is not supported.Right, I think that wiki page is trying to refer only to the notation used by the toolstack. I imagine that could be clearer. Ian.
On 20 February 2012 08:45, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Fri, 2012-02-17 at 22:56 +0000, Matthew Hook wrote: > > I think this is a bug in the xl toolchain. > > I agree. > > Which version of Xen are you running? There have been various fixes to > the stuff in xen-unstable but from your error message (which doesn''t > appear in the unstable code base AFAICT) I guess you are running 4.1. > Are you able to try unstable?I''m running release Xen 4.1.2 with some AMD graphics patches applied. This is the hypervisor installed onto Ubuntu 10.04 LTS with Linux Kernel from the jeremy branch 2.6.32.48> lspci -s 0000:12:0.* > > 12:00.0 Display controller: ATI Technologies Inc Device 671d > > 12:00.1 Audio device: ATI Technologies Inc Device aa80 > > > > sudo xm pci-list-assignable-devices > > NB: it is not recommended to mix and match xm and xl. > > I found some odd behaviour with XL in that it wouldn''t clean up thedomains properly after shutting down. I wasn''t sure whether it was a bug in XL or related to the AMD passthru graphics patches that I have applied. Using XM and I wasn''t having that issue. I will be testing this without the patches today to narrow that issue down.> > 0000:13:00.0 > > 0000:13:00.1 > > 0000:12:00.0 > > 0000:12:00.1 > > > > So there is a multi-function device (actually one ASIC of a AMD Radeon > > 6990 card). > > It has two function addresses 0 = Display controller, 1 = Audio Device. > > > > However when running xl create where the line for pci is of the form > > pci=[''12:00.*''] > > Where * means pass through all devices, you get an error as below. > > > > Parsing config file xenwin7-1.cfg > > xc: info: VIRTUAL MEMORY ARRANGEMENT: > > Loader: 0000000000100000->0000000000179830 > > TOTAL: 0000000000000000->000000007f800000 > > ENTRY ADDRESS: 0000000000100000 > > xc: info: PHYSICAL MEMORY ALLOCATION: > > 4KB PAGES: 0x0000000000000200 > > 2MB PAGES: 0x00000000000003fb > > 1GB PAGES: 0x0000000000000000 > > libxl: error: libxl_pci.c:790:libxl__device_pci_add PCI device > > 0:12:0.70 is not assignable > > > > There is no function 70 on that device. > > The * turns into a "LIBXL_PCI_FUNC_ALL (~0U)" in the internal > represenation, I bet this is some sort unhelpful and confusing of > formatting snafu related to that. >I guess. For this version I just have to explicitly list every PCI address, function.> > > In relation to that, the BDF documentation > > (http://wiki.xensource.com/xenwiki/BDFNotation) seems to imply that > > you can specify > > the extended BDF notation on your kernel boot line in grub. i.e. > > xen-pciback.hide=(0000:12:00.*) does not work. You''ll get an error in > > the kernel log. > > That is the old wiki, the new page seems to be > http://wiki.xen.org/wiki/Bus:Device.Function_(BDF)_Notation -- does this > still give you that impression? Please feel free to update to clarify. > > Yes, the new wiki still gives the impression you can use the extendednotation everywhere. It''d be useful of the kernel boot line could take the extended notation. We are passing through many multi-function devices so the list gets quite large.> Actually, if you look at the code for pci_stub.c in the linux kernel > > (at least for Jeremy branch 2.6.32.48), extended BDF is not supported. > > Right, I think that wiki page is trying to refer only to the notation > used by the toolstack. I imagine that could be clearer. > > Ian. > > I might add some comments about the kernel boot line to the BDF wiki.Thanks for your input. Matt _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2012-Feb-20 19:48 UTC
Re: libxl: error: ... PCI Device is not assignable
> > > In relation to that, the BDF documentation > > > (http://wiki.xensource.com/xenwiki/BDFNotation) seems to imply that > > > you can specify > > > the extended BDF notation on your kernel boot line in grub. i.e. > > > xen-pciback.hide=(0000:12:00.*) does not work. You''ll get an error in > > > the kernel log. > > > > That is the old wiki, the new page seems to be > > http://wiki.xen.org/wiki/Bus:Device.Function_(BDF)_Notation -- does this > > still give you that impression? Please feel free to update to clarify. > > > > Yes, the new wiki still gives the impression you can use the extended > notation everywhere. > It''d be useful of the kernel boot line could take the extended notation. > We are passing through many multi-function devices so the list gets quite > large.Patches are welcome..