Hi everyone I''ve finally got my VMX guest working great (WinXP Pro) on here. However, I''ve got one more thing that I''d like to do. I have a secondary PCI-E video card that I want to pass through to the VM. I am using xen-unstable, so I know passthru is available... I have 2 graphics cards installed: an old 4MB PCI SiS 6326, and a PCI-E Geforce 6200. The PCI card is initialized first by the BIOS, so it is the only one being used by Xen afaict. I have setup my kernel cmdline options to include: pciback.hide=(06:00.0) 06:00.0 is the address reported by lspci for the nVidia card. I''ve also got a line in my vmx guest''s config: pci = [ ''06:00.0'' ] I still see the device in lspci (I suspect that I shouldn''t here!). I can''t get Windows to see the device at all, either. Anyone have any idea what could be preventing the device from being hidden? Is pci passthru supposed to work with VMX guests (ones w/o the pci frontend)? Thanks! Dave -- Dave Goodlad dgoodlad@gmail.com or dave@goodlad.ca http://david.goodlad.ca/ _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 3/20/06, Frank DiRocco <ofanged1@gmail.com> wrote: <snip>> change pci = [ ''06:00.0'' ] > to > pci = [ ''06,00,0'' ]<snip> I tried this, to no avail. I''ve checked that the pciback driver is taking control of the device now: dmesg | grep pciback pciback: seizing PCI device 0000:06:00.0 However, looking at my xend.log, I can''t see anything about the pci device. Even when it spits out the config at me, I don''t see anything related to pci... Dave> i think... > goodluck > > On 3/20/06, David Goodlad <dgoodlad@gmail.com> wrote: > > > > Hi everyone > > > > I''ve finally got my VMX guest working great (WinXP Pro) on here. > > However, I''ve got one more thing that I''d like to do. I have a > > secondary PCI-E video card that I want to pass through to the VM. I > > am using xen-unstable, so I know passthru is available... > > > > I have 2 graphics cards installed: an old 4MB PCI SiS 6326, and a > > PCI-E Geforce 6200. The PCI card is initialized first by the BIOS, so > > it is the only one being used by Xen afaict. I have setup my kernel > > cmdline options to include: > > > > pciback.hide=(06:00.0) > > > > 06:00.0 is the address reported by lspci for the nVidia card. I''ve > > also got a line in my vmx guest''s config: > > > > pci = [ ''06:00.0'' ] > > > > I still see the device in lspci (I suspect that I shouldn''t here!). I > > can''t get Windows to see the device at all, either. > > > > Anyone have any idea what could be preventing the device from being > > hidden? Is pci passthru supposed to work with VMX guests (ones w/o > > the pci frontend)? > > > > Thanks! > > Dave > > > > -- > > Dave Goodlad > > dgoodlad@gmail.com or dave@goodlad.ca > > http://david.goodlad.ca/ > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > > > > > > -- > Thank you, > Frank Di Rocco > > "Does an optimistic person look at a hard drive as half-full or > half-empty?" > -ofanged1-at-gmail.com > >-- Dave Goodlad dgoodlad@gmail.com or dave@goodlad.ca http://david.goodlad.ca/ _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Sorry, PCI passthrough to VMX guests isn''t supported - the guest has to be paravirt and be running the PCI frontend driver. The VMX spec does include some helpful features for passing PCI devices through to a guest, and I''m sure it''s possible to make it work - in principle. I don''t know of anyone actually planning to hack on it though :-( However, there are plans to improve the display performance for VMX guests, so you should find the virtual framebuffer will improve in future releases. I understand that using rdesktop in domain0 to connect to the guest also works very well and provides quite snappy UI performance. Cheers, mark On Mar 21 2006, David Goodlad wrote:>Hi everyone > >I''ve finally got my VMX guest working great (WinXP Pro) on here. >However, I''ve got one more thing that I''d like to do. I have a >secondary PCI-E video card that I want to pass through to the VM. I >am using xen-unstable, so I know passthru is available... > >I have 2 graphics cards installed: an old 4MB PCI SiS 6326, and a >PCI-E Geforce 6200. The PCI card is initialized first by the BIOS, so >it is the only one being used by Xen afaict. I have setup my kernel >cmdline options to include: > >pciback.hide=(06:00.0) > >06:00.0 is the address reported by lspci for the nVidia card. I''ve >also got a line in my vmx guest''s config: > >pci = [ ''06:00.0'' ] > >I still see the device in lspci (I suspect that I shouldn''t here!). I >can''t get Windows to see the device at all, either. > >Anyone have any idea what could be preventing the device from being >hidden? Is pci passthru supposed to work with VMX guests (ones w/o >the pci frontend)? > >Thanks! >Dave > >-- >Dave Goodlad >dgoodlad@gmail.com or dave@goodlad.ca >http://david.goodlad.ca/ > >_______________________________________________ >Xen-users mailing list >Xen-users@lists.xensource.com >http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> -----Original Message----- > From: xen-users-bounces@lists.xensource.com > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of > M.A. Williamson > Sent: 21 March 2006 15:46 > To: David Goodlad > Cc: xen-users@lists.xensource.com > Subject: Re: [Xen-users] PCI Passthrough to VMX Guest > > Sorry, PCI passthrough to VMX guests isn''t supported - the > guest has to be paravirt and be running the PCI frontend driver. > > The VMX spec does include some helpful features for passing > PCI devices through to a guest, and I''m sure it''s possible to > make it work - in principle. I don''t know of anyone actually > planning to hack on it though :-(One of the problems with this is that the OS/Driver that supports the nVidia (or other graphics adapter) will need to actually know it''s physical addresses in memory - something that it doesn''t, because the HVM solution may well tell the OS that it''s got 512MB of memory from 0 to 512M, but it''s ACTUALLY living at 512M to 1G. So when the graphics driver says "You have a bitmap at 128MB", it should actually say "You have a bitmap at 640MB". Until there''s an IOMMU implementation, there''s nothing we can do about this. So even if you COULD assign your PCI device to the DomU, it still wouldn''t do the right thing... :-( So until then, there''s a bit of a problem implementing any complex hardware support in a virtual machine. There may be ways to solve this, but they are non-trivial (and most like specific to the particular hardware...). -- Mats _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 21 Mar 2006 15:46:27 +0000, M.A. Williamson <maw48@cam.ac.uk> wrote:> Sorry, PCI passthrough to VMX guests isn''t supported - the guest has to be > paravirt and be running the PCI frontend driver. > > The VMX spec does include some helpful features for passing PCI devices > through to a guest, and I''m sure it''s possible to make it work - in > principle. I don''t know of anyone actually planning to hack on it though > :-( > > However, there are plans to improve the display performance for VMX guests, > so you should find the virtual framebuffer will improve in future releases. > I understand that using rdesktop in domain0 to connect to the guest also > works very well and provides quite snappy UI performance.Hi Mark That''s too bad that it''s not supported. It might be a good idea to drop a note in the driver domains/PCI passthrough section of the docs :) I just got rdesktop going on here yesterday after having a heck of a time trying to get the network working. Apparently having an ethernet device driven by the sky2 driver attached to a bridge makes the kernel rather unhappy right now! I''ll confirm that about rdesktop, it is, as you say, quite snappy. Thanks for clearing this up for me! Dave> > Cheers, > mark > > On Mar 21 2006, David Goodlad wrote: > > >Hi everyone > > > >I''ve finally got my VMX guest working great (WinXP Pro) on here. > >However, I''ve got one more thing that I''d like to do. I have a > >secondary PCI-E video card that I want to pass through to the VM. I > >am using xen-unstable, so I know passthru is available... > > > >I have 2 graphics cards installed: an old 4MB PCI SiS 6326, and a > >PCI-E Geforce 6200. The PCI card is initialized first by the BIOS, so > >it is the only one being used by Xen afaict. I have setup my kernel > >cmdline options to include: > > > >pciback.hide=(06:00.0) > > > >06:00.0 is the address reported by lspci for the nVidia card. I''ve > >also got a line in my vmx guest''s config: > > > >pci = [ ''06:00.0'' ] > > > >I still see the device in lspci (I suspect that I shouldn''t here!). I > >can''t get Windows to see the device at all, either. > > > >Anyone have any idea what could be preventing the device from being > >hidden? Is pci passthru supposed to work with VMX guests (ones w/o > >the pci frontend)? > > > >Thanks! > >Dave > > > >-- > >Dave Goodlad > >dgoodlad@gmail.com or dave@goodlad.ca > >http://david.goodlad.ca/ > > > >_______________________________________________ > >Xen-users mailing list > >Xen-users@lists.xensource.com > >http://lists.xensource.com/xen-users > > >-- Dave Goodlad dgoodlad@gmail.com or dave@goodlad.ca http://david.goodlad.ca/ _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 3/21/06, Petersson, Mats <Mats.Petersson@amd.com> wrote:> > > > -----Original Message----- > > From: xen-users-bounces@lists.xensource.com > > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of > > M.A. Williamson > > Sent: 21 March 2006 15:46 > > To: David Goodlad > > Cc: xen-users@lists.xensource.com > > Subject: Re: [Xen-users] PCI Passthrough to VMX Guest > > > > Sorry, PCI passthrough to VMX guests isn''t supported - the > > guest has to be paravirt and be running the PCI frontend driver. > > > > The VMX spec does include some helpful features for passing > > PCI devices through to a guest, and I''m sure it''s possible to > > make it work - in principle. I don''t know of anyone actually > > planning to hack on it though :-( > > One of the problems with this is that the OS/Driver that supports the > nVidia (or other graphics adapter) will need to actually know it''s > physical addresses in memory - something that it doesn''t, because the > HVM solution may well tell the OS that it''s got 512MB of memory from 0 > to 512M, but it''s ACTUALLY living at 512M to 1G. So when the graphics > driver says "You have a bitmap at 128MB", it should actually say "You > have a bitmap at 640MB". Until there''s an IOMMU implementation, there''s > nothing we can do about this. > > So even if you COULD assign your PCI device to the DomU, it still > wouldn''t do the right thing... :-( > > So until then, there''s a bit of a problem implementing any complex > hardware support in a virtual machine. There may be ways to solve this, > but they are non-trivial (and most like specific to the particular > hardware...). > > -- > Mats > >Ahh, that clears things up. I never thought of the memory addressing issue! I assume that this issue is handled in paravirtualized machines by the PCI frontend driver? Dave -- Dave Goodlad dgoodlad@gmail.com or dave@goodlad.ca http://david.goodlad.ca/ _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
>One of the problems with this is that the OS/Driver that supports the >nVidia (or other graphics adapter) will need to actually know it''s >physical addresses in memory - something that it doesn''t, because the >HVM solution may well tell the OS that it''s got 512MB of memory from 0 >to 512M, but it''s ACTUALLY living at 512M to 1G. So when the graphics >driver says "You have a bitmap at 128MB", it should actually say "You >have a bitmap at 640MB". Until there''s an IOMMU implementation, there''s >nothing we can do about this.OK, good point! Although IIRC, you guys have a solution for this on the way ;-)>So even if you COULD assign your PCI device to the DomU, it still >wouldn''t do the right thing... :-( > >So until then, there''s a bit of a problem implementing any complex >hardware support in a virtual machine. There may be ways to solve this, >but they are non-trivial (and most like specific to the particular >hardware...).I guess in principle we could port the PCI frontend to run in an unmodified guest... (?) It could then perform IOMMU functionality in software by hooking the right places in the DMA API (and arranging bounce buffering if necessary). It''s not entirely clear to me that this would be worth it, though. Cheers, Mark _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> -----Original Message----- > From: David Goodlad [mailto:dgoodlad@gmail.com] > Sent: 21 March 2006 17:02 > To: Petersson, Mats > Cc: mark.williamson@cl.cam.ac.uk; xen-users@lists.xensource.com > Subject: Re: PCI Passthrough to VMX Guest > > On 3/21/06, Petersson, Mats <Mats.Petersson@amd.com> wrote: > > > > > > > -----Original Message----- > > > From: xen-users-bounces@lists.xensource.com > > > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of M.A. > > > Williamson > > > Sent: 21 March 2006 15:46 > > > To: David Goodlad > > > Cc: xen-users@lists.xensource.com > > > Subject: Re: [Xen-users] PCI Passthrough to VMX Guest > > > > > > Sorry, PCI passthrough to VMX guests isn''t supported - > the guest has > > > to be paravirt and be running the PCI frontend driver. > > > > > > The VMX spec does include some helpful features for passing PCI > > > devices through to a guest, and I''m sure it''s possible to make it > > > work - in principle. I don''t know of anyone actually planning to > > > hack on it though :-( > > > > One of the problems with this is that the OS/Driver that > supports the > > nVidia (or other graphics adapter) will need to actually know it''s > > physical addresses in memory - something that it doesn''t, > because the > > HVM solution may well tell the OS that it''s got 512MB of > memory from 0 > > to 512M, but it''s ACTUALLY living at 512M to 1G. So when > the graphics > > driver says "You have a bitmap at 128MB", it should > actually say "You > > have a bitmap at 640MB". Until there''s an IOMMU implementation, > > there''s nothing we can do about this. > > > > So even if you COULD assign your PCI device to the DomU, it still > > wouldn''t do the right thing... :-( > > > > So until then, there''s a bit of a problem implementing any complex > > hardware support in a virtual machine. There may be ways to solve > > this, but they are non-trivial (and most like specific to the > > particular hardware...). > > > > -- > > Mats > > > > > > Ahh, that clears things up. I never thought of the memory > addressing issue! I assume that this issue is handled in > paravirtualized machines by the PCI frontend driver?Actually, in the paravirtualized case, it''s MOSTLY the para-virtualized driver for the hardware and kernel modifications in the para-virt kernel that fixes up things. Since the para-virt kernel KNOWS that it''s virtual, it can fix things up for itself. When you''re running under HVM, the kernel doesn''t actually know that it''s not running on a real hardware, and thus can''t fix things like this up. The IOMMU that AMD is going to put in the NEXT generation of SVM processors (not the ones coming out this year, next years models), will be able to fix this up. I think Intel has some similar spec, but I''m not an expert on that. For now, I think we''ll need to fix things up with para-virtualized drivers for HVM systems, which means that you''d have to install a special driver once you''ve got your unmodified kernel working as a virtual machine... -- Mats> > Dave > > > -- > Dave Goodlad > dgoodlad@gmail.com or dave@goodlad.ca > http://david.goodlad.ca/ > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> -----Original Message----- > From: xen-users-bounces@lists.xensource.com > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of > M.A. Williamson > Sent: 21 March 2006 17:35 > To: Petersson, Mats > Cc: mark.williamson@cl.cam.ac.uk; > xen-users@lists.xensource.com; David Goodlad > Subject: RE: [Xen-users] PCI Passthrough to VMX Guest > > > >One of the problems with this is that the OS/Driver that > supports the > >nVidia (or other graphics adapter) will need to actually know it''s > >physical addresses in memory - something that it doesn''t, > because the > >HVM solution may well tell the OS that it''s got 512MB of > memory from 0 > >to 512M, but it''s ACTUALLY living at 512M to 1G. So when the > graphics > >driver says "You have a bitmap at 128MB", it should actually > say "You > >have a bitmap at 640MB". Until there''s an IOMMU > implementation, there''s > >nothing we can do about this. > > OK, good point! Although IIRC, you guys have a solution for > this on the way > ;-)Yes, the GART IOMMU work is being done by Mark Langsdorf, and we''ll have a "proper" IOMMU for next generation processors.> > >So even if you COULD assign your PCI device to the DomU, it still > >wouldn''t do the right thing... :-( > > > >So until then, there''s a bit of a problem implementing any complex > >hardware support in a virtual machine. There may be ways to > solve this, > >but they are non-trivial (and most like specific to the particular > >hardware...). > > I guess in principle we could port the PCI frontend to run in > an unmodified guest... (?) It could then perform IOMMU > functionality in software by hooking the right places in the > DMA API (and arranging bounce buffering if necessary). > > It''s not entirely clear to me that this would be worth it, though.Not for Windows, that''s for sure. I''m not that familiar with how the drivers for Linux works, but essentially, in Windows you can get a call to BLT a bitmap from system memory into graphics memory. The way that works is that the driver would ask the OS for the physical address (which the OS think is somewhere between 0 and 512MB, say), and with suitable math put that into the command stream of the graphics processor, together with the "MOVE these pixels from here to there with the following BLTmode" command. There will then be DMA operations performed by the graphics processor to read the physical memory. But I don''t think there''s any way for the hypervisor to understand that the OS is asking for the physical address of the bitmap, and thus the DMA operation would happen from the wrong address [unless the Device Exclusion Vector is set up to prevent that from happening]. -- Mats> > Cheers, > Mark > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Tue, Mar 21, 2006 at 06:50:15PM +0100, Petersson, Mats wrote:> Yes, the GART IOMMU work is being done by Mark Langsdorf, and we''ll have > a "proper" IOMMU for next generation processors.How''s the GART work coming along? I haven''t seen a patch from Mark in a while. Cheers, Muli -- Muli Ben-Yehuda http://www.mulix.org | http://mulix.livejournal.com/ _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I think Mark is on sabatical from AMD (after having worked 7 years in the US organisation, you''re entitled to take 7 weeks off work - more if you have vacation time to take off too), so this may be the reason there''s no posts on the subject. I know Mark from ways back, but we''re not working so closely together that I know exactly what he''s doing. -- Mats> -----Original Message----- > From: Muli Ben-Yehuda [mailto:mulix@mulix.org] > Sent: 21 March 2006 22:04 > To: Petersson, Mats > Cc: mark.williamson@cl.cam.ac.uk; > xen-users@lists.xensource.com; David Goodlad > Subject: Re: [Xen-users] PCI Passthrough to VMX Guest > > On Tue, Mar 21, 2006 at 06:50:15PM +0100, Petersson, Mats wrote: > > > Yes, the GART IOMMU work is being done by Mark Langsdorf, and we''ll > > have a "proper" IOMMU for next generation processors. > > How''s the GART work coming along? I haven''t seen a patch from > Mark in a while. > > Cheers, > Muli > -- > Muli Ben-Yehuda > http://www.mulix.org | http://mulix.livejournal.com/ > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users