Hi,
I''m trying to port AMD VGA passthru patch to the latest XEN and vanila
kernel
and I got SIGSEGV in
static void ati_hw_out(uint16_t hport, uint32_t data)
{
ioperm(gfx_info.host_pio_base, gfx_info.pio_size, 1);
asm volatile ("out %1,
%0"::"Nd"(hport),"a"(data));
ioperm(gfx_info.host_pio_base, gfx_info.pio_size, 0);
}
I tried old 2.6.32 XEN kernel and there is no such problem.
It looks related to arch/x86/kernel/ioport.c but I''m not sure.
Is anyone here familiar with that code?
--
Pavel Mateja
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
On Sun, Nov 13, 2011 at 10:19:06PM +0100, Pavel Matěja wrote:> Hi, > I''m trying to port AMD VGA passthru patch to the latest XEN and vanila kernel > and I got SIGSEGV in > > static void ati_hw_out(uint16_t hport, uint32_t data) > { > ioperm(gfx_info.host_pio_base, gfx_info.pio_size, 1); > asm volatile ("out %1, %0"::"Nd"(hport),"a"(data)); > ioperm(gfx_info.host_pio_base, gfx_info.pio_size, 0); > }Does it work under baremetal? What is the host_pio_base? Is the host_pio_base part of the permitted IO ports? (you can see that if you run ''xl debug-keys q'' and it should show you something like this: (XEN) Rangesets belonging to domain 1: (XEN) I/O Ports { b400-b41f, b800-b81f } (XEN) Interrupts { 18-19, 54-55 } (XEN) I/O Memory { fe940-fe9ff } (XEN) Memory pages belonging to domain 1: (you can get that from xm dmesg). As you can see, the b400->b41f are allowed in the domain 1. Is your host_pio_base in there?> > I tried old 2.6.32 XEN kernel and there is no such problem. > It looks related to arch/x86/kernel/ioport.c but I''m not sure. > Is anyone here familiar with that code?Yes, and I think I saw somebody ask me about that too. Lets rope them in this converstation - they got it to work but my memory is foggy at what was required. Ben, Thomas, I remember you guys had a tough time with vd86 which did something similar and it ultimately was due to to /dev/mem not passing in VM_IO. But the ioperm/outb sounds familiar too. Was there a missing hypercall when forking/copying the ioperm bitmap? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, Nov 17, 2011 at 08:56:34AM -0500, Ben Guthro wrote:> Attached is our patch to work around issues with the ioports with > some older nVidia cards. > > This, admittedly is a bit of a hack, and not exactly something that > I would see upstream wanting to carry, for a variety of reasons. It > really should all be predicated on whether the kernel is the initial > domain, etc. > > This opens up the legacy VGA ports in the VGA arbiter code.Was there a userspace program that did this? As in it would call ioperm?> > Konrad - I think that you had suggested an alternate way of doing > this, IIRC, but I can''t seem to find it in my inbox. Due to > competing demands, and my nVidia hardware disappearing, I never went > back to rework this code.Ah, I might have some code that I just uncovered from Jeremy''s old git tree. I''ve put it up on devel/ioperm - it nicely wraps the ioperm call to go the native or paravirt.> > As written, however, it did solve the problem at hand - however > hacky it may be.<nods> Pavel, Can you try to merge the #devel/ioperm patches in your tree and see if they fix your problem? The git tree is at git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git> > /btg > > On 11/16/2011 09:57 AM, Konrad Rzeszutek Wilk wrote: > >On Sun, Nov 13, 2011 at 10:19:06PM +0100, Pavel Matěja wrote: > >>Hi, > >>I''m trying to port AMD VGA passthru patch to the latest XEN and vanila kernel > >>and I got SIGSEGV in > >> > >>static void ati_hw_out(uint16_t hport, uint32_t data) > >>{ > >> ioperm(gfx_info.host_pio_base, gfx_info.pio_size, 1); > >> asm volatile ("out %1, %0"::"Nd"(hport),"a"(data)); > >> ioperm(gfx_info.host_pio_base, gfx_info.pio_size, 0); > >>} > > > >Does it work under baremetal? > > > >What is the host_pio_base? > > > >Is the host_pio_base part of the permitted IO ports? (you can > >see that if you run ''xl debug-keys q'' and it should show you something > >like this: > > > >(XEN) Rangesets belonging to domain 1: > >(XEN) I/O Ports { b400-b41f, b800-b81f } > >(XEN) Interrupts { 18-19, 54-55 } > >(XEN) I/O Memory { fe940-fe9ff } > >(XEN) Memory pages belonging to domain 1: > > > >(you can get that from xm dmesg). > > > >As you can see, the b400->b41f are allowed in the domain 1. Is your > >host_pio_base in there? > > > > > >> > >>I tried old 2.6.32 XEN kernel and there is no such problem. > >>It looks related to arch/x86/kernel/ioport.c but I''m not sure. > >>Is anyone here familiar with that code? > > > >Yes, and I think I saw somebody ask me about that too. > > > >Lets rope them in this converstation - they got it to work > >but my memory is foggy at what was required. > > > >Ben, Thomas, > > > >I remember you guys had a tough time with vd86 which did something similar > >and it ultimately was due to to /dev/mem not passing in VM_IO. But the > >ioperm/outb sounds familiar too. Was there a missing hypercall when > >forking/copying the ioperm bitmap?> diff --git a/drivers/gpu/vga/vgaarb.c b/drivers/gpu/vga/vgaarb.c > index ace2b16..d488426 100644 > --- a/drivers/gpu/vga/vgaarb.c > +++ b/drivers/gpu/vga/vgaarb.c > @@ -421,6 +421,50 @@ bail: > } > EXPORT_SYMBOL(vga_put); > > +#ifdef CONFIG_XEN > +#include <xen/xen.h> > +#include <xen/interface/physdev.h> > +#include <xen/interface/xen.h> > +#include <asm/xen/hypervisor.h> > +#include <asm/xen/hypercall.h> > + > +#define IO_BITMAP_BITS 65536 > +#define IO_BITMAP_BYTES (IO_BITMAP_BITS/8) > + > +#define VGA_START_PORT 0x3B0 > +#define VGA_END_PORT 0x3DF > + > +static void > +vga_ioport_map(struct pci_dev *pdev) > +{ > + unsigned int i, j; > + struct physdev_set_iobitmap op; > + uint8_t *bitmap; > + > + bitmap = kmalloc(IO_BITMAP_BYTES, GFP_KERNEL); > + memset(bitmap, 0xff, IO_BITMAP_BYTES); > + > + /* PCI io bars */ > + for (i = 0; i < PCI_NUM_RESOURCES; i++) > + if (pci_resource_flags(pdev, i) & IORESOURCE_IO) > + for (j = pci_resource_start(pdev, i); > + j <= pci_resource_end(pdev, i); j++) > + __clear_bit(j, (unsigned long*)bitmap); > + > + /* legacy vga */ > + for (i = VGA_START_PORT; i <= VGA_END_PORT; i++) > + __clear_bit(i, (unsigned long*)bitmap); > + > + op.bitmap = bitmap; > + op.nr_ports = IO_BITMAP_BITS; > + if(HYPERVISOR_physdev_op(PHYSDEVOP_set_iobitmap, &op) != 0) > + pr_info("PHYSDEVOP_set_iobitmap failed\n"); > + > +} > +#else > +#define vga_ioport_map(x) > +#endif > + > /* > * Currently, we assume that the "initial" setup of the system is > * not sane, that is we come up with conflicting devices and let > @@ -509,6 +553,7 @@ static bool vga_arbiter_add_pci_device(struct pci_dev *pdev) > vga_iostate_to_str(vgadev->owns), > vga_iostate_to_str(vgadev->locks)); > > + vga_ioport_map(pdev); > spin_unlock_irqrestore(&vga_lock, flags); > return true; > fail:> _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > This opens up the legacy VGA ports in the VGA arbiter code. > > Was there a userspace program that did this? As in it would > call ioperm?Not necessarily there is also a priviledged interface via the console for historical reasons For the legacy VGA ports the proper approach is almost certainly to scan the PCI bus routing data and work out which device they are routed to and then attach them to that physical device. That may be a dynamic configuration but you''ll see the virtualisation of the PCI config changes any guest tries to do if so. Alan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 11/17/2011 09:30 AM, Konrad Rzeszutek Wilk wrote:> On Thu, Nov 17, 2011 at 08:56:34AM -0500, Ben Guthro wrote: >> Attached is our patch to work around issues with the ioports with >> some older nVidia cards. >> >> This, admittedly is a bit of a hack, and not exactly something that >> I would see upstream wanting to carry, for a variety of reasons. It >> really should all be predicated on whether the kernel is the initial >> domain, etc. >> >> This opens up the legacy VGA ports in the VGA arbiter code. > Was there a userspace program that did this? As in it would > call ioperm? > >> Konrad - I think that you had suggested an alternate way of doing >> this, IIRC, but I can''t seem to find it in my inbox. Due to >> competing demands, and my nVidia hardware disappearing, I never went >> back to rework this code. > Ah, I might have some code that I just uncovered from Jeremy''s old > git tree. > > I''ve put it up on devel/ioperm - it nicely wraps the ioperm > call to go the native or paravirt.Yeah, I''d sort of let that sit, since there''s nothing in a "modern" system which should require it. Allowing usermode to poke at ioports from within a domain is inherently suspect, after all. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, Nov 17, 2011 at 10:05:57AM -0800, Jeremy Fitzhardinge wrote:> On 11/17/2011 09:30 AM, Konrad Rzeszutek Wilk wrote: > > On Thu, Nov 17, 2011 at 08:56:34AM -0500, Ben Guthro wrote: > >> Attached is our patch to work around issues with the ioports with > >> some older nVidia cards. > >> > >> This, admittedly is a bit of a hack, and not exactly something that > >> I would see upstream wanting to carry, for a variety of reasons. It > >> really should all be predicated on whether the kernel is the initial > >> domain, etc. > >> > >> This opens up the legacy VGA ports in the VGA arbiter code. > > Was there a userspace program that did this? As in it would > > call ioperm? > > > >> Konrad - I think that you had suggested an alternate way of doing > >> this, IIRC, but I can''t seem to find it in my inbox. Due to > >> competing demands, and my nVidia hardware disappearing, I never went > >> back to rework this code. > > Ah, I might have some code that I just uncovered from Jeremy''s old > > git tree. > > > > I''ve put it up on devel/ioperm - it nicely wraps the ioperm > > call to go the native or paravirt. > > Yeah, I''d sort of let that sit, since there''s nothing in a "modern" > system which should require it. Allowing usermode to poke at ioports > from within a domain is inherently suspect, after all.There are three different cases here: - Nvidia and VBE tool. They seem to do a lot of poking and Ben needed some way of making it work. Naturally the nouveau driver is not doing this.. This is in Dom0. - DVB cards. I *think* that the problem some folks have with passing in the DVB cards is that the ''scan'' tools used to jump frequencies poke at those IO-ports. But I am not 100% sure about it, and need to look in more details at the drivers itself. The person reporting this was running the DVB card in Dom0. - Lastly the PCI passthrough of NVidia/ATI card in the guest. I think one person is trying to do it while running the guest in paravirtualized mode, in which case some help in doing those hypercall is needed. But if one were to do this in fully virtualized that means QEMU ends up trapping those I/O port accesses and it itself pokes them in the PCI card. But since QEMU is running in dom0 and dom0 is PV, that leads back to the VGA I/O ports not being allowed to poke as we don''t have the ioperm hypercall. This all sounds right, but in practice it might be something entirely different.. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Looking back through the bug database, the problem started in uvesafb,
while running in the nVidia closed source driver
The user reported very long boots, and I found the following repeated
over, and over in the syslog:
Sep 8 20:09:46 Qosmio-X505-1A033757W kernel: [ 29.677547] uvesafb:
Getting mode info block for mode 0x106 failed (eax=0x4f01, err=1)
Sep 8 20:09:46 Qosmio-X505-1A033757W kernel: [ 29.685667] v86d[363]
general protection ip:7f0acfdbdb43 sp:7fff100d3748 error:0 in
libx86.so.1[7f0acfdb6000+1f000]
This continues every 5s until:
Sep 8 20:09:46 Qosmio-X505-1A033757W kernel: [ 84.677745] uvesafb:
Getting mode info block for mode 0x161 failed (eax=0x4f01, err=1)
Sep 8 20:09:46 Qosmio-X505-1A033757W kernel: [ 84.692267] uvesafb:
VBIOS/hardware doesn''t support DDC transfers
Sep 8 20:09:46 Qosmio-X505-1A033757W kernel: [ 84.692275] uvesafb: no
monitor limits have been set, default refresh rate will be used
Sep 8 20:09:46 Qosmio-X505-1A033757W kernel: [ 84.692369] v86d[396]
general protection ip:7ff77908bb43 sp:7fffa8b7e708 error:0 in
libx86.so.1[7ff779084000+1f000]
Chasing this down, I found it all started from drivers/video/uvesafb.c: 506:
uvesafb_reset(task);
task->t.regs.eax = 0x4f01;
task->t.regs.ecx = (u32) *mode;
task->t.flags = TF_BUF_RET | TF_BUF_ESDI;
task->t.buf_len = sizeof(struct vbe_mode_ib);
task->buf = par->vbe_modes + off;
err = uvesafb_exec(task);
if (err || (task->t.regs.eax & 0xffff) != 0x004f) {
printk(KERN_WARNING "uvesafb: Getting mode info block "
"for mode 0x%x failed (eax=0x%x, err=%d)\n",
*mode, (u32)task->t.regs.eax, err);
mode++;
par->vbe_modes_cnt--;
continue;
}
(the same code also exists in testvbe.c - line 55)
This executes in the v86d userspace process
v86d/v86_common.c:
int v86_task(struct uvesafb_task *tsk, u8 *buf)
uvesafb was executing the video BIOS code, as mapped in /dev/mem
However, it was unable to access the ioport, causing a GPF.
uvesafb would loop on that, with a sleep(5)
The end result was a colossally long boot process seen by the end user.
The (albeit hacky) solution to open up the VGA ioports resolved the
seemed hang on this older nVidia card.
Hope this explanation make some semblance of sense.
/btg
On Thu, Nov 17, 2011 at 1:05 PM, Jeremy Fitzhardinge <jeremy@goop.org>
wrote:>
> On 11/17/2011 09:30 AM, Konrad Rzeszutek Wilk wrote:
> > On Thu, Nov 17, 2011 at 08:56:34AM -0500, Ben Guthro wrote:
> >> Attached is our patch to work around issues with the ioports with
> >> some older nVidia cards.
> >>
> >> This, admittedly is a bit of a hack, and not exactly something
that
> >> I would see upstream wanting to carry, for a variety of reasons.
It
> >> really should all be predicated on whether the kernel is the
initial
> >> domain, etc.
> >>
> >> This opens up the legacy VGA ports in the VGA arbiter code.
> > Was there a userspace program that did this? As in it would
> > call ioperm?
> >
> >> Konrad - I think that you had suggested an alternate way of doing
> >> this, IIRC, but I can''t seem to find it in my inbox. Due
to
> >> competing demands, and my nVidia hardware disappearing, I never
went
> >> back to rework this code.
> > Ah, I might have some code that I just uncovered from
Jeremy''s old
> > git tree.
> >
> > I''ve put it up on devel/ioperm - it nicely wraps the ioperm
> > call to go the native or paravirt.
>
> Yeah, I''d sort of let that sit, since there''s nothing in
a "modern"
> system which should require it. Allowing usermode to poke at ioports
> from within a domain is inherently suspect, after all.
>
> J
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
On Thu 17. of November 2011 18:30:10 Konrad Rzeszutek Wilk wrote:> Pavel, > > Can you try to merge the #devel/ioperm patches in your tree and see > if they fix your problem? > > The git tree is at > git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.gitHi Konrad, yes, it fixed the problem. I can see BSOD during Windows boot now. Thanks a lot -- Pavel Mateja _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Apparently Analagous Threads
- Does xen-4.2.0 support VGA passthrough with the virtual machine created by xl command?
- [PATCH v4 10/10] x86: Support compiling out userspace IO (iopl and ioperm)
- [PATCH v4 10/10] x86: Support compiling out userspace IO (iopl and ioperm)
- Use of tools:::httpdPort in a package for CRAN.
- [PATCH v4 10/10] x86: Support compiling out userspace IO (iopl and ioperm)