Hi!
The following simple program (below) mmaps PCI I/O memory
into userspace. This works fine under linux, but not in Xen dom0.
Linux:
# ./test1 /sys/bus/pci/devices/0000\:07\:00.0/resource0 0
0 0
Xen dom0:
# ./test1 /sys/bus/pci/devices/0000\:07\:00.0/resource0 0
0 0xf000eef3
Where does one start looking at fixing this?
Thanks,
MST
---
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
#include <stdio.h>
#include <time.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <string.h>
/* Usage: test0 file offset */
int main(int argc, char** argv)
{
int fd;
if (argc < 3) {
fprintf(stderr, "Usage: %s file offset\n", argv[0]);
return 2;
}
fd = open(argv[1], O_RDWR | O_SYNC);
if (fd < 0) {
perror("open");
return errno;
}
long o = strtol(argv[2], NULL, 0);
long* ptr = mmap(NULL, 0x1000, PROT_READ, MAP_SHARED, fd, o);
if ( (! ptr) || (ptr == MAP_FAILED) ) {
perror("mmap");
return errno;
}
printf("%#lx %#lx\n", o, *ptr);
return 0;
}
--
MST
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
> The following simple program (below) mmaps PCI I/O memory > into userspace. This works fine under linux, but not in Xen dom0. > > Linux: > > # ./test1 /sys/bus/pci/devices/0000\:07\:00.0/resource0 0 0 0 > > Xen dom0: > > # ./test1 /sys/bus/pci/devices/0000\:07\:00.0/resource0 0 0 0xf000eef3Interesting -- I didn''t know you could use /sys/bus/pci/devices/ for mapping memory. Change pci_mmap_page_range in arch/xen/arch/pci/i386.c to use io_remap_pfn_range I''ll check something in. Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Quoting r. Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk>:> Subject: RE: [Xen-devel] Userspace PIO access under xen > > > > > The following simple program (below) mmaps PCI I/O memory > > into userspace. This works fine under linux, but not in Xen dom0. > > > > Linux: > > > > # ./test1 /sys/bus/pci/devices/0000\:07\:00.0/resource0 0 0 0 > > > > Xen dom0: > > > > # ./test1 /sys/bus/pci/devices/0000\:07\:00.0/resource0 0 0 0xf000eef3 > > > Interesting -- I didn''t know you could use /sys/bus/pci/devices/ for > mapping memory.You can also use /proc/bus/pci/ for this, the difference is that you have to give full bus address to /proc/bus/pci/ , while /sys/bus/pci/devices/ needs an offset within the resource.> Change pci_mmap_page_range in arch/xen/arch/pci/i386.c to use > io_remap_pfn_rangeI''ve written a small kernel module to map PIO to userspace using remap_page_range, and that doesnt seem to work, either.> > I''ll check something in. > > Ian >OK, let me know. -- MST _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> You can also use /proc/bus/pci/ for this, the difference is > that you have to give full bus address to /proc/bus/pci/ , > while /sys/bus/pci/devices/ needs an offset within the resource. > > > Change pci_mmap_page_range in arch/xen/arch/pci/i386.c to use > > io_remap_pfn_range > > I''ve written a small kernel module to map PIO to userspace > using remap_page_range, and that doesnt seem to work, either.You have to use io_remap_pfn_range for mapping IO space. Using remap_pfn_range is technically a bug in the i386 code. I believe the pfn_range functions are preferred over page_range functions anyhow. Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Quoting r. Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk>:> Subject: RE: [Xen-devel] Userspace PIO access under xen > > > > > You can also use /proc/bus/pci/ for this, the difference is > > that you have to give full bus address to /proc/bus/pci/ , > > while /sys/bus/pci/devices/ needs an offset within the resource. > > > > > Change pci_mmap_page_range in arch/xen/arch/pci/i386.c to use > > > io_remap_pfn_range > > > > I''ve written a small kernel module to map PIO to userspace > > using remap_page_range, and that doesnt seem to work, either. > > You have to use io_remap_pfn_range for mapping IO space. Using > remap_pfn_range is technically a bug in the i386 code.#define io_remap_page_range(vma, vaddr, paddr, size, prot) \ remap_pfn_range(vma, vaddr, (paddr) >> PAGE_SHIFT, size, prot) So I just tested again and of course io_remap_pfn_range gives the same result.> I believe the pfn_range functions are preferred over page_range > functions anyhow.I actually use remap_pfn_range, I mistyped that.> Ian >Point is, none of these works. -- MST _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > You have to use io_remap_pfn_range for mapping IO space. Using > > remap_pfn_range is technically a bug in the i386 code. > > #define io_remap_page_range(vma, vaddr, paddr, size, prot) \ > remap_pfn_range(vma, vaddr, (paddr) >> > PAGE_SHIFT, size, prot) > > So I just tested again and of course io_remap_pfn_range gives > the same result. > > > I believe the pfn_range functions are preferred over page_range > > functions anyhow. > > I actually use remap_pfn_range, I mistyped that.Look at arch/xen/kernel/devmem.c and include/asm-xen/asm-i386/pgtable.h #define io_remap_page_range(vma,from,phys,size,prot) \ direct_remap_area_pages(vma->vm_mm,from,phys,size,prot,DOMID_IO) It works fine for /dev/mem. It''s possible there''s a definition missing for io_remap_pfn_range. Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Quoting r. Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk>:> Subject: RE: [Xen-devel] Userspace PIO access under xen > > > > > You have to use io_remap_pfn_range for mapping IO space. Using > > > remap_pfn_range is technically a bug in the i386 code. > > > > #define io_remap_page_range(vma, vaddr, paddr, size, prot) \ > > remap_pfn_range(vma, vaddr, (paddr) >> > > PAGE_SHIFT, size, prot) > > > > So I just tested again and of course io_remap_pfn_range gives > > the same result. > > > > > I believe the pfn_range functions are preferred over page_range > > > functions anyhow. > > > > I actually use remap_pfn_range, I mistyped that. > > Look at arch/xen/kernel/devmem.c and include/asm-xen/asm-i386/pgtable.h > > #define io_remap_page_range(vma,from,phys,size,prot) \ > direct_remap_area_pages(vma->vm_mm,from,phys,size,prot,DOMID_IO) > > It works fine for /dev/mem. It''s possible there''s a definition missing > for io_remap_pfn_range. > > Ian >Hmm. I have /lib/modules/2.6.12-xen0/source/include/asm-xen/asm-i386/pgtable.h -> ../../../../linux-2.6-xen-sparse/./include/asm-xen/asm-i386/./pgtable.h But # ls -l /lib/modules/2.6.12-xen0/source/include/asm/pgtable.h -rw-r--r-- 4 root root 14618 Jun 17 22:48 /lib/modules/2.6.12-xen0/source/include/asm/pgtable.h so it seems that the install script didnt link asm/pgtable.h in place correctly? -- MST _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Quoting r. Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk>:> Subject: RE: [Xen-devel] Userspace PIO access under xen > > > > > The following simple program (below) mmaps PCI I/O memory > > into userspace. This works fine under linux, but not in Xen dom0. > > > > Linux: > > > > # ./test1 /sys/bus/pci/devices/0000\:07\:00.0/resource0 0 0 0 > > > > Xen dom0: > > > > # ./test1 /sys/bus/pci/devices/0000\:07\:00.0/resource0 0 0 0xf000eef3 > > > Interesting -- I didn''t know you could use /sys/bus/pci/devices/ for > mapping memory. > > Change pci_mmap_page_range in arch/xen/arch/pci/i386.c to use > io_remap_pfn_range > > I''ll check something in. > > Ian >OK, the following seems to work. Thanks! --- Make pci_mmap_page_range use io_remap_pfn_range. This fixes mmap for resources under /sys/bus/pci/devices/ Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il> Index: linux-2.6.12-xen0/arch/i386/pci/i386.c ==================================================================--- linux-2.6.12-xen0/arch/i386/pci/i386.c +++ linux-2.6.12-xen0/arch/i386/pci/i386.c @@ -295,7 +295,7 @@ int pci_mmap_page_range(struct pci_dev * /* Write-combine setting is ignored, it is changed via the mtrr * interfaces on this platform. */ - if (remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff, + if (io_remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff, vma->vm_end - vma->vm_start, vma->vm_page_prot)) return -EAGAIN; -- MST _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel