Kashmira. K.
2010-Jan-13 21:03 UTC
[Xen-devel] Contiguous DMAble memory in Dom0 user space linux process
Hello Xen experts,
I am trying to get contiguous DMAble memory in user space application which is a
processes running on xen capable Linux.
I thought I could use xen_create_contiguous_region(), but it appears that is for
linux kernel modules.
I need to get about 500MB of contiguous DMAble memory in my application. This
application will run in Dom0. This memory is not to be shared between Domains.
Is it possible to use the HYPERVISOR_grant_table_op hypercall passing
GMTTABOP_map_grant_ref as command to achieve this. If so, to use this hypercall
the grant reference is required. How do I get the grant reference? Would
appreciate if I can get an example code or pointer to one to clearly understand
how to achieve this.
If you have any other suggestion of a better way to get contiguous memory in
user space app in Dom0, I am open to that.
On a different note, where can I get detailed hypercall reference guide. Most
palaces I''ve searched have a list of hypercalls but not the description
of each of the commands of arguments of the hypercall.
Thanks in advance.
Regards
Kashmira
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Jan-13 21:31 UTC
Re: [Xen-devel] Contiguous DMAble memory in Dom0 user space linux process
On Wed, Jan 13, 2010 at 01:03:39PM -0800, Kashmira. K. wrote:> Hello Xen experts, > > I am trying to get contiguous DMAble memory in user space application which is a processes running on xen capable Linux.Why? Is the driver you are communicating with doing DMA of huge chunks of memory (bigger than 2MB?) There surely must be another way to make your application do what it has to do than allocating such a large swath of memory.> Is it possible to use the HYPERVISOR_grant_table_op hypercall passing GMTTABOP_map_grant_ref as command to achieve this. If so, to use this hypercall the grant reference is required. How do I get the grant reference? Would appreciate if I can get an example code or pointer to one to clearly understand how to achieve this.Grant reference won''t get you contiguous memory. It will get you memory shared between domains. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Kashmira. K.
2010-Jan-15 02:16 UTC
Re: [Xen-devel] Contiguous DMAble memory in Dom0 user space linux process
Hi Konard, Please see response inline --- On Wed, 1/13/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> > Subject: Re: [Xen-devel] Contiguous DMAble memory in Dom0 user space linux process > To: "Kashmira. K." <chimnee@yahoo.com> > Cc: xen-devel@lists.xensource.com, winterfi@cisco.com > Date: Wednesday, January 13, 2010, 1:31 PM > On Wed, Jan 13, 2010 at 01:03:39PM > -0800, Kashmira. K. wrote: > > Hello Xen experts, > > > > I am trying to get contiguous DMAble memory in user > space application which is a processes running on xen > capable Linux. > > Why? Is the driver you are communicating with doing DMA of > huge chunks of memory (bigger than 2MB?) >Actually I''m not trying to communicate with a driver. The user space application is kind of a driver that lives outside of the kernel and communicates with the device directly. On a linux system (non xen) the bigphysarea patch was used to get large contiguous DMA memory in user space. I am trying to do the equivalant on Xen capable linux in Dom0 so the same application will work. Kind of port it from ''linux'' to ''Xen capable linux''.> There surely must be another way to make your application > do what it has > to do than allocating such a large swath of memory. > > > > Is it possible to use the > HYPERVISOR_grant_table_op hypercall passing > GMTTABOP_map_grant_ref as command to achieve this. If so, to > use this hypercall the grant reference is required. How do I > get the grant reference? Would appreciate if I can get > an example code or pointer to one to clearly understand how > to achieve this. > > Grant reference won''t get you contiguous memory. It will > get you memory > shared between domains. >Yes I see now. I had confused the flags options GNTMAP_device_map to be DMAble memory. There is a HYPERVISOR_memory_op which can exchange memory for contiguous one. But then this does not help me direcltly since a hypercall can only be used in the kernel. Thanks Kashmira>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Jan-15 16:59 UTC
Re: [Xen-devel] Contiguous DMAble memory in Dom0 user space linux process
> Actually I''m not trying to communicate with a driver. The user space application is kind of a driver that lives outside of the kernel and communicates with the device directly.OK, but isn''t the hardware visible via the PCI bus? Does it have any PCI BARs defined?> On a linux system (non xen) the bigphysarea patch was used to get large contiguous DMA memory in user space. > I am trying to do the equivalant on Xen capable linux in Dom0 so the same application will work. Kind of port it from ''linux'' to ''Xen capable linux''.Oh boy. I think you have two options: 1) Play with stubdomains. They are based on Mini-OS and allow you to create applications + MiniOS (that runs under Xen). An example of this is the QEMU stubdomain. You can modify your MiniOS to get you virtual addresses that are mapped directly to contingous memory areas. 2). Look at fbmem.c and write a device driver that allows the userspace application to mmap the driver (similar to fb_mem), which in turn sets up a vmalloc-ed area that is physically contigous (by using the xen_create_contiguous_region function to exchange PFNs). _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Kashmira. K.
2010-Jan-20 21:10 UTC
Re: [Xen-devel] Contiguous DMAble memory in Dom0 user space linux process
Hi Konard, Thanks you for your reply. On the (non xen) linux system, the hardware was visible to the userspace app. The PCI bars were mapped. So I do need to get the same functionality on xen capable linux. I''m still a newbie to the xen internals, but I was wondering if I could write a simple linux kernel driver for a xen capable linux to do the following - Map the PCI address (BAR). - Get contiguous memory. - Provide user space ioctl call to get the mapped PCI BAR - Provide user space ioctl to get pointer to the contiguous memory. Will that work? Thanks Kashmira ----- Original Message ---- From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> To: Kashmira. K. <chimnee@yahoo.com> Cc: winterfi@cisco.com; xen-devel@lists.xensource.com Sent: Fri, January 15, 2010 8:59:17 AM Subject: Re: [Xen-devel] Contiguous DMAble memory in Dom0 user space linux process> Actually I''m not trying to communicate with a driver. The user space application is kind of a driver that lives outside of the kernel and communicates with the device directly.OK, but isn''t the hardware visible via the PCI bus? Does it have any PCI BARs defined?> On a linux system (non xen) the bigphysarea patch was used to get large contiguous DMA memory in user space. > I am trying to do the equivalant on Xen capable linux in Dom0 so the same application will work. Kind of port it from ''linux'' to ''Xen capable linux''.Oh boy. I think you have two options: 1) Play with stubdomains. They are based on Mini-OS and allow you to create applications + MiniOS (that runs under Xen). An example of this is the QEMU stubdomain. You can modify your MiniOS to get you virtual addresses that are mapped directly to contingous memory areas. 2). Look at fbmem.c and write a device driver that allows the userspace application to mmap the driver (similar to fb_mem), which in turn sets up a vmalloc-ed area that is physically contigous (by using the xen_create_contiguous_region function to exchange PFNs). _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Jan-20 21:42 UTC
Re: [Xen-devel] Contiguous DMAble memory in Dom0 user space linux process
On Wed, Jan 20, 2010 at 01:10:24PM -0800, Kashmira. K. wrote:> Hi Konard, > > Thanks you for your reply. > > On the (non xen) linux system, the hardware was visible to the userspace app. The PCI bars were mapped. So I do need to get the same functionality on xen capable linux. > > I''m still a newbie to the xen internals, but I was wondering if I could write a simple linux kernel driver for a xen capable linux to do the following > - Map the PCI address (BAR). > - Get contiguous memory. > - Provide user space ioctl call to get the mapped PCI BAR > - Provide user space ioctl to get pointer to the contiguous memory. > > Will that work?Sure. I would recommend you look at the 3w-xxxx.c driver for a simple ioctl subsystem and how to get contiguous memory mapped in. For mmap functionality hpet.c is pretty simple. Foremost, make sure you get a PDF copy of "Understanding the Linux Kernel" as that will answer a lot of your questions. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel