This is a followup to our conversation at the Xen summit about userspace passing virtual addresses to the hypervisor. We talked about an API where data structures would be allocated from a special area of memory. This API became rather hairy, which we can talk about. However, it seems to me that the simplest way to handle this is to disallow pointers entirely, instead embedding the structures in the higher-level structure. I''ve had a look through this, and I actually don''t think that would be too bad. When performing this conversion, we could initially exempt the arch-specific hcalls. For consistency I think we''d want to do them all, but that''s not necessary for correctness. Also, constraining these expanded structures to a single page would be best. So I had a look through most (all?) of the hcalls in xen/include/public/*.h to see which would pose problems. I don''t see any show-stoppers: - some hcalls, such as dom0_setvcpucontext and dom0_getvcpucontext, would be trivial to convert. - I haven''t looked at the ACM hcalls in detail, but I think they would be trivial as well. - xc_readconsolering() would need to copy up to a page of data into the caller''s buffer. - it would introduce a hard restriction on the size of the extent array in the memory operations (though it''s worth noting that the balloon driver already limits this to PAGE_SIZE). - dom0_perfccontrol and dom0_getdomaininfolist would gain restrictions similar to the memory ops. Thoughts? -- Hollis Blanchard IBM Linux Technology Center _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > Message: 1 > Date: Tue, 07 Feb 2006 13:58:05 +1100 > From: Hollis Blanchard <hollisb@us.ibm.com> > Subject: [Xen-devel] pointers in hcalls > To: Keir Fraser <Keir.Fraser@cl.cam.ac.uk> > Cc: xen-devel <xen-devel@lists.xensource.com> > Message-ID: <1139281085.13776.29.camel@localhost.localdomain> > Content-Type: text/plain > > This is a followup to our conversation at the Xen summit about userspace > passing virtual addresses to the hypervisor. > > We talked about an API where data structures would be allocated from a > special area of memory. This API became rather hairy, which we can talk > about. However, it seems to me that the simplest way to handle this is > to disallow pointers entirely, instead embedding the structures in the > higher-level structure. I''ve had a look through this, and I actually > don''t think that would be too bad. > > When performing this conversion, we could initially exempt the > arch-specific hcalls. For consistency I think we''d want to do them all, > but that''s not necessary for correctness. Also, constraining these > expanded structures to a single page would be best. > > So I had a look through most (all?) of the hcalls in > xen/include/public/*.h to see which would pose problems. I don''t see any > show-stoppers: > - some hcalls, such as dom0_setvcpucontext and dom0_getvcpucontext, > would be trivial to convert. > - I haven''t looked at the ACM hcalls in detail, but I think they would > be trivial as well.The acm hypercalls uses pointers when setting and reading the policy from the hypervisor and for dumping statistics. A policy might not necessarily be less than one page. I don''t remember the conversation on the Xen summit and probably wasn''t involved. Would you mind summarizing briefly the discussion?> - xc_readconsolering() would need to copy up to a page of data into the > caller''s buffer. > - it would introduce a hard restriction on the size of the extent array > in the memory operations (though it''s worth noting that the balloon > driver already limits this to PAGE_SIZE). > - dom0_perfccontrol and dom0_getdomaininfolist would gain restrictions > similar to the memory ops. > > Thoughts? > > -- > Hollis Blanchard > IBM Linux TechnologyThanks Reiner _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, 2006-02-07 at 20:20 -0500, Reiner Sailer wrote:> > The acm hypercalls uses pointers when setting and reading the policy > from the hypervisor and for dumping statistics. A policy might not > necessarily be less than one page. > > I don''t remember the conversation on the Xen summit and probably > wasn''t involved. Would you mind summarizing briefly the discussion?(Please CC me on replies.) The domains (especially the management tools in dom0) are passing virtual pointers into the hypervisor. This assumes that the hypervisor is running in the same address space as the userland tools. That assumption is not valid on PPC (we run in real mode). One other port (x86-64 VT iirc) has the same problem, but since an x86 MMU tablewalk is pretty straightforward, they do that to translate the address by hand. This is not feasible on PPC. It would dramatically simplify the problem if these memory areas were limited to one page. Once you go over that, the area could be machine discontiguous, which greatly complicates Xen''s copy_to/from_user(). One possibility would be to install ACM policies in chunks (using multiple hcalls) followed by a "commit" hcall. That would introduce some concurrency issues though (e.g. installing two multi-page policies at once). -- Hollis Blanchard IBM Linux Technology Center _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
hollisb@us.ltcfwd.linux.ibm.com wrote on 02/08/2006 12:14:01 AM:> On Tue, 2006-02-07 at 20:20 -0500, Reiner Sailer wrote: > > > > The acm hypercalls uses pointers when setting and reading the policy > > from the hypervisor and for dumping statistics. A policy might not > > necessarily be less than one page. > > > > I don''t remember the conversation on the Xen summit and probably > > wasn''t involved. Would you mind summarizing briefly the discussion? > > (Please CC me on replies.)sorry.> The domains (especially the management tools in dom0) are passing > virtual pointers into the hypervisor. > > This assumes that the hypervisor is running in the same address space as > the userland tools. That assumption is not valid on PPC (we run in real > mode). One other port (x86-64 VT iirc) has the same problem, but since > an x86 MMU tablewalk is pretty straightforward, they do that to > translate the address by hand. This is not feasible on PPC. > > It would dramatically simplify the problem if these memory areas were > limited to one page. Once you go over that, the area could be machine > discontiguous, which greatly complicates Xen''s copy_to/from_user().Thanks. This info is helpful.> One possibility would be to install ACM policies in chunks (using > multiple hcalls) followed by a "commit" hcall.This is a possibility. The chunks can be committed with the last part. The same way we could retrieve the current policy as well as statistic dumps.>That would introduce some > concurrency issues though (e.g. installing two multi-page policies at > once).Yes, we will need to introduce additional synchronization. i) The ACM could reject a new policy while it is accumulating another policy. Then we need to protect from "hanging" policy updates where the policy download is never completed and blocks future policy updates. ii) The ACM could discard ongoing (i.e., not committed) policy loading if a new policy load is started. This works around the "hanging" policy update problem.> -- > Hollis Blanchard > IBM Linux Technology Center >It''s not elegant but if your solution is the one that is being implemented, then we will find a work-around for the ACM policy calls. Reiner _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
xen-devel-bounces@lists.xensource.com wrote on 02/08/2006 12:14:01 AM:> On Tue, 2006-02-07 at 20:20 -0500, Reiner Sailer wrote: > > > > The acm hypercalls uses pointers when setting and reading the policy > > from the hypervisor and for dumping statistics. A policy might not > > necessarily be less than one page. > > > > I don''t remember the conversation on the Xen summit and probably > > wasn''t involved. Would you mind summarizing briefly the discussion? > > (Please CC me on replies.) > > The domains (especially the management tools in dom0) are passing > virtual pointers into the hypervisor. > > This assumes that the hypervisor is running in the same address space as > the userland tools. That assumption is not valid on PPC (we run in real > mode). One other port (x86-64 VT iirc) has the same problem, but since > an x86 MMU tablewalk is pretty straightforward, they do that to > translate the address by hand. This is not feasible on PPC. > > It would dramatically simplify the problem if these memory areas were > limited to one page. Once you go over that, the area could be machine > discontiguous, which greatly complicates Xen''s copy_to/from_user().I wonder whether we could not implement a wrapper for the copy_from_user function that copies the passed data on a page-by-page basis for those arrays that stretch multiple pages? Stefan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel