I''m working on a small OS which is similar in many ways to the Mini-OS
(and
I''ve borrowed much code for the time being). One important difference
is
that it supports multiple processes, each with its own address space.
I''m
having problems setting this up under Xen.
This OS is designed for 32-bit x86 (no PAE), so I intend to use a two-level
paging system. Each process has a page directory - one page with entries
which point to page tables, whose entries describe actual pages. As far as I
can tell, this is what''s done in the Mini-OS source, with page
directories
referred to as L1 page tables, and what I''m calling page tables
referred to
as L2 page tables.
So when creating a process, I do the following:
allocate a page P for the process''s page directory
fill P with zeros
update the existing page table mapping for P (previously set up by what
is essentially the build_pagetable() code from Mini-OS) to give it L1_PROT &
~_PAGE_RW
(using HYPERVISOR_mmu_update(...))
pin the page (using HYPERVISOR_mmuext_op({MMUEXT_PIN_L1_TABLE, ...}...))
All of these steps succeed (the hypercall returns 0, and the argument
indicating how many updates succeeded is 1 in both cases). But later, when
switching into the context of this process, I try to update the user base
pointer with
HYPERVISOR_mmuext_op({MMUEXT_NEW_USER_BASEPTR, mfn of P}...).
This call fails (returns -1, and indicates that 0 operations succeeded). Why
might that be? As I''ve established, the page is zero''d out (so
it''s not
invalid), and it seems to be mapped properly (not writable, pinned L1 page
table). So why would this operation fail?
Thanks in advance,
Dave Pacheco
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
On 27/4/07 21:11, "Dave Pacheco" <dap@cs.brown.edu> wrote:> All of these steps succeed (the hypercall returns 0, and the argument > indicating how many updates succeeded is 1 in both cases). But later, when > switching into the context of this process, I try to update the user base > pointer with > > HYPERVISOR_mmuext_op({MMUEXT_NEW_USER_BASEPTR, mfn of P}...).Only x86/64 has a separate USER_BASEPTR. You want to use just MMUEXT_NEW_BASEPTR. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 4/27/07, Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:> > On 27/4/07 21:11, "Dave Pacheco" <dap@cs.brown.edu> wrote: > > All of these steps succeed (the hypercall returns 0, and the argument > indicating how many updates succeeded is 1 in both cases). But later, when > switching into the context of this process, I try to update the user base > pointer with > > HYPERVISOR_mmuext_op({MMUEXT_NEW_USER_BASEPTR, mfn of P}...). > > > Only x86/64 has a separate USER_BASEPTR. You want to use just > MMUEXT_NEW_BASEPTR. >When I use that, the domain immediately crashes. I would have expected a page fault, since the page tables aren''t actually set up yet, but I don''t even get that. (It''s possible that it faulted again trying to load the fault handler.) /var/log/xen/xend.log doesn''t say much useful, other than that the domain crashed. Is there a way to tell why a domain crashed (i.e. executing an illegal instruction, or triple fault, or something like that)? Thanks, Dave _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2007-Apr-28 01:14 UTC
Re: [Xen-devel] Setting up page directories and tables
Dave Pacheco wrote:> > /var/log/xen/xend.log doesn''t say much useful, other than that the > domain crashed. Is there a way to tell why a domain crashed (i.e. > executing an illegal instruction, or triple fault, or something like > that)?If you compile Xen with debug mode turned on, and then you get useful stuff on the console. It''s generally fairly cryptic, but enough to give you some hints. Also, you can use gdbserver-xen to poke around in your domain so long as it has valid pagetables. It''s useful for stepping up to your cr3 load and looking at the state of things. I see from your description that you''re pinning L1 pages. That''s generally not how Xen linux kernels do it these days; they just pin the whole pagetable from L2 down (in your non-PAE case). Since pinning from top-down is the same as loading cr3 from a validity-checking perspective, it''s useful to see if Xen likes your pagetables without actually trashing them (if you load a bad cr3, you end up in limbo with no useful pagetables, and Xen can''t even print a useful stack backtrace as it destroys your domain). Also, remember that all pages in your new pagetable have to be mapped RO in *all* other pagetables. The easiest way to achieve that is to make all pagetables share the same L1 pages in your "kernel" address space, assuming you have such a thing. Otherwise when you''re allocating a new page for a pagetable, you have to walk through all other pagetables and be sure to update any mappings for that page to RO. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Mark Williamson
2007-May-01 16:27 UTC
Re: [Xen-devel] Setting up page directories and tables
> This OS is designed for 32-bit x86 (no PAE), so I intend to use a two-level > paging system. Each process has a page directory - one page with entries > which point to page tables, whose entries describe actual pages. As far as > I can tell, this is what''s done in the Mini-OS source, with page > directories referred to as L1 page tables, and what I''m calling page tables > referred to as L2 page tables.I think you''ve got this backwards: in Xen code, L1 page tables are the leaf nodes, L2 are their parents, L3 are their parents, etc etc. Cheers, Mark> So when creating a process, I do the following: > allocate a page P for the process''s page directory > fill P with zeros > update the existing page table mapping for P (previously set up by what > is essentially the build_pagetable() code from Mini-OS) to give it L1_PROT > & ~_PAGE_RW > (using HYPERVISOR_mmu_update(...)) > pin the page (using HYPERVISOR_mmuext_op({MMUEXT_PIN_L1_TABLE, > ...}...)) > > All of these steps succeed (the hypercall returns 0, and the argument > indicating how many updates succeeded is 1 in both cases). But later, when > switching into the context of this process, I try to update the user base > pointer with > > HYPERVISOR_mmuext_op({MMUEXT_NEW_USER_BASEPTR, mfn of P}...). > > This call fails (returns -1, and indicates that 0 operations succeeded). > Why might that be? As I''ve established, the page is zero''d out (so it''s not > invalid), and it seems to be mapped properly (not writable, pinned L1 page > table). So why would this operation fail? > > Thanks in advance, > Dave Pacheco-- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 4/27/07, Jeremy Fitzhardinge <jeremy@xensource.com> wrote:> I see from your description that you''re pinning L1 pages. That''s > generally not how Xen linux kernels do it these days; they just pin the > whole pagetable from L2 down (in your non-PAE case). Since pinning from > top-down is the same as loading cr3 from a validity-checking > perspective, it''s useful to see if Xen likes your pagetables without > actually trashing them (if you load a bad cr3, you end up in limbo with > no useful pagetables, and Xen can''t even print a useful stack backtrace > as it destroys your domain).I don''t really understand what you mean here. To "pin the whole pagetable from L2 down" - does that mean pinning the L2 pagetable page itself and then all of the L1 pages, or just the L2 pagetable page? What exactly does pinning do? Does it simply ensure that the page won''t be paged out and that Xen knows it''s a page table? If that''s the case, it seems like you still want to pin L1 pages so that you know that Xen is okay with your L1 page table. Thanks, Dave _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2007-May-01 21:42 UTC
Re: [Xen-devel] Setting up page directories and tables
Dave Pacheco wrote:> I don''t really understand what you mean here. To "pin the whole > pagetable from L2 down" - does that mean pinning the L2 pagetable page > itself and then all of the L1 pages, or just the L2 pagetable page? > > What exactly does pinning do? Does it simply ensure that the page > won''t be paged out and that Xen knows it''s a page table? If that''s the > case, it seems like you still want to pin L1 pages so that you know > that Xen is okay with your L1 page table."Pinning" is a bit of a misnomer. What it means is that the pinned page is always treated as part of an active pagetable (ie, pointed to by %cr3), so all the normal rules apply: it must be RO, and all updates must go through Xen and be validated. This means that Xen doesn''t need to revalidate it when it does become an active pagetable. By definition, if you pin a pagetable page, you also pin all the other pagetable pages it refers to. So if you pin the L2 (L3 for PAE) root page, then you''re pinning the entire pagetable. You could pin the leaf L1 pages (ie the ones actually containing pte entries), but there''s not much point. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Mark Williamson
2007-May-02 00:41 UTC
Re: [Xen-devel] Setting up page directories and tables
On Tuesday 01 May 2007, Dave Pacheco wrote:> On 4/27/07, Jeremy Fitzhardinge <jeremy@xensource.com> wrote: > > I see from your description that you''re pinning L1 pages. That''s > > generally not how Xen linux kernels do it these days; they just pin the > > whole pagetable from L2 down (in your non-PAE case). Since pinning from > > top-down is the same as loading cr3 from a validity-checking > > perspective, it''s useful to see if Xen likes your pagetables without > > actually trashing them (if you load a bad cr3, you end up in limbo with > > no useful pagetables, and Xen can''t even print a useful stack backtrace > > as it destroys your domain). > > I don''t really understand what you mean here. To "pin the whole pagetable > from L2 down" - does that mean pinning the L2 pagetable page itself and > then all of the L1 pages, or just the L2 pagetable page?L2 is the top level of the hierarchy, so if you pin it all the L1 (leaf) pagetables will get validated in the process.> What exactly does pinning do? Does it simply ensure that the page won''t be > paged out and that Xen knows it''s a page table? If that''s the case, it > seems like you still want to pin L1 pages so that you know that Xen is okay > with your L1 page table.Pinning gets Xen to validate the pagetable. It can subsequently be reloaded without validation until it''s unpinned (otherwise Xen would need to validate it each time it was loaded into CR3). It basically does this by validating the pagetable, then bumping the refcounts so that the pages can''t be used in inappropriate ways (e.g. won''t let them be mapped RW because that would allow malicious guests to break out of their memory). Pinning an L2 table which references L1 tables will cause the L1 tables to be validated as part of this process (ensuring that they don''t contain any inappropriate mappings). Cheers, Mark -- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel