Hi, Trying to boot a PV 64bit guest with 255GB mem it hangs in extend_init_mapping() in init-xen.c. The hang is in the loop: /* Finally, blow away any spurious initial mappings. */ while (1) { pmd = (pmd_t *)&page[pmd_index(va)]; if (pmd_none(*pmd)) break; HYPERVISOR_update_va_mapping(va, __pte_ma(0), 0); va += PAGE_SIZE; } More details: tables_space : 0x1ff05000 __START_KERNEL_map == 0xffffffff80000000 &_text == 0xffffffff80200000 start_pfn == 0x20cf8 The va going into above loop is 0xffffffffc0cf5000. This is L3 at 511 which is NULL and so bad things happening. I''m still trying to figure where the initial PTEs are being setup, I don''t see in hypervisor, my fear is libxc, or worse python :).... Could the fix be made in the loop above the above mentioned loop in the function where it''s ensuring init mappings cover kernel+tables to check for pud also? /* Ensure init mappings cover kernel text/data and initial * tables. */ while (va < (__START_KERNEL_map + (start_pfn << PAGE_SHIFT) + tables_space)) { check for pud_none() <======= ???????????? pmd = (pmd_t *)&page[pmd_index(va)]; if (pmd_none(*pmd)) { pte_page = alloc_static_page(&phys); ................ Thanks a lot, Mukesh _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 17/02/2009 04:00, "Mukesh Rathor" <mukesh.rathor@oracle.com> wrote:> Could the fix be made in the loop above the above mentioned loop in the > function where it''s ensuring init mappings cover kernel+tables to check > for pud also?That would be the correct thing to do. I think the kernel should be self-sufficient in this respect, rather than relying on libxc to do this pud setup for it. It should be an easy kernel fix, so long as you can guarantee to be able to easily allocate the page for the pud. :-) -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
hi sir, i am currently doing in m.a.m college of engg in trichy. i am using final year project using xen 3.3.1. the xen 3.3.1 source run on following error displayed how to solve the problem plz tell me.. the following error make world command running:select-repository: Searching `.:..'' for linux-2.6.18-xen.hg select-repository: Ignoring `.'' Unable to determine path to Linux source tree. Falling back to linux-2.6.18-xen Mercurial repository. Cloning http://xenbits.xensource.com/linux-2.6.18-xen.hg to linux-2.6.18-xen.hg. /bin/sh: hg: not found make[3]: *** [linux-2.6.18-xen.hg/.valid-src] Error 127 make[3]: Leaving directory `/home/mamce/Desktop/Missel/xen-3.3.1'' make[2]: *** [linux-2.6-xen-install] Error 2 make[2]: Leaving directory `/home/mamce/Desktop/Missel/xen-3.3.1'' make[1]: *** [install-kernels] Error 1 make[1]: Leaving directory `/home/mamce/Desktop/Missel/xen-3.3.1'' make: *** [world] Error 2 On 2/17/09, Keir Fraser <keir.fraser@eu.citrix.com> wrote:> On 17/02/2009 04:00, "Mukesh Rathor" <mukesh.rathor@oracle.com> wrote: > >> Could the fix be made in the loop above the above mentioned loop in the >> function where it''s ensuring init mappings cover kernel+tables to check >> for pud also? > > That would be the correct thing to do. I think the kernel should be > self-sufficient in this respect, rather than relying on libxc to do this pud > setup for it. It should be an easy kernel fix, so long as you can guarantee > to be able to easily allocate the page for the pud. :-) > > -- Keir > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2009-Feb-17 11:56 UTC
[Xen-devel] Please do not post homework questions to this mailing list
venkatesh k writes ("Re: [Xen-devel] PV domU with 255GB boot failure"):> hi sir, i am currently doing in m.a.m college of engg in trichy. i am > using final year project using xen 3.3.1. the xen 3.3.1 source run on > following error displayed how to solve the problem plz tell me.. > the following error make world command running:select-repository: > Searching `.:..'' for linux-2.6.18-xen.hg > select-repository: Ignoring `.'' > Unable to determine path to Linux source tree. > Falling back to linux-2.6.18-xen Mercurial repository. > Cloning http://xenbits.xensource.com/linux-2.6.18-xen.hg to linux-2.6.18-xen.hg. > /bin/sh: hg: not foundYou have posted this question three times now, each time as a response to some other message chosen apparently at random. Also you have posted it to entirely the wrong place. This is all very rude. The reason we didn''t answer your question the first two times is this: If you can''t figure out what''s wrong from the transcript you have posted, you need the help of your teachers, not of the Xen developers. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser wrote: > On 17/02/2009 04:00, "Mukesh Rathor" <mukesh.rathor@oracle.com> wrote: > >> Could the fix be made in the loop above the above mentioned loop in the >> function where it''s ensuring init mappings cover kernel+tables to check >> for pud also? > > That would be the correct thing to do. I think the kernel should be > self-sufficient in this respect, rather than relying on libxc to do this pud > setup for it. It should be an easy kernel fix, so long as you can guarantee > to be able to easily allocate the page for the pud. :-) > > -- Keir > In that case, attached please find proposed patch. I''m having difficulty currently with building the latest tree, so it was tested on an older version where the function is the same. Please let me know if it looks good. I can resubmit with any changes. thanks, Mukesh _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 18/02/2009 03:39, "Mukesh Rathor" <mukesh.rathor@oracle.com> wrote:> In that case, attached please find proposed patch. I''m having difficulty > currently with building the latest tree, so it was tested on an older > version where the function is the same. Please let me know if it > looks good. I can resubmit with any changes.Looks okay to me. Jan is the only other person I know might remember the Linux x86_64 page-table init code. Please submit with a proper description and sign-off, and double-check your use of tabs versus spaces. Thanks, Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>>> Mukesh Rathor <mukesh.rathor@oracle.com> 18.02.09 04:39 >>>First a general remark: You''re doing this patch to support 256G domains, but by keeping extend_init_mapping() there''ll continue to be no way to support domains with close to or above 512G (or, if making use of XEN_ELFNOTE_INIT_P2M, 1T). This function, rather than needing fixes, really just needs to go away. I''ve done this in our forward ported 2.6.27+ kernels, but unfortunately can''t really contribute the changes to the 2.6.18 tree, as there are too many differences, and I''m simply lacking the time (and, honestly, interest) to work out all the issues. I could post the respective patch if you (or someone else) care(s).>--- init-xen.c.orig 2009-02-17 18:58:58.716954000 -0800 >+++ init-xen.c 2009-02-17 19:33:57.310074000 -0800 >@@ -587,35 +587,45 @@ void __init xen_init_pt(void) > static void __init extend_init_mapping(unsigned long tables_space) > { > unsigned long va = __START_KERNEL_map; >- unsigned long phys, addr, *pte_page; >+ unsigned long phys, addr, *pte_page, *pmd_page; >+ pud_t *pud; > pmd_t *pmd; > pte_t *pte, new_pte; >- unsigned long *page = (unsigned long *)init_level4_pgt; >+ unsigned long *pud_page = (unsigned long *)init_level4_pgt;This is certainly confusing. Please use typeless names here, or names that properly reflect what they''re used for.> >- addr = page[pgd_index(va)]; >- addr_to_page(addr, page); >- addr = page[pud_index(va)]; >- addr_to_page(addr, page); >- >- /* Kill mapping of low 1MB. */ >+ /* Kill low mappings */ > while (va < (unsigned long)&_text) { > if (HYPERVISOR_update_va_mapping(va, __pte_ma(0), 0)) > BUG(); > va += PAGE_SIZE; > } >+ addr = pud_page[pgd_index(va)]; /* get pud entry from pgd tbl */ >+ addr_to_page(addr, pud_page); /* pud_page now va of pud tbl */If you follow above request for using proper (or type-neutral) names, you won''t need extra comments here. Also, the code could stay at the place it was so far (since clearly pgd_index() can never be different for __START_KERNEL_map and _text).> > /* Ensure init mappings cover kernel text/data and initial tables. */ > while (va < (__START_KERNEL_map > + (start_pfn << PAGE_SHIFT) > + tables_space)) { >- pmd = (pmd_t *)&page[pmd_index(va)]; >+ >+ pud = (pud_t *)&pud_page[pud_index(va)]; /* get pud entry */ >+ if (pud_none(*pud)) { >+ pmd_page = alloc_static_page(&phys); >+ early_make_page_readonly( >+ pmd_page, XENFEAT_writable_page_tables); >+ set_pud(pud, __pud(phys | _KERNPG_TABLE)); >+ } else { >+ addr = pud_page[pud_index(va)]; >+ addr_to_page(addr, pmd_page); >+ } >+ >+ pmd = (pmd_t *)&pmd_page[pmd_index(va)]; > if (pmd_none(*pmd)) { > pte_page = alloc_static_page(&phys); > early_make_page_readonly( > pte_page, XENFEAT_writable_page_tables); > set_pmd(pmd, __pmd(phys | _KERNPG_TABLE)); > } else { >- addr = page[pmd_index(va)]; >+ addr = pmd_page[pmd_index(va)]; > addr_to_page(addr, pte_page); > } > pte = (pte_t *)&pte_page[pte_index(va)]; >@@ -630,7 +640,7 @@ static void __init extend_init_mapping(u > > /* Finally, blow away any spurious initial mappings. */ > while (1) {Didn''t you indicate in an earlier mail that we''re lacking a pud_none() check here? Just by adding the pud allocation above, you''re not excluding to run over a pud entry boundary here. Likewise would pmd_page need to be updated here when you cross a pud entry boundary.>- pmd = (pmd_t *)&page[pmd_index(va)]; >+ pmd = (pmd_t *)&pmd_page[pmd_index(va)]; > if (pmd_none(*pmd)) > break; > if (HYPERVISOR_update_va_mapping(va, __pte_ma(0), 0)) >@@ -719,6 +729,7 @@ static void xen_finish_init_mapping(void > table_end = start_pfn; > } > >+ > /* Setup the direct mapping of the physical memory at PAGE_OFFSET. > This runs before bootmem is initialized and gets pages directly from the > physical memory. To access them they are temporarily mapped. */Please omit this last hunk altogether. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, It looks like you don''t have Mercurial (the hg command) installed. You should install your distribution''s package for this, it is used to manage the Xen source and the build process uses it to download the Linux kernel. The appropriate place to post questions of a "How does this work?" / "What does this mean?" nature is to the xen-users mailing list. Also, you''re more likely to get an answer if you start your own e-mail thread - send an e-mail to xen-users@lists.xensource.com with a Subject that briefly describes your question. Don''t reply to another e-mail, else people won''t notice your question is separate. Cheers, Mark On Tuesday 17 February 2009 09:29:34 venkatesh k wrote:> hi sir, i am currently doing in m.a.m college of engg in trichy. i am > using final year project using xen 3.3.1. the xen 3.3.1 source run on > following error displayed how to solve the problem plz tell me.. > the following error make world command running:select-repository: > Searching `.:..'' for linux-2.6.18-xen.hg > select-repository: Ignoring `.'' > Unable to determine path to Linux source tree. > Falling back to linux-2.6.18-xen Mercurial repository. > Cloning http://xenbits.xensource.com/linux-2.6.18-xen.hg to > linux-2.6.18-xen.hg. /bin/sh: hg: not found > make[3]: *** [linux-2.6.18-xen.hg/.valid-src] Error 127 > make[3]: Leaving directory `/home/mamce/Desktop/Missel/xen-3.3.1'' > make[2]: *** [linux-2.6-xen-install] Error 2 > make[2]: Leaving directory `/home/mamce/Desktop/Missel/xen-3.3.1'' > make[1]: *** [install-kernels] Error 1 > make[1]: Leaving directory `/home/mamce/Desktop/Missel/xen-3.3.1'' > make: *** [world] Error 2 > > On 2/17/09, Keir Fraser <keir.fraser@eu.citrix.com> wrote: > > On 17/02/2009 04:00, "Mukesh Rathor" <mukesh.rathor@oracle.com> wrote: > >> Could the fix be made in the loop above the above mentioned loop in the > >> function where it''s ensuring init mappings cover kernel+tables to check > >> for pud also? > > > > That would be the correct thing to do. I think the kernel should be > > self-sufficient in this respect, rather than relying on libxc to do this > > pud setup for it. It should be an easy kernel fix, so long as you can > > guarantee to be able to easily allocate the page for the pud. :-) > > > > -- Keir > > > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich wrote: >>>> Mukesh Rathor <mukesh.rathor@oracle.com> 18.02.09 04:39 >>> > > First a general remark: You''re doing this patch to support 256G domains, > but by keeping extend_init_mapping() there''ll continue to be no way to > support domains with close to or above 512G (or, if making use of > XEN_ELFNOTE_INIT_P2M, 1T). This function, rather than needing fixes, > really just needs to go away. > > I''ve done this in our forward ported 2.6.27+ kernels, but unfortunately > can''t really contribute the changes to the 2.6.18 tree, as there are too > many differences, and I''m simply lacking the time (and, honestly, interest) > to work out all the issues. I could post the respective patch if you (or > someone else) care(s). > I came up with this patch trying to fix the hang on less than 256 GB. With 256 GB it''s not even coming this far, pl see another thread. Since 256 GB was the original bug, we definitely need to support that. So please post your patches with any relevant pointers and I''ll take a crack at it... thanks, Mukesh >> --- init-xen.c.orig 2009-02-17 18:58:58.716954000 -0800 >> +++ init-xen.c 2009-02-17 19:33:57.310074000 -0800 >> @@ -587,35 +587,45 @@ void __init xen_init_pt(void) >> static void __init extend_init_mapping(unsigned long tables_space) >> { >> unsigned long va = __START_KERNEL_map; >> - unsigned long phys, addr, *pte_page; >> + unsigned long phys, addr, *pte_page, *pmd_page; >> + pud_t *pud; >> pmd_t *pmd; >> pte_t *pte, new_pte; >> - unsigned long *page = (unsigned long *)init_level4_pgt; >> + unsigned long *pud_page = (unsigned long *)init_level4_pgt; > > This is certainly confusing. Please use typeless names here, or names > that properly reflect what they''re used for. > >> - addr = page[pgd_index(va)]; >> - addr_to_page(addr, page); >> - addr = page[pud_index(va)]; >> - addr_to_page(addr, page); >> - >> - /* Kill mapping of low 1MB. */ >> + /* Kill low mappings */ >> while (va < (unsigned long)&_text) { >> if (HYPERVISOR_update_va_mapping(va, __pte_ma(0), 0)) >> BUG(); >> va += PAGE_SIZE; >> } >> + addr = pud_page[pgd_index(va)]; /* get pud entry from pgd tbl */ >> + addr_to_page(addr, pud_page); /* pud_page now va of pud tbl */ > > If you follow above request for using proper (or type-neutral) names, you > won''t need extra comments here. > > Also, the code could stay at the place it was so far (since clearly pgd_index() > can never be different for __START_KERNEL_map and _text). > >> /* Ensure init mappings cover kernel text/data and initial tables. */ >> while (va < (__START_KERNEL_map >> + (start_pfn << PAGE_SHIFT) >> + tables_space)) { >> - pmd = (pmd_t *)&page[pmd_index(va)]; >> + >> + pud = (pud_t *)&pud_page[pud_index(va)]; /* get pud entry */ >> + if (pud_none(*pud)) { >> + pmd_page = alloc_static_page(&phys); >> + early_make_page_readonly( >> + pmd_page, XENFEAT_writable_page_tables); >> + set_pud(pud, __pud(phys | _KERNPG_TABLE)); >> + } else { >> + addr = pud_page[pud_index(va)]; >> + addr_to_page(addr, pmd_page); >> + } >> + >> + pmd = (pmd_t *)&pmd_page[pmd_index(va)]; >> if (pmd_none(*pmd)) { >> pte_page = alloc_static_page(&phys); >> early_make_page_readonly( >> pte_page, XENFEAT_writable_page_tables); >> set_pmd(pmd, __pmd(phys | _KERNPG_TABLE)); >> } else { >> - addr = page[pmd_index(va)]; >> + addr = pmd_page[pmd_index(va)]; >> addr_to_page(addr, pte_page); >> } >> pte = (pte_t *)&pte_page[pte_index(va)]; >> @@ -630,7 +640,7 @@ static void __init extend_init_mapping(u >> >> /* Finally, blow away any spurious initial mappings. */ >> while (1) { > > Didn''t you indicate in an earlier mail that we''re lacking a pud_none() check > here? Just by adding the pud allocation above, you''re not excluding to > run over a pud entry boundary here. > > Likewise would pmd_page need to be updated here when you cross a > pud entry boundary. > >> - pmd = (pmd_t *)&page[pmd_index(va)]; >> + pmd = (pmd_t *)&pmd_page[pmd_index(va)]; >> if (pmd_none(*pmd)) >> break; >> if (HYPERVISOR_update_va_mapping(va, __pte_ma(0), 0)) >> @@ -719,6 +729,7 @@ static void xen_finish_init_mapping(void >> table_end = start_pfn; >> } >> >> + >> /* Setup the direct mapping of the physical memory at PAGE_OFFSET. >> This runs before bootmem is initialized and gets pages directly from the >> physical memory. To access them they are temporarily mapped. */ > > Please omit this last hunk altogether. > > Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>>> Mukesh Rathor <mukesh.rathor@oracle.com> 18.02.09 20:12 >>> >Jan Beulich wrote: > > First a general remark: You''re doing this patch to support 256G domains, > > but by keeping extend_init_mapping() there''ll continue to be no way to > > support domains with close to or above 512G (or, if making use of > > XEN_ELFNOTE_INIT_P2M, 1T). This function, rather than needing fixes, > > really just needs to go away. > > > > I''ve done this in our forward ported 2.6.27+ kernels, but unfortunately > > can''t really contribute the changes to the 2.6.18 tree, as there are too > > many differences, and I''m simply lacking the time (and, honestly, interest) > > to work out all the issues. I could post the respective patch if you (or > > someone else) care(s). > > > >I came up with this patch trying to fix the hang on less than 256 GB. >With 256 GB it''s not even coming this far, pl see another thread. Since >256 GB was the original bug, we definitely need to support that. So please >post your patches with any relevant pointers and I''ll take a crack at it...Attached. Just to repeat - they are against a 2.6.27+ kernel that has various other patches enabled, so I can''t easily tell whether they have dependencies on changes made elsewhere. The order we have them applied here is: xen-x86-bigmem xen-x86_64-init-memmap.patch xen-x86_64-note-init-p2m.patch Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel