Not very long ago there was a change to this function appearantly to make available the first megabyte of memory. While this is a nice attempt, I wonder how this worked out for anyone not using shadow mode: Since the initial allocation is provided as a read-write mapping, the occasional use of memory from the corresponding physical range to populate page tables (or descriptor tables) results on BUG_ON-s being triggered in hypervisor.c because these pages then cannot be pinned (after converting their phys-to-virt mapping to read-only they still have the initial mapping left read-write, which prevents the necessary type transition from happening). While resolving this I wanted to avoid adding code to make_page_readonly (much like i386 has a special case already, but appearantly to deal with a different scenario), I wondered whether one couldn''t make these unused portions of the initial mapping readonly during early setup; since I wasn''t sure whether I''d encounter any problems with that I didn''t attempt to do so, yet, and rather reverted the original change. Finally I wonder why this first Mb was taken care about, but the potentially larger area at the end of the initial allocation (which to my understanding is currently lost, too) wasn''t. And I am having the understanding that the respective memory ranges for i386 are also lost. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 30 Aug 2005, at 14:19, Jan Beulich wrote:> Finally I wonder why this first Mb was taken care about, but the > potentially larger area at the end of the initial allocation (which to > my understanding is currently lost, too) wasn''t. And I am having the > understanding that the respective memory ranges for i386 are also lost.x86_64 now completely unmaps low 1MB from the initial mapping area at end of extend_init_mapping(). I don''t think any memory is now lost in either i386 or x86_64 (i386 already freed up the low 1MB). -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>x86_64 now completely unmaps low 1MB from the initial mapping area at >end of extend_init_mapping(). I don''t think any memory is now lost in>either i386 or x86_64 (i386 already freed up the low 1MB).Hmm, interesting. If that change was done along with the one to contig_initmem_init, it would have saved me well over a day of debugging... Anyway, I can''t see where the respective freeing is done on i386, and I still can''t see how the tail part of the initial mapping is being made use of on either 32- or 64bits. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 30 Aug 2005, at 14:53, Jan Beulich wrote:> Hmm, interesting. If that change was done along with the one to > contig_initmem_init, it would have saved me well over a day of > debugging...It needed to be debugged here too. I can''t foresee all bugs unfortunately. :-)> Anyway, I can''t see where the respective freeing is done on i386, and I > still can''t see how the tail part of the initial mapping is being made > use of on either 32- or 64bits.For i386, it is freed to the bootmem allocator in register_bootmem_low_pages(), which walks a fake e820 table that is created by machine_specific_memory_setup() in include/asm-xen/asm-i386/mach-xen/setup_arch_post.h. The tail part of the initial mapping has no special handling on i386 nor on x86_64. It just gets freed up when we free from 0 up to max_pfn, and it never gets reserved (the reserved region precisely covers kernel text/data and initial page tables). Actually, that could be another bug on x86/64 -- I may need to truncate the initial mapping, or we may be ending up with spurious extra writable mappings to some pages... I''ll take a look and see if this is the case. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 30 Aug 2005, at 15:07, Keir Fraser wrote:> The tail part of the initial mapping has no special handling on i386 > nor on x86_64. It just gets freed up when we free from 0 up to > max_pfn, and it never gets reserved (the reserved region precisely > covers kernel text/data and initial page tables). > > Actually, that could be another bug on x86/64 -- I may need to > truncate the initial mapping, or we may be ending up with spurious > extra writable mappings to some pages... I''ll take a look and see if > this is the case.Okay, I see what you mean. init_memory_mapping() sets start_pfn to something after the domain-builder-provided mapping. So we probably lose around 4MB of RAM per domain on x86/64.... that''ll have to be fixed. I didn''t originally write this code, btw. ;-) i386 does not have this concept of an initial/kernel mapping separate from the 1:1 mapping of all phys mem. So this particular problem cannot occur. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>The tail part of the initial mapping has no special handling on i386 >nor on x86_64. It just gets freed up when we free from 0 up tomax_pfn,>and it never gets reserved (the reserved region precisely coverskernel>text/data and initial page tables).For i386 I''m not certain, but for x86-64 I doubt that: init_memory_mapping, which runs before contig_initmem_init, re-initializes start_pfn (which is what in turn gets used to set up the bootmem reservation) from the result of scanning the initial page tables. These, as I understand it, extend to the 4-Mb-rounded end of the initial mapping (which, if the unused tail turns out to be less than 512k, even gets extended by an extra 4M).>Actually, that could be another bug on x86/64 -- I may need totruncate>the initial mapping, or we may be ending up with spurious extra >writable mappings to some pages... I''ll take a look and see if this is>the case.If the above wasn''t true (or was fixed), then I''d assume such a bug would surface (and again I''m not sure why i386 wouldn''t surface it, as I can''t see where these mappings get torn down). Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich wrote:>> The tail part of the initial mapping has no special handling on i386 >> nor on x86_64. It just gets freed up when we free from 0 up to >> max_pfn, and it never gets reserved (the reserved region precisely >> covers kernel text/data and initial page tables). > > For i386 I''m not certain, but for x86-64 I doubt that: > init_memory_mapping, which runs before contig_initmem_init, > re-initializes start_pfn (which is what in turn gets used to set up > the bootmem reservation) from the result of scanning the initial page > tables. These, as I understand it, extend to the 4-Mb-rounded end of > the initial mapping (which, if the unused tail turns out to be less > than 512k, even gets extended by an extra 4M). >Okay, I wrote the code originally. It''s extended to access all the pge, pud, pmd, pte pages when establishing 1:1 direct mapping against the guest physical memory. Unlike the native x86_64 linux, the current x86_64 xenlinux cannot use 2MB, so we need to allocate a lot of (extra) L1 pages if the guest memory is large. For those page table pages, I set RO at contig_initmem_init. I don''t think it''s 4-Mb-rounded, but I''ll take a look at the code. BTW, when do/did you start seeing the problme?>> Actually, that could be another bug on x86/64 -- I may need to >> truncate the initial mapping, or we may be ending up with spurious >> extra writable mappings to some pages... I''ll take a look and see if >> this is > >> the case. > > If the above wasn''t true (or was fixed), then I''d assume such a bug > would surface (and again I''m not sure why i386 wouldn''t surface it, > as I can''t see where these mappings get torn down). > > Jan >Jun --- Intel Open Source Technology Center _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>BTW, when do/did you start seeing the problme?You mean the original one? When we first upgraded to a changeset new enough to contain the HIGH_MEMORY change in contig_initmem_init. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich wrote:>> BTW, when do/did you start seeing the problme? > > You mean the original one? When we first upgraded to a changeset new > enough to contain the HIGH_MEMORY change in contig_initmem_init. > > Jan >Maybe the changeset is missing something. I''ll take a look at it. Do you have any setups (e.g. dom0_mem= , mem=, size of RAM, etc.) that help me reproduce the problem? I use a couple of different machines with different configurations (1GB-8GB, UP-8-way), but I don''t see the problem yet. Jun --- Intel Open Source Technology Center _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>>> BTW, when do/did you start seeing the problme? >> >> You mean the original one? When we first upgraded to a changesetnew>> enough to contain the HIGH_MEMORY change in contig_initmem_init. >> >> Jan >> >Maybe the changeset is missing something. I''ll take a look at it. Doyou>have any setups (e.g. dom0_mem= , mem=, size of RAM, etc.) that helpme>reproduce the problem? I use a couple of different machines with >different configurations (1GB-8GB, UP-8-way), but I don''t see the >problem yet.I''m not sure we talk about the same thing: The mentioned problem was already fixed by a later change. The remaining part I was talking about refer to just observations on the sources, not problems I experienced. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel