Xen devel - Apr 2012 - (XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016

I am not sure how this is happening, but I can only reproduce this
is I use the "max" parameter in the
''dom0_mem=*:max:*'' combination:

sh-4.1# xl info | grep dom0_mem
xen_commandline        : cpuinfo dom0_mem=1G,max:2711M cpufreq=verbose
com1=115200,8n1 console=com1,vga loglvl=all guest_loglvl

02:32:22 # 19 :/sys/devices/system/xen_memory/xen_memory0/
> cat /proc/meminfo |grep Direct
DirectMap4k:     2776516 kB
DirectMap2M:           0 kB

02:33:11 # 23
:/sys/devices/system/xen_memory/xen_memory0/
> echo 2776516 > target_kb
(XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016
(XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 (239 of
256)
(XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016
(XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 (0 of
17)
(XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016
(XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 (0 of
17)
>
The value that it decided it was Ok with was: 2776510kB which is 6 pages short
of the goal and that makes the messages disappear.

Any ideas of what that might be? Could it be the shared_info, hypercall page,
start_info, xenconsole and some other ones are the magic 6 pages which
inhibit how much we can balloon up to?

>>> On 27.04.12 at 04:34, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> I am not sure how this is happening, but I can only reproduce this
> is I use the "max" parameter in the
''dom0_mem=*:max:*'' combination:
Of course - without the ",max:" there just isn''t any limit
for Dom0.
> sh-4.1# xl info | grep dom0_mem
> xen_commandline        : cpuinfo dom0_mem=1G,max:2711M cpufreq=verbose 
> com1=115200,8n1 console=com1,vga loglvl=all guest_loglvl> 
> 
> 02:32:22 # 19 :/sys/devices/system/xen_memory/xen_memory0/ 
>> cat /proc/meminfo |grep Direct
> DirectMap4k:     2776516 kB
> DirectMap2M:           0 kB
> 
> 02:33:11 # 23 :/sys/devices/system/xen_memory/xen_memory0/
>> echo 2776516 > target_kb
> 
> (XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016
> (XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 
> (239 of 256)
> (XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016
> (XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 (0
> of 17)
> (XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 694017 > 694016
> (XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 (0
> of 17)
> 
>>
> The value that it decided it was Ok with was: 2776510kB which is 6 pages 
> short of the goal and that makes the messages disappear.
How would that be? 2711MiB = 2776064kiB, which 446k off the value
above. And apart from that, the value above isn''t even divisible by 4
(i.e. not an even number of pages).
> Any ideas of what that might be? Could it be the shared_info, hypercall
page,
> start_info, xenconsole and some other ones are the magic 6 pages which
> inhibit how much we can balloon up to?
Not likely: The hypercall page is in kernel (image) memory, and there''s
no console page at all fro Dom0.

Jan

> How would that be? 2711MiB = 2776064kiB, which 446k off the value
> above. And apart from that, the value above isn''t even divisible
by 4
I messed up on that. Redid the numbers and I was off.
> (i.e. not an even number of pages).
To make this a bit easier I used ''dom0_max=max:3G'', which
means
(with this swiss-cheese type E820 on this Intel box):

[    0.000000] Released 75745 pages of unused memory

so I should have 75745 pages left to play with. But what I found is that
I can only go up to 786415 which is 17 pages short of the 786432 goal.

Here are the steps:

$cat `find /sys -name current_kb`
2842816
$echo $((3*1024*1024))
3145728
$echo "3145728" > `find /sys -name target_kb`
$cat `find /sys -name current_kb`
3145660
$xl dmesg | tail
(XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 786433 (786432) >
786432
(XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 (0 of
17)
> > Any ideas of what that might be? Could it be the shared_info,
hypercall page,
> > start_info, xenconsole and some other ones are the magic 6 pages which
> > inhibit how much we can balloon up to?
> 
> Not likely: The hypercall page is in kernel (image) memory, and
there''s
> no console page at all fro Dom0.
17 pages.. Hmm
> 
> Jan

On Fri, Apr 27, 2012 at 10:31:22AM -0400, Konrad Rzeszutek Wilk
wrote:
> > How would that be? 2711MiB = 2776064kiB, which 446k off the value
> > above. And apart from that, the value above isn''t even
divisible by 4
> 
> I messed up on that. Redid the numbers and I was off.
> 
> > (i.e. not an even number of pages).
> 
> To make this a bit easier I used ''dom0_max=max:3G'', which
means
> (with this swiss-cheese type E820 on this Intel box):
> 
> [    0.000000] Released 75745 pages of unused memory
> 
> so I should have 75745 pages left to play with. But what I found is that
> I can only go up to 786415 which is 17 pages short of the 786432 goal.
> 
> Here are the steps:
> 
> $cat `find /sys -name current_kb`
> 2842816
> $echo $((3*1024*1024))
> 3145728
> $echo "3145728" > `find /sys -name target_kb`
> $cat `find /sys -name current_kb`
> 3145660
> $xl dmesg | tail
> (XEN) page_alloc.c:1148:d0 Over-allocation for domain 0: 786433 (786432)
> 786432
> (XEN) memory.c:133:d0 Could not allocate order=0 extent: id=0 memflags=0 (0
of 17)
> 
> > > Any ideas of what that might be? Could it be the shared_info,
hypercall page,
> > > start_info, xenconsole and some other ones are the magic 6 pages
which
> > > inhibit how much we can balloon up to?
> > 
> > Not likely: The hypercall page is in kernel (image) memory, and
there''s
> > no console page at all fro Dom0.
> 
> 17 pages.. Hmm
I am still not exactly sure what the problem is, but by running this on
various machines I found that I can be off by 1,2,3, 4 or 17 pages. It
looked to vary based on the amount of ACPI tables that showed up in MADT.

So I think what is happening is that the initial domain gets the ACPI
regions (which are shared) accounted in its d->tot_pages. I can''t
pinpoint the exact piece of code in the hypervisor for this.

But what I did do on the Linux side was using the current_reservation
hypercall (so d->tot_pages) and based on that would populate exactly
up to start_info->nr_pages - d->tot_pages pages and did not get any of
those errors.
> > 
> > Jan
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel