Hi,
Moving on to 256 GB guest, the hyp is failing the XENMEM_populate_physmap
hcall in compat_memory_op(). The problem is size too large for continuation
encoding:
/* Is size too large for us to encode a continuation? */
if ( cmp.rsrv.nr_extents > (UINT_MAX >> MEMOP_EXTENT_SHIFT))
return start_extent;
for 256 GB : nr_extents == 0x4000000
Currently at a loss on this one!
Thanks,
Mukesh
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
On 18/02/2009 03:39, "Mukesh Rathor" <mukesh.rathor@oracle.com> wrote:> Moving on to 256 GB guest, the hyp is failing the XENMEM_populate_physmap > hcall in compat_memory_op(). The problem is size too large for continuation > encoding: > > /* Is size too large for us to encode a continuation? */ > if ( cmp.rsrv.nr_extents > (UINT_MAX >> MEMOP_EXTENT_SHIFT)) > return start_extent; > > for 256 GB : nr_extents == 0x4000000 > > Currently at a loss on this one!Well, who''s making the compat call? Not the guest itself presumably since it is 64-bit? So it''s probably dom0? But I would think that dom0 would only do large amounts of allocation for the new domU in xc_hvm_build.c, and that is careful to allocate memory in batches of 8MB at a time. Basically you need to track down the call site of the failed compat_memory_op(). -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser wrote:
> On 18/02/2009 03:39, "Mukesh Rathor"
<mukesh.rathor@oracle.com> wrote:
>
>> Moving on to 256 GB guest, the hyp is failing the
XENMEM_populate_physmap
>> hcall in compat_memory_op(). The problem is size too large for
continuation
>> encoding:
>>
>> /* Is size too large for us to encode a continuation? */
>> if ( cmp.rsrv.nr_extents > (UINT_MAX >>
MEMOP_EXTENT_SHIFT))
>> return start_extent;
>>
>> for 256 GB : nr_extents == 0x4000000
>>
>> Currently at a loss on this one!
>
> Well, who''s making the compat call? Not the guest itself
presumably since it
> is 64-bit? So it''s probably dom0? But I would think that dom0
would only do
> large amounts of allocation for the new domU in xc_hvm_build.c, and that
is
> careful to allocate memory in batches of 8MB at a time.
>
> Basically you need to track down the call site of the failed
> compat_memory_op().
>
> -- Keir
>
Hi Keir,
I see this in domain-builder-ng.log:
elf_xen_addr_calc_check: addresses:
virt_base = 0xffffffff80000000
elf_paddr_offset = 0xffffffff80000000
virt_offset = 0x0
virt_kstart = 0xffffffff80200000
virt_kend = 0xffffffff807014e4
virt_entry = 0xffffffff80200000
xc_dom_parse_elf_kernel: xen-3.0-x86_64: 0xffffffff80200000 ->
0xffffffff807014e4
xc_dom_mem_init: mem 262144 MB, pages 0x4000000 pages, 4k each
xc_dom_mem_init: 0x4000000 pages
xc_dom_boot_mem_init: called
x86_compat: guest xen-3.0-x86_64, address size 64
xc_dom_malloc : 256 MB
xc_dom_boot.c:143: panic: xc_dom_boot_mem_init: can''t allocate low
memory for domain
I set a breakpoint in compat_memory_op() and stepping thru it, noticed
it was bailing out at the if statement, and nr_extents was 0x4000000.
This is a PV domain BTW, I guess xc_hvm_build.c wont'' be involved.
I''ll
instrument libxc and debug more... meanwhile I thought I''d post this.
Thanks,
Mukesh
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
On 18/02/2009 19:52, "Mukesh Rathor" <mukesh.rathor@oracle.com> wrote:> This is a PV domain BTW, I guess xc_hvm_build.c wont'' be involved. I''ll > instrument libxc and debug more... meanwhile I thought I''d post this.Yes, my mistake, and this points us straight at the problem. You see the xc_domain_memory_populate_physmap() call at the end of libxc/xc_dom_x86.c:arch_setup_meminit()? That needs to be put in a for loop, allocating no more than, say, a million pages at a time. The patch should be very simple, so I''ll leave the rest to you. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Keir,
Attached please find my patch to allocate 1M pages at a time. With this patch
on 3.3.1 and unstable, I''m able to bring up the hypervisor with 512GB
memory
and start a PV 64 guest (with my pud entry fix).
The only issue is performance related. As somewhat expected, after guest
shutdown it takes a while for pages to be freed and guest to disappear. Until
this time, system is fine, but xm will hang.
In my kdb debugger, I see cpu''s in :
[0]ffff828c80110f52: page_scrub_softirq+22 test %al, %al
[1]ffff828c80110f52: page_scrub_softirq+22 test %al, %al
[2]ffff828c80110f52: page_scrub_softirq+22 test %al, %al
...
page_scrub_softirq():
/* free_heap_pages() does not parallelise well. Serialise this function. */
if ( !spin_trylock(&serialise_lock) )
I guess a future work item to look into parallelizing this some day....
Thanks a lot,
Mukesh
Keir Fraser wrote:
> On 18/02/2009 19:52, "Mukesh Rathor"
<mukesh.rathor@oracle.com> wrote:
>
>> This is a PV domain BTW, I guess xc_hvm_build.c wont'' be
involved. I''ll
>> instrument libxc and debug more... meanwhile I thought I''d
post this.
>
> Yes, my mistake, and this points us straight at the problem.
>
> You see the xc_domain_memory_populate_physmap() call at the end of
> libxc/xc_dom_x86.c:arch_setup_meminit()? That needs to be put in a for
loop,
> allocating no more than, say, a million pages at a time.
>
> The patch should be very simple, so I''ll leave the rest to you.
>
> -- Keir
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel