Xen devel - Oct 2010 - rootfs as RAMDisk + Hypervisor: Cannot allocate memory

Hello world,

i have the follow problem and hope to find help here.

Facts:
- XenparavirtOps-Kernel 2.6.32.23 (jeremys git)
- Xen 4.0.1 (from tarball at xensource)
- rootfs is a ext3 initrd RAMDisk (size 640M (655360)) with 130MB free space
- Hypervisor, Kernel and Ramdisk are booting from a TFTPServer with Syslinux
Multiboot (mboot.c32)
- Boot parameters: $hypervisor dom0_mem=2048M loglvl=all guest_loglvl=all
dom0_max_vcpus=1 dom0_vcpus_pin --- $kernel ramdisk_size=655360
root=/dev/ram0 rw $otheroptions --- $ramdisk

The boot-progress and the system integrity without the XEN-Hypervisor are
fine, all works without any errors or problems.
Required modules are loaded without any problems.

With Hypervisor i get various error messages on boot or in system from user
space utilities and their modules.
e.g. bonding or iscsi-module:

-----------------------------------------------------------
[hostname ~]# modprobe iscsi_tcp
Warning: at mm/vmalloc.c:108 vmap_page_range_noflush+0x22/0x2e2()
Hardware name: IBM eServer BladeCenter HS21 -[7995A1G]-
Pid: 2344, comm: modprobe Tainted: C       W 2.6.32.23-pvops #2
Call Trace:
[<ffffffffxxxxxxxx>] ? warn_slowpath_common
[<ffffffffxxxxxxxx>] ? vmap_page_range_noflush
[<ffffffffxxxxxxxx>] ? map_vm_area
[<ffffffffxxxxxxxx>] ? __vmalloc_area_node
[<ffffffffxxxxxxxx>] ? load_module
[<ffffffffxxxxxxxx>] ? do_page_fault
[<ffffffffxxxxxxxx>] ? sys_init_module
[<ffffffffxxxxxxxx>] ? sys_call_fastpath
---[ end trace xxxxxxxx ]---
FATAL: Error inserting iscsi_tcp
(/lib/modules/2.6.32.23-pvops/kernel/driver/scsi/iscsi_tcp.ko): Cannot
allocate memory
-----------------------------------------------------------
(or see http://picfront.de/d/7S63 for screenshot of management console)


I have that problem with different hardware.
But the funny is, if the rootfs size 490000 (478MB) and the ramdisk_size
550000 (537MB), all work! (but need more space in filesystem), the
filesystem use 476MB).

Perhaps is the problem based on free memory management of xen-hypervisor?
(no-bootscrub param. was tested).



Best regards
Thank you for your time!

Robyn


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>>> On 07.10.10 at 13:21, Robyn Bachofer
<r.bachofer@googlemail.com> wrote:
> [hostname ~]# modprobe iscsi_tcp
> Warning: at mm/vmalloc.c:108 vmap_page_range_noflush+0x22/0x2e2()
> Hardware name: IBM eServer BladeCenter HS21 -[7995A1G]-
> Pid: 2344, comm: modprobe Tainted: C       W 2.6.32.23-pvops #2
> Call Trace:
> [<ffffffffxxxxxxxx>] ? warn_slowpath_common
> [<ffffffffxxxxxxxx>] ? vmap_page_range_noflush
> [<ffffffffxxxxxxxx>] ? map_vm_area
> [<ffffffffxxxxxxxx>] ? __vmalloc_area_node
> [<ffffffffxxxxxxxx>] ? load_module
> [<ffffffffxxxxxxxx>] ? do_page_fault
> [<ffffffffxxxxxxxx>] ? sys_init_module
> [<ffffffffxxxxxxxx>] ? sys_call_fastpath
> ---[ end trace xxxxxxxx ]---
> FATAL: Error inserting iscsi_tcp
> (/lib/modules/2.6.32.23-pvops/kernel/driver/scsi/iscsi_tcp.ko): Cannot
> allocate memory
Can you match this to a source line? Namely, if it maps to

		if (WARN_ON(!pte_none(*pte)))

in vmap_pte_range(), it would suggest there''s some page table
cleanup missing after the initrd is no longer needed at its original
location (with the sizes you provided its quite certain that it runs
into the virtual address range used for modules). For native,
there''s no mapping of the initrd following the kernel image (and
honestly I don''t know why Xen specifies it as being mapped
there - if you had one of about 2G in size, Xen would refuse to
load your Dom0 altogether; perhaps time for another ELF note
indicating that the kernel can do without the initrd being
mapped into virtual space), so Xen kernels need to clean up
the left over page table entries, and apparently the pv-ops
code didn''t inherit the respective XenoLinux bits.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hello Jan,

that''s a good guess.
I found your pasted line in mm/vmalloc.c, line 108.
What can i do? A workaround is to change from ramdisk to nfs, but this is
not fine.
Or a way to "fix" this problem?

Robyn


2010/10/7 Jan Beulich <JBeulich@novell.com>
> >>> On 07.10.10 at 13:21, Robyn Bachofer
<r.bachofer@googlemail.com>
> wrote:
> > [hostname ~]# modprobe iscsi_tcp
> > Warning: at mm/vmalloc.c:108 vmap_page_range_noflush+0x22/0x2e2()
> > Hardware name: IBM eServer BladeCenter HS21 -[7995A1G]-
> > Pid: 2344, comm: modprobe Tainted: C       W 2.6.32.23-pvops #2
> > Call Trace:
> > [<ffffffffxxxxxxxx>] ? warn_slowpath_common
> > [<ffffffffxxxxxxxx>] ? vmap_page_range_noflush
> > [<ffffffffxxxxxxxx>] ? map_vm_area
> > [<ffffffffxxxxxxxx>] ? __vmalloc_area_node
> > [<ffffffffxxxxxxxx>] ? load_module
> > [<ffffffffxxxxxxxx>] ? do_page_fault
> > [<ffffffffxxxxxxxx>] ? sys_init_module
> > [<ffffffffxxxxxxxx>] ? sys_call_fastpath
> > ---[ end trace xxxxxxxx ]---
> > FATAL: Error inserting iscsi_tcp
> > (/lib/modules/2.6.32.23-pvops/kernel/driver/scsi/iscsi_tcp.ko): Cannot
> > allocate memory
>
> Can you match this to a source line? Namely, if it maps to
>
>                if (WARN_ON(!pte_none(*pte)))
>
> in vmap_pte_range(), it would suggest there''s some page table
> cleanup missing after the initrd is no longer needed at its original
> location (with the sizes you provided its quite certain that it runs
> into the virtual address range used for modules). For native,
> there''s no mapping of the initrd following the kernel image (and
> honestly I don''t know why Xen specifies it as being mapped
> there - if you had one of about 2G in size, Xen would refuse to
> load your Dom0 altogether; perhaps time for another ELF note
> indicating that the kernel can do without the initrd being
> mapped into virtual space), so Xen kernels need to clean up
> the left over page table entries, and apparently the pv-ops
> code didn''t inherit the respective XenoLinux bits.
>
> Jan
>
>

-- 
Robyn Bachofer

Pearl-S.-Buck-Str. 12
D-73037 Göppingen
Telefon: +49 (0) 7161 4018708
Mobil: +49 (0) 176 49230082
E-Mail: r.bachofer@googlemail.com


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>>> On 08.10.10 at 09:38, Robyn Bachofer
<r.bachofer@googlemail.com> wrote:
> Hello Jan,
> 
> that''s a good guess.
> I found your pasted line in mm/vmalloc.c, line 108.
> What can i do? A workaround is to change from ramdisk to nfs, but this is
> not fine.
> Or a way to "fix" this problem?
Since you posted to xen-devel (and not xen-users) I supposed you
would be able to do some work here. That''s why I outlined potential
fixes (and the page table cleanup one shouldn''t be more than a
couple of lines). If that''s not the case, you''ll need to talk
someone
into doing this work for you.

Oh, actually I just thought of a possible workaround: Likely your
initrd is over 500Mb in size only in uncompressed form. If in
compressed form it''s well below that size, you could either change
the compression type to anything but gzip (so that GrUB won''t try
to expand it) or use "modulenounzip" instead of "module" in
grub.conf for specifying the initrd.

Jan
> 2010/10/7 Jan Beulich <JBeulich@novell.com>
> 
>> >>> On 07.10.10 at 13:21, Robyn Bachofer
<r.bachofer@googlemail.com>
>> wrote:
>> > [hostname ~]# modprobe iscsi_tcp
>> > Warning: at mm/vmalloc.c:108 vmap_page_range_noflush+0x22/0x2e2()
>> > Hardware name: IBM eServer BladeCenter HS21 -[7995A1G]-
>> > Pid: 2344, comm: modprobe Tainted: C       W 2.6.32.23-pvops #2
>> > Call Trace:
>> > [<ffffffffxxxxxxxx>] ? warn_slowpath_common
>> > [<ffffffffxxxxxxxx>] ? vmap_page_range_noflush
>> > [<ffffffffxxxxxxxx>] ? map_vm_area
>> > [<ffffffffxxxxxxxx>] ? __vmalloc_area_node
>> > [<ffffffffxxxxxxxx>] ? load_module
>> > [<ffffffffxxxxxxxx>] ? do_page_fault
>> > [<ffffffffxxxxxxxx>] ? sys_init_module
>> > [<ffffffffxxxxxxxx>] ? sys_call_fastpath
>> > ---[ end trace xxxxxxxx ]---
>> > FATAL: Error inserting iscsi_tcp
>> > (/lib/modules/2.6.32.23-pvops/kernel/driver/scsi/iscsi_tcp.ko):
Cannot
>> > allocate memory
>>
>> Can you match this to a source line? Namely, if it maps to
>>
>>                if (WARN_ON(!pte_none(*pte)))
>>
>> in vmap_pte_range(), it would suggest there''s some page table
>> cleanup missing after the initrd is no longer needed at its original
>> location (with the sizes you provided its quite certain that it runs
>> into the virtual address range used for modules). For native,
>> there''s no mapping of the initrd following the kernel image
(and
>> honestly I don''t know why Xen specifies it as being mapped
>> there - if you had one of about 2G in size, Xen would refuse to
>> load your Dom0 altogether; perhaps time for another ELF note
>> indicating that the kernel can do without the initrd being
>> mapped into virtual space), so Xen kernels need to clean up
>> the left over page table entries, and apparently the pv-ops
>> code didn''t inherit the respective XenoLinux bits.
>>
>> Jan
>>
>>
> 




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel