Hello. Does linux kernel 2.6.37 suitable to run under xen in PV mode? Does it has now any bugs or regressions? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
FWIW, I was able to successfully boot PV domU guests while using Xen 4.0.2-rc2-pre and 2.6.37-rc7 from the git repo The problem I encountered was the lack of netback support. Couldn''t get any network interfaces to come up. Whoops :) Otherwise it seemed fine, but I didnt run the guests very long. Hope that helps! - Don>>> Vasiliy G Tolstov 01/20/11 2:03 PM >>>Hello. Does linux kernel 2.6.37 suitable to run under xen in PV mode? Does it has now any bugs or regressions? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 2011-01-20 at 14:09 -0600, Don Brearley wrote:> > FWIW, I was able to successfully boot PV domU guests while using Xen > 4.0.2-rc2-pre and > 2.6.37-rc7 from the git repo > > The problem I encountered was the lack of netback support. Couldn''t > get any > network interfaces to come up. Whoops :) Otherwise it seemed fine, > but I didnt > run the guests very long. > > Hope that helps! > - DonSorry, i mean domU, not dom0 =). I can try to move from 2.6.32 from jeremy to vanilla 2.6.37... _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, Jan 20, 2011 at 11:11:09PM +0300, Vasiliy G Tolstov wrote:> On Thu, 2011-01-20 at 14:09 -0600, Don Brearley wrote: > > > > FWIW, I was able to successfully boot PV domU guests while using Xen > > 4.0.2-rc2-pre and > > 2.6.37-rc7 from the git repo > > > > The problem I encountered was the lack of netback support. Couldn''t > > get any > > network interfaces to come up. Whoops :) Otherwise it seemed fine, > > but I didnt > > run the guests very long. > > > > Hope that helps! > > - Don > > Sorry, i mean domU, not dom0 =). I can try to move from 2.6.32 from > jeremy to vanilla 2.6.37... >Yeah, please try it and let us know if you find any issues. One day in the (hopefully near) future we''ll set up automatic upstream (Linus) kernel testing.. so we can catch possible regressions/bugs early during the -rc phase and "final" releases are known to be good.. -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 2011-01-20 at 22:19 +0200, Pasi Kärkkäinen wrote:> On Thu, Jan 20, 2011 at 11:11:09PM +0300, Vasiliy G Tolstov wrote: > > On Thu, 2011-01-20 at 14:09 -0600, Don Brearley wrote: > > > > > > FWIW, I was able to successfully boot PV domU guests while using Xen > > > 4.0.2-rc2-pre and > > > 2.6.37-rc7 from the git repo > > > > > > The problem I encountered was the lack of netback support. Couldn''t > > > get any > > > network interfaces to come up. Whoops :) Otherwise it seemed fine, > > > but I didnt > > > run the guests very long. > > > > > > Hope that helps! > > > - Don > > > > Sorry, i mean domU, not dom0 =). I can try to move from 2.6.32 from > > jeremy to vanilla 2.6.37... > > > > Yeah, please try it and let us know if you find any issues. > > One day in the (hopefully near) future we''ll set up automatic > upstream (Linus) kernel testing.. so we can catch possible regressions/bugs early > during the -rc phase and "final" releases are known to be good.. > > -- Pasi >Now (about 1 week testing) i have now issues. Memory hotplug work''s fine (i''m add xen dom0 support for that :)). Live migration does not have any issues. Xenbus watch work''s fine (in some e-mails i send message, that xenbus wath does not work in my kernel module undex 2.6.32 from jeremy and 2.6.36 from linus. Now i have not tested second serial console... My kernel config , xen section is CONFIG_XEN=y CONFIG_XEN_DOM0=y CONFIG_XEN_PRIVILEGED_GUEST=y CONFIG_XEN_PVHVM=y CONFIG_XEN_MAX_DOMAIN_MEMORY=128 CONFIG_XEN_SAVE_RESTORE=y CONFIG_PCI_XEN=y CONFIG_XEN_PCIDEV_FRONTEND=y CONFIG_XEN_BLKDEV_FRONTEND=y CONFIG_XEN_NETDEV_FRONTEND=y CONFIG_XEN_KBDDEV_FRONTEND=y CONFIG_HVC_XEN=y CONFIG_XEN_FBDEV_FRONTEND=y # Xen driver support CONFIG_XEN_BALLOON=y CONFIG_XEN_SCRUB_PAGES=y CONFIG_XEN_DEV_EVTCHN=y CONFIG_XENFS=y CONFIG_XEN_COMPAT_XENFS=y CONFIG_XEN_SYS_HYPERVISOR=y CONFIG_XEN_PLATFORM_PCI=y CONFIG_SWIOTLB_XEN=y Works fine under Exherbo linux (systemd based init system requires>=2.6.36 kernel)_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 2011-01-20 at 22:19 +0200, Pasi Kärkkäinen wrote:> On Thu, Jan 20, 2011 at 11:11:09PM +0300, Vasiliy G Tolstov wrote: > > On Thu, 2011-01-20 at 14:09 -0600, Don Brearley wrote: > > > > > > FWIW, I was able to successfully boot PV domU guests while using Xen > > > 4.0.2-rc2-pre and > > > 2.6.37-rc7 from the git repo > > > > > > The problem I encountered was the lack of netback support. Couldn''t > > > get any > > > network interfaces to come up. Whoops :) Otherwise it seemed fine, > > > but I didnt > > > run the guests very long. > > > > > > Hope that helps! > > > - Don > > > > Sorry, i mean domU, not dom0 =). I can try to move from 2.6.32 from > > jeremy to vanilla 2.6.37... > > > > Yeah, please try it and let us know if you find any issues. > > One day in the (hopefully near) future we''ll set up automatic > upstream (Linus) kernel testing.. so we can catch possible regressions/bugs early > during the -rc phase and "final" releases are known to be good.. > > -- Pasi >Interesting thing appeared: domU pv running with memory=512 and maxmem=4096 after kernel is booted i see in /var/log/messages Memory: 377324k/4202496k available (4125k kernel code, 448k absent, 3824724k reserved, 1636k data, 640k init) total used free shared buffers cached Mem: 372 264 107 0 29 119 -/+ buffers/cache: 116 256 Swap: 4996 0 4996 xm list says that xen allocated 512M of ram to domain Why xen steals 140M of memory? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, Jan 21, 2011 at 01:42:01AM +0300, Vasiliy G Tolstov wrote:> On Thu, 2011-01-20 at 22:19 +0200, Pasi Kärkkäinen wrote: > > On Thu, Jan 20, 2011 at 11:11:09PM +0300, Vasiliy G Tolstov wrote: > > > On Thu, 2011-01-20 at 14:09 -0600, Don Brearley wrote: > > > > > > > > FWIW, I was able to successfully boot PV domU guests while using Xen > > > > 4.0.2-rc2-pre and > > > > 2.6.37-rc7 from the git repo > > > > > > > > The problem I encountered was the lack of netback support. Couldn''t > > > > get any > > > > network interfaces to come up. Whoops :) Otherwise it seemed fine, > > > > but I didnt > > > > run the guests very long. > > > > > > > > Hope that helps! > > > > - Don > > > > > > Sorry, i mean domU, not dom0 =). I can try to move from 2.6.32 from > > > jeremy to vanilla 2.6.37... > > > > > > > Yeah, please try it and let us know if you find any issues. > > > > One day in the (hopefully near) future we''ll set up automatic > > upstream (Linus) kernel testing.. so we can catch possible regressions/bugs early > > during the -rc phase and "final" releases are known to be good.. > > > > -- Pasi > > > > > Interesting thing appeared: > domU pv running with memory=512 and maxmem=4096 > > after kernel is booted i see in /var/log/messages > Memory: 377324k/4202496k available (4125k kernel code, 448k absent, > 3824724k reserved, 1636k data, 640k init) > > total used free shared buffers cached > Mem: 372 264 107 0 29 > 119 > -/+ buffers/cache: 116 256 > Swap: 4996 0 4996 > > xm list says that xen allocated 512M of ram to domain > Why xen steals 140M of memory?Do you see any messages during bootup about memory being returned? Look for xen_release_chunk .. and pages freed ?> > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, 2011-01-21 at 10:30 -0500, Konrad Rzeszutek Wilk wrote:> Do you see any messages during bootup about memory being returned? Look for > xen_release_chunk .. and pages freed ?xen_release_chunk not found, grep by pages returns: [ 0.000000] released 0 pages of unused memory [ 0.000000] On node 0 totalpages: 1050512 [ 0.000000] DMA zone: 56 pages used for memmap [ 0.000000] DMA zone: 6 pages reserved [ 0.000000] DMA zone: 3922 pages, LIFO batch:0 [ 0.000000] DMA32 zone: 14280 pages used for memmap [ 0.000000] DMA32 zone: 1030200 pages, LIFO batch:31 [ 0.000000] Normal zone: 28 pages used for memmap [ 0.000000] Normal zone: 2020 pages, LIFO batch:0 [ 0.000000] PERCPU: Embedded 27 pages/cpu @ffff88001fe62000 s78656 r8192 d23744 u110592 [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 1036142 [ 0.076752] HugeTLB registered 2 MB page size, pre-allocated 0 pages full dmesg: [ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Linux version 2.6.37 (root@21-477.clodo.ru) (gcc version 4.4.4 (Gentoo 4.4.4-r2 p1.2, pie-0.4.5) ) #1 SMP Thu Jan 6 00:29:34 UTC 2011 [ 0.000000] Command line: root=/dev/xvda1 tmem=1 console=hvc0 xencons=hvc0 earlyprintk=xen debug loglevel=10 elevator=noop [ 0.000000] ACPI in unprivileged domain disabled [ 0.000000] released 0 pages of unused memory [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable) [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved) [ 0.000000] Xen: 0000000000100000 - 0000000100800000 (usable) [ 0.000000] bootconsole [xenboot0] enabled [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] DMI not present or invalid. [ 0.000000] e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved) [ 0.000000] e820 remove range: 00000000000a0000 - 0000000000100000 (usable) [ 0.000000] No AGP bridge found [ 0.000000] last_pfn = 0x100800 max_arch_pfn = 0x400000000 [ 0.000000] last_pfn = 0x100000 max_arch_pfn = 0x400000000 [ 0.000000] initial memory mapped : 0 - 027ff000 [ 0.000000] init_memory_mapping: 0000000000000000-0000000100000000 [ 0.000000] 0000000000 - 0100000000 page 4k [ 0.000000] kernel direct mapping tables up to 100000000 @ 1ffa000-27ff000 [ 0.000000] init_memory_mapping: 0000000100000000-0000000100800000 [ 0.000000] 0100000000 - 0100800000 page 4k [ 0.000000] kernel direct mapping tables up to 100800000 @ 1f7f6000-20000000 [ 0.000000] RAMDISK: 017fe000 - 01b38000 [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at 0000000000000000-0000000100800000 [ 0.000000] Initmem setup node 0 0000000000000000-0000000100800000 [ 0.000000] NODE_DATA [000000001fffb000 - 000000001fffffff] [ 0.000000] Zone PFN ranges: [ 0.000000] DMA 0x00000010 -> 0x00001000 [ 0.000000] DMA32 0x00001000 -> 0x00100000 [ 0.000000] Normal 0x00100000 -> 0x00100800 [ 0.000000] Movable zone start PFN for each node [ 0.000000] early_node_map[2] active PFN ranges [ 0.000000] 0: 0x00000010 -> 0x000000a0 [ 0.000000] 0: 0x00000100 -> 0x00100800 [ 0.000000] On node 0 totalpages: 1050512 [ 0.000000] DMA zone: 56 pages used for memmap [ 0.000000] DMA zone: 6 pages reserved [ 0.000000] DMA zone: 3922 pages, LIFO batch:0 [ 0.000000] DMA32 zone: 14280 pages used for memmap [ 0.000000] DMA32 zone: 1030200 pages, LIFO batch:31 [ 0.000000] Normal zone: 28 pages used for memmap [ 0.000000] Normal zone: 2020 pages, LIFO batch:0 [ 0.000000] SMP: Allowing 14 CPUs, 0 hotplug CPUs [ 0.000000] No local APIC present [ 0.000000] APIC: disable apic facility [ 0.000000] APIC: switched to apic NOOP [ 0.000000] nr_irqs_gsi: 16 [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000 [ 0.000000] PCI: Warning: Cannot find a gap in the 32bit address range [ 0.000000] PCI: Unassigned devices with 32bit resource registers may break! [ 0.000000] Allocating PCI resources starting at 100900000 (gap: 100900000:400000) [ 0.000000] Booting paravirtualized kernel on Xen [ 0.000000] Xen version: 4.0.0_21091_06-0.2 (preserve-AD) [ 0.000000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:14 nr_node_ids:1 [ 0.000000] PERCPU: Embedded 27 pages/cpu @ffff88001fe62000 s78656 r8192 d23744 u110592 [ 0.000000] pcpu-alloc: s78656 r8192 d23744 u110592 alloc=27*4096 [ 0.000000] pcpu-alloc: [0] 00 [0] 01 [0] 02 [0] 03 [0] 04 [0] 05 [0] 06 [0] 07 [ 0.000000] pcpu-alloc: [0] 08 [0] 09 [0] 10 [0] 11 [0] 12 [0] 13 [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 1036142 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: root=/dev/xvda1 tmem=1 console=hvc0 xencons=hvc0 earlyprintk=xen debug loglevel=10 elevator=noop [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] Checking aperture... [ 0.000000] No AGP bridge found [ 0.000000] Memory: 377324k/4202496k available (4125k kernel code, 448k absent, 3824724k reserved, 1636k data, 640k init) [ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=14, Nodes=1 [ 0.000000] Hierarchical RCU implementation. [ 0.000000] RCU-based detection of stalled CPUs is disabled. [ 0.000000] NR_IRQS:4352 nr_irqs:384 16 [ 0.000000] Console: colour dummy device 80x25 [ 0.000000] console [tty0] enabled [ 0.000000] console [hvc0] enabled, bootconsole disabled [ 0.000000] allocated 43253760 bytes of page_cgroup [ 0.000000] please try ''cgroup_disable=memory'' option if you don''t want memory cgroups [ 0.000000] Xen: using vcpuop timer interface [ 0.000000] installing Xen timer for CPU 0 [ 0.000000] Detected 2266.814 MHz processor. [ 0.004000] Calibrating delay loop (skipped), value calculated using timer frequency.. 4533.62 BogoMIPS (lpj=9067256) [ 0.004000] pid_max: default: 32768 minimum: 301 [ 0.004000] Security Framework initialized [ 0.004000] SELinux: Initializing. [ 0.004000] SELinux: Starting in permissive mode [ 0.004000] AppArmor: AppArmor disabled by boot time parameter [ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes) [ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes) [ 0.004000] Mount-cache hash table entries: 256 [ 0.004000] Initializing cgroup subsys ns [ 0.004000] ns_cgroup deprecated: consider using the ''clone_children'' flag without the ns_cgroup. [ 0.004000] Initializing cgroup subsys cpuacct [ 0.004000] Initializing cgroup subsys memory [ 0.004000] Initializing cgroup subsys devices [ 0.004000] Initializing cgroup subsys freezer [ 0.004000] Initializing cgroup subsys net_cls [ 0.004000] Initializing cgroup subsys blkio [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.004000] Performance Events: unsupported p6 CPU model 26 no PMU driver, software events only. [ 0.004000] SMP alternatives: switching to UP code [ 0.004057] cpu 0 spinlock event irq 382 [ 0.004321] installing Xen timer for CPU 1 [ 0.004337] cpu 1 spinlock event irq 376 [ 0.004393] SMP alternatives: switching to SMP code [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.008260] installing Xen timer for CPU 2 [ 0.008282] cpu 2 spinlock event irq 370 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.008552] installing Xen timer for CPU 3 [ 0.008569] cpu 3 spinlock event irq 364 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.008869] installing Xen timer for CPU 4 [ 0.008885] cpu 4 spinlock event irq 358 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.009256] installing Xen timer for CPU 5 [ 0.009274] cpu 5 spinlock event irq 352 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.012667] installing Xen timer for CPU 6 [ 0.012685] cpu 6 spinlock event irq 346 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.013028] installing Xen timer for CPU 7 [ 0.013046] cpu 7 spinlock event irq 340 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.013388] installing Xen timer for CPU 8 [ 0.013407] cpu 8 spinlock event irq 334 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.013783] installing Xen timer for CPU 9 [ 0.013801] cpu 9 spinlock event irq 328 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.017342] installing Xen timer for CPU 10 [ 0.017364] cpu 10 spinlock event irq 322 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.017714] installing Xen timer for CPU 11 [ 0.017733] cpu 11 spinlock event irq 316 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.017967] installing Xen timer for CPU 12 [ 0.017986] cpu 12 spinlock event irq 310 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.018208] installing Xen timer for CPU 13 [ 0.018228] cpu 13 spinlock event irq 304 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.018401] Brought up 14 CPUs [ 0.018474] devtmpfs: initialized [ 0.018474] Grant table initialized [ 0.018474] NET: Registered protocol family 16 [ 0.020513] PCI: setting up Xen PCI frontend stub [ 0.020521] PCI: pci_cache_line_size set to 64 bytes [ 0.024503] bio: create slab <bio-0> at 0 [ 0.024503] ACPI: Interpreter disabled. [ 0.024503] xen_balloon: Initialising balloon driver. [ 0.024503] last_pfn = 0x100800 max_arch_pfn = 0x400000000 [ 0.041822] vgaarb: loaded [ 0.041822] PCI: System does not support PCI [ 0.041822] PCI: System does not support PCI [ 0.044638] Switching to clocksource xen [ 0.045008] pnp: PnP ACPI: disabled [ 0.054879] NET: Registered protocol family 2 [ 0.055036] IP route cache hash table entries: 32768 (order: 6, 262144 bytes) [ 0.056093] TCP established hash table entries: 131072 (order: 9, 2097152 bytes) [ 0.056790] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) [ 0.057004] TCP: Hash tables configured (established 131072 bind 65536) [ 0.057014] TCP reno registered [ 0.057037] UDP hash table entries: 2048 (order: 4, 65536 bytes) [ 0.057079] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes) [ 0.057258] NET: Registered protocol family 1 [ 0.057274] PCI: CLS 0 bytes, default 64 [ 0.057339] Unpacking initramfs... [ 0.060911] Freeing initrd memory: 3304k freed [ 0.061734] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 0.061745] Placing 64MB software IO TLB between ffff880017000000 - ffff88001b000000 [ 0.061755] software IO TLB at phys 0x17000000 - 0x1b000000 [ 0.062964] platform rtc_cmos: registered platform RTC device (no PNP device found) [ 0.075440] Intel AES-NI instructions are not detected. [ 0.076049] audit: initializing netlink socket (disabled) [ 0.076074] type=2000 audit(1295626324.019:1): initialized [ 0.076752] HugeTLB registered 2 MB page size, pre-allocated 0 pages [ 0.085565] VFS: Disk quotas dquot_6.5.2 [ 0.085752] Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.087813] fuse init (API version 7.15) [ 0.088874] SGI XFS with ACLs, security attributes, realtime, large block/inode numbers, no debug enabled [ 0.091672] SGI XFS Quota Management subsystem [ 0.092568] Btrfs loaded [ 0.092584] msgmni has been set to 743 [ 0.092804] SELinux: Registering netfilter hooks [ 0.093918] io scheduler noop registered (default) [ 0.093929] io scheduler deadline registered [ 0.094150] io scheduler cfq registered [ 0.094374] pci-stub: invalid id string "" [ 0.101781] Console: switching to colour frame buffer device 100x37 [ 0.106605] Event-channel device installed. [ 0.109154] Non-volatile memory driver v1.3 [ 0.109396] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 0.388486] Initialising Xen virtual ethernet driver. [ 0.389595] Equalizer2002: Simon Janes (simon@ncm.com) and David S. Miller (davem@redhat.com) [ 0.390154] console [netcon0] enabled [ 0.390190] netconsole: network logging started [ 0.391719] PNP: No PS/2 controller found. Probing ports directly. [ 0.393746] i8042.c: No controller found. [ 0.397083] mice: PS/2 mouse device common for all mice [ 0.399430] input: Xen Virtual Keyboard as /devices/virtual/input/input0 [ 0.400862] input: Xen Virtual Pointer as /devices/virtual/input/input1 [ 0.405469] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0 [ 0.406619] rtc_cmos: probe of rtc_cmos failed with error -38 [ 0.408056] cpuidle: using governor ladder [ 0.409113] cpuidle: using governor menu [ 0.410201] Netfilter messages via NETLINK v0.30. [ 0.411491] IPv4 over IPv4 tunneling driver [ 0.414384] GRE over IPv4 demultiplexor driver [ 0.414462] Initializing XFRM netlink socket [ 0.416423] NET: Registered protocol family 17 [ 0.417398] Registering the dns_resolver key type [ 0.418806] registered taskstats version 1 [ 0.419736] IMA: No TPM chip found, activating TPM-bypass! [ 0.490508] blkfront: xvda: barriers enabled [ 0.508209] xvda: xvda1 [ 0.594112] blkfront: xvdb: barriers enabled [ 0.604629] xvdb: xvdb1 [ 0.620211] XENBUS: Device with no driver: device/console/0 [ 0.621407] Freeing unused kernel memory: 640k freed [ 0.628115] xenmgm: version magic ''2.6.32.26 SMP mod_unload '' should be ''2.6.37 SMP mod_unload '' [ 0.657069] EXT3-fs: barriers not enabled [ 0.676979] kjournald starting. Commit interval 5 seconds [ 0.677041] EXT3-fs (xvda1): mounted filesystem with writeback data mode [ 1.633598] NET: Registered protocol family 10 [ 1.634930] lo: Disabled Privacy Extensions [ 1.636080] tunl0: Disabled Privacy Extensions [ 2.861406] EXT3-fs (xvda1): using internal journal [ 3.752996] udevd[596]: specified group ''dialout'' unknown [ 3.753018] systemd-logger[599]: Got error on stream: No such process [ 4.282012] Adding 5116664k swap on /dev/xvdb1. Priority:0 extents:1 across:5116664k SS [ 6.740770] ntpd[843]: ntp engine ready [ 6.878340] sshd (842): /proc/842/oom_adj is deprecated, please use /proc/842/oom_score_adj instead. [ 8.241998] systemd[1]: nginx@multi-user.service: main process exited, code=exited, status=1 [ 8.496974] systemd[1]: pdns.service: control process exited, code=exited status=1 [ 8.560268] systemd[1]: Unit pdns.service entered failed state. [ 8.560576] systemd[1]: nginx@multi-user.service: control process exited, code=exited status=1 [ 8.624358] systemd[1]: Unit nginx@multi-user.service entered failed state. [ 9.671924] systemd[1]: php.service: control process exited, code=exited status=1 [ 9.728545] systemd[1]: Unit php.service entered failed state. [ 16.868127] eth0: no IPv6 routers present [ 62.070790] systemd[1]: Job dev-hvc0.device/start timed out. [ 62.132469] systemd[1]: Startup finished in 1s 649ms 262us (kernel) + 1min 483ms 16us (userspace) = 1min 2s 132ms 278us. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, 2011-01-21 at 19:16 +0300, Vasiliy G Tolstov wrote:> On Fri, 2011-01-21 at 10:30 -0500, Konrad Rzeszutek Wilk wrote: > > > Do you see any messages during bootup about memory being returned? Look for > > xen_release_chunk .. and pages freed ?[snip] Hello. Konrad, do You have any progress about this issue? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, Jan 24, 2011 at 11:19:02PM +0300, Vasiliy G Tolstov wrote:> On Fri, 2011-01-21 at 19:16 +0300, Vasiliy G Tolstov wrote: > > On Fri, 2011-01-21 at 10:30 -0500, Konrad Rzeszutek Wilk wrote: > > > > > Do you see any messages during bootup about memory being returned? Look for > > > xen_release_chunk .. and pages freed ? > [snip] > > Hello. Konrad, do You have any progress about this issue?Try booting your kernel with ''debug loglevel=10 memblock=debug''. I know that at least 64MB is used for SWIOTLB (which you don''t need - you can disable that since you are not doing any PCI passthrough), 6MB for kernel + kernel data. The "memblock=debug" will give you the details to figure out who else is eating the big swaths of memory. Is this also a problem if you boot the guest as HVM? What happens if you boot your PV guest without ''maxmem=4096'' and .. oh, I think I know why this is happening = the kernel allocates pagetables as if you have 4GB of memory and that eats much of low memory. If you decrease the ''maxmem'' you should see a smaller amount being used. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, 2011-01-25 at 09:45 -0500, Konrad Rzeszutek Wilk wrote:> On Mon, Jan 24, 2011 at 11:19:02PM +0300, Vasiliy G Tolstov wrote: > > On Fri, 2011-01-21 at 19:16 +0300, Vasiliy G Tolstov wrote: > > > On Fri, 2011-01-21 at 10:30 -0500, Konrad Rzeszutek Wilk wrote: > > > > > > > Do you see any messages during bootup about memory being returned? Look for > > > > xen_release_chunk .. and pages freed ? > > [snip] > > > > Hello. Konrad, do You have any progress about this issue? > > Try booting your kernel with ''debug loglevel=10 memblock=debug''. I know that at least > 64MB is used for SWIOTLB (which you don''t need - you can disable that since you are > not doing any PCI passthrough), 6MB for kernel + kernel data. The "memblock=debug" will > give you the details to figure out who else is eating the big swaths of memory. >Nice suggestion about swiotlb. Can You check my debug output and helps me to find what else eats memory? [ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Linux version 2.6.37 (root@21-477.clodo.ru) (gcc version 4.4.4 (Gentoo 4.4.4-r2 p1.2, pie-0.4.5) ) #1 SMP Thu Jan 6 00:29:34 UTC 2011 [ 0.000000] Command line: root=/dev/xvda1 tmem=1 console=hvc0 xencons=hvc0 earlyprintk=xen debug loglevel=10 elevator=noop memblock=debug iommu=off [ 0.000000] ACPI in unprivileged domain disabled [ 0.000000] released 0 pages of unused memory [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable) [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved) [ 0.000000] Xen: 0000000000100000 - 0000000100800000 (usable) [ 0.000000] bootconsole [xenboot0] enabled [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] DMI not present or invalid. [ 0.000000] e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved) [ 0.000000] e820 remove range: 00000000000a0000 - 0000000000100000 (usable) [ 0.000000] No AGP bridge found [ 0.000000] last_pfn = 0x100800 max_arch_pfn = 0x400000000 [ 0.000000] last_pfn = 0x100000 max_arch_pfn = 0x400000000 [ 0.000000] memblock_x86_reserve_range: [0x016d3000-0x016defff] BRK [ 0.000000] MEMBLOCK configuration: [ 0.000000] memory size = 0x100790000 [ 0.000000] memory.cnt = 0x2 [ 0.000000] memory[0x0] [0x00000000010000-0x0000000009ffff], 0x90000 bytes [ 0.000000] memory[0x1] [0x00000000100000-0x000001007fffff], 0x100700000 bytes [ 0.000000] reserved.cnt = 0x4 [ 0.000000] reserved[0x0] [0x00000001000000-0x000000016d207f], 0x6d2080 bytes [ 0.000000] reserved[0x1] [0x000000016d3000-0x000000016defff], 0xc000 bytes [ 0.000000] reserved[0x2] [0x000000017fe000-0x00000001c4dfff], 0x450000 bytes [ 0.000000] reserved[0x3] [0x00000020000000-0x000001007fffff], 0xe0800000 bytes [ 0.000000] initial memory mapped : 0 - 027ff000 [ 0.000000] memblock_x86_reserve_range: [0x0009e000-0x0009ffff] TRAMPOLINE [ 0.000000] memblock_x86_reserve_range: [0x0009a000-0x0009dfff] ACPI WAKEUP [ 0.000000] init_memory_mapping: 0000000000000000-0000000100000000 [ 0.000000] 0000000000 - 0100000000 page 4k [ 0.000000] kernel direct mapping tables up to 100000000 @ 1ffa000-27ff000 [ 0.000000] memblock_x86_reserve_range: [0x01ffa000-0x027e8fff] PGTABLE [ 0.000000] init_memory_mapping: 0000000100000000-0000000100800000 [ 0.000000] 0100000000 - 0100800000 page 4k [ 0.000000] kernel direct mapping tables up to 100800000 @ 1f7f6000-20000000 [ 0.000000] memblock_x86_reserve_range: [0x1f7f6000-0x1f7fafff] PGTABLE [ 0.000000] RAMDISK: 017fe000 - 01b38000 [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at 0000000000000000-0000000100800000 [ 0.000000] Initmem setup node 0 0000000000000000-0000000100800000 [ 0.000000] memblock_x86_reserve_range: [0x1fffb000-0x1fffffff] NODE_DATA [ 0.000000] NODE_DATA [000000001fffb000 - 000000001fffffff] [ 0.000000] memblock_x86_reserve_range: [0x1fffa000-0x1fffafff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbfa000-0x1fff9fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbf9cc0-0x1fbf9fd7] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1f3f6000-0x1f7f5fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1b000000-0x1f1fffff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbf8000-0x1fbf8fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbf7000-0x1fbf7fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbf6000-0x1fbf6fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbf5000-0x1fbf5fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbf4000-0x1fbf4fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbf3000-0x1fbf3fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbf2000-0x1fbf2fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbf1000-0x1fbf1fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbf0000-0x1fbf0fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbef000-0x1fbeffff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbee000-0x1fbeefff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbed000-0x1fbedfff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbec000-0x1fbecfff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbeb000-0x1fbebfff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbea000-0x1fbeafff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbe9000-0x1fbe9fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbe8000-0x1fbe8fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbe7000-0x1fbe7fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbe6000-0x1fbe6fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbe5000-0x1fbe5fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbe4000-0x1fbe4fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbe3000-0x1fbe3fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbe2000-0x1fbe2fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbe1000-0x1fbe1fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbe0000-0x1fbe0fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbdf000-0x1fbdffff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbde000-0x1fbdefff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbdd000-0x1fbddfff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbdc000-0x1fbdcfff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbdb000-0x1fbdbfff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fbda000-0x1fbdafff] BOOTMEM [ 0.000000] memblock_x86_free_range: [0x1e9c0000-0x1f1fffff] [ 0.000000] memblock_x86_free_range: [0x1f3f6000-0x1f7f5fff] [ 0.000000] memblock_x86_free_range: [0x1fbfa000-0x1fff9fff] [ 0.000000] Zone PFN ranges: [ 0.000000] DMA 0x00000010 -> 0x00001000 [ 0.000000] DMA32 0x00001000 -> 0x00100000 [ 0.000000] Normal 0x00100000 -> 0x00100800 [ 0.000000] Movable zone start PFN for each node [ 0.000000] early_node_map[2] active PFN ranges [ 0.000000] 0: 0x00000010 -> 0x000000a0 [ 0.000000] 0: 0x00000100 -> 0x00100800 [ 0.000000] On node 0 totalpages: 1050512 [ 0.000000] DMA zone: 56 pages used for memmap [ 0.000000] DMA zone: 6 pages reserved [ 0.000000] DMA zone: 3922 pages, LIFO batch:0 [ 0.000000] memblock_x86_reserve_range: [0x1fff9e80-0x1fff9fff] BOOTMEM [ 0.000000] DMA32 zone: 14280 pages used for memmap [ 0.000000] DMA32 zone: 1030200 pages, LIFO batch:31 [ 0.000000] memblock_x86_reserve_range: [0x1ffe1e80-0x1fff9e7f] BOOTMEM [ 0.000000] Normal zone: 28 pages used for memmap [ 0.000000] Normal zone: 2020 pages, LIFO batch:0 [ 0.000000] memblock_x86_reserve_range: [0x1ffe1dc0-0x1ffe1e7f] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1ffe0000-0x1ffe0fff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1ffdf000-0x1ffdffff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1ffde000-0x1ffdefff] BOOTMEM [ 0.000000] SMP: Allowing 14 CPUs, 0 hotplug CPUs [ 0.000000] No local APIC present [ 0.000000] APIC: disable apic facility [ 0.000000] APIC: switched to apic NOOP [ 0.000000] nr_irqs_gsi: 16 [ 0.000000] memblock_x86_reserve_range: [0x1ffe1cc0-0x1ffe1d9f] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1ffe1c40-0x1ffe1ca7] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1ffe1bc0-0x1ffe1c27] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1ffe1b40-0x1ffe1ba7] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1ffe1b00-0x1ffe1b1f] BOOTMEM [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000 [ 0.000000] PCI: Warning: Cannot find a gap in the 32bit address range [ 0.000000] PCI: Unassigned devices with 32bit resource registers may break! [ 0.000000] Allocating PCI resources starting at 100900000 (gap: 100900000:400000) [ 0.000000] Booting paravirtualized kernel on Xen [ 0.000000] Xen version: 4.0.0_21091_06-0.2 (preserve-AD) [ 0.000000] memblock_x86_reserve_range: [0x1ffe1a80-0x1ffe1afb] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1ffe1a00-0x1ffe1a7b] BOOTMEM [ 0.000000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:14 nr_node_ids:1 [ 0.000000] memblock_x86_reserve_range: [0x1ffdd000-0x1ffddfff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1ffdc000-0x1ffdcfff] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1fe62000-0x1ffdbfff] BOOTMEM [ 0.000000] PERCPU: Embedded 27 pages/cpu @ffff88001fe62000 s78656 r8192 d23744 u110592 [ 0.000000] memblock_x86_reserve_range: [0x1ffe19c0-0x1ffe19c7] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1ffe1980-0x1ffe1987] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1ffe1940-0x1ffe1977] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1ffe18c0-0x1ffe192f] BOOTMEM [ 0.000000] pcpu-alloc: s78656 r8192 d23744 u110592 alloc=27*4096 [ 0.000000] pcpu-alloc: [0] 00 [0] 01 [0] 02 [0] 03 [0] 04 [0] 05 [0] 06 [0] 07 [ 0.000000] pcpu-alloc: [0] 08 [0] 09 [0] 10 [0] 11 [0] 12 [0] 13 [ 0.000000] memblock_x86_reserve_range: [0x1ffe17c0-0x1ffe18bf] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1ffe1740-0x1ffe1787] BOOTMEM [ 0.000000] memblock_x86_reserve_range: [0x1ffe16c0-0x1ffe1707] BOOTMEM [ 0.000000] memblock_x86_free_range: [0x1ffdd000-0x1ffddfff] [ 0.000000] memblock_x86_free_range: [0x1ffdc000-0x1ffdcfff] [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 1036142 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: root=/dev/xvda1 tmem=1 console=hvc0 xencons=hvc0 earlyprintk=xen debug loglevel=10 elevator=noop memblock=debug iommu=off [ 0.000000] memblock_x86_reserve_range: [0x1fe5a000-0x1fe61fff] BOOTMEM [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] Checking aperture... [ 0.000000] No AGP bridge found [ 0.000000] Subtract (25 early reservations) [ 0.000000] [000009a000-000009ffff] [ 0.000000] [0001000000-00016d207f] [ 0.000000] [00016d3000-00016defff] [ 0.000000] [00017fe000-0001c4dfff] [ 0.000000] [0001ffa000-00027e8fff] [ 0.000000] [001b000000-001e9bffff] [ 0.000000] [001f7f6000-001f7fafff] [ 0.000000] [001fbda000-001fbf8fff] [ 0.000000] [001fbf9cc0-001fbf9fd7] [ 0.000000] [001fe5a000-001ffdbfff] [ 0.000000] [001ffde000-001ffe0fff] [ 0.000000] [001ffe16c0-001ffe1707] [ 0.000000] [001ffe1740-001ffe1787] [ 0.000000] [001ffe17c0-001ffe192f] [ 0.000000] [001ffe1940-001ffe1977] [ 0.000000] [001ffe1980-001ffe1987] [ 0.000000] [001ffe19c0-001ffe19c7] [ 0.000000] [001ffe1a00-001ffe1a7b] [ 0.000000] [001ffe1a80-001ffe1afb] [ 0.000000] [001ffe1b00-001ffe1b1f] [ 0.000000] [001ffe1b40-001ffe1ba7] [ 0.000000] [001ffe1bc0-001ffe1c27] [ 0.000000] [001ffe1c40-001ffe1ca7] [ 0.000000] [001ffe1cc0-001ffe1d9f] [ 0.000000] [001ffe1dc0-01007fffff] [ 0.000000] Memory: 443276k/4202496k available (4125k kernel code, 448k absent, 3758772k reserved, 1636k data, 640k init) [ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=14, Nodes=1 [ 0.000000] Hierarchical RCU implementation. [ 0.000000] RCU-based detection of stalled CPUs is disabled. [ 0.000000] NR_IRQS:4352 nr_irqs:384 16 [ 0.000000] Console: colour dummy device 80x25 [ 0.000000] console [tty0] enabled [ 0.000000] console [hvc0] enabled, bootconsole disabled [ 0.000000] allocated 43253760 bytes of page_cgroup [ 0.000000] please try ''cgroup_disable=memory'' option if you don''t want memory cgroups [ 0.000000] Xen: using vcpuop timer interface [ 0.000000] installing Xen timer for CPU 0 [ 0.000000] Detected 2266.836 MHz processor. [ 0.004000] Calibrating delay loop (skipped), value calculated using timer frequency.. 4533.67 BogoMIPS (lpj=9067344) [ 0.004000] pid_max: default: 32768 minimum: 301 [ 0.004000] Security Framework initialized [ 0.004000] SELinux: Initializing. [ 0.004000] SELinux: Starting in permissive mode [ 0.004000] AppArmor: AppArmor disabled by boot time parameter [ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes) [ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes) [ 0.004000] Mount-cache hash table entries: 256 [ 0.004000] Initializing cgroup subsys ns [ 0.004000] ns_cgroup deprecated: consider using the ''clone_children'' flag without the ns_cgroup. [ 0.004000] Initializing cgroup subsys cpuacct [ 0.004000] Initializing cgroup subsys memory [ 0.004000] Initializing cgroup subsys devices [ 0.004000] Initializing cgroup subsys freezer [ 0.004000] Initializing cgroup subsys net_cls [ 0.004000] Initializing cgroup subsys blkio [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.004000] Performance Events: unsupported p6 CPU model 26 no PMU driver, software events only. [ 0.004000] SMP alternatives: switching to UP code [ 0.008069] cpu 0 spinlock event irq 382 [ 0.008382] installing Xen timer for CPU 1 [ 0.008398] cpu 1 spinlock event irq 376 [ 0.008428] SMP alternatives: switching to SMP code [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.012269] installing Xen timer for CPU 2 [ 0.012293] cpu 2 spinlock event irq 370 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.012656] installing Xen timer for CPU 3 [ 0.012674] cpu 3 spinlock event irq 364 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.013017] installing Xen timer for CPU 4 [ 0.013034] cpu 4 spinlock event irq 358 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.015738] installing Xen timer for CPU 5 [ 0.015768] cpu 5 spinlock event irq 352 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.016261] installing Xen timer for CPU 6 [ 0.016288] cpu 6 spinlock event irq 346 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.017027] installing Xen timer for CPU 7 [ 0.017052] cpu 7 spinlock event irq 340 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.017350] installing Xen timer for CPU 8 [ 0.017377] cpu 8 spinlock event irq 334 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.017714] installing Xen timer for CPU 9 [ 0.017740] cpu 9 spinlock event irq 328 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.018029] installing Xen timer for CPU 10 [ 0.018056] cpu 10 spinlock event irq 322 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.018286] installing Xen timer for CPU 11 [ 0.018313] cpu 11 spinlock event irq 316 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.018632] installing Xen timer for CPU 12 [ 0.018659] cpu 12 spinlock event irq 310 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.019058] installing Xen timer for CPU 13 [ 0.019084] cpu 13 spinlock event irq 304 [ 0.004000] CPU: Unsupported number of siblings 16 [ 0.019316] Brought up 14 CPUs [ 0.019398] devtmpfs: initialized [ 0.020299] Grant table initialized [ 0.020299] NET: Registered protocol family 16 [ 0.020513] PCI: setting up Xen PCI frontend stub [ 0.020521] PCI: pci_cache_line_size set to 64 bytes [ 0.028030] bio: create slab <bio-0> at 0 [ 0.028093] ACPI: Interpreter disabled. [ 0.028093] xen_balloon: Initialising balloon driver. [ 0.028367] last_pfn = 0x100800 max_arch_pfn = 0x400000000 [ 0.044669] vgaarb: loaded [ 0.044669] PCI: System does not support PCI [ 0.044669] PCI: System does not support PCI [ 0.044669] Switching to clocksource xen [ 0.044669] pnp: PnP ACPI: disabled [ 0.059138] NET: Registered protocol family 2 [ 0.059305] IP route cache hash table entries: 32768 (order: 6, 262144 bytes) [ 0.060283] TCP established hash table entries: 131072 (order: 9, 2097152 bytes) [ 0.061111] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) [ 0.061422] TCP: Hash tables configured (established 131072 bind 65536) [ 0.061433] TCP reno registered [ 0.061457] UDP hash table entries: 2048 (order: 4, 65536 bytes) [ 0.061503] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes) [ 0.061703] NET: Registered protocol family 1 [ 0.061720] PCI: CLS 0 bytes, default 64 [ 0.061787] Unpacking initramfs... [ 0.065754] Freeing initrd memory: 3304k freed [ 0.067573] platform rtc_cmos: registered platform RTC device (no PNP device found) [ 0.079443] Intel AES-NI instructions are not detected. [ 0.079988] audit: initializing netlink socket (disabled) [ 0.080082] type=2000 audit(1295968813.070:1): initialized [ 0.080611] HugeTLB registered 2 MB page size, pre-allocated 0 pages [ 0.092066] VFS: Disk quotas dquot_6.5.2 [ 0.092278] Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.094579] fuse init (API version 7.15) [ 0.095184] SGI XFS with ACLs, security attributes, realtime, large block/inode numbers, no debug enabled [ 0.097683] SGI XFS Quota Management subsystem [ 0.098553] Btrfs loaded [ 0.098568] msgmni has been set to 872 [ 0.098796] SELinux: Registering netfilter hooks [ 0.099798] io scheduler noop registered (default) [ 0.099808] io scheduler deadline registered [ 0.099964] io scheduler cfq registered [ 0.100226] pci-stub: invalid id string "" [ 0.109773] Console: switching to colour frame buffer device 100x37 [ 0.113403] Event-channel device installed. [ 0.115096] Non-volatile memory driver v1.3 [ 0.115342] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 0.376418] Initialising Xen virtual ethernet driver. [ 0.378610] Equalizer2002: Simon Janes (simon@ncm.com) and David S. Miller (davem@redhat.com) [ 0.379119] console [netcon0] enabled [ 0.379155] netconsole: network logging started [ 0.380636] PNP: No PS/2 controller found. Probing ports directly. [ 0.382550] i8042.c: No controller found. [ 0.384154] mice: PS/2 mouse device common for all mice [ 0.387897] input: Xen Virtual Keyboard as /devices/virtual/input/input0 [ 0.389874] input: Xen Virtual Pointer as /devices/virtual/input/input1 [ 0.394445] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0 [ 0.395599] rtc_cmos: probe of rtc_cmos failed with error -38 [ 0.397054] cpuidle: using governor ladder [ 0.398111] cpuidle: using governor menu [ 0.399227] Netfilter messages via NETLINK v0.30. [ 0.400562] IPv4 over IPv4 tunneling driver [ 0.402211] GRE over IPv4 demultiplexor driver [ 0.402289] Initializing XFRM netlink socket [ 0.404255] NET: Registered protocol family 17 [ 0.405227] Registering the dns_resolver key type [ 0.406866] registered taskstats version 1 [ 0.407802] IMA: No TPM chip found, activating TPM-bypass! [ 0.481165] blkfront: xvda: barriers enabled [ 0.499229] xvda: xvda1 [ 0.588349] blkfront: xvdb: barriers enabled [ 0.617933] xvdb: xvdb1 [ 0.708239] XENBUS: Device with no driver: device/console/0 [ 0.709469] Freeing unused kernel memory: 640k freed [ 0.758412] EXT3-fs: barriers not enabled [ 0.773576] kjournald starting. Commit interval 5 seconds [ 0.773618] EXT3-fs (xvda1): mounted filesystem with writeback data mode [ 2.098341] NET: Registered protocol family 10 [ 2.099747] lo: Disabled Privacy Extensions [ 2.100923] tunl0: Disabled Privacy Extensions [ 3.398405] EXT3-fs (xvda1): using internal journal> Is this also a problem if you boot the guest as HVM?Not. Now i''m use only PVM mode.> What happens if you boot your > PV guest without ''maxmem=4096'' and .. oh, I think I know why this is happening = the > kernel allocates pagetables as if you have 4GB of memory and that eats much of low memory. > If you decrease the ''maxmem'' you should see a smaller amount being used.I understand this. Does xen PVM 2.6.37 have any features that can be disabled to free memory? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, 2011-01-25 at 09:45 -0500, Konrad Rzeszutek Wilk wrote:> I think I know why this is happening = the > kernel allocates pagetables as if you have 4GB of memory and that eats much of low memory. > If you decrease the ''maxmem'' you should see a smaller amount being used. >And why xenlinux kernel from centos (2.6.18-194.26.1.el5xen) displays all memory as available: cat /proc/meminfo under 2.6.37 (iommu=off acpi=off): MemTotal: 447220 kB cat /proc/meminfo under 2.6.18-194.26.1.el5xen: MemTotal: 524288 kB ? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, Jan 25, 2011 at 07:33:14PM +0300, Vasiliy G Tolstov wrote:> On Tue, 2011-01-25 at 09:45 -0500, Konrad Rzeszutek Wilk wrote: > > On Mon, Jan 24, 2011 at 11:19:02PM +0300, Vasiliy G Tolstov wrote: > > > On Fri, 2011-01-21 at 19:16 +0300, Vasiliy G Tolstov wrote: > > > > On Fri, 2011-01-21 at 10:30 -0500, Konrad Rzeszutek Wilk wrote: > > > > > > > > > Do you see any messages during bootup about memory being returned? Look for > > > > > xen_release_chunk .. and pages freed ? > > > [snip] > > > > > > Hello. Konrad, do You have any progress about this issue? > > > > Try booting your kernel with ''debug loglevel=10 memblock=debug''. I know that at least > > 64MB is used for SWIOTLB (which you don''t need - you can disable that since you are > > not doing any PCI passthrough), 6MB for kernel + kernel data. The "memblock=debug" will > > give you the details to figure out who else is eating the big swaths of memory. > > > > Nice suggestion about swiotlb. Can You check my debug output and helps > me to find what else eats memory?Why don''t you run this through | grep memblock, then awk - extract those regions .. something like this: grep memblock | serve_range:// | sed s/\\[// | sed s/\\]// | sed s/-/\ / | awk --non-decimal-data '' { a=$2;b=$1;printf "%s %s\n", a-b, $3 } that should give you an idea what is eating what memory. For example NODE_DATA looks to be eating a bunch of 20 pages (20*4096)= 81920 bytes. thought later on it free''s some of them.> > [ 0.000000] Initializing cgroup subsys cpuset > [ 0.000000] Initializing cgroup subsys cpu > [ 0.000000] Linux version 2.6.37 (root@21-477.clodo.ru) (gcc version > 4.4.4 (Gentoo 4.4.4-r2 p1.2, pie-0.4.5) ) #1 SMP Thu Jan 6 00:29:34 UTC > 2011 > [ 0.000000] Command line: root=/dev/xvda1 tmem=1 console=hvc0 > xencons=hvc0 earlyprintk=xen debug loglevel=10 elevator=noop > memblock=debug iommu=off > [ 0.000000] ACPI in unprivileged domain disabled > [ 0.000000] released 0 pages of unused memory > [ 0.000000] BIOS-provided physical RAM map: > [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable) > [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved) > [ 0.000000] Xen: 0000000000100000 - 0000000100800000 (usable) > [ 0.000000] bootconsole [xenboot0] enabled > [ 0.000000] NX (Execute Disable) protection: active > [ 0.000000] DMI not present or invalid. > [ 0.000000] e820 update range: 0000000000000000 - 0000000000010000 > (usable) ==> (reserved) > [ 0.000000] e820 remove range: 00000000000a0000 - 0000000000100000 > (usable) > [ 0.000000] No AGP bridge found > [ 0.000000] last_pfn = 0x100800 max_arch_pfn = 0x400000000 > [ 0.000000] last_pfn = 0x100000 max_arch_pfn = 0x400000000 > [ 0.000000] memblock_x86_reserve_range: [0x016d3000-0x016defff] > BRK > [ 0.000000] MEMBLOCK configuration: > [ 0.000000] memory size = 0x100790000 > [ 0.000000] memory.cnt = 0x2 > [ 0.000000] memory[0x0] [0x00000000010000-0x0000000009ffff], 0x90000 > bytes > [ 0.000000] memory[0x1] [0x00000000100000-0x000001007fffff], > 0x100700000 bytes > [ 0.000000] reserved.cnt = 0x4 > [ 0.000000] reserved[0x0] [0x00000001000000-0x000000016d207f], > 0x6d2080 bytes > [ 0.000000] reserved[0x1] [0x000000016d3000-0x000000016defff], > 0xc000 bytes > [ 0.000000] reserved[0x2] [0x000000017fe000-0x00000001c4dfff], > 0x450000 bytes > [ 0.000000] reserved[0x3] [0x00000020000000-0x000001007fffff], > 0xe0800000 bytes > [ 0.000000] initial memory mapped : 0 - 027ff000 > [ 0.000000] memblock_x86_reserve_range: [0x0009e000-0x0009ffff] > TRAMPOLINE > [ 0.000000] memblock_x86_reserve_range: [0x0009a000-0x0009dfff] > ACPI WAKEUP > [ 0.000000] init_memory_mapping: 0000000000000000-0000000100000000 > [ 0.000000] 0000000000 - 0100000000 page 4k > [ 0.000000] kernel direct mapping tables up to 100000000 @ > 1ffa000-27ff000 > [ 0.000000] memblock_x86_reserve_range: [0x01ffa000-0x027e8fff] > PGTABLE > [ 0.000000] init_memory_mapping: 0000000100000000-0000000100800000 > [ 0.000000] 0100000000 - 0100800000 page 4k > [ 0.000000] kernel direct mapping tables up to 100800000 @ > 1f7f6000-20000000 > [ 0.000000] memblock_x86_reserve_range: [0x1f7f6000-0x1f7fafff] > PGTABLE > [ 0.000000] RAMDISK: 017fe000 - 01b38000 > [ 0.000000] No NUMA configuration found > [ 0.000000] Faking a node at 0000000000000000-0000000100800000 > [ 0.000000] Initmem setup node 0 0000000000000000-0000000100800000 > [ 0.000000] memblock_x86_reserve_range: [0x1fffb000-0x1fffffff] > NODE_DATA > [ 0.000000] NODE_DATA [000000001fffb000 - 000000001fffffff] > [ 0.000000] memblock_x86_reserve_range: [0x1fffa000-0x1fffafff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbfa000-0x1fff9fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbf9cc0-0x1fbf9fd7] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1f3f6000-0x1f7f5fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1b000000-0x1f1fffff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbf8000-0x1fbf8fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbf7000-0x1fbf7fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbf6000-0x1fbf6fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbf5000-0x1fbf5fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbf4000-0x1fbf4fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbf3000-0x1fbf3fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbf2000-0x1fbf2fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbf1000-0x1fbf1fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbf0000-0x1fbf0fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbef000-0x1fbeffff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbee000-0x1fbeefff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbed000-0x1fbedfff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbec000-0x1fbecfff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbeb000-0x1fbebfff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbea000-0x1fbeafff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbe9000-0x1fbe9fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbe8000-0x1fbe8fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbe7000-0x1fbe7fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbe6000-0x1fbe6fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbe5000-0x1fbe5fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbe4000-0x1fbe4fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbe3000-0x1fbe3fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbe2000-0x1fbe2fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbe1000-0x1fbe1fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbe0000-0x1fbe0fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbdf000-0x1fbdffff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbde000-0x1fbdefff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbdd000-0x1fbddfff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbdc000-0x1fbdcfff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbdb000-0x1fbdbfff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fbda000-0x1fbdafff] > BOOTMEM > [ 0.000000] memblock_x86_free_range: [0x1e9c0000-0x1f1fffff] > [ 0.000000] memblock_x86_free_range: [0x1f3f6000-0x1f7f5fff] > [ 0.000000] memblock_x86_free_range: [0x1fbfa000-0x1fff9fff] > [ 0.000000] Zone PFN ranges: > [ 0.000000] DMA 0x00000010 -> 0x00001000 > [ 0.000000] DMA32 0x00001000 -> 0x00100000 > [ 0.000000] Normal 0x00100000 -> 0x00100800 > [ 0.000000] Movable zone start PFN for each node > [ 0.000000] early_node_map[2] active PFN ranges > [ 0.000000] 0: 0x00000010 -> 0x000000a0 > [ 0.000000] 0: 0x00000100 -> 0x00100800 > [ 0.000000] On node 0 totalpages: 1050512 > [ 0.000000] DMA zone: 56 pages used for memmap > [ 0.000000] DMA zone: 6 pages reserved > [ 0.000000] DMA zone: 3922 pages, LIFO batch:0 > [ 0.000000] memblock_x86_reserve_range: [0x1fff9e80-0x1fff9fff] > BOOTMEM > [ 0.000000] DMA32 zone: 14280 pages used for memmap > [ 0.000000] DMA32 zone: 1030200 pages, LIFO batch:31 > [ 0.000000] memblock_x86_reserve_range: [0x1ffe1e80-0x1fff9e7f] > BOOTMEM > [ 0.000000] Normal zone: 28 pages used for memmap > [ 0.000000] Normal zone: 2020 pages, LIFO batch:0 > [ 0.000000] memblock_x86_reserve_range: [0x1ffe1dc0-0x1ffe1e7f] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1ffe0000-0x1ffe0fff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1ffdf000-0x1ffdffff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1ffde000-0x1ffdefff] > BOOTMEM > [ 0.000000] SMP: Allowing 14 CPUs, 0 hotplug CPUs > [ 0.000000] No local APIC present > [ 0.000000] APIC: disable apic facility > [ 0.000000] APIC: switched to apic NOOP > [ 0.000000] nr_irqs_gsi: 16 > [ 0.000000] memblock_x86_reserve_range: [0x1ffe1cc0-0x1ffe1d9f] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1ffe1c40-0x1ffe1ca7] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1ffe1bc0-0x1ffe1c27] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1ffe1b40-0x1ffe1ba7] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1ffe1b00-0x1ffe1b1f] > BOOTMEM > [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - > 0000000000100000 > [ 0.000000] PCI: Warning: Cannot find a gap in the 32bit address > range > [ 0.000000] PCI: Unassigned devices with 32bit resource registers may > break! > [ 0.000000] Allocating PCI resources starting at 100900000 (gap: > 100900000:400000) > [ 0.000000] Booting paravirtualized kernel on Xen > [ 0.000000] Xen version: 4.0.0_21091_06-0.2 (preserve-AD) > [ 0.000000] memblock_x86_reserve_range: [0x1ffe1a80-0x1ffe1afb] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1ffe1a00-0x1ffe1a7b] > BOOTMEM > [ 0.000000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:14 > nr_node_ids:1 > [ 0.000000] memblock_x86_reserve_range: [0x1ffdd000-0x1ffddfff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1ffdc000-0x1ffdcfff] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1fe62000-0x1ffdbfff] > BOOTMEM > [ 0.000000] PERCPU: Embedded 27 pages/cpu @ffff88001fe62000 s78656 > r8192 d23744 u110592 > [ 0.000000] memblock_x86_reserve_range: [0x1ffe19c0-0x1ffe19c7] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1ffe1980-0x1ffe1987] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1ffe1940-0x1ffe1977] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1ffe18c0-0x1ffe192f] > BOOTMEM > [ 0.000000] pcpu-alloc: s78656 r8192 d23744 u110592 alloc=27*4096 > [ 0.000000] pcpu-alloc: [0] 00 [0] 01 [0] 02 [0] 03 [0] 04 [0] 05 [0] > 06 [0] 07 > [ 0.000000] pcpu-alloc: [0] 08 [0] 09 [0] 10 [0] 11 [0] 12 [0] 13 > [ 0.000000] memblock_x86_reserve_range: [0x1ffe17c0-0x1ffe18bf] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1ffe1740-0x1ffe1787] > BOOTMEM > [ 0.000000] memblock_x86_reserve_range: [0x1ffe16c0-0x1ffe1707] > BOOTMEM > [ 0.000000] memblock_x86_free_range: [0x1ffdd000-0x1ffddfff] > [ 0.000000] memblock_x86_free_range: [0x1ffdc000-0x1ffdcfff] > [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. > Total pages: 1036142 > [ 0.000000] Policy zone: Normal > [ 0.000000] Kernel command line: root=/dev/xvda1 tmem=1 console=hvc0 > xencons=hvc0 earlyprintk=xen debug loglevel=10 elevator=noop > memblock=debug iommu=off > [ 0.000000] memblock_x86_reserve_range: [0x1fe5a000-0x1fe61fff] > BOOTMEM > [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) > [ 0.000000] Checking aperture... > [ 0.000000] No AGP bridge found > [ 0.000000] Subtract (25 early reservations) > [ 0.000000] [000009a000-000009ffff] > [ 0.000000] [0001000000-00016d207f] > [ 0.000000] [00016d3000-00016defff] > [ 0.000000] [00017fe000-0001c4dfff] > [ 0.000000] [0001ffa000-00027e8fff] > [ 0.000000] [001b000000-001e9bffff] > [ 0.000000] [001f7f6000-001f7fafff] > [ 0.000000] [001fbda000-001fbf8fff] > [ 0.000000] [001fbf9cc0-001fbf9fd7] > [ 0.000000] [001fe5a000-001ffdbfff] > [ 0.000000] [001ffde000-001ffe0fff] > [ 0.000000] [001ffe16c0-001ffe1707] > [ 0.000000] [001ffe1740-001ffe1787] > [ 0.000000] [001ffe17c0-001ffe192f] > [ 0.000000] [001ffe1940-001ffe1977] > [ 0.000000] [001ffe1980-001ffe1987] > [ 0.000000] [001ffe19c0-001ffe19c7] > [ 0.000000] [001ffe1a00-001ffe1a7b] > [ 0.000000] [001ffe1a80-001ffe1afb] > [ 0.000000] [001ffe1b00-001ffe1b1f] > [ 0.000000] [001ffe1b40-001ffe1ba7] > [ 0.000000] [001ffe1bc0-001ffe1c27] > [ 0.000000] [001ffe1c40-001ffe1ca7] > [ 0.000000] [001ffe1cc0-001ffe1d9f] > [ 0.000000] [001ffe1dc0-01007fffff] > [ 0.000000] Memory: 443276k/4202496k available (4125k kernel code, > 448k absent, 3758772k reserved, 1636k data, 640k init) > [ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, > CPUs=14, Nodes=1 > [ 0.000000] Hierarchical RCU implementation. > [ 0.000000] RCU-based detection of stalled CPUs is disabled. > [ 0.000000] NR_IRQS:4352 nr_irqs:384 16 > [ 0.000000] Console: colour dummy device 80x25 > [ 0.000000] console [tty0] enabled > [ 0.000000] console [hvc0] enabled, bootconsole disabled > [ 0.000000] allocated 43253760 bytes of page_cgroup > [ 0.000000] please try ''cgroup_disable=memory'' option if you don''t > want memory cgroups > [ 0.000000] Xen: using vcpuop timer interface > [ 0.000000] installing Xen timer for CPU 0 > [ 0.000000] Detected 2266.836 MHz processor. > [ 0.004000] Calibrating delay loop (skipped), value calculated using > timer frequency.. 4533.67 BogoMIPS (lpj=9067344) > [ 0.004000] pid_max: default: 32768 minimum: 301 > [ 0.004000] Security Framework initialized > [ 0.004000] SELinux: Initializing. > [ 0.004000] SELinux: Starting in permissive mode > [ 0.004000] AppArmor: AppArmor disabled by boot time parameter > [ 0.004000] Dentry cache hash table entries: 524288 (order: 10, > 4194304 bytes) > [ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 > bytes) > [ 0.004000] Mount-cache hash table entries: 256 > [ 0.004000] Initializing cgroup subsys ns > [ 0.004000] ns_cgroup deprecated: consider using the ''clone_children'' > flag without the ns_cgroup. > [ 0.004000] Initializing cgroup subsys cpuacct > [ 0.004000] Initializing cgroup subsys memory > [ 0.004000] Initializing cgroup subsys devices > [ 0.004000] Initializing cgroup subsys freezer > [ 0.004000] Initializing cgroup subsys net_cls > [ 0.004000] Initializing cgroup subsys blkio > [ 0.004000] CPU: Unsupported number of siblings 16 > [ 0.004000] Performance Events: unsupported p6 CPU model 26 no PMU > driver, software events only. > [ 0.004000] SMP alternatives: switching to UP code > [ 0.008069] cpu 0 spinlock event irq 382 > [ 0.008382] installing Xen timer for CPU 1 > [ 0.008398] cpu 1 spinlock event irq 376 > [ 0.008428] SMP alternatives: switching to SMP code > [ 0.004000] CPU: Unsupported number of siblings 16 > [ 0.012269] installing Xen timer for CPU 2 > [ 0.012293] cpu 2 spinlock event irq 370 > [ 0.004000] CPU: Unsupported number of siblings 16 > [ 0.012656] installing Xen timer for CPU 3 > [ 0.012674] cpu 3 spinlock event irq 364 > [ 0.004000] CPU: Unsupported number of siblings 16 > [ 0.013017] installing Xen timer for CPU 4 > [ 0.013034] cpu 4 spinlock event irq 358 > [ 0.004000] CPU: Unsupported number of siblings 16 > [ 0.015738] installing Xen timer for CPU 5 > [ 0.015768] cpu 5 spinlock event irq 352 > [ 0.004000] CPU: Unsupported number of siblings 16 > [ 0.016261] installing Xen timer for CPU 6 > [ 0.016288] cpu 6 spinlock event irq 346 > [ 0.004000] CPU: Unsupported number of siblings 16 > [ 0.017027] installing Xen timer for CPU 7 > [ 0.017052] cpu 7 spinlock event irq 340 > [ 0.004000] CPU: Unsupported number of siblings 16 > [ 0.017350] installing Xen timer for CPU 8 > [ 0.017377] cpu 8 spinlock event irq 334 > [ 0.004000] CPU: Unsupported number of siblings 16 > [ 0.017714] installing Xen timer for CPU 9 > [ 0.017740] cpu 9 spinlock event irq 328 > [ 0.004000] CPU: Unsupported number of siblings 16 > [ 0.018029] installing Xen timer for CPU 10 > [ 0.018056] cpu 10 spinlock event irq 322 > [ 0.004000] CPU: Unsupported number of siblings 16 > [ 0.018286] installing Xen timer for CPU 11 > [ 0.018313] cpu 11 spinlock event irq 316 > [ 0.004000] CPU: Unsupported number of siblings 16 > [ 0.018632] installing Xen timer for CPU 12 > [ 0.018659] cpu 12 spinlock event irq 310 > [ 0.004000] CPU: Unsupported number of siblings 16 > [ 0.019058] installing Xen timer for CPU 13 > [ 0.019084] cpu 13 spinlock event irq 304 > [ 0.004000] CPU: Unsupported number of siblings 16 > [ 0.019316] Brought up 14 CPUs > [ 0.019398] devtmpfs: initialized > [ 0.020299] Grant table initialized > [ 0.020299] NET: Registered protocol family 16 > [ 0.020513] PCI: setting up Xen PCI frontend stub > [ 0.020521] PCI: pci_cache_line_size set to 64 bytes > [ 0.028030] bio: create slab <bio-0> at 0 > [ 0.028093] ACPI: Interpreter disabled. > [ 0.028093] xen_balloon: Initialising balloon driver. > [ 0.028367] last_pfn = 0x100800 max_arch_pfn = 0x400000000 > [ 0.044669] vgaarb: loaded > [ 0.044669] PCI: System does not support PCI > [ 0.044669] PCI: System does not support PCI > [ 0.044669] Switching to clocksource xen > [ 0.044669] pnp: PnP ACPI: disabled > [ 0.059138] NET: Registered protocol family 2 > [ 0.059305] IP route cache hash table entries: 32768 (order: 6, > 262144 bytes) > [ 0.060283] TCP established hash table entries: 131072 (order: 9, > 2097152 bytes) > [ 0.061111] TCP bind hash table entries: 65536 (order: 8, 1048576 > bytes) > [ 0.061422] TCP: Hash tables configured (established 131072 bind > 65536) > [ 0.061433] TCP reno registered > [ 0.061457] UDP hash table entries: 2048 (order: 4, 65536 bytes) > [ 0.061503] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes) > [ 0.061703] NET: Registered protocol family 1 > [ 0.061720] PCI: CLS 0 bytes, default 64 > [ 0.061787] Unpacking initramfs... > [ 0.065754] Freeing initrd memory: 3304k freed > [ 0.067573] platform rtc_cmos: registered platform RTC device (no PNP > device found) > [ 0.079443] Intel AES-NI instructions are not detected. > [ 0.079988] audit: initializing netlink socket (disabled) > [ 0.080082] type=2000 audit(1295968813.070:1): initialized > [ 0.080611] HugeTLB registered 2 MB page size, pre-allocated 0 pages > [ 0.092066] VFS: Disk quotas dquot_6.5.2 > [ 0.092278] Dquot-cache hash table entries: 512 (order 0, 4096 bytes) > [ 0.094579] fuse init (API version 7.15) > [ 0.095184] SGI XFS with ACLs, security attributes, realtime, large > block/inode numbers, no debug enabled > [ 0.097683] SGI XFS Quota Management subsystem > [ 0.098553] Btrfs loaded > [ 0.098568] msgmni has been set to 872 > [ 0.098796] SELinux: Registering netfilter hooks > [ 0.099798] io scheduler noop registered (default) > [ 0.099808] io scheduler deadline registered > [ 0.099964] io scheduler cfq registered > [ 0.100226] pci-stub: invalid id string "" > [ 0.109773] Console: switching to colour frame buffer device 100x37 > [ 0.113403] Event-channel device installed. > [ 0.115096] Non-volatile memory driver v1.3 > [ 0.115342] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled > [ 0.376418] Initialising Xen virtual ethernet driver. > [ 0.378610] Equalizer2002: Simon Janes (simon@ncm.com) and David S. > Miller (davem@redhat.com) > [ 0.379119] console [netcon0] enabled > [ 0.379155] netconsole: network logging started > [ 0.380636] PNP: No PS/2 controller found. Probing ports directly. > [ 0.382550] i8042.c: No controller found. > [ 0.384154] mice: PS/2 mouse device common for all mice > [ 0.387897] input: Xen Virtual Keyboard > as /devices/virtual/input/input0 > [ 0.389874] input: Xen Virtual Pointer > as /devices/virtual/input/input1 > [ 0.394445] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0 > [ 0.395599] rtc_cmos: probe of rtc_cmos failed with error -38 > [ 0.397054] cpuidle: using governor ladder > [ 0.398111] cpuidle: using governor menu > [ 0.399227] Netfilter messages via NETLINK v0.30. > [ 0.400562] IPv4 over IPv4 tunneling driver > [ 0.402211] GRE over IPv4 demultiplexor driver > [ 0.402289] Initializing XFRM netlink socket > [ 0.404255] NET: Registered protocol family 17 > [ 0.405227] Registering the dns_resolver key type > [ 0.406866] registered taskstats version 1 > [ 0.407802] IMA: No TPM chip found, activating TPM-bypass! > [ 0.481165] blkfront: xvda: barriers enabled > [ 0.499229] xvda: xvda1 > [ 0.588349] blkfront: xvdb: barriers enabled > [ 0.617933] xvdb: xvdb1 > [ 0.708239] XENBUS: Device with no driver: device/console/0 > [ 0.709469] Freeing unused kernel memory: 640k freed > [ 0.758412] EXT3-fs: barriers not enabled > [ 0.773576] kjournald starting. Commit interval 5 seconds > [ 0.773618] EXT3-fs (xvda1): mounted filesystem with writeback data > mode > [ 2.098341] NET: Registered protocol family 10 > [ 2.099747] lo: Disabled Privacy Extensions > [ 2.100923] tunl0: Disabled Privacy Extensions > [ 3.398405] EXT3-fs (xvda1): using internal journal > > > > Is this also a problem if you boot the guest as HVM? > > Not. Now i''m use only PVM mode. > > > What happens if you boot your > > PV guest without ''maxmem=4096'' and .. oh, I think I know why this is happening = the > > kernel allocates pagetables as if you have 4GB of memory and that eats much of low memory. > > If you decrease the ''maxmem'' you should see a smaller amount being used. > > I understand this. Does xen PVM 2.6.37 have any features that can be > disabled to free memory? > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> From: Vasiliy G Tolstov [mailto:v.tolstov@selfip.ru] > Sent: Tuesday, January 25, 2011 10:35 AM > To: Konrad Rzeszutek Wilk > Cc: Don Brearley; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] linux kernel 2.6.37 > > On Tue, 2011-01-25 at 09:45 -0500, Konrad Rzeszutek Wilk wrote: > > I think I know why this is happening = the > > kernel allocates pagetables as if you have 4GB of memory and that > eats much of low memory. > > If you decrease the ''maxmem'' you should see a smaller amount being > used. > > > > And why xenlinux kernel from centos (2.6.18-194.26.1.el5xen) displays > all memory as available: > > cat /proc/meminfo under 2.6.37 (iommu=off acpi=off): > MemTotal: 447220 kB > > cat /proc/meminfo under 2.6.18-194.26.1.el5xen: > MemTotal: 524288 kB > > ?FYI, I''ve seen a similar problem with 2.6.37 but noticed it when manually ballooning a PV guest. For example: # echo $((256*1024)) > /sys/devices/system/xen_memory/xen_memory0/target_kb should reduce memory to 256M (or at least somewhere close) but instead /proc/meminfo shows 217812 kB, or about 15% lower than expected. This looks like approximately the same percentage of memory loss that Vasiliy is reporting on boot. /me wonders if it might be related to this thread: http://lists.xensource.com/archives/html/xen-devel/2010-09/msg00987.html assuming the related patch got upstream. Jeremy cc''ed just in case. I do NOT see this issue with 2.6.35.4. In fact, I now see a MemTotal boot-time difference when I boot both with mem=512 (in vm.cfg): 2.6.35.4 kernel shows 506864, 2.6.37 shows 495456. Not as much as Vasiliy reports, but still a big mystery. If this problem is real and widespread, I hope nobody is comparing KVM vs Xen (or Xen vs native) on 2.6.37 as that much memory loss could result in a significant performance loss for a Xen guest. Dan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, 2011-01-25 at 13:03 -0800, Dan Magenheimer wrote:> /me wonders if it might be related to this thread: > > http://lists.xensource.com/archives/html/xen-devel/2010-09/msg00987.html > > assuming the related patch got upstream. Jeremy cc''ed just in case. >I''m read mail thread and nothing found about patch. Does is not written yet?> I do NOT see this issue with 2.6.35.4. In fact, I now see > a MemTotal boot-time difference when I boot both with mem=512 > (in vm.cfg): 2.6.35.4 kernel shows 506864, 2.6.37 shows 495456. > Not as much as Vasiliy reports, but still a big mystery.2.6.32.26 already have this issue. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, Jan 25, 2011 at 01:43:42PM -0500, Konrad Rzeszutek Wilk wrote:> On Tue, Jan 25, 2011 at 07:33:14PM +0300, Vasiliy G Tolstov wrote: > > On Tue, 2011-01-25 at 09:45 -0500, Konrad Rzeszutek Wilk wrote: > > > On Mon, Jan 24, 2011 at 11:19:02PM +0300, Vasiliy G Tolstov wrote: > > > > On Fri, 2011-01-21 at 19:16 +0300, Vasiliy G Tolstov wrote: > > > > > On Fri, 2011-01-21 at 10:30 -0500, Konrad Rzeszutek Wilk wrote: > > > > > > > > > > > Do you see any messages during bootup about memory being returned? Look for > > > > > > xen_release_chunk .. and pages freed ? > > > > [snip] > > > > > > > > Hello. Konrad, do You have any progress about this issue? > > > > > > Try booting your kernel with ''debug loglevel=10 memblock=debug''. I know that at least > > > 64MB is used for SWIOTLB (which you don''t need - you can disable that since you are > > > not doing any PCI passthrough), 6MB for kernel + kernel data. The "memblock=debug" will > > > give you the details to figure out who else is eating the big swaths of memory. > > > > > > > Nice suggestion about swiotlb. Can You check my debug output and helps > > me to find what else eats memory? > > Why don''t you run this through | grep memblock, > then awk - extract those regions .. something like this: > > grep memblock | serve_range:// | sed s/\\[// | sed s/\\]// | sed s/-/\ / | awk --non-decimal-data '' { a=$2;b=$1;printf "%s %s\n", a-b, $3 } > > that should give you an idea what is eating what memory. For example > NODE_DATA looks to be eating a bunch of 20 pages (20*4096)= 81920 bytes. > > thought later on it free''s some of them.Did you do what I asked you try? Also what is the memory consumption when you boot Linux as HVM guest? You MUST have the same amount of memory in both guests, so try the test with mem=512 maxmem=512 for both HVM and PV. What is the difference in the log console when you use memblock=debug? Do you see any big differences? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel