Hi,
Me and another sysadmin has independently been researching a problem where DomU
randomly locks (Can’t reach it via xl console, no ping / SSH connection, shown
as stuck in running-state in xentop) on two of our separate machines (installed
completely independently):
Dom0:
Debian 7.0 with Xen version: 4.1.4 and xen-utils 4.1.4-3+deb7u1
Debian 7.1 with Xen version: 4.1
DomU:
7.0
7.1(.3)
Common denominator appears to be qemu-dm consuming (leaking?) memory until the
Dom0 swaps. When the Dom0 swap is full, the domU appears to be locked (see
above) Dom0, at which time a hard reboot a.ka. xl destroy + xl create is the
only way to get it back.
This *could* be related to "[Xen-devel] qemu-system-i386: memory
leak?" http://xen.markmail.org/message/chqpifrj46lxdxx2
DomU by themselves doesn’t use any abnormal memory or swap.
To give an overview, currently Dom0 uses 26GB of swap with 8 active domU. Swap
per process:
Pid Swap Process Uptime
3766 98452 kB qemu-dm -d 29 -domain-name [hostname] -nographic -M xenpv 160
days
6100 276988 kB qemu-dm -d 42 -domain-name [hostname] -nographic -M xenpv 108
days
6790 121620 kB qemu-dm -d 46 -domain-name [hostname] -nographic -M xenpv 95
days
10616 791616 kB qemu-dm -d 51 -domain-name [hostname] -nographic -M xenpv 32
days
11588 3514436 kB qemu-dm -d 49 -domain-name [hostname] -nographic -M xenpv 73
days
16290 170436 kB qemu-dm -d 43 -domain-name [hostname] -nographic -M xenpv 107
days
26974 1647248 kB qemu-dm -d 48 -domain-name [hostname] -nographic -M xenpv 92
days
32403 21147060 kB qemu-dm -d 52 -domain-name [hostname] -nographic -M xenpv
29 days
Generally, the higher usage the higher swap. Possibly, the higher IO the higher
swap.
Any ideas on how to solve the issue?
Regards,
Niklas
