hallo,
I have updated xen server to
xen-4.2.1-r2
xen-tools-4.2.1-r1
kernel gentoo-sources-3.2.39
I have found 2 problems
1) Can not exactly set dom0, domU memory
I dedicate 4G of memory to dom0 via kernel parameter:
dom0_mem=4G,max:4G dom0_max_vcpus=4 dom0_vcpus_pin
Bud the real total memory is set to 2618M after reboot. I have found
article describing this behavior
(http://wiki.xen.org/wiki/Do%EF%BB%BFm0_Memory_%E2%80%94_Where_It_Has_Not_Gone),
so I have run
echo $((4*1024*1024)) > /sys/devices/system/xen_memory/xen_memory0/target_kb
The total memory was se to 3687M. This is better, but not enough.
I have tested kernel 3.7, there is not such problem. I would like to use
longterm kernel 3.2 so I would like to stay with 3.2.
Similar problem exists with domU, config looks like
kernel = ''/etc/xen/kernels/linux-3.2.39-gentoo''
name = ''test1''
memory = ''4000''
maxmem = ''12000''
vcpus = ''8''
cpus = ''^0,^1,^2,^3''
extra = ''elevator=noop gentoo=nodevfs''
root = ''/dev/xvda1 ro''
disk = [
''phy:/dev/vg1/test1_root,xvda1,w'',
''phy:/dev/vg1/test1_var,xvda2,w'',
]
vif = [
''mac=00:16:3e:24:11:38,bridge=br0'',
]
After boot, test1 domain has 3704M total memory (free -m), xm list test1
shows 4000M.
When I increase memory amount via xm mem-set test1 8000, then xm list
test1 really shows 8000M, but domU has 7704M total memory (free -m).
from xend.log
[2013-03-04 18:10:18 4627] DEBUG (XendDomainInfo:1467) Setting memory
target of domain test1 (1) to 8000 MiB.
2) I am not able to increase vcpu count to domU.
After boot domU test1, it has really 8 vcpus, I am able to decrease
vcpus by xm vcpu-set test1 4. Cpus count inside test1 really decreases
to 4. Now I would like to increase vcpu count via xm vcpu-list test1 6,
but unsuccessfully.
rajvir3 configs # xm vcpu-list test1
Name ID VCPU CPU State Time(s) CPU
Affinity
test1 1 0 15 -b- 3.7 12-23
test1 1 1 23 -b- 2.8 12-23
test1 1 2 18 -b- 1.2 12-23
test1 1 3 20 -b- 1.2 12-23
test1 1 4 16 -b- 1.0 12-23
test1 1 5 12 -b- 3.5 12-23
test1 1 6 19 -b- 0.9 12-23
test1 1 7 14 -b- 0.9 12-23
rajvir3 configs # xm vcpu-set test1 4
rajvir3 configs # xm vcpu-list test1
Name ID VCPU CPU State Time(s) CPU
Affinity
test1 1 0 15 -b- 3.7 12-23
test1 1 1 23 -b- 2.9 12-23
test1 1 2 18 -b- 1.3 12-23
test1 1 3 20 -b- 1.2 12-23
test1 1 4 - --p 1.0 12-23
test1 1 5 - --p 3.5 12-23
test1 1 6 - --p 1.0 12-23
test1 1 7 - --p 0.9 12-23
rajvir3 configs # xm vcpu-set test1 6
rajvir3 configs # xm vcpu-list test1
Name ID VCPU CPU State Time(s) CPU
Affinity
test1 1 0 15 -b- 3.8 12-23
test1 1 1 23 -b- 2.9 12-23
test1 1 2 18 -b- 1.3 12-23
test1 1 3 20 -b- 1.3 12-23
test1 1 4 - --p 1.0 12-23
test1 1 5 - --p 3.5 12-23
test1 1 6 - --p 1.0 12-23
test1 1 7 - --p 0.9 12-23
from xend.log
[2013-03-04 18:12:58 4627] INFO (XendDomainInfo:1978) Set VCPU count on
domain test1 to 4
[2013-03-04 18:14:51 4627] INFO (XendDomainInfo:1978) Set VCPU count on
domain test1 to 6
I was not able to increase vcpu count also in kernel 3.7.
Where could be the problem?
Thanks
Michal
Hi Michal, 1) Could you please post your dom0 kernel config options related to XEN? Simply with "zgrep XEN /proc/config.gz" or "grep XEN /usr/src/linux/.config" ? 2) Is CONFIG_HOTPLUG_CPU enabled in your kernel image? Thanks Jan On 03/04/2013 06:23 PM, Michal Fiala wrote:> hallo, > > I have updated xen server to > > xen-4.2.1-r2 > xen-tools-4.2.1-r1 > kernel gentoo-sources-3.2.39 > > I have found 2 problems > > 1) Can not exactly set dom0, domU memory > > I dedicate 4G of memory to dom0 via kernel parameter: > > dom0_mem=4G,max:4G dom0_max_vcpus=4 dom0_vcpus_pin > > Bud the real total memory is set to 2618M after reboot. I have found > article describing this behavior > (http://wiki.xen.org/wiki/Do%EF%BB%BFm0_Memory_%E2%80%94_Where_It_Has_Not_Gone), > so I have run > > echo $((4*1024*1024)) > /sys/devices/system/xen_memory/xen_memory0/target_kb > > The total memory was se to 3687M. This is better, but not enough. > > I have tested kernel 3.7, there is not such problem. I would like to use > longterm kernel 3.2 so I would like to stay with 3.2. > > Similar problem exists with domU, config looks like > > kernel = ''/etc/xen/kernels/linux-3.2.39-gentoo'' > name = ''test1'' > memory = ''4000'' > maxmem = ''12000'' > vcpus = ''8'' > cpus = ''^0,^1,^2,^3'' > extra = ''elevator=noop gentoo=nodevfs'' > root = ''/dev/xvda1 ro'' > disk = [ > ''phy:/dev/vg1/test1_root,xvda1,w'', > ''phy:/dev/vg1/test1_var,xvda2,w'', > ] > vif = [ > ''mac=00:16:3e:24:11:38,bridge=br0'', > ] > > After boot, test1 domain has 3704M total memory (free -m), xm list test1 > shows 4000M. > > When I increase memory amount via xm mem-set test1 8000, then xm list > test1 really shows 8000M, but domU has 7704M total memory (free -m). > > from xend.log > [2013-03-04 18:10:18 4627] DEBUG (XendDomainInfo:1467) Setting memory > target of domain test1 (1) to 8000 MiB. > > 2) I am not able to increase vcpu count to domU. > After boot domU test1, it has really 8 vcpus, I am able to decrease > vcpus by xm vcpu-set test1 4. Cpus count inside test1 really decreases > to 4. Now I would like to increase vcpu count via xm vcpu-list test1 6, > but unsuccessfully. > > rajvir3 configs # xm vcpu-list test1 > Name ID VCPU CPU State Time(s) CPU > Affinity > test1 1 0 15 -b- 3.7 12-23 > test1 1 1 23 -b- 2.8 12-23 > test1 1 2 18 -b- 1.2 12-23 > test1 1 3 20 -b- 1.2 12-23 > test1 1 4 16 -b- 1.0 12-23 > test1 1 5 12 -b- 3.5 12-23 > test1 1 6 19 -b- 0.9 12-23 > test1 1 7 14 -b- 0.9 12-23 > > rajvir3 configs # xm vcpu-set test1 4 > > rajvir3 configs # xm vcpu-list test1 > Name ID VCPU CPU State Time(s) CPU > Affinity > test1 1 0 15 -b- 3.7 12-23 > test1 1 1 23 -b- 2.9 12-23 > test1 1 2 18 -b- 1.3 12-23 > test1 1 3 20 -b- 1.2 12-23 > test1 1 4 - --p 1.0 12-23 > test1 1 5 - --p 3.5 12-23 > test1 1 6 - --p 1.0 12-23 > test1 1 7 - --p 0.9 12-23 > > rajvir3 configs # xm vcpu-set test1 6 > > rajvir3 configs # xm vcpu-list test1 > Name ID VCPU CPU State Time(s) CPU > Affinity > test1 1 0 15 -b- 3.8 12-23 > test1 1 1 23 -b- 2.9 12-23 > test1 1 2 18 -b- 1.3 12-23 > test1 1 3 20 -b- 1.3 12-23 > test1 1 4 - --p 1.0 12-23 > test1 1 5 - --p 3.5 12-23 > test1 1 6 - --p 1.0 12-23 > test1 1 7 - --p 0.9 12-23 > > from xend.log > > [2013-03-04 18:12:58 4627] INFO (XendDomainInfo:1978) Set VCPU count on > domain test1 to 4 > [2013-03-04 18:14:51 4627] INFO (XendDomainInfo:1978) Set VCPU count on > domain test1 to 6 > > I was not able to increase vcpu count also in kernel 3.7. > > Where could be the problem? > > Thanks > > Michal > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
On 03/04/2013 07:08 PM, Jan Hejl wrote:> Hi Michal,Hi> > 1) Could you please post your dom0 kernel config options related to XEN? > Simply with "zgrep XEN /proc/config.gz" or "grep XEN > /usr/src/linux/.config" ?rajvir3 dom0 # grep XEN .config CONFIG_XEN=y CONFIG_XEN_DOM0=y CONFIG_XEN_PRIVILEGED_GUEST=y CONFIG_XEN_PVHVM=y CONFIG_XEN_MAX_DOMAIN_MEMORY=128 CONFIG_XEN_SAVE_RESTORE=y # CONFIG_XEN_DEBUG_FS is not set CONFIG_PCI_XEN=y CONFIG_XEN_PCIDEV_FRONTEND=y CONFIG_XEN_BLKDEV_FRONTEND=y CONFIG_XEN_BLKDEV_BACKEND=y CONFIG_XEN_NETDEV_FRONTEND=y CONFIG_XEN_NETDEV_BACKEND=y CONFIG_HVC_XEN=y # CONFIG_XEN_WDT is not set CONFIG_XEN_BALLOON=y CONFIG_XEN_SCRUB_PAGES=y CONFIG_XEN_DEV_EVTCHN=y CONFIG_XEN_BACKEND=y CONFIG_XENFS=y CONFIG_XEN_COMPAT_XENFS=y CONFIG_XEN_SYS_HYPERVISOR=y CONFIG_XEN_XENBUS_FRONTEND=y CONFIG_XEN_GNTDEV=m CONFIG_XEN_GRANT_DEV_ALLOC=m CONFIG_SWIOTLB_XEN=y CONFIG_XEN_PCIDEV_BACKEND=m> > 2) Is CONFIG_HOTPLUG_CPU enabled in your kernel image?Yes rajvir3 dom0 # grep CONFIG_HOTPLUG_CPU .config CONFIG_HOTPLUG_CPU=y Michal> > Thanks > Jan > > On 03/04/2013 06:23 PM, Michal Fiala wrote: >> hallo, >> >> I have updated xen server to >> >> xen-4.2.1-r2 >> xen-tools-4.2.1-r1 >> kernel gentoo-sources-3.2.39 >> >> I have found 2 problems >> >> 1) Can not exactly set dom0, domU memory >> >> I dedicate 4G of memory to dom0 via kernel parameter: >> >> dom0_mem=4G,max:4G dom0_max_vcpus=4 dom0_vcpus_pin >> >> Bud the real total memory is set to 2618M after reboot. I have found >> article describing this behavior >> (http://wiki.xen.org/wiki/Do%EF%BB%BFm0_Memory_%E2%80%94_Where_It_Has_Not_Gone), >> >> so I have run >> >> echo $((4*1024*1024)) > >> /sys/devices/system/xen_memory/xen_memory0/target_kb >> >> The total memory was se to 3687M. This is better, but not enough. >> >> I have tested kernel 3.7, there is not such problem. I would like to use >> longterm kernel 3.2 so I would like to stay with 3.2. >> >> Similar problem exists with domU, config looks like >> >> kernel = ''/etc/xen/kernels/linux-3.2.39-gentoo'' >> name = ''test1'' >> memory = ''4000'' >> maxmem = ''12000'' >> vcpus = ''8'' >> cpus = ''^0,^1,^2,^3'' >> extra = ''elevator=noop gentoo=nodevfs'' >> root = ''/dev/xvda1 ro'' >> disk = [ >> ''phy:/dev/vg1/test1_root,xvda1,w'', >> ''phy:/dev/vg1/test1_var,xvda2,w'', >> ] >> vif = [ >> ''mac=00:16:3e:24:11:38,bridge=br0'', >> ] >> >> After boot, test1 domain has 3704M total memory (free -m), xm list test1 >> shows 4000M. >> >> When I increase memory amount via xm mem-set test1 8000, then xm list >> test1 really shows 8000M, but domU has 7704M total memory (free -m). >> >> from xend.log >> [2013-03-04 18:10:18 4627] DEBUG (XendDomainInfo:1467) Setting memory >> target of domain test1 (1) to 8000 MiB. >> >> 2) I am not able to increase vcpu count to domU. >> After boot domU test1, it has really 8 vcpus, I am able to decrease >> vcpus by xm vcpu-set test1 4. Cpus count inside test1 really decreases >> to 4. Now I would like to increase vcpu count via xm vcpu-list test1 6, >> but unsuccessfully. >> >> rajvir3 configs # xm vcpu-list test1 >> Name ID VCPU CPU State Time(s) CPU >> Affinity >> test1 1 0 15 -b- 3.7 12-23 >> test1 1 1 23 -b- 2.8 12-23 >> test1 1 2 18 -b- 1.2 12-23 >> test1 1 3 20 -b- 1.2 12-23 >> test1 1 4 16 -b- 1.0 12-23 >> test1 1 5 12 -b- 3.5 12-23 >> test1 1 6 19 -b- 0.9 12-23 >> test1 1 7 14 -b- 0.9 12-23 >> >> rajvir3 configs # xm vcpu-set test1 4 >> >> rajvir3 configs # xm vcpu-list test1 >> Name ID VCPU CPU State Time(s) CPU >> Affinity >> test1 1 0 15 -b- 3.7 12-23 >> test1 1 1 23 -b- 2.9 12-23 >> test1 1 2 18 -b- 1.3 12-23 >> test1 1 3 20 -b- 1.2 12-23 >> test1 1 4 - --p 1.0 12-23 >> test1 1 5 - --p 3.5 12-23 >> test1 1 6 - --p 1.0 12-23 >> test1 1 7 - --p 0.9 12-23 >> >> rajvir3 configs # xm vcpu-set test1 6 >> >> rajvir3 configs # xm vcpu-list test1 >> Name ID VCPU CPU State Time(s) CPU >> Affinity >> test1 1 0 15 -b- 3.8 12-23 >> test1 1 1 23 -b- 2.9 12-23 >> test1 1 2 18 -b- 1.3 12-23 >> test1 1 3 20 -b- 1.3 12-23 >> test1 1 4 - --p 1.0 12-23 >> test1 1 5 - --p 3.5 12-23 >> test1 1 6 - --p 1.0 12-23 >> test1 1 7 - --p 0.9 12-23 >> >> from xend.log >> >> [2013-03-04 18:12:58 4627] INFO (XendDomainInfo:1978) Set VCPU count on >> domain test1 to 4 >> [2013-03-04 18:14:51 4627] INFO (XendDomainInfo:1978) Set VCPU count on >> domain test1 to 6 >> >> I was not able to increase vcpu count also in kernel 3.7. >> >> Where could be the problem? >> >> Thanks >> >> Michal >> >> _______________________________________________ >> Xen-users mailing list >> Xen-users@lists.xen.org >> http://lists.xen.org/xen-users > > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users >
solved, see bellow On 03/04/2013 07:26 PM, Michal Fiala wrote:> On 03/04/2013 07:08 PM, Jan Hejl wrote: >> Hi Michal, > > Hi > >> >> 1) Could you please post your dom0 kernel config options related to XEN? >> Simply with "zgrep XEN /proc/config.gz" or "grep XEN >> /usr/src/linux/.config" ? > > rajvir3 dom0 # grep XEN .config > CONFIG_XEN=y > CONFIG_XEN_DOM0=y > CONFIG_XEN_PRIVILEGED_GUEST=y > CONFIG_XEN_PVHVM=y > CONFIG_XEN_MAX_DOMAIN_MEMORY=128 > CONFIG_XEN_SAVE_RESTORE=y > # CONFIG_XEN_DEBUG_FS is not set > CONFIG_PCI_XEN=y > CONFIG_XEN_PCIDEV_FRONTEND=y > CONFIG_XEN_BLKDEV_FRONTEND=y > CONFIG_XEN_BLKDEV_BACKEND=y > CONFIG_XEN_NETDEV_FRONTEND=y > CONFIG_XEN_NETDEV_BACKEND=y > CONFIG_HVC_XEN=y > # CONFIG_XEN_WDT is not set > CONFIG_XEN_BALLOON=y > CONFIG_XEN_SCRUB_PAGES=y > CONFIG_XEN_DEV_EVTCHN=y > CONFIG_XEN_BACKEND=y > CONFIG_XENFS=y > CONFIG_XEN_COMPAT_XENFS=y > CONFIG_XEN_SYS_HYPERVISOR=y > CONFIG_XEN_XENBUS_FRONTEND=y > CONFIG_XEN_GNTDEV=m > CONFIG_XEN_GRANT_DEV_ALLOC=m > CONFIG_SWIOTLB_XEN=y > CONFIG_XEN_PCIDEV_BACKEND=m > >> >> 2) Is CONFIG_HOTPLUG_CPU enabled in your kernel image? > > Yes > > rajvir3 dom0 # grep CONFIG_HOTPLUG_CPU .config > CONFIG_HOTPLUG_CPU=y > > Michal > >> >> Thanks >> Jan >> >> On 03/04/2013 06:23 PM, Michal Fiala wrote: >>> hallo, >>> >>> I have updated xen server to >>> >>> xen-4.2.1-r2 >>> xen-tools-4.2.1-r1 >>> kernel gentoo-sources-3.2.39 >>> >>> I have found 2 problems >>> >>> 1) Can not exactly set dom0, domU memory >>> >>> I dedicate 4G of memory to dom0 via kernel parameter: >>> >>> dom0_mem=4G,max:4G dom0_max_vcpus=4 dom0_vcpus_pin >>> >>> Bud the real total memory is set to 2618M after reboot. I have found >>> article describing this behavior >>> (http://wiki.xen.org/wiki/Do%EF%BB%BFm0_Memory_%E2%80%94_Where_It_Has_Not_Gone), >>> >>> so I have run >>> >>> echo $((4*1024*1024)) > >>> /sys/devices/system/xen_memory/xen_memory0/target_kb >>> >>> The total memory was se to 3687M. This is better, but not enough. >>> >>> I have tested kernel 3.7, there is not such problem. I would like to use >>> longterm kernel 3.2 so I would like to stay with 3.2. >>> >>> Similar problem exists with domU, config looks like >>> >>> kernel = ''/etc/xen/kernels/linux-3.2.39-gentoo'' >>> name = ''test1'' >>> memory = ''4000'' >>> maxmem = ''12000'' >>> vcpus = ''8'' >>> cpus = ''^0,^1,^2,^3'' >>> extra = ''elevator=noop gentoo=nodevfs'' >>> root = ''/dev/xvda1 ro'' >>> disk = [ >>> ''phy:/dev/vg1/test1_root,xvda1,w'', >>> ''phy:/dev/vg1/test1_var,xvda2,w'', >>> ] >>> vif = [ >>> ''mac=00:16:3e:24:11:38,bridge=br0'', >>> ] >>> >>> After boot, test1 domain has 3704M total memory (free -m), xm list test1 >>> shows 4000M. >>> >>> When I increase memory amount via xm mem-set test1 8000, then xm list >>> test1 really shows 8000M, but domU has 7704M total memory (free -m). >>> >>> from xend.log >>> [2013-03-04 18:10:18 4627] DEBUG (XendDomainInfo:1467) Setting memory >>> target of domain test1 (1) to 8000 MiB.Xen wiki recommends using last stable kernel, it means 3.8. I have used 3.8.2, where is the problem with dom0 memory solved. domU memory allocation was changed, see http://www.gossamer-threads.com/lists/xen/users/270487?do=post_view_flat>>> >>> 2) I am not able to increase vcpu count to domU. >>> After boot domU test1, it has really 8 vcpus, I am able to decrease >>> vcpus by xm vcpu-set test1 4. Cpus count inside test1 really decreases >>> to 4. Now I would like to increase vcpu count via xm vcpu-list test1 6, >>> but unsuccessfully. >>> >>> rajvir3 configs # xm vcpu-list test1 >>> Name ID VCPU CPU State Time(s) CPU >>> Affinity >>> test1 1 0 15 -b- 3.7 12-23 >>> test1 1 1 23 -b- 2.8 12-23 >>> test1 1 2 18 -b- 1.2 12-23 >>> test1 1 3 20 -b- 1.2 12-23 >>> test1 1 4 16 -b- 1.0 12-23 >>> test1 1 5 12 -b- 3.5 12-23 >>> test1 1 6 19 -b- 0.9 12-23 >>> test1 1 7 14 -b- 0.9 12-23 >>> >>> rajvir3 configs # xm vcpu-set test1 4 >>> >>> rajvir3 configs # xm vcpu-list test1 >>> Name ID VCPU CPU State Time(s) CPU >>> Affinity >>> test1 1 0 15 -b- 3.7 12-23 >>> test1 1 1 23 -b- 2.9 12-23 >>> test1 1 2 18 -b- 1.3 12-23 >>> test1 1 3 20 -b- 1.2 12-23 >>> test1 1 4 - --p 1.0 12-23 >>> test1 1 5 - --p 3.5 12-23 >>> test1 1 6 - --p 1.0 12-23 >>> test1 1 7 - --p 0.9 12-23 >>> >>> rajvir3 configs # xm vcpu-set test1 6 >>> >>> rajvir3 configs # xm vcpu-list test1 >>> Name ID VCPU CPU State Time(s) CPU >>> Affinity >>> test1 1 0 15 -b- 3.8 12-23 >>> test1 1 1 23 -b- 2.9 12-23 >>> test1 1 2 18 -b- 1.3 12-23 >>> test1 1 3 20 -b- 1.3 12-23 >>> test1 1 4 - --p 1.0 12-23 >>> test1 1 5 - --p 3.5 12-23 >>> test1 1 6 - --p 1.0 12-23 >>> test1 1 7 - --p 0.9 12-23 >>> >>> from xend.log >>> >>> [2013-03-04 18:12:58 4627] INFO (XendDomainInfo:1978) Set VCPU count on >>> domain test1 to 4 >>> [2013-03-04 18:14:51 4627] INFO (XendDomainInfo:1978) Set VCPU count on >>> domain test1 to 6 >>> >>> I was not able to increase vcpu count also in kernel 3.7.My domU did not have udev rules to hotplug cpus, follow these instructions http://wiki.xen.org/wiki/Paravirt_Linux_CPU_Hotplug>>> >>> Where could be the problem? >>> >>> Thanks >>> >>> Michal >>> >>> _______________________________________________ >>> Xen-users mailing list >>> Xen-users@lists.xen.org >>> http://lists.xen.org/xen-users >> >> >> >> >> _______________________________________________ >> Xen-users mailing list >> Xen-users@lists.xen.org >> http://lists.xen.org/xen-users >> > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users >