Jordi Moles Blanco
2011-Nov-16 13:24 UTC
[Xen-users] understanding memory hot-plug in a paravitual domU
Hello everyone, I''ve been fighting for days with this issue and I would like to ask you directly if what i intend to do is actually possible. let''s say I''ve got a paravirtual domU created with a conf file similar to this (this is not a complete one): *********** kernel=''/boot/vmlinuz-2.6.26-2-xen-amd64'' ramdisk=''/boot/initrd.img-2.6.26-2-xen-amd64'' memory=''2048'' name=''server'' vcpus=1 .... extra = "console=tty xencons=tty" ************ so... obviously when i do ******** # xm list Name ID Mem VCPUs State Time(s) Domain-0 0 1536 8 r----- 850454.2 server 1 2048 1 -b---- 1003142.4 ********* i get that ''server'' has 2048MB of memory. Then i start playing with: ********** # xm mem-max server 4096 # xm mem-max server 4096 ********** and everything seems to work like a charm, none of the above commands actually fails or gives any warnings. but then I do "xm top" or I ssh into ''server'' and do "top" and those new MB of memory are not present, ''server'' hasn''t hot-plugged the new memory. so... is there a way to do that? i''ve tried with different domUs (debian, fedora, ...) and also different versions of xen: 3.3 i 4.0. The result is always the same, i always need to reboot the machine. I''ve read all the documentation I''ve come across and found nothing that works. I''ve seen some documents with references to acpi hot plug and memory hotplug kernel modules, which I''ve tried to load with no luck. I''m successful in loading these modules into the dom0''s kernel, but it fails in the domU. Actually, the kernel that paravirtual machines use: /boot/vmlinuz-2.6.26-2-xen-amd64 is compiled with hotplug flags to yes, but it just seems that this is not supported in virtualization. So, i just would like to know if this is possible without rebooting or if its a feature in the to-do list. Thanks in advance. -- Jordi Moles Blanco Sistemes Cdmon.com ___________________________ Tlf: 902 36 41 38 Tlf: 93 567 75 77 mailto: jordi@cdmon.com http://www.cdmon.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
James Harper
2011-Nov-16 22:33 UTC
RE: [Xen-users] understanding memory hot-plug in a paravitual domU
> i get that ''server'' has 2048MB of memory. > > Then i start playing with: > > ********** > # xm mem-max server 4096 > # xm mem-max server 4096 > ********** >Did you really do mem-max twice, or was the second one actually mem-set? James _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Jordi Moles Blanco
2011-Nov-17 08:14 UTC
Re: understanding memory hot-plug in a paravitual domU
Al 16/11/11 23:33, En/na James Harper ha escrit:>> i get that ''server'' has 2048MB of memory. >> >> Then i start playing with: >> >> ********** >> # xm mem-max server 4096 >> # xm mem-max server 4096 >> ********** >> > Did you really do mem-max twice, or was the second one actually mem-set? > > JamesHi, sorry, obviously, that was a mistake. actually, i did this: ********** # xm mem-max server 4096 # xm mem-set server 4096 ********** can i suppose then that this should work? the thing is that xen doesn''t complain, but both ''xen top'' and the machine itself don''t seem to show any increase in the memory. However, it works when setting a lower value. For example, I can create ''server'' with 2048 and then go down to 1024 and then to 2048 again, but never to a higher value. So... i guess that mem-hotplug works because I can go down and up, but It can never go up higher than the initial memory value. that''s why yesterday I was asking if that''s is even possible, I don''t know if it has anything to do, but dom0 is Debian Squeeze and the domUs i''ve tried with are Debian Squeeze, Ubuntu Natty and Fedora 15. Thanks. -- Jordi Moles Blanco Sistemes Cdmon.com ___________________________ Tlf: 902 36 41 38 Tlf: 93 567 75 77 mailto: jordi@cdmon.com http://www.cdmon.com
Hi, Everything that I''ve read said that you can''t balloon the memory beyond the maximum you specified in the config file when you created the domain. (eg. http://www.centos.org/docs/5/html/Virtualization-en-US/virt-task-xm-create-manage-doms.html#id2645186) So, no, based on the docs, the behaviour that you''re seeing is expected, and, no, you can''t balloon higher than the initial memory value. Strange that xm mem-max isn''t giving errors though. ...> ********** > # xm mem-max server 4096 > # xm mem-set server 4096 > ********** > > > can i suppose then that this should work? > > the thing is that xen doesn''t complain, but both ''xen top'' and the machine > itself don''t seem to show any increase in the memory. > > However, it works when setting a lower value. > > For example, I can create ''server'' with 2048 and then go down to 1024 and > then to 2048 again, but never to a higher value. So... i guess that > mem-hotplug works because I can go down and up, but It can never go up > higher than the initial memory value. > > that''s why yesterday I was asking if that''s is even possible,
Jordi Moles Blanco
2011-Nov-18 11:42 UTC
Re: understanding memory hot-plug in a paravitual domU
Hi, thanks Kyle for the information. I had read a lot of documentation and posts in forums and somehow missed that. As you say, I feel that people would be less confused with this if mem-max clearly said that you can''t grow the balloon higher than the initial memory value. Thanks. Al 18/11/11 11:47, En/na Kyle Lexmond ha escrit:> Hi, > Everything that I''ve read said that you can''t balloon the memory > beyond the maximum you specified in the config file when you created > the domain. (eg. > http://www.centos.org/docs/5/html/Virtualization-en-US/virt-task-xm-create-manage-doms.html#id2645186) > > So, no, based on the docs, the behaviour that you''re seeing is > expected, and, no, you can''t balloon higher than the initial memory > value. > > Strange that xm mem-max isn''t giving errors though. > > ... >> ********** >> # xm mem-max server 4096 >> # xm mem-set server 4096 >> ********** >> >> >> can i suppose then that this should work? >> >> the thing is that xen doesn''t complain, but both ''xen top'' and the machine >> itself don''t seem to show any increase in the memory. >> >> However, it works when setting a lower value. >> >> For example, I can create ''server'' with 2048 and then go down to 1024 and >> then to 2048 again, but never to a higher value. So... i guess that >> mem-hotplug works because I can go down and up, but It can never go up >> higher than the initial memory value. >> >> that''s why yesterday I was asking if that''s is even possible,-- Jordi Moles Blanco Sistemes Cdmon.com ___________________________ Tlf: 902 36 41 38 Tlf: 93 567 75 77 mailto: jordi@cdmon.com http://www.cdmon.com
George Shuklin
2011-Nov-18 12:05 UTC
Re: understanding memory hot-plug in a paravitual domU
AFAIK there is two different operational mode. When we running classic old kernels we using just ballooning. And newer kernel (3.1, 3.2) supports for memory-hotplug which allow to grow domU memory beyond initial memory. Unfortunately I was not succeed at making memory-hotplug works. Sad. But some older -xen kernel allow little trick - domain starts with ''pre-inflated'' balloon, allowing (not really, but looking like) grow at values higher then initial memory in domU. On 18.11.2011 15:42, Jordi Moles Blanco wrote:> Hi, > > thanks Kyle for the information. > > I had read a lot of documentation and posts in forums and somehow > missed that. > > As you say, I feel that people would be less confused with this if > mem-max clearly said that you can''t grow the balloon higher than the > initial memory value. > > Thanks. > > > > Al 18/11/11 11:47, En/na Kyle Lexmond ha escrit: >> Hi, >> Everything that I''ve read said that you can''t balloon the memory >> beyond the maximum you specified in the config file when you created >> the domain. (eg. >> http://www.centos.org/docs/5/html/Virtualization-en-US/virt-task-xm-create-manage-doms.html#id2645186) >> >> >> So, no, based on the docs, the behaviour that you''re seeing is >> expected, and, no, you can''t balloon higher than the initial memory >> value. >> >> Strange that xm mem-max isn''t giving errors though. >> >> ... >>> ********** >>> # xm mem-max server 4096 >>> # xm mem-set server 4096 >>> ********** >>> >>> >>> can i suppose then that this should work? >>> >>> the thing is that xen doesn''t complain, but both ''xen top'' and the >>> machine >>> itself don''t seem to show any increase in the memory. >>> >>> However, it works when setting a lower value. >>> >>> For example, I can create ''server'' with 2048 and then go down to >>> 1024 and >>> then to 2048 again, but never to a higher value. So... i guess that >>> mem-hotplug works because I can go down and up, but It can never go up >>> higher than the initial memory value. >>> >>> that''s why yesterday I was asking if that''s is even possible, > >
Jordi Moles Blanco
2011-Nov-18 12:26 UTC
Re: understanding memory hot-plug in a paravitual domU
Hi, George, thanks for that information as well. It''s good to know that this feature is in the roadmap. I''m also working on resezing disk partitions without having to reboot the domU. It works for 3.X kernels pretty much "out of the box". The problem, though, is that running a debian wheeze dom0 with kernel 3.X and xen 4.1 is a bit unstable to me. I''ve run some stress tests under those software versions and came across some problems that i had never seen. Anyway, if kernel 3.1 comes with memory hotplug i think I''ll have to switch to get all these improvements: memory hotplug, hot disk resize, new xl scripts, etc. thanks. Al 18/11/11 13:05, En/na George Shuklin ha escrit:> AFAIK there is two different operational mode. When we running classic > old kernels we using just ballooning. And newer kernel (3.1, 3.2) > supports for memory-hotplug which allow to grow domU memory beyond > initial memory. > > Unfortunately I was not succeed at making memory-hotplug works. Sad. > > But some older -xen kernel allow little trick - domain starts with > ''pre-inflated'' balloon, allowing (not really, but looking like) grow > at values higher then initial memory in domU. > > On 18.11.2011 15:42, Jordi Moles Blanco wrote: >> Hi, >> >> thanks Kyle for the information. >> >> I had read a lot of documentation and posts in forums and somehow >> missed that. >> >> As you say, I feel that people would be less confused with this if >> mem-max clearly said that you can''t grow the balloon higher than the >> initial memory value. >> >> Thanks. >> >> >> >> Al 18/11/11 11:47, En/na Kyle Lexmond ha escrit: >>> Hi, >>> Everything that I''ve read said that you can''t balloon the memory >>> beyond the maximum you specified in the config file when you created >>> the domain. (eg. >>> http://www.centos.org/docs/5/html/Virtualization-en-US/virt-task-xm-create-manage-doms.html#id2645186) >>> >>> >>> So, no, based on the docs, the behaviour that you''re seeing is >>> expected, and, no, you can''t balloon higher than the initial memory >>> value. >>> >>> Strange that xm mem-max isn''t giving errors though. >>> >>> ... >>>> ********** >>>> # xm mem-max server 4096 >>>> # xm mem-set server 4096 >>>> ********** >>>> >>>> >>>> can i suppose then that this should work? >>>> >>>> the thing is that xen doesn''t complain, but both ''xen top'' and the >>>> machine >>>> itself don''t seem to show any increase in the memory. >>>> >>>> However, it works when setting a lower value. >>>> >>>> For example, I can create ''server'' with 2048 and then go down to >>>> 1024 and >>>> then to 2048 again, but never to a higher value. So... i guess that >>>> mem-hotplug works because I can go down and up, but It can never go up >>>> higher than the initial memory value. >>>> >>>> that''s why yesterday I was asking if that''s is even possible, >> >> > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users-- Jordi Moles Blanco Sistemes Cdmon.com ___________________________ Tlf: 902 36 41 38 Tlf: 93 567 75 77 mailto: jordi@cdmon.com http://www.cdmon.com