Hi everyone, I''m running 10 VMs now quite happily, using LVM2 with distinct storage areas for each VM filesystem. I suspect I''ll be able to run around 30 VMs on the dual xeon 2.8GHz machines successfully, but I''m wondering how to manage the RAM allocations for each. I calculate that each VM will have around 110MB RAM if I run that many, including a healthy amount of RAM for Dom-0. I am wondering how this can be enough for LAMP VMs. Some of the systems I''m running now don''t seem to perform very well with less than 400MB of RAM, and I can only really do that if some kind soul fixes Xen so I can use the full 12GB capacity of my Xen servers (they have 4GB now -- no good wasting money!). This is probably a stupid question, but can I allocate more RAM to each VM than arithmetic would suggest? Is there some kind of compression or COW that goes on in the Xen memory management code? Would it be best to run one replicated MySQL server (or several with different versions) loaded into a VM on each physical machine, and have all the other machines use that over virtual network connections? I suppose this would be perfectly okay, and I could do the same with Apache too I guess. But as I am going to be migrating the sites from many real machines into Xen, splitting them up into individual site machines as I go (at the moment our servers can run many sites), it would take a lot of work to change their configuration so that they connected to a networked copy of MySQL or Apache rather than a local one. Another option, and I''m thinking you are all going to say ''go for that one!'', would be to use 64bit machines for our main cluster, perhaps just using the two dual Xeon machines I have now for development purposes. That way I could use lots of RAM and presumably load up even more VMs into each. If I go down this road, I''m wondering what kind of hardware I should get. Does Xen work well with Opterons? I''m an AMD fan and would be very happy to use them if I went for this option. I''m after inexpensive commodity servers, so if anyone has any recommendations (especially if you are in NZ or Australia), I''m all ears. May I ask how much RAM I could use with a dual Opteron system in a sensible Xen configuration? What''s been tested? One final question: if I were to ask the directors of the company if they would be interested in sponsoring big memory on Xeon-based machines, what kind of commitment would we be talking about? It could be cheaper to do this than to buy a cluster of dual Opteron machines! I do realise though that the true impediment will probably be the amount of time you developers have, so I''ll brace myself for disappointment! :o) Thanks everyone for your valuable time. Regards, Paul ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark A. Williamson
2004-Sep-29 03:30 UTC
Re: [Xen-devel] RAM allocation and miscellaneous questions
> I''m running 10 VMs now quite happily, using LVM2 with distinct storage > areas for each VM filesystem.Cool!> can I allocate more RAM to each VM than arithmetic would suggest?Yes and no ;-) Xen does not do paging to disk, so you can''t (for instance) tell a domain it has 256MB but only give it 128MB and then fake out the extra capacity using disk. The solution under Xen would be to just get the domain to do its own paging - this has the same end result and avoids certain performance and implementation nasties associated with paging in the VMM. Using the balloon driver, you can have a setup where you add and remove memory to / from domains according to their need, so that you can shrink the memory footprints of domains that don''t (currently) need as much in core. This is done manually. Of course, you can also use suspend / resume and live migration to move domains around in order to balance memory load.> Is there some kind of compression or COW > that goes on in the Xen memory management code?Not right now. There is a plan to implement a shared buffer cache, which would allow domains to share unmodified pages of data that have been read from disk. This is planned as a performance optimisation not a space optimisation, however.> Would it be best to run one replicated MySQL server (or several with > different versions) loaded into a VM on each physical machine, and have all > the other machines use that over virtual network connections?It probably depends. I guess you''d save some memory footprint that way (compared to running many copies of the servers) but I really don''t know how much.> [64 bit machines] > Does Xen work well with Opterons?In 32-bit mode, it works. In 64-bit mode, not quite yet. x86_64 support is under development at the moment and is scheduled as a post-2.0 feature. However that will be the preferred path to large memory configurations. HTH, Mark ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Paul Dorman
2004-Sep-29 04:57 UTC
Re: [Xen-devel] RAM allocation and miscellaneous questions
It certainly does Mark (help, that is). I would like to learn more about this balloon driver, which could make things very managable. Is there any documentation or tutorial-like info about? Also, though my domains are running happily, I am seeing plenty of memory-related errors. I''ve just hooked up a serial cable between my development servers and my desktop (I live with fans) so I''ll be able to furnish the list with more details. I have been wondering why my Domain-0 machine hasn''t been using swap! :o) So I should just dump that partition? I suppose I''ll have to be very rigorous when it comes to keeping that domain lean yes? (Especially as I only have 4GB to play with!). Out side of basic OS functionality, are there any Xen or LVM2 related processes I have to account for when setting the amount of RAM I can use for Domain-0? I don''t suppose I could use the ballon driver for Domain-0 too!?! Thanks for your reply. Regards, Paul On Wednesday 29 September 2004 03:30 pm, Mark A. Williamson wrote:> > I''m running 10 VMs now quite happily, using LVM2 with distinct storage > > areas for each VM filesystem. > > Cool! > > > can I allocate more RAM to each VM than arithmetic would suggest? > > Yes and no ;-) > > Xen does not do paging to disk, so you can''t (for instance) tell a domain > it has 256MB but only give it 128MB and then fake out the extra capacity > using disk. The solution under Xen would be to just get the domain to do > its own paging - this has the same end result and avoids certain > performance and implementation nasties associated with paging in the VMM. > > Using the balloon driver, you can have a setup where you add and remove > memory to / from domains according to their need, so that you can shrink > the memory footprints of domains that don''t (currently) need as much in > core. This is done manually. > > Of course, you can also use suspend / resume and live migration to move > domains around in order to balance memory load. > > > Is there some kind of compression or COW > > that goes on in the Xen memory management code? > > Not right now. There is a plan to implement a shared buffer cache, which > would allow domains to share unmodified pages of data that have been read > from disk. This is planned as a performance optimisation not a space > optimisation, however. > > > Would it be best to run one replicated MySQL server (or several with > > different versions) loaded into a VM on each physical machine, and have > > all the other machines use that over virtual network connections? > > It probably depends. I guess you''d save some memory footprint that way > (compared to running many copies of the servers) but I really don''t know > how much. > > > [64 bit machines] > > Does Xen work well with Opterons? > > In 32-bit mode, it works. In 64-bit mode, not quite yet. x86_64 support > is under development at the moment and is scheduled as a post-2.0 feature. > However that will be the preferred path to large memory configurations. > > HTH, > Mark------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Ian Pratt
2004-Sep-29 07:55 UTC
Re: [Xen-devel] RAM allocation and miscellaneous questions
> It certainly does Mark (help, that is). I would like to learn more about this > balloon driver, which could make things very managable. Is there any > documentation or tutorial-like info about?The balloon driver hasn''t been forward ported from 2.4 to 2.6 -- it''s on the todo list (it''ll probably work unmodified modulo changes to the way proc fs works).> I have been wondering why my Domain-0 machine hasn''t been using swap! :o) So I > should just dump that partition?If you''ve configured swap, domain 0 should be using just like any normal linux kernel. The balloon driver works in dom0 just like any other domain.> I suppose I''ll have to be very rigorous when > it comes to keeping that domain lean yes? (Especially as I only have 4GB to > play with!). Out side of basic OS functionality, are there any Xen or LVM2 > related processes I have to account for when setting the amount of RAM I can > use for Domain-0?-xen linux kernels don''t have any special processes that take any noticeable extra resources. In domain 0 you''ll have to worry about the footprint of xend, but it''s not that big. Ian ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Keir Fraser
2004-Sep-29 08:37 UTC
Re: [Xen-devel] RAM allocation and miscellaneous questions
> Another option, and I''m thinking you are all going to say ''go for that one!'', > would be to use 64bit machines for our main cluster, perhaps just using the > two dual Xeon machines I have now for development purposes. That way I could > use lots of RAM and presumably load up even more VMs into each. If I go down > this road, I''m wondering what kind of hardware I should get. Does Xen work > well with Opterons? I''m an AMD fan and would be very happy to use them if I > went for this option. I''m after inexpensive commodity servers, so if anyone > has any recommendations (especially if you are in NZ or Australia), I''m all > ears.Opteron isn''t supported yet in 64-bit mode, but it /is/ on the todo list to get it finished off. As opposed to PAE36 mode, which it is highly likley that we will never support.> May I ask how much RAM I could use with a dual Opteron system in a sensible > Xen configuration? What''s been tested?You will be able to stuff in as much RAM as the machine will take. :-)> One final question: if I were to ask the directors of the company if they > would be interested in sponsoring big memory on Xeon-based machines, what > kind of commitment would we be talking about? It could be cheaper to do this > than to buy a cluster of dual Opteron machines! I do realise though that the > true impediment will probably be the amount of time you developers have, so > I''ll brace myself for disappointment! :o)So you already have an existing 32-bit cluster that you would have to sideline? -- Keir ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On September 29, 9:55 am Ian Pratt <Ian.Pratt@cl.cam.ac.uk> wrote:> > It certainly does Mark (help, that is). I would like to learn more > > about this balloon driver, which could make things very managable. Is > > there any documentation or tutorial-like info about? > > The balloon driver hasn''t been forward ported from 2.4 to 2.6 -- > it''s on the todo list (it''ll probably work unmodified modulo > changes to the way proc fs works). >For people that still use linux 2.4 (like me) is there a tutorial or some documentation about the ballon driver ?> > I have been wondering why my Domain-0 machine hasn''t been using swap! > > :o) So I should just dump that partition? > > If you''ve configured swap, domain 0 should be using just like any > normal linux kernel. The balloon driver works in dom0 just like > any other domain.It seems logical, but due to the fact that only a few process are running ind dom0, swap file don''t need to be huge is it ?> > > I suppose I''ll have to be very rigorous when > > it comes to keeping that domain lean yes? (Especially as I only have > > 4GB to play with!). Out side of basic OS functionality, are there any > > Xen or LVM2 related processes I have to account for when setting the > > amount of RAM I can use for Domain-0? > > -xen linux kernels don''t have any special processes that take any > noticeable extra resources. > > In domain 0 you''ll have to worry about the footprint of xend, but > it''s not that big. >When starting with xen one or two month ago, i''ve created a dom0 with only 32 Mo of ram, and when i launched xend, it was so slow that i tought i had a problem. I didn''t take time to investigate and didn''t remember how much memory was used and if the system was swaping or not. I just have done a test now with my current xen putting 32768 Mo on xen command line (not the kernel line in grub). And the system booted a bit slower and i suspect that durring the launch of xend it has done some swap. free return this just after the boot : mat@zeus:/$ free total used free shared buffers cached Mem: 32768 28572 4196 0 0 7172 -/+ buffers/cache: 21400 11368 Swap: 506008 312 505696 Having this process: 207 ? Ss 0:00 dhclient -e -pf /var/run/dhclient.eth0.pid -lf /var/ 253 ? Ss 0:00 /sbin/syslogd 256 ? Ss 0:00 /sbin/klogd 261 ? Ss 0:00 /usr/sbin/inetd 268 ? Ss 0:00 /usr/sbin/sshd 274 ? S 0:00 xfrd 275 ? S 0:00 python /usr/sbin/xend start 288 ? Ss 0:00 /usr/sbin/cron 327 tty1 Ss+ 0:00 /sbin/getty 38400 tty1 328 tty2 Ss+ 0:00 /sbin/getty 38400 tty2 329 tty3 Ss+ 0:00 /sbin/getty 38400 tty3 330 tty4 Ss+ 0:00 /sbin/getty 38400 tty4 331 tty5 Ss+ 0:00 /sbin/getty 38400 tty5 332 tty6 Ss+ 0:00 /sbin/getty 38400 tty6 333 ? Ss 0:00 sshd: mat [priv] 335 ? R 0:00 sshd: mat@pts/0 336 pts/0 Ss 0:00 -bash 344 pts/0 R+ 0:00 ps ax i guess i could reduce the number of getty and get rid of dhclient (the adress of dom0 is a static one ...). It was my 2cents thoughts ... Matthieu ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark A. Williamson
2004-Sep-29 12:26 UTC
Re: [Xen-devel] RAM allocation and miscellaneous questions
> It certainly does Mark (help, that is). I would like to learn more about > this balloon driver, which could make things very managable. Is there any > documentation or tutorial-like info about?AFAIK, there''s a file in /proc/xen called memory_target or something similar. You just write your memory target into there as ASCII text. e.g. echo target > /proc/xen/memory_target The idea is that there''ll (at some stage) be control messages from dom0 to do this so that you don''t have to log into a domain to change it''s memory target. As Ian said, the driver isn''t yet ported to 2.6 (I forgot that) but it shouldn''t take long when someone gets around to it :-)> Also, though my domains are running happily, I am seeing plenty of > memory-related errors. I''ve just hooked up a serial cable between my > development servers and my desktop (I live with fans) so I''ll be able to > furnish the list with more details.I can relate to living with fans :-) It''s useful to have a serial console available in case of machine crashes / hangs. When the machine is still operational, xm dmesg will retrive Xen''s boot output.> I have been wondering why my Domain-0 machine hasn''t been using swap! :o)I should have explained this better: domains can swap their own memory to / from disk as much as they like. In contrast, Xen itself does not do any swapping to provide the illusion of bigger physical memory. If domains want a bigger memory than you gave them, it''s up to them to do all their own swapping. If you want dom0 to swap, you should be able to set up swap in the usual way (if it worked on vanilla linux, it should just keep working). If you want other domains to swap, you can give them VBDs to swap to - you just have to mkswap on them and configure it in the fstab, same as usual.> So I should just dump that partition? I suppose I''ll have to be very > rigorous when it comes to keeping that domain lean yes? (Especially as I > only have 4GB to play with!). Out side of basic OS functionality, are there > any Xen or LVM2 related processes I have to account for when setting the > amount of RAM I can use for Domain-0? > > I don''t suppose I could use the ballon driver for Domain-0 too!?! > > Thanks for your reply. > > Regards, > Paul > > On Wednesday 29 September 2004 03:30 pm, Mark A. Williamson wrote: > > > I''m running 10 VMs now quite happily, using LVM2 with distinct storage > > > areas for each VM filesystem. > > > > Cool! > > > > > can I allocate more RAM to each VM than arithmetic would suggest? > > > > Yes and no ;-) > > > > Xen does not do paging to disk, so you can''t (for instance) tell a domain > > it has 256MB but only give it 128MB and then fake out the extra capacity > > using disk. The solution under Xen would be to just get the domain to do > > its own paging - this has the same end result and avoids certain > > performance and implementation nasties associated with paging in the VMM. > > > > Using the balloon driver, you can have a setup where you add and remove > > memory to / from domains according to their need, so that you can shrink > > the memory footprints of domains that don''t (currently) need as much in > > core. This is done manually. > > > > Of course, you can also use suspend / resume and live migration to move > > domains around in order to balance memory load. > > > > > Is there some kind of compression or COW > > > that goes on in the Xen memory management code? > > > > Not right now. There is a plan to implement a shared buffer cache, which > > would allow domains to share unmodified pages of data that have been read > > from disk. This is planned as a performance optimisation not a space > > optimisation, however. > > > > > Would it be best to run one replicated MySQL server (or several with > > > different versions) loaded into a VM on each physical machine, and have > > > all the other machines use that over virtual network connections? > > > > It probably depends. I guess you''d save some memory footprint that way > > (compared to running many copies of the servers) but I really don''t know > > how much. > > > > > [64 bit machines] > > > Does Xen work well with Opterons? > > > > In 32-bit mode, it works. In 64-bit mode, not quite yet. x86_64 support > > is under development at the moment and is scheduled as a post-2.0 > > feature. However that will be the preferred path to large memory > > configurations. > > > > HTH, > > Mark------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Paul Dorman
2004-Sep-29 19:16 UTC
Re: [Xen-devel] RAM allocation and miscellaneous questions
Hmmmmmm. Lots of RAM :o) Yes, I do have a ''cluster'', but only two machines so far. The problem for me is that multi-processor Opteron 1u machines (or 2u or 4u for that matter) are hard to come by here in New Zealand, unless we pay through the nose for an HP or other big-name machine. I can get dual Xeon machines pretty cheap though. If I look at the Opteron option, I''ll have to compare it to just buying more Xeons. Either way, it''s going to cost more for our initial cluster than it would if I could just use 12GB in a cheap Xeon unit. Don''t get me wrong though, if Opteron units were available here I would have got them from the outset. Personally I have no issue with sourcing machines from Australia or somewhere, but my bosses will be reluctant due to support issues, etc. Regards, Paul On Wednesday 29 September 2004 08:37 pm, Keir Fraser wrote:> > Another option, and I''m thinking you are all going to say ''go for that > > one!'', would be to use 64bit machines for our main cluster, perhaps just > > using the two dual Xeon machines I have now for development purposes. > > That way I could use lots of RAM and presumably load up even more VMs > > into each. If I go down this road, I''m wondering what kind of hardware I > > should get. Does Xen work well with Opterons? I''m an AMD fan and would be > > very happy to use them if I went for this option. I''m after inexpensive > > commodity servers, so if anyone has any recommendations (especially if > > you are in NZ or Australia), I''m all ears. > > Opteron isn''t supported yet in 64-bit mode, but it /is/ on the todo > list to get it finished off. As opposed to PAE36 mode, which it is > highly likley that we will never support. > > > May I ask how much RAM I could use with a dual Opteron system in a > > sensible Xen configuration? What''s been tested? > > You will be able to stuff in as much RAM as the machine will take. :-) > > > One final question: if I were to ask the directors of the company if they > > would be interested in sponsoring big memory on Xeon-based machines, what > > kind of commitment would we be talking about? It could be cheaper to do > > this than to buy a cluster of dual Opteron machines! I do realise though > > that the true impediment will probably be the amount of time you > > developers have, so I''ll brace myself for disappointment! :o) > > So you already have an existing 32-bit cluster that you would have to > sideline? > > -- Keir------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Ronald G. Minnich
2004-Sep-29 19:53 UTC
Re: [Xen-devel] RAM allocation and miscellaneous questions
> On Wednesday 29 September 2004 08:37 pm, Keir Fraser wrote: > > Opteron isn''t supported yet in 64-bit mode, but it /is/ on the todo > > list to get it finished off. As opposed to PAE36 mode, which it is > > highly likley that we will never support.^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ a truly excellent decision. I mean it. PAE36 is just too ugly to deal with, and we don''t need it now with Opterons out there as cheap or cheaper than Pentiums. ron ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Ian Pratt
2004-Sep-30 08:26 UTC
Re: [Xen-devel] RAM allocation and miscellaneous questions
> On September 29, 9:55 am Ian Pratt <Ian.Pratt@cl.cam.ac.uk> wrote: > > > It certainly does Mark (help, that is). I would like to learn more > > > about this balloon driver, which could make things very managable. Is > > > there any documentation or tutorial-like info about? > > > > The balloon driver hasn''t been forward ported from 2.4 to 2.6 -- > > it''s on the todo list (it''ll probably work unmodified modulo > > changes to the way proc fs works). > > > For people that still use linux 2.4 (like me) is there a tutorial or some > documentation about the ballon driver ?Mark is currently forward porting it to 2.6, and we''re planing on adding a couple of features to enable the memory footprint of a domain to be controlled from xm/xend rather than just from just within a domain.> > If you''ve configured swap, domain 0 should be using just like any > > normal linux kernel. The balloon driver works in dom0 just like > > any other domain. > It seems logical, but due to the fact that only a few process are running > ind dom0, swap file don''t need to be huge is it ?Correct.> When starting with xen one or two month ago, i''ve created a dom0 with only > 32 Mo of ram, and when i launched xend, it was so slow that i tought i had > a problem. > I didn''t take time to investigate and didn''t remember how much memory was > used and if the system was swaping or not. > I just have done a test now with my current xen putting 32768 Mo on xen > command line (not the kernel line in grub). > And the system booted a bit slower and i suspect that durring the launch of > xend it has done some swap. > free return this just after the boot : > mat@zeus:/$ free > total used free shared buffers cached > Mem: 32768 28572 4196 0 0 7172 > -/+ buffers/cache: 21400 11368 > Swap: 506008 312 505696The resident working set of xend once it''s started isn''t too bad, but all the library loading (particularly twisted) does lead to quite a large virtual memory footprint. Perhaps one day, when xend has all the features we want, it''ll be time to re-write it in C, or OCAML... Ian ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Keir Fraser
2004-Sep-30 09:02 UTC
Re: [Xen-devel] RAM allocation and miscellaneous questions
How odd -- I expected that in most places there would be multiple white-box vendors selling Opteron-based systems. It''s not like the component parts are in short supply. Except maybe in NZ. :-) -- Keir> Hmmmmmm. Lots of RAM :o) > > Yes, I do have a ''cluster'', but only two machines so far. The problem for me > is that multi-processor Opteron 1u machines (or 2u or 4u for that matter) are > hard to come by here in New Zealand, unless we pay through the nose for an HP > or other big-name machine. I can get dual Xeon machines pretty cheap though. > If I look at the Opteron option, I''ll have to compare it to just buying more > Xeons. Either way, it''s going to cost more for our initial cluster than it > would if I could just use 12GB in a cheap Xeon unit. Don''t get me wrong > though, if Opteron units were available here I would have got them from the > outset. > > Personally I have no issue with sourcing machines from Australia or somewhere, > but my bosses will be reluctant due to support issues, etc. > > Regards, > Paul------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark A. Williamson
2004-Sep-30 13:26 UTC
Re: [Xen-devel] RAM allocation and miscellaneous questions
> For people that still use linux 2.4 (like me) is there a tutorial or some > documentation about the ballon driver ?Documentation, 2.6 support and facilities to control other domain''s balloons using xm in domain 0 are cooking and should be ready shortly. In the meantime, do: # echo target_megabytes > /proc/xen/memory_target To set a new memory footprint. There''s a Linux kernel param (mem=..., I think) for setting the maximum amount of memory a domain can handle, if larger than it''s initial allocation (i.e. if you want to start with x megabytes but grow it later). Trying to grow the domain''s memory beyond what it''s expecting (the initial allocation or the kernel param if you specified that) will not work. Use: # cat /proc/xen/memory_target To get the current footprint. HTH, Mark> > > I have been wondering why my Domain-0 machine hasn''t been using swap! > > > > > > :o) So I should just dump that partition? > > > > If you''ve configured swap, domain 0 should be using just like any > > normal linux kernel. The balloon driver works in dom0 just like > > any other domain. > > It seems logical, but due to the fact that only a few process are running > ind dom0, swap file don''t need to be huge is it ? > > > > I suppose I''ll have to be very rigorous when > > > it comes to keeping that domain lean yes? (Especially as I only have > > > 4GB to play with!). Out side of basic OS functionality, are there any > > > Xen or LVM2 related processes I have to account for when setting the > > > amount of RAM I can use for Domain-0? > > > > -xen linux kernels don''t have any special processes that take any > > noticeable extra resources. > > > > In domain 0 you''ll have to worry about the footprint of xend, but > > it''s not that big. > > When starting with xen one or two month ago, i''ve created a dom0 with only > 32 Mo of ram, and when i launched xend, it was so slow that i tought i had > a problem. > I didn''t take time to investigate and didn''t remember how much memory was > used and if the system was swaping or not. > I just have done a test now with my current xen putting 32768 Mo on xen > command line (not the kernel line in grub). > And the system booted a bit slower and i suspect that durring the launch of > xend it has done some swap. > free return this just after the boot : > mat@zeus:/$ free > total used free shared buffers cached > Mem: 32768 28572 4196 0 0 7172 > -/+ buffers/cache: 21400 11368 > Swap: 506008 312 505696 > > Having this process: > 207 ? Ss 0:00 dhclient -e -pf /var/run/dhclient.eth0.pid -lf > /var/ > 253 ? Ss 0:00 /sbin/syslogd > 256 ? Ss 0:00 /sbin/klogd > 261 ? Ss 0:00 /usr/sbin/inetd > 268 ? Ss 0:00 /usr/sbin/sshd > 274 ? S 0:00 xfrd > 275 ? S 0:00 python /usr/sbin/xend start > 288 ? Ss 0:00 /usr/sbin/cron > 327 tty1 Ss+ 0:00 /sbin/getty 38400 tty1 > 328 tty2 Ss+ 0:00 /sbin/getty 38400 tty2 > 329 tty3 Ss+ 0:00 /sbin/getty 38400 tty3 > 330 tty4 Ss+ 0:00 /sbin/getty 38400 tty4 > 331 tty5 Ss+ 0:00 /sbin/getty 38400 tty5 > 332 tty6 Ss+ 0:00 /sbin/getty 38400 tty6 > 333 ? Ss 0:00 sshd: mat [priv] > 335 ? R 0:00 sshd: mat@pts/0 > 336 pts/0 Ss 0:00 -bash > 344 pts/0 R+ 0:00 ps ax > > i guess i could reduce the number of getty and get rid of dhclient (the > adress of dom0 is a static one ...). > > It was my 2cents thoughts ... > Matthieu > > > ------------------------------------------------------- > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > Use IT products in your business? Tell us what you think of them. Give us > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more > http://productguide.itmanagersjournal.com/guidepromo.tmpl > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Brian Wolfe
2004-Sep-30 20:21 UTC
Re: [Xen-devel] RAM allocation and miscellaneous questions
Question from a security standpoint... How exactly does this proc control relate to the memory parameter specified when you star tthe domain? Can an unprivileged domain increase it''s memory footprint to MORE than the originally specified mem_size? Or is this just a local control of up to the originally specified mem_size? If it''s arbitrary (not constrained by mem_size) then I''d have to point out that some of us do NOT want this control to be allowed to go beyond what mem_size originally sets the domain at in unprivileged domains. Privileged domains would be exempted from this max limit of course. ;) If i''m just not paying attention and this was already explained, feel free to smack me. :) On Thu, 2004-09-30 at 08:26, Mark A. Williamson wrote:> > For people that still use linux 2.4 (like me) is there a tutorial or some > > documentation about the ballon driver ? > > Documentation, 2.6 support and facilities to control other domain''s balloons > using xm in domain 0 are cooking and should be ready shortly. > > In the meantime, do: > # echo target_megabytes > /proc/xen/memory_target > To set a new memory footprint. There''s a Linux kernel param (mem=..., I > think) for setting the maximum amount of memory a domain can handle, if > larger than it''s initial allocation (i.e. if you want to start with x > megabytes but grow it later). > > Trying to grow the domain''s memory beyond what it''s expecting (the initial > allocation or the kernel param if you specified that) will not work. > > Use: > # cat /proc/xen/memory_target > To get the current footprint. > > HTH, > Mark > > > > > I have been wondering why my Domain-0 machine hasn''t been using swap! > > > > > > > > :o) So I should just dump that partition? > > > > > > If you''ve configured swap, domain 0 should be using just like any > > > normal linux kernel. The balloon driver works in dom0 just like > > > any other domain. > > > > It seems logical, but due to the fact that only a few process are running > > ind dom0, swap file don''t need to be huge is it ? > > > > > > I suppose I''ll have to be very rigorous when > > > > it comes to keeping that domain lean yes? (Especially as I only have > > > > 4GB to play with!). Out side of basic OS functionality, are there any > > > > Xen or LVM2 related processes I have to account for when setting the > > > > amount of RAM I can use for Domain-0? > > > > > > -xen linux kernels don''t have any special processes that take any > > > noticeable extra resources. > > > > > > In domain 0 you''ll have to worry about the footprint of xend, but > > > it''s not that big. > > > > When starting with xen one or two month ago, i''ve created a dom0 with only > > 32 Mo of ram, and when i launched xend, it was so slow that i tought i had > > a problem. > > I didn''t take time to investigate and didn''t remember how much memory was > > used and if the system was swaping or not. > > I just have done a test now with my current xen putting 32768 Mo on xen > > command line (not the kernel line in grub). > > And the system booted a bit slower and i suspect that durring the launch of > > xend it has done some swap. > > free return this just after the boot : > > mat@zeus:/$ free > > total used free shared buffers cached > > Mem: 32768 28572 4196 0 0 7172 > > -/+ buffers/cache: 21400 11368 > > Swap: 506008 312 505696 > > > > Having this process: > > 207 ? Ss 0:00 dhclient -e -pf /var/run/dhclient.eth0.pid -lf > > /var/ > > 253 ? Ss 0:00 /sbin/syslogd > > 256 ? Ss 0:00 /sbin/klogd > > 261 ? Ss 0:00 /usr/sbin/inetd > > 268 ? Ss 0:00 /usr/sbin/sshd > > 274 ? S 0:00 xfrd > > 275 ? S 0:00 python /usr/sbin/xend start > > 288 ? Ss 0:00 /usr/sbin/cron > > 327 tty1 Ss+ 0:00 /sbin/getty 38400 tty1 > > 328 tty2 Ss+ 0:00 /sbin/getty 38400 tty2 > > 329 tty3 Ss+ 0:00 /sbin/getty 38400 tty3 > > 330 tty4 Ss+ 0:00 /sbin/getty 38400 tty4 > > 331 tty5 Ss+ 0:00 /sbin/getty 38400 tty5 > > 332 tty6 Ss+ 0:00 /sbin/getty 38400 tty6 > > 333 ? Ss 0:00 sshd: mat [priv] > > 335 ? R 0:00 sshd: mat@pts/0 > > 336 pts/0 Ss 0:00 -bash > > 344 pts/0 R+ 0:00 ps ax > > > > i guess i could reduce the number of getty and get rid of dhclient (the > > adress of dom0 is a static one ...). > > > > It was my 2cents thoughts ... > > Matthieu > > > > > > ------------------------------------------------------- > > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > > Use IT products in your business? Tell us what you think of them. Give us > > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more > > http://productguide.itmanagersjournal.com/guidepromo.tmpl > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/xen-devel > > > ------------------------------------------------------- > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > Use IT products in your business? Tell us what you think of them. Give us > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more > http://productguide.itmanagersjournal.com/guidepromo.tmpl > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark A. Williamson
2004-Sep-30 20:24 UTC
Re: [Xen-devel] RAM allocation and miscellaneous questions
Domains can''t allocate more than you set using the ''xm maxmem'' command. By default AFAIK they can only shrink themselves. I''m not sure this has been mentioned particularly explicitly alhtough the area got discussed a while back. Cheers, Mark On Thursday 30 September 2004 20:21, Brian Wolfe wrote:> Question from a security standpoint... > > How exactly does this proc control relate to the memory parameter > specified when you star tthe domain? Can an unprivileged domain increase > it''s memory footprint to MORE than the originally specified mem_size? Or > is this just a local control of up to the originally specified mem_size? > > If it''s arbitrary (not constrained by mem_size) then I''d have to point > out that some of us do NOT want this control to be allowed to go beyond > what mem_size originally sets the domain at in unprivileged domains. > Privileged domains would be exempted from this max limit of course. ;) > > If i''m just not paying attention and this was already explained, feel > free to smack me. :) > > On Thu, 2004-09-30 at 08:26, Mark A. Williamson wrote: > > > For people that still use linux 2.4 (like me) is there a tutorial or > > > some documentation about the ballon driver ? > > > > Documentation, 2.6 support and facilities to control other domain''s > > balloons using xm in domain 0 are cooking and should be ready shortly. > > > > In the meantime, do: > > # echo target_megabytes > /proc/xen/memory_target > > To set a new memory footprint. There''s a Linux kernel param (mem=..., I > > think) for setting the maximum amount of memory a domain can handle, if > > larger than it''s initial allocation (i.e. if you want to start with x > > megabytes but grow it later). > > > > Trying to grow the domain''s memory beyond what it''s expecting (the > > initial allocation or the kernel param if you specified that) will not > > work. > > > > Use: > > # cat /proc/xen/memory_target > > To get the current footprint. > > > > HTH, > > Mark > > > > > > > I have been wondering why my Domain-0 machine hasn''t been using > > > > > swap! > > > > > > > > > > :o) So I should just dump that partition? > > > > > > > > If you''ve configured swap, domain 0 should be using just like any > > > > normal linux kernel. The balloon driver works in dom0 just like > > > > any other domain. > > > > > > It seems logical, but due to the fact that only a few process are > > > running ind dom0, swap file don''t need to be huge is it ? > > > > > > > > I suppose I''ll have to be very rigorous when > > > > > it comes to keeping that domain lean yes? (Especially as I only > > > > > have 4GB to play with!). Out side of basic OS functionality, are > > > > > there any Xen or LVM2 related processes I have to account for when > > > > > setting the amount of RAM I can use for Domain-0? > > > > > > > > -xen linux kernels don''t have any special processes that take any > > > > noticeable extra resources. > > > > > > > > In domain 0 you''ll have to worry about the footprint of xend, but > > > > it''s not that big. > > > > > > When starting with xen one or two month ago, i''ve created a dom0 with > > > only 32 Mo of ram, and when i launched xend, it was so slow that i > > > tought i had a problem. > > > I didn''t take time to investigate and didn''t remember how much memory > > > was used and if the system was swaping or not. > > > I just have done a test now with my current xen putting 32768 Mo on xen > > > command line (not the kernel line in grub). > > > And the system booted a bit slower and i suspect that durring the > > > launch of xend it has done some swap. > > > free return this just after the boot : > > > mat@zeus:/$ free > > > total used free shared buffers > > > cached Mem: 32768 28572 4196 0 0 > > > 7172 -/+ buffers/cache: 21400 11368 > > > Swap: 506008 312 505696 > > > > > > Having this process: > > > 207 ? Ss 0:00 dhclient -e -pf /var/run/dhclient.eth0.pid > > > -lf /var/ > > > 253 ? Ss 0:00 /sbin/syslogd > > > 256 ? Ss 0:00 /sbin/klogd > > > 261 ? Ss 0:00 /usr/sbin/inetd > > > 268 ? Ss 0:00 /usr/sbin/sshd > > > 274 ? S 0:00 xfrd > > > 275 ? S 0:00 python /usr/sbin/xend start > > > 288 ? Ss 0:00 /usr/sbin/cron > > > 327 tty1 Ss+ 0:00 /sbin/getty 38400 tty1 > > > 328 tty2 Ss+ 0:00 /sbin/getty 38400 tty2 > > > 329 tty3 Ss+ 0:00 /sbin/getty 38400 tty3 > > > 330 tty4 Ss+ 0:00 /sbin/getty 38400 tty4 > > > 331 tty5 Ss+ 0:00 /sbin/getty 38400 tty5 > > > 332 tty6 Ss+ 0:00 /sbin/getty 38400 tty6 > > > 333 ? Ss 0:00 sshd: mat [priv] > > > 335 ? R 0:00 sshd: mat@pts/0 > > > 336 pts/0 Ss 0:00 -bash > > > 344 pts/0 R+ 0:00 ps ax > > > > > > i guess i could reduce the number of getty and get rid of dhclient (the > > > adress of dom0 is a static one ...). > > > > > > It was my 2cents thoughts ... > > > Matthieu > > > > > > > > > ------------------------------------------------------- > > > This SF.net email is sponsored by: IT Product Guide on > > > ITManagersJournal Use IT products in your business? Tell us what you > > > think of them. Give us Your Opinions, Get Free ThinkGeek Gift > > > Certificates! Click to find out more > > > http://productguide.itmanagersjournal.com/guidepromo.tmpl > > > _______________________________________________ > > > Xen-devel mailing list > > > Xen-devel@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/xen-devel > > > > ------------------------------------------------------- > > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > > Use IT products in your business? Tell us what you think of them. Give us > > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out > > more http://productguide.itmanagersjournal.com/guidepromo.tmpl > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mike Brady
2004-Oct-01 21:20 UTC
Re: [Xen-devel] RAM allocation and miscellaneous questions
The parts are readily available in NZ and I would be surprised if you couldn''t find a local supplier in Wellington to put a system together for you. You would be getting support for the parts rather than support for the system in this case though. Which is what I assume the issue would be. On Thursday 30 September 2004 21:02, Keir Fraser wrote:> How odd -- I expected that in most places there would be multiple > white-box vendors selling Opteron-based systems. It''s not like the > component parts are in short supply. Except maybe in NZ. :-) > > -- Keir > > > Hmmmmmm. Lots of RAM :o) > > > > Yes, I do have a ''cluster'', but only two machines so far. The problem for > > me is that multi-processor Opteron 1u machines (or 2u or 4u for that > > matter) are hard to come by here in New Zealand, unless we pay through > > the nose for an HP or other big-name machine. I can get dual Xeon > > machines pretty cheap though. If I look at the Opteron option, I''ll have > > to compare it to just buying more Xeons. Either way, it''s going to cost > > more for our initial cluster than it would if I could just use 12GB in a > > cheap Xeon unit. Don''t get me wrong though, if Opteron units were > > available here I would have got them from the outset. > > > > Personally I have no issue with sourcing machines from Australia or > > somewhere, but my bosses will be reluctant due to support issues, etc. > > > > Regards, > > Paul > > ------------------------------------------------------- > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > Use IT products in your business? Tell us what you think of them. Give us > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more > http://productguide.itmanagersjournal.com/guidepromo.tmpl > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel