Hello, I have just started to try Xen (running the live CD) and so I am fairly new to this great solution... I can actually create virtual machines (domains) without problems, it''s working great, and I can migrate domains between physical computers... Everything runs smoothly, so I guess this is a great solution :) Now I have a few questions due to my inexperience with the product : -> How is it possible to load balance virtual machines between physical hosts ? And more generally is it possible to interconnect/autodiscover a network of physical hosts (like OpenMosix for example) ? On the RedHat flash on virtualization, it seems possible... -> How is it possible to manage the fail over ? (like physical host A is down, let''s migrate its virtual machines to physical host B for example) -> Is it possible to "burst" RAM usage ? Here''s my more general problem, I would like to host game servers (Quake, Counter Strike, etc...) applications.... The common problem with hosting those servers, is that most of the time they are not used, so basically you have to load balance the servers by hand (like putting 8 game servers per machine and that''s all)... so you have a lot of manual work to do... and you can''t really balance correctly the load amongst the server (eg : a machine can have 7 servers out of 8 who are running with players, another one totally free, etc...).... That''s why I was looking at cluster solutions.... I first started with Single System Image clusters like OpenMosix and OpenSSI, but I don''t really think the way socket migration is handled is correct for such I/O and Network intensive applications... maybe I am wrong.... but from what I''ve read the socket has to stay on the first machine..... so it could lead to a saturation of the NIC. That''s why I ended with Xen, I could basically set up one server per virtual machine, and migrate the virtual machine amongst the physical servers to load balance, the only problem is that I can''t predict how much RAM a virtual machine could take, if there is 0 players the process is quite lite, is there are tons of players it can be quite heavy.... And setting 256M of RAM per virtual machine just to be sure, would be a total waste of memory.... If you have any ideas that would help :), Thanks a lot, Ugo PARSI _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi> And setting 256M of RAM per virtual machine just to be sure, > would be a total waste of memory....You need to give every domU a certain amount of memory on startup. As far as I know, you cannot change this later. So you NEED to give every domain as much memory as it will ever need. Regrads, Steffen _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi> > You need to give every domU a certain amount of memory on startup. > > As far as I know, you cannot change this later. > > > > So you NEED to give every domain as much memory as it will > > ever need. > > Yeah that sucks (for my use) :( > > No way of doing a no limit thing in order to share the RAM ? > Like each domU can take as many RAM as it wants and so it > would automagically share the RAM load ?I am also somehow new to xen, so somebody correct me if I am wrong. But as fas as I understand: Every domU has it''s own kernel which manages it''s own memory. So every domU also has it''s own drive cache. Most OSs tend to simly use all free memory for drive caches. So does Windows and so does linux. This makes sence for real hardware, since unused free memory is a waste of resources and using every available memory as cache speeds up the mashine extremely. However, if there are other systems that share the same memory, It might be better for one system to have less cache memory and therefor preventing another system from swaping. But to take control of this, every system would need to know about the memory usage of the other systems. That would be totally against system separation. Regards, Steffen _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Tuesday 21 March 2006 7:49 am, Steffen Heil wrote:> Hi > > > > You need to give every domU a certain amount of memory on startup. > > > As far as I know, you cannot change this later. > > > > > > So you NEED to give every domain as much memory as it will > > > ever need. > > > > Yeah that sucks (for my use) :( > > > > No way of doing a no limit thing in order to share the RAM ? > > Like each domU can take as many RAM as it wants and so it > > would automagically share the RAM load ? > > I am also somehow new to xen, so somebody correct me if I am wrong. > But as fas as I understand: > > Every domU has it''s own kernel which manages it''s own memory. > So every domU also has it''s own drive cache. > > Most OSs tend to simly use all free memory for drive caches. So does > Windows and so does linux. This makes sence for real hardware, since unused > free memory is a waste of resources and using every available memory as > cache speeds up the mashine extremely. However, if there are other systems > that share the same memory, It might be better for one system to have less > cache memory and therefor preventing another system from swaping. But to > take control of this, every system would need to know about the memory > usage of the other systems. > That would be totally against system separation.that''s all correct, but there''s a nice hack to get that RAM back: the balloon driver. it''s a kernel module that just sits there in a domU taking up a variable amount of RAM, releasing or reclaiming it as commanded by messages from dom0. so, you would start up many domUs giving each a big RAM allocation, but the balloon driver eats most of it. when a given domU needs more RAM, the Xen hypervisor maps some physical pages to that domU, and the balloon driver shrinks by the same amount, making the RAM usable by the rest of the kernel. -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
>-> How is it possible to load balance virtual machines between >physical hosts ? And more generally is it possible to >interconnect/autodiscover a network of physical hosts (like OpenMosix >for example) ? >On the RedHat flash on virtualization, it seems possible...Currently you need to do this manually. More powerful automatic load balancers are likely to appear on the market at some point, as an additional management product. It should be possible to script some sort of basic management application that would suit your needs, however.>-> How is it possible to manage the fail over ? (like physical host A >is down, let''s migrate its virtual machines to physical host B for >example)Two aspects: for a planned shutdown you''d want the management software to live migrate domains off a server you were planning to take down. You''d also like it to detect a crash, and reboot the domains on other hosts (and lock out the downed server from shared storage so that if it comes back on the network any remaining domain images it''s running can''t confuse the situation)>-> Is it possible to "burst" RAM usage ?You can change the RAM footprint using xm mem-set (or something like that). Again, this is manual but it''s possible in principle to script a tool that will automatically allocate memory to a domain which is swapping more, and revoke memory from domains with low swap activity. Again, this is a management plane function that isn''t provided by the current set of tools, but it is possible to write in principle. HTH, Mark>Here''s my more general problem, I would like to host game servers >(Quake, Counter Strike, etc...) applications.... >The common problem with hosting those servers, is that most of the >time they are not used, so basically you have to load balance the >servers by hand (like putting 8 game servers per machine and that''s >all)... so you have a lot of manual work to do... and you can''t really >balance correctly the load amongst the server (eg : a machine can have >7 servers out of 8 who are running with players, another one totally >free, etc...).... > >That''s why I was looking at cluster solutions.... > >I first started with Single System Image clusters like OpenMosix and >OpenSSI, but I don''t really think the way socket migration is handled >is correct for such I/O and Network intensive applications... maybe I >am wrong.... but from what I''ve read the socket has to stay on the >first machine..... so it could lead to a saturation of the NIC. > >That''s why I ended with Xen, I could basically set up one server per >virtual machine, and migrate the virtual machine amongst the physical >servers to load balance, the only problem is that I can''t predict how >much RAM a virtual machine could take, if there is 0 players the >process is quite lite, is there are tons of players it can be quite >heavy.... >And setting 256M of RAM per virtual machine just to be sure, would be >a total waste of memory.... > >If you have any ideas that would help :), > >Thanks a lot, > >Ugo PARSI > >_______________________________________________ >Xen-users mailing list >Xen-users@lists.xensource.com >http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
>> And setting 256M of RAM per virtual machine just to be sure, >> would be a total waste of memory.... > >You need to give every domU a certain amount of memory on startup. >As far as I know, you cannot change this later.You can use the balloon driver - xm mem-set (or something like that) It allows you to grow / shrink domains on request. If you want to grow beyond the initial allocation, I think you still need to pass an extra argument to the guest kernel at boot time telling it how large it might grow (otherwise it doesn''t make its memory management structures big enough), but that''s it. Cheers, mark>So you NEED to give every domain as much memory as it will ever need. > >Regrads, > Steffen >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
>I am also somehow new to xen, so somebody correct me if I am wrong. >But as fas as I understand: > >Every domU has it''s own kernel which manages it''s own memory. >So every domU also has it''s own drive cache.Yep.> Most OSs tend to simly use all free memory for drive caches. So does > Windows and so does linux. This makes sence for real hardware, since > unused free memory is a waste of resources and using every available > memory as cache speeds up the mashine extremely. However, if there are > other systems that share the same memory, It might be better for one > system to have less cache memory and therefor preventing another system > from swaping. But to take control of this, every system would need to > know about the memory usage of the other systems. That would be totally > against system separation.It depends. If you''re (for instance) a hosting company, you might want to prevent domains from impacting each other completely. If you run your own services in multiple domains, you might want to allow them to grow / shrink according to demand, but with limits on how far one domain could grow at the expense of others. This is (more or less) something that can be solved in the management tools - each domain would report its current load statistics and the management tools would reallocate memory between them accordingly. You might want an administrator alarm bell so that potential resource hogs can be checked out and killed by the sysadmin ;-) Another thing you''d want to do is share the IO caches between domains - if they have a common basic root filesystem, this could be a big win in performance and memory utilisation. Cheers, Mark _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> Currently you need to do this manually. More powerful automatic load > balancers are likely to appear on the market at some point, as an > additional management product.Ok, so you''re saying there''s currently no solution at all ? Any opensource project (even beta) that you are aware of ? Basically, I will need this feature as part of my whole project....so I''m planning to code my own Xen load balancer... but I would prefer to help an existing project rather than starting from scratch my own solution....> > Two aspects: for a planned shutdown you''d want the management software to > live migrate domains off a server you were planning to take down. You''d > also like it to detect a crash, and reboot the domains on other hosts (and > lock out the downed server from shared storage so that if it comes back on > the network any remaining domain images it''s running can''t confuse the > situation) >Ok, actually, if you can load balance, it''s pretty easy to detect that a machine is not responding and correctly manage the failure by sharing the remaining load amongst machines that are alive. So my question was stupid, sorry :)> > You can change the RAM footprint using xm mem-set (or something like that). > Again, this is manual but it''s possible in principle to script a tool that > will automatically allocate memory to a domain which is swapping more, and > revoke memory from domains with low swap activity. >Yeah, I have tried it and it works pretty well.... :) It''s pretty easy to script such a thing when you plan the RAM modification. Like getting the virtual machine to 128 Megs -> 256 Megs... But for programs that are quite "memory volatile" it''s rather hard... for example a game server that is growing from 30 Megs to 80 Megs and that can''t stop doing that depending on the number of users.... To add some complexity my main problem is to detect how much RAM is really needed by the virtual machine, because of the way Linux is often caching objects in memory, I''m getting a little bit confused... Any idea on getting all these informations from the Dom0 ? Thanks a lot, Ugo PARSI _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> > Currently you need to do this manually. More powerful automatic load > > balancers are likely to appear on the market at some point, as an > > additional management product. > > Ok, so you''re saying there''s currently no solution at all ? > Any opensource project (even beta) that you are aware of ?http://www.enomalism.com - an opensource Xen management tool. Not released yet, not clear if it does load balancing anyhow. http://www.xensource.com - to be releasing a product called "Optimizer", will include management functions. I don''t know about load balancing for this either.> Basically, I will need this feature as part of my whole project....so > I''m planning to code my own Xen load balancer... but I would prefer to > help an existing project rather than starting from scratch my own > solution....Indeed.> > Two aspects: for a planned shutdown you''d want the management software to > > live migrate domains off a server you were planning to take down. You''d > > also like it to detect a crash, and reboot the domains on other hosts > > (and lock out the downed server from shared storage so that if it comes > > back on the network any remaining domain images it''s running can''t > > confuse the situation) > > Ok, actually, if you can load balance, it''s pretty easy to detect that > a machine is not responding and correctly manage the failure by > sharing the remaining load amongst machines that are alive. > So my question was stupid, sorry :)Well, if a machine has just been partitioned from the network (and not actually brought down) you have to make sure the domains on it *really* die, else they''ll corrupt storage if that machine comes back after the domains have been evacuated.> It''s pretty easy to script such a thing when you plan the RAM modification. > Like getting the virtual machine to 128 Megs -> 256 Megs... > > But for programs that are quite "memory volatile" it''s rather hard... > for example a game server that is growing from 30 Megs to 80 Megs and > that can''t stop doing that depending on the number of users.... > > To add some complexity my main problem is to detect how much RAM is > really needed by the virtual machine, because of the way Linux is > often caching objects in memory, I''m getting a little bit confused...You need to the other domUs to report their swap usage, or something like that. vmstat ? Then balloon based on that (or on their non-cache memory usage, as reported by "free", or whatever). A network monitoring tool would also work. Cheers, Mark -- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users