Kip Macy wrote:> At the risk of starting the construction of a bikeshed, what kind of > hardware are you running on? The reason for the increased appeal of > virtual machines over the past few years is that hw performance and > memory availability has increased faster than applications'' needs.I am running on 2.4GHz Dell Optiplexes with 512MB of memory and no swap configured for dom0. Currently I have 16 of them, but in the long run I am hoping to run this on 1000+ nodes with similar config. That means any memory I waste I can multiply with an arbitrarily large number to make it sound scary. My approach has (to me) many other nice properties, apart from the memory footprint, some of the more important ones being security (much reduced trusted computing base) and performance+isolation (not using shadow page tables, not relying on an external checkpointing service in dom0, being able to delay scheduling badly behaving processes during migration, knowing who is in my ARP cache, etc.), as well as overall better alignment with Saltzer''s end-to-end argument. I would like to create a platform where anyone can purchase resources, without any kind of trust that customer are playing nice, and I think that I am on the right path. Before Xen came along, I created a similar platform (called NomadBIOS), based on L4, sporting what I now call hosted or managed migration. The system I am building now is my ''second system'', for better or worse ;-) I probably spent more time thinking about these issues than most people (though that is no guarantee that my conclusions are correct). > -Kip Jacob ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Will Xen run if compiled against uClibc? What about with the busybox toolset? I am looking to create a minimal dom0 image that simply controls the creation and loading of new domains, and that allows migration to occur. -- Jerome Brown Technology Integrator Hub.Net P: +64 3 961-5116 M: +64 29 453 7663 F: +64 3 961-5129 E: jerome@concepts.net.nz W: http://www.concepts.net.nz/ O: Level 4, 818 Colombo St, Christchurch, New Zealand S: PO Box 1879, Christchurch, New Zealand ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Jerome Brown wrote:> Will Xen run if compiled against uClibc? What about with the busybox > toolset?I don''t think Xen itself is compiled against any libc. I have a busybox-based (though using glibc right now) initrd which you can use for your unprivileged domains. You can download it from http://www.diku.dk/~jacobg/self-migration/> I am looking to create a minimal dom0 image that simply controls the > creation and loading of new domains, and that allows migration to occur.I am working on exactly the same thing, though I am using self-migration in XenLinux rather than the managed migration in Xen. I am already able to migrate to a minimal Xen-host (the network-facing privileged code is about 50 lines of C, there is not even a TCP/IP stack running), and I am currenly working on being able to create new domains using the same mechanism (almost there). I am currently on Xen 1.3, because the current Xend is too resource-hungry and complex for my needs. I am crossing my fingers for someone to implement the basics of Xend in C, and when that happens I plan to move to Xen 2.0. Input and cooperation is welcomed. Jacob ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Jacob Gorm Hansen wrote:> I don''t think Xen itself is compiled against any libc. I have a > busybox-based (though using glibc right now) initrd which you can use > for your unprivileged domains. You can download it from > http://www.diku.dk/~jacobg/self-migration/What size does this run at? I am looking to make dom0 as small as possible, and run all the applications that I require under other domains, to allow moving them to other machines if required.> I am working on exactly the same thing, though I am using self-migration > in XenLinux rather than the managed migration in Xen. I am already able > to migrate to a minimal Xen-host (the network-facing privileged code is > about 50 lines of C, there is not even a TCP/IP stack running), and I am > currenly working on being able to create new domains using the same > mechanism (almost there).How does the XenLinux self-migration differ from the Xen managed migration? What are the advantages of each option. I have only just started to look into this, so am not stuck to a specific way of doing things at this stage :)> I am currently on Xen 1.3, because the current Xend is too > resource-hungry and complex for my needs. I am crossing my fingers for > someone to implement the basics of Xend in C, and when that happens I > plan to move to Xen 2.0.How do the resources used differ? Is Xend currently in python? Is there a plan to change this? What has been the reasoning behind developing it in python?> Input and cooperation is welcomed.I''d love to help. I like the idea of a ''self installing'' dom0 that sets up another control domain, thereby abstracting all the running processes into unprivileged domains for security reasons. As part of this I will look to create a HowTo document that details how I acheive this :) Regards Jerome ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Jerome Brown wrote:> What size does this run at? I am looking to make dom0 as small as > possible, and run all the applications that I require under other > domains, to allow moving them to other machines if required.I would image this to be the sum of Xen, a XenLinux with the TCP-stack removed, and a few hundred kilobytes of initrd with my tools.> How does the XenLinux self-migration differ from the Xen managed > migration? What are the advantages of each option. I have only just > started to look into this, so am not stuck to a specific way of doing > things at this stage :)Self-migration means that the unprivileged domain uses its own resources (pager and TCP stack are the main ones), to checkpoint itself onto another host. There are pros and cons to both approaches, I refer you to my SIGOPS European Workshop paper, available from http://www.diku.dk/~jacobg/ for more details.> How do the resources used differ? Is Xend currently in python? Is there > a plan to change this? What has been the reasoning behind developing it > in python?I am not sure what the exact reasoning behind this decision was, but I am sure other people on this list would like to comment. I rewrote all my own tools in C a while back, something I do not regret at this point. From what I hear dom0 with Xend needs 32-64 megs to run. In my setup 4 megs should be a realistic target.> I''d love to help. I like the idea of a ''self installing'' dom0 that sets > up another control domain, thereby abstracting all the running processes > into unprivileged domains for security reasons. As part of this I will > look to create a HowTo document that details how I acheive this :)See if you can get any of the stuff on my page to run (both sources and binaries are provided), and feel free to ask questions if you can''t (though I am on my way home fairly soon). Best regards, Jacob ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> I am not sure what the exact reasoning behind this decision was, but I > am sure other people on this list would like to comment. I rewrote all > my own tools in C a while back, something I do not regret at this point. > > From what I hear dom0 with Xend needs 32-64 megs to run. In my setup 4 > megs should be a realistic target.At the risk of starting the construction of a bikeshed, what kind of hardware are you running on? The reason for the increased appeal of virtual machines over the past few years is that hw performance and memory availability has increased faster than applications'' needs. -Kip ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Kip Macy wrote:> At the risk of starting the construction of a bikeshed, what kind of > hardware are you running on? The reason for the increased appeal of > virtual machines over the past few years is that hw performance and > memory availability has increased faster than applications'' needs.I am running on 2.4GHz Dell Optiplexes with 512MB of memory and no swap configured for dom0. Currently I have 16 of them, but in the long run I am hoping to run this on 1000+ nodes with similar config. That means any memory I waste I can multiply with an arbitrarily large number to make it sound scary. My approach has (to me) many other nice properties, apart from the memory footprint, some of the more important ones being security (much reduced trusted computing base) and performance+isolation (not using shadow page tables, not relying on an external checkpointing service in dom0, being able to delay scheduling badly behaving processes during migration, knowing who is in my ARP cache, etc.), as well as overall better alignment with Saltzer''s end-to-end argument. I would like to create a platform where anyone can purchase resources, without any kind of trust that customer are playing nice, and I think that I am on the right path. Before Xen came along, I created a similar platform (called NomadBIOS), based on L4, sporting what I now call hosted or managed migration. The system I am building now is my ''second system'', for better or worse ;-) I probably spent more time thinking about these issues than most people (though that is no guarantee that my conclusions are correct).> -KipJacob ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> How do the resources used differ?I can''t put exact numbers on it but using Python will certainly result in a larger memory footprint being necessary for domain 0. Python''s not an enormously fast language but that doesn''t matter too much for control plane functionality (performance critical stuff is coded in C already).> Is Xend currently in python?The control tools and Xend are written in Python. The migration / suspend daemon (xfrd) is also coded in C for performance reasons.> Is there > a plan to change this? What has been the reasoning behind developing it > in python?The use of Python in Xend enables rapid development - Xend does _a lot_ of different tasks, so it''s good to have as much high-level goodness on your side as possible. I don''t think we''re likely to change language in the near future. Compared to the amount of memory spent on other domains, Xend''s footprint is arguably still likely to be pretty small, although obviously it depends somewhat on the application. The lowlevel functions for interfacing to Xen are available in a separate C library so it''s possible to hook arbitrary languages into Xen. It''s certainly possible for Xend to be rewritten in C but it would be a large job to provide equivalent functionality. An alternative control daemon called "x2d2" is in the unstable tree (what about the testing tree? not sure...) but this is intended for developer experimentation and is not a full-blown Xend replacement. Cheers, Mark> > > Input and cooperation is welcomed. > > I''d love to help. I like the idea of a ''self installing'' dom0 that sets > up another control domain, thereby abstracting all the running processes > into unprivileged domains for security reasons. As part of this I will > look to create a HowTo document that details how I acheive this :) > > Regards > > Jerome > > > ------------------------------------------------------- > SF email is sponsored by - The IT Product Guide > Read honest & candid reviews on hundreds of IT Products from real users. > Discover which products truly live up to the hype. Start reading now. > http://productguide.itmanagersjournal.com/ > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel