In Re: [Xen-devel] Network issues with SuSE firewall, On Sat, Nov 08, 2003 at 03:13:57AM +0000, Ian Pratt wrote: ... lots of helpful stuff omitted ...> Let''s get a domain running off ramdisk/CD then we''ll walk you > through the next bit... > > IanThanks, Ian. I''m grateful that running off of CD now works. I''m ready to move towards a workable/usable system, and will describe the outcome I''m thinking of. From what I''ve read in README.CD, this is clearly possible but there are questions left unanswered. Questions: 1) For IP addresses, do they need to be sequential? I have 9 sequential ones, but the first (DOM0) is not. I can renumber if needed, but it looks like Xen just have separate startup scripts with hard-coded IP addresses for each non-DOM0 domain. 2) Can my current /usr (and the rest of the distro) co-exist with Xen, or will I be unable to boot a "regular" Linux after all this? Setup: - I want to run 9+1 domains on one system (9 virtual plus the 0th domain). They''ll be IP-based. - I''d like them to share as much of the core OS as is reasonable: /, /boot, /etc, /usr, /usr/local etc. This can be by NFS, or (better) mounting ro. This is for convenience in administration. - They must have one or more shared data directories (/globus and /shared), again via NFS or ro. (r+w is not needed). This is for getting applications running. - They''ll each have their own /data partition, r+w, which will be a physically separate drive. /dev/sdk1, /dev/sdl1, /dev/sdm1, etc. - I''m not sure how to handle /home. It would be nice for a user to get his *same* home directory r+w on each system. NFS r+w? So, things I need to do are: 1) Get a correct /usr etc. on hard drive, so I can boot virtual domains that will run off of hard drive. (Currently, they''re running in RAMfs via CD.) 2) Give each virtual domain a different /etc/fstab or other configuration so that they mount the right drives ro and rw and NFS. 3) Code a xenctl startup script for each IP-based domain to insure each one is booted properly. For example, "grid-11" should always go to the same IP address, not a different one if, for example, one of the other domains fails to start or is accidentally started twice. For illustration purposes, grid-11.arsc.edu = 137.229.71.11. The main system, peabody.arsc.edu = 137.229.71.6 As you see, the main parts I don''t quite see yet are the interaction between the system filesystems (/usr etc.), and the mechanics of startup for a hard-drive based system. I promise to write up all these steps, for others to use, once I get things running. Thanks again for your help, this is truly going to be extremely useful software for my Grid applications development. -- Greg ------------------------------------------------------- This SF.Net email sponsored by: ApacheCon 2003, 16-19 November in Las Vegas. Learn firsthand the latest developments in Apache, PHP, Perl, XML, Java, MySQL, WebDAV, and more! http://www.apachecon.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On 10 Nov 2003, at 23:30, Gregory Newby wrote:> 3) Code a xenctl startup script for each IP-based domain to insure > each one is booted properly. For example, "grid-11" should always go > to the same IP address, not a different one if, for example, one of > the other domains fails to start or is accidentally started twice. > > For illustration purposes, grid-11.arsc.edu = 137.229.71.11. > The main system, peabody.arsc.edu = 137.229.71.6In /etc/xenctl.xml, if you write <nw_ip>137.229.71.0+</nw_ip> <nw_gw>137.229.71.6</nw_gw> <nw_mask>255.255.0.0</nw_mask> would automatically assign IP addresses to newly created domains based on their domain ID. e.g. domain N has IP address 137.229.71.N. However, your Domain 0 IP address 137.229.71.6 conflicts with the automatic assigned IP address for domain 6. You have to work around that. In a typical ''/etc/xen-mynewdom'', you have: domain new domain start A useful feature of this script is that: each time you run ''xenctl script -f/etc/xen-mynewdom'', it creates and starts a new domain with an ''always-increasing'' domain id, i.e. you run ''xenctl'' for the Nth time, it creates a domain with domain id ''N'', even if some previous domains (with domain id < N) have been destroyed. Hope this information helps. Bin ------------------------------------------------------- This SF.Net email sponsored by: ApacheCon 2003, 16-19 November in Las Vegas. Learn firsthand the latest developments in Apache, PHP, Perl, XML, Java, MySQL, WebDAV, and more! http://www.apachecon.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Tvrtko A. Uršulin
2003-Nov-11 08:47 UTC
Re: [Xen-devel] Setting up a more complex system scenario
> A useful feature of this script is that: each time you run > ''xenctl script -f/etc/xen-mynewdom'', it creates and starts > a new domain with an ''always-increasing'' domain id, i.e. > you run ''xenctl'' for the Nth time, it creates a domain with > domain id ''N'', even if some previous domains (with domain > id < N) have been destroyed.How to reset that counter? -- Croadria Internet usluge <http://www.croadria.com> - Web hosting (Linux & Windows), E-commerce Podrska: <http://podrska.croadria.com> Tel: 01/4657 818, 01/4655 601 Fax: 01/4657 819 ------------------------------------------------------- This SF.Net email sponsored by: ApacheCon 2003, 16-19 November in Las Vegas. Learn firsthand the latest developments in Apache, PHP, Perl, XML, Java, MySQL, WebDAV, and more! http://www.apachecon.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Steven Hand
2003-Nov-11 08:55 UTC
Re: [Xen-devel] Setting up a more complex system scenario
>> A useful feature of this script is that: each time you run >> ''xenctl script -f/etc/xen-mynewdom'', it creates and starts >> a new domain with an ''always-increasing'' domain id, i.e. >> you run ''xenctl'' for the Nth time, it creates a domain with >> domain id ''N'', even if some previous domains (with domain >> id < N) have been destroyed. > >How to reset that counter?Currently there is no way to do so (apart from hacking the xen code :-) It''s possible that we''ll modify this in the future, but for the near future it''s not likely. A better solution is to modify the user-space tools (xenctl etc) to handle allocation of resources in a less fragile way; this may happen soonish as we''re in the process of designing a new set of user-space tools with a better notion of allocation, etc. cheers, S.
> > > A useful feature of this script is that: each time you run > > ''xenctl script -f/etc/xen-mynewdom'', it creates and starts > > a new domain with an ''always-increasing'' domain id, i.e. > > you run ''xenctl'' for the Nth time, it creates a domain with > > domain id ''N'', even if some previous domains (with domain > > id < N) have been destroyed. > > How to reset that counter?You can control what value xenctl adds to any fields with a ''+'' in them using the "-xN" option. Ian ------------------------------------------------------- This SF.Net email sponsored by: ApacheCon 2003, 16-19 November in Las Vegas. Learn firsthand the latest developments in Apache, PHP, Perl, XML, Java, MySQL, WebDAV, and more! http://www.apachecon.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On 10 Nov 2003, at 23:30, Gregory Newby wrote:> - I''d like them to share as much of the core OS as is reasonable: /, > /boot, /etc, /usr, /usr/local etc. This can be by NFS, or (better) > mounting ro. This is for convenience in administration. > > - They must have one or more shared data directories (/globus and > /shared), again via NFS or ro. (r+w is not needed). This is > for getting applications running. > > - They''ll each have their own /data partition, r+w, which will be a > physically separate drive. /dev/sdk1, /dev/sdl1, /dev/sdm1, etc. > > - I''m not sure how to handle /home. It would be nice for a user to > get his *same* home directory r+w on each system. NFS r+w? > > So, things I need to do are: > > 1) Get a correct /usr etc. on hard drive, so I can boot virtual > domains that will run off of hard drive. (Currently, they''re > running in RAMfs via CD.) > > 2) Give each virtual domain a different /etc/fstab or other > configuration so that they mount the right drives ro and rw and NFS.To summarize, you want new domains to share /, /boot, /etc, /usr etc but mount different partitions under /data for ''personal'' use. The main problem is: because new domains share /etc, they will read the same /etc/fstab. How to have domain-specific /etc/fstab automatically? Fortunately, you can use ClusterNFS (http://clusternfs.sourceforge.net) ClusterNFS is a patched version of "Universal NFS Daemon" (UNFSD) server to allow multiple diskless clients to NFS mount the same root filesystem by providing "interpreted" file names. When a client requests the file "/path/filename" , the ClusterNFS server checks for the existence of files of the form "/path/filename$$KEY=value$$". If such a file exists and the clients has a matching value for KEY, this file is returned. If the client does not have the matching value or no such file exists, the file request proceeds as normal. Currently supported keys include, HOST (hostname), IP (IP number), UID (user id), GID (group id), and CLIENT (matches any NFS client). ClusterNFS does not require any kenel changes. It runs only on the server. Clients run (any) standard NFS client. As a result, you can download the latest release of ClusterNFS, compile it and use it to export a shared root partition. In this shared root partition, you have a series of /etc/fstab$$IP=137.229.71.N$$ files, each containing domain specific mount table entry such as mounting /dev/sdN1 on /data (N stands for domain ID). So when doman N (assigned IP address 137.229.71.N) accesses the root partition via ClusterNFS server and tries to read mount table from /etc/fstab, it actally reads /etc/fstab$$IP=137.229.71.N$$ and mount /dev/sdN1 on /data for personal use. It should work and I''ll try it myself later today. Bin ------------------------------------------------------- This SF.Net email sponsored by: ApacheCon 2003, 16-19 November in Las Vegas. Learn firsthand the latest developments in Apache, PHP, Perl, XML, Java, MySQL, WebDAV, and more! http://www.apachecon.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> Questions: > > 1) For IP addresses, do they need to be sequential? I have 9 > sequential ones, but the first (DOM0) is not. I can renumber if > needed, but it looks like Xen just have separate startup scripts with > hard-coded IP addresses for each non-DOM0 domain.You can set them to anything you like. Sequential is just a convenience provided by the tools.> 2) Can my current /usr (and the rest of the distro) co-exist with Xen, > or will I be unable to boot a "regular" Linux after all this?You should be able to co-exist just fine -- all of our systems boot plain Linux from time to time too. Various commands often run in the startup scripts e.g. "gpm", "hwclock" will complain, but the errors are harmless. You can suppress them by using a different init runlevel for linux and xenolinux boots (as per the demo CD).> - I''d like them to share as much of the core OS as is reasonable: /, > /boot, /etc, /usr, /usr/local etc. This can be by NFS, or (better) > mounting ro. This is for convenience in administration.I''ve setup a number of systems with a ro /usr on a raw disk partition, but with each domain having it''s own rw /. The / partition can either be on a raw partition, virtual disk, or NFS mount. If you wish to make the / partition smaller, you can copy the /bin /sbin and /lib directories into /usr/root and create soft links in /. (You need to be a little careful with this as the startup scripts need a couple of libraries before executing the mount of /usr -- see README.CD)> - They must have one or more shared data directories (/globus and > /shared), again via NFS or ro. (r+w is not needed). This is > for getting applications running.Either will work.> - They''ll each have their own /data partition, r+w, which will be a > physically separate drive. /dev/sdk1, /dev/sdl1, /dev/sdm1, etc.Virtual disks or raw partitions work well for this.> - I''m not sure how to handle /home. It would be nice for a user to > get his *same* home directory r+w on each system. NFS r+w?NFS rw works very well.> So, things I need to do are: > > 1) Get a correct /usr etc. on hard drive, so I can boot virtual > domains that will run off of hard drive. (Currently, they''re > running in RAMfs via CD.)Yep. I''d do a normal redhat install and then use "cp -a" to move the data between partitions.> 2) Give each virtual domain a different /etc/fstab or other > configuration so that they mount the right drives ro and rw and NFS.I''d give each domain it'' s own / partition. (with links for /bin /lib and /sbin).> 3) Code a xenctl startup script for each IP-based domain to insure > each one is booted properly. For example, "grid-11" should always go > to the same IP address, not a different one if, for example, one of > the other domains fails to start or is accidentally started twice. > > For illustration purposes, grid-11.arsc.edu = 137.229.71.11. > The main system, peabody.arsc.edu = 137.229.71.6This should work fine. We''re developing a copy-on-write file system that should make all of this configuration rather easier in the future...> I promise to write up all these steps, for others to use, once I get > things running. Thanks again for your help, this is truly going > to be extremely useful software for my Grid applications development.Thanks, Ian ------------------------------------------------------- This SF.Net email sponsored by: ApacheCon 2003, 16-19 November in Las Vegas. Learn firsthand the latest developments in Apache, PHP, Perl, XML, Java, MySQL, WebDAV, and more! http://www.apachecon.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Gregory Newby
2003-Nov-13 08:58 UTC
Re: [Xen-devel] Setting up a more complex system scenario
I''m looking to set up a virtual domain that boots ro from the same partitions as domain0, then I''ll add rw access to a partition for its own purposes. Later, I''ll add NFS for /home and other stuff. The easiest way, it seems, is to modify these 3 files for each domain and boot from the same / as domain0: /etc/fstab /etc/xenctl.xml /etc/xen-mynewdom (or whatever I want to call it...) I''m avoiding Bin''s excellent suggestion of ClusterNFS just to make my life easier for now....creating an alternate fstab is not risky, since the worst that will happen if the system crashes is I''ll accidentally be mounted ro rather than rw. Let''s say I want to boot a domain with hostname "grid-14" (I''d like it to be DOMID=14). It seems I''m unable to find /boot, probably because I''m not sure what the -d and boot options and others refer to (i.e., is "root directory" / or /boot or /dev/sda2 or what?). Please take a look. For now, let''s just look at / (which includes /usr and /usr/local/bin etc.) and /boot:>From /etc/fstab:/dev/sda2 / reiserfs defaults,ro 0 2 /dev/sda1 /boot ext2 defaults,ro 0 2 Here''s xen-mynewdom: domain new -hgrid-14.arsc.edu -ixenolinux.gz -dsda1 -t"ro root=/dev/sda2 DOMID=\14" physical grant -phdc -r physical grant -sda2 -r physical grant -sda1 -r domain start Here''s xenctl.xml: <?xml version="1.0"?> <domctl_defaults> <domain_name>XenoLinux</domain_name> <domain_image>xenolinux.gz</domain_image> <domain_vifs>1</domain_vifs> <root_device>/dev/sda1</root_device> <root_args>r</root_args> <nw_ip>137.229.71.14</nw_ip> <nw_gw>=</nw_gw> <nw_mask>=</nw_mask> <nw_nfs_server>169.254.1.0</nw_nfs_server> <max_domain_number>50</max_domain_number> <xi_tools_dir>/usr/local/bin/</xi_tools_dir> </domctl_defaults> I know these files could be a little different (I''ve been trying a few things, and some options can be switched between xenctl.xml and the xen-mynewdom script). When I run: Here is output from "xenctl script -f/etc/xen-mynewdom14" with the files above. I''ve tried variations on sda1 and sda2 with the same "FileNotFoundException". I think mine is a pretty simple approach, but there don''t seem to be any examples so I keep missing the syntax. (Or maybe I need different physical grant lines). Thanks for pointing the way... -- Greg Domain defaults: name XenoLinux size 0 vifs 1 domainImage xenolinux.gz domainInitRD null rootDevice /dev/sda1 rootArgs r usrDevice null NWIP 137.229.71.14 NWGW 137.229.71.1 NWMask 255.255.255.240 MaxDomainNumber 50 NWNFSServer 169.254.1.0 NWNFSRoot null XIToolsDir /usr/local/bin/ args java.io.FileNotFoundException: xenolinux.gz (No such file or directory) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.<init>(FileInputStream.java:103) at java.io.FileInputStream.<init>(FileInputStream.java:66) at org.xenoserver.control.CommandDomainNew.getUncompressed(CommandDomainNew.java:402) at org.xenoserver.control.CommandDomainNew.execute(CommandDomainNew.java:172) at org.xenoserver.cmdline.ParseDomainNew.parse(ParseDomainNew.java:53) at org.xenoserver.cmdline.ParseGroup.parse(ParseGroup.java:44) at org.xenoserver.cmdline.ParseGroup.parse(ParseGroup.java:44) at org.xenoserver.cmdline.Main.executeArgList(Main.java:59) at org.xenoserver.cmdline.ParseScript.parse(ParseScript.java:49) at org.xenoserver.cmdline.ParseGroup.parse(ParseGroup.java:44) at org.xenoserver.cmdline.Main.executeArgList(Main.java:59) at org.xenoserver.cmdline.Main.main(Main.java:91) Could not create new domain (java.io.FileNotFoundException: xenolinux.gz (No such file or directory)) ------------------------------------------------------- This SF.Net email sponsored by: ApacheCon 2003, 16-19 November in Las Vegas. Learn firsthand the latest developments in Apache, PHP, Perl, XML, Java, MySQL, WebDAV, and more! http://www.apachecon.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> I''m looking to set up a virtual domain that boots ro from the same > partitions as domain0, then I''ll add rw access to a partition for its > own purposes. Later, I''ll add NFS for /home and other stuff. > > The easiest way, it seems, is to modify these 3 files for each domain > and boot from the same / as domain0: > /etc/fstab > /etc/xenctl.xml > /etc/xen-mynewdom (or whatever I want to call it...)Hmm, I''m not quite sure what you''re planning here, but it doesn''t sound right. I wouldn''t recommend sharing a common /. If you want to save space, have a common /usr and links from / into /usr/root/xxx for /lib /bin and /sbin. You''ll want to have separate /etc /tmp and /var directories and these easiest way is with separate roots. /etc/fstab will need to be tailored accordingly in each root, unless you''re using virtual disks in which case you can arrange for every virtual disk to appear as e.g. /dev/xvda /etc/xenctl.xml and xen-mynewdom is only used by domain 0 -- you don''t need these in any other domain. I''d be inclined to make seperate xen-mynewdomX scripts for all your new domains and use a common xenctl-xml (for avoidance of doubt, all in the same directory). This will enable you to have precise control over which IP addresses go with which root file systems etc (by default xenctl will assign them sequentially).> Here''s xen-mynewdom: > domain new -hgrid-14.arsc.edu -ixenolinux.gz -dsda1 -t"ro > root=/dev/sda2 DOMID=\14"You''ll need a full path to the xenolinux.gz image. I expect you''ll also want to set the IP address with "-4" to override any defaults if present in /etc/xenctl.xml> physical grant -phdc -r > physical grant -sda2 -r > physical grant -sda1 -rYou''re not mounting any of these partitions rw. Are you still intending on using the CD as /usr? I''d certainly recommend against doing this -- the machine will be much more responsive with the file system properly installed. Best, Ian ------------------------------------------------------- This SF.Net email sponsored by: ApacheCon 2003, 16-19 November in Las Vegas. Learn firsthand the latest developments in Apache, PHP, Perl, XML, Java, MySQL, WebDAV, and more! http://www.apachecon.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Tue, Nov 11, 2003 at 11:55:50PM +0000, Ian Pratt wrote:> We''re developing a copy-on-write file system [for virtual disks...]How is this going? If I can get this I can use Xen to replace User Mode Linux... Some related things that would help with that aim: (1) Implement lazy allocation (as in the way a sparse file works) for vbd blocks. (Or is this already implemented and I haven''t found it in the list traffic?) This would allow, say, 100 vbd''s, 4Gb each, to be allocated on a 100Gb partition, as long as the average filesystem utilization in those vbd''s stayed below 25%. The extra redirection would incur a slight performance penalty, but there is a balancing economic driver -- see below. (2) Also implement lazy allocation for COW backing store, unless that would already be taken care of by (1). Example scenario -- an ISP sells Xen guests. Each guest is sold as a machine with "4Gb of disk space". The Xen host hardware has 100Gb of free physical disk. The average guest filesystem is only about 25% full, so the ISP can actually "overbook", i.e. provision 100 guests (100 vbd''s, each with their own 4Gb filesystem) on this host. Lazy allocation and/or COW would make this possible. A combination of both would allow the ISP to efficiently keep a backing store for each of several major Linux distros, giving the customers a choice. This is something I can do now with User Mode Linux. It works like this: # On the parent host, create an empty sparse file that will store our # filesystem, 4GB in size: # $ dd if=/dev/zero of=ubd0 count=0 bs=1M seek=4096 0+0 records in 0+0 records out # Note that this consumed zero data blocks so far: # $ ls -ls ubd0 0 -rw-r--r-- 1 stevegt stevegt 4294967296 Jan 15 21:01 ubd0 # Format it: # $ mkfs -j ubd0 # The superblock and copies etc. only used a few blocks: # $ ls -ls ubd0 99040 -rw-r--r-- 1 stevegt stevegt 4294967296 Jan 15 21:06 ubd0 # Mount it via loopback, copy in filesystem contents, etc. At this # point the filesystem is now usable as a UML root filesystem, and the # parent will continue to allocate blocks as they are dirtied. It''s # still pretty small: # $ ls -ls ubd0 202128 -rw-r--r-- 1 stevegt stevegt 4294967296 Jan 15 21:10 ubd0 # Start the UML: # $ linux ubd0=ubd0 ... # Or better yet, start UML, automatically creating a new copy on write # filesystem in a file named ''cow0'', using the above ubd0 as read-only # backing store: # $ chmod 444 ubd0 $ linux ubd0=ubd0,cow0 # Later, check the sizes of backing store and COW -- note the huge # amount of disk space we''re saving by not allocating those blocks: # $ ls -ls ubd0 cow0 202128 -rw-r--r-- 1 stevegt stevegt 4294967296 Jan 15 21:10 ubd0 14256 -rw-r--r-- 1 stevegt stevegt 4294967296 Jan 15 21:30 cow0 In other words, this all works right now in User Mode Linux. These semantics work well for hosting lots of independent guests who each want their own root filesystem. But the performance penalty of UML is high enough that I''d like to be able to do this under Xen instead. I''m a strong Perl/Python guy (with rusty C and kernel internals), so am likely to want to work on the provisioning tools side of making this happen for Xen if you folks implement the lower layers. Steve -- Stephen G. Traugott (KG6HDQ) UNIX/Linux Infrastructure Architect, TerraLuna LLC stevegt@TerraLuna.Org http://www.stevegt.com -- http://Infrastructures.Org ------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. http://www.eclipsecon.org/osdn _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On 16 Jan 2004, at 05:52, stevegt@TerraLuna.Org wrote:> On Tue, Nov 11, 2003 at 11:55:50PM +0000, Ian Pratt wrote: >> We''re developing a copy-on-write file system [for virtual disks...] > > How is this going? If I can get this I can use Xen to replace User > Mode > Linux...It''s actually not a file system. It''s a multiple-device (md) Linux kernel driver, derived from RAID1. I finished writing the CoW device driver for 2.4.24 and modified the ''raidtools-1.00.3'' to support CoW. Currently, the CoW device can be automatically recognized at boot time, and be started and stopped by ''raidstart'' and ''raidstop'' respectively. However, It hasn''t worked properly as expected. I''ve been debugging the kernel, which is a bit complicated.> Some related things that would help with that aim: > > (1) Implement lazy allocation (as in the way a sparse file works) for > vbd blocks. (Or is this already implemented and I haven''t found > it in > the list traffic?) This would allow, say, 100 vbd''s, 4Gb each, to > be > allocated on a 100Gb partition, as long as the average filesystem > utilization in those vbd''s stayed below 25%. The extra redirection > would incur a slight performance penalty, but there is a balancing > economic driver -- see below.In VMWare and UML, virtual disks are implemented as files in real disks. They can grow by seizing more space from the underlying real disks. In Xen, VBDs are implemented as ''extents''. One ''extent'' is a group of contiguous sectors of the real disks. VBDs can grow by adding extra extents from the underlying real disks. Obviously, VBDs work at a much lower level. Therefore, you can say, you have created a VBD of 4G but actually starts with only 500M. Whenever the free space drops below a threshold, extra extents are automatically added. This function is already present in 1.2 and 1.3 BTW, this idea is quite similar to LVM.> (2) Also implement lazy allocation for COW backing store, unless that > would already be taken care of by (1).In CoW, we typically have two devices. One is the original one, say hda, and the other is the ''dirty'' one,say hdb. When read is attempted for a sector, hdb is searched and returns the content of the sector if present. Otherwise, the content is read from hda. On the other hand, all writes go to hdb. Theoretically, hdb must be NO smaller than hda in order to cache all possible writes. But practically, only a small working set exists. That''s why hdb could be much smaller than hda but still sufficient. So, ''lazy allocation'' for CoW means: hdb starts small, but becomes larger by adding extra extents. So, why is CoW useful? 1) To remove redundancy in content. When different clients use almost the same files, CoW can significantly reduces the total storage requirement. 2) CoW can work as a huge cache. For example, you make a CoW out of a ramdisk and hard disk. Maybe even multiple layers of cache: memory, magnetic disks, optical disks ... Anway, I''m debugging the CoW driver. Hopefully to make it ASAP. -- Bin http://www.cl.cam.ac.uk/~br260 ------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. http://www.eclipsecon.org/osdn _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> In Xen, VBDs are implemented as ''extents''. One ''extent'' is a group of > contiguous sectors of the real disks. VBDs can grow by adding extra > extents from the underlying real disks. Obviously, VBDs work at a > much lower level. > > Therefore, you can say, you have created a VBD of 4G but actually > starts with only 500M. Whenever the free space drops below a > threshold, extra extents are automatically added. This function is > already present in 1.2 and 1.3Erm, no. virtual disks may be extended or contracted, but this is not currently automatic: the vd tools must be used to extend the virtual disk, and then typically the guest will need to resize the file system on the virtual disk (I believe ext3 supports on-line resizing, but not automatically).> BTW, this idea is quite similar to LVM.Yes, just like LVM i.e. manual. Right now, the best way to get ''statistical multiplexing'' of disk space is to give everyone a directory tree in domain 0 and export it to other domains via local NFS (169.254.1.x). This doesn''t currently get you CoW, but Russ is working on a CoW capable user-space NFS server. We''ll then be able to bakeoff Russ and Bin''s approaches. Ian ------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. http://www.eclipsecon.org/osdn _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On 16 Jan 2004, at 16:36, Ian Pratt wrote:>> In Xen, VBDs are implemented as ''extents''. One ''extent'' is a group of >> contiguous sectors of the real disks. VBDs can grow by adding extra >> extents from the underlying real disks. Obviously, VBDs work at a >> much lower level. >> >> Therefore, you can say, you have created a VBD of 4G but actually >> starts with only 500M. Whenever the free space drops below a >> threshold, extra extents are automatically added. This function is >> already present in 1.2 and 1.3 > > Erm, no. > > virtual disks may be extended or contracted, but this is not > currently automatic: the vd tools must be used to extend the > virtual disk, and then typically the guest will need to resize > the file system on the virtual disk (I believe ext3 supports > on-line resizing, but not automatically).Ian, thanks for pointing this out. I messed the words up when I tried to say: the function extent adding/removing has been present; but for automatic resizing, a script should be created. In addition to ext3, reiserfs has good support for online resizing, i.e. without unmount. -- Bin ------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. http://www.eclipsecon.org/osdn _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel