Hi all, I am having following deployment scenario, - 2 Xen hosts (running Xen 3.1) - 1 Image server (NFS) Requirements: ----------------- 1. Have an file system image of a linux distribution say Fedora 8 on Image (NFS) Server 2. I want to have 4 guests running Fedora 8 but for 4 different users. 3. Base file system image (Fedora 8 image on Image server) shared amongst 4 users in Read only fashion 4. All 4 users will have separate VBD''s for user specific data. Question: ------------ Is there any way to force users write on the data VBD''s exported for each VM? The whole point is that guest user should not worry about where to write, every write call should be targeted towards data VBD and this needs to be achieved without changing anything in the guest. Thanks and Regards Chetan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
chetan saundankar schreef:> - 1 Image server (NFS)If you go this way, test this with bonnie++ first, Linux will overcommit memory, and in my experience this will crash your Dom0. Stefan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Stefan de Konink schrieb:> chetan saundankar schreef: >> - 1 Image server (NFS) > > If you go this way, test this with bonnie++ first, Linux will overcommit > memory, and in my experience this will crash your Dom0.I''ve migrated a few diskless machines without problems. These were formerly booted via PXE, by now kernel and ramdisk are located at the respective dom0''s, also swap (formerly an nbd device) is now local at dom0. The rootfs has been changed as for every other migration (adding kernel modules, using xen-aware libc / moving /lib/tls out of reach, etc..) I don''t see any problems running these domU''s. Well, disk I/O is about 40-90 MB/sec. during everyday use (not really much, but you get what you pay for, and there are plenty of these machines sharing two GBit NIC''s at physical level). Below is my skeleton domU config: #-------------------------------------------- kernel = ''/boot/vmlinuz-2.6.18-xen_3.1.0'' vcpus = ''1'' ramdisk = ''/boot/initrd.img-2.6.18-xen-3.1.0'' memory = ''640'' #xenbr0 = external, xenbr1 = internal vif = [ ''bridge=xenbr1, mac=%%INT_MAC%%'', ''ip=%%EXT_IP%%, bridge=xenbr0, mac=%%EXT_MAC%%'' ] root = ''/dev/nfs nfsroot=192.168.100.1:/exports/sharedhosting/filesystems/%%HOSTNAME%%/root-x86_32,rsize=8192,wsize=8192 ip=::::::dhcp ro ramdisk_size=65535'' dhcp = ''dhcp'' disk = [ ''phy:vg0/%%HOSTNAME%%-swap,sda2,w'' ] name = ''%%HOSTNAME%%'' on_poweroff = ''destroy'' on_reboot = ''restart'' on_crash = ''restart'' #-------------------------------------------- Regards, Stephan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
It sounds like you need a COW filesystem. COW stands for copy on write. This means that you have a single root filesystem that is shared by all guests, and each guest has an extra volume that he will use to write any changes to (instead of the root filesystem). it''s a bit like a snapshot works. This enables you to have a single image for multiple machines, and you can even have local changes for each machine. Note that this will be a very bad idea if performance is important to you. I''m sure this worked with xen2.0, i''m not sure of the support on xen 3.0/3.1, but you should be able to find more info here: http://wiki.xensource.com/xenwiki/COWHowTo S On Jan 3, 2008 11:35 PM, chetan saundankar <chetan.lists@gmail.com> wrote:> Hi all, > I am having following deployment scenario, > - 2 Xen hosts (running Xen 3.1) > - 1 Image server (NFS) > > Requirements: > ----------------- > 1. Have an file system image of a linux distribution say Fedora 8 on > Image (NFS) Server > 2. I want to have 4 guests running Fedora 8 but for 4 different users. > 3. Base file system image (Fedora 8 image on Image server) shared > amongst 4 users in Read only fashion > 4. All 4 users will have separate VBD''s for user specific data. > > Question: > ------------ > Is there any way to force users write on the data VBD''s exported for > each VM? > The whole point is that guest user should not worry about where to > write, every write call should be targeted towards data VBD and this > needs to be achieved without changing anything in the guest. > > Thanks and Regards > Chetan > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Sven, Thanks a lot for replying. Yes I am looking for the solution that you have described in your mail. Now I would like to know couple of things regarding COW, 1. Is there any requirement of installing any package or updating configurations within the guest VM? if yes is there any work around. 2. How to deploy Fully Virtualized guests in the COW fashion? Regards Chetan On Jan 4, 2008 3:50 PM, sven waeyenbergh <sven.waeyenbergh@gmail.com> wrote:> It sounds like you need a COW filesystem. > COW stands for copy on write. This means that you have a single root > filesystem that is shared by all guests, and each guest has an extra volume > that he will use to write any changes to (instead of the root filesystem). > it''s a bit like a snapshot works. > This enables you to have a single image for multiple machines, and you can > even have local changes for each machine. > Note that this will be a very bad idea if performance is important to you. > > I''m sure this worked with xen2.0, i''m not sure of the support on xen > 3.0/3.1, but you should be able to find more info here: > http://wiki.xensource.com/xenwiki/COWHowTo > > S > > > > On Jan 3, 2008 11:35 PM, chetan saundankar <chetan.lists@gmail.com> wrote: > > > > > > > > Hi all, > > I am having following deployment scenario, > > - 2 Xen hosts (running Xen 3.1) > > - 1 Image server (NFS) > > > > Requirements: > > ----------------- > > 1. Have an file system image of a linux distribution say Fedora 8 on > > Image (NFS) Server > > 2. I want to have 4 guests running Fedora 8 but for 4 different users. > > 3. Base file system image (Fedora 8 image on Image server) shared > > amongst 4 users in Read only fashion > > 4. All 4 users will have separate VBD''s for user specific data. > > > > Question: > > ------------ > > Is there any way to force users write on the data VBD''s exported for > > each VM? > > The whole point is that guest user should not worry about where to > > write, every write call should be targeted towards data VBD and this > > needs to be achieved without changing anything in the guest. > > > > Thanks and Regards > > Chetan > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Chetan, answers below On Jan 4, 2008 11:47 AM, chetan saundankar <chetan.lists@gmail.com> wrote:> Hi Sven, > Thanks a lot for replying. Yes I am looking for the solution that you > have described in your mail. > Now I would like to know couple of things regarding COW, > > 1. Is there any requirement of installing any package or updating > configurations within the guest VM? if yes is there any work around.Yes, you will have to install COW-capable drivers on domU. There are several different possibilities, they are listed on the link in my previous mail. I think the best option is o use this one : http://wiki.xensource.com/xenwiki/InstructionsFromMichaelLang That one uses unionFS. I think that is available from the standard redhat distribution.> > 2. How to deploy Fully Virtualized guests in the COW fashion?i doubt very much that this is possible on HVM domains. The question if this is possible or not is actually more dependant on the distribution that you will use on domU than on xen itself. If you choose a distribution that can be installed on a COW filesystem, it should also be possible in xen. I personally think a redhat system is the best choise, ince there are a lot of readme''s for redhat out there.> > > Regards > > Chetan > > On Jan 4, 2008 3:50 PM, sven waeyenbergh <sven.waeyenbergh@gmail.com> > wrote: > > It sounds like you need a COW filesystem. > > COW stands for copy on write. This means that you have a single root > > filesystem that is shared by all guests, and each guest has an extra > volume > > that he will use to write any changes to (instead of the root > filesystem). > > it''s a bit like a snapshot works. > > This enables you to have a single image for multiple machines, and you > can > > even have local changes for each machine. > > Note that this will be a very bad idea if performance is important to > you. > > > > I''m sure this worked with xen2.0, i''m not sure of the support on xen > > 3.0/3.1, but you should be able to find more info here: > > http://wiki.xensource.com/xenwiki/COWHowTo > > > > S > > > > > > > > On Jan 3, 2008 11:35 PM, chetan saundankar <chetan.lists@gmail.com> > wrote: > > > > > > > > > > > > Hi all, > > > I am having following deployment scenario, > > > - 2 Xen hosts (running Xen 3.1) > > > - 1 Image server (NFS) > > > > > > Requirements: > > > ----------------- > > > 1. Have an file system image of a linux distribution say Fedora 8 on > > > Image (NFS) Server > > > 2. I want to have 4 guests running Fedora 8 but for 4 different users. > > > 3. Base file system image (Fedora 8 image on Image server) shared > > > amongst 4 users in Read only fashion > > > 4. All 4 users will have separate VBD''s for user specific data. > > > > > > Question: > > > ------------ > > > Is there any way to force users write on the data VBD''s exported for > > > each VM? > > > The whole point is that guest user should not worry about where to > > > write, every write call should be targeted towards data VBD and this > > > needs to be achieved without changing anything in the guest. > > > > > > Thanks and Regards > > > Chetan > > > > > > _______________________________________________ > > > Xen-users mailing list > > > Xen-users@lists.xensource.com > > > http://lists.xensource.com/xen-users > > > > > > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Thanx Sven, 1. I would have liked to have a non intrusive solution for implementing COW but it seems that for the time being I need to install COW capable drivers in domU which means when I try to deploy guests in automated fashion, I will have to carry out something like remote installations into guest domains which is going be a tedious task as there are no standard API''s for such a task (or is it there? if no error/exception handling is a worry) 2. COW for HVMs as you said is distro specific, which means storage optimization using COW for deploying Windows guests is not possible. 3. You have also stated in your mail that performance of guests within COW solutions suffers to an extent. I would like to have more pointers on this performance issue and if there are any optimizations that can be done in order to achieve a better performance. On Jan 4, 2008 4:35 PM, sven waeyenbergh <sven.waeyenbergh@gmail.com> wrote:> Chetan, > > answers below > > > On Jan 4, 2008 11:47 AM, chetan saundankar <chetan.lists@gmail.com> wrote: > > Hi Sven, > > Thanks a lot for replying. Yes I am looking for the solution that you > > have described in your mail. > > Now I would like to know couple of things regarding COW, > > > > 1. Is there any requirement of installing any package or updating > > configurations within the guest VM? if yes is there any work around. > > Yes, you will have to install COW-capable drivers on domU. There are several > different possibilities, they are listed on the link in my previous mail. I > think the best option is o use this one : > http://wiki.xensource.com/xenwiki/InstructionsFromMichaelLang > That one uses unionFS. I think that is available from the standard redhat > distribution. > > > > > 2. How to deploy Fully Virtualized guests in the COW fashion? > > i doubt very much that this is possible on HVM domains. The question if this > is possible or not is actually more dependant on the distribution that you > will use on domU than on xen itself. If you choose a distribution that can > be installed on a COW filesystem, it should also be possible in xen. I > personally think a redhat system is the best choise, ince there are a lot of > readme''s for redhat out there. > > > > > > > > Regards > > > > Chetan > > > > > > > > > > On Jan 4, 2008 3:50 PM, sven waeyenbergh <sven.waeyenbergh@gmail.com> > wrote: > > > It sounds like you need a COW filesystem. > > > COW stands for copy on write. This means that you have a single root > > > filesystem that is shared by all guests, and each guest has an extra > volume > > > that he will use to write any changes to (instead of the root > filesystem). > > > it''s a bit like a snapshot works. > > > This enables you to have a single image for multiple machines, and you > can > > > even have local changes for each machine. > > > Note that this will be a very bad idea if performance is important to > you. > > > > > > I''m sure this worked with xen2.0, i''m not sure of the support on xen > > > 3.0/3.1, but you should be able to find more info here: > > > http://wiki.xensource.com/xenwiki/COWHowTo > > > > > > S > > > > > > > > > > > > On Jan 3, 2008 11:35 PM, chetan saundankar <chetan.lists@gmail.com> > wrote: > > > > > > > > > > > > > > > > Hi all, > > > > I am having following deployment scenario, > > > > - 2 Xen hosts (running Xen 3.1) > > > > - 1 Image server (NFS) > > > > > > > > Requirements: > > > > ----------------- > > > > 1. Have an file system image of a linux distribution say Fedora 8 on > > > > Image (NFS) Server > > > > 2. I want to have 4 guests running Fedora 8 but for 4 different users. > > > > 3. Base file system image (Fedora 8 image on Image server) shared > > > > amongst 4 users in Read only fashion > > > > 4. All 4 users will have separate VBD''s for user specific data. > > > > > > > > Question: > > > > ------------ > > > > Is there any way to force users write on the data VBD''s exported for > > > > each VM? > > > > The whole point is that guest user should not worry about where to > > > > write, every write call should be targeted towards data VBD and this > > > > needs to be achieved without changing anything in the guest. > > > > > > > > Thanks and Regards > > > > Chetan > > > > > > > > _______________________________________________ > > > > Xen-users mailing list > > > > Xen-users@lists.xensource.com > > > > http://lists.xensource.com/xen-users > > > > > > > > > > > > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 1/4/08, chetan saundankar <chetan.lists@gmail.com> wrote:> 1. I would have liked to have a non intrusive solution for > implementing COW but it seems that for the time being I need to > install COW capable drivers in domU which means when I try to deploy > guests in automated fashion, I will have to carry out something like > remote installations into guest domains which is going be a tedious > task as there are no standard API''s for such a task (or is it there? > if no error/exception handling is a worry)since you''re using file-backed storage for DomU''s, try qemu''s qcow (or qcow2) format. it let''s you define a ''backing'' storage for each image.> 3. You have also stated in your mail that performance of guests within > COW solutions suffers to an extent. I would like to have more pointers > on this performance issue and if there are any optimizations that can > be done in order to achieve a better performance.i think that by choosing to use file images over NFS, you''ve already chosen on low performance. COW overhead might not be significant after that. -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> I am having following deployment scenario, > - 2 Xen hosts (running Xen 3.1) > - 1 Image server (NFS) > > Requirements: > ----------------- > 1. Have an file system image of a linux distribution say Fedora 8 on > Image (NFS) Server > 2. I want to have 4 guests running Fedora 8 but for 4 different users. > 3. Base file system image (Fedora 8 image on Image server) shared > amongst 4 users in Read only fashion > 4. All 4 users will have separate VBD''s for user specific data. > > Question: > ------------ > Is there any way to force users write on the data VBD''s exported for > each VM? > The whole point is that guest user should not worry about where to > write, every write call should be targeted towards data VBD and this > needs to be achieved without changing anything in the guest.As others have mentioned, using blktap with QCow disks seems like it might do what you want in terms of copy-on-write disks. Serving disk images off NFS can be problematic sometimes. *don''t* use "file:" VBDs on top of NFS. Using blktap should be ("tap:aio:" or "tap:qcow:") better. I don''t think using NFS is the best choice for performance and robustness but it is mighty convenient and you may find it works for you. If you want to try other ways of sharing storage, you could try a cluster filesytem (e.g. OCFS2, GFS) on a shared block device (shared with iSCSI or NBD). Or you could try to share the block level data directly rather than using a shared filesystem - that would probably have the best performance as it cuts out a layer of overhead. Another thought: you could take a look at using some kind of storage server with disk snapshotting at the backend, in order to take care of all the COW operations on behalf of the clients. Then you''d just get the image server to export all the domU block devices and it would take of the COW transparently. The Zumastor project provides COW block devices over the network - I''m not sure how ready for deployment it is, but it could be worth checking out. Hope that helps give you a bit more background. Good luck with your deployment! Cheers, Mark -- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi. Over the past few weeks ive noticed if a xen is rebooted it increments the interface of the network sometimes by 3-4 We started with eth0, Thats fine then it decided it wanted to be eth1 after a reboot Then eth3 Then eth4 and now.... eth6 Is there any way to force it to eth0 i suspect after a reboot or 2 we will soon be up to eth20 Or a possible bug? Mitch _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Le Tue, Jan 08, 2008 at 06:08:18PM +0900, Mitch Kelly [mitchkelly@westnet.com.au] a écrit:> Hi. > > Over the past few weeks ive noticed if a xen is rebooted it increments the > interface of the network sometimes by 3-4 > We started with eth0, Thats fine then it decided it wanted to be eth1 after > a rebootudev is evil when you use randomly generated MAC addresses :-) -- Dominique Rousseau Neuronnexion, Prestataire Internet & Intranet 57, route de Paris 80000 Amiens tel: 03 22 71 61 90 - fax: 03 22 71 61 99 - http://www.neuronnexion.fr _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Mitch,> Is there any way to force it to eth0 i suspect after a reboot or 2 we > will soon be up to eth20Specify a unique MAC address for every vif in the domU''s config file, like so: vif=[ ''mac=00:16:3e:01:01:01,bridge=br0'' ] (only use bridge= if you need to) -Anatol _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users