Hi guys, my apologies for the question probably often asked, but I cannot make up my mind and I am kind of new in Xen. To cut a long story short, I have a Xen 3 installation on the local server, with two fully operational CentOS 4 guests. One guest is intended for local LAMP + Samba, the other will be local network router/firewall/internet gateway with 2 NICs. Had both setup to mount from file, now created one using LVMs. I had problems with dd if-ing a 150 GB samba.img file to be used for Samba mount (file too large), so went playing to do the same with LVMs. Main OS (0) is mounted from /dev/sda (SCSI), the guests are mounted off software-raided SATA disks (sdb and sdc) - /dev/md0 (sdb1 + sdc1) is a 30 GB partition mounted as /xenstuff (no LVM, used to store images and stuff) and /dev/md1 (sdb2 + sdc2) is a LVM partition, used to boot guests off. /dev/md1 is a LVM partition, where for ex. guest1 has a logical group guest1, with logical volumes root, swap, tmp, samba. Recreated with LVM support, all set and mounted using separate logical volumes. Working beautifully. So does with image files, though. I don''t have anything on /dev/md1 mounted permanently in domain0, can mount it like mount /dev/guest1/root /xendomains/guest1 if needed, but what I really liked about using file images for guests is that I could easily move them around or have them rsynced (or backed up in any way) at a breeze. What I like about LVM is that I can easily and more efficiently manage large partitions, extend them and manipulate them, but I lose that simple copy-the-image-files convenience. Load wise, loop doesn''t make loads anymore (since not used with LVM; this box has an old Pentium IV 2.8 GHz CPU without HT so loop loads are kind of noticeable at times) and I *think* the CPU usage is hence lower (haven''t tested performance yet, but I think LVM storage is faster for Samba operations - opening a folder full of images in thumbnail view from Windows box seems faster than with file image mounts). Honestly, I can''t decide whether to incline to file images (visible as stored files, can copy/move around easier, the location where images are stored is mounted hence accessible in a simpler way, had above mentioned problems with images too big like samba one), or LVM (seems more flexible for resizing, large partitions OK, can''t have their container location mounted permanently, can''t simply move around/backup images, loop doesn''t load, not sure how to backup those yet - from what I see, LVM snapshots are still not too stable? In any case, burning the file image to DVD and/or transporting it anywhere seems simpler than whatever used for LVM). Any advice pro/against any of storage options - file or LVM? Thanks, Boris PS. I am really having fun with Xen, really fascinating :) -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.441 / Virus Database: 268.17.35/680 - Release Date: 10.2.2007 21:15 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sun, Feb 11, 2007 at 07:27:55PM +0100, Boris Senker wrote:> Load wise, loop doesn''t make loads anymore (since not used with LVM; > this box has an old Pentium IV 2.8 GHz CPU without HT so loop loads > are kind of noticeable at times) and I *think* the CPU usage is hence > lower (haven''t tested performance yet, but I think LVM storage is > faster for Samba operations - opening a folder full of images in > thumbnail view from Windows box seems faster than with file image mounts).The reason loop appears to not generate as much load is that it is not writing your data out to disk. It is cached in the memory by the loop driver and only flushed periodically. Needless to say this is playing russian roulette with your data - if you experiance an outage on Dom0 chances are that your guest filesystems will experiance *catastrophic* data loss. Not even journalling in the guest FS will help you here since the journall writes will simply be cached in memory in the loop driver. If you want to compare performance of real block devices, vs a file backed image use the blocktap driver instead of the loop driver. eg Instead of file:/path/to/image/file,xvda,w Use the path like tap:aio:/path/to/image/file,xvda,w Also, I''d recommend fully allocating the disk space for your file image, rather than using sparse files - there is significant overhead involved in extending the sparse files at runtime which can lead to unexpected performance degradation. Sparse is fine for development/testing, but in production you want non-sparse files. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=| _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Daniel P. Berrange wrote:> Also, I''d recommend fully allocating the disk space for your file image, > rather than using sparse files - there is significant overhead involved > in extending the sparse files at runtime which can lead to unexpected > performance degradation. Sparse is fine for development/testing, but in > production you want non-sparse files. > > Regards, > Dan. >Not to mention some OSes don''t support sparse files, like NetBSD. For my machines I use LVM, it makes backups easier and giving a VM more space is a simple lvextend/growfs operation. You can always run LVM on top of RAID, at least that will give you some redundancy. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
>Guys,thanks for your replies.>The reason loop appears to not generate as much load is that it is not >writing your data out to disk. It is cached in the memory by the loop >driver and only flushed periodically. Needless to say this is playing >russian roulette with your data - if you experiance an outage on Dom0 >chances are that your guest filesystems will experiance *catastrophic* >data loss. Not even journalling in the guest FS will help you here >since the journall writes will simply be cached in memory in the loop >driver.I''d definitely opt for reliable data storage, for the guests to be properly restorable at any point.>If you want to compare performance of real block devices, vs a file >backed image use the blocktap driver instead of the loop driver. > >eg Instead of > > file:/path/to/image/file,xvda,w > >Use the path like > > tap:aio:/path/to/image/file,xvda,wWill surely try.>Also, I''d recommend fully allocating the disk space for your file image, >rather than using sparse files - there is significant overhead involved >in extending the sparse files at runtime which can lead to unexpected >performance degradation. Sparse is fine for development/testing, but in >production you want non-sparse files.Actually all file images were done with full allocations already. Saw the sparse vs. full discussion somewhere before.>Not to mention some OSes don''t support sparse files, like NetBSD. > >For my machines I use LVM, it makes backups easier and giving a VM >more space is a simple lvextend/growfs operation. You can always >run LVM on top of RAID, at least that will give you some redundancy.This is what attracted me towards LVM (especially in regards to that 150GB Samba partition). I am running LVM on top of raid already - the LV group and volumes were initially created on a /dev/md1. But honesty, I am still not sure what storage type to incline towards. Thanks again, Boris _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 11 Feb 2007 at 19:57, Michael Watters wrote:> Not to mention some OSes don''t support sparse files, like NetBSD. >I always thought DOS FAT (and derived) is the only filesystem that can''t do sparse files. ;-) Ulrich _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ulrich Windl wrote:> On 11 Feb 2007 at 19:57, Michael Watters wrote: > > >> Not to mention some OSes don''t support sparse files, like NetBSD. >> >> > > I always thought DOS FAT (and derived) is the only filesystem that can''t do sparse > files. ;-) > > UlrichYou can create them, but the domU won''t boot because of I/O errors. I''m not sure if NetBSD 4 will fix this, but if you want to run a BSD dom0 you need fully zeroed out files. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> -----Original Message----- > From: xen-users-bounces@lists.xensource.com > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of > Michael Watters > Sent: 12 February 2007 13:07 > To: Ulrich Windl > Cc: xen-users@lists.xensource.com > Subject: Re: [Xen-users] LVM or file storage? > > Ulrich Windl wrote: > > On 11 Feb 2007 at 19:57, Michael Watters wrote: > > > > > >> Not to mention some OSes don''t support sparse files, like NetBSD. > >> > >> > > > > I always thought DOS FAT (and derived) is the only > filesystem that can''t do sparse > > files. ;-) > > > > Ulrich > You can create them, but the domU won''t boot because of I/O > errors. I''m > not sure if NetBSD 4 will fix this, but if you want to run a BSD dom0 > you need fully zeroed out files.Sounds like a bug in the file-system - surely the content of "not yet used" sections of a sparse file should return zero''s (or some otherwise defined data)... -- Mats> > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Petersson, Mats wrote:> > Sounds like a bug in the file-system - surely the content of "not yet > used" sections of a sparse file should return zero''s (or some otherwise > defined data)... > > -- > Mats >It has to do with how NetBSD handles vnd devices, I was trying to set up some guest domains and newfs/mke2fs would error out trying to format the device. I also had problems using templates that I created on a CentOS box. It''s not really a big deal, it just takes slightly longer to create a non-sparse file. Most likely it will be different in the next release, they''ll also have full support for Xen 3. I still wish LVM would get ported to BSD, then I wouldn''t have to worry about it. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users