Hi, I''m running a Xen (Debian Lenny / 3.3.1 / 2.6.18.8 / 2x1T SATA RAID1 / 8G RAM)-Server with ~8-12 VMs (LVM based - NOOP shedule). Each VM is running Postgre, Tomcat, Apache. If we do our daily testings, the disk-io-performance is terrible. As far as I understood reads are cached within the VMs, so this should not be the problem. I think the high number of writes is killing the hdd. Is there a way to do something like "async write", or what else could I do to speedup the VMs even if there is a lot of disk-io? Thanks, Fabian _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Le Fri, 03 Apr 2009 08:30:17 +0200, Fabian Zimmermann <xen-users@z-technologies.de> a écrit :> Hi, > > I''m running a Xen (Debian Lenny / 3.3.1 / 2.6.18.8 / 2x1T SATA RAID1 / > 8G RAM)-Server with ~8-12 VMs (LVM based - NOOP shedule). Each VM is > running Postgre, Tomcat, Apache. If we do our daily testings, the > disk-io-performance is terrible. > > As far as I understood reads are cached within the VMs, so this should > not be the problem. I think the high number of writes is killing the > hdd. Is there a way to do something like "async write", or what else > could I do to speedup the VMs even if there is a lot of disk-io?Hi Fabian, You could start to investigate with the "top" command and "iostat". Iostat will show you the actual disk throughput. If you near your disk maximal throughput speed, then, you can expect more from SATA HDs on RAID1. Using top on each host, pay attention to the "%wa", it''s the percentage of wait, meaning the system wait too much the storage. On a perfect system, "%wa" should stay down to 0. First of all, I will buy another SATA disk to setup a RAID5 instead of RAID1, this will improve (a little bit) I/O performance or better, RAID0+1. Do you use RAID Software between ? Maybe you could increase database memory to decrease read I/O too. Finally, if you have some extra $, try to get a SAS RAID Controller card such as [1] and a couple of SAS HDs (I would say at least 4) to setup a fast RAID5 storage group. Good luck, [1] : http://www.tigerdirect.com/applications/SearchTools/item-details.asp?EdpNo=4525300&CatId=109 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
If all of your guest domain VM''s are running on one or two HDD''s... then it''s no good as VM''s are trying to read/write all over the HDD''s. (e.g. lots and lots of random accesses across the disk) If you''ve got multiple HDD''s, then have your Xen guest domains running on top of each disk. If one of your VM is very much disk I/O intensive, then you would dedicate a HDD to that VM, and have other less disk I/O intensive VM''s on another HDD. Just do whatever you can to balance out loads on your physical HDD''s. Or if you could have SSD (Solid State Disks) then you wouldn''t have to think about this issue as it''s good for random accesses. /Taeho On Fri, Apr 3, 2009 at 3:30 PM, Fabian Zimmermann < xen-users@z-technologies.de> wrote:> Hi, > > I''m running a Xen (Debian Lenny / 3.3.1 / 2.6.18.8 / 2x1T SATA RAID1 / 8G > RAM)-Server with ~8-12 VMs (LVM based - NOOP shedule). Each VM is running > Postgre, Tomcat, Apache. If we do our daily testings, the > disk-io-performance is terrible. > > As far as I understood reads are cached within the VMs, so this should not > be the problem. I think the high number of writes is killing the hdd. Is > there a way to do something like "async write", or what else could I do to > speedup the VMs even if there is a lot of disk-io? > > Thanks, > Fabian > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
2009/4/3 L.M.J <linuxmasterjedi@free.fr>:> > First of all, I will buy another SATA disk to setup a RAID5 instead of RAID1, this will improve (a > little bit) I/O performance or better, RAID0+1.Note that RAID5 will have poorer write performance than RAID1. For a 3 disk array each write requires 2 reads, followed by the parity calculations, then the write. Compare this to simply 2 write actions required for RAID1. -- Please keep list traffic on the list. Rob MacGregor Whoever fights monsters should see to it that in the process he doesn''t become a monster. Friedrich Nietzsche _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I have the same performance issues on systems using fakeraid. Systems with a real active raid controller using SAS harddrives haven''t this problem. They are running as mooth as a standalone system. If you too using fakeraid (meaning that you use dodmraid) or software raid then I''m afraid that you can''t do much here. This is because the raid system consumes some amount of your cpu power to calculate and maintain the raid system. Regards, Martin> Hi, > > I''m running a Xen (Debian Lenny / 3.3.1 / 2.6.18.8 / 2x1T SATA RAID1 / > 8G RAM)-Server with ~8-12 VMs (LVM based - NOOP shedule). Each VM is > running Postgre, Tomcat, Apache. If we do our daily testings, the > disk-io-performance is terrible. > > As far as I understood reads are cached within the VMs, so this should > not be the problem. I think the high number of writes is killing the > hdd. Is there a way to do something like "async write", or what else > could I do to speedup the VMs even if there is a lot of disk-io? > > Thanks, > Fabian > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I have seen lower performance with fake/software raid (Higher CPU load and no raid 1 read interleaving) but not enough to be a serious deal breaker for light use. A decent cached raid controller should help in the situation below as it should reorder writes and reduce disk write thrashing. SSDs would pretty much eliminate the problem but since the user has 1Tb disks I assume SSDs will not provide enough storage for the money. Rob -----Original Message----- From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Martin Fernau Sent: 03 April 2009 08:19 To: xen-users@lists.xensource.com Subject: Re: [Xen-users] How to get rid of IO-Bottleneck? I have the same performance issues on systems using fakeraid. Systems with a real active raid controller using SAS harddrives haven''t this problem. They are running as mooth as a standalone system. If you too using fakeraid (meaning that you use dodmraid) or software raid then I''m afraid that you can''t do much here. This is because the raid system consumes some amount of your cpu power to calculate and maintain the raid system. Regards, Martin> Hi, > > I''m running a Xen (Debian Lenny / 3.3.1 / 2.6.18.8 / 2x1T SATA RAID1 / > 8G RAM)-Server with ~8-12 VMs (LVM based - NOOP shedule). Each VM is > running Postgre, Tomcat, Apache. If we do our daily testings, the > disk-io-performance is terrible. > > As far as I understood reads are cached within the VMs, so this should > not be the problem. I think the high number of writes is killing the > hdd. Is there a way to do something like "async write", or what else > could I do to speedup the VMs even if there is a lot of disk-io? > > Thanks, > Fabian > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users The SAQ Group Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ SAQ is the trading name of SEMTEC Limited. Registered in England & Wales Company Number: 06481952 http://www.saqnet.co.uk AS29219 SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business. Broadband : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support. ISPA Member Find us in http://www.thebestof.co.uk/petersfield _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> I''m running a Xen (Debian Lenny / 3.3.1 / 2.6.18.8 / 2x1T SATA RAID1 / > 8G RAM)-Server with ~8-12 VMs (LVM based - NOOP shedule). Each VM is > running Postgre, Tomcat, Apache. If we do our daily testings, the > disk-io-performance is terrible.Two 7200RPM disks, mirrored, and you have bad i/o? Look no further than your disks. John -- John Madden Sr UNIX Systems Engineer Ivy Tech Community College of Indiana jmadden@ivytech.edu _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi all, for the archive: I installed a good (with RAM) hardware-raid and my performance is back. Thanks to everyone. Fabian _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users