I''ve got 12Gb or so of db+web in a zone on a ZFS filesystem on a mirrored zpool. Noticed during some performance testing today that its i/o bound but using hardly any CPU, so I thought turning on compression would be a quick win. I know I''ll have to copy files for existing data to be compressed, so I was going to make a new filesystem, enable compression and rysnc everything in, then drop the old filesystem and mount the new one (with compressed blocks) in its place. But I''m going to be hooking in faster LUNs later this week. The plan was to remove half of the mirror, attach a new disk, remove the last old disk and attach the second half of the mirror (again on a faster disk). Will this do the same job? i.e. will I see the benefit of compression on the blocks that are copied by the mirror being resilvered? -- Rasputin :: Jack of All Trades - Master of Nuns http://number9.hellooperator.net/
On 9/11/07, Dick Davies <rasputnik at gmail.com> wrote:> > I''ve got 12Gb or so of db+web in a zone on a ZFS filesystem on a mirrored > zpool. > Noticed during some performance testing today that its i/o bound but > using hardly > any CPU, so I thought turning on compression would be a quick win. > > I know I''ll have to copy files for existing data to be compressed, so > I was going to > make a new filesystem, enable compression and rysnc everything in, then > drop the > old filesystem and mount the new one (with compressed blocks) in its > place. > > But I''m going to be hooking in faster LUNs later this week. The plan > was to remove > half of the mirror, attach a new disk, remove the last old disk and > attach the second > half of the mirror (again on a faster disk). > > Will this do the same job? i.e. will I see the benefit of compression > on the blocks > that are copied by the mirror being resilvered?No; resilvering just re-copies the existing blocks, in whatever compression state they are in. You need to re-write the files *at the filesystem layer* to get the blocks compressed. Cheers, - jonathan -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070911/5989bdb6/attachment.html>
> I''ve got 12Gb or so of db+web in a zone on a ZFS > filesystem on a mirrored zpool. > Noticed during some performance testing today that > its i/o bound but > using hardly > any CPU, so I thought turning on compression would be > a quick win.If it is io bound won''t compression make it worse?> > I know I''ll have to copy files for existing data to > be compressed, so > I was going to > make a new filesystem, enable compression and rysnc > everything in, then drop the > old filesystem and mount the new one (with compressed > blocks) in its place. > > But I''m going to be hooking in faster LUNs later this > week. The plan > was to remove > half of the mirror, attach a new disk, remove the > last old disk and > attach the second > half of the mirror (again on a faster disk). > > Will this do the same job? i.e. will I see the > benefit of compression > on the blocks > that are copied by the mirror being resilvered?No! Since you are doing a block for block mirror of the data, this would not could not compress the data.> > > -- > Rasputin :: Jack of All Trades - Master of Nuns > http://number9.hellooperator.net/ > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discu > ssThis message posted from opensolaris.org
On 11/09/2007, Mike DeMarco <mikejd42 at yahoo.com> wrote:> > I''ve got 12Gb or so of db+web in a zone on a ZFS > > filesystem on a mirrored zpool. > > Noticed during some performance testing today that > > its i/o bound but > > using hardly > > any CPU, so I thought turning on compression would be > > a quick win. > > If it is io bound won''t compression make it worse?Well, the CPUs are sat twiddling their thumbs. I thought reducing the amount of data going to disk might help I/O - is that unlikely?> > benefit of compression > > on the blocks > > that are copied by the mirror being resilvered? > > No! Since you are doing a block for block mirror of the data, this would not could not compress the data.No problem, another job for rsync then :) -- Rasputin :: Jack of All Trades - Master of Nuns http://number9.hellooperator.net/
> On 11/09/2007, Mike DeMarco <mikejd42 at yahoo.com> > wrote: > > > I''ve got 12Gb or so of db+web in a zone on a ZFS > > > filesystem on a mirrored zpool. > > > Noticed during some performance testing today > that > > > its i/o bound but > > > using hardly > > > any CPU, so I thought turning on compression > would be > > > a quick win. > > > > If it is io bound won''t compression make it worse? > > Well, the CPUs are sat twiddling their thumbs. > I thought reducing the amount of data going to disk > might help I/O - > is that unlikely?IO bottle necks are usually caused by a slow disk or one that has heavy workloads reading many small files. Two factors that need to be considered are Head seek latency and spin latency. Head seek latency is the amount of time it takes for the head to move to the track that is to be written, this is a eternity for the system (usually around 4 or 5 milliseconds). Spin latency is the amount of time it takes for the spindle to spin the track to be read or written over the head. Ideally you only want to pay the latency penalty once. If you have large reads and writes going to the disk then compression may help a little but if you have many small reads or writes it will do nothing more than to burden your CPU with a no gain amount of work to do since your are going to be paying Mr latency for each read or write. Striping several disks together with a stripe width that is tuned for your data model is how you could get your performance up. Stripping has been left out of the ZFS model for some reason. Where it is true that RAIDZ will stripe the data across a given drive set it does not give you the option to tune the stripe width. Do to the write performance problems of RAIDZ you may not get a performance boost from it stripping if your write to read ratio is too high since the driver has to calculate parity for each write.> > > > benefit of compression > > > on the blocks > > > that are copied by the mirror being resilvered? > > > > No! Since you are doing a block for block mirror of > the data, this would not could not compress the data. > > No problem, another job for rsync then :) > > > -- > Rasputin :: Jack of All Trades - Master of Nuns > http://number9.hellooperator.net/ > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discu > ssThis message posted from opensolaris.org
On 9/12/07, Mike DeMarco <mikejd42 at yahoo.com> wrote:> Striping several disks together with a stripe width that is tuned for your data > model is how you could get your performance up. Stripping has been left out > of the ZFS model for some reason. Where it is true that RAIDZ will stripe > the data across a given drive set it does not give you the option to tune the > stripe width. Do to the write performance problems of RAIDZ you may not > get a performance boost from it stripping if your write to read ratio is too > high since the driver has to calculate parity for each write.I am not sure why you think striping has been left out of the ZFS model. If you create a ZFS pool without the "raidz" or "mirror" keywords, the pool will be striped. Also, the "recordsize" tunable can be useful for matching up application I/O to physical I/O. Thanks, - Ryan -- UNIX Administrator http://prefetch.net
Mike DeMarco wrote:> IO bottle necks are usually caused by a slow disk or one that has heavy > workloads reading many small files. Two factors that need to be considered > are Head seek latency and spin latency. Head seek latency is the amount > of time it takes for the head to move to the track that is to be written, > this is a eternity for the system (usually around 4 or 5 milliseconds).For most modern disks, writes are cached in a buffer. But reads still take the latency hit. The trick is to ensure that writes are committed to media, which ZFS will do for you.> Spin latency is the amount of time it takes for the spindle to spin the > track to be read or written over the head. Ideally you only want to pay the > latency penalty once. If you have large reads and writes going to the disk > then compression may help a little but if you have many small reads or > writes it will do nothing more than to burden your CPU with a no gain > amount of work to do since your are going to be paying Mr latency for each > read or write. > > Striping several disks together with a stripe width that is tuned for your > data model is how you could get your performance up. Stripping has been > left out of the ZFS model for some reason. Where it is true that RAIDZ will > stripe the data across a given drive set it does not give you the option to > tune the stripe width.It is called "dynamic striping" and a write is not compelled to be spread across all vdevs. This opens up an interesting rat hole conversation about whether stochastic spreading is always better than an efficient, larger block write. Our grandchildren might still be arguing this when they enter the retirement home. In general, for ZFS, the top-level dynamic stripe interlace is 1 MByte which seems to fit well with the 128kByte block size. YMMV.> Do to the write performance problems of RAIDZ you may not get a performance > boost from it stripping if your write to read ratio is too high since the > driver has to calculate parity for each write.Write performance for raidz is generally quite good, better than most other RAID-5 implementations which are bit by the read-modify-write cycle (added latency). raidz can pay for this optimization when doing small, random reads, TANSTAAFL. -- richard
> On 9/12/07, Mike DeMarco <mikejd42 at yahoo.com> wrote: > > > Striping several disks together with a stripe width > that is tuned for your data > > model is how you could get your performance up. > Stripping has been left out > > of the ZFS model for some reason. Where it is true > that RAIDZ will stripe > > the data across a given drive set it does not give > you the option to tune the > > stripe width. Do to the write performance problems > of RAIDZ you may not > > get a performance boost from it stripping if your > write to read ratio is too > > high since the driver has to calculate parity for > each write. > > I am not sure why you think striping has been left > out of the ZFS > model. If you create a ZFS pool without the "raidz" > or "mirror" > keywords, the pool will be striped. Also, the > "recordsize" tunable can > be useful for matching up application I/O to physical > I/O. > > Thanks, > - Ryan > -- > UNIX Administrator > http://prefetch.net > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discu > ssOh... How right you are. I dug into the PDFs and read up on Dynamic striping. My bad. ZFS rocks. This message posted from opensolaris.org