Do both compression and fixed record sizes work together? Our Directory Server uses a fixed page size (8KB by default) for database records, so I''m in the habit of setting the ZFS recordsize to equal the database page size. However, we also typically use compression because it often helps improve write performance. Given that compression would inherently result in variable-sized records (based on how well the data can be compressed), is there any advantage or disadvantage to us continuing to set the record size? Neil
On Tue, Aug 22, 2006 at 04:04:50PM -0500, Neil A. Wilson wrote:> Do both compression and fixed record sizes work together?Yes.> Our Directory Server uses a fixed page size (8KB by default) for > database records, so I''m in the habit of setting the ZFS recordsize to > equal the database page size. However, we also typically use > compression because it often helps improve write performance. Given > that compression would inherently result in variable-sized records > (based on how well the data can be compressed), is there any advantage > or disadvantage to us continuing to set the record size?Yes, you should continue to set recordsize.. The recordsize property sets the *logical* block size used by ZFS. If compression is turned on, each block (which may be at most eg. 8k if you have set recordsize=8k) is compressed and stored as a smaller block on-disk if possible. This does not change the record size, only the amount of space used to store that block on disk. Compression and the recordsize property therefore don''t interfere with one another at all. Bottom line: If you are doing record-structured access, set recordsize. If you want to compress your data, set compression. --matt
If you issue aligned, full-record write requests, there is a definite advantage to continuing to set the record size. It allows ZFS to process the write without the read-modify-write cycle that would be required for the default 128K record size. (While compression results in records of variable physical length, it''s the logical length which is important here, since it determines whether part of an on-disk block needs to be read before the block can be rewritten.) Two notes -- First, the version of ZFS in the initial S10U2 release can perform the read-modify-write even if the record size matches. This is fixed in later OpenSolaris builds, I''m not sure if it''s been patched back yet. Second, I believe the eventual goal is for ZFS to detect when a smaller record size would be advantageous, but this has not yet been done. This message posted from opensolaris.org
Anton B. Rang wrote:>If you issue aligned, full-record write requests, there is a definite advantage to continuing to set the record size. It allows ZFS to process the write without the read-modify-write cycle that would be required for the default 128K record size. (While compression results in records of variable physical length, it''s the logical length which is important here, since it determines whether part of an on-disk block needs to be read before the block can be rewritten.) > >Two notes -- First, the version of ZFS in the initial S10U2 release can perform the read-modify-write even if the record size matches. This is fixed in later OpenSolaris builds, I''m not sure if it''s been patched back yet. Second, I believe the eventual goal is for ZFS to detect when a smaller record size would be advantageous, but this has not yet been done. > >Yep, 6424554 full block re-writes need not read data in will be in U3. eric