Hi, We were trying out the "compression=on" feature of ZFS and were wondering if it would make sense to have ZFS do compression only on a certain kind of files (or rather the otherway around). Our observation : - If ZFS finds that it cannot achieve a certain amount of compression it does not compress the file. However, to figure this out ZFS would have to first compress the block''s data. This means ZFS ends up consuming resources and that overhead is not worth it. Is the above observation correct. If so, is there someway this can be tuned/controlled ? If not how about having a feature where we could tell ZFS something like : " Don''t try to compress the .gz files" ? We could probably use the magic number of the file to identify the type. Does this make sense ? Or is this a stupid idea ? Thanks and regards, Sanjeev.
Hello Sanjeev,
Wednesday, August 30, 2006, 3:26:52 PM, you wrote:
SB> Hi,
SB> We were trying out the "compression=on" feature of ZFS and were
SB> wondering if it would make
SB> sense to have ZFS do compression only on a certain kind of files (or
SB> rather the otherway around).
SB> Our observation :
SB> - If ZFS finds that it cannot achieve a certain amount of compression it
SB> does not compress the file.
SB> However, to figure this out ZFS would have to first compress the
SB> block''s data. This means ZFS ends
SB> up consuming resources and that overhead is not worth it.
SB> Is the above observation correct. If so, is there someway this can be
SB> tuned/controlled ?
right now it''s 12% minimum compression gain required.
Unfortunately last I checked it was hard-coded.
Additionally it''s not file based but block based so for large files
which are already compressed zfs during writing will try to compress
block by block.
SB> If not how about having a feature where we could tell ZFS something like
:
SB> " Don''t try to compress the .gz files" ? We could
probably use the
SB> magic number of the file to
SB> identify the type.
SB> Does this make sense ? Or is this a stupid idea ?
I guess some functionality like this would be great.
IIRC it was also proposed by someone from ZFS team long time ago :)
--
Best regards,
Robert mailto:rmilkowski at task.gda.pl
http://milek.blogspot.com
Robert Milkowski wrote:> Hello Sanjeev, > > Wednesday, August 30, 2006, 3:26:52 PM, you wrote: > > SB> Hi, > > SB> We were trying out the "compression=on" feature of ZFS and were > SB> wondering if it would make > SB> sense to have ZFS do compression only on a certain kind of files (or > SB> rather the otherway around). > > SB> Our observation : > SB> - If ZFS finds that it cannot achieve a certain amount of compression it > SB> does not compress the file. > SB> However, to figure this out ZFS would have to first compress the > SB> block''s data. This means ZFS ends > SB> up consuming resources and that overhead is not worth it. > SB> Is the above observation correct. If so, is there someway this can be > SB> tuned/controlled ? > > right now it''s 12% minimum compression gain required. > Unfortunately last I checked it was hard-coded.RFE 6444911 ''zfs lzjb compression needs to be tunable'' has been filed for this. Regards, Manoj
Manoj Joseph wrote:> Robert Milkowski wrote: >> Hello Sanjeev, >> >> Wednesday, August 30, 2006, 3:26:52 PM, you wrote: >> >> SB> Hi, >> >> SB> We were trying out the "compression=on" feature of ZFS and were >> SB> wondering if it would make >> SB> sense to have ZFS do compression only on a certain kind of files >> (or SB> rather the otherway around). >> >> SB> Our observation : >> SB> - If ZFS finds that it cannot achieve a certain amount of >> compression it >> SB> does not compress the file. >> SB> However, to figure this out ZFS would have to first compress the >> SB> block''s data. This means ZFS ends >> SB> up consuming resources and that overhead is not worth it. >> SB> Is the above observation correct. If so, is there someway this >> can be >> SB> tuned/controlled ? >> >> right now it''s 12% minimum compression gain required. >> Unfortunately last I checked it was hard-coded. > > RFE 6444911 ''zfs lzjb compression needs to be tunable'' has been filed > for this.Thanks ! But, I don''t think it completely addresses the issue I highlighted above. Probably another RFE could be filed for this. Or add these requirements to the current RFE. Regards, Sanjeev.> > Regards, > Manoj
Sanjeev Bagewadi wrote:> Manoj Joseph wrote: >> Robert Milkowski wrote: >>> Hello Sanjeev, >>> >>> Wednesday, August 30, 2006, 3:26:52 PM, you wrote: >>> >>> SB> Hi, >>> >>> SB> We were trying out the "compression=on" feature of ZFS and were >>> SB> wondering if it would make >>> SB> sense to have ZFS do compression only on a certain kind of files >>> (or SB> rather the otherway around). >>> >>> SB> Our observation : >>> SB> - If ZFS finds that it cannot achieve a certain amount of >>> compression it >>> SB> does not compress the file. >>> SB> However, to figure this out ZFS would have to first compress the >>> SB> block''s data. This means ZFS ends >>> SB> up consuming resources and that overhead is not worth it. >>> SB> Is the above observation correct. If so, is there someway this >>> can be >>> SB> tuned/controlled ? >>> >>> right now it''s 12% minimum compression gain required. >>> Unfortunately last I checked it was hard-coded. >> >> RFE 6444911 ''zfs lzjb compression needs to be tunable'' has been filed >> for this. > Thanks ! But, I don''t think it completely addresses the issue I > highlighted above. > Probably another RFE could be filed for this. Or add these > requirements to the current > RFE.I''d say its a new RFE. You''d probably want whitelist and blacklist functionality too. -- Torrey McMahon Sun Microsystems Inc.