zfs discuss - Oct 2009 - compressed fs taking up more space than uncompressed equivalent

Salute.

I have a filesystem where I store various source repositories (cvs +
git). I have compression enabled on and zfs get compressratio reports
1.46x. When I copy all the stuff to another filesystem without
compression, the data take up _less_ space (3.5GB vs 2.5GB). How''s
that possible ?


Best regards,
Stathis

Stathis Kamperis wrote:
> Salute.
> 
> I have a filesystem where I store various source repositories (cvs +
> git). I have compression enabled on and zfs get compressratio reports
> 1.46x. When I copy all the stuff to another filesystem without
> compression, the data take up _less_ space (3.5GB vs 2.5GB). How''s
> that possible ?
just a few thoughts:
- how do you measure how much space your data consumes?
- how do you copy?
- is the other FS also ZFS?

Michael
-- 
Michael Schuster     http://blogs.sun.com/recursion
Recursion, n.: see ''Recursion''

2009/10/23 michael schuster <Michael.Schuster at
sun.com>:
> Stathis Kamperis wrote:
>>
>> Salute.
>>
>> I have a filesystem where I store various source repositories (cvs +
>> git). I have compression enabled on and zfs get compressratio reports
>> 1.46x. When I copy all the stuff to another filesystem without
>> compression, the data take up _less_ space (3.5GB vs 2.5GB).
How''s
>> that possible ?
>
> just a few thoughts:
> - how do you measure how much space your data consumes?
With zfs list, under the ''USED'' column. du(1) gives the same
results
as well (the individual fs sizes aren''t enterily identical with those
that zfs list reports , but the difference still exists).

tank/sources               3.73G   620G  3.73G  /export/sources
  <--- compressed
tank/test                  2.32G   620G  2.32G  /tank/test
      <--- uncompressed
> - how do you copy?
With cp(1). Should I be using zfs send | zfs receive ?
> - is the other FS also ZFS?
Yes. And they both live under the same pool.

If it matters, I don''t have any snapshots on neither of the
filesystems.

Thank you for your time.

Best regards,
Stathis Kamperis

Stathis Kamperis wrote:
> 2009/10/23 michael schuster <Michael.Schuster at sun.com>:
>> Stathis Kamperis wrote:
>>> Salute.
>>>
>>> I have a filesystem where I store various source repositories (cvs
+
>>> git). I have compression enabled on and zfs get compressratio
reports
>>> 1.46x. When I copy all the stuff to another filesystem without
>>> compression, the data take up _less_ space (3.5GB vs 2.5GB).
How''s
>>> that possible ?
>> just a few thoughts:
>> - how do you measure how much space your data consumes?
> With zfs list, under the ''USED'' column. du(1) gives the
same results
> as well (the individual fs sizes aren''t enterily identical with
those
> that zfs list reports , but the difference still exists).
> 
> tank/sources               3.73G   620G  3.73G  /export/sources
>   <--- compressed
> tank/test                  2.32G   620G  2.32G  /tank/test
>       <--- uncompressed
obvious, but still: you did make sure that the compressed one doesn''t
have
any other data lying around, right?
> 
>> - how do you copy?
> With cp(1). Should I be using zfs send | zfs receive ?
I don''t know :-) I was just (still am) thinking out loud.
>> - is the other FS also ZFS?
> Yes. And they both live under the same pool.
> 
> If it matters, I don''t have any snapshots on neither of the
filesystems.
"zfs list -t all" might still be revealing ...

Michael
-- 
Michael Schuster     http://blogs.sun.com/recursion
Recursion, n.: see ''Recursion''

Le 23 oct. 09 ? 08:46, Stathis Kamperis a ?crit :
> 2009/10/23 michael schuster <Michael.Schuster at sun.com>:
>> Stathis Kamperis wrote:
>>>
>>> Salute.
>>>
>>> I have a filesystem where I store various source repositories (cvs
+
>>> git). I have compression enabled on and zfs get compressratio  
>>> reports
>>> 1.46x. When I copy all the stuff to another filesystem without
>>> compression, the data take up _less_ space (3.5GB vs 2.5GB).
How''s
>>> that possible ?
>>
>> just a few thoughts:
>> - how do you measure how much space your data consumes?
> With zfs list, under the ''USED'' column. du(1) gives the
same results
> as well (the individual fs sizes aren''t enterily identical with
those
> that zfs list reports , but the difference still exists).
>
> tank/sources               3.73G   620G  3.73G  /export/sources
>  <--- compressed
> tank/test                  2.32G   620G  2.32G  /tank/test
>      <--- uncompressed
>
USED includes the size of the children and the size of the snapshot. I  
see below that you don''t have snapshots on that pull, but in general,  
I found more useful to use

   zfs list -o space,compress,ratio

to look at how the space is used.
>> - how do you copy?
> With cp(1). Should I be using zfs send | zfs receive ?
zfs send/receive or rsync -aH may do a better job by preserving hard  
links.

Regards,

Ga?tan

-- 
Ga?tan Lehmann
Biologie du D?veloppement et de la Reproduction
INRA de Jouy-en-Josas (France)
tel: +33 1 34 65 29 66    fax: 01 34 65 29 09
http://voxel.jouy.inra.fr  http://www.itk.org
http://www.mandriva.org  http://www.bepo.fr

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: Ceci est une signature ?lectronique PGP
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20091023/e7cef8ae/attachment.bin>

2009/10/23 Ga?tan Lehmann <gaetan.lehmann at
jouy.inra.fr>:
>
> Le 23 oct. 09 ? 08:46, Stathis Kamperis a ?crit :
>
>> 2009/10/23 michael schuster <Michael.Schuster at sun.com>:
>>>
>>> Stathis Kamperis wrote:
>>>>
>>>> Salute.
>>>>
>>>> I have a filesystem where I store various source repositories
(cvs +
>>>> git). I have compression enabled on and zfs get compressratio
reports
>>>> 1.46x. When I copy all the stuff to another filesystem without
>>>> compression, the data take up _less_ space (3.5GB vs 2.5GB).
How''s
>>>> that possible ?
>>>
>>> just a few thoughts:
>>> - how do you measure how much space your data consumes?
>>
>> With zfs list, under the ''USED'' column. du(1) gives
the same results
>> as well (the individual fs sizes aren''t enterily identical
with those
>> that zfs list reports , but the difference still exists).
>>
>> tank/sources ? ? ? ? ? ? ? 3.73G ? 620G ?3.73G ?/export/sources
>> ?<--- compressed
>> tank/test ? ? ? ? ? ? ? ? ?2.32G ? 620G ?2.32G ?/tank/test
>> ? ? <--- uncompressed
>>
>
> USED includes the size of the children and the size of the snapshot. I see
> below that you don''t have snapshots on that pull, but in general,
I found
> more useful to use
>
> ?zfs list -o space,compress,ratio
>
> to look at how the space is used.
>
>>> - how do you copy?
>>
>> With cp(1). Should I be using zfs send | zfs receive ?
>
> zfs send/receive or rsync -aH may do a better job by preserving hard links.
I destroyed the test fs, recreated it and did an rsync. The size of
the uncompressed filesystem is now larger than the compressed one. I
guess cp(1) missed a great deal of stuff, which is weird because I
didn''t get any error/warning on the console output. All good now.

Thanks Ga?tan and Michael for your time and sorry to the rest of the
list readers for the noise.

Best regards,
Stathis Kamperis