Carsten Aulbert
2009-Mar-16 20:54 UTC
[zfs-discuss] seeking in ZFS when data is compressed
Hi all, I was just reading http://blogs.sun.com/dap/entry/zfs_compression and would like to know what the experience of people is about enabling compression in ZFS. In principle I don''t think it''s a bad thing, especially not when the CPUs are fast enough to improve the performance as the hard drives might be too slow. However, I''m missing two aspects: o what happens when a user opens the file and does a lot of seeking inside the file? For example our scientists use a data format where quite compressible data is contained in stretches and the file header contains a dictionary where each stretch of data starts. If these files are compressed on disk, what will happen with ZFS? Will it just make educated guesses, or does it have to read all of the typically 30-150 MB of the file and then does the seeking from buffer caches? o Another problem I see (but probably isn''t): A user is accessing a file via a NFS-exported ZFS, appending a line of text, closing the file (and hopefully also flushing everything correctly. However, then the user opens it again appends another line of text, ... Imagine this happening a few times per second. How will ZFS react to this pattern? Will it only opens the final record of the file, uncompress it, adds data, recompresses it, flushes it to disk and reports that back to the user''s processes? Is there a potential problem here? Cheers (and sorry if these questions are stupid ones) Carsten
A Darren Dunham
2009-Mar-16 21:18 UTC
[zfs-discuss] seeking in ZFS when data is compressed
On Mon, Mar 16, 2009 at 09:54:57PM +0100, Carsten Aulbert wrote:> o what happens when a user opens the file and does a lot of seeking > inside the file? For example our scientists use a data format where > quite compressible data is contained in stretches and the file header > contains a dictionary where each stretch of data starts. If these files > are compressed on disk, what will happen with ZFS? Will it just make > educated guesses, or does it have to read all of the typically 30-150 MB > of the file and then does the seeking from buffer caches?Individual ZFS blocks are compressed. So seeking isn''t expensive. It doesn''t have to decompress earlier blocks to find the correct offset.> o Another problem I see (but probably isn''t): A user is accessing a file > via a NFS-exported ZFS, appending a line of text, closing the file (and > hopefully also flushing everything correctly. However, then the user > opens it again appends another line of text, ... Imagine this happening > a few times per second. How will ZFS react to this pattern? Will it only > opens the final record of the file, uncompress it, adds data, > recompresses it, flushes it to disk and reports that back to the user''s > processes? Is there a potential problem here?If it''s truly an append, then no. Only the last block has to be rewritten. -- Darren
Carsten Aulbert wrote:> Hi all, > > I was just reading > http://blogs.sun.com/dap/entry/zfs_compression > > and would like to know what the experience of people is about enabling > compression in ZFS. > > In principle I don''t think it''s a bad thing, especially not when the > CPUs are fast enough to improve the performance as the hard drives might > be too slow. However, I''m missing two aspects: > > o what happens when a user opens the file and does a lot of seeking > inside the file? For example our scientists use a data format where > quite compressible data is contained in stretches and the file header > contains a dictionary where each stretch of data starts. If these files > are compressed on disk, what will happen with ZFS? Will it just make > educated guesses, or does it have to read all of the typically 30-150 MB > of the file and then does the seeking from buffer caches? >Files are not compressed in ZFS. Blocks are compressed. If the compression of the blocks cannot gain more than 12.5% space savings, then the block will not be compressed. If your file contains compressable parts and uncompressable parts, then (depending on the size/blocks) it may be partially compressed.> o Another problem I see (but probably isn''t): A user is accessing a file > via a NFS-exported ZFS, appending a line of text, closing the file (and > hopefully also flushing everything correctly. However, then the user > opens it again appends another line of text, ... Imagine this happening > a few times per second. How will ZFS react to this pattern? Will it only > opens the final record of the file, uncompress it, adds data, > recompresses it, flushes it to disk and reports that back to the user''s > processes? Is there a potential problem here? >The file will be cached in RAM. When the file is closed and synced, the data will be written to the ZIL and ultimately to the data set. I don''t think there is a fundamental problem here... you should notice the NFS sync behaviour whether the backing store is ZFS or some other file system. Using a slog or nonvolatile write cache will help performance for such workloads.> Cheers (and sorry if these questions are stupid ones) >They are good questions :-) -- richard
Carsten Aulbert
2009-Mar-16 21:34 UTC
[zfs-discuss] seeking in ZFS when data is compressed
Hi Richard, Richard Elling wrote:> > Files are not compressed in ZFS. Blocks are compressed.Sorry, yes, I was not specific enough.> > If the compression of the blocks cannot gain more than 12.5% space savings, > then the block will not be compressed. If your file contains > compressable parts > and uncompressable parts, then (depending on the size/blocks) it may be > partially compressed. >I guess the block size is related (or equal) to the record size set for this file system, right? What will happen then if I have a file which contains a header which fits into 1 or 2 blocks, and is followed by stretches of data which are say 500kB each (for simplicity) which could be visualized as sitting in a rectangle with M rows and N columns. Since the file system has no way of knowing details on the file, it will "cut" the file into blocks and store it compressed or uncompressed as you have written. However, what happens if the typical usage pattern is read only columns of the "rectangle", i.e. read the header, seek to the start of stretch #1, then seeking to stretch #N+1, ... Can ZFS make educated guesses where the seek targets might be or will it read the file block by block until it reaches the target position, in the latter case it might be quite inefficient if the file is huge and has a large variance in compressibility.> > The file will be cached in RAM. When the file is closed and synced, the > data > will be written to the ZIL and ultimately to the data set. I don''t > think there > is a fundamental problem here... you should notice the NFS sync behaviour > whether the backing store is ZFS or some other file system. Using a slog > or nonvolatile write cache will help performance for such workloads. >Thanks, that''s answer I was hoping for :)> They are good questions :-)Good :) Cheers Carsten
Carsten Aulbert wrote:> Hi Richard, > > Richard Elling wrote: > >> Files are not compressed in ZFS. Blocks are compressed. >> > > Sorry, yes, I was not specific enough. > > >> If the compression of the blocks cannot gain more than 12.5% space savings, >> then the block will not be compressed. If your file contains >> compressable parts >> and uncompressable parts, then (depending on the size/blocks) it may be >> partially compressed. >> >> > > I guess the block size is related (or equal) to the record size set for > this file system, right? >The block size is dynamic, but for large files will likely top out at the recordsize.> What will happen then if I have a file which contains a header which > fits into 1 or 2 blocks, and is followed by stretches of data which are > say 500kB each (for simplicity) which could be visualized as sitting in > a rectangle with M rows and N columns. Since the file system has no way > of knowing details on the file, it will "cut" the file into blocks and > store it compressed or uncompressed as you have written. However, what > happens if the typical usage pattern is read only columns of the > "rectangle", i.e. read the header, seek to the start of stretch #1, then > seeking to stretch #N+1, ... >File systems do this even if the blocks are not compressed. I can read your question as asking whether or not the file will always be stored in contiguous blocks. The general answer is no, especially since ZFS has a COW architecture. But the practical impacts of this are difficult to predict, because there is so much caching occuring at all levels of the data path. Suffice to say, if you think your disks are seeking themselves to death, there is a simple dtrace script, iopattern, which will prove or disprove the case.> Can ZFS make educated guesses where the seek targets might be or will it > read the file block by block until it reaches the target position, in > the latter case it might be quite inefficient if the file is huge and > has a large variance in compressibility. >This isn''t a ZFS function, per se. If a program is written to seek(), then it can seek. If a program is written to sequentially read, then it will do that. The reason I say "per se" above, is because there is, by default, some prefetching which can occur at the device or file level. For magnetic disks, the prefetching is usually free because it costs so much time to move the head. For SSDs, I think the jury is still out... we just don''t have enough collective experience to know that prefetching will never be a win. -- richard
A Darren Dunham
2009-Mar-16 21:44 UTC
[zfs-discuss] seeking in ZFS when data is compressed
On Mon, Mar 16, 2009 at 10:34:54PM +0100, Carsten Aulbert wrote:> Can ZFS make educated guesses where the seek targets might be or will it > read the file block by block until it reaches the target position, in > the latter case it might be quite inefficient if the file is huge and > has a large variance in compressibility.Imagine a file that isn''t compressed. You don''t have to read earlier blocks to find data further into the file. If you want data from a particular offset, the filesystem can direct you to the block containing that offset, and the byte within the block. Now, since only the individual ZFS blocks are compressed, nothing changes when compression is enabled. The filesystem can still compute the ZFS block containing any offset. You might have to decompress the final block to read the data, but that''s the only file data block that has to be read. This is different from something like a .Z file where the entire file is compressed. For that, to read data at the end you either need a directory or to decompress earlier data. -- Darren
Carsten Aulbert
2009-Mar-16 21:59 UTC
[zfs-discuss] seeking in ZFS when data is compressed
Darren, Richard, thanks a lot for the very good answers. Regarding the seeking I was probably mislead by the believe that the block size was like an impenetrable block where as much data as possible is being squeezed into (like .Z files would be if you first compressed and then cut the data into blocks). Thanks a lot! Carsten
Hey all, I have a question/puzzle with zfs. See the following: bash-3.00# df -h | grep d25 ; zfs list | grep d25 FILESYSTEM SIZE USED AVAIL CAPACITY MOUNTED ON r12_data/d25 *659G* 40G *63G* 39% /opt/d25/oakwc12 df -h says the d25 file system is 659GB?; 40GB used and 63GB available? r12_data/d24 42G 40G 2.1G 95% /opt/d24/oakwcr12 NAME USED AVAIL REFER MOUNTPOINT r12_data/d25 760K *62.7G* 39.9G /opt/d25/oakwc12 zfs list says the db25 file system has 63GB available? r12_data/d24 39.9G 2.14G 39.9G /opt/d24/oakwcr12 Shouldn''t the new filesystem (d25) size be what the clone was allocated?
Grant Lowe wrote:> Hey all, > > I have a question/puzzle with zfs. See the following: > > bash-3.00# df -h | grep d25 ; zfs list | grep d25 > > FILESYSTEM SIZE USED AVAIL CAPACITY MOUNTED ON > r12_data/d25 *659G* 40G *63G* 39% /opt/d25/oakwc12 > df -h says the d25 file system is 659GB?; 40GB used and 63GB available? > r12_data/d24 42G 40G 2.1G 95% /opt/d24/oakwcr12 > > NAME USED AVAIL REFER MOUNTPOINT > r12_data/d25 760K *62.7G* 39.9G /opt/d25/oakwc12 > zfs list says the db25 file system has 63GB available? > r12_data/d24 39.9G 2.14G 39.9G /opt/d24/oakwcr12 > > > Shouldn''t the new filesystem (d25) size be what the clone was allocated? > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >Hi Grant, We''d need more info than that to figure what''s actually going on. Is d25 a clone of something? If so what? Can we see the specs of that as well. Does d25 have any reservations or a quota? What does zfs list of r12_data show? Do you have snapshots? zfs list -t all will show you them. Finally, the clone will only be the size of it''s delta from the source. HTH -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090317/73bb48dc/attachment.bin>
Hi Mike, Yes, d25 is a clone of d24. Here are some data points about it: bash-3.00# zfs get reservation r12_data/d25 NAME PROPERTY VALUE SOURCE r12_data/d25 reservation none default bash-3.00# zfs get quota r12_data/d25 NAME PROPERTY VALUE SOURCE r12_data/d25 quota none default bash-3.00# bash-3.00# zfs list r12_data NAME USED AVAIL REFER MOUNTPOINT r12_data 596G 62.7G 24.5K none bash-3.00# bash-3.00# zfs list -t snapshot r12_data/d24 at A NAME USED AVAIL REFER MOUNTPOINT r12_data/d24 at A 904K - 39.9G - bash-3.00# Thanks for the response. Did you need any more data points from me? ----- Original Message ---- From: Michael Ramchand <michael at ramchand.net> To: Grant Lowe <glowe at sbcglobal.net> Cc: zfs-discuss at opensolaris.org Sent: Tuesday, March 17, 2009 12:40:53 AM Subject: Re: [zfs-discuss] Disk usage Grant Lowe wrote:> Hey all, > > I have a question/puzzle with zfs. See the following: > > bash-3.00# df -h | grep d25 ; zfs list | grep d25 > > FILESYSTEM SIZE USED AVAIL CAPACITY MOUNTED ON > r12_data/d25 *659G* 40G *63G* 39% /opt/d25/oakwc12 > df -h says the d25 file system is 659GB?; 40GB used and 63GB available? > r12_data/d24 42G 40G 2.1G 95% /opt/d24/oakwcr12 > > NAME USED AVAIL REFER MOUNTPOINT > r12_data/d25 760K *62.7G* 39.9G /opt/d25/oakwc12 > zfs list says the db25 file system has 63GB available? > r12_data/d24 39.9G 2.14G 39.9G /opt/d24/oakwcr12 > > > Shouldn''t the new filesystem (d25) size be what the clone was allocated? > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >Hi Grant, We''d need more info than that to figure what''s actually going on. Is d25 a clone of something? If so what? Can we see the specs of that as well. Does d25 have any reservations or a quota? What does zfs list of r12_data show? Do you have snapshots? zfs list -t all will show you them. Finally, the clone will only be the size of it''s delta from the source. HTH
Well, it is kinda confusing... In short, df -h will always return the size of the WHOLE pool for "size" (unless you''ve set a quota on the dataset in which case it says that), the amount of space that particular dataset is using for "used", and the total amount of free space on the WHOLE pool for "avail" (unless you''ve got a quota or reservation set). So in your original mail. 695G is the size of the DATA on the r12_data pool. zfs list r12_data says you are using 596G. I think this is the RAW capacity used, and I reckon you are using compression. (zfs get compressratio r12_data will give you something like 1.1). However, df -h of r12_data/d24 should have the identical 1st and 3rd fields, but they don''t. (Could you re-run?) Same goes for the zfs list commands. Could you try doing zfs list -o space to get a fuller breakdown of how the space is being used. Mike Grant Lowe wrote:> Hi Mike, > > Yes, d25 is a clone of d24. Here are some data points about it: > > bash-3.00# zfs get reservation r12_data/d25 > NAME PROPERTY VALUE SOURCE > r12_data/d25 reservation none default > bash-3.00# zfs get quota r12_data/d25 > NAME PROPERTY VALUE SOURCE > r12_data/d25 quota none default > bash-3.00# > bash-3.00# zfs list r12_data > NAME USED AVAIL REFER MOUNTPOINT > r12_data 596G 62.7G 24.5K none > bash-3.00# > bash-3.00# zfs list -t snapshot r12_data/d24 at A > NAME USED AVAIL REFER MOUNTPOINT > r12_data/d24 at A 904K - 39.9G - > bash-3.00# > > Thanks for the response. Did you need any more data points from me? > > > > ----- Original Message ---- > From: Michael Ramchand <michael at ramchand.net> > To: Grant Lowe <glowe at sbcglobal.net> > Cc: zfs-discuss at opensolaris.org > Sent: Tuesday, March 17, 2009 12:40:53 AM > Subject: Re: [zfs-discuss] Disk usage > > Grant Lowe wrote: > >> Hey all, >> >> I have a question/puzzle with zfs. See the following: >> >> bash-3.00# df -h | grep d25 ; zfs list | grep d25 >> >> FILESYSTEM SIZE USED AVAIL CAPACITY MOUNTED ON >> r12_data/d25 *659G* 40G *63G* 39% /opt/d25/oakwc12 >> df -h says the d25 file system is 659GB?; 40GB used and 63GB available? >> r12_data/d24 42G 40G 2.1G 95% /opt/d24/oakwcr12 >> >> NAME USED AVAIL REFER MOUNTPOINT >> r12_data/d25 760K *62.7G* 39.9G /opt/d25/oakwc12 >> zfs list says the db25 file system has 63GB available? >> r12_data/d24 39.9G 2.14G 39.9G /opt/d24/oakwcr12 >> >> >> Shouldn''t the new filesystem (d25) size be what the clone was allocated? >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >> >> > Hi Grant, > > We''d need more info than that to figure what''s actually going on. > > Is d25 a clone of something? If so what? Can we see the specs of that as > well. > > Does d25 have any reservations or a quota? > > What does zfs list of r12_data show? > > Do you have snapshots? zfs list -t all will show you them. > > Finally, the clone will only be the size of it''s delta from the source. > > HTH > >-------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090317/ede64223/attachment.bin>
Hi Mike, Yes, that does help things. Thanks. bash-3.00# zfs get compression r12_data/d25 NAME PROPERTY VALUE SOURCE r12_data/d25 compression off default bash-3.00# zfs get compression r12_data/d24 NAME PROPERTY VALUE SOURCE r12_data/d24 compression on local bash-3.00# bash-3.00# df -h | grep d24 r12_data/d24 42G 40G 2.1G 95% /opt/d24/oakwcr12 bash-3.00# df -h | grep d25 r12_data/d25 659G 40G 63G 39% /opt/d25/oakwc12 bash-3.00# When you asked me to do zfs list -o space, what option did you mean. space isn''t an option. ----- Original Message ---- From: Michael Ramchand <michael at ramchand.net> To: Grant Lowe <glowe at sbcglobal.net> Cc: zfs-discuss at opensolaris.org Sent: Tuesday, March 17, 2009 8:32:49 AM Subject: Re: [zfs-discuss] Disk usage Well, it is kinda confusing... In short, df -h will always return the size of the WHOLE pool for "size" (unless you''ve set a quota on the dataset in which case it says that), the amount of space that particular dataset is using for "used", and the total amount of free space on the WHOLE pool for "avail" (unless you''ve got a quota or reservation set). So in your original mail. 695G is the size of the DATA on the r12_data pool. zfs list r12_data says you are using 596G. I think this is the RAW capacity used, and I reckon you are using compression. (zfs get compressratio r12_data will give you something like 1.1). However, df -h of r12_data/d24 should have the identical 1st and 3rd fields, but they don''t. (Could you re-run?) Same goes for the zfs list commands. Could you try doing zfs list -o space to get a fuller breakdown of how the space is being used. Mike Grant Lowe wrote:> Hi Mike, > > Yes, d25 is a clone of d24. Here are some data points about it: > > bash-3.00# zfs get reservation r12_data/d25 > NAME PROPERTY VALUE SOURCE > r12_data/d25 reservation none default > bash-3.00# zfs get quota r12_data/d25 > NAME PROPERTY VALUE SOURCE > r12_data/d25 quota none default > bash-3.00# > bash-3.00# zfs list r12_data > NAME USED AVAIL REFER MOUNTPOINT > r12_data 596G 62.7G 24.5K none > bash-3.00# > bash-3.00# zfs list -t snapshot r12_data/d24 at A > NAME USED AVAIL REFER MOUNTPOINT > r12_data/d24 at A 904K - 39.9G - > bash-3.00# > > Thanks for the response. Did you need any more data points from me? > > > > ----- Original Message ---- > From: Michael Ramchand <michael at ramchand.net> > To: Grant Lowe <glowe at sbcglobal.net> > Cc: zfs-discuss at opensolaris.org > Sent: Tuesday, March 17, 2009 12:40:53 AM > Subject: Re: [zfs-discuss] Disk usage > > Grant Lowe wrote: > >> Hey all, >> >> I have a question/puzzle with zfs. See the following: >> >> bash-3.00# df -h | grep d25 ; zfs list | grep d25 >> >> FILESYSTEM SIZE USED AVAIL CAPACITY MOUNTED ON >> r12_data/d25 *659G* 40G *63G* 39% /opt/d25/oakwc12 >> df -h says the d25 file system is 659GB?; 40GB used and 63GB available? >> r12_data/d24 42G 40G 2.1G 95% /opt/d24/oakwcr12 >> >> NAME USED AVAIL REFER MOUNTPOINT >> r12_data/d25 760K *62.7G* 39.9G /opt/d25/oakwc12 >> zfs list says the db25 file system has 63GB available? >> r12_data/d24 39.9G 2.14G 39.9G /opt/d24/oakwcr12 >> >> >> Shouldn''t the new filesystem (d25) size be what the clone was allocated? >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >> >> > Hi Grant, > > We''d need more info than that to figure what''s actually going on. > > Is d25 a clone of something? If so what? Can we see the specs of that as > well. > > Does d25 have any reservations or a quota? > > What does zfs list of r12_data show? > > Do you have snapshots? zfs list -t all will show you them. > > Finally, the clone will only be the size of it''s delta from the source. > > HTH > >
If you meant available, here''s the output of that: bash-3.00# zfs list -o available r12_data AVAIL 62.7G bash-3.00# zfs list -o available r12_data/d24 AVAIL 2.14G bash-3.00# zfs list -o available r12_data/d25 AVAIL 62.7G bash-3.00# ----- Original Message ---- From: Michael Ramchand <michael at ramchand.net> To: Grant Lowe <glowe at sbcglobal.net> Cc: zfs-discuss at opensolaris.org Sent: Tuesday, March 17, 2009 8:32:49 AM Subject: Re: [zfs-discuss] Disk usage Well, it is kinda confusing... In short, df -h will always return the size of the WHOLE pool for "size" (unless you''ve set a quota on the dataset in which case it says that), the amount of space that particular dataset is using for "used", and the total amount of free space on the WHOLE pool for "avail" (unless you''ve got a quota or reservation set). So in your original mail. 695G is the size of the DATA on the r12_data pool. zfs list r12_data says you are using 596G. I think this is the RAW capacity used, and I reckon you are using compression. (zfs get compressratio r12_data will give you something like 1.1). However, df -h of r12_data/d24 should have the identical 1st and 3rd fields, but they don''t. (Could you re-run?) Same goes for the zfs list commands. Could you try doing zfs list -o space to get a fuller breakdown of how the space is being used. Mike Grant Lowe wrote:> Hi Mike, > > Yes, d25 is a clone of d24. Here are some data points about it: > > bash-3.00# zfs get reservation r12_data/d25 > NAME PROPERTY VALUE SOURCE > r12_data/d25 reservation none default > bash-3.00# zfs get quota r12_data/d25 > NAME PROPERTY VALUE SOURCE > r12_data/d25 quota none default > bash-3.00# > bash-3.00# zfs list r12_data > NAME USED AVAIL REFER MOUNTPOINT > r12_data 596G 62.7G 24.5K none > bash-3.00# > bash-3.00# zfs list -t snapshot r12_data/d24 at A > NAME USED AVAIL REFER MOUNTPOINT > r12_data/d24 at A 904K - 39.9G - > bash-3.00# > > Thanks for the response. Did you need any more data points from me? > > > > ----- Original Message ---- > From: Michael Ramchand <michael at ramchand.net> > To: Grant Lowe <glowe at sbcglobal.net> > Cc: zfs-discuss at opensolaris.org > Sent: Tuesday, March 17, 2009 12:40:53 AM > Subject: Re: [zfs-discuss] Disk usage > > Grant Lowe wrote: > >> Hey all, >> >> I have a question/puzzle with zfs. See the following: >> >> bash-3.00# df -h | grep d25 ; zfs list | grep d25 >> >> FILESYSTEM SIZE USED AVAIL CAPACITY MOUNTED ON >> r12_data/d25 *659G* 40G *63G* 39% /opt/d25/oakwc12 >> df -h says the d25 file system is 659GB?; 40GB used and 63GB available? >> r12_data/d24 42G 40G 2.1G 95% /opt/d24/oakwcr12 >> >> NAME USED AVAIL REFER MOUNTPOINT >> r12_data/d25 760K *62.7G* 39.9G /opt/d25/oakwc12 >> zfs list says the db25 file system has 63GB available? >> r12_data/d24 39.9G 2.14G 39.9G /opt/d24/oakwcr12 >> >> >> Shouldn''t the new filesystem (d25) size be what the clone was allocated? >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >> >> > Hi Grant, > > We''d need more info than that to figure what''s actually going on. > > Is d25 a clone of something? If so what? Can we see the specs of that as > well. > > Does d25 have any reservations or a quota? > > What does zfs list of r12_data show? > > Do you have snapshots? zfs list -t all will show you them. > > Finally, the clone will only be the size of it''s delta from the source. > > HTH > >
Sorry, no , I assume you are on Sol 10.
o the value "space" to display space usage
properties on file systems and volumes.
This is a shortcut for "-o
name,avail,used,usedsnap,usedds,
usedrefreserv,usedchild -t
filesystem,volume".
Grant Lowe wrote:> If you meant available, here''s the output of that:
>
> bash-3.00# zfs list -o available r12_data
> AVAIL
> 62.7G
> bash-3.00# zfs list -o available r12_data/d24
> AVAIL
> 2.14G
> bash-3.00# zfs list -o available r12_data/d25
> AVAIL
> 62.7G
> bash-3.00#
>
>
>
>
> ----- Original Message ----
> From: Michael Ramchand <michael at ramchand.net>
> To: Grant Lowe <glowe at sbcglobal.net>
> Cc: zfs-discuss at opensolaris.org
> Sent: Tuesday, March 17, 2009 8:32:49 AM
> Subject: Re: [zfs-discuss] Disk usage
>
> Well, it is kinda confusing...
>
> In short, df -h will always return the size of the WHOLE pool for
"size"
> (unless you''ve set a quota on the dataset in which case it says
that),
> the amount of space that particular dataset is using for "used",
and the
> total amount of free space on the WHOLE pool for "avail" (unless
you''ve
> got a quota or reservation set).
>
> So in your original mail. 695G is the size of the DATA on the r12_data
> pool. zfs list r12_data says you are using 596G. I think this is the
> RAW capacity used, and I reckon you are using compression. (zfs get
> compressratio r12_data will give you something like 1.1).
>
> However, df -h of r12_data/d24 should have the identical 1st and 3rd
> fields, but they don''t. (Could you re-run?)
>
> Same goes for the zfs list commands.
>
> Could you try doing zfs list -o space to get a fuller breakdown of how
> the space is being used.
>
> Mike
>
>
>
> Grant Lowe wrote:
>
>> Hi Mike,
>>
>> Yes, d25 is a clone of d24. Here are some data points about it:
>>
>> bash-3.00# zfs get reservation r12_data/d25
>> NAME PROPERTY VALUE SOURCE
>> r12_data/d25 reservation none default
>> bash-3.00# zfs get quota r12_data/d25
>> NAME PROPERTY VALUE SOURCE
>> r12_data/d25 quota none default
>> bash-3.00#
>> bash-3.00# zfs list r12_data
>> NAME USED AVAIL REFER MOUNTPOINT
>> r12_data 596G 62.7G 24.5K none
>> bash-3.00#
>> bash-3.00# zfs list -t snapshot r12_data/d24 at A
>> NAME USED AVAIL REFER MOUNTPOINT
>> r12_data/d24 at A 904K - 39.9G -
>> bash-3.00#
>>
>> Thanks for the response. Did you need any more data points from me?
>>
>>
>>
>> ----- Original Message ----
>> From: Michael Ramchand <michael at ramchand.net>
>> To: Grant Lowe <glowe at sbcglobal.net>
>> Cc: zfs-discuss at opensolaris.org
>> Sent: Tuesday, March 17, 2009 12:40:53 AM
>> Subject: Re: [zfs-discuss] Disk usage
>>
>> Grant Lowe wrote:
>>
>>
>>> Hey all,
>>>
>>> I have a question/puzzle with zfs. See the following:
>>>
>>> bash-3.00# df -h | grep d25 ; zfs list | grep d25
>>>
>>> FILESYSTEM SIZE USED AVAIL CAPACITY MOUNTED ON
>>> r12_data/d25 *659G* 40G *63G* 39%
/opt/d25/oakwc12
>>> df -h says the d25 file system is 659GB?; 40GB used and 63GB
available?
>>> r12_data/d24 42G 40G 2.1G 95%
/opt/d24/oakwcr12
>>>
>>> NAME USED AVAIL REFER MOUNTPOINT
>>> r12_data/d25 760K *62.7G* 39.9G /opt/d25/oakwc12
>>> zfs list says the db25 file system has 63GB available?
>>> r12_data/d24 39.9G 2.14G 39.9G /opt/d24/oakwcr12
>>>
>>>
>>> Shouldn''t the new filesystem (d25) size be what the clone
was allocated?
>>> _______________________________________________
>>> zfs-discuss mailing list
>>> zfs-discuss at opensolaris.org
>>> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>>>
>>>
>>>
>> Hi Grant,
>>
>> We''d need more info than that to figure what''s
actually going on.
>>
>> Is d25 a clone of something? If so what? Can we see the specs of that
as
>> well.
>>
>> Does d25 have any reservations or a quota?
>>
>> What does zfs list of r12_data show?
>>
>> Do you have snapshots? zfs list -t all will show you them.
>>
>> Finally, the clone will only be the size of it''s delta from
the source.
>>
>> HTH
>>
>>
>>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090317/2d78ea8b/attachment.bin>
Another newbie question: I have a new system with zfs. I create a directory: bash-3.00# mkdir -p /opt/mis/oracle/data/db1 I do my zpool: bash-3.00# zpool create -f oracle c2t5006016B306005AAd0 c2t5006016B306005AAd1 c2t5006016B306005AAd3 c2t5006016B306005AAd4 c2t5006016B306005AAd5 c2t5006016B306005AAd6 c2t5006016B306005AAd7 c2t5006016B306005AAd8 c2t5006016B306005AAd9 c2t5006016B306005AAd10 c2t5006016B306005AAd11 c2t5006016B306005AAd12 c2t5006016B306005AAd13 c2t5006016B306005AAd14 c2t5006016B306005AAd15 c2t5006016B306005AAd16 c2t5006016B306005AAd17 c2t5006016B306005AAd18 c2t5006016B306005AAd19 bash-3.00# zfs create oracle/prd_data bash-3.00# zfs create -b 8192 -V 44Gb oracle/prd_data/db1 I''m trying to set a mountpoint. But trying to mount it doesn''t work. bash-3.00# zfs list NAME USED AVAIL REFER MOUNTPOINT oracle 44.0G 653G 25.5K /oracle oracle/prd_data 44.0G 653G 24.5K /oracle/prd_data oracle/prd_data/db1 22.5K 697G 22.5K - bash-3.00# zfs set mountpoint=/opt/mis/oracle/data/db1 oracle/prd_data/db1 cannot set property for ''oracle/prd_data/db1'': ''mountpoint'' does not apply to datasets of this type bash-3.00# What''s the correct syntax?
Grant, If I''m following correctly, you can''t mount a ZFS resource outside of the pool from which the resource resides. Is this a UFS directory, here: # mkdir -p /opt/mis/oracle/data/db1 What are you trying to do? Cindy Grant Lowe wrote:> Another newbie question: > > I have a new system with zfs. I create a directory: > > bash-3.00# mkdir -p /opt/mis/oracle/data/db1 > > I do my zpool: > > bash-3.00# zpool create -f oracle c2t5006016B306005AAd0 c2t5006016B306005AAd1 c2t5006016B306005AAd3 c2t5006016B306005AAd4 c2t5006016B306005AAd5 c2t5006016B306005AAd6 c2t5006016B306005AAd7 c2t5006016B306005AAd8 c2t5006016B306005AAd9 c2t5006016B306005AAd10 c2t5006016B306005AAd11 c2t5006016B306005AAd12 c2t5006016B306005AAd13 c2t5006016B306005AAd14 c2t5006016B306005AAd15 c2t5006016B306005AAd16 c2t5006016B306005AAd17 c2t5006016B306005AAd18 c2t5006016B306005AAd19 > bash-3.00# zfs create oracle/prd_data > bash-3.00# zfs create -b 8192 -V 44Gb oracle/prd_data/db1 > > I''m trying to set a mountpoint. But trying to mount it doesn''t work. > > bash-3.00# zfs list > NAME USED AVAIL REFER MOUNTPOINT > oracle 44.0G 653G 25.5K /oracle > oracle/prd_data 44.0G 653G 24.5K /oracle/prd_data > oracle/prd_data/db1 22.5K 697G 22.5K - > bash-3.00# zfs set mountpoint=/opt/mis/oracle/data/db1 oracle/prd_data/db1 > cannot set property for ''oracle/prd_data/db1'': ''mountpoint'' does not apply to datasets of this type > bash-3.00# > > What''s the correct syntax? > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Ok, Cindy. Thanks. I would like to have one big pool and divide it into separate file systems for an Oracle database. What I had before was a separate pool for each file system. So does it look I have to go back to what I had before? ----- Original Message ---- From: "Cindy.Swearingen at Sun.COM" <Cindy.Swearingen at Sun.COM> To: Grant Lowe <glowe at sbcglobal.net> Cc: zfs-discuss at opensolaris.org Sent: Tuesday, March 17, 2009 2:20:18 PM Subject: Re: [zfs-discuss] Mounting zfs file systems Grant, If I''m following correctly, you can''t mount a ZFS resource outside of the pool from which the resource resides. Is this a UFS directory, here: # mkdir -p /opt/mis/oracle/data/db1 What are you trying to do? Cindy Grant Lowe wrote:> Another newbie question: > > I have a new system with zfs. I create a directory: > > bash-3.00# mkdir -p /opt/mis/oracle/data/db1 > > I do my zpool: > > bash-3.00# zpool create -f oracle c2t5006016B306005AAd0 c2t5006016B306005AAd1 c2t5006016B306005AAd3 c2t5006016B306005AAd4 c2t5006016B306005AAd5 c2t5006016B306005AAd6 c2t5006016B306005AAd7 c2t5006016B306005AAd8 c2t5006016B306005AAd9 c2t5006016B306005AAd10 c2t5006016B306005AAd11 c2t5006016B306005AAd12 c2t5006016B306005AAd13 c2t5006016B306005AAd14 c2t5006016B306005AAd15 c2t5006016B306005AAd16 c2t5006016B306005AAd17 c2t5006016B306005AAd18 c2t5006016B306005AAd19 > bash-3.00# zfs create oracle/prd_data > bash-3.00# zfs create -b 8192 -V 44Gb oracle/prd_data/db1 > > I''m trying to set a mountpoint. But trying to mount it doesn''t work. > > bash-3.00# zfs list > NAME USED AVAIL REFER MOUNTPOINT > oracle 44.0G 653G 25.5K /oracle > oracle/prd_data 44.0G 653G 24.5K /oracle/prd_data > oracle/prd_data/db1 22.5K 697G 22.5K - > bash-3.00# zfs set mountpoint=/opt/mis/oracle/data/db1 oracle/prd_data/db1 > cannot set property for ''oracle/prd_data/db1'': ''mountpoint'' does not apply to datasets of this type > bash-3.00# > > What''s the correct syntax? > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
no, this is an incorrect diagnosis. The problem is that by using the -V option, you created a volume, not a file system. That is, you created a raw device. You could then newfs a ufs file system within the volume, but that is almost certainly not what you want. Don''t use -V when you create the oracle/prd_data/db1 dataset. Then it will be a mountable file system. You will need to give it a mount point however by setting the mountpoint property, since the default mountpoint won''t be what you want. Lori On 03/17/09 15:45, Grant Lowe wrote:> Ok, Cindy. Thanks. I would like to have one big pool and divide it into separate file systems for an Oracle database. What I had before was a separate pool for each file system. So does it look I have to go back to what I had before? > > > > ----- Original Message ---- > From: "Cindy.Swearingen at Sun.COM" <Cindy.Swearingen at Sun.COM> > To: Grant Lowe <glowe at sbcglobal.net> > Cc: zfs-discuss at opensolaris.org > Sent: Tuesday, March 17, 2009 2:20:18 PM > Subject: Re: [zfs-discuss] Mounting zfs file systems > > Grant, > > If I''m following correctly, you can''t mount a ZFS resource > outside of the pool from which the resource resides. > > Is this a UFS directory, here: > > # mkdir -p /opt/mis/oracle/data/db1 > > What are you trying to do? > > Cindy > > Grant Lowe wrote: > >> Another newbie question: >> >> I have a new system with zfs. I create a directory: >> >> bash-3.00# mkdir -p /opt/mis/oracle/data/db1 >> >> I do my zpool: >> >> bash-3.00# zpool create -f oracle c2t5006016B306005AAd0 c2t5006016B306005AAd1 c2t5006016B306005AAd3 c2t5006016B306005AAd4 c2t5006016B306005AAd5 c2t5006016B306005AAd6 c2t5006016B306005AAd7 c2t5006016B306005AAd8 c2t5006016B306005AAd9 c2t5006016B306005AAd10 c2t5006016B306005AAd11 c2t5006016B306005AAd12 c2t5006016B306005AAd13 c2t5006016B306005AAd14 c2t5006016B306005AAd15 c2t5006016B306005AAd16 c2t5006016B306005AAd17 c2t5006016B306005AAd18 c2t5006016B306005AAd19 >> bash-3.00# zfs create oracle/prd_data >> bash-3.00# zfs create -b 8192 -V 44Gb oracle/prd_data/db1 >> >> I''m trying to set a mountpoint. But trying to mount it doesn''t work. >> >> bash-3.00# zfs list >> NAME USED AVAIL REFER MOUNTPOINT >> oracle 44.0G 653G 25.5K /oracle >> oracle/prd_data 44.0G 653G 24.5K /oracle/prd_data >> oracle/prd_data/db1 22.5K 697G 22.5K - >> bash-3.00# zfs set mountpoint=/opt/mis/oracle/data/db1 oracle/prd_data/db1 >> cannot set property for ''oracle/prd_data/db1'': ''mountpoint'' does not apply to datasets of this type >> bash-3.00# >> >> What''s the correct syntax? >> >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >> > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090317/9d6a2d7a/attachment.html>
On Mar 17, 2009, at 4:45 PM, Grant Lowe wrote:>> >> bash-3.00# zfs create -b 8192 -V 44Gb oracle/prd_data/db1 >> >> I''m trying to set a mountpoint. But trying to mount it doesn''t work. >> >> bash-3.00# zfs list >> NAME USED AVAIL REFER MOUNTPOINT >> oracle 44.0G 653G 25.5K /oracle >> oracle/prd_data 44.0G 653G 24.5K /oracle/prd_data >> oracle/prd_data/db1 22.5K 697G 22.5K - >> bash-3.00# zfs set mountpoint=/opt/mis/oracle/data/db1 oracle/ >> prd_data/db1 >> cannot set property for ''oracle/prd_data/db1'': ''mountpoint'' does >> not apply to datasets of this typeThe issue is, you can''t set a mountpoint on a zvol, there''s no filesystem on there yet. Once you''ve created whatever (non-ZFS) filesystem on that zvol, then you can either mount it manually or set up an entry in /etc/vfstab. -Chris
Great explanation. Thanks, Lori. ________________________________ From: Lori Alt <Lori.Alt at Sun.COM> To: Grant Lowe <glowe at sbcglobal.net> Cc: Cindy.Swearingen at Sun.COM; zfs-discuss at opensolaris.org Sent: Tuesday, March 17, 2009 2:52:04 PM Subject: Re: [zfs-discuss] Mounting zfs file systems no, this is an incorrect diagnosis. The problem is that by using the -V option, you created a volume, not a file system. That is, you created a raw device. You could then newfs a ufs file system within the volume, but that is almost certainly not what you want. Don''t use -V when you create the oracle/prd_data/db1 dataset. Then it will be a mountable file system. You will need to give it a mount point however by setting the mountpoint property, since the default mountpoint won''t be what you want. Lori On 03/17/09 15:45, Grant Lowe wrote: Ok, Cindy. Thanks. I would like to have one big pool and divide it into separate file systems for an Oracle database. What I had before was a separate pool for each file system. So does it look I have to go back to what I had before? ----- Original Message ---- From: "Cindy.Swearingen at Sun.COM" <Cindy.Swearingen at Sun.COM> To: Grant Lowe <glowe at sbcglobal.net> Cc: zfs-discuss at opensolaris.org Sent: Tuesday, March 17, 2009 2:20:18 PM Subject: Re: [zfs-discuss] Mounting zfs file systems Grant, If I''m following correctly, you can''t mount a ZFS resource outside of the pool from which the resource resides. Is this a UFS directory, here: # mkdir -p /opt/mis/oracle/data/db1 What are you trying to do? Cindy Grant Lowe wrote: Another newbie question: I have a new system with zfs. I create a directory: bash-3.00# mkdir -p /opt/mis/oracle/data/db1 I do my zpool: bash-3.00# zpool create -f oracle c2t5006016B306005AAd0 c2t5006016B306005AAd1 c2t5006016B306005AAd3 c2t5006016B306005AAd4 c2t5006016B306005AAd5 c2t5006016B306005AAd6 c2t5006016B306005AAd7 c2t5006016B306005AAd8 c2t5006016B306005AAd9 c2t5006016B306005AAd10 c2t5006016B306005AAd11 c2t5006016B306005AAd12 c2t5006016B306005AAd13 c2t5006016B306005AAd14 c2t5006016B306005AAd15 c2t5006016B306005AAd16 c2t5006016B306005AAd17 c2t5006016B306005AAd18 c2t5006016B306005AAd19 bash-3.00# zfs create oracle/prd_data bash-3.00# zfs create -b 8192 -V 44Gb oracle/prd_data/db1 I''m trying to set a mountpoint. But trying to mount it doesn''t work. bash-3.00# zfs list NAME USED AVAIL REFER MOUNTPOINT oracle 44.0G 653G 25.5K /oracle oracle/prd_data 44.0G 653G 24.5K /oracle/prd_data oracle/prd_data/db1 22.5K 697G 22.5K - bash-3.00# zfs set mountpoint=/opt/mis/oracle/data/db1 oracle/prd_data/db1 cannot set property for ''oracle/prd_data/db1'': ''mountpoint'' does not apply to datasets of this type bash-3.00# What''s the correct syntax? _______________________________________________ zfs-discuss mailing list zfs-discuss at opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss _______________________________________________ zfs-discuss mailing list zfs-discuss at opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090317/e992b21c/attachment.html>