How can I test if a file on ZFS has holes, i.e. is a sparse file,
using the C api?
Olga
--
? ? ? ,?? _? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? _?? ,
? ?? { \/`o;====-? ? Olga Kryzhanovska?? -====;o`\/ }
.----''-/`-/? ?? olga.kryzhanovska at gmail.com?? \-`\-''----.
?`''-..-| /? ? ?? http://twitter.com/fleyta? ?? \ |-..-''`
? ? ? /\/\? ?? Solaris/BSD//C/C++ programmer?? /\/\
? ? ? `--`? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? `--`
2012/3/26 ????? ???????????? <olga.kryzhanovska at gmail.com>:> How can I test if a file on ZFS has holes, i.e. is a sparse file, > using the C api?See SEEK_HOLE in lseek(2). -- Mike Gerdts http://mgerdts.blogspot.com/
Mike, I was hoping that some one has a complete example for a bool has_file_one_or_more_holes(const char *path) function. Olga 2012/3/26 Mike Gerdts <mgerdts at gmail.com>:> 2012/3/26 ????? ???????????? <olga.kryzhanovska at gmail.com>: >> How can I test if a file on ZFS has holes, i.e. is a sparse file, >> using the C api? > > See SEEK_HOLE in lseek(2). > > -- > Mike Gerdts > http://mgerdts.blogspot.com/-- ? ? ? ,?? _? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? _?? , ? ?? { \/`o;====-? ? Olga Kryzhanovska?? -====;o`\/ } .----''-/`-/? ?? olga.kryzhanovska at gmail.com?? \-`\-''----. ?`''-..-| /? ? ?? http://twitter.com/fleyta? ?? \ |-..-''` ? ? ? /\/\? ?? Solaris/BSD//C/C++ programmer?? /\/\ ? ? ? `--`? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? `--`
????? ???????????? <olga.kryzhanovska at gmail.com> wrote:> How can I test if a file on ZFS has holes, i.e. is a sparse file, > using the C api?See star ..... ftp://ftp.berlios.de/pub/star/ or http://hg.berlios.de/repos/schillix-on/file/e3829115a7a4/usr/src/cmd/star/hole.c The interface was defined for star in September 2004, star added support in May 2005 after the interface was implemented. J?rg -- EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin js at cs.tu-berlin.de (uni) joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
I just played and knocked this up (note the stunning lack of comments,
missing optarg processing, etc)...
Give it a list of files to check...
#define _FILE_OFFSET_BITS 64
#include<sys/types.h>
#include<unistd.h>
#include<stdio.h>
#include<sys/stat.h>
#include<fcntl.h>
int
main(int argc, char **argv)
{
int i;
for (i = 1; i< argc; i++) {
int fd;
fd = open(argv[i], O_RDONLY);
if (fd< 0) {
perror(argv[i]);
} else {
off_t eof;
off_t hole;
if (((eof = lseek(fd, 0, SEEK_END))< 0) ||
lseek(fd, 0, SEEK_SET)< 0) {
perror(argv[i]);
} else if (eof == 0) {
printf("%s: empty\n", argv[i]);
} else {
hole = lseek(fd, 0, SEEK_HOLE);
if (hole< 0) {
perror(argv[i]);
} else if (hole< eof) {
printf("%s: sparse\n",
argv[i]);
} else {
printf("%s: not sparse\n",
argv[i]);
}
}
close(fd);
}
}
return 0;
}
On 03/26/12 10:06 PM, ????? ???????????? wrote:> Mike, I was hoping that some one has a complete example for a bool
> has_file_one_or_more_holes(const char *path) function.
>
> Olga
>
> 2012/3/26 Mike Gerdts<mgerdts at gmail.com>:
>> 2012/3/26 ????? ????????????<olga.kryzhanovska at gmail.com>:
>>> How can I test if a file on ZFS has holes, i.e. is a sparse file,
>>> using the C api?
>> See SEEK_HOLE in lseek(2).
>>
>> --
>> Mike Gerdts
>> http://mgerdts.blogspot.com/
>
>
On Mon, 26 Mar 2012, Andrew Gabriel wrote:> I just played and knocked this up (note the stunning lack of comments, > missing optarg processing, etc)... > Give it a list of files to check...This is a cool program, but programmers were asking (and answering) this same question 20+ years ago before there was anything like SEEK_HOLE. If file space usage is less than file directory size then it must contain a hole. Even for compressed files, I am pretty sure that Solaris reports the uncompressed space usage. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
On Mon, Mar 26, 2012 at 6:18 PM, Bob Friesenhahn <bfriesen at simple.dallas.tx.us> wrote:> On Mon, 26 Mar 2012, Andrew Gabriel wrote: > >> I just played and knocked this up (note the stunning lack of comments, >> missing optarg processing, etc)... >> Give it a list of files to check... > > > This is a cool program, but programmers were asking (and answering) this > same question 20+ years ago before there was anything like SEEK_HOLE. > > If file space usage is less than file directory size then it must contain a > hole. ?Even for compressed files, I am pretty sure that Solaris reports the > uncompressed space usage.That''s not the case. # zfs create -o compression=on rpool/junk # perl -e ''print "foo" x 100000''> /rpool/junk/foo # ls -ld /rpool/junk/foo -rw-r--r-- 1 root root 300000 Mar 26 18:25 /rpool/junk/foo # du -h /rpool/junk/foo 16K /rpool/junk/foo # truss -t stat -v stat du /rpool/junk/foo ... lstat64("foo", 0x08047C40) = 0 d=0x02B90028 i=8 m=0100644 l=1 u=0 g=0 sz=300000 at = Mar 26 18:25:25 CDT 2012 [ 1332804325.742827733 ] mt = Mar 26 18:25:25 CDT 2012 [ 1332804325.889143166 ] ct = Mar 26 18:25:25 CDT 2012 [ 1332804325.889143166 ] bsz=131072 blks=32 fs=zfs Notice that it says it has 32 512 byte blocks. The mechanism you suggest does work for every other file system that I''ve tried it on. -- Mike Gerdts http://mgerdts.blogspot.com/
On Mar 26, 2012, at 4:18 PM, Bob Friesenhahn wrote:> On Mon, 26 Mar 2012, Andrew Gabriel wrote: > >> I just played and knocked this up (note the stunning lack of comments, missing optarg processing, etc)... >> Give it a list of files to check... > > This is a cool program, but programmers were asking (and answering) this same question 20+ years ago before there was anything like SEEK_HOLE. > > If file space usage is less than file directory size then it must contain a hole. Even for compressed files, I am pretty sure that Solaris reports the uncompressed space usage.+1 Also, prior to ZFS, you could look at the length of the file (ls -l or stat struct st_size) and compare to the size (ls -ls or stat struct st_blocks). If length > size (unit adjusted, rounded up) then there are holes. In ZFS, this can be more difficult, because the size can be larger than the length (!) due to copies. Also, if you have compression enabled, size can be < length. -- richard -- DTrace Conference, April 3, 2012, http://wiki.smartos.org/display/DOC/dtrace.conf ZFS Performance and Training Richard.Elling at RichardElling.com +1-760-896-4422 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20120326/dd9a7d37/attachment.html>
>On Mon, 26 Mar 2012, Andrew Gabriel wrote: > >> I just played and knocked this up (note the stunning lack of comments, >> missing optarg processing, etc)... >> Give it a list of files to check... > >This is a cool program, but programmers were asking (and answering) >this same question 20+ years ago before there was anything like >SEEK_HOLE. > >If file space usage is less than file directory size then it must >contain a hole. Even for compressed files, I am pretty sure that >Solaris reports the uncompressed space usage. >Unfortunately not true with filesystems which compress data. Casper
On Mon, 26 Mar 2012, Mike Gerdts wrote:>> >> If file space usage is less than file directory size then it must contain a >> hole. ?Even for compressed files, I am pretty sure that Solaris reports the >> uncompressed space usage. > > That''s not the case.You are right. I should have tested this prior to posting. :-( Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/