OK, so this may be a little off-topic, but here goes: The reason I switched to OpenSolaris was primarily to take advantage of ZFS''s features when storing my digital imaging collection. I switched from a pretty stock Linux setup, but it left me at one disadvantage. I had been using inotify under Linux to trigger a series of Ruby scripts that would do all the basic ingestion/setup for me (renaming files, converting to DNG, adding bulk metadata). The scripts will run under OpenSolaris, except for the inotify part. Question: Is there a facility similar to inotify that I can use to monitor a directory structure in OpenSolaris/ZFS, such that it will block until a file is modified (added, deleted, etc), and then pass the state along (STDOUT is fine)? One other requirement: inotify can handle subdirectories being added on the fly. So if you use it to monitor, for example, /data/images/incoming, and a /data/images/incoming/100canon directory gets created, then the files under that directory will automatically be monitored as well. Thanks, Paul Archer
On Sun, Oct 25, 2009 at 4:16 PM, Paul Archer <paul at paularcher.org> wrote:> OK, so this may be a little off-topic, but here goes: > The reason I switched to OpenSolaris was primarily to take advantage of > ZFS''s features when storing my digital imaging collection. > > I switched from a pretty stock Linux setup, but it left me at one > disadvantage. I had been using inotify under Linux to trigger a series of > Ruby scripts that would do all the basic ingestion/setup for me (renaming > files, converting to DNG, adding bulk metadata). The scripts will run under > OpenSolaris, except for the inotify part. > > Question: Is there a facility similar to inotify that I can use to monitor a > directory structure in OpenSolaris/ZFS, such that it will block until a file > is modified (added, deleted, etc), and then pass the state along (STDOUT is > fine)? One other requirement: inotify can handle subdirectories being added > on the fly. So if you use it to monitor, for example, /data/images/incoming, > and a /data/images/incoming/100canon directory gets created, then the files > under that directory will automatically be monitored as well.Paul, while there is no inotify for Solaris, there are similar technologies available. Check port_create(3C) and gam_server(1) -- Regards, Cyril
5:12pm, Cyril Plisko wrote:>> Question: Is there a facility similar to inotify that I can use to monitor a >> directory structure in OpenSolaris/ZFS, such that it will block until a file >> is modified (added, deleted, etc), and then pass the state along (STDOUT is >> fine)? One other requirement: inotify can handle subdirectories being added >> on the fly. So if you use it to monitor, for example, /data/images/incoming, >> and a /data/images/incoming/100canon directory gets created, then the files >> under that directory will automatically be monitored as well. > > > while there is no inotify for Solaris, there are similar technologies available. > > Check port_create(3C) and gam_server(1) >I can''t find much on gam_server on Solaris (couldn''t find too much on it at all, really), and port_create is apparently a system call. (I''m not a developer--if I can''t write it in BASH, Perl, or Ruby, I can''t write it.) I appreciate the suggestions, but I need something a little more pret-a-porte. Does anyone have any dtrace experience? I figure this could probably be done with dtrace, but I don''t know enough about it to write a dtrace script (although I may learn if that turns out to be the best way to go). I was hoping that there''d be a script out there already, but I haven''t turned up anything yet. Paul
Paul Being a script hacker like you the only kludge I can think of. A script that does something like ls > /tmp/foo sleep ls /tmp/foo.new diff /tmp/foo /tmp/foo.new > /tmp/files_that_have_changed mv /tmp/foo.new /tmp/foo Or you might be able to knock something up with bart nd zfs snapshots. I did write this which may help????? #!/bin/sh #set -x # Note: No implied warranty etc. applies. # Don''t cry if it does not work. I''m an SE not a programmer! # ################################################################### # # Version 29th Jan. 2009 # # GOAL: Show what files have changed between snapshots # # But of course it could be any two directories!! # ################################################################### # ## Set some variables # SCRIPT_NAME=$0 FILESYSTEM=$1 SNAPSHOT=$2 FILESYSTEM_BART_FILE=/tmp/filesystem.$$ SNAPSHOT_BART_FILE=/tmp/snapshot.$$ CHANGED_FILES=/tmp/changes.$$ ## Declare some commands (just in case PATH is wrong, like cron) # BART=/bin/bart ## Usage # Usage() { echo "" echo "" echo "Usage: $SCRIPT_NAME -q filesystem snapshot " echo "" echo " -q will stop all echos and just list the changes" echo "" echo "Examples" echo " $SCRIPT_NAME /home/fred /home/.zfs/snapshot/fred " echo " $SCRIPT_NAME . /home/.zfs/snapshot/fred " echo "" echo "" exit 1 } ########### Main Part ################### ## Check Usage # if [ $# -ne 2 ]; then Usage fi ## Check we have different directories # if [ "$1" = "$2" ]; then Usage fi ## Handle dot # if [ "$FILESYSTEM" = "." ]; then cd $FILESYSTEM ; FILESYSTEM=`pwd` fi if [ "$SNAPSHOT" = "." ]; then cd $SNAPSHOT ; SNAPSHOT=`pwd` fi ## Check the filesystems exists It should be a directory # and it should have some files # for FS in "$FILESYSTEM" "$SNAPSHOT" do if [ ! -d "$FS" ]; then echo "" echo "ERROR file system $FS does not exist" echo "" exit 1 fi if [ X"`/bin/ls "$FS"`" = "X" ]; then echo "" echo "ERROR file system $FS seems to be empty" exit 1 echo "" fi done ## Create the bart files # echo "" echo "Creating bart file for $FILESYSTEM can take a while......" cd "$FILESYSTEM" ; $BART create -R . > $FILESYSTEM_BART_FILE echo "" echo "Creating bart file for $SNAPSHOT can take a while......" cd "$SNAPSHOT" ; $BART create -R . > $SNAPSHOT_BART_FILE ## Compare them and report the diff # echo "" echo "Changes...." echo "" $BART compare -p $FILESYSTEM_BART_FILE $SNAPSHOT_BART_FILE | awk ''{print $1}'' > $CHANGED_FILES /bin/more $CHANGED_FILES echo "" echo "" echo "" ## Tidy kiwi # /bin/rm $FILESYSTEM_BART_FILE /bin/rm $SNAPSHOT_BART_FILE /bin/rm $CHANGED_FILES exit 0 Paul Archer wrote: 5:12pm, Cyril Plisko wrote: Question: Is there a facility similar to inotify that I can use to monitor a directory structure in OpenSolaris/ZFS, such that it will block until a file is modified (added, deleted, etc), and then pass the state along (STDOUT is fine)? One other requirement: inotify can handle subdirectories being added on the fly. So if you use it to monitor, for example, /data/images/incoming, and a /data/images/incoming/100canon directory gets created, then the files under that directory will automatically be monitored as well. while there is no inotify for Solaris, there are similar technologies available. Check port_create(3C) and gam_server(1) all, really), and port_create is apparently a system call. (I''m not a developer--if I can''t write it in BASH, Perl, or Ruby, I can''t write it.) I appreciate the suggestions, but I need something a little more pret-a-porte. Does anyone have any dtrace experience? I figure this could probably be done with dtrace, but I don''t know enough about it to write a dtrace script (although I may learn if that turns out to be the best way to go). I was hoping that there''d be a script out there already, but I haven''t turned up anything yet. Paul _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org mail.opensolaris.org/mailman/listinfo/zfs-discuss eagle.co.nz This email is confidential and may be legally privileged. If received in error please destroy and immediately notify us. _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org mail.opensolaris.org/mailman/listinfo/zfs-discuss
On 10/25/09 5:38 PM, Paul Archer wrote:> 5:12pm, Cyril Plisko wrote:>> while there is no inotify for Solaris, there are similar technologies >> available. >> >> Check port_create(3C) and gam_server(1) >> > I can''t find much on gam_server on Solaris (couldn''t find too much on it > at all, really), and port_create is apparently a system call. (I''m not a > developer--if I can''t write it in BASH, Perl, or Ruby, I can''t write it.) > I appreciate the suggestions, but I need something a little more > pret-a-porte.Your Google-fu needs work ;-) Main Gamin page: gnome.org/~veillard/gamin/index.html Perl module: search.cpan.org/~garnacho/Sys-Gamin-0.1/lib/Sys/Gamin.pm libev (and the EV perl module) will hide port_create() from you, but from a quick skim it may not have the functionality you want. -- Carson
How about a consumer for gvfs-monitor-dir(1) or gvfs-monitor- file(1)? :-) -- richard On Oct 26, 2009, at 3:17 PM, Carson Gaspar wrote:> On 10/25/09 5:38 PM, Paul Archer wrote: >> 5:12pm, Cyril Plisko wrote: > >>> while there is no inotify for Solaris, there are similar >>> technologies >>> available. >>> >>> Check port_create(3C) and gam_server(1) >>> >> I can''t find much on gam_server on Solaris (couldn''t find too much >> on it >> at all, really), and port_create is apparently a system call. (I''m >> not a >> developer--if I can''t write it in BASH, Perl, or Ruby, I can''t >> write it.) >> I appreciate the suggestions, but I need something a little more >> pret-a-porte. > > Your Google-fu needs work ;-) > > Main Gamin page: gnome.org/~veillard/gamin/index.html > Perl module: search.cpan.org/~garnacho/Sys-Gamin-0.1/lib/Sys/Gamin.pm > > libev (and the EV perl module) will hide port_create() from you, but > from a quick skim it may not have the functionality you want. > > -- > Carson > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > mail.opensolaris.org/mailman/listinfo/zfs-discuss
On 10/26/09 3:31 PM, Richard Elling wrote:> How about a consumer for gvfs-monitor-dir(1) or gvfs-monitor-file(1)? :-)The docs are... ummm... "skimpy" is being rather polite. The docs I can find via Google say that they will launch some random unspecified daemons via d-bus (I assume gvfsd ans gvfsd-${accessmethod}). This implies that you need to start a d-bus session to use them. gvfsd (no man page or docs of any kind that I can find) is linked against libgio, which has unresolved symbols against port_create() and friends, which is a good sign that they don''t just poll. -- Carson
On Oct 26, 2009, at 3:56 PM, Carson Gaspar wrote:> On 10/26/09 3:31 PM, Richard Elling wrote: >> How about a consumer for gvfs-monitor-dir(1) or gvfs-monitor- >> file(1)? :-) > > The docs are... ummm... "skimpy" is being rather polite. The docs I > can find via Google say that they will launch some random > unspecified daemons via d-bus (I assume gvfsd ans gvfsd-$ > {accessmethod}). This implies that you need to start a d-bus session > to use them. gvfsd (no man page or docs of any kind that I can find) > is linked against libgio, which has unresolved symbols against > port_create() and friends, which is a good sign that they don''t just > poll.I haven''t dug into the details, and this has nothing to do with ZFS, but observe the following example: $ gvfs-monitor-file /zwimming/whee File Monitor Event: File = /zwimming/whee Event = ATTRIB CHANGED File Monitor Event: File = /zwimming/whee Event = CHANGED File Monitor Event: File = /zwimming/whee Event = CHANGES_DONE_HINT ...while in another tab I simply did "touch /zwimming/whee" gvfs-* commands seem more suitable for scripts or programs than humans. But it doesn''t look like a difficult script to write in any of the scripting languages. I presume the gvfs-* commands will be more portable than inotify & others...? -- richard
>> I can''t find much on gam_server on Solaris (couldn''t find too much on it >> at all, really), and port_create is apparently a system call. (I''m not a >> developer--if I can''t write it in BASH, Perl, or Ruby, I can''t write >> it.) >> I appreciate the suggestions, but I need something a little more >> pret-a-porte. > > Your Google-fu needs work ;-) > > Main Gamin page: gnome.org/~veillard/gamin/index.htmlActually, I found this page, which has this gem: "At this point Gamin is fairly tied to Linux, portability is not a primary goal at this stage but if you have portability patches they are welcome." Unfortunately, I''m trying for a Solaris solution. I already had a Linux solution (the ''inotify'' I started out with). Paul
On 10/26/09 5:33 PM, paul at paularcher.org wrote:>>> I can''t find much on gam_server on Solaris (couldn''t find too much on it >>> at all, really), and port_create is apparently a system call. (I''m not a >>> developer--if I can''t write it in BASH, Perl, or Ruby, I can''t write >>> it.) >>> I appreciate the suggestions, but I need something a little more >>> pret-a-porte. >> >> Your Google-fu needs work ;-) >> >> Main Gamin page: gnome.org/~veillard/gamin/index.html > > Actually, I found this page, which has this gem: "At this point Gamin is > fairly tied to Linux, portability is not a primary goal at this stage but > if you have portability patches they are welcome."Much has changed since that text was written, including support for the event completion framework (port_create() and friends, introduced with Sol 10) on Solaris, thus the recommendation for gam_server / gamin. $ nm /usr/lib/gam_server | grep port_create [458] | 134589544| 0|FUNC |GLOB |0 |UNDEF |port_create> Unfortunately, I''m trying for a Solaris solution. I already had a Linux > solution (the ''inotify'' I started out with).And we''re on a Solaris mailing list, trying to give you solutions that work on Solaris. Don''t believe everything you read on the Internet ;-) -- Carson
On Oct 26, 2009, at 20:42, Carson Gaspar wrote:>> Unfortunately, I''m trying for a Solaris solution. I already had a >> Linux >> solution (the ''inotify'' I started out with). > > And we''re on a Solaris mailing list, trying to give you solutions > that work on Solaris. Don''t believe everything you read on the > Internet ;-)Gamin is also more portable than ''inotify'', so you could have one set of code for multiple platforms: freshports.org/search.php?query=gamin
I haven''t tried this, but this must be very easy with dtrace. How come no one mentioned it yet? :) You would have to monitor some specific syscalls... -- This message posted from opensolaris.org
On Mon, Oct 26, 2009 at 08:53:50PM -0700, Anil wrote:> I haven''t tried this, but this must be very easy with dtrace. How come > no one mentioned it yet? :) You would have to monitor some specific > syscalls...DTrace is not reliable in this sense: it will drop events rather than overburden the system. Also, system calls are not the only thing you want to watch for -- you should really trace the VFS/fop rather than syscalls for this. In any case, port_create(3C) and gamin are the way forward. port_create(3C) is rather easy to use. Searching the web for PORT_SOURCE_FILE you''ll find useful docs like: blogs.sun.com/praks/entry/file_events_notification which has example code too. I do think it''d be useful to have command-line utility in core Solaris that uses this facility, something like the example in Prakash''s blog (which, incidentally, _works_), but perhaps a bit more complete. Nico --
I''d hoped this script would work for me as a "snapshot diff" script, but it seems that bart doesn''t play well with large filesystems (don''t know the cutoff, but my zfs pools (other than rpool) are all well over 4TB). ''bart create'' fails immediately with a "Value too large for defined data type" error, and this is in fact mentioned in the Solaris 10 10/09 release notes: ===Possible Error With 32-bit Applications Getting File System State on Large File Systems (6468905) When run on large file systems, for example ZFS, applications using statvfs(2) or statfs(2) to get information about the state of the file system exhibit an error. The following error message is displayed: Value too large for defined data type Workaround: Applications should use statvfs64() instead. ===from docs.sun.com/app/docs/doc/821-0381/gdzmr?l=en&a=view and in fact, if I invoke bart via truss, I see it calls statvfs() and fails. Way to keep up with the times, Sun! Is there a 64-bit version of bart, or a better recommendation for comparing snapshots? My current backup strategy uses rsync, which I''d like to replace with zfs send/receive, but I need a way to see what changed in the past day. Thanks, Andrew Daugherity Systems Analyst Division of Research & Graduate Studies Texas A&M University>>> Trevor Pretty <trevor_pretty at eagle.co.nz> 10/26/2009 5:16 PM >>>Paul Being a script hacker like you the only kludge I can think of. A script that does something like ls > /tmp/foo sleep ls /tmp/foo.new diff /tmp/foo /tmp/foo.new >/tmp/files_that_have_changed mv /tmp/foo.new /tmp/foo Or you might be able to knock something up with bart nd zfs snapshots. I did write this which may help????? #!/bin/sh #set -x # Note: No implied warranty etc. applies. # Don''t cry if it does not work. I''m an SE not a programmer! # ################################################################### # # Version 29th Jan. 2009 # # GOAL: Show what files have changed between snapshots # # But of course it could be any two directories!! # ################################################################### # ## Set some variables # SCRIPT_NAME=$0 FILESYSTEM=$1 SNAPSHOT=$2 FILESYSTEM_BART_FILE=/tmp/filesystem.$$ SNAPSHOT_BART_FILE=/tmp/snapshot.$$ CHANGED_FILES=/tmp/changes.$$ ## Declare some commands (just in case PATH is wrong, like cron) # BART=/bin/bart ## Usage # Usage() { echo "" echo "" echo "Usage: $SCRIPT_NAME -q filesystem snapshot " echo "" echo " -q will stop all echos and just list the changes" echo "" echo "Examples" echo " $SCRIPT_NAME /home/fred /home/.zfs/snapshot/fred " echo " $SCRIPT_NAME . /home/.zfs/snapshot/fred " echo "" echo "" exit 1 } ########### Main Part ################### ## Check Usage # if [ $# -ne 2 ]; then Usage fi ## Check we have different directories # if [ "$1" = "$2" ]; then Usage fi ## Handle dot # if [ "$FILESYSTEM" = "." ]; then cd $FILESYSTEM ; FILESYSTEM=`pwd` fi if [ "$SNAPSHOT" = "." ]; then cd $SNAPSHOT ; SNAPSHOT=`pwd` fi ## Check the filesystems exists It should be a directory # and it should have some files # for FS in "$FILESYSTEM" "$SNAPSHOT" do if [ ! -d "$FS" ]; then echo "" echo "ERROR file system $FS does not exist" echo "" exit 1 fi if [ X"`/bin/ls "$FS"`" = "X" ]; then echo "" echo "ERROR file system $FS seems to be empty" exit 1 echo "" fi done ## Create the bart files # echo "" echo "Creating bart file for $FILESYSTEM can take a while......" cd "$FILESYSTEM" ; $BART create -R . > $FILESYSTEM_BART_FILE echo "" echo "Creating bart file for $SNAPSHOT can take a while......" cd "$SNAPSHOT" ; $BART create -R . > $SNAPSHOT_BART_FILE ## Compare them and report the diff # echo "" echo "Changes...." echo "" $BART compare -p $FILESYSTEM_BART_FILE $SNAPSHOT_BART_FILE | awk ''{print $1}'' > $CHANGED_FILES /bin/more $CHANGED_FILES echo "" echo "" echo "" ## Tidy kiwi # /bin/rm $FILESYSTEM_BART_FILE /bin/rm $SNAPSHOT_BART_FILE /bin/rm $CHANGED_FILES exit 0
Andrew Daugherity wrote:> if I invoke bart via truss, I see it calls statvfs() and fails. Way to keep up with the times, Sun!% file /bin/truss /bin/amd64/truss /bin/truss: ELF 32-bit LSB executable 80386 Version 1 [FPU], dynamically linked, not stripped, no debugging information available /bin/amd64/truss: ELF 64-bit LSB executable AMD64 Version 1 [SSE2 SSE FXSR CMOV FPU], dynamically linked, not stripped, no debugging information available Rob T
On Mon, Nov 09, 2009 at 03:25:02PM -0700, Robert Thurlow wrote:> Andrew Daugherity wrote: > > >if I invoke bart via truss, I see it calls statvfs() and fails. Way to > >keep up with the times, Sun! > > % file /bin/truss /bin/amd64/truss > > /bin/truss: ELF 32-bit LSB executable 80386 Version 1 [FPU], > dynamically linked, not stripped, no debugging information available > > /bin/amd64/truss: ELF 64-bit LSB executable AMD64 Version 1 [SSE2 > SSE FXSR CMOV FPU], dynamically linked, not stripped, no debugging > information availableI''m pretty sure he means that ''bart'' is failing, not truss. /bin/truss is just a link to /usr/lib/isaexec, which will run the 64-bit version when appropriate. -- Darren
On Nov 9, 2009, at 2:06 PM, Andrew Daugherity wrote:> I''d hoped this script would work for me as a "snapshot diff" script, > but it seems that bart doesn''t play well with large filesystems > (don''t know the cutoff, but my zfs pools (other than rpool) are all > well over 4TB). > > ''bart create'' fails immediately with a "Value too large for defined > data type" error, and this is in fact mentioned in the Solaris 10 > 10/09 release notes: > ===> Possible Error With 32-bit Applications Getting File System State on > Large File Systems (6468905) > > When run on large file systems, for example ZFS, applications using > statvfs(2) or statfs(2) to get information about the state of the > file system exhibit an error. The following error message is > displayed: > > Value too large for defined data type > > Workaround: Applications should use statvfs64() instead. > ===> from > docs.sun.com/app/docs/doc/821-0381/gdzmr?l=en&a=view > > and in fact, if I invoke bart via truss, I see it calls statvfs() > and fails. Way to keep up with the times, Sun! > > > Is there a 64-bit version of bart, or a better recommendation for > comparing snapshots? My current backup strategy uses rsync, which > I''d like to replace with zfs send/receive, but I need a way to see > what changed in the past day.find /filesystem -mtime -1 -- richard> > Thanks, > > > Andrew Daugherity > Systems Analyst > Division of Research & Graduate Studies > Texas A&M University > > > > > > >>>> Trevor Pretty <trevor_pretty at eagle.co.nz> 10/26/2009 5:16 PM >>> > Paul > > Being a script hacker like you the only kludge I can think of. > > A script that does something like > > ls > /tmp/foo > sleep > ls /tmp/foo.new > diff /tmp/foo /tmp/foo.new >/tmp/files_that_have_changed > mv /tmp/foo.new /tmp/foo > > Or you might be able to knock something up with bart nd zfs > snapshots. I did write this which may help????? > > #!/bin/sh > > #set -x > > # Note: No implied warranty etc. applies. > # Don''t cry if it does not work. I''m an SE not a programmer! > # > ################################################################### > # > # Version 29th Jan. 2009 > # > # GOAL: Show what files have changed between snapshots > # > # But of course it could be any two directories!! > # > ################################################################### > # > > ## Set some variables > # > SCRIPT_NAME=$0 > FILESYSTEM=$1 > SNAPSHOT=$2 > FILESYSTEM_BART_FILE=/tmp/filesystem.$$ > SNAPSHOT_BART_FILE=/tmp/snapshot.$$ > CHANGED_FILES=/tmp/changes.$$ > > > ## Declare some commands (just in case PATH is wrong, like cron) > # > BART=/bin/bart > > > ## Usage > # > Usage() > { > echo "" > echo "" > echo "Usage: $SCRIPT_NAME -q filesystem snapshot " > echo "" > echo " -q will stop all echos and just list the changes" > echo "" > echo "Examples" > echo " $SCRIPT_NAME /home/fred /home/.zfs/snapshot/fred " > echo " $SCRIPT_NAME . /home/.zfs/snapshot/fred " > echo "" > echo "" > exit 1 > } > > ########### Main Part ################### > > > ## Check Usage > # > if [ $# -ne 2 ]; then > Usage > fi > > ## Check we have different directories > # > if [ "$1" = "$2" ]; then > Usage > fi > > > ## Handle dot > # > if [ "$FILESYSTEM" = "." ]; then > cd $FILESYSTEM ; FILESYSTEM=`pwd` > fi > if [ "$SNAPSHOT" = "." ]; then > cd $SNAPSHOT ; SNAPSHOT=`pwd` > fi > > ## Check the filesystems exists It should be a directory > # and it should have some files > # > for FS in "$FILESYSTEM" "$SNAPSHOT" > do > if [ ! -d "$FS" ]; then > echo "" > echo "ERROR file system $FS does not exist" > echo "" > exit 1 > fi > if [ X"`/bin/ls "$FS"`" = "X" ]; then > echo "" > echo "ERROR file system $FS seems to be empty" > exit 1 > echo "" > fi > done > > > > ## Create the bart files > # > > echo "" > echo "Creating bart file for $FILESYSTEM can take a while......" > cd "$FILESYSTEM" ; $BART create -R . > $FILESYSTEM_BART_FILE > echo "" > echo "Creating bart file for $SNAPSHOT can take a while......" > cd "$SNAPSHOT" ; $BART create -R . > $SNAPSHOT_BART_FILE > > > ## Compare them and report the diff > # > echo "" > echo "Changes...." > echo "" > $BART compare -p $FILESYSTEM_BART_FILE $SNAPSHOT_BART_FILE | awk > ''{print $1}'' > $CHANGED_FILES > /bin/more $CHANGED_FILES > echo "" > echo "" > echo "" > > ## Tidy kiwi > # > /bin/rm $FILESYSTEM_BART_FILE > /bin/rm $SNAPSHOT_BART_FILE > /bin/rm $CHANGED_FILES > > exit 0 > > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > mail.opensolaris.org/mailman/listinfo/zfs-discuss
>>> Robert Thurlow <robert.thurlow at sun.com> 11/9/2009 4:25 PM >>>% file /bin/truss /bin/amd64/truss /bin/truss: ELF 32-bit LSB executable 80386 Version 1 [FPU], dynamically linked, not stripped, no debugging information available /bin/amd64/truss: ELF 64-bit LSB executable AMD64 Version 1 [SSE2 SSE FXSR CMOV FPU], dynamically linked, not stripped, no debugging information available It doesn''t make any difference if I invoke it with the amd64 truss. The only bart binary I can find on the system (Sol 10u8) is /usr/bin/bart, and it definitely calls statvfs(). Truss log follows at the end. I know all about ''find -mtime ...'', but that doesn''t show which files have been deleted, whereas ''rsync -av --delete --backup-dir=`date +%Y%m%d`'' does. (When users delete files and then need them restored a week later, it''s very helpful to know which day they were deleted, as I can avoid running a find that could take quite a while. I think incremental zfs snapshots are a better strategy but there are little hurdles like this to be crossed.) bart (or something faster than running ''gdiff -qr snap1 snap2'' on a snapshots of a 2.1TB-and-growing FS) seems like a great idea, if I could find a working tool. It looks like dircmp(1) might be a possibility, but I''m open to suggestions. I suppose I could use something like AIDE or tripwire, although that seems a bit like swatting a fly with a sledgehammer. Thanks, Andrew andrew at imsfs-new:~$ /usr/bin/amd64/truss bart create -R /export/ims > /tmp/bart-ims execve("/usr/bin/bart", 0x08047D6C, 0x08047D80) argc = 4 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEFF0000 resolvepath("/usr/lib/ld.so.1", "/lib/ld.so.1", 1023) = 12 resolvepath("/usr/bin/bart", "/usr/bin/bart", 1023) = 13 sysconfig(_CONFIG_PAGESIZE) = 4096 stat64("/usr/bin/bart", 0x08047B00) = 0 open("/var/ld/ld.config", O_RDONLY) Err#2 ENOENT stat64("/lib/libsec.so.1", 0x080473A0) = 0 resolvepath("/lib/libsec.so.1", "/lib/libsec.so.1", 1023) = 16 open("/lib/libsec.so.1", O_RDONLY) = 3 mmap(0x00010000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_ALIGN, 3, 0) = 0xFEFB0000 mmap(0x00010000, 143360, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFEF80000 mmap(0xFEF80000, 50487, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_TEXT, 3, 0) = 0xFEF80000 mmap(0xFEF9D000, 11909, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_INITDATA, 3, 53248) = 0xFEF9D000 mmap(0xFEFA0000, 8296, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANON, -1, 0) = 0xFEFA0000 munmap(0xFEF8D000, 65536) = 0 memcntl(0xFEF80000, 8844, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 close(3) = 0 stat64("/lib/libmd.so.1", 0x080473A0) = 0 resolvepath("/lib/libmd.so.1", "/lib/libmd.so.1", 1023) = 15 open("/lib/libmd.so.1", O_RDONLY) = 3 mmap(0xFEFB0000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0xFEFB0000 mmap(0x00010000, 126976, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFEF60000 mmap(0xFEF60000, 56424, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_TEXT, 3, 0) = 0xFEF60000 mmap(0xFEF7E000, 552, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_INITDATA, 3, 57344) = 0xFEF7E000 munmap(0xFEF6E000, 65536) = 0 memcntl(0xFEF60000, 1464, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 close(3) = 0 stat64("/lib/libc.so.1", 0x080473A0) = 0 resolvepath("/lib/libc.so.1", "/lib/libc.so.1", 1023) = 14 open("/lib/libc.so.1", O_RDONLY) = 3 mmap(0xFEFB0000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0xFEFB0000 mmap(0x00010000, 1208320, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFEE30000 mmap(0xFEE30000, 1099077, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_TEXT, 3, 0) = 0xFEE30000 mmap(0xFEF4D000, 30183, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_INITDATA, 3, 1101824) = 0xFEF4D000 mmap(0xFEF55000, 4240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANON, -1, 0) = 0xFEF55000 munmap(0xFEF3D000, 65536) = 0 memcntl(0xFEE30000, 124080, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 close(3) = 0 stat64("/lib/libavl.so.1", 0x080473A0) = 0 resolvepath("/lib/libavl.so.1", "/lib/libavl.so.1", 1023) = 16 open("/lib/libavl.so.1", O_RDONLY) = 3 mmap(0xFEFB0000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0xFEFB0000 mmap(0x00010000, 73728, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFEE10000 mmap(0xFEE10000, 2788, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_TEXT, 3, 0) = 0xFEE10000 mmap(0xFEE21000, 204, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_INITDATA, 3, 4096) = 0xFEE21000 munmap(0xFEE11000, 65536) = 0 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEFC0000 memcntl(0xFEE10000, 1056, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 close(3) = 0 mmap(0x00010000, 24576, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFEF90000 munmap(0xFEFB0000, 32768) = 0 getcontext(0x08047970) getrlimit(RLIMIT_STACK, 0x08047968) = 0 getpid() = 14812 [14811] lwp_private(0, 1, 0xFEF92A00) = 0x000001C3 setustack(0xFEF92A60) sysi86(SI86FPSTART, 0xFEF55740, 0x0000133F, 0x00001F80) = 0x00000001 brk(0x08086428) = 0 brk(0x08088428) = 0 sysconfig(_CONFIG_PAGESIZE) = 4096 getcwd("/export/home/andrew", 1024) = 0 chdir("/export/ims") = 0 getcwd("/export/ims", 1024) = 0 chdir("/export/home/andrew") = 0 pipe() = 3 [4] schedctl() = 0xFEFBC000 fork1() = 14813 lwp_sigmask(SIG_SETMASK, 0x00000000, 0x00000000) = 0xFFBFFEFF [0x0000FFFF] fcntl(3, F_DUP2FD, 0x00000001) = 1 close(3) = 0 close(4) = 0 statvfs("/export/ims", 0x08085DB0) Err#79 EOVERFLOW /export/imswrite(2, " / e x p o r t / i m s", 11) = 11 : write(2, " : ", 2) = 2 Value too large for defined data typewrite(2, " V a l u e t o o l a".., 37) = 37 write(2, "\n", 1) = 1 close(1) = 0 waitid(P_ALL, 0, 0x08047C50, WEXITED|WTRAPPED) = 0 _exit(-1)
Seems to me that you really want auditing. You can configure the audit system to only record the events you are interested in. docs.sun.com/app/docs/doc/816-4557/auditov-1?l=en&a=view -- richard On Nov 9, 2009, at 4:55 PM, Andrew Daugherity wrote:>>>> Robert Thurlow <robert.thurlow at sun.com> 11/9/2009 4:25 PM >>> > % file /bin/truss /bin/amd64/truss > /bin/truss: ELF 32-bit LSB executable 80386 Version 1 [FPU], > dynamically linked, not stripped, no debugging information available > > /bin/amd64/truss: ELF 64-bit LSB executable AMD64 Version 1 > [SSE2 > SSE FXSR CMOV FPU], dynamically linked, not stripped, no debugging > information available > > > > > It doesn''t make any difference if I invoke it with the amd64 truss. > The only bart binary I can find on the system (Sol 10u8) is /usr/bin/ > bart, and it definitely calls statvfs(). Truss log follows at the > end. > > I know all about ''find -mtime ...'', but that doesn''t show which > files have been deleted, whereas ''rsync -av --delete --backup- > dir=`date +%Y%m%d`'' does. (When users delete files and then need > them restored a week later, it''s very helpful to know which day they > were deleted, as I can avoid running a find that could take quite a > while. I think incremental zfs snapshots are a better strategy but > there are little hurdles like this to be crossed.) > > bart (or something faster than running ''gdiff -qr snap1 snap2'' on a > snapshots of a 2.1TB-and-growing FS) seems like a great idea, if I > could find a working tool. It looks like dircmp(1) might be a > possibility, but I''m open to suggestions. I suppose I could use > something like AIDE or tripwire, although that seems a bit like > swatting a fly with a sledgehammer. > > > Thanks, > > Andrew > > > > andrew at imsfs-new:~$ /usr/bin/amd64/truss bart create -R /export/ims > > /tmp/bart-ims > execve("/usr/bin/bart", 0x08047D6C, 0x08047D80) argc = 4 > mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE| > MAP_ANON, -1, 0) = 0xFEFF0000 > resolvepath("/usr/lib/ld.so.1", "/lib/ld.so.1", 1023) = 12 > resolvepath("/usr/bin/bart", "/usr/bin/bart", 1023) = 13 > sysconfig(_CONFIG_PAGESIZE) = 4096 > stat64("/usr/bin/bart", 0x08047B00) = 0 > open("/var/ld/ld.config", O_RDONLY) Err#2 ENOENT > stat64("/lib/libsec.so.1", 0x080473A0) = 0 > resolvepath("/lib/libsec.so.1", "/lib/libsec.so.1", 1023) = 16 > open("/lib/libsec.so.1", O_RDONLY) = 3 > mmap(0x00010000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_ALIGN, > 3, 0) = 0xFEFB0000 > mmap(0x00010000, 143360, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE| > MAP_ANON|MAP_ALIGN, -1, 0) = 0xFEF80000 > mmap(0xFEF80000, 50487, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED| > MAP_TEXT, 3, 0) = 0xFEF80000 > mmap(0xFEF9D000, 11909, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED| > MAP_INITDATA, 3, 53248) = 0xFEF9D000 > mmap(0xFEFA0000, 8296, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED| > MAP_ANON, -1, 0) = 0xFEFA0000 > munmap(0xFEF8D000, 65536) = 0 > memcntl(0xFEF80000, 8844, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 > close(3) = 0 > stat64("/lib/libmd.so.1", 0x080473A0) = 0 > resolvepath("/lib/libmd.so.1", "/lib/libmd.so.1", 1023) = 15 > open("/lib/libmd.so.1", O_RDONLY) = 3 > mmap(0xFEFB0000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, > 3, 0) = 0xFEFB0000 > mmap(0x00010000, 126976, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE| > MAP_ANON|MAP_ALIGN, -1, 0) = 0xFEF60000 > mmap(0xFEF60000, 56424, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED| > MAP_TEXT, 3, 0) = 0xFEF60000 > mmap(0xFEF7E000, 552, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED| > MAP_INITDATA, 3, 57344) = 0xFEF7E000 > munmap(0xFEF6E000, 65536) = 0 > memcntl(0xFEF60000, 1464, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 > close(3) = 0 > stat64("/lib/libc.so.1", 0x080473A0) = 0 > resolvepath("/lib/libc.so.1", "/lib/libc.so.1", 1023) = 14 > open("/lib/libc.so.1", O_RDONLY) = 3 > mmap(0xFEFB0000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, > 3, 0) = 0xFEFB0000 > mmap(0x00010000, 1208320, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE| > MAP_ANON|MAP_ALIGN, -1, 0) = 0xFEE30000 > mmap(0xFEE30000, 1099077, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED| > MAP_TEXT, 3, 0) = 0xFEE30000 > mmap(0xFEF4D000, 30183, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED| > MAP_INITDATA, 3, 1101824) = 0xFEF4D000 > mmap(0xFEF55000, 4240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED| > MAP_ANON, -1, 0) = 0xFEF55000 > munmap(0xFEF3D000, 65536) = 0 > memcntl(0xFEE30000, 124080, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 > close(3) = 0 > stat64("/lib/libavl.so.1", 0x080473A0) = 0 > resolvepath("/lib/libavl.so.1", "/lib/libavl.so.1", 1023) = 16 > open("/lib/libavl.so.1", O_RDONLY) = 3 > mmap(0xFEFB0000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, > 3, 0) = 0xFEFB0000 > mmap(0x00010000, 73728, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE| > MAP_ANON|MAP_ALIGN, -1, 0) = 0xFEE10000 > mmap(0xFEE10000, 2788, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED| > MAP_TEXT, 3, 0) = 0xFEE10000 > mmap(0xFEE21000, 204, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED| > MAP_INITDATA, 3, 4096) = 0xFEE21000 > munmap(0xFEE11000, 65536) = 0 > mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE| > MAP_ANON, -1, 0) = 0xFEFC0000 > memcntl(0xFEE10000, 1056, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 > close(3) = 0 > mmap(0x00010000, 24576, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE| > MAP_ANON|MAP_ALIGN, -1, 0) = 0xFEF90000 > munmap(0xFEFB0000, 32768) = 0 > getcontext(0x08047970) > getrlimit(RLIMIT_STACK, 0x08047968) = 0 > getpid() = 14812 [14811] > lwp_private(0, 1, 0xFEF92A00) = 0x000001C3 > setustack(0xFEF92A60) > sysi86(SI86FPSTART, 0xFEF55740, 0x0000133F, 0x00001F80) = 0x00000001 > brk(0x08086428) = 0 > brk(0x08088428) = 0 > sysconfig(_CONFIG_PAGESIZE) = 4096 > getcwd("/export/home/andrew", 1024) = 0 > chdir("/export/ims") = 0 > getcwd("/export/ims", 1024) = 0 > chdir("/export/home/andrew") = 0 > pipe() = 3 [4] > schedctl() = 0xFEFBC000 > fork1() = 14813 > lwp_sigmask(SIG_SETMASK, 0x00000000, 0x00000000) = 0xFFBFFEFF > [0x0000FFFF] > fcntl(3, F_DUP2FD, 0x00000001) = 1 > close(3) = 0 > close(4) = 0 > statvfs("/export/ims", 0x08085DB0) Err#79 EOVERFLOW > /export/imswrite(2, " / e x p o r t / i m s", 11) = 11 > : write(2, " : ", 2) = 2 > Value too large for defined data typewrite(2, " V a l u e t o o > l a".., 37) = 37 > > write(2, "\n", 1) = 1 > close(1) = 0 > waitid(P_ALL, 0, 0x08047C50, WEXITED|WTRAPPED) = 0 > _exit(-1) > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > mail.opensolaris.org/mailman/listinfo/zfs-discuss
Thanks for info, although the audit system seems a lot more complex than what I need. Would still be nice if they fixed bart to work on large filesystems, though. Turns out the solution was right under my nose -- rsync in dry-run mode works quite well as a "snapshot diff" tool. I''ll share this with the list, in case it helps anyone else. For example: rsync -avn --delete-before /export/ims/.zfs/snapshot/zfs-auto-snap_daily-2009-11-09-1900/ /export/ims/.zfs/snapshot/zfs-auto-snap_daily-2009-11-08-1900/ This makes a list of what files have been changed, added, or deleted between these two snapshots, and runs in about 10 seconds. If you cared to see changes within files (I don''t), it would be trivial to add a loop along the lines of ''$rsync_cmd | while read file; do diff $snap1/$file $snap2/file; done''. Note the trailing slashes (otherwise rsync works one directory higher and considers the snapshot directory name, which we don''t want) and that the newer snapshot is the "source" and the older snapshot, the destination. I''m [ab]using rsync to have it tell me exactly how it would make the "destination" be a replica of the "source". FWIW, I''m using rsync 3.0.6 from opencsw. Older rsync should work fine but may take longer to run. -Andrew>>> Richard Elling <richard.elling at gmail.com> 11/9/2009 7:33 PM >>>Seems to me that you really want auditing. You can configure the audit system to only record the events you are interested in. docs.sun.com/app/docs/doc/816-4557/auditov-1?l=en&a=view -- richard
On Nov 10, 2009, at 10:23 AM, Andrew Daugherity wrote:> > For example: > rsync -avn --delete-before /export/ims/.zfs/snapshot/zfs-auto- > snap_daily-2009-11-09-1900/ /export/ims/.zfs/snapshot/zfs-auto- > snap_daily-2009-11-08-1900/ > > [...]> If you cared to see changes within files (I don''t),toss the -W flag on there then and it should run even faster, methinks, as it won''t compare file contents at all. -Jeremy
Carson Gaspar wrote:> On 10/26/09 5:33 PM, paul at paularcher.org wrote: >>>> I can''t find much on gam_server on Solaris (couldn''t find too much >>>> on it >>>> at all, really), and port_create is apparently a system call. (I''m >>>> not a >>>> developer--if I can''t write it in BASH, Perl, or Ruby, I can''t write >>>> it.) >>>> I appreciate the suggestions, but I need something a little more >>>> pret-a-porte. >>> >>> Your Google-fu needs work ;-) >>> >>> Main Gamin page: gnome.org/~veillard/gamin/index.html >> >> Actually, I found this page, which has this gem: "At this point Gamin is >> fairly tied to Linux, portability is not a primary goal at this stage but >> if you have portability patches they are welcome." > > Much has changed since that text was written, including support for the > event completion framework (port_create() and friends, introduced with > Sol 10) on Solaris, thus the recommendation for gam_server / gamin. > > $ nm /usr/lib/gam_server | grep port_create > [458] | 134589544| 0|FUNC |GLOB |0 |UNDEF |port_createThe patch for port_create has never gone upstream however, while gvfs uses glib''s gio, which has backends for inotify, solaris, fam and win32. -- James Andrewartha