Hi, I've been using rsync (OSX Tiger now Leopard) to backup my home folder daily using -a -H -A -X link-dest=dir to make incremental backups. There was a problem though since many files especially images, movies etc would be recopied each time instead of creating hard links. I have been testing the pre5 release and found that it seems to make hard links correctly for all files. I am hoping rsync 3.0 can replace the Apple version which has been so flawed. I tried the osx-create-time.diff patch too and it works but it took twice as long to copy my home folder as without and ground to a halt the last time. I know the creation date issue is somewhat "fringe" for rsync but it does matter to a lot of OSX folk so I don't know if there is any way to speed it up. I've made some small backup wrapper applications for people and they always want the creation date..... I noticed the rsync version that is used now in Carbon Copy Cloner is pretty "clean" with meatdata and saves the creation date and is very fast..... Just some thoughts-I don't have any experience with this code so can't help in that way. Thanks, Rob
Looking at this patch from a performance perspective, it appears that
getattrlist is called twice for every file:
23:57:24.341 lstat 00-basic-
permissions/owned-by-
root
0.000011
rsync
23:57:24.341 listxattr 00-basic-
permissions/owned-by-
root
0.000006
rsync
23:57:24.341 getattrlist 00-basic-
permissions/owned-by-
root
0.000006
rsync
23:57:24.341 getattrlist 00-basic-
permissions/owned-by-
root
0.000005
rsync
23:57:24.341 lstat 00-basic-
permissions/owned-by-
www
0.000008
rsync
23:57:24.341 listxattr 00-basic-
permissions/owned-by-
www
0.000005
rsync
23:57:24.341 getattrlist 00-basic-
permissions/owned-by-
www
0.000006
rsync
23:57:24.341 getattrlist 00-basic-
permissions/owned-by-
www
0.000005
rsync
The first time it is called by
sys_llistxattr()
getCreationTime()
-- basically we determine if the file has a creation date. If it
does, then add the CRTIME_XATTR string to the xattr list. The
creation date isn't actually cached here, though. To get the actual
creation date, getattrlist is called again via xattrs.c
get_xattr_data()
sys_lgetxattr()
get_crtime_xattr()
getCreationTime()
The performance hit is significant, and I'm wondering how safe it is
to simply assume that every file has a creation date (given that this
section is wrapped in #if HAVE_OSX_XATTRS), therefore, drop the first
getCreationDate and add the CRTIME_XATTR string to the xattr list by
default. For example:
// sysxattrs.c:150
ssize_t sys_llistxattr(const char *path, char *list, size_t size)
{
ssize_t ret = listxattr(path, list, size, XATTR_NOFOLLOW);
if (ret < 0)
return ret;
// if (getCreationTime(path) != NULL) {
ret += sizeof CRTIME_XATTR;
if (list) {
if ((size_t)ret > size) {
errno = ERANGE;
return -1;
}
memcpy(list + ret - sizeof CRTIME_XATTR,
CRTIME_XATTR, sizeof CRTIME_XATTR);
}
// }
return ret;
}
Or would this bomb out running on MOSX with a non-HFS volume as the
source? Or is there a better way to avoid this call (e.g. determine
the underlying filesystem)?
Mike
On Dec 1, 2007, at 10:45 PM, Robert DuToit wrote:
> Hi,
> I've been using rsync (OSX Tiger now Leopard) to backup my home
> folder daily using -a -H -A -X link-dest=dir to make incremental
> backups. There was a problem though since many files especially
> images, movies etc would be recopied each time instead of creating
> hard links. I have been testing the pre5 release and found that it
> seems to make hard links correctly for all files. I am hoping rsync
> 3.0 can replace the Apple version which has been so flawed.
>
> I tried the osx-create-time.diff patch too and it works but it took
> twice as long to copy my home folder as without and ground to a halt
> the last time. I know the creation date issue is somewhat
"fringe"
> for rsync but it does matter to a lot of OSX folk so I don't know if
> there is any way to speed it up. I've made some small backup wrapper
> applications for people and they always want the creation date.....
>
> I noticed the rsync version that is used now in Carbon Copy Cloner
> is pretty "clean" with meatdata and saves the creation date and
is
> very fast..... Just some thoughts-I don't have any experience with
> this code so can't help in that way. Thanks, Rob--
> To unsubscribe or change options:
https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>
-------------- next part --------------
HTML attachment scrubbed and removed
Hi Wayne, I tried the crtimes patch and it compiles and works but when I tried to add the flags patch too I came up with this make error. Though flags compiloes by itself or with the osx-create-time.diff patch. Rob robert-dutoits-powerbook-g4-15:rsync-3.0.0pre8 astrid$ make perl ./mkproto.pl ./*.c ./lib/compat.c gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c flist.c -o flist.o In file included from flist.c:24: ifuncs.h: In function ?f_crtime?: ifuncs.h:65: error: ?crtimes_ndx? undeclared (first use in this function) ifuncs.h:65: error: (Each undeclared identifier is reported only once ifuncs.h:65: error: for each function it appears in.) ifuncs.h: In function ?f_crtime_set?: ifuncs.h:76: error: ?crtimes_ndx? undeclared (first use in this function) flist.c: In function ?send_file_entry?: flist.c:467: error: ?XMIT_CRTIME_EQ_MTIME? undeclared (first use in this function) flist.c: In function ?recv_file_entry?: flist.c:770: error: ?XMIT_CRTIME_EQ_MTIME? undeclared (first use in this function) make: *** [flist.o] Error 1> > On Feb 2, 2008, at 7:02 PM, Wayne Davison wrote: > >> On Sat, Feb 02, 2008 at 12:18:07AM -0600, Mike Bombich wrote: >>> The performance hit is significant, and I'm wondering how safe it >>> is to >>> simply assume that every file has a creation date? >> >> I've been thinking about rewriting that patch so that the creation >> time >> is sent as a time attribute, like the atimes patch, instead of >> going to >> the trouble of faking an xattr value. This has several advantages, >> including only needing a single call to read the value, and also it >> might avoid the creation of so many separate xattr objects for the >> various files. The patch for this is now a patch/crtimes branch in >> git, >> and can also be seen here: >> >> http://rsync.samba.org/ftp/rsync/patches/crtimes.diff >> >> This requires the use of a new option, --crtimes (-N -- think >> "newness") >> to get the create times preserved. If you want -N anytime -t is >> used, >> you might want to add a couple popt aliases to your ~/.popt (or even >> /etc/popt) file: >> >> rsync alias -a -aN >> rsync alias -t -tN >> >> ..wayne.. >> -- >> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync >> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html >
On Sat, Feb 02, 2008 at 09:38:20PM -0500, Robert DuToit wrote:> I tried the crtimes patch and it compiles and works but when I tried > to add the flags patch too I came up with this make error.You apparently missed the patch errors before that. You need to resolve all the failed hunks from the second patch before trying to run make. Look for the *.rej files. ..wayne..
Apparently Analagous Threads
- DO NOT REPLY [Bug 6276] New: crtimes.patch does not preserve creation dates on Mac x86_64 only
- DO NOT REPLY [Bug 7685] New: rsync should not set the creation data on the root folder of an HFS+ volume
- fileflags.diff patch
- Fwd: Making rsync compile under Mac OS X 10.3.9 with extended attributes
- RFC: __attribute__((ifunc("resolver")) representation in LLVM IR