rsync - Mar 2004 - HFS+ resource forks: WIP patch included

As you all know, rsync doesn't have any special handling
for Mac OS X HFS+ resource forks.  Kevin Boyd made RsyncX
and rsync_hfs, to address this gap, but they only work when
the destination filesystem is also HFS+.  I haven't been 
able to find any references to an rsync that is capable of
syncing from HFS+ to UFS (etc).  The only solutions I've
seen involve lots of preprocessing chicanery and additional
disk space consumption (basically, dupe the resource forks
or entire files first, then copy them over), which doesn't
suit my needs.

So I'm working on a patch to the mainline of rsync-2.6.0
that will recognize a resource fork on an HFS+ filesystem,
and copy the data to a separate file on the destination FS.
This will allow me to back up Mac desktops to larger hard
drives on Solaris and Linux file servers.

Many people seem to have a need for this, but I don't know
if it's a feature that belongs in the standard distribution.
Definitely not in the overly-specific form I've approached
the problem, but I could imagine a more general solution to
this and similar situations (NTFS streams, other FS metadata)
being a worthwhile project.

Anyway, I'm not sure how many people on this list run HFS+.
I'm guessing not many, or someone would have already written
a (better) version of this patch.  But I'm sending it in 
hopes of getting some feedback.  My C skills have atrophied
over the past ten years or so, and I only spent a few hours
looking over the flow of the rsync code, so I may have made
some very bad decisions.

That said, it does work, for me.  I've synced ~200GB of HFS+
filesystems, several times, from a few different machines,
and it sure seems like everything's OK.

If you have an interest and can test the code, please do so.
If you can offer rsync architectural advice, please do so.
If you can offer C syntax or style critique, please do so.

It looks like I'm going to be using this pretty heavily, and
at some point depending on it, so I have significant interest
in making it work as well as it can.

Thanks for your help,
Andrew
reynhout@quesera.com


OVERVIEW OF PATCH
================
If --hfs-mode switch is specified, check every regular file
on the local filesystem (no FStype checks) for the presence
of a resource fork.  If found, add it as an additional file
to the flist, but change the name it will be synced to on
the destination FS.

On HFS+, a resource fork for <filename> is accessed via
<filename>/..namedfork/rsrc .  Obviously, we can't send the
same filename to the destination, because an inode can't be
a regular file and a directory simultaneously.

Map:  <filename>/..namedfork/rsrc
to:   <filename>.~~~namedfork.rsrc

It appears to be critical that the local and destination
filenames sort into identical positions in the flist.  I
see the utility of sorting filenames -- to guarantee that
directories are treated before the files they contain, to
make removing duplicates easier -- but I'm less clear on
why (it appears) that *both* sides sort the flist.

Since the file-id is implicit (the offset in the flist),
the original and new filenames MUST sort into the same
position in the flist, or filesystem corruption will
result on the destination FS (files will have the wrong
filenames).  I think this could be fixed with an EXPLICIT
file-id, but of course rsync was designed with traffic
minimization in mind, so the protocol exchanges as little
data as possible.

Anyway, I picked .~~~namedfork.rsrc because it sorts into
the same position as /..namedfork/rsrc in almost all cases.
(Extended ASCII chars in filenames (and EBCDIC systems!)
complicate this assertion, but I haven't decided exactly
what to do about it yet.)

So then the only trick is to get send_file_entry to use
the DESTINATION filename isntead of the local filename
when sending the flist to the other side.  This is done
by replacing the call to f_name in send_file_entry with
f_name_dst, which duplicates f_name except for the above
change.  It's ugly, but it works for now.

NOTE: this is currently a ONE-WAY, SEND-ONLY operation.
Files will have to be manually reassembled if restoration
is required.  It would be easy to add automatic reassembly,
but it's not in here yet.  Also, the current code DOES NOT
require that the destination rsync also be patched.  It
works just fine with stock rsync-2.6.0.  This is a valuable
feature for me right now, so I'm resisting altering it.

Current (WIP!) version of patch is included below.
It's also available and will be kept updated at:
http://www.quesera.com/~reynhout/misc/rsync-hfs-mode-patch

The switch to turn the new stuff on is: --hfs-mode=darsplit
I wanted to include mode "appledot" to set the resource fork
filename to ._<filename> on the other side, but the sorting
issue described above got in the way...for now, at least.

Thanks again for any help you can offer.  Don't be gentle.


diff -u rsync-2.6.0/flist.c rsync-2.6.0-dar/flist.c
--- rsync-2.6.0/flist.c	Mon Dec 15 03:10:31 2003
+++ rsync-2.6.0-dar/flist.c	Wed Mar 10 11:29:32 2004
@@ -382,7 +382,7 @@
 
 	io_write_phase = "send_file_entry";
 
-	fname = f_name(file);
+	fname = f_name_dst(file);
 
 	flags = base_flags;
 
@@ -737,7 +737,7 @@
 		if (lastdir && strcmp(fname, lastdir) == 0) {
 			file->dirname = lastdir;
 		} else {
-			file->dirname = strdup(fname);
+			file->dirname = STRDUP(ap,fname);
 			lastdir = file->dirname;
 		}
 		file->basename = STRDUP(ap, p + 1);
@@ -804,6 +804,9 @@
 	struct file_struct *file;
 	extern int delete_excluded;
 
+	struct file_struct *file_hfsrf;
+	extern char *hfs_mode;
+
 	/* f is set to -1 when calculating deletion file list */
 	file = make_file(fname, &flist->string_area,
 			 f == -1 && delete_excluded? SERVER_EXCLUDES
@@ -831,8 +834,77 @@
 		local_exclude_list = last_exclude_list;
 		return;
 	}
+
+	if ( file->basename[0] && S_ISREG(file->mode) &&
hfs_mode ) {
+		/** check for rsrc fork, and add if found */
+		file_hfsrf = hfs_rsrc_fork(f,flist,file,fname);
+		if ( file_hfsrf ) {
+			flist->files[flist->count++] = file_hfsrf;
+			send_file_entry(file_hfsrf, f, base_flags);
+		}
+	}
 }
 
+struct file_struct *hfs_rsrc_fork(int f, struct file_list *flist,
+	struct file_struct *file, char *fname)
+{
+	extern char *hfs_mode;
+	extern int delete_excluded;
+	struct file_struct *file_rf;
+	char *fname_rf;
+	struct stat statbuf;
+	mode_t modes;
+	off_t size;
+	char *suffix=".~~~namedfork.rsrc";
+
+	/**	HFS+ resource fork handler code:
+
+		HFS+ exposes resource fork as <filename>/..namedfork/rsrc
+
+		If >0bytes, make new file_struct. Populate dirname_dst
+		with regular file's path, and basename_dst with arbitrary
+		destination filename: <filename>.~~~namedfork.rsrc
+
+		Detailed rationale for this weird decision presented at
+		http://www.quesera.com/~reynhout/misc/rsync-hfs-mode-patch
+
+		Then add the file_struct to flist, and sort out the
+		"which-name-to-use" problem in send_file_entry() by
+		calling f_name_dst() instead of f_name(), which is ugly.
+
+		Andrew Reynhout <reynhout@quesera.com>
+	**/
+
+	if ( hfs_mode && (!S_ISDIR(file->mode)) ) {
+		fname_rf=strcat(fname,"/..namedfork/rsrc");
+		stat(fname_rf,&statbuf);
+		modes=statbuf.st_mode;
+		size=statbuf.st_size;
+
+		if ( (S_ISREG(modes)) && (size > 0) ) {
+			file_rf = make_file(fname_rf, &flist->string_area,
+				f == -1 && delete_excluded? SERVER_EXCLUDES
+				: ALL_EXCLUDES);
+
+			file_rf->dirname_dst = file->dirname;
+
+			if ( strcmp(hfs_mode,"darsplit") == 0 ) {
+				file_rf->basename_dst = new_array(char,
+					MAXPATHLEN); 
+				if (!file_rf->basename_dst)
+					out_of_memory("hfs_rsrc_fork 2a");
+				sprintf(file_rf->basename_dst, "%s%s",
+					file->basename,suffix);
+			}
+
+			/* rprintf(FERROR,"darDEBUG: %s,%s to %s,%s\n",
+				file_rf->dirname,file_rf->basename,
+				file_rf->dirname_dst,file_rf->basename_dst); */
+			return file_rf;
+		}
+	}
+	return NULL;
+}
 
 
 static void send_directory(int f, struct file_list *flist, char *dir)
@@ -1409,3 +1481,35 @@
 
 	return p;
 }
+
+char *f_name_dst(struct file_struct *f)
+{
+	static char names[10][MAXPATHLEN];
+	static int n;
+	char *p = names[n];
+	char *dname, *bname;
+
+	if (!f || !f->basename)
+		return NULL;
+
+	dname=f->dirname;
+	bname=f->basename;
+	if ( f->basename_dst ) {
+		dname=f->dirname_dst;
+		bname=f->basename_dst;
+	}
+
+	n = (n + 1) % 10;
+
+	if (dname) {
+		int off;
+
+		off = strlcpy(p, dname, MAXPATHLEN);
+		off += strlcpy(p + off, "/", MAXPATHLEN - off);
+		off += strlcpy(p + off, bname, MAXPATHLEN - off);
+	} else {
+		strlcpy(p, bname, MAXPATHLEN);
+	}
+
+	return p;
+} 
diff -u rsync-2.6.0/loadparm.c rsync-2.6.0-dar/loadparm.c
--- rsync-2.6.0/loadparm.c	Sat Dec  6 16:07:27 2003
+++ rsync-2.6.0-dar/loadparm.c	Sat Mar  6 16:35:39 2004
@@ -135,6 +135,7 @@
 	char *include;
 	char *include_from;
 	char *log_format;
+	char *hfs_mode;
 	char *refuse_options;
 	char *dont_compress;
 	int timeout;
@@ -293,6 +294,7 @@
   {"transfer logging", P_BOOL,    P_LOCAL, 
&sDefault.transfer_logging,NULL,0},
   {"ignore errors",    P_BOOL,    P_LOCAL, 
&sDefault.ignore_errors,NULL,0},
   {"log format",       P_STRING,  P_LOCAL,  &sDefault.log_format,
NULL,   0},
+  {"HFS mode",         P_STRING,  P_LOCAL,  &sDefault.hfs_mode,  
NULL,   0},
   {"refuse options",   P_STRING,  P_LOCAL, 
&sDefault.refuse_options,NULL, 0},
   {"dont compress",    P_STRING,  P_LOCAL, 
&sDefault.dont_compress,NULL,  0},
   {NULL,               P_BOOL,    P_NONE,   NULL,                  NULL,   0}
@@ -370,6 +372,7 @@
 FN_LOCAL_STRING(lp_include, include)
 FN_LOCAL_STRING(lp_include_from, include_from)
 FN_LOCAL_STRING(lp_log_format, log_format)
+FN_LOCAL_STRING(lp_hfs_mode, hfs_mode)
 FN_LOCAL_STRING(lp_refuse_options, refuse_options)
 FN_LOCAL_STRING(lp_dont_compress, dont_compress)
 FN_LOCAL_INTEGER(lp_timeout, timeout)
diff -u rsync-2.6.0/options.c rsync-2.6.0-dar/options.c
--- rsync-2.6.0/options.c	Tue Dec 30 13:16:25 2003
+++ rsync-2.6.0-dar/options.c	Mon Mar  8 16:51:57 2004
@@ -113,6 +113,7 @@
 char *config_file = NULL;
 char *shell_cmd = NULL;
 char *log_format = NULL;
+char *hfs_mode = NULL;
 char *password_file = NULL;
 char *rsync_path = RSYNC_PATH;
 char *backup_dir = NULL;
@@ -171,10 +172,11 @@
 	/* Note that this field may not have type ino_t.  It depends
 	 * on the complicated interaction between largefile feature
 	 * macros. */
-	rprintf(f, "              %sIPv6, %d-bit system inums, %d-bit internal
inums\n",
+	rprintf(f, "              %sIPv6, %d-bit system inums, %d-bit internal
inums,\n",
 		ipv6,
 		(int) (sizeof(dumstat->st_ino) * 8),
 		(int) (sizeof(INO64_T) * 8));
+	rprintf(f, "              HFS+ (Mac OS X) resource forks\n");
 #ifdef MAINTAINER_MODE
 	rprintf(f, "              panic action: \"%s\"\n",
 		get_panic_action());
@@ -277,6 +279,7 @@
   rprintf(F,"     --stats                 give some file transfer
stats\n");
   rprintf(F,"     --progress              show progress during
transfer\n");
   rprintf(F,"     --log-format=FORMAT     log file transfers using
specified format\n");
+  rprintf(F,"     --hfs-mode=MODE         handle MacOS HFS+ resource
forks\n");
   rprintf(F,"     --password-file=FILE    get password from FILE\n");
   rprintf(F,"     --bwlimit=KBPS          limit I/O bandwidth, KBytes per
second\n");
   rprintf(F,"     --write-batch=PREFIX    write batch fileset starting
with PREFIX\n");
@@ -295,7 +298,7 @@
 
 enum {OPT_VERSION = 1000, OPT_SENDER, OPT_EXCLUDE, OPT_EXCLUDE_FROM,
       OPT_DELETE_AFTER, OPT_DELETE_EXCLUDED, OPT_LINK_DEST,
-      OPT_INCLUDE, OPT_INCLUDE_FROM, OPT_MODIFY_WINDOW,
+      OPT_INCLUDE, OPT_INCLUDE_FROM, OPT_MODIFY_WINDOW, OPT_HFS_MODE,
       OPT_READ_BATCH, OPT_WRITE_BATCH};
 
 static struct poptOption long_options[] = {
@@ -366,6 +369,7 @@
   {"config",           0,  POPT_ARG_STRING, &config_file, 0, 0, 0
},
   {"port",             0,  POPT_ARG_INT,    &rsync_port, 0, 0, 0
},
   {"log-format",       0,  POPT_ARG_STRING, &log_format, 0, 0, 0
},
+  {"hfs-mode",         0,  POPT_ARG_STRING, &hfs_mode,
OPT_HFS_MODE, 0, 0 },
   {"bwlimit",          0,  POPT_ARG_INT,    &bwlimit, 0, 0, 0 },
   {"address",          0,  POPT_ARG_STRING, &bind_address, 0, 0,
0 },
   {"backup-dir",       0,  POPT_ARG_STRING, &backup_dir, 0, 0, 0
},
@@ -516,6 +520,16 @@
 		case OPT_INCLUDE_FROM:
 			add_exclude_file(&exclude_list, poptGetOptArg(pc),
 					 MISSING_FATAL, ADD_INCLUDE);
+			break;
+
+		case OPT_HFS_MODE:
+			if ( (strcmp(hfs_mode,"none") != 0) &&
+				(strcmp(hfs_mode,"darsplit") != 0) ) {
+				snprintf(err_buf, sizeof err_buf,
+				"unsupported hfs-mode: \"%s\"\n",hfs_mode);
+				rprintf(FERROR, "ERROR: %s", err_buf);
+				exit_cleanup(RERR_UNSUPPORTED);
+			}
 			break;
 
 		case 'h':
diff -u rsync-2.6.0/proto.h rsync-2.6.0-dar/proto.h
--- rsync-2.6.0/proto.h	Sat Dec  6 16:07:27 2003
+++ rsync-2.6.0-dar/proto.h	Mon Mar  8 13:37:07 2004
@@ -77,8 +77,28 @@
 int link_stat(const char *path, STRUCT_STAT * buffer);
 struct file_struct *make_file(char *fname, struct string_area **ap,
 			      int exclude_level);
+void send_file_name_ifhasrsrcfork(int f, struct file_list *flist, char *fname,
+		    int recursive, unsigned base_flags);
+void send_file_name(int f, struct file_list *flist, char *fname,
+		    int recursive, unsigned base_flags);
+struct file_list *send_file_list(int f, int argc, char *argv[]);
+struct file_list *recv_file_list(int f);
+int file_compare(struct file_struct **f1, struct file_struct **f2);
+int flist_find(struct file_list *flist, struct file_struct *f);
+void free_file(struct file_struct *file);
+struct file_list *flist_new(void);
+void flist_free(struct file_list *flist);
+char *f_name(struct file_struct *f);
+char *f_name_dst(struct file_struct *f);
+void show_flist_stats(void);
+int readlink_stat(const char *path, STRUCT_STAT * buffer, char *linkbuf);
+int link_stat(const char *path, STRUCT_STAT * buffer);
+struct file_struct *make_file(char *fname, struct string_area **ap,
+			      int exclude_level);
 void send_file_name(int f, struct file_list *flist, char *fname,
 		    int recursive, unsigned base_flags);
+struct file_struct *hfs_rsrc_fork(int f, struct file_list *flist,
+	struct file_struct *file, char *fname);
 struct file_list *send_file_list(int f, int argc, char *argv[]);
 struct file_list *recv_file_list(int f);
 int file_compare(struct file_struct **f1, struct file_struct **f2);
@@ -87,6 +107,7 @@
 struct file_list *flist_new(void);
 void flist_free(struct file_list *flist);
 char *f_name(struct file_struct *f);
+char *f_name_dst(struct file_struct *f);
 void write_sum_head(int f, struct sum_struct *sum);
 void recv_generator(char *fname, struct file_list *flist, int i, int f_out);
 void generate_files(int f,struct file_list *flist,char *local_name,int f_recv);
@@ -142,6 +163,7 @@
 char *lp_include(int );
 char *lp_include_from(int );
 char *lp_log_format(int );
+char *lp_hfs_mode(int );
 char *lp_refuse_options(int );
 char *lp_dont_compress(int );
 int lp_timeout(int );
diff -u rsync-2.6.0/rsync.1 rsync-2.6.0-dar/rsync.1
--- rsync-2.6.0/rsync.1	Thu Jan  1 14:00:11 2004
+++ rsync-2.6.0-dar/rsync.1	Mon Mar  8 16:34:23 2004
@@ -385,6 +385,7 @@
      --bwlimit=KBPS          limit I/O bandwidth, KBytes per second
      --write-batch=PREFIX    write batch fileset starting with PREFIX
      --read-batch=PREFIX     read batch fileset starting with PREFIX
+     --hfs-mode=MODE         handle Mac OS X HFS+ resource forks in MODE
  -h, --help                  show this help screen
 
 
@@ -985,6 +986,34 @@
 using the fileset whose filenames start with PREFIX\&. See the "BATCH
 MODE" section for details\&.
 .IP 
+.IP "\fB--hfs-mode=MODE\fP" 
+Handle Mac OS X HFS+ files with resource forks according to MODE\&.
+This is currently a \fBONE-WAY\fP, \fBSEND-ONLY\fP operation, and
+is only useful when backing up a Mac OS X machine to a non-HFS+
+filesystem (e\&.g\&. a Linux fileserver)\&.  The rsync process must
+be initiated from the Mac\&.  It is \fINOT\fP necessary for the
+remote rsync to also have this patch in place\&.
+.IP
+MODE "darsplit" will save the resource fork of <filename> to
+.nf
+<filename>\&.~~~namedfork\&.rsrc on the destination
filesystem\&.
+.fi
+.IP
+No other MODEs are valid at this time\&.  See web page for
explanation\&.
+.RS
+http://www\&.quesera\&.com/~reynhout/misc/rsync-hfs-mode-patch
+.RE
+.IP
+I use this command line to back up my home directory to fileserver:
+.IP
+.nf
+rsync --archive --delete --verbose --hfs-mode=darsplit \\
+                /Users/reynhout fileserver:/backups/mac
+.fi
+.IP 
+(hfs-mode is a patch to rsync, not part of the standard distribution.
+For more info and the latest version, see the web page listed above.)
+.IP
 .PP 
 .SH "EXCLUDE PATTERNS" 
 .PP 
diff -u rsync-2.6.0/rsync.h rsync-2.6.0-dar/rsync.h
--- rsync-2.6.0/rsync.h	Tue Dec 16 18:04:59 2003
+++ rsync-2.6.0-dar/rsync.h	Sun Mar  7 19:04:50 2004
@@ -381,6 +381,11 @@
 	char *basedir;
 	char *link;
 	char *sum;
+
+	/** these are used by the hfs-mode switch to copy HFS+
+	*** resource forks to a different filename on the dest */
+	char *dirname_dst;
+	char *basename_dst;
 };
 
 
diff -u rsync-2.6.0/rsync.yo rsync-2.6.0-dar/rsync.yo
--- rsync-2.6.0/rsync.yo	Thu Jan  1 14:00:11 2004
+++ rsync-2.6.0-dar/rsync.yo	Mon Mar  8 16:34:25 2004
@@ -348,6 +348,7 @@
      --bwlimit=KBPS          limit I/O bandwidth, KBytes per second
      --write-batch=PREFIX    write batch fileset starting with PREFIX
      --read-batch=PREFIX     read batch fileset starting with PREFIX
+     --hfs-mode=MODE         handle Mac OS X HFS+ resource forks in MODE
  -h, --help                  show this help screen
 
 
@@ -854,6 +855,26 @@
 dit(bf(--read-batch=PREFIX)) Apply a previously generated change batch,
 using the fileset whose filenames start with PREFIX. See the "BATCH
 MODE" section for details.
+
+dit(bf(--hfs-mode=MODE)) Handle Mac OS X HFS+ files with resource
+forks according to MODE.  This is currently a ONE-WAY, SEND-ONLY
+operation, and is only useful when backing up a Mac OS X machine
+to a non-HFS+ filesystem (e.g. a Linux fileserver).  It is NOT
+necessary for the remote rsync to also have this patch in place.
+
+MODE "darsplit" will save the resource fork of <filename> to
+<filename>.~~~namedfork.rsrc on the destination filesystem.
+
+No other MODEs are valid at this time.  See web page for explanation.
+http://www.quesera.com/~reynhout/misc/rsync-hfs-mode-patch
+
+I use this command line to back up my home directory to fileserver:
+
+    rsync --archive --delete --verbose --hfs-mode=darsplit \
+        /Users/reynhout fileserver:/backups/mac
+
+(hfs-mode is a patch to rsync, not part of the standard distribution.
+For more info and the latest version, see the web page listed above.)
 
 enddit()

On Wed, 10 Mar 2004, D Andrew Reynhout wrote:
>
> On HFS+, a resource fork for <filename> is accessed via
> <filename>/..namedfork/rsrc .  Obviously, we can't send the
> same filename to the destination, because an inode can't be
> a regular file and a directory simultaneously.
>
> Map:  <filename>/..namedfork/rsrc
> to:   <filename>.~~~namedfork.rsrc
>
There are chflags and finder metadata to be mindful of as well.

On your website, you state:

"There are a few ways to approach the problem.  RsyncX and rsync_hfs
(by Kevin Boyd) turn the files into one of the single-fork formats
of olde (e.g.: macbinary, applesingle, etc) and transfer them that
way, then unwrap them on the other side.  This method requires a
HFS+ filesystem on the both sides, so it didn't suit my needs."

This is untrue. rsync_hfs does not encode or split the file at all, it
does a read in place.  rsync_hfs with the --eahfs flag requires rsync_hfs
on the other end because it has to catch all of the extra unencoded data
that it throws.

For --eahfs, what needs to be implemented is catching extra data on
non-HFS+ filesystems, say storing them in a standard Apple alien FS
format. If you change the file (encode split, etc), you are altering the
source folders, which is not desirable.  You could encode the files and
trick rsync into using applesingle files, but that would incur a space and
encode time penalty.  You could encode (or split) files into appledouble
format, which would incur less of a time and space penalty, but would
still require additional space on the source machine.

Both of these options do require a lot of 'tricking' rsync into thinking
it is processing one set of files instead of another.  I've considered
taking the penalty for appledouble, storing the created metadata resource
files in a temporary, non-source tree location (like /tmp/), given the
demand for cross-platform storage.  I'd say most people would prefer the
source to be untouched, and let the destination create the appropriate
repository for additional data (encode metadata and resource fork in
memory at source, perform rsync algo with dest, and then send what is
needed.)

The agreed (and best) thought would be for SUS accepted format EA and ACL
support in rsync, and for HFS+ patches for EA and ACL support a la:

http://acl.bestbits.at/

to support other filesystems.

Kevin Boyd
OS X Deployment Coordinator
Sys Adm UMIT Contract Services

I use rsync to distribute files from a non-OS X server to
OS X clients.  I think using ._ files would clearly be the
best solution for the non-OS X side if you can get the
sorting issue worked out.  That way the files on the non-HFS+
side of the rsync could be accessed directly by the Mac if
mounted via SMB or NFS.


I also wanted to share a little known tool I found through
Googling that is the hack I use to make rsync work for me
currently.  "FixupResourceForks" pushes metadata and resource
forks stored in ._ files back into the filesystem.  For the
record, this is how I currently use rsync to copy files
between OS X machines:

1. Use the Finder (or CpMac, or other utils) to copy my data
    from an HFS partition to a UFS or NFS partition on a server.
2. Use rsync to copy the files from the non-HFS+ partition to
    an HFS+ partition on a Mac
3. On the client OS X machine, run
    /System/Library/CoreServices/FixupResourceForks on the
    root of the synced tree on the HFS+ partition.

The disadvantage to this method is that all resource forks
and metadata (all ._ files) are re-synced every time rsync
runs because FixupResourceForks deletes the ._ files in
the process of pushing the data into the filesystem, but
this is fortunately a small amount of data for me.

Hope this helps someone until the *real* rsync solution is
developed.  (I *do* need type/creator/icon information, so
this patch wouldn't work for me just yet.)

Greg

On Wed, Mar 10, 2004 at 01:11:35PM -0500, D Andrew Reynhout
wrote:
> but I'm less clear on
> why (it appears) that *both* sides sort the flist.
This is because the flist gets sent as it is created, so it is sent
unsorted.  Without this rule it would be necessary to wait until the
list had been fully read from disk and sorted before it was transferred,
making the use of time much less optimal (it's better to have the disk
I/O and network I/O overlap rather than be consecutive).

..wayne..

On Sat, 13 Mar 2004, Wayne Davison wrote:
>
> I have no desire to see the current protocol get explicit IDs.  I think
> it would be better to simply ensure that the lists sort identically on
> each side, and the best way to do that is to ensure that both sides sort
> the exact same list.
>
I agree.
> For instance, in the prior example of munging the names using
".~~~" in
> place of "/..", I can think of two better solutions than having
the
> lists differ on each side prior to the sort:
>
> (1) The sender would create the file list pre-munged (with a simpler
> naming scheme) but flagged in such a way that it would know that it had
> to tweak the name back into its unmunged form before opening it.  (This
> solution avoids needing a modified rsync on the receiving side.)
>
Or to solve the naming issue completely, rsync on an HFS+ volume could
generate a file list that represents the data as it should appear
losslessly on a UFS volume.  If the files are received by an HFS+ volume,
they could be stored in their native format, if this version of rsync is
present on both HFS+ volumes.  If the files are received by a UFS machine,
or a machine without this version of rsync installed, then the files will
be stored in the format Apple currently uses for UFS volumes.  This would
require on the fly encoding of HFS+ files to UFS, on the fly decoding of
UFS formatted HFS+ files back to native HFS+, and an flist comparison of
HFS+ volumes in UFS format.
> (2) The sender sends the list unmunged but flagged as "needing to be
> transformed".  Such entries would then be munged sometime after the
> sort.  (This solution has the disadvantage of requiring both sides to be
> upgraded, but it does do less munging, especially if the receiving side
> had native support for the resource-fork files.)
>
This is what occurs in rsync_hfs, which can be browsed via cvs at
www.opendarwin.org.

Kevin Boyd
OS X Deployment Coordinator
Sys Adm UMIT Contract Services