rsync Digest, Vol 62, Issue 1

Zane Brady
Fri, 01 Feb 2008 12:27:24 -0000

Yep

Zane

-----Original Message-----
From: rsync-bounces+zane_brady=trimble.com@lists.samba.org
[mailto:rsync-bounces+zane_brady=trimble.com@lists.samba.org] On Behalf Of
rsync-request@lists.samba.org
Sent: Friday, February 01, 2008 7:01 AM
To: rsync@lists.samba.org
Subject: rsync Digest, Vol 62, Issue 1

Send rsync mailing list submissions to
	rsync@lists.samba.org

To subscribe or unsubscribe via the World Wide Web, visit
	https://lists.samba.org/mailman/listinfo/rsync
or, via email, send a message with subject or body 'help' to
	rsync-request@lists.samba.org

You can reach the person managing the list at
	rsync-owner@lists.samba.org

When replying, please edit your Subject line so it is more specific than
"Re: Contents of rsync digest..."


To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

---------------------------------------

Today's Topics:

   1. Re: OS X xattr troubles (was Re: --exclude patterns)
      (Mike Bombich)
   2. Re: rsync-ing from two locations with same filenames (at
      different	versions) (Mojca Miklavec)
   3. Re: rsync-ing from two locations with same filenames (at
      different	versions) (Matt McCutchen)
   4. Re: rsync-ing from two locations with same filenames (at
      different	versions) (Mojca Miklavec)


----------------------------------------------------------------------

Message: 1
Date: Thu, 31 Jan 2008 06:54:44 -0600
From: Mike Bombich <mike@bombich.com>
Subject: Re: OS X xattr troubles (was Re: --exclude patterns)
To: Anthony Morton <amorton@fastmail.fm>
Cc: rsync@lists.samba.org
Message-ID: <3C41FC6F-4A4A-44D9-AC1C-B06A00E0B985@bombich.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes


On Jan 30, 2008, at 11:03 PM, Anthony Morton wrote:

>>> I have a similar problem.  I'm trying to specify a custom per-
>>> directory filter using
>>>
>>> 	--filter='dir-merge .rsync-filter-m'
>>>
>>> but because the whole thing is double-quoted the filter rule
arrives
>>> in single quotes.  I can't simply leave out the quotes here
because
>>> the --filter option only gets the first token as its argument.....
>>>
>
>> You can use an underscore instead of a space after the
"dir-merge".
>
> Thanks, that's solved it.
>
> Now there's a new problem I'd appreciate any help with.  I've
just
> started out with 3.0.0pre8 on OS X Leopard 10.5.1, patched according
> to Mike Bombich's recipe on this list at
> <http://lists.samba.org/archive/rsync/2008-January/019618.html>
>
> I have also run the backup-bouncer test as suggested (using -aHAX
> and --fileflags) and obtained results identical to Rob DuToit's.
> (Devices and fifos did not copy with their xattrs but all else was OK
> - this may be related to the bug Wayne opened.)

I think the errors that rsync complains about on devices and fifos should be
ignored.  I applied this patch and the errors go away and the backup-bouncer
test succeeds (e.g. the fifos and devices are recreated without error):

diff -Naur rsync-3.0.0pre8/xattrs.c rsync-3.0.0pre8_mod/xattrs.c
--- rsync-3.0.0pre8/xattrs.c	2008-01-12 11:14:56.000000000 -0600
+++ rsync-3.0.0pre8_mod/xattrs.c	2008-01-28 22:31:11.000000000 -0600
@@ -128,7 +128,7 @@
  	}
  	if (list_len >= 0)
  		return list_len;
-	if (errno == ENOTSUP)
+	if (errno == ENOTSUP || errno == EPERM)
  		return 0;
  	if (errno == ERANGE) {
  		list_len = sys_llistxattr(fname, NULL, 0); @@ -766,6 +766,8 @@
  		}

  		if (sys_lsetxattr(fname, name, rxas[i].datum, rxas[i].datum_len) <
0) {
+			if (errno == EPERM)
+				return 0;
  			rsyserr(FERROR_XFER, errno,
  				"rsync_xal_set: lsetxattr(\"%s\",\"%s\")
failed",
  				fname, name);


I plan to submit that, I just haven't had a spare moment.

>
>
> However, trying a local rsync on my own home directory with -X and the
> destination set to a directory on a local HFS+-formatted FireWire disk
> (actually an iPod) immediately errors out with
>
> [sender] could not find xattr #2 for .
> rsync error: error in rsync protocol data stream (code 12) at
> xattrs.c(561) [sender=3.0.0pre8]
>
> Things seem to be OK if I omit -X to leave the xattrs out.
>
> Is this a bug or is there something up with my filesystem?  (My
> installation's only a month old.)

I'm getting the same error with pre8 and the nightly from the 27th.  I
haven't boiled it down to a reproducible case yet, though.  I tried to do
that last night but ran out of time...

Mike

>
>
> Tony M.
>
> --
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read:
> http://www.catb.org/~esr/faqs/smart-questions.html
>



------------------------------

Message: 2
Date: Fri, 1 Feb 2008 02:06:23 +0100
From: "Mojca Miklavec" <mojca.miklavec.lists@gmail.com>
Subject: Re: rsync-ing from two locations with same filenames (at
	different	versions)
To: "Matt McCutchen" <matt@mattmccutchen.net>
Cc: rsync@lists.samba.org
Message-ID:
	<6faad9f00801311706o7d341e6an4affeb85c4f60256@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

On Jan 30, 2008 2:38 PM, Matt McCutchen wrote:
> On Wed, 2008-01-30 at 09:48 +0100, Mojca Miklavec wrote:
> > Neither helps. Even if I have a file of differest size and with a
> > different timestamp, and even if I add --checksum or --ignore-times,
> > the old file in dest won't be modified (overwritten by a newer
file).
>
> I can't reproduce the problem.  I ran the script in your original
> message, except I changed "b2" to "b22" and waited a
few seconds
> before running that line so the file would get a later mtime.  Both
> rsync 2.6.9 and the latest development rsync correctly replaced b.txt
> with the version from new/ on the second run.  Am I missing something?

I don't know. Somtimes it works and sometimes not (but mostly not as a rule
of thumb). Even if I wait for a few minutes inbetween, the new file won't be
chosen.

> rsync --version
rsync  version 2.6.3  protocol version 28

That might be old, but that was the default that came with fink on Mac OS X (if
the error has been fixed in the meantime, I will upgrade).


> ll new/dir1/
skupno 4,0K
-rw-r--r-- 1 mojca wheel 6 feb  1 02:00 b.txt

> ll full/dir1/
skupno 8,0K
-rw-r--r-- 1 mojca wheel 2 jan 30 09:36 a.txt
-rw-r--r-- 1 mojca wheel 4 feb  1 01:52 b.txt

> cat full/dir1/b.txt
b12

> cat new/dir1/b.txt
b2222

> rsync -rpztlv --delete --checksum new/dir1/ new/dir2/ full/dir1/
> full/dir2/ dest
building file list ... done
./

sent 261 bytes  received 20 bytes  562.00 bytes/sec total size is 24  speedup is
0.09

> cat dest/b.txt
b12

Thanks a lot,
   Mojca


------------------------------

Message: 3
Date: Thu, 31 Jan 2008 20:36:26 -0500
From: Matt McCutchen <matt@mattmccutchen.net>
Subject: Re: rsync-ing from two locations with same filenames (at
	different	versions)
To: Mojca Miklavec <mojca.miklavec.lists@gmail.com>
Cc: rsync@lists.samba.org
Message-ID: <1201829786.7077.72.camel@localhost>
Content-Type: text/plain; charset=UTF-8

On Fri, 2008-02-01 at 02:06 +0100, Mojca Miklavec wrote:
> I don't know. Somtimes it works and sometimes not (but mostly not as a
> rule of thumb). Even if I wait for a few minutes inbetween, the new
> file won't be chosen.
>
> > rsync --version
> rsync  version 2.6.3  protocol version 28
>
> That might be old, but that was the default that came with fink on Mac
> OS X (if the error has been fixed in the meantime, I will upgrade).

Duh.  I realize now that it's perfectly reasonable for you to be able to
reproduce the problem while I can't.  Versions 2.6.9 and earlier of rsync
sort the file-list using the C library's quicksort, an unstable sort, ???so
the results in case of duplicate files are highly sensitive to both the C
library implementation and the order of directory entries in the source (which
in turn is sensitive to the filesystem implementation).  You probably have both
a different C library and a different filesystem than I do.

In any case, since rsync 3.0.0pre1, the default file-list sorting algorithm is a
mergesort, which is stable, so files from earlier source arguments take
priority.  If you upgrade to an rsync 3.0.0pre* version, your scenario should
work consistently.  If it doesn't, that's a bug we should try to fix.

Matt



------------------------------

Message: 4
Date: Fri, 1 Feb 2008 09:46:41 +0100
From: "Mojca Miklavec" <mojca.miklavec.lists@gmail.com>
Subject: Re: rsync-ing from two locations with same filenames (at
	different	versions)
To: "Matt McCutchen" <matt@mattmccutchen.net>
Cc: rsync@lists.samba.org
Message-ID:
	<6faad9f00802010046t7629e852jfda267de42a3dd77@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

On Feb 1, 2008 2:36 AM, Matt McCutchen wrote:
> On Fri, 2008-02-01 at 02:06 +0100, Mojca Miklavec wrote:
> > I don't know. Somtimes it works and sometimes not (but mostly not
as
> > a rule of thumb). Even if I wait for a few minutes inbetween, the
> > new file won't be chosen.
> >
> > > rsync --version
> > rsync  version 2.6.3  protocol version 28
> >
> > That might be old, but that was the default that came with fink on
> > Mac OS X (if the error has been fixed in the meantime, I will
upgrade).
>
> Duh.  I realize now that it's perfectly reasonable for you to be able
> to reproduce the problem while I can't.  Versions 2.6.9 and earlier of
> rsync sort the file-list using the C library's quicksort, an unstable
> sort, ???so the results in case of duplicate files are highly
> sensitive to both the C library implementation and the order of
> directory entries in the source (which in turn is sensitive to the
> filesystem implementation).  You probably have both a different C
> library and a different filesystem than I do.
>
> In any case, since rsync 3.0.0pre1, the default file-list sorting
> algorithm is a mergesort, which is stable, so files from earlier
> source arguments take priority.  If you upgrade to an rsync 3.0.0pre*
> version, your scenario should work consistently.  If it doesn't,
> that's a bug we should try to fix.

Oh, thanks a lot for the explanation :)
I will try to figure out how to build it and test then.

In this particular case it is not needed, but what about a switch:

--always-take-the-latest-file-version-no-matter-of-order-of-supplied-arguments
?

Two problems still remain then: one is that I cannot rely on the fact that users
will have the latest version (at least for a while, but that argument will
disapear with time), and the other one - it would simplify the work a lot if I
didn't have to reproduce a bunch of empty folders in "new" (for
which you have said: "I agree that this feature is useful, and I might
implement it at some point as a patch.")

But thanks a lot for the explanation. You just gave me a really nice idea for a
real-life task for the forthcomming informatics competition here :)

Mojca

------------------------------

_______________________________________
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

rsync mailing list
rsync@lists.samba.org
https://lists.samba.org/mailman/listinfo/rsync


End of rsync Digest, Vol 62, Issue 1
************************************