Raphael Becker
2008-May-31 19:48 UTC
rsync 3.0.2 with --fileflags on FreeBSD: cannot rsync hardlinked immutable files
Hi *,
it seems rsync with --fileflags isn't able to work on (already)
hardlinked and immutable ("schg") files on FreeBSD. The following
scripts will create a simple example for this behaviour:
--------------------------------------------------------------
#! /bin/sh
#
#
set -x
DIR="/var/tmp/rsync_$(date +%s)/"
mkdir "${DIR}/"
# Preparing dir_A
mkdir "${DIR}/dir_A/"
touch "${DIR}/dir_A/file_A"
ln "${DIR}/dir_A/file_A" "${DIR}/dir_A/file_B"
chflags schg "${DIR}/dir_A/file_A"
ls -laio "${DIR}/dir_A/"
# Try rsync: will fail because of schg'ed hardlinked file
/usr/local/bin/rsync -avHWx --fileflags "${DIR}/dir_A/"
"${DIR}/dir_B/"
ls -laio "${DIR}/dir_B/"
# Try cpdup: will work
/usr/local/bin/cpdup "${DIR}/dir_A/" "${DIR}/dir_C/"
ls -laio "${DIR}/dir_C/"
--------------------------------------------------------------
will output:
--------------------------------------------------------------
+ date +%s
+ DIR=/var/tmp/rsync_1212259951/
+ mkdir /var/tmp/rsync_1212259951//
+ mkdir /var/tmp/rsync_1212259951//dir_A/
+ touch /var/tmp/rsync_1212259951//dir_A/file_A
+ ln /var/tmp/rsync_1212259951//dir_A/file_A
/var/tmp/rsync_1212259951//dir_A/file_B
+ chflags schg /var/tmp/rsync_1212259951//dir_A/file_A
+ ls -laio /var/tmp/rsync_1212259951//dir_A/
total 4
188674 drwxr-xr-x 2 root wheel - 512 May 31 20:52 .
188673 drwxr-xr-x 3 root wheel - 512 May 31 20:52 ..
188676 -rw-r--r-- 2 root wheel schg 0 May 31 20:52 file_A
188676 -rw-r--r-- 2 root wheel schg 0 May 31 20:52 file_B
+ /usr/local/bin/rsync -avHWx --fileflags /var/tmp/rsync_1212259951//dir_A/
/var/tmp/rsync_1212259951//dir_B/
sending incremental file list
created directory /var/tmp/rsync_1212259951//dir_B
./
file_B
rsync: link "/var/tmp/rsync_1212259951/dir_B/file_A" => file_B
failed: Operation not permitted (1)
sent 95 bytes received 34 bytes 258.00 bytes/sec
total size is 0 speedup is 0.00
rsync error: some files could not be transferred (code 23) at main.c(1031)
[sender=3.0.2]
+ ls -laio /var/tmp/rsync_1212259951//dir_B/
total 4
188677 drwxr-xr-x 2 root wheel - 512 May 31 20:52 .
188673 drwxr-xr-x 4 root wheel - 512 May 31 20:52 ..
188678 -rw-r--r-- 1 root wheel schg 0 May 31 20:52 file_B
+ /usr/local/bin/cpdup /var/tmp/rsync_1212259951//dir_A/
/var/tmp/rsync_1212259951//dir_C/
+ ls -laio /var/tmp/rsync_1212259951//dir_C/
total 4
188679 drwxr-xr-x 2 root wheel - 512 May 31 20:52 .
188673 drwxr-xr-x 5 root wheel - 512 May 31 20:52 ..
188680 -rw-r--r-- 2 root wheel schg 0 May 31 20:52 file_A
188680 -rw-r--r-- 2 root wheel schg 0 May 31 20:52 file_B
--------------------------------------------------------------
It seems rsync first sets the immutable-flag ("schg") and try to
hardlink the second file on the (already) schg'ed file, which fails as
expected.
A workaround would be to set alle hardlinks before setting the
immutable-flags on the files.
I got a patched patch for "flags.diff" by Rolf Grossmann for rsync
2.6.9, which handled hardlinks on schg'ed files gracefully (using
--flags instead of --fileflags), see
http://rabe.uugrn.org/files/replacement_for_net_rsync_work_rsync-2.6.9_patches_flags.diff
--------------------------------------------------------------
+ date +%s
+ DIR=/var/tmp/rsync_1212261208/
+ mkdir /var/tmp/rsync_1212261208//
+ mkdir /var/tmp/rsync_1212261208//dir_A/
+ touch /var/tmp/rsync_1212261208//dir_A/file_A
+ ln /var/tmp/rsync_1212261208//dir_A/file_A
/var/tmp/rsync_1212261208//dir_A/file_B
+ chflags schg /var/tmp/rsync_1212261208//dir_A/file_A
+ ls -laio /var/tmp/rsync_1212261208//dir_A/
total 4
212060 drwxr-xr-x 2 root wheel - 512 May 31 21:13 .
212059 drwxr-xr-x 3 root wheel - 512 May 31 21:13 ..
212061 -rw-r--r-- 2 root wheel schg 0 May 31 21:13 file_A
212061 -rw-r--r-- 2 root wheel schg 0 May 31 21:13 file_B
+ /usr/local/bin/rsync -avHWx --flags /var/tmp/rsync_1212261208//dir_A/
/var/tmp/rsync_1212261208//dir_B/
building file list ... done
created directory /var/tmp/rsync_1212261208//dir_B
./
file_B
file_A => file_B
sent 148 bytes received 61 bytes 418.00 bytes/sec
total size is 0 speedup is 0.00
+ ls -laio /var/tmp/rsync_1212261208//dir_B/
total 4
212062 drwxr-xr-x 2 root wheel - 512 May 31 21:13 .
212059 drwxr-xr-x 4 root wheel - 512 May 31 21:13 ..
212063 -rw-r--r-- 2 root wheel schg 0 May 31 21:13 file_A
212063 -rw-r--r-- 2 root wheel schg 0 May 31 21:13 file_B
+ /usr/local/bin/cpdup /var/tmp/rsync_1212261208//dir_A/
/var/tmp/rsync_1212261208//dir_C/
+ ls -laio /var/tmp/rsync_1212261208//dir_C/
total 4
212064 drwxr-xr-x 2 root wheel - 512 May 31 21:13 .
212059 drwxr-xr-x 5 root wheel - 512 May 31 21:13 ..
212065 -rw-r--r-- 2 root wheel schg 0 May 31 21:13 file_A
212065 -rw-r--r-- 2 root wheel schg 0 May 31 21:13 file_B
--------------------------------------------------------------
root@bottom:~# pkg_info | grep rsync
rsync-2.6.9_2 A network file distribution/synchronization utility
--------------------------------------------------------------
I use rsync for lot of things, especially rsyncing FreeBSD jails between
jailhosts (via ssh) needs the hardlinks on schg'ed files, for
local-to-local syncing cpdup is doing fine here.
Regards
Raphael Becker
--
--
Raphael Becker <rabe@uugrn.org> http://rabe.uugrn.org/
GnuPG: E7B2 1D66 3AF2 EDC7 9828 6D7A 9CDA 3E7B 10CA 9F2D
.........|.........|.........|.........|.........|.........|.........|..
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url :
http://lists.samba.org/archive/rsync/attachments/20080531/8254cf10/attachment.bin
Raphael Becker
2008-Jun-07 22:55 UTC
rsync 3.0.2 with --fileflags on FreeBSD: cannot rsync hardlinked immutable files
On Sat, May 31, 2008 at 09:16:43PM +0200, I wrote:> Hi *, > > it seems rsync with --fileflags isn't able to work on (already) > hardlinked and immutable ("schg") files on FreeBSD. The following > scripts will create a simple example for this behaviour:I forgot to check --force-change but this didn't help either. I googled around and found http://rsync.samba.org/ftp/rsync/patches/fileflags.diff ... which doesn't seem to work on the current(?) sources: root@daemon:/usr/ports/net/rsync# make extract ===> Vulnerability check disabled, database not found ===> Found saved configuration for rsync-3.0.2_1 => rsync-3.0.2.tar.gz doesn't seem to exist in /data/ftp/pub/FreeBSD/distfiles//. => Attempting to fetch from http://ftp2.de.freebsd.org/FreeBSD/distfiles//. fetch: http://ftp2.de.freebsd.org/FreeBSD/distfiles//rsync-3.0.2.tar.gz: Not Found => Attempting to fetch from http://rsync.samba.org/ftp/rsync/. rsync-3.0.2.tar.gz 100% of 747 kB 21 MBps => rsync-patches-3.0.2.tar.gz doesn't seem to exist in /data/ftp/pub/FreeBSD/distfiles//. => Attempting to fetch from http://ftp2.de.freebsd.org/FreeBSD/distfiles//. fetch: http://ftp2.de.freebsd.org/FreeBSD/distfiles//rsync-patches-3.0.2.tar.gz: Not Found => Attempting to fetch from http://rsync.samba.org/ftp/rsync/. rsync-patches-3.0.2.tar.gz 100% of 150 kB 17 MBps ===> Extracting for rsync-3.0.2_1 => MD5 Checksum OK for rsync-3.0.2.tar.gz. => SHA256 Checksum OK for rsync-3.0.2.tar.gz. => MD5 Checksum OK for rsync-patches-3.0.2.tar.gz. => SHA256 Checksum OK for rsync-patches-3.0.2.tar.gz. ===> rsync-3.0.2_1 depends on file: /usr/local/bin/perl5.8.8 - found Overriding the patch from the distfile with the patch from the web: root@daemon:/usr/ports/net/rsync# wget -O /data/tmp/usr/ports/net/rsync/work/rsync-3.0.2/patches/fileflags.diff http://rsync.samba.org/ftp/rsync/patches/fileflags.diff But this patch doesn't match the current(?) sources: root@daemon:/usr/ports/net/rsync# make patch ===> Patching for rsync-3.0.2_1 ===> rsync-3.0.2_1 depends on file: /usr/local/bin/perl5.8.8 - found ===> Applying extra patch /data/tmp/usr/ports/net/rsync/work/rsync-3.0.2/patches/fileflags.diff 1 out of 2 hunks failed--saving rejects to t_stub.c.rej 1 out of 4 hunks failed--saving rejects to util.c.rej *** Error code 2 Stop in /usr/ports/net/rsync. *** Error code 1 Stop in /usr/ports/net/rsync. The two distfiles are MD5 (rsync-3.0.2.tar.gz) = fd4c5d77d8cb7bb86ab209076fa214d9 MD5 (rsync-patches-3.0.2.tar.gz) = c076661c447ddc165f4ff427e7264a31 Does anyone know if there's a way to rsync hardlinks on immutable files? Wayne Davison? Rolf Grossmann? Anyone else? Help would be greatly appreciated. Regards Raphael Becker -- Raphael Becker <rabe@uugrn.org> http://rabe.uugrn.org/ GnuPG: E7B2 1D66 3AF2 EDC7 9828 6D7A 9CDA 3E7B 10CA 9F2D .........|.........|.........|.........|.........|.........|.........|.. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.samba.org/archive/rsync/attachments/20080608/01a02f2f/attachment.bin