Daniel Maher
2007-Nov-05 13:19 UTC
problem restoring via rsync when using stored ACLs (--fake-super)
Hello, In summary, i appear to have encountered a bug when attempting to restore files using rsync 3.0.0pre4 (matt) with stored ACLs (via --fake-super). The ACL information appears to be correctly stored as xattrs on the backup server; however, when i attempt to restore while preserving the ACL data, i get the following error: ERROR: out of memory in get_xattr_data [sender] Questions: 1. Is this a bug in rsync? The filesystem? Malloc? 2. If this is not how i'm supposed to restore the files with their ACLs intact, how should i do it? A complete run down, including commands, output, and other supporting data, is located below. Finally, i apologise if this is not an appropriate place to discuss this particular release of rsync on (since it is, of course, Mr. McCutchen's release...). Thank you. ------------------------- Running rsync 3.0.0pre4 "matt" on both machines (BACKUP-SERVER and FILE-SERVER). Commandline run on BACKUP-SERVER: /usr/bin/sudo /usr/bin/rsync --stats -A -X -bavzR --fake-super --delete --numeric-ids --rsync-path="/usr/bin/sudo /usr/bin/rsync" --log-file=/opt/rsync-backup/rsync-backup.log --backup-dir=/opt/rsync-backup/FILE-SERVER/2007-11-05T11:31:43 -e "/usr/bin/ssh -i /home/rsync-backup/.ssh/id_rsa" rsync-backup@FILE-SERVER:/tmp/test /opt/rsync-backup/FILE-SERVER/CURRENT/ Output of backup command: receiving incremental file list tmp/ tmp/test/ tmp/test/t1/ tmp/test/t2/ tmp/test/t2/something tmp/test/t3/ Number of files: 6 Number of files transferred: 1 Total file size: 7 bytes Total transferred file size: 7 bytes Literal data: 7 bytes Matched data: 0 bytes File list size: 191 File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 51 Total bytes received: 287 sent 51 bytes received 287 bytes 676.00 bytes/sec total size is 7 speedup is 0.02 /tmp/test on FILE-SERVER: # ls -l /tmp/test total 24 drwxr-x---+ 2 rol samba 4096 Oct 30 13:17 t1 drwxr-x---+ 2 dma samba 4096 Oct 30 13:19 t2 drwxr-x---+ 2 jbc samba 4096 Oct 29 19:51 t3 # getfacl /tmp/test/t1 getfacl: Removing leading '/' from absolute path names # file: tmp/test/t1 # owner: rol # group: samba user::rwx user:720:r-x group::--- mask::r-x other::--- Result on BACKUP-SERVER: # ls -l /opt/rsync-backup/FILE-SERVER/CURRENT/tmp/test total 24 drwxr-x--- 2 root root 4096 2007-10-30 13:17 t1 drwxr-x--- 2 root root 4096 2007-10-30 13:19 t2 drwxr-x--- 2 root root 4096 2007-10-29 19:51 t3 # getfacl /opt/rsync-backup/FILE-SERVER/CURRENT/tmp/test/t1 getfacl: Removing leading '/' from absolute path names # file: opt/rsync-backup/FILE-SERVER/CURRENT/tmp/test/t1 # owner: root # group: root user::rwx group::r-x other::--- # attr -l /opt/rsync-backup/FILE-SERVER/CURRENT/tmp/test/t1 Attribute "rsync.%aacl" has a 24 byte value for /opt/rsync-backup/FILE-SERVER/CURRENT/tmp/test/t1 Attribute "rsync.%stat" has a 18 byte value for /opt/rsync-backup/FILE-SERVER/CURRENT/tmp/test/t1 # attr -g "rsync.% aacl" /opt/rsync-backup/FILE-SERVER/CURRENT/tmp/test/t1 Attribute "rsync.%aacl" had a 24 byte value for /opt/rsync-backup/FILE-SERVER/CURRENT/tmp/test/t1: ??? ? # attr -g "rsync.% stat" /opt/rsync-backup/FILE-SERVER/CURRENT/tmp/test/t1 Attribute "rsync.%stat" had a 18 byte value for /opt/rsync-backup/FILE-SERVER/CURRENT/tmp/test/t1: 40750 0,0 600:4999 Commandline of attempted restore of /tmp/test to FILE-SERVER: $ /usr/bin/sudo /usr/bin/rsync --fake-super --stats -A -X -a --rsync-path="/usr/bin/sudo /usr/bin/rsync" -e "/usr/bin/ssh -i /home/rsync-backup/.ssh/id_rsa" /opt/rsync-backup/FILE-SERVER/CURRENT/tmp/test rsync-backup@FILE-SERVER:/tmp Output of attempted restore command: ERROR: out of memory in get_xattr_data [sender] rsync error: error allocating core memory buffers (code 22) at util.c(116) [sender=3.0.0pre4] (Additional verbosity provides only the "opening connection using" and "sending incremental file list" lines - nothing more.) If --fake-super is removed from the commandline of attempted restore, the files are transferred as expected: Number of files: 5 Number of files transferred: 1 Total file size: 7 bytes Total transferred file size: 7 bytes Literal data: 7 bytes Matched data: 0 bytes File list size: 113 File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 192 Total bytes received: 47 sent 192 bytes received 47 bytes 159.33 bytes/sec total size is 7 speedup is 0.03 HOWEVER, the ACL information is not set. /tmp/test on FILE-SERVER after restore: # ls -l /tmp/test/ total 12 drwxr-x--- 2 root root 4096 Oct 30 13:17 t1 drwxr-x--- 2 root root 4096 Oct 30 13:19 t2 drwxr-x--- 2 root root 4096 Oct 29 19:51 t3 # getfacl t1 # file: t1 # owner: root # group: root user::rwx group::r-x other::---
Wayne Davison
2007-Nov-05 15:08 UTC
problem restoring via rsync when using stored ACLs (--fake-super)
On Mon, Nov 05, 2007 at 02:12:42PM +0100, Daniel Maher wrote:> when i attempt to restore while preserving the ACL data, i get the > following error: > > ERROR: out of memory in get_xattr_data [sender]Turns out the get_xattr_acl() routine needed to zero out the *len_p variable before calling get_xattr_data(). (My testing must have gotten lucky with a small value in the uninitialized memory.) The latest CVS version has this fixed. ..wayne..
Daniel Maher
2007-Nov-05 16:27 UTC
problem restoring via rsync when using stored ACLs (--fake-super)
On Mon, 2007-05-11 at 07:08 -0800, Wayne Davison wrote:> On Mon, Nov 05, 2007 at 02:12:42PM +0100, Daniel Maher wrote: > > when i attempt to restore while preserving the ACL data, i get the > > following error: > > > > ERROR: out of memory in get_xattr_data [sender] > > Turns out the get_xattr_acl() routine needed to zero out the *len_p > variable before calling get_xattr_data(). (My testing must have gotten > lucky with a small value in the uninitialized memory.) The latest CVS > version has this fixed. > > ..wayne..I made the change to xattrs.c and recompiled rsync on my backup server. Now when i run the backup command (which previously worked), i get this: $ /usr/bin/sudo /usr/bin/rsync --stats -A -X -bavzR --fake-super --delete --numeric-ids --rsync-path="/usr/bin/sudo /usr/bin/rsync" --log-file=/opt/rsync-backup/rsync-backup.log --backup-dir=/opt/rsync-backup/FILE-SERVER/2007-11-05T16:23:57 -e "/usr/bin/ssh -i /home/rsync-backup/.ssh/id_rsa" rsync-backup@FILE-SERVER:/tmp/test /opt/rsync-backup/FILE-SERVER/CURRENT/ receiving incremental file list *** glibc detected *** /usr/bin/rsync: free(): invalid pointer: 0x083bfeb0 *** ======= Backtrace: ========/lib/libc.so.6[0xa79df1] /lib/libc.so.6(cfree+0x90)[0xa7d430] /usr/bin/rsync[0x8067339] /usr/bin/rsync[0x8067b9f] /usr/bin/rsync[0x8052852] /usr/bin/rsync[0x8052c44] /usr/bin/rsync[0x805795f] /usr/bin/rsync[0x8061886] /usr/bin/rsync[0x8061b14] /usr/bin/rsync[0x8062c62] /lib/libc.so.6(__libc_start_main+0xe0)[0xa27f70] /usr/bin/rsync[0x804a621] ======= Memory map: =======00110000-00111000 r-xp 00110000 00:00 0 [vdso] 00111000-00118000 r-xp 00000000 fd:03 1730976 /usr/lib/libpopt.so.0.0.0 00118000-00119000 rwxp 00006000 fd:03 1730976 /usr/lib/libpopt.so.0.0.0 009f3000-00a0e000 r-xp 00000000 fd:03 1239106 /lib/ld-2.6.so 00a0e000-00a0f000 r-xp 0001a000 fd:03 1239106 /lib/ld-2.6.so 00a0f000-00a10000 rwxp 0001b000 fd:03 1239106 /lib/ld-2.6.so 00a12000-00b60000 r-xp 00000000 fd:03 1239122 /lib/libc-2.6.so 00b60000-00b62000 r-xp 0014e000 fd:03 1239122 /lib/libc-2.6.so 00b62000-00b63000 rwxp 00150000 fd:03 1239122 /lib/libc-2.6.so 00b63000-00b66000 rwxp 00b63000 00:00 0 00c16000-00c1a000 r-xp 00000000 fd:03 1239218 /lib/libattr.so.1.1.0 00c1a000-00c1b000 rwxp 00003000 fd:03 1239218 /lib/libattr.so.1.1.0 00c1d000-00c23000 r-xp 00000000 fd:03 1239210 /lib/libacl.so.1.1.0 00c23000-00c24000 rwxp 00005000 fd:03 1239210 /lib/libacl.so.1.1.0 00dc3000-00dce000 r-xp 00000000 fd:03 1239172 /lib/libgcc_s-4.1.2-20070925.so.1 00dce000-00dcf000 rwxp 0000a000 fd:03 1239172 /lib/libgcc_s-4.1.2-20070925.so.1 08048000-0809c000 r-xp 00000000 fd:03 1739864 /usr/bin/rsync 0809c000-0809f000 rw-p 00054000 fd:03 1739864 /usr/bin/rsync 0809f000-080af000 rw-p 0809f000 00:00 0 0834b000-08401000 rw-p 0834b000 00:00 0 b7b00000-b7b21000 rw-p b7b00000 00:00 0 b7b21000-b7c00000 ---p b7b21000 00:00 0 b7c0e000-b7c50000 rw-p b7c0e000 00:00 0 b7c71000-b7c72000 rw-p b7c71000 00:00 0 b7c72000-b7c79000 r--s 00000000 fd:03 1761571 /usr/lib/gconv/gconv-modules.cache b7c79000-b7c7a000 rw-p b7c79000 00:00 0 b7c7a000-b7d5c000 r--p 009df000 fd:03 1730802 /usr/lib/locale/locale-archive b7d5c000-b7f5c000 r--p 00000000 fd:03 1730802 /usr/lib/locale/locale-archive b7f5c000-b7f5e000 rw-p b7f5c000 00:00 0 bf82b000-bf841000 rw-p bf82b000 00:00 0 [stack] rsync: writefd_unbuffered failed to write 8 bytes [receiver]: Broken pipe (32) rsync error: error in rsync protocol data stream (code 12) at io.c(1501) [receiver=3.0.0pre4] Any ideas?
Daniel Maher
2007-Nov-06 09:59 UTC
problem restoring via rsync when using stored ACLs (--fake-super)
On Mon, 2007-05-11 at 17:27 +0100, Daniel Maher wrote:> On Mon, 2007-05-11 at 07:08 -0800, Wayne Davison wrote: > > On Mon, Nov 05, 2007 at 02:12:42PM +0100, Daniel Maher wrote: > > > when i attempt to restore while preserving the ACL data, i get the > > > following error: > > > > > > ERROR: out of memory in get_xattr_data [sender] > > > > Turns out the get_xattr_acl() routine needed to zero out the *len_p > > variable before calling get_xattr_data(). (My testing must have gotten > > lucky with a small value in the uninitialized memory.) The latest CVS > > version has this fixed. > > > > ..wayne.. > > I made the change to xattrs.c and recompiled rsync on my backup server. > Now when i run the backup command (which previously worked), i get this:After re-compiling with the latest xattrs.c from CVS, everything appears to work as it should. The ACL data, stored as xattrs via --fake-super, is properly restored on the target server. Thank you Mr. Davison for your rapid response. :) -- dan.