rsync - Jul 2009 - DO NOT REPLY [Bug 6590] New: [sender] could not find xattr #1 for home/jdoe/TheFresh

https://bugzilla.samba.org/show_bug.cgi?id=6590

           Summary: [sender] could not find xattr #1 for home/jdoe/TheFresh
           Product: rsync
           Version: 3.0.5
          Platform: Other
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P3
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: stlman at poczta.fm
         QAContact: rsync-qa at samba.org


I get a message like the one in the summary when I try to copy files with their
EAs (xattrs.c:591 in 3.0.6, I run 3.0.5). However, "TheFresh"
directory has got
no attributes (empty output of getfattr -dm. /home/jdoe/TheFresh).

The error does not occur during a dry-run, although it shows a line like this
(-i)
.d........x home/jdoe/TheFresh/
when neither local nor remote (target) copy of the directory have any EAs.
The command is:
/usr/bin/rsync -XAHain --stats --delete-excluded --delete --iconv=utf8,utf8
--password-file=/root/rsync/rsync.backup.cred --filter='.
/root/rsync/system.exclude' / backup at 10.1.2.6::violetta/0

There seem to be completely nothing special about this /home/jdoe/TheFresh
directory whatsoever.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=6590





------- Comment #1 from stlman at poczta.fm  2009-07-29 18:19 CST -------
OK I managed to work it around somehow. I removed trusted.SGI_ACL_DEFAULT
attribute from several files in the /home/jdoe directory (it hasn't been set
on
TheFresh, I think).

getfattr -dm. didn't show it but it was there as I moved the files from XFS.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=6590

SATOH Fumiyasu <fumiyas at osstech.co.jp> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fumiyas at osstech.co.jp

--- Comment #2 from SATOH Fumiyasu <fumiyas at osstech.co.jp> 2011-04-18
23:53:39 UTC ---
I met the same problem here with transferring from rsync 3.0.8 + Solaris 10
xattrs patch (backported from 3.1.0dev) on Solaris 10 to rsync 3.0.8 on
Debian GNU/Linux.

I'm using the following command-line:

# rsync \
    --password-file /srv/osstech-work/etc/rsync.password \
    --iconv UTF-8,EUC-JP-MS \
    --archive \
    --acls \
    --xattrs \
    --hard-links \
    --numeric-ids \
    --delete \
    --stats \
    --partial \
    --progress \
    remote::home \
    /export/home \
   ;

When I remove "--iconv UTF-8,EUC-JP-MS" from the above command-line,
the problem is gone.

Can you try to remove --iconv option from your command-line?

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=6590

Mantas M. <grawity at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |grawity at gmail.com

--- Comment #3 from Mantas M. <grawity at gmail.com> 2014-04-24 10:20:31
UTC ---
I'm hitting the same (or at least a very similar) bug when trying to use
--fake-super when backing up a server that runs Samba 4, and I always get the
following error when rsync reaches the Samba "sysvol":

| [sender] could not find xattr #1 for Adm
| rsync error: protocol incompatibility (code 2) at xattrs.c(620)
[sender=3.1.0]

(Both ends run 3.1.0; I also tried 3.1.1pre1.)

It seems this only happens when the "Adm" directory has xattrs from
multiple
namespaces, security.*, system.* and user.* ... (specifically, Samba sets
security.NTACL and user.DOSATTRIB), and if I remove either security.NTACL _or_
user.DOSATTRIB, it works fine.

To reproduce:

#!/bin/sh -ex
mkdir Adm
sudo setfattr --restore=- <<'EOF'
# file: Adm
security.NTACL=0sBAAEAAAAAgAEAAIAAQAIlTbcbYS4nDSXwRaCyfUXTGYaUGme0BW6093Nz6ifNwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAcG9zaXhfYWNsAHhnWbOkV88BuuACskYEKF40Vz8II2s/PkYW216aikVHSnkIiA2SLHYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEABIC0AAAAxAAAAAAAAADgAAAAAQIAAAAAAAUgAAAAIAIAAAEFAAAAAAAFFQAAAJn1WnqkTiRm4eaHmAECAAACALgABwAAAAADJAD/AR8AAQUAAAAAAAUVAAAAmfVaeqROJGbh5oeYAAIAAAADJAD/AR8AAQUAAAAAAAUVAAAAmfVaeqROJGbh5oeYBwIAAAAAGAD/AR8AAQIAAAAAAAUgAAAAIAIAAAALFAD/AR8AAQEAAAAAAAMAAAAAAAMUAP8BHwABAQAAAAAABRIAAAAAAxQAqQASAAEBAAAAAAAFCwAAAAADFACpABIAAQEAAAAAAAUJAAAA
system.posix_acl_access=0sAgAAAAEABwD/////AgAHAMLGLQACAAUAw8YtAAIABwDGxi0AAgAHAMjGLQACAAUAysYtAAQAAAD/////CAAAAGQAAAAIAAcAwMYtAAgABwDCxi0ACAAFAMPGLQAIAAcAxsYtAAgABwDIxi0ACAAFAMrGLQAQAAcA/////yAAAAD/////
system.posix_acl_default=0sAgAAAAEABwD/////AgAHAMDGLQACAAcAwsYtAAIABQDDxi0AAgAHAMbGLQACAAcAyMYtAAIABQDKxi0ABAAAAP////8IAAAAZAAAAAgABwDCxi0ACAAFAMPGLQAIAAcAxsYtAAgABwDIxi0ACAAFAMrGLQAQAAcA/////yAAAAD/////
user.DOSATTRIB=0sMHgxMAAAAwADAAAAEQAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA42UbOkV88BAAAAAAAAAAAEOF
sudo rsync -a -X --fake-super Adm/ Adm-new/

[receiver] could not find xattr #1 for .
rsync error: protocol incompatibility (code 2) at xattrs.c(620)
[receiver=3.1.0]

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=6590

--- Comment #4 from Mantas M. <grawity at gmail.com> 2014-04-24 10:23:21
UTC ---
Ah, on the second thought, I'm actually looking for bug 9594...

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=6590

--- Comment #5 from Hans-Kristian <hkbakke at gmail.com> ---
I am running into this on rsync v3.1.1 in Debian Jessie.

The symptoms are the same as #3, and as 3.1.1 have the patch in
https://bugzilla.samba.org/show_bug.cgi?id=9594 as I have understood I think
this is another issue.

Command:
rsync -avihh --stats --out-format=%i %C %n%L --numeric-ids
--partial-dir=.rsync-partial --timeout=600 --acls --xattrs --fake-super -e ssh
-i <key> -f merge <rule_file> root@<source>:/
<destination>

Result:
[sender] could not find xattr #1 for root/test
rsync error: protocol incompatibility (code 2) at xattrs.c(622) [sender=3.1.1]
2015-08-15 17:50:54 -> Rsync returned non-zero exit code [ 2 ]

The issue is caused by attributes set via Samba4 and only when
"security.NTACL"
and "user.DOSATTRIB" coexists. Other xattrs set on the same file
doesn't seem
to matter.

Reproducer for a problematic source file:
---
#!/bin/bash

set -e

dest="/root/test"

[[ -f ${dest} ]] && rm -f "${dest}"
echo "1" > "${dest}"

setfattr -n "security.NTACL" -v
"0sBAAEAAAAAgAEAAIAAQCN16Y6BFqNVEMomAM+v42pydr5hYsoWGyPpwyCBv866wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAcG9zaXhfYWNsAJzHXbI01dABZggOgcJ1PT/NJyBGmeTVwbN/3SXfnPAVXgM5MFQOssUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEABIC0AAAA0AAAAAAAAADsAAAAAQUAAAAAAAUVAAAAeg7SFXszBjXjUnZeUAQAAAEFAAAAAAAFFQAAAHoO0hV7MwY141J2XgECAAACAHgABAAAAAAAJAD/AR8AAQUAAAAAAAUVAAAAeg7SFXszBjXjUnZeUAQAAAAAFAD/AR8AAQEAAAAAAAUSAAAAAAAkAAAAAAABBQAAAAAABRUAAAB6DtIVezMGNeNSdl4BAgAAAAAUAAAAAAABAQAAAAAAAQAAAAA="
"${dest}"
setfattr -n "user.DOSATTRIB" -v
"0sMHgyMAAAAwADAAAAEQAAACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABE9FdC2w9ABAAAAAAAAAAA="
"${dest}"

---

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=6590

--- Comment #6 from SATOH Fumiyasu <fumiyas at osstech.co.jp> ---
You can reproduce this bug by the following:

# rm -rf src dst
# mkdir src dst
# touch src/f
# setfattr -n security.a -v 0sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
src/f
# setfattr -n user.a -v 0sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA src/f
# rsync --fake-super -X src/f dst
[sender] could not find xattr #1 for f
rsync error: protocol incompatibility (code 2) at xattrs.c(622) [sender=3.1.2]

In the first case, "security.a" and "user.a" have 33 bytes
each other.
If "security.a" and/or "user.a" is reduced to 32 or shorter,
rsync does not fail as the following:

# rm -rf src dst
# mkdir src dst
# touch src/f
# setfattr -n security.a -v 0sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
src/f
# setfattr -n user.a -v 0sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= src/f
# rsync --fake-super -X src/f dst
(no output, thus no error)

And then, if you change "user.a" to "user.s" (or
"user.t", "user.u" ...)
from the first case, rsync does not fail too:

# rm -rf src dst
# mkdir src dst
# touch src/f
# setfattr -n security.a -v 0sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
src/f
# setfattr -n user.s -v 0sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA src/f
# rsync --fake-super -X src/f dst
(no output, thus no error)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=6590

--- Comment #7 from SATOH Fumiyasu <fumiyas at osstech.co.jp> ---
(In reply to SATOH Fumiyasu from comment #6)

I can reproduce this bug on Debian sid (amd64) with Linux kernel 4.3.3,
ext4 and XFS.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=6590

--- Comment #8 from SATOH Fumiyasu <fumiyas at osstech.co.jp> ---
(In reply to SATOH Fumiyasu from comment #7)

# rm -rf src dst
# mkdir src dst
# touch src/f
# setfattr -n security.a -v 0sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
src/f
# setfattr -n user.rsync -v 0sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
src/f
# rsync "$@" --fake-super -X src/f dst
[sender] could not find xattr #1 (2) for f cnt=1
rsync error: protocol incompatibility (code 2) at xattrs.c(622) [sender=3.1.2]

Change "user.rsync" to "user.rsynd":

# rm -rf src dst
# mkdir src dst
# touch src/f
# setfattr -n security.a -v 0sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
src/f
# setfattr -n user.rsynd -v 0sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
src/f
# rsync "$@" --fake-super -X src/f dst
(no output, thus no error)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=6590

--- Comment #9 from SATOH Fumiyasu <fumiyas at osstech.co.jp> ---
(In reply to SATOH Fumiyasu from comment #6)

The boundary "33 bytes" depends on "#define MAX_FULL_DATUM
32" in xattrs.c.
If I increase it to 64, the boundary value becomes 65 bytes
(i.e, MAX_FULL_DATUM+1 bytes).

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=6590

--- Comment #10 from SATOH Fumiyasu <fumiyas at osstech.co.jp> ---
Created attachment 11780
  --> https://bugzilla.samba.org/attachment.cgi?id=11780&action=edit
Concept patch

This is a concept patch to describe what the problem is.
This fixes this bug, but does NOT a fundamental problem I think.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=6590

--- Comment #11 from SATOH Fumiyasu <fumiyas at osstech.co.jp> ---
(In reply to SATOH Fumiyasu from comment #10)
> This fixes this bug, but does NOT a fundamental problem I think.
Oops... Correction: This patch does NOT fix a fundamental problem I think.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=6590

--- Comment #12 from Roman Dubtsov <dubtsov at gmail.com> ---
Hi, I think I've found the source of this bug. 

send_xattr_request() skips xattrs that are abbreviated:

552         case XSTATE_ABBREV:
553             /* Items left abbreviated matched the sender's checksum, so
554              * the receiver will cache the local data for future use. */
555             if (am_generator)
556                 rxa->datum[0] = XSTATE_DONE;
557             continue;

But recv_xattrs_request() does not have a good way to detect skipped elements
because it is only able to traverse the xattrs list once.

I do not understand the code well enough to understand if it's reasonable to
skip abbreviated xattrs, so I do not have a better solution than the one by
SATOH Fumiyasu which modifies recv_xattrs_request() to loop over xattrs list
multiple times.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=6590

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #13 from Wayne Davison <wayned at samba.org> ---
I have committed a fix for this bug to git which will be in the next release.
Thanks for the reproducing script, diagnosis, and suggested patch. My apologies
for the delay in getting this fixed.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.