rsync - Jun 2008 - DO NOT REPLY [Bug 5520] New: Rsync cannot copy to a vfat partition on kernel 2.6.25+

https://bugzilla.samba.org/show_bug.cgi?id=5520

           Summary: Rsync cannot copy to a vfat partition on kernel 2.6.25+
           Product: rsync
           Version: 3.0.3
          Platform: All
               URL: https://bugzilla.redhat.com/show_bug.cgi?id=449080
        OS/Version: Linux
            Status: NEW
          Severity: major
          Priority: P3
         Component: core
        AssignedTo: wayned@samba.org
        ReportedBy: piskozub@iopan.gda.pl
         QAContact: rsync-qa@samba.org


This was reported by me on the RedHat Bugzilla (URL given) and reclassified as
a kernel bug. However, this seems to be caused by a conscious decision of the
vfat Linux kernel developers and therefore may be easier to solve from the
rsync end.

In a nutshell: Rsync cannot now copy to a vfat partition even if the user is
root (yes!). This is due to new and more strict vfat operations security model
adopted in the kernel. When rsync tries to mkstemp it gets a "not
permitted"
error.

This bug was discussed on LKML (the best URL is
http://www.gossamer-threads.com/lists/linux/kernel/927626) with n conclusions
so far. However it is clear that:

1) Kernel 2.6.25 (and newer) does not let "allows only acceptable chmod(),
and
if not acceptable, it returns -EPERM. Old one allows even if it can't store
the
disk inode."

2) As Hirofumi Ogawa explained to me in an email: 
"rsync will try to set permission to 0700 or something. But, FAT can't
store almost permissions to storage, because FAT doesn't have permission
field on disk inode (FAT have permission field on only memory
inode). So, chmod() returns -EPERM."

3) It is not clear how and whether at all it will be solved in the kernel
because as Pavel Machek put it: "So we have choice between security hole
and
regression...?"

The question is: Can this be solved from the rsync end?


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5520


wayned@samba.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED




------- Comment #1 from wayned@samba.org  2008-06-14 13:13 CST -------
I assume that the -EPERM error is at the kernel level, and the application gets
a return of -1 with errno = EPERM.  It would be much better to return errno
EINVAL so that the error could be easily distinguished from a real EPERM error
where the user doesn't have permission to operate on the file.  Since the
kernel call is complaining about being unable to support the requested
permissions, EINVAL seems like a much more logical error to me.  But let's
leave that aside for now.

I assume it is the fchmod() function that is returning the error (sadly, the
strace in the original bug report did not contain the process that was actually
failing).  If so, you can avoid the mkstemp failure by not asking rsync to
preserve permissions (e.g. --no-p).

So, is the issue that rsync is trying to create a mode 0600 file and the mount
options are set to make the mount world-readable/writable?  I'd be loath to
make rsync retry the fchmod() call with 0666 unless the user asked for that
explicitly.  Or perhaps have an option to just skip the fchmod() on the temp
file?

Any no-option ideas that are safe?  Perhaps test if the new temp file has bits
set outside the umask, and if so, skip the fchmod()?

You might also try changing your mount options to mount the media as a single
user, which I would imagine would make a mode of 0600 acceptable to the kernel.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5520





------- Comment #2 from piskozub@iopan.gda.pl  2008-06-14 16:26 CST -------
Changing the mount options and/or rsyncing as root did not help in my original
tests. But I probably missed a thing with the options reporting they do not
affect the problem (see below).

I did more tests today under the newest Fedora 9 kernel
(kernel-2.6.25.6-55.fc9).
You are right about the options. This happens only with -p (or -a which is
probably why I missed it before). I actually could successfully sync the vfat
partition replacing -a with -rt (because I did not need the other components of
-a on a FAT partition, anyway).

This means there is a simple workaround but still either rsync or the kernel
should be MHO modified to avoid the -p problem.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5520


devzero at web.de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |devzero at web.de




------- Comment #3 from devzero at web.de  2009-08-07 04:42 CST -------
i`m getting

neoware:/tmp # mount -o loop vfat.dat /mnt/
neoware:/tmp # rsync -av log /mnt
sending incremental file list
log
rsync: chown "/mnt/.log.X0sasg" failed: Operation not permitted (1)

sent 859 bytes  received 31 bytes  1780.00 bytes/sec
total size is 773  speedup is 0.87
rsync error: some files/attrs were not transferred (see previous errors) (code
23) at main.c(1040) [sender=3.0.4]

which is 

[pid 23439] lstat64("log", {st_mode=S_IFREG|0755, st_size=773, ...}) =
0
[pid 23439] lchown32("log", 1000, 100)  = -1 EPERM (Operation not
permitted)
[pid 23439] select(4, [3], [1], [1], {60, 0}) = 1 (out [1], left {60, 0})
[pid 23439] write(1, "<\0\0\10rsync: chown \"/mnt/log\"
fail"..., 64) = 64
[pid 23439] time(NULL)                  = 1249636822
[pid 23439] time(NULL)                  = 1249636822
[pid 23439] select(4, [3], [1], [1], {60, 0}) = 1 (out [1], left {60, 0})
[pid 23439] write(1, "\4\0\0\7\2p\0\0", 8) = 8
[pid 23439] time(NULL)                  = 1249636822
[pid 23439] munmap(0xb7ae5000, 266240)  = 0
[pid 23439] munmap(0xb7b90000, 135168)  = 0
[pid 23439] select(4, [3], [], NULL, {60, 0} <unfinished ...>
[pid 23438] <... select resumed> )      = 1 (in [5], left {59, 992000})
[pid 23438] read(5, "<\0\0\10", 4)      = 4
[pid 23438] select(6, [5], [], NULL, {60, 0}) = 1 (in [5], left {60, 0})
[pid 23438] read(5, "rsync: chown \"/mnt/log\" failed: "...,
60) = 60
[pid 23438] write(2, "rsync: chown \"/mnt/log\" failed:
"..., 59rsync: chown
"/mnt/log" failed: Operation
 not permitted (1)) = 59


i`m also wondering, why we get "EPERM". i would expect
"EINVAL" here.


Anyway - rsync tries to do chown on a filesystem where it`s not meant to do
that.

-p is invalid here, so is -a - but since -p is hidden in "-a" which is
commonly
and widely used default - and the problem is, that this is not obvious to the
average end user.

does rsync have filesystem capabilities detection ? rdiff-backup handles such
things very intelligently, since it checks what features exist on the target
filesystem. 

anyway - what about adding a hint to chown/chmod related error messages?
if rsync does not intelligently handle that, at least the user should be given
a hint where to look at.

neoware:/tmp # rsync -av log /mnt
sending incremental file list
rsync: chown "/mnt/log" failed: Operation not permitted (1)
HINT: not all filesystems support attributs like owner or acl`s, so you may
accidentally transfer permission information where you shouldn`t. (options like
-p or -a)


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5520





------- Comment #4 from devzero at web.de  2010-08-22 07:03 CST -------
Any update on this?


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5520


rsync at sanitarium.net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |rsync at sanitarium.net




------- Comment #5 from rsync at sanitarium.net  2011-02-28 19:22 CST -------
I believe that this question has been answered...

Using rsync's -a or -p is instructing it to do something that is impossible
to
do on a vfat (and a few others btw) filesystem.  Therefore rsync fails to do
what it was told to do and errors appropriately.

The alternative of having rsync ignore part of the user's instruction based
on
the fact that it will not work and therefore the user must just not be paying
attention is wrong and could easily lead to a security issue.  If someone were
to rsync a private file to a vfat mount point with insecure permissions that
file could then silently become public on the system.

There are many other rsync flags that will not work on a vfat formatted drive
and they should all produce appropriate errors too (assuming they encounter a
file that triggers that feature).

The reason that rdiff-backup handles this is that it is specifically a backup
program.  A backup of data doesn't have to look exactly like the data.  It
could for instance be a tarball on a vfat mount that contains hard links,
ownerships, permissions and other things that vfat can't handle.  Rsync
otoh,
is for synchronizing files and directories.  If the target filesystem can't
handle what the source filesystem is storing then rsync can't function.

So, the solution is to use -rt instead of -a.  Note that you will probably need
--modify-window as well.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.