rsync - Jun 2014 - [Bug 10675] New: rsyncing >2GB file onto fat32 partition should fail earlier

https://bugzilla.samba.org/show_bug.cgi?id=10675

           Summary: rsyncing >2GB file onto fat32 partition should fail
                    earlier
           Product: rsync
           Version: 3.1.0
          Platform: x86
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: darko.veberic at ijs.si
         QAContact: rsync-qa at samba.org


i have noticed that while rsyncing (-av) a whole directory (from linux ext4)
onto a fat32 filesystem on a usb disk there was a large >4GB file among the
files to be copied. while rsync should fail immediately encountering such a job
what really happened was strange: rsync continued to read all the other files
in the job but nothing has been written to the usb disk anymore. i have noticed
this since the i/o on my internal disk went to the max of 120 MB/s and the i/o
on the usb disk stayed at 0. at that point i pressed ctrl-c and, remarkably
late, got the corresponding error:

rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(632)
[sender=3.1.0]
rsync: write failed on "/media/luser/usbdisk/folder/dvd.iso": File too
large
(27)
rsync error: error in file IO (code 11) at receiver.c(389) [receiver=3.1.0]



how to reproduce: format a usb disk with mkfs.vfat and try to rsync a directory
containing some files plus a large file over the >2GB fat32 limit.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

On 6/26/2014 5:36 PM, samba-bugs at samba.org wrote:
> https://bugzilla.samba.org/show_bug.cgi?id=10675
>
>             Summary: rsyncing >2GB file onto fat32 partition should fail
>                      earlier
>             Product: rsync
>             Version: 3.1.0
>            Platform: x86
>          OS/Version: Linux
>              Status: NEW
>            Severity: normal
>            Priority: P5
>           Component: core
>          AssignedTo: wayned at samba.org
>          ReportedBy: darko.veberic at ijs.si
>           QAContact: rsync-qa at samba.org
>
>
> i have noticed that while rsyncing (-av) a whole directory (from linux
ext4)
> onto a fat32 filesystem on a usb disk there was a large >4GB file among
the
> files to be copied. while rsync should fail immediately encountering such a
job
> what really happened was strange: rsync continued to read all the other
files
> in the job but nothing has been written to the usb disk anymore. i have
noticed
> this since the i/o on my internal disk went to the max of 120 MB/s and the
i/o
> on the usb disk stayed at 0. at that point i pressed ctrl-c and, remarkably
> late, got the corresponding error:
>
> rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(632)
> [sender=3.1.0]
> rsync: write failed on "/media/luser/usbdisk/folder/dvd.iso":
File too large
> (27)
> rsync error: error in file IO (code 11) at receiver.c(389) [receiver=3.1.0]
>
>
>
> how to reproduce: format a usb disk with mkfs.vfat and try to rsync a
directory
> containing some files plus a large file over the >2GB fat32 limit.
>

I don't think it's reasonable to expect rsync, or any other such tool, 
to include a database of all the quirks and limitations of all the 
different filesystems in existence, including all their different 
versions and options.

But it might be possible for rsync to fail earlier on this particular 
problem by just attempting to allocate the space when it fist goes to 
transfer the file, and aborting if that fails. It wouldn't know or care 
what kind of filesystem the destination has. It wouldn't fail at the 
beginning of the rsync job, just at the beginning of that file.

I don't know how to handle --inplace, just add space to the end of the file?

-- 
bkw

https://bugzilla.samba.org/show_bug.cgi?id=10675

--- Comment #1 from Daniel <daniel-inet at gmx.net> ---
If the Topic should mean ">4GB":
I can confirm the problem.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.