rsync - Feb 2008 - Rsync 2.6.9 does not skip any files based on modification time

Hi

I am trying to rsync some ghost images from a windows client running Windows
XP to my Linux server. The problem is that rsync sends the complete files
again even if nothing changed on the client side. The only way to avoid this
is to use the "-c"-option but this takes nearly as long as uploading
the
files would.

The server is running rsync-2.6.9, /etc/rsyncd.conf looks as follows.
/mnt/D/ is a FAT32 partition.

read only = false
use chroot = true
log format = %h %o %f %l %b
log file = /var/log/rsyncd.log
secrets file = /etc/rsyncd.secrets

[Backup]
path = /mnt/D/Backup
auth users = stefan
uid = stefan

[HDD-Abbilder]
path = /mnt/D/HDD-Abbilder
auth users = stefan
uid = stefan


The client is using cwRsync_2.0.10, which also uses rsync 2.6.9. I am using
the patched cygwin1.dll from
http://www.okisoft.co.jp/esc/utf8-cygwin/download.html in order to preserve
filenames with german umlauts when uploading to the server. This is the line
used to call rsync from cwrsync.cmd
rsync -rvvvv --del --modify-window=3 /cygdrive/E/HDD-Abbilder/
stefan@192.168.99.10::HDD-Abbilder

How can I get rsync to skip files based on modification time ?

Bye
Stefan
-------------- next part --------------
HTML attachment scrubbed and removed

On Fri 08 Feb 2008, Stefan Malte Schumacher wrote:
> 
> I am trying to rsync some ghost images from a windows client running
Windows
> XP to my Linux server. The problem is that rsync sends the complete files
> again even if nothing changed on the client side. The only way to avoid
this
> is to use the "-c"-option but this takes nearly as long as
uploading the
> files would.
> 
> The server is running rsync-2.6.9, /etc/rsyncd.conf looks as follows.
> /mnt/D/ is a FAT32 partition.
[...]
> used to call rsync from cwrsync.cmd
> rsync -rvvvv --del --modify-window=3 /cygdrive/E/HDD-Abbilder/
> stefan@192.168.99.10::HDD-Abbilder
You're correctly using --modify-window, which is otherwise the probable
solution to your problem. However, somehow rsync still considers an
update necessary. You can use -i (--itemize-changes) to let rsync show
what attributes are incorrect, triggering the update.  Once you know
that, you can try to fix it.


Paul Slootman

>
>
> Hmm, it does seem that Stefan is missing the -t option.
> As I basically always do "-a", I overlooked this.
>

Hello guys

Thanks for the quick replies. I just added the -tO option as well as the -i
option.
The output looks this way :
<f..t..... Avalon/filename.gho>
After this rsync complains that it cannot set file names on
"/Avalon/filename.gho" because the Operation is not permitted. The
FAT-partition is mounted with " rw,user,nodev,nosuid,umask=000" so in
theory
everything the user stefan is able to - as whom rsync is doing file
operations if I understand the uid-option correctly - should be possibly for
rsync as well. Should I try to set the uid in the rsyncd.conf to root ? Is
it possible that the chroot-option could cause this behavior ?

Bye
Stefan
-------------- next part --------------
HTML attachment scrubbed and removed

On Fri 08 Feb 2008, Stefan Malte Schumacher wrote:
> >
> > Hmm, it does seem that Stefan is missing the -t option.
> > As I basically always do "-a", I overlooked this.
> 
> Thanks for the quick replies. I just added the -tO option as well as the -i
> option.
> The output looks this way :
> <f..t..... Avalon/filename.gho>
> After this rsync complains that it cannot set file names on
> "/Avalon/filename.gho" because the Operation is not permitted.
The
I think you mean file owner? Of file times?
"cannot set file names" seems unlikely :-)


Paul Slootman

Paul Slootman wrote:
> 
> I think you mean file owner? Of file times?
> "cannot set file names" seems unlikely :-)
File  OWNER
File  GROUP

and DOS/Windows  complains
Me, I let it complain   --- much more complaining with cp than with rsync

DOS and Windows have different ideas of what file attributes are worth
preserving.
NTFS has some elaborate ACLs, but far too much trouble to do what is trivial
on *nix with users and groups.

??On Fri, 2008-02-08 at 10:59 +0100, Stefan Malte Schumacher wrote:
> Thanks for the quick replies. I just added the -tO option as well as
> the -i option. 
> The output looks this way :
> <f..t..... Avalon/filename.gho> 
> After this rsync complains that it cannot set file names on
> "/Avalon/filename.gho" because the Operation is not permitted.
The
> FAT-partition is mounted with " rw,user,nodev,nosuid,umask=000"
so in
> theory everything the user stefan is able to - as whom rsync is doing
> file operations if I understand the uid-option correctly - should be
> possibly for rsync as well. Should I try to set the uid in the
> rsyncd.conf to root ? Is it possible that the chroot-option could
> cause this behavior ?
The trouble is that FAT does not store file ownership, so Linux shows
all the files on the partition as belonging to a single user and group,
by default root.  User stefan can create files due to the 777
permissions, but ownership of the files is immediately given to root,
which prevents stefan from setting their mtimes arbitrarily (since Linux
requires that a non-root user own a file in order to set its mtime
arbitrarily).

To work around this irritating incompatibility, you could set the uid in
rsyncd.conf to root, or (safer) you could mount the partition with the
"uid=stefan" mount option to make all the files owned by stefan.

Matt

On Sat, 2008-02-09 at 01:49 +0100, Stefan Malte Schumacher
wrote:?
> Thanks, mounting the partition with uid=stefan did the trick. Probably
> more a cygwin than a rsync-question, but 
> does anybody have any suggestions how I can make rsync accept "E:
> \B?cher & Comics" as source path ?
How is rsync rejecting it?  "File has vanished"?  If so, see this
message:

http://lists.samba.org/archive/rsync/2008-January/019696.html

Matt

For these cases, I add the option --size-only.  I guarantee it will
fix it.  rsync sees files as different b/c of mismatch between fat and
linux filesystems. using -t also helps. -a does not because it tries
to do impossible things.

Hth

PJ

On 2/8/08, Matt McCutchen <matt@mattmccutchen.net>
wrote:
> On Sat, 2008-02-09 at 01:49 +0100, Stefan Malte Schumacher wrote:?
> > Thanks, mounting the partition with uid=stefan did the trick. Probably
> > more a cygwin than a rsync-question, but
> > does anybody have any suggestions how I can make rsync accept "E:
> > \B?cher & Comics" as source path ?
>
> How is rsync rejecting it?  "File has vanished"?  If so, see this
> message:
>
> http://lists.samba.org/archive/rsync/2008-January/019696.html
>
> Matt
>
> --
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>

-- 
Paul E. Johnson
Professor, Political Science
1541 Lilac Lane, Room 504
University of Kansas

>
>
> How is rsync rejecting it?  "File has vanished"?  If so, see this
> message:
>
> http://lists.samba.org/archive/rsync/2008-January/019696.html
>
> Matt


rsync outputs the following :

building file list ... rsync: link_stat "/cygdrive/E/B\#302\#263cher "
failed:
No such file or directory (2)
rsync error: some files could not be transferred (code 23) at main.c(977)
[sender=2.6.9]
...

It seems that the problem are not only the spaces but the Umlaute in the
filename. I renamed the folder from "B?cher & Comics" to
"B?cher", but the
error message stays the same. I have read the link from Matt but the
codepage for non-unicode apps was already set to german. FYI : I am already
using the hacked cygwin1.dll.

Bye
Stefan
-------------- next part --------------
HTML attachment scrubbed and removed

>
> It seems that the problem are not only the spaces but the Umlaute in the
> filename. I renamed the folder from "B?cher & Comics" to
"B?cher", but the
> error message stays the same. I have read the link from Matt but the
> codepage for non-unicode apps was already set to german. FYI : I am already
> using the hacked cygwin1.dll.
>

I just renamed "B?cher & Comics" to "Buecher" and then
tried to rsync again.
This time it works and rsync even copies the subdirectories  - one of which
is named "B?cher & Rollenspiele" correctly. So the only problem to
be solved
is how to get rsync to accept "B?cher & Comics" as command line
parameter.

Bye
Stefan
-------------- next part --------------
HTML attachment scrubbed and removed

>
>
> rsync -rvtO --del --files-from=/cygdrive/C/Programme/cwRsync/rsync.txt
> \       --modify-window=3 / stefan@192.168.99.10::Backup
>
> > How should I list E:\B?cher & Comics in rsync.txt ?
> > As /cygdrive/E/B?cher & Comics or should I just use single or
double
> > quotes ?
>
> As you said.  rsync.txt should contain these two lines:
>
> ?/cygdrive/D/Temp
> /cygdrive/E/B?cher & Comics

I just tried what you suggested Matt but unfortunately Rsync again complains
about the filenames -
"B?cher & Comics" changes to "B_cher & Comics" and
for the subdirectory
"B?cher & Rollenspiele" I get a "file has vanished"  for
each file in it and
even the directory itself is not created.  If I change the name of the
directory from "B?cher & Comics" to "Bucher" and call it
directly from the
command line instead of using --files-from "B?cher & Rollenspiele"
is
created and the files in it are copied. I have read the posting you read
concerning the "file has vanished"-message and the language for
non-unicode
apps is set to german !

Bye
Stefan
-------------- next part --------------

3j?Zr???
???y??v?????