William R. Dickson
2002-Jul-23 18:31 UTC
--delete doesn't work when transferring many files
This is a curious thing. I'm syncing two directories, containing some
250 subdirectories totalling around 11,000 files. This:
rsync --password-file=/usr/local/etc/rsync/.passwd --delete -rtv
--exclude-from=/usr/local/etc/rsync/excludes /usr/local/sourcedir
username@destserver.com::modulename
...works fine if both directories are already nearly in sync. Deleting a
few files from the source causes matching deletions on the destination.
But recently I made major changes to the source directory, which should
have resulted in nearly everything being deleted from the destination
server. However, it doesn't -- it simply starts uploading files.
If I change "-rtv" to "-rtvn", rsync correctly reports a
whole slew of
planned deletions -- but as soon as I try to run it for real, it simply
fails to do the actual deleting.
If it helps at all, the source system is a NetBSD/MacPPC machine, and the
destination is a FreeBSD/i386 machine.
Any ideas? I searched the list archives and google, but was unable to
find any similar issues.
Thanks,
-Bill
--
William R. Dickson -- Consuming the Earth's limited resources since 1968
wrd@awenet.com http://www.seavoy-dickson.org/
But I'm the luckiest guy on the lower east side
'Cuz I've got wheels, and you wanna go for a ride
- The Magnetic Fields
Granzow, Doug (NCI)
2002-Jul-24 12:08 UTC
--delete doesn't work when transferring many files
Is it possible that the destination directory is filling up? I believe
rsync saves deletes until the end, so that it knows there are no IO error
conditions that would cause it to exit without deleting. Filling up the
destination filesystem is probably one of those error conditions. Another
possibility would be permissions problems on the destination preventing
rsync from writing to the target directory.
Try adding another v to your command line ("-rtvv") to get more
verbosity;
rsync may tell you why it is skipping the deletes.
You could add --ignore-errors to force the --delete, but I would recommend
finding the cause of the problem first.
-----Original Message-----
From: William R. Dickson [mailto:wrd@awenet.com]
Sent: Tuesday, July 23, 2002 9:30 PM
To: rsync@lists.samba.org
Subject: --delete doesn't work when transferring many files
This is a curious thing. I'm syncing two directories, containing some 250
subdirectories totalling around 11,000 files. This:
rsync --password-file=/usr/local/etc/rsync/.passwd --delete -rtv
--exclude-from=/usr/local/etc/rsync/excludes /usr/local/sourcedir
username@destserver.com::modulename
...works fine if both directories are already nearly in sync. Deleting a
few files from the source causes matching deletions on the destination. But
recently I made major changes to the source directory, which should have
resulted in nearly everything being deleted from the destination server.
However, it doesn't -- it simply starts uploading files.
If I change "-rtv" to "-rtvn", rsync correctly reports a
whole slew of
planned deletions -- but as soon as I try to run it for real, it simply
fails to do the actual deleting.
If it helps at all, the source system is a NetBSD/MacPPC machine, and the
destination is a FreeBSD/i386 machine.
Any ideas? I searched the list archives and google, but was unable to find
any similar issues.
Thanks,
-Bill
--
William R. Dickson -- Consuming the Earth's limited resources since 1968
wrd@awenet.com http://www.seavoy-dickson.org/
But I'm the luckiest guy on the lower east side
'Cuz I've got wheels, and you wanna go for a ride
- The Magnetic Fields
--
To unsubscribe or change options:
http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
tim.conway@philips.com
2002-Jul-25 06:49 UTC
--delete doesn't work when transferring many files
That wouldn't be a problem, because rsync generates the lists and does the
compares, and if there are no errors or "--ignore-errors" is invoked
(logical or... no errors, ignore-errors is not tested), and
"--delete-after" is not invoked, the deletions are performed, followed
by
the sends. I haven't read the code, but i think that since
--ignore-errors is documented as applying only to the deletion phase, i
would expect that creation errors might not count against --delete.
It's all clearly documented in the man page.
Perhaps in his larger runs, he's running into items for which he has no
write permission. Sometimes, adding "--force" can overcome that,
though
that is more commonly used to force the wipeout of a dir being replaced
with a symlink.
I'd suggest that he choose a filesystem with plenty of empty space to
recieve a complete trace of his run, all threads (-f option to truss and
strace), and less that trace, searching for "unlink". Somewhere
around an
unlink line, he should see something that failed.
Tim Conway
tim.conway@philips.com
303.682.4917 office, 3039210301 cell
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(nnnnnnnnnnnn,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
".\n" '
"There are some who call me.... Tim?"
"Granzow, Doug (NCI)" <granzowd@mail.nih.gov>
Sent by: rsync-admin@lists.samba.org
07/24/2002 01:05 PM
To: "'William R. Dickson'" <wrd@awenet.com>
"'rsync@lists.samba.org'" <rsync@lists.samba.org>
cc: (bcc: Tim Conway/LMT/SC/PHILIPS)
Subject: RE: --delete doesn't work when transferring many
files
Classification:
Is it possible that the destination directory is filling up? I believe
rsync saves deletes until the end, so that it knows there are no IO error
conditions that would cause it to exit without deleting. Filling up the
destination filesystem is probably one of those error conditions. Another
possibility would be permissions problems on the destination preventing
rsync from writing to the target directory.
Try adding another v to your command line ("-rtvv") to get more
verbosity;
rsync may tell you why it is skipping the deletes.
You could add --ignore-errors to force the --delete, but I would recommend
finding the cause of the problem first.
-----Original Message-----
From: William R. Dickson [mailto:wrd@awenet.com]
Sent: Tuesday, July 23, 2002 9:30 PM
To: rsync@lists.samba.org
Subject: --delete doesn't work when transferring many files
This is a curious thing. I'm syncing two directories, containing some 250
subdirectories totalling around 11,000 files. This:
rsync --password-file=/usr/local/etc/rsync/.passwd --delete -rtv
--exclude-from=/usr/local/etc/rsync/excludes /usr/local/sourcedir
username@destserver.com::modulename
...works fine if both directories are already nearly in sync. Deleting a
few files from the source causes matching deletions on the destination.
But
recently I made major changes to the source directory, which should have
resulted in nearly everything being deleted from the destination server.
However, it doesn't -- it simply starts uploading files.
If I change "-rtv" to "-rtvn", rsync correctly reports a
whole slew of
planned deletions -- but as soon as I try to run it for real, it simply
fails to do the actual deleting.
If it helps at all, the source system is a NetBSD/MacPPC machine, and the
destination is a FreeBSD/i386 machine.
Any ideas? I searched the list archives and google, but was unable to
find
any similar issues.
Thanks,
-Bill
--
William R. Dickson -- Consuming the Earth's limited resources since 1968
wrd@awenet.com http://www.seavoy-dickson.org/
But I'm the luckiest guy on the lower east side
'Cuz I've got wheels, and you wanna go for a ride
- The Magnetic Fields
--
To unsubscribe or change options:
http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
--
To unsubscribe or change options:
http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html