The background:
I host one of the unofficial slackware mirrors & push to a high speed
(unlisted) mirror that several other unofficials use for pulls. To save
time/disk/bandwidth I use a custom script to hardlink the trees prior to pushing
the updates to said unlisted mirror.
The commandline (linewrap from hell, sorry):
(note: username/password/hostname/modulename deliberately obfuscated)
export RSYNC_PASSWORD="password"
rsync --archive --compress --hard-links --partial --progress --sparse --verbose
--delete-after --include /.pending --include /slackware-8.1 --include
/slackware-9.0 --include /slackware-9.1 --include /slackware-10.0 --include
/slackware-10.1 --include /slackware-current --exclude ".*.??????"
--exclude "/*" /pub/mirrors/ftp.slackware.com/pub/slackware/
rsync://username@hostname/modulename/
The errors (more linewrap hell):
rsync: link "/slackware-10.1/patches/source/cvs/slack-desc" (in
modulename) => slackware-10.0/source/d/cvs/slack-desc failed: No such file or
directory (2)
rsync: link "/slackware-10.1/patches/source/python/slack-desc" (in
modulename) => slackware-10.0/source/d/python/slack-desc failed: No such file
or directory (2)
rsync: link "/slackware-8.1/patches/source/python/slack-desc" (in
modulename) => slackware-10.0/source/d/python/slack-desc failed: No such file
or directory (2)
rsync: link "/slackware-9.0/patches/source/python/slack-desc" (in
modulename) => slackware-10.0/source/d/python/slack-desc failed: No such file
or directory (2)
rsync: link "/slackware-9.1/patches/source/python/slack-desc" (in
modulename) => slackware-10.0/source/d/python/slack-desc failed: No such file
or directory (2)
rsync: link
"/slackware-10.1/patches/source/python/slack-desc.python-demo" (in
modulename) => slackware-10.0/source/d/python/slack-desc.python-demo failed:
No such file or directory (2)
rsync: link
"/slackware-9.1/patches/source/python/slack-desc.python-demo" (in
modulename) => slackware-10.0/source/d/python/slack-desc.python-demo failed:
No such file or directory (2)
rsync: link
"/slackware-10.1/patches/source/python/slack-desc.python-tools" (in
modulename) => slackware-10.0/source/d/python/slack-desc.python-tools failed:
No such file or directory (2)
rsync: link
"/slackware-9.1/patches/source/python/slack-desc.python-tools" (in
modulename) => slackware-10.0/source/d/python/slack-desc.python-tools failed:
No such file or directory (2)
rsync error: some files could not be transferred (code 23) at main.c(789)
Observations & misc info:
In all, 26 unique files (going by inodes) were hardlinked by my custom script
prior to the push. The above represents exactly 4 of them. Curious that the
other 22 didn't generate errors during the push.
There appears to be no rhyme nor reason to single these files out. They're
neither the most linked nor the least, not the largest nor the smallest. Nothing
particularly unique about their permissions, paths, whatever.. just day-to-day
mirror update stuff.
I've been using the above-quoted commandline, with obvious
'--include' changes as time goes by, for the last several years. The
only other time I saw this error was while pushing the previous slackware update
(2005-apr-05), but assumed it was a burp caused by something I had done so
didn't report it.
$ rsync --version
rsync version 2.6.4 protocol version 29
Copyright (C) 1996-2005 by Andrew Tridgell and others
<http://rsync.samba.org/>
Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles,
inplace, IPv6, 64-bit system inums, 64-bit internal inums
This version known to be in use at both ends since 2005-mar-31.
Shameless plug:
http://alphageek.dyndns.org/linux/slackware-mirrors.shtml
Erik
--
"I really want a license to do just two things: make the code available
to others, and make sure that improvements stay that way. That's really
it. Nothing more, nothing less. Everything else is fluff."
-- Linus Torvalds
The background:
I host one of the unofficial slackware mirrors & push to a high speed
(unlisted) mirror that several other unofficials use for pulls. To save
time/disk/bandwidth I use a custom script to hardlink the trees prior to pushing
the updates to said unlisted mirror.
The commandline (linewrap from hell, sorry):
(note: username/password/hostname/modulename deliberately obfuscated)
export RSYNC_PASSWORD="password"
rsync --archive --compress --hard-links --partial --progress --sparse --verbose
--delete-after --include /.pending --include /slackware-8.1 --include
/slackware-9.0 --include /slackware-9.1 --include /slackware-10.0 --include
/slackware-10.1 --include /slackware-current --exclude ".*.??????"
--exclude "/*" /pub/mirrors/ftp.slackware.com/pub/slackware/
rsync://username@hostname/modulename/
The errors (more linewrap hell):
rsync: link "/slackware-10.1/patches/source/cvs/slack-desc" (in
modulename) => slackware-10.0/source/d/cvs/slack-desc failed: No such file or
directory (2)
rsync: link "/slackware-10.1/patches/source/python/slack-desc" (in
modulename) => slackware-10.0/source/d/python/slack-desc failed: No such file
or directory (2)
rsync: link "/slackware-8.1/patches/source/python/slack-desc" (in
modulename) => slackware-10.0/source/d/python/slack-desc failed: No such file
or directory (2)
rsync: link "/slackware-9.0/patches/source/python/slack-desc" (in
modulename) => slackware-10.0/source/d/python/slack-desc failed: No such file
or directory (2)
rsync: link "/slackware-9.1/patches/source/python/slack-desc" (in
modulename) => slackware-10.0/source/d/python/slack-desc failed: No such file
or directory (2)
rsync: link
"/slackware-10.1/patches/source/python/slack-desc.python-demo" (in
modulename) => slackware-10.0/source/d/python/slack-desc.python-demo failed:
No such file or directory (2)
rsync: link
"/slackware-9.1/patches/source/python/slack-desc.python-demo" (in
modulename) => slackware-10.0/source/d/python/slack-desc.python-demo failed:
No such file or directory (2)
rsync: link
"/slackware-10.1/patches/source/python/slack-desc.python-tools" (in
modulename) => slackware-10.0/source/d/python/slack-desc.python-tools failed:
No such file or directory (2)
rsync: link
"/slackware-9.1/patches/source/python/slack-desc.python-tools" (in
modulename) => slackware-10.0/source/d/python/slack-desc.python-tools failed:
No such file or directory (2)
rsync error: some files could not be transferred (code 23) at main.c(789)
Observations & misc info:
In all, 26 unique files (going by inodes) were hardlinked by my custom script
prior to the push. The above represents exactly 4 of them. Curious that the
other 22 didn't generate errors during the push.
There appears to be no rhyme nor reason to single these files out. They're
neither the most linked nor the least, not the largest nor the smallest. Nothing
particularly unique about their permissions, paths, whatever.. just day-to-day
mirror update stuff.
I've been using the above-quoted commandline, with obvious
'--include' changes as time goes by, for the last several years. The
only other time I saw this error was while pushing the previous slackware update
(2005-apr-05), but assumed it was a burp caused by something I had done so
didn't report it.
$ rsync --version
rsync version 2.6.4 protocol version 29
Copyright (C) 1996-2005 by Andrew Tridgell and others
<http://rsync.samba.org/>
Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles,
inplace, IPv6, 64-bit system inums, 64-bit internal inums
This version known to be in use at both ends since 2005-mar-31.
Shameless plug:
http://alphageek.dyndns.org/linux/slackware-mirrors.shtml
Erik
--
"I really want a license to do just two things: make the code available
to others, and make sure that improvements stay that way. That's really
it. Nothing more, nothing less. Everything else is fluff."
-- Linus Torvalds
It would help to know what rsync was doing with the files when the error
happened, if all the hard-links for a particular hard-link group are
represented in the errors, and what the results were after the transfer.
The error messages reveal these files, separated into linked groups:
slackware-10.0/source/d/cvs/slack-desc
slackware-10.1/patches/source/cvs/slack-desc
slackware-10.0/source/d/python/slack-desc
slackware-10.1/patches/source/python/slack-desc
slackware-8.1/patches/source/python/slack-desc
slackware-9.0/patches/source/python/slack-desc
slackware-9.1/patches/source/python/slack-desc
slackware-10.0/source/d/python/slack-desc.python-demo
slackware-10.1/patches/source/python/slack-desc.python-demo
slackware-9.1/patches/source/python/slack-desc.python-demo
slackware-10.0/source/d/python/slack-desc.python-tools
slackware-10.1/patches/source/python/slack-desc.python-tools
slackware-9.1/patches/source/python/slack-desc.python-tools
- Are any other files (that were not excluded from the transfer) hard-
linked to these files?
- I assume that at least the first file in each list existed at the time
of the transfer (because the first file alphabetically was chosen as
the group lead), but it would help to know how many of the files
existed beforehand.
- I also assume that each of these lead files was changed in some way,
correct? I.e. did the first file in each group get mentioned as being
updated prior to the errors being output?
- After the transfer, are any of these files present on the destination
system (in either the old or new form)?
- If you re-run the command, do the same errors repeat?
I haven't seen any problems in my testing, so I'll need to figure out a
way to duplicate the failure before I can fix it.
..wayne..