rsync - Aug 2017 - hard-link reference error when trying to delete

Hello, all.  I've encountered an odd error that I haven't been able to
identify and resolve myself.  Hoping someone here could provide some
guidance.

I'm using the following command to rsync files between two servers, over
the internet (if that matters), via SSH:

rsync -ahu --timeout=600 --bwlimit=5120 -v -e "ssh -i /path/to/key"
--filter=". /path/to/filter" /path/to/source/ user at
remote.server.com:dest

This works perfectly fine.  However, I recently realized that I forgot
to the --delete option, so there are a great number of stale files on
the remote server at this point.

Adding --delete is when I run into my problem:

sending incremental file list
hard-link reference out of range: 105 (10)
rsync error: protocol incompatibility (code 2) at flist.c(769)
[Receiver=3.1.2]

The source and destination servers are running the same versions of
rsync, ssh, etc., so I think that last line can be ignored.  The
"hard-link reference out of range" line seems to be the main issue,
but
I cannot figure out what would be causing this.  I enabled various debug
options in both rsync and ssh, but I can't find anything in the debug
output that would indicate the problem.  I tried searching, but the only
result I've found outside of source code
(https://github.com/xonsh/xonsh/issues/2134) is for something that seems
unrelated - it's a python project that calls rsync, apparently
incorrectly, resulting in that error.  In my case I'm calling rsync
directly from bash, so the given solution in that bug report doesn't
seem to apply here.

Any suggestions?  I'm stumped, and I really need to figure out how to
clean up the stale files on the remote server as it's running low on
space  I'm happy to provide any additional information regarding debug
logs or whatever if it would help, just didn't want to spam the list
with a bunch of potentially irrelevant information.

Thanks.  Appreciate any and all guidance.

-- 
Jared

Have you tried doing the clean shell test?

ssh -i /path/to/key user at remote.server.com true > testfile
if testfile isn't 0 bytes then whatever is producing the data that ended
up in there is your problem.


On 08/15/2017 04:13 PM, jared via rsync wrote:
> Hello, all.  I've encountered an odd error that I haven't been able
to
> identify and resolve myself.  Hoping someone here could provide some
> guidance.
> 
> I'm using the following command to rsync files between two servers,
over
> the internet (if that matters), via SSH:
> 
> rsync -ahu --timeout=600 --bwlimit=5120 -v -e "ssh -i
/path/to/key"
> --filter=". /path/to/filter" /path/to/source/ user at
remote.server.com:dest
> 
> This works perfectly fine.  However, I recently realized that I forgot
> to the --delete option, so there are a great number of stale files on
> the remote server at this point.
> 
> Adding --delete is when I run into my problem:
> 
> sending incremental file list
> hard-link reference out of range: 105 (10)
> rsync error: protocol incompatibility (code 2) at flist.c(769)
> [Receiver=3.1.2]
> 
> The source and destination servers are running the same versions of
> rsync, ssh, etc., so I think that last line can be ignored.  The
> "hard-link reference out of range" line seems to be the main
issue, but
> I cannot figure out what would be causing this.  I enabled various debug
> options in both rsync and ssh, but I can't find anything in the debug
> output that would indicate the problem.  I tried searching, but the only
> result I've found outside of source code
> (https://github.com/xonsh/xonsh/issues/2134) is for something that seems
> unrelated - it's a python project that calls rsync, apparently
> incorrectly, resulting in that error.  In my case I'm calling rsync
> directly from bash, so the given solution in that bug report doesn't
> seem to apply here.
> 
> Any suggestions?  I'm stumped, and I really need to figure out how to
> clean up the stale files on the remote server as it's running low on
> space  I'm happy to provide any additional information regarding debug
> logs or whatever if it would help, just didn't want to spam the list
> with a bunch of potentially irrelevant information.
> 
> Thanks.  Appreciate any and all guidance.
> 
-- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 224 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20170815/50eab3d5/signature.sig>

Hi, Kevin.  Thank you for the suggestion.  It triggered a memory that I
had set some restrictions on this rsync copy a while back.  Sure enough,
in ~/.ssh/authorized_keys:

command="rsync --server -vulogDtpre.iLsfxC --timeout=600 --bwlimit=5120
. dest" ssh-rsa

Tacking on --delete in the appropriate spot fixed the issue.

That's a big whoops on my part.  :-)  Appreciate the pointer.

-- 
Jared

On 08/15/2017 07:20 PM, Kevin Korb via rsync wrote:
> Have you tried doing the clean shell test?
> 
> ssh -i /path/to/key user at remote.server.com true > testfile
> if testfile isn't 0 bytes then whatever is producing the data that
ended
> up in there is your problem.