Hello all,
I'm using rsync over a fairly expensive satellite network that has
intermittent connectivity and a lot of packet loss.? I'm getting a lot
of corrupted file names, which is making me wonder if this is just the
one in 64k-odd packets that are corrupted with a matching TCP checksum
or if there's something else going on.
My server is using 3.1.2-2.1ubuntu1_amd64 (it's an old version of
Ubuntu, so I hack-upgraded it) run from xinetd and the clients are using
3.1.2 from Cygwin.
For example, in one case a file is expected to have a name similar to:
SH02491_20191021203744-20191021211456_1WJ1170GTKL003151.hpr
but I find:
170GTKL003151.hpr^G?~]<F3>^G<D0><F0><A4><C2>56>
^Q,6000000-20191116
In another case, this should have no subdirectories, but I find this
(with ls -alFb)
-rw-rw-rw- 1 nobody users 211346 Aug 4 18:05 3074777997.hpr
-rw-rw-rw- 1 nobody users 255851 Aug 4 19:05 3074777998.hpr
-rw-rw-rw- 1 nobody users 276949 Aug 4 20:05 3074777999.hpr
-rw-rw-rw- 1 nobody users 242214 May 30 2019 307477799.hpr
-rw-rw-rw- 1 nobody users 102301 May 23 2019 30747779.hpr
-rw-rw-rw- 1 nobody users 66197 May 22 2019 3074777.hpr
drwx------ 2 nobody users 4096 Dec 23 13:44 3.mom/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ \t0.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ \t1.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ \t2.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ \t3.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ \t4.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ \t5.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ \t6.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ \t7.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\
\t\0057.hpr\004\356\210]\036\2066\360h,F\ \>\
\t\0058.hpr\004#,]\260\2306\360T\=pr\002\255+]\264\3306\354\020L4\>\
\t\0058.hpr\001[\353]2.7\360\264ut%\>\ \t\0059.hpr\004?]D<7\360X/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\
\t\0057.hpr\004\356\210]\036\2066\360h,F\ \>\
\t\0058.hpr\004#,]\260\2306\360T\=pr\002\255+]\264\3306\354\020L4\>\
\t\t\0059.hpr\003\334t]\261\3377\360\254?\035\>\ \b\0057.hpr\004\305X\\/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ 80.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ 81.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ 82.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ 83.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ 84.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ 85.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ 86.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ 87.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ 88.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ 89.hpr/
drwx------ 2 nobody users 4096 Dec 22 07:51 6\360\004[Q\030\>\ \t8.hpr/
I haven't dug deeply into the rsync sources, but I have a few questions
or suggestions
1. Does each logical transaction between rsync server and client employ
any integrity guarantees beyond the TCP checksum?
2. Can the file list be xferred as a regular /tmp file?? (I'm guessing
the file would have to include file times, sizes, permissions, acls,
attrs, etc.)
3. Is there any option (that I've missed) to give rsync the hint that
we're in the 1980s with a dial-up-like connection and we need more
to eat more CPU (for compression and such) to save bandwidth?
4. Is the file list compressed when using -z?? In our case, this would
help a LOT.
5. I don't suppose there's an option to specify the compression
mechanism or level is there?
6. When using -z and if files are similar enough, would it be possible
to reuse the dictionary from a previously compressed file to send
less data over the wire?
I have recently added "max verbosity = 3" to /etc/rsyncd.conf and
"--verbose" to the arguments when launched from xinetd and I'm
getting a
dump of all the recv_file_name() calls now, but I haven't seen a
corrupted file name in the log yet.
Thanks
Daniel
