Hi,
I used --partial to transfer files from my local computer (rsync 3.1.2,
Debian) to a remote computer (rsync 3.1.1 WD MyPassport Storage device)
The files get transferred, but after successful transfer, the files
are not renamed from .<tmpfile> to <file>.
Where to go next?
Here is the verbose output after transferring an empty file
heiko at blade:~/Pictures$ rsync -iavvvvvvvP ./x
platte.local:/shares/Storage/Bilder\\\ von\\\ Heikos\\\ Computer/
FILE_STRUCT_LEN=24, EXTRA_LEN=4
cmd=<NULL> machine=platte.local user=<NULL>
path=/shares/Storage/Bilder\ von\ Heikos\ Computer/
cmd[0]=ssh cmd[1]=platte.local cmd[2]=rsync cmd[3]=--server
cmd[4]=-vvvvvvvlogDtpre.iLsfxC cmd[5]=--log-format=%i cmd[6]=--partial cmd[7]=.
cmd[8]=/shares/Storage/Bilder\ von\ Heikos\ Computer/
opening connection using: ssh platte.local rsync --server
-vvvvvvvlogDtpre.iLsfxC "--log-format=%i" --partial .
"/shares/Storage/Bilder\ von\ Heikos\ Computer/" (9 args)
msg checking charset: UTF-8
FILE_STRUCT_LEN=16, EXTRA_LEN=4
(Server) Protocol versions: remote=31, negotiated=31
(Client) Protocol versions: remote=31, negotiated=31
sending incremental file list
[sender] change_dir(/home/heiko/Pictures)
[sender] make_file(x,*,0)
[sender] flist start=1, used=1, low=0, high=0
[sender] i=1 . x mode=0100644 len=0 uid=1000 gid=1000 flags=1005
send_file_list done
[sender] flist_eof=1
file list sent
send_files starting
server_recv(2) starting pid=21983
uid 1000(heiko) maps to 1000
gid 1000(heiko) maps to 1000
recv_file_name(x)
received 1 names
[Receiver] flist start=1, used=1, low=0, high=0
[Receiver] i=1 1 x mode=0100644 len=0 uid=1000 gid=1000 flags=1000
recv_file_list done
[Receiver] flist_eof=1
get_local_name count=1 /shares/Storage/Bilder von Heikos Computer/
[Receiver] change_dir(/shares/Storage/Bilder von Heikos Computer)
generator starting pid=21983
delta-transmission enabled
recv_generator(x,1)
send_files(1, ./x)
count=0 n=0 rem=0
send_files mapped ./x of size 0
calling match_sums ./x
<f+++++++++ x
sending file_sum
false_alarms=0 hash_hits=0 matches=0
0 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=0/1)
sender finished ./x
generate_files phase=1
recv_files(1) starting
recv_files(x)
got file_sum
set uid of .x.VITBUv from 0 to 1000
set gid of .x.VITBUv from 0 to 1000
set modtime of .x.VITBUv to (1546208608) Sun Dec 30 14:23:28 2018
<rename is missing here>
send_files phase=1
recv_files phase=1
generate_files phase=2
send_files phase=2
send files finished
total: matches=0 hash_hits=0 false_alarms=0 data=0
recv_files phase=2
recv_files finished
generate_files phase=3
generate_files finished
client_run waiting on 9125
sent 96 bytes received 874 bytes 1,940.00 bytes/sec
total size is 0 speedup is 0.00
[sender] _exit_cleanup(code=0, file=main.c, line=1196): entered
[sender] _exit_cleanup(code=0, file=main.c, line=1196): about to call
exit(0)
Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
--
SCHLITTERMANN.de ---------------------------- internet & unix support -
Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
gnupg encrypted messages are welcome --------------- key ID: F69376CE -
! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ -
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20181230/80ba3b95/signature.sig>
I think --partial might be a red herring here. It only applies to what happens when rsync is aborted in the middle of a file. What happens without -P? Also, it is worth trying with --inplace. On 12/30/18 5:32 PM, Heiko Schlittermann via rsync wrote:> Hi, > > I used --partial to transfer files from my local computer (rsync 3.1.2, > Debian) to a remote computer (rsync 3.1.1 WD MyPassport Storage device) > > The files get transferred, but after successful transfer, the files > are not renamed from .<tmpfile> to <file>. > > Where to go next? > > Here is the verbose output after transferring an empty file > > heiko at blade:~/Pictures$ rsync -iavvvvvvvP ./x platte.local:/shares/Storage/Bilder\\\ von\\\ Heikos\\\ Computer/ > FILE_STRUCT_LEN=24, EXTRA_LEN=4 > cmd=<NULL> machine=platte.local user=<NULL> path=/shares/Storage/Bilder\ von\ Heikos\ Computer/ > cmd[0]=ssh cmd[1]=platte.local cmd[2]=rsync cmd[3]=--server cmd[4]=-vvvvvvvlogDtpre.iLsfxC cmd[5]=--log-format=%i cmd[6]=--partial cmd[7]=. cmd[8]=/shares/Storage/Bilder\ von\ Heikos\ Computer/ > opening connection using: ssh platte.local rsync --server -vvvvvvvlogDtpre.iLsfxC "--log-format=%i" --partial . "/shares/Storage/Bilder\ von\ Heikos\ Computer/" (9 args) > msg checking charset: UTF-8 > FILE_STRUCT_LEN=16, EXTRA_LEN=4 > (Server) Protocol versions: remote=31, negotiated=31 > (Client) Protocol versions: remote=31, negotiated=31 > sending incremental file list > [sender] change_dir(/home/heiko/Pictures) > [sender] make_file(x,*,0) > [sender] flist start=1, used=1, low=0, high=0 > [sender] i=1 . x mode=0100644 len=0 uid=1000 gid=1000 flags=1005 > send_file_list done > [sender] flist_eof=1 > file list sent > send_files starting > server_recv(2) starting pid=21983 > uid 1000(heiko) maps to 1000 > gid 1000(heiko) maps to 1000 > recv_file_name(x) > received 1 names > [Receiver] flist start=1, used=1, low=0, high=0 > [Receiver] i=1 1 x mode=0100644 len=0 uid=1000 gid=1000 flags=1000 > recv_file_list done > [Receiver] flist_eof=1 > get_local_name count=1 /shares/Storage/Bilder von Heikos Computer/ > [Receiver] change_dir(/shares/Storage/Bilder von Heikos Computer) > generator starting pid=21983 > delta-transmission enabled > recv_generator(x,1) > send_files(1, ./x) > count=0 n=0 rem=0 > send_files mapped ./x of size 0 > calling match_sums ./x > <f+++++++++ x > sending file_sum > false_alarms=0 hash_hits=0 matches=0 > 0 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=0/1) > sender finished ./x > generate_files phase=1 > recv_files(1) starting > recv_files(x) > got file_sum > set uid of .x.VITBUv from 0 to 1000 > set gid of .x.VITBUv from 0 to 1000 > set modtime of .x.VITBUv to (1546208608) Sun Dec 30 14:23:28 2018 > <rename is missing here> > send_files phase=1 > recv_files phase=1 > generate_files phase=2 > send_files phase=2 > send files finished > total: matches=0 hash_hits=0 false_alarms=0 data=0 > recv_files phase=2 > recv_files finished > generate_files phase=3 > generate_files finished > client_run waiting on 9125 > > sent 96 bytes received 874 bytes 1,940.00 bytes/sec > total size is 0 speedup is 0.00 > [sender] _exit_cleanup(code=0, file=main.c, line=1196): entered > [sender] _exit_cleanup(code=0, file=main.c, line=1196): about to call exit(0) > > Best regards from Dresden/Germany > Viele Grüße aus Dresden > Heiko Schlittermann > -- > SCHLITTERMANN.de ---------------------------- internet & unix support - > Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - > gnupg encrypted messages are welcome --------------- key ID: F69376CE - > ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - > >-- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: https://sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/rsync/attachments/20181230/5f1c8917/signature.sig>
Kevin Korb via rsync <rsync at lists.samba.org> (So 30 Dez 2018 23:56:44 CET):> I think --partial might be a red herring here. It only applies to what > happens when rsync is aborted in the middle of a file. What happens > without -P?Same happens w/o --partial. I append 2 logs: - a from localhost to remote server, exposing the missing "rename" - b from localhost to localhost In both cases the file itself is empty, and didn't exist on the receivers side. The unified diff looks like this: --- /tmp/a 2018-12-31 18:30:38.747209487 +0100 +++ /tmp/b 2018-12-31 18:30:45.623719009 +0100 @@ -1,6 +1,6 @@ -cmd=<NULL> machine=platte user=<NULL> path=/tmp/y -cmd[0]=ssh cmd[1]=platte cmd[2]=rsync cmd[3]=--server cmd[4]=-vvvve.LsfxC cmd[5]=. cmd[6]=/tmp/y -opening connection using: ssh platte rsync --server -vvvve.LsfxC . /tmp/y (7 args) +cmd=<NULL> machine=localhost user=<NULL> path=/tmp/y +cmd[0]=ssh cmd[1]=localhost cmd[2]=rsync cmd[3]=--server cmd[4]=-vvvve.LsfxC cmd[5]=. cmd[6]=/tmp/y +opening connection using: ssh localhost rsync --server -vvvve.LsfxC . /tmp/y (7 args) msg checking charset: UTF-8 (Client) Protocol versions: remote=31, negotiated=31 [sender] make_file(x,*,0) @@ -10,7 +10,7 @@ [sender] flist_eof=1 file list sent send_files starting -server_recv(2) starting pid=16534 +server_recv(2) starting pid=18896 recv_file_name(x) received 1 names [Receiver] flist_eof=1 @@ -18,7 +18,7 @@ [Receiver] i=0 1 x mode=0100644 len=0 flags=1000 recv_file_list done get_local_name count=1 /tmp/y -generator starting pid=16534 +generator starting pid=18896 delta-transmission enabled recv_generator(y,0) send_files(0, ./x) @@ -30,10 +30,11 @@ false_alarms=0 hash_hits=0 matches=0 sender finished ./x generate_files phase=1 -recv_files(1) starting send_files phase=1 +recv_files(1) starting recv_files(y) got file_sum * +renaming .y.Oq45md to y recv_files phase=1 generate_files phase=2 send_files phase=2 @@ -43,9 +44,9 @@ recv_files finished generate_files phase=3 generate_files finished -client_run waiting on 18876 +client_run waiting on 18878 -sent 75 bytes received 533 bytes 1,216.00 bytes/sec +sent 75 bytes received 561 bytes 424.00 bytes/sec total size is 0 speedup is 0.00 [sender] _exit_cleanup(code=0, file=main.c, line=1196): entered [sender] _exit_cleanup(code=0, file=main.c, line=1196): about to call exit(0) Best regards from Dresden/Germany Viele Grüße aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- cmd=<NULL> machine=platte user=<NULL> path=/tmp/y cmd[0]=ssh cmd[1]=platte cmd[2]=rsync cmd[3]=--server cmd[4]=-vvvve.LsfxC cmd[5]=. cmd[6]=/tmp/y opening connection using: ssh platte rsync --server -vvvve.LsfxC . /tmp/y (7 args) msg checking charset: UTF-8 (Client) Protocol versions: remote=31, negotiated=31 [sender] make_file(x,*,0) [sender] flist start=0, used=1, low=0, high=0 [sender] i=0 . x mode=0100644 len=0 flags=1000 send_file_list done [sender] flist_eof=1 file list sent send_files starting server_recv(2) starting pid=16534 recv_file_name(x) received 1 names [Receiver] flist_eof=1 [Receiver] flist start=0, used=1, low=0, high=0 [Receiver] i=0 1 x mode=0100644 len=0 flags=1000 recv_file_list done get_local_name count=1 /tmp/y generator starting pid=16534 delta-transmission enabled recv_generator(y,0) send_files(0, ./x) count=0 n=0 rem=0 send_files mapped ./x of size 0 calling match_sums ./x x sending file_sum false_alarms=0 hash_hits=0 matches=0 sender finished ./x generate_files phase=1 recv_files(1) starting send_files phase=1 recv_files(y) got file_sum recv_files phase=1 generate_files phase=2 send_files phase=2 send files finished total: matches=0 hash_hits=0 false_alarms=0 data=0 recv_files phase=2 recv_files finished generate_files phase=3 generate_files finished client_run waiting on 18876 sent 75 bytes received 533 bytes 1,216.00 bytes/sec total size is 0 speedup is 0.00 [sender] _exit_cleanup(code=0, file=main.c, line=1196): entered [sender] _exit_cleanup(code=0, file=main.c, line=1196): about to call exit(0) -------------- next part -------------- cmd=<NULL> machine=localhost user=<NULL> path=/tmp/y cmd[0]=ssh cmd[1]=localhost cmd[2]=rsync cmd[3]=--server cmd[4]=-vvvve.LsfxC cmd[5]=. cmd[6]=/tmp/y opening connection using: ssh localhost rsync --server -vvvve.LsfxC . /tmp/y (7 args) msg checking charset: UTF-8 (Client) Protocol versions: remote=31, negotiated=31 [sender] make_file(x,*,0) [sender] flist start=0, used=1, low=0, high=0 [sender] i=0 . x mode=0100644 len=0 flags=1000 send_file_list done [sender] flist_eof=1 file list sent send_files starting server_recv(2) starting pid=18896 recv_file_name(x) received 1 names [Receiver] flist_eof=1 [Receiver] flist start=0, used=1, low=0, high=0 [Receiver] i=0 1 x mode=0100644 len=0 flags=1000 recv_file_list done get_local_name count=1 /tmp/y generator starting pid=18896 delta-transmission enabled recv_generator(y,0) send_files(0, ./x) count=0 n=0 rem=0 send_files mapped ./x of size 0 calling match_sums ./x x sending file_sum false_alarms=0 hash_hits=0 matches=0 sender finished ./x generate_files phase=1 send_files phase=1 recv_files(1) starting recv_files(y) got file_sum renaming .y.Oq45md to y recv_files phase=1 generate_files phase=2 send_files phase=2 send files finished total: matches=0 hash_hits=0 false_alarms=0 data=0 recv_files phase=2 recv_files finished generate_files phase=3 generate_files finished client_run waiting on 18878 sent 75 bytes received 561 bytes 424.00 bytes/sec total size is 0 speedup is 0.00 [sender] _exit_cleanup(code=0, file=main.c, line=1196): entered [sender] _exit_cleanup(code=0, file=main.c, line=1196): about to call exit(0) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: <http://lists.samba.org/pipermail/rsync/attachments/20181231/d583e67e/signature.sig>