search for: updating_basis_fil

Hi, I recently came across a situation where "rsync --inplace" performs very poorly. If both the source and destination files contain long sequences of identical blocks, but not necessarily in the same location, the sender can spend an inordinate amount of CPU time finding matching blocks. In my case, I came across this problem while backing up multi-hundred-gigabyte MySQL database

...d/match.c --- rsync-3.2.7.patch_orig/match.c 2023-05-16 14:17:31.082521665 -0400 +++ rsync-3.2.7.patched/match.c 2023-05-16 15:36:46.781895867 -0400 @@ -32,7 +32,7 @@ extern int append_mode; extern struct name_num_item *xfer_sum_nni; -extern int xfer_sum_len; +extern size_t xfer_sum_len; int updating_basis_file; char sender_file_sum[MAX_DIGEST_LEN]; diff -Naur rsync-3.2.7.patch_orig/receiver.c rsync-3.2.7.patched/receiver.c --- rsync-3.2.7.patch_orig/receiver.c 2023-05-16 14:17:31.082521665 -0400 +++ rsync-3.2.7.patched/receiver.c 2023-05-16 15:37:10.401929552 -0400 @@ -75,7 +75,7 @@ extern OFF_T preal...

...Running strace on the process shows no activity. The hanging process is the rsync with the lowest PID. I attached gdb to the hanging rsync. Here is the back trace: 0x000000000041e5c4 in hash_search (f=4, s=0xcc1ec0, buf=0xcc1ef0, len=2000967340) at match.c:211 211 if (updating_basis_file && s->sums[i].offset < offset (gdb) back #0 0x000000000041e5c4 in hash_search (f=4, s=0xcc1ec0, buf=0xcc1ef0, len=2000967340) at match.c:211 #1 match_sums (f=4, s=0xcc1ec0, buf=0xcc1ef0, len=2000967340) at match.c:369 #2 0x0000000000414491 in send_files (f_in=5, f_out=4) at sende...

...09-04-12 22:48:59.000000000 +0300 +++ rsync-3.0.6.fast/sender.c 2009-09-23 22:24:59.000000000 +0300 @@ -35,6 +35,8 @@ extern int allowed_lull; extern int preserve_xattrs; extern int protocol_version; +extern int local_server; +extern int local_socket; extern int remove_source_files; extern int updating_basis_file; extern int make_backups; @@ -319,6 +321,16 @@ path,slash,fname, (double)st.st_size); } + /* send_fd for fast copy before sending sum head, + * which will trigger the recv_fd at the receiver -goldor */ + if (local_server && local_socket && s->count == 0) { + if...

https://bugzilla.samba.org/show_bug.cgi?id=7778 Summary: --inplace does extra WRITE operations Product: rsync Version: 3.0.7 Platform: Other OS/Version: Linux Status: NEW Severity: minor Priority: P3 Component: core AssignedTo: wayned at samba.org ReportedBy: ildar at altlinux.ru

...: external declaration in primary source file extern int protocol_version; ^ sender.c(37): remark #1419: external declaration in primary source file extern int remove_source_files; ^ sender.c(38): remark #1419: external declaration in primary source file extern int updating_basis_file; ^ sender.c(39): remark #1419: external declaration in primary source file extern int make_backups; ^ sender.c(40): remark #1419: external declaration in primary source file extern int do_progress; ^ sender.c(41): remark #1419: external declaration in...