Thorsten Schöning
2020-Nov-08 16:31 UTC
How to make generator and receiver processes strictly sequential?
Hi all, I have one server hosting virtual machines and one older NAS Synology DS1512+ used as backup target for those virtual machines. The NAS uses BTRFS for its internal storage because I want to use compression and at best make effecient use of snapshots. When using "--inplace --whole-file" the backup succeeds in ~2 hours, which is OK to fit into avarious historically grown time windows for different backups onto the NAS and from the NAs to USB itself. When using "--inplace" alone OTOH the necessary time increases to ~16 hours, which is unacceptable long currently. The bottleneck seems to be the NAS: With "--whole-file" it simply writes somewhat constantly with 100+ MiB/s, while without it reads drop to some broad range of ~35 to at best 80/100 MiB/s. Those read numbers are not too constant anymore as well. Looking at HTOP on the NAS, I can see two RSYNC processes most of the time concurrently reading subsequent files. After asking about this behaviour on SU[1], it's most likely that those processes are the generator calculating block-based checksums and the receiver itself. It seems even though the amount of changed data on the files is pretty low and therefore writes occur pretty seldom, the concurrent reads do make overall performance worse. So, is there some option to make generator and receiver on the backup target strictly sequential? Thanks! [1]: https://superuser.com/questions/1600336/why-does-rsync-use-two-reading-processes-with-different-files-on-my-backup-targe Mit freundlichen Gr??en, Thorsten Sch?ning -- Thorsten Sch?ning E-Mail: Thorsten.Schoening at AM-SoFT.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...........05151- 9468- 55 Fax...............05151- 9468- 88 Mobil..............0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Gesch?ftsf?hrer: Andreas Muchow