mail at crcomp.net
2017-Sep-04 22:45 UTC
Is it possible to transfer a large, dynamic file in a piecemeal fashion?
Greetings, Is it possible to use rsync to transmit a large, dynamic 2TB file in a piecemeal fashion during daylight hours over the course of a dozen days? On a good day, about 200GB of data can be transferred before rsync times out to enable a nightly local backup to complete. The local backup changes a few 512 byte blocks in the 2TB file. That's why the 2TB file is dynamic. As an experiment, rsync was started on the client to sync the 2TB file. At that point, rsyncd on the backup server created a temporarily file with the 2TB file name appended with a dot followed by six characters. After about 100MB was transferred the rsync client was interrupted and stopped in order to simulate the nightly shutdown. When rsync was restarted it erased the original 100MB temporary file and started over with a new, empty temporarily file. In summary: * rsync apparently erases temporarily files upon restart * it takes about a dozen days to transfer the 2TB file Unless there's an option to have rsync pick up where it left off it seems impossible to transmit a large file in a piecemeal fashion. Because the (nearly) same 200GB at the beginning of the 2TB file gets retransmitted over and over with each restart. Thank you, -- Don Kuenz KB7RPU Genius is nothing but a greater aptitude for patience. - de Buffon
Possibly Parallel Threads
- Is it possible to transfer a large, dynamic file in a piecemeal fashion? Yes it is.
- Is it possible to transfer a large, dynamic file in a piecemeal fashion? Yes it is.
- Is it possible to transfer a large, dynamic file in a piecemeal fashion?
- samba_backup script doesn't use tdbbackup for tdb files?
- How to add rfc2307 after a join