Evan Harris
2005-Mar-16 05:44 UTC
Problem with rsync --inplace very slow/hung on large files
I'm trying to rsync a very large (62gig) file from one machine to another as part of a nightly backup. If the file does not exist at the destination, it takes about 2.5 hours to copy in my environment. But, if the file does exist and --inplace is specified, and the file contents differ, rsync either is so significantly slowed as to take more than 30 hours (the longest I've let an instance run), or it is just hung. Running with -vvv gives this as the last few lines of the output: match at 205401064 last_match=205401064 j=821 len=250184 n=0 match at 205651248 last_match=205651248 j=822 len=250184 n=0 match at 205901432 last_match=205901432 j=823 len=250184 n=0 match at 206151616 last_match=206151616 j=824 len=250184 n=0 at which point it has not printed anything else since I last looked at the current run attempt about 8 hours ago. Doing an strace on the rsync processes on the sending and receiving machines it appears that there is still reading and writing going on, but there isn't any output from the -vvv and I can't tell if it's really doing anything. Is this excessive slowness just an artifact of doing an rsync --inplace on such a large file, and it will eventually complete if let run long enough? I would try testing without the --inplace, but the system in question doesn't have enough disk space for two copies of that size file, which is why I am using --inplace. Using 2.6.3, on Debian. Any help appreciated. Evan
Maybe Matching Threads
- The --inplace is very different from the behaviour of --partial when resuming a complex case transfer.
- ERROR: writefd_unbuffered failed to write ... when COMPRESS
- exit status 12 when transferring a large file
- Rsync lets user corrupt dest by applying non-inplace batch in inplace mode
- DO NOT REPLY [Bug 5201] New: Rsync lets user corrupt dest by applying non-inplace batch in inplace mode