Granzow, Doug (NCI)
2002-Apr-24 05:40 UTC
memory requirements was RE: out of memory in build_hash_table
Well, for reference, here's what I'm running: The mirror is running Solaris 8, the source is running Solaris 7. Both are running the same version of rsync: rsync version 2.5.4 protocol version 26 Copyright (C) 1996-2002 by Andrew Tridgell and others <http://rsync.samba.org/> Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles, no IPv6, 64-bit system inums, 64-bit internal inums The command being run on the mirror is: /usr/local/bin/rsync -qau --delete --rsh=/usr/bin/ssh source:/share/abc/. /share/abc/
Eric Whiting
2002-Apr-24 07:28 UTC
memory requirements was RE: out of memory in build_hash_table
"Granzow, Doug (NCI)" wrote:> > From what I've observed by running top while rsync is running, its memory > usage appears to grow gradually, not exponentially. A rsync may take > several hours to complete. (I have one running now that started over four > hours ago. The filesystem contains 236 GB of data in 2.4 million files. It > is currently taking up 1351MB of memory on the mirror server and 646M on the > source server.) All filesystems are veritas filesystem, in case that is > relevant.I just killed a 750M rsync (2.5.5) process. It had been runing for over 12 hours. (around 62G in 1.2 million files).. This was a local rsync without any ssh/remsh/rsyncd involved. I went back to the 'for file in *;do rsync blaa;done' method. This helps it run in a reasonable amount of time and RAM. I'd still like to be able to run a rsync on the toplevel (without the for loop...) eric