Displaying 2 results from an estimated 2 matches for "3mil".
Did you mean:
1mil
2004 Jul 22
0
ERROR: out of memory in receive_file_entry
...Server: HP-UX 11.11
rsync version: 2.6.2
Directory size: 400Gbytes
number of files: 3273133
rsync cmd: rsync -avRx --progress --stats --numeric-ids --blocking-io --delete -e ssh hp-ux.server:/path /local/linux/
It seems to fail after consuming roughly 250M of memory and recieving a
little over 3mil files in the file list. Both client and server have 1G
of RAM and 2G of swap. Here is the error message from rsync:
<snip>
ERROR: out of memory in receive_file_entry
rsync error: error allocating core memory buffers (code 22) at util.c(116)
rsync: connection unexpectedly closed (63453387 b...
2004 Aug 10
2
out of memory in receive_file_entry rsync-2.6.2
...xist in a single directory several levels below the top. Anyways,
i'll take the function out of my script and paste it here but it may not
work right taken out of context. YMMV.
<snip>
function DC {
# "Divide and Conquer"
# evil hack to get around rsync's 3mil file limit
# Accepts two arguments as the srcpath and dstpath. Will not work
# if srcpath is local.
srchost=`echo $1 | awk -F: '{ print $1 }'`
srcdir=`echo $1 | awk -F: '{ print $2 }'`
num_files=`ssh root@$srchost "find $srcdir | wc -l"...