Hello list, I get the following error using rsync 3.0.3 (both sides): ... data/xxxxxxxx/admin_docs/2/8/9/9/0/7/9/msg-75682-748.msg is uptodate data/xxxxxxxx/admin_docs/2/8/9/9/0/8/0/msg-75682-749.txt is uptodate ERROR: out of memory in flist_expand [generator] rsync error: error allocating core memory buffers (code 22) at util.c(117) [generator=3.0.3] rsync error: received SIGUSR1 (code 19) at main.c(1286) [receiver=3.0.3] Command was: rsync -avvz remotehost:/var/www /backup These things I have tried to fix the problem: After starting rsync, it says 'receiving incremental file list', which I read indicates it won't consume too much memory. I have increased memory by adding an extra swap space of 4GB (I know, overdone..). There is 1GB of RAM in the system. There is plenty of diskspace as well. I have also tried modifying the maximum stack size, which is now 512 MB for a program. In the Rsync FAQ I read Rsync uses approximately 100 bytes per file for indexing. The number of files to copy from the remote server is 3902364, so rsync would need about 400MB... right? Both sides run FreeBSD 6.x, and rsync 3.0.3. What am I missing? -- Rob klein Gunnewiek BWSS B.V. Deventer (http://www.bwss.nl) Tel +(31) 0570-665140 Fax +(31) 0570-665141
Could someone please help me with this problem? Rob klein Gunnewiek wrote:> Hello list, > > I get the following error using rsync 3.0.3 (both sides): > > ... > data/xxxxxxxx/admin_docs/2/8/9/9/0/7/9/msg-75682-748.msg is uptodate > data/xxxxxxxx/admin_docs/2/8/9/9/0/8/0/msg-75682-749.txt is uptodate > ERROR: out of memory in flist_expand [generator] > rsync error: error allocating core memory buffers (code 22) at util.c(117) > [generator=3.0.3] > rsync error: received SIGUSR1 (code 19) at main.c(1286) [receiver=3.0.3] > > Command was: rsync -avvz remotehost:/var/www /backup > > These things I have tried to fix the problem: > > After starting rsync, it says 'receiving incremental file list', which I read > indicates it won't consume too much memory. > > I have increased memory by adding an extra swap space of 4GB (I know, > overdone..). There is 1GB of RAM in the system. There is plenty of diskspace as > well. I have also tried modifying the maximum stack size, which is now 512 MB > for a program. > > In the Rsync FAQ I read Rsync uses approximately 100 bytes per file for > indexing. The number of files to copy from the remote server is 3902364, so > rsync would need about 400MB... right? > > Both sides run FreeBSD 6.x, and rsync 3.0.3. > > What am I missing? >-- Rob klein Gunnewiek BWSS B.V. Deventer (http://www.bwss.nl) Tel +(31) 0570-665140 Fax +(31) 0570-665141
On Fri, Aug 15, 2008 at 05:28:43PM +0200, Rob klein Gunnewiek wrote:> ERROR: out of memory in flist_expand [generator]With -vv, you should be seeing messages like this: [generator] expand file_list pointer array to N bytes, did move That will tell you what the last realloc attempt was before the error. If it is a really huge value, check to see if you have a directory with a huge number of files in it. You should also check your process's memory ulimit values to see if they are too small, and try using a different compiler or malloc package to see if your malloc is having trouble with memory fragmentation. ..wayne..