rsync - Jun 2005 - Cannot Allocate Memory

Hello -

I've just upgraded to 2.6.5.

On a new set of servers I'm trying to sync I seem to be running into a log
of memory allocation errors.

With some of the servers, I start the rsync session, and it counts so many
files and then stops. At that point I'm unable to control-c out of the
process. From another session I can't issue a kill command either...very
strange.

On other servers, the first few files/directories issue a 'cannot allocate
memory', but strangly enough, it continues on.

Does this simply mean that the 256mb memory isn't sufficient? I am trying
to sync a very large folder structure.

Is there anyway around this using tmp disk space or something until I can
get more memory?

Thanks,
Max

On Tue, 7 Jun 2005, Max Kipness <max@kipness.com>
wrote:
> 
> I've just upgraded to 2.6.5.
> 
> On a new set of servers I'm trying to sync I seem to be running into a
log
> of memory allocation errors.
> 
> With some of the servers, I start the rsync session, and it counts so many
> files and then stops. At that point I'm unable to control-c out of the
> process. From another session I can't issue a kill command
either...very
> strange.
A control-C might not always work, but a  kill -9 {PID}  should.  If it
doesn't, there may be some OS-related problem in your brand-new servers.
> On other servers, the first few files/directories issue a 'cannot
allocate
> memory', but strangly enough, it continues on.
If that's an actual quote of the error message, it's not from rsync.
Rsync uses "ERROR: out of memory in {NameOfRoutine}", and then exits.
> Does this simply mean that the 256mb memory isn't sufficient? I am
trying
> to sync a very large folder structure.
At an average of 100 bytes per file, you could do 1 million files in
100 MB, so unless you have a real whopper, 256 MB should be just fine
especially if you also have some swap space.
> Is there anyway around this using tmp disk space or something until I can
> get more memory?
Using swap should effectively use temp disk space.

What are your ulimit values?  If you are running as root it shouldn't
matter,
but small memory limits might trigger a problem.

But the overall symptoms imply a deeper problem, IMO.

    John