Hi:
I have an application where there is a single file-server and
approximately 500 clients that are writing a continuous stream of
moderate-sized data files. We have been using samba for this, but the
problem is each smbd process on the server side ends up chewing up
50-60 megs worth of memory after a while. With 500 clients that takes
a good chunk of the memory away :-)
I am exploring using rsync to move this files. They (can be) written
locally on each client (to a ramdisk) and then copied to the server
using rsync. I was optimistic that this would be the solution to my
problem, but I'm experiencing reliability issues. The server-side
error messages occasionally report the problem referenced in the
subject stat: no such file or directory. There is no reported error
on the client-side.
I'm running rysnc version 2.5.7. Client-machines are linux (2.6.12
kernel) and the server is a RedHat Enterprise Linux AS3 (update 5)
machine. I haven't played with rsync much before, but I did search
through the archives and googled around looking for clues when I
started seeing this problem. The filenames I'm using do not contain
spaces (one of the problems I saw mentioned often), and I have not been
able to tell ahead of time which file transfers will have an error
(i.e. I don't think there is anything special about the files
themselves, it may just be a rate problem).
There is an application running on the server machine that is watching
for new files and moves them elsewhere when they appear. As near as I
can tell, rsync creates the file with a temporary name, and then
renames it to the final name when the file transfer is complete. I am
not sure of the convention used, but I believe rsync is using mkstemp,
which should be affecting the last part of the filename. As long as
the temporary file does not have the same extension, then the other
application should not be yanking the files out in transit.
Also, the rsync module is named like: [abc] with a path
= /path/to/common and the rsync copy command is something like:
rsync myfile.xml rsync://host/abc/mydir/myfile.xml. The directory abc
exists under /path/to/common on the server. The filename referenced in
the stat error is: /mydir/myfile.xml. I would have thought this
should reference the full pathname on the server, not just the relative
path within the rsync module --- can anyone clarify this? The machine
that has the exact error message and configuration is being moved to a
different location at the moment, so unfortunately I cannot access the
exact information, but I'm pretty sure the info I have above is right.
I've tried to keep this email as short as possible and still provide
relevant info, I know it isn't a short question. Can anyone shed any
light on this problem?
thanks,
bob
--
Bob Robison bob.robison@swri.org
Staff Engineer 210-522-3935
Southwest Research Institute San Antonio, TX