samba - Nov 2003 - Is there a buffer or cache setting in samba?

Hi. 

Can anyone tell me if samba has any settings that determine how much data 
gets cached or buffered in RAM before being written to the computer's hard 
drives? 

I'm having a strange problem and I suspect that the explanation has to do 
with that kind of setting. 

I am using a Linux system (P4-3.06 Ghz, 1 GB RAM, 2.4.22 kernel, samba 
2.2.8a) to store video and audio files that can be accesssed by a group of 
Windows-based video editing systems. I got the whole system up and running a
week ago
and it was working perfectly (my storage devices, by the way, are a series of 
firewire drives arranged into a RAID 10 array) 

When I tested the system with "disk testing program" on the Windows
side, I
got a transfer rate of 22 MB/sec for a 1 GB test. And in real life, I could 
sustain a rate of at least 18 MB/second for 20 minutes over my gigabit network. 
That's what is required for my application -- digitizing uncompressed video.

But now things have suddenly fallen apart. Yesterday I had to reinstall 
Mandrake 9.2 because I had been moving firewire and ethernet cards around to 
different PCI slots to optimize the system and I just messed things up too much.
So
I reinstalled and went back to the same card configuration I had when I got 
the 18 MB/second. And now it doesn't work. 
I know that I am using a DIFFERENT smb.conf file now compared to before. I 
don't think I have the old one that I had made with SWAT. The one I'm
using
right now is very simple and it forces a user and group name on all files
written
to the Linux share. 

Looking at a Linux monitoring program -- I believe it's called XOSVIEW -- I 
think I can see the problem. 

Yesterday when I tested the system I saw that all the RAM had to "fill
up"
completely (took about 40 seconds at 18 MB/sec) before Linux started writing to 
the hard drives. And shortly after that my Windows video program would abort, 
telling me the data wasn't getting transferred fast. 

Last week, when things were working -- and I was using the same monitoring 
program --Linux would start writing to the drives after about just a few seconds
rather than buffering or caching so much data in RAM. And I could see in the 
monitoring program that there was more RAM free. 

There must be a setting in samba that determines how much data is cached or 
buffered in RAM before writing it to the drives. Do you know anything about 
this? 

Your advice would be very much appreciated. 

Andy Liebman

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
> Message: 18
> Date: Wed, 5 Nov 2003 06:21:04 EST
> From: AndyLiebman@aol.com
> Subject: [Samba] Is there a buffer or cache setting in samba?
> To: samba@lists.samba.org
> Message-ID: <1c6.1154b750.2cda3720@aol.com>
> Content-Type: text/plain; charset="US-ASCII"
>
> Hi.
>
> Can anyone tell me if samba has any settings that determine how much data
> gets cached or buffered in RAM before being written to the computer's
hard
> drives?
>
> I'm having a strange problem and I suspect that the explanation has to
do
> with that kind of setting.
>
> I am using a Linux system (P4-3.06 Ghz, 1 GB RAM, 2.4.22 kernel, samba
> 2.2.8a) to store video and audio files that can be accesssed by a
group of
> Windows-based video editing systems. I got the whole system up and
running a week ago
> and it was working perfectly (my storage devices, by the way, are a
series of
> firewire drives arranged into a RAID 10 array)
>
> When I tested the system with "disk testing program" on the
Windows
side, I
> got a transfer rate of 22 MB/sec for a 1 GB test. And in real life, I
could
> sustain a rate of at least 18 MB/second for 20 minutes over my gigabit
network.
> That's what is required for my application -- digitizing uncompressed
video.
>
> But now things have suddenly fallen apart. Yesterday I had to reinstall
> Mandrake 9.2 because I had been moving firewire and ethernet cards
around to
> different PCI slots to optimize the system and I just messed things up
too much.

Hmmm, reinstalling is normally not a good solution on unix, and if you
do, it's advisable to at least backup all configuration files (tar -cjvf
 /some/safe/place/etc`date +%Y%m%d`.tar.bz2 /etc).

BTW, Mandrake 9.2 has a parallel-installable version of samba-3.0.0
available in contrib:
# urpmi samba3-server
(assuming you have a contrib urpmi medium available, see
http://plf.zarb.org/~nanardon if you don't know how to do this)

Samba3 may perform better than 2.2.x (possibly mainly since sendfile is
enabled by default). Just be careful to only run one at a time (unless
you have been even more careful to set them up to run in parallel).
> So
> I reinstalled and went back to the same card configuration I had when
I got
> the 18 MB/second. And now it doesn't work.
> I know that I am using a DIFFERENT smb.conf file now compared to
before. I
> don't think I have the old one that I had made with SWAT. The one
I'm
using
> right now is very simple and it forces a user and group name on all
files written
> to the Linux share.
>
> Looking at a Linux monitoring program -- I believe it's called XOSVIEW
- -- I
> think I can see the problem.
>
> Yesterday when I tested the system I saw that all the RAM had to "fill
up"
> completely (took about 40 seconds at 18 MB/sec) before Linux started
writing to
> the hard drives. And shortly after that my Windows video program would
abort,
> telling me the data wasn't getting transferred fast.
>
> Last week, when things were working -- and I was using the same
monitoring
> program --Linux would start writing to the drives after about just a
few seconds
> rather than buffering or caching so much data in RAM. And I could see
in the
> monitoring program that there was more RAM free.
>
> There must be a setting in samba that determines how much data is
cached or
> buffered in RAM before writing it to the drives. Do you know anything
about
> this?
The only thing (AFAIK) samba does regarding caching is calling sync, see
the 'strict sync' and 'sync always' options). But, this may harm
performance (as the kernel normally has a better idea about when it
should write what to disk than a client program does), but it's worth a
shot. You probably want to try 'strict sync = yes'.

Regards,
Buchan

- --
|--------------Another happy Mandrake Club member--------------|
Buchan Milne                Mechanical Engineer, Network Manager
Cellphone * Work            +27 82 472 2231 * +27 21 8828820x202
Stellenbosch Automotive Engineering         http://www.cae.co.za
GPG Key                   http://ranger.dnsalias.com/bgmilne.asc
1024D/60D204A7 2919 E232 5610 A038 87B1 72D6 AC92 BA50 60D2 04A7
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQE/qSI2rJK6UGDSBKcRApAoAJ91xsYEP7loSTx1vYgn9nalGPUx1gCgutkh
jAW6YOkUUNeRoVXOS3d1DTM=bGaS
-----END PGP SIGNATURE-----

*****************************************************************
Please click on http://www.cae.co.za/disclaimer.htm to read our
e-mail disclaimer or send an e-mail to info@cae.co.za for a copy.
*****************************************************************