samba - Jan 2011 - Question on how to get Samba to use larger pread/write calls.

We are testing Samba 3 (and 4) on Fedora Core 13,
10Gbit connection with a Mac OS 10.6.4 system
as the client.  We will be adding some Windows
machines sooner or later with 10Gbit interfaces.

We are seeing 100-150MBytes/sec read or write
performance between the Mac and the FC13 system
over 10Gbit interface but it should be capable of
400-500MBytes/sec.  We have a local raid
on the FC13 system that runs 1GByte/sec locally
using an Areca 1880-ix-16 raid card (6Gbit version).
It has 16 fast Hitachi disks in a Raid5 format
using xfs filesystem.

The problem here is that samba is poking the Areca
at 128KByte I/O's on preads and writes, i.e.
shown to us using strace on the smbd daemons
that are running.  Using vmstat/iostat/sar utilities,
we see 100% utilization of the Areca card because the
average wait time is real high and the average
queue length to it is also high......too many
small I/O's.....
This is not the case if I run "fio" or "dd" locally
to/from the Areca's raid using 1-4MByte I/O's.
I see fast I/O...

I do not see any way to increase the size of Samba's
pread/write's in the smb.conf documentation.  I
am sure it may be just a matter of getting the source
code and making some changes to allow larger
sized IO's but........and of course I suppose Windows
clients may complain but....

I remember that back in the old Irix days with the
group of engineers in Australia I worked with,
we had Samba screaming fast but not sure if they
tweaked the version of Samba on Irix to do this.

Can you guys come up with a way for us to allow
the reads/writes to/from the disks to be tunable
up to say 4MB in size?

Thx
chris duffy

On Mon, Jan 17, 2011 at 10:14:04AM -0600, Chris Duffy
wrote:
> We are testing Samba 3 (and 4) on Fedora Core 13,
> 10Gbit connection with a Mac OS 10.6.4 system
> as the client.  We will be adding some Windows
> machines sooner or later with 10Gbit interfaces.
> 
> We are seeing 100-150MBytes/sec read or write
> performance between the Mac and the FC13 system
> over 10Gbit interface but it should be capable of
> 400-500MBytes/sec.  We have a local raid
> on the FC13 system that runs 1GByte/sec locally
> using an Areca 1880-ix-16 raid card (6Gbit version).
> It has 16 fast Hitachi disks in a Raid5 format
> using xfs filesystem.
> 
> The problem here is that samba is poking the Areca
> at 128KByte I/O's on preads and writes, i.e.
> shown to us using strace on the smbd daemons
> that are running.  Using vmstat/iostat/sar utilities,
> we see 100% utilization of the Areca card because the
> average wait time is real high and the average
> queue length to it is also high......too many
> small I/O's.....
> This is not the case if I run "fio" or "dd" locally
> to/from the Areca's raid using 1-4MByte I/O's.
> I see fast I/O...
> 
> I do not see any way to increase the size of Samba's
> pread/write's in the smb.conf documentation.  I
> am sure it may be just a matter of getting the source
> code and making some changes to allow larger
> sized IO's but........and of course I suppose Windows
> clients may complain but....
> 
> I remember that back in the old Irix days with the
> group of engineers in Australia I worked with,
> we had Samba screaming fast but not sure if they
> tweaked the version of Samba on Irix to do this.
> 
> Can you guys come up with a way for us to allow
> the reads/writes to/from the disks to be tunable
> up to say 4MB in size?
You might want to play with "write cache size". This will
only tune writes though. And it will only work for oplocked
files. I'm not 100% sure that OS/X plays nicely wrt oplocks.

For reads, we need to take a much closer look at your real
workload and see if we need to use some kind of preopen,
prefetch or so module. We need to closely work with the
kernel buffer cache and potential readahead kernel
algorithms.

With best regards,

Volker Lendecke

-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 G?ttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG G?ttingen, HRB 2816, GF: Dr. Johannes Loxen

On 17 January 2011 08:14, Chris Duffy <cjd at small-tree.com>
wrote:
> We are testing Samba 3 (and 4) on Fedora Core 13,
> 10Gbit connection with a Mac OS 10.6.4 system
> as the client. ?We will be adding some Windows
> machines sooner or later with 10Gbit interfaces.
>
> We are seeing 100-150MBytes/sec read or write
> performance between the Mac and the FC13 system
> over 10Gbit interface but it should be capable of
> 400-500MBytes/sec. ?We have a local raid
> on the FC13 system that runs 1GByte/sec locally
> using an Areca 1880-ix-16 raid card (6Gbit version).
> It has 16 fast Hitachi disks in a Raid5 format
> using xfs filesystem.
>
> The problem here is that samba is poking the Areca
> at 128KByte I/O's on preads and writes, i.e.
> shown to us using strace on the smbd daemons
> that are running. ?Using vmstat/iostat/sar utilities,
> we see 100% utilization of the Areca card because the
> average wait time is real high and the average
> queue length to it is also high......too many
> small I/O's.....
> This is not the case if I run "fio" or "dd" locally
> to/from the Areca's raid using 1-4MByte I/O's.
> I see fast I/O...
>
> I do not see any way to increase the size of Samba's
> pread/write's in the smb.conf documentation. ?I
> am sure it may be just a matter of getting the source
> code and making some changes to allow larger
> sized IO's but........and of course I suppose Windows
> clients may complain but....
>
> I remember that back in the old Irix days with the
> group of engineers in Australia I worked with,
> we had Samba screaming fast but not sure if they
> tweaked the version of Samba on Irix to do this.
Hi Chris :)

IIRC, we used a carefully tweaked read ahead module to make sure that
I/Os were hitting the disks properly aligned and of reasonable size.

http://www.samba.org/samba/docs/man/manpages-3/vfs_readahead.8.html
>
> Can you guys come up with a way for us to allow
> the reads/writes to/from the disks to be tunable
> up to say 4MB in size?
>
> Thx
> chris duffy
>
> --
> To unsubscribe from this list go to the following URL and read the
> instructions: ?https://lists.samba.org/mailman/options/samba
>


-- 
James Peach | jorgar at gmail.com