samba - Apr 2016 - poor samba performance with many smaller files

Hi,
I need to to open existing readonly files. Yes you are right, not the open
call takes the time, but stat() system call.
I looked at aio_pthread source, the lines which needs to be removed to make
it work are just the lines which tests for O_CREAT|O_EXCL? What problems
can cause such solution for readonly share?

On Tue, Apr 26, 2016 at 8:42 PM, Jeremy Allison <jra at samba.org> wrote:
> On Tue, Apr 26, 2016 at 03:49:44PM +0200, Peter Bulin wrote:
> > Hi,
> >
> > I have problem with my samba. I have shared folder which contains many
> > smaller files (few MB each), this files are stored on very fast disk
> array.
> > Locally, from that machine I can read this files in parallel with
speed
> > about 500-1000 MB/s, but when I read them through samba, I get only
> 50-100
> > MB/s on a 10 gigabit network. I tried to find the problem, so I
analyzed
> > strace logs and found out, that files are open synchronous and it
takes
> > about 15 ms to open one file, so most of the time is spend with
openings
> of
> > files. Now my question is if I can do somethings about that, is there
> some
> > settings which turns on asynchronous open of files?
>
> If you're on Linux then use the module vfs_aio_pthread
> and set "aio_pthread:aio open = true"
>
> This makes open(O_CREAT|O_EXCL) calls async on Linux
> only (as it allows per-thread credentials).
>
> If it's opening existing files then this module
> currently doesn't help (although removal of a
> couple of lines of code would make it work).
>
> Are you *sure* it's the open() calls that
> take the time ? Or is it file name lookups ?
>

On Tue, Apr 26, 2016 at 09:44:57PM +0200, Peter Bulin
wrote:
> Hi,
> I need to to open existing readonly files. Yes you are right, not the open
> call takes the time, but stat() system call.
> I looked at aio_pthread source, the lines which needs to be removed to make
> it work are just the lines which tests for O_CREAT|O_EXCL? What problems
> can cause such solution for readonly share?
So ignore the aio_pthread advice - that really isn't relevent.
What's causing the problem is the lookup time for cache misses
with a really large directory.

See here:

https://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/largefile.html

for details.

The directory is small, it contains only one file. stat() takes 15ms and if
I read 1000 files it sums to 15 seconds. So the issue is really that open
file is synchronous.

On Tue, Apr 26, 2016 at 9:57 PM, Jeremy Allison <jra at samba.org> wrote:
> On Tue, Apr 26, 2016 at 09:44:57PM +0200, Peter Bulin wrote:
> > Hi,
> > I need to to open existing readonly files. Yes you are right, not the
> open
> > call takes the time, but stat() system call.
> > I looked at aio_pthread source, the lines which needs to be removed to
> make
> > it work are just the lines which tests for O_CREAT|O_EXCL? What
problems
> > can cause such solution for readonly share?
>
> So ignore the aio_pthread advice - that really isn't relevent.
> What's causing the problem is the lookup time for cache misses
> with a really large directory.
>
> See here:
>
> https://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/largefile.html
>
> for details.
>