Hello list, we have a small NAS-Box here in our office, running Linux 2.6.13 and Samba 3 (exactly version string is not avalilable for me at moment). Is there a limit, how many files samba will store in one folder? We recognize a massive CPU-Load of the smbd-process, when accessing a folder which stores round about 60 000 small text-files. Is this a samba-Limit or a bug? The kernel and samba is compiled by the NAS-manufactur, so no cimpiler-options are available for me. So long, Markus -- Markus Neviadomski lists @ dieitexperten.de
Sorry I haven't answered your question. I would go and check the docs
or search the code.
When you say "accessing a folder" do you really mean browsing in
Explorer?
If you're using Explorer, does it know they're text files? Do they
have .txt extension?
If it doesn't - if it has an extension which requires it - it could
open each one to find out what it is - to generate a thumbnail etc.
I've just created a directory containing 60,000 small text files with
names of the form "textfile${N}.txt" and can navigate it with only
momentary delays. I must admit, I'm using a dual xeon with fast scsi
raid.
The interesting thing for me was that when I hit 32768 files in my
creation loop, there was a delay of about a second - possibly as an
extra inode was allocated to the directory list.
When you access a large directory in samba, one cpu intensive part of
the process is likely to be the mapping of user names and permissions.
Do you have many users?
I've just doubled the size of my directory (120,000 small text files)
and it still performs quite well - when accessing from Explorer over a
samba share (3.0.24) I see a burst of activity on the Linux box and
then a long delay as Windows arranges the list.
Will the NAS manufacturer be able to give you more info?
A common way around the problem of large directories is to use
subdirectories based on the first letter or digit from the filename.
This is only good to you if you can get to the code of your
application and the files names do not change.
Regards,
Quinn
On 12/11/2007, lists@dieitexperten.de <lists@dieitexperten.de>
wrote:>
> Hello list,
>
> we have a small NAS-Box here in our office, running Linux 2.6.13 and
> Samba 3 (exactly version string is not avalilable for me at moment).
>
> Is there a limit, how many files samba will store in one folder? We
> recognize a massive CPU-Load of the smbd-process, when accessing a
> folder which stores round about 60 000 small text-files.
>
> Is this a samba-Limit or a bug? The kernel and samba is compiled by the
> NAS-manufactur, so no cimpiler-options are available for me.
>
> So long,
> Markus
>
> --
> Markus Neviadomski
> lists @ dieitexperten.de
> --
> To unsubscribe from this list go to the following URL and read the
> instructions: https://lists.samba.org/mailman/listinfo/samba
>
On Mon, Nov 12, 2007 at 08:24:05AM +0100, lists@dieitexperten.de wrote:> > Hello list, > > we have a small NAS-Box here in our office, running Linux 2.6.13 and > Samba 3 (exactly version string is not avalilable for me at moment). > > Is there a limit, how many files samba will store in one folder? We > recognize a massive CPU-Load of the smbd-process, when accessing a > folder which stores round about 60 000 small text-files. > > Is this a samba-Limit or a bug? The kernel and samba is compiled by the > NAS-manufactur, so no cimpiler-options are available for me.There are no hard coded limits, only what the OS restricts. However, storing large numbers of files in a folder is a bad idea unless Samba is set up specially to do this. See here : http://us1.samba.org/samba/ftp/HOWTO/Samba-LargeDirectory-HOWTO for details. Jeremy.
On Mon, 2007-11-12 at 17:39 -0800, Jeremy Allison wrote:> On Mon, Nov 12, 2007 at 08:24:05AM +0100, lists@dieitexperten.de wrote: > > > > Hello list, > > > > we have a small NAS-Box here in our office, running Linux 2.6.13 and > > Samba 3 (exactly version string is not avalilable for me at moment). > > > > Is there a limit, how many files samba will store in one folder? We > > recognize a massive CPU-Load of the smbd-process, when accessing a > > folder which stores round about 60 000 small text-files. > > > > Is this a samba-Limit or a bug? The kernel and samba is compiled by the > > NAS-manufactur, so no cimpiler-options are available for me. > > There are no hard coded limits, only what the OS restricts. > However, storing large numbers of files in a folder is a bad > idea unless Samba is set up specially to do this. See here : > > http://us1.samba.org/samba/ftp/HOWTO/Samba-LargeDirectory-HOWTO > > for details.IIRC ext3 has a (compile time changeable) limit of 32k files per dir by default ... Simo. -- Simo Sorce Samba Team GPL Compliance Officer <simo@samba.org> Senior Software Engineer at Red Hat Inc. <ssorce@redhat.com>
simo schrieb:> On Mon, 2007-11-12 at 17:39 -0800, Jeremy Allison wrote: > >> On Mon, Nov 12, 2007 at 08:24:05AM +0100, lists@dieitexperten.de wrote: >> >>> Hello list, >>> >>> we have a small NAS-Box here in our office, running Linux 2.6.13 and >>> Samba 3 (exactly version string is not avalilable for me at moment). >>> >>> Is there a limit, how many files samba will store in one folder? We >>> recognize a massive CPU-Load of the smbd-process, when accessing a >>> folder which stores round about 60 000 small text-files. >>> >>> Is this a samba-Limit or a bug? The kernel and samba is compiled by the >>> NAS-manufactur, so no cimpiler-options are available for me. >>> >> There are no hard coded limits, only what the OS restricts. >> However, storing large numbers of files in a folder is a bad >> idea unless Samba is set up specially to do this. See here : >> >> http://us1.samba.org/samba/ftp/HOWTO/Samba-LargeDirectory-HOWTO >> >> for details. >> > > IIRC ext3 has a (compile time changeable) limit of 32k files per dir by > default ... >Ah, nice to know. That could be the real answer for my problem. I will ask the manufactur of the NAS-Box. Greets, Markus> Simo. > >