Hi Joel, We essentially have the following directory structure: /images/yyyy/mm/dd/nn For each day of the year we get in the region of 120,000 images, which are split down into 100 subdirectories to prevent their being too many files in one directory. The directory /images is shared out as follows: [images] comment = Imaging Files path = /images public = yes writable = yes printable = no browseable = no valid users = root [imaging] comment = Imaging Files path = /images public = yes writable = no printable = no guest ok = yes So essentially there is a read only copy of the images which any user can access as guest, and a hidden share which only one user - the processes which write out the images can write to... I'm currently trying to work out whether we can change things so that we can share out each month as opposed to the whole lot, however that will probably mean an application rewrite. I'm also copying all of the data over onto a RedHat 7.1 box, to see if the issue is confined to Tru64 Unix 4.0F. Regards, Rob -----Original Message----- From: Joel@HammersHome.com [SMTP:MIME :Joel@HammersHome.com] Sent: Saturday, September 22, 2001 2:34 AM To: samba@lists.samba.org Subject: Re: URGENT REPOST: Maximum number of files i For my own education, why are you serving so many shares from one box? Don't you get performance degradation just from that alone? Is it 6.4 millions files in all shares or just in one share? Could you set up shares to serve up fractions of the 6.4 millions files, like a-d, e-h, etc? Why don't you set up separate servers? You could put separate servers on the same box if you want, with each server responsible for a fraction of the files. Are there different classes of user? Does each class of user need to access all six million files? If not, configure samba to offer shares containing files needed by a particular class of user, instead of all 6.4 million. Joel On Fri, Sep 21, 2001 at 01:48:36PM +0000, rob.leadbeater@lynx.co.uk wrote:> Hi, > > Is anyone out there using samba to share out over 6.4 million files ? > We've just had our production system grind to a halt again when we hitthis, apparently samba, limitation. -- To unsubscribe from this list go to the following URL and read the instructions: http://lists.samba.org/mailman/listinfo/samba This message is intended only for the use of the person(s) ("The intended Recipient(s)") to whom it is addressed. It may contain information which is privileged and confidential within the meaning of applicable law. If you are not the intended recipient, please contact the sender as soon as possible. The views expressed in this communication are not necessarily those held by LYNX Express Limited.
Hi Don,
The man page for setrlimit shows that there is a maximum for Digital Unix (aka
Compaq Tru 64, aka HP-UX ;-) of 4096, so I've set the max open files option
accordingly.
This would explain the warnings I was getting.
I'm just setting up the logging so that I can trap what is happening when we
get up to 6.4 million files again..
Regards,
Rob
-----Original Message-----
From: don_mccall@hp.com [SMTP:MIME :don_mccall@hp.com]
Sent: Friday, September 21, 2001 6:12 PM
To: Rob Leadbeater; samba@lists.samba.org; samba-technical@samba.org
Subject: RE: URGENT REPOST: Maximum number of files i
Hi Rob,
The message you are getting is basically a 'warning' from samba; samba
uses
the system
call setrlimit() to try to set the RLIMIT_NOFILE based on the max open
files
smb.conf parameter; by default max open files = 10000, and the samba
routine
set_maxfiles that calls
setrlimit() tacks on 10 additional to take care of files it knows it's
going
to have to have open as well; what the message is saying is that
setrlimit() was called with a value of 10010, and your system returned the
error 'invalid argument'; take a look at the man page for setrlimit() to
see
what's going on there.
In general, the number of files in a SHARE is unlimited; where you run
into
problems is when you have a really large number of files in a single
directory; this slows samba down in a number of ways, as when it is
processing certain smb requests that have to scan the directory looking
for
a file.
Question; when your smbd was taking up so much time, how many files was
the
user trying to copy? And how many files were in the particular directory
that he was trying to copy into/out of? You said that you could delete
some
files from the share and it would work?
The files deleted were from the same directory that the copy was trying to
use?
If you can 'force' this problem to occur, you might want to do a debug
session with a particular client, setting log level = 10 for that
particular
client, then 'forcing' the problem, and take a look at the
log.machinename for that client when the cpu
useage goes up; that will give you a better handle on WHAT all that
processing time is being used for.
Hope this helps,
Don
-----Original Message-----
From: rob.leadbeater@lynx.co.uk [mailto:rob.leadbeater@lynx.co.uk]
Sent: Friday, September 21, 2001 9:49 AM
To: samba@lists.samba.org; samba-technical@samba.org
Subject: URGENT REPOST: Maximum number of files i
Hi,
Is anyone out there using samba to share out over 6.4 million files ?
We've just had our production system grind to a halt again when we hit
this,
apparently samba, limitation.
The only noticeable thing that I get from the log files is the following
messages:
[2001/09/20 00:32:03, 3] lib/util.c:(1627)
set_maxfiles: setrlimit for RLIMIT_NOFILE for 10010 max files failed
with
error Invalid argument
[2001/09/20 00:32:03, 1] smbd/files.c:(151)
file_init: Information only: requested 10000 open files, 4086 are
available.
However as I understand it, the max files option is the number of open
files
per process, rather than the number of actual files...
Could anyone give me some pointers ??
TIA,
Rob Leadbeater
-----Original Message-----
From: rob.leadbeater@lynx.co.uk [SMTP:MIME :rob.leadbeater@lynx.co.uk]
Sent: Thursday, September 06, 2001 5:24 PM
To: samba@lists.samba.org
Subject: Maximum number of files in a samba share
Hi,
Is there a maximum number of files that can be located in a samba share ?
We're running Samba 2.2.0 on Compaq Tru64 Unix 4.0F.
Everything had been running fine up until yesterday when suddenly we were
getting lots of odd network looking faults.
We eventually pin pointed that an smbd process was taking 99.9% CPU time,
which was related to a Windows machine attempting to copy files onto the
Unix
box.
The number of files in the file system which is shared out is currently
around
6.4 million.
I'm pretty sure that we've not got a unix problem, as creating files
directly
from unix works fine.
Attempting to create files via Windows (NT) eventually gives an
"Unexpected
network error".
If I delete some files from the samba share, then the Windows to Unix copy
starts working again...
Any ideas would be greatly welcomed.
Regards,
Rob Leadbeater
This message is intended only for the use of the person(s) ("The intended
Recipient(s)") to whom it is addressed. It may contain information which
is privileged and confidential within the meaning of applicable law. If
you are not the intended recipient, please contact the sender as soon as
possible. The views expressed in this communication are not necessarily
those held by LYNX Express Limited.
-- To unsubscribe from this list go to the following URL and read the
instructions: http://lists.samba.org/mailman/listinfo/samba
-- To unsubscribe from this list go to the following URL and read the
instructions: http://lists.samba.org/mailman/listinfo/samba
This message is intended only for the use of the person(s) ("The intended
Recipient(s)") to whom it is addressed. It may contain information which
is privileged and confidential within the meaning of applicable law. If
you are not the intended recipient, please contact the sender as soon as
possible. The views expressed in this communication are not necessarily
those held by LYNX Express Limited.
Hi,
We've eventually pin pointed this to be a problem with the underlying file
system, rather than Samba.
Hopefully Compaq will come up with a solution before too long !!
Regards,
Rob Leadbeater
-----Original Message-----
From: rob.leadbeater@lynx.co.uk [SMTP:MIME :rob.leadbeater@lynx.co.uk]
Sent: Friday, September 21, 2001 1:50 PM
To: samba@lists.samba.org; samba-technical@samba.org
Subject: URGENT REPOST: Maximum number of files i
Hi,
Is anyone out there using samba to share out over 6.4 million files ?
We've just had our production system grind to a halt again when we hit
this, apparently samba, limitation.
The only noticeable thing that I get from the log files is the following
messages:
[2001/09/20 00:32:03, 3] lib/util.c:(1627)
set_maxfiles: setrlimit for RLIMIT_NOFILE for 10010 max files failed
with error Invalid argument
[2001/09/20 00:32:03, 1] smbd/files.c:(151)
file_init: Information only: requested 10000 open files, 4086 are
available.
However as I understand it, the max files option is the number of open
files per process, rather than the number of actual files...
Could anyone give me some pointers ??
TIA,
Rob Leadbeater
-----Original Message-----
From: rob.leadbeater@lynx.co.uk [SMTP:MIME :rob.leadbeater@lynx.co.uk]
Sent: Thursday, September 06, 2001 5:24 PM
To: samba@lists.samba.org
Subject: Maximum number of files in a samba share
Hi,
Is there a maximum number of files that can be located in a samba share ?
We're running Samba 2.2.0 on Compaq Tru64 Unix 4.0F.
Everything had been running fine up until yesterday when suddenly we were
getting lots of odd network looking faults.
We eventually pin pointed that an smbd process was taking 99.9% CPU time,
which was related to a Windows machine attempting to copy files onto the
Unix
box.
The number of files in the file system which is shared out is currently
around
6.4 million.
I'm pretty sure that we've not got a unix problem, as creating files
directly
from unix works fine.
Attempting to create files via Windows (NT) eventually gives an
"Unexpected
network error".
If I delete some files from the samba share, then the Windows to Unix copy
starts working again...
Any ideas would be greatly welcomed.
Regards,
Rob Leadbeater
This message is intended only for the use of the person(s) ("The intended
Recipient(s)") to whom it is addressed. It may contain information which
is privileged and confidential within the meaning of applicable law. If
you are not the intended recipient, please contact the sender as soon as
possible. The views expressed in this communication are not necessarily
those held by LYNX Express Limited.
-- To unsubscribe from this list go to the following URL and read the
instructions: http://lists.samba.org/mailman/listinfo/samba
This message is intended only for the use of the person(s) ("The intended
Recipient(s)") to whom it is addressed. It may contain information which
is privileged and confidential within the meaning of applicable law. If
you are not the intended recipient, please contact the sender as soon as
possible. The views expressed in this communication are not necessarily
those held by LYNX Express Limited.