samba - Apr 2004 - Slow Directory listing for large number of files

Hello,

I have a samba server which has a directory with 11764 files. (The
program stores information about jobs and each job has 3 files, nothing
I can do about it).

When I do a listing of that directory using smbclient, it takes 15+
seconds to complete, same thing on Windows XP.

Does anyone have any ideas on how I can speed this up?

I did a log on log level 10, it is accessible at
http://www.enger.org/backup.log (10761841 bytes).

Timeline I have noticed:

[0 sec] Received the request
[0 sec] Starts running on all files:
[2004/04/19 16:31:51, 8] lib/util.c:is_in_path(1496)
  is_in_path: A0000009.job
[2004/04/19 16:31:51, 8] lib/util.c:is_in_path(1500)
  is_in_path: no name list.
[2 sec] starts doing this on all files:
[2004/04/19 16:31:53, 5] smbd/trans2.c:get_lanman2_dir_entry(606)
  get_lanman2_dir_entry found Detail/A000MOON.job fname=A000MOON.job
[2004/04/19 16:31:53, 8] smbd/trans2.c:get_lanman2_dir_entry(521)
  get_lanman2_dir_entry:readdir on dirptr 0x87f3960 now at offset 8
[2004/04/19 16:31:53, 10] lib/ms_fnmatch.c:ms_fnmatch(240)
  ms_fnmatch(*,A000RAJA.pfl) -> 0
[2004/04/19 16:31:53, 8] smbd/dosmode.c:dos_mode(122)
  dos_mode: Detail/A000RAJA.pfl
[2004/04/19 16:31:53, 8] lib/util.c:is_in_path(1496)
  is_in_path: Detail/A000RAJA.pfl
[2004/04/19 16:31:53, 8] lib/util.c:is_in_path(1500)
  is_in_path: no name list.
[2004/04/19 16:31:53, 8] smbd/dosmode.c:dos_mode(170)
  dos_mode returning
[27 sec] finishes off
[2004/04/19 16:32:18, 9] smbd/trans2.c:send_trans2_replies(178)
  t2_rep: params_sent_thistime = 8, data_sent_thistime = 4200,
useable_space = 131012
[2004/04/19 16:32:18, 9] smbd/trans2.c:send_trans2_replies(180)
  t2_rep: params_to_send = 8, data_to_send = 4200, paramsize = 8,
datasize = 4200
[2004/04/19 16:32:18, 6] lib/util_sock.c:write_socket(407)
  write_socket(5,4268)
[2004/04/19 16:32:18, 6] lib/util_sock.c:write_socket(410)
  write_socket(5,4268) wrote 4268
[2004/04/19 16:32:18, 3] smbd/trans2.c:call_trans2findnext(1369)
  SMBtrans2 mask=* directory=Detail dirtype=22 numentries=35

If anyone has any ideas that would be great.

I have tried Samba 2.2.7a and 3.0.2 which both have the same issue. Log
is from 3.0.2. Running Fedora Core 1 and ext3 file system.

Thanks,
	Matthew Enger
	m.enger@xi.com.au



-- 
Matthew Enger
m.enger@xi.com.au
Mob: 0412 463 080
Direct: (03) 9747 4001
X Integration
A Netcruiser Pty Ltd business
Ph: 1300 730 997
Fax: 1300 136 720

Matthew Enger wrote:
> I have a samba server which has a directory with 11764 files. (The
> program stores information about jobs and each job has 3 files, nothing
> I can do about it).
> 
> When I do a listing of that directory using smbclient, it takes 15+
> seconds to complete, same thing on Windows XP.
>
> I have tried Samba 2.2.7a and 3.0.2 which both have the same issue. Log
> is from 3.0.2. Running Fedora Core 1 and ext3 file system.
How fast do you get a listing when directly on the server? Most 
filesystems become really slow once you get more than a couple thousand 
files in them.

If listings are slow on the server, the only thing you can do is divide 
up the files into subdirectories or switch to something like reiserfs 
that can handle large directories.

I don't know if anything can help XP though. I once had a directory with 
~25,000 files on a samba server, and XP would just hang when trying to 
list it.

m.