Matthew Enger
2004-Apr-19 12:29 UTC
[Samba] Slow Directory listing for large number of files
Hello, I have a samba server which has a directory with 11764 files. (The program stores information about jobs and each job has 3 files, nothing I can do about it). When I do a listing of that directory using smbclient, it takes 15+ seconds to complete, same thing on Windows XP. Does anyone have any ideas on how I can speed this up? I did a log on log level 10, it is accessible at http://www.enger.org/backup.log (10761841 bytes). Timeline I have noticed: [0 sec] Received the request [0 sec] Starts running on all files: [2004/04/19 16:31:51, 8] lib/util.c:is_in_path(1496) is_in_path: A0000009.job [2004/04/19 16:31:51, 8] lib/util.c:is_in_path(1500) is_in_path: no name list. [2 sec] starts doing this on all files: [2004/04/19 16:31:53, 5] smbd/trans2.c:get_lanman2_dir_entry(606) get_lanman2_dir_entry found Detail/A000MOON.job fname=A000MOON.job [2004/04/19 16:31:53, 8] smbd/trans2.c:get_lanman2_dir_entry(521) get_lanman2_dir_entry:readdir on dirptr 0x87f3960 now at offset 8 [2004/04/19 16:31:53, 10] lib/ms_fnmatch.c:ms_fnmatch(240) ms_fnmatch(*,A000RAJA.pfl) -> 0 [2004/04/19 16:31:53, 8] smbd/dosmode.c:dos_mode(122) dos_mode: Detail/A000RAJA.pfl [2004/04/19 16:31:53, 8] lib/util.c:is_in_path(1496) is_in_path: Detail/A000RAJA.pfl [2004/04/19 16:31:53, 8] lib/util.c:is_in_path(1500) is_in_path: no name list. [2004/04/19 16:31:53, 8] smbd/dosmode.c:dos_mode(170) dos_mode returning [27 sec] finishes off [2004/04/19 16:32:18, 9] smbd/trans2.c:send_trans2_replies(178) t2_rep: params_sent_thistime = 8, data_sent_thistime = 4200, useable_space = 131012 [2004/04/19 16:32:18, 9] smbd/trans2.c:send_trans2_replies(180) t2_rep: params_to_send = 8, data_to_send = 4200, paramsize = 8, datasize = 4200 [2004/04/19 16:32:18, 6] lib/util_sock.c:write_socket(407) write_socket(5,4268) [2004/04/19 16:32:18, 6] lib/util_sock.c:write_socket(410) write_socket(5,4268) wrote 4268 [2004/04/19 16:32:18, 3] smbd/trans2.c:call_trans2findnext(1369) SMBtrans2 mask=* directory=Detail dirtype=22 numentries=35 If anyone has any ideas that would be great. I have tried Samba 2.2.7a and 3.0.2 which both have the same issue. Log is from 3.0.2. Running Fedora Core 1 and ext3 file system. Thanks, Matthew Enger m.enger@xi.com.au -- Matthew Enger m.enger@xi.com.au Mob: 0412 463 080 Direct: (03) 9747 4001 X Integration A Netcruiser Pty Ltd business Ph: 1300 730 997 Fax: 1300 136 720
Michael Carmack
2004-Apr-19 15:59 UTC
[Samba] Slow Directory listing for large number of files
Matthew Enger wrote:> I have a samba server which has a directory with 11764 files. (The > program stores information about jobs and each job has 3 files, nothing > I can do about it). > > When I do a listing of that directory using smbclient, it takes 15+ > seconds to complete, same thing on Windows XP. > > I have tried Samba 2.2.7a and 3.0.2 which both have the same issue. Log > is from 3.0.2. Running Fedora Core 1 and ext3 file system.How fast do you get a listing when directly on the server? Most filesystems become really slow once you get more than a couple thousand files in them. If listings are slow on the server, the only thing you can do is divide up the files into subdirectories or switch to something like reiserfs that can handle large directories. I don't know if anything can help XP though. I once had a directory with ~25,000 files on a samba server, and XP would just hang when trying to list it. m.