thr3ads.net - samba - [Samba] Poor performance getting lots of small files with WinXP/Win2k vs OS2 [Sep 2006]

If this information is useful, please help other people find it:
Share via:
Alain.Gorlier@altissemiconductor.com
2006-Sep-22 04:56 UTC
[Samba] Poor performance getting lots of small files with WinXP/Win2k vs OS2

Hi,
We're doing some of performance tests with Linux Samba v3.
We're comparing WinXP/Win2k clients vs OS2 clients accessing a samba 
server.

For one test, we're coping 10.000 small files.
Let's say, we run the command copy \\sambaserver\10000files\*.* 
c:\data\10000files

With an OS2 client it takes 23 seconds to complete.
With an Windows client it takes  460 seconds to complete.

Doing a Ethereal capture, the network usage is very different.
OS2 :
No.     Time        Source                Destination           Protocol 
Info

     15 0.588766    10.16.43.123          10.240.47.86          SMB Open 
AndX Request, Path: \10000\0511_3008.IN; Read AndX, FID: 0x0000, 4096 
bytes at offset 0
     16 0.589008    10.240.47.86          10.16.43.123          SMB Open 
AndX Response, FID: 0x2a81; Read AndX, FID: 0x0000, 74 bytes
     17 0.589251    10.16.43.123          10.240.47.86          SMB Close 
Request, FID: 0x2a81
     18 0.589494    10.240.47.86          10.16.43.123          SMB Close 
Response

WinXP SP2:


Trace Windows XP:
No.     Time        Source                Destination           Protocol 
Info
     16 1.001734    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_PATH_INFO, Query File Basic Info, Path: \1000\0D02_1707.IN
     17 1.001968    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_PATH_INFO
     18 1.002454    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_PATH_INFO, Query File Basic Info, Path: \1000\0D02_1707.IN
     19 1.002697    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_PATH_INFO
     20 1.002940    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_PATH_INFO, Query File Basic Info, Path: \1000
     21 1.003184    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_PATH_INFO
     22 1.003427    10.16.37.96           10.240.47.86          SMB Trans2 
Request, FIND_FIRST2, Pattern: \1000\0D02_1707.IN
     23 1.003670    10.240.47.86          10.16.37.96           SMB Trans2 
Response, FIND_FIRST2, Files: 0D02_1707.IN
     24 1.003913    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_PATH_INFO, Query File Basic Info, Path: \1000\0D02_1707.IN
     25 1.004157    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_PATH_INFO
     26 1.004400    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_PATH_INFO, Query File Standard Info, Path: 
\1000\0D02_1707.IN
     27 1.004402    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_PATH_INFO
     28 1.004886    10.16.37.96           10.240.47.86          SMB NT 
Create AndX Request, Path: \1000\0D02_1707.IN
     29 1.005129    10.240.47.86          10.16.37.96           SMB NT 
Create AndX Response, FID: 0x1cf0
     30 1.005373    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf0, Query File Internal Info
     31 1.005615    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
     32 1.006102    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf0, Query File Basic Info
     33 1.006104    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
     34 1.006345    10.16.37.96           10.240.47.86          SMB Trans2 
Request, SET_FILE_INFO, FID: 0x1cf0
     35 1.006589    10.240.47.86          10.16.37.96           SMB Trans2 
Response, SET_FILE_INFO
     36 1.007075    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf0, 222 bytes at offset 0
     37 1.007318    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf0, 222 bytes
     38 1.007804    10.16.37.96           10.240.47.86          SMB Close 
Request, FID: 0x1cf0
     39 1.008292    10.240.47.86          10.16.37.96           SMB Close 
Response
     40 1.008294    10.16.37.96           10.240.47.86          SMB NT 
Create AndX Request, Path: \1000\0D02_1707.IN
     41 1.008778    10.240.47.86          10.16.37.96           SMB NT 
Create AndX Response, FID: 0x1cf1
     42 1.008780    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf1, Query File Basic Info
     43 1.009021    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
     44 1.009263    10.16.37.96           10.240.47.86          SMB Close 
Request, FID: 0x1cf1
     45 1.009750    10.240.47.86          10.16.37.96           SMB Close 
Response
     46 1.009994    10.16.37.96           10.240.47.86          SMB NT 
Create AndX Request, Path: \1000\0D02_1707.IN
     47 1.010236    10.240.47.86          10.16.37.96           SMB NT 
Create AndX Response, FID: 0x1cf2
     48 1.010479    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf2, 222 bytes at offset 0
     49 1.010723    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf2, 222 bytes
     50 1.011209    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_PATH_INFO, Query File Basic Info, Path: \1000\0D02_1707.IN
     51 1.011452    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_PATH_INFO
     52 1.011696    10.16.37.96           10.240.47.86          SMB NT 
Create AndX Request, Path: \1000\0D02_1707.IN
     53 1.011939    10.240.47.86          10.16.37.96           SMB 
Locking AndX Request, FID: 0x1cf2
     54 1.012182    10.16.37.96           10.240.47.86          SMB 
Locking AndX Request, FID: 0x1cf2
     55 1.012669    10.240.47.86          10.16.37.96           SMB NT 
Create AndX Response, FID: 0x1cf4
     56 1.013155    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Basic Info
     57 1.013157    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
     58 1.013398    10.16.37.96           10.240.47.86          SMB Trans2 
Request, SET_FILE_INFO, FID: 0x1cf4
     59 1.013641    10.240.47.86          10.16.37.96           SMB Trans2 
Response, SET_FILE_INFO
     60 1.013884    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
     61 1.014127    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
     62 1.014370    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 256 bytes at offset 0
     63 1.014614    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 222 bytes
     64 1.014857    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 34 bytes at offset 222
     65 1.015100    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 0 bytes
     66 1.015344    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 256 bytes at offset 1
     67 1.015587    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 221 bytes
     68 1.015830    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 35 bytes at offset 222
     69 1.016074    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 0 bytes
     70 1.016563    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 512 bytes at offset 0
     71 1.016565    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 222 bytes
     72 1.016803    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 290 bytes at offset 222
     73 1.017045    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 0 bytes
     74 1.017533    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 512 bytes at offset 0
     75 1.017776    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 222 bytes
     76 1.017778    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 290 bytes at offset 222
     77 1.018019    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 0 bytes
     78 1.018262    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
     79 1.018505    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
     80 1.018993    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 2 bytes at offset 0
     81 1.018995    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 2 bytes
     82 1.019235    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
     83 1.019478    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
     84 1.019722    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 512 bytes at offset 0
     85 1.019965    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 222 bytes
     86 1.020208    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 290 bytes at offset 222
     87 1.020451    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 0 bytes
     88 1.020694    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
     89 1.020937    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
     90 1.021424    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
     91 1.021426    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
     92 1.021667    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 222 bytes at offset 0
     93 1.021910    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 222 bytes
     94 1.022153    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 4096 bytes at offset 0
     95 1.022396    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 222 bytes
     96 1.022640    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 3874 bytes at offset 222
     97 1.022883    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 0 bytes
     98 1.023369    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 512 bytes at offset 0
     99 1.023613    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 222 bytes
    100 1.023616    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 290 bytes at offset 222
    101 1.023856    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 0 bytes
    102 1.024099    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 64 bytes at offset 0
    103 1.024346    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 64 bytes
    104 1.024587    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
    105 1.024830    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
    106 1.025317    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 512 bytes at offset 0
    107 1.025561    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 222 bytes
    108 1.025564    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf4, 290 bytes at offset 222
    109 1.025803    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf4, 0 bytes
    110 1.026291    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
    111 1.026778    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
    112 1.026782    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
    113 1.027019    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
    114 1.027262    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
    115 1.027505    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
    116 1.027748    10.16.37.96           10.240.47.86          SMB Close 
Request, FID: 0x1cf4
    117 1.028236    10.240.47.86          10.16.37.96           SMB Close 
Response
    118 1.028479    10.16.37.96           10.240.47.86          SMB NT 
Create AndX Request, Path: \1000\0D02_1707.IN
    119 1.028723    10.240.47.86          10.16.37.96           SMB NT 
Create AndX Response, FID: 0x1cf5
    120 1.028727    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf5, Query File Basic Info
    121 1.028965    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
    122 1.029207    10.16.37.96           10.240.47.86          SMB Close 
Request, FID: 0x1cf5
    123 1.029698    10.240.47.86          10.16.37.96           SMB Close 
Response
    124 1.029938    10.16.37.96           10.240.47.86          SMB NT 
Create AndX Request, Path: \1000\0D02_1707.IN
    125 1.030181    10.240.47.86          10.16.37.96           SMB NT 
Create AndX Response, FID: 0x1cf6
    126 1.030184    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf6, Query File Standard Info
    127 1.030423    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
    128 1.030666    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf6, Query File Basic Info
    129 1.031156    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
    130 1.031159    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf6, Query File Stream Info
    131 1.031396    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
    132 1.031640    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf6, Query File Basic Info
    133 1.031883    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
    134 1.032126    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf6, Query File EA Info
    135 1.032369    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
    136 1.033108    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FS_INFO, Query FS Attribute Info
    137 1.033111    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FS_INFO
    138 1.033589    10.16.37.96           10.240.47.86          SMB Trans2 
Request, QUERY_FILE_INFO, FID: 0x1cf6, Query File Standard Info
    139 1.033592    10.240.47.86          10.16.37.96           SMB Trans2 
Response, QUERY_FILE_INFO
    140 1.034074    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf6, 4096 bytes at offset 0
    141 1.034317    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf6, 222 bytes
    142 1.034320    10.16.37.96           10.240.47.86          SMB Read 
AndX Request, FID: 0x1cf6, 3874 bytes at offset 222
    143 1.034558    10.240.47.86          10.16.37.96           SMB Read 
AndX Response, FID: 0x1cf6, 0 bytes
    144 1.035293    10.16.37.96           10.240.47.86          SMB Close 
Request, FID: 0x1cf6
    145 1.035775    10.240.47.86          10.16.37.96           SMB Close 
Response
    146 1.035779    10.16.37.96           10.240.47.86          SMB Close 
Request, FID: 0x1cf2
    147 1.036265    10.240.47.86          10.16.37.96           SMB Close 
Response

So, the performance difference is easy to explain....

Is there a way to avoid such network usage on a winXP client ?  At the 
client side ? At the Samba server side ?
Looks like it's related to the Windows NTFS environment....
We've tried to define the Samba share drive as a samba drive (fstype = 
samba in smb.conf), but the performance remains the same ....

We're currently looking for a migration plan from OS2 clients/OS2 Lan 
servers to Windows Clients/Samba servers. We may encountered response time 
problems at the Win client side and CPU/Network load problem on the samba 
server side....

Is there some recommandations regarding copying lots of files for Windows 
clients ?
Is there some recommendation regarding Application Developpement for 
Windows accessing Samba servers ?

Regards,
Possibly Parallel Threads

Search for more maybe matching threads
samba - Sep 2006 - Poor performance getting lots of small files with WinXP/Win2k vs OS2

[Samba] Poor performance getting lots of small files with WinXP/Win2k vs OS2

Possibly Parallel Threads

Wisdom of the Ancients
