Alain.Gorlier@altissemiconductor.com
2006-Sep-21 09:32 UTC
[Samba] Fw: Poor performance getting lots of small files with WinXP/Win2k vs OS2
Hi,
We're doing some of performance tests with Linux Samba v3.
We're comparing WinXP/Win2k clients vs OS2 clients accessing a samba
server.
For one test, we're coping 10.000 small files.
Let's say, we run the command copy \\sambaserver\10000files\*.*
c:\data\10000files
With an OS2 client it takes 23 seconds to complete.
With an Windows client it takes 460 seconds to complete.
Doing a Ethereal capture, the network usage is very different.
OS2 :
No. Time Source Destination Protocol
Info
15 0.588766 10.16.43.123 10.240.47.86 SMB Open
AndX Request, Path: \10000\0511_3008.IN; Read AndX, FID: 0x0000, 4096
bytes at offset 0
16 0.589008 10.240.47.86 10.16.43.123 SMB Open
AndX Response, FID: 0x2a81; Read AndX, FID: 0x0000, 74 bytes
17 0.589251 10.16.43.123 10.240.47.86 SMB Close
Request, FID: 0x2a81
18 0.589494 10.240.47.86 10.16.43.123 SMB Close
Response
WinXP SP2:
Trace Windows XP:
No. Time Source Destination Protocol
Info
16 1.001734 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_PATH_INFO, Query File Basic Info, Path: \1000\0D02_1707.IN
17 1.001968 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_PATH_INFO
18 1.002454 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_PATH_INFO, Query File Basic Info, Path: \1000\0D02_1707.IN
19 1.002697 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_PATH_INFO
20 1.002940 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_PATH_INFO, Query File Basic Info, Path: \1000
21 1.003184 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_PATH_INFO
22 1.003427 10.16.37.96 10.240.47.86 SMB Trans2
Request, FIND_FIRST2, Pattern: \1000\0D02_1707.IN
23 1.003670 10.240.47.86 10.16.37.96 SMB Trans2
Response, FIND_FIRST2, Files: 0D02_1707.IN
24 1.003913 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_PATH_INFO, Query File Basic Info, Path: \1000\0D02_1707.IN
25 1.004157 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_PATH_INFO
26 1.004400 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_PATH_INFO, Query File Standard Info, Path:
\1000\0D02_1707.IN
27 1.004402 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_PATH_INFO
28 1.004886 10.16.37.96 10.240.47.86 SMB NT
Create AndX Request, Path: \1000\0D02_1707.IN
29 1.005129 10.240.47.86 10.16.37.96 SMB NT
Create AndX Response, FID: 0x1cf0
30 1.005373 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf0, Query File Internal Info
31 1.005615 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
32 1.006102 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf0, Query File Basic Info
33 1.006104 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
34 1.006345 10.16.37.96 10.240.47.86 SMB Trans2
Request, SET_FILE_INFO, FID: 0x1cf0
35 1.006589 10.240.47.86 10.16.37.96 SMB Trans2
Response, SET_FILE_INFO
36 1.007075 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf0, 222 bytes at offset 0
37 1.007318 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf0, 222 bytes
38 1.007804 10.16.37.96 10.240.47.86 SMB Close
Request, FID: 0x1cf0
39 1.008292 10.240.47.86 10.16.37.96 SMB Close
Response
40 1.008294 10.16.37.96 10.240.47.86 SMB NT
Create AndX Request, Path: \1000\0D02_1707.IN
41 1.008778 10.240.47.86 10.16.37.96 SMB NT
Create AndX Response, FID: 0x1cf1
42 1.008780 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf1, Query File Basic Info
43 1.009021 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
44 1.009263 10.16.37.96 10.240.47.86 SMB Close
Request, FID: 0x1cf1
45 1.009750 10.240.47.86 10.16.37.96 SMB Close
Response
46 1.009994 10.16.37.96 10.240.47.86 SMB NT
Create AndX Request, Path: \1000\0D02_1707.IN
47 1.010236 10.240.47.86 10.16.37.96 SMB NT
Create AndX Response, FID: 0x1cf2
48 1.010479 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf2, 222 bytes at offset 0
49 1.010723 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf2, 222 bytes
50 1.011209 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_PATH_INFO, Query File Basic Info, Path: \1000\0D02_1707.IN
51 1.011452 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_PATH_INFO
52 1.011696 10.16.37.96 10.240.47.86 SMB NT
Create AndX Request, Path: \1000\0D02_1707.IN
53 1.011939 10.240.47.86 10.16.37.96 SMB
Locking AndX Request, FID: 0x1cf2
54 1.012182 10.16.37.96 10.240.47.86 SMB
Locking AndX Request, FID: 0x1cf2
55 1.012669 10.240.47.86 10.16.37.96 SMB NT
Create AndX Response, FID: 0x1cf4
56 1.013155 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Basic Info
57 1.013157 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
58 1.013398 10.16.37.96 10.240.47.86 SMB Trans2
Request, SET_FILE_INFO, FID: 0x1cf4
59 1.013641 10.240.47.86 10.16.37.96 SMB Trans2
Response, SET_FILE_INFO
60 1.013884 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
61 1.014127 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
62 1.014370 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 256 bytes at offset 0
63 1.014614 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 222 bytes
64 1.014857 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 34 bytes at offset 222
65 1.015100 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 0 bytes
66 1.015344 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 256 bytes at offset 1
67 1.015587 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 221 bytes
68 1.015830 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 35 bytes at offset 222
69 1.016074 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 0 bytes
70 1.016563 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 512 bytes at offset 0
71 1.016565 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 222 bytes
72 1.016803 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 290 bytes at offset 222
73 1.017045 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 0 bytes
74 1.017533 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 512 bytes at offset 0
75 1.017776 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 222 bytes
76 1.017778 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 290 bytes at offset 222
77 1.018019 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 0 bytes
78 1.018262 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
79 1.018505 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
80 1.018993 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 2 bytes at offset 0
81 1.018995 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 2 bytes
82 1.019235 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
83 1.019478 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
84 1.019722 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 512 bytes at offset 0
85 1.019965 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 222 bytes
86 1.020208 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 290 bytes at offset 222
87 1.020451 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 0 bytes
88 1.020694 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
89 1.020937 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
90 1.021424 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
91 1.021426 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
92 1.021667 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 222 bytes at offset 0
93 1.021910 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 222 bytes
94 1.022153 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 4096 bytes at offset 0
95 1.022396 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 222 bytes
96 1.022640 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 3874 bytes at offset 222
97 1.022883 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 0 bytes
98 1.023369 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 512 bytes at offset 0
99 1.023613 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 222 bytes
100 1.023616 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 290 bytes at offset 222
101 1.023856 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 0 bytes
102 1.024099 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 64 bytes at offset 0
103 1.024346 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 64 bytes
104 1.024587 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
105 1.024830 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
106 1.025317 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 512 bytes at offset 0
107 1.025561 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 222 bytes
108 1.025564 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf4, 290 bytes at offset 222
109 1.025803 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf4, 0 bytes
110 1.026291 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
111 1.026778 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
112 1.026782 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
113 1.027019 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
114 1.027262 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf4, Query File Standard Info
115 1.027505 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
116 1.027748 10.16.37.96 10.240.47.86 SMB Close
Request, FID: 0x1cf4
117 1.028236 10.240.47.86 10.16.37.96 SMB Close
Response
118 1.028479 10.16.37.96 10.240.47.86 SMB NT
Create AndX Request, Path: \1000\0D02_1707.IN
119 1.028723 10.240.47.86 10.16.37.96 SMB NT
Create AndX Response, FID: 0x1cf5
120 1.028727 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf5, Query File Basic Info
121 1.028965 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
122 1.029207 10.16.37.96 10.240.47.86 SMB Close
Request, FID: 0x1cf5
123 1.029698 10.240.47.86 10.16.37.96 SMB Close
Response
124 1.029938 10.16.37.96 10.240.47.86 SMB NT
Create AndX Request, Path: \1000\0D02_1707.IN
125 1.030181 10.240.47.86 10.16.37.96 SMB NT
Create AndX Response, FID: 0x1cf6
126 1.030184 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf6, Query File Standard Info
127 1.030423 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
128 1.030666 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf6, Query File Basic Info
129 1.031156 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
130 1.031159 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf6, Query File Stream Info
131 1.031396 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
132 1.031640 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf6, Query File Basic Info
133 1.031883 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
134 1.032126 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf6, Query File EA Info
135 1.032369 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
136 1.033108 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FS_INFO, Query FS Attribute Info
137 1.033111 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FS_INFO
138 1.033589 10.16.37.96 10.240.47.86 SMB Trans2
Request, QUERY_FILE_INFO, FID: 0x1cf6, Query File Standard Info
139 1.033592 10.240.47.86 10.16.37.96 SMB Trans2
Response, QUERY_FILE_INFO
140 1.034074 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf6, 4096 bytes at offset 0
141 1.034317 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf6, 222 bytes
142 1.034320 10.16.37.96 10.240.47.86 SMB Read
AndX Request, FID: 0x1cf6, 3874 bytes at offset 222
143 1.034558 10.240.47.86 10.16.37.96 SMB Read
AndX Response, FID: 0x1cf6, 0 bytes
144 1.035293 10.16.37.96 10.240.47.86 SMB Close
Request, FID: 0x1cf6
145 1.035775 10.240.47.86 10.16.37.96 SMB Close
Response
146 1.035779 10.16.37.96 10.240.47.86 SMB Close
Request, FID: 0x1cf2
147 1.036265 10.240.47.86 10.16.37.96 SMB Close
Response
So, the performance difference is easy to explain....
Is there a way to avoid such network usage on a winXP client ? At the
client side ? At the Samba server side ?
Looks like it's related to the Windows NTFS environment....
We've tried to define the Samba share drive as a samba drive (fstype =
samba in smb.conf), but the performance remains the same ....
We're currently looking for a migration plan from OS2 clients/OS2 Lan
servers to Windows Clients/Samba servers. We may encountered response time
problems at the Win client side and CPU/Network load problem on the samba
server side....
Is there some recommandations regarding copying lots of files for Windows
clients ?
Is there some recommendation regarding Application Developpement for
Windows accessing Samba servers ?
Regards,
Jeremy Allison
2006-Sep-21 19:04 UTC
[Samba] Fw: Poor performance getting lots of small files with WinXP/Win2k vs OS2
On Thu, Sep 21, 2006 at 11:31:57AM +0200, Alain.Gorlier@altissemiconductor.com wrote:> Hi, > We're doing some of performance tests with Linux Samba v3. > We're comparing WinXP/Win2k clients vs OS2 clients accessing a samba > server. > > For one test, we're coping 10.000 small files. > Let's say, we run the command copy \\sambaserver\10000files\*.* > c:\data\10000files > > With an OS2 client it takes 23 seconds to complete. > With an Windows client it takes 460 seconds to complete. > > Doing a Ethereal capture, the network usage is very different. > OS2 : > No. Time Source Destination Protocol > Info > > So, the performance difference is easy to explain.... > > Is there a way to avoid such network usage on a winXP client ? At the > client side ? At the Samba server side ? > Looks like it's related to the Windows NTFS environment....This is breakage on the Windows XP client. There's nothing you can do except not use Windows, sorry. This is just what those clients do. Jeremy.
Apparently Analagous Threads
- Poor performance getting lots of small files with WinXP/Win2k vs OS2
- File writing strangeness
- Poor performance when accessing Linux from Windows XP because of too many QUERY_FILE_INFO requests
- Long Delays Displaying Small Folders in Windows Explorer (samba-3.0.10-1 FC2)
- Access Denied, Roaming Profile -- no apparent reason...