samba - Aug 2017 - Windows SMB2 client doing excessive, inefficient SMB2 Find (and other) requests

Andreas,

On Thu, Aug 24, 2017 at 03:37:07PM +0200, awl1 via samba
wrote:
> Before I follow your advice to move this whole issue/topic to the
> samba-technical alias and start over there with a clean description of the
> scenario and the findings, I have two simple questions:
> 
> 1) I plan to use a new, reproducible test scenario with 2000 random small
> files with a file length between 1 and 2048 bytes, created along the lines
> of the following:
> 
> for i in $(seq -f "%04g" 1 2000) ; do
>   length=`shuf -i 1-2048 -n 1`
>   head -c $length < /dev/urandom > file${i}.rnd
> done
This is overly complicated for this I guess, a simple touch file$i should do it.
> in order to make test data non-confidential (unfortunately, my previous
test
> files/packet traces were confidential). Do you agree that the above
> procedure is fine to create the test scenario?
> 
> 2) What about confidential data from SMB/SMB2 sessions (i.e. Samba
> usernames/passwords)? What do I meed to do to filter all information from
> Wireshark traces that points to users and passwords?
Passwords are not send over the wire, the user name is as are IP addresses. Use
a test user and setup VMs is a private test network if you care.
> More specifically, would filtering all SMB2 "SessionSetup"
request and
> response packets from Wireshark traces be sufficient to do so? What about
> machine names/IP addresses from my LAN? Any other such IP addresses/machine
> names contained in any packets other than (obviously) those of my
particular
> SMB2 client and Samba server (and Windows SMB2 server to compare)?
For the trace it's okay if it starts with the reproducer without session
setup.

If you can send a brief description of the issue alongside the traces to
samba-technical, I can try to get this to the right people.

Cheerio!
-slow

Hello Ralph,

many thanks for your fast reply and help with this issue! :-)

Am 31.08.2017 um 15:43 schrieb Ralph Böhme:
> On Thu, Aug 24, 2017 at 03:37:07PM +0200, awl1 via samba wrote:
>> Before I follow your advice to move this whole issue/topic to the
>> samba-technical alias and start over there with a clean description of
the
>> scenario and the findings, I have two simple questions:
>>
>> 1) I plan to use a new, reproducible test scenario with 2000 random
small
>> files with a file length between 1 and 2048 bytes, created along the
lines
>> of the following:
>>
>> for i in $(seq -f "%04g" 1 2000) ; do
>>    length=`shuf -i 1-2048 -n 1`
>>    head -c $length < /dev/urandom > file${i}.rnd
>> done
> This is overly complicated for this I guess, a simple touch file$i should
do it.
I see. While I fear that some logic might detect that we are about to 
send empty files and possibly use some short-cut paths, I will compare 
traces for the random content/length files and the zero-length files 
scenarios and decide based on my findings whether a huge number of 
zero-length files are indeed sufficient to reproduce the issue.
>> 2) What about confidential data from SMB/SMB2 sessions (i.e. Samba
>> usernames/passwords)? What do I meed to do to filter all information
from
>> Wireshark traces that points to users and passwords?
> Passwords are not send over the wire, the user name is as are IP addresses.
Use
> a test user and setup VMs is a private test network if you care.
>
> For the trace it's okay if it starts with the reproducer without
session setup.
OK, fine. I will then use a test user name and a specific private test 
IP range, and remove SessionSetup packets from the packet trace.
> If you can send a brief description of the issue alongside the traces to
> samba-technical, I can try to get this to the right people.
That's great. I hope to get all 12 scenarios traced over this coming 
weekend:

W1) copy from Win 10 SMB1 client (Explorer, xcopy /s) to SMB1 3.6.15 
server share
W2) copy from Win 10 SMB2 client (Explorer, xcopy /s) to SMB2 4.6.5 
server share
W3) copy from Win 10 SMB2 client (Explorer, xcopy /s) to Win 10 SMB2 
server share
W4) copy from Linux SMB1 client (cp -r, krusader) to SMB1 3.6.15 server 
share
W5) copy from Linux SMB2 client (cp -r, krusader) to SMB2 4.6.5 server share
W6) copy from Linux SMB2 client (cp -r, krusader) to Win 10 SMB2 server 
share

R1) copy to Win 10 SMB1 client (Explorer, xcopy /s) from SMB1 3.6.15 
server share
R2) copy to Win 10 SMB2 client (Explorer, xcopy /s) from SMB2 4.6.5 
server share
R3) copy to Win 10 SMB2 client (Explorer, xcopy /s) from Win 10 SMB2 
server share
R4) copy to Linux SMB1 client (cp -r, krusader) from SMB1 3.6.15 server 
share
R5) copy to Linux SMB2 client (cp -r, krusader) from SMB2 4.6.5 server 
share
R6) copy to Linux SMB2 client (cp -r, krusader) from Win 10 SMB2 server 
share

and will hopefully be able to produce a brief, but still meaningful 
summary/description of the differences/inefficiencies seen in 
client-side SMB2 requests that make SMB2 so much slower than SMB1, 
especially with Samba (but also with Windows SMB2 server)...

So please stay tuned - I'll get back to you soon on samba-technical... ;-)

Many thanks & best regards
Andreas

On Thu, Aug 31, 2017 at 06:04:04PM +0200, awl1 wrote:
> Am 31.08.2017 um 15:43 schrieb Ralph Böhme:
> > On Thu, Aug 24, 2017 at 03:37:07PM +0200, awl1 via samba wrote:
> > > 2) What about confidential data from SMB/SMB2 sessions (i.e.
Samba
> > > usernames/passwords)? What do I meed to do to filter all
information from
> > > Wireshark traces that points to users and passwords?
> > Passwords are not send over the wire, the user name is as are IP
addresses. Use
> > a test user and setup VMs is a private test network if you care.
> > 
> > For the trace it's okay if it starts with the reproducer without
session setup.
> OK, fine. I will then use a test user name and a specific private test IP
> range, and remove SessionSetup packets from the packet trace.
Why? Not that it really matters.
> > If you can send a brief description of the issue alongside the traces
to
> > samba-technical, I can try to get this to the right people.
>
> That's great. I hope to get all 12 scenarios traced over this coming
> weekend:
oh, that's way more then required. Just a trace of W3 will do.

Cheerio!
-slow

Ralph, All,

this is just to let you know that I haven't lost interest in this topic, 
but I have been and still am fighting some health issues at the moment. :-(

I hope to be able to start the thread on samba-technical about the 
Windows SMB2 performance issues due to inefficient Find requests soon 
(hopefully within the next two weeks)...

Sorry & best regards
Andreas


Am 31.08.2017 um 18:04 schrieb awl1:
> Hello Ralph,
>
> many thanks for your fast reply and help with this issue! :-)
>
> Am 31.08.2017 um 15:43 schrieb Ralph Böhme:
>> On Thu, Aug 24, 2017 at 03:37:07PM +0200, awl1 via samba wrote:
>>> Before I follow your advice to move this whole issue/topic to the
>>> samba-technical alias and start over there with a clean description
>>> of the
>>> scenario and the findings, I have two simple questions:
>>>
>>> 1) I plan to use a new, reproducible test scenario with 2000 random
>>> small
>>> files with a file length between 1 and 2048 bytes, created along
the
>>> lines
>>> of the following:
>>>
>>> for i in $(seq -f "%04g" 1 2000) ; do
>>>    length=`shuf -i 1-2048 -n 1`
>>>    head -c $length < /dev/urandom > file${i}.rnd
>>> done
>> This is overly complicated for this I guess, a simple touch file$i 
>> should do it.
> I see. While I fear that some logic might detect that we are about to 
> send empty files and possibly use some short-cut paths, I will compare 
> traces for the random content/length files and the zero-length files 
> scenarios and decide based on my findings whether a huge number of 
> zero-length files are indeed sufficient to reproduce the issue.
>
>>> 2) What about confidential data from SMB/SMB2 sessions (i.e. Samba
>>> usernames/passwords)? What do I meed to do to filter all
information
>>> from
>>> Wireshark traces that points to users and passwords?
>> Passwords are not send over the wire, the user name is as are IP 
>> addresses. Use
>> a test user and setup VMs is a private test network if you care.
>>
>> For the trace it's okay if it starts with the reproducer without 
>> session setup.
> OK, fine. I will then use a test user name and a specific private test 
> IP range, and remove SessionSetup packets from the packet trace.
>
>> If you can send a brief description of the issue alongside the traces
to
>> samba-technical, I can try to get this to the right people.
> That's great. I hope to get all 12 scenarios traced over this coming 
> weekend:
>
> W1) copy from Win 10 SMB1 client (Explorer, xcopy /s) to SMB1 3.6.15 
> server share
> W2) copy from Win 10 SMB2 client (Explorer, xcopy /s) to SMB2 4.6.5 
> server share
> W3) copy from Win 10 SMB2 client (Explorer, xcopy /s) to Win 10 SMB2 
> server share
> W4) copy from Linux SMB1 client (cp -r, krusader) to SMB1 3.6.15 
> server share
> W5) copy from Linux SMB2 client (cp -r, krusader) to SMB2 4.6.5 server 
> share
> W6) copy from Linux SMB2 client (cp -r, krusader) to Win 10 SMB2 
> server share
>
> R1) copy to Win 10 SMB1 client (Explorer, xcopy /s) from SMB1 3.6.15 
> server share
> R2) copy to Win 10 SMB2 client (Explorer, xcopy /s) from SMB2 4.6.5 
> server share
> R3) copy to Win 10 SMB2 client (Explorer, xcopy /s) from Win 10 SMB2 
> server share
> R4) copy to Linux SMB1 client (cp -r, krusader) from SMB1 3.6.15 
> server share
> R5) copy to Linux SMB2 client (cp -r, krusader) from SMB2 4.6.5 server 
> share
> R6) copy to Linux SMB2 client (cp -r, krusader) from Win 10 SMB2 
> server share
>
> and will hopefully be able to produce a brief, but still meaningful 
> summary/description of the differences/inefficiencies seen in 
> client-side SMB2 requests that make SMB2 so much slower than SMB1, 
> especially with Samba (but also with Windows SMB2 server)...
>
> So please stay tuned - I'll get back to you soon on samba-technical... 
> ;-)
>
> Many thanks & best regards
> Andreas
>