samba - Feb 2019 - 32 seconds vs 72 minutes -- expected performance difference?

>
> When you provide an exact vers= then no auto-negotiation happens (unless
> you pass "vers=3" which essentially means 3.x: use 3.0 or above).
You
> either get the connection or mount fails. None the less, you can dump
> the current SMB ressources managed by the kernel by looking at
> /proc/fs/cifs/DebugData if your kernel is recent enough it should show
> the SMB version.
>
There is something going horribly wrong with my configuration. Notice that
CIFS version in the debug output below --

Display Internal CIFS Data Structures for Debugging
---------------------------------------------------
CIFS Version 2.10
Features: dfs fscache lanman posix spnego xattr acl
Active VFS Requests: 0
Servers:
Number of credits: 510
1) Name: [REDACTED] Uses: 1 Capability: 0x300047     Session Status: 1
 TCP status: 1
        Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0
        Shares:
        0) IPC: \\[REDACTED]\IPC$ Mounts: 1 DevInfo: 0x0 Attributes: 0x0
        PathComponentMax: 0 Status: 1 type: 0
        Share Capabilities: None        Share Flags: 0x8000

        1) \\[REDACTED]\uploaded_files Mounts: 1 DevInfo: 0x20 Attributes:
0x1006f
        PathComponentMax: 255 Status: 1 type: DISK
        Share Capabilities: None Aligned, Partition Aligned,    Share
Flags: 0x8000     Optimal sector size: 0x200

        MIDs:


> * nounix means the server doesn't support unix extensions. These
> extensions are only
>   supported by samba and for SMB1 only at the moment. Work is in progress
>   to add them to SMB2+.
>
Will this in any way interfere with "case sensitive = yes" -
https://www.samba.org/samba/docs/current/man-html/smb.conf.5.html#NAMEMANGLINGSECT
?

> *  mapposix has been default for a while, it maps reserved characters (: ?
>   \ " etc) to reserved unicode characters to/from the servers so you
can
>   use those characters in file names transparently from linux, without
>   needing to rely on protocol extensions.
>
Again, this sounds like it directly conflicts with `nounix`!

> If you provide a trace we could see if any unecesarry packets are
> exchanged or if something goes in loop, or if due to some error, the
> kernel close and reopens the connection often and looses time, ...
>
I have a 1.3 GB capture.pcap from the command you shared earlier. `ls -lR`
ran only for a few minutes, and ended up generating so much network
chatter!

Btw, I have `smb encrypt = required` on the server config. Ideally, I would
hope, the network capture would be useless!

Apart from network capture, is there any debug logging that I can enable to
give similar insights?

-- Saurabh.

> Apart from network capture, is there any debug logging that I can enable
> to give similar insights?
>
I found something interesting in /proc/fs/cifs/Stats. Notice the
"QueryDirectories > Failed" number. This keeps increasing as along
as `ls
-lR` is running.

Resources in use
CIFS Session: 1
Share (unique mount targets): 2
SMB Request/Response Buffer: 1 Pool size: 5
SMB Small Req/Resp Buffer: 2 Pool size: 30
Operations (MIDs): 1

2 session 4 share reconnects
Total vfs operations: 615094 maximum at one time: 2

1) \\[REDACTED]\uploaded_files
SMBs: 124332
Negotiates: 0 sent 0 failed
SessionSetups: 0 sent 0 failed
Logoffs: 0 sent 0 failed
TreeConnects: 0 sent 0 failed
TreeDisconnects: 0 sent 0 failed
Creates: 0 sent 1 failed
Closes: 0 sent 0 failed
Flushes: 0 sent 0 failed
Reads: 0 sent 0 failed
Writes: 0 sent 0 failed
Locks: 0 sent 0 failed
IOCTLs: 0 sent 0 failed
Cancels: 0 sent 0 failed
Echos: 0 sent 0 failed
QueryDirectories: 0 sent 29672 failed
ChangeNotifies: 0 sent 0 failed
QueryInfos: 0 sent 0 failed
SetInfos: 0 sent 0 failed
OplockBreaks: 0 sent 0 failed


-- Saurabh.

Saurabh Nanda <saurabhnanda at gmail.com> writes:
> There is something going horribly wrong with my configuration. Notice that
> CIFS version in the debug output below --
CIFS version is the kernel module version, not the protocol version used
for the connection. Nothing horribly wrong there.
> Will this in any way interfere with "case sensitive = yes" -
>
https://www.samba.org/samba/docs/current/man-html/smb.conf.5.html#NAMEMANGLINGSECT
> ?
The kernel client tries to be case sensitive by default as well, I don't
think there's a problem there either.
>
>> *  mapposix has been default for a while, it maps reserved characters
(: ?
>>   \ " etc) to reserved unicode characters to/from the servers so
you can
>>   use those characters in file names transparently from linux, without
>>   needing to rely on protocol extensions.
>>
>
> Again, this sounds like it directly conflicts with `nounix`!
No, nounix means server doesn't support unix extensions (cannot create
symlinks, block/char devices, unix sockets, ...)

mapposix is just character remapping in the filenames and doesn't
require any specific extensions. You write to a file "foo*bar.txt",
kernel will actually send "foo<some weird unicode char>bar.txt"
and when
it receives filenames it does the opposite remapping.
> I have a 1.3 GB capture.pcap from the command you shared earlier. `ls -lR`
> ran only for a few minutes, and ended up generating so much network
> chatter!
Yes, I warned you about the size. As I said in my previous email, you
can try to compress it (gzip mycapture.pcap) or capture for a smaller
amount of time.
> Btw, I have `smb encrypt = required` on the server config. Ideally, I would
> hope, the network capture would be useless!
Try disabling encryption then. See if it works better. If it doesn't at
least the capture will be in clear for analysis. We could also decrypt
it by dumping the session keys but this will require building your own
samba or your own kernel. [1]
> Apart from network capture, is there any debug logging that I can enable to
> give similar insights?
You can make the kernel more verbose, but given the amount of data it
will be extremely noisy. See [2].

1: https://wiki.samba.org/index.php/Wireshark_Decryption
2: https://wiki.samba.org/index.php/Bug_Reporting#cifs.ko

Cheers,
-- 
Aurélien Aptel / SUSE Labs Samba Team
GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
SUSE Linux GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

Saurabh Nanda <saurabhnanda at gmail.com> writes:
> I found something interesting in /proc/fs/cifs/Stats. Notice the
> "QueryDirectories > Failed" number. This keeps increasing as
along as `ls
> -lR` is running.
That's interesting indeed. The verbose logs and network trace would tell
us more.

-- 
Aurélien Aptel / SUSE Labs Samba Team
GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
SUSE Linux GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)