Saurabh Nanda
2019-Feb-18 10:43 UTC
[Samba] 32 seconds vs 72 minutes -- expected performance difference?
> > I tried looking at the network capture using `tcpdump -A -X` but wasn't > able to understand anything. I tried installing wireshark on a throwaway > cloud instance, but realised that it's a GUI program. Can you please help > be with the network trace at > https://www.dropbox.com/s/r8cn0qggrvmrpc3/dump.pcap?dl=0 ? It's capturing > the network chatter for ~1min of `ls -lR` on the shared folder. Does this > help in getting to the bottom of this problem? >Help in analysing the tcpdump to get to the bottom of this perf issue would be appreciated. -- Saurabh.
Aurélien Aptel
2019-Feb-21 18:56 UTC
[Samba] 32 seconds vs 72 minutes -- expected performance difference?
Saurabh Nanda <saurabhnanda at gmail.com> writes:> Help in analysing the tcpdump to get to the bottom of this perf issue would > be appreciated.I've looked at the trace but didn't find anything out of the ordinary. The "STATUS_NO_MORE_FILES" is erroneously taken into account in the stats because your kernel is missing this patch: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8e6e72aeceaa So another red herring I'm afraid :( Looking at the Service Response Time (SRT, time between request & response) stat in wireshark we see this: SMB2 SRT Statistics: Filter: smb2.cmd Index Commands Calls Min SRT Max SRT Avg SRT Sum SRT 5 Create 14759 0.000387 0.035268 0.001288 19.003702 6 Close 14713 0.000253 0.029454 0.000629 9.260241 14 Find 27468 0.000237 0.035798 0.000678 18.634827 16 GetInfo 983 0.000276 0.020349 0.000553 0.543487 The way SMB works, you call Create() to open the directory, then call Find() repeatedly on the handle you get to list its content, then Close(). In your trace each Create operation takes about 0.0013s on average, which I can't really say if it's excessive but its taking a lot of the capture length when you sum them all (19s out of the minute-long trace you sent). You should look on the server side maybe, in the logs (try to raise "log level" in your smb.conf). I'm less knowledgeable on samba perf tweaks though so hopefully someone else will shine some light. Cheers, -- Aurélien Aptel / SUSE Labs Samba Team GPG: 1839 CB5F 9F5B FB9B AA97 8C99 03C8 A49B 521B D5D3 SUSE Linux GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
Saurabh Nanda
2019-Feb-22 06:05 UTC
[Samba] 32 seconds vs 72 minutes -- expected performance difference?
> > I've looked at the trace but didn't find anything out of the > ordinary. The "STATUS_NO_MORE_FILES" is erroneously taken into account > in the stats because your kernel is missing this patch: >Thank you for looking into this, Aurélien. Unfortunately, I've already configured NFS and it is working slightly better than Samba, for now. It takes anywhere from 30 - 40 minutes for the same operation, as against 60-70 minutes that Samba takes. I'm guessing traversing lots of files and directories is just not what network filesystems are meant for. -- Saurabh.