samba - Apr 2021 - Samba 4.7.6-ubuntu taking 100% CPU

Il 02/04/2021 23:04, Jeremy Allison ha scritto:
> On Fri, Apr 02, 2021 at 06:56:47PM +0200, Olaf Marzocchi via samba wrote:
>> Dear all,
>> I already sent this email but I wrongly added attachments, resulting 
>> in the first email being rejected. Sorry if you read the followup with 
>> ZFS properties.
>> I'm sending it again using external storage where needed.
>>
>> I recently incurred in an issue with Samba 4.7.6-ubuntu (Ubuntu 
>> 18.04.05 LTS, with latest updates).
>> I'm not sure what caused the issue, since at the same time I
performed
>> updates to Ubuntu (apt upgrade) and I separately updated the zfs 
>> kernel module to 2.0.4 from whatever version available mid last year, 
>> also some time passed before I noticed the present issue.
>>
>> The issue manifests itself as following: when I try to open some files 
>> (not all of them) from any SMB share (I tried two different Windows 10 
>> machines and one macOS 10.14), the smb3 process takes 100% CPU, it 
>> doesn't seem to retrieve the file, and I have to kill it
"-9", the
>> hard way. I found that, sometimes, after I kill smbd3 Windows gets the 
>> file I tried to open, but it's not a way that always works:
sometimes
>> it's corrupted (cut too short).
>> I tested locally from ssh and the same files have no issues 
>> whatsoever, it's not a hardware issue. I can read and copy them 
>> without problems.
> 
> Use strace -p <spinning process id> to get system call
> traces to see what is going on.
Thanks for the reply!
I tried just now:

sendfile(36, 30, [360448], 262144)      = -1 EAGAIN (Resource 
temporarily unavailable)

I disabled sendfile and the issue is solved.
I'm very thankful for the help!

I tried transferring some big files and I still saturate gigabit, while 
smaller files are disk-limited anyway. It doesn't look like I actually 
need sendfile active.

I'll investigate the sendfile issue with recent ZFS and if needed I'll 
report it on their mailing list.

Cheers!
Olaf

On Fri, Apr 02, 2021 at 11:16:10PM +0200, Olaf Marzocchi via samba
wrote:
>
>
>Il 02/04/2021 23:04, Jeremy Allison ha scritto:
>>On Fri, Apr 02, 2021 at 06:56:47PM +0200, Olaf Marzocchi via samba
wrote:
>>>Dear all,
>>>I already sent this email but I wrongly added attachments, 
>>>resulting in the first email being rejected. Sorry if you read the 
>>>followup with ZFS properties.
>>>I'm sending it again using external storage where needed.
>>>
>>>I recently incurred in an issue with Samba 4.7.6-ubuntu (Ubuntu 
>>>18.04.05 LTS, with latest updates).
>>>I'm not sure what caused the issue, since at the same time I 
>>>performed updates to Ubuntu (apt upgrade) and I separately updated 
>>>the zfs kernel module to 2.0.4 from whatever version available mid 
>>>last year, also some time passed before I noticed the present 
>>>issue.
>>>
>>>The issue manifests itself as following: when I try to open some 
>>>files (not all of them) from any SMB share (I tried two different 
>>>Windows 10 machines and one macOS 10.14), the smb3 process takes 
>>>100% CPU, it doesn't seem to retrieve the file, and I have to
kill
>>>it "-9", the hard way. I found that, sometimes, after I
kill smbd3
>>>Windows gets the file I tried to open, but it's not a way that 
>>>always works: sometimes it's corrupted (cut too short).
>>>I tested locally from ssh and the same files have no issues 
>>>whatsoever, it's not a hardware issue. I can read and copy them 
>>>without problems.
>>
>>Use strace -p <spinning process id> to get system call
>>traces to see what is going on.
>
>Thanks for the reply!
>I tried just now:
>
>sendfile(36, 30, [360448], 262144)      = -1 EAGAIN (Resource 
>temporarily unavailable)
>
>I disabled sendfile and the issue is solved.
>I'm very thankful for the help!
>
>I tried transferring some big files and I still saturate gigabit, 
>while smaller files are disk-limited anyway. It doesn't look like I 
>actually need sendfile active.
>
>I'll investigate the sendfile issue with recent ZFS and if needed
I'll
>report it on their mailing list.
>
Hmmm. That may be an old bug (Samba 4.7.6 is quite old
now) when using sendfile() with non-blocking sockets.

It might not be sendfile, but Samba here. I'd advise
trying a later Samba version.