[Samba] CTDB problems

Alex Crow
Wed, 19 Apr 2017 16:29:29 -0000

On 19/04/17 17:06, David Disseldorp via samba wrote:
> Hi Alex,
>
> On Wed, 19 Apr 2017 12:55:45 +0100, Alex Crow via samba wrote:
>
>> ----- Process in D state, printing kernel stack only
>> [<ffffffffa05b253d>] __fuse_request_send+0x13d/0x2c0 [fuse]
>> [<ffffffffa05b26d2>] fuse_request_send+0x12/0x20 [fuse]
>> [<ffffffffa05bb66c>] fuse_setlk+0x16c/0x1a0 [fuse]
>> [<ffffffffa05bc40f>] fuse_file_lock+0x5f/0x210 [fuse]
>> [<ffffffff81253a73>] vfs_lock_file+0x23/0x40
>> [<ffffffff81255069>] fcntl_setlk+0x159/0x310
>> [<ffffffff81210fe1>] SyS_fcntl+0x3e1/0x610
>> [<ffffffff816968c9>] system_call_fastpath+0x16/0x1b
>> [<ffffffffffffffff>] 0xffffffffffffffff
> Are your tdb database files placed on a FUSE filesystem mount? If so
> I'd suggest testing POSIX locking on your filesystem using a tool like
> https://wiki.samba.org/index.php/Ping_pong .
>
>> ----- Stack trace for PID=8990 -----
>> 2017/04/19 10:40:31.294250 [ 7423]: /etc/ctdb/debug_locks.sh: line 73:
>> gstack: command not found
> This script attempts to dump the stack trace of the blocked process,
> but can't as gstack isn't installed - it should be available in the
> gdb package.
>
> @Martin: would the attached (untested) patch make sense?
>
> Cheers, David
Hi David,

I've install gstack now, we should have more debugging should it happen
again.

And no, the tdb files are on XFS locally on each machine. I think this
is an actual client smbd process in the above - one that got stuck and
triggered a reboot as we have this in sysctl.conf:

# Reboot 5 seconds after panic
kernel.panic = 5

# Panic if a hung task was found
kernel.hung_task_panic = 1

# Setup timeout for hung task to 300 seconds
kernel.hung_task_timeout_secs = 1200

PS I didn't see any attachment.

Many thanks

Alex