samba - Mar 2011 - Clustered Samba: smbd process deadlocks when doing an /etc/init.d/samba stop on gentoo

Hi All,

i have a problem with the SMBD processes on my clustered Samba setup.
The setup contains:
- 2  gentoo linux nodes (kernel 2.6.33.2)
- ctdb 1.0.114_p1
- samba 3.5.8
- glusterfs 2.0.8

sometimes i want to do some maitenance on one of the nodes, then i do
the following:
- /etc/init.d/samba stop (to stop samba, and move all the users to the
other nodes)
- /etc/iniit.d/ctdb stop (to move the floating ip to the other host)

After the first command, i see a big decline in the amount of smbd
processes; from ~45 to ~15.
but after 45 seconds the number stays the same, and after ~60 seconds
the stopping process fails;
leaving the remaining 15 processes either in S or in D state is `ps aux`

i cannot kill -9 those processes, they either switch to deadlock or
just stay deadlocked.
The only resolution i have is to do a very ugly `reboot -nfi`

so far i can provide the output of testparm, and also the relevant
logs of loglevel 1, but i have the feeling that might not be enough?

any help is very much appreciated!
Erik

On Wed, Mar 23, 2011 at 02:56:39PM +0100, erik bergsma
wrote:
> i cannot kill -9 those processes, they either switch to deadlock or
> just stay deadlocked.
If you can't kill -9 processes for an extended period of
time, it's a kernel or file system bug. IIRC gluster is
fuse-based, maybe the processes hang somewhere in a request
to gluster.

Volker

-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 G?ttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG G?ttingen, HRB 2816, GF: Dr. Johannes Loxen