[Samba] Also seeing high winbindd CPU usage

David Eisner
Tue, 05 Aug 2008 21:01:57 -0000

I think somebody had a similar problem (also on Solaris), but that
thread seemed to die.

I've compiled (with Sun Studio cc) and installed samba-3.2.1 on a
Solaris 10 x64 box, which is a member of a (Windows Server 2003
controlled) domain.  I previously had samba 3.0.28a running on the
same machine without any problems.  Now winbindd is eating up all of
the CPU (on the CPU it's assigned to).

I configured samba like so:

  ./configure --prefix=/usr/local/samba-3.2.1  --with-ads --with-pam
--with-acl-support \
     --with-shared-modules=idmap_ad  --enable-socket-wrapper
--enable-nss-wrapper --with-krb5=/usr/local/heimdal/heimdal-1.1

I successfully built it, and it passed 'make test', too.  Then I
installed it (into a new location).  Rather than copy over the
secrets.tdb, I repeated the net ads join, which succeeded.  'net ads
testjoin' tells me the 'Join is OK.'

However, when I tried wbinfo -u, it just hung.  Then I noticed
winbindd was eating up CPU.  Actually, it appears to have PANIC'd and
crashed just now.  Here's the winbindd log:

[2008/08/05 16:12:09,  0] winbindd/winbindd.c:(1126)
  winbindd version 3.2.1 started.
  Copyright Andrew Tridgell and the Samba Team 1992-2008
[2008/08/05 16:20:29,  0] winbindd/winbindd.c:(1126)
  winbindd version 3.2.1 started.
  Copyright Andrew Tridgell and the Samba Team 1992-2008
[2008/08/05 16:20:29,  0] winbindd/winbindd_cache.c:(2353)
  initialize_winbindd_cache: clearing cache and re-creating with
version number 1
[2008/08/05 16:21:24,  0] winbindd/winbindd.c:(632)
  request_len_recv: Invalid request size received: 2088 (expected 2096)
[2008/08/05 16:35:42,  0] winbindd/winbindd.c:(632)
  request_len_recv: Invalid request size received: 2088 (expected 2096)
[2008/08/05 16:35:42,  0] winbindd/winbindd.c:(632)
  request_len_recv: Invalid request size received: 2088 (expected 2096)
[2008/08/05 16:35:42,  0] winbindd/winbindd.c:(632)
  request_len_recv: Invalid request size received: 2088 (expected 2096)
[2008/08/05 16:37:42,  0] winbindd/winbindd_dual.c:(272)
  PANIC: assert failed at winbindd/winbindd_dual.c(272):
cache_retrieve_response(state->child_pid, state->response)
[2008/08/05 16:37:42,  0] winbindd/winbindd_util.c:(275)
  Got invalid trustdom response
[2008/08/05 16:40:02,  0] winbindd/winbindd.c:(632)
  request_len_recv: Invalid request size received: 2088 (expected 2096)
[2008/08/05 16:42:42,  0] lib/fault.c:(40)
  ==============================================================[2008/08/05
16:42:42,  0] lib/fault.c:(41)
  INTERNAL ERROR: Signal 6 in pid 20965 (3.2.1)
  Please read the Trouble-Shooting section of the Samba3-HOWTO
[2008/08/05 16:42:42,  0] lib/fault.c:(43)

  From: http://www.samba.org/samba/docs/Samba3-HOWTO.pdf
[2008/08/05 16:42:42,  0] lib/fault.c:(44)
  ==============================================================[2008/08/05
16:42:42,  0] lib/util.c:(1666)
  PANIC (pid 20965): internal error
[2008/08/05 16:42:42,  0] lib/util.c:(1820)
  unable to produce a stack trace on this platform
[2008/08/05 16:42:42,  0] lib/fault.c:(201)
  dumping core in /usr/local/samba-3.2.1/var/cores/winbindd

I restarted winbindd just now, same story.  Here's the truss output.
It looks like this, repeated:

...
time()                                          = 1217969117
pollsys(0x08047710, 4, 0x080477F0, 0x00000000)  = 1
write(16, " 0\b\0\013\0\0\0\0\0\0\0".., 2096)   = 2096
time()                                          = 1217969117
brk(0x0DA3EC28)                                 = 0
brk(0x0DA40C28)                                 = 0
pollsys(0x08047710, 4, 0x080477F0, 0x00000000)  = 1
read(16, "02\0\0\0", 4)                         = 4
fcntl(12, F_SETLKW64, 0x080475E4)               = 0
fcntl(12, F_SETLKW64, 0x08047664)               = 0
fcntl(12, F_SETLKW64, 0x080475E4)               = 0
fcntl(12, F_SETLKW64, 0x08047664)               = 0
fcntl(12, F_SETLKW64, 0x080475F4)               = 0
fcntl(12, F_SETLK64, 0x08047644)                = 0
fcntl(12, F_SETLK64, 0x08047644)                = 0
fcntl(12, F_SETLKW64, 0x08047594)               = 0
fcntl(12, F_SETLKW64, 0x080475D4)               = 0
fcntl(12, F_SETLKW64, 0x08047674)               = 0
fcntl(12, F_SETLKW64, 0x080475F4)               = 0
fcntl(12, F_SETLK64, 0x08047644)                = 0
fcntl(12, F_SETLK64, 0x08047644)                = 0
fcntl(12, F_SETLKW64, 0x08047594)               = 0
fcntl(12, F_SETLKW64, 0x080475D4)               = 0
fcntl(12, F_SETLKW64, 0x08047674)               = 0
fcntl(12, F_SETLKW64, 0x080475E4)               = 0
fcntl(12, F_SETLKW64, 0x08047664)               = 0
time()                                          = 1217969117
pollsys(0x08047710, 4, 0x080477F0, 0x00000000)  = 1
write(16, " 0\b\0\013\0\0\0\0\0\0\0".., 2096)   = 2096
time()                                          = 1217969117
pollsys(0x08047710, 4, 0x080477F0, 0x00000000)  = 1
read(16, "02\0\0\0", 4)                         = 4
fcntl(12, F_SETLKW64, 0x080475E4)               = 0
brk(0x0DA40C28)                                 = 0
brk(0x0DA42C28)                                 = 0
fcntl(12, F_SETLKW64, 0x08047664)               = 0
fcntl(12, F_SETLKW64, 0x080475E4)               = 0
fcntl(12, F_SETLKW64, 0x08047664)               = 0
fcntl(12, F_SETLKW64, 0x080475F4)               = 0
...

Though it's spinning its wheels, so far this is the entirety of the
winbindd log after restarting it just now:

[2008/08/05 16:44:59,  0] winbindd/winbindd.c:(1126)
  winbindd version 3.2.1 started.
  Copyright Andrew Tridgell and the Samba Team 1992-2008
[2008/08/05 16:44:59,  0] winbindd/winbindd_cache.c:(2353)
  initialize_winbindd_cache: clearing cache and re-creating with
version number 1

Finally, here's the setup:

[global]
        unix charset = LOCALE
        workgroup = OURDOMAIN
        realm = OURDOMAIN.UMD.EDU
        server string = OUR Samba Server
        security = ADS
        allow trusted domains = No
        use kerberos keytab = Yes
        log level = 1
        syslog = 0
        log file = /usr/local/samba-current/var/log.%m
        printcap name = CUPS
        dns proxy = No
        idmap domains = OURDOMAIN
        idmap alloc backend = tdb
        template homedir = /home/windows/%D/%U
        template shell = /bin/bash
        winbind cache time = 0
        winbind enum users = Yes
        winbind enum groups = Yes
        winbind use default domain = Yes
        winbind nss info = rfc2307
        idmap alloc config:range = 5000 - 9999
        idmap config OURDOMAIN:range = 10000 - 300000000
        idmap config OURDOMAIN:schema_mode = rfc2307
        idmap config OURDOMAIN:default = yes
        idmap config OURDOMAIN:backend = ad
        hosts allow = XXX.Y., ZZZ.X., 127.
        cups options = raw

[homes]
        comment = Home Directories
        valid users = %D\%U
        read only = No
        browseable = No

[printers]
        comment = All Printers
        path = /var/spool/samba
        printable = Yes
        browseable = No

[voltest]
        comment = DELOS www
        path = /path/to/share/dir
        admin users = OURDOMAIN\administrator
        read only = No
        inherit acls = Yes
        map acl inherit = Yes
        store dos attributes = Yes


Is this potentially a known bug?

-David

--
David Eisner http://cradle.brokenglass.com