samba - Feb 2008 - Samba 3.0.25b on centos 5.1 a lot of signal 11 very unstable!!!

Hi.
    We add a new virtual machine with vmware server 1.0.4, centos 5.x
client, we update the server with yum and receive samba
3.0.25b-1.el5_1.4, we have a PDC running Gentoo with Samba
3.0.24+ldap, all the windows clients and other Linux boxes(.3.0.10 +
3.0.23 + 3.0.24) are working very good and stable.

    The only problem is this new server, i read about some changes
with samba 3.0.25b and oldest version, since we add this server to the
domain we had been having problems, we enable the roaming profile to
our windows clients, but some times the server doesn't update the user
profile, on other situations we lost the profile, example firefox
settings, or if  the user update some excel file next day appear with
no changes.

   We have almost 3GB of core dumps since we setup samba inside
winbind folder, look this is my smb.conf file:

[global]
        workgroup = mydomain
        server string = mydomain VM DBA Server
        interfaces = eth0, lo
        bind interfaces only = Yes
        security = DOMAIN
        username map = /etc/samba/smbusers
        log level = 10
        log file = /var/log/samba/%m.%U.log
        max log size = 150
        smb ports = 139 445
        name resolve order = wins hosts lmhosts bcast
        socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192
        printcap name = CUPS
        preferred master = No
        local master = No
        domain master = No
        wins server = 192.168.1.7
        ldap admin dn = cn=root,dc=mydomain,dc=com
        ldap group suffix = ou=Groups
        ldap idmap suffix = ou=Idmap
        ldap machine suffix = ou=Computers
        ldap suffix = dc=oakwest,dc=com
        ldap user suffix = ou=Users
        utmp = Yes
        remote announce = 192.168.1.255
        idmap backend = "ldap:ldap://192.168.1.7/
ldap://192.168.1.150"
        idmap uid = 10000-20000
        idmap gid = 10000-20000
        hosts allow = 127.0.0.1, 192.168.1.0/24, 192.168.2.0/24, 192.168.10.0/24
        hosts deny = 0.0.0.0
        printing = cups
        print command         lpq command = %p
        lprm command 
[homes]
        comment = Home Directories Oakwest
        valid users = mydomain\%U
        read only = No
        create mask = 0664
        directory mask = 0775
        browseable = No

[profiles]
        path = /home/samba/profiles
        valid users = mydomain\%U, "@Domain Admins"
        force user = %U
        read only = No
        create mask = 0600
        directory mask = 0700
        profile acls = Yes
        browseable = No
        csc policy = disable

Look now, this winbind logs:

-----------------------------
 wb-mydomain.log

  rpc_pipe_bind: Remote machine PDC-SRV pipe \lsarpc fnum 0x7628 bind
request returned ok.
[2008/02/26 21:58:43, 1]
rpc_client/cli_pipe.c:cli_pipe_validate_current_pdu(625)
  cli_pipe_validate_current_pdu: RPC fault code
DCERPC_FAULT_OP_RNG_ERROR received from remote machine PDC-SRV pipe
\lsarpc fnum 0x7628!
[2008/02/26 21:58:43, 3] rpc_client/cli_pipe.c:rpc_pipe_bind(2081)
  rpc_pipe_bind: Remote machine PDC-SRV pipe \lsarpc fnum 0x7629 bind
request returned ok.
[2008/02/26 21:58:43, 1]
rpc_client/cli_pipe.c:cli_pipe_validate_current_pdu(625)
  cli_pipe_validate_current_pdu: RPC fault code
DCERPC_FAULT_OP_RNG_ERROR received from remote machine PDC-SRV pipe
\lsarpc fnum 0x7629!
[2008/02/26 21:58:43, 3] rpc_client/cli_pipe.c:rpc_pipe_bind(2081)
  rpc_pipe_bind: Remote machine PDC-SRV pipe \lsarpc fnum 0x762a bind
request returned ok.
[2008/02/26 21:58:43, 1]
rpc_client/cli_pipe.c:cli_pipe_validate_current_pdu(625)
  cli_pipe_validate_current_pdu: RPC fault code
DCERPC_FAULT_OP_RNG_ERROR received from remote machine PDC-SRV pipe
\lsarpc fnum 0x762a!
[2008/02/26 21:58:43, 3] rpc_client/cli_pipe.c:rpc_pipe_bind(2081)
  rpc_pipe_bind: Remote machine PDC-SRV pipe \lsarpc fnum 0x762b bind
request returned ok.
[2008/02/26 21:58:43, 1]
rpc_client/cli_pipe.c:cli_pipe_validate_current_pdu(625)
  cli_pipe_validate_current_pdu: RPC fault code
DCERPC_FAULT_OP_RNG_ERROR received from remote machine PDC-SRV pipe
\lsarpc fnum 0x762b!
[2008/02/26 21:58:43, 3] nsswitch/winbindd_rpc.c:trusted_domains(909)
  rpc: trusted_domains
[2008/02/26 21:58:43, 3] rpc_client/cli_pipe.c:rpc_pipe_bind(2081)
  rpc_pipe_bind: Remote machine PDC-SRV pipe \lsarpc fnum 0x762c bind
request returned ok.
[2008/02/26 21:58:43, 1]
rpc_client/cli_pipe.c:cli_pipe_validate_current_pdu(625)
  cli_pipe_validate_current_pdu: RPC fault code
DCERPC_FAULT_OP_RNG_ERROR received from remote machine PDC-SRV pipe
\lsarpc fnum 0x762c!
[2008/02/26 21:58:43, 3] rpc_client/cli_pipe.c:rpc_pipe_bind(2081)
  rpc_pipe_bind: Remote machine PDC-SRV pipe \lsarpc fnum 0x762d bind
request returned ok.
[2008/02/26 21:58:43, 1]
rpc_client/cli_pipe.c:cli_pipe_validate_current_pdu(625)
  cli_pipe_validate_current_pdu: RPC fault code
DCERPC_FAULT_OP_RNG_ERROR received from remote machine PDC-SRV pipe
\lsarpc fnum 0x762d!
-------------------------------------
 winbindd-dc-connect.log

[2008/02/26 21:38:46, 3] nsswitch/winbindd_cm.c:connection_ok(1447)
  connection_ok: Connection to PDC-SRV for domain OAKWEST has died or
was never started (fd == -1)
[2008/02/26 21:38:46, 1] libsmb/clientgen.c:cli_rpc_pipe_close(387)
  cli_rpc_pipe_close: cli_close failed on pipe \samr, fnum 0x7405 to
machine PDC-SRV.  Error was SUCCESS - 0
[2008/02/26 21:38:46, 1] libsmb/clientgen.c:cli_rpc_pipe_close(387)
  cli_rpc_pipe_close: cli_close failed on pipe \NETLOGON, fnum 0x7401
to machine PDC-SRV.  Error was SUCCESS - 0
[2008/02/26 21:38:46, 3] libsmb/namequery.c:resolve_wins(863)
  resolve_wins: Attempting wins lookup for name PDC-SRV<0x20>
[2008/02/26 21:38:46, 3] libsmb/namequery.c:resolve_wins(902)
  resolve_wins: using WINS server 192.168.1.7 and tag '*'
[2008/02/26 21:38:46, 2] libsmb/namequery.c:name_query(604)
  Got a positive name query response from 192.168.1.7 ( 192.168.1.7 )
[2008/02/26 21:38:46, 3] nsswitch/winbindd_cm.c:cm_get_ipc_userpass(507)
  cm_get_ipc_userpass: No auth-user defined
[2008/02/26 21:38:46, 3] libsmb/cliconnect.c:cli_session_setup_spnego(789)
  Doing spnego session setup (blob length=58)
[2008/02/26 21:38:46, 3] libsmb/cliconnect.c:cli_session_setup_spnego(814)
  got OID=1 3 6 1 4 1 311 2 2 10
[2008/02/26 21:38:46, 3] libsmb/cliconnect.c:cli_session_setup_spnego(822)
  got principal=NONE
[2008/02/26 21:38:46, 3] libsmb/ntlmssp.c:ntlmssp_client_challenge(1018)
  Got challenge flags:
[2008/02/26 21:38:46, 3] libsmb/ntlmssp.c:debug_ntlmssp_flags(63)
  Got NTLMSSP neg_flags=0x60898215
[2008/02/26 21:38:46, 3] libsmb/ntlmssp.c:ntlmssp_client_challenge(1040)
  NTLMSSP: Set final flags:
[2008/02/26 21:38:46, 3] libsmb/ntlmssp.c:debug_ntlmssp_flags(63)
  Got NTLMSSP neg_flags=0x60088215
[2008/02/26 21:38:46, 3] libsmb/ntlmssp_sign.c:ntlmssp_sign_init(338)
  NTLMSSP Sign/Seal - Initialising with flags:
[2008/02/26 21:38:46, 3] libsmb/ntlmssp.c:debug_ntlmssp_flags(63)
  Got NTLMSSP neg_flags=0x60088215
[2008/02/26 21:38:46, 3] rpc_client/cli_pipe.c:rpc_pipe_bind(2081)
  rpc_pipe_bind: Remote machine PDC-SRV pipe \lsarpc fnum 0x74f6 bind
request returned ok.
[2008/02/26 21:38:46, 1]
rpc_client/cli_pipe.c:cli_pipe_validate_current_pdu(625)
  cli_pipe_validate_current_pdu: RPC fault code
DCERPC_FAULT_OP_RNG_ERROR received from remote machine PDC-SRV pipe
\lsarpc fnum 0x74f6!
[2008/02/26 21:38:46, 3] rpc_client/cli_pipe.c:rpc_pipe_bind(2081)
  rpc_pipe_bind: Remote machine PDC-SRV pipe \NETLOGON fnum 0x74f7
bind request returned ok.
[2008/02/26 21:38:46, 3] rpc_client/cli_pipe.c:rpc_pipe_bind(2081)
  rpc_pipe_bind: Remote machine PDC-SRV pipe \NETLOGON fnum 0x74f8
bind request returned ok.
[2008/02/26 21:38:46, 1]
rpc_client/cli_pipe.c:cli_pipe_validate_current_pdu(625)
  cli_pipe_validate_current_pdu: RPC fault code
DCERPC_FAULT_OP_RNG_ERROR received from remote machine PDC-SRV pipe
\NETLOGON fnum 0x74f8!
[2008/02/26 21:38:46, 3] libsmb/namequery.c:get_dc_list(1489)
  get_dc_list: preferred server list: ", *"
[2008/02/26 21:38:46, 3] libsmb/namequery.c:resolve_wins(863)
  resolve_wins: Attempting wins lookup for name MUEBLEX<0x1c>
[2008/02/26 21:38:46, 3] libsmb/namequery.c:resolve_wins(902)
  resolve_wins: using WINS server 192.168.1.7 and tag '*'
[2008/02/26 21:38:46, 3] libsmb/namequery.c:name_query(554)
  Negative name query response, rcode 0x03: The name requested does not exist.
[2008/02/26 21:38:46, 3] libsmb/namequery.c:resolve_lmhosts(966)
  resolve_lmhosts: Attempting lmhosts lookup for name MUEBLEX<0x1c>
[2008/02/26 21:38:46, 3] libsmb/namequery.c:name_resolve_bcast(805)
  name_resolve_bcast: Attempting broadcast lookup for name MUEBLEX<0x1c>

-----------------------------
winbindd-idmap.log

[root@oak-vmdba samba]# vim winbindd-idmap.log
   #19 winbindd [0x80b3f2]
   #20 winbindd [0x7dc368]
   #21 winbindd(main+0x94d) [0x7dcdbd]
   #22 /lib/libc.so.6(__libc_start_main+0xdc) [0x26fdec]
   #23 winbindd [0x7db061]
[2008/02/26 21:50:43, 0] lib/fault.c:dump_core(181)
  dumping core in /var/log/samba/cores/winbindd
[2008/02/26 21:50:43, 3] nsswitch/winbindd_async.c:winbindd_dual_sid2gid(558)
  [ 2097]: sid to gid S-1-5-32-546
[2008/02/26 21:50:43, 1] nsswitch/idmap.c:idmap_init(343)
  Initializing idmap domains
[2008/02/26 21:50:43, 0] passdb/secrets.c:fetch_ldap_pw(769)
  fetch_ldap_pw: neither ldap secret retrieved!
[2008/02/26 21:50:43, 2] nsswitch/idmap_ldap.c:get_credentials(101)
  get_credentials: Failed to lookup ldap bind creds.  Using anonymous
connection.
[2008/02/26 21:50:43, 0] lib/fault.c:fault_report(41)
  ==============================================================[2008/02/26
21:50:43, 0] lib/fault.c:fault_report(42)
  INTERNAL ERROR: Signal 11 in pid 2441 (3.0.25b-1.el5_1.4)
  Please read the Trouble-Shooting section of the Samba3-HOWTO
[2008/02/26 21:50:43, 0] lib/fault.c:fault_report(44)

  From: http://www.samba.org/samba/docs/Samba3-HOWTO.pdf
[2008/02/26 21:50:43, 0] lib/fault.c:fault_report(45)
  ==============================================================[2008/02/26
21:50:43, 0] lib/util.c:smb_panic(1654)
  PANIC (pid 2441): internal error
[2008/02/26 21:50:43, 0] lib/util.c:log_stack_trace(1758)
  BACKTRACE: 24 stack frames:
   #0 winbindd(log_stack_trace+0x2d) [0x87486d]
   #1 winbindd(smb_panic+0x5d) [0x87499d]
   #2 winbindd [0x85f38a]
   #3 [0xd62420]
   #4 /lib/libc.so.6(strlen+0x33) [0x2c6fd3]
   #5 /lib/libc.so.6(__strdup+0x25) [0x2c6d15]
   #6 winbindd(smbldap_set_creds+0x92) [0x985cb2]
   #7 winbindd [0x9ab630]
   #8 winbindd [0x9abd78]
   #9 winbindd(idmap_init+0x401) [0x9a5a31]
   #10 winbindd(idmap_sids_to_unixids+0x25) [0x9a81c5]
   #11 winbindd(idmap_sid_to_gid+0x65) [0x9aad05]
   #12 winbindd(winbindd_dual_sid2gid+0x17e) [0x80e9ae]
   #13 winbindd [0x80a152]
   #14 winbindd [0x80cad5]
   #15 winbindd(winbindd_sid2gid_async+0xf7) [0x80f9f7]
   #16 winbindd [0x7f83bd]
   #17 winbindd [0x80c195]
   #18 winbindd [0x80cc97]
   #19 winbindd [0x80b3f2]
   #20 winbindd [0x7dc368]
   #21 winbindd(main+0x94d) [0x7dcdbd]
   #22 /lib/libc.so.6(__libc_start_main+0xdc) [0x26fdec]
   #23 winbindd [0x7db061]
[2008/02/26 21:50:43, 0] lib/fault.c:dump_core(181)
  dumping core in /var/log/samba/cores/winbindd
------------------------------
Now this is what i receive in my mails:
auth/auth.c:check_ntlm_password(221) check_ntlm_password: Checking
password for unmapped user [mydomain]\[userx]@[userx] with the new
password interface : 1 Time(s)
auth/auth.c:check_ntlm_password(221) check_ntlm_password: Checking
password for unmapped user []\[]@[userx] with the new password
interface : 1 Time(s)
auth/auth.c:check_ntlm_password(224) check_ntlm_password: mapped user
is: [mydomain]\[]@[userx] : 1 Time(s)
auth/auth.c:check_ntlm_password(224) check_ntlm_password: mapped user
is: [mydomain]\[userx]@[userx] : 1 Time(s)
auth/auth.c:check_ntlm_password(270) check_ntlm_password: guest
authentication for user [] succeeded : 1 Time(s)
auth/auth.c:check_ntlm_password(270) check_ntlm_password: winbind
authentication for user [userx] succeeded : 1 Time(s)
auth/auth.c:check_ntlm_password(309) check_ntlm_password:
authentication for user [userx] -> [userx] -> [userx] succeeded : 1
Time(s)
auth/auth_util.c:create_builtin_administrators(792)
create_builtin_administrators: Failed to create Administrators : 2
Time(s)
auth/auth_util.c:create_builtin_users(758) create_builtin_users:
Failed to create Users : 2 Time(s)
auth/auth_util.c:create_local_nt_token(914) create_local_nt_token:
Failed to create BUILTIN\Administrators group! : 2 Time(s)
auth/auth_util.c:create_local_nt_token(941) create_local_nt_token:
Failed to create BUILTIN\Users group! : 2 Time(s)
groupdb/mapping.c:pdb_create_builtin_alias(722)
pdb_create_builtin_alias: Could not get a gid out of winbind : 4
Time(s)
lib/access.c:check_access(312) check_access: no hostnames in host
allow/deny list. : 4 Time(s)
lib/access.c:check_access(323) Allowed connection from (192.168.1.52)
: 3 Time(s)
lib/fault.c:dump_core(181) dumping core in
/var/log/samba/cores/winbindd : 17 Time(s)
lib/fault.c:fault_report(42) INTERNAL ERROR: Signal 11 in pid 9008
(3.0.25b-1.el5_1.4) Please read the Trouble-Shooting section of the
Samba3-HOWTO : 1 Time(s)
lib/fault.c:fault_report(42) INTERNAL ERROR: Signal 11 in pid 9009
(3.0.25b-1.el5_1.4) Please read the Trouble-Shooting section of the
Samba3-HOWTO : 1 Time(s)
.
.
.
lib/fault.c:fault_report(42) INTERNAL ERROR: Signal 11 in pid 9010
(3.0.25b-1.el5_1.4) Please read the Trouble-Shooting section of the
Samba3-HOWTO : 1 Time(s)
lib/fault.c:fault_report(42) INTERNAL ERROR: Signal 11 in pid 9011
(3.0.25b-1.el5_1.4) Please read the Trouble-Shooting section of the
Samba3-HOWTO : 1 Time(s)
lib/fault.c:fault_report(42) INTERNAL ERROR: Signal 11 in pid 9025
(3.0.25b-1.el5_1.4) Please read the Trouble-Shooting section of the
Samba3-HOWTO : 1 Time(s)
lib/util.c:log_stack_trace(1758) BACKTRACE: 21 stack frames: #0
winbindd(log_stack_trace+0x2d) [0x6ad86d] #1 winbindd(smb_panic+0x5d)
[0x6ad99d] #2 winbindd [0x69838a]
#3 [0x42c420] #4 /lib/libc.so.6(strlen+0x33) [0x276fd3] #5
/lib/libc.so.6(__strdup+0x25) [0x276d15] #6
winbindd(smbldap_set_creds+0x92) [0x7becb2] #7 winbindd [0x7e46
30] #8 winbindd [0x7e4d78] #9 winbindd(idmap_init+0x401) [0x7dea31]
#10 winbindd [0x7e0c17] #11 winbindd(idmap_allocate_gid+0x28)
[0x7e10a8] #12 winbindd(winbindd_dua
l_allocate_gid+0x24) [0x6301b4] #13 winbindd [0x643152] #14
winbindd(sendto_child+0x46) [0x644996] #15
winbindd(winbindd_allocate_gid+0x97) [0x630277] #16 winbindd [0x6
14ee0] #17 winbindd [0x615368] #18 winbindd(main+0x94d) [0x615dbd] #19
/lib/libc.so.6(__libc_start_main+0xdc) [0x21fdec] #20 winbindd
[0x614061] : 4 Time(s)
lib/util.c:log_stack_trace(1758) BACKTRACE: 24 stack frames: #0
winbindd(log_stack_trace+0x2d) [0x6ad86d] #1 winbindd(smb_panic+0x5d)
[0x6ad99d] #2 winbindd [0x69838a]
#3 [0x42c420] #4 /lib/libc.so.6(strlen+0x33) [0x276fd3] #5
/lib/libc.so.6(__strdup+0x25) [0x276d15] #6
winbindd(smbldap_set_creds+0x92) [0x7becb2] #7 winbindd [0x7e46
30] #8 winbindd [0x7e4d78] #9 winbindd(idmap_init+0x401) [0x7dea31]
#10 winbindd(idmap_sids_to_unixids+0x25) [0x7e11c5] #11
winbindd(idmap_sid_to_gid+0x65) [0x7e3d05]
#12 winbindd(winbindd_dual_sid2gid+0x17e) [0x6479ae] #13 winbindd
[0x643152] #14 winbindd [0x645ad5] #15
winbindd(winbindd_sid2gid_async+0xf7) [0x6489f7] #16 winbindd
[0x6313bd] #17 winbindd [0x645195] #18 winbindd [0x645c97] #19
winbindd [0x6443f2] #20 winbindd [0x615368] #21 winbindd(main+0x94d)
[0x615dbd] #22 /lib/libc.so.6(__
libc_start_main+0xdc) [0x21fdec] #23 winbindd [0x614061] : 13 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9008): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9009): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9010): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9011): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9012): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9013): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9014): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9015): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9016): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9017): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9018): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9020): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9021): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9022): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9023): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9024): internal error : 1 Time(s)
lib/util.c:smb_panic(1654) PANIC (pid 9025): internal error : 1 Time(s)
lib/util_seaccess.c:se_access_check(250) : 5 Time(s)
------------------------------

Hope this info give some point to start debugging this problem, does
someone see what is causing the problem?
Thanks all for your time, if u need more info please let me know, thanks!!!

-- 
LIving the dream...

On Tue, 26 Feb 2008 22:12:53 -0800, Alberto Moreno wrote:

> 
>     The only problem is this new server, i read about some changes
> with samba 3.0.25b and oldest version, since we add this server to the
> domain we had been having problems, we enable the roaming profile to our
> windows clients, but some times the server doesn't update the user
> profile, on other situations we lost the profile, example firefox
> settings, or if  the user update some excel file next day appear with no
> changes.
Roaming profiles are just problematic, even on native Windows servers.  I 
have seen several roaming profiles implode on WinXP client boxes.  I've 
also seen bad behavior with Win2k client/server setups as well.  Symptoms 
include the client creating new profiles, ignoring existing profiles, or 
dialogs indicating profile corruption.
> 
>    We have almost 3GB of core dumps since we setup samba inside
> winbind folder, look this is my smb.conf file:
[ lots of stuff snipped out ]

> > lib/fault.c:dump_core(181) dumping core in /var/log/samba/cores/
winbindd

[ even more stuff snipped out]
> [0x645c97] #19 winbindd [0x6443f2] #20 winbindd [0x615368] #21
> winbindd(main+0x94d) [0x615dbd] #22 /lib/libc.so.6(__
> libc_start_main+0xdc) [0x21fdec] #23 winbindd [0x614061] : 13 Time(s)
> ------------------------------
> 
> Hope this info give some point to start debugging this problem, does
> someone see what is causing the problem? Thanks all for your time, if u
> need more info please let me know, thanks!!!
I'm no Samba or programming expert, but that last line looks like a libc 
segfault.  Sig 11 errors a long time ago used to implicate RAM issues, 
usually due to bad contacts or faulty RAM chips.    This may sound silly 
but try powering down the machine, unseating and reseating all of your 
RAM.  If it continues, try reducing the RAM and see if the issue goes 
away (due to a bad RAM stick).  Just my .02 cents.