samba - Apr 2016 - Nonfunctional linux/CIFS mounts after update (ADS / windows DC auth)

I have been running in loglevel 10 and looking at the logs, but as I said
in my initial post the credentials function fine with nautilus. Appended is
a connection log snippet where samba walks through a cascade of
authentication methods and finally fails.

Earlier in the log, samba successfully determines which domain controller
to talk to and pulls its information. However, just before the failure
there is a line which I feel is the root cause of the problem.
>   domain_client_validate: Domain password server not available.
>
Taken at face value (not the best idea as it seems to be walking through a
set of authentication methods), it indicates that somehow the information
about the domain controller doing the authentication has been lost.

The password is good: It works on Windows and it works with nautilus. Samba
says the password is bad when using cifs or smbclient. What gives?

For the situational awareness of others affected by this issue, I hoped it
was sssd or the associated name service switch libraries, so I built and
installed rpms for an older version of sssd. No dice.

Patrick

[2016/04/26 17:06:13.912141,  3, pid=11263, effective(0, 0), real(0,
0)]
> ../auth/ntlmssp/ntlmssp_util.c:69(debug_ntlmssp_flags)
>   Got NTLMSSP neg_flags=0x62088a15
>     NTLMSSP_NEGOTIATE_UNICODE
>     NTLMSSP_REQUEST_TARGET
>     NTLMSSP_NEGOTIATE_SIGN
>     NTLMSSP_NEGOTIATE_NTLM
>     NTLMSSP_ANONYMOUS
>     NTLMSSP_NEGOTIATE_ALWAYS_SIGN
>     NTLMSSP_NEGOTIATE_EXTENDED_SESSIONSECURITY
>     NTLMSSP_NEGOTIATE_VERSION
>     NTLMSSP_NEGOTIATE_128
>     NTLMSSP_NEGOTIATE_KEY_EXCH
> [2016/04/26 17:06:13.913179,  3, pid=11263, effective(0, 0), real(0, 0)]
> ../source3/libsmb/cliconnect.c:2173(cli_session_setup_done_spnego)
>   SPNEGO login failed: Logon failure
> [2016/04/26 17:06:13.913395,  0, pid=11263, effective(0, 0), real(0, 0),
> class=auth] ../source3/auth/auth_domain.c:184(domain_client_validate)
>   domain_client_validate: Domain password server not available.
> [2016/04/26 17:06:13.915495,  5, pid=11263, effective(0, 0), real(0, 0),
> class=auth] ../source3/auth/auth.c:252(auth_check_ntlm_password)
>   check_ntlm_password: winbind authentication for user [testuser] FAILED
> with error NT_STATUS_LOGON_FAILURE
> [2016/04/26 17:06:13.915542,  2, pid=11263, effective(0, 0), real(0, 0),
> class=auth] ../source3/auth/auth.c:315(auth_check_ntlm_password)
>   check_ntlm_password:  Authentication for user [testuser] -> [testuser]
> FAILED with error NT_STATUS_LOGON_FAILURE
> [2016/04/26 17:06:13.915561,  5, pid=11263, effective(0, 0), real(0, 0)]
> ../source3/auth/auth_ntlmssp.c:188(auth3_check_password)
>   Checking NTLMSSP password for workgroup\testuser failed:
> NT_STATUS_LOGON_FAILURE
> [2016/04/26 17:06:13.915600,  3, pid=11263, effective(0, 0), real(0, 0)]
> ../source3/smbd/error.c:82(error_packet_set)
>   NT error packet at ../source3/smbd/sesssetup.c(934) cmd=115
> (SMBsesssetupX) NT_STATUS_LOGON_FAILURE
>


On Wed, Apr 27, 2016 at 2:50 PM, Jeremy Allison <jra at samba.org> wrote:
> On Tue, Apr 26, 2016 at 05:08:48PM -0400, Glomski, Patrick wrote:
> > Failure for me is always:
> >
> > SMB PACKET: SMBsesssetupX (REPLY)
> > > SMB Command   =  0x73
> > > Error class   =  0x6D
> > > Error code    =  49152 (0xc000)
> > > Flags1        =  0x80
> > > Flags2        =  0x3
> > > Tree ID       =  0 (0x0)
> > > Proc ID       =  12056 (0x2f18)
> > > UID           =  29165 (0x71ed)
> > > MID           =  3 (0x3)
> > > Word Count    =  0 (0x0)
> > > NTError = STATUS_LOGON_FAILURE
> > > smb_bcc=0
> > >
> >
> > Credentials are correct; it works through nautilus' smb://...
> >
> > Let me know what else would help to diagnose. I can also privately
share
> > verbose samba or other logs.
>
> Debug level 10 on smbd. Look into the cause
> of the STATUS_LOGON_FAILURE in SMBsesssetupX
> (should be pretty obvious from that).
>

On Wed, Apr 27, 2016 at 03:18:18PM -0400, Glomski, Patrick
wrote:
> I have been running in loglevel 10 and looking at the logs, but as I said
> in my initial post the credentials function fine with nautilus. Appended is
> a connection log snippet where samba walks through a cascade of
> authentication methods and finally fails.
> 
> Earlier in the log, samba successfully determines which domain controller
> to talk to and pulls its information. However, just before the failure
> there is a line which I feel is the root cause of the problem.
> 
> >   domain_client_validate: Domain password server not available.
> >
> Taken at face value (not the best idea as it seems to be walking through a
> set of authentication methods), it indicates that somehow the information
> about the domain controller doing the authentication has been lost.
> 
> The password is good: It works on Windows and it works with nautilus. Samba
> says the password is bad when using cifs or smbclient. What gives?
> 
> For the situational awareness of others affected by this issue, I hoped it
> was sssd or the associated name service switch libraries, so I built and
> installed rpms for an older version of sssd. No dice.
What is your smb.conf. Is winbindd running ? If so you might
want to look in the winbindd logs, as this is an NTLM auth
which should be passed to winbindd in the domain case.

Both answered in my initial post; see below:

On Wed, Apr 27, 2016 at 4:02 PM, Jeremy Allison <jra at samba.org> wrote:
> On Wed, Apr 27, 2016 at 03:18:18PM -0400, Glomski, Patrick wrote:
> > I have been running in loglevel 10 and looking at the logs, but as I
said
> > in my initial post the credentials function fine with nautilus.
Appended
> is
> > a connection log snippet where samba walks through a cascade of
> > authentication methods and finally fails.
> >
> > Earlier in the log, samba successfully determines which domain
controller
> > to talk to and pulls its information. However, just before the failure
> > there is a line which I feel is the root cause of the problem.
> >
> > >   domain_client_validate: Domain password server not available.
> > >
> > Taken at face value (not the best idea as it seems to be walking
through
> a
> > set of authentication methods), it indicates that somehow the
information
> > about the domain controller doing the authentication has been lost.
> >
> > The password is good: It works on Windows and it works with nautilus.
> Samba
> > says the password is bad when using cifs or smbclient. What gives?
> >
> > For the situational awareness of others affected by this issue, I
hoped
> it
> > was sssd or the associated name service switch libraries, so I built
and
> > installed rpms for an older version of sssd. No dice.
>
> What is your smb.conf.


[global]
workgroup = WORKGROUP
server string = Linux Server
netbios name = SRVNAME
log level = 1
security = ads
dedicated keytab file = /etc/krb5.keytab
kerberos method = system keytab
realm = WORKGROUP.COM
passdb backend = tdbsam
socket options = TCP_NODELAY IPTOS_LOWDELAY
client NTLMv2 auth = yes

oplocks = False
level2oplocks = False
posix locking = no

log file = /var/log/samba/log.%m
max log size = 5000
include = /etc/samba/rhs-samba.conf

[test]
 path = /home/test
 inherit permissions = yes
 inherit acls = yes
 public = yes
 only guest = no
 writable = yes
 printable = no
 browseable = yes
 strict locking = no

> Is winbindd running ? If so you might
> want to look in the winbindd logs, as this is an NTLM auth
> which should be passed to winbindd in the domain case.
>
We don't and have never run winbind on the system, so winbind is off.

El 27/04/16 a les 22:02, Jeremy Allison ha escrit:
> What is your smb.conf. Is winbindd running ? If so you might
> want to look in the winbindd logs, as this is an NTLM auth
> which should be passed to winbindd in the domain case.
Is this so even for an NT domain member?
Because I tried running winbind but it segfaults immediately.
This is on ubuntu 14.04, winbind/samba 2:4.3.8+dfsg-0ubuntu.

Just before the segfault I see a strange user_name:

[0000] 13 00 5C 00 5C 00 48 00   50 00 00 00 00 49 00 42   ..\.\.H. P....I.B
[0010] 00 52 00 45 00 24 00 00   57 00 45 00 54 00 52 00   .R.E.$.. W.E.T.R.
[0020] 4F 00 4E 00 00 00 01 00   00 00 FF FF FF FF         O.N..... ......
      &response->data.nt4: struct NETLOGON_SAM_LOGON_RESPONSE_NT40
         command                  : LOGON_SAM_LOGON_RESPONSE (19)
         pdc_name                 : '\\HP'
         user_name                : '䤀䈀刀䔀␀'
         domain_name              : 'WETRON'
         nt_version               : 0x00000001 (1)
                1: NETLOGON_NT_VERSION_1
                0: NETLOGON_NT_VERSION_5
                0: NETLOGON_NT_VERSION_5EX
                0: NETLOGON_NT_VERSION_5EX_WITH_IP
                0: NETLOGON_NT_VERSION_WITH_CLOSEST_SITE
                0: NETLOGON_NT_VERSION_AVOID_NT4EMUL
                0: NETLOGON_NT_VERSION_PDC
                0: NETLOGON_NT_VERSION_IP
                0: NETLOGON_NT_VERSION_LOCAL
                0: NETLOGON_NT_VERSION_GC
         lmnt_token               : 0xffff (65535)
         lm20_token               : 0xffff (65535)


It seems there's an extra 0 (instead of an 'L') spoiling the utf-16 
interpretation, however I couldn't see it in wireshark traces, so I 
don't know where it comes from.

Bye
-- 
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007