samba - Nov 2020 - NT_STATUS_NO_LOGON_SERVERS with domain joined member samba server

Hello, 

We had a strange thing happen today - we shut down our system (virtual machines)
so that some work could be done in the server closet and everything was shut
down gracefully but when everything was started back up I can no longer access
my file server from windows. I get the following message in windows:

\\fs1 is not accessible. You might not have permission to use this network
resource. Contact
the administrator of this server to find out if you have access permissions. 

We can't sign you in with this credential because your domain isn't
available. Make sure your
device is connected to your organization's network and try again. If you
previously signed in
on this device with another credential, you can sign in with that credential. 

I am able to see and access netlogon and sysvol on the samba 4 ADDC (DC1) from
this same computer.

My log.smbd shows the following: 

#> tail log.smbd 
check_ntlm_password: Authentication for user [tech] -> [tech] FAILED with
error NT_STATUS_NO_LOGON_SERVERS, authoritative=1
[2020/11/09 20:58:45.144324, 2]
../../auth/auth_log.c:653(log_authentication_event_human_readable)
Auth: [SMB2,(null)] user [CROSSFIRE]\[tech] at [Mon, 09 Nov 2020 20:58:45.144299
EST] with [NTLMv2] status [NT_STATUS_NO_LOGON_SERVERS] workstation
[CROSSFIRE-EDITI] remote host [ipv4:192.168.11.20:58682] mapped to
[CROSSFIRE]\[tech]. local host [ipv4:192.168.11.3:445]
{"timestamp": "2020-11-09T20:58:45.144420-0500",
"type": "Authentication", "Authentication":
{"version": {"major": 1, "minor": 2},
"eventId": 4625, "logonId": "0",
"logonType": 3, "status":
"NT_STATUS_NO_LOGON_SERVERS", "localAddress":
"ipv4:192.168.11.3:445", "remoteAddress":
"ipv4:192.168.11.20:58682", "serviceDescription":
"SMB2", "authDescription": null, "clientDomain":
"CROSSFIRE", "clientAccount": "tech",
"workstation": "CROSSFIRE-EDITI", "becameAccount":
null, "becameDomain": null, "becameSid": null,
"mappedAccount": "tech", "mappedDomain":
"CROSSFIRE", "netlogonComputer": null,
"netlogonTrustAccount": null, "netlogonNegotiateFlags":
"0x00000000", "netlogonSecureChannelType": 0,
"netlogonTrustAccountSid": null, "passwordType":
"NTLMv2", "duration": 3439}}
[2020/11/09 20:58:45.144546, 3]
../../auth/gensec/spnego.c:1444(gensec_spnego_server_negTokenTarg_step)
gensec_spnego_server_negTokenTarg_step: SPNEGO(ntlmssp) login failed:
NT_STATUS_NO_LOGON_SERVERS
[2020/11/09 20:58:45.144634, 3]
../../source3/smbd/smb2_server.c:3256(smbd_smb2_request_error_ex)
smbd_smb2_request_error_ex: smbd_smb2_request_error_ex: idx[1]
status[NT_STATUS_NO_LOGON_SERVERS] || at ../../source3/smbd/smb2_sesssetup.c:149
[2020/11/09 20:58:45.145740, 3]
../../source3/smbd/server_exit.c:244(exit_server_common)
Server exit (NT_STATUS_CONNECTION_RESET) 

CROSSFIRE-EDITI is the windows machine that I'm attempting to access fs1
from.

All of this was working fine before I shut down the servers... according to the
process list smbd and nmbd are both running. I am able to log into the domain
from the workstation and it authenticates me without error so I think the DC is
working properly.

The DNS on FS1 is pointing to the DC1 server - verified that. 
I'm not out of disk space. 

Tried to check the simple stuff. This server hasn't been rebooted in quite a
while and has been rock solid for several months.

Thanks, 
Rich

----- On Nov 9, 2020, at 9:20 PM, samba samba at lists.samba.org wrote:
> Hello,
> 
> We had a strange thing happen today - we shut down our system (virtual
machines)
> so that some work could be done in the server closet and everything was
shut
> down gracefully but when everything was started back up I can no longer
access
> my file server from windows. I get the following message in windows:
> 
> \\fs1 is not accessible. You might not have permission to use this network
> resource. Contact
> the administrator of this server to find out if you have access
permissions.
> 
> We can't sign you in with this credential because your domain isn't
available.
> Make sure your
> device is connected to your organization's network and try again. If
you
> previously signed in
> on this device with another credential, you can sign in with that
credential.
> 
> I am able to see and access netlogon and sysvol on the samba 4 ADDC (DC1)
from
> this same computer.
> 
> My log.smbd shows the following:
> 
> #> tail log.smbd
> check_ntlm_password: Authentication for user [tech] -> [tech] FAILED
with error
> NT_STATUS_NO_LOGON_SERVERS, authoritative=1
> [2020/11/09 20:58:45.144324, 2]
> ../../auth/auth_log.c:653(log_authentication_event_human_readable)
> Auth: [SMB2,(null)] user [CROSSFIRE]\[tech] at [Mon, 09 Nov 2020
20:58:45.144299
> EST] with [NTLMv2] status [NT_STATUS_NO_LOGON_SERVERS] workstation
> [CROSSFIRE-EDITI] remote host [ipv4:192.168.11.20:58682] mapped to
> [CROSSFIRE]\[tech]. local host [ipv4:192.168.11.3:445]
> {"timestamp": "2020-11-09T20:58:45.144420-0500",
"type": "Authentication",
> "Authentication": {"version": {"major": 1,
"minor": 2}, "eventId": 4625,
> "logonId": "0", "logonType": 3,
"status": "NT_STATUS_NO_LOGON_SERVERS",
> "localAddress": "ipv4:192.168.11.3:445",
"remoteAddress":
> "ipv4:192.168.11.20:58682", "serviceDescription":
"SMB2", "authDescription":
> null, "clientDomain": "CROSSFIRE",
"clientAccount": "tech", "workstation":
> "CROSSFIRE-EDITI", "becameAccount": null,
"becameDomain": null, "becameSid":
> null, "mappedAccount": "tech",
"mappedDomain": "CROSSFIRE", "netlogonComputer":
> null, "netlogonTrustAccount": null,
"netlogonNegotiateFlags": "0x00000000",
> "netlogonSecureChannelType": 0,
"netlogonTrustAccountSid": null,
> "passwordType": "NTLMv2", "duration": 3439}}
> [2020/11/09 20:58:45.144546, 3]
> ../../auth/gensec/spnego.c:1444(gensec_spnego_server_negTokenTarg_step)
> gensec_spnego_server_negTokenTarg_step: SPNEGO(ntlmssp) login failed:
> NT_STATUS_NO_LOGON_SERVERS
> [2020/11/09 20:58:45.144634, 3]
> ../../source3/smbd/smb2_server.c:3256(smbd_smb2_request_error_ex)
> smbd_smb2_request_error_ex: smbd_smb2_request_error_ex: idx[1]
> status[NT_STATUS_NO_LOGON_SERVERS] || at
> ../../source3/smbd/smb2_sesssetup.c:149
> [2020/11/09 20:58:45.145740, 3]
> ../../source3/smbd/server_exit.c:244(exit_server_common)
> Server exit (NT_STATUS_CONNECTION_RESET)
> 
> CROSSFIRE-EDITI is the windows machine that I'm attempting to access
fs1 from.
> 
> All of this was working fine before I shut down the servers... according to
the
> process list smbd and nmbd are both running. I am able to log into the
domain
> from the workstation and it authenticates me without error so I think the
DC is
> working properly.
> 
> The DNS on FS1 is pointing to the DC1 server - verified that.
> I'm not out of disk space.
> 
> Tried to check the simple stuff. This server hasn't been rebooted in
quite a
> while and has been rock solid for several months.
> 
> Thanks,
> Rich
Further investigation revealed that Winbindd was not running and so I looked at
the log for that and it had an error indicated that we weren't joined to an
AD.  Through some searching I found that the error indicating "Did We
Join?" was a clear sign that the machine was not part of a join.  I did a
net ads join -U and the join worked.  I was then able to start winbind and
things started working.

Any idea why my server would just "forget" that it was / is joined to
the domain?

Rich