samba - Nov 2023 - windows workstations needing reboot to validate passwords. -- ERROR MESSAGE

On Tue, 2023-11-21 at 09:19 -0800, Ray Klassen via samba
wrote:
> 
> 
> On Tue, 2023-11-21 at 12:00 -0500, James Atwell via samba wrote:
> > 
> > 
> > > -----Original Message-----
> > > From: samba <samba-bounces at lists.samba.org> On Behalf Of
Ray
> > > Klassen via
> > > samba
> > > Sent: Monday, November 20, 2023 7:39 PM
> > > To: samba at lists.samba.org
> > > Subject: Re: [Samba] windows workstations needing reboot to
> > > validate
> > > passwords. --ADDENDUM
> > > 
> > > 
> > > 
> > > On Mon, 2023-11-20 at 15:19 -0500, James Atwell via samba wrote:
> > > > > -----Original Message-----
> > > > > From: samba <samba-bounces at lists.samba.org> On
Behalf Of Ray
> > > > > Klassen
> > > > > via samba
> > > > > Sent: Monday, November 20, 2023 2:10 PM
> > > > > To: samba at lists.samba.org
> > > > > Subject: Re: [Samba] windows workstations needing
reboot to
> > > > > validate
> > > > > passwords. --ADDENDUM
> > > > > 
> > > > > 
> > > > > 
> > > > > On Mon, 2023-11-20 at 13:43 -0500, James Atwell via
samba
> > > > > wrote:
> > > > > > 
> > > > > > 
> > > > > > > -----Original Message-----
> > > > > > > From: samba <samba-bounces at
lists.samba.org> On Behalf Of
> > > > > > > Ray
> > > > > > > Klassen via samba
> > > > > > > Sent: Monday, November 20, 2023 1:09 PM
> > > > > > > To: samba at lists.samba.org
> > > > > > > Subject: Re: [Samba] windows workstations
needing reboot
> > > > > > > to
> > > > > > > validate passwords. --ADDENDUM
> > > > > > > 
> > > > > > > Audit logging has been a bust. The failed
attempt by the
> > > > > > > workstation to validate the password does not
show up in
> > > > > > > the
> > > > > > > logs.
> > > > > > > 
> > > > > > > 
> > > > > > > On Thu, 2023-11-16 at 10:38 -0800, Ray
Klassen via samba
> > > > > > > wrote:
> > > > > > > > Thank you for the suggestion. Audit
logging enabled.
> > > > > > > > 
> > > > > > > > On Thu, 2023-11-16 at 13:27 -0500, James
Atwell via
> > > > > > > > samba
> > > > > > > > wrote:
> > > > > > > > > Have you setup Samba audit logging?
This may aid in
> > > > > > > > > your
> > > > > > > > > efforts to see the reasons for not
authenticating
> > > > > > > > > from
> > > > > > > > > the
> > > > > > > > > servers perspective.
> > > > > > > > > 
> > > > > > > > >
https://wiki.samba.org/index.php/Setting_up_Audit_Logging
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: samba <samba-bounces at
lists.samba.org> On Behalf
> > > > > > > > > Of Ray
> > > > > > > > > Klassen via samba
> > > > > > > > > Sent: Thursday, November 16, 2023
1:11 PM
> > > > > > > > > To: samba at lists.samba.org
> > > > > > > > > Subject: [Samba] windows
workstations needing reboot
> > > > > > > > > to
> > > > > > > > > validate passwords. --ADDENDUM
> > > > > > > > > 
> > > > > > > > > I am (earlier reported under the
subject "Peculiar
> > > > > > > > > Problem")
> > > > > > > > > having an issue that started
several weeks ago, where
> > > > > > > > > windows
> > > > > > > > > (10 pro, server
> > > > > > > > > 2019) computers randomly get into a
state where they
> > > > > > > > > refuse
> > > > > > > > > to validate passwords. Rebooting
(sometimes several
> > > > > > > > > times)
> > > > > > > > > makes the problem go away. You can
also log in if you
> > > > > > > > > disconnect the PC from the network
and then
> > > > > > > > > reconnect.
> > > > > > > > > 
> > > > > > > > > List of changes around the time it
started.
> > > > > > > > > 
> > > > > > > > > Samba upgrade to 4.19.2
> > > > > > > > > Samba schema upgrade to 2012_R2
functional level
> > > > > > > > > Samba
> > > > > > > > > upgrade to
> > > > > > > > > 2008 functional level
> > > > > > > > > 
> > > > > > > > > List of measures taken (hoping that
if best practises
> > > > > > > > > are
> > > > > > > > > not being observed, implementing
them will fix
> > > > > > > > > things!!)
> > > > > > > > > 
> > > > > > > > > Moved DNS from SAMBA_INTERNAL to
BIND_DLZ Moved ntp
> > > > > > > > > from
> > > > > ntpsec
> > > > > > > to
> > > > > > > > > chrony
> > > > > > > > > 
> > > > > > > > > Diagnostic steps
> > > > > > > > > 
> > > > > > > > > Packet dumps (decoded with keytab)
and loglevel 255
> > > > > > > > > show no
> > > > > > > > > glaring issues or errors.
> > > > > > > > > 
> > > > > > > > > Going to try restarting all of the
DC's next time it
> > > > > > > > > happens
> > > > > > > > > to determine if the
miscommunication originates with
> > > > > > > > > windows
> > > > > > > > > or samba.
> > > > > > > > > 
> > > > > > > > > Windows Eventviewer lists failure
as Event ID 4625
> > > > > > > > > Status
> > > > > > > > > 0xC000006D Sub Status 0x0 Failure
reason %%2304
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > Any other suggestions welcome!!
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > --
> > > > > > > > > To unsubscribe from this list go to
the following URL
> > > > > > > > > and
> > > > > > > > > read the
> > > > > > > > > instructions:
> > > > > > > > >
https://lists.samba.org/mailman/options/samba
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > --
> > > > > > > To unsubscribe from this list go to the
following URL and
> > > > > > > read
> > > > > > > the
> > > > > > > instructions:?
> > > > > > > https://lists.samba.org/mailman/options/samba
> > > > > > 
> > > > > > You mentioned restarting all your DC's. I
assume you have
> > > > > > more
> > > > > > than 1 DC and enabled audit logging on all your
DC's. I
> > > > > > also
> > > > > > assume you verified on all DC's the logs do
not exist if
> > > > > > enabled
> > > > > > on all?
> > > > > > 
> > > > > > 
> > > > > > I have 4 DC's. I've got auditing enabled
on all of them.
> > > > > > And
> > > > > > seeing audit entries on all of them regarding
other
> > > > > > traffic.
> > > > > > The
> > > > > > wkstation that misbehaved this morning shows
entries on
> > > > > > some
> > > > > > of
> > > > > > them over the weekend 'NT_STATUS_OK'and
earlier. It looks
> > > > > > like it
> > > > > > doing a machine password update.
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > --
> > > > > To unsubscribe from this list go to the following URL
and
> > > > > read
> > > > > the
> > > > > instructions:?
https://lists.samba.org/mailman/options/samba
> > > > 
> > > > 
> > > > The fact that you can unplug the device and log back in
tells
> > > > me
> > > > the
> > > > workstation is using cached credentials to log back in.
> > > > 
> > > > Try authenticating to the netlogon share from each of your
DC's
> > > > with
> > > > one of the affected usernames.
> > > > 
> > > > smbclient //localhost/netlogon -Uusername -c 'ls'
> > > > 
> > > 
> > > 
> > > 
> > > > I would also check replication is working as expected and
all
> > > > databases match.
> > > > 
> > > > https://wiki.samba.org/index.php/Samba-tool_ldapcmp
> > > > 
> > > > The biggest change you made was upgrading the schema. Did
you
> > > > ensure
> > > > to include
> > > > 
> > > > ad dc functional level = 2016
> > > > 
> > > > in the smb.conf file on all your DC's?
> > > > 
> > > > Without log files its hard to troubleshoot. You need to pull
> > > > the
> > > > authentication attempt failure to analyze. Do you have other
> > > > services
> > > > that use your DC for authentication that exhibit similar
> > > > behavior?
> > > > 
> > > > 
> > > 
> > > 
> > > > The schema upgrade was described in the following wiki page
> > > > without
> > > > reference to upping the actual domain functional level. once
> > > > the
> > > > schema upgrade was successful I upped samba to the maximum
> > > > allowed --
> > > > 2008. Does samba level need to be equal to its schema?
Should
> > > > we
> > > > update the wiki page to include that?
> > > https://wiki.samba.org/index.php/Azure_AD_Connect_Cloud_sync>
> > > 
> > > FYI samba-tool ldapcmp registers SUCCESS between the main DC and
> > > the
> > > others on all comparisons samba-tool drs showrepl (something I
> > > check
> > > everytime I install a new
> > > version) is showing 0 failures across the board.
> > > 
> > > I've got a server that has the problem... I'm looking for
ways to
> > > remotely reset
> > > the machine password to see if that's the issue. I don't
think
> > > it's
> > > using cached
> > > credentials for the user. If it was, it would work, as
> > > disconnecting the box from
> > > the LAN and forcing cached credentials works every time.
> > > 
> > > 
> > 
> > The link you provided refers to Azure AD Cloud Sync. For my schema
> > upgrade I used the following link
> > https://wiki.samba.org/index.php/AD_Schema_Version_Support
> > and version notes from 4.19.0.
> > https://www.samba.org/samba/history/samba-4.19.0.html
> > 
> > 
> 
> 
> Okay. Domain Functional level now equals schema upgrade. I want to
> wait
> on the 2016 schema and functional level as the release note classify
> that as initial. The only reason I upgraded the schema in the first
> place to was to be ready to use Cloud Sync if necessary. I'm guessing
> that 2012_R2 has the chance of being more complete -- I assume there
> are fewer changes from earlier functional levels. If this works and
> my
> problem goes away, I'd really like to know what association my
> problem
> had with this as a solution.
> > 

well that didn't fix the problem. not sure where to go from
here.
>

On Wed, 2023-11-22 at 15:02 -0800, Ray Klassen via samba
wrote:
> 
> 
> On Tue, 2023-11-21 at 09:19 -0800, Ray Klassen via samba wrote:
> > 
> > 
> > On Tue, 2023-11-21 at 12:00 -0500, James Atwell via samba wrote:
> > > 
> > > 
> > > > -----Original Message-----
> > > > From: samba <samba-bounces at lists.samba.org> On
Behalf Of Ray
> > > > Klassen via
> > > > samba
> > > > Sent: Monday, November 20, 2023 7:39 PM
> > > > To: samba at lists.samba.org
> > > > Subject: Re: [Samba] windows workstations needing reboot to
> > > > validate
> > > > passwords. --ADDENDUM
> > > > 
> > > > 
> > > > 
> > > > On Mon, 2023-11-20 at 15:19 -0500, James Atwell via samba
> > > > wrote:
> > > > > > -----Original Message-----
> > > > > > From: samba <samba-bounces at
lists.samba.org> On Behalf Of
> > > > > > Ray
> > > > > > Klassen
> > > > > > via samba
> > > > > > Sent: Monday, November 20, 2023 2:10 PM
> > > > > > To: samba at lists.samba.org
> > > > > > Subject: Re: [Samba] windows workstations needing
reboot to
> > > > > > validate
> > > > > > passwords. --ADDENDUM
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > On Mon, 2023-11-20 at 13:43 -0500, James Atwell
via samba
> > > > > > wrote:
> > > > > > > 
> > > > > > > 
> > > > > > > > -----Original Message-----
> > > > > > > > From: samba <samba-bounces at
lists.samba.org> On Behalf
> > > > > > > > Of
> > > > > > > > Ray
> > > > > > > > Klassen via samba
> > > > > > > > Sent: Monday, November 20, 2023 1:09 PM
> > > > > > > > To: samba at lists.samba.org
> > > > > > > > Subject: Re: [Samba] windows
workstations needing
> > > > > > > > reboot
> > > > > > > > to
> > > > > > > > validate passwords. --ADDENDUM
> > > > > > > > 
> > > > > > > > Audit logging has been a bust. The
failed attempt by
> > > > > > > > the
> > > > > > > > workstation to validate the password
does not show up
> > > > > > > > in
> > > > > > > > the
> > > > > > > > logs.
> > > > > > > > 
> > > > > > > > 
> > > > > > > > On Thu, 2023-11-16 at 10:38 -0800, Ray
Klassen via
> > > > > > > > samba
> > > > > > > > wrote:
> > > > > > > > > Thank you for the suggestion. Audit
logging enabled.
> > > > > > > > > 
> > > > > > > > > On Thu, 2023-11-16 at 13:27 -0500,
James Atwell via
> > > > > > > > > samba
> > > > > > > > > wrote:
> > > > > > > > > > Have you setup Samba audit
logging? This may aid in
> > > > > > > > > > your
> > > > > > > > > > efforts to see the reasons for
not authenticating
> > > > > > > > > > from
> > > > > > > > > > the
> > > > > > > > > > servers perspective.
> > > > > > > > > > 
> > > > > > > > > >
https://wiki.samba.org/index.php/Setting_up_Audit_Logging
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > -----Original Message-----
> > > > > > > > > > From: samba <samba-bounces
at lists.samba.org> On
> > > > > > > > > > Behalf
> > > > > > > > > > Of Ray
> > > > > > > > > > Klassen via samba
> > > > > > > > > > Sent: Thursday, November 16,
2023 1:11 PM
> > > > > > > > > > To: samba at lists.samba.org
> > > > > > > > > > Subject: [Samba] windows
workstations needing
> > > > > > > > > > reboot
> > > > > > > > > > to
> > > > > > > > > > validate passwords. --ADDENDUM
> > > > > > > > > > 
> > > > > > > > > > I am (earlier reported under
the subject "Peculiar
> > > > > > > > > > Problem")
> > > > > > > > > > having an issue that started
several weeks ago,
> > > > > > > > > > where
> > > > > > > > > > windows
> > > > > > > > > > (10 pro, server
> > > > > > > > > > 2019) computers randomly get
into a state where
> > > > > > > > > > they
> > > > > > > > > > refuse
> > > > > > > > > > to validate passwords.
Rebooting (sometimes several
> > > > > > > > > > times)
> > > > > > > > > > makes the problem go away. You
can also log in if
> > > > > > > > > > you
> > > > > > > > > > disconnect the PC from the
network and then
> > > > > > > > > > reconnect.
> > > > > > > > > > 
> > > > > > > > > > List of changes around the
time it started.
> > > > > > > > > > 
> > > > > > > > > > Samba upgrade to 4.19.2
> > > > > > > > > > Samba schema upgrade to
2012_R2 functional level
> > > > > > > > > > Samba
> > > > > > > > > > upgrade to
> > > > > > > > > > 2008 functional level
> > > > > > > > > > 
> > > > > > > > > > List of measures taken (hoping
that if best
> > > > > > > > > > practises
> > > > > > > > > > are
> > > > > > > > > > not being observed,
implementing them will fix
> > > > > > > > > > things!!)
> > > > > > > > > > 
> > > > > > > > > > Moved DNS from SAMBA_INTERNAL
to BIND_DLZ Moved ntp
> > > > > > > > > > from
> > > > > > ntpsec
> > > > > > > > to
> > > > > > > > > > chrony
> > > > > > > > > > 
> > > > > > > > > > Diagnostic steps
> > > > > > > > > > 
> > > > > > > > > > Packet dumps (decoded with
keytab) and loglevel 255
> > > > > > > > > > show no
> > > > > > > > > > glaring issues or errors.
> > > > > > > > > > 
> > > > > > > > > > Going to try restarting all of
the DC's next time
> > > > > > > > > > it
> > > > > > > > > > happens
> > > > > > > > > > to determine if the
miscommunication originates
> > > > > > > > > > with
> > > > > > > > > > windows
> > > > > > > > > > or samba.
> > > > > > > > > > 
> > > > > > > > > > Windows Eventviewer lists
failure as Event ID 4625
> > > > > > > > > > Status
> > > > > > > > > > 0xC000006D Sub Status 0x0
Failure reason %%2304
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > Any other suggestions
welcome!!
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > --
> > > > > > > > > > To unsubscribe from this list
go to the following
> > > > > > > > > > URL
> > > > > > > > > > and
> > > > > > > > > > read the
> > > > > > > > > > instructions:
> > > > > > > > > >
https://lists.samba.org/mailman/options/samba
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > --
> > > > > > > > To unsubscribe from this list go to the
following URL
> > > > > > > > and
> > > > > > > > read
> > > > > > > > the
> > > > > > > > instructions:?
> > > > > > > >
https://lists.samba.org/mailman/options/samba
> > > > > > > 
> > > > > > > You mentioned restarting all your DC's. I
assume you have
> > > > > > > more
> > > > > > > than 1 DC and enabled audit logging on all
your DC's. I
> > > > > > > also
> > > > > > > assume you verified on all DC's the logs
do not exist if
> > > > > > > enabled
> > > > > > > on all?
> > > > > > > 
> > > > > > > 
> > > > > > > I have 4 DC's. I've got auditing
enabled on all of them.
> > > > > > > And
> > > > > > > seeing audit entries on all of them regarding
other
> > > > > > > traffic.
> > > > > > > The
> > > > > > > wkstation that misbehaved this morning shows
entries on
> > > > > > > some
> > > > > > > of
> > > > > > > them over the weekend
'NT_STATUS_OK'and earlier. It looks
> > > > > > > like it
> > > > > > > doing a machine password update.
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > --
> > > > > > To unsubscribe from this list go to the following
URL and
> > > > > > read
> > > > > > the
> > > > > > instructions:?
> > > > > > https://lists.samba.org/mailman/options/samba
> > > > > 
> > > > > 
> > > > > The fact that you can unplug the device and log back in
tells
> > > > > me
> > > > > the
> > > > > workstation is using cached credentials to log back in.
> > > > > 
> > > > > Try authenticating to the netlogon share from each of
your
> > > > > DC's
> > > > > with
> > > > > one of the affected usernames.
> > > > > 
> > > > > smbclient //localhost/netlogon -Uusername -c
'ls'
> > > > > 
> > > > 
> > > > 
> > > > 
> > > > > I would also check replication is working as expected
and all
> > > > > databases match.
> > > > > 
> > > > > https://wiki.samba.org/index.php/Samba-tool_ldapcmp
> > > > > 
> > > > > The biggest change you made was upgrading the schema.
Did you
> > > > > ensure
> > > > > to include
> > > > > 
> > > > > ad dc functional level = 2016
> > > > > 
> > > > > in the smb.conf file on all your DC's?
> > > > > 
> > > > > Without log files its hard to troubleshoot. You need to
pull
> > > > > the
> > > > > authentication attempt failure to analyze. Do you have
other
> > > > > services
> > > > > that use your DC for authentication that exhibit
similar
> > > > > behavior?
> > > > > 
> > > > > 
> > > > 
> > > > 
> > > > > The schema upgrade was described in the following wiki
page
> > > > > without
> > > > > reference to upping the actual domain functional level.
once
> > > > > the
> > > > > schema upgrade was successful I upped samba to the
maximum
> > > > > allowed --
> > > > > 2008. Does samba level need to be equal to its schema?
Should
> > > > > we
> > > > > update the wiki page to include that?
> > > >
https://wiki.samba.org/index.php/Azure_AD_Connect_Cloud_sync>
> > > > 
> > > > FYI samba-tool ldapcmp registers SUCCESS between the main DC
> > > > and
> > > > the
> > > > others on all comparisons samba-tool drs showrepl (something
I
> > > > check
> > > > everytime I install a new
> > > > version) is showing 0 failures across the board.
> > > > 
> > > > I've got a server that has the problem... I'm
looking for ways
> > > > to
> > > > remotely reset
> > > > the machine password to see if that's the issue. I
don't think
> > > > it's
> > > > using cached
> > > > credentials for the user. If it was, it would work, as
> > > > disconnecting the box from
> > > > the LAN and forcing cached credentials works every time.
> > > > 
> > > > 
> > > 
> > > The link you provided refers to Azure AD Cloud Sync. For my
> > > schema
> > > upgrade I used the following link
> > > https://wiki.samba.org/index.php/AD_Schema_Version_Support
> > > and version notes from 4.19.0.
> > > https://www.samba.org/samba/history/samba-4.19.0.html
> > > 
> > > 
> > 
> > 
> > Okay. Domain Functional level now equals schema upgrade. I want to
> > wait
> > on the 2016 schema and functional level as the release note
> > classify
> > that as initial. The only reason I upgraded the schema in the first
> > place to was to be ready to use Cloud Sync if necessary. I'm
> > guessing
> > that 2012_R2 has the chance of being more complete -- I assume
> > there
> > are fewer changes from earlier functional levels. If this works and
> > my
> > problem goes away, I'd really like to know what association my
> > problem
> > had with this as a solution.
> > > 
> 
> 
> well that didn't fix the problem. not sure where to go from here.
Finally have an error message!
> > ?{"timestamp": "2023-11-22T12:55:27.227588-0800",
"type": "KDC
> > Authorization", "KDC Authorization":
{"version": {"major": 1,
> > "minor": 0}, "status":
"NT_STATUS_NO_SUCH_USER", "localAddress":
> > null, "remoteAddress": "ipv4:172.19.2.130:62219",
> > "serviceDescription": null, "authType":
"TGS-REQ with Ticket-
> > Granting Ticket", "domain": null, "account":
null, "sid": null,
> > "logonServer": "ADMIRAL", "authTime":
"2023-11-22T12:55:27.226868-
> > 0800", "serverPolicyAccessCheck": null}}