samba - Dec 2016 - winbind terminates after machine password change and needs domain rejoin

Hello,

Samba 4.4.7 AD member on Linux SLES 12 here ...

We've been running flawlessly for weeks with version 4.4.5 until we updated
to 4.4.6 and experienced this bug:
https://bugzilla.samba.org/show_bug.cgi?id=12369
So we updated to 4.4.7 in which this issue was fixed with an interim downgrade
to version 4.4.5 until 4.4.7 was available.

Now, we're experiencing another issue and it seems related to machine
(trusted account) password change.
When this happens:
- users get an 'access denied' error to their home directory.
- winbindd is not running anymore on the Samba server
- restarting winbindd is not enough to fix the issue. We also need to join the
domain again.

We first had the issue Mon 28th early in the afternoon and then yesterday early
in the afternoon which is exactly 7 days after.

log.wb-{DOMAINNAME} showed the same lines in either case:
[2016/11/30 10:25:26.114186,  1]
../source3/libsmb/trusts_util.c:264(trust_pw_change)
  2016/11/30 10:25:26 : trust_pw_change(UNIV-LR): Changed password locally
[2016/11/30 10:25:26.179269,  1]
../source3/libsmb/trusts_util.c:278(trust_pw_change)
  2016/11/30 10:25:26 : trust_pw_change(UNIV-LR): Changed password remotely.
[2016/11/30 10:25:26.516562,  0]
../source3/winbindd/winbindd.c:280(winbindd_sig_term_handler)
  Got sig[15] terminate (is_parent=0)

The 'machine password timeout' parameter has the default value of 604800
seconds which is exactly 7 days.

I'm not sure about disabling password change setting a 0 value to the
machine password timeout parameters because it's a security feature and
because it just worked before.
Maybe I can try to force the password setting debug level to 10  using 'net
ads changetrustpw' and see if I can reproduce the issue (users may be angry
with another outage ...)

Any help appreciated

Thank you
Alban

On Tue, 6 Dec 2016, Rodriguez Alban via samba wrote:
> Hello,
>
> Samba 4.4.7 AD member on Linux SLES 12 here ...
>
> We've been running flawlessly for weeks with version 4.4.5 until we 
> updated to 4.4.6 and experienced this bug: 
> https://bugzilla.samba.org/show_bug.cgi?id=12369 So we updated to 4.4.7 
> in which this issue was fixed with an interim downgrade to version 4.4.5 
> until 4.4.7 was available.
>
> Now, we're experiencing another issue and it seems related to machine 
> (trusted account) password change.
> When this happens:
> - users get an 'access denied' error to their home directory.
> - winbindd is not running anymore on the Samba server
> - restarting winbindd is not enough to fix the issue. We also need to join
the domain again.
>
> We first had the issue Mon 28th early in the afternoon and then 
> yesterday early in the afternoon which is exactly 7 days after.
>
> log.wb-{DOMAINNAME} showed the same lines in either case:
> [2016/11/30 10:25:26.114186,  1]
../source3/libsmb/trusts_util.c:264(trust_pw_change)
>  2016/11/30 10:25:26 : trust_pw_change(UNIV-LR): Changed password locally
> [2016/11/30 10:25:26.179269,  1]
../source3/libsmb/trusts_util.c:278(trust_pw_change)
>  2016/11/30 10:25:26 : trust_pw_change(UNIV-LR): Changed password remotely.
> [2016/11/30 10:25:26.516562,  0]
../source3/winbindd/winbindd.c:280(winbindd_sig_term_handler)
>  Got sig[15] terminate (is_parent=0)
>
> The 'machine password timeout' parameter has the default value of
604800
> seconds which is exactly 7 days.
>
> I'm not sure about disabling password change setting a 0 value to the 
> machine password timeout parameters because it's a security feature and
> because it just worked before. Maybe I can try to force the password 
> setting debug level to 10 using 'net ads changetrustpw' and see if
I can
> reproduce the issue (users may be angry with another outage ...)
>
> Any help appreciated
>
> Thank you
> Alban
I'm seeing weird behavior with winbind around machine account password 
changes too.  See my thread with subject "winbind trust account password 
management" (no one has responded yet).

I'm running v4.4.4 right now.  I'm planning to upgrade to v4.5.1 in a
few
weeks with the (misguided?) hope that it will work better in the latest 
version.

 	Andy

Le 6 déc. 2016 à 18:18, Andrew Morgan <morgan at orst.edu> a écrit :
> On Tue, 6 Dec 2016, Rodriguez Alban via samba wrote:
> 
>> Hello,
>> 
>> Samba 4.4.7 AD member on Linux SLES 12 here ...
>> 
>> We've been running flawlessly for weeks with version 4.4.5 until we
updated to 4.4.6 and experienced this bug:
https://bugzilla.samba.org/show_bug.cgi?id=12369 So we updated to 4.4.7 in which
this issue was fixed with an interim downgrade to version 4.4.5 until 4.4.7 was
available.
>> 
>> Now, we're experiencing another issue and it seems related to
machine (trusted account) password change.
>> When this happens:
>> - users get an 'access denied' error to their home directory.
>> - winbindd is not running anymore on the Samba server
>> - restarting winbindd is not enough to fix the issue. We also need to
join the domain again.
>> 
>> We first had the issue Mon 28th early in the afternoon and then
yesterday early in the afternoon which is exactly 7 days after.
>> 
>> log.wb-{DOMAINNAME} showed the same lines in either case:
>> [2016/11/30 10:25:26.114186,  1]
../source3/libsmb/trusts_util.c:264(trust_pw_change)
>> 2016/11/30 10:25:26 : trust_pw_change(UNIV-LR): Changed password
locally
>> [2016/11/30 10:25:26.179269,  1]
../source3/libsmb/trusts_util.c:278(trust_pw_change)
>> 2016/11/30 10:25:26 : trust_pw_change(UNIV-LR): Changed password
remotely.
>> [2016/11/30 10:25:26.516562,  0]
../source3/winbindd/winbindd.c:280(winbindd_sig_term_handler)
>> Got sig[15] terminate (is_parent=0)
>> 
>> The 'machine password timeout' parameter has the default value
of 604800 seconds which is exactly 7 days.
>> 
>> I'm not sure about disabling password change setting a 0 value to
the machine password timeout parameters because it's a security feature and
because it just worked before. Maybe I can try to force the password setting
debug level to 10 using 'net ads changetrustpw' and see if I can
reproduce the issue (users may be angry with another outage ...)
>> 
>> Any help appreciated
>> 
>> Thank you
>> Alban
> 
> I'm seeing weird behavior with winbind around machine account password
changes too.  See my thread with subject "winbind trust account password
management" (no one has responded yet).
> 
> I'm running v4.4.4 right now.  I'm planning to upgrade to v4.5.1 in
a few weeks with the (misguided?) hope that it will work better in the latest
version.
> 
> 	Andy

Andy,

In fact, I've seen your post while searching for a known bug about my
current issue. But it seemed different. Maybe both are related.
What is weird in your log is (simplified):

Changed password locally
Changed password remotely
Maybe ... the trust account password was changed and we didn't know it.

So the trust account password is changed (which I believe is triggered by the
client or in this case member server) and then it pretends it didn't know !?

Also, I don't see winbindd receiving signal 15 just after password change on
your side. So I wonder why, and which process is sending a terminate signal to
winbind on mine ?

Anyways, I'll probably fill a bug report for that issue because I really
think it's a new bug since 4.4.5 and I will  probably downgrade to 4.4.5
(again).

Cheers

Alban