samba - Sep 2018 - winbindd crashing -- how to auto-heal?

El 2/9/18 a les 10:39, Rowland Penny via samba ha escrit:
> All of this is just a sticking plaster on the problem, if winbind is
> crashing on a regular basis, we need to know this and will need
> level 10 logs, debug info etc. Without this info, it will never get
> fixed.
Meanwhile, I need my server to keep running, so the plaster looks fine.
Besides, winbind isn't crashing, just stops resolving some groups/users.
I have several domain members and this is the only one that does it, but 
that's probably because it's our mail server and it is constantly 
authenticating users and resolving groups.
> 
> However, it may have already been fixed in a later version, so if you
> can upgrade and use a version that isn't experimental (This means, do
> not try to run a DC on red-hat using MIT)
This is a different distro (mageia 6), a different version of samba 
(4.6.12), not a dc but a domain member and I will stick to what the 
distro provides.

Bye
-- 
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007

Thanks for the workaround, Luca. I might end up going with:

#!/bin/bash
getent group | grep -q 'Domain Users' && exit 0
echo "restarting winbind"
sudo systemctl restart winbind


Rowland, it crashed again. Here's some info. Please let me know if I should
provide more:
https://gist.github.com/jamiejackson/8aa60fb0ac1f407ba73776aaaeaa542f

On Sun, Sep 2, 2018 at 4:50 AM Luca Olivetti via samba <
samba at lists.samba.org> wrote:
> El 2/9/18 a les 10:39, Rowland Penny via samba ha escrit:
>
> > All of this is just a sticking plaster on the problem, if winbind is
> > crashing on a regular basis, we need to know this and will need
> > level 10 logs, debug info etc. Without this info, it will never get
> > fixed.
>
> Meanwhile, I need my server to keep running, so the plaster looks fine.
> Besides, winbind isn't crashing, just stops resolving some
groups/users.
> I have several domain members and this is the only one that does it, but
> that's probably because it's our mail server and it is constantly
> authenticating users and resolving groups.
>
> >
> > However, it may have already been fixed in a later version, so if you
> > can upgrade and use a version that isn't experimental (This means,
do
> > not try to run a DC on red-hat using MIT)
>
> This is a different distro (mageia 6), a different version of samba
> (4.6.12), not a dc but a domain member and I will stick to what the
> distro provides.
>
> Bye
> --
> Luca Olivetti
> Wetron Automation Technology http://www.wetron.es/
> Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
>
> --
> To unsubscribe from this list go to the following URL and read the
> instructions:  https://lists.samba.org/mailman/options/samba
>

On Sun, 2 Sep 2018 22:37:05 -0400
Jamie Jackson via samba <samba at lists.samba.org> wrote:
> Thanks for the workaround, Luca. I might end up going with:
> 
> #!/bin/bash
> getent group | grep -q 'Domain Users' && exit 0
> echo "restarting winbind"
> sudo systemctl restart winbind
> 
> 
> Rowland, it crashed again. Here's some info. Please let me know if I
> should provide more:
> https://gist.github.com/jamiejackson/8aa60fb0ac1f407ba73776aaaeaa542f
> 
The last few lines of the log are these:

[2018/09/01 23:19:27.748555,  3, pid=9795]
../source3/winbindd/winbindd_msrpc.c:244(msrpc_name_to_sid)
  msrpc_name_to_sid: name=REDACTED\ROOT
[2018/09/01 23:19:27.748585,  3, pid=9795]
../source3/winbindd/winbindd_msrpc.c:258(msrpc_name_to_sid)
  name_to_sid [rpc] REDACTED\ROOT for domain REDACTED
[2018/09/01 23:20:28.475336,  0, pid=9795]
../source3/winbindd/winbindd.c:281(winbindd_sig_term_handler)
  Got sig[15] terminate (is_parent=0)

It looks like you ran you script/commands on Sun Sep  2 21:35:39 EDT
2018

To my eye, there is nothing really wrong, there certainly isn't any
evidence of a crash, the only evidence that I can see is that winbind
has exited.

So I went back to the info you posted earlier, at that time I
concentrated on the smb.conf, this time I looked a lot more closely at
this:

$ ps -aux | grep '\(samba\|smb\|winbind\)'
root     12600  0.0  0.0 399284  2324 ?        Ss   Aug31   0:16
/usr/sbin/winbindd --foreground --no-process-group
root     12602  0.0  0.0 424328  5948 ?        S    Aug31   0:01
/usr/sbin/winbindd --foreground --no-process-group
root     12857  0.0  0.0 406020   964 ?        S    Aug31   0:03
/usr/sbin/winbindd --foreground --no-process-group
root     12858  0.0  0.0 399144  1416 ?        S    Aug31   0:00
/usr/sbin/winbindd --foreground --no-process-group
root     12859  0.0  0.0 399144  1504 ?        S    Aug31   0:00 
/usr/sbin/winbindd --foreground --no-process-group

If I run the same command on a Unix domain member, I get this:

root      2231  0.0  0.1 398220 17912 ?        Ss   Aug29   0:03
/usr/sbin/winbindd
root      2593  0.0  0.1 455004 20328 ?        S    Aug29   0:10
/usr/sbin/winbindd
root      2630  0.0  0.0 307728 15336 ?        Ss   Aug29   0:00 /usr/sbin/smbd
-D
root      2683  0.0  0.0 296524  4516 ?        S    Aug29   0:00 /usr/sbin/smbd
-D
root      2684  0.0  0.0 296524  4516 ?        S    Aug29   0:00 /usr/sbin/smbd
-D
root      2730  0.0  0.0 265088  9696 ?        S    Aug29   0:00
/usr/sbin/winbindd
root      2745  0.0  0.0 265220  9560 ?        S    Aug29   0:00
/usr/sbin/winbindd
root      2746  0.0  0.1 403600 16412 ?        S    Aug29   0:01
/usr/sbin/winbindd

So, from that, can I ask these questions:

Why isn't 'smbd' running ?
Is 'nmbd' running ?
Why aren't you running 'winbind' as a daemon ?
Why do feel you don't need a process group for winbind ?

Rowland

I did have an extra look in that debug log. 

And i noticed: 
2018/08/30 14:24:36 : trust_pw_change(REDACTED): Verified new password remotely
using netlogon_creds_cli:CLI[RWHUDXDKRDEV/RWHUDXDKRDEV$]/SRV[RWGOV-DC1/REDACTED]
346  [2018/09/01 10:14:49.046206,  1, pid=12602]
../source3/libads/ldap_utils.c:93(ads_do_search_retry_internal)
347    Reducing LDAP page size from 1000 to 500 due to IO_TIMEOUT 
348  [2018/09/01 10:14:49.055826,  1, pid=12602]
../source3/libads/ldap_utils.c:109(ads_do_search_retry_internal)
349    ads_search_retry: failed to reconnect (No logon servers are currently
available to service the logon request.)
350  [2018/09/01 13:40:24.015312,  1, pid=12602]
../source3/winbindd/winbindd_cm.c:3128(cm_connect_lsa)

Somewhere between 2018/08/30 14:24:36 and 2018/09/01 10:14:49. 
Then its reconnecting. 

 [2018/09/01 13:40:24.
   Unwilling to make LSA connection to domain REDACTED without connection level
security, must set 'winbind sealed pipes = false' and 'require
strong key = false' to proceed: NT_STATUS_DOWNGRADE_DETECTED

2018/09/01 14:13:54.
winbindd_cm.c:2085(connection_ok) 
355    connection_ok: Connection to (null) for domain REDACTED is not connected 

get_dc_list: preferred server list: ", *"
Why are we seeing ", *"  and not "hostname, *"

Now  [2018/09/01 14:14:03 
It shows the correct preffered list. 
   get_dc_list: preferred server list: "RWGOV-DC1.REDACTED.wan, *" 

And as of this point its ok again untill  [2018/09/01 14:31:45 

ads: fetch sequence_number for REDACTED

msrpc_sid_to_name: S-0-0 for domain REDACTED

msrpc_sid_to_name: failed to lookup sids: NT_STATUS_INVALID_PARAMETER

msrpc_sid_to_name: S-1-5-21-314559009-3729260175-93040071-513 for domain
REDACTED

And repeat the above. 


When i look at above, i would say, i need these to say more, but my first guess,
errors in resolving or you did hit some winbind bugs, its a 50% 50% here. 

Check these first. 
1) Check Nsswitch.conf  ( post it to the list. ) 
2) check resolv.conf ( post it to the list. ) 
3) change smb.conf  ( already done, but post the result on the list again. )
In smb.conf, i would change autorid to rid. 

I do believe the case of this error is a configuration thing. 
Not a user error, but a combination of settings that is causing this bug. 


Quote Rowland:  On Mon, 3 Sep 2018 16:45:36 +0200 
>Yes, but it depends on how you run Samba. If you run Samba as a
>standalone server you only need to run 'smbd', but running
'nmbd' as
>well would be a good idea.
>
>If you run Samba as a PDC or BDC, the same as a standalone server goes.
>
>Anything else needs both smbd and winbind running.
>
>Rowland  
This really depends. 
In all cases, you dont "need" nmbd. Its just handy to "see"
you computers.
But its not needed, dns proxy = yes is helpfull to resolve the hostnames over
dns.
If you disable file and printer sharing in windows you also dont "see"
the computers.

Now taking in account also that there was an samba version with lots of winbind
bug,
i just cant remember the number (version) but i was in the low range of 4.6.x or
4.7.x .

But still i would try a setup with winbind only and these configs. 
By example, how i run my proxy with winbind for auth. 

search primaryZone.yourdomain.tld
nameserver ip_of_AD_DC_server
( or if you run caching dns, 127.0.0.1, then use a forward zone to the AD
DC's )

/etc/hosts
127.0.0.1   localhost
192.168.0.1 proxy1.primaryZone.yourdomain.tld  proxy1

Now the most important one in smb.conf. 

netbios name =  Its often not defined and that "should" be fine
normaly, but i advice to set it manualy.


Smb.conf
   workgroup = NTDOM
    security = ads
    realm = PRIMARYZONE.YOURKERBEROSDOMAIN.TLD
	
    netbios name = PROXY1 	# DEFINE IT MANUALY

# Note this : 
# The maximum length of the host name and of the fully qualified domain name
(FQDN) is 63 bytes per label
# and 255 bytes per FQDN. Note Windows does not permit computer names that
exceed 15 characters,
# and you cannot specify a DNS host name that differs from the NETBIOS host
name.

    preferred master = no
    domain master = no
    host msdfs = no
    interfaces = 192.168.0.1 127.0.0.1
    bind interfaces only = yes
    dns proxy = yes

    ## map id's outside to domain to tdb files.
    idmap config *:backend = tdb
    idmap config *:range = 2000-9999

    ## map ids from the domains, the range may not overlap !
    idmap config NTDOM : backend = rid
    idmap config NTDOM : range = 10000-3999999

    dedicated keytab file = /etc/krb5.keytab
    kerberos method = secrets and keytab

    # renew the kerberos ticket
    winbind refresh tickets = yes
    winbind use default domain = yes  # or no what you want/need here.

    # show users with getent passwd ( handy for debugging, then set yes. ) 
    winbind enum users  = no
    winbind enum groups = no

    # enable offline logins
    winbind offline logon = yes

    # user Administrator workaround, without it you are unable to set privileges
    username map = /etc/samba/samba_usermapping

    # disable usershares creating, when set empty no error log messages.
    usershare path 
    # Disable printing completely
    load printers = no
    printing = bsd
    printcap name = /dev/null
    disable spoolss = yes


I really suggest try these above settings and post the results. 

Greetz, 

Louis


> -----Oorspronkelijk bericht-----
> Van: samba [mailto:samba-bounces at lists.samba.org] Namens 
> Jamie Jackson via samba
> Verzonden: maandag 3 september 2018 4:37
> Aan: luca at wetron.es
> CC: samba at lists.samba.org
> Onderwerp: Re: [Samba] winbindd crashing -- how to auto-heal?
> 
> Thanks for the workaround, Luca. I might end up going with:
> 
> #!/bin/bash
> getent group | grep -q 'Domain Users' && exit 0
> echo "restarting winbind"
> sudo systemctl restart winbind
> 
> 
> Rowland, it crashed again. Here's some info. Please let me 
> know if I should
> provide more:
> https://gist.github.com/jamiejackson/8aa60fb0ac1f407ba73776aaaeaa542f
> 
> On Sun, Sep 2, 2018 at 4:50 AM Luca Olivetti via samba <
> samba at lists.samba.org> wrote:
> 
> > El 2/9/18 a les 10:39, Rowland Penny via samba ha escrit:
> >
> > > All of this is just a sticking plaster on the problem, if 
> winbind is
> > > crashing on a regular basis, we need to know this and will need
> > > level 10 logs, debug info etc. Without this info, it will 
> never get
> > > fixed.
> >
> > Meanwhile, I need my server to keep running, so the plaster 
> looks fine.
> > Besides, winbind isn't crashing, just stops resolving some 
> groups/users.
> > I have several domain members and this is the only one that 
> does it, but
> > that's probably because it's our mail server and it is
constantly
> > authenticating users and resolving groups.
> >
> > >
> > > However, it may have already been fixed in a later 
> version, so if you
> > > can upgrade and use a version that isn't experimental 
> (This means, do
> > > not try to run a DC on red-hat using MIT)
> >
> > This is a different distro (mageia 6), a different version of samba
> > (4.6.12), not a dc but a domain member and I will stick to what the
> > distro provides.
> >
> > Bye
> > --
> > Luca Olivetti
> > Wetron Automation Technology http://www.wetron.es/
> > Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
> >
> > --
> > To unsubscribe from this list go to the following URL and read the
> > instructions:  https://lists.samba.org/mailman/options/samba
> >
> -- 
> To unsubscribe from this list go to the following URL and read the
> instructions:  https://lists.samba.org/mailman/options/samba
> 
>