samba - May 2016 - After some time 4.3.9 Member Server in different Subnet than ADS controller loses trust

Hello everybody,

I hope someone can help me with this or point me into the right
direction since I am not being able to solve it since weeks.

Since last year I was running Samba 4.1.6 on Ubuntu 14.04 LTS without
issues as a active directory domain controller as well as member
servers. Trouble started with the upgrade to Samba 4.3.8 (now 4.3.9).

The ADS controller and most member servers are sharing the same subnet. For
security reasons I pushed one of the member servers into a DMZ. I am
using Kerberos, Winbind and Samba to integrate to the ADS.
What has worked with 4.1.6 seems not to work anymore with 4.3.8 and
4.3.9. While all member servers on the same subnet work fine the machine
in the DMZ looses connection to the ADS after some time.

On the member server in the DMZ, from a shell I can successfully
- obtain Kerberos tickets
- join to the domain via (net ads join ...)
- After join do a testjoin
- obtain domain information
- get users via >wbinfo -u< and groups via >wbinfo -g<
- create a keytab file for kerberos ticket update

After some time (several hours, I found it hard to track) I experience
the following issues:
- net ads testjoin
      > ads_connect: No logon servers
      > Join to domain is not valid: No logon servers
- wbinfo -g and wbinfo -u
      > provide no output anymore.

What I checked and did not change situation:
- name resolution (forward, backward, all ok to ADS controller as well
as domain name)
- disabled ALL firewall rules between the systems (ADS controller and
member server)




My kerberos configuration on the client looks like this:
[libdefaults]
          default_realm = DOMAIN.DE
          dns_lookup_realm = false # also tried this to set to true
          dns_lookup_kdc = true

[realms]
      DOMAIN.DE = {
          kdc = dc.domain.de
          admin_server = dc.domain.de
          master_kdc = dc.domain.de
      }

[domain_realm]
      domain.de = DOMAIN.DE


This is the smb.conf:
######## GLOBAL
[global]
      #### GLOBAL SETTINGS
      netbios name = HOSTNAME
      server string = HOSTNAME
      workgroup = DOMAIN
      realm = DOMAIN.DE
      server role = MEMBER SERVER
      name resolve order = hosts wins bcast

      #### SECURITY SETTINGS
      security = ads
      allow trusted domains = Yes
      map untrusted to domain = Yes
      encrypt passwords = yes
      client use spnego = yes
      client ntlmv2 auth = yes
      client ldap sasl wrapping = sign
      restrict anonymous = 2
      acl map full control = yes

      #### SERVER SETTINGS
      dns proxy = yes
      domain master = no
      local master = no
      preferred master = no
      os level = 0
      follow symlinks = yes
      veto files
/.AppleDouble/.DS_Store/._.DS_Store/.fseventsd/.notfirsttime/.Spotlight-V100/.TemporaryItems/.Trash/.Trashes/Thumbs.db/thumbs.db/._*/~$*/System\

Volume\ Information/
      delete veto files = yes
      server min protocol = SMB2
      server max protocol = SMB3

      #### KERBEROS
      dedicated keytab file = /etc/krb5.keytab
      kerberos method = secrets and keytab

      #### WINBIND CONFIGURATION
      winbind enum users = yes
      winbind enum groups = yes
      winbind offline logon = no
      winbind reconnect delay = 30
      winbind refresh tickets = yes
      winbind nested groups = yes
      idmap config *:backend = tdb
      idmap config *:range = 70001-80000
      idmap config DOMAIN:backend = rid
      idmap config DOMAIN:schema_mode = rfc2307
      idmap config DOMAIN:range = 20000-40000
      idmap cache time = 604800
      winbind separator = /
      winbind use default domain = no
      #### HOME DIRECTORIES
      template shell = /bin/bash
      template homedir = /home/%U

      #### PRINTING
      disable spoolss = yes
      load printers = no
      idmap_ldb:use rfc2307 = yes

      #### LOGGING
      log level = 2
      username level = 3
      log file = /var/log/samba/log.%m
      max log size = 50
      syslog only = no
      syslog = 2
      panic action = /usr/share/samba/panic-action %d


the resolv.conf:
nameserver 10.14.11.5 # This is the ADS Controller
nameserver 10.14.12.1 # This is an alternate nameserver
search domain.de




In /var/log/syslog I can see various messages that caught my attention
but neither of those helped me in my research. Don´t give to much about
date/time. I just copied them as I found them:

1. "Could not receive Trustdoms".
May 16 06:58:43 hostname winbindd[820]: [2016/05/16 06:58:43.776831,  1]
../source3/winbindd/winbindd_util.c:351(trustdom_list_done)
May 16 06:58:43 hostname winbindd[820]:   Could not receive trustdoms

2. "Check connection to trusted domain"
May 22 06:10:23 hostname winbindd[840]: [2016/05/22 06:10:23.784860,  0]
../source3/winbindd/winbindd_group.c:45(fill_grent)
May 22 06:10:23 hostname winbindd[840]:   Failed to find domain 'Unix
Group'. Check connection to trusted domains!

3. This is indicating a name resolution issue but I have checked that
already:
May 22 06:44:52 hostname winbindd[24623]:   ads_find_dc: name resolution
for realm 'domain.de' (domain 'DOMAIN') failed:
NT_STATUS_NO_LOGON_SERVERS

4. "failed to reconnect (No logon servers)"
May 22 21:09:51 hostname winbindd[971]: [2016/05/22 21:09:51.487192,  1]
../source3/libads/ldap_utils.c:107(ads_do_search_retry_internal)
May 22 21:09:51 hostname winbindd[971]:   ads_search_retry: failed to
reconnect (No logon servers)

5. "ads_connect for domain DOMAIN failed: No logon servers"
May 22 21:10:07 hostname winbindd[971]: [2016/05/22 21:10:07.493461,  1]
../source3/winbindd/winbindd_ads.c:136(ads_cached_connection_connect)
May 22 21:10:07 hostname winbindd[971]:   ads_connect for domain DOMAIN
failed: No logon servers




Any pointers are greatly appreciated.
Best regards

Thomas

Hello Louis, hello Rowland,

I have worked on this topic the last days trying your suggestions, but 
stumbled accidentially over two things.

1. on my DC in /etc/hosts the entry for the FQDN and shortname of the DC 
referred to 127.0.0.1 (which is clearly not what should be there - I 
guess this has forgotten during my reinstall of the ADS controller). I 
changed this to reflect the real IP-Address as mentioned as well in 
https://help.ubuntu.com/community/ActiveDirectoryWinbindHowto .

2. I could not get rid of the feeling that something is wrong with the 
naming resolution and while digging a little deeper into other smb.conf 
options I recognized a typo in my config:
"name resolve order = hosts wins bcast" must not have the
"s" at the end
of the "host" so I changed it to "name resolve order = host wins
bcast"

I can´t say 100% that these two changes solved my problems because 
Ubuntu just released another 4.3.9 Version of their Samba package while 
I was doing the change (and of course I tried installing it) but so far 
it looks promising.

Again thanks for your help. Will keep it in mind if this turns out not 
to be as hoped.
Best regards and have a great weekend

Thomas