samba - Oct 2006 - ADS authentication in disjointed, multiple forest

All,

I have recently come across a problem I have been unable to resolve.  I
have been running Samba with Winbind ADS authentication for the last
couple of years with good success until recently.  About 2 weeks ago, I
restarted the Samba and Winbind services on a Fedora Core 3 machine
runing Samba 3.0.10-1 (installed from FC3 RPMs) and the active directory
authentication has started failing.

The environment is two domains sitting in separate forests: cit.univ.edu
and dept.univ.edu.  (These are separate active directory forests because
that is the environment dictated by the university central IT. 
Cit.univ.edu is also a disjoined domain where all the computers for the
domain sit in the univ.edu DNS zone.)  There is a one way trust so the
dept domain can use the cit domain for user authentication to computer
resources in the dept domain.  (The domain controllers for DEPT.UNIV.EDU
are DCA.DEPT.UNIV.EDU and DCB.DEPT.UNIV.EDU.  The domain controllers for
UNIV.EDU are DC1.UNIV.EDU, DC2.UNIV.EDU, DC3.UNIV.EDU, and
DC4.UNIV.EDU.  The Samba server in the domain DEPT.UNIV.EDU is
SAMBA.DEPT.UNIV.EDU.)

The last time I can verify everything was working correctly was about
two months ago when the Samba server was last rebooted.  The krb5.conf
that was working at the time is

    [logging]
      default = FILE:/var/log/krb5libs.log
      kdc = FILE:/var/log/krb5kdc.log
      admin_server = FILE:/var/log/kadmind.log

    [libdefaults]
      default_realm = DEPT.UNIV.EDU
      dns_lookup_realm = true
      dns_lookup_kdc = true

    [realms]
      DEPT.UNIV.EDU = {
        admin_server = dca.dept.univ.edu
        default_domain = dept.univ.edu
      }
      UNIV.EDU = {
        admin_server = dc1.univ.edu
        kdc = dc1.univ.edu
        default_domain = univ.edu
      }

    [domain_realm]
      .dept.univ.edu = DEPT.UNIV.EDU
      dept.univ.edu = DEPT.UNIV.EDU
      .univ.edu = UNIV.EDU
      univ.edu = UNIV.EDU

    [kdc]
      profile = /var/kerberos/krb5kdc/kdc.conf

    [appdefaults]
      pam = {
         debug = false
         ticket_lifetime = 36000
         renew_lifetime = 36000
         forwardable = true
         krb4_convert = true
      }

And Samba/Winbind was configured with the following:

    security = ads
    realm = DEPT.UNIV.EDU
    password server = *

As of a couple of weeks ago, my Samba servers are still able to connect
to the dept.univ.edu domain but unable to connect to the cit.univ.edu
domain.  The command "wbinfo -g --domain UNIV" now results in a
"Error
looking up domain groups" error and

    [2006/10/12 15:07:58, 1] libsmb/clikrb5.c:ads_krb5_mk_req(390)
      ads_krb5_mk_req: krb5_get_credentials failed for dc1$@CIT.UNIV.EDU
    (Server not found in Kerberos database)
    [2006/10/12 15:07:58, 1]
    libsmb/cliconnect.c:cli_session_setup_kerberos(544)
      spnego_gen_negTokenTarg failed: Server not found in Kerberos database
    [2006/10/12 15:07:58, 1] libsmb/clikrb5.c:ads_krb5_mk_req(390)
      ads_krb5_mk_req: krb5_get_credentials failed for dc1$@CIT.UNIV.EDU
    (Server not found in Kerberos database)
    [2006/10/12 15:07:58, 1] libsmb/clikrb5.c:ads_krb5_mk_req(390)
      ads_krb5_mk_req: krb5_get_credentials failed for dc1$@CIT.UNIV.EDU
    (Server not found in Kerberos database)
    [2006/10/12 15:07:58, 1]
    nsswitch/winbindd_ads.c:ads_cached_connection(81)
      ads_connect for domain UNIV failed: Server not found in Kerberos
    database

in the winbind.log file.

So far, my searches for a resolution to the problem have suggested a
problem in correctly resolving the host name for the domain
controllers.  On the host involved, the gethostbyname function correctly
returns the appropriate IP address for dc1.univ.edu and the reverse
lookup, with gethostbyaddr, of that IP address returns dc1.univ.edu.  (I
don't manage dc1.univ.edu but those who do tell me the machine itself
believes its name is also dc1.univ.edu.  Unfortunately I can't check
dc1.univ.edu to make sure it is correctly resolving dca.dept.univ.edu or
samba.dept.univ.edu.)

The authentication for Windows XP and Windows 2003 machines still works
correctly.  I have also been having this same problem with a new server
on which I installed CentOS 4.4 (with all the updates) using either
Samba 3.0.10-1 from the CentOS repository or 3.0.23c installed from
source downloaded from samba.org.

I have also tried the following krb5.conf with no luck:

    [logging]
     default = FILE:/var/log/krb5libs.log
     kdc = FILE:/var/log/krb5kdc.log
     admin_server = FILE:/var/log/kadmind.log

    [libdefaults]
      default_realm = DEPT.UNIV.EDU
      dns_lookup_realm = false
      dns_lookup_kdc = false

    [realms]
      DEPT.UNIV.EDU = {
        admin_server = dca.dept.univ.edu
        kdc = dca.dept.univ.edu
        kdc = dcb.dept.univ.edu
        default_domain = dept.univ.edu
      }
      UNIV.EDU = {
        admin_server = dc1.univ.edu
        kdc = dc1.univ.edu
        kdc = dc2.univ.edu
        kdc = dc3.univ.edu
        kdc = dc4.univ.edu
        default_domain = cit.univ.edu
      }
      CIT.UNIV.EDU = {
        admin_server = dc1.univ.edu
        kdc = dc1.univ.edu
        kdc = dc2.univ.edu
        kdc = dc3.univ.edu
        kdc = dc4.univ.edu
        default_domain = cit.univ.edu
      }

    [domain_realm]
      .dept.univ.edu = DEPT.UNIV.EDU
      dept.univ.edu = DEPT.UNIV.EDU
      .univ.edu = CIT.UNIV.EDU
      univ.edu = CIT.UNIV.EDU
      .cit.univ.edu = CIT.UNIV.EDU
      cit.univ.edu = CIT.UNIV.EDU
      dc1.univ.edu = CIT.UNIV.EDU
      dc2.univ.edu = CIT.UNIV.EDU
      dc3.univ.edu = CIT.UNIV.EDU
      dc4.univ.edu = CIT.UNIV.EDU

    [kdc]
      profile = /var/kerberos/krb5kdc/kdc.conf

    [appdefaults]
      pam = {
        debug = false
        ticket_lifetime = 36000
        renew_lifetime = 36000
        forwardable = true
        krb4_convert = false
      }

The Fedora Core 3 (existing) machine is running kernel version
2.6.11-1.27_FC3smp, krb5-libs version 1.3.6-5, and Samba version
3.0.10-1 installed. (All installed from FC3 RPMs.)

The CentOS 4.4 (new) machine is running kernel version 2.6.9-42EL,
krb5-libs version 1.3.4-33, and Samba 3.0.10-1.4E9 installed.  (Again,
all installed from CentOS RPMs.  However, as mentioned above, I am also
seeing this problem with Samba 3.0.23c installed from source on this
machine.)

Any suggestions that would point me in the right direction to resolve
this problem would be greatly appreciated.

Thanks,
Scott



-- 
------------------------------------------------------------------------
Scott Parrill 	| 	When the legends die, the dreams end.
Area Technology & Communications Manager 	| 	When the dreams end, there
is no more greatness.
Wyoming GISciences Center 	| 	
University of Wyoming 	| 	
Laramie, WY 82071 	| 	                                When the Legends Die
PH: (307)766-2524 FAX: (307)766-2744 	|
                                Hal Borland
sparrill@uwyo.edu 	|