samba - Nov 2015 - Authentication to Secondary Domain Controller initially fails when PDC is offline

Hi,

I tested the AD (Samba4) domain log-in on Windows 7 clients and Linux 
member servers with my PDC being offline (plugged the cable). It is not 
working so well.

On Windows it initially takes forever. It works again after rebooting 
the client, which seems to be the easiest solution (can be performed by 
the user).

On Linux member servers, ssh log-in eventually times out. It works 
again, after I manually swap the DNS server order in the 
/etc/resolv.conf and the KDC provider order in the /etc/krb5.conf. But 
manual intervention is clearly not preferred here.

According to the sanity checks for domain controllers and members 
servers on the wiki setup and troubleshooting pages, my domain is 
working at its best.

Is this due to DNS and kerberos timeouts accumulating? What is the best 
way of dealing with this?

Best,
Ole

On 11/11/15 15:20, Ole Traupe wrote:
> Hi,
>
> I tested the AD (Samba4) domain log-in on Windows 7 clients and Linux 
> member servers with my PDC being offline (plugged the cable). It is 
> not working so well.
>
> On Windows it initially takes forever. It works again after rebooting 
> the client, which seems to be the easiest solution (can be performed 
> by the user).
>
> On Linux member servers, ssh log-in eventually times out. It works 
> again, after I manually swap the DNS server order in the 
> /etc/resolv.conf and the KDC provider order in the /etc/krb5.conf. But 
> manual intervention is clearly not preferred here.
What have you got in /etc/resolv.conf on your first DC (please don't 
call it a PDC) , your second DC and a Unix client.

Your /etc/krb5.conf only needs to look like this:

libdefaults]
         default_realm = SAMDOM.EXAMPLE.COM
         dns_lookup_realm = false
         dns_lookup_kdc = true

DNS should find your DCs

Are you running ntp on all the Unix machines?

Rowland
>
> According to the sanity checks for domain controllers and members 
> servers on the wiki setup and troubleshooting pages, my domain is 
> working at its best.
>
> Is this due to DNS and kerberos timeouts accumulating? What is the 
> best way of dealing with this?
>
> Best,
> Ole
>
>
>

Am 11.11.2015 um 17:05 schrieb Rowland Penny:
> On 11/11/15 15:20, Ole Traupe wrote:
>> Hi,
>>
>> I tested the AD (Samba4) domain log-in on Windows 7 clients and Linux 
>> member servers with my PDC being offline (plugged the cable). It is 
>> not working so well.
>>
>> On Windows it initially takes forever. It works again after rebooting 
>> the client, which seems to be the easiest solution (can be performed 
>> by the user).
>>
>> On Linux member servers, ssh log-in eventually times out. It works 
>> again, after I manually swap the DNS server order in the 
>> /etc/resolv.conf and the KDC provider order in the /etc/krb5.conf. 
>> But manual intervention is clearly not preferred here.
>
> What have you got in /etc/resolv.conf on your first DC (please don't 
> call it a PDC) , your second DC and a Unix client.
My resolv.conf files are "crossed":

# First_DC:
nameserver IP_OF_SECOND_DC
nameserver IP_OF_FIRST_DC
search my.domain.com

# Second_DC _AND_ member servers:
nameserver IP_OF_FIRST_DC
nameserver IP_OF_SECOND_DC
search my.domain.com

>
> Your /etc/krb5.conf only needs to look like this:
>
> libdefaults]
>         default_realm = SAMDOM.EXAMPLE.COM
>         dns_lookup_realm = false
>         dns_lookup_kdc = true
It is, on the DCs. On the member server it is like this:

[logging]
  default = FILE:/var/log/krb5libs.log
  kdc = FILE:/var/log/krb5kdc.log
  admin_server = FILE:/var/log/kadmind.log

[libdefaults]
  default_realm = MY.DOMAIN.COM
  dns_lookup_realm = false
  dns_lookup_kdc = true
  ticket_lifetime = 24h
  renew_lifetime = 7d
  forwardable = true

[realms]
  MY.DOMAIN.COM = {
   kdc = first_dc.my.domain.com
   kdc = second_dc.my.domain.com
   admin_server = first_dc.my.domain.com
   default_domain = my.domain.com
  }

[domain_realm]
  my.domain.com = MY.DOMAIN.COM
  .my.domain.com = MY.DOMAIN.COM

If the First_DC is online, it is working perfectly.

The above "swapping" of the config lines was meant for the member 
server. Without swapping the lines in the resolv.conf I can ping the 
Second_DC (if the First_DC is offline), but it takes 5+ seconds before I 
get a response (DNS related)?. So I figured the issue might be a too 
long timeout.

I am running ntp on all linux machines, and my time is in sync.

Thanks for your help, Rowland!

>
> DNS should find your DCs
>
> Are you running ntp on all the Unix machines?
>
> Rowland
>
>>
>> According to the sanity checks for domain controllers and members 
>> servers on the wiki setup and troubleshooting pages, my domain is 
>> working at its best.
>>
>> Is this due to DNS and kerberos timeouts accumulating? What is the 
>> best way of dealing with this?
>>
>> Best,
>> Ole
>>
>>
>>
>
>

On 11:06:29 wrote Ole Traupe:
> Hi,
> 
> I tested the AD (Samba4) domain log-in on Windows 7 clients and Linux
> member servers with my PDC being offline (plugged the cable). It is
> not working so well.
> 
> On Windows it initially takes forever. It works again after rebooting
> the client, which seems to be the easiest solution (can be performed
> by the user).
> 
> On Linux member servers, ssh log-in eventually times out. It works
> again, after I manually swap the DNS server order in the
> /etc/resolv.conf and the KDC provider order in the /etc/krb5.conf.
> But manual intervention is clearly not preferred here.
> 
> According to the sanity checks for domain controllers and members
> servers on the wiki setup and troubleshooting pages, my domain is
> working at its best.
> 
> Is this due to DNS and kerberos timeouts accumulating?
It is DNS related.
> What is the best way of dealing with this?
The *best way* is a HA solution for your DNS Servers, but its expensive.

The DNS client (resolver) caches the srv records for 15 minutes aka 900 
seconds.

ipconfig /flushdns drops the cache. Reboot does the same.

On server side you may set shorter TTL for the server records, but then 
you have more DNS traffic. On small netwoks (sites up to 20 clients, no 
wifi) I have good experience with a TTL of 180.
 
> Best,
> Ole

-- 

Gruss
	Harry Jede

Am 12.11.2015 um 11:22 schrieb Harry Jede:
> On 11:06:29 wrote Ole Traupe:
>> Hi,
>>
>> I tested the AD (Samba4) domain log-in on Windows 7 clients and Linux
>> member servers with my PDC being offline (plugged the cable). It is
>> not working so well.
>>
>> On Windows it initially takes forever. It works again after rebooting
>> the client, which seems to be the easiest solution (can be performed
>> by the user).
>>
>> On Linux member servers, ssh log-in eventually times out. It works
>> again, after I manually swap the DNS server order in the
>> /etc/resolv.conf and the KDC provider order in the /etc/krb5.conf.
>> But manual intervention is clearly not preferred here.
>>
>> According to the sanity checks for domain controllers and members
>> servers on the wiki setup and troubleshooting pages, my domain is
>> working at its best.
>>
>> Is this due to DNS and kerberos timeouts accumulating?
> It is DNS related.
>
>> What is the best way of dealing with this?
> The *best way* is a HA solution for your DNS Servers, but its expensive.
>
> The DNS client (resolver) caches the srv records for 15 minutes aka 900
> seconds.
>
> ipconfig /flushdns drops the cache. Reboot does the same.
Will try this, thank you!
>
> On server side you may set shorter TTL for the server records, but then
> you have more DNS traffic. On small netwoks (sites up to 20 clients, no
> wifi) I have good experience with a TTL of 180.
Ok. So I do this on my Samba DCs (my domain DNS servers), and this will 
affect Windows and Linux domain clients/member servers likewise?

> It is DNS related.
>
>> What is the best way of dealing with this?
> The *best way* is a HA solution for your DNS Servers, but its expensive.
>
> The DNS client (resolver) caches the srv records for 15 minutes aka 900
> seconds.
>
> ipconfig /flushdns drops the cache. Reboot does the same.
>
> On server side you may set shorter TTL for the server records, but then
> you have more DNS traffic. On small netwoks (sites up to 20 clients, no
> wifi) I have good experience with a TTL of 180.
Harry, I tried this - unsuccessfully.

I have TTL settings in a) the SOA and b) the NS record of the FQDN and 
the _msdcs.FQDN sections in my Windows RSAT DNS console. None of these 4 
entries I can change: I get something like "The Source Of Authority 
(SOA) cannot be updated. The record already exists."

Do you have an idea how to accomplish this? Currently the setting is 1h, 
which is pretty long.

Ole

> Hi, If you can bear with me, I am trying to get the join to add the NS 
> for the joining DC to the SOA, I believe I may be near to get this 
> working (after leading myself down the garden path, what I tried 
> previously, didn't work), once it does, I should be able answer your 
> question, my test domain is using the internal dns.
>
> Rowland

I am happy to hear that and hope that solves the problem! I have tested 
fail-over now with the new NS record, but the situation is more or less 
the same:

- created the NS record and waited until I found the record to be replicated
- restarted the windows machine I wanted to test this on
- suspended the 1st DC (currently a VM)
- tried to log-on to the windows test machine
- results:

1. first log-on for a user takes ~30 seconds (on a second test it was up 
to 60 s)
2. following second log-on takes only 5 s
3. third log-on takes 2-3 s

Confirmed this with a second user, the same time-out pattern. Seems to 
me that Windows 7 keeps its default DC but is willing to make exceptions 
on a user basis?

However, I cannot say whether this actually is a server authentication 
or an offline log-on. I looked into the Windows logs ("Security") but 
didn't find anything conclusive.


Two other things to mention:

- From Windows, I can access my home and other network shares (located 
on a Samba 4 member server) as usual with out any problem (which is good!!)

- But when I try to ssh to a member server, it still takes forever, and 
a 'kinit' on a member server gives this:
   "kinit: Cannot contact any KDC for realm 'MY.DOMAIN.TLD' while 
getting initial credentials"


My /etc/krb5.conf looks like this (following your suggestions, Rowland, 
as everything else are defaults):

[libdefaults]
  default_realm = MY.DOMAIN.TLD

And my /etc/resolv.conf is this:

search my.domain.tld
nameserver IP_of_1st_DC
nameserver IP_of_2nd_DC


So from a Windows client point of view, I am more or less fine (even 
without restarting the machines). But it would be great if I could 
log-in to the Linux member servers as well.

Ole

On 04/12/15 16:20, Ole Traupe wrote:
>
>> Hi, If you can bear with me, I am trying to get the join to add the 
>> NS for the joining DC to the SOA, I believe I may be near to get this 
>> working (after leading myself down the garden path, what I tried 
>> previously, didn't work), once it does, I should be able answer
your
>> question, my test domain is using the internal dns.
>>
>> Rowland
>
>
> I am happy to hear that and hope that solves the problem! I have 
> tested fail-over now with the new NS record, but the situation is more 
> or less the same:
>
> - created the NS record and waited until I found the record to be 
> replicated
> - restarted the windows machine I wanted to test this on
> - suspended the 1st DC (currently a VM)
> - tried to log-on to the windows test machine
> - results:
>
> 1. first log-on for a user takes ~30 seconds (on a second test it was 
> up to 60 s)
> 2. following second log-on takes only 5 s
> 3. third log-on takes 2-3 s
>
> Confirmed this with a second user, the same time-out pattern. Seems to 
> me that Windows 7 keeps its default DC but is willing to make 
> exceptions on a user basis?
>
> However, I cannot say whether this actually is a server authentication 
> or an offline log-on. I looked into the Windows logs ("Security")
but
> didn't find anything conclusive.
>
>
> Two other things to mention:
>
> - From Windows, I can access my home and other network shares (located 
> on a Samba 4 member server) as usual with out any problem (which is 
> good!!)
>
> - But when I try to ssh to a member server, it still takes forever, 
> and a 'kinit' on a member server gives this:
>   "kinit: Cannot contact any KDC for realm 'MY.DOMAIN.TLD'
while
> getting initial credentials"
>
>
> My /etc/krb5.conf looks like this (following your suggestions, 
> Rowland, as everything else are defaults):
>
> [libdefaults]
>  default_realm = MY.DOMAIN.TLD
>
> And my /etc/resolv.conf is this:
>
> search my.domain.tld
> nameserver IP_of_1st_DC
> nameserver IP_of_2nd_DC
>
>
> So from a Windows client point of view, I am more or less fine (even 
> without restarting the machines). But it would be great if I could 
> log-in to the Linux member servers as well.
>
> Ole
>
>
>
I am getting nearer, I can now add another NS record to the SOA whilst 
joining a DC, it's the wrong record, but it was added :-D

Now to get it to add the correct NS record (after I figure out just 
where I went wrong).

Rowland

To check which DC was used to connect on simply type "set" in MSDOS
console
(cmd). Then look for a line which contain a DC name.

For Windows they should try to find a DC at logon time, according to their
IP address and AD sites configuration as explained earlier I think. This
process includes DNS SRV request to find LDAP server list and then LDAP
requests are sent to received SRV to find one working server, something
like one replying the quicker (that's a foggy notion for me :)

For Linux and kinit that should be based on DNS resolution and caching if
some. Now how kinit chose a Kerberos server from DNS I no real idea.
It is possible to force usage of one particular kerberos server forcing it
in some configuration file and then using that file in $KRB5_CONFIG
environment variable. At least you could use that to test if kinit works
when forced on the remaining server.

But that does not answer the question of failover for Linux parts :(


2015-12-04 17:20 GMT+01:00 Ole Traupe <ole.traupe at tu-berlin.de>:
>
> Hi, If you can bear with me, I am trying to get the join to add the NS for
>> the joining DC to the SOA, I believe I may be near to get this working
>> (after leading myself down the garden path, what I tried previously,
didn't
>> work), once it does, I should be able answer your question, my test
domain
>> is using the internal dns.
>>
>> Rowland
>>
>
>
> I am happy to hear that and hope that solves the problem! I have tested
> fail-over now with the new NS record, but the situation is more or less the
> same:
>
> - created the NS record and waited until I found the record to be
> replicated
> - restarted the windows machine I wanted to test this on
> - suspended the 1st DC (currently a VM)
> - tried to log-on to the windows test machine
> - results:
>
> 1. first log-on for a user takes ~30 seconds (on a second test it was up
> to 60 s)
> 2. following second log-on takes only 5 s
> 3. third log-on takes 2-3 s
>
> Confirmed this with a second user, the same time-out pattern. Seems to me
> that Windows 7 keeps its default DC but is willing to make exceptions on a
> user basis?
>
> However, I cannot say whether this actually is a server authentication or
> an offline log-on. I looked into the Windows logs ("Security")
but didn't
> find anything conclusive.
>
>
> Two other things to mention:
>
> - From Windows, I can access my home and other network shares (located on
> a Samba 4 member server) as usual with out any problem (which is good!!)
>
> - But when I try to ssh to a member server, it still takes forever, and a
> 'kinit' on a member server gives this:
>   "kinit: Cannot contact any KDC for realm 'MY.DOMAIN.TLD'
while getting
> initial credentials"
>
>
> My /etc/krb5.conf looks like this (following your suggestions, Rowland, as
> everything else are defaults):
>
> [libdefaults]
>  default_realm = MY.DOMAIN.TLD
>
> And my /etc/resolv.conf is this:
>
> search my.domain.tld
> nameserver IP_of_1st_DC
> nameserver IP_of_2nd_DC
>
>
> So from a Windows client point of view, I am more or less fine (even
> without restarting the machines). But it would be great if I could log-in
> to the Linux member servers as well.
>
>
> Ole
>
>
>
> --
> To unsubscribe from this list go to the following URL and read the
> instructions:  https://lists.samba.org/mailman/options/samba
>

> - But when I try to ssh to a member server, it still takes forever, 
> and a 'kinit' on a member server gives this:
>   "kinit: Cannot contact any KDC for realm 'MY.DOMAIN.TLD'
while
> getting initial credentials"
>
>
> My /etc/krb5.conf looks like this (following your suggestions, 
> Rowland, as everything else are defaults):
>
> [libdefaults]
>  default_realm = MY.DOMAIN.TLD
>
> And my /etc/resolv.conf is this:
>
> search my.domain.tld
> nameserver IP_of_1st_DC
> nameserver IP_of_2nd_DC
Any idea why I still get this when trying to log on to a member server 
while the first DC is down?

# kinit: Cannot contact any KDC for realm 'MY.DOMAIN.TLD' while getting 
initial credentials

Ole

Hai Ole, 

Can you run on the member where you logged in. 

host -t SRV _ldap._tcp.samdom.example.com.
host -t SRV _kerberos._udp.samdom.example.com. 

host -t A dc1.samdom.example.com.
host -t A dc2.samdom.example.com.

and again with 
search my.domain.tld
nameserver IP_of_2st_DC
nameserver IP_of_1nd_DC

looks ok to me sofare. 

Greetz, 

Louis

> -----Oorspronkelijk bericht-----
> Van: samba [mailto:samba-bounces at lists.samba.org] Namens Ole Traupe
> Verzonden: woensdag 9 december 2015 17:33
> Aan: samba at lists.samba.org
> Onderwerp: Re: [Samba] Authentication to Secondary Domain Controller
> initially fails when PDC is offline
> 
> 
> > - But when I try to ssh to a member server, it still takes forever,
> > and a 'kinit' on a member server gives this:
> >   "kinit: Cannot contact any KDC for realm
'MY.DOMAIN.TLD' while
> > getting initial credentials"
> >
> >
> > My /etc/krb5.conf looks like this (following your suggestions,
> > Rowland, as everything else are defaults):
> >
> > [libdefaults]
> >  default_realm = MY.DOMAIN.TLD
> >
> > And my /etc/resolv.conf is this:
> >
> > search my.domain.tld
> > nameserver IP_of_1st_DC
> > nameserver IP_of_2nd_DC
> 
> Any idea why I still get this when trying to log on to a member server
> while the first DC is down?
> 
> # kinit: Cannot contact any KDC for realm 'MY.DOMAIN.TLD' while
getting
> initial credentials
> 
> Ole
> 
> 
> 
> --
> To unsubscribe from this list go to the following URL and read the
> instructions:  https://lists.samba.org/mailman/options/samba