samba - Jan 2016 - Authentication to Secondary Domain Controller initially fails when PDC is offline

Hi all,

Wish you a happy new year altogether!

Mathias, James, let me first say that I highly appreciate your help with 
all your testing and writing up your thoughts.

Here are my responses:

A. I have no different sites, no various subnets; so I don't really know 
what to do.

B. I don't understand the purpose of setting my domain up with different 
sites with associated networks, if on failure the client looks into 
Default-First-Site-Name, where all DCs are known, and is able to pick a 
working one. Because all my DCs already are in one place (i.e. 
_tcp.Default-First-Site-Name._sites.FQDN)

C. After I followed Rowland's advice to add an NS record for the 2nd DC, 
and after manually creating all the other DNS records for my 2nd DC, 
logon from Windows clients works with the 1st DC offline (as I stated 
earlier). "set" shows logon to 2nd DC, and I get no type 11 login
event
(I get type 2: interactive). This works "right away" - if that term 
allows for ignoring the fact that the first login took 74 seconds 
(according to Windows logs).

D. My remaining problem is that while Windows (7) clients appear to be 
somewhat flexible, this clearly is not the case for my Linux member 
servers. They insist on failing to ssh-login after some 20 or more 
seconds. However, I can 'kinit' now - thanks to the created DNS records,
I suppose.

I wonder, whether this might be due to TTL (time-to-live) settings of 
DNS records, or DNS request timeouts accumulating (and hitting an ssh 
login time threshold?). I noticed that I can change the default TTL 
settings of an SOA - with the effect the the actually TTL of *newly 
created* DNS records is set accordingly. However, to test this, I would 
have to recreate all the DNS records for my DCs again. It is likely I'll 
try this sooner or later.

Ole


Am 23.12.2015 um 16:51 schrieb mathias dufresne:
> Hi all,
>
> Firs I apologize I did not manage to find time to reply earlier.
>
> The initial issue was about how Samba AD react when one DC is out and 
> more specifically about what happen when FSMO ower is unreachable 
> (poweroff in Ole tests).
>
> This issue is solved using a correct AD Sites configuration.
>
> Here I kept 3 DCs in my domain.
> Sites:
> I set up a second site named "authentication" and I've added
in that
> site 2 DCs, including FSMO owner.
> On that "authentication" site I've added the network on which
my
> clients are.
> On "Default-First-Site-Name" I do not configured any network 
> addresses. All 3 DCs are declared in "Default-First-Site-Name".
>
> All DC up behaviour:
> The windows client when connecting ask to AD for a DC, AD answer this 
> client it depends of "authentication" site and this client
re-launch
> DNS requests taking in account AD Site information to retrieve DC list 
> for that site.
> Then the client connects to AD.
>
> To check which AD DC was used to connected on: launch "cmd" then
in
> that window type "set". In "set" result there is a line
> LOGONSERVER=<DCname>
> This is the DC used for connection (and later normally).
>
> The test:
> I powered off both DC in "authentication" site. So only one DC is
up
> and running, the one *outside* of that site.
> Login in Windows works.
> Using cmd -> set -> LOGONSERVER=<the last remaining DC>
>
> I waited something like one hour then I rebooted my MS Windows client. 
> I can still log in using differnt accounts (administrator then my own 
> account).
> This MS Windows client is still using the only running DC, the one 
> outside of client's site, this because this DC is in 
> Default-First-Site-Name.
>
> This behaviour is normal. It is Site behaviour. It is AD Sites purpose.
>
> So configuring AD Sites correctly would solve the issue about failover.
>
>
> DNS issue:
> SOA and NS records are used when DNS servers are discussing together 
> but not when a DNS client is asking for records (execpt for SOA or NS 
> obvisouly).
> SOA and NS records are, in my understanding, used when the DNS server 
> receive a request for a zone this server does not know. In that 
> specific case the DNS server must find a name server for that zone, so 
> our DNS server is asking to upper level for a NS in the mentioned zone.
>
> So, if my understanding is correct, client never uses NS record (not 
> in normal mode). And so there is no much issue that Samba DNS has only 
> one SOA and one NS.
>
> Regarding NS records, I create them manually because it matches our 
> real configuration (each DC is DNS, each DNS as NS record).
>
> The point to make Samba AD works with internal is to work around the 
> issue of samba_dnsupdate. At least it is all I have to do on all 
> domains I tested, even with the 4.3.3.
>
> samba_dnsupdate could work with some option in smb.conf to grant 
> unsecure update of DNS zones.
> If you don't want to allow unsecure update, use the awk script I 
> provided in that thread days (or weeks) ago.
>
> I use Samba AD with Internal DNS with no issue except for 
> samba_dnsupdate which not able to create internal record. To solve 
> that issue I provided here a awk script which extract from 
> samba_dnsupdate needed information to force recrods creation using 
> samba-tool. This awk script is not dangerous: you can run it as much 
> you want, it just tries to create entries. If entry exists, an error 
> is displayed:
> ERROR: Record already exists
>
> If you are afraid of that script you can modify its behaviour replacing:
> cmd = "samba-tool dns add "
> with
> cmd = "echo samba-tool dns add "
>
> Then the script will do nothing except display what command could be 
> run to force DNS records creation.
>
> With all needed DNS records and a well configured AD Sites failover 
> works. Nicely.
>
> A last note: 3 DCs means 3 DNS servers. You want your DC can be down 
> so your clients MUST have all DC declared as "nameserver" in 
> /etc/resolv.conf.
>
> Another strategy is to build one Bind server with one zone configured 
> with the exact same name as AD domain, this zone will do forward only 
> and will forward to all DCs.
> Doing that your clients can have only one DNS configured: the one with 
> Bind forwarding to DCs.
> This bind zone config:
> ------------------
> zone "samba.domain.tld" IN {
>   type forward;
>   forward only;
>   forwarders {
>     IP_DC1;
>     IP_DC2;
>     IP_DC3;
>   };
> };
> -------------------
>
> I hope you will finally be able to have failover working Ole.
>
>
>
> 2015-12-22 11:44 GMT+01:00 Ole Traupe <ole.traupe at tu-berlin.de 
> <mailto:ole.traupe at tu-berlin.de>>:
>
>
>
>                 Can I suggest that you do what I did, create your own
>                 small test domain in VMs using Bind9
>
>
>             Yes, that is a good idea. However, from what I had read
>             before, much of it on the Samba wiki, I was expecting
>             Samba4 to just work with multiple DCs. I still wonder why
>             no one ever seems to have tested or questioned that
>             (publicly). And I don't feel that I have to question
>             something myself that is broadly recommended: use the
>             internal DNS unless you really have to do otherwise (even
>             by the developers, it seems). In addition, bind9 working
>             with multiple DC's does not necessarily mean that internal
>             DNS won't.
>
>
>         I am going to discuss this with Marc and the rest of the team,
>         like you, I am surprised that nobody has raised this before. I
>         have always used Samba with Bind9, so was unaware of this
>         possible problem, it only came to head for me when you
>         mentioned it. I then found I only had one NS  record in the
>         SOA and this lead to where we are now.
>
>
>     Hi Rowland,
>
>     Again: thanks a lot for your support.
>
>     Merry Christmas and good holidays to the list!
>
>     Ole
>
>
>
>             I also feel the need to would like to state that I am a
>             part-time admin and I can't test something for a year or
>             so (like others) before I go into production. With Samba 4
>             I was rather happy to find something that won't require so
>             much work (although it feels differently now, partially
>             due to me being more or less a newbee to unix-based
>             systems, I guess).
>
>
>         It doesn't need much looking after, once you have got it up
>         and running :-)
>
>         Rowland
>
>
>
>     -- 
>     To unsubscribe from this list go to the following URL and read the
>     instructions: https://lists.samba.org/mailman/options/samba
>
>

On 1/4/2016 12:23 PM, Ole Traupe wrote:
> Hi all,
>
> Wish you a happy new year altogether!
>
> Mathias, James, let me first say that I highly appreciate your help 
> with all your testing and writing up your thoughts.
>
> Here are my responses:
>
> A. I have no different sites, no various subnets; so I don't really 
> know what to do.
>
> B. I don't understand the purpose of setting my domain up with 
> different sites with associated networks, if on failure the client 
> looks into Default-First-Site-Name, where all DCs are known, and is 
> able to pick a working one. Because all my DCs already are in one 
> place (i.e. _tcp.Default-First-Site-Name._sites.FQDN)
>
> C. After I followed Rowland's advice to add an NS record for the 2nd 
> DC, and after manually creating all the other DNS records for my 2nd 
> DC, logon from Windows clients works with the 1st DC offline (as I 
> stated earlier). "set" shows logon to 2nd DC, and I get no type
11
> login event (I get type 2: interactive). This works "right away"
- if
> that term allows for ignoring the fact that the first login took 74 
> seconds (according to Windows logs).
>
> D. My remaining problem is that while Windows (7) clients appear to be 
> somewhat flexible, this clearly is not the case for my Linux member 
> servers. They insist on failing to ssh-login after some 20 or more 
> seconds. However, I can 'kinit' now - thanks to the created DNS 
> records, I suppose.
>
> I wonder, whether this might be due to TTL (time-to-live) settings of 
> DNS records, or DNS request timeouts accumulating (and hitting an ssh 
> login time threshold?). I noticed that I can change the default TTL 
> settings of an SOA - with the effect the the actually TTL of *newly 
> created* DNS records is set accordingly. However, to test this, I 
> would have to recreate all the DNS records for my DCs again. It is 
> likely I'll try this sooner or later.
>
> Ole
>
>
> Am 23.12.2015 um 16:51 schrieb mathias dufresne:
>> Hi all,
>>
>> Firs I apologize I did not manage to find time to reply earlier.
>>
>> The initial issue was about how Samba AD react when one DC is out and 
>> more specifically about what happen when FSMO ower is unreachable 
>> (poweroff in Ole tests).
>>
>> This issue is solved using a correct AD Sites configuration.
>>
>> Here I kept 3 DCs in my domain.
>> Sites:
>> I set up a second site named "authentication" and I've
added in that
>> site 2 DCs, including FSMO owner.
>> On that "authentication" site I've added the network on
which my
>> clients are.
>> On "Default-First-Site-Name" I do not configured any network 
>> addresses. All 3 DCs are declared in
"Default-First-Site-Name".
>>
>> All DC up behaviour:
>> The windows client when connecting ask to AD for a DC, AD answer this 
>> client it depends of "authentication" site and this client
re-launch
>> DNS requests taking in account AD Site information to retrieve DC 
>> list for that site.
>> Then the client connects to AD.
>>
>> To check which AD DC was used to connected on: launch "cmd"
then in
>> that window type "set". In "set" result there is a
line
>> LOGONSERVER=<DCname>
>> This is the DC used for connection (and later normally).
>>
>> The test:
>> I powered off both DC in "authentication" site. So only one
DC is up
>> and running, the one *outside* of that site.
>> Login in Windows works.
>> Using cmd -> set -> LOGONSERVER=<the last remaining DC>
>>
>> I waited something like one hour then I rebooted my MS Windows 
>> client. I can still log in using differnt accounts (administrator 
>> then my own account).
>> This MS Windows client is still using the only running DC, the one 
>> outside of client's site, this because this DC is in 
>> Default-First-Site-Name.
>>
>> This behaviour is normal. It is Site behaviour. It is AD Sites purpose.
>>
>> So configuring AD Sites correctly would solve the issue about failover.
>>
>>
>> DNS issue:
>> SOA and NS records are used when DNS servers are discussing together 
>> but not when a DNS client is asking for records (execpt for SOA or NS 
>> obvisouly).
>> SOA and NS records are, in my understanding, used when the DNS server 
>> receive a request for a zone this server does not know. In that 
>> specific case the DNS server must find a name server for that zone, 
>> so our DNS server is asking to upper level for a NS in the mentioned 
>> zone.
>>
>> So, if my understanding is correct, client never uses NS record (not 
>> in normal mode). And so there is no much issue that Samba DNS has 
>> only one SOA and one NS.
>>
>> Regarding NS records, I create them manually because it matches our 
>> real configuration (each DC is DNS, each DNS as NS record).
>>
>> The point to make Samba AD works with internal is to work around the 
>> issue of samba_dnsupdate. At least it is all I have to do on all 
>> domains I tested, even with the 4.3.3.
>>
>> samba_dnsupdate could work with some option in smb.conf to grant 
>> unsecure update of DNS zones.
>> If you don't want to allow unsecure update, use the awk script I 
>> provided in that thread days (or weeks) ago.
>>
>> I use Samba AD with Internal DNS with no issue except for 
>> samba_dnsupdate which not able to create internal record. To solve 
>> that issue I provided here a awk script which extract from 
>> samba_dnsupdate needed information to force recrods creation using 
>> samba-tool. This awk script is not dangerous: you can run it as much 
>> you want, it just tries to create entries. If entry exists, an error 
>> is displayed:
>> ERROR: Record already exists
>>
>> If you are afraid of that script you can modify its behaviour
replacing:
>> cmd = "samba-tool dns add "
>> with
>> cmd = "echo samba-tool dns add "
>>
>> Then the script will do nothing except display what command could be 
>> run to force DNS records creation.
>>
>> With all needed DNS records and a well configured AD Sites failover 
>> works. Nicely.
>>
>> A last note: 3 DCs means 3 DNS servers. You want your DC can be down 
>> so your clients MUST have all DC declared as "nameserver" in 
>> /etc/resolv.conf.
>>
>> Another strategy is to build one Bind server with one zone configured 
>> with the exact same name as AD domain, this zone will do forward only 
>> and will forward to all DCs.
>> Doing that your clients can have only one DNS configured: the one 
>> with Bind forwarding to DCs.
>> This bind zone config:
>> ------------------
>> zone "samba.domain.tld" IN {
>>   type forward;
>>   forward only;
>>   forwarders {
>>     IP_DC1;
>>     IP_DC2;
>>     IP_DC3;
>>   };
>> };
>> -------------------
>>
>> I hope you will finally be able to have failover working Ole.
>>
>>
>>
>> 2015-12-22 11:44 GMT+01:00 Ole Traupe <ole.traupe at tu-berlin.de 
>> <mailto:ole.traupe at tu-berlin.de>>:
>>
>>
>>
>>                 Can I suggest that you do what I did, create your own
>>                 small test domain in VMs using Bind9
>>
>>
>>             Yes, that is a good idea. However, from what I had read
>>             before, much of it on the Samba wiki, I was expecting
>>             Samba4 to just work with multiple DCs. I still wonder why
>>             no one ever seems to have tested or questioned that
>>             (publicly). And I don't feel that I have to question
>>             something myself that is broadly recommended: use the
>>             internal DNS unless you really have to do otherwise (even
>>             by the developers, it seems). In addition, bind9 working
>>             with multiple DC's does not necessarily mean that
internal
>>             DNS won't.
>>
>>
>>         I am going to discuss this with Marc and the rest of the team,
>>         like you, I am surprised that nobody has raised this before. I
>>         have always used Samba with Bind9, so was unaware of this
>>         possible problem, it only came to head for me when you
>>         mentioned it. I then found I only had one NS  record in the
>>         SOA and this lead to where we are now.
>>
>>
>>     Hi Rowland,
>>
>>     Again: thanks a lot for your support.
>>
>>     Merry Christmas and good holidays to the list!
>>
>>     Ole
>>
>>
>>
>>             I also feel the need to would like to state that I am a
>>             part-time admin and I can't test something for a year
or
>>             so (like others) before I go into production. With Samba 4
>>             I was rather happy to find something that won't require
so
>>             much work (although it feels differently now, partially
>>             due to me being more or less a newbee to unix-based
>>             systems, I guess).
>>
>>
>>         It doesn't need much looking after, once you have got it up
>>         and running :-)
>>
>>         Rowland
>>
>>
>>
>>     --     To unsubscribe from this list go to the following URL and 
>> read the
>>     instructions: https://lists.samba.org/mailman/options/samba
>>
>>
>
Ole,

     I can't recall but are you able to get a packet trace? This may 
help further troubleshoot.

Just to recap you do you both servers listed as available DNS servers on 
your workstations? As well as your member server? I made a small tweak 
but haven't fully tested is adding the following options to my resolv.conf.

cat /etc/resolvconf/resolv.conf.d/tail
options timeout:1
options edns0

timeout:n
                      sets the amount of time the resolver will wait for 
a response from a remote name server before retrying  the query  via  a  
different  name
                      server.  Measured in seconds, the default is 
RES_TIMEOUT (currently 5, see <resolv.h>).  The value for this option is 
silently capped to 30.

edns0 (since glibc 2.6)
                      sets RES_USE_EDNSO in _res.options.  This enables 
support for the DNS extensions described in RFC 2671.

 From what I researched, this is the intended behavior on a Microsoft 
Server. Again I can disable my "PDC" and log in from a windows 
workstation just fine. It appears for some users after a hour or so they 
run into issues.



-- 
-James

>
>     I can't recall but are you able to get a packet trace? This may 
> help further troubleshoot.
I'll look into this. However, Rowland stated that bind9 will be the only 
solution.

>
> Just to recap you do you both servers listed as available DNS servers 
> on your workstations? As well as your member server? 
Yes, of course. For member servers, this is the content of /etc/resolv.conf:

search my.domain.tld
nameserver IP_of_1st_DC
nameserver IP_of_2nd_DC

> I made a small tweak but haven't fully tested is adding the following 
> options to my resolv.conf.
>
> cat /etc/resolvconf/resolv.conf.d/tail
> options timeout:1
Great, this sounds exactly as what I need! However, I tried this: no 
effect. I created this file and restarted the network service. But I 
still get long timeouts and can't login via ssh, when I suspend my 1st DC.

# cat /etc/resolvconf/resolv.conf.d/tail
options timeout:1
options edns0

Or do I need Network Manager for that?

> options edns0
What's that for, particularly?

>
> timeout:n
>                      sets the amount of time the resolver will wait 
> for a response from a remote name server before retrying  the query  
> via  a  different  name
>                      server.  Measured in seconds, the default is 
> RES_TIMEOUT (currently 5, see <resolv.h>).  The value for this option
> is silently capped to 30.
>
> edns0 (since glibc 2.6)
>                      sets RES_USE_EDNSO in _res.options.  This enables 
> support for the DNS extensions described in RFC 2671.
>
> From what I researched, this is the intended behavior on a Microsoft 
> Server. Again I can disable my "PDC" and log in from a windows 
> workstation just fine. It appears for some users after a hour or so 
> they run into issues 
I thought this was only happening with roaming machines resulting in 
cached logins.

For the member servers, to reduce timeouts etc when one DC is down.

Change your resolv.conf to : 
domain internal.domain.tld
search internal.domain.tld

nameserver IP_DC1
nameserver IP_DC2

options timeout:2
options attempts:2
options rotate
options edns0

see man resolv.conf for the options explained. 

Ow.. and .. 

domain and search are NOT exclusive anymore in Debian Jessie and up. 
At least, i didnt find it anymore. 

Greetz, 

Louis


> -----Oorspronkelijk bericht-----
> Van: samba [mailto:samba-bounces at lists.samba.org] Namens Ole Traupe
> Verzonden: dinsdag 5 januari 2016 12:30
> Aan: samba at lists.samba.org
> Onderwerp: Re: [Samba] Authentication to Secondary Domain Controller
> initially fails when PDC is offline
> 
> 
> >
> >     I can't recall but are you able to get a packet trace? This
may
> > help further troubleshoot.
> 
> I'll look into this. However, Rowland stated that bind9 will be the
only
> solution.
> 
> 
> >
> > Just to recap you do you both servers listed as available DNS servers
> > on your workstations? As well as your member server?
> 
> Yes, of course. For member servers, this is the content of
> /etc/resolv.conf:
> 
> search my.domain.tld
> nameserver IP_of_1st_DC
> nameserver IP_of_2nd_DC
> 
> 
> > I made a small tweak but haven't fully tested is adding the
following
> > options to my resolv.conf.
> >
> > cat /etc/resolvconf/resolv.conf.d/tail
> > options timeout:1
> 
> Great, this sounds exactly as what I need! However, I tried this: no
> effect. I created this file and restarted the network service. But I
> still get long timeouts and can't login via ssh, when I suspend my 1st
DC.
> 
> # cat /etc/resolvconf/resolv.conf.d/tail
> options timeout:1
> options edns0
> 
> Or do I need Network Manager for that?
> 
> 
> > options edns0
> 
> What's that for, particularly?
> 
> 
> >
> > timeout:n
> >                      sets the amount of time the resolver will wait
> > for a response from a remote name server before retrying  the query
> > via  a  different  name
> >                      server.  Measured in seconds, the default is
> > RES_TIMEOUT (currently 5, see <resolv.h>).  The value for this
option
> > is silently capped to 30.
> >
> > edns0 (since glibc 2.6)
> >                      sets RES_USE_EDNSO in _res.options.  This enables
> > support for the DNS extensions described in RFC 2671.
> >
> > From what I researched, this is the intended behavior on a Microsoft
> > Server. Again I can disable my "PDC" and log in from a
windows
> > workstation just fine. It appears for some users after a hour or so
> > they run into issues
> 
> I thought this was only happening with roaming machines resulting in
> cached logins.
> 
> 
> 
> --
> To unsubscribe from this list go to the following URL and read the
> instructions:  https://lists.samba.org/mailman/options/samba