On 7/25/19 4:07 PM, Giles Coochey wrote:> > On 25/07/2019 13:51, hw wrote: >> Hi, >> >> how can DNS reliability, as experienced by clients on the LAN who are >> sending queries, be increased? >> >> Would I have to set up some sort of cluster consisting of several >> servers all providing DNS services which is reachable under a single >> IP address known to the clients? >> >> Just setting up several name servers and making them known to the clients >> for the clients to automatically switch isn't a good solution because >> the clients take their timeouts and users lacking even the most basic >> knowledge inevitably panic when the first name server does not answer >> queries. > > Sounds like you're performing maintenance on your servers > > (a) too often > (b) during office / peak hoursI can't help it when the primary name server goes down because the UPS fails the self test and tells the server it has 2 minutes or so left in wich case the server figures it needs to shut down. I wanted better UPSs ...> You could load balance multiple servers (using lots of available load-balancing technologies) to allow you to perform maintenance at certain times, but it has its own issues.Load balancing or clustering? At least clustering seems not entirely trivial to do.> I've recently been looking at PowerDNS, which separates the recursor and the authoritative server into two distinct packages. I'm just running the authoritative server as a master, and keeping my old bind/named servers as recursors / slaves.This can be done with bind, how does it require something called PowerDNS?> It's a home > office network, but I only have issues when I'm tinkering, and if I were to be doing this kind of work in a larger commercial environment, then I would not be doing DNS server maintenance while others were relying on them.The maintenance didn't cause any problems. You can edit the configuration just fine and restart the server when done ... :)> For much of the back end infrastructure I use IP addresses rather than DNS names in their configuration, just to take DNS issues out of the equation completely.I think this is a very bad idea because it causes lots of work and is likely to cause issues. What if you, for example, migrate remote logging to another server? All the time, you have to document every place where you put an IP address; you have to keep the documentation always up to date and then change the address at every place when you make a change. Forget one place, and things break. But when you use names instead of addresses, like 'log.example.com', you only need to make a single change at a single place such as you alter the address in your name server config. DNS can be difficult to get right, though it's not all that difficult, and once it's working, there aren't really any issues other than that a server can become unreachable.
hw wrote:> On 7/25/19 4:07 PM, Giles Coochey wrote:<snip>>> Sounds like you're performing maintenance on your servers >> >> >> (a) too often >> (b) during office / peak hours >> > > I can't help it when the primary name server goes down because the UPS > fails the self test and tells the server it has 2 minutes or so left in > wich case the server figures it needs to shut down. I wanted better UPSs > ...<snip> Change that. Are you using apcupsd? You can set the config from SHUTDOWN=/sbin/shutdown to /bin/false. Then, the next time you see the UPS, change the battery. If it's just started to complain, it's not dead yet! Works for me with all of our mostly APC SmartUPS 3000 rackmounts. mark
On Thu, Jul 25, 2019 at 10:32 AM hw <hw at gc-24.de> wrote:> I can't help it when the primary name server goes down because the UPS > fails > the self test and tells the server it has 2 minutes or so left in wich case > the server figures it needs to shut down. I wanted better UPSs ... >critical infrastructure servers should have redudant PSUs, on seperate UPSs. my last rack builds, I had 2 Eaton PowerWare 7KVA 4U UPS's in the bottom of each rack. one fed the left side PDUs, the other fed the right side PDUs, and each server had redundant PSU's, one plugged into each PDU. those Eaton UPS's had hotswappable batteries, too. -- -john r pierce recycling used bits in santa cruz
John Pierce wrote:> On Thu, Jul 25, 2019 at 10:32 AM hw <hw at gc-24.de> wrote: > >> I can't help it when the primary name server goes down because the UPS >> fails the self test and tells the server it has 2 minutes or so left in >> wich case the server figures it needs to shut down. I wanted better >> UPSs ... >> > critical infrastructure servers should have redudant PSUs, on seperate > UPSs. > > my last rack builds, I had 2 Eaton PowerWare 7KVA 4U UPS's in the bottom > of each rack. one fed the left side PDUs, the other fed the right side > PDUs, and each server had redundant PSU's, one plugged into each PDU. > those Eaton UPS's had hotswappable batteries, too.*shrug* All UPSes have hot-swappable. Mine beep while you disconnect the batteries, pull out the sled, replace all 8, shove it back in, and reconnect, and it shuts up. For those that haven't done it, though, DO NOT BELIEVE WHAT ANYONE SAYS, DO NOT USE *ANYTHING* BUT HR (high rate) batteries in a UPS (maybe in a home one, but...). APC, for example, simply stays red, and insists that you still need to change them. *Good* battery vendors know this.
On 7/25/19 9:11 PM, mark wrote:> hw wrote: >> On 7/25/19 4:07 PM, Giles Coochey wrote: > <snip> >>> Sounds like you're performing maintenance on your servers >>> >>> >>> (a) too often >>> (b) during office / peak hours >>> >> >> I can't help it when the primary name server goes down because the UPS >> fails the self test and tells the server it has 2 minutes or so left in >> wich case the server figures it needs to shut down. I wanted better UPSs >> ... > <snip> > Change that. Are you using apcupsd? You can set the config from > SHUTDOWN=/sbin/shutdown to /bin/false. Then, the next time you see the > UPS, change the battery. If it's just started to complain, it's not dead > yet! > > Works for me with all of our mostly APC SmartUPS 3000 rackmounts.I don't remember which UPS it was, either the crappy one for which a replacement battery was already waiting to be put in, or the normal one that already had a new battery in it which is either broken or doesn't get charged ... That's how I rather have not everything go dark even when Murphy comes along. I have generally deprecated all non-rackmount UPSs, and being able to change batteries without outage has become a requirement.
On 7/25/19 9:39 PM, John Pierce wrote:> On Thu, Jul 25, 2019 at 10:32 AM hw <hw at gc-24.de> wrote: > >> I can't help it when the primary name server goes down because the UPS >> fails >> the self test and tells the server it has 2 minutes or so left in wich case >> the server figures it needs to shut down. I wanted better UPSs ... >> > > critical infrastructure servers should have redudant PSUs, on seperate UPSs.right, with hot swappable batteries ...> my last rack builds, I had 2 Eaton PowerWare 7KVA 4U UPS's in the bottom of > each rack. one fed the left side PDUs, the other fed the right side PDUs, > and each server had redundant PSU's, one plugged into each PDU. > > those Eaton UPS's had hotswappable batteries, too.... like this
On 25/07/2019 20:39, John Pierce wrote:> On Thu, Jul 25, 2019 at 10:32 AM hw <hw at gc-24.de> wrote: > >> I can't help it when the primary name server goes down because the UPS >> fails >> the self test and tells the server it has 2 minutes or so left in wich case >> the server figures it needs to shut down. I wanted better UPSs ... >> > critical infrastructure servers should have redudant PSUs, on seperate UPSs.Separate DNS servers must be on a different subnet according to RFC2182 (https://tools.ietf.org/html/rfc2182): Secondary servers must be placed at both topologically and geographically dispersed locations on the Internet, to minimise the likelihood of a single failure disabling all of them. I know that UPSs are physical, and subnets are logical, but the reasoning behind the requirement is due to having to be on a different infrastructure.