thr3ads.net - nsd users - [nsd-users] Trying to understand a SERVFAIL [Dec 2009]

If this information is useful, please help other people find it:
Share via:

Pim van Pelt

2009-Dec-28 18:25 UTC

[nsd-users] Trying to understand a SERVFAIL

Hoi,

I've loaded two zones, paphosting.net and l.paphosting.net (see [1])
on three nameservers (ns.paphosting.{net,nl.eu}). These nameservers
serve requests generally fine and authoritatively (for example,
http0.l.paphosting.net resolves to two A and two AAAA records).
However, on some resolvers (and some, but not all client),
intermittently, I get unexpected answers (SERVFAIL), and looking at
tcpdump output it seems that perhaps there is a bit of a weird answer
coming from NSD.

The client (on hispeed.ch, a caching nameserver running bind 9.4.2 on
OpenBSD 4.3) is speaking to an NSD (on bit.nl, an nsd authoritative
nameserver running 3.2.2 on Linux 2.6/Ubuntu LTS 8.04). Using host(1),
I can expose this issue, while using dig(1) I cannot. So running
"""host -t A www.paphosting.net 192.168.2.1""",
here's the resulting
UDP conversation[2] - it shows SERVFAIL, and seems to try each of the
3 nameservers twice, in turn, before giving up.

I observed each NSD giving an odd answer:
http0.l.paphosting.net. A nlede01.paphosting.net.122.109.193.in-addr.arpa
http0.l.paphosting.net. A http.weirdnet.nl

the first record is not my intention, and perhaps a clue. There are
sibling domains on the NSD authoritative nameservers (paphosting.nl
and paphosting.eu) and for them, looking up www.paphosting.{nl,eu}
works fine and those lookups do not show the .arpa reply.

This is an intriguing problem to me, because a simple method exists
for populating the cache, see here:
$ host www.paphosting.net 192.168.2.1
Using domain server:
Name: 192.168.2.1
Address: 192.168.2.1#53
Aliases:

Host www.paphosting.net not found: 2(SERVFAIL)
$ dig @192.168.2.1 ANY www.paphosting.net

; <<>> DiG 9.4.2 <<>> @192.168.2.1 ANY
www.paphosting.net
; (1 server found)
;; global options: ?printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44766
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 5

;; QUESTION SECTION:
;www.paphosting.net. ? ? ? ? ? ?IN ? ? ?ANY

;; ANSWER SECTION:
www.paphosting.net. ? ? 86400 ? IN ? ? ?CNAME ? http0.l.paphosting.net.

;; AUTHORITY SECTION:
paphosting.net. ? ? ? ? 84971 ? IN ? ? ?NS ? ? ?ns.paphosting.eu.
paphosting.net. ? ? ? ? 84971 ? IN ? ? ?NS ? ? ?ns.paphosting.net.
paphosting.net. ? ? ? ? 84971 ? IN ? ? ?NS ? ? ?ns.paphosting.nl.

;; ADDITIONAL SECTION:
ns.paphosting.nl. ? ? ? 84971 ? IN ? ? ?A ? ? ? 94.142.245.3
ns.paphosting.nl. ? ? ? 84971 ? IN ? ? ?AAAA ? ?2a02:898:28::3
ns.paphosting.net. ? ? ?84971 ? IN ? ? ?AAAA ? ?2001:7b8:3:47:20d:b9ff:fe14:70d4
ns.paphosting.eu. ? ? ? 84970 ? IN ? ? ?A ? ? ? 62.220.146.194
ns.paphosting.eu. ? ? ? 84971 ? IN ? ? ?AAAA ? ?2001:788:2:117::2

;; Query time: 25 msec
;; SERVER: 192.168.2.1#53(192.168.2.1)
;; WHEN: Mon Dec 28 19:13:39 2009
;; MSG SIZE ?rcvd: 251

$ host www.paphosting.net 192.168.2.1
Using domain server:
Name: 192.168.2.1
Address: 192.168.2.1#53
Aliases:

www.paphosting.net is an alias for http0.l.paphosting.net.
http0.l.paphosting.net has address 94.142.245.2
http0.l.paphosting.net has address 193.109.122.243
http0.l.paphosting.net has IPv6 address 2a02:898:28::2
http0.l.paphosting.net has IPv6 address 2001:7b8:3:4f:216:3eff:fe4b:ae79

and from then on, the cache understands what I mean. Does anybody
have any clues? Are there things I can do to better understand this
issue? This issue can be observed from the internet - I'd be happy to
facilitate any necessary debugging (for example, I can try to load this
data on another auth nameserver like bind or powerdns).

groet,
Pim

[1]
$ORIGIN paphosting.net.
$TTL 86400
@ ? ? ? IN ? ? ?SOA ? ? ns.paphosting.net. hostmaster.paphosting.net. (
? ? ? ? ? ? ? ?2009122501
? ? ? ? ? ? ? ?28800
? ? ? ? ? ? ? ?7200
? ? ? ? ? ? ? ?604800
? ? ? ? ? ? ? ?86400
? ? ? ? ? ? ? ?)

@ ? ? ? ? ? ? ? NS ? ? ?ns.paphosting.nl.
@ ? ? ? ? ? ? ? NS ? ? ?ns.paphosting.net.
@ ? ? ? ? ? ? ? NS ? ? ?ns.paphosting.eu.
@ ? ? ? ? ? ? ? MX ? ? ?10 ? ? ?mx0.paphosting.net.
@ ? ? ? ? ? ? ? MX ? ? ?20 ? ? ?mx1.paphosting.net.
@ ? ? ? ? ? ? ? MX ? ? ?30 ? ? ?mx2.paphosting.net.
; DNS
ns ? ? ? ? ? ? ?AAAA ? ?2001:7b8:3:47:20d:b9ff:fe14:70d4
ns ? ? ? ? ? ? ?A ? ? ? 213.154.229.21
l ? ? ? ? ? ? ? NS ? ? ?ns.paphosting.nl.
l ? ? ? ? ? ? ? NS ? ? ?ns.paphosting.net.
l ? ? ? ? ? ? ? NS ? ? ?ns.paphosting.eu.

www ? ? ? ? ? ? CNAME ? http0.l.paphosting.net.

$ORIGIN l.paphosting.net.
$TTL 300
@ ? ? ? IN ? ? ?SOA ? ? ns.paphosting.net. hostmaster.paphosting.net. (
? ? ? ? ? ? ? ?2009121300
? ? ? ? ? ? ? ?28800 ? ? ? ? ? ? ? ? ? ? ? ? ; Refresh
? ? ? ? ? ? ? ?7200 ? ? ? ? ? ? ? ? ? ? ? ? ?; Retry
? ? ? ? ? ? ? ?604800 ? ? ? ? ? ? ? ? ? ? ? ?; Expire
? ? ? ? ? ? ? ?300 ? ? ? ? ? ? ? ? ? ? ? ? ? ; Minimum
? ? ? ? ? ? ? ?)
@ ? ? ? IN ? ? ?NS ? ? ?ns.paphosting.nl.
@ ? ? ? IN ? ? ?NS ? ? ?ns.paphosting.net.
@ ? ? ? IN ? ? ?NS ? ? ?ns.paphosting.eu.
; TODO: Autogenerate this zonefile based on a configfile

; nl.ede, BIT, Ede, Netherlands
http0 ? IN ? ? ?A ? ? ? 193.109.122.243
http0 ? IN ? ? ?AAAA ? ?2001:7b8:3:4f:216:3eff:fe4b:ae79

; nl.ams, Coloclue, Amsterdam, Netherlands
http0 ? IN ? ? ?A ? ? ? 94.142.245.2
http0 ? IN ? ? ?AAAA ? ?2a02:898:28::2

[2] tcpdump while typing 'host www.paphosting.net 192.168.2.1' on an
empty bind 9.4.2 resolver
18:51:08.146580 00:0d:b9:14:70:70 00:17:10:00:c9:34 ip 89:
77-56-102-109.dclient.hispeed.ch.45465 > 62.220.146.194.domain: [udp
sum ok] 53168% [1au] A? www.paphosting.net. ar: . OPT UDPsize=4096
(47) (ttl 64, id 46989, len 75)
18:51:08.164377 00:17:10:00:c9:34 00:0d:b9:14:70:70 ip 324:
62.220.146.194.domain > 77-56-102-109.dclient.hispeed.ch.45465: [udp
sum ok] 53168*- q: A? www.paphosting.net. 3/6/3 www.paphosting.net.
CNAME http0.l.paphosting.net., http0.l.paphosting.net. A
nlede01.paphosting.net.122.109.193.in-addr.arpa,
http0.l.paphosting.net. A http.weirdnet.nl ns: l.paphosting.net. NS
ns.paphosting.nl., l.paphosting.net. NS ns.paphosting.net.,
l.paphosting.net. NS ns.paphosting.eu., paphosting.net. NS
ns.paphosting.nl., paphosting.net. NS ns.paphosting.net.,
paphosting.net. NS ns.paphosting.eu. ar: ns.paphosting.net. A
console.colo.bit.nl, ns.paphosting.net. AAAA
2001:7b8:3:47:20d:b9ff:fe14:70d4, . OPT UDPsize=4096 (282) [tos 0x20]
(ttl 53, id 8308, len 310)
18:51:08.165146 00:0d:b9:14:70:70 00:17:10:00:c9:34 ip 89:
77-56-102-109.dclient.hispeed.ch.45465 > console.colo.bit.nl.domain:
[udp sum ok] 64115% [1au] A? www.paphosting.net. ar: . OPT
UDPsize=4096 (47) (ttl 64, id 15411, len 75)
18:51:08.202528 00:17:10:00:c9:34 00:0d:b9:14:70:70 ip 324:
console.colo.bit.nl.domain > 77-56-102-109.dclient.hispeed.ch.45465:
[udp sum ok] 64115*- q: A? www.paphosting.net. 3/6/3
www.paphosting.net. CNAME http0.l.paphosting.net.,
http0.l.paphosting.net. A
nlede01.paphosting.net.122.109.193.in-addr.arpa,
http0.l.paphosting.net. A http.weirdnet.nl ns: l.paphosting.net. NS
ns.paphosting.nl., l.paphosting.net. NS ns.paphosting.net.,
l.paphosting.net. NS ns.paphosting.eu., paphosting.net. NS
ns.paphosting.nl., paphosting.net. NS ns.paphosting.net.,
paphosting.net. NS ns.paphosting.eu. ar: ns.paphosting.net. A
console.colo.bit.nl, ns.paphosting.net. AAAA
2001:7b8:3:47:20d:b9ff:fe14:70d4, . OPT UDPsize=4096 (282) (ttl 55, id
35079, len 310)
18:51:08.203184 00:0d:b9:14:70:70 00:17:10:00:c9:34 ip 89:
77-56-102-109.dclient.hispeed.ch.45465 > ns1.weirdnet.nl.domain: [udp
sum ok] 29356% [1au] A? www.paphosting.net. ar: . OPT UDPsize=4096
(47) (ttl 64, id 1335, len 75)
18:51:08.241402 00:17:10:00:c9:34 00:0d:b9:14:70:70 ip 324:
ns1.weirdnet.nl.domain > 77-56-102-109.dclient.hispeed.ch.45465: [udp
sum ok] 29356*- q: A? www.paphosting.net. 3/6/3 www.paphosting.net.
CNAME http0.l.paphosting.net., http0.l.paphosting.net. A
nlede01.paphosting.net.122.109.193.in-addr.arpa,
http0.l.paphosting.net. A http.weirdnet.nl ns: l.paphosting.net. NS
ns.paphosting.nl., l.paphosting.net. NS ns.paphosting.net.,
l.paphosting.net. NS ns.paphosting.eu., paphosting.net. NS
ns.paphosting.nl., paphosting.net. NS ns.paphosting.net.,
paphosting.net. NS ns.paphosting.eu. ar: ns.paphosting.net. A
console.colo.bit.nl, ns.paphosting.net. AAAA
2001:7b8:3:47:20d:b9ff:fe14:70d4, . OPT UDPsize=4096 (282) (ttl 54, id
57236, len 310)
18:51:08.249549 00:0d:b9:14:70:70 00:17:10:00:c9:34 ip 89:
77-56-102-109.dclient.hispeed.ch.45465 > 62.220.146.194.domain: [udp
sum ok] 64826% [1au] A? www.paphosting.net. ar: . OPT UDPsize=4096
(47) (ttl 64, id 7106, len 75)
18:51:08.266204 00:17:10:00:c9:34 00:0d:b9:14:70:70 ip 324:
62.220.146.194.domain > 77-56-102-109.dclient.hispeed.ch.45465: [udp
sum ok] 64826*- q: A? www.paphosting.net. 3/6/3 www.paphosting.net.
CNAME http0.l.paphosting.net., http0.l.paphosting.net. A
nlede01.paphosting.net.122.109.193.in-addr.arpa,
http0.l.paphosting.net. A http.weirdnet.nl ns: l.paphosting.net. NS
ns.paphosting.nl., l.paphosting.net. NS ns.paphosting.net.,
l.paphosting.net. NS ns.paphosting.eu., paphosting.net. NS
ns.paphosting.nl., paphosting.net. NS ns.paphosting.net.,
paphosting.net. NS ns.paphosting.eu. ar: ns.paphosting.net. A
console.colo.bit.nl, ns.paphosting.net. AAAA
2001:7b8:3:47:20d:b9ff:fe14:70d4, . OPT UDPsize=4096 (282) [tos 0x20]
(ttl 53, id 34156, len 310)
18:51:08.266877 00:0d:b9:14:70:70 00:17:10:00:c9:34 ip 89:
77-56-102-109.dclient.hispeed.ch.45465 > console.colo.bit.nl.domain:
[udp sum ok] 34265% [1au] A? www.paphosting.net. ar: . OPT
UDPsize=4096 (47) (ttl 64, id 56895, len 75)
18:51:08.304342 00:17:10:00:c9:34 00:0d:b9:14:70:70 ip 324:
console.colo.bit.nl.domain > 77-56-102-109.dclient.hispeed.ch.45465:
[udp sum ok] 34265*- q: A? www.paphosting.net. 3/6/3
www.paphosting.net. CNAME http0.l.paphosting.net.,
http0.l.paphosting.net. A
nlede01.paphosting.net.122.109.193.in-addr.arpa,
http0.l.paphosting.net. A http.weirdnet.nl ns: l.paphosting.net. NS
ns.paphosting.nl., l.paphosting.net. NS ns.paphosting.net.,
l.paphosting.net. NS ns.paphosting.eu., paphosting.net. NS
ns.paphosting.nl., paphosting.net. NS ns.paphosting.net.,
paphosting.net. NS ns.paphosting.eu. ar: ns.paphosting.net. A
console.colo.bit.nl, ns.paphosting.net. AAAA
2001:7b8:3:47:20d:b9ff:fe14:70d4, . OPT UDPsize=4096 (282) (ttl 55, id
62916, len 310)
18:51:08.304972 00:0d:b9:14:70:70 00:17:10:00:c9:34 ip 89:
77-56-102-109.dclient.hispeed.ch.45465 > ns1.weirdnet.nl.domain: [udp
sum ok] 43899% [1au] A? www.paphosting.net. ar: . OPT UDPsize=4096
(47) (ttl 64, id 36756, len 75)
18:51:08.343809 00:17:10:00:c9:34 00:0d:b9:14:70:70 ip 324:
ns1.weirdnet.nl.domain > 77-56-102-109.dclient.hispeed.ch.45465: [udp
sum ok] 43899*- q: A? www.paphosting.net. 3/6/3 www.paphosting.net.
CNAME http0.l.paphosting.net., http0.l.paphosting.net. A
nlede01.paphosting.net.122.109.193.in-addr.arpa,
http0.l.paphosting.net. A http.weirdnet.nl ns: l.paphosting.net. NS
ns.paphosting.nl., l.paphosting.net. NS ns.paphosting.net.,
l.paphosting.net. NS ns.paphosting.eu., paphosting.net. NS
ns.paphosting.nl., paphosting.net. NS ns.paphosting.net.,
paphosting.net. NS ns.paphosting.eu. ar: ns.paphosting.net. A
console.colo.bit.nl, ns.paphosting.net. AAAA
2001:7b8:3:47:20d:b9ff:fe14:70d4, . OPT UDPsize=4096 (282) (ttl 54, id
24040, len 310)

-- 
Pim van Pelt <pim at ipng.nl>
PBVP1-RIPE - http://www.ipng.nl/

Pim van Pelt

2009-Dec-28 20:53 UTC

head link

[nsd-users] Trying to understand a SERVFAIL

Hoi Jeremy,

On Mon, Dec 28, 2009 at 8:07 PM, Jeremy C. Reed <reed at reedmedia.net>
wrote:> multiple NS RRsets in authority sectionThis seems correct to me.
> ;; AUTHORITY SECTION:
> l.paphosting.net. ? ? ? 300 ? ? IN ? ? ?NS ? ? ?ns.paphosting.nl.
> l.paphosting.net. ? ? ? 300 ? ? IN ? ? ?NS ? ? ?ns.paphosting.net.
> l.paphosting.net. ? ? ? 300 ? ? IN ? ? ?NS ? ? ?ns.paphosting.eu.
> paphosting.net. ? ? ? ? 86400 ? IN ? ? ?NS ? ? ?ns.paphosting.nl.
> paphosting.net. ? ? ? ? 86400 ? IN ? ? ?NS ? ? ?ns.paphosting.net.
> paphosting.net. ? ? ? ? 86400 ? IN ? ? ?NS ? ? ?ns.paphosting.eu.
>
> (I don't know why.)this is because www.paphosting.net is a CNAME to
http0.l.paphosting.net (which is a different zone, the nameservers of
which are the same but their TTL is 300s).

groet,
Pim
-- 
Pim van Pelt <pim at ipng.nl>
PBVP1-RIPE - http://www.ipng.nl/

Pim van Pelt

2009-Dec-28 22:03 UTC

head link

[nsd-users] Trying to understand a SERVFAIL

Hoi,

Addendum, see inline:

On Mon, Dec 28, 2009 at 7:25 PM, Pim van Pelt <pim at ipng.nl>
wrote:> Hoi,
>
> I've loaded two zones, paphosting.net and l.paphosting.net (see [1])
> on three nameservers (ns.paphosting.{net,nl.eu}). These nameservers
> serve requests generally fine and authoritatively (for example,
> http0.l.paphosting.net resolves to two A and two AAAA records).
> However, on some resolvers (and some, but not all client),
> intermittently, I get unexpected answers (SERVFAIL), and looking at
> tcpdump output it seems that perhaps there is a bit of a weird answer
> coming from NSD.
>
> The client (on hispeed.ch, a caching nameserver running bind 9.4.2 on
> OpenBSD 4.3) is speaking to an NSD (on bit.nl, an nsd authoritative
> nameserver running 3.2.2 on Linux 2.6/Ubuntu LTS 8.04). Using host(1),
> I can expose this issue, while using dig(1) I cannot. So running
> """host -t A www.paphosting.net
192.168.2.1""", here's the resulting
> UDP conversation[2] - it shows SERVFAIL, and seems to try each of the
> 3 nameservers twice, in turn, before giving up.
>
> I observed each NSD giving an odd answer:
> http0.l.paphosting.net. A nlede01.paphosting.net.122.109.193.in-addr.arpa
> http0.l.paphosting.net. A http.weirdnet.nl
>
> the first record is not my intention, and perhaps a clue.This is not a clue, because in 122.109.193.in-addr.arpa., also served
by the same nameservers, I had omitted the trailing '.' behind
nlede01.paphosting.net; which made it become that *.arpa address. This
is now fixed,
but the issue remains. The rest is simply an artifact from tcpdump,
which translates all IP addresses to names :)

The issue itself, as described in the original post, remains.
-- 
Pim van Pelt <pim at ipng.nl>
PBVP1-RIPE - http://www.ipng.nl/

W.C.A. Wijngaards

2010-Jan-01 00:07 UTC

head link

[nsd-users] Trying to understand a SERVFAIL

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Pim, Jeremy,

This response looks like a corner case.  I think it may trigger that bad
behaviour in some resolvers.  This may be something that is caused by
new 'Kaminksy-era-paranoia' fixes in resolvers.  I see that this
response triggers drastic cutting measures in unbound (but it does work
there), perhaps BIND does something as well.

One operational fix would be to integrate both zones - no zone cut, SOA
or NS records for l.paphosting.net.  Simply put the information into the
paphosting.net zone.  This is an operational fix, perhaps BIND and NSD
code need fixes.

http0.l   IN      A       193.109.122.243
http0.l   IN      AAAA    2001:7b8:3:4f:216:3eff:fe4b:ae79
http0.l   IN      A       94.142.245.2
http0.l   IN      AAAA    2a02:898:28::2

Since many resolvers won't use the information after the CNAME from the
first response anyway, perhaps NSD should not descend into the next zone.

Best regards,
   Wouter

On 12/28/2009 09:53 PM, Pim van Pelt wrote:> Hoi Jeremy,
> 
> On Mon, Dec 28, 2009 at 8:07 PM, Jeremy C. Reed <reed at
reedmedia.net> wrote:
>> multiple NS RRsets in authority section
> This seems correct to me.
> 
>> ;; AUTHORITY SECTION:
>> l.paphosting.net.       300     IN      NS      ns.paphosting.nl.
>> l.paphosting.net.       300     IN      NS      ns.paphosting.net.
>> l.paphosting.net.       300     IN      NS      ns.paphosting.eu.
>> paphosting.net.         86400   IN      NS      ns.paphosting.nl.
>> paphosting.net.         86400   IN      NS      ns.paphosting.net.
>> paphosting.net.         86400   IN      NS      ns.paphosting.eu.
>>
>> (I don't know why.)
> this is because www.paphosting.net is a CNAME to
> http0.l.paphosting.net (which is a different zone, the nameservers of
> which are the same but their TTL is 300s).
> 
> groet,
> Pim
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iEYEARECAAYFAks9PMIACgkQkDLqNwOhpPhADwCghDuzfLJTUbz49h2VUGA15Mdf
+PYAoJi78H4zRBHtuZBhWkQpjJHzqSBR
=NdOx
-----END PGP SIGNATURE-----

nsd users - Dec 2009 - Trying to understand a SERVFAIL

[nsd-users] Trying to understand a SERVFAIL

[nsd-users] Trying to understand a SERVFAIL

[nsd-users] Trying to understand a SERVFAIL

[nsd-users] Trying to understand a SERVFAIL