Patrick Goetz
2022-Aug-26 23:15 UTC
[Samba] After a power outage Samba DNS is no longer working properly
Oh, I should add to this that is searching the web I found this command: root at samba-dc:~# samba_dnsupdate --verbose --all-names That results in, among other things, errors that look like this: ---------------------------------------------- ; TSIG error with server: tsig verify failure Failed nsupdate: 2 update(nsupdate): SRV _ldap._tcp.Default-First-Site-Name._sites.ForestDnsZones.ea.my_org.org samba-dc.ea.my_org.org 389 Calling nsupdate for SRV _ldap._tcp.Default-First-Site-Name._sites.ForestDnsZones.ea.my_orgs.org samba-dc.ea.my_org.org 389 (add) Successfully obtained Kerberos ticket to DNS/samba-dc.ea.myorg.org as SAMBA-DC$ Outgoing update query: ;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id: 0 ;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0 ;; UPDATE SECTION: _ldap._tcp.Default-First-Site-Name._sites.ForestDnsZones.ea.my_org.org. 900 IN SRV 0 100 389 samba-dc.ea.my_org.org. ; TSIG error with server: tsig verify failure Failed nsupdate: 2 Failed update of 29 entries ---------------------------------------------- <rant>I ran a Samba 3 server for this org for 12 years. In their previous locations they had power outages on an almost weekly basis and yet Samba 3 *never* crashed and burned. Now running Samba 4 in a new location, and the very first time there's a power hiccup it falls part. There is literally nothing which should make the software fail like this.</rant> On 8/26/22 15:05, Patrick Goetz via samba wrote:> > ? OS: Ubuntu 20.04.3 > ? Samba version: 4.15.2 from Louis' repo > > We suffered a power outage after which Samba DNS resolution no longer > works and I can't figure out why.? The domain controller continues to be > aware of its client machines: > > root at samba-dc:~# samba-tool dns query samba-dc ea.my_org.org ea124 A -U > Administrator > Password for [EA\Administrator]: > ? Name=, Records=1, Children=0 > ??? A: 172.18.90.124 (flags=f0, serial=110, ttl=1200) > > > root at samba-dc:~# samba-tool dns query samba-dc 90.18.172.in-addr.arpa > 124 PTR -U Administrator > Password for [EA\Administrator]: > ? Name=, Records=1, Children=0 > ??? PTR: EA124.ea.my_org.org (flags=f0, serial=7, ttl=900) > > However, on the Windows 10 clients (specifically ea124) > > ? net use G: \\data2\share > > gives a characteristically cryptic Microsoft error message: Error 53 > which turns out to mean it can't resolve the host name.? If I substitute > the share server's IP address: > > ? net use G: \\127.18.90.30\share > > then the mount executes as one would expect. > > The Wiki page here: > https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.samba.org%2Findex.php%2FDNS_Administration&data=05%7C01%7C%7C72149244adb241bddc9d08da87a0b21e%7C31d7e2a5bdd8414e9e97bea998ebdfe1%7C0%7C0%7C637971421504331373%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UhNlXcoit6hdPl%2Bs2HteeiG6hYR%2FAavVPN0eypf8jSc%3D&reserved=0 > > > is very sparse on details of how one would go about debugging or > repairing this issue.? I seem to recall people running into this on much > larger networks than mine, but googling and searching the list didn't > bring up any useful information. > > Anyone have any ideas? > > >
Patrick Goetz
2022-Aug-29 16:17 UTC
[Samba] After a power outage Samba DNS is no longer working properly
So, no one has any thoughts on my DNS corruption problem? I'm still scratching my head as to how this could happen, but am guessing it has something to do with DynDNS. I tried removing one of the workstations from the domain and rejoining, but this didn't fix the problem for that workstation. In the absence of any kind of debugging or mitigation, it looks like I'll have to rebuild the domain controller from scratch (I guess an opportunity to upgrade to 4.16 or 4.17) but will henceforth aggressively snapshot the container the DC runs in. I have a snapshot, but this was before adding users and computers to the domain. On 8/26/22 18:15, Patrick Goetz via samba wrote:> Oh, I should add to this that is searching the web I found this command: > > ? root at samba-dc:~# samba_dnsupdate --verbose --all-names > > That results in, among other things, errors that look like this: > ---------------------------------------------- > ? ; TSIG error with server: tsig verify failure > Failed nsupdate: 2 > update(nsupdate): SRV > _ldap._tcp.Default-First-Site-Name._sites.ForestDnsZones.ea.my_org.org > samba-dc.ea.my_org.org 389 > Calling nsupdate for SRV > _ldap._tcp.Default-First-Site-Name._sites.ForestDnsZones.ea.my_orgs.org > samba-dc.ea.my_org.org 389 (add) > Successfully obtained Kerberos ticket to DNS/samba-dc.ea.myorg.org as > SAMBA-DC$ > Outgoing update query: > ;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id:????? 0 > ;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0 > ;; UPDATE SECTION: > _ldap._tcp.Default-First-Site-Name._sites.ForestDnsZones.ea.my_org.org. > 900 IN SRV 0 100 389 samba-dc.ea.my_org.org. > > ; TSIG error with server: tsig verify failure > Failed nsupdate: 2 > Failed update of 29 entries > ---------------------------------------------- > > <rant>I ran a Samba 3 server for this org for 12 years. In their > previous locations they had power outages on an almost weekly basis and > yet Samba 3 *never* crashed and burned.? Now running Samba 4 in a new > location, and the very first time there's a power hiccup it falls part. > ?There is literally nothing which should make the software fail like > this.</rant> > > On 8/26/22 15:05, Patrick Goetz via samba wrote: >> >> ?? OS: Ubuntu 20.04.3 >> ?? Samba version: 4.15.2 from Louis' repo >> >> We suffered a power outage after which Samba DNS resolution no longer >> works and I can't figure out why.? The domain controller continues to >> be aware of its client machines: >> >> root at samba-dc:~# samba-tool dns query samba-dc ea.my_org.org ea124 A >> -U Administrator >> Password for [EA\Administrator]: >> ?? Name=, Records=1, Children=0 >> ???? A: 172.18.90.124 (flags=f0, serial=110, ttl=1200) >> >> >> root at samba-dc:~# samba-tool dns query samba-dc 90.18.172.in-addr.arpa >> 124 PTR -U Administrator >> Password for [EA\Administrator]: >> ?? Name=, Records=1, Children=0 >> ???? PTR: EA124.ea.my_org.org (flags=f0, serial=7, ttl=900) >> >> However, on the Windows 10 clients (specifically ea124) >> >> ?? net use G: \\data2\share >> >> gives a characteristically cryptic Microsoft error message: Error 53 >> which turns out to mean it can't resolve the host name.? If I >> substitute the share server's IP address: >> >> ?? net use G: \\127.18.90.30\share >> >> then the mount executes as one would expect. >> >> The Wiki page here: >> https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.samba.org%2Findex.php%2FDNS_Administration&data=05%7C01%7C%7Ccb66d6b8be4346cd8ff808da87b90dc0%7C31d7e2a5bdd8414e9e97bea998ebdfe1%7C0%7C0%7C637971526112010134%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FxJ3a3omipQG4vmlh7Sae0bynVcLUM6IFXo6bZSTQxw%3D&reserved=0 >> >> >> is very sparse on details of how one would go about debugging or >> repairing this issue.? I seem to recall people running into this on >> much larger networks than mine, but googling and searching the list >> didn't bring up any useful information. >> >> Anyone have any ideas? >> >> >> >
Patrick Goetz
2022-Sep-05 17:16 UTC
[Samba] After a power outage Samba DNS is no longer working properly
Answering my own question in case someone with the same problem is
searching through the list archive.
The domain under consideration here has 2 servers: data2 and staging,
from which SMBx shares are mounted on the windows workstations.
Everything was working fine until an overnight power outage took out the
whole network, after which users could no longer mount shares using the
hostnames of the servers; i.e. internal DNS record corruption.
Not sure why it took me this long to notice, but it turns out the "A"
records for both data2 and staging were no longer present in the
internal DNS server; the PTR records were intact. All the records for
the Windows 10 clients were also unaffected.
So fixing the problem was a simple matter of recreating the A records
for the file servers:
samba-tool dns add <SAMBA-DC> <DOMAIN> data2 A xxx.xxx.xxx.xxx -U
Administrator
and
samba-tool dns add <SAMBA-DC> <DOMAIN> staging A
xxx.xxx.xxx.yyy -U
Administrator
I have no idea how the records could just disappear like that; perhaps
has something to do with dynDNS.
Speaking of which, this command:
samba_dnsupdate --verbose --all-names
still shows these errors:
; TSIG error with server: tsig verify failure
Failed nsupdate: 2
update(nsupdate): SRV
_ldap._tcp.Default-First-Site-Name._sites.ForestDnsZones.ea.example.org
samba-dc.ea.example.org 389
Calling nsupdate for SRV
_ldap._tcp.Default-First-Site-Name._sites.ForestDnsZones.ea.example.org
samba-dc.ea.example.org 389 (add)
Successfully obtained Kerberos ticket to DNS/samba-dc.ea.example.org as
SAMBA-DC$
Outgoing update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id: 0
;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0
;; UPDATE SECTION:
_ldap._tcp.Default-First-Site-Name._sites.ForestDnsZones.ea.example.org.
900 IN SRV 0 100 389 samba-dc.ea.example.org.
I'm not sure what this means, but if it doesn't affect operations,
I'm
not terribly concerned.
On 8/29/22 11:17, Patrick Goetz wrote:> So, no one has any thoughts on my DNS corruption problem?? I'm still
> scratching my head as to how this could happen, but am guessing it has
> something to do with DynDNS.
>
> I tried removing one of the workstations from the domain and rejoining,
> but this didn't fix the problem for that workstation. In the absence of
> any kind of debugging or mitigation, it looks like I'll have to rebuild
> the domain controller from scratch (I guess an opportunity to upgrade to
> 4.16 or 4.17) but will henceforth aggressively snapshot the container
> the DC runs in.? I have a snapshot, but this was before adding users and
> computers to the domain.
>
> On 8/26/22 18:15, Patrick Goetz via samba wrote:
>> Oh, I should add to this that is searching the web I found this
command:
>>
>> ?? root at samba-dc:~# samba_dnsupdate --verbose --all-names
>>
>> That results in, among other things, errors that look like this:
>> ----------------------------------------------
>> ?? ; TSIG error with server: tsig verify failure
>> Failed nsupdate: 2
>> update(nsupdate): SRV
>> _ldap._tcp.Default-First-Site-Name._sites.ForestDnsZones.ea.my_org.org
>> samba-dc.ea.my_org.org 389
>> Calling nsupdate for SRV
>> _ldap._tcp.Default-First-Site-Name._sites.ForestDnsZones.ea.my_orgs.org
samba-dc.ea.my_org.org
>> 389 (add)
>> Successfully obtained Kerberos ticket to DNS/samba-dc.ea.myorg.org as
>> SAMBA-DC$
>> Outgoing update query:
>> ;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id:????? 0
>> ;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0
>> ;; UPDATE SECTION:
>> _ldap._tcp.Default-First-Site-Name._sites.ForestDnsZones.ea.my_org.org.
900
>> IN SRV 0 100 389 samba-dc.ea.my_org.org.
>>
>> ; TSIG error with server: tsig verify failure
>> Failed nsupdate: 2
>> Failed update of 29 entries
>> ----------------------------------------------
>>
>> <rant>I ran a Samba 3 server for this org for 12 years. In their
>> previous locations they had power outages on an almost weekly basis
>> and yet Samba 3 *never* crashed and burned.? Now running Samba 4 in a
>> new location, and the very first time there's a power hiccup it
falls
>> part. ??There is literally nothing which should make the software fail
>> like this.</rant>
>>
>> On 8/26/22 15:05, Patrick Goetz via samba wrote:
>>>
>>> ?? OS: Ubuntu 20.04.3
>>> ?? Samba version: 4.15.2 from Louis' repo
>>>
>>> We suffered a power outage after which Samba DNS resolution no
longer
>>> works and I can't figure out why.? The domain controller
continues to
>>> be aware of its client machines:
>>>
>>> root at samba-dc:~# samba-tool dns query samba-dc ea.my_org.org
ea124 A
>>> -U Administrator
>>> Password for [EA\Administrator]:
>>> ?? Name=, Records=1, Children=0
>>> ???? A: 172.18.90.124 (flags=f0, serial=110, ttl=1200)
>>>
>>>
>>> root at samba-dc:~# samba-tool dns query samba-dc
90.18.172.in-addr.arpa
>>> 124 PTR -U Administrator
>>> Password for [EA\Administrator]:
>>> ?? Name=, Records=1, Children=0
>>> ???? PTR: EA124.ea.my_org.org (flags=f0, serial=7, ttl=900)
>>>
>>> However, on the Windows 10 clients (specifically ea124)
>>>
>>> ?? net use G: \\data2\share
>>>
>>> gives a characteristically cryptic Microsoft error message: Error
53
>>> which turns out to mean it can't resolve the host name.? If I
>>> substitute the share server's IP address:
>>>
>>> ?? net use G: \\127.18.90.30\share
>>>
>>> then the mount executes as one would expect.
>>>
>>> The Wiki page here:
>>>
https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.samba.org%2Findex.php%2FDNS_Administration&data=05%7C01%7C%7Ccb66d6b8be4346cd8ff808da87b90dc0%7C31d7e2a5bdd8414e9e97bea998ebdfe1%7C0%7C0%7C637971526112010134%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FxJ3a3omipQG4vmlh7Sae0bynVcLUM6IFXo6bZSTQxw%3D&reserved=0
>>>
>>>
>>> is very sparse on details of how one would go about debugging or
>>> repairing this issue.? I seem to recall people running into this on
>>> much larger networks than mine, but googling and searching the list
>>> didn't bring up any useful information.
>>>
>>> Anyone have any ideas?
>>>
>>>
>>>
>>