samba - Dec 2023 - Failed to store repsFrom - Indexed and full searches both failed!

hello,

Using the same server, hostname, and IP, I performed an offline demotion of
dc4 because the online demotion resulted in an error.

After the offline demotion, I checked via RSAT, and the entries had been
removed.

Later...
- I deleted the .tdb and .ldb files.
- I executed the provisioning command.
- I backed up the idmap.ldb and copied it to dc4 using rsync.
- I performed rsync of the sysvol folder.
- I executed the command "samba-tool ntacl sysvolreset."
- I recreated the bind entries via "samba_upgradedns
--dns-backend=SAMBA_INTERNAL" and again with "samba_upgradedns
--dns-backend=BIND9_DLZ."

I restarted bind and samba, tested "drs showrepl" and
"samba_dnsupdate,"
and they were okay.

I left a tail running on the log, and after some time, the error "Failed to
store repsFrom - Indexed and full searches both failed!" returned.

Is there still a chance to use the same hostname and IP, or do I need to
change everything?

On Wed, Dec 6, 2023 at 6:41?PM Elias Pereira <empbilly at gmail.com>
wrote:
> Thanks Andrew,
>
> Can I demote and use same hostname and IP?
>
> On Wed, Dec 6, 2023 at 6:05?PM Andrew Bartlett <abartlet at
samba.org> wrote:
>
>> "Indexed and full searches both failed" is not a good sign.
>>
>> Failed in this case doesn't just mean 'returned no
results', it means
>> 'database error'.  It could be on any record, as the filtering
for a
>> full search has to happen across the whole DB and if any of those
>> filter tests fail, it will do this.
>>
>> I think you have another working DC, if so I would demote this broken
>> DC and replace it.
>>
>> Andrew Bartlett
>>
>> On Wed, 2023-12-06 at 12:45 -0300, Elias Pereira via samba wrote:
>> > Any thoughts? :D
>> >
>> > On Tue, Dec 5, 2023 at 4:59?PM Elias Pereira <
>> > empbilly at gmail.com
>> > > wrote:
>> >
>> > > hi,
>> > >
>> > > After an update to our DC4, I started to notice the error
"Failed
>> > > to store
>> > > repsFrom - Indexed and full searches both failed!" in
the logs.
>> > >
>> > > root at dc4:~# tail -f /var/log/samba/log.samba
>> > >   Copyright Andrew Tridgell and the Samba Team 1992-2023
>> > > [2023/12/05 14:09:06.191978,  0]
>> > > ../../lib/util/become_daemon.c:150(daemon_status)
>> > >   daemon_status: daemon 'samba' : Starting process...
>> > > [2023/12/05 14:09:06.373562,  0]
>> > > ../../source4/samba/server.c:896(binary_smbd_main)
>> > >   binary_smbd_main: samba: using 'prefork' process
model
>> > > [2023/12/05 16:05:04.141027,  0]
>> > > ../../source4/dsdb/kcc/kcc_periodic.c:790(samba_kcc_done)
>> > >   ../../source4/dsdb/kcc/kcc_periodic.c:790: Failed samba_kcc
-
>> > > NT_STATUS_IO_TIMEOUT
>> > > [2023/12/05 16:06:30.931069,  0]
>> > > ../../source4/dsdb/common/util.c:3482(dsdb_savereps)
>> > >   Failed to store repsFrom - Indexed and full searches both
failed!
>> > >
>> > > I tried running a "samba-tool drs replicate...",
but the error
>> > > occurs:
>> > >
>> > > ERROR(<class 'samba.drs_utils.drsException'>):
DsReplicaSync failed
>> > > -
>> > > drsException: DsReplicaSync failed (3221225653, '{Device
Timeout}
>> > > The
>> > > specified I/O operation on %hs was not completed before the
time-
>> > > out period
>> > > expired.')
>> > >   File
"/usr/lib/python3/dist-packages/samba/netcmd/drs.py", line
>> > > 567, in
>> > > run
>> > >     drs_utils.sendDsReplicaSync(server_bind,
server_bind_handle,
>> > > source_dsa_guid, NC, req_options)
>> > >   File
"/usr/lib/python3/dist-packages/samba/drs_utils.py", line
>> > > 100, in
>> > > sendDsReplicaSync
>> > >     raise drsException("DsReplicaSync failed %s" %
estr)
>> > >
>> > > Pastebin with -d10 drs replicate:
>> > > https://pastebin.com/raw/LZNTqssa
>> > >
>> > >
>> > > Some points:
>> > > - samba_dnsupdate --verbose --all-names is ok
>> > > - samba-tool dbcheck --cross-ncs --reset-well-known-acls
--fix is
>> > > ok
>> > > - samba-tool drs showrepl is ok (it takes +/- 1 minute for
the
>> > > result)
>> > >
>> > > Researching, I saw that it could be the corrupted database,
and
>> > > only
>> > > demote the DC in question would solve it.
>> > >
>> > > Is that really the case? If so, can I demote and use the same
>> > > server,
>> > > including the same hostname and IP?
>> > >
>> > > Thanks!!!
>> > >
>> > > --
>> > > Elias Pereira
>> > >
>> >
>> >
>> > --
>> > Elias Pereira
>> >
>> --
>> Andrew Bartlett (he/him)       https://samba.org/~abartlet/
>> Samba Team Member (since 2001) https://samba.org
>> Samba Team Lead                https://catalyst.net.nz/services/samba
>> Catalyst.Net <https://catalyst.net.nz/services/sambaCatalyst.Net>
Ltd
>>
>> Proudly developing Samba for Catalyst.Net Ltd - a Catalyst IT group
>> company
>>
>> Samba Development and Support: https://catalyst.net.nz/services/samba
>>
>> Catalyst IT - Expert Open Source Solutions
>>
>>
>>
>>
>
> --
> Elias Pereira
>

-- 
Elias Pereira

On Sat, 9 Dec 2023 13:20:39 -0300
Elias Pereira via samba <samba at lists.samba.org> wrote:
> hello,
> 
> Using the same server, hostname, and IP, I performed an offline
> demotion of dc4 because the online demotion resulted in an error.
> 
> After the offline demotion, I checked via RSAT, and the entries had
> been removed.
> 
> Later...
> - I deleted the .tdb and .ldb files.
> - I executed the provisioning command.
I thought that you were replacing one existing DC in a multi DC domain,
if so, you do not 'provision', you 'join' a new DC.
Can you please post the exact command you used.
 
> - I backed up the idmap.ldb and copied it to dc4 using rsync.
> - I performed rsync of the sysvol folder.
> - I executed the command "samba-tool ntacl sysvolreset."
> - I recreated the bind entries via "samba_upgradedns
> --dns-backend=SAMBA_INTERNAL" and again with "samba_upgradedns
> --dns-backend=BIND9_DLZ."
Not sure why you are doing that, if you are going to use Bind9, why not
just join with '--dns-backend=BIND9_DLZ' ?
> 
> I restarted bind and samba, tested "drs showrepl" and
> "samba_dnsupdate," and they were okay.
> 
> I left a tail running on the log, and after some time, the error
> "Failed to store repsFrom - Indexed and full searches both
failed!"
> returned.
> 
> Is there still a chance to use the same hostname and IP, or do I need
> to change everything?
Provided that there are no records in AD for the DC, you should be able
to use the old hostname and IP.

Rowland