hi,
My saga continues...
I've configured the audit log for drs_repl in smb.conf, and below is the
log generated.
https://transfer.sh/7fen4qCNIQ/drs_repl.log
The log level was 5.
drs_repl:5@/var/log/samba/drs_repl.log
Could someone take a look and help me understand the log?
On Sat, Feb 10, 2024 at 11:29?AM Elias Pereira <empbilly at gmail.com>
wrote:
> Hi samba list!!!
>
> Douglas, /usr/sbin/samba_kcc is made in python. Does it have a link to
> source4/dsdb/kcc/kcc_periodic.c which is made in C?
>
> The errors that appear in my DCs have their output in the C code.
>
> Correct me if I'm wrong, but I read in some old posts on the list that
> samba would have a new code for kcc (python code?), which would be
> closer to what M$ uses.
>
> Could this have anything to do with the errors I'm seeing?
>
> On Thu, Feb 8, 2024 at 7:42?PM Elias Pereira <empbilly at gmail.com>
wrote:
>
>> What other ways would we have of debugging this?
>>
>> On Thu, Feb 8, 2024 at 5:39?PM Douglas Bagnall <
>> douglas.bagnall at catalyst.net.nz> wrote:
>>
>>> On 9/02/24 06:35, Elias Pereira wrote:
>>> > for i in $(seq 1 10); do time /usr/sbin/samba_kcc --debug;
sleep
>>> 5;done
>>> >
>>> > I ran the command and in one run it lasted more than 2
minutes.
>>> >
>>> > I've sent you the file if you'd like to take a look.
>>>
>>> To summarise for the list, one of the runs took 3 minutes, and the
rest
>>> of them
>>> took half a second.
>>>
>>> $ grep -A2 real Downloads/kcc_output.txt
>>> real 2m59.646s
>>> user 0m0.387s
>>> sys 0m0.163s
>>> --
>>> real 0m0.504s
>>> user 0m0.413s
>>> sys 0m0.078s
>>> --
>>> real 0m0.496s
>>> user 0m0.371s
>>> sys 0m0.112s
>>> --
>>> [...]
>>>
>>> The debug output of each run is identical.
>>>
>>> Looking at the 'user' and 'sys' lines, we can see
that samba_kcc wasn't
>>> doing
>>> anything extra in those 3 minutes. It was just waiting. For what, I
>>> don't know.
>>>
>>> Douglas
>>>
>>>
>>>
>>> >
>>> > On Wed, Feb 7, 2024 at 8:03?PM Douglas Bagnall <
>>> douglas.bagnall at catalyst.net.nz
>>> > <mailto:douglas.bagnall at catalyst.net.nz>> wrote:
>>> >
>>> > On 8/02/24 11:17, Elias Pereira wrote:
>>> > > Hi Douglas, thanks for the answer!!!
>>> > >
>>> > > samba kcc command = /path/to/samba_kcc --debug
>>> > >
>>> > >
>>> > > I ran the command and the result is in the pastebin
below, if
>>> you can
>>> > > take a look.
>>> > > https://pastebin.com/x6W2yWm7
<https://pastebin.com/x6W2yWm7>
>>> > <https://pastebin.com/x6W2yWm7
<https://pastebin.com/x6W2yWm7>>
>>> >
>>> > It looks to me like that was a successful run, but I
can't see if
>>> it was
>>> > slow.
>>> >
>>> > If I understand correctly, sometimes it is bad, and
sometimes it
>>> is not.
>>> > If you run in a loop, something like
>>> >
>>> > for i in $(seq 1 10); do time /usr/sbin/samba_kcc --debug;
sleep
>>> 5;done
>>> >
>>> > then you might find one that takes more than 40 seconds --
that is
>>> the
>>> > NT_STATUS_IO_TIMEOUT.
>>> >
>>> > Douglas
>>> >
>>> >
>>> > >
>>> > > But anyway, I don't think this is a
samba_kcc specific
>>> problem, and
>>> > > https://bugzilla.samba.org/show_bug.cgi?id=15568
>>> > <https://bugzilla.samba.org/show_bug.cgi?id=15568>
>>> > >
<https://bugzilla.samba.org/show_bug.cgi?id=15568
>>> >
<https://bugzilla.samba.org/show_bug.cgi?id=15568>>is due to the
>>> same
>>> > > underlying issue. I don't know what that is.
>>> > >
>>> > >
>>> > > I opened this bug, but no one has replied yet.
>>> > >
>>> > > >> UpdateRefs failed with
NT_STATUS_IO_TIMEOUT
>>> > > This is a different replication failure,
independent of the
>>> KCC.
>>> > > samba_kcc works out where the DC should
replicate from, and
>>> the drepl
>>> > > code does the actual replication.
>>> > > Both could fail with network weirdness, or if
either the
>>> local or remote
>>> > > database is locked up for some reason.
>>> > >
>>> > >
>>> > > And how can I debug this problem? I've searched
everywhere and
>>> found
>>> > > nothing about it.
>>> > >
>>> > > On Wed, Feb 7, 2024 at 6:40?PM Douglas Bagnall
>>> > > <douglas.bagnall at catalyst.net.nz <mailto:
>>> douglas.bagnall at catalyst.net.nz>
>>> > > <mailto:douglas.bagnall at catalyst.net.nz
>>> > <mailto:douglas.bagnall at catalyst.net.nz>>>
wrote:
>>> > >
>>> > > On 8/02/24 02:11, Elias Pereira via samba wrote:
>>> > > > Any idea how I can debug this?
>>> > >
>>> > >
>>> > > >>> I don't know the answer to
that, but I think setting
>>> > > >>>
>>> > > >>> samba kcc command =
/usr/sbin/samba_kcc -d5
>>> > > >>>
>>> > >
>>> > > I looked it up. Actually
>>> > >
>>> > > samba kcc command = /path/to/samba_kcc
--debug
>>> > >
>>> > > *might* produce more interesting results
(you'll probably
>>> want to use
>>> > > `less -R` to view the logs).
>>> > >
>>> > > You can run samba_kcc independently -- it should
not do any
>>> harm, but
>>> > > you can use the --readonly flag if you want to
be careful.
>>> > >
>>> > > But anyway, I don't think this is a
samba_kcc specific
>>> problem, and
>>> > > https://bugzilla.samba.org/show_bug.cgi?id=15568
>>> > <https://bugzilla.samba.org/show_bug.cgi?id=15568>
>>> > >
<https://bugzilla.samba.org/show_bug.cgi?id=15568
>>> >
<https://bugzilla.samba.org/show_bug.cgi?id=15568>>
>>> > > is due to the same underlying issue. I don't
know what that
>>> is.
>>> > >
>>> > > >> [2024/02/02 20:05:17.752322, 0]
>>> > > >>
>>> > >
>>> >
>>>
../../source4/dsdb/repl/drepl_out_helpers.c:1300(dreplsrv_update_refs_done)
>>> > > >> UpdateRefs failed with
NT_STATUS_IO_TIMEOUT
>>> > >
>>> > > This is a different replication failure,
independent of the
>>> KCC.
>>> > >
>>> > > samba_kcc works out where the DC should
replicate from, and
>>> the drepl
>>> > > code does the actual replication.
>>> > >
>>> > > Both could fail with network weirdness, or if
either the
>>> local or
>>> > > remote
>>> > > database is locked up for some reason.
>>> > >
>>> > > >> What does this part of the code do?
>>> > > >> source4/dsdb/kcc/kcc_periodic.c:790
>>> > >
>>> > > That just reports that samba_kcc failed.
>>> > >
>>> > > cheers,
>>> > > Douglas
>>> > >
>>> > >
>>> > >
>>> > > --
>>> > > Elias Pereira
>>> >
>>> >
>>> >
>>> > --
>>> > Elias Pereira
>>>
>>>
>>
>> --
>> Elias Pereira
>>
>
>
> --
> Elias Pereira
>
--
Elias Pereira