Hai, I snapped this part of you logs.> Successful AuthZ: [DCE/RPC,krb5] user [COMPANY]\[DC6$] [S-1-5-21-2660373802-310620142-1895175072-6626] at [Fri, 26 Apr 2019 10:16:20.224329 PDT] Remote host [ipv4:10.14.16.11:35006] local host [ipv4:10.12.16.11:49153] > [2019/04/26 10:16:23.503632, 0] ../../source4/lib/cmdline/popt_common.c:74(popt_s4_talloc_log_fn) > Bad talloc magic value - unknown value > [2019/04/26 10:16:23.503698, 0] ../../lib/util/fault.c:128(smb_panic_default) > smb_panic_default: PANIC (pid 8888): Bad talloc magic value - unknown value > [2019/04/26 10:16:23.505811, 0] ../../lib/util/fault.c:261(log_stack_trace) > BACKTRACE: 50 stack frames: > #0 /usr/lib/x86_64-linux-gnu/libsamba-util.so.0(log_stack_trace+0x30) [0x7fe1294e7ba0] > #1 /usr/lib/x86_64-linux-gnu/libsamba-util.so.0(smb_panic+0x4b) [0x7fe1294e7cab] > #2 /usr/lib/x86_64-linux-gnu/libtalloc.so.2(talloc_strdup+0x305) [0x7fe127677d15] > #3 /usr/lib/x86_64-linux-gnu/libldb.so.1(+0x15f4f) [0x7fe12724bf4f] > #4 /usr/lib/x86_64-linux-gnu/ldb/modules/ldb/samba/objectclass_attrs.so(+0x2461) [0x7fe10fd6f461] > . > . > . > [2019/04/26 10:37:29.854836, 0] ../../source4/smbd/process_standard.c:160(standard_child_pipe_handler) > standard_child_pipe_handler: Child 9937 (kcc) terminated with signal 6 > > ==> This last line about (kcc) terminated may hold more cluesYes, not only the last line, this complete part, this is an ubuntu server and debian/ubuntu these kind of errors.. Well, thats long ago that i have seen things like that. Anf because of that i can see im 90% sure your problem is due to the DNS setup. If its wrong packages, based on this, that post the requested package version info, i'll check.> Bad talloc magic value - unknown valuewhich version of talloc is used/installed? And to be sure, run this. Run : dpkg -l |egrep "samba|winbin|?db|tevent|talloc|nss|wrapper" And post it on the list. Now going through the logs i noticed that. 10.14.16.11, the problem ip is a DC and the DC's are NOT supported in bind9_flat files.> Successful AuthZ: [DCE/RPC,krb5] user [COMPANY]\[DC6$] [S-1-5-21-2660373802-310620142-1895175072-6626] at [Fri, 26 Apr 2019 10:16:20.224329 PDT] Remote host [ipv4:10.14.16.11:35006] local host [ipv4:10.12.16.11:49153]Line 855: >> forwarders { 10.14.16.11; 10.14.16.12; }; zone "sql01.company.tld" {>> type forward; >> >> forwarders { 10.14.16.11; 10.14.16.12; }; >> >> };>> zone "14.10.in-addr.arpa" { >> type forward; >> >> forwarders { 10.14.16.11; 10.14.16.12; }; >> >> };So basicly, for every zone where you use samba AD, these must be in bind_DLZ and not in flat files. Review you setup base on this, and if you have question ask again. Greetz, Louis
Hi Louis, In the past few days I’ve removed all bind flat file configs from my environment, and I’ve checked carefully that all DCs are replicating and that all changes on any DC eventually replicate cleanly to all other DCs, I’ve checked resolv.conf on all the DCs as well and they all have at least two other IPs of other DC in them. I believe you said that the first IP should be the IP of the local host, but I haven’t done that on every server yet. I’m running dc4 on Ubuntu 18.04 using your samba packages. All other samba DCs are running 4.9.3 that I’ve compiled previously on Ubuntu 16.04. This same 4.9.3 package is running without any kcc errors or process PANICs on another site I manage. Also, one DC is Windows 2008 R2 (WDC1) Every time I start samba AD DC on 18.04 with your packages or on 16.04 with my own packages, the samba kccsvr ( ├─6615 samba: task[kccsrv] ) task starts with all other samba components and runs for about 10-12 seconds and then goes to PANIC and crashes as shown in the logs below. After that ‘samba-tool drs showrepl’ always fails. I don’t know how to tell if I’m using talloc/tdb from Samba source or from the OS. I believe it’s from source because I always compile on a new, clean system and I don’t install any talloc/tdb or samba packages to prepare the system for compile. I’ve checked versions as you’ve requested. This version list is from DC4, with your packages. ubuntu at dc4:~$ dpkg -l |egrep "samba|winbin|?db|tevent|talloc|nss|wrapper" ii dbus 1.12.2-1ubuntu1 amd64 simple interprocess messaging system (daemon and utilities) ii ldb-tools 2:1.5.4-0ubuntu1.1 amd64 LDAP-like embedded database - tools ii libdb5.3:amd64 5.3.28-13.1ubuntu1 amd64 Berkeley v5.3 Database Libraries [runtime] ii libdbus-1-3:amd64 1.12.2-1ubuntu1 amd64 simple interprocess messaging system (library) ii libgdbm-compat4:amd64 1.14.1-6 amd64 GNU dbm database routines (legacy support runtime version) ii libgdbm5:amd64 1.14.1-6 amd64 GNU dbm database routines (runtime version) ii libjansson4:amd64 2.11-1 amd64 C library for encoding, decoding and manipulating JSON data ii libkdb5-9:amd64 1.16-2ubuntu0.1 amd64 MIT Kerberos runtime libraries - Kerberos database ii libldb1:amd64 2:1.5.4-0ubuntu1.1 amd64 LDAP-like embedded database - shared library ii libnss-systemd:amd64 237-3ubuntu10.21 amd64 nss module providing dynamic user and group name resolution ii libnss-winbind:amd64 2:4.10.2+nmu-0ubuntu1 amd64 Samba nameservice integration plugins ii libnss3:amd64 2:3.35-2ubuntu2.2 amd64 Network Security Service libraries ii libpam-winbind:amd64 2:4.10.2+nmu-0ubuntu1 amd64 Windows domain authentication integration plugin ii libsasl2-modules-db:amd64 2.1.27~101-g0780600+dfsg-3ubuntu2 amd64 Cyrus SASL - pluggable authentication modules (DB) ii libtalloc2:amd64 2.1.16-0ubuntu1.1 amd64 hierarchical pool based memory allocator ii libtdb1:amd64 1.3.18-0ubuntu1.1 amd64 Trivial Database - shared library ii libtevent0:amd64 0.9.39-0ubuntu1.1 amd64 talloc-based event loop library - shared library ii libwbclient0:amd64 2:4.10.2+nmu-0ubuntu1 amd64 Samba winbind client library ii libwrap0:amd64 7.6.q-27 amd64 Wietse Venema's TCP wrappers library ii libxmlsec1-openssl:amd64 1.2.25-1build1 amd64 Openssl engine for the XML security library ii man-db 2.8.3-2ubuntu0.1 amd64 on-line manual pager ii openssh-client 1:7.6p1-4ubuntu0.3 amd64 secure shell (SSH) client, for secure access to remote machines ii openssh-server 1:7.6p1-4ubuntu0.3 amd64 secure shell (SSH) server, for secure access from remote machines ii openssh-sftp-server 1:7.6p1-4ubuntu0.3 amd64 secure shell (SSH) sftp server module, for SFTP access from remote machines ii openssl 1.1.0g-2ubuntu4.3 amd64 Secure Sockets Layer toolkit - cryptographic utility ii python-dbus 1.2.6-1 amd64 simple interprocess messaging system (Python interface) ii python-gnupg 0.4.1-1ubuntu1 all Python wrapper for the GNU Privacy Guard (Python 2.x) ii python-m2crypto 0.27.0-5 amd64 Python wrapper for the OpenSSL library ii python-openssl 17.5.0-1ubuntu1 all Python 2 wrapper around the OpenSSL library ii python3-click 6.7-3 all Simple wrapper around optparse for powerful command line utilities - Python 3.x ii python3-dbus 1.2.6-1 amd64 simple interprocess messaging system (Python 3 interface) ii python3-gdbm:amd64 3.6.7-1~18.04 amd64 GNU dbm database support for Python 3.x ii python3-ldb 2:1.5.4-0ubuntu1.1 amd64 Python 3 bindings for LDB ii python3-openssl 17.5.0-1ubuntu1 all Python 3 wrapper around the OpenSSL library ii python3-samba 2:4.10.2+nmu-0ubuntu1 amd64 Python 3 bindings for Samba ii python3-talloc 2.1.16-0ubuntu1.1 amd64 hierarchical pool based memory allocator - Python3 bindings ii python3-tdb 1.3.18-0ubuntu1.1 amd64 Python3 bindings for TDB ii samba 2:4.10.2+nmu-0ubuntu1 amd64 SMB/CIFS file, print, and login server for Unix ii samba-common 2:4.10.2+nmu-0ubuntu1 all common files used by both the Samba server and client ii samba-common-bin 2:4.10.2+nmu-0ubuntu1 amd64 Samba common files used by both the server and the client ii samba-dsdb-modules:amd64 2:4.10.2+nmu-0ubuntu1 amd64 Samba Directory Services Database ii samba-libs:amd64 2:4.10.2+nmu-0ubuntu1 amd64 Samba core libraries ii samba-vfs-modules:amd64 2:4.10.2+nmu-0ubuntu1 amd64 Samba Virtual FileSystem plugins ii tdb-tools 1.3.18-0ubuntu1.1 amd64 Trivial Database - bundled binaries ii winbind 2:4.10.2+nmu-0ubuntu1 amd64 service to resolve user and group information from Windows NT servers ii wireless-regdb 2018.05.09-0ubuntu1~18.04.1 all wireless regulatory database This is from DC5 with my packages. You’ll note that this list shows "samba-common 2:4.3.11+dfsg-0ubuntu0.16.04.12” but this is only the folder structure and file structure created by 4.3.11 Ubuntu package. I found out the hard way that if I purge that package, it deletes my entire /var/lib/samba directory, so I had to re-build one of my DC’s from scratch. :( =ubuntu at dc5:~$ dpkg -l |egrep "samba|winbin|?db|tevent|talloc|nss|wrapper" ii dbus 1.10.6-1ubuntu3.3 amd64 simple interprocess messaging system (daemon and utilities) ii insserv 1.14.0-5ubuntu3 amd64 boot sequence organizer using LSB init.d script dependency information ii libdb5.3:amd64 5.3.28-11ubuntu0.1 amd64 Berkeley v5.3 Database Libraries [runtime] ii libdbus-1-3:amd64 1.10.6-1ubuntu3.3 amd64 simple interprocess messaging system (library) ii libdbus-glib-1-2:amd64 0.106-1 amd64 simple interprocess messaging system (GLib-based shared library) ii libevent-openssl-2.0-5:amd64 2.0.21-stable-2ubuntu0.16.04.1 amd64 Asynchronous event notification library (openssl) ii libgdbm3:amd64 1.8.3-13.1 amd64 GNU dbm database routines (runtime version) ii libgmpxx4ldbl:amd64 2:6.1.0+dfsg-2 amd64 Multiprecision arithmetic library (C++ bindings) ii libgnutls-openssl27:amd64 3.4.10-4ubuntu1.4 amd64 GNU TLS library - OpenSSL wrapper ii libjansson-dev:amd64 2.7-3ubuntu0.1 amd64 C library for encoding, decoding and manipulating JSON data (dev) ii libjansson4:amd64 2.7-3ubuntu0.1 amd64 C library for encoding, decoding and manipulating JSON data ii libkdb5-8:amd64 1.13.2+dfsg-5ubuntu2.1 amd64 MIT Kerberos runtime libraries - Kerberos database ii liblmdb-dev:amd64 0.9.17-3 amd64 Lightning Memory-Mapped Database development files ii liblmdb0:amd64 0.9.17-3 amd64 Lightning Memory-Mapped Database shared library ii libpython-dbg:amd64 2.7.12-1~16.04 amd64 debug build of the Python Interpreter (version 2.7) ii libpython2.7-dbg:amd64 2.7.12-1ubuntu0~16.04.4 amd64 Debug Build of the Python Interpreter (version 2.7) ii libsasl2-modules-db:amd64 2.1.26.dfsg1-14ubuntu0.1 amd64 Cyrus SASL - pluggable authentication modules (DB) ii libtalloc2:amd64 2.1.5-2 amd64 hierarchical pool based memory allocator ii libwrap0:amd64 7.6.q-25 amd64 Wietse Venema's TCP wrappers library ii libxmlsec1-openssl 1.2.20-2ubuntu4 amd64 Openssl engine for the XML security library ii lmdb-doc 0.9.17-3 all Lightning Memory-Mapped Database doxygen documentation ii lmdb-utils 0.9.17-3 amd64 Lightning Memory-Mapped Database Utilities ii man-db 2.7.5-1 amd64 on-line manual pager ii openssh-client 1:7.2p2-4ubuntu2.8 amd64 secure shell (SSH) client, for secure access to remote machines ii openssh-server 1:7.2p2-4ubuntu2.8 amd64 secure shell (SSH) server, for secure access from remote machines ii openssh-sftp-server 1:7.2p2-4ubuntu2.8 amd64 secure shell (SSH) sftp server module, for SFTP access from remote machines ii openssl 1.0.2g-1ubuntu4.15 amd64 Secure Sockets Layer toolkit - cryptographic utility ii python-dbg 2.7.12-1~16.04 amd64 debug build of the Python Interpreter (version 2.7) ii python-gnupg 0.3.8-2 all Python wrapper for the GNU Privacy Guard (Python 2.x) ii python-gpgme 0.3-1.1 amd64 python wrapper for the GPGME library ii python-m2crypto 0.22.6~rc4-1ubuntu1 amd64 Python wrapper for the OpenSSL library ii python-openssl 0.15.1-2ubuntu0.2 all Python 2 wrapper around the OpenSSL library ii python2.7-dbg 2.7.12-1ubuntu0~16.04.4 amd64 Debug Build of the Python Interpreter (version 2.7) ii python3-dbus 1.2.0-3 amd64 simple interprocess messaging system (Python 3 interface) ii python3-gdbm:amd64 3.5.1-1 amd64 GNU dbm database support for Python 3.x ii python3-gpgme 0.3-1.1 amd64 python wrapper for the GPGME library (Python 3) hi samba 4.9.3-1 amd64 samba build by myCompany hc samba-common 2:4.3.11+dfsg-0ubuntu0.16.04.12 all common files used by both the Samba server and client ii tcpd 7.6.q-25 amd64 Wietse Venema's TCP wrapper utilities ii wireless-regdb 2018.05.09-0ubuntu1~16.04.1 all wireless regulatory database> On Apr 29, 2019, at 12:37 AM, L.P.H. van Belle via samba <samba at lists.samba.org> wrote: > > Hai, > > I snapped this part of you logs. > >> Successful AuthZ: [DCE/RPC,krb5] user [COMPANY]\[DC6$] [S-1-5-21-2660373802-310620142-1895175072-6626] at [Fri, 26 Apr 2019 10:16:20.224329 PDT] Remote host [ipv4:10.14.16.11:35006] local host [ipv4:10.12.16.11:49153] >> [2019/04/26 10:16:23.503632, 0] ../../source4/lib/cmdline/popt_common.c:74(popt_s4_talloc_log_fn) >> Bad talloc magic value - unknown value >> [2019/04/26 10:16:23.503698, 0] ../../lib/util/fault.c:128(smb_panic_default) >> smb_panic_default: PANIC (pid 8888): Bad talloc magic value - unknown value >> [2019/04/26 10:16:23.505811, 0] ../../lib/util/fault.c:261(log_stack_trace) >> BACKTRACE: 50 stack frames: >> #0 /usr/lib/x86_64-linux-gnu/libsamba-util.so.0(log_stack_trace+0x30) [0x7fe1294e7ba0] >> #1 /usr/lib/x86_64-linux-gnu/libsamba-util.so.0(smb_panic+0x4b) [0x7fe1294e7cab] >> #2 /usr/lib/x86_64-linux-gnu/libtalloc.so.2(talloc_strdup+0x305) [0x7fe127677d15] >> #3 /usr/lib/x86_64-linux-gnu/libldb.so.1(+0x15f4f) [0x7fe12724bf4f] >> #4 /usr/lib/x86_64-linux-gnu/ldb/modules/ldb/samba/objectclass_attrs.so(+0x2461) [0x7fe10fd6f461] >> . >> . >> . >> [2019/04/26 10:37:29.854836, 0] ../../source4/smbd/process_standard.c:160(standard_child_pipe_handler) >> standard_child_pipe_handler: Child 9937 (kcc) terminated with signal 6 >> >> ==>> This last line about (kcc) terminated may hold more clues > > > Yes, not only the last line, this complete part, this is an ubuntu server and debian/ubuntu these kind of errors.. > Well, thats long ago that i have seen things like that. > Anf because of that i can see im 90% sure your problem is due to the DNS setup. > > If its wrong packages, based on this, that post the requested package version info, i'll check. >> Bad talloc magic value - unknown value > which version of talloc is used/installed? > > And to be sure, run this. > Run : dpkg -l |egrep "samba|winbin|?db|tevent|talloc|nss|wrapper" > > And post it on the list. > > Now going through the logs i noticed that. > > > 10.14.16.11, the problem ip is a DC and the DC's are NOT supported in bind9_flat files. > > >> Successful AuthZ: [DCE/RPC,krb5] user [COMPANY]\[DC6$] [S-1-5-21-2660373802-310620142-1895175072-6626] at [Fri, 26 Apr 2019 10:16:20.224329 PDT] Remote host [ipv4:10.14.16.11:35006] local host [ipv4:10.12.16.11:49153] > Line 855: >> forwarders { 10.14.16.11; 10.14.16.12; }; > > zone "sql01.company.tld" { >>> type forward; >>> >>> forwarders { 10.14.16.11; 10.14.16.12; }; >>> >>> }; > >>> zone "14.10.in-addr.arpa" { >>> type forward; >>> >>> forwarders { 10.14.16.11; 10.14.16.12; }; >>> >>> }; > > So basicly, for every zone where you use samba AD, these must be in bind_DLZ and not in flat files. > > Review you setup base on this, and if you have question ask again. > > Greetz, > > Louis > > > > -- > To unsubscribe from this list go to the following URL and read the > instructions: https://lists.samba.org/mailman/options/samba
Hai Mason,> -----Oorspronkelijk bericht----- > Van: M B [mailto:mmx at exm0.net] > Verzonden: dinsdag 30 april 2019 20:42 > Aan: L.P.H. van Belle; samba at lists.samba.org > Onderwerp: Re: [Samba] Replication failures > > Hi Louis, > > In the past few days I’ve removed all bind flat file configs > from my environment, and I’ve checked carefully that all DCs > are replicating and that all changes on any DC eventually > replicate cleanly to all other DCsOk, so to confirm, your replication is ok now? If you think yes, then get en review the setting in this script. wget https://raw.githubusercontent.com/thctlo/samba4/master/samba-check-db-repl.sh Run it from every dc and post the outputs.> > I’ve checked resolv.conf on all the DCs as well and they all > have at least two other IPs of other DC in them. I believe > you said that the first IP should be the IP of the local > host, but I haven’t done that on every server yet.Yes, but you change that after the join and after you check replication is ok. What i always do is, join, reboot, check replication, change dns, reboot, and verify replication again. This order.> > I’m running dc4 on Ubuntu 18.04 using your samba packages. > All other samba DCs are running 4.9.3 that I’ve compiled > previously on Ubuntu 16.04. This same 4.9.3 package is > running without any kcc errors or process PANICs on another > site I manage. > Also, one DC is Windows 2008 R2 (WDC1) > > Every time I start samba AD DC on 18.04 with your packages or > on 16.04 with my own packages, the samba kccsvr ( ??????6615 > samba: task[kccsrv] ) task starts with all other samba > components and runs for about 10-12 seconds and then goes to > PANIC and crashes as shown in the logs below. After that > ‘samba-tool drs showrepl’ always fails.On the server, set log level = 10 A pain yes, but i dont see directly whats wrong here. Before a log level 10 post, run on the DC with my packages this again. https://github.com/thctlo/samba4/blob/master/samba-collect-debug-info.sh Pm me the unmodified output, i'll re-check that. What i suspect is a damaged AD or DNS or both. It's just hard to find, but if AD is replication now, it must be something in the DNS. I can't tell jet.> > I don’t know how to tell if I’m using talloc/tdb from Samba > source or from the OS. I believe it’s from source because I > always compile on a new, clean system and I don’t install any > talloc/tdb or samba packages to prepare the system for compile. > > I’ve checked versions as you’ve requested. This version list > is from DC4, with your packages. > > ubuntu at dc4:~$ dpkg -l |egrep > "samba|winbin|?db|tevent|talloc|nss|wrapper" > ii dbus 1.12.2-1ubuntu1 > amd64 simple interprocess messaging > system (daemon and utilities) > .... Shorted this a bit. > 2018.05.09-0ubuntu1~18.04.1 all wireless > regulatory database >This looks ok.> > This is from DC5 with my packages. You’ll note that this list > shows "samba-common 2:4.3.11+dfsg-0ubuntu0.16.04.12” but > this is only the folder structure and file structure created > by 4.3.11 Ubuntu package. I found out the hard way that if I > purge that package, it deletes my entire /var/lib/samba > directory, so I had to re-build one of my DC’s from scratch. :(Au, yes, the other option was to run : apt dist-upgrade What should have upgraded that package. Hard, but this way we learn quicker, and.. I know you feeling ;-)> => ubuntu at dc5:~$ dpkg -l |egrep > "samba|winbin|?db|tevent|talloc|nss|wrapper" > ii dbus 1.10.6-1ubuntu3.3 > amd64 simple interprocess....> 2018.05.09-0ubuntu1~16.04.1 all > wireless regulatory databaseHere also left overs. In samba packages. The sources build does include tallec/tevent/tdb/ldb so you dont see these in the list. And i dont know how you create your samba 4.9.3 package so this is a bit hard to tell. I suggest, Stop samba, backup you /var/{lib,cache}/samba/ and /etc/samba apt remove --purge samba-common samba --autoremove And install the 4.9.3 back. Or, upgrade to ubuntu 18.04 and setup my 4.9 repo. Or use my repo and rebuild the packages for your own. Greetz, Louis> > > On Apr 29, 2019, at 12:37 AM, L.P.H. van Belle via samba > <samba at lists.samba.org> wrote: > > > > Hai, > > > > I snapped this part of you logs. > > > >> Successful AuthZ: [DCE/RPC,krb5] user [COMPANY]\[DC6$] > [S-1-5-21-2660373802-310620142-1895175072-6626] at [Fri, 26 > Apr 2019 10:16:20.224329 PDT] Remote host > [ipv4:10.14.16.11:35006] local host [ipv4:10.12.16.11:49153] > >> [2019/04/26 10:16:23.503632, 0] > ../../source4/lib/cmdline/popt_common.c:74(popt_s4_talloc_log_fn) > >> Bad talloc magic value - unknown value > >> [2019/04/26 10:16:23.503698, 0] > ../../lib/util/fault.c:128(smb_panic_default) > >> smb_panic_default: PANIC (pid 8888): Bad talloc magic > value - unknown value > >> [2019/04/26 10:16:23.505811, 0] > ../../lib/util/fault.c:261(log_stack_trace) > >> BACKTRACE: 50 stack frames: > >> #0 > /usr/lib/x86_64-linux-gnu/libsamba-util.so.0(log_stack_trace+0 > x30) [0x7fe1294e7ba0] > >> #1 > /usr/lib/x86_64-linux-gnu/libsamba-util.so.0(smb_panic+0x4b) > [0x7fe1294e7cab] > >> #2 > /usr/lib/x86_64-linux-gnu/libtalloc.so.2(talloc_strdup+0x305) > [0x7fe127677d15] > >> #3 /usr/lib/x86_64-linux-gnu/libldb.so.1(+0x15f4f) > [0x7fe12724bf4f] > >> #4 > /usr/lib/x86_64-linux-gnu/ldb/modules/ldb/samba/objectclass_at > trs.so(+0x2461) [0x7fe10fd6f461] > >> . > >> . > >> . > >> [2019/04/26 10:37:29.854836, 0] > ../../source4/smbd/process_standard.c:160(standard_child_pipe_handler) > >> standard_child_pipe_handler: Child 9937 (kcc) terminated > with signal 6 > >> > >> ==> >> This last line about (kcc) terminated may hold more clues > > > > > > Yes, not only the last line, this complete part, this is an > ubuntu server and debian/ubuntu these kind of errors.. > > Well, thats long ago that i have seen things like that. > > Anf because of that i can see im 90% sure your problem is > due to the DNS setup. > > > > If its wrong packages, based on this, that post the > requested package version info, i'll check. > >> Bad talloc magic value - unknown value > > which version of talloc is used/installed? > > > > And to be sure, run this. > > Run : dpkg -l |egrep "samba|winbin|?db|tevent|talloc|nss|wrapper" > > > > And post it on the list. > > > > Now going through the logs i noticed that. > > > > > > 10.14.16.11, the problem ip is a DC and the DC's are NOT > supported in bind9_flat files. > > > > > >> Successful AuthZ: [DCE/RPC,krb5] user [COMPANY]\[DC6$] > [S-1-5-21-2660373802-310620142-1895175072-6626] at [Fri, 26 > Apr 2019 10:16:20.224329 PDT] Remote host > [ipv4:10.14.16.11:35006] local host [ipv4:10.12.16.11:49153] > > Line 855: >> forwarders { 10.14.16.11; 10.14.16.12; }; > > > > zone "sql01.company.tld" { > >>> type forward; > >>> > >>> forwarders { 10.14.16.11; 10.14.16.12; }; > >>> > >>> }; > > > >>> zone "14.10.in-addr.arpa" { > >>> type forward; > >>> > >>> forwarders { 10.14.16.11; 10.14.16.12; }; > >>> > >>> }; > > > > So basicly, for every zone where you use samba AD, these > must be in bind_DLZ and not in flat files. > > > > Review you setup base on this, and if you have question ask again. > > > > Greetz, > > > > Louis > > > > > > > > -- > > To unsubscribe from this list go to the following URL and read the > > instructions: https://lists.samba.org/mailman/options/samba > >
New observations: 1. "samba: task[kccsrv]" always goes to PANIC around 15-16 seconds after samba starts 2. I have three sites and the automatic" NTDS Settings" links between sites are not being generated consistently. I’ve had to manually create some NTDS Seting replication links, especially after I demote/rejoin any DC. I’m guessing the “kccsrv” process should manage these links automatically but it’s crashing so it can not create appropriate links. It seems that links within a site are created automatically, but not necessarily links between sites. I’ve seen links created automatically in some newly re-joined DCs, but not in existing DCs back to the newly re-joined DCs samba-check-db-repl.sh output pasted below. I pasted results from only one DC. All others are similar. I do get some replication inconsistencies in DNS, but those go away if I run the script again as the differences get resolved> On May 1, 2019, at 2:25 AM, L.P.H. van Belle via samba <samba at lists.samba.org> wrote: > > Hai Mason, > > >> -----Oorspronkelijk bericht----- >> Van: M B [mailto:mmx at exm0.net <mailto:mmx at exm0.net>] >> Verzonden: dinsdag 30 april 2019 20:42 >> Aan: L.P.H. van Belle; samba at lists.samba.org <mailto:samba at lists.samba.org> >> Onderwerp: Re: [Samba] Replication failures >> >> Hi Louis, >> >> In the past few days I’ve removed all bind flat file configs >> from my environment, and I’ve checked carefully that all DCs >> are replicating and that all changes on any DC eventually >> replicate cleanly to all other DCs > > Ok, so to confirm, your replication is ok now? > If you think yes, then get en review the setting in this script. > wget https://raw.githubusercontent.com/thctlo/samba4/master/samba-check-db-repl.sh <https://raw.githubusercontent.com/thctlo/samba4/master/samba-check-db-repl.sh> > Run it from every dc and post the outputs. > >> >> I’ve checked resolv.conf on all the DCs as well and they all >> have at least two other IPs of other DC in them. I believe >> you said that the first IP should be the IP of the local >> host, but I haven’t done that on every server yet. > > Yes, but you change that after the join and after you check replication is ok. > What i always do is, join, reboot, check replication, change dns, reboot, and verify replication again. > This order. > >> >> I’m running dc4 on Ubuntu 18.04 using your samba packages. >> All other samba DCs are running 4.9.3 that I’ve compiled >> previously on Ubuntu 16.04. This same 4.9.3 package is >> running without any kcc errors or process PANICs on another >> site I manage. >> Also, one DC is Windows 2008 R2 (WDC1) >> >> Every time I start samba AD DC on 18.04 with your packages or >> on 16.04 with my own packages, the samba kccsvr ( ??????6615 >> samba: task[kccsrv] ) task starts with all other samba >> components and runs for about 10-12 seconds and then goes to >> PANIC and crashes as shown in the logs below. After that >> ‘samba-tool drs showrepl’ always fails. > > On the server, set log level = 10 > A pain yes, but i dont see directly whats wrong here. > Before a log level 10 post, run on the DC with my packages this again. > https://github.com/thctlo/samba4/blob/master/samba-collect-debug-info.sh <https://github.com/thctlo/samba4/blob/master/samba-collect-debug-info.sh> > Pm me the unmodified output, i'll re-check that. > > What i suspect is a damaged AD or DNS or both. > It's just hard to find, but if AD is replication now, it must be something in the DNS. > I can't tell jet. > >> >> I don’t know how to tell if I’m using talloc/tdb from Samba >> source or from the OS. I believe it’s from source because I >> always compile on a new, clean system and I don’t install any >> talloc/tdb or samba packages to prepare the system for compile. >> >> I’ve checked versions as you’ve requested. This version list >> is from DC4, with your packages. >> >> ubuntu at dc4:~$ dpkg -l |egrep >> "samba|winbin|?db|tevent|talloc|nss|wrapper" >> ii dbus 1.12.2-1ubuntu1 >> amd64 simple interprocess messaging >> system (daemon and utilities) >> .... Shorted this a bit. >> 2018.05.09-0ubuntu1~18.04.1 all wireless >> regulatory database >> > > This looks ok. > >> >> This is from DC5 with my packages. You’ll note that this list >> shows "samba-common 2:4.3.11+dfsg-0ubuntu0.16.04.12” but >> this is only the folder structure and file structure created >> by 4.3.11 Ubuntu package. I found out the hard way that if I >> purge that package, it deletes my entire /var/lib/samba >> directory, so I had to re-build one of my DC’s from scratch. :( > > Au, yes, the other option was to run : apt dist-upgrade > What should have upgraded that package. > Hard, but this way we learn quicker, and.. I know you feeling ;-) > > >> =>> ubuntu at dc5:~$ dpkg -l |egrep >> "samba|winbin|?db|tevent|talloc|nss|wrapper" >> ii dbus 1.10.6-1ubuntu3.3 >> amd64 simple interprocess > .... >> 2018.05.09-0ubuntu1~16.04.1 all >> wireless regulatory database > > Here also left overs. In samba packages. > The sources build does include tallec/tevent/tdb/ldb so you dont see these in the list. > And i dont know how you create your samba 4.9.3 package so this is a bit hard to tell. > > I suggest, > Stop samba, backup you /var/{lib,cache}/samba/ and /etc/samba > apt remove --purge samba-common samba --autoremove > And install the 4.9.3 back. > Or, upgrade to ubuntu 18.04 and setup my 4.9 repo. > Or use my repo and rebuild the packages for your own. > > > > Greetz, > > Louis > > >Typical output from script: Running with with console output Checking the DC_With_FSMO (dc1) with SAMBA DC: dc5.my.company.tld dc4.my.company.tld dc7.my.company.tld dc6.my.company.tld dc2.my.company.tld Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc5.my.company.tld Please wait.. this can take a while.. cat /tmp/samba_ldapcmp_checkdb * Comparing [DOMAIN] context... * Objects to be compared: 1321 * Result for [DOMAIN]: SUCCESS * Comparing [CONFIGURATION] context... * Objects to be compared: 1713 * Result for [CONFIGURATION]: SUCCESS * Comparing [SCHEMA] context... * Objects to be compared: 1550 * Result for [SCHEMA]: SUCCESS * Comparing [DNSDOMAIN] context... * Objects to be compared: 1691 * Result for [DNSDOMAIN]: SUCCESS * Comparing [DNSFOREST] context... * Objects to be compared: 49 * Result for [DNSFOREST]: SUCCESS Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc4.my.company.tld Please wait.. this can take a while.. cat /tmp/samba_ldapcmp_checkdb * Comparing [DOMAIN] context... * Objects to be compared: 1321 * Result for [DOMAIN]: SUCCESS * Comparing [CONFIGURATION] context... * Objects to be compared: 1713 * Result for [CONFIGURATION]: SUCCESS * Comparing [SCHEMA] context... * Objects to be compared: 1550 * Result for [SCHEMA]: SUCCESS * Comparing [DNSDOMAIN] context... * Objects to be compared: 1691 * Result for [DNSDOMAIN]: SUCCESS * Comparing [DNSFOREST] context... * Objects to be compared: 49 * Result for [DNSFOREST]: SUCCESS Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc7.my.company.tld Please wait.. this can take a while.. cat /tmp/samba_ldapcmp_checkdb * Comparing [DOMAIN] context... * Objects to be compared: 1321 * Result for [DOMAIN]: SUCCESS * Comparing [CONFIGURATION] context... * Objects to be compared: 1713 * Result for [CONFIGURATION]: SUCCESS * Comparing [SCHEMA] context... * Objects to be compared: 1550 * Result for [SCHEMA]: SUCCESS * Comparing [DNSDOMAIN] context... * Objects to be compared: 1691 * Result for [DNSDOMAIN]: SUCCESS * Comparing [DNSFOREST] context... * Objects to be compared: 49 * Result for [DNSFOREST]: SUCCESS Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc6.my.company.tld Please wait.. this can take a while.. cat /tmp/samba_ldapcmp_checkdb * Comparing [DOMAIN] context... * Objects to be compared: 1321 * Result for [DOMAIN]: SUCCESS * Comparing [CONFIGURATION] context... * Objects to be compared: 1714 * Result for [CONFIGURATION]: SUCCESS * Comparing [SCHEMA] context... * Objects to be compared: 1550 * Result for [SCHEMA]: SUCCESS * Comparing [DNSDOMAIN] context... * Objects to be compared: 1691 * Result for [DNSDOMAIN]: SUCCESS * Comparing [DNSFOREST] context... * Objects to be compared: 49 * Result for [DNSFOREST]: SUCCESS Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc2.my.company.tld Please wait.. this can take a while.. cat /tmp/samba_ldapcmp_checkdb * Comparing [DOMAIN] context... * Objects to be compared: 1321 * Result for [DOMAIN]: SUCCESS * Comparing [CONFIGURATION] context... * Objects to be compared: 1714 * Result for [CONFIGURATION]: SUCCESS * Comparing [SCHEMA] context... * Objects to be compared: 1550 * Result for [SCHEMA]: SUCCESS * Comparing [DNSDOMAIN] context... * Objects to be compared: 1691 * Result for [DNSDOMAIN]: SUCCESS * Comparing [DNSFOREST] context... * Objects to be compared: 49 * Result for [DNSFOREST]: SUCCESS .. Next check.. Running : samba-tool drs showrepl grep -c "failed" /tmp/samba_drs_showrepl grep -c "successful" /tmp/samba_drs_showrepl failures don't match successes don't match failures don't match successes don't match failures don't match successes don't match failures don't match successes don't match failures don't match successes don't match if [ "${EMAIL_REPORT_ALWAYS}" = "yes" ] && [ -n "${EMAIL_REPORT_ADDRESS}" ]; then #cat /tmp/samba_drs_showrepl | ${SET_MAILTOOL} -s "SAMBA CHECK DB : showrepl results" $EMAIL_REPORT_ADDRESS ${SET_MAILTOOL} -s "SAMBA CHECK DB : showrepl results" $EMAIL_REPORT_ADDRESS < /tmp/samba_drs_showrepl #cat /tmp/samba_ldapcmp_checkdb | ${SET_MAILTOOL} -s "SAMBA CHECK DB : ldapcmp results" $EMAIL_REPORT_ADDRESS ${SET_MAILTOOL} -s "SAMBA CHECK DB : ldapcmp results" $EMAIL_REPORT_ADDRESS < /tmp/samba_ldapcmp_checkdb fi if [ "${SETREMOVELOG}" = "yes" ]; then if [ -f /tmp/samba_ldapcmp_checkdb ]; then rm /tmp/samba_ldapcmp_checkdb fi if [ -f /tmp/samba_drs_showrepl ]; then rm /tmp/samba_drs_showrepl fi fi
Hai Mason, I had a look at the debug output. on 1) why around 15-16 second, that i really dont know. im trying to figure that out. on 2) if DNS is inconsistance, that everything is unrelayable. This is really the first the that needs fixing. then we look again at the replication. The debug output still shows several messages about zones in flat files. I still do believe also that this has impact on your problem. Your bind config is still not correct, set it exactly as i''ve dont in the howto. first get everything running error free, then, add you own setting to them. for example: You ad-dc managed zones still need auth-nxdomain yes; And i'll have a look at the debug script again since i see it fails at the end. Im failing to see the big picture here, how your setup is done .. and that does not happen often. :-/ (hint https://www.diagrameditor.com/ ) What i suggest, or what i would do. Verify all needed dns records per server, per host. I'll sleep a night over this and maybe i can come up with some more. Your problem is not your samba, but DNS settings and maybe an inheratence from the past.. Greetz, Louis Van: M B [mailto:mmx at exm0.net] Verzonden: woensdag 1 mei 2019 14:44 Aan: L.P.H. van Belle; samba at lists.samba.org Onderwerp: Re: [Samba] Replication failures New observations: 1. "samba: task[kccsrv]" always goes to PANIC around 15-16 seconds after samba starts 2. I have three sites and the automatic" NTDS Settings" links between sites are not being generated consistently. I’ve had to manually create some NTDS Seting replication links, especially after I demote/rejoin any DC. I’m guessing the “kccsrv” process should manage these links automatically but it’s crashing so it can not create appropriate links. It seems that links within a site are created automatically, but not necessarily links between sites. I’ve seen links created automatically in some newly re-joined DCs, but not in existing DCs back to the newly re-joined DCs samba-check-db-repl.sh output pasted below. I pasted results from only one DC. All others are similar. I do get some replication inconsistencies in DNS, but those go away if I run the script again as the differences get resolved On May 1, 2019, at 2:25 AM, L.P.H. van Belle via samba <samba at lists.samba.org> wrote: Hai Mason, -----Oorspronkelijk bericht----- Van: M B [ MailScanner heeft een e-mail met mogelijk een poging tot fraude gevonden van "exm0.net" mailto:mmx at exm0.net] Verzonden: dinsdag 30 april 2019 20:42 Aan: L.P.H. van Belle; samba at lists.samba.org Onderwerp: Re: [Samba] Replication failures Hi Louis, In the past few days I’ve removed all bind flat file configs from my environment, and I’ve checked carefully that all DCs are replicating and that all changes on any DC eventually replicate cleanly to all other DCs Ok, so to confirm, your replication is ok now? If you think yes, then get en review the setting in this script. wget https://raw.githubusercontent.com/thctlo/samba4/master/samba-check-db-repl.sh Run it from every dc and post the outputs. I’ve checked resolv.conf on all the DCs as well and they all have at least two other IPs of other DC in them. I believe you said that the first IP should be the IP of the local host, but I haven’t done that on every server yet. Yes, but you change that after the join and after you check replication is ok. What i always do is, join, reboot, check replication, change dns, reboot, and verify replication again. This order. I’m running dc4 on Ubuntu 18.04 using your samba packages. All other samba DCs are running 4.9.3 that I’ve compiled previously on Ubuntu 16.04. This same 4.9.3 package is running without any kcc errors or process PANICs on another site I manage. Also, one DC is Windows 2008 R2 (WDC1) Every time I start samba AD DC on 18.04 with your packages or on 16.04 with my own packages, the samba kccsvr ( ??????6615 samba: task[kccsrv] ) task starts with all other samba components and runs for about 10-12 seconds and then goes to PANIC and crashes as shown in the logs below. After that ‘samba-tool drs showrepl’ always fails. On the server, set log level = 10 A pain yes, but i dont see directly whats wrong here. Before a log level 10 post, run on the DC with my packages this again. https://github.com/thctlo/samba4/blob/master/samba-collect-debug-info.sh Pm me the unmodified output, i'll re-check that. What i suspect is a damaged AD or DNS or both. It's just hard to find, but if AD is replication now, it must be something in the DNS. I can't tell jet. I don’t know how to tell if I’m using talloc/tdb from Samba source or from the OS. I believe it’s from source because I always compile on a new, clean system and I don’t install any talloc/tdb or samba packages to prepare the system for compile. I’ve checked versions as you’ve requested. This version list is from DC4, with your packages. ubuntu at dc4:~$ dpkg -l |egrep "samba|winbin|?db|tevent|talloc|nss|wrapper" ii dbus 1.12.2-1ubuntu1 amd64 simple interprocess messaging system (daemon and utilities) .... Shorted this a bit. 2018.05.09-0ubuntu1~18.04.1 all wireless regulatory database This looks ok. This is from DC5 with my packages. You’ll note that this list shows "samba-common 2:4.3.11+dfsg-0ubuntu0.16.04.12” but this is only the folder structure and file structure created by 4.3.11 Ubuntu package. I found out the hard way that if I purge that package, it deletes my entire /var/lib/samba directory, so I had to re-build one of my DC’s from scratch. :( Au, yes, the other option was to run : apt dist-upgrade What should have upgraded that package. Hard, but this way we learn quicker, and.. I know you feeling ;-) =ubuntu at dc5:~$ dpkg -l |egrep "samba|winbin|?db|tevent|talloc|nss|wrapper" ii dbus 1.10.6-1ubuntu3.3 amd64 simple interprocess .... 2018.05.09-0ubuntu1~16.04.1 all wireless regulatory database Here also left overs. In samba packages. The sources build does include tallec/tevent/tdb/ldb so you dont see these in the list. And i dont know how you create your samba 4.9.3 package so this is a bit hard to tell. I suggest, Stop samba, backup you /var/{lib,cache}/samba/ and /etc/samba apt remove --purge samba-common samba --autoremove And install the 4.9.3 back. Or, upgrade to ubuntu 18.04 and setup my 4.9 repo. Or use my repo and rebuild the packages for your own. Greetz, Louis Typical output from script: Running with with console output Checking the DC_With_FSMO (dc1) with SAMBA DC: dc5.my.company.tld dc4.my.company.tld dc7.my.company.tld dc6.my.company.tld dc2.my.company.tld Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc5.my.company.tld Please wait.. this can take a while.. cat /tmp/samba_ldapcmp_checkdb * Comparing [DOMAIN] context... * Objects to be compared: 1321 * Result for [DOMAIN]: SUCCESS * Comparing [CONFIGURATION] context... * Objects to be compared: 1713 * Result for [CONFIGURATION]: SUCCESS * Comparing [SCHEMA] context... * Objects to be compared: 1550 * Result for [SCHEMA]: SUCCESS * Comparing [DNSDOMAIN] context... * Objects to be compared: 1691 * Result for [DNSDOMAIN]: SUCCESS * Comparing [DNSFOREST] context... * Objects to be compared: 49 * Result for [DNSFOREST]: SUCCESS Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc4.my.company.tld Please wait.. this can take a while.. cat /tmp/samba_ldapcmp_checkdb * Comparing [DOMAIN] context... * Objects to be compared: 1321 * Result for [DOMAIN]: SUCCESS * Comparing [CONFIGURATION] context... * Objects to be compared: 1713 * Result for [CONFIGURATION]: SUCCESS * Comparing [SCHEMA] context... * Objects to be compared: 1550 * Result for [SCHEMA]: SUCCESS * Comparing [DNSDOMAIN] context... * Objects to be compared: 1691 * Result for [DNSDOMAIN]: SUCCESS * Comparing [DNSFOREST] context... * Objects to be compared: 49 * Result for [DNSFOREST]: SUCCESS Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc7.my.company.tld Please wait.. this can take a while.. cat /tmp/samba_ldapcmp_checkdb * Comparing [DOMAIN] context... * Objects to be compared: 1321 * Result for [DOMAIN]: SUCCESS * Comparing [CONFIGURATION] context... * Objects to be compared: 1713 * Result for [CONFIGURATION]: SUCCESS * Comparing [SCHEMA] context... * Objects to be compared: 1550 * Result for [SCHEMA]: SUCCESS * Comparing [DNSDOMAIN] context... * Objects to be compared: 1691 * Result for [DNSDOMAIN]: SUCCESS * Comparing [DNSFOREST] context... * Objects to be compared: 49 * Result for [DNSFOREST]: SUCCESS Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc6.my.company.tld Please wait.. this can take a while.. cat /tmp/samba_ldapcmp_checkdb * Comparing [DOMAIN] context... * Objects to be compared: 1321 * Result for [DOMAIN]: SUCCESS * Comparing [CONFIGURATION] context... * Objects to be compared: 1714 * Result for [CONFIGURATION]: SUCCESS * Comparing [SCHEMA] context... * Objects to be compared: 1550 * Result for [SCHEMA]: SUCCESS * Comparing [DNSDOMAIN] context... * Objects to be compared: 1691 * Result for [DNSDOMAIN]: SUCCESS * Comparing [DNSFOREST] context... * Objects to be compared: 49 * Result for [DNSFOREST]: SUCCESS Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc2.my.company.tld Please wait.. this can take a while.. cat /tmp/samba_ldapcmp_checkdb * Comparing [DOMAIN] context... * Objects to be compared: 1321 * Result for [DOMAIN]: SUCCESS * Comparing [CONFIGURATION] context... * Objects to be compared: 1714 * Result for [CONFIGURATION]: SUCCESS * Comparing [SCHEMA] context... * Objects to be compared: 1550 * Result for [SCHEMA]: SUCCESS * Comparing [DNSDOMAIN] context... * Objects to be compared: 1691 * Result for [DNSDOMAIN]: SUCCESS * Comparing [DNSFOREST] context... * Objects to be compared: 49 * Result for [DNSFOREST]: SUCCESS .. Next check.. Running : samba-tool drs showrepl grep -c "failed" /tmp/samba_drs_showrepl grep -c "successful" /tmp/samba_drs_showrepl failures don't match successes don't match failures don't match successes don't match failures don't match successes don't match failures don't match successes don't match failures don't match successes don't match if [ "${EMAIL_REPORT_ALWAYS}" = "yes" ] && [ -n "${EMAIL_REPORT_ADDRESS}" ]; then #cat /tmp/samba_drs_showrepl | ${SET_MAILTOOL} -s "SAMBA CHECK DB : showrepl results" $EMAIL_REPORT_ADDRESS ${SET_MAILTOOL} -s "SAMBA CHECK DB : showrepl results" $EMAIL_REPORT_ADDRESS < /tmp/samba_drs_showrepl #cat /tmp/samba_ldapcmp_checkdb | ${SET_MAILTOOL} -s "SAMBA CHECK DB : ldapcmp results" $EMAIL_REPORT_ADDRESS ${SET_MAILTOOL} -s "SAMBA CHECK DB : ldapcmp results" $EMAIL_REPORT_ADDRESS < /tmp/samba_ldapcmp_checkdb fi if [ "${SETREMOVELOG}" = "yes" ]; then if [ -f /tmp/samba_ldapcmp_checkdb ]; then rm /tmp/samba_ldapcmp_checkdb fi if [ -f /tmp/samba_drs_showrepl ]; then rm /tmp/samba_drs_showrepl fi fi