Rowland penny
2016-Mar-02 20:59 UTC
[Samba] Samba AD/DC crashed again, third time in as many months
On 02/03/16 20:46, IT Admin wrote:> Any further information required? Any troubleshooting steps? I need to > get this domain functional ASAP. > > Please advise. > > JS > On Mar 2, 2016 11:39 AM, "IT Admin" <it at cliffbells.com> wrote: > >> Hello to the samba users list again. I've got an emergency here, a Samba >> AD DC I've got deployed has crashed again, this is the third time since >> 12/17/2015 that this domain has failed completely. >> >> No power outages or unexpected shutdowns, samba simply fails to start, it >> appears my database is yet again corrupt: >> >> [2016/03/02 11:27:17.025235, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: dsdb_get_schema: refresh_fn() failed >> [2016/03/02 11:27:17.025346, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.025377, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module schema_load initialization failed : Operations error >> [2016/03/02 11:27:17.025404, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module rootdse initialization failed : Operations error >> [2016/03/02 11:27:17.025431, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module samba_dsdb initialization failed : Operations error >> [2016/03/02 11:27:17.025460, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: Unable to load modules for /usr/local/samba/private/sam.ldb: >> schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.032681, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: dsdb_get_schema: refresh_fn() failed >> [2016/03/02 11:27:17.033626, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> [2016/03/02 11:27:17.033776, 0] >> ../source4/smbd/service_task.c:35(task_server_terminate) >> task_server_terminate: [nbtd failed to open samdb] >> ldb: schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.034676, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module schema_load initialization failed : Operations error >> [2016/03/02 11:27:17.035224, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module rootdse initialization failed : Operations error >> [2016/03/02 11:27:17.035838, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module samba_dsdb initialization failed : Operations error >> [2016/03/02 11:27:17.036392, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: Unable to load modules for /usr/local/samba/private/sam.ldb: >> schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.038675, 0] >> ../source4/smbd/service_task.c:35(task_server_terminate) >> [2016/03/02 11:27:17.040353, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> [2016/03/02 11:27:17.042764, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: dsdb_get_schema: refresh_fn() failed >> [2016/03/02 11:27:17.042857, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.042887, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module schema_load initialization failed : Operations error >> [2016/03/02 11:27:17.042917, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module rootdse initialization failed : Operations error >> [2016/03/02 11:27:17.042942, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module samba_dsdb initialization failed : Operations error >> [2016/03/02 11:27:17.042971, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: Unable to load modules for /usr/local/samba/private/sam.ldb: >> schema_load_init: dsdb_get_schema failed >> ldb: dsdb_get_schema: refresh_fn() failed >> [2016/03/02 11:27:17.045981, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.046598, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module schema_load initialization failed : Operations error >> [2016/03/02 11:27:17.047302, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> [2016/03/02 11:27:17.047523, 0] >> ../source4/smbd/service_task.c:35(task_server_terminate) >> task_server_terminate: [dreplsrv: Failed to connect to local samdb: >> WERR_DS_UNAVAILABLE >> ] >> ldb: module rootdse initialization failed : Operations error >> [2016/03/02 11:27:17.048503, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module samba_dsdb initialization failed : Operations error >> [2016/03/02 11:27:17.048540, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: Unable to load modules for /usr/local/samba/private/sam.ldb: >> schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.050402, 0] >> ../source4/smbd/service_task.c:35(task_server_terminate) >> task_server_terminate: [kdc: krb5_init_context samdb connect failed] >> task_server_terminate: [cldapd failed to open samdb] >> [2016/03/02 11:27:17.072670, 0] >> ../source4/smbd/server.c:211(samba_terminate) >> samba_terminate: dreplsrv: Failed to connect to local samdb: >> WERR_DS_UNAVAILABLE >> >> [2016/03/02 11:27:17.073896, 0] >> ../source4/smbd/server.c:211(samba_terminate) >> samba_terminate: nbtd failed to open samdb >> [2016/03/02 11:27:17.075031, 0] >> ../source4/smbd/server.c:211(samba_terminate) >> samba_terminate: cldapd failed to open samdb >> [2016/03/02 11:27:17.075614, 0] >> ../source4/smbd/server.c:211(samba_terminate) >> samba_terminate: kdc: krb5_init_context samdb connect failed >> [2016/03/02 11:27:17.086051, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: dsdb_get_schema: refresh_fn() failed >> [2016/03/02 11:27:17.091786, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: dsdb_get_schema: refresh_fn() failed >> [2016/03/02 11:27:17.091860, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.091890, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module schema_load initialization failed : Operations error >> [2016/03/02 11:27:17.091919, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module rootdse initialization failed : Operations error >> [2016/03/02 11:27:17.091946, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module samba_dsdb initialization failed : Operations error >> [2016/03/02 11:27:17.091974, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: Unable to load modules for /usr/local/samba/private/sam.ldb: >> schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.093919, 0] >> ../source4/smbd/service_task.c:35(task_server_terminate) >> task_server_terminate: [dns: samdb_connect failed] >> [2016/03/02 11:27:17.095175, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.095232, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module schema_load initialization failed : Operations error >> [2016/03/02 11:27:17.095261, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module rootdse initialization failed : Operations error >> [2016/03/02 11:27:17.095288, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module samba_dsdb initialization failed : Operations error >> [2016/03/02 11:27:17.095317, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: Unable to load modules for /usr/local/samba/private/sam.ldb: >> schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.095940, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: dsdb_get_schema: refresh_fn() failed >> [2016/03/02 11:27:17.097255, 0] >> ../source4/smbd/service_task.c:35(task_server_terminate) >> [2016/03/02 11:27:17.098404, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> task_server_terminate: [kccsrv: Failed to connect to local samdb: >> WERR_DS_UNAVAILABLE >> ] >> [2016/03/02 11:27:17.102052, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.102608, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module schema_load initialization failed : Operations error >> [2016/03/02 11:27:17.103164, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module rootdse initialization failed : Operations error >> [2016/03/02 11:27:17.103210, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module samba_dsdb initialization failed : Operations error >> [2016/03/02 11:27:17.103241, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: Unable to load modules for /usr/local/samba/private/sam.ldb: >> schema_load_init: dsdb_get_schema failed >> ldb: dsdb_get_schema: refresh_fn() failed >> [2016/03/02 11:27:17.103939, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.103974, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module schema_load initialization failed : Operations error >> [2016/03/02 11:27:17.104003, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module rootdse initialization failed : Operations error >> [2016/03/02 11:27:17.107759, 0] >> ../source4/smbd/service_task.c:35(task_server_terminate) >> task_server_terminate: [dnsupdate: Failed to connect to local samdb >> ] >> [2016/03/02 11:27:17.109612, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module samba_dsdb initialization failed : Operations error >> [2016/03/02 11:27:17.109672, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: Unable to load modules for /usr/local/samba/private/sam.ldb: >> schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.114754, 0] >> ../source4/smbd/service_task.c:35(task_server_terminate) >> task_server_terminate: [ntp_signd failed to open samdb] >> [2016/03/02 11:27:17.731614, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: dsdb_get_schema: refresh_fn() failed >> [2016/03/02 11:27:17.731721, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.731750, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module schema_load initialization failed : Operations error >> [2016/03/02 11:27:17.731779, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module rootdse initialization failed : Operations error >> [2016/03/02 11:27:17.731807, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module samba_dsdb initialization failed : Operations error >> [2016/03/02 11:27:17.731835, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: Unable to load modules for /usr/local/samba/private/sam.ldb: >> schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.762162, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: dsdb_get_schema: refresh_fn() failed >> [2016/03/02 11:27:17.762247, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.762276, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module schema_load initialization failed : Operations error >> [2016/03/02 11:27:17.762305, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module rootdse initialization failed : Operations error >> [2016/03/02 11:27:17.762332, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: module samba_dsdb initialization failed : Operations error >> [2016/03/02 11:27:17.762361, 0] >> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug) >> ldb: Unable to load modules for /usr/local/samba/private/sam.ldb: >> schema_load_init: dsdb_get_schema failed >> [2016/03/02 11:27:17.763923, 0] >> ../source4/smbd/service_task.c:35(task_server_terminate) >> task_server_terminate: [Failed to startup ldap server task] >> [2016/03/02 11:27:18.247740, 0] >> ../file_server/file_server.c:46(file_server_smbd_done) >> file_server smbd daemon died with exit status 1 >> [2016/03/02 11:27:18.247875, 0] >> ../source4/smbd/service_task.c:35(task_server_terminate) >> task_server_terminate: [smbd child process exited] >> [2016/03/02 11:27:23.987725, 0] >> ../source4/winbind/winbindd.c:49(winbindd_done) >> winbindd daemon exited normally >> [2016/03/02 11:27:23.989217, 0] >> ../source4/smbd/service_task.c:35(task_server_terminate) >> task_server_terminate: [winbindd child process exited] >> >> Relevant info: >> >> itwerks at filer:~$ samba -V >> Version 4.3.3 >> itwerks at filer:~$ lsb_release -a >> No LSB modules are available. >> Distributor ID: Ubuntu >> Description: Ubuntu 15.10 >> Release: 15.10 >> Codename: wily >> itwerks at filer:~$ uname -a >> Linux filer 4.2.0-30-generic #36-Ubuntu SMP Fri Feb 26 00:58:07 UTC 2016 >> x86_64 x86_64 x86_64 GNU/Linux >> itwerks at filer:~$ >> >> Please advise. It is end of year for this company, I simply cannot afford >> failures like this every three to four weeks, and I cannot blow the entire >> domain out again and rebuild from scratch. >> >> Frustrated. >> >> JS >>Have you considered that it may be a hardware problem ? or is your raid setup ok . You could also try turning Apparmor off as a test (Is there anything in syslog about Samba and Apparmor). Your problem is not a normal one, or one that is common, it may help if we could see your smb.conf (you may have posted it before, but if so, please post it again)
> On 02/03/16 20:46, IT Admin wrote: >> Any further information required? Any troubleshooting steps? I need to >> get this domain functional ASAP. >> >> Please advise. >> >> JSI would also install a second DC. It would give you redundancy, and if the second one does NOT fail when the first one does...then at least you know that the problem is specific to that one DC. (i realise that this does not help you to get the domain back up, now)
IT Admin
2016-Mar-02 21:42 UTC
[Samba] Samba AD/DC crashed again, third time in as many months
I built this machine, and while it isn't the most robust box in the world
it has been stable otherwise. The RAID array is configured RAID1, I can't
see how that could cause corruption issues and I haven't experienced any
other data corruption issues apart from SAMBA collapsing. I did start
seeing samba failures about two weeks ago, restarting the machine brought
the domain up in those cases, associated errors revolved around samba dns
functions iirc. I have intended to upgrade to the latest release (which
went public a week after the last rebuild), I had hoped that update would
eradicate the errors I was experiencing.
My smb.conf:
cat /etc/samba/smb.conf
# Global parameters
[global]
workgroup = CB
realm = CB.CLIFFBELLS.COM
netbios name = FILER
server role = active directory domain controller
dns forwarder = 192.168.37.254
allow dns updates = nonsecure and secure
idmap_ldb:use rfc2307 = yes
printing = CUPS
printcap name = /dev/null
# below added from dragon.org guide
idmap config *:backend = tdb
# This local range must not overlap the BLACK:range above
idmap config *:range = 5000-9999
idmap config CB:backend = ad
idmap config CB:schema_mode = rfc2307
idmap config CB:range = 10000-29999
# Use home directory and shell information from AD
winbind nss info = rfc2307
[netlogon]
path = /usr/local/samba/var/locks/sysvol/cb.cliffbells.com/scripts
read only = No
[sysvol]
path = /usr/local/samba/var/locks/sysvol
read only = No
[Users]
directory_mode: parameter = 0700
read only = no
path = /storage/Users
csc policy = documents
The above smb.conf has been in use since late January, and the following I
cannot explain... There should be declarations for "data" and
"backups" in
that file but they have magically vanished. I worked at this location last
weekend and know for a fact that those shares were live and accessible.
It appears an update mangled the smb.conf file on the 17th of february:
ls -la /etc/samba/
total 68
drwxr-xr-x 3 root root 4096 Mar 2 16:34 .
drwxr-xr-x 135 root root 12288 Mar 2 06:51 ..
-rw-r--r-- 1 root root 8 Nov 12 13:07 gdbcommands
-rw-r--r-- 1 root root 972 Feb 17 06:34 smb.conf
-rw-r--r-- 1 root root 998 Dec 13 19:44 smb.conf.BAD
-rw-r--r-- 1 root root 345 Jan 10 04:36 smb.conf.BADDY
-rw-r--r-- 1 root root 1529 Jan 6 06:54 smb.conf.ONE.CLIFFBELLS.COM
-rw-r--r-- 1 root root 9542 Dec 7 18:43 smb.conf.ORIG
-rw-r--r-- 1 root root 345 Jan 8 21:24 smb.conf.SRC
-rw-r--r-- 1 root root 9535 Feb 17 06:34 smb.conf.ucf-dist
drwxr-xr-x 2 root root 4096 Nov 12 13:06 tls
I'm going to restore it to an earlier version and reboot.
JS
On Mar 2, 2016 4:01 PM, "Rowland penny" <rpenny at samba.org>
wrote:
> On 02/03/16 20:46, IT Admin wrote:
>
>> Any further information required? Any troubleshooting steps? I need
to
>> get this domain functional ASAP.
>>
>> Please advise.
>>
>> JS
>> On Mar 2, 2016 11:39 AM, "IT Admin" <it at
cliffbells.com> wrote:
>>
>> Hello to the samba users list again. I've got an emergency here, a
Samba
>>> AD DC I've got deployed has crashed again, this is the third
time since
>>> 12/17/2015 that this domain has failed completely.
>>>
>>> No power outages or unexpected shutdowns, samba simply fails to
start, it
>>> appears my database is yet again corrupt:
>>>
>>> [2016/03/02 11:27:17.025235, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.025346, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.025377, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.025404, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.025431, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.025460, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.032681, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.033626, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> [2016/03/02 11:27:17.033776, 0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>> task_server_terminate: [nbtd failed to open samdb]
>>> ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.034676, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.035224, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.035838, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.036392, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.038675, 0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>> [2016/03/02 11:27:17.040353, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> [2016/03/02 11:27:17.042764, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.042857, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.042887, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.042917, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.042942, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.042971, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.045981, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.046598, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.047302, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> [2016/03/02 11:27:17.047523, 0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>> task_server_terminate: [dreplsrv: Failed to connect to local
samdb:
>>> WERR_DS_UNAVAILABLE
>>> ]
>>> ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.048503, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.048540, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.050402, 0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>> task_server_terminate: [kdc: krb5_init_context samdb connect
failed]
>>> task_server_terminate: [cldapd failed to open samdb]
>>> [2016/03/02 11:27:17.072670, 0]
>>> ../source4/smbd/server.c:211(samba_terminate)
>>> samba_terminate: dreplsrv: Failed to connect to local samdb:
>>> WERR_DS_UNAVAILABLE
>>>
>>> [2016/03/02 11:27:17.073896, 0]
>>> ../source4/smbd/server.c:211(samba_terminate)
>>> samba_terminate: nbtd failed to open samdb
>>> [2016/03/02 11:27:17.075031, 0]
>>> ../source4/smbd/server.c:211(samba_terminate)
>>> samba_terminate: cldapd failed to open samdb
>>> [2016/03/02 11:27:17.075614, 0]
>>> ../source4/smbd/server.c:211(samba_terminate)
>>> samba_terminate: kdc: krb5_init_context samdb connect failed
>>> [2016/03/02 11:27:17.086051, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.091786, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.091860, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.091890, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.091919, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.091946, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.091974, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.093919, 0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>> task_server_terminate: [dns: samdb_connect failed]
>>> [2016/03/02 11:27:17.095175, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.095232, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.095261, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.095288, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.095317, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.095940, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.097255, 0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>> [2016/03/02 11:27:17.098404, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> task_server_terminate: [kccsrv: Failed to connect to local
samdb:
>>> WERR_DS_UNAVAILABLE
>>> ]
>>> [2016/03/02 11:27:17.102052, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.102608, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.103164, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.103210, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.103241, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.103939, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.103974, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.104003, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.107759, 0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>> task_server_terminate: [dnsupdate: Failed to connect to local
samdb
>>> ]
>>> [2016/03/02 11:27:17.109612, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.109672, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.114754, 0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>> task_server_terminate: [ntp_signd failed to open samdb]
>>> [2016/03/02 11:27:17.731614, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.731721, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.731750, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.731779, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.731807, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.731835, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.762162, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.762247, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.762276, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.762305, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.762332, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.762361, 0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.763923, 0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>> task_server_terminate: [Failed to startup ldap server task]
>>> [2016/03/02 11:27:18.247740, 0]
>>> ../file_server/file_server.c:46(file_server_smbd_done)
>>> file_server smbd daemon died with exit status 1
>>> [2016/03/02 11:27:18.247875, 0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>> task_server_terminate: [smbd child process exited]
>>> [2016/03/02 11:27:23.987725, 0]
>>> ../source4/winbind/winbindd.c:49(winbindd_done)
>>> winbindd daemon exited normally
>>> [2016/03/02 11:27:23.989217, 0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>> task_server_terminate: [winbindd child process exited]
>>>
>>> Relevant info:
>>>
>>> itwerks at filer:~$ samba -V
>>> Version 4.3.3
>>> itwerks at filer:~$ lsb_release -a
>>> No LSB modules are available.
>>> Distributor ID: Ubuntu
>>> Description: Ubuntu 15.10
>>> Release: 15.10
>>> Codename: wily
>>> itwerks at filer:~$ uname -a
>>> Linux filer 4.2.0-30-generic #36-Ubuntu SMP Fri Feb 26 00:58:07 UTC
2016
>>> x86_64 x86_64 x86_64 GNU/Linux
>>> itwerks at filer:~$
>>>
>>> Please advise. It is end of year for this company, I simply cannot
>>> afford
>>> failures like this every three to four weeks, and I cannot blow the
>>> entire
>>> domain out again and rebuild from scratch.
>>>
>>> Frustrated.
>>>
>>> JS
>>>
>>>
> Have you considered that it may be a hardware problem ? or is your raid
> setup ok . You could also try turning Apparmor off as a test (Is there
> anything in syslog about Samba and Apparmor).
>
> Your problem is not a normal one, or one that is common, it may help if we
> could see your smb.conf (you may have posted it before, but if so, please
> post it again)
>
> --
> To unsubscribe from this list go to the following URL and read the
> instructions: https://lists.samba.org/mailman/options/samba
>
IT Admin
2016-Mar-02 22:07 UTC
[Samba] Samba AD/DC crashed again, third time in as many months
I forgot that when compiled from source samba looks at /usr/local/samba/etc
for smb.conf, not /etc/samba, ignore my confusion re: mangled conf file,
the actual smb.conf is correct and follows:
cat /usr/local/samba/etc/smb.conf
# Global parameters
[global]
workgroup = CB
realm = CB.CLIFFBELLS.COM
netbios name = FILER
server role = active directory domain controller
dns forwarder = 192.168.37.254
idmap_ldb:use rfc2307 = yes
printing = CUPS
printcap name = /dev/null
# below added from dragon.org guide
idmap config *:backend = tdb
# This local range must not overlap the BLACK:range above
idmap config *:range = 5000-9999
idmap config CB:backend = ad
idmap config CB:schema_mode = rfc2307
idmap config CB:range = 10000-29999
# Use home directory and shell information from AD
winbind nss info = rfc2307
[netlogon]
path = /usr/local/samba/var/locks/sysvol/cb.cliffbells.com/scripts
read only = No
[sysvol]
path = /usr/local/samba/var/locks/sysvol
read only = No
[Users]
directory_mode: parameter = 0700
read only = no
path = /storage/Users
csc policy = documents
[data]
writeable = yes
directory mode = 777
delete readonly = yes
comment = Cliff Bell's Shared Data Directory
path = /storage/data
create mode = 777
[backups]
comment = Cliff Bell's Backup Directory
path = /storage/backups
directory mode = 777
delete readonly = yes
writeable = yes
create mode = 777
JS
On Mar 2, 2016 4:42 PM, "IT Admin" <it at cliffbells.com> wrote:
> I built this machine, and while it isn't the most robust box in the
world
> it has been stable otherwise. The RAID array is configured RAID1, I
can't
> see how that could cause corruption issues and I haven't experienced
any
> other data corruption issues apart from SAMBA collapsing. I did start
> seeing samba failures about two weeks ago, restarting the machine brought
> the domain up in those cases, associated errors revolved around samba dns
> functions iirc. I have intended to upgrade to the latest release (which
> went public a week after the last rebuild), I had hoped that update would
> eradicate the errors I was experiencing.
>
> My smb.conf:
>
> cat /etc/samba/smb.conf
> # Global parameters
> [global]
> workgroup = CB
> realm = CB.CLIFFBELLS.COM
> netbios name = FILER
> server role = active directory domain controller
> dns forwarder = 192.168.37.254
> allow dns updates = nonsecure and secure
> idmap_ldb:use rfc2307 = yes
> printing = CUPS
> printcap name = /dev/null
> # below added from dragon.org guide
> idmap config *:backend = tdb
> # This local range must not overlap the BLACK:range above
> idmap config *:range = 5000-9999
> idmap config CB:backend = ad
> idmap config CB:schema_mode = rfc2307
> idmap config CB:range = 10000-29999
> # Use home directory and shell information from AD
> winbind nss info = rfc2307
>
> [netlogon]
> path = /usr/local/samba/var/locks/sysvol/cb.cliffbells.com/scripts
> read only = No
>
> [sysvol]
> path = /usr/local/samba/var/locks/sysvol
> read only = No
> [Users]
> directory_mode: parameter = 0700
> read only = no
> path = /storage/Users
> csc policy = documents
>
> The above smb.conf has been in use since late January, and the following I
> cannot explain... There should be declarations for "data" and
"backups" in
> that file but they have magically vanished. I worked at this location last
> weekend and know for a fact that those shares were live and accessible.
>
> It appears an update mangled the smb.conf file on the 17th of february:
>
> ls -la /etc/samba/
> total 68
> drwxr-xr-x 3 root root 4096 Mar 2 16:34 .
> drwxr-xr-x 135 root root 12288 Mar 2 06:51 ..
> -rw-r--r-- 1 root root 8 Nov 12 13:07 gdbcommands
> -rw-r--r-- 1 root root 972 Feb 17 06:34 smb.conf
> -rw-r--r-- 1 root root 998 Dec 13 19:44 smb.conf.BAD
> -rw-r--r-- 1 root root 345 Jan 10 04:36 smb.conf.BADDY
> -rw-r--r-- 1 root root 1529 Jan 6 06:54 smb.conf.ONE.CLIFFBELLS.COM
> -rw-r--r-- 1 root root 9542 Dec 7 18:43 smb.conf.ORIG
> -rw-r--r-- 1 root root 345 Jan 8 21:24 smb.conf.SRC
> -rw-r--r-- 1 root root 9535 Feb 17 06:34 smb.conf.ucf-dist
> drwxr-xr-x 2 root root 4096 Nov 12 13:06 tls
>
> I'm going to restore it to an earlier version and reboot.
>
> JS
> On Mar 2, 2016 4:01 PM, "Rowland penny" <rpenny at
samba.org> wrote:
>
>> On 02/03/16 20:46, IT Admin wrote:
>>
>>> Any further information required? Any troubleshooting steps? I
need to
>>> get this domain functional ASAP.
>>>
>>> Please advise.
>>>
>>> JS
>>> On Mar 2, 2016 11:39 AM, "IT Admin" <it at
cliffbells.com> wrote:
>>>
>>> Hello to the samba users list again. I've got an emergency
here, a Samba
>>>> AD DC I've got deployed has crashed again, this is the
third time since
>>>> 12/17/2015 that this domain has failed completely.
>>>>
>>>> No power outages or unexpected shutdowns, samba simply fails to
start,
>>>> it
>>>> appears my database is yet again corrupt:
>>>>
>>>> [2016/03/02 11:27:17.025235, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: dsdb_get_schema: refresh_fn() failed
>>>> [2016/03/02 11:27:17.025346, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.025377, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module schema_load initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.025404, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module rootdse initialization failed : Operations error
>>>> [2016/03/02 11:27:17.025431, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module samba_dsdb initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.025460, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>>> schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.032681, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: dsdb_get_schema: refresh_fn() failed
>>>> [2016/03/02 11:27:17.033626, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> [2016/03/02 11:27:17.033776, 0]
>>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>> task_server_terminate: [nbtd failed to open samdb]
>>>> ldb: schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.034676, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module schema_load initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.035224, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module rootdse initialization failed : Operations error
>>>> [2016/03/02 11:27:17.035838, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module samba_dsdb initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.036392, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>>> schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.038675, 0]
>>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>> [2016/03/02 11:27:17.040353, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> [2016/03/02 11:27:17.042764, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: dsdb_get_schema: refresh_fn() failed
>>>> [2016/03/02 11:27:17.042857, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.042887, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module schema_load initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.042917, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module rootdse initialization failed : Operations error
>>>> [2016/03/02 11:27:17.042942, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module samba_dsdb initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.042971, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>>> schema_load_init: dsdb_get_schema failed
>>>> ldb: dsdb_get_schema: refresh_fn() failed
>>>> [2016/03/02 11:27:17.045981, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.046598, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module schema_load initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.047302, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> [2016/03/02 11:27:17.047523, 0]
>>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>> task_server_terminate: [dreplsrv: Failed to connect to local
samdb:
>>>> WERR_DS_UNAVAILABLE
>>>> ]
>>>> ldb: module rootdse initialization failed : Operations error
>>>> [2016/03/02 11:27:17.048503, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module samba_dsdb initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.048540, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>>> schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.050402, 0]
>>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>> task_server_terminate: [kdc: krb5_init_context samdb connect
failed]
>>>> task_server_terminate: [cldapd failed to open samdb]
>>>> [2016/03/02 11:27:17.072670, 0]
>>>> ../source4/smbd/server.c:211(samba_terminate)
>>>> samba_terminate: dreplsrv: Failed to connect to local samdb:
>>>> WERR_DS_UNAVAILABLE
>>>>
>>>> [2016/03/02 11:27:17.073896, 0]
>>>> ../source4/smbd/server.c:211(samba_terminate)
>>>> samba_terminate: nbtd failed to open samdb
>>>> [2016/03/02 11:27:17.075031, 0]
>>>> ../source4/smbd/server.c:211(samba_terminate)
>>>> samba_terminate: cldapd failed to open samdb
>>>> [2016/03/02 11:27:17.075614, 0]
>>>> ../source4/smbd/server.c:211(samba_terminate)
>>>> samba_terminate: kdc: krb5_init_context samdb connect failed
>>>> [2016/03/02 11:27:17.086051, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: dsdb_get_schema: refresh_fn() failed
>>>> [2016/03/02 11:27:17.091786, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: dsdb_get_schema: refresh_fn() failed
>>>> [2016/03/02 11:27:17.091860, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.091890, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module schema_load initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.091919, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module rootdse initialization failed : Operations error
>>>> [2016/03/02 11:27:17.091946, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module samba_dsdb initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.091974, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>>> schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.093919, 0]
>>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>> task_server_terminate: [dns: samdb_connect failed]
>>>> [2016/03/02 11:27:17.095175, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.095232, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module schema_load initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.095261, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module rootdse initialization failed : Operations error
>>>> [2016/03/02 11:27:17.095288, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module samba_dsdb initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.095317, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>>> schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.095940, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: dsdb_get_schema: refresh_fn() failed
>>>> [2016/03/02 11:27:17.097255, 0]
>>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>> [2016/03/02 11:27:17.098404, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> task_server_terminate: [kccsrv: Failed to connect to local
samdb:
>>>> WERR_DS_UNAVAILABLE
>>>> ]
>>>> [2016/03/02 11:27:17.102052, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.102608, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module schema_load initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.103164, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module rootdse initialization failed : Operations error
>>>> [2016/03/02 11:27:17.103210, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module samba_dsdb initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.103241, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>>> schema_load_init: dsdb_get_schema failed
>>>> ldb: dsdb_get_schema: refresh_fn() failed
>>>> [2016/03/02 11:27:17.103939, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.103974, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module schema_load initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.104003, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module rootdse initialization failed : Operations error
>>>> [2016/03/02 11:27:17.107759, 0]
>>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>> task_server_terminate: [dnsupdate: Failed to connect to
local samdb
>>>> ]
>>>> [2016/03/02 11:27:17.109612, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module samba_dsdb initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.109672, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>>> schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.114754, 0]
>>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>> task_server_terminate: [ntp_signd failed to open samdb]
>>>> [2016/03/02 11:27:17.731614, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: dsdb_get_schema: refresh_fn() failed
>>>> [2016/03/02 11:27:17.731721, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.731750, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module schema_load initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.731779, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module rootdse initialization failed : Operations error
>>>> [2016/03/02 11:27:17.731807, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module samba_dsdb initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.731835, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>>> schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.762162, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: dsdb_get_schema: refresh_fn() failed
>>>> [2016/03/02 11:27:17.762247, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.762276, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module schema_load initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.762305, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module rootdse initialization failed : Operations error
>>>> [2016/03/02 11:27:17.762332, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: module samba_dsdb initialization failed : Operations
error
>>>> [2016/03/02 11:27:17.762361, 0]
>>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>> ldb: Unable to load modules for
/usr/local/samba/private/sam.ldb:
>>>> schema_load_init: dsdb_get_schema failed
>>>> [2016/03/02 11:27:17.763923, 0]
>>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>> task_server_terminate: [Failed to startup ldap server task]
>>>> [2016/03/02 11:27:18.247740, 0]
>>>> ../file_server/file_server.c:46(file_server_smbd_done)
>>>> file_server smbd daemon died with exit status 1
>>>> [2016/03/02 11:27:18.247875, 0]
>>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>> task_server_terminate: [smbd child process exited]
>>>> [2016/03/02 11:27:23.987725, 0]
>>>> ../source4/winbind/winbindd.c:49(winbindd_done)
>>>> winbindd daemon exited normally
>>>> [2016/03/02 11:27:23.989217, 0]
>>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>> task_server_terminate: [winbindd child process exited]
>>>>
>>>> Relevant info:
>>>>
>>>> itwerks at filer:~$ samba -V
>>>> Version 4.3.3
>>>> itwerks at filer:~$ lsb_release -a
>>>> No LSB modules are available.
>>>> Distributor ID: Ubuntu
>>>> Description: Ubuntu 15.10
>>>> Release: 15.10
>>>> Codename: wily
>>>> itwerks at filer:~$ uname -a
>>>> Linux filer 4.2.0-30-generic #36-Ubuntu SMP Fri Feb 26 00:58:07
UTC 2016
>>>> x86_64 x86_64 x86_64 GNU/Linux
>>>> itwerks at filer:~$
>>>>
>>>> Please advise. It is end of year for this company, I simply
cannot
>>>> afford
>>>> failures like this every three to four weeks, and I cannot blow
the
>>>> entire
>>>> domain out again and rebuild from scratch.
>>>>
>>>> Frustrated.
>>>>
>>>> JS
>>>>
>>>>
>> Have you considered that it may be a hardware problem ? or is your raid
>> setup ok . You could also try turning Apparmor off as a test (Is there
>> anything in syslog about Samba and Apparmor).
>>
>> Your problem is not a normal one, or one that is common, it may help if
>> we could see your smb.conf (you may have posted it before, but if so,
>> please post it again)
>>
>> --
>> To unsubscribe from this list go to the following URL and read the
>> instructions: https://lists.samba.org/mailman/options/samba
>>
>
Rowland penny
2016-Mar-02 22:12 UTC
[Samba] Samba AD/DC crashed again, third time in as many months
On 02/03/16 21:42, IT Admin wrote:> > I built this machine, and while it isn't the most robust box in the > world it has been stable otherwise. The RAID array is configured > RAID1, I can't see how that could cause corruption issues and I > haven't experienced any other data corruption issues apart from SAMBA > collapsing. I did start seeing samba failures about two weeks ago, > restarting the machine brought the domain up in those cases, > associated errors revolved around samba dns functions iirc. I have > intended to upgrade to the latest release (which went public a week > after the last rebuild), I had hoped that update would eradicate the > errors I was experiencing. >I have never trusted raid since the server at the place I was working went down, this was traced to one of the HDs dying. the raid was composed of 4 HDs, two in stripped mode, backed up by the other two, if one set was faulty the other set was supposed to take over, only they didn't, took three days to get the server back up.> My smb.conf: > > cat /etc/samba/smb.conf > # Global parameters > [global] > workgroup = CB > realm = CB.CLIFFBELLS.COM <http://CB.CLIFFBELLS.COM> > netbios name = FILER > server role = active directory domain controller > dns forwarder = 192.168.37.254 > allow dns updates = nonsecure and secure > idmap_ldb:use rfc2307 = yes > printing = CUPS > printcap name = /dev/null >> # below added from dragon.org <http://dragon.org> guide > idmap config *:backend = tdb > # This local range must not overlap the BLACK:range above > idmap config *:range = 5000-9999 > idmap config CB:backend = ad > idmap config CB:schema_mode = rfc2307 > idmap config CB:range = 10000-29999 > # Use home directory and shell information from AD > winbind nss info = rfc2307 >You might as well remove the above 9 lines from smb.conf, they do *nothing* on a DC.> [netlogon] > path = > /usr/local/samba/var/locks/sysvol/cb.cliffbells.com/scripts > <http://cb.cliffbells.com/scripts> > read only = No > > [sysvol] > path = /usr/local/samba/var/locks/sysvol > read only = No > [Users] > directory_mode: parameter = 0700 > read only = no > path = /storage/Users > csc policy = documents > > The above smb.conf has been in use since late January, and the > following I cannot explain... There should be declarations for "data" > and "backups" in that file but they have magically vanished. I worked > at this location last weekend and know for a fact that those shares > were live and accessible. > > It appears an update mangled the smb.conf file on the 17th of february: >AHA, I take you are talking about an Ubuntu update, as you are using a self compiled version of Samba, no Ubuntu update should touch smb.conf, or are you talking about some other update ?> ls -la /etc/samba/ >Why is smb.conf in /etc/samba/ ? It should be in /usr/local/samba/etc/ Rowland> total 68 > drwxr-xr-x 3 root root 4096 Mar 2 16:34 . > drwxr-xr-x 135 root root 12288 Mar 2 06:51 .. > -rw-r--r-- 1 root root 8 Nov 12 13:07 gdbcommands > -rw-r--r-- 1 root root 972 Feb 17 06:34 smb.conf > -rw-r--r-- 1 root root 998 Dec 13 19:44 smb.conf.BAD > -rw-r--r-- 1 root root 345 Jan 10 04:36 smb.conf.BADDY > -rw-r--r-- 1 root root 1529 Jan 6 06:54 > smb.conf.ONE.CLIFFBELLS.COM <http://smb.conf.ONE.CLIFFBELLS.COM> > -rw-r--r-- 1 root root 9542 Dec 7 18:43 smb.conf.ORIG > -rw-r--r-- 1 root root 345 Jan 8 21:24 smb.conf.SRC > -rw-r--r-- 1 root root 9535 Feb 17 06:34 smb.conf.ucf-dist > drwxr-xr-x 2 root root 4096 Nov 12 13:06 tls > > I'm going to restore it to an earlier version and reboot. > > JS > >
Andrew Bartlett
2016-Mar-06 23:56 UTC
[Samba] Samba AD/DC crashed again, third time in as many months
On Wed, 2016-03-02 at 16:42 -0500, IT Admin wrote:> I built this machine, and while it isn't the most robust box in the > world > it has been stable otherwise. The RAID array is configured RAID1, I > can't > see how that could cause corruption issues and I haven't experienced > any > other data corruption issues apart from SAMBA collapsingI know it is hard to swallow, but I really think this is hardware, or the OS configuration under it, combined with unexpected shutdown or some other corruption vector. We have at this stage 10,000 or more domains running Samba4, and this is only the second I've heard of with this kind of symptom. The first I blamed on a use of DRDB that I postulated was not preserving 'write barriers' (that is, the thing that makes fsync() work) and a poweroff, but I didn't really have any proof. You do need to run a second DC, as well as run tools like memcheck on this DC. Make sure you regularly run the backup script, so you can work out when the corruption happens, and verify your DB with dbcheck. The second DC has the advantage that this kind of low-level corruption doesn't easily spread across DRS replication (it would instead fail replication). The error shown indicates that for some reason or other, it can't read the schema. This is very odd, as the schema doesn't change! We would love to get to the bottom of this. Unlike others, I don't think this has anything to do with packaging (that would just make us not start at all), but a clean install on a clean machine is my best advise, keeping the rest aside (and off) for forensics if you have the patience. Finally, always keep the steps simple - otherwise we might start confusing admin errors for hardware errors or vice verca. The things we all do in the panic are always the hardest to de-construct in the cold light of day. Thanks, Andrew Bartlett -- Andrew Bartlett https://samba.org/~abartlet/ Authentication Developer, Samba Team https://samba.org Samba Development and Support, Catalyst IT https://catalyst.net.nz/services/samba
Possibly Parallel Threads
- Samba AD/DC crashed again, third time in as many months
- Samba AD/DC crashed again, third time in as many months
- Samba AD/DC crashed again, third time in as many months
- Samba AD/DC crashed again, third time in as many months
- Samba AD/DC crashed again, third time in as many months