I believe a schema change on a Windows DC (2008rc) has broken replication with our S4 DCs. Anyone have any tips or pointers to resolve this? I have three S4 DCs [CentOS6] and one Windows 2008R2 DC. The Windows 2008R2 DC has the schema master FSMO, and I believe the Exchange schema was added. I am willing to pay US dollars to get this issue resolved. I need the replication restored, the Windows 2008DC, and the Samba4 DCs demoted. [root at larkin27 ~]# samba-tool ldapcmp ldap://temp2008r2dc.micore.us lda p://larkin27.micore.us schema * Comparing [SCHEMA] context... * DN lists have different size: 4267 != 4014 ... -- Adam Tauno Williams <mailto:awilliam at whitemice.org> GPG D95ED383 Systems Administrator, Python Developer, LPI / NCLA
On Wed, 2016-11-16 at 16:04 -0500, Adam Tauno Williams via samba wrote:> I believe a schema change on a Windows DC (2008rc) has broken > replication with our S4 DCs. Anyone have any tips or pointers to > resolve this?Sadly this isn't totally unexpected: the exchange schema has quite a few elements, and while I understand it has worked in the past, it isn't part of our automatic tests and our schema code certainly needs work.> I have three S4 DCs [CentOS6] and one Windows 2008R2 DC. The Windows > 2008R2 DC has the schema master FSMO, and I believe the Exchange > schema > was added.Can you clarify exactly what versions of Samba is involved here? What messages are in the log?> I am willing to pay US dollars to get this issue resolved. I need > the > replication restored, the Windows 2008DC, and the Samba4 DCs demoted.Can you clarify a bit further what is your goal? To keep Samba after fixing this issue, possibly after a demote and fresh replicate, or just get it out of the domain long-term? Commercial support services are listed at: https://www.samba.org/samba/support/globalsupport.html Thanks, Andrew Bartlett -- Andrew Bartlett http://samba.org/~abartlet/ Authentication Developer, Samba Team http://samba.org Samba Developer, Catalyst IT http://catalyst.net.nz/services/samba
Adam Tauno Williams
2016-Nov-20 20:55 UTC
[Samba] 4.5.1 Upgrade Breaks Samba [Was: Schema Change Breaks Replication]
On Sat, 2016-11-19 at 09:57 +1300, Andrew Bartlett wrote:> On Fri, 2016-11-18 at 09:41 -0500, Adam Tauno Williams wrote: > > On Fri, 2016-11-18 at 21:32 +1300, Andrew Bartlett wrote: > > > I believe a schema change on a Windows DC (2008rc) has > > > > > > broken > > > > sernet-samba-4.2.14-23.el6.x86_64 - the same package on all > > > > three > > > > LINUX DC. All DCs are virtualized CentOS6. > > > This is likely the major issue. Running a current Samba version > > > would be a very good idea, for things like this. > > Yep... I have just purchased a SAMBA+ subscription so I will have > > the > > 4.5 packages. Should I begin by updating the software on the DCs? > Yes.An upgrade of one of the S4 DCs to Samba 4.5.1 appears to result in a non-operational server. Winbind is not working with a log message of - [root at larkin26 samba]# tail log.winbindd [2016/11/20 15:38:58.229223, 0] ../lib/util/become_daemon.c:124(daemon_ready) STATUS=daemon 'winbindd' finished starting up and ready to serve connections [2016/11/20 15:38:58.252934, 1] ../source3/winbindd/winbindd_util.c:352(trustdom_list_done) trustdom_list_done: Could not receive trusts for domain BACKBONE [2016/11/20 15:40:33.495158, 0] ../source3/winbindd/winbindd_cache.c:3244(initialize_winbindd_cache) initialize_winbindd_cache: clearing cache and re-creating with version number 2 [2016/11/20 15:40:36.106151, 0] ../lib/util/become_daemon.c:124(daemon_ready) STATUS=daemon 'winbindd' finished starting up and ready to serve connections [2016/11/20 15:40:36.129472, 1] ../source3/winbindd/winbindd_util.c:352(trustdom_list_done) trustdom_list_done: Could not receive trusts for domain BACKBONE "samba-tool drs showrepl" fails - [root at larkin26 ~]# samba-tool drs showrepl Failed to connect host 172.31.7.50 on port 135 - NT_STATUS_CONNECTION_REFUSED Failed to connect host 172.31.7.50 (larkin26.micore.us) on port 135 - NT_STATUS_CONNECTION_REFUSED. ERROR(<class 'samba.drs_utils.drsException'>): DRS connection to larkin26.micore.us failed - drsException: DRS connection to larkin26.micore.us failed: (-1073741258, 'The connection was refused') File "/usr/lib64/python2.6/site-packages/samba/netcmd/drs.py", line 41, in drsuapi_connect (ctx.drsuapi, ctx.drsuapi_handle, ctx.bind_supported_extensions) drs_utils.drsuapi_connect(ctx.server, ctx.lp, ctx.creds) File "/usr/lib64/python2.6/site-packages/samba/drs_utils.py", line 54, in drsuapi_connect raise drsException("DRS connection to %s failed: %s" % (server, e)) And it looks like nobody is listening on port 135 - [root at larkin26 ~]# netstat --listen --inet --program --numeric Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.0.1:199 0.0.0.0:* LISTEN 1577/snmpd tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1607/sshd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1697/master udp 0 0 0.0.0.0:34659 0.0.0.0:* 1271/rsyslogd udp 0 0 0.0.0.0:631 0.0.0.0:* 1261/portreserve udp 0 0 172.31.7.50:123 0.0.0.0:* 1618/ntpd udp 0 0 127.0.0.1:123 0.0.0.0:* 1618/ntpd udp 0 0 0.0.0.0:123 0.0.0.0:* 1618/ntpd udp 0 0 0.0.0.0:161 0.0.0.0:* 1577/snmpd -- Adam Tauno Williams <mailto:awilliam at whitemice.org> GPG D95ED383 Systems Administrator, Python Developer, LPI / NCLA
Rowland Penny
2016-Nov-20 21:11 UTC
[Samba] 4.5.1 Upgrade Breaks Samba [Was: Schema Change Breaks Replication]
On Sun, 20 Nov 2016 15:55:08 -0500 Adam Tauno Williams via samba <samba at lists.samba.org> wrote:> > And it looks like nobody is listening on port 135 - > > [root at larkin26 ~]# netstat --listen --inet --program --numeric > Active Internet connections (only servers) > Proto Recv-Q Send-Q Local Address Foreign > Address State PID/Program name > tcp 0 0 127.0.0.1:199 > 0.0.0.0:* LISTEN 1577/snmpd > tcp 0 0 0.0.0.0:22 > 0.0.0.0:* LISTEN 1607/sshd > tcp 0 0 127.0.0.1:25 > 0.0.0.0:* LISTEN 1697/master > udp 0 0 0.0.0.0:34659 > 0.0.0.0:* 1271/rsyslogd > udp 0 0 0.0.0.0:631 > 0.0.0.0:* 1261/portreserve > udp 0 0 172.31.7.50:123 > 0.0.0.0:* 1618/ntpd > udp 0 0 127.0.0.1:123 > 0.0.0.0:* 1618/ntpd > udp 0 0 0.0.0.0:123 > 0.0.0.0:* 1618/ntpd > udp 0 0 0.0.0.0:161 > 0.0.0.0:* 1577/snmpd >If that is the entire output, not only is nothing listening on port 135, Samba doesn't seem to be listening at all, is it running ? Rowland
After upgrade to Samba 4.5.1 the replication is still accumulating errors related to schema. ... [2016/11/21 09:30:45.571862, 0] ../source4/rpc_server/drsuapi/getncchanges.c:1905(dcesrv_drsuapi_DsGetN CChanges) ../source4/rpc_server/drsuapi/getncchanges.c:1905: DsGetNCChanges 2nd replication on DN DC=micore,DC=us older highwatermark (last_dn CN=Personel,CN=Users,DC=micore,DC=us) [2016/11/21 09:30:45.573362, 2] ../source4/rpc_server/drsuapi/getncchanges.c:1490(getncchanges_collect_ objects) ../source4/rpc_server/drsuapi/getncchanges.c:1490: getncchanges on DC=micore,DC=us using filter (uSNChanged>=1) [2016/11/21 09:30:45.731436, 3] ../source4/dsdb/samdb/ldb_modules/repl_meta_data.c:5103(replmd_replicat ed_apply_merge) Discarding older DRS attribute update to dnsRecord on DC=PC03574,DC=micore.us,CN=MicrosoftDNS,DC=DomainDnsZones,DC=micore,DCus from eae9ebb0-429b-44de-8582-c58abdf611ce [2016/11/21 09:30:45.731489, 3] ../source4/dsdb/samdb/ldb_modules/repl_meta_data.c:5103(replmd_replicat ed_apply_merge) Discarding older DRS attribute update to dNSTombstoned on DC=PC03574,DC=micore.us,CN=MicrosoftDNS,DC=DomainDnsZones,DC=micore,DCus from eae9ebb0-429b-44de-8582-c58abdf611ce [2016/11/21 09:30:45.900456, 2] ../source4/dsdb/repl/replicated_objects.c:1016(dsdb_replicated_objects_ commit) Replicated 1 objects (0 linked attributes) for DC=DomainDnsZones,DC=micore,DC=us [2016/11/21 09:30:46.201141, 0] ../source4/dsdb/schema/schema_syntax.c:2752(dsdb_attribute_drsuapi_remo te_to_local) ../source4/dsdb/schema/schema_syntax.c:2752: Unknown local attributeID_id 0x80820E75 remote 0x80820E75 [2016/11/21 09:30:46.204447, 0] ../source4/dsdb/repl/replicated_objects.c:734(dsdb_replicated_objects_c onvert) Failed to convert object CN=Recipient Management,OU=Microsoft Exchange Security Groups,DC=micore,DC=us: WERR_DS_ATT_NOT_DEF_IN_SCHEMA [2016/11/21 09:30:46.204626, 0] ../source4/dsdb/repl/drepl_out_helpers.c:908(dreplsrv_op_pull_source_ap ply_changes_trigger) Failed to convert objects: WERR_DS_ATT_NOT_DEF_IN_SCHEMA/NT_STATUS_INVALID_NETWORK_RESPONSE [2016/11/21 09:30:46.506030, 2] ../source4/dsdb/repl/replicated_objects.c:1016(dsdb_replicated_objects_ commit) Replicated 0 objects (0 linked attributes) for DC=DomainDnsZones,DC=micore,DC=us [2016/11/21 09:30:46.606197, 2] ../source4/rpc_server/drsuapi/getncchanges.c:516(get_nc_changes_add_la) Search of guid 30181ffd-4d35-4f45-9352-37b2019a617a returned 0 objects, skipping it ! [root at larkin26 ~]# samba-tool drs showrepl larkin26 ldb_wrap open of secrets.ldb GENSEC backend 'gssapi_spnego' registered GENSEC backend 'gssapi_krb5' registered GENSEC backend 'gssapi_krb5_sasl' registered GENSEC backend 'spnego' registered GENSEC backend 'schannel' registered GENSEC backend 'naclrpc_as_system' registered GENSEC backend 'sasl-EXTERNAL' registered GENSEC backend 'ntlmssp' registered GENSEC backend 'ntlmssp_resume_ccache' registered GENSEC backend 'http_basic' registered GENSEC backend 'http_ntlm' registered GENSEC backend 'krb5' registered GENSEC backend 'fake_gssapi_krb5' registered Using binding ncacn_ip_tcp:larkin26[,seal] resolve_lmhosts: Attempting lmhosts lookup for name larkin26<0x20> resolve_lmhosts: Attempting lmhosts lookup for name larkin26<0x20> resolve_lmhosts: Attempting lmhosts lookup for name larkin26<0x20> Default-First-Site-Name\LARKIN26 DSA Options: 0x00000001 DSA object GUID: 3a74ac28-1613-471f-ac3d-1b8932eeb167 DSA invocationId: 7e0030ce-36a1-439c-91b6-f1de73ff7a81 ==== INBOUND NEIGHBORS === ... CN=Schema,CN=Configuration,DC=micore,DC=us Default-First-Site-Name\TEMP2008R2DC via RPC DSA object GUID: c8d5c583-a097-4265-858a-cb67797ebb05 Last attempt @ Mon Nov 21 09:30:40 2016 EST failed, result 58 (WERR_BAD_NET_RESP) 4625 consecutive failure(s). ... DC=micore,DC=us Default-First-Site-Name\TEMP2008R2DC via RPC DSA object GUID: c8d5c583-a097-4265-858a-cb67797ebb05 Last attempt @ Mon Nov 21 09:33:26 2016 EST failed, result 58 (WERR_BAD_NET_RESP) 10458 consecutive failure(s). Last success @ Fri Nov 4 15:03:27 2016 EDT ... CN=Configuration,DC=micore,DC=us Default-First-Site-Name\TEMP2008R2DC via RPC DSA object GUID: c8d5c583-a097-4265-858a-cb67797ebb05 Last attempt @ Mon Nov 21 09:30:44 2016 EST failed, result 58 (WERR_BAD_NET_RESP) 4621 consecutive failure(s). Last success @ Fri Nov 4 14:48:55 2016 EDT -- Adam Tauno Williams <mailto:awilliam at whitemice.org> GPG D95ED383 Systems Administrator, Python Developer, LPI / NCLA
Apparently Analagous Threads
- Schema Change Breaks Replication
- Removed old DC, now replication hopelessly broken [HELP]
- Removed old DC, now replication hopelessly broken [SOLVED]
- 4.5.1 Upgrade Breaks Samba [Was: Schema Change Breaks Replication]
- Error "Failed extended allocation RID pool operation..."