Andrew Bartlett
2016-Apr-14 19:20 UTC
[Samba] Previously extended schema not working in 4.4.0
On Thu, 2016-04-14 at 18:07 +0100, Jonathan Hunter wrote:> On 14 April 2016 at 13:37, Jonathan Hunter <jmhunter1 at gmail.com> > wrote: > > > # samba-tool dbcheck --cross-ncs > > Checking 4079 objects > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x00290001 > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x0029000a > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x00290004 > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x0009030e > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x00090001 > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x00020119 > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x00020002 > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x00020001 > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x00000000 > > > ERROR: incorrect attributeID values in replPropertyMetaData on > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk > > > > Not fixing incorrect value 0x00290004 with 0xbd27f4d3 for myAttr in > > replPropertyMetaData on > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk > > > > Going back over the results of 'samba-tool dbcheck', it struck me > just now > that the errors flagged up only appear on objects previously created > using > my extended schema - these are exactly the same type of errors I am > now > getting when trying to create more of these objects. > > So I think that 'samba-tool dbcheck' is displaying the symptom, and > in fact > running 'samba-tool dbcheck' probably won't help my situation. > > What could cause the errors shown via 'samba-tool dbcheck'?Our DRS replication code with extended schema has been pretty badly broken in a number of releases, and so we fixed the bugs and added dbcheck rules to fix the damage. We also added code in Samba to refuse to operate when we detect damage at runtime. Once you run with --fix it should all get back to normal - thankfully we have enough information, just a little scrambled, to fix this up. (Those rules are actually some of the best-tested in dbcheck). We continue to improve our extended schema code. Hopefully we will have it all solid for 4.5, but it is much, much better in 4.4 than 4.2 was. Andrew Bartlett -- Andrew Bartlett http://samba.org/~abartlet/ Authentication Developer, Samba Team http://samba.org Samba Developer, Catalyst IT http://catalyst.net.nz/services/samba
Jonathan Hunter
2016-Apr-14 23:32 UTC
[Samba] Previously extended schema not working in 4.4.0
Thank you Andrew, really appreciated. I have now run 'samba-tool dbcheck --cross-ncs --fix' and it has successfully fixed some errors; there were 110 previously, however there are still 69 remaining after a second pass of dbcheck --fix. The remaining errors seem to be mainly of this form: ERROR: duplicate attributeID values for myattrib in replPropertyMetaData on MYOBJ=object1,OU=myou,DC=mydomain,DC=org,DC=uk Fix replPropertyMetaData on MYOBJ=object1,OU=myou,DC=mydomain,DC=org,DC=uk by removing the duplicate value 0x00290003 for myattrib (keeping 0xbd27f44d5)? [YES] [...] ERROR: incorrect attributeID values in replPropertyMetaData on MYOBJ=object1,OU=myou,DC=mydomain,DC=org,DC=uk Fix replPropertyMetaData on MYOBJ=object1,OU=myou,DC=mydomain,DC=org,DC=uk by replacing incorrect value 0x00290001 for et (new 0x00290001)? [YES] No rDN found in replPropertyMetaData for MYOBJ=object1,OU=myou,DC=mydomain,DC=org,DC=uk! Failed to fix attribute replPropertyMetaData : (19, 'replmd_update_rpmd: No rDN found in replPropertyMetaData for MYOBJ=object1,OU=myou,DC=mydomain,DC=org,DC=uk [YES] I've had a brief look at one of the objects in question (myobj=object1,ou=myou) using ldbsearch, and it looks OK to my untrained eye, there is a dn: of MYOBJ=object1,OU=myou,DC=mydomain,DC=org,DC=uk and also a distringuishedName: of the same; there is a "myobj: object1" attribute, and the usual objectClass/GUID/etc.. Do you know precisely what it is looking for in terms of rDN in replPropertyMetaData? I can have a look there and see if I can find it. Or - given that I have taken a backup via 'ldbsearch -s sub -b ou=myou,dc=...' - am I better off removing this entire OU (which is the only place I have created these objects), and restoring it? Can I play back an LDIF generated via ldbsearch safely - will I get the same GUIDs, creation dates, etc.? That does feel a little like 'giving up'; and I am very happy to investigate further if it will help find any gaps or corner cases that could be used to improve the codebase - but equally, if this isn't particularly interesting and it can be quickly fixed by a delete / restore, then I'm happy to do that also :) Many thanks Jonathan On 14 April 2016 at 20:20, Andrew Bartlett <abartlet at samba.org> wrote:> On Thu, 2016-04-14 at 18:07 +0100, Jonathan Hunter wrote: > > On 14 April 2016 at 13:37, Jonathan Hunter <jmhunter1 at gmail.com> > > wrote: > > > > > # samba-tool dbcheck --cross-ncs > > > Checking 4079 objects > > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x00290001 > > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x0029000a > > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x00290004 > > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x0009030e > > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x00090001 > > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x00020119 > > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x00020002 > > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x00020001 > > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk: 0x00000000 > > > > > ERROR: incorrect attributeID values in replPropertyMetaData on > > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk > > > > > > Not fixing incorrect value 0x00290004 with 0xbd27f4d3 for myAttr in > > > replPropertyMetaData on > > > MYOBJ=value,OU=myou,DC=mydomain,DC=org,DC=uk > > > > > > > Going back over the results of 'samba-tool dbcheck', it struck me > > just now > > that the errors flagged up only appear on objects previously created > > using > > my extended schema - these are exactly the same type of errors I am > > now > > getting when trying to create more of these objects. > > > > So I think that 'samba-tool dbcheck' is displaying the symptom, and > > in fact > > running 'samba-tool dbcheck' probably won't help my situation. > > > > What could cause the errors shown via 'samba-tool dbcheck'? > > Our DRS replication code with extended schema has been pretty badly > broken in a number of releases, and so we fixed the bugs and added > dbcheck rules to fix the damage. We also added code in Samba to refuse > to operate when we detect damage at runtime. > > Once you run with --fix it should all get back to normal - thankfully > we have enough information, just a little scrambled, to fix this up. > > (Those rules are actually some of the best-tested in dbcheck). > > We continue to improve our extended schema code. Hopefully we will > have it all solid for 4.5, but it is much, much better in 4.4 than 4.2 > was. > > Andrew Bartlett > > -- > Andrew Bartlett http://samba.org/~abartlet/ > Authentication Developer, Samba Team http://samba.org > Samba Developer, Catalyst IT > http://catalyst.net.nz/services/samba > > > >-- "If we knew what it was we were doing, it would not be called research, would it?" - Albert Einstein
Andrew Bartlett
2016-Apr-15 01:31 UTC
[Samba] Previously extended schema not working in 4.4.0
On Fri, 2016-04-15 at 00:32 +0100, Jonathan Hunter wrote:> Thank you Andrew, really appreciated. > > I have now run 'samba-tool dbcheck --cross-ncs --fix' and it has > successfully fixed some errors; there were 110 previously, however > there are still 69 remaining after a second pass of dbcheck --fix. > > The remaining errors seem to be mainly of this form: > > ERROR: duplicate attributeID values for myattrib in > replPropertyMetaData on > MYOBJ=object1,OU=myou,DC=mydomain,DC=org,DC=uk > > Fix replPropertyMetaData on > MYOBJ=object1,OU=myou,DC=mydomain,DC=org,DC=uk by removing the > duplicate value 0x00290003 for myattrib (keeping 0xbd27f44d5)? [YES] > [...] > ERROR: incorrect attributeID values in replPropertyMetaData on > MYOBJ=object1,OU=myou,DC=mydomain,DC=org,DC=uk > > Fix replPropertyMetaData > on MYOBJ=object1,OU=myou,DC=mydomain,DC=org,DC=uk by replacing > incorrect value 0x00290001 for et (new 0x00290001)? [YES] > No rDN found in replPropertyMetaData > for MYOBJ=object1,OU=myou,DC=mydomain,DC=org,DC=uk! > > Failed to fix attribute replPropertyMetaData : (19, > 'replmd_update_rpmd: No rDN found in replPropertyMetaData > for MYOBJ=object1,OU=myou,DC=mydomain,DC=org,DC=uk [YES] > > I've had a brief look at one of the objects in question > (myobj=object1,ou=myou) using ldbsearch, and it looks OK to my > untrained eye, there is a dn: of > MYOBJ=object1,OU=myou,DC=mydomain,DC=org,DC=uk and also a > distringuishedName: of the same; there is a "myobj: object1" > attribute, and the usual objectClass/GUID/etc.. > > Do you know precisely what it is looking for in terms of rDN in > replPropertyMetaData? I can have a look there and see if I can find > it. > > Or - given that I have taken a backup via 'ldbsearch -s sub -b > ou=myou,dc=...' - am I better off removing this entire OU (which is > the only place I have created these objects), and restoring it? Can I > play back an LDIF generated via ldbsearch safely - will I get the > same GUIDs, creation dates, etc.? > > That does feel a little like 'giving up'; and I am very happy to > investigate further if it will help find any gaps or corner cases > that could be used to improve the codebase - but equally, if this > isn't particularly interesting and it can be quickly fixed by a > delete / restore, then I'm happy to do that also :)No, a delete probably won't help, the deleted object stays around as a tombstone, and you can't recreate it with the same guid. Because the custom schema attribute myobj was also the RDN, it hit a case we haven't tested yet. We probably need to fix further our test scripts. Please file a bug, with the relevent replPropertyMetaData in base64 and with the --show-binary argument to ldbsearch if possible. Andrew Bartlett -- Andrew Bartlett https://samba.org/~abartlet/ Authentication Developer, Samba Team https://samba.org Samba Development and Support, Catalyst IT https://catalyst.net.nz/services/samba