Fedora directory users - Jul 2006 - Strange problem -- LDAP server hosed

Hey guys... I hope I can provide sufficient detail to get a clue here, but I
don''t have much info about what''s happening yet.

We are using Fedora DS v1.0.2, and the client is a Java application using
JNDI.  The client is doing some tests that involve manipulating the schema,
adding/removing attributes, adding/modifying/removing object classes.
During this process, objects of these types are created in the directory,
too.

What''s happening is that it seems like objects with duplicate names are
being created, i.e. cn=object1 is created twice.  The second time it gets
created, its name is nsuniqueid=<alphanumeric string>.  I''m not
sure how
this could happen, because typically if you tried to create a duplicate
entry, you''d get a javax.naming.directory.NameAlreadyBoundException.

What''s worse, I can''t delete any of these entries.  When I try
to, it says
"Operation not allowed on nonleaf" (doing this via the graphical
console),
although the object in question is a leaf.  Typically, even for nonleafs,
the GUI would recursively delete everything.

The only fix for this problem was to delete the underlying database behind
the root suffix, and recreate it fresh.  Obviously this is a serious
problem, in a production environment, we can''t afford to be doing
something
like this.  This has happened on two of our servers now, and on the second
one, I''m unable to even delete the database!  It got halfway through,
and
then sits there hanging.  That server is completely out of commision now.

Any information would be appreciated!!

Mike

Follow-up:  I was able to finish deleting and recreating the database on the
second server by restarting the server (/etc/init.d/ns-slapd restart).

Mike

On 7/7/06, Mike Mueller <bitdumpster@gmail.com>
wrote:
>
> Hey guys... I hope I can provide sufficient detail to get a clue here, but
> I don''t have much info about what''s happening yet.
>
> We are using Fedora DS v1.0.2, and the client is a Java application using
> JNDI.  The client is doing some tests that involve manipulating the schema,
> adding/removing attributes, adding/modifying/removing object classes.
> During this process, objects of these types are created in the directory,
> too.
>
> What''s happening is that it seems like objects with duplicate
names are
> being created, i.e. cn=object1 is created twice.  The second time it gets
> created, its name is nsuniqueid=<alphanumeric string>.  I''m
not sure how
> this could happen, because typically if you tried to create a duplicate
> entry, you''d get a
javax.naming.directory.NameAlreadyBoundException.
>
> What''s worse, I can''t delete any of these entries.  When
I try to, it says
> "Operation not allowed on nonleaf" (doing this via the graphical
console),
> although the object in question is a leaf.  Typically, even for nonleafs,
> the GUI would recursively delete everything.
>
> The only fix for this problem was to delete the underlying database behind
> the root suffix, and recreate it fresh.  Obviously this is a serious
> problem, in a production environment, we can''t afford to be doing
something
> like this.  This has happened on two of our servers now, and on the second
> one, I''m unable to even delete the database!  It got halfway
through, and
> then sits there hanging.  That server is completely out of commision now.
>
> Any information would be appreciated!!
>
> Mike
>

Mike Mueller wrote:
> Hey guys... I hope I can provide sufficient detail to get a clue here, 
> but I don''t have much info about what''s happening yet.
>
> We are using Fedora DS v1.0.2, and the client is a Java application 
> using JNDI.  The client is doing some tests that involve manipulating 
> the schema, adding/removing attributes, adding/modifying/removing 
> object classes.  During this process, objects of these types are 
> created in the directory, too.
>
> What''s happening is that it seems like objects with duplicate
names
> are being created, i.e. cn=object1 is created twice.  The second time 
> it gets created, its name is nsuniqueid=<alphanumeric string>. 
I''m
> not sure how this could happen, because typically if you tried to 
> create a duplicate entry, you''d get a 
> javax.naming.directory.NameAlreadyBoundException.
Are you are using multi-master replication?  It sounds like these 
entries you are seeing are replication conflict entries.  You can read 
about dealing with them in the Administrator''s Guide.  Here is a link
to
the relevant section:

    
http://www.redhat.com/docs/manuals/dir-server/ag/7.1/replicat.html#1106141
>
> What''s worse, I can''t delete any of these entries.  When
I try to, it
> says "Operation not allowed on nonleaf" (doing this via the
graphical
> console), although the object in question is a leaf.  Typically, even 
> for nonleafs, the GUI would recursively delete everything.
What happens when you try to delete the entry with ldapdelete?  Also, 
did you verify that the entry is indeed a leaf entry with ldapsearch as 
"cn=directory manager"?

-NGK
>
> The only fix for this problem was to delete the underlying database 
> behind the root suffix, and recreate it fresh.  Obviously this is a 
> serious problem, in a production environment, we can''t afford to
be
> doing something like this.  This has happened on two of our servers 
> now, and on the second one, I''m unable to even delete the
database!
> It got halfway through, and then sits there hanging.  That server is 
> completely out of commision now.
>
> Any information would be appreciated!!
>
> Mike
> ------------------------------------------------------------------------
>
> --
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>

On 7/7/06, Nathan Kinder <nkinder@redhat.com>
wrote:
>
> Are you are using multi-master replication?  It sounds like these
> entries you are seeing are replication conflict entries.  You can read
> about dealing with them in the Administrator''s Guide.  Here is a
link to
> the relevant section:
>
http://www.redhat.com/docs/manuals/dir-server/ag/7.1/replicat.html#1106141


No, replication is currently not active on these machines...

What happens when you try to delete the entry with ldapdelete? 
Also,
> did you verify that the entry is indeed a leaf entry with ldapsearch as
> "cn=directory manager"?
>
> -NGK

Unfortunately, since I''ve deleted and recreated the databases, I
haven''t
reproduced this problem.  I will try this the next time it happens, assuming
it ever happens again.

Thanks for the quick response.

Mike

Mike Mueller wrote:
>
>
> On 7/7/06, *Nathan Kinder* <nkinder@redhat.com 
> <mailto:nkinder@redhat.com>> wrote:
>
>     Are you are using multi-master replication?  It sounds like these
>     entries you are seeing are replication conflict entries.  You can read
>     about dealing with them in the Administrator''s Guide.  Here is
a
>     link to
>     the relevant section:
>
>    
http://www.redhat.com/docs/manuals/dir-server/ag/7.1/replicat.html#1106141
>
>
> No, replication is currently not active on these machines...
You say currently, was it once?


-- 
Pete

Ok, I just double checked, and apparently it never got turned off.  The two
machines involved were using multiple-master replication.  Thanks for the
insightful replies, guys.  Seems like Nathan described exactly what was
happening.

Mike

On 7/7/06, Pete Rowley <prowley@redhat.com> wrote:
>
> You say currently, was it once?
>
> --
> Pete
>