Fedora directory users - Jul 2008 - Case Sensitive Lookup and Searching

Hi,

I''m running Fedora-Directory/1.0.2 B2006.111.2147, and talking to it
via a Java App. Previously the app was talking to an OpenLDAP 2.3.x
server.

My problem is with this:

Object o = ctx.lookup("memberUid=steves,ou=People");

In OpenLDAP, it returns the correct user (steves). In FDS, it returns
the wrong user, ''Steves''. Yes, unfortunately our data is like
that,
where case sensitivity is important. In fact, as a side issue, when we
import the data from ldif into FDS, the ldif2db process ignores
duplicate entries (i.e. steves was inserted, but Steves ignored as it
was considered a duplicate).

ldif2db Error: import company: WARNING: Skipping duplicate entry
"memberUid=steves,ou=People,o=company.com"

As you might imagine, I''d like to get it so both ldif2db and lookups
are case sensitive. However, it seems like ldapsearch is case
sensitive.

# ./ldapsearch  -h 127.0.0.1  -b "o=company.com"  memberUid=steves
# ./ldapsearch  -h 127.0.0.1  -b "o=company.com"  memberUid=Steves
version: 1
dn: memberUid=Steves,ou=People,o=company.com
personalTitle: Mr
etc...

So, the question goes, what am I missing? I''ve even tried changing the
definition of memberUid in config/schema/10rfc2307.ldif to use

attributeTypes: (
  1.3.6.1.1.1.1.12
  NAME ''memberUid''
  DESC ''Standard LDAP attribute type''
  EQUALITY caseExactIA5Match
  SUBSTRINGS caseExactIA5SubstringsMatch
  SYNTAX ''IA5String''
)

Ideas?

Thanks,

Mike

Mike C wrote:
> 
> Object o = ctx.lookup("memberUid=steves,ou=People");
Attribute ''memberUid'' was never meant to be used within a user
entry.

So general advice is to define a better schema and sanitize your data. 
You probably already know that. ;-)
> I''ve even tried changing the
> definition of memberUid in config/schema/10rfc2307.ldif to use
> 
> attributeTypes: (
>   1.3.6.1.1.1.1.12
>   NAME ''memberUid''
>   DESC ''Standard LDAP attribute type''
>   EQUALITY caseExactIA5Match
>   SUBSTRINGS caseExactIA5SubstringsMatch
>   SYNTAX ''IA5String''
> )
> 
> Ideas?
Well, looking at the schema in FDS there''s no such matching rule named 
''caseExactIA5Match'' (IMO the server shouldn''t even
start with such a
mis-defined schema element declaration). The only caseExact* matching 
rules listed in the subschema are ''caseExactOrderingMatch-en''
and
''caseExactSubstringMatch-en'' which both does not look suitable
to me.

Strange enough there''s not even an EQUALITY matching rule defined for 
attribute type ''memberUid'' at all...

I really wonder whether default matching rules are applied for certain 
LDAP syntaxes and how to find out which these are.

Ciao, Michael.

Michael Ströder wrote:
> Mike C wrote:
>>
>> Object o = ctx.lookup("memberUid=steves,ou=People");
>
> Attribute ''memberUid'' was never meant to be used within a
user entry.
>
> So general advice is to define a better schema and sanitize your data. 
> You probably already know that. ;-)
>
>> I''ve even tried changing the
>> definition of memberUid in config/schema/10rfc2307.ldif to use
>>
>> attributeTypes: (
>>   1.3.6.1.1.1.1.12
>>   NAME ''memberUid''
>>   DESC ''Standard LDAP attribute type''
>>   EQUALITY caseExactIA5Match
>>   SUBSTRINGS caseExactIA5SubstringsMatch
>>   SYNTAX ''IA5String''
>> )
>>
>> Ideas?
>
> Well, looking at the schema in FDS there''s no such matching rule
named
> ''caseExactIA5Match'' (IMO the server shouldn''t
even start with such a
> mis-defined schema element declaration). The only caseExact* matching 
> rules listed in the subschema are
''caseExactOrderingMatch-en'' and
> ''caseExactSubstringMatch-en'' which both does not look
suitable to me.
>
> Strange enough there''s not even an EQUALITY matching rule defined
for
> attribute type ''memberUid'' at all...
>
> I really wonder whether default matching rules are applied for certain 
> LDAP syntaxes and how to find out which these are.
>
If there is no matching rule, it just goes by the most appropriate 
internal matching rule that corresponds to the SYNTAX.
> Ciao, Michael.
>
> -- 
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

Rich Megginson wrote:
> Michael Ströder wrote:
>> I really wonder whether default matching rules are applied for certain 
>> LDAP syntaxes and how to find out which these are.
>>
> If there is no matching rule, it just goes by the most appropriate 
> internal matching rule that corresponds to the SYNTAX.
Any description how the "most appropriate internal matching rule" is 
chosen? Is the list of matching rules in the subschema subentry complete?

Ciao, Michael.

Michael Ströder wrote:
> Rich Megginson wrote:
>> Michael Ströder wrote:
>>> I really wonder whether default matching rules are applied for 
>>> certain LDAP syntaxes and how to find out which these are.
>>>
>> If there is no matching rule, it just goes by the most appropriate 
>> internal matching rule that corresponds to the SYNTAX.
>
> Any description how the "most appropriate internal matching rule"
is
> chosen?
No, not really, afaik.  I suppose it attempts to use "common sense"
e.g.
if the syntax is for case sensitive string, it uses a matching rule for 
case sensitive string comparison, and uses indexers for case sensitive 
strings.
> Is the list of matching rules in the subschema subentry complete?
Complete as in "implements every matching rule defined in every LDAP 
RFC" - no.
>
> Ciao, Michael.
>
> -- 
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

Rich Megginson wrote:
> Michael Ströder wrote:
>> Rich Megginson wrote:
>>> Michael Ströder wrote:
>>>> I really wonder whether default matching rules are applied for 
>>>> certain LDAP syntaxes and how to find out which these are.
>>>>
>>> If there is no matching rule, it just goes by the most appropriate 
>>> internal matching rule that corresponds to the SYNTAX.
>>
>> Any description how the "most appropriate internal matching
rule" is
>> chosen?
> No, not really, afaik.  I suppose it attempts to use "common
sense" e.g.
> if the syntax is for case sensitive string, it uses a matching rule for 
> case sensitive string comparison, and uses indexers for case sensitive 
> strings.
And how to determine whether a syntax is used for case sensitive 
strings? That''s exactly the problem of the original poster I guess.
>> Is the list of matching rules in the subschema subentry complete?
> Complete as in "implements every matching rule defined in every LDAP 
> RFC" - no.
Complete in the sense: The subschema subentry lists all the matching 
rules which are implemented, no more, no less.

Ciao, Michael.

Michael Ströder wrote:
> Rich Megginson wrote:
>> Michael Ströder wrote:
>>> Rich Megginson wrote:
>>>> Michael Ströder wrote:
>>>>> I really wonder whether default matching rules are applied
for
>>>>> certain LDAP syntaxes and how to find out which these are.
>>>>>
>>>> If there is no matching rule, it just goes by the most
appropriate
>>>> internal matching rule that corresponds to the SYNTAX.
>>>
>>> Any description how the "most appropriate internal matching
rule" is
>>> chosen?
>> No, not really, afaik.  I suppose it attempts to use "common
sense"
>> e.g. if the syntax is for case sensitive string, it uses a matching 
>> rule for case sensitive string comparison, and uses indexers for case 
>> sensitive strings.
>
> And how to determine whether a syntax is used for case sensitive 
> strings? That''s exactly the problem of the original poster I
guess.
I think IA5String is case sensitive, and Directory String is case 
insensitive.
>
>>> Is the list of matching rules in the subschema subentry complete?
>> Complete as in "implements every matching rule defined in every
LDAP
>> RFC" - no.
>
> Complete in the sense: The subschema subentry lists all the matching 
> rules which are implemented, no more, no less.
Not sure.
>
> Ciao, Michael.
>
> -- 
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

Rich Megginson wrote:
> Michael Ströder wrote:
>> Rich Megginson wrote:
>>> Michael Ströder wrote:
>>>> Rich Megginson wrote:
>>>>> Michael Ströder wrote:
>>>>>> I really wonder whether default matching rules are
applied for
>>>>>> certain LDAP syntaxes and how to find out which these
are.
>>>>>>
>>>>> If there is no matching rule, it just goes by the most
appropriate
>>>>> internal matching rule that corresponds to the SYNTAX.
>>>>
>>>> Any description how the "most appropriate internal
matching rule" is
>>>> chosen?
>>> No, not really, afaik.  I suppose it attempts to use "common
sense"
>>> e.g. if the syntax is for case sensitive string, it uses a matching
>>> rule for case sensitive string comparison, and uses indexers for
case
>>> sensitive strings.
>>
>> And how to determine whether a syntax is used for case sensitive 
>> strings? That''s exactly the problem of the original poster I
guess.
 >
> I think IA5String is case sensitive, and Directory String is case 
> insensitive.
I don''t think so (see section 4.2. of RFC 4517).

Ciao, Michael.

Michael Ströder wrote:
> Rich Megginson wrote:
>> Michael Ströder wrote:
>>> Rich Megginson wrote:
>>>> Michael Ströder wrote:
>>>>> Rich Megginson wrote:
>>>>>> Michael Ströder wrote:
>>>>>>> I really wonder whether default matching rules are
applied for
>>>>>>> certain LDAP syntaxes and how to find out which
these are.
>>>>>>>
>>>>>> If there is no matching rule, it just goes by the most 
>>>>>> appropriate internal matching rule that corresponds to
the SYNTAX.
>>>>>
>>>>> Any description how the "most appropriate internal
matching rule"
>>>>> is chosen?
>>>> No, not really, afaik.  I suppose it attempts to use
"common sense"
>>>> e.g. if the syntax is for case sensitive string, it uses a
matching
>>>> rule for case sensitive string comparison, and uses indexers
for
>>>> case sensitive strings.
>>>
>>> And how to determine whether a syntax is used for case sensitive 
>>> strings? That''s exactly the problem of the original poster
I guess.
> >
>> I think IA5String is case sensitive, and Directory String is case 
>> insensitive.
>
> I don''t think so (see section 4.2. of RFC 4517).
Ok.  But the the way Fedora DS works is that it treats IA5String as case 
sensitive, and Directory String as case insensitive - see 
ldap/servers/plugins/syntaxes/ces.c and cis.c
>
> Ciao, Michael.
>
> -- 
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

Rich Megginson wrote:
> Michael Ströder wrote:
>> Rich Megginson wrote:
>>> I think IA5String is case sensitive, and Directory String is case 
>>> insensitive.
>> I don''t think so (see section 4.2. of RFC 4517).
> Ok.  But the the way Fedora DS works is that it treats IA5String as case 
> sensitive, and Directory String as case insensitive - see 
> ldap/servers/plugins/syntaxes/ces.c and cis.c
Hmm, but then the problem of the original poster is that the matching 
rule applied to an attribute value (based on your rule above) has 
nothing to with the normalization of the entry''s RDN.
Because he was asking about memberUid=steves vs. memberUid=Steves which 
are two different entries in his data (based on caseExactIA5Match) but 
are treated as the same entry in FDS.

(Nevertheless he should get his data fixed for various reasons.)

Ciao, Michael.

On Tue, Jul 8, 2008 at 6:24 AM, Michael Ströder <michael@stroeder.com>
wrote:
> Hmm, but then the problem of the original poster is that the matching rule
> applied to an attribute value (based on your rule above) has nothing to
with
> the normalization of the entry''s RDN.
> Because he was asking about memberUid=steves vs. memberUid=Steves which are
> two different entries in his data (based on caseExactIA5Match) but are
> treated as the same entry in FDS.
>
> (Nevertheless he should get his data fixed for various reasons.)
I agree, my schema (and data) are terrible. It''s an artifact from
openldap not being as conforming as fds.

My main concern is that sanitizing my repository would require
changing usernames for a hundred odd external users, something I wish
to avoid. But given how memberUid''s case sensitivity is nullified when
part of a dn, migration it is.

Thank you all for your help,

Regards,

Mike

Mike C wrote:
> I agree, my schema (and data) are terrible. It''s an artifact from
> openldap not being as conforming as fds.
I assume you meant the opposite.  OpenLDAP is operating according to the 
specification of memberUid as in RFC 2307 (case-sensitive), while FDS 
isn''t.  Whether you might consider this a feature or not, this is a
fact.

p.


Ing. Pierangelo Masarati
OpenLDAP Core Team

SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office:  +39 02 23998309
Mobile:  +39 333 4963172
Email:   ando@sys-net.it
-----------------------------------

Mike C wrote:
> I agree, my schema (and data) are terrible.
Then fix it.
> It''s an artifact from openldap not being as conforming as fds.
Hmm, from the LDAPv3 standard''s standpoint I strongly disagree.
Strictly speaking it''s the other way round.
> My main concern is that sanitizing my repository would require
> changing usernames for a hundred odd external users, something I wish
> to avoid.
Why do you have to change the user names? You could split the data 
across different subtrees and change the clients'' configuration 
accordingly. Or you could merge entries. Or whatever...
That''s just meant as a start to think about what you can do without 
changing what the end-user has to type in. Your mileage may vary...

Ciao, Michael.