Fedora directory users - Jan 2008 - Paged Results support?

Does FDS support Paged Results?

I ask this because with FDS configured with "nsslapd-sizelimit: 2000" 
and a client search configured to return 1000 results per page it 
appears that 2000 results are returned and a "Sizelimit exceeded"
error
is returned by the server.

Attached is a perl snippet to test paged results.

Thanks,

Ben

Ben wrote:
> Does FDS support Paged Results?
>
> I ask this because with FDS configured with "nsslapd-sizelimit:
2000"
> and a client search configured to return 1000 results per page it 
> appears that 2000 results are returned and a "Sizelimit exceeded"
> error is returned by the server.
Fedora DS does not support the LDAPv3 Paged Results feature.  Fedora DS 
does support VLV (Virtual List View) which can be used to page through 
many entries.  This is what the console refers to as a "Browsing
Index".
>
> Attached is a perl snippet to test paged results.
>
> Thanks,
>
> Ben
> ------------------------------------------------------------------------
>
> --
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

Rich Megginson wrote:
> Ben wrote:
>> Does FDS support Paged Results?
> Fedora DS does not support the LDAPv3 Paged Results feature.  Fedora 
> DS does support VLV (Virtual List View) which can be used to page 
> through many entries.  This is what the console refers to as a 
> "Browsing Index".
Thanks for the info.

OK, so how would i go about using VLV in ''perl'' and
''php''?  And how does
that affect ''nss_ldap'' with "nss_paged_results yes"
and "pagesize 1000" set?

Ben

Ben wrote:
> Rich Megginson wrote:
>> Ben wrote:
>>> Does FDS support Paged Results?
>> Fedora DS does not support the LDAPv3 Paged Results feature.  Fedora 
>> DS does support VLV (Virtual List View) which can be used to page 
>> through many entries.  This is what the console refers to as a 
>> "Browsing Index".
> Thanks for the info.
>
> OK, so how would i go about using VLV in ''perl'' and
''php''?
You can use Net::LDAP to create and parse the VLV controls using the 
Convert::ASN1 package.  Not sure about php.
> And how does that affect ''nss_ldap'' with
"nss_paged_results yes" and
> "pagesize 1000" set?
If the paged results control is not marked critical, fedora ds will 
ignore it.  So it will have no effect.
>
> Ben
>
> -- 
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

Rich Megginson wrote:
> Ben wrote:
>> Rich Megginson wrote:
>>> Ben wrote:
>>>> Does FDS support Paged Results?
>>> Fedora DS does not support the LDAPv3 Paged Results feature. 
Fedora
>>> DS does support VLV (Virtual List View) which can be used to page 
>>> through many entries.  This is what the console refers to as a 
>>> "Browsing Index".
>> Thanks for the info.
>>
>> OK, so how would i go about using VLV in ''perl'' and
''php''?
> You can use Net::LDAP to create and parse the VLV controls using the 
> Convert::ASN1 package.  Not sure about php.
There is a "Net::LDAP::Control::VLV" perl module.  i have not been
able
to get it work.  Is there some preparation that needs to be done on the 
FDS server to allow/support it?
>> And how does that affect ''nss_ldap'' with
"nss_paged_results yes" and
>> "pagesize 1000" set?
> If the paged results control is not marked critical, fedora ds will 
> ignore it.  So it will have no effect.
I assume you mean in bugzilla?

Ben

Hi,

How would this be set at the client end? Ie it seems a bit silly to have
ldap.conf like this,

=====URI ldap://vuwunicvfdsm001.vuw.ac.nz/
BASE dc=vuw,dc=ac,dc=nz
TLS_CACERTDIR /etc/openldap/cacerts
ssl start_tls
=====
As if I lose the master (I assume) the slave (vuwunicvfdss001) wont be
queried....

Regards

Steven Jones

Ben wrote:
> Rich Megginson wrote:
>> Ben wrote:
>>> Rich Megginson wrote:
>>>> Ben wrote:
>>>>> Does FDS support Paged Results?
>>>> Fedora DS does not support the LDAPv3 Paged Results feature.  
>>>> Fedora DS does support VLV (Virtual List View) which can be
used to
>>>> page through many entries.  This is what the console refers to
as a
>>>> "Browsing Index".
>>> Thanks for the info.
>>>
>>> OK, so how would i go about using VLV in ''perl''
and ''php''?
>> You can use Net::LDAP to create and parse the VLV controls using the 
>> Convert::ASN1 package.  Not sure about php.
> There is a "Net::LDAP::Control::VLV" perl module.  i have not
been
> able to get it work.  Is there some preparation that needs to be done 
> on the FDS server to allow/support it?
Yes.  Check the documentation.  Not sure where it is documented though.  
You have to configure it then use the db_index command to create the
index.
>>> And how does that affect ''nss_ldap'' with
"nss_paged_results yes" and
>>> "pagesize 1000" set?
>> If the paged results control is not marked critical, fedora ds will 
>> ignore it.  So it will have no effect.
> I assume you mean in bugzilla?
? bugzilla ?
>
> Ben
>
> -- 
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

Rich Megginson wrote:
> Ben wrote:
>> Rich Megginson wrote:
>>> Ben wrote:
>>>> Rich Megginson wrote:
>>>>> Ben wrote:
>>>>>> Does FDS support Paged Results?
>>>>> Fedora DS does not support the LDAPv3 Paged Results
feature.
>>>>> Fedora DS does support VLV (Virtual List View) which can be
used
>>>>> to page through many entries.  This is what the console
refers to
>>>>> as a "Browsing Index".
>>>> Thanks for the info.
>>>>
>>>> OK, so how would i go about using VLV in
''perl'' and ''php''?
>>> You can use Net::LDAP to create and parse the VLV controls using
the
>>> Convert::ASN1 package.  Not sure about php.
>> There is a "Net::LDAP::Control::VLV" perl module.  i have not
been
>> able to get it work.  Is there some preparation that needs to be done 
>> on the FDS server to allow/support it?
> Yes.  Check the documentation.  Not sure where it is documented 
> though.  You have to configure it then use the db_index command to 
> create the index.
>>>> And how does that affect ''nss_ldap'' with
"nss_paged_results yes"
>>>> and "pagesize 1000" set?
>>> If the paged results control is not marked critical, fedora ds will
>>> ignore it.  So it will have no effect.
>> I assume you mean in bugzilla?
> ? bugzilla ?
Sorry i thought you were referring to getting paged support added to 
FDS, what do you mean "If the paged results control is not marked 
critical, fedora ds will ignore it." with reference to
''nss_ldap''?

Ben

Ben wrote:
> Rich Megginson wrote:
>> Ben wrote:
>>> Rich Megginson wrote:
>>>> Ben wrote:
>>>>> Rich Megginson wrote:
>>>>>> Ben wrote:
>>>>>>> Does FDS support Paged Results?
>>>>>> Fedora DS does not support the LDAPv3 Paged Results
feature.
>>>>>> Fedora DS does support VLV (Virtual List View) which
can be used
>>>>>> to page through many entries.  This is what the console
refers to
>>>>>> as a "Browsing Index".
>>>>> Thanks for the info.
>>>>>
>>>>> OK, so how would i go about using VLV in
''perl'' and ''php''?
>>>> You can use Net::LDAP to create and parse the VLV controls
using
>>>> the Convert::ASN1 package.  Not sure about php.
>>> There is a "Net::LDAP::Control::VLV" perl module.  i have
not been
>>> able to get it work.  Is there some preparation that needs to be 
>>> done on the FDS server to allow/support it?
>> Yes.  Check the documentation.  Not sure where it is documented 
>> though.  You have to configure it then use the db_index command to 
>> create the index.
>>>>> And how does that affect ''nss_ldap'' with
"nss_paged_results yes"
>>>>> and "pagesize 1000" set?
>>>> If the paged results control is not marked critical, fedora ds
will
>>>> ignore it.  So it will have no effect.
>>> I assume you mean in bugzilla?
>> ? bugzilla ?
>
> Sorry i thought you were referring to getting paged support added to 
> FDS, what do you mean "If the paged results control is not marked 
> critical, fedora ds will ignore it." with reference to
''nss_ldap''?
Sorry, that''s LDAP-speak.  Paged Results is an LDAPv3 Control that is 
added to the LDAP Search Request.  Controls can be marked as Critical or 
not.  If a control is marked as Critical, the server will return an 
error if unsupported, otherwise the server will ignore it.

If you want to file a bugzilla requesting that Fedora DS should support 
the Paged Results control, please do.
>
> Ben
>
> -- 
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

> Rich Megginson wrote:
> > Ben wrote:
> >> Rich Megginson wrote:
> >>> Ben wrote:
> >>>> Rich Megginson wrote:
> >>>>> Ben wrote:
> >>>>>> Does FDS support Paged Results?
> >>>>> Fedora DS does not support the LDAPv3 Paged Results
feature.
> >>>>> Fedora DS does support VLV (Virtual List View) which
can be used
> >>>>> to page through many entries.  This is what the
console refers to
> >>>>> as a "Browsing Index".
> >>>> Thanks for the info.
> >>>>
> >>>> OK, so how would i go about using VLV in
''perl'' and ''php''?
> >>> You can use Net::LDAP to create and parse the VLV controls
using the
> >>> Convert::ASN1 package.  Not sure about php.
> >> There is a "Net::LDAP::Control::VLV" perl module.  i
have not been
> >> able to get it work.  Is there some preparation that needs to be
done
> >> on the FDS server to allow/support it?
> > Yes.  Check the documentation.  Not sure where it is documented 
> > though.  You have to configure it then use the db_index command to 
> > create the index.
> >>>> And how does that affect ''nss_ldap'' with
"nss_paged_results yes"
> >>>> and "pagesize 1000" set?
> >>> If the paged results control is not marked critical, fedora ds
will
> >>> ignore it.  So it will have no effect.
> >> I assume you mean in bugzilla?
> > ? bugzilla ?
> 
> Sorry i thought you were referring to getting paged support added to 
> FDS, what do you mean "If the paged results control is not marked 
> critical, fedora ds will ignore it." with reference to
''nss_ldap''?
nss_ldap doesn''t seem to set the criticality of the paged result
control:

  rc = ldap_carete_page_control(__session.ls_conn,
                                __session.ls_config->ldc_pagesize,
                                NULL, 0, &serverCtrls[0]);

which means fedora-ds just ignore the control.

Masato Taruishi wrote:
>> Rich Megginson wrote:
>>> Ben wrote:
>>>> Rich Megginson wrote:
>>>>> Ben wrote:
>>>>>> Rich Megginson wrote:
>>>>>>> Ben wrote:
>>>>>>>> Does FDS support Paged Results?
>>>>>>> Fedora DS does not support the LDAPv3 Paged Results
feature.
>>>>>>> Fedora DS does support VLV (Virtual List View)
which can be used
>>>>>>> to page through many entries.  This is what the
console refers to
>>>>>>> as a "Browsing Index".
>>>>>> Thanks for the info.
>>>>>>
>>>>>> OK, so how would i go about using VLV in
''perl'' and ''php''?
>>>>> You can use Net::LDAP to create and parse the VLV controls
using the
>>>>> Convert::ASN1 package.  Not sure about php.
>>>> There is a "Net::LDAP::Control::VLV" perl module.  i
have not been
>>>> able to get it work.  Is there some preparation that needs to
be done
>>>> on the FDS server to allow/support it?
>>> Yes.  Check the documentation.  Not sure where it is documented 
>>> though.  You have to configure it then use the db_index command to 
>>> create the index.
>>>>>> And how does that affect ''nss_ldap''
with "nss_paged_results yes"
>>>>>> and "pagesize 1000" set?
>>>>> If the paged results control is not marked critical, fedora
ds will
>>>>> ignore it.  So it will have no effect.
>>>> I assume you mean in bugzilla?
>>> ? bugzilla ?
>> Sorry i thought you were referring to getting paged support added to 
>> FDS, what do you mean "If the paged results control is not marked 
>> critical, fedora ds will ignore it." with reference to
''nss_ldap''?
> 
> nss_ldap doesn''t seem to set the criticality of the paged result
> control:
> 
>   rc = ldap_carete_page_control(__session.ls_conn,
>                                 __session.ls_config->ldc_pagesize,
>                                 NULL, 0, &serverCtrls[0]);
> 
> which means fedora-ds just ignore the control.
I don''t think any client really needs that control.  The only reason
many were forced into at least __knowing__ about it is that Active
Directory returns pagedResults response even if not requested, in
blatant violation of LDAPv3.  So both the users and the developers of
FDS can safely ignore its existence.  Only client developers need to
know about it if they want their clients to be able to interoperate with AD.

p.



Ing. Pierangelo Masarati
OpenLDAP Core Team

SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
---------------------------------------
Office:  +39 02 23998309
Mobile:  +39 333 4963172
Email:   pierangelo.masarati@sys-net.it
---------------------------------------

Pierangelo Masarati wrote:
> Masato Taruishi wrote:
>   
>>> Rich Megginson wrote:
>>>       
>>>> Ben wrote:
>>>>         
>>>>> Rich Megginson wrote:
>>>>>           
>>>>>> Ben wrote:
>>>>>>             
>>>>>>> Rich Megginson wrote:
>>>>>>>               
>>>>>>>> Ben wrote:
>>>>>>>>                 
>>>>>>>>> Does FDS support Paged Results?
>>>>>>>>>                   
>>>>>>>> Fedora DS does not support the LDAPv3 Paged
Results feature.
>>>>>>>> Fedora DS does support VLV (Virtual List View)
which can be used
>>>>>>>> to page through many entries.  This is what the
console refers to
>>>>>>>> as a "Browsing Index".
>>>>>>>>                 
>>>>>>> Thanks for the info.
>>>>>>>
>>>>>>> OK, so how would i go about using VLV in
''perl'' and ''php''?
>>>>>>>               
>>>>>> You can use Net::LDAP to create and parse the VLV
controls using the
>>>>>> Convert::ASN1 package.  Not sure about php.
>>>>>>             
>>>>> There is a "Net::LDAP::Control::VLV" perl module.
i have not been
>>>>> able to get it work.  Is there some preparation that needs
to be done
>>>>> on the FDS server to allow/support it?
>>>>>           
>>>> Yes.  Check the documentation.  Not sure where it is documented
>>>> though.  You have to configure it then use the db_index command
to
>>>> create the index.
>>>>         
>>>>>>> And how does that affect
''nss_ldap'' with "nss_paged_results yes"
>>>>>>> and "pagesize 1000" set?
>>>>>>>               
>>>>>> If the paged results control is not marked critical,
fedora ds will
>>>>>> ignore it.  So it will have no effect.
>>>>>>             
>>>>> I assume you mean in bugzilla?
>>>>>           
>>>> ? bugzilla ?
>>>>         
>>> Sorry i thought you were referring to getting paged support added
to
>>> FDS, what do you mean "If the paged results control is not
marked
>>> critical, fedora ds will ignore it." with reference to
''nss_ldap''?
>>>       
>> nss_ldap doesn''t seem to set the criticality of the paged
result
>> control:
>>
>>   rc = ldap_carete_page_control(__session.ls_conn,
>>                                 __session.ls_config->ldc_pagesize,
>>                                 NULL, 0, &serverCtrls[0]);
>>
>> which means fedora-ds just ignore the control.
>>     
>
> I don''t think any client really needs that control.  The only
reason
> many were forced into at least __knowing__ about it is that Active
> Directory returns pagedResults response even if not requested, in
> blatant violation of LDAPv3.  So both the users and the developers of
> FDS can safely ignore its existence.  Only client developers need to
> know about it if they want their clients to be able to interoperate with
AD.
>
> p.
>   
If i recall correctly the reason i started using paged results was to 
exceed the ''sizelimit'' restriction when i was using OpenLDAP. 
i.e. if
"sizelimit = 100" you could get 300 results using paged results.

So in relation to ''nss_ldap'' isn''t that what the
"nss_paged_results yes"
and "pagesize 1000" options are good for?  So with a directory of
>2000
users nss_ldap could get them all when the server has sizelimit set to 
say 1000.

Ben

Ben wrote:
> If i recall correctly the reason i started using paged results was to
> exceed the ''sizelimit'' restriction when i was using
OpenLDAP.  i.e. if
> "sizelimit = 100" you could get 300 results using paged results.
> 
> So in relation to ''nss_ldap'' isn''t that what the
"nss_paged_results yes"
> and "pagesize 1000" options are good for?  So with a directory of
>2000
> users nss_ldap could get them all when the server has sizelimit set to
> say 1000.
That would work with AD, not with OL.  With OL, the sizelimit refers to
the total amount of entries returned by a search, no matter what
fraction is returned in each page.

p.



Ing. Pierangelo Masarati
OpenLDAP Core Team

SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
---------------------------------------
Office:  +39 02 23998309
Mobile:  +39 333 4963172
Email:   pierangelo.masarati@sys-net.it
---------------------------------------