Fedora directory users - Mar 2007 - lookthrough vs. sizelimit

Something I''ve been wondering about:
It seems like nsslapd-lookthroughlimit and nsslapd-sizelimit effectively 
do the same thing, but just return a different error code.

If nsslapd-lookthroughlimit is lower, the error code is 11 and the error 
message is:
ldap_search: Administrative limit exceeded

If nsslapd-sizelimit is lower, the error code is 4 and the error message is:
ldap_search: Sizelimit exceeded

I''ve read the description of both of these variables many times in the 
documentation, and I think I understand the theoretical difference.  But 
in practical terms, it still seems like whichever has the higher value 
will never have an effect, since the lower limit on the other is always 
hit first.

Can anyone describe a practical situation where both the lookthrough and 
size limits would come into play?
Is there any particular reason to prefer one or the other to enforce 
maximum search result limits?


Thank you!
-- George

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


As I understand it, sizelimit determines the maximum number of results that 
are returned from the search, whereas lookthroughlimit determines the 
maximum number of things that will be searched in the first place.

Frankly, in our setup I have lookthroughlimit set to -1 (unlimited). Since 
the order of the searching is non-deterministic, I can''t fathom any use
for
it. It has to be at least as large as your largest searchable tree, or else 
there will be entries that can never be returned in a search. If anyone out 
there is using this parameter, can you explain how/why?

  -paul

- --On Wednesday, March 14, 2007 12:45:49 PM -0700 George Holbert 
<gholbert@broadcom.com> wrote:
> Something I''ve been wondering about:
> It seems like nsslapd-lookthroughlimit and nsslapd-sizelimit effectively
> do the same thing, but just return a different error code.
>
> If nsslapd-lookthroughlimit is lower, the error code is 11 and the error
> message is:
> ldap_search: Administrative limit exceeded
>
> If nsslapd-sizelimit is lower, the error code is 4 and the error message
> is:
> ldap_search: Sizelimit exceeded
>
> I''ve read the description of both of these variables many times in
the
> documentation, and I think I understand the theoretical difference.  But
> in practical terms, it still seems like whichever has the higher value
> will never have an effect, since the lower limit on the other is always
> hit first.
>
> Can anyone describe a practical situation where both the lookthrough and
> size limits would come into play?
> Is there any particular reason to prefer one or the other to enforce
> maximum search result limits?
>
>
> Thank you!
> -- George
>
>
> --
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>


- -- 
Paul D. Engle                | Rice University
Sr. Systems Administrator    | Information Technology - MS119
(713) 348-4702               | P.O. Box 1892
pengle@rice.edu              | Houston, TX 77251-1892
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)

iD8DBQFF+FadCpkISWtyHNsRAuUVAKC3jFoDbyrl9ut37XhwySrBMX4MOQCcCton
eggDv1KLhHc1Y8dctEjZIq4=XnpW
-----END PGP SIGNATURE-----

The notion behind lookthrough limit is that the administrator
can dermine an upper bound for the amount of WORK that
the server will perform for a given client''s search. This is
basically a simple form of denial of service control.
So clients that hit the limit are not expected to receive
useful results at all. The client should say something like
''the server didn''t complete your search because you burned
too much gas''.

I believe it is fairly common to want to set a lookthrough limit
for ''ordinary'' users, but have an infinite limit for special
accounts
that are expected to perform expensive searches.

There are other ways to skin the cat, for example denying
certain users the ability to perform un-indexed searches at all.


Paul Engle wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>
>As I understand it, sizelimit determines the maximum number of results that 
>are returned from the search, whereas lookthroughlimit determines the 
>maximum number of things that will be searched in the first place.
>
>Frankly, in our setup I have lookthroughlimit set to -1 (unlimited). Since 
>the order of the searching is non-deterministic, I can''t fathom any
use for
>it. It has to be at least as large as your largest searchable tree, or else 
>there will be entries that can never be returned in a search. If anyone out 
>there is using this parameter, can you explain how/why?
>
>  -paul
>
>- --On Wednesday, March 14, 2007 12:45:49 PM -0700 George Holbert 
><gholbert@broadcom.com> wrote:
>
>  
>
>>Something I''ve been wondering about:
>>It seems like nsslapd-lookthroughlimit and nsslapd-sizelimit effectively
>>do the same thing, but just return a different error code.
>>
>>If nsslapd-lookthroughlimit is lower, the error code is 11 and the error
>>message is:
>>ldap_search: Administrative limit exceeded
>>
>>If nsslapd-sizelimit is lower, the error code is 4 and the error message
>>is:
>>ldap_search: Sizelimit exceeded
>>
>>I''ve read the description of both of these variables many times
in the
>>documentation, and I think I understand the theoretical difference.  But
>>in practical terms, it still seems like whichever has the higher value
>>will never have an effect, since the lower limit on the other is always
>>hit first.
>>
>>Can anyone describe a practical situation where both the lookthrough and
>>size limits would come into play?
>>Is there any particular reason to prefer one or the other to enforce
>>maximum search result limits?
>>
>>
>>Thank you!
>>-- George
>>
>>
>>--
>>Fedora-directory-users mailing list
>>Fedora-directory-users@redhat.com
>>https://www.redhat.com/mailman/listinfo/fedora-directory-users
>>
>>    
>>
>
>
>
>- -- 
>Paul D. Engle                | Rice University
>Sr. Systems Administrator    | Information Technology - MS119
>(713) 348-4702               | P.O. Box 1892
>pengle@rice.edu              | Houston, TX 77251-1892
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.2.6 (GNU/Linux)
>
>iD8DBQFF+FadCpkISWtyHNsRAuUVAKC3jFoDbyrl9ut37XhwySrBMX4MOQCcCton
>eggDv1KLhHc1Y8dctEjZIq4>=XnpW
>-----END PGP SIGNATURE-----
>
>--
>Fedora-directory-users mailing list
>Fedora-directory-users@redhat.com
>https://www.redhat.com/mailman/listinfo/fedora-directory-users
>  
>

> The notion behind lookthrough limit is that the administrator
> can dermine an upper bound for the amount of WORK that
> the server will perform for a given client''s search.
That makes sense.
Does this mean if a sizelimit (not lookthrough) is hit, the server 
continues searching the database, even though it has already returned 
error code 4 to the client?

Thanks for the responses,
-- George


David Boreham wrote:
>
> The notion behind lookthrough limit is that the administrator
> can dermine an upper bound for the amount of WORK that
> the server will perform for a given client''s search. This is
> basically a simple form of denial of service control.
> So clients that hit the limit are not expected to receive
> useful results at all. The client should say something like
> ''the server didn''t complete your search because you
burned
> too much gas''.
>
> I believe it is fairly common to want to set a lookthrough limit
> for ''ordinary'' users, but have an infinite limit for
special accounts
> that are expected to perform expensive searches.
>
> There are other ways to skin the cat, for example denying
> certain users the ability to perform un-indexed searches at all.
>
>
> Paul Engle wrote:
>
>> As I understand it, sizelimit determines the maximum number of 
>> results that are returned from the search, whereas lookthroughlimit 
>> determines the maximum number of things that will be searched in the 
>> first place.
>>
>> Frankly, in our setup I have lookthroughlimit set to -1 (unlimited). 
>> Since the order of the searching is non-deterministic, I can''t
fathom
>> any use for it. It has to be at least as large as your largest 
>> searchable tree, or else there will be entries that can never be 
>> returned in a search. If anyone out there is using this parameter, 
>> can you explain how/why?
>>
>>  -paul
>>
>> - --On Wednesday, March 14, 2007 12:45:49 PM -0700 George Holbert 
>> <gholbert@broadcom.com> wrote:
>>
>>  
>>
>>> Something I''ve been wondering about:
>>> It seems like nsslapd-lookthroughlimit and nsslapd-sizelimit 
>>> effectively
>>> do the same thing, but just return a different error code.
>>>
>>> If nsslapd-lookthroughlimit is lower, the error code is 11 and the 
>>> error
>>> message is:
>>> ldap_search: Administrative limit exceeded
>>>
>>> If nsslapd-sizelimit is lower, the error code is 4 and the error 
>>> message
>>> is:
>>> ldap_search: Sizelimit exceeded
>>>
>>> I''ve read the description of both of these variables many
times in the
>>> documentation, and I think I understand the theoretical difference.
>>> But
>>> in practical terms, it still seems like whichever has the higher
value
>>> will never have an effect, since the lower limit on the other is
always
>>> hit first.
>>>
>>> Can anyone describe a practical situation where both the
lookthrough
>>> and
>>> size limits would come into play?
>>> Is there any particular reason to prefer one or the other to
enforce
>>> maximum search result limits?
>>>
>>>
>>> Thank you!
>>> -- George
>>>
>>>
>>>   
>>
>>
>>
>> - -- Paul D. Engle                | Rice University
>> Sr. Systems Administrator    | Information Technology - MS119
>> (713) 348-4702               | P.O. Box 1892
>> pengle@rice.edu              | Houston, TX 77251-1892

George Holbert wrote:
>> The notion behind lookthrough limit is that the administrator
>> can dermine an upper bound for the amount of WORK that
>> the server will perform for a given client''s search.
>
>
> That makes sense.
> Does this mean if a sizelimit (not lookthrough) is hit, the server 
> continues searching the database, even though it has already returned 
> error code 4 to the client?
No. That''d be quite silly, wouldn''t it ?

It _might_ do a bunch of work up front to service a search
only to discover when sending entries back to the client that the
size limit is exceeded.

> No. That''d be quite silly, wouldn''t it ?
Absolutely :), that''s why I was curious.
So correct me if this wrong, but sounds like either of the two can be 
used to limit how much the server works on a search, but they each take 
effect at a different part of the search algorithm.
I still wonder why you''d choose one over the other to implement result 
limits?  Seems kind of like a door with two knobs.  Maybe there''s some 
specific cases where one is preferable.

Thanks again for the replies,
-- George


David Boreham wrote:
> George Holbert wrote:
>
>>> The notion behind lookthrough limit is that the administrator
>>> can dermine an upper bound for the amount of WORK that
>>> the server will perform for a given client''s search.
>>
>>
>> That makes sense.
>> Does this mean if a sizelimit (not lookthrough) is hit, the server 
>> continues searching the database, even though it has already returned 
>> error code 4 to the client?
>
> No. That''d be quite silly, wouldn''t it ?
>
> It _might_ do a bunch of work up front to service a search
> only to discover when sending entries back to the client that the
> size limit is exceeded.
>
>
>
>
>

George Holbert wrote:
>> No. That''d be quite silly, wouldn''t it ?
>
> Absolutely :), that''s why I was curious.
> So correct me if this wrong, but sounds like either of the two can be 
> used to limit how much the server works on a search, but they each 
> take effect at a different part of the search algorithm.
> I still wonder why you''d choose one over the other to implement
result
> limits?  Seems kind of like a door with two knobs.  Maybe there''s
some
> specific cases where one is preferable.
In general, lookthroughlimit is much stricter than sizelimit.

For example, let''s say a user wants to do an unindexed search for 
(description=*something*).  Let''s say that there are 5000 users and
1000
users who have a description attribute that matches *something*.  The 
server will have to search through every entry in sequential 
(indeterminate) order to find matches.

If you set lookthroughlimit to be 1000, and set sizelimit to be 
unlimited, the server will look at up to 1000 entries looking for 
description=*something*.  Some of them may match, some of them may not, 
and the server will return 1000 or fewer entries (indeterminate).  The 
server is limited in the amount of work it performs searching through 
the database.

If you set sizelimit to be 1000, and set lookthroughlimit to be 
unlimited, the server could look at all 5000 user entries, until it 
finds 1000 entries which match, at which point it will terminate the 
search and return the 1000 entries to the user.
>
> Thanks again for the replies,
> -- George
>
>
> David Boreham wrote:
>> George Holbert wrote:
>>
>>>> The notion behind lookthrough limit is that the administrator
>>>> can dermine an upper bound for the amount of WORK that
>>>> the server will perform for a given client''s search.
>>>
>>>
>>> That makes sense.
>>> Does this mean if a sizelimit (not lookthrough) is hit, the server 
>>> continues searching the database, even though it has already 
>>> returned error code 4 to the client?
>>
>> No. That''d be quite silly, wouldn''t it ?
>>
>> It _might_ do a bunch of work up front to service a search
>> only to discover when sending entries back to the client that the
>> size limit is exceeded.
>>
>>
>>
>>
>>
>
>
>
> -- 
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

That clarifies it perfectly.
Thanks for the example!

Richard Megginson wrote:
> In general, lookthroughlimit is much stricter than sizelimit.
>
> For example, let''s say a user wants to do an unindexed search for 
> (description=*something*).  Let''s say that there are 5000 users
and
> 1000 users who have a description attribute that matches *something*.  
> The server will have to search through every entry in sequential 
> (indeterminate) order to find matches.
>
> If you set lookthroughlimit to be 1000, and set sizelimit to be 
> unlimited, the server will look at up to 1000 entries looking for 
> description=*something*.  Some of them may match, some of them may 
> not, and the server will return 1000 or fewer entries 
> (indeterminate).  The server is limited in the amount of work it 
> performs searching through the database.
>
> If you set sizelimit to be 1000, and set lookthroughlimit to be 
> unlimited, the server could look at all 5000 user entries, until it 
> finds 1000 entries which match, at which point it will terminate the 
> search and return the 1000 entries to the user.