Hi! I was spending some time today trying to make sure that I was getting the most bang for my buck today an my replica''s and I notices two items of interest that I was wondering if anyone else had input on! Firstly, after creating a number of indexs, my performance seems to be really good, the exception that I noticed was "finger" I noticed that finger takes a couple of seconds to return the data on RHDS whereas on OpenLDAP, it pops right now in real time! My first though was that I was doing an un-indexed search, but I can''t for the life of me figure out what I might not be indexing that I should be! The second thing I noticed was that on my servers, which are RHEL5, running 32bit OS''s with the PAE Kernels, RHDS doesn''t ever actually address more then 3 gig of ram! I was looking through the documentations, and it looks like by raising the "Maximum Cache Size" I''ll be able to allow RHDS to use more of the available memory.. did I get that right? Anyway, as always thanks in advance for all the help! This list has been a tremendous resource for an application that keeps on showing it''s value in huge ways! Best, Tim
Andrey Ivanov
2009-Jun-26 06:35 UTC
Re: [389-users] Finger slow and optimizing performance
Hi, There may be several attributes of interest to you as far as the memory consumption is concerned ( http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_File_Reference-Plug_in_Implemented_Server_Functionality_Reference-Database_Plug_in_Attributes.html) : nsslapd-dbcachesize nsslapd-cachememsize for every backend (by default, your data is in cn=userRoot,cn=ldbm database,cn=plugins,cn=config) nsslapd-import-cachesize (used only during ldif import) You can adjust the corresponding values by monitoring the attributes like currententrycachesize or entrycachehitratio of cn=monitor,cn=userRoot,cn=ldbm database,cn=plugins,cn=config ( http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_File_Reference-Plug_in_Implemented_Server_Functionality_Reference-Database_Plug_in_Attributes.html#Configuration_Command_File_Reference-Database_Plug_in_Attributes-Database_Attributes_under_cnmonitor_cnldbm_database_cnplugins_cnconfig ) 2009/6/26 Tim Hartmann <hartmann@fas.harvard.edu>> Hi! > > > I was spending some time today trying to make sure that I was getting the > most bang for my buck today an my replica''s and I notices two items of > interest that I was wondering if anyone else had input on! > > Firstly, after creating a number of indexs, my performance seems to be > really good, the exception that I noticed was "finger" I noticed that finger > takes a couple of seconds to return the data on RHDS whereas on OpenLDAP, it > pops right now in real time! My first though was that I was doing an > un-indexed search, but I can''t for the life of me figure out what I might > not be indexing that I should be! > > The second thing I noticed was that on my servers, which are RHEL5, running > 32bit OS''s with the PAE Kernels, RHDS doesn''t ever actually address more > then 3 gig of ram! I was looking through the documentations, and it looks > like by raising the "Maximum Cache Size" I''ll be able to allow RHDS to use > more of the available memory.. did I get that right? > > > Anyway, as always thanks in advance for all the help! This list has been a > tremendous resource for an application that keeps on showing it''s value in > huge ways! > > > Best, > > Tim > > -- > 389 users mailing list > 389-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >
Rich Megginson
2009-Jun-26 14:20 UTC
Re: [389-users] Finger slow and optimizing performance
Andrey Ivanov wrote:> Hi, > > > There may be several attributes of interest to you as far as the > memory consumption is concerned > (http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_File_Reference-Plug_in_Implemented_Server_Functionality_Reference-Database_Plug_in_Attributes.html) > : > nsslapd-dbcachesize > nsslapd-cachememsize for every backend (by default, your data is in > cn=userRoot,cn=ldbm database,cn=plugins,cn=config) > nsslapd-import-cachesize (used only during ldif import)Start with nsslapd-cachememsize - make that as large as possible and minimize nsslapd-dbcachesize> > You can adjust the corresponding values by monitoring the attributes > like currententrycachesize or entrycachehitratio of > cn=monitor,cn=userRoot,cn=ldbm database,cn=plugins,cn=config > (http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_File_Reference-Plug_in_Implemented_Server_Functionality_Reference-Database_Plug_in_Attributes.html#Configuration_Command_File_Reference-Database_Plug_in_Attributes-Database_Attributes_under_cnmonitor_cnldbm_database_cnplugins_cnconfig) >You can also use the logconv.pl script to examine the access log to see what types of searches are being done and which are not indexed properly.> > > 2009/6/26 Tim Hartmann <hartmann@fas.harvard.edu > <mailto:hartmann@fas.harvard.edu>> > > Hi! > > > I was spending some time today trying to make sure that I was > getting the most bang for my buck today an my replica''s and I > notices two items of interest that I was wondering if anyone else > had input on! > > Firstly, after creating a number of indexs, my performance seems > to be really good, the exception that I noticed was "finger" I > noticed that finger takes a couple of seconds to return the data > on RHDS whereas on OpenLDAP, it pops right now in real time! My > first though was that I was doing an un-indexed search, but I > can''t for the life of me figure out what I might not be indexing > that I should be! > > The second thing I noticed was that on my servers, which are > RHEL5, running 32bit OS''s with the PAE Kernels, RHDS doesn''t ever > actually address more then 3 gig of ram! I was looking through the > documentations, and it looks like by raising the "Maximum Cache > Size" I''ll be able to allow RHDS to use more of the available > memory.. did I get that right? > > > Anyway, as always thanks in advance for all the help! This list > has been a tremendous resource for an application that keeps on > showing it''s value in huge ways! > > > Best, > > Tim > > -- > 389 users mailing list > 389-users@redhat.com <mailto:389-users@redhat.com> > https://www.redhat.com/mailman/listinfo/fedora-directory-users > > > ------------------------------------------------------------------------ > > -- > 389 users mailing list > 389-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >
Rich Megginson wrote:> Andrey Ivanov wrote: >> Hi, >> >> >> There may be several attributes of interest to you as far as the >> memory consumption is concerned >> (http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_File_Reference-Plug_in_Implemented_Server_Functionality_Reference-Database_Plug_in_Attributes.html) >> : >> nsslapd-dbcachesize >> nsslapd-cachememsize for every backend (by default, your data is in >> cn=userRoot,cn=ldbm database,cn=plugins,cn=config) >> nsslapd-import-cachesize (used only during ldif import) > Start with nsslapd-cachememsize - make that as large as possible and > minimize nsslapd-dbcachesize >> >> You can adjust the corresponding values by monitoring the attributes >> like currententrycachesize or entrycachehitratio of >> cn=monitor,cn=userRoot,cn=ldbm database,cn=plugins,cn=config >> (http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_File_Reference-Plug_in_Implemented_Server_Functionality_Reference-Database_Plug_in_Attributes.html#Configuration_Command_File_Reference-Database_Plug_in_Attributes-Database_Attributes_under_cnmonitor_cnldbm_database_cnplugins_cnconfig) >> >> > You can also use the logconv.pl script to examine the access log to > see what types of searches are being done and which are not indexed > properly. >So after playing with logconv a bit, it looked like finger was making this call in the logs... [26/Jun/2009:10:59:36 -0400] conn=283289 op=-1 fd=80 closed - B1 [26/Jun/2009:10:59:36 -0400] conn=283289 op=2 RESULT err=11 tag=101 nentries=0 etime=1 notes=U [26/Jun/2009:10:59:35 -0400] conn=283289 op=2 SRCH base="ou=really,ou=long,o=name,dc=school,dc=edu" scope=2 filter="(objectClass=posixAccount)" attrs="uid userPassword uidNumber gidNumber cn homeDirectory loginShell gecos description objectClass" [26/Jun/2009:10:59:35 -0400] conn=283289 op=1 RESULT err=0 tag=101 nentries=1 etime=0 [26/Jun/2009:10:59:35 -0400] conn=283289 op=1 SRCH base="ou=systems,ou=services,o=hascs,dc=fas,dc=harvard,dc=edu" scope=2 filter="(&(objectClass=posixAccount)(uid=foo))" attrs="uid userPassword uidNumber gidNumber cn homeDirectory loginShell gecos description objectClass" [26/Jun/2009:10:59:35 -0400] conn=283289 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="" [26/Jun/2009:10:59:35 -0400] conn=283289 op=0 BIND dn="" method=128 version=3 [26/Jun/2009:10:59:35 -0400] conn=283289 fd=80 slot=80 connection from 1.2.3.4 to 4.3.2.1 But even after indexing "uid userPassword uidNumber gidNumber cn homeDirectory loginShell gecos description objectClass" finger still responds slower then it does on comparison to the older openLDAP servers... where we don''t do indexing on all of these attributes, AND still claims that I''m running an search that hasn''t been indexed! I''m I missing something glaringly obvious? Thanks! Tim
Rich Megginson
2009-Jun-26 20:14 UTC
Re: [389-users] Finger slow and optimizing performance
Tim Hartmann wrote:> Rich Megginson wrote: > >> Andrey Ivanov wrote: >> >>> Hi, >>> >>> >>> There may be several attributes of interest to you as far as the >>> memory consumption is concerned >>> (http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_File_Reference-Plug_in_Implemented_Server_Functionality_Reference-Database_Plug_in_Attributes.html) >>> : >>> nsslapd-dbcachesize >>> nsslapd-cachememsize for every backend (by default, your data is in >>> cn=userRoot,cn=ldbm database,cn=plugins,cn=config) >>> nsslapd-import-cachesize (used only during ldif import) >>> >> Start with nsslapd-cachememsize - make that as large as possible and >> minimize nsslapd-dbcachesize >> >>> You can adjust the corresponding values by monitoring the attributes >>> like currententrycachesize or entrycachehitratio of >>> cn=monitor,cn=userRoot,cn=ldbm database,cn=plugins,cn=config >>> (http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_File_Reference-Plug_in_Implemented_Server_Functionality_Reference-Database_Plug_in_Attributes.html#Configuration_Command_File_Reference-Database_Plug_in_Attributes-Database_Attributes_under_cnmonitor_cnldbm_database_cnplugins_cnconfig) >>> >>> >>> >> You can also use the logconv.pl script to examine the access log to >> see what types of searches are being done and which are not indexed >> properly. >> >> > > So after playing with logconv a bit, it looked like finger was making > this call in the logs... > > [26/Jun/2009:10:59:36 -0400] conn=283289 op=-1 fd=80 closed - B1 > [26/Jun/2009:10:59:36 -0400] conn=283289 op=2 RESULT err=11 tag=101 > nentries=0 etime=1 notes=U > [26/Jun/2009:10:59:35 -0400] conn=283289 op=2 SRCH > base="ou=really,ou=long,o=name,dc=school,dc=edu" scope=2 > filter="(objectClass=posixAccount)" attrs="uid userPassword uidNumber > gidNumber cn homeDirectory loginShell gecos description objectClass" > [26/Jun/2009:10:59:35 -0400] conn=283289 op=1 RESULT err=0 tag=101 > nentries=1 etime=0 > [26/Jun/2009:10:59:35 -0400] conn=283289 op=1 SRCH > base="ou=systems,ou=services,o=hascs,dc=fas,dc=harvard,dc=edu" scope=2 > filter="(&(objectClass=posixAccount)(uid=foo))" attrs="uid userPassword > uidNumber gidNumber cn homeDirectory loginShell gecos description > objectClass" > [26/Jun/2009:10:59:35 -0400] conn=283289 op=0 RESULT err=0 tag=97 > nentries=0 etime=0 dn="" > [26/Jun/2009:10:59:35 -0400] conn=283289 op=0 BIND dn="" method=128 > version=3 > [26/Jun/2009:10:59:35 -0400] conn=283289 fd=80 slot=80 connection from > 1.2.3.4 to 4.3.2.1 > > > > But even after indexing "uid userPassword uidNumber gidNumber cn > homeDirectory loginShell gecos description objectClass" finger still > responds slower then it does on comparison to the older openLDAP > servers... where we don''t do indexing on all of these attributes, AND > still claims that I''m running an search that hasn''t been indexed! I''m > I missing something glaringly obvious? >You only need to index the attributes used for searching: (&(objectClass=posixAccount)(uid=foo)) You need an equality index on objectClass (which should already be there, it is one of the default indexes) and an equality index on uid (again, should already be there). The problem is this: [26/Jun/2009:10:59:36 -0400] conn=283289 op=2 RESULT err=11 tag=101 nentries=0 etime=1 notes=U [26/Jun/2009:10:59:35 -0400] conn=283289 op=2 SRCH base="ou=really,ou=long,o=name,dc=school,dc=edu" scope=2 filter="(objectClass=posixAccount)" attrs="uid userPassword uidNumber gidNumber cn homeDirectory loginShell gecos description objectClass" The notes=U and err=11 mean that either the lookthrough limit has been exceeded, or you need to increase your nsslapd-idlistscanlimit: http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_File_Reference-Plug_in_Implemented_Server_Functionality_Reference-Database_Plug_in_Attributes.html#Configuration_Command_File_Reference-Database_Attributes_under_cnconfig_cnldbm_database_cnplugins_cnconfig-nsslapd_idlistscanlimit This is not a good search anyway - the client is basically asking for all entries that match objectClass=posixAccount which could be thousands or more - what does the client intend to do with all of those entries?> Thanks! > > Tim > > -- > 389 users mailing list > 389-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >
Rich Megginson wrote:> Tim Hartmann wrote: >> >>> >> >> So after playing with logconv a bit, it looked like finger was making >> this call in the logs... >> >> [26/Jun/2009:10:59:36 -0400] conn=283289 op=-1 fd=80 closed - B1 >> [26/Jun/2009:10:59:36 -0400] conn=283289 op=2 RESULT err=11 tag=101 >> nentries=0 etime=1 notes=U >> [26/Jun/2009:10:59:35 -0400] conn=283289 op=2 SRCH >> base="ou=really,ou=long,o=name,dc=school,dc=edu" scope=2 >> filter="(objectClass=posixAccount)" attrs="uid userPassword uidNumber >> gidNumber cn homeDirectory loginShell gecos description objectClass" >> [26/Jun/2009:10:59:35 -0400] conn=283289 op=1 RESULT err=0 tag=101 >> nentries=1 etime=0 >> [26/Jun/2009:10:59:35 -0400] conn=283289 op=1 SRCH >> base="ou=systems,ou=services,o=hascs,dc=fas,dc=harvard,dc=edu" scope=2 >> filter="(&(objectClass=posixAccount)(uid=foo))" attrs="uid userPassword >> uidNumber gidNumber cn homeDirectory loginShell gecos description >> objectClass" >> [26/Jun/2009:10:59:35 -0400] conn=283289 op=0 RESULT err=0 tag=97 >> nentries=0 etime=0 dn="" >> [26/Jun/2009:10:59:35 -0400] conn=283289 op=0 BIND dn="" method=128 >> version=3 >> [26/Jun/2009:10:59:35 -0400] conn=283289 fd=80 slot=80 connection from >> 1.2.3.4 to 4.3.2.1 >> >> >> >> But even after indexing "uid userPassword uidNumber gidNumber cn >> homeDirectory loginShell gecos description objectClass" finger still >> responds slower then it does on comparison to the older openLDAP >> servers... where we don''t do indexing on all of these attributes, AND >> still claims that I''m running an search that hasn''t been indexed! I''m >> I missing something glaringly obvious? > You only need to index the attributes used for searching: > (&(objectClass=posixAccount)(uid=foo)) > You need an equality index on objectClass (which should already be > there, it is one of the default indexes) and an equality index on uid > (again, should already be there). >Ok, thats cool, those are both being indexed correctly....> The problem is this: > [26/Jun/2009:10:59:36 -0400] conn=283289 op=2 RESULT err=11 tag=101 > nentries=0 etime=1 notes=U > [26/Jun/2009:10:59:35 -0400] conn=283289 op=2 SRCH > base="ou=really,ou=long,o=name,dc=school,dc=edu" scope=2 > filter="(objectClass=posixAccount)" attrs="uid userPassword uidNumber > gidNumber cn homeDirectory loginShell gecos description objectClass" > > The notes=U and err=11 mean that either the lookthrough limit has been > exceeded, or you need to increase your nsslapd-idlistscanlimit: > http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_File_Reference-Plug_in_Implemented_Server_Functionality_Reference-Database_Plug_in_Attributes.html#Configuration_Command_File_Reference-Database_Attributes_under_cnconfig_cnldbm_database_cnplugins_cnconfig-nsslapd_idlistscanlimit > > > This is not a good search anyway - the client is basically asking for > all entries that match objectClass=posixAccount which could be > thousands or more - what does the client intend to do with all of > those entries? >Thats apparently the search that "finger foo" on RHEL 5.2 generates! "finger foo" on Ubunutu 8.04 responds notibly faster, so I''m assuming that it''s generating a different search... hmmm
Nalin Dahyabhai
2009-Jun-26 20:59 UTC
Re: [389-users] Finger slow and optimizing performance
On Fri, Jun 26, 2009 at 02:14:06PM -0600, Rich Megginson wrote:> This is not a good search anyway - the client is basically asking for > all entries that match objectClass=posixAccount which could be thousands > or more - what does the client intend to do with all of those entries?The finger command also tries to match your query against the first or last name of a user, as stored in the GECOS field of user entries. On Fedora and derived systems (including RHEL), this part of its function can be turned off with the -m switch. HTH, Nalin