As I understand it, OpenLDAP doesn''t perform all that well under a high load. How does FDS perform in comparison to other LDAP implmentations like OpenLDAP and Sun? -- "Do the actors on Unsolved Mysteries ever get arrested because they look just like the criminal they are playing?" Christopher
Vampire D wrote:> As I understand it, OpenLDAP doesn''t perform all that well under a > high load.OpenLDAP 2.3 does.> How does FDS perform in comparison to other LDAP implmentations like > OpenLDAP and Sun?It depends. What performance characteristics do you require?> > -- > "Do the actors on Unsolved Mysteries ever get arrested because they > look just like the criminal they are playing?" > > Christopher > > ------------------------------------------------------------------------ > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >
We will not be using LDAP in the traditional sense. Instead of authentication, we will be using it to perform lookups upon incoming mail. We plan on having tens of thousands of email addresses stored in LDAP, every message that comes in is verified via LDAP that it is allowed, and then it is processed by our system. We plan on caching entries (positive and negative) for 24 hours, so as long as the look up has been done in the last 24 hours and the 1M record cache isn''t exhausted it will not perform a look up. This should cut down a lot of the demand. Initially we are looking at about 100k lookups an hour, as we expand the service that can go up by 50-100k at a time. On 7/13/07, Richard Megginson <rmeggins@redhat.com> wrote:> > Vampire D wrote: > > As I understand it, OpenLDAP doesn''t perform all that well under a > > high load. > OpenLDAP 2.3 does. > > How does FDS perform in comparison to other LDAP implmentations like > > OpenLDAP and Sun? > It depends. What performance characteristics do you require? > > > > -- > > "Do the actors on Unsolved Mysteries ever get arrested because they > > look just like the criminal they are playing?" > > > > Christopher > > > > ------------------------------------------------------------------------ > > > > -- > > Fedora-directory-users mailing list > > Fedora-directory-users@redhat.com > > https://www.redhat.com/mailman/listinfo/fedora-directory-users > > > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users > > >-- "Do the actors on Unsolved Mysteries ever get arrested because they look just like the criminal they are playing?" Christopher
Vampire D wrote:> We will not be using LDAP in the traditional sense. > > Instead of authentication, we will be using it to perform lookups upon > incoming mail. > We plan on having tens of thousands of email addresses stored in LDAP, > every message that comes in is verified via LDAP that it is allowed, > and then it is processed by our system. We plan on caching entries > (positive and negative) for 24 hours, so as long as the look up has > been done in the last 24 hours and the 1M record cache isn''t exhausted > it will not perform a look up. This should cut down a lot of the > demand. Initially we are looking at about 100k lookups an hour, as we > expand the service that can go up by 50-100k at a time.Fedora DS, Sun DS, and OpenLDAP should all be able to handle this load very well.> > > > On 7/13/07, *Richard Megginson* <rmeggins@redhat.com > <mailto:rmeggins@redhat.com>> wrote: > > Vampire D wrote: > > As I understand it, OpenLDAP doesn''t perform all that well under a > > high load. > OpenLDAP 2.3 does. > > How does FDS perform in comparison to other LDAP implmentations like > > OpenLDAP and Sun? > It depends. What performance characteristics do you require? > > > > -- > > "Do the actors on Unsolved Mysteries ever get arrested because they > > look just like the criminal they are playing?" > > > > Christopher > > > > > ------------------------------------------------------------------------ > > > > -- > > Fedora-directory-users mailing list > > Fedora-directory-users@redhat.com > <mailto:Fedora-directory-users@redhat.com> > > https://www.redhat.com/mailman/listinfo/fedora-directory-users > <https://www.redhat.com/mailman/listinfo/fedora-directory-users> > > > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > <mailto:Fedora-directory-users@redhat.com> > https://www.redhat.com/mailman/listinfo/fedora-directory-users > > > > > > -- > "Do the actors on Unsolved Mysteries ever get arrested because they > look just like the criminal they are playing?" > > Christopher > ------------------------------------------------------------------------ > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >
On Fri, 13 Jul 2007, Vampire D wrote:> We will not be using LDAP in the traditional sense. > > Instead of authentication, we will be using it to perform lookups upon > incoming mail. > We plan on having tens of thousands of email addresses stored in LDAP, every > message that comes in is verified via LDAP that it is allowed, and then it > is processed by our system. We plan on caching entries (positive and > negative) for 24 hours, so as long as the look up has been done in the last > 24 hours and the 1M record cache isn''t exhausted it will not perform a look > up. This should cut down a lot of the demand. Initially we are looking at > about 100k lookups an hour, as we expand the service that can go up by > 50-100k at a time.Lots of people (myself included) use LDAP for this, among other things. Fedora DS will not blink at 100K searches per hour. I have seen 50-80K ops/minute on our LDAP servers, which are HP DL145s with 2 cores and 4 Gb memory, without any performance degradation, and I''ve spoken with people doing far more than that on comparable hardware. At the rates you''re talking about, performance will be a non-issue. Chris St. Pierre Unix Systems Administrator Nebraska Wesleyan University ---------------------------- LOPSA Sysadmin Days: Professional Training for Professional SysAdmins August 6-7, Cherry Hill, NJ http://lopsa.org/SysadminDays
Vampire D wrote:> We will not be using LDAP in the traditional sense. > > Instead of authentication, we will be using it to perform lookups upon > incoming mail. > We plan on having tens of thousands of email addresses stored in LDAP, > every message that comes in is verified via LDAP that it is allowed, > and then it is processed by our system. We plan on caching entries > (positive and negative) for 24 hours, so as long as the look up has > been done in the last 24 hours and the 1M record cache isn''t exhausted > it will not perform a look up. This should cut down a lot of the > demand. Initially we are looking at about 100k lookups an hour, as we > expand the service that can go up by 50-100k at a time.It isn''t clear to me how often you''ll be adding, deleting or modifying the LDAP entries. Your search workload seems very low -- you should be able to achieve on the order of 10-50k searches/s on modern hardware with on the order of a million entries. However, like all databases that use a WAL, you will struggle to achieve a few hundred writes/s without very high end storage hardware (solid state disks for example). It may not be worthwhile caching search results inside your application because the LDAP server is caching too. You''d only save the network round trip overhead.
Would you recommend more than 2GB for 100k/hr? On 7/13/07, Chris St. Pierre <stpierre@nebrwesleyan.edu> wrote:> > On Fri, 13 Jul 2007, Vampire D wrote: > > > We will not be using LDAP in the traditional sense. > > > > Instead of authentication, we will be using it to perform lookups upon > > incoming mail. > > We plan on having tens of thousands of email addresses stored in LDAP, > every > > message that comes in is verified via LDAP that it is allowed, and then > it > > is processed by our system. We plan on caching entries (positive and > > negative) for 24 hours, so as long as the look up has been done in the > last > > 24 hours and the 1M record cache isn''t exhausted it will not perform a > look > > up. This should cut down a lot of the demand. Initially we are looking > at > > about 100k lookups an hour, as we expand the service that can go up by > > 50-100k at a time. > > Lots of people (myself included) use LDAP for this, among other > things. > > Fedora DS will not blink at 100K searches per hour. I have seen > 50-80K ops/minute on our LDAP servers, which are HP DL145s with 2 > cores and 4 Gb memory, without any performance degradation, and I''ve > spoken with people doing far more than that on comparable hardware. > > At the rates you''re talking about, performance will be a non-issue. > > Chris St. Pierre > Unix Systems Administrator > Nebraska Wesleyan University > ---------------------------- > LOPSA Sysadmin Days: Professional Training for Professional SysAdmins > August 6-7, Cherry Hill, NJ > http://lopsa.org/SysadminDays > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >-- "Do the actors on Unsolved Mysteries ever get arrested because they look just like the criminal they are playing?" Christopher
Vampire D wrote:> Would you recommend more than 2GB for 100k/hr?Memory usage is unrelated to search workload. You need more memory to achieve good performance with a larger set of active data. I''d recommend you do some load testing. There are plenty of good tools available that make testing fairly easy.
Can anyone recommend any tools that are available for testing LDAP? - dc On 7/13/07, David Boreham <david_list@boreham.org> wrote:> > Vampire D wrote: > > Would you recommend more than 2GB for 100k/hr? > Memory usage is unrelated to search workload. > You need more memory to achieve good performance > with a larger set of active data. > > I''d recommend you do some load testing. There are > plenty of good tools available that make testing fairly easy. > > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >
On 7/13/07, Vampire D <vampired@gmail.com> wrote:> As I understand it, OpenLDAP doesn''t perform all that well under a high > load. How does FDS perform in comparison to other LDAP implmentations like > OpenLDAP and Sun?Interesting. Where did you get the information that OpenLDAP does not perform under load? I was always under the impression that OpenLDAP was the fastest and most scalable LDAP server around. For example: http://www.symas.com/benchmark-auth.shtml I recall reading another benchmark somewhere comparing it with FDS but can''t find it at the moment. -- Norman Gaywood, Systems Administrator University of New England, Armidale, NSW 2351, Australia
I heard it from Cisco when working with them on a project as they claims it has a hard time keeping up under a heavy load. On 7/16/07, Norman Gaywood <ngaywood@une.edu.au> wrote:> > On 7/13/07, Vampire D <vampired@gmail.com> wrote: > > As I understand it, OpenLDAP doesn''t perform all that well under a high > > load. How does FDS perform in comparison to other LDAP implmentations > like > > OpenLDAP and Sun? > > Interesting. Where did you get the information that OpenLDAP does not > perform under load? I was always under the impression that OpenLDAP > was the fastest and most scalable LDAP server around. For example: > > http://www.symas.com/benchmark-auth.shtml > > I recall reading another benchmark somewhere comparing it with FDS but > can''t find it at the moment. > > -- > Norman Gaywood, Systems Administrator > University of New England, Armidale, > NSW 2351, Australia > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >-- "Do the actors on Unsolved Mysteries ever get arrested because they look just like the criminal they are playing?" Christopher
Chun Tat David Chu wrote:> Can anyone recommend any tools that are available for testing LDAP?SLAMD is popular, but I still like to use the simple command line tools like rsearch and its siblings : http://docs.sun.com/source/816-6400-10/rsearch.html http://docs.sun.com/source/816-5615-10/srchrate.htm http://docs.sun.com/source/816-5615-10/modrate.htm http://docs.sun.com/source/816-5615-10/authrate.htm http://docs.sun.com/source/816-5615-10/infadd.htm For me it''s easier to craft a workload matching my requirements using these tools.
Norman Gaywood wrote:> On 7/13/07, Vampire D <vampired@gmail.com> wrote: >> As I understand it, OpenLDAP doesn''t perform all that well under a high >> load. How does FDS perform in comparison to other LDAP >> implmentations like >> OpenLDAP and Sun? > > Interesting. Where did you get the information that OpenLDAP does not > perform under load? I was always under the impression that OpenLDAP > was the fastest and most scalable LDAP server around. For example: > > http://www.symas.com/benchmark-auth.shtml > > I recall reading another benchmark somewhere comparing it with FDS but > can''t find it at the moment.That looks to be a read-only test. What happens when you throw some updates at it? And are there any benchmarks for FDS running in multi-master mode with update activity? -- Les Mikesell lesmikesell@gmail.com
On Mon, Jul 16, 2007 at 10:20:23AM -0500, Les Mikesell wrote:> Norman Gaywood wrote: > >perform under load? I was always under the impression that OpenLDAP > >was the fastest and most scalable LDAP server around. For example: > > > >http://www.symas.com/benchmark-auth.shtml > > > >I recall reading another benchmark somewhere comparing it with FDS but > >can''t find it at the moment. > > That looks to be a read-only test. What happens when you throw some > updates at it? And are there any benchmarks for FDS running in > multi-master mode with update activity?Yes it was a read-only test. But then that''s the main application of LDAP servers. Are there applications that require high LDAP write performance? I found the other benchmark paper here: http://highlandsun.com/hyc/SambaXP.pdf It includes figures for FDS. A summary can be found here: http://www.mail-archive.com/ldap@umich.edu/msg01151.html According to that paper, OpenLDAP pretty much blows away everyone else in performance and scalability. Nothing else is even close. Of course it is a benchmark. I''m sure someone will find some flaws :-) -- Norman Gaywood, Systems Administrator School of Mathematics, Statistics and Computer Science University of New England, Armidale, NSW 2351, Australia norm@turing.une.edu.au Phone: +61 (0)2 6773 2412 http://turing.une.edu.au/~norm Fax: +61 (0)2 6773 3312 Please avoid sending me Word or PowerPoint attachments. See http://www.gnu.org/philosophy/no-word-attachments.html
Norman Gaywood wrote:> Yes it was a read-only test. But then that''s the main application of > LDAP servers. Are there applications that require high LDAP write > performance? >It''s pretty easy to achieve performance in excess of most applications'' requirements for reads, but write performance it typically much lower (due to the need to maintain the WAL with many indices, usually). Replication makes the situation worse because the replication changelog also has to be written, reducing the available I/O resources for primary database writes. So in any given real-world application, it''s often the write capacity that determines overall system capacity.
On Fri, 13 Jul 2007, Vampire D wrote:> Would you recommend more than 2GB for 100k/hr?If you can afford it, I''d recommend enough memory to keep your entire database resident in memory. That''s obviously not a function of queries per hour, but of number of entries, entry size, etc. Chris St. Pierre Unix Systems Administrator Nebraska Wesleyan University ---------------------------- LOPSA Sysadmin Days: Professional Training for Professional SysAdmins August 6-7, Cherry Hill, NJ http://lopsa.org/SysadminDays
You may want to look at SLAMD. Chun Tat David Chu <beyonddc.storage@gmail.com> wrote: Can anyone recommend any tools that are available for testing LDAP? - dc On 7/13/07, David Boreham <david_list@boreham.org > wrote:Vampire D wrote:> Would you recommend more than 2GB for 100k/hr?Memory usage is unrelated to search workload. You need more memory to achieve good performance with a larger set of active data. I''d recommend you do some load testing. There are plenty of good tools available that make testing fairly easy. -- Fedora-directory-users mailing list Fedora-directory-users@redhat.com https://www.redhat.com/mailman/listinfo/fedora-directory-users -- Fedora-directory-users mailing list Fedora-directory-users@redhat.com https://www.redhat.com/mailman/listinfo/fedora-directory-users
Hello, I receipt error when I execute start-slapd Enter PIN for Internal (Software) Token: [19/Jul/2007:12:21:41 +0200] - SSL alert: CERT_VerifyCertificateNow: verify certificate failed for cert Cert-DS-Server of family cn=RSA,cn=encryption,cn=config (Netscape Portable Runtime error -8182 - Peer''s certificate has an invalid signature.) [19/Jul/2007:12:21:41 +0200] - SSL failure: None of the cipher are valid I following this document: http://www.csse.uwa.edu.au/~ashley/fedora-ds/fedora-ds-26072006.html I have been one week to synchronize password de AD to FDS. In order to synchronize the users I did not have problems. exists any document to synchronize users and passwords step by step? :( bye. Esteban Torres Rodríguez ÁREA DE SOPORTE TÉCNICO - Administración de Servidores Subdirección de Sistemas Informáticos Empresa Pública Desarrollo Agrario y Pesquero, email: etorres@dap.es
Some pros with SLAMD include: 1. Detailed resource statistics are built-in 2. It came with a relatively large set of tests. 3. Addition of new tests are relatively easy with some Java skills Anh, David Boreham <david_list@boreham.org> wrote: Chun Tat David Chu wrote:> Can anyone recommend any tools that are available for testing LDAP?SLAMD is popular, but I still like to use the simple command line tools like rsearch and its siblings : http://docs.sun.com/source/816-6400-10/rsearch.html http://docs.sun.com/source/816-5615-10/srchrate.htm http://docs.sun.com/source/816-5615-10/modrate.htm http://docs.sun.com/source/816-5615-10/authrate.htm http://docs.sun.com/source/816-5615-10/infadd.htm For me it''s easier to craft a workload matching my requirements using these tools. -- Fedora-directory-users mailing list Fedora-directory-users@redhat.com https://www.redhat.com/mailman/listinfo/fedora-directory-users
Richard Megginson wrote:> Vampire D wrote: >> As I understand it, OpenLDAP doesn''t perform all that well under a >> high load. > OpenLDAP 2.3 does.Howard Chu''s "SambaXP" key notes certainly seem to make that argument. He makes the bold claim that "OpenLDAP is the only directory software that matters." Do you agree? Is there a future for Fedora DS, or will OpenLDAP own the Free Software directory service market? Will the two projects share technology and converge?
I was under the impression FDS was better implementation than OpenLDAP in terms of Performance, Reliability, and especially replication? On 7/21/07, Gordon Messmer <yinyang@eburg.com> wrote:> > Richard Megginson wrote: > > Vampire D wrote: > >> As I understand it, OpenLDAP doesn''t perform all that well under a > >> high load. > > OpenLDAP 2.3 does. > > Howard Chu''s "SambaXP" key notes certainly seem to make that argument. > He makes the bold claim that "OpenLDAP is the only directory software > that matters." > > Do you agree? Is there a future for Fedora DS, or will OpenLDAP own the > Free Software directory service market? Will the two projects share > technology and converge? > > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >-- "Do the actors on Unsolved Mysteries ever get arrested because they look just like the criminal they are playing?" Christopher
Another member on the list who does consulting with LDAP as well as someone who works for Cisco/IronPort. That''s why we have been looking at FDS. On 7/23/07, Pierangelo Masarati <ando@sys-net.it> wrote:> > Vampire D wrote: > > I was under the impression FDS was better implementation than OpenLDAP > > in terms of Performance, Reliability, and especially replication? > > It would be interesting to know what/where you got that impression from. > > p. > > > > Ing. Pierangelo Masarati > OpenLDAP Core Team > > SysNet s.r.l. > via Dossi, 8 - 27100 Pavia - ITALIA > http://www.sys-net.it > --------------------------------------- > Office: +39 02 23998309 > Mobile: +39 333 4963172 > Email: pierangelo.masarati@sys-net.it > --------------------------------------- > > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >-- "Do the actors on Unsolved Mysteries ever get arrested because they look just like the criminal they are playing?" Christopher
Vampire D wrote:> I was under the impression FDS was better implementation than OpenLDAP > in terms of Performance, Reliability, and especially replication?It would be interesting to know what/where you got that impression from. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it ---------------------------------------
Vampire D wrote:> Another member on the list who does consulting with LDAP as well as > someone who works for Cisco/IronPort. > That''s why we have been looking at FDS.If you Howard Chu is to be believed, Cisco is a lousy reference for LDAP (see message from 7/18). http://highlandsun.com/hyc/SambaXP.pdf Howard notes that Richard Megginson reviewed the configuration of the FDS server used in the benchmark. In my own experience, OpenLDAP leaked memory, was prone to db corruption, and was a lousy development platform (attributes requested by alias were returned by canonical name, which makes aliases useless in application development). Those were pretty serious problems, but I haven''t used OpenLDAP in several years. Symas *looks* like it''s pretty serious about making OpenLDAP a better platform. A lot can change in several years'' time. I''m comfortable with FDS. It''s more than fast enough for what I need, it''s stable, and applications developed on FDS are easily portable to Sun''s directory server. However, if OpenLDAP is developing into a compelling platform, then I''m more inclined to test my applications against it, and consider it for future deployments.
Gordon Messmer wrote:> Richard Megginson wrote: >> Vampire D wrote: >>> As I understand it, OpenLDAP doesn''t perform all that well under a >>> high load. >> OpenLDAP 2.3 does. > > Howard Chu''s "SambaXP" key notes certainly seem to make that > argument. He makes the bold claim that "OpenLDAP is the only > directory software that matters." > > Do you agree?No.> Is there a future for Fedora DS,Yes. Red Hat has invested, and continues to invest, a lot of resources in Fedora DS. Since December, 2004, the directory server team at Red Hat has spent the majority of effort on making everything about Fedora DS open source developer friendly: * replace the proprietary admin server with Apache * use FHS style paths * use autotools for building * including the software in the Fedora OS distribution This has represented an enormous amount of work, and we''re almost finished. At the same time, we''ve also managed to add some new features (password syntax checking, ldapi, distributed numeric assignment, bitwise matching rules, other features) as well as many bug fixes. We have a lot of ideas for the future after we complete this work. In general, we want to make LDAP easier to use, easier to deploy, and easier to fit in with other applications. We also want to find out what features you want. We really want to make this a community effort. We are not going anywhere - we are committed to continual improvement of Fedora DS.> or will OpenLDAP own the Free Software directory service market?I think there is room for both projects. Some people prefer OpenLDAP, and some prefer Fedora DS. Although it''s too early to tell, some may prefer Sun''s OpenDS or Apache DS.> Will the two projects share technology and converge?I don''t know, but this is something I would like to pursue, to find ways that we can share technology.> > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users