Jeff Tharp
2008-Jan-16 19:08 UTC
[Fedora-directory-users] RE: Contemplating an upgrade to Fedora DS 1.1
>Rich Megginson wrote:>Can you be more specific about your planned deployment? Number ofentries? Average entry size? Search rate? >Update rate? Number of masters? Total number of replicas? Number and type of clients? Sorry, I have my list subscription set to digest mode, but I saw that you sent this question, so I thought I''d respond ahead of actually receiving the email :-) Let''s see...number of entries = 529,809 (I was a little off in my estimate earlier, it''s been a while since I checked the exact count). Average entry size...not sure exactly how to calculate this, if you mean by memory size. We maintain 45 attributes per entry. My particular entry (which could be considered average, I suppose) is ~4 KB when exported to LDIF. I''ll run some performance monitoring today as far as the number of search and update operations. We only have two masters, no other replicas. One master is active and gets all the load, the other is on standby (via Heartbeat) ready to take over if the first fails. All requests come from two front-end application servers and one back-end administration web application. The authentication application always bind''s as the admin user then does a compare on the password hash of a particular user. In case you''re wondering, current production hardware is a pair of Dell PowerEdge 2850''s, each with 1x3.2 GHz CPU, 2 GB RAM, single RAID 10 array with 15K RPM disks. One other consideration I want to look into is enabling Berkeley DB transaction logging--last year we had a power outage that caused the servers to go down hard (our generator was on the fritz :-P). The database on both masters was corrupted and had to be restored. I do nightly backups but lost all the account creations/updates for the day of the outage. I''d like to setup transaction logs and write these off to tape or another server so that we can recover to within 30 min to 1 hour of an outage. I looked into this with our current setup, but I wasn''t able to get more than 1 transaction log written per day, which doesn''t help much beyond our current nightly backups. Thanks for your help, Jeff Tharp System Administrator ESRI - Redlands, CA http://www.esri.com> -----Original Message----- > From: Jeff Tharp > Sent: Tuesday, January 15, 2008 8:38 PM > To: ''Fedora-directory-users@redhat.com'' > Subject: Contemplating an upgrade to Fedora DS 1.1 > > I am looking into the feasibility of upgrading the LDAP > backend used for authentication on many of our web sites > (roughly 300K users). Currently we are using FedoraDS 1.0.2 > running on RHEL 4 in a multi-master configuration of two > nodes configured as a high-availability cluster using > Heartbeat from the Linux-HA project. My underlying database > is Berkeley DB 4.2.52. My goal would be to upgrade to > FedoraDS 1.1 running on RHEL 5.1. I have managed to complete > the initial installation on my test system and so I''m now > digging into the details of the migration. > > Some questions that have come up: > 1. RHEL5.1 ships with Berkeley DB 4.3 and I noticed a note > that this has been found subpar for production use in large > environments. Should I consider reverting back to Berkeley > DB 4.2.52 or should I look into installing Berkeley DB 4.5 or > 4.6? If I installed the FedoraDS 1.1 fc6 binary packages, do > I need to be worried that these were built against a specific > Berkeley DB version? > > 2. Most of the migration notes I see on the site mention > migrating from 1.0.4 to 1.1. Is it necessary to migrate our > current 1.0.2 install to 1.0.4 as an intermediate step to > upgrading to 1.1? Or should the 1.0.4 migration steps be sufficient? > > 3. Previously, we had separate physical filesystems for / and > /opt, so that the directory server files were separated from > the system files. I understand that in FedoraDS 1.1 the > decision has been to standardize the pathing so this is no > longer feasible. If I still wanted at least the > instance-specific files (or at least the instance-specific > database files) to be in a separate filesystem, say /data, > what would be the recommended way of accomplishing this? Or > should I just go crazy with symbolic links to accomplish the > structure I want? :-) > > I greatly appreciate any advice you can provide regarding > these questions. I must say that we originally deployed > FedoraDS 1.0.2 two years ago to replace a much older OpenLDAP > 2.0 implementation and have generally been happy with both > its performance and stability. > > Thanks, > Jeff Tharp > System Administrator > ESRI - Redlands, CA > http://www.esri.com
Rich Megginson
2008-Jan-16 22:44 UTC
Re: [Fedora-directory-users] RE: Contemplating an upgrade to Fedora DS 1.1
Jeff Tharp wrote:>> Rich Megginson wrote: >> > > >> Can you be more specific about your planned deployment? Number of >> > entries? Average entry size? Search rate? >Update rate? Number of > masters? Total number of replicas? Number and type of clients? > > Sorry, I have my list subscription set to digest mode, but I saw that > you sent this question, so I thought I''d respond ahead of actually > receiving the email :-) Let''s see...number of entries = 529,809 (I was > a little off in my estimate earlier, it''s been a while since I checked > the exact count). Average entry size...not sure exactly how to > calculate this, if you mean by memory size. We maintain 45 attributes > per entry. My particular entry (which could be considered average, I > suppose) is ~4 KB when exported to LDIF. >We have performed several stress tests with several million entries each on RHEL5.1 using the standard bdb 4.3 that is included with RHEL5.1. There was no data loss, corruption, or other such problems.> I''ll run some performance monitoring today as far as the number of > search and update operations. We only have two masters, no other > replicas. One master is active and gets all the load, the other is on > standby (via Heartbeat) ready to take over if the first fails. All > requests come from two front-end application servers and one back-end > administration web application. The authentication application always > bind''s as the admin user then does a compare on the password hash of a > particular user. > > In case you''re wondering, current production hardware is a pair of Dell > PowerEdge 2850''s, each with 1x3.2 GHz CPU, 2 GB RAM, single RAID 10 > array with 15K RPM disks. > > One other consideration I want to look into is enabling Berkeley DB > transaction logging--last year we had a power outage that caused the > servers to go down hard (our generator was on the fritz :-P). The > database on both masters was corrupted and had to be restored. I do > nightly backups but lost all the account creations/updates for the day > of the outage. I''d like to setup transaction logs and write these off > to tape or another server so that we can recover to within 30 min to 1 > hour of an outage. I looked into this with our current setup, but I > wasn''t able to get more than 1 transaction log written per day, which > doesn''t help much beyond our current nightly backups. >These settings are available, but not exposed in the console. See http://www.redhat.com/docs/manuals/dir-server/cli/8.0/Configuration_Command_File_Reference-Plug_in_Implemented_Server_Functionality_Reference-Database_Plug_in_Attributes.html for more information.> Thanks for your help, > Jeff Tharp > System Administrator > ESRI - Redlands, CA > http://www.esri.com > > > >> -----Original Message----- >> From: Jeff Tharp >> Sent: Tuesday, January 15, 2008 8:38 PM >> To: ''Fedora-directory-users@redhat.com'' >> Subject: Contemplating an upgrade to Fedora DS 1.1 >> >> I am looking into the feasibility of upgrading the LDAP >> backend used for authentication on many of our web sites >> (roughly 300K users). Currently we are using FedoraDS 1.0.2 >> running on RHEL 4 in a multi-master configuration of two >> nodes configured as a high-availability cluster using >> Heartbeat from the Linux-HA project. My underlying database >> is Berkeley DB 4.2.52. My goal would be to upgrade to >> FedoraDS 1.1 running on RHEL 5.1. I have managed to complete >> the initial installation on my test system and so I''m now >> digging into the details of the migration. >> >> Some questions that have come up: >> 1. RHEL5.1 ships with Berkeley DB 4.3 and I noticed a note >> that this has been found subpar for production use in large >> environments. Should I consider reverting back to Berkeley >> DB 4.2.52 or should I look into installing Berkeley DB 4.5 or >> 4.6? If I installed the FedoraDS 1.1 fc6 binary packages, do >> I need to be worried that these were built against a specific >> Berkeley DB version? >> >> 2. Most of the migration notes I see on the site mention >> migrating from 1.0.4 to 1.1. Is it necessary to migrate our >> current 1.0.2 install to 1.0.4 as an intermediate step to >> upgrading to 1.1? Or should the 1.0.4 migration steps be sufficient? >> >> 3. Previously, we had separate physical filesystems for / and >> /opt, so that the directory server files were separated from >> the system files. I understand that in FedoraDS 1.1 the >> decision has been to standardize the pathing so this is no >> longer feasible. If I still wanted at least the >> instance-specific files (or at least the instance-specific >> database files) to be in a separate filesystem, say /data, >> what would be the recommended way of accomplishing this? Or >> should I just go crazy with symbolic links to accomplish the >> structure I want? :-) >> >> I greatly appreciate any advice you can provide regarding >> these questions. I must say that we originally deployed >> FedoraDS 1.0.2 two years ago to replace a much older OpenLDAP >> 2.0 implementation and have generally been happy with both >> its performance and stability. >> >> Thanks, >> Jeff Tharp >> System Administrator >> ESRI - Redlands, CA >> http://www.esri.com >> > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >