I recently did an ldif backup of our iplanet 52 database. Its about an 88 MB ldif file. I took this to a new FDS server Dell 850 3 ghz duel core 2 sata hard disks. I ran an ldapadd the data imported perfectly. Then I tried to cutover some systems and give the database some load. System went 200% processor Eventually I realized I was missing indexes so I added them through the graphical tool. The log seemed to do something like this generating index 1% generating index 2% .... generating index 49% Done Seemed weird that they would jump from 49% to Done At this point the new system was running at 100% processor But the queries are running faster on our old 440 MHZ sparc t1 server52 database I ran DB ERROR: db_verify: Page 30: out-of-order key at entry 498 DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4: DB_VERIFY_BAD: Database verification failed then I tried db2_index. The program seemed to be in a tight loop complaining about 1 missing entry. I do not realize how the data can be so corrupted right after an import. These are someone generic symptoms. Any ideas? Thanks
Richard Megginson
2006-Dec-15 23:52 UTC
Re: [Fedora-directory-users] Migration from i-planet 52
Eddie C wrote:> I recently did an ldif backup of our iplanet 52 database. Its about an > 88 MB ldif file. > I took this to a new FDS server Dell 850 3 ghz duel core 2 sata hard > disks. > I ran an ldapadd the data imported perfectly. > Then I tried to cutover some systems and give the database some load. > > System went 200% processor > > Eventually I realized I was missing indexes so I added them through > the graphical tool. > > The log seemed to do something like this > generating index 1% > generating index 2% > .... > generating index 49% > Done > Seemed weird that they would jump from 49% to Done > At this point the new system was running at 100% processor > But the queries are running faster on our old 440 MHZ sparc t1 > server52 database > > I ran > DB ERROR: db_verify: Page 30: out-of-order key at entry 498 > DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4: > DB_VERIFY_BAD: Database verification failed > > then I tried db2_index. The program seemed to be in a tight loop > complaining about 1 missing entry. > > I do not realize how the data can be so corrupted right after an import. > > These are someone generic symptoms. Any ideas? ThanksTry creating all of the required indexes first, then doing the import of your original LDIF. Not only will the import+index creation be much faster (than doing the import then creating the indexes one at a time), but I think your database corruption problems will vanish.> ------------------------------------------------------------------------ > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >
Noriko Hosoi
2006-Dec-17 00:39 UTC
Re: [Fedora-directory-users] Migration from i-planet 52
Richard Megginson wrote:> Eddie C wrote: > >> I recently did an ldif backup of our iplanet 52 database. Its about >> an 88 MB ldif file. >> I took this to a new FDS server Dell 850 3 ghz duel core 2 sata hard >> disks. >> I ran an ldapadd the data imported perfectly. >Are there any reason to use ldapadd instead of ldif2db? ldif2db should be much faster...>> Then I tried to cutover some systems and give the database some load. >> >> System went 200% processor >> >> Eventually I realized I was missing indexes so I added them through >> the graphical tool. >> >> The log seemed to do something like this >> generating index 1% >> generating index 2% >> .... >> generating index 49% >> Done >> Seemed weird that they would jump from 49% to Done >> At this point the new system was running at 100% processor >> But the queries are running faster on our old 440 MHZ sparc t1 >> server52 database >> >> I ran >> DB ERROR: db_verify: Page 30: out-of-order key at entry 498 >> DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4: >> DB_VERIFY_BAD: Database verification failed >> >> then I tried db2_index. The program seemed to be in a tight loop >> complaining about 1 missing entry. >> >> I do not realize how the data can be so corrupted right after an import. >> >> These are someone generic symptoms. Any ideas? Thanks > > Try creating all of the required indexes first, then doing the import > of your original LDIF. Not only will the import+index creation be > much faster (than doing the import then creating the indexes one at a > time), but I think your database corruption problems will vanish. > >> ------------------------------------------------------------------------ >> >> -- >> Fedora-directory-users mailing list >> Fedora-directory-users@redhat.com >> https://www.redhat.com/mailman/listinfo/fedora-directory-users >> > >------------------------------------------------------------------------ > >-- >Fedora-directory-users mailing list >Fedora-directory-users@redhat.com >https://www.redhat.com/mailman/listinfo/fedora-directory-users > >
The document I had suggested using ldapsearch and ldapadd to migrate data. If lidf2db commands are faster/better I will use them.>> Try creating all of the required indexes first, then doing the import of >> your original LDIF.I am willing to try this, but It is scary to me. I would have rather you said I must be doing something wrong...because.... Our LDAP database has been in production for 6 years. We add indexes to our i-planet on average twice a year due to new software or new features. Your advice is almost suggesting that adding new indexes can corrupt the database. I will try again from scratch using everyones advice of course. Thank you, Edward On 12/16/06, Noriko Hosoi <nhosoi@redhat.com> wrote:> > Richard Megginson wrote: > > > Eddie C wrote: > > > >> I recently did an ldif backup of our iplanet 52 database. Its about > >> an 88 MB ldif file. > >> I took this to a new FDS server Dell 850 3 ghz duel core 2 sata hard > >> disks. > >> I ran an ldapadd the data imported perfectly. > > > Are there any reason to use ldapadd instead of ldif2db? ldif2db should > be much faster... > > >> Then I tried to cutover some systems and give the database some load. > >> > >> System went 200% processor > >> > >> Eventually I realized I was missing indexes so I added them through > >> the graphical tool. > >> > >> The log seemed to do something like this > >> generating index 1% > >> generating index 2% > >> .... > >> generating index 49% > >> Done > >> Seemed weird that they would jump from 49% to Done > >> At this point the new system was running at 100% processor > >> But the queries are running faster on our old 440 MHZ sparc t1 > >> server52 database > >> > >> I ran > >> DB ERROR: db_verify: Page 30: out-of-order key at entry 498 > >> DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4: > >> DB_VERIFY_BAD: Database verification failed > >> > >> then I tried db2_index. The program seemed to be in a tight loop > >> complaining about 1 missing entry. > >> > >> I do not realize how the data can be so corrupted right after an > import. > >> > >> These are someone generic symptoms. Any ideas? Thanks > > > > Try creating all of the required indexes first, then doing the import > > of your original LDIF. Not only will the import+index creation be > > much faster (than doing the import then creating the indexes one at a > > time), but I think your database corruption problems will vanish. > > > >> > ------------------------------------------------------------------------ > >> > >> -- > >> Fedora-directory-users mailing list > >> Fedora-directory-users@redhat.com > >> https://www.redhat.com/mailman/listinfo/fedora-directory-users > >> > > > >------------------------------------------------------------------------ > > > >-- > >Fedora-directory-users mailing list > >Fedora-directory-users@redhat.com > >https://www.redhat.com/mailman/listinfo/fedora-directory-users > > > > > > > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users > > > >
Richard Megginson
2006-Dec-18 15:06 UTC
Re: [Fedora-directory-users] Migration from i-planet 52
Eddie C wrote:> The document I had suggested using ldapsearch and ldapadd to migrate > data. If lidf2db commands are faster/better I will use them. > > >> Try creating all of the required indexes first, then doing the > import of > >> your original LDIF. > > I am willing to try this, but It is scary to me. I would have rather > you said I must be doing something wrong...because.... > > Our LDAP database has been in production for 6 years. We add indexes > to our i-planet on average twice a year due to new software or new > features. Your advice is almost suggesting that adding new indexes can > corrupt the database.No, not exactly. I''m not really sure what went wrong. Feel free to try it again. It''s just that for the initial data import, it''s much faster to configure the indexes first, then use ldif2db to import the data and create the indexes at the same time.> > I will try again from scratch using everyones advice of course. > > Thank you, > Edward > > > > > > On 12/16/06, *Noriko Hosoi* < nhosoi@redhat.com > <mailto:nhosoi@redhat.com>> wrote: > > Richard Megginson wrote: > > > Eddie C wrote: > > > >> I recently did an ldif backup of our iplanet 52 database. Its about > >> an 88 MB ldif file. > >> I took this to a new FDS server Dell 850 3 ghz duel core 2 sata > hard > >> disks. > >> I ran an ldapadd the data imported perfectly. > > > Are there any reason to use ldapadd instead of ldif2db? ldif2db > should > be much faster... > > >> Then I tried to cutover some systems and give the database some > load. > >> > >> System went 200% processor > >> > >> Eventually I realized I was missing indexes so I added them through > >> the graphical tool. > >> > >> The log seemed to do something like this > >> generating index 1% > >> generating index 2% > >> .... > >> generating index 49% > >> Done > >> Seemed weird that they would jump from 49% to Done > >> At this point the new system was running at 100% processor > >> But the queries are running faster on our old 440 MHZ sparc t1 > >> server52 database > >> > >> I ran > >> DB ERROR: db_verify: Page 30: out-of-order key at entry 498 > >> DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4: > >> DB_VERIFY_BAD: Database verification failed > >> > >> then I tried db2_index. The program seemed to be in a tight loop > >> complaining about 1 missing entry. > >> > >> I do not realize how the data can be so corrupted right after > an import. > >> > >> These are someone generic symptoms. Any ideas? Thanks > > > > Try creating all of the required indexes first, then doing the > import > > of your original LDIF. Not only will the import+index creation be > > much faster (than doing the import then creating the indexes one > at a > > time), but I think your database corruption problems will vanish. > > > >> > ------------------------------------------------------------------------ > > >> > >> -- > >> Fedora-directory-users mailing list > >> Fedora-directory-users@redhat.com > <mailto:Fedora-directory-users@redhat.com> > >> https://www.redhat.com/mailman/listinfo/fedora-directory-users > >> > > > >------------------------------------------------------------------------ > > > >-- > >Fedora-directory-users mailing list > >Fedora-directory-users@redhat.com > <mailto:Fedora-directory-users@redhat.com> > >https://www.redhat.com/mailman/listinfo/fedora-directory-users > <https://www.redhat.com/mailman/listinfo/fedora-directory-users> > > > > > > > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > <mailto:Fedora-directory-users@redhat.com> > https://www.redhat.com/mailman/listinfo/fedora-directory-users > > > > > ------------------------------------------------------------------------ > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >
David Boreham
2006-Dec-18 15:38 UTC
Re: [Fedora-directory-users] Migration from i-planet 52
Eddie C wrote:> I ran > DB ERROR: db_verify: Page 30: out-of-order key at entry 498 > DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4: > DB_VERIFY_BAD: Database verification failedI''m assuming that you are running the correct version of db_verify (it should perform a version check on the magic number in the region files, so I think it has to be the right one). My best guess here is that the process by which a new file is added to a running DB environment went wrong somehow. The details have changed a few times over the years and it is just possible that the current FDS code is not up to date. Potentially the ''problem'' would be fixed by simply stopping and re-starting the server (because the environment would see all the files closed and then re-opened). I do know that the process for adding a new file to the environment is correctly followed in the newer code that deals with individual back-end restore from archive. Perhaps the online index code is not using the same underlying function. I haven''t taken the time to examine the code though. The problem I''m thinking about arises when a file that was created using one db environment (a temporary one, such as is used when building an index), is opened within another environment, ''bad stuff'' can happen along the lines of what you are seeing. There is state in the file that references the old environment, which is now stale vs. the new one. db_verify sees that and barfs. It is possible to avoid this happening but one has to be careful to make the right calls in the correct order (the details of which I forget now). As Rich and Noriko said, if you just re-import after creating the indices this issue (if it is indeed the one I''m thinking of) will be avoided because all the files are created at the same time under the same db environment.
All, I tested this from scratch. I used the ldif_2db function which did work much faster! 112 seconds...rather then about 20 minutes. However I think the verify_db and db2_index functions are not in agreement. Create indexes. After my initial import. [root@ldap3 slapd-ldap3]# more out.after.final.txt ***************************************************************** verify-db: This tool should only be run if recovery start fails and the server is down. If you run this tool while the server is running, you may get false reports of corrupted files or other false errors. ***************************************************************** Verify log files in db ... Good Verify db/o_idsk_com/id2entry.db4 ... Good Verify db/userRoot/ancestorid.db4 ... Good Verify db/userRoot/entrydn.db4 ... Good Verify db/userRoot/cn.db4 ... Good Verify db/userRoot/numsubordinates.db4 ... Good Verify db/userRoot/aci.db4 ... Good Verify db/userRoot/parentid.db4 ... Good Verify db/userRoot/objectclass.db4 ... Good Verify db/userRoot/id2entry.db4 ... Good Verify db/userRoot/nsUniqueId.db4 ... Good Verify db/idsk_services/ancestorid.db4 ... DB ERROR: db_verify: Page 4: out-of-order key at entry 252 DB ERROR: db_verify: Page 7: out-of-order key at entry 194 DB ERROR: db_verify: Page 7: out-of-order key at entry 450 DB ERROR: db_verify: Page 11: out-of-order key at entry 69 DB ERROR: db_verify: Page 11: out-of-order key at entry 325 DB ERROR: db_verify: Page 11: out-of-order key at entry 581 DB ERROR: db_verify: Page 12: out-of-order key at entry 22 DB ERROR: db_verify: Page 16: out-of-order key at entry 249 DB ERROR: db_verify: Page 16: out-of-order key at entry 498 DB ERROR: db_verify: Page 16: out-of-order key at entry 754 DB ERROR: db_verify: Page 17: out-of-order key at entry 195 DB ERROR: db_verify: Page 17: out-of-order key at entry 451 DB ERROR: db_verify: Page 17: out-of-order key at entry 707 DB ERROR: db_verify: Page 18: out-of-order key at entry 148 DB ERROR: db_verify: Page 21: out-of-order key at entry 254 DB ERROR: db_verify: Page 21: out-of-order key at entry 510 DB ERROR: db_verify: Page 21: out-of-order key at entry 766 DB ERROR: db_verify: Page 22: out-of-order key at entry 207 DB ERROR: db_verify: Page 22: out-of-order key at entry 463 DB ERROR: db_verify: Page 22: out-of-order key at entry 719 DB ERROR: db_verify: Page 23: out-of-order key at entry 160 DB ERROR: db_verify: DB->verify: db/idsk_services/ancestorid.db4: DB_VERIFY_BAD: Database verification failed Secondary index file ancestorid.db4 in db/idsk_services is corrupted. Please run db2index(.pl) for reindexing. So then i reran db2index....verify_db again...same result. Edward On 12/15/06, Eddie C <edlinuxguru@gmail.com> wrote:> > I recently did an ldif backup of our iplanet 52 database. Its about an 88 > MB ldif file. > I took this to a new FDS server Dell 850 3 ghz duel core 2 sata hard > disks. > I ran an ldapadd the data imported perfectly. > Then I tried to cutover some systems and give the database some load. > > System went 200% processor > > Eventually I realized I was missing indexes so I added them through the > graphical tool. > > The log seemed to do something like this > generating index 1% > generating index 2% > .... > generating index 49% > Done > Seemed weird that they would jump from 49% to Done > At this point the new system was running at 100% processor > But the queries are running faster on our old 440 MHZ sparc t1 server52 > database > > I ran > DB ERROR: db_verify: Page 30: out-of-order key at entry 498 > DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4: > DB_VERIFY_BAD: Database verification failed > > then I tried db2_index. The program seemed to be in a tight loop > complaining about 1 missing entry. > > I do not realize how the data can be so corrupted right after an import. > > These are someone generic symptoms. Any ideas? Thanks >
Noriko Hosoi
2006-Dec-18 22:35 UTC
Re: [Fedora-directory-users] Re: Migration from i-planet 52
Eddie C wrote:> All, > > I tested this from scratch. > I used the ldif_2db function which did work much faster! 112 > seconds...rather then about 20 minutes. > However I think the verify_db and db2_index functions are not in > agreement. > > Create indexes. > After my initial import. > [root@ldap3 slapd-ldap3]# more out.after.final.txt > ***************************************************************** > verify-db: This tool should only be run if recovery start fails > and the server is down. If you run this tool while the server is > running, you may get false reports of corrupted files or other > false errors. > ***************************************************************** > Verify log files in db ... Good > Verify db/o_idsk_com/id2entry.db4 ... Good > Verify db/userRoot/ancestorid.db4 ... Good > Verify db/userRoot/entrydn.db4 ... Good > Verify db/userRoot/cn.db4 ... Good > Verify db/userRoot/numsubordinates.db4 ... Good > Verify db/userRoot/aci.db4 ... Good > Verify db/userRoot/parentid.db4 ... Good > Verify db/userRoot/objectclass.db4 ... Good > Verify db/userRoot/id2entry.db4 ... Good > Verify db/userRoot/nsUniqueId.db4 ... Good > Verify db/idsk_services/ancestorid.db4 ... > DB ERROR: db_verify: Page 4: out-of-order key at entry 252 > DB ERROR: db_verify: Page 7: out-of-order key at entry 194 > DB ERROR: db_verify: Page 7: out-of-order key at entry 450 > DB ERROR: db_verify: Page 11: out-of-order key at entry 69 > DB ERROR: db_verify: Page 11: out-of-order key at entry 325 > DB ERROR: db_verify: Page 11: out-of-order key at entry 581 > DB ERROR: db_verify: Page 12: out-of-order key at entry 22 > DB ERROR: db_verify: Page 16: out-of-order key at entry 249 > DB ERROR: db_verify: Page 16: out-of-order key at entry 498 > DB ERROR: db_verify: Page 16: out-of-order key at entry 754 > DB ERROR: db_verify: Page 17: out-of-order key at entry 195 > DB ERROR: db_verify: Page 17: out-of-order key at entry 451 > DB ERROR: db_verify: Page 17: out-of-order key at entry 707 > DB ERROR: db_verify: Page 18: out-of-order key at entry 148 > DB ERROR: db_verify: Page 21: out-of-order key at entry 254 > DB ERROR: db_verify: Page 21: out-of-order key at entry 510 > DB ERROR: db_verify: Page 21: out-of-order key at entry 766 > DB ERROR: db_verify: Page 22: out-of-order key at entry 207 > DB ERROR: db_verify: Page 22: out-of-order key at entry 463 > DB ERROR: db_verify: Page 22: out-of-order key at entry 719 > DB ERROR: db_verify: Page 23: out-of-order key at entry 160 > DB ERROR: db_verify: DB->verify: db/idsk_services/ancestorid.db4: > DB_VERIFY_BAD: Database verification failed > Secondary index file ancestorid.db4 in db/idsk_services is corrupted. > Please run db2index(.pl) for reindexing. >Is this after you imported your ldif file to the backend ''idsk_services''? When you imported the ldif file, did you get any errors from ldif2db? This is an sample output when we import Example.ldif. Did you see any messages other than these? [18/Dec/2006:04:00:48 -0800] - dblayer_instance_start: pagesize: 4096, pages: 1009265, procpages: 23476 [18/Dec/2006:04:00:48 -0800] - cache autosizing: import cache: 204800k [18/Dec/2006:04:00:48 -0800] - li_import_cache_autosize: 50, import_pages: 51200, pagesize: 4096 [18/Dec/2006:04:00:48 -0800] - WARNING: Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database [18/Dec/2006:04:00:48 -0800] - dblayer_instance_start: pagesize: 4096, pages: 1009265, procpages: 23476 [18/Dec/2006:04:00:48 -0800] - cache autosizing: import cache: 204800k [18/Dec/2006:04:00:48 -0800] - li_import_cache_autosize: 50, import_pages: 51200, pagesize: 4096 [18/Dec/2006:04:00:49 -0800] - import example: Beginning import job... [18/Dec/2006:04:00:49 -0800] - import example: Index buffering enabled with bucket size 100 [18/Dec/2006:04:00:49 -0800] - import example: Processing file "/export/DS7.2-24471/server/usr/share/fedora-ds/data/Example.ldif" [18/Dec/2006:04:00:49 -0800] - import example: Finished scanning file "/export/DS7.2-24471/server/usr/share/fedora-ds/data/Example.ldif" (160 entries) [18/Dec/2006:04:00:50 -0800] - import example: Workers finished; cleaning up... [18/Dec/2006:04:00:50 -0800] - import example: Workers cleaned up. [18/Dec/2006:04:00:50 -0800] - import example: Cleaning up producer thread... [18/Dec/2006:04:00:50 -0800] - import example: Indexing complete. Post-processing... [18/Dec/2006:04:00:50 -0800] - import example: Flushing caches... [18/Dec/2006:04:00:50 -0800] - import example: Closing files... [18/Dec/2006:04:00:51 -0800] - All database threads now stopped [18/Dec/2006:04:00:51 -0800] - import example: Import complete. Processed 160 entries in 2 seconds. (80.00 entries/sec) Thanks, --noriko> So then i reran db2index....verify_db again...same result. > > Edward > > > On 12/15/06, *Eddie C* <edlinuxguru@gmail.com > <mailto:edlinuxguru@gmail.com>> wrote: > > I recently did an ldif backup of our iplanet 52 database. Its > about an 88 MB ldif file. > I took this to a new FDS server Dell 850 3 ghz duel core 2 sata > hard disks. > I ran an ldapadd the data imported perfectly. > Then I tried to cutover some systems and give the database some load. > > System went 200% processor > > Eventually I realized I was missing indexes so I added them > through the graphical tool. > > The log seemed to do something like this > generating index 1% > generating index 2% > .... > generating index 49% > Done > Seemed weird that they would jump from 49% to Done > At this point the new system was running at 100% processor > But the queries are running faster on our old 440 MHZ sparc t1 > server52 database > > I ran > DB ERROR: db_verify: Page 30: out-of-order key at entry 498 > DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4: > DB_VERIFY_BAD: Database verification failed > > then I tried db2_index. The program seemed to be in a tight loop > complaining about 1 missing entry. > > I do not realize how the data can be so corrupted right after an > import. > > These are someone generic symptoms. Any ideas? Thanks > > > ------------------------------------------------------------------------ > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >
>>Any errors?No as far as I can tell the import goes smooth. Our upstream applications seem to have no problem with the data it seems to be all imported. I am really troubleshooting and index and performance issues, I figure out of order keys could be slowing searches down. This a fairly large db 160000 entires) some objects have upwards of 20 attributes. db_verify and db_2index both run fairly quickly. It just seems like they cant both agree on what clean data looks like. FWI I upgraded to the latest 1.0.4 before all this testing. [18/Dec/2006:15:29:50 -0500] - WARNING: Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database [18/Dec/2006:15:29:50 -0500] - import idsk_services: Beginning import job... [18/Dec/2006:15:29:50 -0500] - import idsk_services: Index buffering enabled with bucket size 15 [18/Dec/2006:15:29:50 -0500] - import idsk_services: Processing file "/opt/idsk/downloads/idsk.services.com.ldif" [18/Dec/2006:15:29:50 -0500] - import idsk_services: Finished scanning file "/opt/idsk/downloads/idsk.services.com.ldif" (2037 entries) [18/Dec/2006:15:29:51 -0500] - import idsk_services: Workers finished; cleaning up... [18/Dec/2006:15:29:51 -0500] - import idsk_services: Workers cleaned up. [18/Dec/2006:15:29:51 -0500] - import idsk_services: Cleaning up producer thread... [18/Dec/2006:15:29:51 -0500] - import idsk_services: Indexing complete. Post-processing... [18/Dec/2006:15:29:51 -0500] - import idsk_services: Flushing caches... [18/Dec/2006:15:29:51 -0500] - import idsk_services: Closing files... [18/Dec/2006:15:29:51 -0500] - import idsk_services: Import complete. Processed 2037 entries in 1 seconds. (2037.00 entries/sec) [18/Dec/2006:15:30:28 -0500] - Bringing idsk_data offline... [18/Dec/2006:15:30:28 -0500] - WARNING: Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database [18/Dec/2006:15:30:28 -0500] - import idsk_data: Beginning import job... [18/Dec/2006:15:30:28 -0500] - import idsk_data: Index buffering enabled with bucket size 15 [18/Dec/2006:15:30:28 -0500] - import idsk_data: Processing file "/opt/idsk/downloads/idsk.com.ldif" [18/Dec/2006:15:30:48 -0500] - import idsk_data: Processed 77288 entries -- average rate 3680.4/sec, recent rate 3680.3/sec, hit ratio 0% [18/Dec/2006:15:31:13 -0500] - import idsk_data: Processed 141956 entries -- average rate 3086.0/sec, recent rate 3086.0/sec, hit ratio 100% [18/Dec/2006:15:31:35 -0500] - import idsk_data: Finished scanning file "/opt/idsk/downloads/idsk.com.ldif" (163684 entries) [18/Dec/2006:15:31:49 -0500] - import idsk_data: Workers finished; cleaning up... [18/Dec/2006:15:31:49 -0500] - import idsk_data: Workers cleaned up. [18/Dec/2006:15:31:49 -0500] - import idsk_data: Cleaning up producer thread... [18/Dec/2006:15:31:49 -0500] - import idsk_data: Indexing complete. Post-processing... [18/Dec/2006:15:32:20 -0500] - import idsk_data: Flushing caches... [18/Dec/2006:15:32:23 -0500] - import idsk_data: Closing files... [18/Dec/2006:15:32:24 -0500] - import idsk_data: Import complete. Processed 163684 entries in 116 seconds. (1411.07 entries/sec) Edward On 12/18/06, Noriko Hosoi <nhosoi@redhat.com> wrote:> > Eddie C wrote: > > All, > > > > I tested this from scratch. > > I used the ldif_2db function which did work much faster! 112 > > seconds...rather then about 20 minutes. > > However I think the verify_db and db2_index functions are not in > > agreement. > > > > Create indexes. > > After my initial import. > > [root@ldap3 slapd-ldap3]# more out.after.final.txt > > ***************************************************************** > > verify-db: This tool should only be run if recovery start fails > > and the server is down. If you run this tool while the server is > > running, you may get false reports of corrupted files or other > > false errors. > > ***************************************************************** > > Verify log files in db ... Good > > Verify db/o_idsk_com/id2entry.db4 ... Good > > Verify db/userRoot/ancestorid.db4 ... Good > > Verify db/userRoot/entrydn.db4 ... Good > > Verify db/userRoot/cn.db4 ... Good > > Verify db/userRoot/numsubordinates.db4 ... Good > > Verify db/userRoot/aci.db4 ... Good > > Verify db/userRoot/parentid.db4 ... Good > > Verify db/userRoot/objectclass.db4 ... Good > > Verify db/userRoot/id2entry.db4 ... Good > > Verify db/userRoot/nsUniqueId.db4 ... Good > > Verify db/idsk_services/ancestorid.db4 ... > > DB ERROR: db_verify: Page 4: out-of-order key at entry 252 > > DB ERROR: db_verify: Page 7: out-of-order key at entry 194 > > DB ERROR: db_verify: Page 7: out-of-order key at entry 450 > > DB ERROR: db_verify: Page 11: out-of-order key at entry 69 > > DB ERROR: db_verify: Page 11: out-of-order key at entry 325 > > DB ERROR: db_verify: Page 11: out-of-order key at entry 581 > > DB ERROR: db_verify: Page 12: out-of-order key at entry 22 > > DB ERROR: db_verify: Page 16: out-of-order key at entry 249 > > DB ERROR: db_verify: Page 16: out-of-order key at entry 498 > > DB ERROR: db_verify: Page 16: out-of-order key at entry 754 > > DB ERROR: db_verify: Page 17: out-of-order key at entry 195 > > DB ERROR: db_verify: Page 17: out-of-order key at entry 451 > > DB ERROR: db_verify: Page 17: out-of-order key at entry 707 > > DB ERROR: db_verify: Page 18: out-of-order key at entry 148 > > DB ERROR: db_verify: Page 21: out-of-order key at entry 254 > > DB ERROR: db_verify: Page 21: out-of-order key at entry 510 > > DB ERROR: db_verify: Page 21: out-of-order key at entry 766 > > DB ERROR: db_verify: Page 22: out-of-order key at entry 207 > > DB ERROR: db_verify: Page 22: out-of-order key at entry 463 > > DB ERROR: db_verify: Page 22: out-of-order key at entry 719 > > DB ERROR: db_verify: Page 23: out-of-order key at entry 160 > > DB ERROR: db_verify: DB->verify: db/idsk_services/ancestorid.db4: > > DB_VERIFY_BAD: Database verification failed > > Secondary index file ancestorid.db4 in db/idsk_services is corrupted. > > Please run db2index(.pl) for reindexing. > > > Is this after you imported your ldif file to the backend ''idsk_services''? > > When you imported the ldif file, did you get any errors from ldif2db? > This is an sample output when we import Example.ldif. Did you see any > messages other than these? > [18/Dec/2006:04:00:48 -0800] - dblayer_instance_start: pagesize: 4096, > pages: 1009265, procpages: 23476 > [18/Dec/2006:04:00:48 -0800] - cache autosizing: import cache: 204800k > [18/Dec/2006:04:00:48 -0800] - li_import_cache_autosize: 50, > import_pages: 51200, pagesize: 4096 > [18/Dec/2006:04:00:48 -0800] - WARNING: Import is running with > nsslapd-db-private-import-mem on; No other process is allowed to access > the database > [18/Dec/2006:04:00:48 -0800] - dblayer_instance_start: pagesize: 4096, > pages: 1009265, procpages: 23476 > [18/Dec/2006:04:00:48 -0800] - cache autosizing: import cache: 204800k > [18/Dec/2006:04:00:48 -0800] - li_import_cache_autosize: 50, > import_pages: 51200, pagesize: 4096 > [18/Dec/2006:04:00:49 -0800] - import example: Beginning import job... > [18/Dec/2006:04:00:49 -0800] - import example: Index buffering enabled > with bucket size 100 > [18/Dec/2006:04:00:49 -0800] - import example: Processing file > "/export/DS7.2-24471/server/usr/share/fedora-ds/data/Example.ldif" > [18/Dec/2006:04:00:49 -0800] - import example: Finished scanning file > "/export/DS7.2-24471/server/usr/share/fedora-ds/data/Example.ldif" (160 > entries) > [18/Dec/2006:04:00:50 -0800] - import example: Workers finished; > cleaning up... > [18/Dec/2006:04:00:50 -0800] - import example: Workers cleaned up. > [18/Dec/2006:04:00:50 -0800] - import example: Cleaning up producer > thread... > [18/Dec/2006:04:00:50 -0800] - import example: Indexing complete. > Post-processing... > [18/Dec/2006:04:00:50 -0800] - import example: Flushing caches... > [18/Dec/2006:04:00:50 -0800] - import example: Closing files... > [18/Dec/2006:04:00:51 -0800] - All database threads now stopped > [18/Dec/2006:04:00:51 -0800] - import example: Import complete. > Processed 160 entries in 2 seconds. (80.00 entries/sec) > > Thanks, > --noriko > > So then i reran db2index....verify_db again...same result. > > > > Edward > > > > > > On 12/15/06, *Eddie C* <edlinuxguru@gmail.com > > <mailto:edlinuxguru@gmail.com>> wrote: > > > > I recently did an ldif backup of our iplanet 52 database. Its > > about an 88 MB ldif file. > > I took this to a new FDS server Dell 850 3 ghz duel core 2 sata > > hard disks. > > I ran an ldapadd the data imported perfectly. > > Then I tried to cutover some systems and give the database some > load. > > > > System went 200% processor > > > > Eventually I realized I was missing indexes so I added them > > through the graphical tool. > > > > The log seemed to do something like this > > generating index 1% > > generating index 2% > > .... > > generating index 49% > > Done > > Seemed weird that they would jump from 49% to Done > > At this point the new system was running at 100% processor > > But the queries are running faster on our old 440 MHZ sparc t1 > > server52 database > > > > I ran > > DB ERROR: db_verify: Page 30: out-of-order key at entry 498 > > DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4: > > DB_VERIFY_BAD: Database verification failed > > > > then I tried db2_index. The program seemed to be in a tight loop > > complaining about 1 missing entry. > > > > I do not realize how the data can be so corrupted right after an > > import. > > > > These are someone generic symptoms. Any ideas? Thanks > > > > > > ------------------------------------------------------------------------ > > > > -- > > Fedora-directory-users mailing list > > Fedora-directory-users@redhat.com > > https://www.redhat.com/mailman/listinfo/fedora-directory-users > > > > > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users > > > >
Richard Megginson
2006-Dec-21 19:16 UTC
Re: [Fedora-directory-users] Re: Migration from i-planet 52
Eddie C wrote:> >>Any errors? > No as far as I can tell the import goes smooth. Our upstream > applications seem to have no problem with the data it seems to be all > imported. > > I am really troubleshooting and index and performance issues, I figure > out of order keys could be slowing searches down. This a fairly large > db 160000 entires) some objects have upwards of 20 attributes. > > db_verify and db_2index both run fairly quickly. It just seems like > they cant both agree on what clean data looks like. > > FWI I upgraded to the latest 1.0.4 before all this testing.Just to confirm, was the server running when you ran db_verify?> > [18/Dec/2006:15:29:50 -0500] - WARNING: Import is running with > nsslapd-db-private-import-mem on; No other process is allowed to > access the database > [18/Dec/2006:15:29:50 -0500] - import idsk_services: Beginning import > job... > [18/Dec/2006:15:29:50 -0500] - import idsk_services: Index buffering > enabled with bucket size 15 > [18/Dec/2006:15:29:50 -0500] - import idsk_services: Processing file > "/opt/idsk/downloads/idsk.services.com.ldif" > [18/Dec/2006:15:29:50 -0500] - import idsk_services: Finished scanning > file "/opt/idsk/downloads/idsk.services.com.ldif" (2037 entries) > [18/Dec/2006:15:29:51 -0500] - import idsk_services: Workers finished; > cleaning up... > [18/Dec/2006:15:29:51 -0500] - import idsk_services: Workers cleaned up. > [18/Dec/2006:15:29:51 -0500] - import idsk_services: Cleaning up > producer thread... > [18/Dec/2006:15:29:51 -0500] - import idsk_services: Indexing > complete. Post-processing... > [18/Dec/2006:15:29:51 -0500] - import idsk_services: Flushing caches... > [18/Dec/2006:15:29:51 -0500] - import idsk_services: Closing files... > [18/Dec/2006:15:29:51 -0500] - import idsk_services: Import complete. > Processed 2037 entries in 1 seconds. ( 2037.00 entries/sec) > [18/Dec/2006:15:30:28 -0500] - Bringing idsk_data offline... > [18/Dec/2006:15:30:28 -0500] - WARNING: Import is running with > nsslapd-db-private-import-mem on; No other process is allowed to > access the database > [18/Dec/2006:15:30:28 -0500] - import idsk_data: Beginning import job... > [18/Dec/2006:15:30:28 -0500] - import idsk_data: Index buffering > enabled with bucket size 15 > [18/Dec/2006:15:30:28 -0500] - import idsk_data: Processing file > "/opt/idsk/downloads/idsk.com.ldif" > [18/Dec/2006:15:30:48 -0500] - import idsk_data: Processed 77288 > entries -- average rate 3680.4/sec, recent rate 3680.3/sec, hit ratio 0% > [18/Dec/2006:15:31:13 -0500] - import idsk_data: Processed 141956 > entries -- average rate 3086.0/sec, recent rate 3086.0/sec, hit ratio 100% > [18/Dec/2006:15:31:35 -0500] - import idsk_data: Finished scanning > file "/opt/idsk/downloads/idsk.com.ldif" (163684 entries) > [18/Dec/2006:15:31:49 -0500] - import idsk_data: Workers finished; > cleaning up... > [18/Dec/2006:15:31:49 -0500] - import idsk_data: Workers cleaned up. > [18/Dec/2006:15:31:49 -0500] - import idsk_data: Cleaning up producer > thread... > [18/Dec/2006:15:31:49 -0500] - import idsk_data: Indexing complete. > Post-processing... > [18/Dec/2006:15:32:20 -0500] - import idsk_data: Flushing caches... > [18/Dec/2006:15:32:23 -0500] - import idsk_data: Closing files... > [18/Dec/2006:15:32:24 -0500] - import idsk_data: Import complete. > Processed 163684 entries in 116 seconds. ( 1411.07 entries/sec) > > Edward > > On 12/18/06, *Noriko Hosoi* <nhosoi@redhat.com > <mailto:nhosoi@redhat.com>> wrote: > > Eddie C wrote: > > All, > > > > I tested this from scratch. > > I used the ldif_2db function which did work much faster! 112 > > seconds...rather then about 20 minutes. > > However I think the verify_db and db2_index functions are not in > > agreement. > > > > Create indexes. > > After my initial import. > > [root@ldap3 slapd-ldap3]# more out.after.final.txt > > ***************************************************************** > > verify-db: This tool should only be run if recovery start fails > > and the server is down. If you run this tool while the server is > > running, you may get false reports of corrupted files or other > > false errors. > > ***************************************************************** > > Verify log files in db ... Good > > Verify db/o_idsk_com/id2entry.db4 ... Good > > Verify db/userRoot/ancestorid.db4 ... Good > > Verify db/userRoot/entrydn.db4 ... Good > > Verify db/userRoot/cn.db4 ... Good > > Verify db/userRoot/numsubordinates.db4 ... Good > > Verify db/userRoot/aci.db4 ... Good > > Verify db/userRoot/parentid.db4 ... Good > > Verify db/userRoot/objectclass.db4 ... Good > > Verify db/userRoot/id2entry.db4 ... Good > > Verify db/userRoot/nsUniqueId.db4 ... Good > > Verify db/idsk_services/ancestorid.db4 ... > > DB ERROR: db_verify: Page 4: out-of-order key at entry 252 > > DB ERROR: db_verify: Page 7: out-of-order key at entry 194 > > DB ERROR: db_verify: Page 7: out-of-order key at entry 450 > > DB ERROR: db_verify: Page 11: out-of-order key at entry 69 > > DB ERROR: db_verify: Page 11: out-of-order key at entry 325 > > DB ERROR: db_verify: Page 11: out-of-order key at entry 581 > > DB ERROR: db_verify: Page 12: out-of-order key at entry 22 > > DB ERROR: db_verify: Page 16: out-of-order key at entry 249 > > DB ERROR: db_verify: Page 16: out-of-order key at entry 498 > > DB ERROR: db_verify: Page 16: out-of-order key at entry 754 > > DB ERROR: db_verify: Page 17: out-of-order key at entry 195 > > DB ERROR: db_verify: Page 17: out-of-order key at entry 451 > > DB ERROR: db_verify: Page 17: out-of-order key at entry 707 > > DB ERROR: db_verify: Page 18: out-of-order key at entry 148 > > DB ERROR: db_verify: Page 21: out-of-order key at entry 254 > > DB ERROR: db_verify: Page 21: out-of-order key at entry 510 > > DB ERROR: db_verify: Page 21: out-of-order key at entry 766 > > DB ERROR: db_verify: Page 22: out-of-order key at entry 207 > > DB ERROR: db_verify: Page 22: out-of-order key at entry 463 > > DB ERROR: db_verify: Page 22: out-of-order key at entry 719 > > DB ERROR: db_verify: Page 23: out-of-order key at entry 160 > > DB ERROR: db_verify: DB->verify: db/idsk_services/ancestorid.db4: > > DB_VERIFY_BAD: Database verification failed > > Secondary index file ancestorid.db4 in db/idsk_services is > corrupted. > > Please run db2index(.pl) for reindexing. > > > Is this after you imported your ldif file to the backend > ''idsk_services''? > > When you imported the ldif file, did you get any errors from ldif2db? > This is an sample output when we import Example.ldif. Did you see any > messages other than these? > [18/Dec/2006:04:00:48 -0800] - dblayer_instance_start: pagesize: 4096, > pages: 1009265, procpages: 23476 > [18/Dec/2006:04:00:48 -0800] - cache autosizing: import cache: > 204800k > [18/Dec/2006:04:00:48 -0800] - li_import_cache_autosize: 50, > import_pages: 51200, pagesize: 4096 > [18/Dec/2006:04:00:48 -0800] - WARNING: Import is running with > nsslapd-db-private-import-mem on; No other process is allowed to > access > the database > [18/Dec/2006:04:00:48 -0800] - dblayer_instance_start: pagesize: 4096, > pages: 1009265, procpages: 23476 > [18/Dec/2006:04:00:48 -0800] - cache autosizing: import cache: 204800k > [18/Dec/2006:04:00:48 -0800] - li_import_cache_autosize: 50, > import_pages: 51200, pagesize: 4096 > [18/Dec/2006:04:00:49 -0800] - import example: Beginning import job... > [18/Dec/2006:04:00:49 -0800] - import example: Index buffering enabled > with bucket size 100 > [18/Dec/2006:04:00:49 -0800] - import example: Processing file > "/export/DS7.2-24471/server/usr/share/fedora-ds/data/Example.ldif" > [18/Dec/2006:04:00:49 -0800] - import example: Finished scanning file > "/export/DS7.2-24471/server/usr/share/fedora-ds/data/Example.ldif" > (160 > entries) > [18/Dec/2006:04:00:50 -0800] - import example: Workers finished; > cleaning up... > [18/Dec/2006:04:00:50 -0800] - import example: Workers cleaned up. > [18/Dec/2006:04:00:50 -0800] - import example: Cleaning up producer > thread... > [18/Dec/2006:04:00:50 -0800] - import example: Indexing complete. > Post-processing... > [18/Dec/2006:04:00:50 -0800] - import example: Flushing caches... > [18/Dec/2006:04:00:50 -0800] - import example: Closing files... > [18/Dec/2006:04:00:51 -0800] - All database threads now stopped > [18/Dec/2006:04:00:51 -0800] - import example: Import complete. > Processed 160 entries in 2 seconds. (80.00 entries/sec) > > Thanks, > --noriko > > So then i reran db2index....verify_db again...same result. > > > > Edward > > > > > > On 12/15/06, *Eddie C* <edlinuxguru@gmail.com > <mailto:edlinuxguru@gmail.com> > > <mailto: edlinuxguru@gmail.com <mailto:edlinuxguru@gmail.com>>> > wrote: > > > > I recently did an ldif backup of our iplanet 52 database. Its > > about an 88 MB ldif file. > > I took this to a new FDS server Dell 850 3 ghz duel core 2 sata > > hard disks. > > I ran an ldapadd the data imported perfectly. > > Then I tried to cutover some systems and give the database > some load. > > > > System went 200% processor > > > > Eventually I realized I was missing indexes so I added them > > through the graphical tool. > > > > The log seemed to do something like this > > generating index 1% > > generating index 2% > > .... > > generating index 49% > > Done > > Seemed weird that they would jump from 49% to Done > > At this point the new system was running at 100% processor > > But the queries are running faster on our old 440 MHZ sparc t1 > > server52 database > > > > I ran > > DB ERROR: db_verify: Page 30: out-of-order key at entry 498 > > DB ERROR: db_verify: DB->verify: > db/o_com/channelcontentowner.db4: > > DB_VERIFY_BAD: Database verification failed > > > > then I tried db2_index. The program seemed to be in a tight loop > > complaining about 1 missing entry. > > > > I do not realize how the data can be so corrupted right after an > > import. > > > > These are someone generic symptoms. Any ideas? Thanks > > > > > > > ------------------------------------------------------------------------ > > > > -- > > Fedora-directory-users mailing list > > Fedora-directory-users@redhat.com > <mailto:Fedora-directory-users@redhat.com> > > https://www.redhat.com/mailman/listinfo/fedora-directory-users > <https://www.redhat.com/mailman/listinfo/fedora-directory-users> > > > > > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > <mailto:Fedora-directory-users@redhat.com> > https://www.redhat.com/mailman/listinfo/fedora-directory-users > > > > > ------------------------------------------------------------------------ > > -- > Fedora-directory-users mailing list > Fedora-directory-users@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users >