Over the weekend my workstation (still on CentOS 3) started showing SMART errors, so we shut it down and moved everything onto a new drive. On reboot, the RPM database reports as corrupted and "rpm --rebuilddb" segmentation faults. Is there any reasonable way to recover from this, short of a complete reinstall? The machine is otherwise up though it''s quite likely some files have been lost.
On Mon, Feb 23, 2009, Bart Schaefer wrote:>Over the weekend my workstation (still on CentOS 3) started showing >SMART errors, so we shut it down and moved everything onto a new >drive. On reboot, the RPM database reports as corrupted and "rpm >--rebuilddb" segmentation faults. > >Is there any reasonable way to recover from this, short of a complete >reinstall? The machine is otherwise up though it''s quite likely some >files have been lost.You may be able to get things going again with the bdb recover script, /usr/lib/rpm/rpmdb_recover. I think this should be a berkeley DB recovery routine that is specific to the version being used by rpm. I have recovered the OpenPKG rpm database using this technique using the standard bdb db_recover program. If I am not mistaken, the appropriate command would be: /usr/lib/rpm/rpmdb_recover -h /var/lib/rpm Bill -- INTERNET: bill at celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way Voice: (206) 236-1676 Mercer Island, WA 98040-0820 Fax: (206) 232-9186 Cutting the space budget really restores my faith in humanity. It eliminates dreams, goals, and ideals and lets us get straight to the business of hate, debauchery, and self-annihilation. -- Johnny Hart
On Mon, Feb 23, 2009 at 8:59 AM, Bill Campbell <centos at celestial.com> wrote:> On Mon, Feb 23, 2009, Bart Schaefer wrote: >>Over the weekend my workstation (still on CentOS 3) started showing >>SMART errors, so we shut it down and moved everything onto a new >>drive. On reboot, the RPM database reports as corrupted and "rpm >>--rebuilddb" segmentation faults. >> >>Is there any reasonable way to recover from this, short of a complete >>reinstall? The machine is otherwise up though it''s quite likely some >>files have been lost. > > You may be able to get things going again with the bdb recover > script, /usr/lib/rpm/rpmdb_recover. I think this should be a > berkeley DB recovery routine that is specific to the version > being used by rpm. I have recovered the OpenPKG rpm database > using this technique using the standard bdb db_recover program. > > If I am not mistaken, the appropriate command would be: > > /usr/lib/rpm/rpmdb_recover -h /var/lib/rpmYou can first verity the database by: cd /var/lib/rpm /usr/lib/rpm/rpmdb_verify Packages If the recover command does not work, then try something like: Check the contents of /var/log/rpmpkgs (and .1 .2 etc). Get all packages listed there into a single directory. cd to that directory and issue the command: rpm -ivh --noscripts --notriggers --justdb *.rpm to restore your /var/lib/rpm/Packages file Akemi
On Mon, Feb 23, 2009 at 8:59 AM, Bill Campbell <centos at celestial.com> wrote:> You may be able to get things going again with the bdb recover > script, /usr/lib/rpm/rpmdb_recover. I think this should be a > berkeley DB recovery routine that is specific to the version > being used by rpm.There doesn''t seem to be an rpmdb_recover script: # ls /usr/lib/rpm athlon-linux find-provides magic.req rpmdb_verify brp-compress find-provides.perl mkinstalldirs rpmdeps brp-python-bytecompile find-req.pl noarch-linux rpme brp-redhat find-requires perl.prov rpmfile brp-sparc64-linux find-requires.perl perl.req rpmi brp-strip freshen.sh perldeps.pl rpmk brp-strip-comment-note get_magic.pl redhat rpmpopt-4.2.3 brp-strip-shared getpo.sh rpm.daily rpmq brp-strip-static-archive http.req rpm.log rpmrc check-files i386-linux rpm.xinetd rpmt check-prereqs i486-linux rpm2cpio.sh rpmu config.guess i586-linux rpmb rpmv config.site i686-linux rpmcache tgpg config.sub javadeps rpmd trpm convertrpmrc.sh macros rpmdb_deadlock u_pkg.sh cross-build magic rpmdb_dump vpkg-provides.sh debugedit magic.mgc rpmdb_load vpkg-provides2.sh find-debuginfo.sh magic.mime rpmdb_loadcvt find-lang.sh magic.mime.mgc rpmdb_stat find-prov.pl magic.prov rpmdb_svc If I try using db_recover directly: # db_recover -h /var/lib/rpm db_recover: unable to join the environment rpmdb_verify still reports DB_VERIFY_BAD afterwards.
On Mon, Feb 23, 2009 at 9:10 AM, Akemi Yagi <amyagi at gmail.com> wrote:> Check the contents of /var/log/rpmpkgs (and .1 .2 etc). > Get all packages listed there into a single directory. > cd to that directory and issue the command: > > rpm -ivh --noscripts --notriggers --justdb *.rpm > > to restore your /var/lib/rpm/Packages fileAre there any files I should remove or zero out or the like before beginning this process?
On Mon, Feb 23, 2009, Bart Schaefer wrote:>On Mon, Feb 23, 2009 at 8:59 AM, Bill Campbell <centos at celestial.com> wrote: >> You may be able to get things going again with the bdb recover >> script, /usr/lib/rpm/rpmdb_recover. I think this should be a >> berkeley DB recovery routine that is specific to the version >> being used by rpm. > >There doesn''t seem to be an rpmdb_recover script:I got that grep rpm -ql output on a CentOS 5 system with rpm-4.4.2-47.el5. I see the same file on rpm-4.4.2-48.el5 on a slightly newer system. It is not there on a CentOS 4.5 system. That machine has /usr/bin/db_recover as part of the package db4-utils-4.2.52-7.1 which may work. Bill -- INTERNET: bill at celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way Voice: (206) 236-1676 Mercer Island, WA 98040-0820 Fax: (206) 232-9186 People who relieve others of their money with guns are called robbers. It does not alter the immorality of the act when the income transfer is carried out by government.
On Mon, Feb 23, 2009 at 10:13 AM, Bill Campbell <centos at celestial.com> wrote:> > That machine has /usr/bin/db_recover as part of the package > db4-utils-4.2.52-7.1 which may work.# db_recover -h /var/lib/rpm db_recover: unable to join the environment rpmdb_verify still reports DB_VERIFY_BAD afterwards. In fact several of the files are trashed, not just Packages: # /usr/lib/rpm/rpmdb_verify Basenames db_verify: Page 2596: overflow page of invalid type 2 db_verify: Page 2466: hash page has bad prev_pgno db_verify: Page 0: non-invalid page 2405 on free list db_verify: DB->verify: Basenames: DB_VERIFY_BAD: Database verification failed # /usr/lib/rpm/rpmdb_verify Filemd5s db_verify: Page 1328: hash page has bad prev_pgno db_verify: Page 0: non-invalid page 1330 on free list db_verify: DB->verify: Filemd5s: DB_VERIFY_BAD: Database verification failed # /usr/lib/rpm/rpmdb_verify Name db_verify: Page 10: hash page has bad prev_pgno db_verify: DB->verify: Name: DB_VERIFY_BAD: Database verification failed /usr/lib/rpm/rpmdb_verify Packages db_verify: Page 9537: overflow page of invalid type 2 (some hundreds of other errors) db_verify: Page 0: non-invalid page 12437 on free list db_verify: DB->verify: Packages: DB_VERIFY_BAD: Database verification failed # /usr/lib/rpm/rpmdb_verify Requirename db_verify: Page 48: overflow page of invalid type 2 db_verify: Page 53: overflow page of invalid type 2 db_verify: Page 52: hash page has bad prev_pgno db_verify: Page 55: overflow page of invalid type 2 db_verify: Page 54: hash page has bad prev_pgno db_verify: Page 50: hash page has bad prev_pgno db_verify: Page 59: hash page has bad prev_pgno db_verify: Page 94: hash page has bad prev_pgno db_verify: Page 0: non-invalid page 96 on free list db_verify: DB->verify: Requirename: DB_VERIFY_BAD: Database verification failed # /usr/lib/rpm/rpmdb_verify Sha1header db_verify: Page 12: hash page has bad prev_pgno db_verify: DB->verify: Sha1header: DB_VERIFY_BAD: Database verification failed
On Mon, Feb 23, 2009 at 9:10 AM, Akemi Yagi <amyagi at gmail.com> wrote:> > Check the contents of /var/log/rpmpkgs (and .1 .2 etc). > Get all packages listed there into a single directory. > cd to that directory and issue the command: > > rpm -ivh --noscripts --notriggers --justdb *.rpm > > to restore your /var/lib/rpm/Packages fileI ended up backing up/removing /var/lib/rpm and then running "rpm --initdb" before running the --justdb command shown above. I was able to pull all but 73 out of more than 1000 packages listed in /var/log/rpmpkgs from /var/cache/yum plus the CentOS 3.9 CD set.