Xapian discuss - Jul 2008 - can a corrupt database be fixed?

Hi.

I have a Xapian database which appears to be corrupt.  I know this is
a long shot, but I'm wondering if there's any way to repair it.

Why do I think it's corrupt?  Let me present some evidence.

If I run `delve' on the database, its output is always the following
two lines:

number of documents = 559147 
Error: The revision being read has been discarded - you should call
Xapian::Database::reopen() and retry the operation

If I try to copy the database using copydatabase, its output always is
the following line:

copydatabase: The revision being read has been discarded - you should call
Xapian::Database::reopen() and retry the operation

If I run the Linux `fuser' command on the database's files, it always
tells me that they're not in use.

The database is used by PHP software running within Apache.  In Apache's
PHP error log from last night, I find a number of messages like the
following which mention "DatabaseCorruptError":

[23-Jul-2008 23:26:39] PHP Fatal error:  DatabaseCorruptError: Unexpected end of
posting list for `feeling' in CONCEALED on line 720

(I have replaced the filename with the string "CONCEALED".)

There are a total of 14 messages mentioning "DatabaseCorruptError",
which I have put in the attached file.

In that same PHP error log, there is a constant stream of messages about
"DatabaseModifiedError: The revision being read has been discarded".
(Although these messages have always been appearing, the rate has been
very high since last night.)  Also, the PHP application isn't working
properly at present.

Taken together, I think the above gives sufficient reason to conclude
that the database is corrupt.

I can probably regenerate the database starting from a backup of 3 months
ago, but it would be much easier if there was some program I could run
to repair it.

Alternatively, if there is some programmatic approach which would salvage
most of the data, that would also be good.  I am reasonably familiar
with Xapian's PHP API, so, if given an algorithm, I should be able to
implement it.

In case it's relevant: the version of the Xapian software I'm using
is 0.9.9.  To be precise: I'm using a fairly old Debian system, and the
relevant Debian packages seem to be libxapian13, which is at version
0.9.9-0stable1, and php4-xapian, which is at version 0.9.9-1stable1.

Regards,

Ronan.
-------------- next part --------------
[23-Jul-2008 23:26:39] PHP Fatal error:  DatabaseCorruptError: Unexpected end of
posting list for `feeling' in CONCEALED on line 720
[23-Jul-2008 23:27:12] PHP Fatal error:  DatabaseCorruptError: Unexpected end of
posting list for `far' in CONCEALED on line 720
[23-Jul-2008 23:29:16] PHP Fatal error:  DatabaseCorruptError: Unexpected end of
posting list for `family' in CONCEALED on line 720
[23-Jul-2008 23:30:04] PHP Fatal error:  DatabaseCorruptError: Unexpected end of
posting list for `far' in CONCEALED on line 660
[23-Jul-2008 23:32:01] PHP Fatal error:  DatabaseCorruptError: Unexpected end of
posting list for `far' in CONCEALED on line 720
[23-Jul-2008 23:36:28] PHP Fatal error:  DatabaseCorruptError: Unexpected end of
posting list for `feeling' in CONCEALED on line 720
[23-Jul-2008 23:36:56] PHP Fatal error:  DatabaseCorruptError: Unexpected end of
posting list for `referendum' in CONCEALED on line 720
[23-Jul-2008 23:37:26] PHP Fatal error:  DatabaseCorruptError: Unexpected end of
posting list for `family' in CONCEALED on line 720
[23-Jul-2008 23:49:05] PHP Fatal error:  DatabaseCorruptError: Db block
overwritten - are there multiple writers? in CONCEALED on line 222
[23-Jul-2008 23:50:28] PHP Fatal error:  DatabaseCorruptError: Unexpected end of
posting list for `family' in CONCEALED on line 720
[23-Jul-2008 23:51:51] PHP Fatal error:  DatabaseCorruptError: Unexpected end of
posting list for `family' in CONCEALED on line 720
[23-Jul-2008 23:53:43] PHP Fatal error:  DatabaseCorruptError: Unexpected end of
posting list for `family' in CONCEALED on line 720
[23-Jul-2008 23:54:03] PHP Fatal error:  DatabaseCorruptError: Db block
overwritten - are there multiple writers? in CONCEALED on line 222
[23-Jul-2008 23:54:05] PHP Fatal error:  DatabaseCorruptError: Db block
overwritten - are there multiple writers? in CONCEALED on line 222

> I have a Xapian database which appears to be corrupt.  I know this is
> a long shot, but I'm wondering if there's any way to repair it.
Never mind; I've managed to regenerate the database.

Ronan.

On Thu, Jul 24, 2008 at 05:39:29PM +0100, Ronan wrote:
> I have a Xapian database which appears to be corrupt.  I know this is
> a long shot, but I'm wondering if there's any way to repair it.
I realise you're no longer needing an answer, but others may want to
know.

Do you know how it became corrupted BTW?

Corrupt Xapian databases are very rare in my experience, and the only
cases I'm aware of which weren't attributable to either hardware failure
or concurrent writing after defeating the database write locking (quartz
used to leave stale locks behind if the Xapian-using process didn't exit
cleanly so people had to delete such lock files and sometimes did so too
keenly) are down to mishandling the base files in corner cases.

Such cases are likely to be recoverable by removing whichever of baseA
or baseB is wedged, to leave a consistent set of tables (but take a copy
before you start deleting!)  Or you might find a newer release of Xapian
can read them the "corrupt" database fine.

There isn't a "database repair tool" currently.  If only the
postlist is
corrupted, then using copydatabase will generate a copy of the database
without looking at the postlist, except to open it.`
> Why do I think it's corrupt?  Let me present some evidence.
> [...]
Running xapian-check is probably the best check.  That checks the
structure of each table, and also performs some cross-checks for
consistency.
> Alternatively, if there is some programmatic approach which would salvage
> most of the data, that would also be good.
This is the main obstacle to writing a "corrupt database fixer" - I
don't know what the appropriate programmatic approach would be as I've
not seen what corrupt databases look like.  And producing a non-corrupt
database mostly full of junk isn't useful.
> In case it's relevant: the version of the Xapian software I'm using
> is 0.9.9.  To be precise: I'm using a fairly old Debian system, and the
> relevant Debian packages seem to be libxapian13, which is at version
> 0.9.9-0stable1, and php4-xapian, which is at version 0.9.9-1stable1.
I'm afraid that's really too old for us to still provide help with.  We
do supply backported packages of newer Xapian versions for Debian and
Ubuntu though - see the download page for details.

Cheers,
    Olly