Xapian discuss - Jun 2006 - xapian-compact seg faulting & Re: Error msg xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation

Olly,

True something wrong with my setup. How I can send you the file (~6.4 MB gz)?

Thanks,
OSC
> ----- Original Message -----
> From: "Olly Betts" <olly@survex.com>
> To: oscaruser@programmer.net
> Subject: Re: [Xapian-discuss] xapian-compact seg faulting & Re:
[Xapian-discuss] Error msg xapian-compact: The revision being read has been
discarded - you should call Xapian::Database::reopen() and retry the operation
> Date: Tue, 13 Jun 2006 02:03:51 +0100
> 
> 
> This is very odd.  Flint currently seems rock solid for everybody using
> it apart from you where it seems very flaky.  If I could reproduce the
> problem, I should be able to pin down what's wrong and fix it, but I
> can't from the information you've provided so far.
> 
> For example, Gmane uses xapian-compact to merge 2 databases every night,
> and even uses the -F option like you do.  For a full rebuild (which I
> must have done at least 10 times) it merges one database per million
> documents, so that's more than 30.  Tweakers.net use flint on a large
> system and run xapian-compact frequently and they're very happy with
> the stability.  I know of several other happy flint users (and there
> are probably more I don't know of).
> 
> I don't understand what's the difference which is causing you these
> problems.  I suspect if we can work that out this will be fairly easy to
> resolve.
> 
> On Mon, Jun 12, 2006 at 11:35:53AM -0800, oscaruser@programmer.net wrote:
> > I think it may also be time to develop a tool to validate the
integrity
> > of a flint index.
> 
> If you can run copydatabase on a database, it's in good shape with the
> possible exception of the postlist table.
> 
> If you want something more specific, quartzcheck is probably a good
> starting point.  The flint and quartz formats still have quite a bit
> in common.
> 
> Cheers,
>      Olly

-- 
___________________________________________________
Play 100s of games for FREE! http://games.mail.com/

On Tue, Jun 13, 2006 at 10:05:34AM -0800, oscaruser@programmer.net
wrote:
> True something wrong with my setup. How I can send you the file (~6.4 MB
gz)?
I'm not sure there's much point - we know it's broken.  The
interesting
question is how it got that way.  What I really need is a way to
reproduce the problem - for example, a program I can run which produces
a bad database after a while.  The bad database itself doesn't really
help.

Is this bad database a "one off" incident currently, or if you restart
the system does it produce another bad database after running for a
while?

I have had one thought about this - what you're doing that is unusual
is opening and closing the database for every document addition or
update.  You could get the problem that you're seeing if there were a
flaw in the flint locking strategy which that usage pattern exposed.
I'll review the code and if I can't see a problem, run some tests.

Cheers,
    Olly