Xapian discuss - Apr 2008 - allterms iterator dangerous?

Is there a safe way of completely deleting a database, but from the
API, not by wiping it on the filesystem* ? This is for a 'rebuild all'
function.

I used to use the all terms iterator to do it but a) that seems slow!
and b) it is now marked "dangerous" (at least in the Ruby API).
Why is it dangerous, and is there an alternative?

The all terms iterator was also useful for the slightly less crazy
thing of looking for stray stuff that is in the index, but no longer
in the database.

Francis

* OK, you can try and convince me I should do this instead if you
like! It's just there might be things reading the database, so you'd
likely get errors pulling the database away erratically under their
feet.

On Apr 24, 2008, at 10:00 AM, Francis Irving wrote:
> Is there a safe way of completely deleting a database, but from the
> API, not by wiping it on the filesystem* ? This is for a 'rebuild
all'
> function.
>
Build a new index in a new place, and then swap them later on?
If your concern is things reading that is.


- andreas

Francis Irving wrote:
> Is there a safe way of completely deleting a database, but from the
> API, not by wiping it on the filesystem* ? This is for a 'rebuild
all'
> function.
> 
> I used to use the all terms iterator to do it but a) that seems slow!
> and b) it is now marked "dangerous" (at least in the Ruby API).
> Why is it dangerous, and is there an alternative?
I've not been following the ruby bindings, but I think it's just that 
the "allterms_begin()" and "allterms_end()" methods have
been deprecated
and replaced by a ruby iterator obtained by calling "allterms()".
> The all terms iterator was also useful for the slightly less crazy
> thing of looking for stray stuff that is in the index, but no longer
> in the database.
That seems perfectly sensible, yes.

> * OK, you can try and convince me I should do this instead if you
> like! It's just there might be things reading the database, so
you'd
> likely get errors pulling the database away erratically under their
> feet.
I'd tend to go this way, simply because it saves a lot of effort.

If you're on Unix, you can use the fact that xapian keeps the database 
file handles open during a search.  This means that if you unlink the 
files which a search is in progress, your search should still complete 
correctly.  There is a brief period where some of the file handles are 
open but others haven't yet been opened, so I'd suggest using a symlink 
to point to the "live database", building a new live database, then 
switching the symlink over, then wait a few seconds (to ensure no 
searches are in the process of opening the database), and then delete 
the old database.

This doesn't work on windows, of course.

-- 
Richard