Xapian discuss - Apr 2011 - How to rebuild database from scratch without preventing searches

I have a process which wants to recreate a database nightly (it takes ~10 
minutes, and is much faster than trying to figure out what to remove and add 
from the existing contents). However, opening the database with the flag 
DB_CREATE_OR_OVERWRITE destroys the existing database and prevents searches 
from taking place until the rebuild process is finished.

We'd like to rebuild the database while still allowing searches to proceed, 
naturally ... what's the, uh, "recommended" way to do this? Build
into a
temporary copy and move a symlink? Is there any way to clear the contents and 
start over without removing the existing version so readers can continue to 
work (other than scanning through all the existing documents and removing them 
...)?

On 12 April 2011 14:10, William Crawford <william at sciencephoto.co.uk>
wrote:
> We'd like to rebuild the database while still allowing searches to
proceed,
> naturally ... what's the, uh, "recommended" way to do this?
Build into a
> temporary copy and move a symlink? Is there any way to clear the contents
and
> start over without removing the existing version so readers can continue to
> work (other than scanning through all the existing documents and removing
them
> ...)?
Don't use a symlink - if a database is being opened for reading while
the symlink is changed, some tables may be opened from the old db, and
some from the new, causing a (false) report of a corrupted database).

What you want to use is a stub database:
http://xapian.org/docs/overview.html (and search for stub database).

Basically, this is a text file which points to one or more other
databases.  You can open a Xapian::Database by pointing to a stub
database, and it will read it and open the databases it points to,
returning them as the Xapian::Database.  Also, because it's a single
file, you can use rename to atomically replace it with a new file, to
swap to a file pointing to a new set of databases.

If you work this out, we really need to write this up in the admin
guide (http://xapian.org/docs/admin_notes.html) - if anyone's able to
provide a patch to add this, that'd be great.

-- 
Richard