Xapian discuss - Jan 2015 - Question on "single writer, multiple reader"

Hi, dear Xapianers!

    I've been using Xapian in my project recently. The feature "single
writer, multiple reader" is one of my favorite, but currently I can't
make
it work.

    My goal is to add more documents to the database increamentally, while
the Xapian search process is not stopped.

    I followed "quickstartindex.cc" and
"quickstartsearch.cc" on the
website (http://xapian.org/docs/quickstart.html), and added a while loop
reading in query text around the core logic of the search process, in order
to simulate the online situation. (code snippet attatched below)

    Here is my question:

        With the search process alive, I added some new documents into the
database, but I can't retrieve them on the search process. However, if I
stop the search process and restart it again, I can retrieve those new ones.

    Could you tell me where I went wrong? How can I make increamental
modifications without stopping the search service?
    Thanks a lot :)

Best regards,
Gang Chen


================================// code snippet for the while loop in
quickstartsearch.cc

Xapian::Database db(argv[1]);
Xapian::Enquire enquire(db);
while (true) {
            string line;
            getline(cin, line);
            Xapian::Query query(line);
            cout << "Performing query `" <<
query.get_description() << "'"
<< endl;

            // Give the query object to the enquire session
            enquire.set_query(query);

            // Get the top 10 results of the query
            Xapian::MSet matches = enquire.get_mset(0, 10);

            // Display the results
            cout << matches.size() << " results found"
<< endl;
            for (Xapian::MSetIterator i = matches.begin(); i !matches.end();
++i) {
                Xapian::Document doc = i.get_document();
                cout << "Document ID " << *i <<
"\t" <<
                        i.get_percent() << "% [" <<
                        doc.get_data() << "]" << endl;
            }
}

On 16 Jan 2015, at 07:39, Gang Chen <pkuchengang at gmail.com> wrote:
> I've been using Xapian in my project recently. The feature "single
> writer, multiple reader" is one of my favorite, but currently I
can't make
> it work.
> 
> With the search process alive, I added some new documents into the
> database, but I can't retrieve them on the search process. However, if
I
> stop the search process and restart it again, I can retrieve those new
ones.
That?s exactly how it?s supposed to work. ?Eventually? (once the writer gets
sufficiently far ahead of the reader), the reader will get a
DatabaseModifiedError and will have to re-open the database, but until then it?s
up to it when it does so. You may wish to do it every N requests, or every K
seconds, or only when you have to handle DatabaseModifiedError; it?s up to you.

We have a note that some more detailed documentation around this would be
helpful. For now, the following should be useful:
<https://getting-started-with-xapian.readthedocs.org/en/latest/concepts/indexing/databases.html?highlight=databasemodifiederror#concurrent-access>.

J

-- 
 James Aylett, occasional trouble-maker
 xapian.org

On Sun, Jan 18, 2015 at 04:25:29PM +0000, James Aylett
wrote:
> That?s exactly how it?s supposed to work. ?Eventually? (once the
> writer gets sufficiently far ahead of the reader), the reader will get
> a DatabaseModifiedError and will have to re-open the database, but
> until then it?s up to it when it does so. You may wish to do it every
> N requests, or every K seconds, or only when you have to handle
> DatabaseModifiedError; it?s up to you.
>
> We have a note that some more detailed documentation around this would
> be helpful. For now, the following should be useful:
>
<https://getting-started-with-xapian.readthedocs.org/en/latest/concepts/indexing/databases.html?highlight=databasemodifiederror#concurrent-access>.
I've just improved this with a note that reopen() is a cheap no-op when
there isn't a newer revision:

https://github.com/jaylett/xapian-docsprint/commit/41bb7a1da61d22e0047a83176386da4db1ee9f15

Cheers,
    Olly