Xapian discuss - Sep 2010 - Search::Xapian really slow compared to C++ Xapian

Hi,

I made a few tests with the Perl module of Xapian and the C++ Version. The
database I'm using consits of 50 000 documents
which are mainly full texts of PFD documents. The database has a size of
about 1,5 GB.

The indexer is written in Perl and does the job pretty fast and uses about
100 MB of RAM which is OK.

Now if I'm searching in the index with a pretty complex search query like
'a* AND b* AND c* AND d* AND e*'
my searcher, which is based on the Perl example code uses about 2GB of RAM
and takes 20 seconds to
finish the search which is pretty much.

The same searcher written in C++ (also based on the example code) uses only
a few MB of RAM and finishes the search in less than
1 second.

Is it normal that the Perl code is so much slower and uses this enormous
amount of RAM?

Cheers,

Gerald

On Tue, Sep 28, 2010 at 08:14:01PM +0200, Hanzz Solo
wrote:
> Is it normal that the Perl code is so much slower and uses this enormous
> amount of RAM?
This doesn't make much sense - the Perl bindings should just be a thin
wrapper around the C++ code, so it's strange that they are so much slower.

All I can think is that either the two versions aren't actually equivalent,
or there's something lazily not done in the C++ version which the Perl
bindings force to be done, or you're pulling a lot of data out and that
ends up adding a lot of overhead in Perl.

Can you post the code for the Perl and C++ versions of the searcher?

Cheers,
    Olly