Xapian discuss - Feb 2006 - xapian-tcpsrv very slow

Hi,

I have been looking at xapian for a project and have so far been very 
impressed.  I have hit one major problem the final architecture requires 
that the web server be a deferent machine from the server that xapian is 
installed on.  With my test code using local database ($db_path = 
"/var/xapain_db/default";    $db = new_database($db_path);) a query
takes
around 0.012 seconds on the same machine with the connection changed to use 
tcp ($db = remote_open('127.0.0.1',3055);) the same query takes 129
seconds
or times out.   I am currently trying to follow tcpsrv via the debuger 
however so far everything looks reasonable !

Any ideas would be well received.

environment: HP G4 2X3.6Ghz 2Gb ram, mandrake 2005, xapian 0.9.2

Roger


-- 
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.1.375 / Virus Database: 267.15.2/252 - Release Date: 06/02/2006

On Wed, Feb 08, 2006 at 03:15:36PM +0000, Roger Fishwick
wrote:
> I have been looking at xapian for a project and have so far been very 
> impressed.  I have hit one major problem the final architecture requires 
> that the web server be a deferent machine from the server that xapian is 
> installed on.  With my test code using local database ($db_path = 
> "/var/xapain_db/default";    $db = new_database($db_path);) a
query takes
> around 0.012 seconds on the same machine with the connection changed to use
> tcp ($db = remote_open('127.0.0.1',3055);) the same query takes 129
seconds
> or times out.
I'd expect some overhead from the remote link, but not that much.  And
you're using loopback so it's not a slow network.

The webtop guys used xapian-tcpsrv to search 500 million documents with
sub-second query times, and that involved a real network (albeit a
fast one).  It's possible something has changed in the code which
is causing a problem now though.  Nobody's using it heavily that I'm
aware of.

Hmm, are you using Omega here?  If so, did you disable $topterms from
the query template?
> Any ideas would be well received.
An alternative approach is to run Omega on the xapian machine searching
databases locally, and use the xml template (or something similar) to
return XML results which the web server machine reads and formats for
display.
> While I think of it socketserver.cc is missing include of iostream if 
> TIMING_PATCH is defined.
Thanks, I'll add it.

Cheers,
    Olly