Xapian discuss - Dec 2005 - Building Xapian and Python Bindings for Windows: Solved

Hello,
I have finally managed to build xapian-core 0.9.2 and xapian python 
bindings for Windows/Python 2.3. Here is a summary of what I did, maybe 
it could be helpful for someone else, who knows.

The following posts where especially useful:
- http://thread.gmane.org/gmane.comp.search.xapian.general/1645 : 
win32/find.exe vs unix/find
- http://thread.gmane.org/gmane.comp.search.xapian.devel/600 : 
pythonXY.a generation and other SWIG tricks.

It is likely some problems I had came from previous tools setup quirks, 
but I cannot tell for sure. I will add '*' before issues I did not find 
anywhere else, maybe someone will know.

xapian-core
-----------
- Compilation platform is WinXP SP2 on IA-32 (Athlon something). I have 
MSVC 7.1 and other unrelated compilation tools already installed.

- Setup cygwin, mingw-4.1.0 and msys-1.0.10. Added all the bin/ 
subdirectories to PATH in this order, before system paths (to solve the 
find.exe mismatch problem)

- Retrieve and extract "xapian-core-0.9.2"

*- Update the root makefile to take in account the backend archive 
targets. It means creating a variable like:
"""
BACKENDS_LA =   backends/quartz/libquartz.la \
                 backends/flint/libflint.la \
                 backends/multi/libmulti.la \
                 backends/inmemory/libinmemory.la
"""
and adding it la_DEPENDENCIES and la_LIBADD actions in makefile.in.
It seems dependency resolution fails to work recursively with dependent 
archives, which happens with the backends: xapian-core <- libbackend <- 
[all backends archives]. So I resolved it manually. I suppose it would 
be better to edit the Makefile.am and let automake do everything but 
when I tried it, "make" seems to run forever so, being rather ignorant
about the GNU toolchain I ended changing the .in instead. Without these, 
the build almost works but ends with missing references when trying to 
link the shared library.

- Open a prompt on xapian-core root directory, started a cygwin sh and 
run "./configure". Then run "make install". Cygwin
/usr/local being not
the same than the MSYS /usr/local, move all the stuff which was 
installed in the second to the first.


Python bindings
---------------
- Instruction below assumes python is setup in "d:/python23".
- Build the libpython23.a as explained in 
http://sebsauvage.net/python/mingw.html [2. Create libpython22.a]. 
Basically, find the python DLL and run:
"""
 > pexports python23.dll > pythonxy.def
 > dlltool --dllname python23.dll --def python23.def --output-lib 
libpython23.a
"""
and move the libpython23.a into Python lib/ directory.

*- Run configure for python bindings:
"""
 > ./configure XAPIAN_CONFIG=/usr/local/bin/xapian-config --without-php 
--without-tcl --without-java --without-guile --without-csharp
"""
Note: at this point configure outputs something like
"""
checking for python... /cygdrive/d/Python23//python
"""
Which seems wrong and will cause some problems later.

- Run make:
"""
 > make install PYTHON_LIBS='-Ld:/python23/libs -lpython23'
"""
Again, make almost works until:
"""
PYTHONPATH=".;.libs" /cygdrive/d/Python23//python -c "import
xapian"
/bin/sh: /cygdrive/d/Python23//python: No such file or directory
"""
The python path detected by configure is definitely wrong. Do not worry, 
it would not have worked with a good one since the installer also miss 
cygwin xapian DLL. Instead, install it manually: copy "xapian.py", 
"_xapian.dll" and "cygxapian-8.dll" in python package path,
then try to
import "xapian" in an interpreter, and run the
"smoketest.py" test.
- Enjoy !


Finally, you can try a still in development higher level wrapper written 
by DivMod here: http://divmod.org/trac/wiki/DivmodXapwrap. Because it 
would be just too easy, it does not work out of the box for Windows, you 
will have to rewrite a custom version of FilesystemLock (using Win32 
named mutexes for example), otherwise it is okay. I will try to send 
them a patch about that.

--
Patrick M?zard

On Tue, Dec 27, 2005 at 10:55:52PM +0100, Patrick M?zard
wrote:
> I have finally managed to build xapian-core 0.9.2 and xapian python 
> bindings for Windows/Python 2.3. Here is a summary of what I did, maybe 
> it could be helpful for someone else, who knows.
Thanks.  I'll work through these and try to fix things so they just work
out of the box, or failing that document the issues.
> The following posts where especially useful:
> - http://thread.gmane.org/gmane.comp.search.xapian.general/1645 : 
> win32/find.exe vs unix/find
That should no longer be an issue in 0.9.2 - I redid the code which used
find and it now looks at the contents of the Makefile.am-s instead.

Did you actually hit this issue (in which case something's amiss)?  Or
did you just read about it and take steps to avoid it?

Cheers,
    Olly

Patrick M?zard a ?crit :
> Finally, you can try a still in development higher level wrapper 
> written by DivMod here: http://divmod.org/trac/wiki/DivmodXapwrap. 
> Because it would be just too easy, it does not work out of the box for 
> Windows, you will have to rewrite a custom version of FilesystemLock 
> (using Win32 named mutexes for example), otherwise it is okay. I will 
> try to send them a patch about that.
FYI, the patch is sent and waiting here:
http://divmod.org/trac/ticket/504

Patrick M?zard

On Tue, Dec 27, 2005 at 10:55:52PM +0100, Patrick M?zard
wrote:
> *- Update the root makefile to take in account the backend archive 
> targets. It means creating a variable like:
> """
> BACKENDS_LA =   backends/quartz/libquartz.la \
>                 backends/flint/libflint.la \
>                 backends/multi/libmulti.la \
>                 backends/inmemory/libinmemory.la
> """
> and adding it la_DEPENDENCIES and la_LIBADD actions in makefile.in.
I suspect that this is a consequence of the "wrong find.exe" problem.
I'm just putting together a mail to send to the libtool list.
> checking for python... /cygdrive/d/Python23//python
This is odd - if that's not a python interpreter then the code where we
check the python version should fail to match the patterns for the
python versions we support and configure should disable the python
bindings.

I wonder if PYTHON is set in your environment - does "echo $PYTHON"
output anything?

Is there anything at all at the path "/cygdrive/d/Python23//python"?
What does "ls -l /cygdrive/d/Python23" output?
> The python path detected by configure is definitely wrong. Do not worry, 
> it would not have worked with a good one since the installer also miss 
> cygwin xapian DLL.
It should work - configure looks for xapian-config which should report
where the xapian library is installed.  If you've installed it using a
prefix such that xapian-config isn't on your PATH, then you need to tell
xapian-binding's configure where xapian-config is like so:

./configure XAPIAN_CONFIG=/path/to/xapian-config

Cheers,
    Olly