Xapian discuss - Feb 2008 - problem compiling static python bindings

Hello -

I'm attempting to compile xapian bindings for python for use on a
shared (university-based) web server, so I've had to compile
everything (python, xapian-core) in my home directory using
"--prefix=$HOME/public_html/local". Here's some system info:

uname -a
... i686 i686 i386 GNU/Linux

gcc --version
gcc (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-59)

$HOME/public_html/local/bin/python2.5
Python 2.5.1 (r251:54863, Feb  5 2008, 13:48:55)
[GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-59)] on linux2

Compiling normally:

wget http://www.oligarchy.co.uk/xapian/1.0.5/xapian-bindings-1.0.5.tar.gz
tar -xzvf xapian-bindings-1.0.5.tar.gz
cd xapian-bindings-1.0.5
export XAPIAN_CONFIG=$HOME/public_html/local/bin/xapian-config
export PYTHON=$HOME/public_html/local/bin/python2.5
./configure --prefix=$HOME/public_html/local --with-python && make
&&
make install

This completes without errors and results in the following:

ldd $HOME/public_html/local/lib/python2.5/site-packages/_xapian.so
        libxapian.so.15 =>
/rc21/d75/labmed1/public_html/local/lib/libxapian.so.15 (0x00cf4000)
        libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x009b1000)
        libm.so.6 => /lib/tls/libm.so.6 (0x0075a000)
        libc.so.6 => /lib/tls/libc.so.6 (0x00111000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00fd0000)
        libz.so.1 => /usr/lib/libz.so.1 (0x006d6000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00ad2000)

However, I believe that I need to compile the python bindings
statically, because attempts to import xapian from the cgi script (I'm
using MoinMoin - moin.cgi, for what it's worth) result in an error:

libxapian.so.15: cannot open shared object file: No such file or directory

I've tried to set LD_LIBRARY_PATH from within the script (using
os.environ['LD_LIBRARY_PATH'] = some_val) to every possible absolute
or relative path that I can think of, but I get the same error. But in
any case, I don't think that the cgi script has access to the
filesystem where libxapian.so.15 is located.

When I try to compile statically, I get an error:

make distclean
./configure --prefix=$HOME/public_html/local --with-python
--enable-shared=no --enable-static=yes && make && make install

...
checking whether to build shared libraries... no
checking whether to build static libraries... yes
...

make[4]: Entering directory
`/nfs/aesop02/rc21/d75/labmed1/src/xapian-bindings-1.0.5/python'
/bin/sh ../libtool --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I.
-I..  -I/rc21/d75/labmed1/public_html/local/include/python2.5
-fno-strict-aliasing -Wall -Wno-unused -Wno-uninitialized
-I/rc21/d75/labmed1/public_html/local/include -g -O2 -MT
xapian_wrap.lo -MD -MP -MF .deps/xapian_wrap.Tpo -c -o xapian_wrap.lo
`test -f 'modern/xapian_wrap.cc' || echo
'./'`modern/xapian_wrap.cc
 g++ -DHAVE_CONFIG_H -I. -I..
-I/rc21/d75/labmed1/public_html/local/include/python2.5
-fno-strict-aliasing -Wall -Wno-unused -Wno-uninitialized
-I/rc21/d75/labmed1/public_html/local/include -g -O2 -MT
xapian_wrap.lo -MD -MP -MF .deps/xapian_wrap.Tpo -c
modern/xapian_wrap.cc -o xapian_wrap.o
mv -f .deps/xapian_wrap.Tpo .deps/xapian_wrap.Plo
/bin/sh ../libtool --tag=CXX   --mode=link g++ -fno-strict-aliasing
-Wall -Wno-unused -Wno-uninitialized
-I/rc21/d75/labmed1/public_html/local/include -g -O2 -avoid-version
-module   -o _xapian.la -rpath
/rc21/d75/labmed1/public_html/local/lib/python2.5/site-packages
xapian_wrap.lo -L/rc21/d75/labmed1/public_html/local/lib -lxapian
-lstdc++
mkdir .libs
ar cru .libs/_xapian.a  xapian_wrap.o
ranlib .libs/_xapian.a
creating _xapian.la
(cd .libs && rm -f _xapian.la && ln -s ../_xapian.la _xapian.la)
cp `test -f modern/xapian.py || echo './'`modern/xapian.py .
PYTHONPATH=".:.libs" /rc21/d75/labmed1/public_html/local/bin/python2.5
-c "import _xapian;import xapian"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named _xapian
make[4]: *** [xapian.pyc] Error 1
make[4]: Leaving directory
`/nfs/aesop02/rc21/d75/labmed1/src/xapian-bindings-1.0.5/python'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory
`/nfs/aesop02/rc21/d75/labmed1/src/xapian-bindings-1.0.5/python'
make[2]: *** [all] Error 2
make[2]: Leaving directory
`/nfs/aesop02/rc21/d75/labmed1/src/xapian-bindings-1.0.5/python'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory
`/nfs/aesop02/rc21/d75/labmed1/src/xapian-bindings-1.0.5'
make: *** [all] Error 2

Can anyone suggest a way to make things work given this information?
I'm at the limit of my understanding here.

Thanks.

On Tue, Feb 05, 2008 at 05:09:44PM -0800, Noah Hoffman wrote:
> I'm attempting to compile xapian bindings for python for use on a
> shared (university-based) web server, so I've had to compile
> everything (python, xapian-core) in my home directory using
> "--prefix=$HOME/public_html/local".
It's a bad idea to put this kind of stuff inside your web
root. $HOME/local would be a better choice (unless your CGIs run
inside a chroot or jail, which is unlikely).
> However, I believe that I need to compile the python bindings
> statically, because attempts to import xapian from the cgi script (I'm
> using MoinMoin - moin.cgi, for what it's worth) result in an error:
> 
> libxapian.so.15: cannot open shared object file: No such file or directory
Hmm. Where is libxapian.so.15? Is there any documentation on strange
restrictions provided by the CGI mechanism?
> When I try to compile statically, I get an error:
> 
> make distclean
> ./configure --prefix=$HOME/public_html/local --with-python
> --enable-shared=no --enable-static=yes && make && make
install
I'm not convinced this is the right approach. It's not that you
don't
want to build static libraries, it's that you want to link against the
static *xapian* library. Try building xapian-core with
--enable-shared=no --enable-static=yes and then build the bindings
normally.

J

-- 
/--------------------------------------------------------------------------\
  James Aylett                                                  xapian.org
  james at tartarus.org                               uncertaintydivision.org

On Tue, Feb 05, 2008 at 05:09:44PM -0800, Noah Hoffman
wrote:
> ldd $HOME/public_html/local/lib/python2.5/site-packages/_xapian.so
>         libxapian.so.15 =>
> /rc21/d75/labmed1/public_html/local/lib/libxapian.so.15 (0x00cf4000)
That's good - libtool has set an rpath so libxapian.so.15 can be found.
> However, I believe that I need to compile the python bindings
> statically, because attempts to import xapian from the cgi script (I'm
> using MoinMoin - moin.cgi, for what it's worth) result in an error:
We don't support building xapian-bindings with a static xapian-core.
For more information see:

http://xapian.org/cgi-bin/bugzilla/show_bug.cgi?id=187
> I've tried to set LD_LIBRARY_PATH from within the script (using
> os.environ['LD_LIBRARY_PATH'] = some_val) to every possible
absolute
> or relative path that I can think of, but I get the same error. But in
> any case, I don't think that the cgi script has access to the
> filesystem where libxapian.so.15 is located.
You shouldn't need to set LD_LIBRARY_PATH - the rpath should do the
trick, and has for me in the past on Linux.  It would be interesting
to work out why this doesn't seem to work.

But if you do set it, you may need to set it before invoking the python
script using a shell script wrapper.

For example:

    #!/bin/sh
    LD_LIBRARY_PATH=/rc21/d75/labmed1/public_html/local/lib
    export LD_LIBRARY_PATH
    exec python /path/to/script.py

Cheers,
    Olly