Francesco - I'm forwarding this on to the xapian-discuss mailing list, which is the best place to get help on this sort of thing. Apologies that this wasn't obvious from our website! On Fri, Mar 25, 2005 at 10:50:27AM -0800, Francesco Garosi wrote:> I'm interested in Xapian as a possible search engine for a > Python project I'm developing. The reason why I write to you, > is that I saw that you've been fixing bugs... so please excuse > me if I'm bothering you: I saw no other contact on the > xapian.org web pages.Olly, and chance of a standard footer across xapian.org which includes a link 'Contact' => lists.php or similar. Also, on lists.php, s/general discussions/general questions and discussions/ or similar? On to the issue itself:> I've been spending the last two days trying to set up Python > bindings for the library on Win32 with MinGW/MSYS. I know, > it's probably not your main platform, but someone might need > it as I do, for whatever reason. The reason why it fails, I think, > is that Xapian::Remote::open is probably not supported on > Win32.The network backend isn't supported on Win32 as I understand it, although that's not noted in the PLATFORMS file. It may be that these days it's been reported to work and I just haven't noticed, however :-). Which version of MSYS/MingW are you using? Which GCC version?> When I realized it, I finally managed to exclude from the > (provided) SWIG-generated xapian_wrap.cc the unsupported > lines, and to "gently" raise an exception when remote_open() > is called.[snip patch] If the remote backend doesn't work on mingw, we should detect this in the bindings configure and disable the wrappers automatically, I think. I'll forward the entire original message to the list in a minute (sorry, this will break threading I suspect :-(. J -- /--------------------------------------------------------------------------\ James Aylett xapian.org james@tartarus.org uncertaintydivision.org
Message body follows:
Hi,
I'm interested in Xapian as a possible search engine for a
Python project I'm developing. The reason why I write to you,
is that I saw that you've been fixing bugs... so please excuse
me if I'm bothering you: I saw no other contact on the
xapian.org web pages.
I've been spending the last two days trying to set up Python
bindings for the library on Win32 with MinGW/MSYS. I know,
it's probably not your main platform, but someone might need
it as I do, for whatever reason. The reason why it fails, I think,
is that Xapian::Remote::open is probably not supported on
Win32.
When I realized it, I finally managed to exclude from the
(provided) SWIG-generated xapian_wrap.cc the unsupported
lines, and to "gently" raise an exception when remote_open()
is called. Here is the patch for
python/modern/xapian_wrap.cc:
--- xapian_wrap.cc_ORIG Fri Mar 25 16:42:49 2005
+++ xapian_wrap.cc Fri Mar 25 17:53:18 2005
@@ -15350 +15350,2 @@
-
+// remote_open should be removed to wrap in
Win32/MinGW32
+#ifndef NO_REMOTE_OPEN
@@ -15597,0 +15599,6 @@
+#else
+static PyObject *_wrap_remote_open(PyObject *self,
PyObject *args) {
+ PyErr_SetString(PyExc_NotImplementedError,"No
matching function for 'remote_open'");
+ return NULL;
+}
+#endif // NO_REMOTE_OPEN
(I hope, the web mail program will not break too many lines).
This patch works in conjunction with a Python distutils
styled "setup.py" script, that follows:
#!/usr/bin/env python
# setup file for xapian
import os, sys
from distutils.core import setup, Extension
XAPIAN_BASE = ""
try:
XAPIAN_BASE = os.environ['XAPIAN_BASE']
except KeyError:
print "warning: XAPIAN_BASE not set, use xapian-config
to determine it"
# please do not change the lines below
EXTRA_LDFLAGS = (
"-v -Wl,--enable-runtime-pseudo-reloc "
"-L%(xapianbase)s/lib -lstdc++ "
"-lxapianqueryparser -lxapian "
% {'xapianbase': XAPIAN_BASE}).split()
EXTRA_CFLAGS = (
"-I%(xapianbase)s/include "
% {'xapianbase': XAPIAN_BASE}).split()
DATA_FILES = []
if sys.platform == "win32":
df = (
"%(xapianbase)s/bin/libxapianqueryparser-5.dll "
"%(xapianbase)s/bin/libxapian-5.dll "
% {'xapianbase': XAPIAN_BASE}).split()
DATA_FILES = [('.', df)]
EXTRA_CFLAGS.append("-DNO_REMOTE_OPEN")
setup(name="Xapian",
version="0.8.5",
description="Xapian bindings for Python",
author="The Xapian Team",
author_email="xapian@users.sourceforge.net",
url="http://xapian.org/",
py_modules=["xapian"],
data_files=DATA_FILES,
ext_modules=[
Extension("_xapian",
["xapian_wrap.cc"],
extra_compile_args=EXTRA_CFLAGS,
extra_link_args=EXTRA_LDFLAGS,
)]
)
# end.
...if a user wants to build it using the standard Python build
system, he just has to patch the .cc file, copy this script in
the python/modern directory, set the XAPIAN_BASE
directory to where Xapian is installed and launch "python
setup.py build -c mingw32 install" from a command line
there. Still don't know how to make it work on other
platforms - the provided build system should work perfectly.
I hope this will help someone if you disclose it...
Thank you for releasing the library as open source,
F.
>>@
Risposte/Replies :
Francesco Garosi
Office: <f.garosi@usl7.toscana.it> (++39/0577/586937)
Home: <franz.g@infinito.it> (++39/0577/47806)
--
This message has been sent to you, a registered SourceForge.net user,
by another site user, through the SourceForge.net site. This message
has been delivered to your SourceForge.net mail alias. You may reply
to this message using the "Reply" feature of your email client, or
using the messaging facility of SourceForge.net at:
https://sourceforge.net/sendmessage.php?touser=328337
Hi Olly,> You need fork, socketpair, gethostbyname and gethostbyaddr > for the remote backend. And if any of these are missing, > configure will disable the remote backend automatically.you are right: AFAIK MinGW neither supports fork() nor is intended to... Cygwin is more an emulation layer, thus many more system calls or posix/unix functions are supported. But for what I could see, the Xapian library *correctly* disables network backend support on Win32/MinGW (btw, I'm using MinGW version 3.1.0, but I also upgraded gcc to 3.4.2). In fact ld reports that Xapian::Remote::open cannot be resolved when linking the high-level language bindings against libxapian*.a. The only problem with the bindings is that they are built with SWIG, and I really do not know it... otherwise I'd be glad to try to contribute a patch that directly affects the interface files, in order to always disable remote backend support on native MinGW. However I'll try to do my best to sort it out ;-) (don't expect much: I've been avoiding C/C++ for years and years when I switched to Python). There are some other problems, however, with the xapian-bindings configure script and generated makefiles, at least for Python: in fact, it assumes that Python is installed "the UNIX way", that is it assumes that libraries are in $PYTHON_PREFIX/lib/python2.3 and so on. That is why I actually submitted a "setup.py" file for Python, which "incidentally" allows building on all Python supported platforms. However I realize that this is not the Xapian distribution style, that tries to possibly build bindings for all available languages at once. I think that it's not going to be a picnic to allow to build everything (with the said limitations) natively on Win32 with MinGW. Thank you again for replying, F.>>@Risposte/Replies : Francesco Garosi Office: <f.garosi@usl7.toscana.it> (++39/0577/586937) Home: <franz.g@infinito.it> (++39/0577/47806)
Hi Olly, sincerely I forgot to consider that there are other platforms, like Mac OS 9 and earlier, that may have different build systems and installation layouts. So I only focused the setup.py script on UNIX (which is straightforward) and Win32 (which requires some extra options). The script could be probably avoided, however it can be helpful in providing the compiler the necessary options to correctly compile for the destination platform. And, regarding your question, yes: it can be used to produce "installers": RPM on Linux, EXE on Windows and binary tarballs on other platforms. Of course on Windows InnoSetup installers are nicer and can achieve the same result. I'm trying right now to learn as quick as possible how to use the Autoconf/Automake tools in order to try to help in finding a solution to make the bindings more portable (requiring MSYS along with MinGW on Win32 platforms). I'd like to contribute, even though I think it will require some time. Unfortunately I only use Python as a scripting language -- well, nowadays I use it also for applications, almost abandoning C and C++ -- so I probably will not be of much help for other bindings. Anyway I'll do my best to give some help, if I can. Cheers, F.
A few thoughts:> #!/usr/bin/env python > # setup file for xapian > > import os, sys > from distutils.core import setup, Extension > > XAPIAN_BASE = "" > try: > XAPIAN_BASE = os.environ['XAPIAN_BASE'] > except KeyError: > print "warning: XAPIAN_BASE not set, use xapian-config > to determine it" > > # please do not change the lines below > EXTRA_LDFLAGS = ( > "-v -Wl,--enable-runtime-pseudo-reloc " > "-L%(xapianbase)s/lib -lstdc++ " > "-lxapianqueryparser -lxapian " > % {'xapianbase': XAPIAN_BASE}).split()Needing to pass --enable-runtime-pseudo-reloc is a win32 specific oddity, and I suspect passing it elsewhere will cause errors. But we can just add that on in the win32 case below. Also -lxapianqueryparser is no longer required - just using `xapian-config --libs` will sort that out and eliminate the need for XAPIAN_BASE here. The option -lstdc++ is GCC specific - currently there shouldn't be any problem with building the bindings with other C++ compilers. Poking around inside the python libraries, it looks like they should notice this is C++ code and link with g++, so I wonder why you needed this. By not using libtool to build the glue library, we force installing of the Xapian library itself before the bindings can be built. That's not a problem if you're just using Xapian, but annoying if you're developing Xapian. So for UNIX, I think it's probably better to stick to the current approach.> [snip] > DATA_FILES = [] > if sys.platform == "win32": > df = ( > "%(xapianbase)s/bin/libxapianqueryparser-5.dll " > "%(xapianbase)s/bin/libxapian-5.dll " > % {'xapianbase': XAPIAN_BASE}).split()We can't go hard coding the library versioning information here or it will get out of step (also, only libxapian is now needed). Also the "split()" approach will cause problems if XAPIAN_BASE contains spaces which is pretty common on Windows (it'll be the case if Xapian is installed in "Program Files").> [snip]Cheers, Olly