Deron Meranda
2007-Aug-28 21:01 UTC
[Xapian-discuss] flintlock fork causes hang in Apache+Python+mod_python
I am trying to use Xapian 1.0.2 with the Python SWIG bindings withn an environment consisting of Apache httpd with mod_python. (not as a CGI) Also this is Linux. Whenever the python code attempts to open a database the entire httpd process will hang indefinitely. The python bindings work outside of the apache/mod_python environment.>From the best I can tell the hang occurs in FlintLock:lock()method. I can see that the httpd process has forked off a child using the "/bin/cat" executable, and the httpd process (which has the embedded Python interpreter) is hung on a FUTEX_WAIT system call). While this process is hung, then attempts to open the same database outside of Apache sometimes result in an error saying that the database lock could not be obtained because it is held by another process. Has anybody experienced this problem? I've never seen a process locking mechanism before which uses this fork/cat style mechanism. Is it possible that this could be confusing the python interpreter GIL or something? Deron Meranda
Richard Boulton
2007-Aug-28 23:45 UTC
[Xapian-discuss] flintlock fork causes hang in Apache+Python+mod_python
Deron Meranda wrote:> I am trying to use Xapian 1.0.2 with the Python SWIG > bindings withn an environment consisting of Apache > httpd with mod_python. (not as a CGI) Also this is > Linux. > > Whenever the python code attempts to open a database > the entire httpd process will hang indefinitely. The > python bindings work outside of the apache/mod_python > environment.I think this is probably a slightly different problem which was identified on the list (or maybe it was just on the bug tracker) a little while ago. I added a piece of documentation on how to work around it to the SVN tree a little while ago, but it's not in a released version yet. So you don't have to check out SVN, the documentation I wrote is as follows: There is an issue with python's threading support and mod-python, which can cause problems when using the Xapian Python bindings from mod-python. If you find that your mod-python application is hanging whenever a Xapian method is called, you have probably run into this problem. By default, mod-python uses a separate instance of the python interpreter for each mod-python application. However, the Xapian python bindings do not know how to get and release control of the thread lock unless they are running in the "main" python interpreter; as a result, deadlocks can occur. To avoid the problem, you need to set the "PythonInterpreter main_interpreter" option in the apache configuration section for all mod_python scripts which use xapian: this ensures that the xapian bindings are called from the main_interpreter, and can handle the thread locks appropriately. An example configuration file using this option is available <a href="http://www.xapian.org/cgi-bin/bugzilla/attachment.cgi?id=121&action=view">here</a>. In the longer term, we hope to be able to resolve this problem, so that arbitrary python interpreters can be used; but this is quite a tricky problem to fix, and is relatively low priority. More details can be found in Xapian's bug tracker, as bug #185: http://www.xapian.org/cgi-bin/bugzilla/show_bug.cgi?id=185 -- Richard
Maybe Matching Threads
- mod_python - apache crash
- [Fwd: CentOS-3 i386 errata: Updated mod_python package fixes security issue]
- CentOS-3 i386 errata: Updated mod_python package fixes security issue
- CentOS-3 i386 errata: Updated mod_python package fixes security issue (resent)
- CentOS-3 x86_64 errata: Updated mod_python package fixes security issue