Xapian discuss - Dec 2006 - Xapian JNI questions

I am using Xapian JNI layer for a while and I am interested in
status of Xapian JNI, documentation, bug lists, references, etc.

Specifically,  is the following correct:
1)  Writable database object must be finalized to release writable
database lock (otherwise database could  become unreadable if
application is restarted (after crash ) ?

2)  finalize method must be called  on any Xapian JNI onject when its
no longer needed ?


I have also encountered  a few bugs

3) the following function in org.xapian.ESetIterator  always returns false.

 public boolean hasNext() {
       try {
           return XapianJNI.esetiterator_equals(id, _end_id);
       } catch (XapianError xe) {
           throw new XapianRuntimeError(xe);


When changed to

 public boolean hasNext() {
       try {
           return !XapianJNI.esetiterator_equals(id, _end_id);
       } catch (XapianError xe) {
           throw new XapianRuntimeError(xe);
       }
   }

it seems to be working correctly.



4) Attempts to use MatchDecider caused an exception:

 Exception in thread "Thread-37" java.lang. accept
       at org.xapian.XapianJNI.enquire_register_match_decider(Native Method)
       at org.xapian.Enquire.registerMatchDecider(Enquire.java:127)
       at
tv.cozmo.neo.logic.search.YouTubeShowIndexManager.isIndexed(YouTubeShowIndexManager.java:286


The JNI code is as follows:

       JavaMatchDecider(JNIEnv *xenv, jclass xclazz, jobject &obj) :
MatchDecider() {
           env = xenv;
           clazz = xclazz;
           _obj = obj;
           TRY
               _deciderclass = env->GetObjectClass(obj);
               if (!_deciderclass) goto on_error;

               _acceptmethod = env->GetMethodID(_deciderclass,
"accept", "(Lorg/xapian/Document;)I");
               if (!_acceptmethod) goto on_error;

on_error:
           CATCH(;)
       }

Since the return type for accept method is boolean, I have replaced
"(Lorg/xapian/Document;)I" with "(Lorg/xapian/Document;)Z" (
and the
same for JavaExpandDecider ).

With this change the NoSuchMethodError exception is no longer thrown,
however, it looks like
the JavaMatchDecider is not called at all.



Regards & Thanks,

Alex Kushkuley

On Wed, Dec 06, 2006 at 04:21:30PM -0500, Alex Kushkuley
wrote:
> I am using Xapian JNI layer for a while and I am interested in
> status of Xapian JNI, documentation, bug lists, references, etc.
The Java bindings are lagging behind the others at the moment.  The
problem is that it's trickier and takes longer to update the JNI code by
hand than it does to update all the SWIG based languages together.

The plan is to move Java from hand-coded JNI to SWIG.  I did a quick
hack set of SWIG generated Java wrappers, and the size was actually
smaller than what we currently have, though I suspect there were a
few bits missing and they'd end up being pretty much the same.

Because of this, I've tended to put off major work on Java recently,
though I'm certainly happy to consider patches.
> Specifically,  is the following correct:
> 1)  Writable database object must be finalized to release writable
> database lock (otherwise database could  become unreadable if
> application is restarted (after crash ) ?
A Xapian database should never become unreadable after a crash (unless
the crash actually scrambles the disk contents in which case nothing would
be immune!) though unflushed changes may be lost.

For a quartz database (currently the default), you need to ensure the
C++ destructor gets called or the database lock won't be released.  You
can just remove the lock file by hand if you're certain that the process
which created it is no longer running.

Flint uses a different locking strategy such that the lock is
automatically released when the process dies.

For both Quartz and Flint, you need to either ensure the destructor
gets called, or explicitly call flush() on each WritableDatabase before
you exit.  It's a good idea to explicitly call flush() anyway, as it
allows you to deal with exceptions better.

So *if* Java doesn't reliably call the C++ destructor, you should use
flint instead of quartz (see http://wiki.xapian.org/FlintBackend for
details of how to do this) and make sure you call flush() on each
WritableDatabase before you exit.

I say "*if*" because I don't know Java and JNI well enough to know
what the
situation is here.  I vaguelly recall that Eric Ridge (who wrote the JNI
bindings originally) once said something about problems with finalisers getting
called, but I can't find the mail.  I think he said there's a Java API
call
which addresses this problem, but that the Sun documentation says it's buggy
and should be avoided or something like that.  Without the mail it's hard to
be sure I'm not misremembering though.
> 2)  finalize method must be called  on any Xapian JNI onject when its
> no longer needed ?
Currently at least, it shouldn't really matter for anything except
WritableDatabase.
> 3) the following function in org.xapian.ESetIterator  always returns false.
Thanks, fixed.

Note that it's much easy for me to apply changes if you send a patch
rather than describing them in English!  You can generate a patch by
keeping a copy of the original and using "diff -u ESetIterator.java.orig
ESetIterator.java" or similar).  Or if working from SVN, just use
"svn diff".
> Since the return type for accept method is boolean, I have replaced
> "(Lorg/xapian/Document;)I" with
"(Lorg/xapian/Document;)Z" ( and the
> same for JavaExpandDecider ).
I've committed these two changes.  But when I tried to write a regression
test
for ExpandDecider, but I got:

Caught unexpected exception java.lang.NullPointerException

So I'm not sure this is enough to make ExpandDecider work.
> With this change the NoSuchMethodError exception is no longer thrown,
> however, it looks like the JavaMatchDecider is not called at all.
I notice you're calling Enquire::register_match_decider() - this
registers a name for a match decider so it can work with the remote
backend (though I don't think the remote backend side is implemented
yet).

If you want to just use a MatchDecider, you need to subclass it and pass an
instance of your subclass to get_mset() like this (this is a pointless
matchdecider which rejects everything):

    class MyMatchDecider implements MatchDecider {
	public boolean accept(Document d) { return false; }
    }

    // ...

    MSet mset = enquire.getMSet(0, 10, null, new MyMatchDecider());

However, this doesn't work for me:

    Exception in thread "main" java.lang.NoSuchMethodError:
<init> (J)
       at org.xapian.Enquire.getMSet(Enquire.java:107)
       at SmokeTest.main(SmokeTest.java:116)
    FAIL: SmokeTest.class

Looking at the code, I think it's trying to find the constructor for
org.xapian.Document as "<init> (J)".  Is that the right thing to
look
for?

Cheers,
    Olly