Xapian discuss - May 2004 - "Error reading block xxx: got end of file"

Xapian (0.7.5) is spitting out this error on a regular basis:

org.xapian.errors.DatabaseError: Error reading block 136618: got end of=20
file
  =A0=A0=A0=A0=A0=A0=A0 at=20
org.xapian.XapianJNI.writabledatabase_repalce_document(Native Method)
  =A0=A0=A0=A0=A0=A0=A0 at=20
org.xapian.WritableDatabase.replaceDocument(WritableDatabase.java:67)

I don't have a gdb backtrace, only the Java stacktrace.  :(

What would cause this "got end of file" error?  The system on which=20
this is running is currently running in "single user mode".  Our
fti=20
stuff is threaded, but we only have 1 writer thread (b/c Quartz doesn't=20
support multiple writers), so I feel pretty good about this not being a=20
concurrency issue of some kind.

I'd provide more details, but I'm not exactly sure what is relevant.

eric

On Tue, May 11, 2004 at 01:37:48PM -0400, Eric Ridge
wrote:
> Xapian (0.7.5) is spitting out this error on a regular basis:
> 
> org.xapian.errors.DatabaseError: Error reading block 136618: got end of 
> file
>          at 
> org.xapian.XapianJNI.writabledatabase_repalce_document(Native Method)
"repalce"?  Is this retyped, or is there really a typo in the JNI
method
name?
>          at 
> org.xapian.WritableDatabase.replaceDocument(WritableDatabase.java:67)
> 
> I don't have a gdb backtrace, only the Java stacktrace.  :(
> 
> What would cause this "got end of file" error?#
Quartz tried to read a block off the end of the Btree file.
> I'd provide more details, but I'm not exactly sure what is
relevant.
First thing to do is to run "quartzcheck" on the database to see if
the
btree's are consistent.

If they are, I need to be able to reproduce this - if you can come up with
a small C++ example, that'd help a lot.

It's possible it's fixed in 0.8.0 - I can't see anything in the
ChangeLog, but quartz changed in a major way from 0.7.5 to 0.8.0...

Cheers,
  Olly

On 5/11/04 1:37 PM, "Eric Ridge" <ebr@tcdi.com> wrote:
> Xapian (0.7.5) is spitting out this error on a regular basis:
> 
> org.xapian.errors.DatabaseError: Error reading block 136618: got end of
file
We've since upgraded to 0.8.0, and completely rewritten our backend indexing
subsystem to eliminate possible concurrent writes and to improve
performance.

Sadly, we're still getting this error.  Again, new version of Xapian, and
new backend code.  :(

It's happening in .replace_document():

org.xapian.errors.DatabaseError: Error reading block 1909: got end of file
   at org.xapian.XapianJNI.writabledatabase_repalce_document(Native Method)
   at org.xapian.WritableDatabase.replaceDocument(WritableDatabase.java:67)

I've been trying to reproduce this standalone, but haven't been able to
break it.  I'm still unclear as to what can cause this error, so I'm not
sure how to test for it.

There must be some corner case in replace_document(), but I can't figure out
how to trigger it.  Any suggestions?

"quartzcheck" checks out okay.  See below.

eric

$ quartzcheck ./backup_index
record:
base66  Revision *81857  levels 2  root [2]  blocksize 8192  items 79379
lastblock 2731
B-tree checked okay
postlist:
base66  Revision *81857  levels 2  root [3]  blocksize 8192  items 137754
lastblock 1822
B-tree checked okay
Postlist structure checked OK
termlist:
base66  Revision *81857  levels 2  root [778]  blocksize 8192  items 79378
lastblock 3675
B-tree checked okay
position:
base66  Revision *81857  levels 2  root [381]  blocksize 8192  items 767394
lastblock 3892
B-tree checked okay
value:
base66  Revision *81857  levels 1  root [202]  blocksize 8192  items 79378
lastblock 777
B-tree checked okay