oscaruser@programmer.net
2006-Jun-07 23:41 UTC
[Xapian-discuss] Error msg xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation
Folks, While running xapian-compact across a number of flint indicies, I receive the following error message. There are no other clients attempting to read or write the databases than xapian-compact. It could be that I killed the scriptindex process while a flint index was being updated, which may have caused corruption. Is there a way to repair the index in that case? Are there other reasons why this could have happened? Is there a way to validate the integrity of an index? Thanks, OSC gamma:/svr/hda1/gigablast/ppa-index3# /home/oscar/xapian/bin/xapian-compact -F -m /svr/hda1/omega/data/bsp*/default /svr/hda1/xapian/default postlist: Reduced by 62.0901% 693328K (1116648K -> 423320K) record .../home/oscar/xapian/bin/xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation gamma:/svr/hda1/gigablast/ppa-index3# -- ___________________________________________________ Play 100s of games for FREE! http://games.mail.com/
oscaruser@programmer.net
2006-Jun-08 00:57 UTC
[Xapian-discuss] Error msg xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation
Folks,
It looks like it is happening with one of the input dbs (bsp0010), and seems to
take about 10 minutes of running before aborting. I ran strace on the call to
see where it was crashing and observed the following output.
Thanks,
OSC
stat64("/svr/hda1/omega/data/bsp0010/default/record.DB",
{st_mode=S_IFREG|0644, st_size=2146304, ...}) = 0
open("/svr/hda1/omega/data/bsp0010/default/record.baseA",
O_RDONLY|O_LARGEFILE) = 4
read(4, "\270\23\5\200@\203\2\1!\266\23\203\2\0\0\270\23\377\377"...,
1024) = 52
read(4, "", 972) = 0
read(4, "", 33) = 0
close(4) = 0
open("/svr/hda1/omega/data/bsp0010/default/record.baseB",
O_RDONLY|O_LARGEFILE) = 4
read(4, "\271\23\5\200@\205\2\1!\267\23\205\2\0\0\271\23\377\377"...,
1024) = 52
read(4, "", 972) = 0
read(4, "", 33) = 0
close(4) = 0
open("/svr/hda1/omega/data/bsp0010/default/record.DB",
O_RDONLY|O_LARGEFILE) = 4
_llseek(4, 2138112, [2138112], SEEK_SET) = 0
read(4, "\0\0\t\271\1\21\353\21\353\2\21\37\371\37\356\37\343\37"...,
8192) = 8192
_llseek(4, 0, [0], SEEK_SET) = 0
read(4, "\0\0\0\25\0\20;\20;\0!\37\371\36b\34\306\33\323\32U\30"...,
8192) = 8192
_llseek(4, 16384, [16384], SEEK_SET) = 0
read(4, "\0\0\0\37\0\20\20\20\20\0\37\0365\34\210\32\330\31O\30"...,
8192) = 8192
_llseek(3, 8667136, [8667136], SEEK_SET) = 0
write(3, "\0\0\0\1\0\0\3\0\3\0005\37\216\35\302\34\36\32X\30\357"...,
8192) = 8192
_llseek(4, 24576, [24576], SEEK_SET) = 0
...
read(4, "\0\0\5c\0\0214\0214\0\35\36R\34\272\33\16\31A\27\256\26"...,
8192) = 8192
_llseek(4, 1163264, [1163264], SEEK_SET) = 0
read(4, "\0\0\5l\0\21c\21c\0\35\36\206\35\1\33*\31h\27\257\25\354"...,
8192) = 8192
_llseek(3, 9232384, [9232384], SEEK_SET) = 0
write(3, "\0\0\0\1\0\0\0\0\0\0005\37\301\36\24\34:\32}\30\343\027"...,
8192) = 8192
_llseek(4, 1171456, [1171456], SEEK_SET) = 0
read(4, "\0\1\253 \3\16\326\20\2\1\306\4\0\1\347\1\0\1\203\24\0"...,
8192) = 8192
close(4) = 0
close(3) = 0
write(2, "/home/oscar/xapian/bin/xapia"...,
41/home/oscar/xapian/bin/xapian-compact) = 41
write(2, ": ", 2: ) = 2
write(2, "The revision being read has been"..., 111The revision being
read has been discarded - you should call Xapian::Database::reopen() and retry
the operation) = 111
write(2, "\n", 1
) = 1
munmap(0x401c5000, 4096) = 0
exit_group(1) = ?
> ----- Original Message -----
> From: oscaruser@programmer.net
> To: xapian-discuss@lists.xapian.org
> Subject: [Xapian-discuss] Error msg xapian-compact: The revision being read
has been discarded - you should call Xapian::Database::reopen() and retry the
operation
> Date: Wed, 07 Jun 2006 14:40:33 -0800
>
>
> Folks,
>
> While running xapian-compact across a number of flint indicies, I
> receive the following error message. There are no other clients
> attempting to read or write the databases than xapian-compact. It
> could be that I killed the scriptindex process while a flint index
> was being updated, which may have caused corruption. Is there a way
> to repair the index in that case? Are there other reasons why this
> could have happened? Is there a way to validate the integrity of an
> index?
>
> Thanks,
> OSC
>
> gamma:/svr/hda1/gigablast/ppa-index3#
> /home/oscar/xapian/bin/xapian-compact -F -m
> /svr/hda1/omega/data/bsp*/default /svr/hda1/xapian/default
> postlist: Reduced by 62.0901% 693328K (1116648K -> 423320K)
> record .../home/oscar/xapian/bin/xapian-compact: The revision being
> read has been discarded - you should call
> Xapian::Database::reopen() and retry the operation
> gamma:/svr/hda1/gigablast/ppa-index3#
>
--
___________________________________________________
Play 100s of games for FREE! http://games.mail.com/
oscaruser@programmer.net
2006-Jun-08 01:19 UTC
[Xapian-discuss] Error msg xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation
I tried to use the copydatabase utility to sort out what the problem is with this db, and found that at record 1112, there seems to be some corruption. How can I fix? Do I need to look at the binary data structure to determine how to fix this issue? Part of the problem is that it is not trivial to regenerate the data that was in the file. It is better to attempt to recover it, but I haven't found yet a recover utility. It may be OK to try and rebuild record 1112 if I can discover which URL it belongs to. Thanks gamma:/svr/hda1/omega/data/bsp0010/default# /home/oscar/xapian/bin/copydatabase /svr/hda1/omega/data/bsp0010/default /tmp /svr/hda1/omega/data/bsp0010/default: 1112 docs to go /home/oscar/xapian/bin/copydatabase: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation gamma:/svr/hda1/omega/data/bsp0010/default#> ----- Original Message ----- > From: oscaruser@programmer.net > To: xapian-discuss@lists.xapian.org > Subject: Re: [Xapian-discuss] Error msg xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation > Date: Wed, 07 Jun 2006 15:57:15 -0800 > > > Folks, > > It looks like it is happening with one of the input dbs (bsp0010), > and seems to take about 10 minutes of running before aborting. I > ran strace on the call to see where it was crashing and observed > the following output. > > Thanks, > OSC > > > stat64("/svr/hda1/omega/data/bsp0010/default/record.DB", > {st_mode=S_IFREG|0644, st_size=2146304, ...}) = 0 > open("/svr/hda1/omega/data/bsp0010/default/record.baseA", > O_RDONLY|O_LARGEFILE) = 4 > read(4, > "\270\23\5\200@\203\2\1!\266\23\203\2\0\0\270\23\377\377"..., 1024) > = 52 > read(4, "", 972) = 0 > read(4, "", 33) = 0 > close(4) = 0 > open("/svr/hda1/omega/data/bsp0010/default/record.baseB", > O_RDONLY|O_LARGEFILE) = 4 > read(4, > "\271\23\5\200@\205\2\1!\267\23\205\2\0\0\271\23\377\377"..., 1024) > = 52 > read(4, "", 972) = 0 > read(4, "", 33) = 0 > close(4) = 0 > open("/svr/hda1/omega/data/bsp0010/default/record.DB", > O_RDONLY|O_LARGEFILE) = 4 > _llseek(4, 2138112, [2138112], SEEK_SET) = 0 > read(4, > "\0\0\t\271\1\21\353\21\353\2\21\37\371\37\356\37\343\37"..., 8192) > = 8192 > _llseek(4, 0, [0], SEEK_SET) = 0 > read(4, > "\0\0\0\25\0\20;\20;\0!\37\371\36b\34\306\33\323\32U\30"..., 8192) > = 8192 > _llseek(4, 16384, [16384], SEEK_SET) = 0 > read(4, > "\0\0\0\37\0\20\20\20\20\0\37\0365\34\210\32\330\31O\30"..., 8192) > = 8192 > _llseek(3, 8667136, [8667136], SEEK_SET) = 0 > write(3, > "\0\0\0\1\0\0\3\0\3\0005\37\216\35\302\34\36\32X\30\357"..., 8192) > = 8192 > _llseek(4, 24576, [24576], SEEK_SET) = 0 > ... > read(4, > "\0\0\5c\0\0214\0214\0\35\36R\34\272\33\16\31A\27\256\26"..., 8192) > = 8192 > _llseek(4, 1163264, [1163264], SEEK_SET) = 0 > read(4, > "\0\0\5l\0\21c\21c\0\35\36\206\35\1\33*\31h\27\257\25\354"..., > 8192) = 8192 > _llseek(3, 9232384, [9232384], SEEK_SET) = 0 > write(3, > "\0\0\0\1\0\0\0\0\0\0005\37\301\36\24\34:\32}\30\343\027"..., 8192) > = 8192 > _llseek(4, 1171456, [1171456], SEEK_SET) = 0 > read(4, "\0\1\253 > \3\16\326\20\2\1\306\4\0\1\347\1\0\1\203\24\0"..., 8192) = 8192 > close(4) = 0 > close(3) = 0 > write(2, "/home/oscar/xapian/bin/xapia"..., > 41/home/oscar/xapian/bin/xapian-compact) = 41 > write(2, ": ", 2: ) = 2 > write(2, "The revision being read has been"..., 111The revision > being read has been discarded - you should call > Xapian::Database::reopen() and retry the operation) = 111 > write(2, "\n", 1 > ) = 1 > munmap(0x401c5000, 4096) = 0 > exit_group(1) = ? > > > ----- Original Message ----- > > From: oscaruser@programmer.net > > To: xapian-discuss@lists.xapian.org > > Subject: [Xapian-discuss] Error msg xapian-compact: The revision > > being read has been discarded - you should call > > Xapian::Database::reopen() and retry the operation > > Date: Wed, 07 Jun 2006 14:40:33 -0800 > > > > > > Folks, > > > > While running xapian-compact across a number of flint indicies, I > > receive the following error message. There are no other clients > > attempting to read or write the databases than xapian-compact. It > > could be that I killed the scriptindex process while a flint > > index was being updated, which may have caused corruption. Is > > there a way to repair the index in that case? Are there other > > reasons why this could have happened? Is there a way to validate > > the integrity of an index? > > > > Thanks, > > OSC > > > > gamma:/svr/hda1/gigablast/ppa-index3# > > /home/oscar/xapian/bin/xapian-compact -F -m > > /svr/hda1/omega/data/bsp*/default /svr/hda1/xapian/default > > postlist: Reduced by 62.0901% 693328K (1116648K -> 423320K) > > record .../home/oscar/xapian/bin/xapian-compact: The revision > > being read has been discarded - you should call > > Xapian::Database::reopen() and retry the operation > > gamma:/svr/hda1/gigablast/ppa-index3# > > > > > > -- > ___________________________________________________ > Play 100s of games for FREE! http://games.mail.com/ > > > _______________________________________________ > Xapian-discuss mailing list > Xapian-discuss@lists.xapian.org > http://lists.xapian.org/mailman/listinfo/xapian-discuss>-- ___________________________________________________ Play 100s of games for FREE! http://games.mail.com/
Olly Betts
2006-Jun-08 02:37 UTC
[Xapian-discuss] Error msg xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation
On Wed, Jun 07, 2006 at 02:40:33PM -0800, oscaruser@programmer.net wrote:> While running xapian-compact across a number of flint indicies, I > receive the following error message. [...]> record .../home/oscar/xapian/bin/xapian-compact: The revision being > read has been discarded - you should call Xapian::Database::reopen() > and retry the operationThis means that while reading from an input database's record table we encountered part of the tree which has a newer revision than the current root block. This is almost invariably caused by updating a database while reading from it. If two updates are committed before the read completes, you get this error (it's DatabaseModifiedError). It's a bit of a pain and will be going away in the future, but it's not too hard to design to avoid it happening at least. The alternative is a low-level bug in Xapian's flint backend, or a hardware or system software problem in the server. I wouldn't rule out a Xapian bug, but the code in question was in the quartz backend too, so it's been well hammered and every previous occurrence of this has been attributed to simultaneous update or ailing hardware.> There are no other clients attempting to read or write the databases > than xapian-compact. It could be that I killed the scriptindex process > while a flint index was being updated, which may have caused > corruption.It shouldn't be possible to cause corruption in this way. Even a power failure should leave the database in a consistent state (assume the power failure doesn't corrupt the actual data being written of course!) The only loophole is that we assume fsync/fdatasync actually syncs data to disk before returning, but the Linux man page notes: In case the hard disk has write cache enabled, the data may not really be on permanent storage when fsync/fdatasync return. After the sync we write a new "baseA" or "baseB" file to point to the new root, so there's perhaps a possibility that the base file could get written before the sync completes if the disk subsystem writes blocks out of order (you'd hope it wouldn't across a flush though), but I think this could only be a problem anyway with a kernel crash or power failure.> Is there a way to repair the index in that case?We just don't get corrupt indexes, so nobody's bothered to write a repair tool! If it's the postlist table that's broken, copydatabase should be able to help, since it effectively reconstructs the postlist table from the termlist table (which is why it's so much slower than xapian-compact and quartzcompact). But the data in the other tables isn't redundant. If the database has a full set of "baseA" and "baseB" files, you could try remove all of one and see if you can run xapian-compact then, then restore all those and remove all the others and see if that works. If this helps, you'll lose the last batch of documents flushed.> Are there other reasons why this could > have happened? Is there a way to validate the integrity of an index?For quartz databases, you can run quartzcheck. I've not yet written a version for flint, since the database format is due to change substantially.> I tried to use the copydatabase utility to sort out what the problem > is with this db, and found that at record 1112, there seems to be some > corruption. How can I fix? Do I need to look at the binary data > structure to determine how to fix this issue? Part of the problem is > that it is not trivial to regenerate the data that was in the file.The database format isn't too easy to follow from a hex dump. It's not impossible to work out what's wrong and probably recover much of the data, but it's likely to be very time consuming, especially if you don't know the format to start with. Sorry not to have better news. Cheers, Olly
Hello. Is it possibly to give exact matches a higher weight / relevance than that of the pure stemmed form? Means, if querying for footballer, I would retrieve documents including terms like 'football', 'footballs' and 'footballer'. Is there a feature in Xapian to give the document including 'footballer' the highest relevance? Thanks. DD
oscaruser@programmer.net
2006-Jun-09 19:52 UTC
[Xapian-discuss] Error msg xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation
Olly, The disk has write cache enabled. I will disable this. I've noticed from strace that when I have many scriptindex processes running concurrently on different flint dbs the fsync operation takes about 30 seconds to a minute to complete at times. This has had a noticable impact on my rate of processing. I was running 150 spiders in parallel, but had to scale back to 30 or less. My hope is with this disabled the updates will be occur at a faster rate, but it could be disabling the writeback cache would not really help that. Can I modify the copy database tool to ignore the revision check in order to attempt to recover the flint index? I have identified which dbs are busted, and which record precisely -- but as you say I haven't studied the index structure at all. Will I need to make specific decisions about each index error in order to rebuild? Thanks, OSC> ----- Original Message ----- > From: "Olly Betts" <olly@survex.com> > To: oscaruser@programmer.net > Subject: Re: [Xapian-discuss] Error msg xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation > Date: Thu, 8 Jun 2006 02:37:45 +0100 > > > On Wed, Jun 07, 2006 at 02:40:33PM -0800, oscaruser@programmer.net wrote: > > While running xapian-compact across a number of flint indicies, I > > receive the following error message. [...] > > > record .../home/oscar/xapian/bin/xapian-compact: The revision being > > read has been discarded - you should call Xapian::Database::reopen() > > and retry the operation > > This means that while reading from an input database's record table we > encountered part of the tree which has a newer revision than the current > root block. > > This is almost invariably caused by updating a database while reading > from it. If two updates are committed before the read completes, you > get this error (it's DatabaseModifiedError). It's a bit of a pain > and will be going away in the future, but it's not too hard to design > to avoid it happening at least. > > The alternative is a low-level bug in Xapian's flint backend, or a > hardware or system software problem in the server. > > I wouldn't rule out a Xapian bug, but the code in question was in > the quartz backend too, so it's been well hammered and every previous > occurrence of this has been attributed to simultaneous update or ailing > hardware. > > > There are no other clients attempting to read or write the databases > > than xapian-compact. It could be that I killed the scriptindex process > > while a flint index was being updated, which may have caused > > corruption. > > It shouldn't be possible to cause corruption in this way. Even a power > failure should leave the database in a consistent state (assume the > power failure doesn't corrupt the actual data being written of course!) > > The only loophole is that we assume fsync/fdatasync actually syncs > data to disk before returning, but the Linux man page notes: > > In case the hard disk has write cache enabled, the data may not > really be on permanent storage when fsync/fdatasync return. > > After the sync we write a new "baseA" or "baseB" file to point to > the new root, so there's perhaps a possibility that the base file could > get written before the sync completes if the disk subsystem writes > blocks out of order (you'd hope it wouldn't across a flush though), but > I think this could only be a problem anyway with a kernel crash or power > failure. > > > Is there a way to repair the index in that case? > > We just don't get corrupt indexes, so nobody's bothered to write a > repair tool! > > If it's the postlist table that's broken, copydatabase should be able > to help, since it effectively reconstructs the postlist table from > the termlist table (which is why it's so much slower than xapian-compact > and quartzcompact). But the data in the other tables isn't redundant. > > If the database has a full set of "baseA" and "baseB" files, you could > try remove all of one and see if you can run xapian-compact then, > then restore all those and remove all the others and see if that works. > If this helps, you'll lose the last batch of documents flushed. > > > Are there other reasons why this could > > have happened? Is there a way to validate the integrity of an index? > > For quartz databases, you can run quartzcheck. I've not yet written a > version for flint, since the database format is due to change > substantially. > > > I tried to use the copydatabase utility to sort out what the problem > > is with this db, and found that at record 1112, there seems to be some > > corruption. How can I fix? Do I need to look at the binary data > > structure to determine how to fix this issue? Part of the problem is > > that it is not trivial to regenerate the data that was in the file. > > The database format isn't too easy to follow from a hex dump. It's not > impossible to work out what's wrong and probably recover much of the > data, but it's likely to be very time consuming, especially if you > don't know the format to start with. > > Sorry not to have better news. > > Cheers, > Olly-- ___________________________________________________ Play 100s of games for FREE! http://games.mail.com/
oscaruser@programmer.net
2006-Jun-10 20:46 UTC
[Xapian-discuss] Error msg xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation
Tried to remove *.baseA or *.baseB, then run xapian-compact, but did not succeed. I may not be understanding what you are telling me to do. Thanks oscar@delta:/svr/hda1/omega/data/dsp0035/default$ mkdir bak oscar@delta:/svr/hda1/omega/data/dsp0035/default$ ls -aFl total 20608 drwxr-xr-x 3 oscar oscar 4096 Jun 10 12:22 ./ drwxr-xr-x 3 oscar oscar 4096 Jun 2 16:29 ../ drwxr-xr-x 2 oscar oscar 4096 Jun 10 12:22 bak/ -rw-r--r-- 1 oscar oscar 12 Jun 2 16:29 iamflint -rw-r--r-- 1 oscar oscar 7577600 Jun 6 08:03 position.DB -rw-r--r-- 1 oscar oscar 136 Jun 6 08:02 position.baseB -rw-r--r-- 1 oscar oscar 7946240 Jun 6 08:02 postlist.DB -rw-r--r-- 1 oscar oscar 130 Jun 6 08:03 postlist.baseA -rw-r--r-- 1 oscar oscar 130 Jun 6 08:02 postlist.baseB -rw-r--r-- 1 oscar oscar 851968 Jun 6 08:02 record.DB -rw-r--r-- 1 oscar oscar 30 Jun 6 08:00 record.baseA -rw-r--r-- 1 oscar oscar 30 Jun 6 08:02 record.baseB -rw-r--r-- 1 oscar oscar 4636672 Jun 6 08:02 termlist.DB -rw-r--r-- 1 oscar oscar 90 Jun 6 08:02 termlist.baseB -rw-r--r-- 1 oscar oscar 0 Jun 2 16:29 value.DB -rw-r--r-- 1 oscar oscar 17 Jun 6 08:00 value.baseA -rw-r--r-- 1 oscar oscar 17 Jun 6 08:02 value.baseB oscar@delta:/svr/hda1/omega/data/dsp0035/default$ ~/xapian/bin/xapian-compact -F /svr/hda1/omega/data/dsp0035/default /tmp/xapian postlist: Reduced by 62.1649% 4824K (7760K -> 2936K) record: Reduced by 50.9615% 424K (832K -> 408K) termlist .../home/oscar/xapian/bin/xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation oscar@delta:/svr/hda1/omega/data/dsp0035/default$ mv *.baseB bak oscar@delta:/svr/hda1/omega/data/dsp0035/default$ ls -aF ./ bak/ position.DB postlist.baseA record.baseA value.DB ../ iamflint postlist.DB record.DB termlist.DB value.baseA oscar@delta:/svr/hda1/omega/data/dsp0035/default$ cd .. oscar@delta:/svr/hda1/omega/data/dsp0035$ ~/xapian/bin/xapian-compact -F /svr/hda1/omega/data/dsp0035/default /tmp/xapian /home/oscar/xapian/bin/xapian-compact: Cannot open database at `/svr/hda1/omega/data/dsp0035/default' - it does not exist oscar@delta:/svr/hda1/omega/data/dsp0035$ cd default oscar@delta:/svr/hda1/omega/data/dsp0035/default$ ls bak position.DB postlist.baseA record.baseA value.DB iamflint postlist.DB record.DB termlist.DB value.baseA oscar@delta:/svr/hda1/omega/data/dsp0035/default$ mv bak/* . oscar@delta:/svr/hda1/omega/data/dsp0035/default$ ls bak position.baseB postlist.baseB record.baseB value.DB iamflint postlist.DB record.DB termlist.DB value.baseA position.DB postlist.baseA record.baseA termlist.baseB value.baseB oscar@delta:/svr/hda1/omega/data/dsp0035/default$ mv *.baseA bak oscar@delta:/svr/hda1/omega/data/dsp0035/default$ ~/xapian/bin/xapian-compact -F /svr/hda1/omega/data/dsp0035/default /tmp/xapian postlist: Reduced by 62.1649% 4824K (7760K -> 2936K) record: Reduced by 50.9615% 424K (832K -> 408K) termlist .../home/oscar/xapian/bin/xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation oscar@delta:/svr/hda1/omega/data/dsp0035/default$> ----- Original Message ----- > From: "Olly Betts" <olly@survex.com> > To: oscaruser@programmer.net > Subject: Re: [Xapian-discuss] Error msg xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation > Date: Sat, 10 Jun 2006 02:07:45 +0100 > > > On Fri, Jun 09, 2006 at 10:52:09AM -0800, oscaruser@programmer.net wrote:...> Did you look to see if you have a full set of both .baseA and .baseB > files? If so, my suggestion from a previous mail is most likely to > save you.-- ___________________________________________________ Play 100s of games for FREE! http://games.mail.com/
oscaruser@programmer.net
2006-Jun-13 00:13 UTC
[Xapian-discuss] Error msg xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation
I can increase this number, but my code stops to run scriptindexer for each single page. If I understand the function of XAPIAN_FLUSH_THRESHOLD, I would see an improved performance if I run scriptindex against several files at once, but not against single pages. I suppose buffering up 100 or so pages would help that. Thanks> ----- Original Message ----- > From: "Olly Betts" <olly@survex.com> > To: oscaruser@programmer.net > Subject: Re: [Xapian-discuss] Error msg xapian-compact: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation > Date: Sat, 10 Jun 2006 02:07:45 +0100 > > > Have you tried setting XAPIAN_FLUSH_THRESHOLD in the environment (and > exporting it)? It defaults to 10000, but it you've plenty of memory > you can go to 100000 or probably more. This increase the interval > between automatic flushes.-- ___________________________________________________ Play 100s of games for FREE! http://games.mail.com/