Hello,
While doing a from scratch sup-sync to benefit for the new version of
the xapian index, sup-sync
crashed like this:
/usr/lib/ruby/site_ruby/1.8/xapian.rb:240:in `doclength'':
DocNotFoundError: Document 3318104838 not found (RuntimeError)
from /usr/lib/ruby/site_ruby/1.8/xapian.rb:240:in `postlist''
from /usr/lib/ruby/site_ruby/1.8/xapian.rb:60:in
`_safelyIterate''
from /usr/lib/ruby/site_ruby/1.8/xapian.rb:238:in `postlist''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:365:in
`term_docids''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:369:in
`find_docid''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:373:in
`find_doc''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:469:in
`index_message''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:469:in
`map''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:469:in
`index_message''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:118:in
`sync_message''
from /usr/lib/ruby/1.8/monitor.rb:242:in `synchronize''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:392:in
`synchronize''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:117:in
`sync_message''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:87:in
`add_message''
from /usr/lib/ruby/gems/1.8/gems/sup-999/bin/sup-sync:202
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/poll.rb:152:in
`each_message_from''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/source.rb:104:in
`each''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/util.rb:560:in
`send''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/util.rb:560:in
`__pass''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/util.rb:547:in
`method_missing''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/poll.rb:140:in
`each_message_from''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/util.rb:520:in
`send''
from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/util.rb:520:in
`method_missing''
from /usr/lib/ruby/gems/1.8/gems/sup-999/bin/sup-sync:146
from /usr/lib/ruby/gems/1.8/gems/sup-999/bin/sup-sync:141:in
`each''
from /usr/lib/ruby/gems/1.8/gems/sup-999/bin/sup-sync:141
from /usr/bin/sup-sync:19:in `load''
from /usr/bin/sup-sync:19
The last status message was:
## read 9996m (about 30%) @ 11.1m/s. 0:15:03 elapsed, about 0:35:00 remaining
And it was processing a 500MB mbox.
I would like to know what are my options from here?
Best regards,
--
Nicolas Pouillard aka Ertai
On Sat, Aug 29, 2009 at 11:18 AM, Nicolas Pouillard<nicolas.pouillard at gmail.com> wrote:> Hello, > > While doing a from scratch sup-sync to benefit for the new version of > the xapian index, sup-sync > crashed like this: >I''ve lauched sup-sync on another bunch of mboxes and got this crash at the end: Deleting missing messages from the index... /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup.rb:17:in `id'': wrong id called on nil (RuntimeError) from /usr/lib/ruby/gems/1.8/gems/sup-999/bin/sup-sync:237 from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/index.rb:150:in `each_message'' from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:141:in `each_id'' from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:141:in `each'' from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:141:in `each_id'' from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/index.rb:149:in `each_message'' from /usr/lib/ruby/gems/1.8/gems/sup-999/bin/sup-sync:235 from /usr/lib/ruby/gems/1.8/gems/sup-999/bin/sup-sync:233:in `each'' from /usr/lib/ruby/gems/1.8/gems/sup-999/bin/sup-sync:233 from /usr/bin/sup-sync:19:in `load'' from /usr/bin/sup-sync:19 -- Nicolas Pouillard aka Ertai
Excerpts from Nicolas Pouillard''s message of Sat Aug 29 08:03:04 -0400 2009:> On Sat, Aug 29, 2009 at 11:18 AM, Nicolas > Pouillard<nicolas.pouillard at gmail.com> wrote: > > Hello, > > > > While doing a from scratch sup-sync to benefit for the new version of > > the xapian index, sup-sync > > crashed like this: > > > > I''ve lauched sup-sync on another bunch of mboxes and got this crash at the end: > > Deleting missing messages from the index... > /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup.rb:17:in `id'': wrong id > called on nil (RuntimeError) > from /usr/lib/ruby/gems/1.8/gems/sup-999/bin/sup-sync:237 > from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/index.rb:150:in > `each_message'' > from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:141:in > `each_id'' > from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:141:in > `each'' > from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/xapian_index.rb:141:in > `each_id'' > from /usr/lib/ruby/gems/1.8/gems/sup-999/lib/sup/index.rb:149:in > `each_message'' > from /usr/lib/ruby/gems/1.8/gems/sup-999/bin/sup-sync:235 > from /usr/lib/ruby/gems/1.8/gems/sup-999/bin/sup-sync:233:in `each'' > from /usr/lib/ruby/gems/1.8/gems/sup-999/bin/sup-sync:233 > from /usr/bin/sup-sync:19:in `load'' > from /usr/bin/sup-sync:19 >Are you using the Flint or Chert Xapian backend?
On Sat, Aug 29, 2009 at 6:25 PM, Rich Lane<rlane at club.cc.cmu.edu> wrote:> Excerpts from Nicolas Pouillard''s message of Sat Aug 29 08:03:04 -0400 2009: >> On Sat, Aug 29, 2009 at 11:18 AM, Nicolas Pouillard<nicolas.pouillard at gmail.com> wrote: >> > Hello, >> > >> > While doing a from scratch sup-sync to benefit for the new version of >> > the xapian index, sup-sync >> > crashed like this: >> >[...]> Are you using the Flint or Chert Xapian backend?The Chert one. -- Nicolas Pouillard aka Ertai
Excerpts from Nicolas Pouillard''s message of Sat Aug 29 12:45:29 -0400 2009:> > Are you using the Flint or Chert Xapian backend? > > The Chert one.Could you try it with Flint? I''ve seen this (nondeterministic) bug when using Chert.
On Sat, Aug 29, 2009 at 6:59 PM, Rich Lane<rlane at club.cc.cmu.edu> wrote:> Excerpts from Nicolas Pouillard''s message of Sat Aug 29 12:45:29 -0400 2009: >> > Are you using the Flint or Chert Xapian backend? >> >> The Chert one. > > Could you try it with Flint? I''ve seen this (nondeterministic) bug when > using Chert.I was currently running it again with no problems so far. Are the nondeterministic bugs only during sync? Chert is supposed to faster? Thanks for your help, -- Nicolas Pouillard
Excerpts from Nicolas Pouillard''s message of Sat Aug 29 13:05:33 -0400 2009:> Are the nondeterministic bugs only during sync?I''ve only seen it during sup-sync, but I''d guess if you ran sup long enough you''d see it there too (after changing labels / polling).> Chert is supposed to faster?Yes, and more space efficient.