Sup had been running for a while, and I just came back to my terminal to find it dead: --- IOError from thread: periodic poll DatabaseError: Error reading block 4279173482: got end of file ./lib/sup/index.rb:271:in `_dangerous_allterms_begin'' ./lib/sup/index.rb:271:in `each_prefixed_term'' ./lib/sup/index.rb:284:in `each_source_info'' ./lib/sup/mbox.rb:147:in `each'' ./lib/sup/mbox.rb:147:in `map'' ./lib/sup/mbox.rb:147:in `last_indexed_message'' ./lib/sup/util.rb:204:in `call'' ./lib/sup/util.rb:204:in `benchmark'' /usr/lib/ruby/1.8/benchmark.rb:293:in `measure'' ./lib/sup/util.rb:204:in `benchmark'' ./lib/sup/mbox.rb:147:in `last_indexed_message'' ./lib/sup/mbox.rb:152:in `first_new_message'' ./lib/sup/mbox.rb:124:in `poll'' ./lib/sup/poll.rb:154:in `poll_from'' ./lib/sup/poll.rb:113:in `do_poll'' ./lib/sup/poll.rb:103:in `each'' ./lib/sup/poll.rb:103:in `do_poll'' ./lib/sup/poll.rb:102:in `synchronize'' ./lib/sup/poll.rb:102:in `do_poll'' ./lib/sup/util.rb:609:in `send'' ./lib/sup/util.rb:609:in `method_missing'' ./lib/sup/modes/poll-mode.rb:15:in `poll'' ./lib/sup/poll.rb:49:in `poll_with_sources'' ./lib/sup/poll.rb:68:in `poll'' ./lib/sup/poll.rb:86:in `start'' ./lib/sup.rb:78:in `reporting_thread'' ./lib/sup.rb:76:in `initialize'' ./lib/sup.rb:76:in `new'' ./lib/sup.rb:76:in `reporting_thread'' ./lib/sup/poll.rb:83:in `start'' ./lib/sup/util.rb:609:in `send'' ./lib/sup/util.rb:609:in `method_missing'' bin/sup:222 I tried to start it again, and got --- IOError from thread: poll after loading inbox DatabaseCorruptError: Db block overwritten - are there multiple writers? /usr/lib/ruby/1.8/xapian.rb:259:in `_dangerous_postlist_begin'' /usr/lib/ruby/1.8/xapian.rb:259:in `postlist'' ./lib/sup/index.rb:530:in `term_docids'' ./lib/sup/index.rb:717:in `index_message_threading'' ./lib/sup/index.rb:633:in `sync_message'' ./lib/sup/util.rb:609:in `send'' ./lib/sup/util.rb:609:in `method_missing'' ./lib/sup/poll.rb:168:in `poll_from'' ./lib/sup/mbox.rb:127:in `poll'' ./lib/sup/poll.rb:154:in `poll_from'' ./lib/sup/poll.rb:113:in `do_poll'' ./lib/sup/poll.rb:103:in `each'' ./lib/sup/poll.rb:103:in `do_poll'' ./lib/sup/poll.rb:102:in `synchronize'' ./lib/sup/poll.rb:102:in `do_poll'' ./lib/sup/util.rb:609:in `send'' ./lib/sup/util.rb:609:in `method_missing'' ./lib/sup/modes/poll-mode.rb:15:in `poll'' ./lib/sup/poll.rb:49:in `poll_with_sources'' ./lib/sup/poll.rb:68:in `poll'' ./lib/sup/util.rb:609:in `send'' ./lib/sup/util.rb:609:in `method_missing'' bin/sup:212 ./lib/sup.rb:78:in `reporting_thread'' ./lib/sup.rb:76:in `initialize'' ./lib/sup.rb:76:in `new'' ./lib/sup.rb:76:in `reporting_thread'' bin/sup:212 ./lib/sup/modes/thread-index-mode.rb:684:in `call'' ./lib/sup/modes/thread-index-mode.rb:684:in `__unprotected_load_threads'' ./lib/sup/modes/thread-index-mode.rb:625:in `call'' ./lib/sup/modes/thread-index-mode.rb:625:in `load_n_threads_background'' ./lib/sup.rb:78:in `reporting_thread'' ./lib/sup.rb:76:in `initialize'' ./lib/sup.rb:76:in `new'' ./lib/sup.rb:76:in `reporting_thread'' ./lib/sup/modes/thread-index-mode.rb:623:in `load_n_threads_background'' ./lib/sup/modes/thread-index-mode.rb:694:in `__unprotected_load_threads'' (eval):12:in `load_threads'' bin/sup:212 My xapian dir looks like: total 334396 -rw------- 1 andrew andrew 0 May 6 14:43 flintlock -rw-rw-r-- 1 andrew andrew 12 May 31 2010 iamflint -rw-rw-r-- 1 andrew andrew 103014400 May 6 13:09 position.DB -rw-rw-r-- 1 andrew andrew 1594 May 6 11:22 position.baseA -rw-rw-r-- 1 andrew andrew 161333248 May 6 13:29 postlist.DB -rw-rw-r-- 1 andrew andrew 2475 May 6 11:22 postlist.baseA -rw-rw-r-- 1 andrew andrew 20807680 May 6 13:04 record.DB -rw-rw-r-- 1 andrew andrew 339 May 6 11:22 record.baseA -rw-rw-r-- 1 andrew andrew 50692096 May 6 13:29 termlist.DB -rw-rw-r-- 1 andrew andrew 795 May 6 11:22 termlist.baseA -rw-rw-r-- 1 andrew andrew 6184960 May 6 13:04 value.DB -rw-rw-r-- 1 andrew andrew 115 May 6 11:22 value.baseA I''m running an old git pull from next, commit 1bafcf6 dated 2010-10-12. Anything I should do to try to diagnose and recover this? Or just load from sup-dump? Andrew
Reformatted excerpts from Andrew Pimlott''s message of 2011-05-06:> Sup had been running for a while, and I just came back to my terminal to > find it dead: > > --- IOError from thread: periodic poll > DatabaseError: Error reading block 4279173482: got end of file > I''m running an old git pull from next, commit 1bafcf6 dated 2010-10-12.Yipes. Are you out of disk space, or anything like that?> Anything I should do to try to diagnose and recover this? Or just > load from sup-dump?Not knowing very much about Xapian, I suggest sup-dump. Not a great solution, I know. Wait for heliotrope to solve your problems (or at least be something I understand and can fix). -- William <wmorgan-sup at masanjin.net>
Excerpts from William Morgan''s message of Tue May 10 17:36:54 -0700 2011:> Reformatted excerpts from Andrew Pimlott''s message of 2011-05-06: > > Sup had been running for a while, and I just came back to my terminal to > > find it dead: > > > > --- IOError from thread: periodic poll > > DatabaseError: Error reading block 4279173482: got end of file > > I''m running an old git pull from next, commit 1bafcf6 dated 2010-10-12. > > Yipes. Are you out of disk space, or anything like that?I wasn''t out of space--unless possibly Xapian caused some transient spike in disk use that was freed when it died. The system does run without a ton of free space though, maybe 100 MB for a Xapian database that is ~300 MB total. I didn''t get any disk errors either. This is on a VPS that could conceivably be hiding some errors.> > Anything I should do to try to diagnose and recover this? Or just > > load from sup-dump? > > Not knowing very much about Xapian, I suggest sup-dump. Not a great > solution, I know.It turned out that sup-dump still worked. Probably some index was corrupt but the data could still be walked. So I was able to restore without losing any data. (Well, it did use most of the 160 MB of memory on the VPS.)> Wait for heliotrope to solve your problems (or at > least be something I understand and can fix).Dum de dum dum... Andrew