search for: firstunused

...tmuch developers, I also had this database corruption, I waited for the fix to land in notmuch 0.26.2, build it, moved the xapian directory away, did a notmuch new and restored the tags from a dump. But the problem remains: ~$ xapian-check ~/Mail/.notmuch/xapian docdata: blocksize=8K items=10841 firstunused=75 revision=82 levels=1 root=2 B-tree checked okay docdata table structure checked OK termlist: blocksize=8K items=1893162 firstunused=368983 revision=82 levels=3 root=177608 xapian-check: DatabaseError: 1 unused block(s) missing from the free list, first is 0 this is very similar to the old dat...

...D_LIBRARY_PATH > ./notmuch new >    Found 20065 total files (that's not much mail). >    Processed 20065 total files in 58s (341 files/sec.). >    Added 19605 new messages to the database. > > $ xapian-check .mail/.notmuch/xapian/ >    docdata: >    blocksize=8K items=63 firstunused=1 revision=2 levels=0 root=0 >    B-tree checked okay >    docdata table structure checked OK >    termlist: >    blocksize=8K items=43520 firstunused=8293 revision=2 levels=2 root=748 >    xapian-check: DatabaseError: 1 unused block(s) missing from the free > list, first is 0 OK...

Matt <mattator at gmail.com> writes: > thanks didn't know about xapian-check ! > the output > === > docdata: > blocksize=8K items=70 firstunused=3 revision=421 levels=0 root=2 > B-tree checked okay > docdata table structure checked OK > > termlist: > blocksize=8K items=186136 firstunused=62058 revision=421 levels=2 root=12260 > B-tree checked okay > termlist table structure checked OK > > postlist: > blocksize=...

...This happens consistently at this point in the process - if I remove the database and start again it will fail at the same place. After the failure the database appears to be okay: > agrajag-testing ~/s/notmuch % xapian-check ~/Maildir/.notmuch/xapian > docdata: > blocksize=8K items=0 firstunused=1 revision=1 levels=0 root=(faked) > void B-tree checked okay > docdata table structure checked OK > > termlist: > blocksize=8K items=0 firstunused=2 revision=1 levels=0 root=(faked) > void B-tree checked okay > termlist table structure checked OK > > postlist: > blo...

...doclen not within bounds [repeated 52498 times] OTOH, notmuch compact completes for me, so that might be unrelated. I'm copying the Xapian list for input. > Compaction failed: A Xapian exception occurred > > Running xapian-check says: > > docdata: > blocksize=8K items=2677 firstunused=19 revision=15425 levels=1 root=17 > B-tree checked okay > docdata table structure checked OK > > termlist: > blocksize=8K items=2986940 firstunused=276475 revision=15425 levels=2 root=271786 > B-tree checked okay > doclen not within bounds > (...) > doclen not within bou...

...s point in the process - if I remove the > database and start again it will fail at the same place. > > After the failure the database appears to be okay: > > > agrajag-testing ~/s/notmuch % xapian-check ~/Maildir/.notmuch/xapian > > docdata: > > blocksize=8K items=0 firstunused=1 revision=1 levels=0 root=(faked) > > void B-tree checked okay > > docdata table structure checked OK > > > > termlist: > > blocksize=8K items=0 firstunused=2 revision=1 levels=0 root=(faked) > > void B-tree checked okay > > termlist table structure check...

...database corruption, I waited for the fix to land > in notmuch 0.26.2, build it, moved the xapian directory away, did > a notmuch new and restored the tags from a dump. But the problem > remains: > > ~$ xapian-check ~/Mail/.notmuch/xapian > docdata: > blocksize=8K items=10841 firstunused=75 revision=82 levels=1 root=2 > B-tree checked okay > docdata table structure checked OK > > termlist: > blocksize=8K items=1893162 firstunused=368983 revision=82 levels=3 root=177608 > xapian-check: DatabaseError: 1 unused block(s) missing from the free list, first is 0 This is...

On Tue, Apr 07, 2020 at 05:21:47PM -0300, David Bremner wrote: > Matt <mattator at gmail.com> writes: [...] > > termlist: > > blocksize=8K items=186136 firstunused=62058 revision=421 levels=2 root=12260 > > B-tree checked okay > > termlist table structure checked OK > > > > postlist: > > blocksize=8K items=2598971 firstunused=61412 revision=421 levels=2 root=49814 > > xapian-check: DatabaseCorruptError: Db block overwritten...

...h new >>    Found 20065 total files (that's not much mail). >>    Processed 20065 total files in 58s (341 files/sec.). >>    Added 19605 new messages to the database. >> >> $ xapian-check .mail/.notmuch/xapian/ >>    docdata: >>    blocksize=8K items=63 firstunused=1 revision=2 levels=0 root=0 >>    B-tree checked okay >>    docdata table structure checked OK >>    termlist: >>    blocksize=8K items=43520 firstunused=8293 revision=2 levels=2 root=748 >>    xapian-check: DatabaseError: 1 unused block(s) missing from the free >&...

...index.py", line 89, in do_update backend.update(index, current_qs) File "/a/mailarch/current/haystack/backends/xapian_backend.py", line 286, in update database.close() xapian.DatabaseCorruptError: Expected block 615203 to be level 0, not 1 docdata: blocksize=8K items=380000 firstunused=21983 revision=38 levels=2 root=21410 Any suggestions for how I could get more information to troubleshoot this failure would be greatly appreciated. Thank You, Ryan

...s not a database.close() anywhere > near line 286 in git master: > > https://github.com/notanumber/xapian-haystack/blob/master/xapian_backend.py#L286 > >> xapian.DatabaseCorruptError: Expected block 615203 to be level 0, not 1 >> docdata: >> blocksize=8K items=380000 firstunused=21983 revision=38 levels=2 root=21410 > > Is that the full output of xapian-check? > >> Any suggestions for how I could get more information to troubleshoot this >> failure would be greatly appreciated. > > Is the data to reproduce this something you can make available...

...of xapian-haystack? There's not a database.close() anywhere near line 286 in git master: https://github.com/notanumber/xapian-haystack/blob/master/xapian_backend.py#L286 > xapian.DatabaseCorruptError: Expected block 615203 to be level 0, not 1 > docdata: > blocksize=8K items=380000 firstunused=21983 revision=38 levels=2 root=21410 Is that the full output of xapian-check? > Any suggestions for how I could get more information to troubleshoot this > failure would be greatly appreciated. Is the data to reproduce this something you can make available? I'd stick with Xapian 1.4....

...where >> near line 286 in git master: >> >> https://github.com/notanumber/xapian-haystack/blob/master/xapian_backend.py#L286 >> >>> xapian.DatabaseCorruptError: Expected block 615203 to be level 0, not 1 >>> docdata: >>> blocksize=8K items=380000 firstunused=21983 revision=38 levels=2 root=21410 >> >> Is that the full output of xapian-check? >> >>> Any suggestions for how I could get more information to troubleshoot this >>> failure would be greatly appreciated. >> >> Is the data to reproduce this some...