search for: docdata

Hi notmuch developers, I also had this database corruption, I waited for the fix to land in notmuch 0.26.2, build it, moved the xapian directory away, did a notmuch new and restored the tags from a dump. But the problem remains: ~$ xapian-check ~/Mail/.notmuch/xapian docdata: blocksize=8K items=10841 firstunused=75 revision=82 levels=1 root=2 B-tree checked okay docdata table structure checked OK termlist: blocksize=8K items=1893162 firstunused=368983 revision=82 levels=3 root=177608 xapian-check: DatabaseError: 1 unused block(s) missing from the free list, first is 0...

...ATH=/hidden-path/notmuch-0.26.1/lib/:$LD_LIBRARY_PATH > ./notmuch new >    Found 20065 total files (that's not much mail). >    Processed 20065 total files in 58s (341 files/sec.). >    Added 19605 new messages to the database. > > $ xapian-check .mail/.notmuch/xapian/ >    docdata: >    blocksize=8K items=63 firstunused=1 revision=2 levels=0 root=0 >    B-tree checked okay >    docdata table structure checked OK >    termlist: >    blocksize=8K items=43520 firstunused=8293 revision=2 levels=2 root=748 >    xapian-check: DatabaseError: 1 unused block(s) miss...

...aystack/management/commands/update_index.py", line 89, in do_update backend.update(index, current_qs) File "/a/mailarch/current/haystack/backends/xapian_backend.py", line 286, in update database.close() xapian.DatabaseCorruptError: Expected block 615203 to be level 0, not 1 docdata: blocksize=8K items=380000 firstunused=21983 revision=38 levels=2 root=21410 Any suggestions for how I could get more information to troubleshoot this failure would be greatly appreciated. Thank You, Ryan

...********************** 452.68user 124.94system 4:42.27elapsed 204%CPU (0avgtext+0avgdata 1055204maxresident)k 0inputs+21046192outputs (0major+41137071minor)pagefaults 0swaps ************************* 793244 /home/dockes/.recoll/xapiandb total 793240 -rw-r--r-- 1 dockes dockes 24150016 Apr 12 10:47 docdata.glass -rw-r--r-- 1 dockes dockes 0 Apr 12 10:47 flintlock -rw-r--r-- 1 dockes dockes 130 Apr 12 10:47 iamglass -rw-r--r-- 1 dockes dockes 577527808 Apr 12 10:47 position.glass -rw-r--r-- 1 dockes dockes 120905728 Apr 12 10:47 postlist.glass -rw-r--r-- 1 dockes dockes 89677824 Apr 12...

Mike Hommey <mh at glandium.org> writes: > Hi, > > When running `notmuch compact` today, it stopped with the following > output: > > Compacting database... > compacting table postlist > Reduced by 25% 648656K (2498904K -> 1850248K) > compacting table docdata > Reduced by 15% 24K (152K -> 128K) > compacting table termlist > Reduced by 1% 27008K (2211800K -> 2184792K) > compacting table position > Error while compacting: Bad position key I had not seen anything like this before, but when I run xapian-check from xapian 1.4....

...the version of xapian-haystack? There's not a database.close() anywhere > near line 286 in git master: > > https://github.com/notanumber/xapian-haystack/blob/master/xapian_backend.py#L286 > >> xapian.DatabaseCorruptError: Expected block 615203 to be level 0, not 1 >> docdata: >> blocksize=8K items=380000 firstunused=21983 revision=38 levels=2 root=21410 > > Is that the full output of xapian-check? > >> Any suggestions for how I could get more information to troubleshoot this >> failure would be greatly appreciated. > > Is the data to...

...database.close() What's the version of xapian-haystack? There's not a database.close() anywhere near line 286 in git master: https://github.com/notanumber/xapian-haystack/blob/master/xapian_backend.py#L286 > xapian.DatabaseCorruptError: Expected block 615203 to be level 0, not 1 > docdata: > blocksize=8K items=380000 firstunused=21983 revision=38 levels=2 root=21410 Is that the full output of xapian-check? > Any suggestions for how I could get more information to troubleshoot this > failure would be greatly appreciated. Is the data to reproduce this something you can mak...

Olly Betts writes: > On Sun, Apr 10, 2016 at 04:47:01PM +0200, Jean-Francois Dockes wrote: > > Some might notice the 50% index size increase. Excessive index size is > > already one relatively rare, but recurring complaint. Except if I did > > something wrong: I'm actually quite surprised by it. > > Did you try compacting the resulting databases? > >

...> agrajag-testing ~/s/notmuch % This happens consistently at this point in the process - if I remove the database and start again it will fail at the same place. After the failure the database appears to be okay: > agrajag-testing ~/s/notmuch % xapian-check ~/Maildir/.notmuch/xapian > docdata: > blocksize=8K items=0 firstunused=1 revision=1 levels=0 root=(faked) > void B-tree checked okay > docdata table structure checked OK > > termlist: > blocksize=8K items=0 firstunused=2 revision=1 levels=0 root=(faked) > void B-tree checked okay > termlist table structure c...

...aystack? There's not a database.close() anywhere >> near line 286 in git master: >> >> https://github.com/notanumber/xapian-haystack/blob/master/xapian_backend.py#L286 >> >>> xapian.DatabaseCorruptError: Expected block 615203 to be level 0, not 1 >>> docdata: >>> blocksize=8K items=380000 firstunused=21983 revision=38 levels=2 root=21410 >> >> Is that the full output of xapian-check? >> >>> Any suggestions for how I could get more information to troubleshoot this >>> failure would be greatly appreciated. &...

...ost a hard drive and after successfully restoring everything, I think, I'm getting "Error opening database `current.1': DatabaseOpeningError: Couldn't detect type of database" The directory current.1 contains the following files: -rw-rw-r-- 1 jwl jwl  30064640 Aug 28 23:44 docdata.glass -rw-rw-r-- 1 jwl jwl       151 Aug 28 23:44 iamglass -rw-rw-r-- 1 jwl jwl 767115264 Aug 28 23:44 position.glass -rw-rw-r-- 1 jwl jwl 383213568 Aug 28 23:44 postlist.glass -rw-rw-r-- 1 jwl jwl 203341824 Aug 28 23:44 termlist.glass -rw-rw-r-- 1 jwl jwl         0 Aug 28 23:44 flintlock I know t...

...h developers, > > I also had this database corruption, I waited for the fix to land > in notmuch 0.26.2, build it, moved the xapian directory away, did > a notmuch new and restored the tags from a dump. But the problem > remains: > > ~$ xapian-check ~/Mail/.notmuch/xapian > docdata: > blocksize=8K items=10841 firstunused=75 revision=82 levels=1 root=2 > B-tree checked okay > docdata table structure checked OK > > termlist: > blocksize=8K items=1893162 firstunused=368983 revision=82 levels=3 root=177608 > xapian-check: DatabaseError: 1 unused block(s) miss...

...6.1/lib/:$LD_LIBRARY_PATH >> ./notmuch new >>    Found 20065 total files (that's not much mail). >>    Processed 20065 total files in 58s (341 files/sec.). >>    Added 19605 new messages to the database. >> >> $ xapian-check .mail/.notmuch/xapian/ >>    docdata: >>    blocksize=8K items=63 firstunused=1 revision=2 levels=0 root=0 >>    B-tree checked okay >>    docdata table structure checked OK >>    termlist: >>    blocksize=8K items=43520 firstunused=8293 revision=2 levels=2 root=748 >>    xapian-check: DatabaseError...

...; > This happens consistently at this point in the process - if I remove the > database and start again it will fail at the same place. > > After the failure the database appears to be okay: > > > agrajag-testing ~/s/notmuch % xapian-check ~/Maildir/.notmuch/xapian > > docdata: > > blocksize=8K items=0 firstunused=1 revision=1 levels=0 root=(faked) > > void B-tree checked okay > > docdata table structure checked OK > > > > termlist: > > blocksize=8K items=0 firstunused=2 revision=1 levels=0 root=(faked) > > void B-tree checked o...

Hi All, I've installed Xapian and the php module. I've set up a script for use with scriptindex and dbi2omega for getting data from the db into the index easily, the script file is as follows: =============================== id : field=id title : index title: field=title description : index description : truncate=50 field=content ============================= However, when querying

Matt <mattator at gmail.com> writes: > thanks didn't know about xapian-check ! > the output > === > docdata: > blocksize=8K items=70 firstunused=3 revision=421 levels=0 root=2 > B-tree checked okay > docdata table structure checked OK > > termlist: > blocksize=8K items=186136 firstunused=62058 revision=421 levels=2 root=12260 > B-tree checked okay > termlist table structure checke...

...e these glass things in the index directories? There is a no new version of Perl Search::Xapian. $ quest -version quest - xapian-core 1.4.0 $ delve --version delve - xapian-core 1.2.23 Exception: Couldn't detect type of database http://pacific-design.com/search?q=kevin 253952 Jul 6 13:55 docdata.glass 0 Jul 6 13:55 flintlock 114 Jul 6 13:55 iamglass 1228800 Jul 6 13:55 position.glass 761856 Jul 6 13:55 postlist.glass 270336 Jul 6 13:55 termlist.glass On Sat, Jun 25, 2016 at 11:58 AM, Emmanuel Engelhart <kelson at kiwix.org> wrote: > This is really great. Th...

...a no new version of > Perl Search::Xapian. > > $ quest -version > quest - xapian-core 1.4.0 > > $ delve --version > delve - xapian-core 1.2.23 > > Exception: Couldn't detect type of database > http://pacific-design.com/search?q=kevin > > 253952 Jul 6 13:55 docdata.glass > 0 Jul 6 13:55 flintlock > 114 Jul 6 13:55 iamglass > 1228800 Jul 6 13:55 position.glass > 761856 Jul 6 13:55 postlist.glass > 270336 Jul 6 13:55 termlist.glass > > > On Sat, Jun 25, 2016 at 11:58 AM, Emmanuel Engelhart <kelson at kiwix.org&gt...

...uccessfully restoring everything, I > think, I'm getting "Error opening database `current.1': > DatabaseOpeningError: Couldn't detect type of database" > > The directory current.1 contains the following files: > > -rw-rw-r-- 1 jwl jwl  30064640 Aug 28 23:44 docdata.glass > -rw-rw-r-- 1 jwl jwl       151 Aug 28 23:44 iamglass > -rw-rw-r-- 1 jwl jwl 767115264 Aug 28 23:44 position.glass > -rw-rw-r-- 1 jwl jwl 383213568 Aug 28 23:44 postlist.glass > -rw-rw-r-- 1 jwl jwl 203341824 Aug 28 23:44 termlist.glass > -rw-rw-r-- 1 jwl jwl         0 Aug 28...

...ed > first. That will tend to mean more good matches are found sooner, which > will help the matcher take short cuts. Thanks, I'll keep that in mind. Btw, is there a way to quickly figure out which sub-DB a retrieved document or mset item belongs to? I suppose I could add that info to docdata since I'm having to reindex, anyways...