similar to: Error while compacting: Bad position key

Hi notmuch developers, I also had this database corruption, I waited for the fix to land in notmuch 0.26.2, build it, moved the xapian directory away, did a notmuch new and restored the tags from a dump. But the problem remains: ~$ xapian-check ~/Mail/.notmuch/xapian docdata: blocksize=8K items=10841 firstunused=75 revision=82 levels=1 root=2 B-tree checked okay docdata table structure checked

Running notmuch from git on Debian testing[1] with the mail and database sitting on a ZFS filesystem, adding mail to a new database: > agrajag-testing ~/s/notmuch % ./notmuch new > Found 605510 total files (that's not much mail). > add_file: A Xapian exception occurred36m 37s remaining). > A Xapian exception occurred adding message: Unexpected end of posting list for

Javier Garcia <javiertury at gmail.com> writes: > I've applied the path to notmuch 0.26.1 without success. > > $ rm -rf ~/.mail/.notmuch > $ LD_LIBRARY_PATH=/hidden-path/notmuch-0.26.1/lib/:$LD_LIBRARY_PATH > ./notmuch new >    Found 20065 total files (that's not much mail). >    Processed 20065 total files in 58s (341 files/sec.). >    Added 19605 new

Matt <mattator at gmail.com> writes: > thanks didn't know about xapian-check ! > the output > === > docdata: > blocksize=8K items=70 firstunused=3 revision=421 levels=0 root=2 > B-tree checked okay > docdata table structure checked OK > > termlist: > blocksize=8K items=186136 firstunused=62058 revision=421 levels=2 root=12260 > B-tree checked okay >

On Fri, Dec 29, 2017 at 03:00:47PM +0000, David Edmondson wrote: > Running notmuch from git on Debian testing[1] with the mail and database > sitting on a ZFS filesystem, adding mail to a new database: > > > agrajag-testing ~/s/notmuch % ./notmuch new > > Found 605510 total files (that's not much mail). > > add_file: A Xapian exception occurred36m 37s remaining).

Unfortunately I can't share my emails without the approval of other parties. The minimum subsets that trigger the error are in the range of 1000-5000 mails, so asking each and everyone of them is out of my reach. I tried to replicate the problem using just spam folders without success. The following is a solid workaround I've stumbled upon. Afew no longer complains and database corruption

Gregor Zattler <telegraph at gmx.net> writes: > Hi notmuch developers, > > I also had this database corruption, I waited for the fix to land > in notmuch 0.26.2, build it, moved the xapian directory away, did > a notmuch new and restored the tags from a dump. But the problem > remains: > > ~$ xapian-check ~/Mail/.notmuch/xapian > docdata: > blocksize=8K

On Tue, Apr 07, 2020 at 05:21:47PM -0300, David Bremner wrote: > Matt <mattator at gmail.com> writes: [...] > > termlist: > > blocksize=8K items=186136 firstunused=62058 revision=421 levels=2 root=12260 > > B-tree checked okay > > termlist table structure checked OK > > > > postlist: > > blocksize=8K items=2598971 firstunused=61412 revision=421

Hello, I am trying to rebuild an index of 2+ million documents and have not been successful. I am running Python 2.7 Django 1.7 Haystack 2.1.1 Xapian 1.2.21 The index rebuild command I’m using is: django-admin.py rebuild_index --noinput --batch-size=100000 The rebuild completes but an immediate xapian-check returns this error: xapian-check ./archive_index record: baseB blocksize=8K

On Mon, Feb 27, 2017 at 10:29:46AM -0800, Ryan Cross wrote: > I am trying to rebuild an index of 2+ million documents and have not been successful. I am running > > Python 2.7 > Django 1.7 > Haystack 2.1.1 > Xapian 1.2.21 > > The index rebuild command I’m using is: django-admin.py rebuild_index --noinput --batch-size=100000 > The rebuild completes but an immediate

Hi Olly, After upgrades my stack is now: Python 2.7 Django 1.8 Haystack 2.6.0 Xapian 1.4.3. (latest xapian haystack backend with some modifications) Using the same rebuild command as below but with —batch-size=50000 The issue has now become one of performance. I am indexing 2.2 million documents. Using delve I can see that performance starts off at about 100,000 records an hour. This is

Hi Olly, Thanks for the detailed response. I hadn’t realized there was a new xapian haystack backend. I’m going to try that but I have some upgrades to do first. Django 1.8, etc. Thanks, Ryan > On Feb 28, 2017, at 3:40 PM, Olly Betts <olly at survex.com> wrote: > > On Mon, Feb 27, 2017 at 10:29:46AM -0800, Ryan Cross wrote: >> I am trying to rebuild an index of 2+

Hi, I was confronted with some trouble, I describe the trouble in my journal http://trac.xapian.org/wiki/GSoC2014/Posting%20list%20encoding%20improvements/Journal#May10 And corresponding code is in my git. Would you like to give me some help? ------------------ Shangtong Zhang,Second Year Undergraduate, School of Computer Science, Fudan University, China. -------------- next part

Olly Betts writes: > On Sun, Apr 10, 2016 at 04:47:01PM +0200, Jean-Francois Dockes wrote: > > Some might notice the 50% index size increase. Excessive index size is > > already one relatively rare, but recurring complaint. Except if I did > > something wrong: I'm actually quite surprised by it. > > Did you try compacting the resulting databases? > >

Hi, I'm trying to use custom weight with python binding. My test code is like this. class TinkerWeight(xapian.Weight): def __init__(self): pass def name(self): return "Tinker" def serialize(self): return "" def get_sumpart(*args): return 1 def get_maxpart(*args): return 1 def get_sumextra(*args):

Greetings, I've been wondering about the index format and backwards compatibility. We're using the dev version (for chert) and each svn up means that any indexes created prior to this revision cannot be read. Is this purely a cautious move to prevent errors, and, barring any obvious index format changes, can I safely force the current revision to read existing indexes? eg, by

Hi I see a changed behaviour in xts indexed on class Date in the latest versions, versus 2. It seems to be related to changes to/from daylight savings time, happens those weekends. Is it not intended that class Date be used like this, or is this new behaviour incorrect? Giles Example: > a<-as.Date(15423:15426) > x<-xts(seq_along(a),a) > print(x) [,1] 2012-03-24

On Thu Apr 23 00:21:30 2020, Olly Betts <olly at survex.com> wrote: > First question: what version of Xapian are you using? On my laptop it's 1.4.15 (arch linux) and the desktop runs 1.4.14 (Gentoo linux) > And second thing to check, are you committing each message separately? No, I sync with mbsync which dosnloads a bunch of mails, then I run notmuch new which indexes all in

Franz Fellner <alpine.art.de at gmail.com> writes: > > On Thu Apr 23 00:21:30 2020, Olly Betts <olly at survex.com> wrote: >> Then I'd try compacting the database (I think there's a "notmuch >> compact" subcommand to do this). > And there we go. Cured the issues. Dropped the very first indexing > from several minutes to 1.5 seconds on the

On Mon, Apr 20, 2020 at 11:36:36AM -0300, David Bremner wrote: > Franz Fellner <alpine.art.de at gmail.com> writes: > > > I also suffer from bad performance of notmuch new. I used notmuch > > some years ago and notmuch new always felt instantanious. Had to stop > > using it because internet was too slow to sync my mails :/ Now (with > > better internet and a