similar to: notmuch: Xapian exception during database creation

On Fri, Dec 29, 2017 at 03:00:47PM +0000, David Edmondson wrote: > Running notmuch from git on Debian testing[1] with the mail and database > sitting on a ZFS filesystem, adding mail to a new database: > > > agrajag-testing ~/s/notmuch % ./notmuch new > > Found 605510 total files (that's not much mail). > > add_file: A Xapian exception occurred36m 37s remaining).

Matt <mattator at gmail.com> writes: > thanks didn't know about xapian-check ! > the output > === > docdata: > blocksize=8K items=70 firstunused=3 revision=421 levels=0 root=2 > B-tree checked okay > docdata table structure checked OK > > termlist: > blocksize=8K items=186136 firstunused=62058 revision=421 levels=2 root=12260 > B-tree checked okay >

Mike Hommey <mh at glandium.org> writes: > Hi, > > When running `notmuch compact` today, it stopped with the following > output: > > Compacting database... > compacting table postlist > Reduced by 25% 648656K (2498904K -> 1850248K) > compacting table docdata > Reduced by 15% 24K (152K -> 128K) > compacting table termlist > Reduced by

Hi notmuch developers, I also had this database corruption, I waited for the fix to land in notmuch 0.26.2, build it, moved the xapian directory away, did a notmuch new and restored the tags from a dump. But the problem remains: ~$ xapian-check ~/Mail/.notmuch/xapian docdata: blocksize=8K items=10841 firstunused=75 revision=82 levels=1 root=2 B-tree checked okay docdata table structure checked

Javier Garcia <javiertury at gmail.com> writes: > I've applied the path to notmuch 0.26.1 without success. > > $ rm -rf ~/.mail/.notmuch > $ LD_LIBRARY_PATH=/hidden-path/notmuch-0.26.1/lib/:$LD_LIBRARY_PATH > ./notmuch new >    Found 20065 total files (that's not much mail). >    Processed 20065 total files in 58s (341 files/sec.). >    Added 19605 new

Unfortunately I can't share my emails without the approval of other parties. The minimum subsets that trigger the error are in the range of 1000-5000 mails, so asking each and everyone of them is out of my reach. I tried to replicate the problem using just spam folders without success. The following is a solid workaround I've stumbled upon. Afew no longer complains and database corruption

On Tue, Apr 07, 2020 at 05:21:47PM -0300, David Bremner wrote: > Matt <mattator at gmail.com> writes: [...] > > termlist: > > blocksize=8K items=186136 firstunused=62058 revision=421 levels=2 root=12260 > > B-tree checked okay > > termlist table structure checked OK > > > > postlist: > > blocksize=8K items=2598971 firstunused=61412 revision=421

On Friday, 2017-12-29 at 22:23:01 UTC, Olly Betts wrote: >> After the failure the database appears to be okay: >> >> > agrajag-testing ~/s/notmuch % xapian-check ~/Maildir/.notmuch/xapian >> > ... >> > position table structure checked OK > > This seems to be for an almost empty database (2 items in the postlist > table and nothing anywhere else)

Gregor Zattler <telegraph at gmx.net> writes: > Hi notmuch developers, > > I also had this database corruption, I waited for the fix to land > in notmuch 0.26.2, build it, moved the xapian directory away, did > a notmuch new and restored the tags from a dump. But the problem > remains: > > ~$ xapian-check ~/Mail/.notmuch/xapian > docdata: > blocksize=8K

Hi! Suppose you have a huge notmuch/Xapian database, built on a 32-bit system (well, actually on x86_64-pc-linux-gnu, but using a years old 32-bit notmuch binary; notmuch 0.9, Xapian 1.2.21 -- don't laugh), and suppose you're finally going to update that years old notmuch installation (release by release, forward-porting a bunch of patches). Naturally, I'd now do a native 64-bit

On Thu Apr 23 00:21:30 2020, Olly Betts <olly at survex.com> wrote: > First question: what version of Xapian are you using? On my laptop it's 1.4.15 (arch linux) and the desktop runs 1.4.14 (Gentoo linux) > And second thing to check, are you committing each message separately? No, I sync with mbsync which dosnloads a bunch of mails, then I run notmuch new which indexes all in

Mueen Nawaz <mueen at nawaz.org> writes: > After a lot of poking around, I figured out the problem, and this may be > of interest to the developers (although not sure if it is a xapian issue > or a notmuch issue). > > Here's why it would freeze: > > I have a post-new hook that runs a Python script. Depending on whether > the new email it is processing matches a

Hello, I am trying to rebuild an index of 2+ million documents and have not been successful. I am running Python 2.7 Django 1.7 Haystack 2.1.1 Xapian 1.2.21 The index rebuild command I’m using is: django-admin.py rebuild_index --noinput --batch-size=100000 The rebuild completes but an immediate xapian-check returns this error: xapian-check ./archive_index record: baseB blocksize=8K

Franz Fellner <alpine.art.de at gmail.com> writes: > I also suffer from bad performance of notmuch new. I used notmuch > some years ago and notmuch new always felt instantanious. Had to stop > using it because internet was too slow to sync my mails :/ Now (with > better internet and a completely new setup using mbsync) indexing one > mail takes at least 10 seconds,

David Bremner <david at tethera.net> writes: >> Here's why it would freeze: >> >> I have a post-new hook that runs a Python script. Depending on >> whether the new email it is processing matches a rule I have, >> it will fire off an email to the sender using the SMTP library >> in Python. >> >> I had recently upgraded my MTA

I hadn't noticed any interactive slowdown, but when I got around to running the notmuch performance suite, there seems to be some noticable slowdown with the glass backend (default in Xapian 1.3.5) compared to chert (using xapian 1.2.22) These tests are on an older i7 with 12G of RAM and an SSD. I'm reasonable confident they are CPU bound. One curious thing is the increase in system time

Hello, Really stumped on this issue. I have samba 4.4.7 running on a new server. Users cannot write to files to which they have write permissions via group. Example: Here's the local filesystem on the samba server. I'm logged in as jmalone : jmalone at canis; cd /home/www.nrao.edu/content/logs/ : jmalone at canis; ls -l total 4 -rw-rw-r-- 1 jmalone nraoweb 0 Nov 10 10:02

On Mon, Apr 20, 2020 at 11:36:36AM -0300, David Bremner wrote: > Franz Fellner <alpine.art.de at gmail.com> writes: > > > I also suffer from bad performance of notmuch new. I used notmuch > > some years ago and notmuch new always felt instantanious. Had to stop > > using it because internet was too slow to sync my mails :/ Now (with > > better internet and a

Olly Betts <olly at survex.com> writes: > > So the T00-new.sh numbers make sense - there's more work to do, and > we need to read existing positional data more to insert the new stuff, > so the increased reads and writes make sense. > > But guessing at what the other two tests do, I wouldn't expect them to > be affected by this. The non-optimized-away cases of

Franz Fellner <alpine.art.de at gmail.com> writes: > > On Thu Apr 23 00:21:30 2020, Olly Betts <olly at survex.com> wrote: >> Then I'd try compacting the database (I think there's a "notmuch >> compact" subcommand to do this). > And there we go. Cured the issues. Dropped the very first indexing > from several minutes to 1.5 seconds on the