similar to: crash after running notmuch new

On Tue, Apr 07, 2020 at 05:21:47PM -0300, David Bremner wrote: > Matt <mattator at gmail.com> writes: [...] > > termlist: > > blocksize=8K items=186136 firstunused=62058 revision=421 levels=2 root=12260 > > B-tree checked okay > > termlist table structure checked OK > > > > postlist: > > blocksize=8K items=2598971 firstunused=61412 revision=421

Running notmuch from git on Debian testing[1] with the mail and database sitting on a ZFS filesystem, adding mail to a new database: > agrajag-testing ~/s/notmuch % ./notmuch new > Found 605510 total files (that's not much mail). > add_file: A Xapian exception occurred36m 37s remaining). > A Xapian exception occurred adding message: Unexpected end of posting list for

Javier Garcia <javiertury at gmail.com> writes: > I've applied the path to notmuch 0.26.1 without success. > > $ rm -rf ~/.mail/.notmuch > $ LD_LIBRARY_PATH=/hidden-path/notmuch-0.26.1/lib/:$LD_LIBRARY_PATH > ./notmuch new >    Found 20065 total files (that's not much mail). >    Processed 20065 total files in 58s (341 files/sec.). >    Added 19605 new

Mike Hommey <mh at glandium.org> writes: > Hi, > > When running `notmuch compact` today, it stopped with the following > output: > > Compacting database... > compacting table postlist > Reduced by 25% 648656K (2498904K -> 1850248K) > compacting table docdata > Reduced by 15% 24K (152K -> 128K) > compacting table termlist > Reduced by

Hi notmuch developers, I also had this database corruption, I waited for the fix to land in notmuch 0.26.2, build it, moved the xapian directory away, did a notmuch new and restored the tags from a dump. But the problem remains: ~$ xapian-check ~/Mail/.notmuch/xapian docdata: blocksize=8K items=10841 firstunused=75 revision=82 levels=1 root=2 B-tree checked okay docdata table structure checked

On Fri, Dec 29, 2017 at 03:00:47PM +0000, David Edmondson wrote: > Running notmuch from git on Debian testing[1] with the mail and database > sitting on a ZFS filesystem, adding mail to a new database: > > > agrajag-testing ~/s/notmuch % ./notmuch new > > Found 605510 total files (that's not much mail). > > add_file: A Xapian exception occurred36m 37s remaining).

Hello, I am trying to rebuild an index of 2+ million documents and have not been successful. I am running Python 2.7 Django 1.7 Haystack 2.1.1 Xapian 1.2.21 The index rebuild command I’m using is: django-admin.py rebuild_index --noinput --batch-size=100000 The rebuild completes but an immediate xapian-check returns this error: xapian-check ./archive_index record: baseB blocksize=8K

Unfortunately I can't share my emails without the approval of other parties. The minimum subsets that trigger the error are in the range of 1000-5000 mails, so asking each and everyone of them is out of my reach. I tried to replicate the problem using just spam folders without success. The following is a solid workaround I've stumbled upon. Afew no longer complains and database corruption

Antoine Beaupré <anarcat at orangeseeds.org> writes: > On 2018-03-19 13:36:49, David Bremner wrote: >> >> I can't duplicate that part. > > That's very strange. I can reproduce this on my workstation here, but > taking the tarball I sent in the original message, I can't reproduce > anymore. So something changed! I suspect it's the

On 2018-03-29 04:17:21, Olly Betts wrote: > On Mon, Mar 19, 2018 at 05:03:21PM -0300, David Bremner wrote: >> I can confirm this reproduces both the xapian-check and the notmuch-show >> error. Olly agrees that whatever notmuch is doing wrong, it shouldn't >> lead to a corrupted database > > There was a Xapian bug here, which I fixed on master last week and will >

Gregor Zattler <telegraph at gmx.net> writes: > Hi notmuch developers, > > I also had this database corruption, I waited for the fix to land > in notmuch 0.26.2, build it, moved the xapian directory away, did > a notmuch new and restored the tags from a dump. But the problem > remains: > > ~$ xapian-check ~/Mail/.notmuch/xapian > docdata: > blocksize=8K

On Mon, Jul 03, 2023 at 02:26:03PM +0200, David Bremner wrote: > "Peter P." <peterparker at fastmail.com> writes: > > > I ran xapian-check on ~/.notmuch/xapian and include its messages > > below at the end of this mail. Everyone please forgive me for > > pasting 1121 there. :) > > H'mm. It doesn't look familiar to me, but I will check with

Franz Fellner <alpine.art.de at gmail.com> writes: > I also suffer from bad performance of notmuch new. I used notmuch > some years ago and notmuch new always felt instantanious. Had to stop > using it because internet was too slow to sync my mails :/ Now (with > better internet and a completely new setup using mbsync) indexing one > mail takes at least 10 seconds,

Hi! Suppose you have a huge notmuch/Xapian database, built on a 32-bit system (well, actually on x86_64-pc-linux-gnu, but using a years old 32-bit notmuch binary; notmuch 0.9, Xapian 1.2.21 -- don't laugh), and suppose you're finally going to update that years old notmuch installation (release by release, forward-porting a bunch of patches). Naturally, I'd now do a native 64-bit

On Thu Apr 23 00:21:30 2020, Olly Betts <olly at survex.com> wrote: > First question: what version of Xapian are you using? On my laptop it's 1.4.15 (arch linux) and the desktop runs 1.4.14 (Gentoo linux) > And second thing to check, are you committing each message separately? No, I sync with mbsync which dosnloads a bunch of mails, then I run notmuch new which indexes all in

Mueen Nawaz <mueen at nawaz.org> writes: > After a lot of poking around, I figured out the problem, and this may be > of interest to the developers (although not sure if it is a xapian issue > or a notmuch issue). > > Here's why it would freeze: > > I have a post-new hook that runs a Python script. Depending on whether > the new email it is processing matches a

Olly Betts <olly at survex.com> writes: > > So the T00-new.sh numbers make sense - there's more work to do, and > we need to read existing positional data more to insert the new stuff, > so the increased reads and writes make sense. > > But guessing at what the other two tests do, I wouldn't expect them to > be affected by this. The non-optimized-away cases of

Hello, I'm having a litle issue with Xapian-1.0.6 with php bindings under php5 when trying to add a new document it throws: Fatal error: Uncaught exception 'Exception' with message 'DatabaseCorruptError: Failed to unlink /var/lib/xapian/trade.ar/termlist.baseA: No such file or directory' in /usr/share/php5/xapian.php:1140 Stack trace: #0 /usr/share/php5/xapian.php(1140):

Hi Olly, Thanks for the detailed response. I hadn’t realized there was a new xapian haystack backend. I’m going to try that but I have some upgrades to do first. Django 1.8, etc. Thanks, Ryan > On Feb 28, 2017, at 3:40 PM, Olly Betts <olly at survex.com> wrote: > > On Mon, Feb 27, 2017 at 10:29:46AM -0800, Ryan Cross wrote: >> I am trying to rebuild an index of 2+

David Bremner <david at tethera.net> writes: >> Here's why it would freeze: >> >> I have a post-new hook that runs a Python script. Depending on >> whether the new email it is processing matches a rule I have, >> it will fire off an email to the sender using the SMTP library >> in Python. >> >> I had recently upgraded my MTA