similar to: slowdown in notmuch perf suite with xapian 1.3.5

Olly Betts <olly at survex.com> writes: > > So the T00-new.sh numbers make sense - there's more work to do, and > we need to read existing positional data more to insert the new stuff, > so the increased reads and writes make sense. > > But guessing at what the other two tests do, I wouldn't expect them to > be affected by this. The non-optimized-away cases of

On Thu, Apr 07, 2016 at 09:40:59PM -0300, David Bremner wrote: > Olly Betts <olly at survex.com> writes: > > > > > So the T00-new.sh numbers make sense - there's more work to do, and > > we need to read existing positional data more to insert the new stuff, > > so the increased reads and writes make sense. > > > > But guessing at what the other

On Thu, Apr 07, 2016 at 08:56:46AM -0300, David Bremner wrote: > I hadn't noticed any interactive slowdown, but when I got around to > running the notmuch performance suite, there seems to be some noticable > slowdown with the glass backend (default in Xapian 1.3.5) compared to > chert (using xapian 1.2.22) Some of this is pretty much expected, though other parts I don't

Hi, R-users, I met a problem: Items:[Anna 'moi =) akku loppu joskus 4ltä. Kestää kauan nää..'\tAmer, Tuusula (0:20)\t20\t12\t16\t00\t00\t11]/Anne 'Ei jakoa,uus päivä muistio et 4n niin peruin. Hups'\t (0:16)\t0\t12\t18\t00\t00\t11/Elina 'Konsertissa. En tod. vastaa teille'\tEtu-Töölö, Helsinki (2:40)\t24\t12\t18\t00\t00\t11 I want to parsing the above data into the

Hi! Suppose you have a huge notmuch/Xapian database, built on a 32-bit system (well, actually on x86_64-pc-linux-gnu, but using a years old 32-bit notmuch binary; notmuch 0.9, Xapian 1.2.21 -- don't laugh), and suppose you're finally going to update that years old notmuch installation (release by release, forward-porting a bunch of patches). Naturally, I'd now do a native 64-bit

Thomas Schwinge <thomas at schwinge.name> writes: > sizes?). Doing some light (read-only!) testing, it seems to work fine to > access the old 32-bit built Xapian "chert" database with 64-bit > notmuch/Xapian. Is that (a) generally safe and expected to work fine, > (b) there may be issues, or (c) don't do that, or don't know? > IIRC, Olly previously

On Friday, 2017-12-29 at 22:23:01 UTC, Olly Betts wrote: >> After the failure the database appears to be okay: >> >> > agrajag-testing ~/s/notmuch % xapian-check ~/Maildir/.notmuch/xapian >> > ... >> > position table structure checked OK > > This seems to be for an almost empty database (2 items in the postlist > table and nothing anywhere else)

Running notmuch from git on Debian testing[1] with the mail and database sitting on a ZFS filesystem, adding mail to a new database: > agrajag-testing ~/s/notmuch % ./notmuch new > Found 605510 total files (that's not much mail). > add_file: A Xapian exception occurred36m 37s remaining). > A Xapian exception occurred adding message: Unexpected end of posting list for

On Fri, Dec 29, 2017 at 03:00:47PM +0000, David Edmondson wrote: > Running notmuch from git on Debian testing[1] with the mail and database > sitting on a ZFS filesystem, adding mail to a new database: > > > agrajag-testing ~/s/notmuch % ./notmuch new > > Found 605510 total files (that's not much mail). > > add_file: A Xapian exception occurred36m 37s remaining).

David Bremner <david at tethera.net> writes: >> Here's why it would freeze: >> >> I have a post-new hook that runs a Python script. Depending on >> whether the new email it is processing matches a rule I have, >> it will fire off an email to the sender using the SMTP library >> in Python. >> >> I had recently upgraded my MTA

On Mon, Apr 20, 2020 at 11:36:36AM -0300, David Bremner wrote: > Franz Fellner <alpine.art.de at gmail.com> writes: > > > I also suffer from bad performance of notmuch new. I used notmuch > > some years ago and notmuch new always felt instantanious. Had to stop > > using it because internet was too slow to sync my mails :/ Now (with > > better internet and a

On Thu Apr 23 00:21:30 2020, Olly Betts <olly at survex.com> wrote: > First question: what version of Xapian are you using? On my laptop it's 1.4.15 (arch linux) and the desktop runs 1.4.14 (Gentoo linux) > And second thing to check, are you committing each message separately? No, I sync with mbsync which dosnloads a bunch of mails, then I run notmuch new which indexes all in

On Tue, Apr 07, 2020 at 05:21:47PM -0300, David Bremner wrote: > Matt <mattator at gmail.com> writes: [...] > > termlist: > > blocksize=8K items=186136 firstunused=62058 revision=421 levels=2 root=12260 > > B-tree checked okay > > termlist table structure checked OK > > > > postlist: > > blocksize=8K items=2598971 firstunused=61412 revision=421

Franz Fellner <alpine.art.de at gmail.com> writes: > > On Thu Apr 23 00:21:30 2020, Olly Betts <olly at survex.com> wrote: >> Then I'd try compacting the database (I think there's a "notmuch >> compact" subcommand to do this). > And there we go. Cured the issues. Dropped the very first indexing > from several minutes to 1.5 seconds on the

Daniel Barlow <dan at telent.net> writes: > There are a number of ways this is currently insecure but the particular > one I want to ask about today is running the notmuch cli commands with > user-supplied arguments and whether there are any particular gotchas in > doing so? I am reasonably sure that my code to invoke notmuch(1) is > calling execve(2) without invoking

Hi all, notmuch[1] becomes more and more popular. At least in geek community. But notmuch is local-only (except ssh)... It looks like it's hard (if even possible) to use IMAP as transport for notmuch-alike work flow. I think it would be great to create a new protocol based global search and message tagging. All core notmuch operations (search, show, count, tag) can be mapped to the new

Mueen Nawaz <mueen at nawaz.org> writes: > After a lot of poking around, I figured out the problem, and this may be > of interest to the developers (although not sure if it is a xapian issue > or a notmuch issue). > > Here's why it would freeze: > > I have a post-new hook that runs a Python script. Depending on whether > the new email it is processing matches a

Mueen Nawaz <mueen at nawaz.org> writes: > > DATABASE = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE) OK. So your code is locking the database, and never unlocking it (because of the hang). So that part is at least not mysterious. > I can think of two experiments: I was thinking more along the lines of something that could be part of the notmuch test suite, i.e. run in

Matt <mattator at gmail.com> writes: > thanks didn't know about xapian-check ! > the output > === > docdata: > blocksize=8K items=70 firstunused=3 revision=421 levels=0 root=2 > B-tree checked okay > docdata table structure checked OK > > termlist: > blocksize=8K items=186136 firstunused=62058 revision=421 levels=2 root=12260 > B-tree checked okay >

Franz Fellner <alpine.art.de at gmail.com> writes: > I also suffer from bad performance of notmuch new. I used notmuch > some years ago and notmuch new always felt instantanious. Had to stop > using it because internet was too slow to sync my mails :/ Now (with > better internet and a completely new setup using mbsync) indexing one > mail takes at least 10 seconds,