similar to: empty FD after reopen since version 1.2.16

I took one of the examples and tried to run against my database ls -l /data1/mail/db/cur.1 total 1129624 -rw-r--r-- 1 jwl jwl 0 2011-06-09 02:27 flintlock -rw-r--r-- 1 jwl jwl 28 2011-06-09 02:27 iamchert -rwxrwxrwx 1 jwl jwl 7258 2011-06-09 02:27 position.baseA -rwxrwxrwx 1 jwl jwl 7046 2011-06-09 02:27 position.baseB -rwxrwxrwx 1 jwl jwl 474226688 2011-06-09 02:28

We've had some catastrophic filesystem failures that have left us with corrupted databases with empty files and no backup for about 15TB of our data. Recreating the 15TB from source data backups is possible but will take a very very long time. I'm hoping that, given all of the .DB files are still intact, there my be some way to extract their contents and rebuild the other tables. This

It took approximately five days, having single process using one core CPU and 6GB of memory to build this giant 607GB single Xapian index, containing 219 million of unique documents (web sites). So far I did not found any other implementation that would enable me to build such a single index containing over 200 million documents, while testing Lucene, Solr, MySQL, Hadoop and Oracle. Probably

Xapian Index 253 million documents = 704G I just build my largest single Xapian index with 253 million unique documents on single server using single hard disk, less that 8G RAM and single processor 2.0 GHz. I do not see any search performance decreases in searching my indexes between 100 million and 250 million, which indicates a good scalability of Xapian and it looks like, I can push it easily

Xapians, I am maintaining about two indexes for my search engines which approximately is each the same size. I would like to share this knowledge with you, since many of you have never seen Xapian index of this size. And of course you can search the index by yourself at - http://myhealthcare.com/ - http://find1friend.com/ I need 2 x 100 million more documents into each index, and I hope it will

I think this is a shining example of how well Xapian works with large document collections. I was just discussing this with my colleagues here and one of the issues that came up is that we'd love Xapian to become really lot more popular but have found that the documentation's a bit difficult to get into, as is the API. So I was wondering: do you have any thoughts on improving this and

hi, i use xapian to index a txt file, it's size is 268M. i take each line as a document, and each line has two field like 13445511 | 111115151. the recored size is 10000000. the XAPIAN_FLUSH_THRESHOLD set 1000000. it takes 1026544ms to index the file, it is more slower than lucene. The lucene speed is about 40000 records per second. code: try { Xapian::WritableDatabase

Folks, Having trouble getting this to work. OMEGA cgi is not reading my stub file properly because it is trying to read it as a directory instead of a file. Is there an easy fix? Here is a transcript. Thanks, OSC oscar@epsilon:/svr/xapian/beta$ ls -aFl total 21335200 drwxr-xr-x 2 oscar oscar 4096 Aug 6 10:15 ./ drwxr-xr-x 5 oscar oscar 4096 Aug 6 12:59 ../ lrwxrwxrwx 1 oscar

> I think this is a shining example of how well Xapian works with large > document collections. I was just discussing this with my colleagues here > and one of the issues that came up is that we'd love Xapian to become > really lot more popular but have found that the documentation's a bit > difficult to get into, as is the API. I agree. There are a few gotchas, as well

On Mon, Apr 27, 2015 at 02:23:25PM +0200, matthias lay wrote: > after upgrading xapian I encountered the same problem as described in > ticket > #645 Read block errors after reopen() > > in our setup its 100% reproducible after each reopen(). I downgraded > again and it seems the problem occurs in Version 1.2.16 and above. > in <=1.2.15 everything works fine without seeing

Most Xapian database files are locked while the database is open, but it seems that .baseA and .baseB files are not, so any other application can delete them (I am talking about the Windows package). Is there a way to protect them as rest of the Xapian database files? Regards, PK

I found something interesting via strace. lda is writing a timestamp with utime before doign the fsync, but I'm really not a C guy, so I have no idea why that's going on via procmail and not via commandline. I assume it's related to the choice of pread64 vs read. when called from commandline (working): read(0, "July 14-20, 2013\n10 courses. Bon"..., 4096) = 4096

I've just noticed that new indexes no longer have position.{DB,baseA,baseB} files, all previous indexes (I roll indexes every week using xapian-compact) have the position files. The index seems to work but it is returning some odd results, for example if I run a query with the phrase "machine learning" it mostly returns documents containing "machine learning" but it also

Hello, i sync with imapsync from an archiopteryx-server to an dovecot-server, for testing until now. The dovecot is inside an VM. storage for mail is on an nfs-share, indexes are local. What i now see, is, on large folder( 58000 messages and more) the load on the imap-process is very high, and slowdown the transferprocess. Is there something, I can do. Whats cause the high load ind the

Hi, I'm using the zlib plugin with dovecot 1.1.2 (on Centos 4.5, x86, ext3) and some bzip2 (version 1.0.2) compressed mail files are causing the following error to show in the logs when I click on the folder in outlook express. Aug 27 11:39:15 server-2 dovecot: IMAP(marchive): fstat(maildir) failed: Bad file descriptor Aug 27 11:39:17 Info: IMAP(marchive): Loading modules from directory:

Hi, A Recoll user is reporting an index corruption problem. In general, index corruption happens from time to time with Recoll, because of crashes, reboots, misc Recoll bugs, etc. The strange thing here is that xapian-check does not seem to detect anything. In a nutshell, some document numbers seem to point to a data blackhole: the docids are returned when searching for the file/doc unique

Olly Betts writes: > On Mon, Apr 11, 2016 at 09:54:36AM +0200, Jean-Francois Dockes wrote: > > The question which remains for me is if I should run xapian-compact > > after an initial indexing operation. I guess that this depends on the > > amount of expected updates and that there is no easy answer ? > > I think it's not obvious whether it's a good plan

Hi I just tried to run the convert plugin as described at http://wiki.dovecot.org/Plugins/Convert (except with mail_location = maildir:~/Mail) It fails with an error message: Eopen(/.temp.falcon.endbracket.net.18618.8d5e0a038da6cf06) failed: Permission denied Error: imap dump-capability process returned 89 It looks like Dovecot execs /usr/libexec/dovecot/imap, which drops root privileges

Experts, (Now that I am able to run imap...) Here's what I am seeing with imap, it seems to die in the latest nightly. (tss, I am still trying to get dbx to work properly, maybe it won't, but I got some more details here.) ... 2 select "INBOX" * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.

Olly Betts writes: > On Wed, May 17, 2017 at 09:08:32PM +0200, Jean-Francois Dockes wrote: > > I have a user reporting the following error during recoll indexing: > > > > flush() failed: Db block overwritten - are there multiple writers? > > > > "flush() failed" is from recoll, the rest is, I think the text of the Xapian > > exception.