dovecot - Apr 2009 - Indexing of mails to speed up the IMAP SEARCH command

Hi all,

I use a local dovecot server which is synchronized with my two imap
accounts using OfflineIMAP.  This works very nice and is highly usable.

But one thing I'd like to improve is the slow IMAP search.  When I
search for a string in the subjects of all messages in a mailbox using
some mail client, dovecot seems to grep all the messages in there.

Is there a way to let dovecot index more informations which are then
used to speed up the SEARCH command?

Bye,
Tassilo

On Wed, 2009-04-08 at 21:09 +0200, Tassilo Horn wrote:
> I use a local dovecot server which is synchronized with my two imap
> accounts using OfflineIMAP.  This works very nice and is highly usable.
> 
> But one thing I'd like to improve is the slow IMAP search.  When I
> search for a string in the subjects of all messages in a mailbox using
> some mail client, dovecot seems to grep all the messages in there.
Subject (or any other header) search should be fast, at least after the
first one. The subjects should then (if not before) be stored in
dovecot.index.cache file, and the search should be over in less than a
second even with tens of thousands of messages. If this isn't happening
with you, something's wrong. What Dovecot version are you using?

For message body indexing there are a couple of choices:
http://wiki.dovecot.org/Plugins/FTS
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20090408/e16ef58e/attachment-0002.bin>

Timo Sirainen wrote:
> For message body indexing there are a couple of choices:
> http://wiki.dovecot.org/Plugins/FTS
>   
Just curious, what is the average disk usage for FTS? I know it's 
usually around 10-15% of an inbox for standard indexing. I'm assuming 
FTS is considerably more.

-- 
===============================David Halik
System Administrator
OIT-CSS Rutgers University
dhalik at jla.rutgers.edu
================================

Timo Sirainen <tss at iki.fi> writes:

Hi Timo,
> On Wed, 2009-04-08 at 21:09 +0200, Tassilo Horn wrote:
>> I use a local dovecot server which is synchronized with my two imap
>> accounts using OfflineIMAP.  This works very nice and is highly
>> usable.
>> 
>> But one thing I'd like to improve is the slow IMAP search.  When I
>> search for a string in the subjects of all messages in a mailbox
>> using some mail client, dovecot seems to grep all the messages in
>> there.
>
> Subject (or any other header) search should be fast, at least after
> the first one.  The subjects should then (if not before) be stored in
> dovecot.index.cache file, and the search should be over in less than a
> second even with tens of thousands of messages.  If this isn't
> happening with you, something's wrong.
Oh, you proved me wrong.  I tried on a folder with about 20.000 messages
and it took about 2 seconds.  So searching for subject is reasonably
fast.
> What Dovecot version are you using?
1.1.13
> For message body indexing there are a couple of choices:
> http://wiki.dovecot.org/Plugins/FTS
Great, that's exactly what I was looking for!

Thanks a lot!
Tassilo
-- 
Chuck Norris actually built the stairway to heaven.

On 4/8/2009, Timo Sirainen (tss at iki.fi) wrote:
> For message body indexing there are a couple of choices:
> http://wiki.dovecot.org/Plugins/FTS
On this page it says that the indexes are not updated with deliver and
suggests using a cron job...

Is this planned to be added anytime soon (1.2? 1.3?)? Or should I go
ahead and plan on doing this via cron as suggested?

I prefer not to do things via cron unless absolutely necessary is why
I'm asking...

Thanks...

-- 

Best regards,

Charles