dovecot - Apr 2012 - Multiply mailboxes vs one huge

Hello,

as I need to store a lot of messages on my IMAP server (order of 
900K-1000K; this is an archive for some time, maybe a year or so), I see 
some "slowness" in dealing with such a huge amount. I mainly need to
do
searches like "get all messages from user1 at domain1.com to 
user2 at domain2.tld recieved between date1 and date2".

So I really interested will it be wise to

a) split all messages into several smaller mailboxes (per-month, or 
per-day, or create 2-level-structure like "month/day/")
b) use dbox (vs currently used mbox) storage scheme (I'm afraid of mdbox 
as I still not sure I'll be able to parse it by scripts later "just in 
case")

Dovecot is the latest one (2.1.3). No compression Dovecot-side, but it 
mails are in zfs volume with compression on. I ask this mainly due to my 
not fully understand how Dovecot indexes are working.

I also test another approach: to use my own index somewhere outside 
Dovecot which will store reference between emails and UIDs, and dates 
and UIDs, so I'll simple query my index for things I need. But then, 
that's exactly what IMAP index can do, so I simple slow my search  down, 
isn't it? The only reason I think about my own index is I won't use
'all
header' as search scope, I need to deal only with From:, To:, Cc:, Bcc: 
(if any), Recieved (if nowehere else I see the from/to info), and date 
field(s) - I doubt IMAP will care for that for me.

Yours,
   Alexander

On 9.4.2012, at 17.58, Alexander Chekalin wrote:
> as I need to store a lot of messages on my IMAP server (order of
900K-1000K; this is an archive for some time, maybe a year or so), I see some
"slowness" in dealing with such a huge amount. I mainly need to do
searches like "get all messages from user1 at domain1.com to user2 at
domain2.tld recieved between date1 and date2".
So by "received between date" you mean the IMAP INTERNALDATE as
opposed to Date: header? These kind of searches are looked up from the
index/cache files, and the performance should be exactly the same with all of
the mailbox formats. It would be useful to figure out what exactly is causing
the slowness. Is the SEARCH command slow? Something else? Is the slowness about
user CPU, system CPU or disk IO?