dovecot - Dec 2010 - Question about "slow" storage but fast cpus, plenty of ram and dovecot

Hello 

 We are using dovecot 1.2.x. In our setup we will have 1200
concurrent imap users (maildirs) and we have 2xraid5 sas 15k diks
mounted by iSCSI. The dovecot server (RHEL 5 x64) is 

a virtual machine
in our vmware esx cluster. We want to minimize disk I/O, what config
options should we use. We can "exchange" CPU & RAM to minimize
disk i/o.


 Should we change to dovecot 2.0? Maybe mdbox can help us? Maybe ext4
instead of ext3? Any idea is welcome. 

 Regards 

 Javier

Quoting javierdemiguel at us.es:
> in our vmware esx cluster. We want to minimize disk I/O, what config
> options should we use. We can "exchange" CPU & RAM to
minimize disk i/o.
Depends on what you are doing -- pop3, imap, both, deliver or some other
LDA?  Do you care if the indexes are lost on reboot or not?

You might try putting the indexes in, memory (either via dovecot settings
or a RAM DISK) or on SSD.

You could also try using ext3/4 with an external journal on a SSD.

"SSD" would preferably be an enterprise SSD, but it could be a lesser
SSD, or even a USB memory stick (replaced periodically).  Which is right
depends on your needs and budget.

Failing that, you should probably put the indexes on the fastest disks
possible (might be local, might be iscsi, you'd have to benchmark).
>  Should we change to dovecot 2.0?
For any new system, I'd start with the most recent dovecot 2.x version.
How easy that is if you are upgrading depends on what version you run now.

-- 
Eric Rostetter
The Department of Physics
The University of Texas at Austin

Go Longhorns!

Patrick Westenberg put forth on 12/11/2010 5:12 AM:
> Stan Hoeppner schrieb:
> 
>> So, either:
>>
>> 2.  Move indexes to memory
> 
> What steps have to be done and what will the configuration look like
> to have your indexes in memory?
Regarding Dovecot 1.2.x, for maildir, I believe it would be something
like this:

mail_location = maildir:~/Maildir:INDEX=MEMORY

The ":INDEX=MEMORY" disables writing the index files to disk, and as
the
name implies, I believe, simply keeps indexes in memory.

The docs say:

"If you really want to, you can also disable the index files completely
by appending :INDEX=MEMORY."

My read of that is that indexing isn't disabled completely, merely
storing the indexes to disk is disables.  The indexes are still built
and maintained in memory.

Timo, is that correct?


Also, due to the potential size of the index files (mine alone are 276
MB on an 877 MB mbox), you'll need to do some additional research to see
if this is a possibility for you.  If using a ramdisk, 100 mail boxen
like mine would require ~ 27.6 GB or RAM just for the index files.  This
would not be logical to do or feasible given the amount of RAM required.
 I don't know if, or how much, storing them in RAM via :INDEX=MEMORY
consumes, as compared to using a ramdisk.  The memory consumption may be
less or it may be more.  Timo should be able to answer this, and give a
recommendation as to whether this is even a sane thing to do.

-- 
Stan