Hello,
I'd like to figure out how to set up a site running dovecot on multiple
computers for load balancing reasons.
I'm currently running dovecot-1.0.14 on a 8GB RAM, 2 quad-core 2.66Ghz Xeon
single server which runs FreeBSD 6.2-STABLE SMP.
. the mailbox format was mbox and I successfully migrated to Maildir only
. mailboxes are on an NFS NetApp server, attribute caching is off
. indexes are locally stored
. the number of users is around 2500, some of them having huge mailboxes (~ 1
or 2 GB)
. a large amount of users is using IMAP but there are also many POP3 users
. the passdb/userdb is openLDAP
. for now, I neither use namespaces, nor quotas nor deliver, nor any plugin,
but I plan to use namespaces and ACL to implement shared mailboxes.
The same machine is running postfix-2.4.6 with amavisd-new-2.5.4 (no
spamassassin) and clamav-0.93.1. I'm using procmail-3.22 as the LDA.
I installed dovecot-1.1.2 on non-standard ports for test and plan to switch to
it as I read on this list it performs better, load wise.
This setup (1.0.14) works great except that, compared to the time mailboxes
were in mbox format, the load average (as shown by 'top') goes higher
from a
significant amout on a regular basis and sometimes goes over the top.
On "normal" circumstances, the number of processes is around 1400 and
the load
oscillates between 1 to 10 and 30 to 60. Most of the time, I'd say the load
average is around 20.
Every friday a message (no attachement, decent size) is sent to all users : in
such circumstances (but only sometimes, not every time : let's say half the
time such a message is sent), the load goes so high that I have to stop dovecot
to let procmail deliver the message or even reboot the machine.
Sometimes, the load climbs up to something like 150, then goes back to the
'normal' case described above.
I know multi-master replication is on the roadmap. I know some dovecot sites
use in the meantime several dovecot servers. I'd like to know how those
sites
do the load balancing, the main problem beeing to be able to direct each user
to the same dovecot server each time as stated in the Wiki
(http://wiki.dovecot.org/NFS).
I don't think that DNS round robin would do the trick because some UA (for
instance Thunderbird) often open up to 5 connexions for the same user, unless
maybe such UA makes a single 'gethostbyname()', thus connecting 5 times
to the
same physical server ?
What are the options to achieve such a setup ? Any successful experiences ?
Thank you
--
Thomas Hummel | Institut Pasteur
<hummel at pasteur.fr> | P?le informatique - syst?mes et r?seau
On Thu, Jul 31, 2008 at 2:15 PM, Thomas Hummel <hummel at pasteur.fr> wrote:> Every friday a message (no attachement, decent size) is sent to all users : in > such circumstances (but only sometimes, not every time : let's say half the > time such a message is sent), the load goes so high that I have to stop dovecot > to let procmail deliver the message or even reboot the machine. >Do you use dovecot's delivery agent (LDA)? Dovecot's LDA updates dovecot's cache file correctly. If you use a different LDA, dovecot has to re-read directory contents every time (and update the cache file).> I know multi-master replication is on the roadmap. I know some dovecot sites > use in the meantime several dovecot servers. I'd like to know how those sites > do the load balancing, the main problem beeing to be able to direct each user > to the same dovecot server each time as stated in the Wiki > (http://wiki.dovecot.org/NFS). >If you use multiple servers you can use dovecot's proxy-feature to redirect the user to the correct server. You just need a database to tell dovecot (and the MTA) on how to find the correct location for that specific user. Chris
On Thu, Jul 31, 2008 at 7:15 AM, Thomas Hummel <hummel at pasteur.fr> wrote:> > I don't think that DNS round robin would do the trick because some UA (for > instance Thunderbird) often open up to 5 connexions for the same user, unless > maybe such UA makes a single 'gethostbyname()', thus connecting 5 times to the > same physical server ? >I am not seeing the issue, your seeing issues with the UA spreading the connects across multiple servers? If so are the message-ids different for imap://hummel@ on one machine versus another? The same filer for a backend right? I assume that authentication, and more, is caching on Dovecot. So the actual LDAP hits are solitary for the whole lifetime of the UA session most likely. Dovecot as a LDA is ideal, as stated above, it will do the initial indexing upon delivery of messages and dovecot-imap will fixup as needed. Perhaps the UUIDS for pop and imap can be looked at for speed ups. Having a lot of processes, and in what state are they in? If the bulk of them are running because of searches (like outlook crazy queries or virus checkers than scan 1gb of mail etc) Thats a problem that is a little outside load balancing it seems. I would see the pre-delivery being more of a issue than dovecot performance wise especially with the LDA, hefty caching, and good maildir file naming. -- Gabriel Millerd
Thomas Hummel wrote:> The same machine is running postfix-2.4.6 with amavisd-new-2.5.4 (no > spamassassin) and clamav-0.93.1. I'm using procmail-3.22 as the LDA. > > I installed dovecot-1.1.2 on non-standard ports for test and plan to > switch to > it as I read on this list it performs better, load wise.Also, Dovecot v1.1 should do better with NFS.> Every friday a message (no attachement, decent size) is sent to all users > : in > such circumstances (but only sometimes, not every time : let's say half > the > time such a message is sent), the load goes so high that I have to stop > dovecot > to let procmail deliver the message or even reboot the machine.How about doing some rate limiting in Postfix, or even moving that to a different server? Anders.