CentOS - Oct 2007 - Large scale Postfix/Cyrus email system for 100,000+ users

I'm trying to set up a large scale email system that supports 100,000+
IMAP accounts.  We have an existing frontend web interface that does a
lookup on a mysql db to figure out which IMAP server to connect to for
each user.  For the email infrastructure we have decided on Postfix
and Cyrus.  We have configured both to use mysql to get the virtual
user information.

Because of the way that the infrastructure is (biz reasons) we are not
doing shared storage, we have numerous IMAP servers that we distribute
accounts across.  As we add more users, we image up a new IMAP server.
 For our business's scaling purposes this was the best plan.

What I am having a problem is how do I get postfix to transfer the
email to the particular IMAP server that the user account is on.  I
know that I need to use lmtp and transport, but all the examples I
have seen show forwarding all email to 1 IMAP server.  I would like
Postfix to do a lookup for each mailbox and determine which IMAP
server to deliver it to.

Anyone have a working example that they could share?  It would be
greatly appreciated.

thanks
-matt

> What I am having a problem is how do I get postfix to transfer the
> email to the particular IMAP server that the user account is on.  I
> know that I need to use lmtp and transport, but all the examples I
> have seen show forwarding all email to 1 IMAP server.  I would like
> Postfix to do a lookup for each mailbox and determine which IMAP
> server to deliver it to.
> 
> Anyone have a working example that they could share?  It would be
> greatly appreciated.
Sorry, never did lmtp but if I read your post properly, you want to do a 
transport map lookup for each mailbox to get the correct lmtp entry.

I suggest using cdb for your transport map database and rebuilding it 
say every eight hours. cdb offers fast lookup and database building. You 
can store the entries in mysql and dump to a file for cdb creation. I do 
not suggest mysql for transport even if you are using mysql connection 
pooling and transport tables get called in a lot of places.

Matt Shields wrote:
> 
> I'm trying to set up a large scale email system that supports 100,000+
> IMAP accounts.  We have an existing frontend web interface that does a
> lookup on a mysql db to figure out which IMAP server to connect to for
> each user.  For the email infrastructure we have decided on Postfix
> and Cyrus.  We have configured both to use mysql to get the virtual
> user information.
> 
> Because of the way that the infrastructure is (biz reasons) we are not
> doing shared storage, we have numerous IMAP servers that we distribute
> accounts across.  As we add more users, we image up a new IMAP server.
>  For our business's scaling purposes this was the best plan.
> 
> What I am having a problem is how do I get postfix to transfer the
> email to the particular IMAP server that the user account is on.  I
> know that I need to use lmtp and transport, but all the examples I
> have seen show forwarding all email to 1 IMAP server.  I would like
> Postfix to do a lookup for each mailbox and determine which IMAP
> server to deliver it to.
> 
> Anyone have a working example that they could share?  It would be
> greatly appreciated.
http://www.postfix.org/MYSQL_README.html

Then you can create a view out of your existing data schema to fit
the postfix needed schema.

-Ross

______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.

Matt Shields wrote:
> I'm trying to set up a large scale email system that supports 100,000+
> IMAP accounts.  We have an existing frontend web interface that does a
> lookup on a mysql db to figure out which IMAP server to connect to for
> each user.  For the email infrastructure we have decided on Postfix
> and Cyrus.  We have configured both to use mysql to get the virtual
> user information.
> 
> Because of the way that the infrastructure is (biz reasons) we are not
> doing shared storage, we have numerous IMAP servers that we distribute
> accounts across.  As we add more users, we image up a new IMAP server.
>  For our business's scaling purposes this was the best plan.
> 
> What I am having a problem is how do I get postfix to transfer the
> email to the particular IMAP server that the user account is on.  I
> know that I need to use lmtp and transport, but all the examples I
> have seen show forwarding all email to 1 IMAP server.  I would like
> Postfix to do a lookup for each mailbox and determine which IMAP
> server to deliver it to.
There are primarily two ways:

[virtual aliase]
you can use virtual_alias_maps to redirect foo at example.com to
foo at hostN.example.com, provided the final server accepts such addresses.

If the final server doesn't accept these, and you use smtp to relay to,
then you can write the addresses back, using smtp_generic_maps.

[transport]
an laternative is to use use (per-user) transport_maps. something like

foo at example.com		relay:[hostN.example.com]


In bothe approaches, the mappings can be generated using sql statements
(mostly CONCAT). something like
...
query = SELECT concat('relay:[', host, '.example.com]')
	FROM User
	where '%u' = user and '%d' = domain

you get the idea I hope.


> 
> Anyone have a working example that they could share?  It would be
> greatly appreciated.
> 
> thanks
> -matt
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
> 
>

Matt Shields wrote:
> Because of the way that the infrastructure is (biz reasons) we are not
> doing shared storage, we have numerous IMAP servers that we distribute
> accounts across.  As we add more users, we image up a new IMAP server.
>  For our business's scaling purposes this was the best plan.
> 
> What I am having a problem is how do I get postfix to transfer the
> email to the particular IMAP server that the user account is on.  I
> know that I need to use lmtp and transport, but all the examples I
> have seen show forwarding all email to 1 IMAP server.  I would like
> Postfix to do a lookup for each mailbox and determine which IMAP
> server to deliver it to.
Having no idea how that fits into your already existing infrastructure,
but the Cyrus IMAPD Aggregator (also known as Cyrus IMAPD Murder) looks
like something which should be evaluated - you probably can even drop
the mysql database, as it really doesn't matter to which of the
lmtp/imapd proxies you connect to.

More information:

<http://cyrusimap.web.cmu.edu/ag.html>

Cheers,

Ralph
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL:
<http://lists.centos.org/pipermail/centos/attachments/20071024/5f480c9d/attachment-0004.sig>

Matt Shields wrote:
> I'm trying to set up a large scale email system that supports 100,000+
> IMAP accounts.  We have an existing frontend web interface that does a
> lookup on a mysql db to figure out which IMAP server to connect to for
> each user.  For the email infrastructure we have decided on Postfix
> and Cyrus.  We have configured both to use mysql to get the virtual
> user information.
> 
> Because of the way that the infrastructure is (biz reasons) we are not
> doing shared storage, we have numerous IMAP servers that we distribute
> accounts across.  As we add more users, we image up a new IMAP server.
>  For our business's scaling purposes this was the best plan.
> 
> What I am having a problem is how do I get postfix to transfer the
> email to the particular IMAP server that the user account is on.  I
> know that I need to use lmtp and transport, but all the examples I
> have seen show forwarding all email to 1 IMAP server.  I would like
> Postfix to do a lookup for each mailbox and determine which IMAP
> server to deliver it to.
I thought the usual ways of doing this were to either use a 
high-performance NFS server (netapp filer...) and maildir format so you 
can run imap from any client facing server, or to keep the delivery host 
information in an LDAP attribute that you find when validating the address.

-- 
   Les Mikesell
    lesmikesell at gmail.com

On 10/24/07, Ralph Angenendt <ra+centos at br-online.de>
wrote:
> Matt Shields wrote:
> > Because of the way that the infrastructure is (biz reasons) we are not
> > doing shared storage, we have numerous IMAP servers that we distribute
> > accounts across.  As we add more users, we image up a new IMAP server.
> >  For our business's scaling purposes this was the best plan.
> >
> > What I am having a problem is how do I get postfix to transfer the
> > email to the particular IMAP server that the user account is on.  I
> > know that I need to use lmtp and transport, but all the examples I
> > have seen show forwarding all email to 1 IMAP server.  I would like
> > Postfix to do a lookup for each mailbox and determine which IMAP
> > server to deliver it to.
>
> Having no idea how that fits into your already existing infrastructure,
> but the Cyrus IMAPD Aggregator (also known as Cyrus IMAPD Murder) looks
> like something which should be evaluated - you probably can even drop
> the mysql database, as it really doesn't matter to which of the
> lmtp/imapd proxies you connect to.
>From what I understand about Cyrus Murder, it is for replicating your
user data across multiple servers, which is good if you want to load
balance multiple IMAP servers and you don't have a shared storage
backend.

As mentioned we have a web frontend that checks mysql when the user
logs in to see which imap server the account resides on.


Everyone, I have figured it out.  I do plan on posting after I finish
documenting the steps (for those interested).

-matt