Does anyone have any idea of how many IMAP connections a single cloud VM (4 vCores at 2.4GHz, 30GB RAM, local SSD storage - non-RAID) can be expected to handle in production. The mailboxes are fairly small (average 5MB total - 50MB max, as I don?t store attachments in Dovecot expect those saved through IMAP in the Sent/Drafts folders) and each user will probably have an average of 2 devices that have the mail clients configured to access each mailbox. Can such a server handle 100,000 mailboxes (200,000 devices/clients)? Or is it more like 10,000? Or, even smaller? I can scale the cloud VM up to 32 vCores and 240GB RAM (at 8 times the price) or split the mailboxes onto multiple VMs. The VM will also be running LMTP and other Dovecot services (I don?t plan on supporting POP3 at this time). The mailboxes will be sync?d to a backup VM running Dovecot for high availability so has some load from this background activity. LMTP will not be that high a load, I think, since most messages will be delivered by at night. But, clients will have IMAP connections 24/7. Just trying to get an idea of the cost of running a potentially huge/growing mail service in the cloud? I?m going to have to support around a million mailboxes before the site will generate significant revenue to support operations. Kevin
Zitat von KT Walrus <kevin at my.walr.us>:> Does anyone have any idea of how many IMAP connections a single > cloud VM (4 vCores at 2.4GHz, 30GB RAM, local SSD storage - > non-RAID) can be expected to handle in production. The mailboxes are > fairly small (average 5MB total - 50MB max, as I don?t store > attachments in Dovecot expect those saved through IMAP in the > Sent/Drafts folders) and each user will probably have an average of > 2 devices that have the mail clients configured to access each > mailbox. > > Can such a server handle 100,000 mailboxes (200,000 > devices/clients)? Or is it more like 10,000? Or, even smaller?The bottleneck of IMAP-operations normally is the number of needed IOPS for accessing the mail storage, which depends on the mean number of concurrent users to be expected. Can such a server handle this number of mailboxes? Most certainly yes. The real question is: can such a server cope up with the expected load of concurrent user sessions you are expecting?
On 04 Jun 2016, at 21:28, KT Walrus <kevin at my.walr.us> wrote:> > Does anyone have any idea of how many IMAP connections a single cloud VM (4 vCores at 2.4GHz, 30GB RAM, local SSD storage - non-RAID) can be expected to handle in production. The mailboxes are fairly small (average 5MB total - 50MB max, as I don?t store attachments in Dovecot expect those saved through IMAP in the Sent/Drafts folders) and each user will probably have an average of 2 devices that have the mail clients configured to access each mailbox. > > Can such a server handle 100,000 mailboxes (200,000 devices/clients)? Or is it more like 10,000? Or, even smaller? > > I can scale the cloud VM up to 32 vCores and 240GB RAM (at 8 times the price) or split the mailboxes onto multiple VMs. The VM will also be running LMTP and other Dovecot services (I don?t plan on supporting POP3 at this time). The mailboxes will be sync?d to a backup VM running Dovecot for high availability so has some load from this background activity. LMTP will not be that high a load, I think, since most messages will be delivered by at night. But, clients will have IMAP connections 24/7. > > Just trying to get an idea of the cost of running a potentially huge/growing mail service in the cloud? I?m going to have to support around a million mailboxes before the site will generate significant revenue to support operations.Do you mean most of the IMAP clients will be IDLEing waiting for new mails, which mostly won't arrive until the next night? imap-hibernate feature will be very helpful there then. Bottlenecks are commonly either the disk IO or the memory usage. With SSD you're probably less likely to run bottleneck in disk IO. Memory usage mainly depends on the number of active (non-hibernated) concurrent connections and also the mailbox sizes of the users. I'd limit a single Dovecot VM to 64 GB of memory. Maybe more would work, but it might run into bottlenecks on the CPU usage side for services that are limited to a single process per instance. Replication with dsync is going to increase the load and I'm not sure how big of an issue that is. Anyway, if it's mostly IDLE connections, I'd expect 100k mailboxes/VM to be fine. Generally I'd expect about 10k active (non-hibernated) IMAP connections/VM for 32 GB of memory, but this depends a lot on the mailbox sizes.
> Anyway, if it's mostly IDLE connections, I'd expect 100k mailboxes/VM to be fine. Generally I'd expect about 10k active (non-hibernated) IMAP connections/VM for 32 GB of memory, but this depends a lot on the mailbox sizes.That is great news. 100k mailboxes/VM is a great number. I do expect most IMAP clients will be IDLEing. Do almost all email clients in use today do the IDLE command? Do most email clients open many connections per mailbox? Perhaps IDLEing on multiple namespaces/folders per mailbox? Would this affect your 100k mailboxes/VM estimate? The cloud VM at 4 vCores, 30GB RAM, and local SSD storage is just $40/month (OVH Public Cloud). I had expected a cost of 10 cents per mailbox per month (with redundancies raising that cost to 25 cents per mailbox per month). But 100k mailboxes/VM would give me a total operating cost of less than 1 cent per month per mailbox, at scale. Maybe even 10 cents per year per mailbox for Public Cloud hosting fees? Does anyone on this list run a large number of mailboxes per server in production? What is the largest number of Dovecot mailboxes/client connections you supported on a single server before you had to upgrade to multiple Dovecot servers? Kevin> On Jun 11, 2016, at 6:37 PM, Timo Sirainen <tss at iki.fi> wrote: > > On 04 Jun 2016, at 21:28, KT Walrus <kevin at my.walr.us <mailto:kevin at my.walr.us>> wrote: >> >> Does anyone have any idea of how many IMAP connections a single cloud VM (4 vCores at 2.4GHz, 30GB RAM, local SSD storage - non-RAID) can be expected to handle in production. The mailboxes are fairly small (average 5MB total - 50MB max, as I don?t store attachments in Dovecot expect those saved through IMAP in the Sent/Drafts folders) and each user will probably have an average of 2 devices that have the mail clients configured to access each mailbox. >> >> Can such a server handle 100,000 mailboxes (200,000 devices/clients)? Or is it more like 10,000? Or, even smaller? >> >> I can scale the cloud VM up to 32 vCores and 240GB RAM (at 8 times the price) or split the mailboxes onto multiple VMs. The VM will also be running LMTP and other Dovecot services (I don?t plan on supporting POP3 at this time). The mailboxes will be sync?d to a backup VM running Dovecot for high availability so has some load from this background activity. LMTP will not be that high a load, I think, since most messages will be delivered by at night. But, clients will have IMAP connections 24/7. >> >> Just trying to get an idea of the cost of running a potentially huge/growing mail service in the cloud? I?m going to have to support around a million mailboxes before the site will generate significant revenue to support operations. > > Do you mean most of the IMAP clients will be IDLEing waiting for new mails, which mostly won't arrive until the next night? imap-hibernate feature will be very helpful there then. > > Bottlenecks are commonly either the disk IO or the memory usage. With SSD you're probably less likely to run bottleneck in disk IO. Memory usage mainly depends on the number of active (non-hibernated) concurrent connections and also the mailbox sizes of the users. > > I'd limit a single Dovecot VM to 64 GB of memory. Maybe more would work, but it might run into bottlenecks on the CPU usage side for services that are limited to a single process per instance. > > Replication with dsync is going to increase the load and I'm not sure how big of an issue that is. > > Anyway, if it's mostly IDLE connections, I'd expect 100k mailboxes/VM to be fine. Generally I'd expect about 10k active (non-hibernated) IMAP connections/VM for 32 GB of memory, but this depends a lot on the mailbox sizes.