similar to: Scalability with high density servers and proxies, TCP port limits

On Fri, 03 Jul 2015 07:05:43 +0200 Urban Loesch wrote: > Hi, > > Am 03.07.2015 um 05:14 schrieb Christian Balzer: > > > > > 2. Here is where the fun starts. > > Each IMAP session that gets proxied to the real mailbox server needs a > > port for the outgoing connection. > > So to support 2 million sessions we need 40 IP addresses here. Ouch. > >

Hi, Am 03.07.2015 um 05:14 schrieb Christian Balzer: > > 2. Here is where the fun starts. > Each IMAP session that gets proxied to the real mailbox server needs a > port for the outgoing connection. > So to support 2 million sessions we need 40 IP addresses here. Ouch. > And from a brief test having multiple IP addresses per server won't help > either (Dovecot

Thanks for the info. I do have one further question for you. On your servers that are currently handling 50k IMAP sessions, how many users does that correspond to? Since many users will have multiple IMAP sessions on multiple devices, I?d like to hear about some real-world numbers that could be used for budgeting a new project like mine. Also, do you use Dovecot IMAP proxies in front of your

Hello, Just to follow up on this, we've hit over 16k (default client limit here) hibernated sessions: --- dovecot 119157 0.1 0.0 63404 56140 ? S Apr01 62:05 dovecot/imap-hibernate [11291 connections] dovecot 877825 0.2 0.0 28512 21224 ? S Apr23 1:34 dovecot/imap-hibernate [5420 connections] --- No issues other than the minor bug I reported, CPU usage is

thanks for your help happy to say that the performance dramatically improved after i use the high performance settings from here http://wiki.dovecot.org/LoginProcess grep Login: /var/log/mail.log.1 |wc -l with the mail.log being of a typical, busy day. 412992 i also picked up the imap and pop3 connections during peak hours [root at ns1 domains]# doveadm who | awk

On 6 Apr 2017, at 9.56, Christian Balzer <chibi at gol.com> wrote: > >> For no particular reason besides wanting to start conservatively, we've got >> client_limit set to 50 on the hibernate procs (with 1100 total hibernated >> connections on the box I'm looking at). At only a little over a meg each, >> I'm fine with those extra processes. >>

christian the servers i currently own are dell servers. The servers i plan to buy are Dell R530, 2U rack servers with 8 x 3.5 inch drives, with 64 gb ram each, Hardware raid. I am thinking of 2 X 300 gb ssds raid1 and 6 x 2 tb drives in raid10 for data. I do not have any experience in setting up drdb (that would be my next step) ... primarily using standalone servers with hardware level

On Mon, 10 Apr 2017 23:11:24 +0300 Timo Sirainen wrote: > On 10 Apr 2017, at 21.49, Mark Moseley <moseleymark at gmail.com> wrote: > > > > Timo, any sense on where (if any) the point is where there are so many > > connections on a given login process that it would get too busy to keep up? > > I.e. where the sheer amount of stuff the login process has to do

Hello, We've got a pretty substantial dovecot installation (versions 2.1.7 and 2.2.13 on the backends, but tested with 2.2.24 as well) and this is the relevant config snippet from 10-master.conf: --- default_process_limit = 1024 default_client_limit = 16384 [...] service imap { # Most of the memory goes to mmap()ing files. You may need to increase this # limit if you have huge

> 1. 256GB of real RAM, swap is for chums. Are you sure that 100,000 IMAP sessions wouldn?t work well with SWAP, especially with fast SSD storage (which is a lot cheaper than RAM)? Seems that these IMAP processes are long lived processes (idling most of the time) that don?t need that much of the contents of real memory available for much of the life of the process. I use a database proxy in

Hello, On Thu, 6 Apr 2017 09:24:23 +0300 Aki Tuomi wrote: > On 06.04.2017 07:02, Christian Balzer wrote: > > > > Hello, > > > > this is on Debian Jessie box, dovecot 2.2.27 from backports, > > imap-hibernate obviously enabled. > > > > I've been seeing a few of these since starting this cluster (see previous > > mail), they all follow the

> 1500 IMAP sessions will eat up about 3GB alone. Are you saying that Dovecot needs 2MB of physical memory per IMAP session? If I want to support a max 100,000 IMAP sessions per server, I should configure the server to have at least 200GBs of SWAP? > On Feb 10, 2017, at 3:58 AM, Christian Balzer <chibi at gol.com> wrote: > > On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote:

Hello, Dovecot 2.2.27 (Debian Jessie backports). When issuing a "doveadm kick" for a specific user I was greeted by: --- "warning: other connections would also be kicked from following users:" --- and a list of 22odd thousand users. As it turns out, kick wants to smack the hibernation proces(ses) that have hibernated sessions for this user, obviously the wrong approach here.

Hello, The users here have been migrated to the new mail system with dovecot 0.99.10.5 (Debian package) on the backends. Storage is maildir, only pop3 access for now. I see very seldom occurrences (compared to the session wide pop lock of qpopper on the old system) like this: --- Jun 16 14:35:52 mb01 pop3(user at gol.com): Timeout while waiting for release of exclusive fcntl() lock for index

Hello, this is on Debian Jessie box, dovecot 2.2.27 from backports, imap-hibernate obviously enabled. I've been seeing a few of these since starting this cluster (see previous mail), they all follow the same pattern, a user who accesses their mailbox with both POP3 and IMAP deletes mails with POP3 and the IMAP (imap-hibernate really) is getting confused and upset about this: --- Apr 6

Hello, On Thu, 6 Apr 2017 20:04:38 +0900 Christian Balzer wrote: > Hello Aki, Timo, > > according to git this fix should be in 2.2.27, which I'm running, so I > guess this isn't it or something else is missing. > See: > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=859700 > Any update on this? Christian > Regards, > > Christian > > On Thu, 6

On 6 Apr 2017, at 21.14, Mark Moseley <moseleymark at gmail.com> wrote: > >> >> imap-hibernate processes are similar to imap-login processes in that they >> should be able to handle thousands or even tens of thousands of connections >> per process. >> > > TL;DR: In a director/proxy setup, what's a good client_limit for >

Hello, With qpopper you get something like this after a successful login: --- +OK test has 6 visible messages (0 hidden) in 11822 octets. --- This gives a clueful user or support people a quick status about the size of this mailbox, very helpful in full quota scenarios. Would be nice if the next dovecot would have such a feature as well. I doubt the same is (easily) applicable for IMAP (though

Hello, running 0.99.10.6 (but seen it before that, too). Also (quite obviously) running 5 auth processes, given the fact that this is a pretty busy box and I didn't want to introduce any artificial bottlenecks. Alas with them eating up half of the free memory (which would go to a much better use as FS cache) I'm getting sorta concerned. If it's not a leak, it's caching something

Hello, I'm willing to swear that with the 0.99.10.4 Debian package in my rather extensive tests with mailbox format dovecot created not only any missing parental directories but also the INBOX file, but the later I can't prove/test now as I upgraded all respective boxes. However 0.99.10.5 will still create (if needed and possible) missing directories for mbox storage, just not an