Hello, Just to follow up on this, we've hit over 16k (default client limit here) hibernated sessions: --- dovecot 119157 0.1 0.0 63404 56140 ? S Apr01 62:05 dovecot/imap-hibernate [11291 connections] dovecot 877825 0.2 0.0 28512 21224 ? S Apr23 1:34 dovecot/imap-hibernate [5420 connections] --- No issues other than the minor bug I reported, CPU usage is slight (at most 2% of a CPU core), memory savings are immense, so I'm a happy camper. Just out of curiosity, how does dovecot decide to split and spread out sessions between hibernate processes? It's clearly something more involved than "fill up one and then fill up the next" or we would see 16k on the old one and a few on the new one. Christian -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/
On 24 Apr 2017, at 4.04, Christian Balzer <chibi at gol.com> wrote:> > > Hello, > > Just to follow up on this, we've hit over 16k (default client limit here) > hibernated sessions: > --- > dovecot 119157 0.1 0.0 63404 56140 ? S Apr01 62:05 dovecot/imap-hibernate [11291 connections] > dovecot 877825 0.2 0.0 28512 21224 ? S Apr23 1:34 dovecot/imap-hibernate [5420 connections] > --- > > No issues other than the minor bug I reported, CPU usage is slight (at > most 2% of a CPU core), memory savings are immense, so I'm a happy camper. > > Just out of curiosity, how does dovecot decide to split and spread out > sessions between hibernate processes? > It's clearly something more involved than "fill up one and then fill up > the next" or we would see 16k on the old one and a few on the new one.New processes aren't created until client_limit is reached in all the existing processes. When there are multiple processes they're all listening for new connections and whichever happens to be fastest gets it. Related to this, I'm thinking about implementing SO_REUSEPORT (https://lwn.net/Articles/542629/ <https://lwn.net/Articles/542629/>) soon that would change the behavior a bit. Although its main purposes would be as a workaround to allow Dovecot restarts to work even though some of the old processes are still keeping the listener port open.
On 24 Apr 2017, at 15.41, Timo Sirainen <tss at iki.fi> wrote:> > On 24 Apr 2017, at 4.04, Christian Balzer <chibi at gol.com <mailto:chibi at gol.com>> wrote: >> >> >> Hello, >> >> Just to follow up on this, we've hit over 16k (default client limit here) >> hibernated sessions: >> --- >> dovecot 119157 0.1 0.0 63404 56140 ? S Apr01 62:05 dovecot/imap-hibernate [11291 connections] >> dovecot 877825 0.2 0.0 28512 21224 ? S Apr23 1:34 dovecot/imap-hibernate [5420 connections] >> --- >> >> No issues other than the minor bug I reported, CPU usage is slight (at >> most 2% of a CPU core), memory savings are immense, so I'm a happy camper. >> >> Just out of curiosity, how does dovecot decide to split and spread out >> sessions between hibernate processes? >> It's clearly something more involved than "fill up one and then fill up >> the next" or we would see 16k on the old one and a few on the new one. > > > New processes aren't created until client_limit is reached in all the existing processes. When there are multiple processes they're all listening for new connections and whichever happens to be fastest gets it. Related to this, I'm thinking about implementing SO_REUSEPORT (https://lwn.net/Articles/542629/ <https://lwn.net/Articles/542629/> <https://lwn.net/Articles/542629/ <https://lwn.net/Articles/542629/>>) soon that would change the behavior a bit. Although its main purposes would be as a workaround to allow Dovecot restarts to work even though some of the old processes are still keeping the listener port open.Huh, SO_REUSEPORT was already implemented in 2013. I completely forgot about that. Would be useful to try if it works better (or at least not worse) and maybe change it to be enabled by default in some version. service ... { inet_listener ... { reuse_port = yes