dovecot - Mar 2015 - [PATCH] increase fd_limit to max_client_limit automatically

Hi, with a low soft limit on file descriptors, dovecot 2.2.15 warns on
startup:

    Warning: fd limit (ulimit -n) is lower than required under max. load
    (256 < 1000), because of default_client_limit

It could try increasing the limit first, and only report the warning if that
fails. I'm attaching a patch that does just this.

Without the patch, the soft fd limit is kept at whatever it was on dovecot
startup:

    % pfexec plimit $(pgrep dovecot)
    18737:  /opt/niksula/sbin/dovecot
       resource              current         maximum
      time(seconds)         unlimited       unlimited
      file(blocks)          unlimited       unlimited
      data(kbytes)          unlimited       unlimited
      stack(kbytes)         10240           unlimited
      coredump(blocks)      unlimited       unlimited
      nofiles(descriptors)  256             65536
      vmemory(kbytes)       unlimited       unlimited

and with patch applied, it's increased to max_client_limit:

    % pfexec plimit $(pgrep dovecot)                                 
    18775:  /opt/niksula/sbin/dovecot
       resource              current         maximum
      time(seconds)         unlimited       unlimited
      file(blocks)          unlimited       unlimited
      data(kbytes)          unlimited       unlimited
      stack(kbytes)         10240           unlimited
      coredump(blocks)      unlimited       unlimited
      nofiles(descriptors)  1000            65536
      vmemory(kbytes)       unlimited       unlimited

It should probably be increased to higher than that to account for fds
other than client sockets, but I don't have insights into that.

-- 
Lauri Tirkkonen
Niksula systems specialist
-------------- next part --------------
diff -r e30e9b2b0e37 src/master/master-settings.c
--- a/src/master/master-settings.c	Wed Jan 21 02:21:35 2015 +0200
+++ b/src/master/master-settings.c	Thu Jan 22 13:39:41 2015 +0200
@@ -18,6 +18,7 @@
 #include <unistd.h>
 #include <sys/stat.h>
 #include <sys/wait.h>
+#include <sys/resource.h>
 
 static bool master_settings_verify(void *_set, pool_t pool,
 				   const char **error_r);
@@ -418,7 +419,7 @@
 #ifdef CONFIG_BINARY
 	const struct service_settings *default_service;
 #else
-	rlim_t fd_limit;
+	struct rlimit fd_limit;
 	const char *max_client_limit_source = "default_client_limit";
 	unsigned int max_client_limit = set->default_client_limit;
 #endif
@@ -591,12 +592,18 @@
 			  client_limit, max_anvil_client_processes);
 	}
 #ifndef CONFIG_BINARY
-	if (restrict_get_fd_limit(&fd_limit) == 0 &&
-	    fd_limit < (rlim_t)max_client_limit) {
-		i_warning("fd limit (ulimit -n) is lower than required "
-			  "under max. load (%u < %u), because of %s",
-			  (unsigned int)fd_limit, max_client_limit,
-			  max_client_limit_source);
+	if (getrlimit(RLIMIT_NOFILE, &fd_limit) == 0) {
+#ifdef HAVE_SETRLIMIT
+		rlim_t old = fd_limit.rlim_cur;
+		fd_limit.rlim_cur = I_MAX(old, max_client_limit);
+		if (setrlimit(RLIMIT_NOFILE, &fd_limit) < 0)
+			fd_limit.rlim_cur = old;
+#endif
+		if (fd_limit.rlim_cur < (rlim_t)max_client_limit)
+			i_warning("fd limit (ulimit -n) is lower than required "
+			    "under max. load (%u < %u), because of %s",
+			    (unsigned int)fd_limit.rlim_cur, max_client_limit,
+			    max_client_limit_source);
 	}
 #endif

On 22 Jan 2015, at 14:01, Lauri Tirkkonen <ltirkkon at niksula.hut.fi>
wrote:
> 
> Hi, with a low soft limit on file descriptors, dovecot 2.2.15 warns on
> startup:
> 
>    Warning: fd limit (ulimit -n) is lower than required under max. load
>    (256 < 1000), because of default_client_limit
> 
> It could try increasing the limit first, and only report the warning if
that
> fails. I'm attaching a patch that does just this.
It could .. But somehow it doesn't seem like a good idea to me to do
automatically. Maybe the limits are there intentionally. Maybe it's the
Dovecot settings that are wrong and not the ulimits.

On Fri, Mar 20 2015 21:06:12 +0200, Timo Sirainen wrote:
> On 22 Jan 2015, at 14:01, Lauri Tirkkonen <ltirkkon at
niksula.hut.fi> wrote:
> > 
> > Hi, with a low soft limit on file descriptors, dovecot 2.2.15 warns on
> > startup:
> > 
> >    Warning: fd limit (ulimit -n) is lower than required under max.
load
> >    (256 < 1000), because of default_client_limit
> > 
> > It could try increasing the limit first, and only report the warning
if that
> > fails. I'm attaching a patch that does just this.
> 
> It could .. But somehow it doesn't seem like a good idea to me to do
> automatically. Maybe the limits are there intentionally.
I would make the argument that that's why there are both soft and hard
limits. Any program can raise their soft ulimits to the current hard
limit - I see the soft limit as more of a guard against misconfiguration
than a resource control, so I don't think there is anything wrong with
raising it when you have a legitimate reason to do so. On systems with
low default fd limits (such as some illumos distributions; see
https://www.illumos.org/issues/3772), the alternative is requiring
administrators to create startup wrappers for dovecot that raise the
limit, but since it's not a privileged operation that just seems silly.

-- 
Lauri Tirkkonen
Niksula systems specialist