dovecot - Dec 2017 - sieve_pipe_socket_dir not created at startup for configured pipe service

Hi, all

I?m new to the list but not to dovecot.  I?ve been using it in a basic
configuration for some time, but finally decided to tweak my deployed system to
take advantage of some more dovecot features.  In particular I?m trying to set
up pigeonhole to implement spam retraining with imapsieve.  All of this is with
dovecot 2.2.31 (65cde28) and pigeonhole 0.4.19.

Before going any further let me start by saying that I have gotten all of this
to work.  It works when I can get dovecot to start up, that is.  My
configuration is pretty much straight from the docs, with a few tweaks for my
particular needs.  I?m trying to set up a pipe service using sieve-extprograms,
and the relevant part of my config looks like this:

plugin {

  sieve_pipe_input_eol = lf

  sieve_pipe_socket_dir = sieve-pipe
  sieve_filter_socket_dir = sieve-filter
  sieve_execute_socket_dir = sieve-execute

  sieve_pipe_bin_dir = /usr/local/libexec/dovecot/sieve-pipe
  sieve_filter_bin_dir = /usr/local/libexec/dovecot/sieve-filter
  sieve_execute_bin_dir = /usr/local/libexec/dovecot/sieve-execute
}

service sieve-train-ham {
  executable = script /usr/local/libexec/dovecot/sieve-pipe/train-ham.sh

  # Needs access to dspam config and lockfiles.
  user = dspam

  # socket name is program-name in Sieve (without sieve-pipe/ prefix)
  unix_listener sieve-pipe/train-ham {
  }
}

It?s my understanding from reading the docs that the sieve_pipe_socket_dir
specifies a directory that is relative to base_dir, which is the default
/var/run/dovecot in my case.  The issue I?m having is that dovecot will not
start, and spews the following errors:

dovecot: master: Error: bind(/var/run/dovecot/sieve-pipe/train-ham) failed: No
such file or directory
dovecot: master: Fatal: Failed to start listeners

Once dovecot fails startup, it leaves /var/run/dovecot around and if I manually
create the sieve-pipe directory there it will start up, create the sockets there
and everything works as intended subsequently.  The problem, though, is that on
normal shutdown all of /var/run/dovecot goes away and the at the next startup it
fails to start again.  Needless to say this isn?t great for unintended reboots,
etc.

So, can anyone see anything obvious that I?m doing wrong?  I?m just assuming
that dovecot should create the needed subdir since I can?t find anything in the
docs to suggest a way to create it otherwise.  I?ve tried all I can think of at
the moment to try to remedy this without success.  I?m happy to provide
additional details as needed to try to track this down.


Thanks in advance,

Garth

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3730 bytes
Desc: not available
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20171217/69144404/attachment-0001.p7s>

In looking at this further, I?m not seeing how this can be configured as
documented and still work.

The sieve_pipe_socket_dir setting, as the documentation suggests, appears to be
used by the plugin to construct a path to find the ?script? (the socket in this
case), and it is always relative to base_dir and must include the contents of
sieve_pipe_socket_dir.

    /* Try socket first */
    if ( ext_config->socket_dir != NULL ) {
        path = t_strconcat(senv->user->set->base_dir, "/",
            ext_config->socket_dir, "/", program_name, NULL);

But nowhere in dovecot can I find a place where this subdir would be created at
startup.  The only subdirs that I can find created are all explicitly hardcoded
in master-settings.c in a function called master_settings_do_fixes():

	/* Make sure our permanent state directory exists */
	if (mkdir_parents(set->state_dir, 0755) < 0 && errno != EEXIST)
		i_fatal("mkdir(%s) failed: %m", set->state_dir);

	mkdir_login_dir(set, t_strconcat(set->base_dir, "/login", NULL));
	mkdir_login_dir(set, t_strconcat(set->base_dir, "/token-login",
NULL));

	empty_dir = t_strconcat(set->base_dir, "/empty", NULL);
	if (safe_mkdir(empty_dir, 0755, master_uid, getegid()) == 0) {
		i_warning("Corrected permissions for empty directory "
			  "%s", empty_dir);
	}

Granted this is from a pretty quick browsing of the sources but I couldn?t find
where subdirs specified in service listeners might be created at startup.

I?m working around this by defining sieve_pipe_socket_dir to the current
directory (?.?), which allows the search path to resolve to base_dir, and find
the socket in there.  Then I just define the listener without the subdir.  This
seems to work fine and allows dovecot to start up.

Unless someone can reveal the magic incantation to use sieve_pipe_socket_dir to
create an actual subdir, though, it seems like either a documentation issue or a
bug.  I?d argue the latter since the configuration variable isn?t actually
useful.00


Garth
> On Dec 17, 2017, at 4:51 PM, Garth Corral <gcorral at abode.com>
wrote:
> 
> 
> Hi, all
> 
> I?m new to the list but not to dovecot.  I?ve been using it in a basic
configuration for some time, but finally decided to tweak my deployed system to
take advantage of some more dovecot features.  In particular I?m trying to set
up pigeonhole to implement spam retraining with imapsieve.  All of this is with
dovecot 2.2.31 (65cde28) and pigeonhole 0.4.19.
> 
> Before going any further let me start by saying that I have gotten all of
this to work.  It works when I can get dovecot to start up, that is.  My
configuration is pretty much straight from the docs, with a few tweaks for my
particular needs.  I?m trying to set up a pipe service using sieve-extprograms,
and the relevant part of my config looks like this:
> 
> plugin {
> 
>  sieve_pipe_input_eol = lf
> 
>  sieve_pipe_socket_dir = sieve-pipe
>  sieve_filter_socket_dir = sieve-filter
>  sieve_execute_socket_dir = sieve-execute
> 
>  sieve_pipe_bin_dir = /usr/local/libexec/dovecot/sieve-pipe
>  sieve_filter_bin_dir = /usr/local/libexec/dovecot/sieve-filter
>  sieve_execute_bin_dir = /usr/local/libexec/dovecot/sieve-execute
> }
> 
> service sieve-train-ham {
>  executable = script /usr/local/libexec/dovecot/sieve-pipe/train-ham.sh
> 
>  # Needs access to dspam config and lockfiles.
>  user = dspam
> 
>  # socket name is program-name in Sieve (without sieve-pipe/ prefix)
>  unix_listener sieve-pipe/train-ham {
>  }
> }
> 
> It?s my understanding from reading the docs that the sieve_pipe_socket_dir
specifies a directory that is relative to base_dir, which is the default
/var/run/dovecot in my case.  The issue I?m having is that dovecot will not
start, and spews the following errors:
> 
> dovecot: master: Error: bind(/var/run/dovecot/sieve-pipe/train-ham) failed:
No such file or directory
> dovecot: master: Fatal: Failed to start listeners
> 
> Once dovecot fails startup, it leaves /var/run/dovecot around and if I
manually create the sieve-pipe directory there it will start up, create the
sockets there and everything works as intended subsequently.  The problem,
though, is that on normal shutdown all of /var/run/dovecot goes away and the at
the next startup it fails to start again.  Needless to say this isn?t great for
unintended reboots, etc.
> 
> So, can anyone see anything obvious that I?m doing wrong?  I?m just
assuming that dovecot should create the needed subdir since I can?t find
anything in the docs to suggest a way to create it otherwise.  I?ve tried all I
can think of at the moment to try to remedy this without success.  I?m happy to
provide additional details as needed to try to track this down.
> 
> 
> Thanks in advance,
> 
> Garth
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3730 bytes
Desc: not available
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20171218/f369d46e/attachment.p7s>

> On December 19, 2017 at 5:06 AM Garth Corral <gcorral at abode.com>
wrote:
> 
> 
> In looking at this further, I?m not seeing how this can be configured as
documented and still work.
> 
> The sieve_pipe_socket_dir setting, as the documentation suggests, appears
to be used by the plugin to construct a path to find the ?script? (the socket in
this case), and it is always relative to base_dir and must include the contents
of sieve_pipe_socket_dir.
> 
Hi!

The socket_dir setting is only needed if you actually want to use unix sockets
for pipe scripts. It's not required for using this plugin.
 
Aki
> > Thanks in advance,
> > 
> > Garth
> > 
>

Op 12/18/2017 om 1:51 AM schreef Garth Corral:
> Hi, all
>
> I?m new to the list but not to dovecot.  I?ve been using it in a basic
configuration for some time, but finally decided to tweak my deployed system to
take advantage of some more dovecot features.  In particular I?m trying to set
up pigeonhole to implement spam retraining with imapsieve.  All of this is with
dovecot 2.2.31 (65cde28) and pigeonhole 0.4.19.
>
> Before going any further let me start by saying that I have gotten all of
this to work.  It works when I can get dovecot to start up, that is.  My
configuration is pretty much straight from the docs, with a few tweaks for my
particular needs.  I?m trying to set up a pipe service using sieve-extprograms,
and the relevant part of my config looks like this:
>
> plugin {
>
>   sieve_pipe_input_eol = lf
>
>   sieve_pipe_socket_dir = sieve-pipe
>   sieve_filter_socket_dir = sieve-filter
>   sieve_execute_socket_dir = sieve-execute
>
>   sieve_pipe_bin_dir = /usr/local/libexec/dovecot/sieve-pipe
>   sieve_filter_bin_dir = /usr/local/libexec/dovecot/sieve-filter
>   sieve_execute_bin_dir = /usr/local/libexec/dovecot/sieve-execute
> }
Define either *bin_dir or *socket_dir; not both. These are two
alternative ways of running scripts, either using a direct fork() or
through a script service, respectively.

https://wiki.dovecot.org/Pigeonhole/Sieve/Plugins/Extprograms
> service sieve-train-ham {
>   executable = script /usr/local/libexec/dovecot/sieve-pipe/train-ham.sh
>
>   # Needs access to dspam config and lockfiles.
>   user = dspam
>
>   # socket name is program-name in Sieve (without sieve-pipe/ prefix)
>   unix_listener sieve-pipe/train-ham {
>   }
> }
>
> It?s my understanding from reading the docs that the sieve_pipe_socket_dir
specifies a directory that is relative to base_dir, which is the default
/var/run/dovecot in my case.  The issue I?m having is that dovecot will not
start, and spews the following errors:
>
> dovecot: master: Error: bind(/var/run/dovecot/sieve-pipe/train-ham) failed:
No such file or directory
> dovecot: master: Fatal: Failed to start listeners
First of all, this directory does not strictly need to be relative to
/var/run/dovecot. It can be an absolute path to somewhere else.

The relative directories are indeed not currently created implicitly.
Hmm, we will discuss this internally.
> Once dovecot fails startup, it leaves /var/run/dovecot around and if I
manually create the sieve-pipe directory there it will start up, create the
sockets there and everything works as intended subsequently.  The problem,
though, is that on normal shutdown all of /var/run/dovecot goes away and the at
the next startup it fails to start again.  Needless to say this isn?t great for
unintended reboots, etc.
>
> So, can anyone see anything obvious that I?m doing wrong?  I?m just
assuming that dovecot should create the needed subdir since I can?t find
anything in the docs to suggest a way to create it otherwise.  I?ve tried all I
can think of at the moment to try to remedy this without success.  I?m happy to
provide additional details as needed to try to track this down.
>

Regards,

Stephan.