dovecot - Apr 2013 - Feature request: Configure CONFIG_MODULE_DIR and AUTH_MODULE_DIR at runtime

Hi!

I am running dovecot 2.1.16 on NixOS (http://nixos.org), and I have been  
fighting the dynamically loaded dovecot modules/plugins a bit.

The problem is that in Nix/NixOS all packages are completely isolated from  
each other (each package has a separate /lib, /libexec, /bin etc, with  
only its own files in it). So dovecot has all its modules under  
/nix/store/xxx-dovecot/{lib,libexec}/dovecot, and the Pigeonhole modules  
are under /nix/store/yyy-pidgeonhole/{lib,libexec}/dovecot.

Now, dovecot wants to load modules only from one directory, so I created a  
"virtual" package that unifies all modules in  
/nix/store/zzz-dovecot-plugins/{lib,libexec}/dovecot. I can then use this  
directory as the "mail_plugin_dir" setting in dovecot.conf. This works
fine, dovecot can then find the "sieve" mail plugin.

The problem is the service and auth modules, that dovecot tries to load  
 from the compile-time set MODULE_DIR/{settings,auth}. This is a problem  
for me, because I can't set the module path during compilation since that  
introduces a circular dependency between the dovecot package and the  
virtual dovecot-plugins package.

I have solved this by hacking dovecot to read an environment variable,  
DOVECOT_CONFIG_MODULE_DIR, in the config_parse_load_modules function. This  
way I can inject my module path during runtime. But this only works for  
the master process, not the config process which seems to reset its  
environment (I assume it does this when dropping privileges). So I did  
another hack which lets me pass an argument to the config process which  
contains the module path. I can then set this argument in dovecot.conf by  
"service config { executable = config /my/module/path }".

These hacks work, and I can now use both the sieve mail plugin and the  
sieve service. I have attached the patch, but this is not a request to add  
it to the source. My hack is a hack because I don't know enough about the  
inner workings of dovecot to solve it in a good way. Rather, I'm asking  
for some way to configure the module path in dovecot, either as an  
argument to the master process or as a setting in dovecot.conf. A bonus  
would be the ability to specify several paths that dovecot would search,  
then I can do away with my virtual module package.



Best regards,
   Rickard Nilsson
-------------- next part --------------
A non-text attachment was scrubbed...
Name: module_dir.patch
Type: application/octet-stream
Size: 4666 bytes
Desc: not available
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20130423/6409aa16/attachment.obj>

On 23.4.2013, at 17.58, Rickard Nilsson <rickard.nilsson at telia.com>
wrote:
> I am running dovecot 2.1.16 on NixOS (http://nixos.org), and I have been
fighting the dynamically loaded dovecot modules/plugins a bit.
> 
> The problem is that in Nix/NixOS all packages are completely isolated from
each other (each package has a separate /lib, /libexec, /bin etc, with only its
own files in it). So dovecot has all its modules under
/nix/store/xxx-dovecot/{lib,libexec}/dovecot, and the Pigeonhole modules are
under /nix/store/yyy-pidgeonhole/{lib,libexec}/dovecot.
> 
> Now, dovecot wants to load modules only from one directory, so I created a
"virtual" package that unifies all modules in
/nix/store/zzz-dovecot-plugins/{lib,libexec}/dovecot. I can then use this
directory as the "mail_plugin_dir" setting in dovecot.conf. This works
fine, dovecot can then find the "sieve" mail plugin.
> 
> The problem is the service and auth modules, that dovecot tries to load
from the compile-time set MODULE_DIR/{settings,auth}. This is a problem for me,
because I can't set the module path during compilation since that introduces
a circular dependency between the dovecot package and the virtual
dovecot-plugins package.
Sounds like a rather generic problem in nixos. Maybe there could be generic way
to handle this for other packages too, like maybe creating a whole new directory
somewhere (e.g. under /var) that has symlinks to all the installed plugins?
> I have solved this by hacking dovecot to read an environment variable,
DOVECOT_CONFIG_MODULE_DIR, in the config_parse_load_modules function. This way I
can inject my module path during runtime. But this only works for the master
process, not the config process which seems to reset its environment (I assume
it does this when dropping privileges). So I did another hack which lets me pass
an argument to the config process which contains the module path. I can then set
this argument in dovecot.conf by "service config { executable = config
/my/module/path }".
You can preserve environments by adding them to import_environment setting.
> These hacks work, and I can now use both the sieve mail plugin and the
sieve service. I have attached the patch, but this is not a request to add it to
the source. My hack is a hack because I don't know enough about the inner
workings of dovecot to solve it in a good way. Rather, I'm asking for some
way to configure the module path in dovecot, either as an argument to the master
process or as a setting in dovecot.conf. A bonus would be the ability to specify
several paths that dovecot would search, then I can do away with my virtual
module package.
I'd rather make those plugin dirs configurable, since it only adds new
settings that just about nobody changes (and perhaps I shouldn't have added
mail_plugin_dir setting either in the first place).

If you start/stop dovecot with an initscript or some other related 
system, you can do what SuSE does, since this problem occurs in lots of 
situations, not just dovecot.

Since you know where all the config files are, you could either have the 
initscript set up a directory with symlinks, as Timo said, or you could 
collect them together and place them all in one file that is in the 
correct location.  Multiple config files for dovecot can be concatenated 
together if you are careful to scan for and accommodate "include" 
directives.

Dem