Howdy folks,
we're using Dovecot 2.3.19 in combination with vmailmgr as the virtual user
management backend (please don't judge, there are plans to move away from it
for various reasons, but as of today it's there), using the wrapper from
https://wiki.dovecot.org/HowTo/VMailMgr - and we're facing an interesting
problem with colons in home directories. I think this problem is not specific to
vmailmgr but happens in any scenario that includes ":" in directory
names.
tl;dr:
- The checkvpw-dovecot-wrapper from the Dovecot wiki converts ":" in
directory names to "::" which works fine with general IMAP use, but at
least the autoexpunge plugin and ManageSieve don't handle this escaping and
thus break on those users.
- When patching the ":"->"::" conversion out of the
checkvpw-dovecot-wrapper, Dovecot itself bails on it, interpreting it as a
sub-setting.
- The latter is quite surprising because a HOME/mail_home directory is supposed
to be exactly one directory (AFAIK), with no sub-settings like e.g.
mail_location, so I did not expect it to be "interpreted" in any way.
Short story long:
When you set up a user with a name containing dots, vmailmgr converts them to
colons, pretty much like qmail does (qmail maps the username "xy.z" to
a file named ".qmail-xy:z"; vmailmgr creates the directory
"./users/xy:z" for that username). The wrapper from the dovecot wiki
escapes ":" into "::" before exporting the user's
directory as the HOME environment variable:
/* escape possible ':' characters before finally exporting */
escape_colon(buffer, MAXLEN);
setenv("HOME", buffer, 1);
The result looks like that:
[root at stardust ~]# doveadm auth login xy.z at coltest.uber.space supersecret
passdb: xy.z at coltest.uber.space auth succeeded
extra fields:
user=xy.z at coltest.uber.space
userdb extra fields:
xy.z at coltest.uber.space
home=/home/coltest/./users/xy::z <---------- notice the double colon here
uid=1009
gid=1010
auth_mech=PLAIN
So far, so good: Working with IMAP generally works fine, but there's a
catch: Not every module seems prepared for this "::" syntax. The first
notable exception is the autoexpunge plugin which simply bails on it:
Error: autoexpunge: Couldn't create dovecot.autoexpunge.lock lock:
file_create_locked(/home/coltest/./users/xy::z/dovecot.autoexpunge.lock) failed:
safe_mkstemp(/home/coltest/./users/xy::z/dovecot.autoexpunge.lock) failed: No
such file or directory
The second notable exception is ManageSieve which takes the escaped directory
name "as is", leading to a situation where the Maildir is in
"xy:z" while the Sieve stuff is being put in "xy::z" where
it effectively is never picked up during delivery (as the stuff is in the wrong
location and the .dovecot.sieve symlink is - silently - not created):
[coltest at stardust ~]$ ls -la users/*
users/xy:z:
total 28
drwx------. 9 coltest coltest 4096 Jun 16 11:39 .
drwxr-xr-x. 4 coltest coltest 31 Jun 16 11:40 ..
drwxr-xr-x. 2 coltest coltest 6 Jun 16 11:39 cur
-rw-------. 1 coltest coltest 376 Jun 16 11:40 dovecot.index.log
-rw-------. 1 coltest coltest 2676 Jun 16 11:39 dovecot.list.index.log
-rw-------. 1 coltest coltest 96 Jun 16 11:39 dovecot.mailbox.log
-rw-------. 1 coltest coltest 51 Jun 16 11:39 dovecot-uidlist
-rw-------. 1 coltest coltest 8 Jun 16 11:39 dovecot-uidvalidity
-r--r--r--. 1 coltest coltest 0 Jun 16 11:39 dovecot-uidvalidity.62aafa70
drwx------. 5 coltest coltest 108 Jun 16 11:39 .Drafts
drwxr-xr-x. 2 coltest coltest 6 Jun 16 11:39 new
drwx------. 5 coltest coltest 108 Jun 16 11:39 .Sent
drwx------. 5 coltest coltest 108 Jun 16 11:39 .Spam
-rw-------. 1 coltest coltest 28 Jun 16 11:39 subscriptions
drwxr-xr-x. 2 coltest coltest 6 Jun 16 11:39 tmp
drwx------. 5 coltest coltest 108 Jun 16 11:39 .Trash
users/xy::z:
total 0
drwx------. 3 coltest coltest 19 Jun 16 11:40 .
drwxr-xr-x. 4 coltest coltest 31 Jun 16 11:40 ..
drwx------. 3 coltest coltest 33 Jun 16 11:40 sieve
So my initial thought was: Why is the ":" even escaped to a
"::" in the first place? It shouldn't be necessary according to
this clean workaround that I found in a posting from 2009 with Timo advising for
this type of handling:
> Hmm. Although now that I looked at your code, you seem to be using only
> HOME. And using ~/ in mail_location should work around any ":"
issues.
> So what exactly is the problem you're seeing?
https://www.dovecot.org/list/dovecot/2009-June/040590.html
(In the same thread, Timo _also_ introduced the "::" handling in
Dovecot.)
This is exactly what we're doing at delivery time: Setting mail_home to the
real pathname including a _single_ colon (not an escaped double colon) and then
pointing mail_location to "~/":
/usr/libexec/dovecot/dovecot-lda -a '$RECIPIENT' -f '$SENDER' -o
mail_home='$MAILDIR' -o mail_location='maildir:~/'
This works well.
So my assumption was: If the checkvpw-dovecot-wrapper returns the HOME directory
of a virtual user; where's even the need to escape a single colon to a
double colon? So I gave it a try and removed that one line of escaping and
recompiled the wrapper. Result:
[root at stardust ~]# doveadm auth login xy.z at coltest.uber.space supersecret
passdb: xy.z at coltest.uber.space auth succeeded
extra fields:
user=xy.z at coltest.uber.space
userdb extra fields:
xy.z at coltest.uber.space
home=/home/coltest/./users/xy:z <---------- single colon here now
uid=1009
gid=1010
auth_mech=PLAIN
Using a real IMAP login, though, this lead to the "interesting"
situation that the login itself worked correctly, but _then_ immediately an
"Internal error occurred":
[root at stardust ~]# nc localhost 143
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SPECIAL-USE
LITERAL+ STARTTLS AUTH=PLAIN] Dovecot ready.
0 LOGIN xy.z at coltest.uber.space supersecret
0 OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT
SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND
URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED
I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH
LIST-STATUS BINARY MOVE SNIPPET=FUZZY PREVIEW=FUZZY PREVIEW STATUS=SIZE SAVEDATE
SPECIAL-USE LITERAL+ NOTIFY SPECIAL-USE] Logged in
* BYE Internal error occurred. Refer to server log for more information.
This is the according log:
Jun 16 10:04:37 stardust.uberspace.de dovecot[15268]: Jun 16 10:04:37
imap-login: Info: Login: user=<xy.z at coltest.uber.space>, method=PLAIN,
rip=::1, lip=::1, mpid=22889, secured
Jun 16 10:04:37 stardust.uberspace.de dovecot[15268]: Jun 16 10:04:37 imap(xy.z
at coltest.uber.space)<22889><utbow43hgJoAAAAAAAAAAAAAAAAAAAAB>:
Error: Namespace '': Unknown setting: z
Jun 16 10:04:37 stardust.uberspace.de dovecot[15268]: Jun 16 10:04:37 imap(xy.z
at coltest.uber.space)<22889><utbow43hgJoAAAAAAAAAAAAAAAAAAAAB>:
Info: Disconnected: Namespace '': Unknown setting: z in=0 out=428
deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0
So it looks like Dovecot tries to do some interpretation on the HOME value,
splitting it at the ":" and interpret the "z" as some kind
of sub-setting - which a HOME variable (in contrast to, e.g. mail_location) is
not supposed to have.
Here I'm a bit stuck. My idea is to work around this by eliminating the
":" from directory names altogether by passing an explicit directory
name to vadduser:
[coltest at stardust ~]$ echo supersecret | vadduser --directory=./users/ab.c
ab.c
vadduser: user 'ab.c' successfully added
[coltest at stardust ~]$ ls -l users
total 0
drwx------. 5 coltest coltest 39 Jun 16 10:26 ab.c
drwx------. 5 coltest coltest 39 Jun 16 09:31 xy:z
But still, it would be nicer if Dovecot cleanly supported the ":" in
directory names, as it seems to _want_ to support it (by introducing the
"::" escaping) but then only does it halfway.
Could this be fixed in whatever direction, or is there any other advice how we
should handle this?
All the best,
Jonas