dovecot - May 2009 - Some questions about deliver

While investigating Dovecot's deliver with Postfix, I encountered some  
behaviors making me wonder whether I really understand the purpose of  
that binary. So, if you allow...

This is from Postfix' main.cf:

     mailbox_command = /usr/local/dovecot/libexec/dovecot/deliver -e -n

This is my quick setup for Dovecot:

     # 1.2.rc4: /usr/local/etc/dovecot.conf
     # OS: Darwin 9.7.0 i386
     protocols: pop3
     ssl: no
     disable_plaintext_auth: no
     login_dir: /usr/local/var/run/dovecot/login
     login_executable: /usr/local/dovecot-1.2.rc4/libexec/dovecot/pop3- 
login
     first_valid_uid: 2001
     mail_location: mbox:~/_mailboxes:INBOX=~/_inbox/mailspool
     mbox_read_locks: flock
     mbox_write_locks: flock dotlock
     mail_executable: /usr/local/dovecot-1.2.rc4/libexec/dovecot/pop3
     mail_plugin_dir: /usr/local/dovecot-1.2.rc4/lib/dovecot/pop3
     pop3_lock_session: yes
     pop3_uidl_format: %08Xv%08Xu
     auth default:
       passdb:
         driver: pam
         args: *
       userdb:
         driver: passwd

Here is an excerpt from mail.log:

     postfix/smtpd[4117]: connect from localhost[127.0.0.1]
     postfix/smtpd[4117]: 163E1CA2675: client=localhost[127.0.0.1]
     postfix/cleanup[4120]: 163E1CA2675:
message-id=<27002A3F-AF44-410D-A500-368D04F5FA9A at almbp.local
 >
     postfix/qmgr[3232]: 163E1CA2675: from=<testuser at almbp.local>,  
size=2650, nrcpt=1 (queue active)
     dovecot[4129]: deliver(www): mail_location: mbox: mkdir(/Library/ 
WebServer/_mailboxes) failed: Permission denied (euid=70(_www)  
egid=70(_www) missing +w perm: /Library/WebServer)
     dovecot[4129]: deliver(www): Fatal: Namespace initialization failed
     postfix/local[4122]: 163E1CA2675: to=<www at almbp.local>,  
relay=local, delay=0.09, delays=0.02/0/0/0.07, dsn=4.3.0,  
status=deferred (temporary failure)
     postfix/smtpd[4117]: disconnect from localhost[127.0.0.1]


1. Shouldn't deliver honor the first_valid_uid setting?

After all, if some user isn't allowed to login, is that user supposed  
to receive mail?

2. What exactly is the -e option supposed to do?

User _www clearly isn't configured for receiving mail, but Postfix  
will nevertheless try to deliver the message until  
maximal_queue_lifetime has expired.
 From the log excerpt, it appears that Postfix has been led to  
consider a temporary failure; unless I'm wrong, that's because Postfix  
didn't receive neither a "D.S.N text" string nor an EX_* fatal
return
code.
 From deliver.c, it seems that many calls to i_fatal() are liable to  
be executed before the -e option is taken into account (through the  
local variable stderr_rejection).
As a result, I'm wondering under which circumstances exactly deliver  
is liable to return EX_NOPERM, so that Postfix may consider a  
permanent failure and bounce the message.

Note that the question may somehow be re-phrased as: when invoked  
without the -e option, under which circumstances will deliver send a  
rejection message?

3. Doesn't failure_exit_callback() in deliver.c tend to merge many  
(possibly dissimilar) errors into the single EX_TEMPFAIL one?

And BTW, is that function guaranteed to be always called with an EX_*  
value? Seems to be the case, but... ;-)


TIA for your patience,
Axel

On Sun, 2009-05-24 at 18:35 +0200, Axel Luttgens wrote:
> 1. Shouldn't deliver honor the first_valid_uid setting?
I'm not sure. Somehow enforcing it there seems like a bad idea to me.
> 2. What exactly is the -e option supposed to do?
..
> Note that the question may somehow be re-phrased as: when invoked  
> without the -e option, under which circumstances will deliver send a  
> rejection message?
Only when user is over quota. If you don't want messages to be delivered
to some users, Postfix shouldn't call deliver for the user.
> 3. Doesn't failure_exit_callback() in deliver.c tend to merge many  
> (possibly dissimilar) errors into the single EX_TEMPFAIL one?
Yes. EX_TEMPFAIL is the safest choice almost always. If something
breaks, you typically want to fix it and get the mail delivered again,
instead of being rejected the first time and never seeing the mail.
> And BTW, is that function guaranteed to be always called with an EX_*  
> value? Seems to be the case, but... ;-)
What do you mean? It's called with FATAL_* values and it replaces them
with EX_TEMPFAIL.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20090525/c97f6a12/attachment-0002.bin>

Le 26 mai 09 ? 02:08, Timo Sirainen a ?crit :
> On Sun, 2009-05-24 at 18:35 +0200, Axel Luttgens wrote:
>> 1. Shouldn't deliver honor the first_valid_uid setting?
>
> I'm not sure. Somehow enforcing it there seems like a bad idea to me.
Yes, could well be that I overlooked some possible side-effects.

On the other hand, this would help to enforce the deliver+Dovecot pair  
consistency (if John Doe can't fetch his mails, should he receive  
mails?). Or help to enforce, even if inefficiently, some local  
policies not easily implemented otherwise.

Someone else? ;-)

>> 2. What exactly is the -e option supposed to do?
> ..
>> Note that the question may somehow be re-phrased as: when invoked
>> without the -e option, under which circumstances will deliver send a
>> rejection message?
>
> Only when user is over quota.
OK, this would thus be intended design.

> If you don't want messages to be delivered
> to some users, Postfix shouldn't call deliver for the user.
I agree that waiting to have a message enqueued for rejecting it  
afterwards is somewhat inefficient.

But... ;-)

Let's first consider deliver as a replacement of Postfix' mailbox  
delivery, because of the nice benefits (indexing, headers  
sanitization, plugins...) it comes with.

With Postfix' own mailbox delivery, i.e.

     mailbox_command 
one gets with my previous example:

     postfix/smtpd[8639]: connect from localhost[127.0.0.1]
     postfix/smtpd[8639]: 1AFE4CA5D97: client=localhost[127.0.0.1]
     postfix/cleanup[8642]: 1AFE4CA5D97:
message-id=<029EE72B-B412-437F-A211-33C3597C8C54 at almbp.local
 >
     postfix/qmgr[8637]: 1AFE4CA5D97: from=<testuser at almbp.local>,  
size=560, nrcpt=1 (queue active)
     postfix/local[8643]: 1AFE4CA5D97: to=<www at almbp.local>,  
relay=local, delay=0.08, delays=0.01/0.01/0/0.06, dsn=5.2.0,  
status=bounced (cannot update mailbox /Library/WebServer/_inbox/ 
mailspool for user www. unable to create lock file /Library/WebServer/ 
_inbox/mailspool.lock: No such file or directory)
     postfix/cleanup[8642]: 2C455CA5D99:
message-id=<20090526075000.2C455CA5D99 at almbp.local
 >
     postfix/qmgr[8637]: 2C455CA5D99: from=<>, size=2896, nrcpt=1  
(queue active)
     postfix/bounce[8644]: 1AFE4CA5D97: sender non-delivery  
notification: 2C455CA5D99
     postfix/local[8643]: 2C455CA5D99: to=<testuser at almbp.local>,  
relay=local, delay=0.01, delays=0/0/0/0, dsn=2.0.0, status=sent  
(delivered to mailbox)
     postfix/qmgr[8637]: 1AFE4CA5D97: removed
     postfix/qmgr[8637]: 2C455CA5D99: removed
     postfix/smtpd[8639]: disconnect from localhost[127.0.0.1]

So, Postfix' default behavior is to bounce the message; strictly  
speaking, deliver can't thus be viewed as a transparent substitute.

Now, let's consider the default behavior of Postfix when facing an  
over-quota recipient:

     postfix/smtpd[8977]: connect from localhost[127.0.0.1]
     postfix/smtpd[8977]: 91CEECA5FDF: client=localhost[127.0.0.1]
     postfix/cleanup[8980]: 91CEECA5FDF:
message-id=<B46CADD6-E42D-493C-AB04-46B0BFF3F78A at swing.be
 >
     postfix/qmgr[8938]: 91CEECA5FDF: from=<AxelLuttgens at swing.be>,  
size=12634, nrcpt=1 (queue active)
     postfix/local[8981]: 91CEECA5FDF: to=<testuser at almbp.local>,  
relay=local, delay=0.04, delays=0.02/0.01/0/0.01, dsn=5.2.2,  
status=bounced (cannot update mailbox /Volumes/ALMbpSpare/People/a/ 
testuser/_inbox/mailspool for user testuser. error writing message:  
File too large)
     postfix/cleanup[8980]: 980EECA5FE1:
message-id=<20090526085344.980EECA5FE1 at almbp.fusl.ac.be
 >
     postfix/bounce[8982]: 91CEECA5FDF: sender non-delivery  
notification: 980EECA5FE1
     postfix/qmgr[8938]: 91CEECA5FDF: removed
     postfix/qmgr[8938]: 980EECA5FE1: from=<>, size=2702, nrcpt=1  
(queue active)
     postfix/smtp[8983]: 980EECA5FE1: to=<AxelLuttgens at swing.be>,  
relay=in.mx.skynet.be[195.238.5.129]:25, delay=2.8,  
delays=0.01/0.01/2.8/0.05, dsn=2.0.0, status=sent (250 ok:  Message  
446634039 accepted)
     postfix/qmgr[8938]: 980EECA5FE1: removed
     postfix/smtpd[8977]: disconnect from localhost[127.0.0.1]

In this case, by default, Posfix adopts the same behavior as deliver.
But I could have needed for some administrative reason to configure  
Postfix with, for example,

     soft_bounce = yes

and then again face a problem when considering to make use of deliver  
as mailbox transport.

More generally, since one of deliver's goal is to replace an MTA's  
local delivery agent, it would be nice to have some ways to fine tune  
deliver's behavior.
That would allow to transparently integrate deliver into an existing  
MTA setup.
Or even to augment the capabilities of that setup, for example by  
refining local policies.

At least, it would be nice to have a very precise description of how  
deliver is supposed to behave when facing various conditions.
This would then be a documentation matter.
Of course, the source code always is the ultimate documentation :-)

>> 3. Doesn't failure_exit_callback() in deliver.c tend to merge many
>> (possibly dissimilar) errors into the single EX_TEMPFAIL one?
>
> Yes. EX_TEMPFAIL is the safest choice almost always. If something
> breaks, you typically want to fix it and get the mail delivered again,
> instead of being rejected the first time and never seeing the mail.
Yes, to be right under any circumstances is rather difficult;  
ultimately, perhaps is this a matter of local policy.

Anyway, let's consider this (stupid) one:

	mailbox_command = /usr/local/dovecot/libexec/dovecot/deliver -e -n -x

Here, deliver immediately returns with EX_USAGE and, by default,  
Postix will reject the message; but this is something I could have  
noticed and fixed in the meantime.

On the other hand, it could also be argued that an over-quota  
recipient is fixable as well: just pick the phone and ask the  
recipient to clean his mailbox. And one could thus conclude that  
deliver's default behavior isn't the right one...

Now, that last point may be controlled thru QUOTA_FULL_TEMPFAIL, so  
that deliver nevertheless appears to be at least partially configurable.
Hence the temptation to ask for more configurability...
(in fact, I was musing on the possibility to modulate the switch  
statement with some config settings)

>> And BTW, is that function guaranteed to be always called with an EX_*
>> value? Seems to be the case, but... ;-)
>
> What do you mean? It's called with FATAL_* values and it replaces them
> with EX_TEMPFAIL.
Sorry, some kind of mental short circuit occurred here; I meant:  
"guaranteed to always return with status set to an EX_* value?".

I asked because of the default case in failure_exit_callback(), which  
just returns and leaves status as it was on entry. Conceptually, one  
could thus enter and leave the function with status set to a value  
differing from one of the FATAL_* and the EX_*, that value being  
ultimately returned to the caller. May this happen in practice? If  
yes, when and which values?


Again, thanks a lot Timo for bearing with a guy who's just a plain  
newbie with Dovecot,
Axel