dovecot - Jul 2009 - dovecot lda and postfix

I do not know if this is a help request to solve a problem but
to understand what is going on. I am going to start with the problem
description as I see it and we will go from there.

I have postfix setup to use dovecot for tls/sasl in addition to its
normal imap/pop3 functions. Postfix is also setup to do virtual
domains, getting its information from the files valias, vmaps, and
vhosts which are all located in its /etc/postfix directory:

# Virtual domain stuff
virtual_mailbox_domains = /etc/postfix/vhosts.txt
virtual_mailbox_base = /var/spool/vmail
virtual_mailbox_maps = hash:/etc/postfix/vmaps.txt
# 1500:1500 is user virtual
virtual_uid_maps = static:1500
virtual_gid_maps = static:1500
virtual_alias_maps = hash:/etc/postfix/valias.txt

Dovecot uses its own unix passwd-like files to do authentication. In
fact, as you can see, I broke them apart so I can make the actual
password file readable only by root:

mail_location = maildir:~/

auth default {
  passdb passwd-file {
     args = /etc/dovecot/passwd
  }
  userdb passwd-file {
     args = /etc/dovecot/users
  }
}

I know the norm is to use a database or at least ldap, but I had my
reasons. Mailscanner is also installed. So, the whay things have been
working so far was, postfix receives an email and checks if it is
recipient exists here. If so, it would pass it to mailscanner and then
get it back, when it would then drop it at the recipient's mailbox in
/var/spool/vmail/domain.com/user. A user would then connect to dovecot
to retrieve the email. And when an email was to be sent, postfix would
use dovecot to do the user authentication. So far so good.

Now I want to add dovecot lda so I can run the cmu sieve plugin (this
is dovecot 1.0.1) primarily to move emails mailscanner (and
spamassassin) thinks that are spam to each virtual  user's  Spam
maildir directory.

So, reading http://wiki.dovecot.org/LDA/Sieve/CMU, I add the following
entries to dovecot.conf:

protocol lda {
  mail_plugin_dir = /usr/lib/dovecot/modules/lda

  sendmail_path = /usr/lib/sendmail

  auth_socket_path = /var/run/dovecot/auth-master
  mail_plugins = cmusieve
  global_script_path = /etc/dovecot/scripts/dovecot.sieve

  mail_debug = yes
  log_path = /var/log/dovecot-lda
  info_log_path = /var/log/dovecot-lda

}

# I am being rather lazy in my sieve config entry. I did make sure the file
# is owned by virtual:virtual
plugin {
  sieve = /etc/dovecot/scripts/dovecot.sieve
}

  socket listen {
    master {
      path = /var/run/dovecot/auth-master
      mode = 0600
      user = virtual # User running Dovecot LDA's deliver
    }
    # Postfix is using dovecot for SMTP AUTH (SASL)
    client {
       path = /var/spool/postfix/private/auth
       mode = 0660
       user = postfix
       group = postfix
    }
  }

Then, based on http://wiki.dovecot.org/LDA/Postfix, I edit postfix's
master.cf:

dovecot   unix  -       n       n       -       -       pipe
  flags=DRhu user=virtual:virtual argv=/usr/lib/dovecot/deliver -f
${sender} -d ${user}@${nexthop}

and main.cf:
dovecot_destination_recipient_limit = 1
virtual_transport = dovecot
# mailbox_virtual_domains was already defined as shown above

The dovecot.sieve file looks like this:

require ["fileinto", "reject"];

# Spam stuff
if header :contains "X-Spam-Level" "********************" {
        discard;
        stop;
}
elsif header :contains "X-Spam-Status" "Yes" {
        fileinto "Spam";
        stop;
}

So, I restart it and try to send a message to my account, say
raub at domain.com, in this machine. It bounces back saying the user does
not exist. I check th elog files and they seem to agree:

dovecot: 2009-06-30 16:31:44 Info: auth(default):
passwd-file(raub at domain.com): lookup: user=raub at domain.com
file=/etc/dovecot/users
dovecot: 2009-06-30 16:31:44 Info: auth(default):
passwd-file(raub at domain.com): unknown user

Now this is where I am trying to understand what is going on. It seems
to me that postfix *or* dovecot went to  /etc/dovecot/users looking
for raub at domain.com. Not finding it there (I had it defined there as
raub simply; I have mail_location = maildir:~/ in dovecot.conf, so the
username in this file does not need to be related to the user's email
address), it decided user does not exist and bounced back.

My testing indicated that happened after postfix passed the email to
dovecot lda (I guess that virtual_transport = dovecot means it goes to
master.cf and looks for the dovecot entry and then passes the message
to it). That would mean postfix is not using vmaps to see if the user
exists anymore. Instead, it is delegating checking if a user exists to
dovecot, which is not what I want to do. I want postfix to worry about
getting mail and making sure the recipient exists. Out of the blue, if
I replaced the virtual_mailbox entry with local_transport = dovecot,
or nothing at all, mail is delivered as it should.

My gut feeling is the master.cf dovecot entry is responsible for the
problem, but I am not sure because I really do not know everything
that is going on there; this is the first time I venture in that file.

So, could anyone shed some light on what is causing the problem? Thanks!

Le 1 juil. 09 ? 16:05, Mauricio Tavares a ?crit :
> [...]
>
> So, could anyone shed some light on what is causing the problem?  
> Thanks!
Hello Mauricio,

It's rather difficult to have a clear view of what's happening.
Could you augment your description with the output of
	postconf -n
	dovecot -n
as well as all the log entries written by both Postfix and Dovecot  
when you send your email to raub at domain.com, please without  
obfuscating the data: its already sufficiently complicated... ;-)

If needed, you could try to augment Postfix' verbosity with the help  
of debug_peer_level and debug_peer_list in main.cf.
The same way, you could rise Dovecot's verbosity with mail_debug in  
dovecot.conf.

Thanks,
Axel

Am 01.07.2009 um 16:05 schrieb Mauricio Tavares:
> My testing indicated that happened after postfix passed the email to
> dovecot lda (I guess that virtual_transport = dovecot means it goes to
> master.cf and looks for the dovecot entry and then passes the message
> to it). That would mean postfix is not using vmaps to see if the user
> exists anymore. Instead, it is delegating checking if a user exists to
> dovecot, which is not what I want to do. I want postfix to worry about
> getting mail and making sure the recipient exists. Out of the blue, if
> I replaced the virtual_mailbox entry with local_transport = dovecot,
> or nothing at all, mail is delivered as it should.
>
> My gut feeling is the master.cf dovecot entry is responsible for the
> problem, but I am not sure because I really do not know everything
> that is going on there; this is the first time I venture in that file.
>
> So, could anyone shed some light on what is causing the problem?  
> Thanks!
If you use the Dovecot LDA,  the delivery is completely handled by  
Dovecot once the message is passed to the Agent. This means the  
maildir location needs to be parsed by Dovecot, either by a static  
userdb or entries in your userdb/passwd files.

Thomas

There isn't enough info for me to say what is going on, but it is a  
postfix problem. I am using 100% postfix virtual users, and it checks  
the virtual tables before passing it to virtual_transport (dovecot lda  
in my case).

The issue is probably the restrictions lines, like  
smtp_recipient_restrictions, not having a line to reject unknown  
recipients. But could be any number of issues, that is just the  
simplest.

Quoting Mauricio Tavares <raubvogel at gmail.com>:
>       I do not know if this is a help request to solve a problem but
> to understand what is going on. I am going to start with the problem
> description as I see it and we will go from there.
>
> I have postfix setup to use dovecot for tls/sasl in addition to its
> normal imap/pop3 functions. Postfix is also setup to do virtual
> domains, getting its information from the files valias, vmaps, and
> vhosts which are all located in its /etc/postfix directory:
>
> # Virtual domain stuff
> virtual_mailbox_domains = /etc/postfix/vhosts.txt
> virtual_mailbox_base = /var/spool/vmail
> virtual_mailbox_maps = hash:/etc/postfix/vmaps.txt
> # 1500:1500 is user virtual
> virtual_uid_maps = static:1500
> virtual_gid_maps = static:1500
> virtual_alias_maps = hash:/etc/postfix/valias.txt
>
> Dovecot uses its own unix passwd-like files to do authentication. In
> fact, as you can see, I broke them apart so I can make the actual
> password file readable only by root:
>
> mail_location = maildir:~/
>
> auth default {
>   passdb passwd-file {
>      args = /etc/dovecot/passwd
>   }
>   userdb passwd-file {
>      args = /etc/dovecot/users
>   }
> }
>
> I know the norm is to use a database or at least ldap, but I had my
> reasons. Mailscanner is also installed. So, the whay things have been
> working so far was, postfix receives an email and checks if it is
> recipient exists here. If so, it would pass it to mailscanner and then
> get it back, when it would then drop it at the recipient's mailbox in
> /var/spool/vmail/domain.com/user. A user would then connect to dovecot
> to retrieve the email. And when an email was to be sent, postfix would
> use dovecot to do the user authentication. So far so good.
>
> Now I want to add dovecot lda so I can run the cmu sieve plugin (this
> is dovecot 1.0.1) primarily to move emails mailscanner (and
> spamassassin) thinks that are spam to each virtual  user's  Spam
> maildir directory.
>
> So, reading http://wiki.dovecot.org/LDA/Sieve/CMU, I add the following
> entries to dovecot.conf:
>
> protocol lda {
>   mail_plugin_dir = /usr/lib/dovecot/modules/lda
>
>   sendmail_path = /usr/lib/sendmail
>
>   auth_socket_path = /var/run/dovecot/auth-master
>   mail_plugins = cmusieve
>   global_script_path = /etc/dovecot/scripts/dovecot.sieve
>
>   mail_debug = yes
>   log_path = /var/log/dovecot-lda
>   info_log_path = /var/log/dovecot-lda
>
> }
>
> # I am being rather lazy in my sieve config entry. I did make sure the file
> # is owned by virtual:virtual
> plugin {
>   sieve = /etc/dovecot/scripts/dovecot.sieve
> }
>
>   socket listen {
>     master {
>       path = /var/run/dovecot/auth-master
>       mode = 0600
>       user = virtual # User running Dovecot LDA's deliver
>     }
>     # Postfix is using dovecot for SMTP AUTH (SASL)
>     client {
>        path = /var/spool/postfix/private/auth
>        mode = 0660
>        user = postfix
>        group = postfix
>     }
>   }
>
> Then, based on http://wiki.dovecot.org/LDA/Postfix, I edit postfix's  
>  master.cf:
>
> dovecot   unix  -       n       n       -       -       pipe
>   flags=DRhu user=virtual:virtual argv=/usr/lib/dovecot/deliver -f
> ${sender} -d ${user}@${nexthop}
>
> and main.cf:
> dovecot_destination_recipient_limit = 1
> virtual_transport = dovecot
> # mailbox_virtual_domains was already defined as shown above
>
> The dovecot.sieve file looks like this:
>
> require ["fileinto", "reject"];
>
> # Spam stuff
> if header :contains "X-Spam-Level"
"********************" {
>         discard;
>         stop;
> }
> elsif header :contains "X-Spam-Status" "Yes" {
>         fileinto "Spam";
>         stop;
> }
>
> So, I restart it and try to send a message to my account, say
> raub at domain.com, in this machine. It bounces back saying the user does
> not exist. I check th elog files and they seem to agree:
>
> dovecot: 2009-06-30 16:31:44 Info: auth(default):
> passwd-file(raub at domain.com): lookup: user=raub at domain.com
> file=/etc/dovecot/users
> dovecot: 2009-06-30 16:31:44 Info: auth(default):
> passwd-file(raub at domain.com): unknown user
>
> Now this is where I am trying to understand what is going on. It seems
> to me that postfix *or* dovecot went to  /etc/dovecot/users looking
> for raub at domain.com. Not finding it there (I had it defined there as
> raub simply; I have mail_location = maildir:~/ in dovecot.conf, so the
> username in this file does not need to be related to the user's email
> address), it decided user does not exist and bounced back.
>
> My testing indicated that happened after postfix passed the email to
> dovecot lda (I guess that virtual_transport = dovecot means it goes to
> master.cf and looks for the dovecot entry and then passes the message
> to it). That would mean postfix is not using vmaps to see if the user
> exists anymore. Instead, it is delegating checking if a user exists to
> dovecot, which is not what I want to do. I want postfix to worry about
> getting mail and making sure the recipient exists. Out of the blue, if
> I replaced the virtual_mailbox entry with local_transport = dovecot,
> or nothing at all, mail is delivered as it should.
>
> My gut feeling is the master.cf dovecot entry is responsible for the
> problem, but I am not sure because I really do not know everything
> that is going on there; this is the first time I venture in that file.
>
> So, could anyone shed some light on what is causing the problem? Thanks!
>

Le 1 juil. 09 ? 18:05, I wrote:
> [...]
> Could you augment your description with the output of
> 	postconf -n
> 	dovecot -n
> as well as all the log entries written by both Postfix and Dovecot  
> when you send your email to raub at domain.com, please without  
> obfuscating the data: its already sufficiently complicated... ;-)
> [...]
I forgot an essential piece of information:
	postconf mail_version

Thanks,
Axel