dovecot - Mar 2015 - Problem with replication ("Fatal: -N parameter requires syncing with remote host")

I have been fighting, for 2 days, with a Dovecot setup and I now
officially need help :)

Basically, I am trying to setup replication between 2 identical Dovecot
servers, on different physical servers (different hostnames/etc, of
course), and although I've managed to solve many problems I can't
overcome the final hurdle.

When a mail comes in to one of the servers, replication over TCP should
kick in and mail should be replicated, but instead I get this in the
mail log:

-------------------------------------------------------------------------------------------------
dovecot2 dovecot: doveadm(user1 at test1.int): Fatal: -N parameter requires
syncing with remote host
-------------------------------------------------------------------------------------------------

Mail is delivered to dovecot2 server, Postfix delivers it via LMTP, and
everything seems to work ok. However, I can't figure out the cause for
the error that I am seeing. I have looked into source code and found
where this error is triggered (line 834 in
doveadm/dsync/doveadm-dsync.c, inside Dovecot 2.2.9 source code), but I
am afraid that I don't understand the context (ctx->run_type
=DSYNC_RUN_TYPE_LOCAL) and it would take me a lot of time to debug the
whole thing from the scratch. I've never looked into Dovecot internals
before, so it's all a mystery to me at this point.

I've tried stracing the process but the flow does not make sense to me,
I can't understand what exactly happens before error is displayed. Also,
there is no traffic to port 12345 on other node, so execution fails
before syncing even starts (and error message is not related with remote
node config/setup/etc).

Triggering replication manually, from command line, works as expected:

# doveadm sync -A tcp:192.168.1.100

(at this point user1's mail folder is properly updated on remote server)

I've tried searching around, but it appears that noone else has this
problem(or isn't complaining about it :).

I am not sure if I am doing something wrong (apart from replication). I
will need to support multiple virtual domains and I am basically using
passwd-file for authentication, so I am using a single passwd file for
all virtual domain users which allows user lookup (doveadm user '*') to
work, which is required by replication.

This is configuration file from dovecot2 server (which I use for testing
and delivering mails):

--- config start ---

# 2.2.9: /etc/dovecot/dovecot.conf
# OS: Linux 3.16.0-30-generic x86_64 Ubuntu 14.04.2 LTS ext4
debug_log_path = /var/log/dovecot_debug.log
doveadm_password = 67890
doveadm_port = 12345
mail_debug = yes
mail_location = maildir:/vmail/mail/%d/%n/Maildir
mail_plugins = " notify replication"
namespace inbox {
  inbox = yes
  location   mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix }
passdb {
  args = username_format=%u /vmail/auth/passwd
  driver = passwd-file
}
plugin {
  mail_replica = 192.168.1.100
}
protocols = imap lmtp
service aggregator {
  fifo_listener replication-notify-fifo {
    user = vmail
  }
  unix_listener replication-notify {
    user = vmail
  }
}
service auth-worker {
  unix_listener auth-worker {
    user = dovecot-auth
  }
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0660
    user = postfix
  }
  user = dovecot-auth
}
service doveadm {
  inet_listener {
    port = 12345
  }
}
service imap-login {
  inet_listener imap {
    port = 0
  }
}
service lmtp {
  inet_listener lmtp {
    address = 127.0.0.1
    port = 10024
  }
  process_min_avail = 3
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
    mode = 0660
    user = postfix
  }
  user = vmail
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
    mode = 0666
  }
}
ssl_cert = </etc/dovecot/dovecot.pem
ssl_key = </etc/dovecot/private/dovecot.pem
userdb {
  args = username_format=%u /vmail/auth/passwd
  default_fields = uid=vmail gid=vmail home=/vmail/mail/%d/%n
  driver = passwd-file
}
protocol lmtp {
  postmaster_address = root at localhost
}


--- config end ---

Configuration file on other node is exactly the same, except different
IP address in mail_replica statement.

Any help is much appreciated.

Thank you.

As it usually happens, I found out the answer after reading my own mail
to mailing list multiple times. I knew it had to be simple.

Somehow, I managed to screw up config and mail_plugin section had the
entry "192.168.1.100" instead of "tcp:192.168.1.100".
Replication
appears to be working now (at least few test mails that I got through
triggered it).

I am aware that I need to prepend tcp: and I am 100% sure I did it
initially, but at some point I removed it and never noticed it (and
never paid attention to it). Whoops.

So... If anyone runs into this problem, fix it easy :)

On 03/16/2015 03:12 PM, Vanja wrote:
> I have been fighting, for 2 days, with a Dovecot setup and I now
> officially need help :)
> 
> Basically, I am trying to setup replication between 2 identical Dovecot
> servers, on different physical servers (different hostnames/etc, of
> course), and although I've managed to solve many problems I can't
> overcome the final hurdle.
> 
> When a mail comes in to one of the servers, replication over TCP should
> kick in and mail should be replicated, but instead I get this in the
> mail log:
> 
>
-------------------------------------------------------------------------------------------------
> dovecot2 dovecot: doveadm(user1 at test1.int): Fatal: -N parameter requires
> syncing with remote host
>
-------------------------------------------------------------------------------------------------
> 
> Mail is delivered to dovecot2 server, Postfix delivers it via LMTP, and
> everything seems to work ok. However, I can't figure out the cause for
> the error that I am seeing. I have looked into source code and found
> where this error is triggered (line 834 in
> doveadm/dsync/doveadm-dsync.c, inside Dovecot 2.2.9 source code), but I
> am afraid that I don't understand the context (ctx->run_type =>
DSYNC_RUN_TYPE_LOCAL) and it would take me a lot of time to debug the
> whole thing from the scratch. I've never looked into Dovecot internals
> before, so it's all a mystery to me at this point.
> 
> I've tried stracing the process but the flow does not make sense to me,
> I can't understand what exactly happens before error is displayed.
Also,
> there is no traffic to port 12345 on other node, so execution fails
> before syncing even starts (and error message is not related with remote
> node config/setup/etc).
> 
> Triggering replication manually, from command line, works as expected:
> 
> # doveadm sync -A tcp:192.168.1.100
> 
> (at this point user1's mail folder is properly updated on remote
server)
> 
> I've tried searching around, but it appears that noone else has this
> problem(or isn't complaining about it :).
> 
> I am not sure if I am doing something wrong (apart from replication). I
> will need to support multiple virtual domains and I am basically using
> passwd-file for authentication, so I am using a single passwd file for
> all virtual domain users which allows user lookup (doveadm user
'*') to
> work, which is required by replication.
> 
> This is configuration file from dovecot2 server (which I use for testing
> and delivering mails):
> 
> --- config start ---
> 
> # 2.2.9: /etc/dovecot/dovecot.conf
> # OS: Linux 3.16.0-30-generic x86_64 Ubuntu 14.04.2 LTS ext4
> debug_log_path = /var/log/dovecot_debug.log
> doveadm_password = 67890
> doveadm_port = 12345
> mail_debug = yes
> mail_location = maildir:/vmail/mail/%d/%n/Maildir
> mail_plugins = " notify replication"
> namespace inbox {
>   inbox = yes
>   location >   mailbox Drafts {
>     special_use = \Drafts
>   }
>   mailbox Junk {
>     special_use = \Junk
>   }
>   mailbox Sent {
>     special_use = \Sent
>   }
>   mailbox "Sent Messages" {
>     special_use = \Sent
>   }
>   mailbox Trash {
>     special_use = \Trash
>   }
>   prefix > }
> passdb {
>   args = username_format=%u /vmail/auth/passwd
>   driver = passwd-file
> }
> plugin {
>   mail_replica = 192.168.1.100
> }
> protocols = imap lmtp
> service aggregator {
>   fifo_listener replication-notify-fifo {
>     user = vmail
>   }
>   unix_listener replication-notify {
>     user = vmail
>   }
> }
> service auth-worker {
>   unix_listener auth-worker {
>     user = dovecot-auth
>   }
> }
> service auth {
>   unix_listener /var/spool/postfix/private/auth {
>     group = postfix
>     mode = 0660
>     user = postfix
>   }
>   user = dovecot-auth
> }
> service doveadm {
>   inet_listener {
>     port = 12345
>   }
> }
> service imap-login {
>   inet_listener imap {
>     port = 0
>   }
> }
> service lmtp {
>   inet_listener lmtp {
>     address = 127.0.0.1
>     port = 10024
>   }
>   process_min_avail = 3
>   unix_listener /var/spool/postfix/private/dovecot-lmtp {
>     group = postfix
>     mode = 0660
>     user = postfix
>   }
>   user = vmail
> }
> service replicator {
>   process_min_avail = 1
>   unix_listener replicator-doveadm {
>     mode = 0666
>   }
> }
> ssl_cert = </etc/dovecot/dovecot.pem
> ssl_key = </etc/dovecot/private/dovecot.pem
> userdb {
>   args = username_format=%u /vmail/auth/passwd
>   default_fields = uid=vmail gid=vmail home=/vmail/mail/%d/%n
>   driver = passwd-file
> }
> protocol lmtp {
>   postmaster_address = root at localhost
> }
> 
> 
> --- config end ---
> 
> Configuration file on other node is exactly the same, except different
> IP address in mail_replica statement.
> 
> Any help is much appreciated.
> 
> Thank you.
>