dovecot - Nov 2021 - dsync error: Received unexpected input d != N

Hi,

I'm trying to set up replication with dsync, and I'm getting errors like
this:

dsync-local(david at mandelberg.org)<...>: Error: dsync(...): Received 
unexpected input d != N

I tried the `doveadm -D sync` command from 
https://doc.dovecot.org/configuration_manual/replication/#administration 
and its output looked like:

...
dsync-local(david at mandelberg.org)<...>: Debug: brain M: Locking done by
remote (local hostname=..., remote hostname=...)
dsync-local(david at mandelberg.org)<...>: Debug: brain M: Local mailbox 
tree: ...
[line above repeated many times for different mailboxes]
dsync-local(david at mandelberg.org)<...>: Error: dsync(...): Received 
unexpected input d != N
dsync-local(david at mandelberg.org)<...>: Error: dsync(...): Received 
unexpected input d != N
dsync-local(david at mandelberg.org)<...>: Debug: auth-master: conn 
unix:/run/dovecot/auth-userdb (pid=47097,uid=0): Disconnected: 
Connection closed (fd=8)

I'm a bit stuck. Does anybody have any ideas what's going on? The output
of `doveconf -n` is attached.
-------------- next part --------------
# 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.13 (cdd19fe3)
# OS: Linux 5.10.0-9-amd64 x86_64 Debian 11.1 ext4
# Hostname: mail-storage-bbf78862.virgo.mandelberg.org
auth_username_chars = +-.0123456789 at
ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz
dsync_remote_cmd = nc.openbsd -U /run/dovecot/dovecot-replication-client
lda_mailbox_autosubscribe = yes
mail_gid = vmail
mail_home = /var/cache/mail/%{domain}/%{username}/dovecot-home
mail_location =
maildir:/var/local/mail/persistent/mail/%{domain}/%{username}/Maildir
mail_plugins = listescape notify replication
mail_uid = vmail
namespace inbox {
  inbox = yes
  location = 
  mailbox Archive {
    special_use = \Archive
  }
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix = 
  separator = /
}
passdb {
  args = /etc/dovecot/conf.d/10-passwd.passdb
  driver = passwd-file
}
plugin {
  mail_replica = remoteprefix:foo at mail-storage-acadbae1.fornax.mandelberg.org
  sieve =
file:/usr/local/etc/mail/%{domain}/%{username}/sieve;active=/var/cache/mail/%{domain}/%{username}/active.sieve;bindir=/var/cache/mail/%{domain}/%{username}/sieve
  sieve_before = file:/etc/dovecot/sieve-before
  sieve_filter_bin_dir = /etc/dovecot/sieve-filter-bin
  sieve_filter_exec_timeout = 540s
  sieve_global_extensions = +vnd.dovecot.filter
  sieve_plugins = sieve_extprograms
  sieve_user_log =
/var/local/mail/persistent/mail/%{domain}/%{username}/sieve.log
}
postmaster_address = postmaster@%{hostname}
protocols = " imap lmtp"
service aggregator {
  fifo_listener replication-notify-fifo {
    user = vmail
  }
  unix_listener replication-notify {
    user = vmail
  }
}
service auth-worker {
  user = $default_internal_user
}
service auth {
  unix_listener auth-userdb {
    group = vmail
    mode = 0660
  }
}
service imap-login {
  inet_listener imaps {
    port = 993
    ssl = yes
  }
}
service lmtp {
  unix_listener lmtp {
    group = stunnel4
    mode = 0660
  }
  user = vmail
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
    mode = 0600
    user = vmail
  }
}
service stats {
  unix_listener stats-writer {
    mode = 0666
  }
}
ssl = required
ssl_cert =
</etc/letsencrypt/live/mail-storage-general.private.mandelberg.org/fullchain.pem
ssl_cipher_list = HIGH:!kRSA:!eNULL:!aNULL:@SECLEVEL=2:@STRENGTH
ssl_key = # hidden, use -P to show it
ssl_min_protocol = TLSv1.2
userdb {
  args = /etc/dovecot/conf.d/10-passwd.userdb
  driver = passwd-file
}
protocol lmtp {
  mail_plugins = listescape notify replication sieve
}
local_name mail-storage-general.private.mandelberg.org {
  ssl_cert =
</etc/letsencrypt/live/mail-storage-general.private.mandelberg.org/fullchain.pem
  ssl_key = # hidden, use -P to show it
}
local_name mail-storage-strict.private.mandelberg.org {
  ssl_cert =
</usr/local/etc/x509/mail-storage-strict.private.mandelberg.org/cert.pem
  ssl_key = # hidden, use -P to show it
}

Op 02-11-2021 om 22:43 schreef David Mandelberg:
> Hi,
> 
> I'm trying to set up replication with dsync, and I'm getting errors
like
> this:
I figured it out :)
> dsync-local(david at mandelberg.org)<...>: Error: dsync(...):
Received
> unexpected input d != N
To debug this, I changed dsync_remote_cmd to a script with these contents:

#!/bin/sh
tee /tmp/tmp.MsfWIQCvNk/outbound.log | nc.openbsd -U 
/run/dovecot/dovecot-replication-client | tee 
/tmp/tmp.MsfWIQCvNk/inbound.log

Then looking at the inbound.log file, the lines that seemed to be 
causing the "d != N" errors looked like this:

dsync-remote(david at mandelberg.org)<...>: Error: dsync(local): Remote 
dsync doesn't use compatible protocol

I had written a wrapper script in python that did this:

username = sys.stdin.readline().rstrip('\n')
os.execvp('doveadm', ('doveadm', 'dsync-server',
'-u', username))

Apparently sys.stdin.readline() reads more than just the one line from 
stdin, so it was consuming the VERSION line.


The actual issue was totally my fault for the wrapper script, but is 
there any chance dovecot could log the "Remote dsync doesn't use 
compatible protocol" lines instead of (or in addition to) sending them 
to the other end? That would have saved some debugging effort.