thr3ads.net - dovecot - [Dovecot] dsync replication errors [Jan 2013]

If this information is useful, please help other people find it:
Share via:

Oli Schacher

2013-Jan-31 10:27 UTC

[Dovecot] dsync replication errors

Hi

I'm trying to build a cluster of two servers with dsync replication
(based on http://wiki2.dovecot.org/Replication). My test setup works
fine for very simple tests, I can log in to both servers, copy a
message to one of the servers and it successfully apperars in the other
account. But, if I try to copy a large amount of messages at once to
one of the accounts, my maillogs get flodded with errors(see below) and
the mailboxes seem to get out of sync and messages are duplicated over
and over again (I originally copied 100 messages and ended up with
thousands in both mailboxes until I killed dovecot)

I'd appreciate if someone could have a look at my config and tell me
what I did wrong.

dovecot.conf of both servers, they are identical except for the target
ip in mail_replica:

dovecot -n
# 2.2.beta1 (070ca24e5846+): /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-279.19.1.el6.x86_64 x86_64 CentOS release 6.3
(Final) disable_plaintext_auth = no
mail_plugins = " notify replication"
namespace {
  inbox = yes
  location = 
  prefix = 
  separator = /
  type = private
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf
  driver = sql
}
plugin {
  mail_replica = remote:vmail at 192.168.23.62
}
protocols = pop3 imap
service aggregator {
  fifo_listener replication-notify-fifo {
    user = vmail
  }
  unix_listener replication-notify {
    user = vmail
  }
}
service auth {
  unix_listener auth-master {
    group = vmail
    mode = 0660
    user = vmail
  }
  user = root
}
service replicator {
  process_min_avail = 1
}
ssl = no
userdb {
  args = /etc/dovecot/dovecot-sql.conf
  driver = sql
}



Log on server1 after I copied 100 messages to an account on that server:

Jan 31 10:41:04 doco1 dovecot: imap-login: Login: user=<user1>,
method=PLAIN, rip=192.168.23.130, lip=192.168.23.61, mpid=1432,
session=<OdjlbJLUmwDAqBeC>
Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning:
Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID
(old uid=72, file=1359625327.M621257P1432.doco1,S=2472,W=2547:2,)
Jan 31 10:42:12 doco1 dovecot: dsync-local(user1): Error: Recent flags state
corrupted for mailbox INBOX
Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning:
Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID
(old uid=73, file=1359625327.M740847P1432.doco1,S=2417,W=2492:2,)
Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning:
Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID
(old uid=74, file=1359625328.M206735P1432.doco1,S=2400,W=2474:2,)
Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning:
Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID
(old uid=75, file=1359625328.M668118P1432.doco1,S=2421,W=2496:2,)
Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning:
Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID
(old uid=76, file=1359625329.M167578P1432.doco1,S=2480,W=2559:2,)
Jan 31 10:42:13 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning:
Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID
(old uid=77, file=1359625329.M520528P1432.doco1,S=2525,W=2604:2,)
Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 132:
1359625329.M520528P1432.doco1,S=2525,W=2604 (uid 77 -> 133)
Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 133:
1359625327.M621257P1432.doco1,S=2472,W=2547 (uid 72 -> 134)
Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 134:
1359625327.M740847P1432.doco1,S=2417,W=2492 (uid 73 -> 135)
Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 135:
1359625328.M206735P1432.doco1,S=2400,W=2474 (uid 74 -> 136)
Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 136:
1359625328.M668118P1432.doco1,S=2421,W=2496 (uid 75 -> 137)
Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 137:
1359625329.M167578P1432.doco1,S=2480,W=2559 (uid 76 -> 138)
Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 139:
1359625329.M782065P1432.doco1,S=2461,W=2539 (uid 78 -> 140)
Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 140:
1359625329.M973834P1432.doco1,S=2523,W=2602 (uid 79 -> 141)
Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 141:
1359625330.M114922P1432.doco1,S=2546,W=2626 (uid 80 -> 142)
Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 142:
1359625330.M255922P1432.doco1,S=2467,W=2546 (uid 81 -> 143)
Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 143:
1359625330.M403257P1432.doco1,S=2534,W=2611 (uid 82 -> 144)
Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 144:
1359625330.M486295P1432.doco1,S=2451,W=2529 (uid 83 -> 145)
Jan 31 10:42:14 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error:
Mailbox INBOX: Unexpected GUID mismatch for UID=85:
1359625327.M740847P1432.doco1,S=2417,W=2492 != 2e1b1ee97994566870c02910ea929091
Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error:
Mailbox INBOX: Remote didn't send mail
GUID=1359625329.M167578P1432.doco1,S=2480,W=2559 (UID=175)
Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error:
Mailbox INBOX: Remote didn't send mail
GUID=1359625329.M167578P1432.doco1,S=2480,W=2559 (UID=150)
Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error:
Mailbox INBOX: Remote didn't send mail
GUID=1359625329.M520528P1432.doco1,S=2525,W=2604 (UID=176)
Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error:
Mailbox INBOX: Remote didn't send mail
GUID=1359625329.M520528P1432.doco1,S=2525,W=2604 (UID=179)
Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error:
Mailbox INBOX: Remote didn't send mail
GUID=1359625327.M740847P1432.doco1,S=2417,W=2492 (UID=123)
Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error:
Mailbox INBOX: Remote didn't send mail
GUID=1359625327.M740847P1432.doco1,S=2417,W=2492 (UID=147)
Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error:
Mailbox INBOX: Remote didn't send mail
GUID=1359625328.M668118P1432.doco1,S=2421,W=2496 (UID=174)
Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error:
Mailbox INBOX: Remote didn't send mail
GUID=1359625328.M668118P1432.doco1,S=2421,W=2496 (UID=149)
Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error:
Mailbox INBOX: Remote didn't send mail
GUID=1359625328.M206735P1432.doco1,S=2400,W=2474 (UID=173)
Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error:
Mailbox INBOX: Remote didn't send mail
GUID=1359625328.M206735P1432.doco1,S=2400,W=2474 (UID=148)
Jan 31 10:42:16 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error:
Importing mailbox INBOX failed
Jan 31 10:42:16 doco1 dovecot: dsync-local(user1): Error: read(vmail at
192.168.23.62) failed: EOF
Jan 31 10:42:16 doco1 dovecot: dsync-local(user1): Error: Remote command
returned error 75
Jan 31 10:42:20 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error:
Mailbox INBOX: Remote didn't send mail
GUID=1359625329.M167578P1432.doco1,S=2480,W=2559 (UID=230)
[...]

Log on server2:
Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 132:
1359625329.M520528P1432.doco1,S=2525,W=2604 (uid 77 -> 133)
Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 133:
1359625327.M621257P1432.doco1,S=2472,W=2547 (uid 72 -> 134)
Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 134:
1359625327.M740847P1432.doco1,S=2417,W=2492 (uid 73 -> 135)
Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 135:
1359625328.M206735P1432.doco1,S=2400,W=2474 (uid 74 -> 136)
Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 136:
1359625328.M668118P1432.doco1,S=2421,W=2496 (uid 75 -> 137)
Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 137:
1359625329.M167578P1432.doco1,S=2480,W=2559 (uid 76 -> 138)
Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 139:
1359625329.M782065P1432.doco1,S=2461,W=2539 (uid 78 -> 140)
Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 140:
1359625329.M973834P1432.doco1,S=2523,W=2602 (uid 79 -> 141)
Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 141:
1359625330.M114922P1432.doco1,S=2546,W=2626 (uid 80 -> 142)
Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 142:
1359625330.M255922P1432.doco1,S=2467,W=2546 (uid 81 -> 143)
Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 143:
1359625330.M403257P1432.doco1,S=2534,W=2611 (uid 82 -> 144)
Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
/mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 144:
1359625330.M486295P1432.doco1,S=2451,W=2529 (uid 83 -> 145)
Jan 31 10:42:15 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID
(old uid=84, file=1359625330.M608391P1432.doco1,S=2480,W=2559:2,)
Jan 31 10:42:15 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID
(old uid=85, file=1359625330.M813949P1432.doco1,S=2427,W=2502:2,)
Jan 31 10:42:15 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID
(old uid=86, file=1359625331.M217320P1432.doco1,S=2547,W=2625:2,)
Jan 31 10:42:15 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning:
Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID
(old uid=87, file=1359625331.M688431P1432.doco1,S=2529,W=2609:2,)
[...]

Thanks

Oli
-- 
message transmitted on 100% recycled electrons

Timo Sirainen

2013-Jan-31 11:53 UTC

head link

[Dovecot] dsync replication errors

On 31.1.2013, at 12.27, Oli Schacher <dovecot at lists.wgwh.ch> wrote:
> I'm trying to build a cluster of two servers with dsync replication
> (based on http://wiki2.dovecot.org/Replication). My test setup works
> fine for very simple tests, I can log in to both servers, copy a
> message to one of the servers and it successfully apperars in the other
> account. But, if I try to copy a large amount of messages at once to
> one of the accounts, my maillogs get flodded with errors(see below) and
> the mailboxes seem to get out of sync and messages are duplicated over
> and over again (I originally copied 100 messages and ended up with
> thousands in both mailboxes until I killed dovecot)
..> Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1):
Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a
new UID (old uid=72, file=1359625327.M621257P1432.doco1,S=2472,W=2547:2,)
Looks like some bug. Possibilities:

a) Use mdbox format instead of maildir. It works better with dsync.

b) Switch to v2.2 (latest hg version). It has a rewritte dsync that works
better.

Ideally do both. :)

Seemingly Similar Threads

Search for more maybe matching threads

dovecot - Jan 2013 - dsync replication errors

[Dovecot] dsync replication errors

[Dovecot] dsync replication errors

Seemingly Similar Threads