Wolfgang Hennerbichler
2015-Jun-20 20:29 UTC
Duplicate mails with pop3 + dsync replication
Hi, When I enable (and it only happens when it is actually running) replication over TCP my users get duplicate mails via pop3. Here?s what happens: * User gets mail * Mail appears in (Maildir) new/ folder on both replicas * user retrieves mail (retr), calls dele. Mail still exists in new/ folder on both replicas. * user runs retr (and gets the duplicate) and dele a second time, the mail is actually removed. If I disable replication this behaviour goes away. I use dovecot from debian stable resources. Any help would _really_ be appreciated. I found one german thread that dealt with the same problem, but no solution for it (http://www.dovecot-buch.de/mailingliste/pop3-fehler-bei-delete-und-dsync-replikation/) wogri # dovecot --version 2.2.13 # dovecot -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.0 ext4 auth_verbose = yes doveadm_password = XXX first_valid_gid = 113 first_valid_uid = 108 last_valid_gid = 113 last_valid_uid = 108 mail_gid = dovecot mail_location = maildir:/var/mail/%n mail_plugins = quota fts fts_lucene virtual notify replication mail_temp_dir = /var/lib/dovecot/tmp mail_uid = dovecot namespace { list = children location = virtual:/var/mail/%n/virtual prefix = virtual. separator = . } namespace inbox { inbox = yes list = yes location = mailbox "Deleted Messages" { auto = subscribe special_use = \Trash } mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } [48/4646] mailbox Trash { special_use = \Trash } prefix = separator = . subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { default_language = de fts = lucene fts_lucene = whitespace_chars=@. mail_replica = tcp:172.16.1.1:12345 quota = maildir:User quota quota_rule = *:storage=4G quota_rule2 = Trash:storage=+100M quota_rule3 = Spam:ignore quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /etc/sieve/%n.sieve sieve_default = /etc/sieve/default.sieve sieve_dir = ~/sieve sieve_extensions = +editheader } postmaster_address = postmaster at wogri.at protocols = " imap lmtp pop3" service aggregator { fifo_listener replication-notify-fifo { user = dovecot } unix_listener replication-notify { user = dovecot } } service doveadm { inet_listener { port = 12345 } } service imap { process_limit = 1024 } } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 2003 } user = dovecot } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service pop3 { process_limit = 1024 } service quota-warning { executable = script /usr/local/sbin/quota-warning.sh unix_listener quota-warning { user = dovecot } user = dovecot } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 } } ssl = required ssl_cert = </etc/exim4/server.cert.pem ssl_key = </etc/exim4/server.key.pem userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = quota sieve virtual notify replication } protocol lda { mail_plugins = quota fts fts_lucene virtual notify replication sieve } protocol imap { mail_max_userip_connections = 50 mail_plugins = quota fts fts_lucene virtual notify replication imap_quota } -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4127 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20150620/57c827b1/attachment-0001.p7s>
Wolfgang Hennerbichler
2015-Jun-21 19:05 UTC
Duplicate mails with pop3 + dsync replication
FWIW I just tried the sdbox mailbox format for a singe user (not the whole installation) by overriding the mail-userattr via ldap and the problem persists, so I don?t think it?s an issue with maildir per se, but rather a pop3 / replication issue or a misconfiguration on my side. Wolfgang> On 20 Jun 2015, at 22:29, Wolfgang Hennerbichler <wogri at wogri.com> wrote: > > Hi, > > When I enable (and it only happens when it is actually running) replication over TCP my users get duplicate mails via pop3. > Here?s what happens: > > * User gets mail > * Mail appears in (Maildir) new/ folder on both replicas > * user retrieves mail (retr), calls dele. Mail still exists in new/ folder on both replicas. > * user runs retr (and gets the duplicate) and dele a second time, the mail is actually removed. > If I disable replication this behaviour goes away. > I use dovecot from debian stable resources. > Any help would _really_ be appreciated. I found one german thread that dealt with the same problem, but no solution for it (http://www.dovecot-buch.de/mailingliste/pop3-fehler-bei-delete-und-dsync-replikation/) > > wogri > > # dovecot --version > 2.2.13 > > # dovecot -n > # 2.2.13: /etc/dovecot/dovecot.conf > # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.0 ext4 > auth_verbose = yes > doveadm_password = XXX > first_valid_gid = 113 > first_valid_uid = 108 > last_valid_gid = 113 > last_valid_uid = 108 > mail_gid = dovecot > mail_location = maildir:/var/mail/%n > mail_plugins = quota fts fts_lucene virtual notify replication > mail_temp_dir = /var/lib/dovecot/tmp > mail_uid = dovecot > namespace { > list = children > location = virtual:/var/mail/%n/virtual > prefix = virtual. > separator = . > } > namespace inbox { > inbox = yes > list = yes > location = > mailbox "Deleted Messages" { > auto = subscribe > special_use = \Trash > } > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > auto = no > special_use = \Sent > } > mailbox Spam { > auto = subscribe > special_use = \Junk > } [48/4646] > mailbox Trash { > special_use = \Trash > } > prefix = > separator = . > subscriptions = yes > type = private > } > passdb { > args = /etc/dovecot/dovecot-ldap.conf.ext > driver = ldap > } > plugin { > default_language = de > fts = lucene > fts_lucene = whitespace_chars=@. > mail_replica = tcp:172.16.1.1:12345 > quota = maildir:User quota > quota_rule = *:storage=4G > quota_rule2 = Trash:storage=+100M > quota_rule3 = Spam:ignore > quota_warning = storage=95%% quota-warning 95 %u > quota_warning2 = storage=80%% quota-warning 80 %u > sieve = /etc/sieve/%n.sieve > sieve_default = /etc/sieve/default.sieve > sieve_dir = ~/sieve > sieve_extensions = +editheader > } > postmaster_address = postmaster at wogri.at > protocols = " imap lmtp pop3" > service aggregator { > fifo_listener replication-notify-fifo { > user = dovecot > } > unix_listener replication-notify { > user = dovecot > } > } > service doveadm { > inet_listener { > port = 12345 > } > } > service imap { > process_limit = 1024 > } > } > service lmtp { > inet_listener lmtp { > address = 127.0.0.1 > port = 2003 > } > user = dovecot > } > service managesieve-login { > inet_listener sieve { > port = 4190 > } > service_count = 1 > } > service pop3 { > process_limit = 1024 > } > service quota-warning { > executable = script /usr/local/sbin/quota-warning.sh > unix_listener quota-warning { > user = dovecot > } > user = dovecot > } > service replicator { > process_min_avail = 1 > unix_listener replicator-doveadm { > mode = 0600 > } > } > ssl = required > ssl_cert = </etc/exim4/server.cert.pem > ssl_key = </etc/exim4/server.key.pem > userdb { > args = /etc/dovecot/dovecot-ldap.conf.ext > driver = ldap > } > protocol lmtp { > mail_plugins = quota sieve virtual notify replication > } > protocol lda { > mail_plugins = quota fts fts_lucene virtual notify replication sieve > } > protocol imap { > mail_max_userip_connections = 50 > mail_plugins = quota fts fts_lucene virtual notify replication imap_quota > } >-------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4127 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20150621/684ab8df/attachment-0001.p7s>
Wolfgang Hennerbichler
2015-Jun-22 06:49 UTC
Duplicate mails with pop3 + dsync replication
It turns out that if I enable this option: pop3_deleted_flag = "$POP3Deleted? The issue no longer persists. I have to manually expunge the kept mails that have been deleted via pop3 though: doveadm expunge mailbox INBOX KEYWORD '$POP3Deleted' -A Wolfgang> On 21 Jun 2015, at 21:05, Wolfgang Hennerbichler <wogri at wogri.com> wrote: > > FWIW I just tried the sdbox mailbox format for a singe user (not the whole installation) by overriding the mail-userattr via ldap and the problem persists, so I don?t think it?s an issue with maildir per se, but rather a pop3 / replication issue or a misconfiguration on my side. > > Wolfgang > >> On 20 Jun 2015, at 22:29, Wolfgang Hennerbichler <wogri at wogri.com> wrote: >> >> Hi, >> >> When I enable (and it only happens when it is actually running) replication over TCP my users get duplicate mails via pop3. >> Here?s what happens: >> >> * User gets mail >> * Mail appears in (Maildir) new/ folder on both replicas >> * user retrieves mail (retr), calls dele. Mail still exists in new/ folder on both replicas. >> * user runs retr (and gets the duplicate) and dele a second time, the mail is actually removed. >> If I disable replication this behaviour goes away. >> I use dovecot from debian stable resources. >> Any help would _really_ be appreciated. I found one german thread that dealt with the same problem, but no solution for it (http://www.dovecot-buch.de/mailingliste/pop3-fehler-bei-delete-und-dsync-replikation/) >> >> wogri >> >> # dovecot --version >> 2.2.13 >> >> # dovecot -n >> # 2.2.13: /etc/dovecot/dovecot.conf >> # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.0 ext4 >> auth_verbose = yes >> doveadm_password = XXX >> first_valid_gid = 113 >> first_valid_uid = 108 >> last_valid_gid = 113 >> last_valid_uid = 108 >> mail_gid = dovecot >> mail_location = maildir:/var/mail/%n >> mail_plugins = quota fts fts_lucene virtual notify replication >> mail_temp_dir = /var/lib/dovecot/tmp >> mail_uid = dovecot >> namespace { >> list = children >> location = virtual:/var/mail/%n/virtual >> prefix = virtual. >> separator = . >> } >> namespace inbox { >> inbox = yes >> list = yes >> location = >> mailbox "Deleted Messages" { >> auto = subscribe >> special_use = \Trash >> } >> mailbox Drafts { >> special_use = \Drafts >> } >> mailbox Junk { >> special_use = \Junk >> } >> mailbox Sent { >> special_use = \Sent >> } >> mailbox "Sent Messages" { >> auto = no >> special_use = \Sent >> } >> mailbox Spam { >> auto = subscribe >> special_use = \Junk >> } [48/4646] >> mailbox Trash { >> special_use = \Trash >> } >> prefix = >> separator = . >> subscriptions = yes >> type = private >> } >> passdb { >> args = /etc/dovecot/dovecot-ldap.conf.ext >> driver = ldap >> } >> plugin { >> default_language = de >> fts = lucene >> fts_lucene = whitespace_chars=@. >> mail_replica = tcp:172.16.1.1:12345 >> quota = maildir:User quota >> quota_rule = *:storage=4G >> quota_rule2 = Trash:storage=+100M >> quota_rule3 = Spam:ignore >> quota_warning = storage=95%% quota-warning 95 %u >> quota_warning2 = storage=80%% quota-warning 80 %u >> sieve = /etc/sieve/%n.sieve >> sieve_default = /etc/sieve/default.sieve >> sieve_dir = ~/sieve >> sieve_extensions = +editheader >> } >> postmaster_address = postmaster at wogri.at >> protocols = " imap lmtp pop3" >> service aggregator { >> fifo_listener replication-notify-fifo { >> user = dovecot >> } >> unix_listener replication-notify { >> user = dovecot >> } >> } >> service doveadm { >> inet_listener { >> port = 12345 >> } >> } >> service imap { >> process_limit = 1024 >> } >> } >> service lmtp { >> inet_listener lmtp { >> address = 127.0.0.1 >> port = 2003 >> } >> user = dovecot >> } >> service managesieve-login { >> inet_listener sieve { >> port = 4190 >> } >> service_count = 1 >> } >> service pop3 { >> process_limit = 1024 >> } >> service quota-warning { >> executable = script /usr/local/sbin/quota-warning.sh >> unix_listener quota-warning { >> user = dovecot >> } >> user = dovecot >> } >> service replicator { >> process_min_avail = 1 >> unix_listener replicator-doveadm { >> mode = 0600 >> } >> } >> ssl = required >> ssl_cert = </etc/exim4/server.cert.pem >> ssl_key = </etc/exim4/server.key.pem >> userdb { >> args = /etc/dovecot/dovecot-ldap.conf.ext >> driver = ldap >> } >> protocol lmtp { >> mail_plugins = quota sieve virtual notify replication >> } >> protocol lda { >> mail_plugins = quota fts fts_lucene virtual notify replication sieve >> } >> protocol imap { >> mail_max_userip_connections = 50 >> mail_plugins = quota fts fts_lucene virtual notify replication imap_quota >> } >> >-------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4127 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20150622/92e1a58e/attachment.p7s>