Hello all, Definitely congrats on the 1.0.0 release! Anyway, I'm having a problem, that may or may not be related to Dovecot. Specifically, duplicate (and sometimes triplicate) emails are appearing. These emails have exactly the same queue ID, so its not that they were sent multiple times. These dup emails do not show up to me ever. Postfix is being used as the SMTP, and it hands it over to procmail. I set up procmail's verbose logging feature for that user and on the duplicate emails that he receives, it shows only ONE write to a file for that specific subject of email. Tonight / this morning I'll be tracking down to see if this duplicate for sure happens in IMAP. Note: Its possible that part of the problem is he leaves two IMAP clients open at the same time (Netscape I think). I'm going to verify that with him. Any ideas? I'm NOT trying to blame Dovecot, I'm just trying to explore options. Right now its looking fairly bad for Dovecot though....I mean, what else is there after the mail is delivered to the appropriate place in the mailbox? Software versions: postfix 2.3.8 procmail 3.22 dovecot 1.0.rc28 Thanks, Phillip
When this happens with sendmail, it can be that the queue handler doesn't finish before a new queue handler is kicked off, so each successive queue handler/runner ends up sending the same mail. Background: sendmail, and perhaps postfix, is configured to a) toss stuff it can't handle right away into the queue and then, every so often (interval to be set by admin), it burps the queue. If the interval is too short, the previous queue handler/runner doesn't complete before a new one is invoked and gets it list of things to mail...with the same email on both queue handler/runners incarnations to do list........... Phillip T. George wrote:> Hello all, > > Definitely congrats on the 1.0.0 release! > > Anyway, I'm having a problem, that may or may not be related to > Dovecot. Specifically, duplicate (and sometimes triplicate) emails > are appearing. These emails have exactly the same queue ID, so its > not that they were sent multiple times. These dup emails do not show > up to me ever. > > Postfix is being used as the SMTP, and it hands it over to procmail. > I set up procmail's verbose logging feature for that user and on the > duplicate emails that he receives, it shows only ONE write to a file > for that specific subject of email. Tonight / this morning I'll be > tracking down to see if this duplicate for sure happens in IMAP. > > Note: Its possible that part of the problem is he leaves two IMAP > clients open at the same time (Netscape I think). I'm going to verify > that with him. > > Any ideas? I'm NOT trying to blame Dovecot, I'm just trying to > explore options. Right now its looking fairly bad for Dovecot > though....I mean, what else is there after the mail is delivered to > the appropriate place in the mailbox? > > Software versions: > postfix 2.3.8 > procmail 3.22 > dovecot 1.0.rc28 > > Thanks, > Phillip-- ===Stewart Dean, Unix System Admin, Henderson Computer Resources Center of Bard College, Annandale-on-Hudson, New York 12504 sdean at bard.edu voice: 845-758-7475, fax: 845-758-7035
Phillip T. George wrote:> Hello all, > > Definitely congrats on the 1.0.0 release! > > Anyway, I'm having a problem, that may or may not be related to > Dovecot. Specifically, duplicate (and sometimes triplicate) emails are > appearing. These emails have exactly the same queue ID, so its not that > they were sent multiple times. These dup emails do not show up to me ever.What are you using for delivery? It seems to me the only way this could be dovecot related is if you are using dovecot-deliver. Am I wrong? -- Best regards, Charles
On Fri, 2007-04-13 at 09:10 -0500, Phillip T. George wrote:> Anyway, I'm having a problem, that may or may not be related to > Dovecot. Specifically, duplicate (and sometimes triplicate) emails are > appearing. These emails have exactly the same queue ID, so its not that > they were sent multiple times. These dup emails do not show up to me ever.What kind of filenames do they have in maildir? Do you see anything in Dovecot's error logs? The only reason why Dovecot would internally duplicate messages is if it detects multiple filenames with same base name, which aren't hardlinks to each others. And that could happen if the maildir filenames aren't truly unique. You said you're using Procmail, are you sure you're not using MH filenames (http://wiki.dovecot.org/MailboxFormat/Maildir)? If that's not the problem, you can enable mail_log plugin and check if it logs that the client does a duplicate COPY. http://wiki.dovecot.org/Plugins/MailLog -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20070413/b2bf1365/attachment.bin>
Timo Sirainen wrote:> On Fri, 2007-04-13 at 09:10 -0500, Phillip T. George wrote: > >> Anyway, I'm having a problem, that may or may not be related to >> Dovecot. Specifically, duplicate (and sometimes triplicate) emails are >> appearing. These emails have exactly the same queue ID, so its not that >> they were sent multiple times. These dup emails do not show up to me ever. >> > > What kind of filenames do they have in maildir? Do you see anything in > Dovecot's error logs? The only reason why Dovecot would internally > duplicate messages is if it detects multiple filenames with same base > name, which aren't hardlinks to each others. And that could happen if > the maildir filenames aren't truly unique. You said you're using > Procmail, are you sure you're not using MH filenames > (http://wiki.dovecot.org/MailboxFormat/Maildir)? > > If that's not the problem, you can enable mail_log plugin and check if > it logs that the client does a duplicate COPY. > http://wiki.dovecot.org/Plugins/MailLog >Here's a snippet from the procmail log file: procmail: Assigning "LASTFOLDER=/home/domain.com/homes/user/Maildir/new/1176426436.9170_1.east.domain.com" procmail: Notified comsat: "user.domain.com at 0:/home/domain.com/homes/user/Maildir/new/1176426436.9170_1.east.domain.com" I do not see any errors near the time the duplicate messages occurred. In /etc/postfix/main.cf : home_mailbox = Maildir/ mailbox_command = /usr/bin/procmail -a "$EXTENSION" I'll try enabling the mail_log plugin. Thanks, Phillip
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip T. George schrieb:> Hello all, > > Definitely congrats on the 1.0.0 release! > > Anyway, I'm having a problem, that may or may not be related to > Dovecot. Specifically, duplicate (and sometimes triplicate) emails are > appearing. These emails have exactly the same queue ID, so its not that > they were sent multiple times. These dup emails do not show up to me ever. > > Postfix is being used as the SMTP, and it hands it over to procmail. I > set up procmail's verbose logging feature for that user and on the > duplicate emails that he receives, it shows only ONE write to a file for > that specific subject of email. Tonight / this morning I'll be tracking > down to see if this duplicate for sure happens in IMAP. > > Note: Its possible that part of the problem is he leaves two IMAP > clients open at the same time (Netscape I think). I'm going to verify > that with him. > > Any ideas? I'm NOT trying to blame Dovecot, I'm just trying to explore > options. Right now its looking fairly bad for Dovecot though....I mean, > what else is there after the mail is delivered to the appropriate place > in the mailbox? > > Software versions: > postfix 2.3.8 > procmail 3.22 > dovecot 1.0.rc28 > > Thanks, > Phillip >Hi Phillip, this surely gets not to dovecot, my first tip goes to procmail read http://www.postfix.org/faq.html#duplicate http://pm-doc.sourceforge.net/pm-tips-body.html search Order of the procmail recipes use google duplicate procmail after all you can double mails with filters like amavis, clamsmtp therefor your have to include -o receive_override_options=no_address_mappings, in master cf in the filter lines, after all apple mail has long list of bug with imap with double emails too, look at you mail log and you will find the reason - -- Mit freundlichen Gruessen Best Regards Robert Schetterer https://www.schetterer.org Munich/Bavaria/Germany -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFGH+yxfGH2AvR16oERAnVIAJ9fRakXwcxXuLAB/+O2E2D18OoL3gCghoof +IGwfVBgaiHnvbt0i3TQ3nU=BkSX -----END PGP SIGNATURE-----
Timo Sirainen wrote:> On Fri, 2007-04-13 at 09:10 -0500, Phillip T. George wrote: > >> Anyway, I'm having a problem, that may or may not be related to >> Dovecot. Specifically, duplicate (and sometimes triplicate) emails are >> appearing. These emails have exactly the same queue ID, so its not that >> they were sent multiple times. These dup emails do not show up to me ever. >> > > What kind of filenames do they have in maildir? Do you see anything in > Dovecot's error logs? The only reason why Dovecot would internally > duplicate messages is if it detects multiple filenames with same base > name, which aren't hardlinks to each others. And that could happen if > the maildir filenames aren't truly unique. You said you're using > Procmail, are you sure you're not using MH filenames > (http://wiki.dovecot.org/MailboxFormat/Maildir)? > > If that's not the problem, you can enable mail_log plugin and check if > it logs that the client does a duplicate COPY. > http://wiki.dovecot.org/Plugins/MailLog >Ok. Using the logging, I saw 2 entries today: Apr 16 12:51:45 east dovecot: IMAP(user.domain.com), IP(70.177.64.93): copy -> Spam: uid=718, msgid=<sameid at domain.com> Apr 16 12:51:45 east dovecot: IMAP(user.domain.com), IP(70.168.68.56): copy -> Spam: uid=718, msgid=<sameid at domain.com> It appears that the problem is that the user has two email programs running the same filters at the same time. Somehow, this creates a duplicate message. Any clues as to why this might be? Are there any options that may cause this behavior to change? I thought maybe the options "maildir_copy_with_hardlinks" and/or "maildir_copy_preserve_filename" may be helpful options...(they're both set to the default of 'no'). Any recommendations? Thanks, Phillip