Jason Forester
2008-May-21 22:21 UTC
[Dovecot] Vanishing maildirsize files with 1.0.7 using 'deliver' via postfix
Aloha, I've searched the list and have found no similar reports, so I'm asking here. We have 3 RH5.1 boxes running the stock Postfix 2.3.3, but due to issues with the stock Dovecot 1.0rc15, Redhat supplied us with the version of Dovecot 1.0.7 that will be in Redhat 5.2. The mail store is automounted per-user from a Sun box via NFS. We were using it with no problems until last Friday, when we attempted to implement quotas. I had done quite a lot of testing on a single system, so I didn't anticipate any issues here. I created maildirsize files for each user in the NFS-mounted /var/spool/mail/$USER and everything seemed fine. I changed mailbox_command to /usr/libexec/dovecot/deliver in postfix main.cf. Mail was delivered, quota files updated, all good. However almost immediately a maildirsize folder vanished. Soon after another, then another, and now we're missing nearly 40 of them. Any ideas as to why they're vanishing? Something to do with NFS? I've tried to trigger it by massively sending to the same user on all three systems but have yet to reproduce it at will. Any clues would be appreciated. For reasons of support I cannot upgrade to a later dovecot at this time. Best Regards, - jason
Charles Marcus
2008-May-22 01:36 UTC
[Dovecot] Vanishing maildirsize files with 1.0.7 using 'deliver' via postfix
Jason Forester wrote:> We have 3 RH5.1 boxes running the stock Postfix 2.3.3, but due to issues > with the stock Dovecot 1.0rc15, Redhat supplied us with the version of > Dovecot 1.0.7 that will be in Redhat 5.2.>> The mail store is automounted per-user from a Sun box via NFS.Timo strongly recommends to use 1.1 if you are using NFS... 1.1 is available via atrpms.net
Jason Forester
2008-May-24 00:58 UTC
[Dovecot] Vanishing maildirsize files with 1.0.7 using 'deliver' via postfix
Charles Marcus wrote:>> Jason Forester wrote: >> We have 3 RH5.1 boxes running the stock Postfix 2.3.3, but due to issues >> with the stock Dovecot 1.0rc15, Redhat supplied us with the version of >> Dovecot 1.0.7 that will be in Redhat 5.2. >> >> The mail store is automounted per-user from a Sun box via NFS. > >Timo strongly recommends to use 1.1 if you are using NFS... > >1.1 is available via atrpms.netI do understand, but as I stated fairly clearly I thought "For reasons of support I cannot upgrade to a later dovecot at this time." Of course I have a ticket open with RedHat, but I thought I might come to the source, find out if it's some form of a known bug, and then perhaps I *could* use that as leverage to get another version installed. So the question still stands: Is this a known bug? Is maildirsize not NFS safe? Knowing that would be a huge first step for me, so thanks in advance for any assistance. Best Regards, -jason On Wed, May 21, 2008 at 12:21 PM, Jason Forester <hilobird at gmail.com> wrote:> > Aloha, > > I've searched the list and have found no similar reports, so I'm asking here. > > We have 3 RH5.1 boxes running the stock Postfix 2.3.3, but due to issues with the stock Dovecot 1.0rc15, Redhat supplied us with the version of Dovecot 1.0.7 that will be in Redhat 5.2. > > The mail store is automounted per-user from a Sun box via NFS. > > We were using it with no problems until last Friday, when we attempted to implement quotas. I had done quite a lot of testing on a single system, so I didn't anticipate any issues here. I created maildirsize files for each user in the NFS-mounted /var/spool/mail/$USER and everything seemed fine. I changed mailbox_command to /usr/libexec/dovecot/deliver in postfix main.cf. Mail was delivered, quota files updated, all good. > > However almost immediately a maildirsize folder vanished. Soon after another, then another, and now we're missing nearly 40 of them. > > Any ideas as to why they're vanishing? Something to do with NFS? I've tried to trigger it by massively sending to the same user on all three systems but have yet to reproduce it at will. > > Any clues would be appreciated. For reasons of support I cannot upgrade to a later dovecot at this time. > > Best Regards, > > - jason
Timo Sirainen
2008-May-25 12:25 UTC
[Dovecot] Vanishing maildirsize files with 1.0.7 using 'deliver' via postfix
On Wed, 2008-05-21 at 12:21 -1000, Jason Forester wrote:> We have 3 RH5.1 boxes running the stock Postfix 2.3.3, but due to issues > with the stock Dovecot 1.0rc15, Redhat supplied us with the version of > Dovecot 1.0.7 that will be in Redhat 5.2.There is one change to Maildir++ quota since v1.0.7: http://hg.dovecot.org/dovecot-1.0/rev/288fd70ac258 But that's mostly an optimization and doesn't cause files to be deleted.> However almost immediately a maildirsize folder vanished. Soon after > another, then another, and now we're missing nearly 40 of them.Maildir++ quota recalculation works like: Go through all files and sum up their sizes. Keep track of the highest directory mtime. Once everything is calculated, again go through all directories and find the highest mtime. If the highest mtime had increased, something had changed, so delete maildirsize file. So could it be simply that the recalculation is slow enough that the maildir changes during it and causes the file to be deleted? Another possibility is that some error happens, but then Dovecot should have logged something. Do you keep your quota limits stored only in the maildirsize file? If not, having the files deleted shouldn't affect functionality, but of course the performance is worse. The rebuild performance could be improved if maildir filenames contain ",S=xxx" containing the file's size so stat() calls can be avoided. But you can't really change existing filenames without having them show up as new mails. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20080525/e24daccd/attachment-0002.bin>
Jason Forester
2008-May-25 19:09 UTC
[Dovecot] Vanishing maildirsize files with 1.0.7 using 'deliver' via postfix
Hi Timo, thanks for replying. My comments are inline: On Sun, May 25, 2008 at 2:25 AM, Timo Sirainen <tss at iki.fi> wrote: <snip>>> However almost immediately a maildirsize folder vanished. Soon after >> another, then another, and now we're missing nearly 40 of them. > > Maildir++ quota recalculation works like: > > Go through all files and sum up their sizes. Keep track of the highest > directory mtime. Once everything is calculated, again go through all > directories and find the highest mtime. If the highest mtime had > increased, something had changed, so delete maildirsize file. > > So could it be simply that the recalculation is slow enough that the > maildir changes during it and causes the file to be deleted? Another > possibility is that some error happens, but then Dovecot should have > logged something.I'm not seeing anything in my logs. It's certainly possible that multiple mails are delivered during a recalculation. I'm not quite sure I understand the mtime explanation above, are you saying that if dovecot thinks that the maildirsize file is inaccurate it deletes it and goes on?> Do you keep your quota limits stored only in the maildirsize file? If > not, having the files deleted shouldn't affect functionality, but of > course the performance is worse.Basically yes, we have many users with different quotas and as such we cannot use a single quota in the config file. So, it sounds like I'm the only person who has encountered this problem? If it's really just a consequence of the maildir changing, wouldn't it be more common? Thanks again Timo, - jason