dovecot - Feb 2012 - Slightly more intelligent way of handling issues in sdbox?

Hi there,

I was doing some testing on sdbox yesterday. Basically I did the 
following procedure:

1) Create new sdbox; deliver 2 messages into it (u.1, u.2)
2) Create a copy of the index file (no cache file created yet)
3) deliver another message to the mailbox (u.3)
4) copy back index file from stage (2)
5) deliver new mail

Then the message delivered in stage 3 ie u.3 gets replaced with the 
message delivered in (5) also called u.3. Is it possible to try an 
open/access call on the mail file before overwriting it with the new 
message in case we have an issue where an older version of the index 
file is present (eg due to nfs latencies) ? I notice when you are 
expunging files you very carefully open them and read the header 
contents to make sure the guid is the same as in the index - any reason 
that this is not done when delivering? This is with lmtp on dovecot 2.0.16.

I also noticed that index corruption in sdbox does not get automatically 
repaired. I know this is because the flags are stored in the index files 
so you'd get some loss of flags, but in many situations for us this 
auto-repair with flag loss would be better than having the mailbox 
locked out until we manually do a force-resync on it. (speaking of 
which, it would be great if force-resync also rebuilt the cache files if 
there are valid cache files around, rather than just doing away with them)

Thanks,

Mark

On 3.2.2012, at 16.16, Mark Zealey wrote:
> I was doing some testing on sdbox yesterday. Basically I did the following
procedure:
> 
> 1) Create new sdbox; deliver 2 messages into it (u.1, u.2)
> 2) Create a copy of the index file (no cache file created yet)
> 3) deliver another message to the mailbox (u.3)
> 4) copy back index file from stage (2)
> 5) deliver new mail
> 
> Then the message delivered in stage 3 ie u.3 gets replaced with the message
delivered in (5) also called u.3.
http://hg.dovecot.org/dovecot-2.1/rev/a765e0a895a9 fixes this.
> Is it possible to try an open/access call on the mail file before
overwriting it with the new message in case we have an issue where an older
version of the index file is present (eg due to nfs latencies) ? I notice when
you are expunging files you very carefully open them and read the header
contents to make sure the guid is the same as in the index - any reason that
this is not done when delivering? This is with lmtp on dovecot 2.0.16.
Hm. Yes, I guess there should be a check to avoid overwriting files.
> I also noticed that index corruption in sdbox does not get automatically
repaired. I know this is because the flags are stored in the index files so
you'd get some loss of flags, but in many situations for us this auto-repair
with flag loss would be better than having the mailbox locked out until we
manually do a force-resync on it.
I'm not entirely sure what you mean by this. Does the above patch help with
this problem also?
> (speaking of which, it would be great if force-resync also rebuilt the
cache files if there are valid cache files around, rather than just doing away
with them)
Well, ideally there shouldn't be so much corruption that this matters..