dovecot - May 2015 - Alternate Storage altmove not working bug

Hello everyone!

I would like to report a bug here. We are using dovecot's sdbox with
alternate storage here. From time to time my SSD partition gets full and I
always find users with lots of e-mails stuck there, and altmove just won't
move them. It moves most of the e-mails correctly, but a few was getting
stuck everyday.

So, I think I have found the problem! When a user moves the message from
one mailbox to another, the move will keep all the flags!
Including MAIL_INDEX_MAIL_FLAG_BACKEND 0x40. So, altmove think this message
was already moved and doesn't do anything.

I did a doveadm dump on the destination mailbox's index and what I see is a
lot of this:

RECORD: seq=104, uid=104, flags=0x48 (Seen)
 - ext 0 keywords  :            (0000)
 - ext 1 modseq    :         12 (0c00000000000000)
 - ext 3 cache     :      14004 (b4360000)
 - cache offset=14004 size=76, prev_offset = 0
    - mime.parts:
(48000000b404000000000000cc04000000000000e542000000000000fd4300000000000018010000)
(pos=0 hdr.p=1204 hdr.v=1228 body.p=17125 body.v=17405 flags=48)
    - size.physical: 18329 (9947000000000000)
    - date.received: 1418235366 (e68d8854)

Although the 0x40 flag is set, the file still lives on the main/ssd storage
area.

My config: http://pastebin.com/8Wk4bfRK
Dovecot version: 2.2.5-0~auto+22
Host is Xeon E3 64bit, running Debian 7.
Filesystem is plain local ext4 on both storage areas

I know my version is outdated but I didn't find anything related to this
there.

Just one last question: how can I remove that flag from the message? I have
taken a look at doveadm-flags(1) but it doesn't seem to allow anything on
that flag.

Thank you very much!

Best,
Daniel Colchete

On 15 May 2015, at 02:50, Daniel van Ham Colchete <daniel.colchete at
gmail.com> wrote:
> 
> Hello everyone!
> 
> I would like to report a bug here. We are using dovecot's sdbox with
> alternate storage here. From time to time my SSD partition gets full and I
> always find users with lots of e-mails stuck there, and altmove just
won't
> move them. It moves most of the e-mails correctly, but a few was getting
> stuck everyday.
> 
> So, I think I have found the problem! When a user moves the message from
> one mailbox to another, the move will keep all the flags!
> Including MAIL_INDEX_MAIL_FLAG_BACKEND 0x40. So, altmove think this message
> was already moved and doesn't do anything.
Thanks, fixed: http://hg.dovecot.org/dovecot-2.2/rev/9ad3f11617f1
> Just one last question: how can I remove that flag from the message? I have
> taken a look at doveadm-flags(1) but it doesn't seem to allow anything
on
> that flag.
You can use doveadm altmove -r parameter to "move" the mail back to
the primary storage. This fixes the flag. Then you can move it to alt storage.

9 hours for a fix and a release! Wow! That's really fast! Thank you very
much!

On Fri, May 15, 2015 at 5:17 AM, Timo Sirainen <tss at iki.fi> wrote:
> On 15 May 2015, at 02:50, Daniel van Ham Colchete <
> daniel.colchete at gmail.com> wrote:
> >
> > Hello everyone!
> >
> > I would like to report a bug here. We are using dovecot's sdbox
with
> > alternate storage here. From time to time my SSD partition gets full
and
> I
> > always find users with lots of e-mails stuck there, and altmove just
> won't
> > move them. It moves most of the e-mails correctly, but a few was
getting
> > stuck everyday.
> >
> > So, I think I have found the problem! When a user moves the message
from
> > one mailbox to another, the move will keep all the flags!
> > Including MAIL_INDEX_MAIL_FLAG_BACKEND 0x40. So, altmove think this
> message
> > was already moved and doesn't do anything.
>
> Thanks, fixed: http://hg.dovecot.org/dovecot-2.2/rev/9ad3f11617f1
>
> > Just one last question: how can I remove that flag from the message? I
> have
> > taken a look at doveadm-flags(1) but it doesn't seem to allow
anything on
> > that flag.
>
> You can use doveadm altmove -r parameter to "move" the mail back
to the
> primary storage. This fixes the flag. Then you can move it to alt storage.
>