dovecot - Sep 2017 - sieve stopped working and doveadm mailbox list without -s shows less folders then with

Hi Aki,

Am 18.09.17 um 11:13 schrieb Aki Tuomi:
> On 18.09.2017 12:10, Ralf Becker wrote:
>> Am 14.09.17 um 01:07 schrieb Timo Sirainen:
>>> On 7 Sep 2017, at 17.42, Ralf Becker <rb at egroupware.org>
wrote:
>>>> Dovecot 2.2.31 with mailboxes in mdbox format.
>>>>
>>>> Since a couple of days some mailboxes have the problem, that
sieve rules
>>>> moving mails to folders stop working and .sieve.log in mailbox
shows:
>>>>
>>>> sieve: info: started log at Sep 07 13:57:17.
>>>> error:
>>>> msgid=<20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de>:
>>>> failed to store into mailbox 'INBOX/Munser': Mailbox
doesn't exist:
>>>> INBOX/Munser.
>>>>
>>>> When I do a doveadm mailbox list -s -u
<user>@<domain> I get all folders
>>>> incl. the one mentioned above, while doveadm mailbox list
without -s
>>>> shows just
>>>> user
>>>> INBOX
>>> Subscriptions are stored independently from the actual folders. So
it looks like the subscription file exists and is correct, but somehow
you've lost all the folders. Do you see the folders in the filesystem under
user/mailboxes/ directory?
>> Yes, the folders exist under
>> /var/dovecot/imap/<domain>/<user>/mdbox/mailboxes/.
>> Just doveadm mailbox list -u <user>@<domain> (without -s)
does only show
>> INBOX and user.
>> (I can send you the list of folders via private mail, but I can not
post
>> them on the list.)
>>
>> Anything I can do to get Dovecot to eg. rescan the folders from the
>> filesystem or any other way to fix that problem?
>> I have it with a couple of mailboxes, so I believe it's some kind
of
>> systematic problem, nothing the users did.
>>
>> Ralf
>>
>>> My guess is that it only has INBOX, which means the folders were
deleted by something (Dovecot corruption can't lose entire folders -
something must explicitly delete them).
> You can always try doveadm force-resync -u victim "*"
>
> You should run it twice, I guess.
Tried that before and just tried it again, no luck :(

root at fra-nfs-mail:/var/dovecot/imap/<domain># doveadm force-resync -u
<user>@<domain> "*"
doveadm(<user>@<domain>): Warning: fscking index file
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index
doveadm(<user>@<domain>): Warning: mdbox
/var/dovecot/imap/<domain>/<user>/mdbox/storage: rebuilding indexes
doveadm(<user>@<domain>): Warning: Transaction log file
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index.log
was locked for 72 seconds (mdbox storage rebuild)
doveadm(<user>@<domain>): Warning: fscking index file
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index

root at fra-nfs-mail:/var/dovecot/imap/<domain># doveadm force-resync -u
<user>@<domain> "*"
doveadm(<user>@<domain>): Warning: fscking index file
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index
doveadm(<user>@<domain>): Warning: mdbox
/var/dovecot/imap/<domain>/<user>/mdbox/storage: rebuilding indexes
doveadm(<user>@<domain>): Warning: fscking index file
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index

root at fra-nfs-mail:/var/dovecot/imap/<domain># doveadm mailbox list -u
<user>@<domain>
user
INBOX

What else can I do to analyse the problem?

Ralf

-- 
Ralf Becker
EGroupware GmbH [www.egroupware.org]
Handelsregister HRB Kaiserslautern 3587
Gesch?ftsf?hrer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 631 31657-0


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20170918/8b0116a7/attachment.sig>

On 18.09.2017 12:20, Ralf Becker wrote:
> Hi Aki,
>
> Am 18.09.17 um 11:13 schrieb Aki Tuomi:
>> On 18.09.2017 12:10, Ralf Becker wrote:
>>> Am 14.09.17 um 01:07 schrieb Timo Sirainen:
>>>> On 7 Sep 2017, at 17.42, Ralf Becker <rb at
egroupware.org> wrote:
>>>>> Dovecot 2.2.31 with mailboxes in mdbox format.
>>>>>
>>>>> Since a couple of days some mailboxes have the problem,
that sieve rules
>>>>> moving mails to folders stop working and .sieve.log in
mailbox shows:
>>>>>
>>>>> sieve: info: started log at Sep 07 13:57:17.
>>>>> error:
>>>>> msgid=<20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz
at somedomain.egroupware.de>:
>>>>> failed to store into mailbox 'INBOX/Munser':
Mailbox doesn't exist:
>>>>> INBOX/Munser.
>>>>>
>>>>> When I do a doveadm mailbox list -s -u
<user>@<domain> I get all folders
>>>>> incl. the one mentioned above, while doveadm mailbox list
without -s
>>>>> shows just
>>>>> user
>>>>> INBOX
>>>> Subscriptions are stored independently from the actual folders.
So it looks like the subscription file exists and is correct, but somehow
you've lost all the folders. Do you see the folders in the filesystem under
user/mailboxes/ directory?
>>> Yes, the folders exist under
>>> /var/dovecot/imap/<domain>/<user>/mdbox/mailboxes/.
>>> Just doveadm mailbox list -u <user>@<domain> (without
-s) does only show
>>> INBOX and user.
>>> (I can send you the list of folders via private mail, but I can not
post
>>> them on the list.)
>>>
>>> Anything I can do to get Dovecot to eg. rescan the folders from the
>>> filesystem or any other way to fix that problem?
>>> I have it with a couple of mailboxes, so I believe it's some
kind of
>>> systematic problem, nothing the users did.
>>>
>>> Ralf
>>>
>>>> My guess is that it only has INBOX, which means the folders
were deleted by something (Dovecot corruption can't lose entire folders -
something must explicitly delete them).
>> You can always try doveadm force-resync -u victim "*"
>>
>> You should run it twice, I guess.
> Tried that before and just tried it again, no luck :(
>
> root at fra-nfs-mail:/var/dovecot/imap/<domain># doveadm force-resync
-u
> <user>@<domain> "*"
> doveadm(<user>@<domain>): Warning: fscking index file
>
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index
> doveadm(<user>@<domain>): Warning: mdbox
> /var/dovecot/imap/<domain>/<user>/mdbox/storage: rebuilding
indexes
> doveadm(<user>@<domain>): Warning: Transaction log file
>
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index.log
> was locked for 72 seconds (mdbox storage rebuild)
> doveadm(<user>@<domain>): Warning: fscking index file
>
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index
>
> root at fra-nfs-mail:/var/dovecot/imap/<domain># doveadm force-resync
-u
> <user>@<domain> "*"
> doveadm(<user>@<domain>): Warning: fscking index file
>
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index
> doveadm(<user>@<domain>): Warning: mdbox
> /var/dovecot/imap/<domain>/<user>/mdbox/storage: rebuilding
indexes
> doveadm(<user>@<domain>): Warning: fscking index file
>
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index
>
> root at fra-nfs-mail:/var/dovecot/imap/<domain># doveadm mailbox list
-u
> <user>@<domain>
> user
> INBOX
>
> What else can I do to analyse the problem?
>
> Ralf
>
It seems you are running into
https://github.com/dovecot/core/commit/c8be39472a93a5de2cc1051bdbd4468bea0ca7ba#diff-664ea8e9082f57f29f8a284ced77d165

Aki

Hi Aki,

Am 18.09.17 um 11:22 schrieb Aki Tuomi:
> On 18.09.2017 12:20, Ralf Becker wrote:
>> Hi Aki,
>>
>> Am 18.09.17 um 11:13 schrieb Aki Tuomi:
>>> On 18.09.2017 12:10, Ralf Becker wrote:
>>>> Am 14.09.17 um 01:07 schrieb Timo Sirainen:
>>>>> On 7 Sep 2017, at 17.42, Ralf Becker <rb at
egroupware.org> wrote:
>>>>>> Dovecot 2.2.31 with mailboxes in mdbox format.
>>>>>>
>>>>>> Since a couple of days some mailboxes have the problem,
that sieve rules
>>>>>> moving mails to folders stop working and .sieve.log in
mailbox shows:
>>>>>>
>>>>>> sieve: info: started log at Sep 07 13:57:17.
>>>>>> error:
>>>>>>
msgid=<20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de>:
>>>>>> failed to store into mailbox 'INBOX/Munser':
Mailbox doesn't exist:
>>>>>> INBOX/Munser.
>>>>>>
>>>>>> When I do a doveadm mailbox list -s -u
<user>@<domain> I get all folders
>>>>>> incl. the one mentioned above, while doveadm mailbox
list without -s
>>>>>> shows just
>>>>>> user
>>>>>> INBOX
>>>>> Subscriptions are stored independently from the actual
folders. So it looks like the subscription file exists and is correct, but
somehow you've lost all the folders. Do you see the folders in the
filesystem under user/mailboxes/ directory?
>>>> Yes, the folders exist under
>>>> /var/dovecot/imap/<domain>/<user>/mdbox/mailboxes/.
>>>> Just doveadm mailbox list -u <user>@<domain>
(without -s) does only show
>>>> INBOX and user.
>>>> (I can send you the list of folders via private mail, but I can
not post
>>>> them on the list.)
>>>>
>>>> Anything I can do to get Dovecot to eg. rescan the folders from
the
>>>> filesystem or any other way to fix that problem?
>>>> I have it with a couple of mailboxes, so I believe it's
some kind of
>>>> systematic problem, nothing the users did.
>>>>
>>>> Ralf
>>>>
>>>>> My guess is that it only has INBOX, which means the folders
were deleted by something (Dovecot corruption can't lose entire folders -
something must explicitly delete them).
>>> You can always try doveadm force-resync -u victim "*"
>>>
>>> You should run it twice, I guess.
>> Tried that before and just tried it again, no luck :(
>>
>> root at fra-nfs-mail:/var/dovecot/imap/<domain># doveadm
force-resync -u
>> <user>@<domain> "*"
>> doveadm(<user>@<domain>): Warning: fscking index file
>>
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index
>> doveadm(<user>@<domain>): Warning: mdbox
>> /var/dovecot/imap/<domain>/<user>/mdbox/storage: rebuilding
indexes
>> doveadm(<user>@<domain>): Warning: Transaction log file
>>
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index.log
>> was locked for 72 seconds (mdbox storage rebuild)
>> doveadm(<user>@<domain>): Warning: fscking index file
>>
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index
>>
>> root at fra-nfs-mail:/var/dovecot/imap/<domain># doveadm
force-resync -u
>> <user>@<domain> "*"
>> doveadm(<user>@<domain>): Warning: fscking index file
>>
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index
>> doveadm(<user>@<domain>): Warning: mdbox
>> /var/dovecot/imap/<domain>/<user>/mdbox/storage: rebuilding
indexes
>> doveadm(<user>@<domain>): Warning: fscking index file
>>
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index
>>
>> root at fra-nfs-mail:/var/dovecot/imap/<domain># doveadm mailbox
list -u
>> <user>@<domain>
>> user
>> INBOX
>>
>> What else can I do to analyse the problem?
>>
>> Ralf
>>
> It seems you are running into
>
https://github.com/dovecot/core/commit/c8be39472a93a5de2cc1051bdbd4468bea0ca7ba#diff-664ea8e9082f57f29f8a284ced77d165
That commit is part of 2.2.32, as far as I can see on Github, so I
*only* need to update?

I'm a bit reluctant to update, after all the problems in the version
bitween 2.2.27 and 2.2.31 ...

You recon the update 2.2.31 to .32 has no know problems so far?

Ralf

-- 
Ralf Becker
EGroupware GmbH [www.egroupware.org]
Handelsregister HRB Kaiserslautern 3587
Gesch?ftsf?hrer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 631 31657-0


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20170918/c770692c/attachment-0001.sig>