dovecot - Sep 2017 - sieve stopped working and doveadm mailbox list without -s shows less folders then with

Am 14.09.17 um 01:07 schrieb Timo Sirainen:
> On 7 Sep 2017, at 17.42, Ralf Becker <rb at egroupware.org> wrote:
>> Dovecot 2.2.31 with mailboxes in mdbox format.
>>
>> Since a couple of days some mailboxes have the problem, that sieve
rules
>> moving mails to folders stop working and .sieve.log in mailbox shows:
>>
>> sieve: info: started log at Sep 07 13:57:17.
>> error:
>> msgid=<20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de>:
>> failed to store into mailbox 'INBOX/Munser': Mailbox
doesn't exist:
>> INBOX/Munser.
>>
>> When I do a doveadm mailbox list -s -u <user>@<domain> I
get all folders
>> incl. the one mentioned above, while doveadm mailbox list without -s
>> shows just
>> user
>> INBOX
> Subscriptions are stored independently from the actual folders. So it looks
like the subscription file exists and is correct, but somehow you've lost
all the folders. Do you see the folders in the filesystem under user/mailboxes/
directory?
Yes, the folders exist under
/var/dovecot/imap/<domain>/<user>/mdbox/mailboxes/.
Just doveadm mailbox list -u <user>@<domain> (without -s) does only
show
INBOX and user.
(I can send you the list of folders via private mail, but I can not post
them on the list.)

Anything I can do to get Dovecot to eg. rescan the folders from the
filesystem or any other way to fix that problem?
I have it with a couple of mailboxes, so I believe it's some kind of
systematic problem, nothing the users did.

Ralf
> My guess is that it only has INBOX, which means the folders were deleted by
something (Dovecot corruption can't lose entire folders - something must
explicitly delete them).
-- 
Ralf Becker
EGroupware GmbH [www.egroupware.org]
Handelsregister HRB Kaiserslautern 3587
Gesch?ftsf?hrer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 631 31657-0


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20170918/6413c125/attachment.sig>

On 18.09.2017 12:10, Ralf Becker wrote:
> Am 14.09.17 um 01:07 schrieb Timo Sirainen:
>> On 7 Sep 2017, at 17.42, Ralf Becker <rb at egroupware.org>
wrote:
>>> Dovecot 2.2.31 with mailboxes in mdbox format.
>>>
>>> Since a couple of days some mailboxes have the problem, that sieve
rules
>>> moving mails to folders stop working and .sieve.log in mailbox
shows:
>>>
>>> sieve: info: started log at Sep 07 13:57:17.
>>> error:
>>> msgid=<20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de>:
>>> failed to store into mailbox 'INBOX/Munser': Mailbox
doesn't exist:
>>> INBOX/Munser.
>>>
>>> When I do a doveadm mailbox list -s -u <user>@<domain>
I get all folders
>>> incl. the one mentioned above, while doveadm mailbox list without
-s
>>> shows just
>>> user
>>> INBOX
>> Subscriptions are stored independently from the actual folders. So it
looks like the subscription file exists and is correct, but somehow you've
lost all the folders. Do you see the folders in the filesystem under
user/mailboxes/ directory?
> Yes, the folders exist under
> /var/dovecot/imap/<domain>/<user>/mdbox/mailboxes/.
> Just doveadm mailbox list -u <user>@<domain> (without -s) does
only show
> INBOX and user.
> (I can send you the list of folders via private mail, but I can not post
> them on the list.)
>
> Anything I can do to get Dovecot to eg. rescan the folders from the
> filesystem or any other way to fix that problem?
> I have it with a couple of mailboxes, so I believe it's some kind of
> systematic problem, nothing the users did.
>
> Ralf
>
>> My guess is that it only has INBOX, which means the folders were
deleted by something (Dovecot corruption can't lose entire folders -
something must explicitly delete them).
You can always try doveadm force-resync -u victim "*"

You should run it twice, I guess.

Aki

Hi Aki,

Am 18.09.17 um 11:13 schrieb Aki Tuomi:
> On 18.09.2017 12:10, Ralf Becker wrote:
>> Am 14.09.17 um 01:07 schrieb Timo Sirainen:
>>> On 7 Sep 2017, at 17.42, Ralf Becker <rb at egroupware.org>
wrote:
>>>> Dovecot 2.2.31 with mailboxes in mdbox format.
>>>>
>>>> Since a couple of days some mailboxes have the problem, that
sieve rules
>>>> moving mails to folders stop working and .sieve.log in mailbox
shows:
>>>>
>>>> sieve: info: started log at Sep 07 13:57:17.
>>>> error:
>>>> msgid=<20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de>:
>>>> failed to store into mailbox 'INBOX/Munser': Mailbox
doesn't exist:
>>>> INBOX/Munser.
>>>>
>>>> When I do a doveadm mailbox list -s -u
<user>@<domain> I get all folders
>>>> incl. the one mentioned above, while doveadm mailbox list
without -s
>>>> shows just
>>>> user
>>>> INBOX
>>> Subscriptions are stored independently from the actual folders. So
it looks like the subscription file exists and is correct, but somehow
you've lost all the folders. Do you see the folders in the filesystem under
user/mailboxes/ directory?
>> Yes, the folders exist under
>> /var/dovecot/imap/<domain>/<user>/mdbox/mailboxes/.
>> Just doveadm mailbox list -u <user>@<domain> (without -s)
does only show
>> INBOX and user.
>> (I can send you the list of folders via private mail, but I can not
post
>> them on the list.)
>>
>> Anything I can do to get Dovecot to eg. rescan the folders from the
>> filesystem or any other way to fix that problem?
>> I have it with a couple of mailboxes, so I believe it's some kind
of
>> systematic problem, nothing the users did.
>>
>> Ralf
>>
>>> My guess is that it only has INBOX, which means the folders were
deleted by something (Dovecot corruption can't lose entire folders -
something must explicitly delete them).
> You can always try doveadm force-resync -u victim "*"
>
> You should run it twice, I guess.
Tried that before and just tried it again, no luck :(

root at fra-nfs-mail:/var/dovecot/imap/<domain># doveadm force-resync -u
<user>@<domain> "*"
doveadm(<user>@<domain>): Warning: fscking index file
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index
doveadm(<user>@<domain>): Warning: mdbox
/var/dovecot/imap/<domain>/<user>/mdbox/storage: rebuilding indexes
doveadm(<user>@<domain>): Warning: Transaction log file
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index.log
was locked for 72 seconds (mdbox storage rebuild)
doveadm(<user>@<domain>): Warning: fscking index file
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index

root at fra-nfs-mail:/var/dovecot/imap/<domain># doveadm force-resync -u
<user>@<domain> "*"
doveadm(<user>@<domain>): Warning: fscking index file
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index
doveadm(<user>@<domain>): Warning: mdbox
/var/dovecot/imap/<domain>/<user>/mdbox/storage: rebuilding indexes
doveadm(<user>@<domain>): Warning: fscking index file
/var/dovecot/imap/<domain>/<user>/mdbox/storage/dovecot.map.index

root at fra-nfs-mail:/var/dovecot/imap/<domain># doveadm mailbox list -u
<user>@<domain>
user
INBOX

What else can I do to analyse the problem?

Ralf

-- 
Ralf Becker
EGroupware GmbH [www.egroupware.org]
Handelsregister HRB Kaiserslautern 3587
Gesch?ftsf?hrer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 631 31657-0


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20170918/8b0116a7/attachment.sig>

On 18 Sep 2017, at 12.10, Ralf Becker <rb at egroupware.org <mailto:rb at
egroupware.org>> wrote:
> 
> Am 14.09.17 um 01:07 schrieb Timo Sirainen:
>> On 7 Sep 2017, at 17.42, Ralf Becker <rb at egroupware.org
<mailto:rb at egroupware.org>> wrote:
>>> Dovecot 2.2.31 with mailboxes in mdbox format.
>>> 
>>> Since a couple of days some mailboxes have the problem, that sieve
rules
>>> moving mails to folders stop working and .sieve.log in mailbox
shows:
>>> 
>>> sieve: info: started log at Sep 07 13:57:17.
>>> error:
>>> msgid=<20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de
<mailto:20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de>>:
>>> failed to store into mailbox 'INBOX/Munser': Mailbox
doesn't exist:
>>> INBOX/Munser.
>>> 
>>> When I do a doveadm mailbox list -s -u <user>@<domain>
I get all folders
>>> incl. the one mentioned above, while doveadm mailbox list without
-s
>>> shows just
>>> user
>>> INBOX
>> Subscriptions are stored independently from the actual folders. So it
looks like the subscription file exists and is correct, but somehow you've
lost all the folders. Do you see the folders in the filesystem under
user/mailboxes/ directory?
> 
> Yes, the folders exist under
> /var/dovecot/imap/<domain>/<user>/mdbox/mailboxes/.
> Just doveadm mailbox list -u <user>@<domain> (without -s) does
only show
> INBOX and user.
> (I can send you the list of folders via private mail, but I can not post
> them on the list.)
> 
> Anything I can do to get Dovecot to eg. rescan the folders from the
> filesystem or any other way to fix that problem?
> I have it with a couple of mailboxes, so I believe it's some kind of
> systematic problem, nothing the users did.
I can't really think of any reason why it wouldn't simply work.
Especially since you're not using v2.2.32, the folder listing is always
performed by listing the directories in filesystem, so there's nothing
really to resync. What's your doveconf -n? You could try with
mailbox_list_index=no if that happens to make any difference, but it
shouldn't.

You could also try what "strace -o log -s 100 doveadm mailbox list -u user
at domain" shows. Is it opening the correct mailboxes/ directory? Maybe the
path is just wrong for some reason (some typo added somewhere)?

Hi Timo,

Am 18.09.17 um 12:03 schrieb Timo Sirainen:
> On 18 Sep 2017, at 12.10, Ralf Becker <rb at egroupware.org
> <mailto:rb at egroupware.org>> wrote:
>>
>> Am 14.09.17 um 01:07 schrieb Timo Sirainen:
>>> On 7 Sep 2017, at 17.42, Ralf Becker <rb at egroupware.org
>>> <mailto:rb at egroupware.org>> wrote:
>>>> Dovecot 2.2.31 with mailboxes in mdbox format.
>>>>
>>>> Since a couple of days some mailboxes have the problem, that
sieve
>>>> rules
>>>> moving mails to folders stop working and .sieve.log in mailbox
shows:
>>>>
>>>> sieve: info: started log at Sep 07 13:57:17.
>>>> error:
>>>> msgid=<20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de
>>>> <mailto:20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de>>:
>>>> failed to store into mailbox 'INBOX/Munser': Mailbox
doesn't exist:
>>>> INBOX/Munser.
>>>>
>>>> When I do a doveadm mailbox list -s -u
<user>@<domain> I get all
>>>> folders
>>>> incl. the one mentioned above, while doveadm mailbox list
without -s
>>>> shows just
>>>> user
>>>> INBOX
>>> Subscriptions are stored independently from the actual folders. So
>>> it looks like the subscription file exists and is correct, but
>>> somehow you've lost all the folders. Do you see the folders in
the
>>> filesystem under user/mailboxes/ directory??
>>
>> Yes, the folders exist under
>> /var/dovecot/imap/<domain>/<user>/mdbox/mailboxes/.
>> Just doveadm mailbox list -u <user>@<domain> (without -s)
does only show
>> INBOX and user.
>> (I can send you the list of folders via private mail, but I can not
post
>> them on the list.)
>>
>> Anything I can do to get Dovecot to eg. rescan the folders from the
>> filesystem or any other way to fix that problem?
>> I have it with a couple of mailboxes, so I believe it's some kind
of
>> systematic problem, nothing the users did.
>
> I can't really think of any reason why it wouldn't simply work.
> Especially since you're not using v2.2.32, the folder listing is
> always performed by listing the directories in filesystem, so there's
> nothing really to resync. What's your doveconf -n? You could try with
> mailbox_list_index=no if that happens to make any difference, but it
> shouldn't.
>
> You could also try what "strace -o log -s 100 doveadm mailbox list -u
> user at domain" shows. Is it opening the correct mailboxes/ directory?
> Maybe the path is just wrong for some reason (some typo added somewhere)?

Nope it lstats the correct directories, but does not show them.

I send you the strace / sysdig output per private mail, as it contains
private information of that user.

Ralf

-- 
Ralf Becker
EGroupware GmbH [www.egroupware.org]
Handelsregister HRB Kaiserslautern 3587
Gesch?ftsf?hrer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 631 31657-0


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20170918/cb6ffef4/attachment.sig>