dovecot - Sep 2017 - sieve stopped working and doveadm mailbox list without -s shows less folders then with

Hi Timo,

Am 18.09.17 um 12:03 schrieb Timo Sirainen:
> On 18 Sep 2017, at 12.10, Ralf Becker <rb at egroupware.org
> <mailto:rb at egroupware.org>> wrote:
>>
>> Am 14.09.17 um 01:07 schrieb Timo Sirainen:
>>> On 7 Sep 2017, at 17.42, Ralf Becker <rb at egroupware.org
>>> <mailto:rb at egroupware.org>> wrote:
>>>> Dovecot 2.2.31 with mailboxes in mdbox format.
>>>>
>>>> Since a couple of days some mailboxes have the problem, that
sieve
>>>> rules
>>>> moving mails to folders stop working and .sieve.log in mailbox
shows:
>>>>
>>>> sieve: info: started log at Sep 07 13:57:17.
>>>> error:
>>>> msgid=<20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de
>>>> <mailto:20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de>>:
>>>> failed to store into mailbox 'INBOX/Munser': Mailbox
doesn't exist:
>>>> INBOX/Munser.
>>>>
>>>> When I do a doveadm mailbox list -s -u
<user>@<domain> I get all
>>>> folders
>>>> incl. the one mentioned above, while doveadm mailbox list
without -s
>>>> shows just
>>>> user
>>>> INBOX
>>> Subscriptions are stored independently from the actual folders. So
>>> it looks like the subscription file exists and is correct, but
>>> somehow you've lost all the folders. Do you see the folders in
the
>>> filesystem under user/mailboxes/ directory??
>>
>> Yes, the folders exist under
>> /var/dovecot/imap/<domain>/<user>/mdbox/mailboxes/.
>> Just doveadm mailbox list -u <user>@<domain> (without -s)
does only show
>> INBOX and user.
>> (I can send you the list of folders via private mail, but I can not
post
>> them on the list.)
>>
>> Anything I can do to get Dovecot to eg. rescan the folders from the
>> filesystem or any other way to fix that problem?
>> I have it with a couple of mailboxes, so I believe it's some kind
of
>> systematic problem, nothing the users did.
>
> I can't really think of any reason why it wouldn't simply work.
> Especially since you're not using v2.2.32, the folder listing is
> always performed by listing the directories in filesystem, so there's
> nothing really to resync. What's your doveconf -n? You could try with
> mailbox_list_index=no if that happens to make any difference, but it
> shouldn't.
>
> You could also try what "strace -o log -s 100 doveadm mailbox list -u
> user at domain" shows. Is it opening the correct mailboxes/ directory?
> Maybe the path is just wrong for some reason (some typo added somewhere)?

Nope it lstats the correct directories, but does not show them.

I send you the strace / sysdig output per private mail, as it contains
private information of that user.

Ralf

-- 
Ralf Becker
EGroupware GmbH [www.egroupware.org]
Handelsregister HRB Kaiserslautern 3587
Gesch?ftsf?hrer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 631 31657-0


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20170918/cb6ffef4/attachment.sig>

On 18 Sep 2017, at 20.12, Ralf Becker <rb at egroupware.org <mailto:rb at
egroupware.org>> wrote:
> 
> Hi Timo,
> 
> Am 18.09.17 um 12:03 schrieb Timo Sirainen:
>> On 18 Sep 2017, at 12.10, Ralf Becker <rb at egroupware.org
<mailto:rb at egroupware.org>
>> <mailto:rb at egroupware.org <mailto:rb at
egroupware.org>>> wrote:
>>> 
>>> Am 14.09.17 um 01:07 schrieb Timo Sirainen:
>>>> On 7 Sep 2017, at 17.42, Ralf Becker <rb at egroupware.org
<mailto:rb at egroupware.org>
>>>> <mailto:rb at egroupware.org <mailto:rb at
egroupware.org>>> wrote:
>>>>> Dovecot 2.2.31 with mailboxes in mdbox format.
>>>>> 
>>>>> Since a couple of days some mailboxes have the problem,
that sieve
>>>>> rules
>>>>> moving mails to folders stop working and .sieve.log in
mailbox shows:
>>>>> 
>>>>> sieve: info: started log at Sep 07 13:57:17.
>>>>> error:
>>>>> msgid=<20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz
at somedomain.egroupware.de
<mailto:20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de>
>>>>>
<mailto:20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de
<mailto:20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de>>>:
>>>>> failed to store into mailbox 'INBOX/Munser':
Mailbox doesn't exist:
>>>>> INBOX/Munser.
>>>>> 
>>>>> When I do a doveadm mailbox list -s -u
<user>@<domain> I get all
>>>>> folders
>>>>> incl. the one mentioned above, while doveadm mailbox list
without -s
>>>>> shows just
>>>>> user
>>>>> INBOX
>>>> Subscriptions are stored independently from the actual folders.
So
>>>> it looks like the subscription file exists and is correct, but
>>>> somehow you've lost all the folders. Do you see the folders
in the
>>>> filesystem under user/mailboxes/ directory? 
>>> 
>>> Yes, the folders exist under
>>> /var/dovecot/imap/<domain>/<user>/mdbox/mailboxes/.
>>> Just doveadm mailbox list -u <user>@<domain> (without
-s) does only show
>>> INBOX and user.
>>> (I can send you the list of folders via private mail, but I can not
post
>>> them on the list.)
>>> 
>>> Anything I can do to get Dovecot to eg. rescan the folders from the
>>> filesystem or any other way to fix that problem?
>>> I have it with a couple of mailboxes, so I believe it's some
kind of
>>> systematic problem, nothing the users did.
>> 
>> I can't really think of any reason why it wouldn't simply work.
>> Especially since you're not using v2.2.32, the folder listing is
>> always performed by listing the directories in filesystem, so
there's
>> nothing really to resync. What's your doveconf -n? You could try
with
>> mailbox_list_index=no if that happens to make any difference, but it
>> shouldn't.
>> 
>> You could also try what "strace -o log -s 100 doveadm mailbox list
-u
>> user at domain" shows. Is it opening the correct mailboxes/
directory?
>> Maybe the path is just wrong for some reason (some typo added
somewhere)?
> 
> 
> Nope it lstats the correct directories, but does not show them.
> 
> I send you the strace / sysdig output per private mail, as it contains
> private information of that user.
Looks like you have some dovecot-acl and dovecot-acl-list files, so it has to be
because Dovecot thinks the ACLs are preventing access to the user. Try deleting
dovecot-acl-list to see if the problem is with that. If not, look at the
dovecot-acl files and/or "doveadm acl debug -u user at domain <folder
that's supposed to exist>" to figure out what's wrong.

Hi Timo,

update to 2.2.32 (suggested by Aki) did not change the situation ...

Am 18.09.17 um 20:49 schrieb Timo Sirainen:
> On 18 Sep 2017, at 20.12, Ralf Becker <rb at egroupware.org
> <mailto:rb at egroupware.org>> wrote:
>>
>> Hi Timo,
>>
>> Am 18.09.17 um 12:03 schrieb Timo Sirainen:
>>> On 18 Sep 2017, at 12.10, Ralf Becker <rb at egroupware.org
>>> <mailto:rb at egroupware.org>
>>> <mailto:rb at egroupware.org>> wrote:
>>>>
>>>> Am 14.09.17 um 01:07 schrieb Timo Sirainen:
>>>>> On 7 Sep 2017, at 17.42, Ralf Becker <rb at
egroupware.org
>>>>> <mailto:rb at egroupware.org>
>>>>> <mailto:rb at egroupware.org>> wrote:
>>>>>> Dovecot 2.2.31 with mailboxes in mdbox format.
>>>>>>
>>>>>> Since a couple of days some mailboxes have the problem,
that sieve
>>>>>> rules
>>>>>> moving mails to folders stop working and .sieve.log in
mailbox shows:
>>>>>>
>>>>>> sieve: info: started log at Sep 07 13:57:17.
>>>>>> error:
>>>>>>
msgid=<20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de
>>>>>>
<mailto:20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de>
>>>>>>
<mailto:20170907155704.EGroupware.S4yThVJRr12WSiJLpKbK0Bz at
somedomain.egroupware.de>>:
>>>>>> failed to store into mailbox 'INBOX/Munser':
Mailbox doesn't exist:
>>>>>> INBOX/Munser.
>>>>>>
>>>>>> When I do a doveadm mailbox list -s -u
<user>@<domain> I get all
>>>>>> folders
>>>>>> incl. the one mentioned above, while doveadm mailbox
list without -s
>>>>>> shows just
>>>>>> user
>>>>>> INBOX
>>>>> Subscriptions are stored independently from the actual
folders. So
>>>>> it looks like the subscription file exists and is correct,
but
>>>>> somehow you've lost all the folders. Do you see the
folders in the
>>>>> filesystem under user/mailboxes/ directory??
>>>>
>>>> Yes, the folders exist under
>>>> /var/dovecot/imap/<domain>/<user>/mdbox/mailboxes/.
>>>> Just doveadm mailbox list -u <user>@<domain>
(without -s) does only
>>>> show
>>>> INBOX and user.
>>>> (I can send you the list of folders via private mail, but I can
not
>>>> post
>>>> them on the list.)
>>>>
>>>> Anything I can do to get Dovecot to eg. rescan the folders from
the
>>>> filesystem or any other way to fix that problem?
>>>> I have it with a couple of mailboxes, so I believe it's
some kind of
>>>> systematic problem, nothing the users did.
>>>
>>> I can't really think of any reason why it wouldn't simply
work.
>>> Especially since you're not using v2.2.32, the folder listing
is
>>> always performed by listing the directories in filesystem, so
there's
>>> nothing really to resync. What's your doveconf -n? You could
try with
>>> mailbox_list_index=no if that happens to make any difference, but
it
>>> shouldn't.
>>>
>>> You could also try what "strace -o log -s 100 doveadm mailbox
list -u
>>> user at domain" shows. Is it opening the correct mailboxes/
directory?
>>> Maybe the path is just wrong for some reason (some typo added
>>> somewhere)?
>>
>>
>> Nope it lstats the correct directories, but does not show them.
>>
>> I send you the strace / sysdig output per private mail, as it contains
>> private information of that user.
>
> Looks like you have some dovecot-acl and dovecot-acl-list files, so it
> has to be because Dovecot thinks the ACLs are preventing access to the
> user. Try deleting dovecot-acl-list to see if the problem is with
> that. If not, look at the dovecot-acl files and/or "doveadm acl debug
> -u user at domain <folder that's supposed to exist>" to
figure out what's
> wrong.

root at fra-nfs-mail:~# doveadm acl debug -u <user>@<domain>
INBOX/AA
doveadm(<user>@<domain>): Info: Mailbox 'AA' is in namespace
'INBOX/'
doveadm(<user>@<domain>): Info: Mailbox path:
/var/dovecot/imap/<domain>/<user>/mdbox/mailboxes/AA/dbox-Mails
doveadm(<user>@<domain>): Info: All message flags are shared across
users in mailbox
doveadm(<user>@<domain>): Info: User <user>@<domain> has
no rights for
mailbox
doveadm(<user>@<domain>): Error: User <user>@<domain> is
missing
'lookup' right
doveadm(<user>@<domain>): Info: Mailbox INBOX/AA is NOT visible in
LIST

Ok, but when I try to fix it:

root at fra-nfs-mail:~# doveadm acl add -u <user>@<domain> INBOX/AA
user=<user>@<domain> admin create delete expunge insert lookup post
read
write write-deleted write-seen

root at fra-nfs-mail:~# doveadm acl add -u <user>@<domain> INBOX/AA
owner
admin create delete expunge insert lookup post read write write-deleted
write-seen

root at fra-nfs-mail:~# doveadm acl get -u <user>@<domain> INBOX/AA
ID????????????????????????????????????????? Global Rights
owner????????????????????????????????????????????? admin create delete
expunge insert lookup post read write write-deleted write-seen
user=<user>@<domain>??????? admin create delete expunge insert
lookup
post read write write-deleted write-seen

Nothing has changed :(

root at fra-nfs-mail:~# doveadm acl debug -u <user>@<domain>
INBOX/AA
doveadm(<user>@<domain>): Info: Mailbox 'AA' is in namespace
'INBOX/'
doveadm(<user>@<domain>): Info: Mailbox path:
/var/dovecot/imap/<domain>/<user>/mdbox/mailboxes/AA/dbox-Mails
doveadm(<user>@<domain>): Info: All message flags are shared across
users in mailbox
doveadm(<user>@<domain>): Info: User <user>@<domain> has
no rights for
mailbox
doveadm(<user>@<domain>): Error: User <user>@<domain> is
missing
'lookup' right
doveadm(<user>@<domain>): Info: Mailbox INBOX/AA is NOT visible in
LIST

How to fix that situation?

Is there a way to reset acl of all folders of a user to all rights for
the owner?

root at fra-nfs-mail:/var/dovecot/imap/<domain>/<user># find -name
"dovecot-acl*"
./mdbox/mailboxes/INBOX/dbox-Mails/dovecot-acl
./mdbox/mailboxes/AA/dbox-Mails/dovecot-acl
./mdbox/dovecot-acl-list

Ralf

-- 
Ralf Becker
EGroupware GmbH [www.egroupware.org]
Handelsregister HRB Kaiserslautern 3587
Gesch?ftsf?hrer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 631 31657-0


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20170919/ed534399/attachment.sig>