dovecot - Jan 2016 - Mapping of INBOX.INBOX breaks imapc_list_prefix=INBOX?

We're running a huge imapc-Migration from an old Courier-IMAP to Dovecot.

Courier always uses "INBOX."-prefix for his folders, so we set

namespace inbox {
[...]
  separator = .
  prefix = INBOX.
[...]
}

on our new Dovecot 2.2.20 system.

And also we added:

imapc_list_prefix=INBOX

to avoid the dublication of the INBOX prefix.

At the end, doveadm/dsync/imapc see "INBOX.Virus", adds his Prefix,
has
"INBOX.INBOX.Virus", deletes his imapc_list_prefix and the end we have
"INBOX.Virus" again which is identical to the source folder from
Courier.

Works perfect BUT:

It looks like there is a "hidden feature" in Dovecot that always maps
"INBOX.INBOX" back to "INBOX". For me that looks like a
workaround for
broken mailclients to fix some of the common problems after changing the
namespace prefix.

I never heard about that "hidden feature", but even if I don't (!)
have
INBOX.INBOX:

root at imap01:~/MIGRATION # doveadm mailbox list -u test
INBOX
INBOX.Virus
INBOX.Spam
INBOX.Trash

I'm always able to access INBOX.INBOX which is identical to INBOX:

root at imap01:~/MIGRATION # doveadm fetch -u test uid mailbox INBOX.INBOX
uid: 1

Looks like there is such a magic-mapping and looks like this "hard coded
mapping" (?) breaks the imapc_list_prefix-magic.

If the user has on the SOURCE-server (Courier):

Source-Server:
 INBOX (5 messages)
 INBOX.Inbox (3 messages)

and I do migrate that user using doveadm/dsync/imapc, this user has on
the DESTINATION-server (Dovecot)

Destination-server:
 INBOX (5 messages)
 INBOX.Inbox (5 messages from "real INBOX")

and his 3 messages from INBOX.Inbox are lost.

I can NOT find any errors in the debug/error-output of
doveadm/dsync/imapc. From the view of doveadm everything worked perfect
as you can see in the attached logfile.

Or am I completly wrong?!

Peer




-- 
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-42
Fax: 030 / 405051-19

Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht
Berlin-Charlottenburg,
Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin

-------------- next part --------------
A non-text attachment was scrubbed...
Name: p246098p2.log
Type: text/x-log
Size: 15894 bytes
Desc: not available
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20160106/368827a3/attachment-0001.bin>

On 06 Jan 2016, at 11:47, Peer Heinlein <p.heinlein at
heinlein-support.de> wrote:
> 
> 
> We're running a huge imapc-Migration from an old Courier-IMAP to
Dovecot.
> 
> Courier always uses "INBOX."-prefix for his folders, so we set
> 
> namespace inbox {
> [...]
>  separator = .
>  prefix = INBOX.
> [...]
> }
> 
> on our new Dovecot 2.2.20 system.
> 
> And also we added:
> 
> imapc_list_prefix=INBOX
> 
> to avoid the dublication of the INBOX prefix.
> 
> At the end, doveadm/dsync/imapc see "INBOX.Virus", adds his
Prefix, has
> "INBOX.INBOX.Virus", deletes his imapc_list_prefix and the end we
have
> "INBOX.Virus" again which is identical to the source folder from
Courier.
> 
> Works perfect BUT:
> 
> It looks like there is a "hidden feature" in Dovecot that always
maps
> "INBOX.INBOX" back to "INBOX". For me that looks like a
workaround for
> broken mailclients to fix some of the common problems after changing the
> namespace prefix.
> 
> I never heard about that "hidden feature", but even if I
don't (!) have
> INBOX.INBOX:
> 
> root at imap01:~/MIGRATION # doveadm mailbox list -u test
> INBOX
> INBOX.Virus
> INBOX.Spam
> INBOX.Trash
> 
> I'm always able to access INBOX.INBOX which is identical to INBOX:
> 
> root at imap01:~/MIGRATION # doveadm fetch -u test uid mailbox INBOX.INBOX
> uid: 1
> 
> Looks like there is such a magic-mapping and looks like this "hard
coded
> mapping" (?) breaks the imapc_list_prefix-magic.
> 
> If the user has on the SOURCE-server (Courier):
> 
> Source-Server:
> INBOX (5 messages)
> INBOX.Inbox (3 messages)
> 
> and I do migrate that user using doveadm/dsync/imapc, this user has on
> the DESTINATION-server (Dovecot)
> 
> Destination-server:
> INBOX (5 messages)
> INBOX.Inbox (5 messages from "real INBOX")
> 
> and his 3 messages from INBOX.Inbox are lost.
> 
> I can NOT find any errors in the debug/error-output of
> doveadm/dsync/imapc. From the view of doveadm everything worked perfect
> as you can see in the attached logfile.
> 
> Or am I completly wrong?!
It's not really a "hidden feature", but more likely a consequence
of the INBOX. prefix being added/removed from folder names. I'd need to look
into it more carefully to see if it's possible to fix easily or not, but I
have a feeling it may not be easy. INBOX.INBOX has caused troubles many times
earlier.. One solution that should work is that for dsync runs you'd simply
remove the INBOX. prefix from both the namespace and the imapc_list_prefix.

> On 06 Jan 2016, at 15:35, Timo Sirainen <tss at iki.fi> wrote:
> 
> On 06 Jan 2016, at 11:47, Peer Heinlein <p.heinlein at
heinlein-support.de> wrote:
>> 
>> 
>> We're running a huge imapc-Migration from an old Courier-IMAP to
Dovecot.
>> 
>> Courier always uses "INBOX."-prefix for his folders, so we
set
>> 
>> namespace inbox {
>> [...]
>> separator = .
>> prefix = INBOX.
>> [...]
>> }
>> 
>> on our new Dovecot 2.2.20 system.
>> 
>> And also we added:
>> 
>> imapc_list_prefix=INBOX
>> 
>> to avoid the dublication of the INBOX prefix.
>> 
>> At the end, doveadm/dsync/imapc see "INBOX.Virus", adds his
Prefix, has
>> "INBOX.INBOX.Virus", deletes his imapc_list_prefix and the
end we have
>> "INBOX.Virus" again which is identical to the source folder
from Courier.
>> 
>> Works perfect BUT:
>> 
>> It looks like there is a "hidden feature" in Dovecot that
always maps
>> "INBOX.INBOX" back to "INBOX". For me that looks
like a workaround for
>> broken mailclients to fix some of the common problems after changing
the
>> namespace prefix.
>> 
>> I never heard about that "hidden feature", but even if I
don't (!) have
>> INBOX.INBOX:
>> 
>> root at imap01:~/MIGRATION # doveadm mailbox list -u test
>> INBOX
>> INBOX.Virus
>> INBOX.Spam
>> INBOX.Trash
>> 
>> I'm always able to access INBOX.INBOX which is identical to INBOX:
>> 
>> root at imap01:~/MIGRATION # doveadm fetch -u test uid mailbox
INBOX.INBOX
>> uid: 1
>> 
>> Looks like there is such a magic-mapping and looks like this "hard
coded
>> mapping" (?) breaks the imapc_list_prefix-magic.
>> 
>> If the user has on the SOURCE-server (Courier):
>> 
>> Source-Server:
>> INBOX (5 messages)
>> INBOX.Inbox (3 messages)
>> 
>> and I do migrate that user using doveadm/dsync/imapc, this user has on
>> the DESTINATION-server (Dovecot)
>> 
>> Destination-server:
>> INBOX (5 messages)
>> INBOX.Inbox (5 messages from "real INBOX")
>> 
>> and his 3 messages from INBOX.Inbox are lost.
>> 
>> I can NOT find any errors in the debug/error-output of
>> doveadm/dsync/imapc. From the view of doveadm everything worked perfect
>> as you can see in the attached logfile.
>> 
>> Or am I completly wrong?!
> 
> It's not really a "hidden feature", but more likely a
consequence of the INBOX. prefix being added/removed from folder names. I'd
need to look into it more carefully to see if it's possible to fix easily or
not, but I have a feeling it may not be easy. INBOX.INBOX has caused troubles
many times earlier.. One solution that should work is that for dsync runs
you'd simply remove the INBOX. prefix from both the namespace and the
imapc_list_prefix.
Or actually that would just add INBOX. prefix to everything. Maybe removing
either namespace prefix or imapc_list_prefix would work?

Am 06.01.2016 um 21:35 schrieb Timo Sirainen:

> It's not really a "hidden feature", but more likely a
consequence of the INBOX. prefix being added/removed from folder names.
I called this a "hidden feature" because for me that's a
non-standard
behavior without any options to switch that on or off.

Maybe a simple "double_inbox_mapping=yes|no" would solve the problem
--
if this feature is off during migration there's no risk for that mapping
mistake.

Peer




-- 
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-42
Fax: 030 / 405051-19

Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht
Berlin-Charlottenburg,
Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin