dovecot - Nov 2015 - Problems Converting from Cyrus to Dovecot (cyrus2dovecot)

Hi,

No, I tried fetching over imapc too exactly as you  suggested.
In my case it was not from cyrus, but from CriticalPath.
isync was finally able to do the job, preserving flags and doing UIDs 
mapping. The most boring part was generating proper config file for 
thousands of accounts.
A working imapc/dsync would have been better.

Emmanuel.

Le 26/11/2015 15:24, Sami Ketola a ?crit :
> Hi,
>
> I think you tried to read cyrus mails folders directly. I was talking about
fething mails from cyrus over imapc connection.
>
> Sami
>
>> On 26 Nov 2015, at 15:36, FUSTE Emmanuel <emmanuel.fuste at
thalesgroup.com> wrote:
>>
>> Hello,
>>
>> Because it did not work ?
>> In a similar situation, we where forced to use isync/mbsync in imap to
>> imap mode because dsync did not work.
>> It was reported here more than a year ago (May 2014).
>> Time to time, I see the same report from other trying to use dsync to
do
>> a migration to dovecot.
>> Dsync is a very appealing and elegant solution to this usage, but it
>> does not always  work in real world.
>>
>> Regards,
>> Emmanuel
>>
>> Le 26/11/2015 12:30, Sami Ketola a ?crit :
>>> Hi,
>>>
>>> With imapsync you will lose message UIDs which means that IMAP
clients need to clear their local caches and redownload all messages. Why not
use dovecot dsync over imapc instead? It tries to preserve UIDs and Flags.
>>>
>>> http://wiki2.dovecot.org/Migration
>>>
>>> Sami
>>>
>>>
>>>> On 07 Nov 2015, at 23:35, Forrest <those.lists at
gmail.com> wrote:
>>>>
>>>> Thank you for the reply.  I did find imapsync whilst perusing
Google.  I will give it a shot, it sounds more realistic/reliable. I have a
hoard of emails going back to 1999, so I want as few errors as possible :)
>>>>
>>>>
>>>>
>>>> On 11/7/15 3:31 PM, Philon wrote:
>>>>> Hi there,
>>>>>
>>>>> I was in the same position, but for mutliple accounts.
Still you might want to look at imapsync (https://github.com/imapsync/imapsync),
isync and offlineimap. There are more alternatives listed at the imapsync
homepage.
>>>>>
>>>>>
>>>>> Philon
>>>>>
>>>>>
>>>>>> Am 04.11.2015 um 20:47 schrieb Forrest <those.lists
at gmail.com>:
>>>>>>
>>>>>> I have been attempting to use the cyrus2dovecot script,
to no avail.
>>>>>>
>>>>>> I have many years of content that I want to convert
from Cyrus to Dovecot; with the above not working, what are other options out
there?  Another idea I had is simply set up another IMAP server (using Dovecot)
and drag-and-drop and just wait, which I may end up doing.
>>>>>>
>>>>>> In the above, I copied over my entire /var/imap and
/var/spool/imap to another system; there is only one account (mine), so calling
the script was fairly easy; it just doesn't work.
>>>>>>
>>>>>>
>>>>>> inboxes=the "myaccount" that was copied over
>>>>>>
>>>>>> /home/myaccount/cyrus2dovecot --cyrus-inbox
/home/myaccount/inboxes/%u     \
>>>>>>                   --cyrus-seen
/home/myaccount/varimap/user/%h/%u.seen    \
>>>>>>                   --cyrus-sub
/home/varimap/user/%h/%u.sub      \
>>>>>>                   --dovecot-inbox
/home/myaccount/dovecot/Maildir \
>>>>>>                   myaccount
>>>>>>
>>>>>>
>>>>>>
>>>>>> the log output complains of:
>>>>>>
>>>>>>     cyrus2dovecot [myaccount]: (warning) Index record
missing for: INBOX/62020.
>>>>>>
>>>>>> and correctly complains about squat indices, as
that's not a file it would handle.  There is no output into the Maildir,
however.
>>>>>>
>>>>>> All directory paths are correct.
>>>>>>
>>>>>>
>>>>>> Thanks.

v2.2.19 has many fixes related to dsync+imapc, which were found while migrating
several million users from GMail. I'm not aware of any problems with it now.
Also even before v2.2.19 dsync+imapc has been used to successfully do many large
migrations.
> On 26 Nov 2015, at 17:49, FUSTE Emmanuel <emmanuel.fuste at
thalesgroup.com> wrote:
> 
> Hi,
> 
> No, I tried fetching over imapc too exactly as you  suggested.
> In my case it was not from cyrus, but from CriticalPath.
> isync was finally able to do the job, preserving flags and doing UIDs 
> mapping. The most boring part was generating proper config file for 
> thousands of accounts.
> A working imapc/dsync would have been better.
> 
> Emmanuel.
> 
> Le 26/11/2015 15:24, Sami Ketola a ?crit :
>> Hi,
>> 
>> I think you tried to read cyrus mails folders directly. I was talking
about fething mails from cyrus over imapc connection.
>> 
>> Sami
>> 
>>> On 26 Nov 2015, at 15:36, FUSTE Emmanuel <emmanuel.fuste at
thalesgroup.com> wrote:
>>> 
>>> Hello,
>>> 
>>> Because it did not work ?
>>> In a similar situation, we where forced to use isync/mbsync in imap
to
>>> imap mode because dsync did not work.
>>> It was reported here more than a year ago (May 2014).
>>> Time to time, I see the same report from other trying to use dsync
to do
>>> a migration to dovecot.
>>> Dsync is a very appealing and elegant solution to this usage, but
it
>>> does not always  work in real world.
>>> 
>>> Regards,
>>> Emmanuel
>>> 
>>> Le 26/11/2015 12:30, Sami Ketola a ?crit :
>>>> Hi,
>>>> 
>>>> With imapsync you will lose message UIDs which means that IMAP
clients need to clear their local caches and redownload all messages. Why not
use dovecot dsync over imapc instead? It tries to preserve UIDs and Flags.
>>>> 
>>>> http://wiki2.dovecot.org/Migration
>>>> 
>>>> Sami
>>>> 
>>>> 
>>>>> On 07 Nov 2015, at 23:35, Forrest <those.lists at
gmail.com> wrote:
>>>>> 
>>>>> Thank you for the reply.  I did find imapsync whilst
perusing Google.  I will give it a shot, it sounds more realistic/reliable. I
have a hoard of emails going back to 1999, so I want as few errors as possible
:)
>>>>> 
>>>>> 
>>>>> 
>>>>> On 11/7/15 3:31 PM, Philon wrote:
>>>>>> Hi there,
>>>>>> 
>>>>>> I was in the same position, but for mutliple accounts.
Still you might want to look at imapsync (https://github.com/imapsync/imapsync),
isync and offlineimap. There are more alternatives listed at the imapsync
homepage.
>>>>>> 
>>>>>> 
>>>>>> Philon
>>>>>> 
>>>>>> 
>>>>>>> Am 04.11.2015 um 20:47 schrieb Forrest
<those.lists at gmail.com>:
>>>>>>> 
>>>>>>> I have been attempting to use the cyrus2dovecot
script, to no avail.
>>>>>>> 
>>>>>>> I have many years of content that I want to convert
from Cyrus to Dovecot; with the above not working, what are other options out
there?  Another idea I had is simply set up another IMAP server (using Dovecot)
and drag-and-drop and just wait, which I may end up doing.
>>>>>>> 
>>>>>>> In the above, I copied over my entire /var/imap and
/var/spool/imap to another system; there is only one account (mine), so calling
the script was fairly easy; it just doesn't work.
>>>>>>> 
>>>>>>> 
>>>>>>> inboxes=the "myaccount" that was copied
over
>>>>>>> 
>>>>>>> /home/myaccount/cyrus2dovecot --cyrus-inbox
/home/myaccount/inboxes/%u     \
>>>>>>>                  --cyrus-seen
/home/myaccount/varimap/user/%h/%u.seen    \
>>>>>>>                  --cyrus-sub
/home/varimap/user/%h/%u.sub      \
>>>>>>>                  --dovecot-inbox
/home/myaccount/dovecot/Maildir \
>>>>>>>                  myaccount
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> the log output complains of:
>>>>>>> 
>>>>>>>    cyrus2dovecot [myaccount]: (warning) Index
record missing for: INBOX/62020.
>>>>>>> 
>>>>>>> and correctly complains about squat indices, as
that's not a file it would handle.  There is no output into the Maildir,
however.
>>>>>>> 
>>>>>>> All directory paths are correct.
>>>>>>> 
>>>>>>> 
>>>>>>> Thanks.

Hello Timo,

Yes, I follow closely the commit messages of the dovecot-cvs list and 
lots of things have moved in this area.
I will try and expect to be able to use dsync+imapc for our futures 
migrations.

Best regards,
Emmanuel.

Le 26/11/2015 21:37, Timo Sirainen a ?crit :
> v2.2.19 has many fixes related to dsync+imapc, which were found while
migrating several million users from GMail. I'm not aware of any problems
with it now. Also even before v2.2.19 dsync+imapc has been used to successfully
do many large migrations.
>
>> On 26 Nov 2015, at 17:49, FUSTE Emmanuel <emmanuel.fuste at
thalesgroup.com> wrote:
>>
>> Hi,
>>
>> No, I tried fetching over imapc too exactly as you  suggested.
>> In my case it was not from cyrus, but from CriticalPath.
>> isync was finally able to do the job, preserving flags and doing UIDs
>> mapping. The most boring part was generating proper config file for
>> thousands of accounts.
>> A working imapc/dsync would have been better.
>>
>> Emmanuel.
>>
>> Le 26/11/2015 15:24, Sami Ketola a ?crit :
>>> Hi,
>>>
>>> I think you tried to read cyrus mails folders directly. I was
talking about fething mails from cyrus over imapc connection.
>>>
>>> Sami
>>>
>>>> On 26 Nov 2015, at 15:36, FUSTE Emmanuel <emmanuel.fuste at
thalesgroup.com> wrote:
>>>>
>>>> Hello,
>>>>
>>>> Because it did not work ?
>>>> In a similar situation, we where forced to use isync/mbsync in
imap to
>>>> imap mode because dsync did not work.
>>>> It was reported here more than a year ago (May 2014).
>>>> Time to time, I see the same report from other trying to use
dsync to do
>>>> a migration to dovecot.
>>>> Dsync is a very appealing and elegant solution to this usage,
but it
>>>> does not always  work in real world.
>>>>
>>>> Regards,
>>>> Emmanuel
>>>>
>>>> Le 26/11/2015 12:30, Sami Ketola a ?crit :
>>>>> Hi,
>>>>>
>>>>> With imapsync you will lose message UIDs which means that
IMAP clients need to clear their local caches and redownload all messages. Why
not use dovecot dsync over imapc instead? It tries to preserve UIDs and Flags.
>>>>>
>>>>> http://wiki2.dovecot.org/Migration
>>>>>
>>>>> Sami
>>>>>
>>>>>
>>>>>> On 07 Nov 2015, at 23:35, Forrest <those.lists at
gmail.com> wrote:
>>>>>>
>>>>>> Thank you for the reply.  I did find imapsync whilst
perusing Google.  I will give it a shot, it sounds more realistic/reliable. I
have a hoard of emails going back to 1999, so I want as few errors as possible
:)
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 11/7/15 3:31 PM, Philon wrote:
>>>>>>> Hi there,
>>>>>>>
>>>>>>> I was in the same position, but for mutliple
accounts. Still you might want to look at imapsync
(https://github.com/imapsync/imapsync), isync and offlineimap. There are more
alternatives listed at the imapsync homepage.
>>>>>>>
>>>>>>>
>>>>>>> Philon
>>>>>>>
>>>>>>>
>>>>>>>> Am 04.11.2015 um 20:47 schrieb Forrest
<those.lists at gmail.com>:
>>>>>>>>
>>>>>>>> I have been attempting to use the cyrus2dovecot
script, to no avail.
>>>>>>>>
>>>>>>>> I have many years of content that I want to
convert from Cyrus to Dovecot; with the above not working, what are other
options out there?  Another idea I had is simply set up another IMAP server
(using Dovecot) and drag-and-drop and just wait, which I may end up doing.
>>>>>>>>
>>>>>>>> In the above, I copied over my entire /var/imap
and /var/spool/imap to another system; there is only one account (mine), so
calling the script was fairly easy; it just doesn't work.
>>>>>>>>
>>>>>>>>
>>>>>>>> inboxes=the "myaccount" that was
copied over
>>>>>>>>
>>>>>>>> /home/myaccount/cyrus2dovecot --cyrus-inbox
/home/myaccount/inboxes/%u     \
>>>>>>>>                   --cyrus-seen
/home/myaccount/varimap/user/%h/%u.seen    \
>>>>>>>>                   --cyrus-sub
/home/varimap/user/%h/%u.sub      \
>>>>>>>>                   --dovecot-inbox
/home/myaccount/dovecot/Maildir \
>>>>>>>>                   myaccount
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> the log output complains of:
>>>>>>>>
>>>>>>>>     cyrus2dovecot [myaccount]: (warning) Index
record missing for: INBOX/62020.
>>>>>>>>
>>>>>>>> and correctly complains about squat indices, as
that's not a file it would handle.  There is no output into the Maildir,
however.
>>>>>>>>
>>>>>>>> All directory paths are correct.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks.