Yes, you are right. The problems are not of technical nature. The reason seems
to be some sort of fear (and "admins").
We have all we need. We have the old dovecot config, we have the mdbox files
and the complete directory structure. We are simply not allowed to do all the
stuff on the live system. Even if dsync backup exists that does not modify
anything. Currently it is planned that the real conversion will take place on a
weekend. We are setting up an environment that enables us to do that (how well
does it work, how long will it take). They are only around 500 users and ca 4TB
(incl public folders). I don't see a problem with that. And if they feel
some sort of fear, then I can understand it (except the "admins" who
should know better). We do what the customer wants us to do. And yes, they pay
pretty well for working on weekends.
What I would like to do is set up the real new machine with everything - the new
live system. Put all the mdbox files/structures into a directory and run the
conversion for each user. So the first question is if the mdbox files can
really be "simple files in folders" or if they have to be
"hosted" by an imap server, too. (As I understood, the destination
can be our final imap server.) And I believe we can have the source in the
filesystem (what would be mach faster). The other question is if we can run
some of these conversion in parallel? I would say "why not".
> Gesendet: Sonntag, 14. August 2022 um 18:24 Uhr
> Von: "Steve Litt" <slitt at troubleshooters.com>
> An: dovecot at dovecot.org
> Betreff: Re: convert mdbox to maildir
>
> On Sat, 2022-08-13 at 18:36 +0200, lutz.niederer at gmx.net wrote:
> >
> > The real problem is that we must not use the running, old dovecot
installation.?
> > So we are not able to connect to the old server, pull all folders and
mails and
> > create a new maildir structure.? Currently, we can't do anything
against it.? What
> > we get are the users' mdbox files.
>
> Why not? Is the old server broken beyond repair? If not, is there an actual
reason
> behind, or is it just a arbitrary decision capable of being swayed by
facts? Is the
> customer willing to pay for the large increase in time to rebuild the whole
thing?
>
> Will they at least let you rsync the old server's entire mdbox
structure to a
> machine where you can do your conversion? I don't know, to me their act
of giving
> you some files and saying "it's your problem now" seems
arbitrary, and you should
> charge them a lot of money.
>
>
> >
> > Is there any way to convert mdbox files and structures to maildir
directly from
> > filesystem?
> > Or do we have to build a copy of the old machine (dovecot only, or
-maybe better-
> > a vm) and then use doveadm backup?
> > Or is it ok to just set up the completely new installation, set
mail_location to
> > where the new Maildirs will be, like maildir:~/Maildir and then run
something like
> > doveadm backup mdbox:/tmp/$user/mdbox -u $user?? Will this transfer
all mails and
> > folders or do we have to keep an eye on some specific things?
>
> All I know about mdbox comes from this document:
> https://doc.dovecot.org/admin_manual/mailbox_formats/dbox/
>
> Quoting a specific sentence: "One of the main reasons for dbox?s high
performance is
> that it uses Dovecot?s index files as the only storage for message flags
and
> keywords, so the indexes don?t have to be ?synchronized?. Dovecot trusts
that
> they?re always up-to-date (unless it sees that something is clearly
broken). This
> also means that you must not lose the dbox index files, as they can?t be
regenerated
> without data loss."
>
> The quote says *dbox*, but it's in a section devoted to both dbox and
mdbox, so I'm
> thinking it might be true of both. Have they given you the index files? If
not, it
> sounds to me like any regeneration would be an approximation at best.
>
> Do you have a way of accurately putting together the directory structure of
the
> former mdbox system?
>
> My experience 10 years ago converting about a quarter million kmail emails
to
> Dovecot Maildir is it takes about an hour to transfer between 25,000 and
50,000
> emails, but of course that was on a much more anemic machine than I have
today. I'd
> guess that if you have both databases on the same machine, the way I did
ten years
> ago, the process will go pretty fast. Here's a count of my Dovecot
Maildir today:
>
> [root at mydesk Maildir]# du -hs
> 16G .
> [root at mydesk Maildir]# find . | wc -l
> 734906
> [root at mydesk Maildir]#
>
> I don't know much about your particular situation, but it seems to me
like the
> majority of your problem isn't technical.
>
> SteveT
>