dovecot - May 2020 - doveadm sync backup from old to new server

Yeah, I tried that already - and it seems to crash more badly...

dsync-remote(use at abc.net)<mTEPAAUCvl4yFgAAb6OASA>: Panic:
epoll_ctl(add, 0) failed: Operation not permitted (fd doesn't support epoll
- instead of '<file', try 'cat file|')
dsync-remote(user at abc.net)<mTEPAAUCvl4yFgAAb6OASA>: Error: Raw
backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x3d)
[0x7fa00465733d] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e)
[0x7fa00465745e] -> /usr/lib/dovecot/libdovecot.so.0(+0xeca6b)
[0x7fa004660a6b] -> /usr/lib/dovecot/libdovecot.so.0(+0xecaa1)
[0x7fa004660aa1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4ec1e)
[0x7fa0045c2c1e] -> /usr/lib/dovecot/libdovecot.so.0(+0x520b9)
[0x7fa0045c60b9] -> /usr/lib/dovecot/libdovecot.so.0(+0x102278)
[0x7fa004676278] -> /usr/lib/dovecot/libdovecot.so.0(io_add_istream_to+0x46)
[0x7fa004676336] -> doveadm(dsync_ibc_init_stream+0x216) [0x5564c8f81f06]
-> doveadm(+0x2f2a6) [0x5564c8f542a6] -> doveadm(+0x30b16)
[0x5564c8f55b16] -> doveadm(+0x3170a) [0x5564c8f5670a] ->
doveadm(doveadm_mail_try_run+0x21e) [0x5564c8f56ffe] -> doveadm(main+0x552)
[0x5564c8f471b2] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb)
[0x7fa0041fb09b] -> doveadm(_start+0x2a) [0x5564c8f472fa]
dsync-local(user at abc.net)<Gc7DAf0Bvl6yFwAAwCTZMw>: Error: read(remote)
failed: EOF (version not received)
dsync-local(user at abc.net)<Gc7DAf0Bvl6yFwAAwCTZMw>: Error: Remote
command returned error 134: ssh root at po.sloop.net docker exec b3093c6200a9
doveadm dsync-server -u user at abc.net dsync-server

I found several other reports that seem similar, some on Ubuntu 18.04 which
match the new platform I'm using.

And I couldn't even find details in the manpage for dsync-server - so I was
totally lost - even worse than I was before. [Blind AND LOST, instead of just
lost.]

doveadm sync *is* documented, so I figured it was "safer" to use.

Sigh.
IMAP sync looks better and better. I'm a little puzzled why I don't just
give up walking on the broken glass and use it - but it just bugs the heck out
of me.

This whole process makes me feel like a total moron. I know it's not true -
but the docs, and such, around doveadm sync are so obtuse and it's so
non-obvious what's wrong, or what one might do to right things - well, it is
hard not to feel like you ought to know, but just can't crack the davinci
code or something...

Thanks again!

Hopefully someone Aki/Timo et al. will pipe up with something helpful soon.
Otherwise, I guess doveadm sync/backup will continue to be unused by any mere
mortals.

-Greg

P> I'm not clear myself on the difference myself between 
P> dsync
P> doveadm sync
P> doveadm dsync-server
P> ... perhaps someone here can explain? However what worked for me
P> was using doveadm dsync-server on the 'receive' end. Maybe that
P> puts it in 'listen' mode? So maybe try
P> doveadm backup -u user at abc.net \
P>   ssh root at po.abc.net \
P>   docker exec b3093cxxxxxx doveadm dsync-server -u user at abc.net

P> I've tried to get along with docker before, but it always just
P> seems to add another level of complexity into everything, so thus
P> far I've managed to avoid it apart from general experiments! 

P> P.

P> On 15/05/2020 07.00, Gregory Sloop wrote:
>> So that was really helpful for me to understand that a lot more
clearly.
>> Thanks! [Many, many thanks @Plutocrat!!]
>> But I'm still getting a similar failure.
>> Let me give the command I'm using.
>> doveadm backup -D -u user at abc.net \
>> ssh root at po.abc.net \
>> docker exec b3093cxxxxxx doveadm sync -D -u user at abc.net
>> The "local" server is where the data/mail currently is.
>> The remote or docker container/volume is where dovecot is installed.
(The data is in a different docker container.)
>> So, I think I'm "backing up" the data from the local
machine and pushing that "backed up" data via SSH to the dovecot
install in docker and attempting to "sync" that data to the remote
dovecot install.
>> However, I immediately get
>> "Error: read(remote) failed: EOF (version not received)"
>> The connection hangs. If I wait a while - perhaps 30-60s, it kills the
SSH connection and aborts the process.
>> Is there a way I can test the remote end's ability to accept the
data.
>> [i.e. Can I do something like
>> ssh root at po.abc.net \
>> docker exec b3093cxxxxxx doveadm sync -D -u user at abc.net
>> And see if it would accept the data. [In short, do I have a local end
problem or a remote end problem - and being able to test both parts individually
would probably help me figure out what's broken. That still won't fix
it, but at least I'll know which end I need to concentrate on.]
>> I'd guess this will all seem pretty obvious in retrospect, but for
the life of me, I'm completely lost and really don't have any idea where
to start to break it down better so I can see how each piece is working or not.
>> TIA
>> -Greg
>> *P> I struggled with this at one time, and with the lack of examples
>> P> around the internet. In fact, I might have written this very post
>> P> myself at one point! In the end, the penny dropped when I saw
that the format was
>> P> (command on local server) + (stuff you need to connect over ssh)
+ (command on remote server)
>> P> So what worked for me was:
>> P> doveadm backup -u *user at domain.com <mailto:user at
domain.com>* \
>> P>    ssh -p 2222 (+ other ssh options) *root at remoteserver.com
<mailto:root at remoteserver.com>* \
>> P>    doveadm dsync-server -u *user2 at domain2.com <mailto:user2
at domain2.com>*
>> P> If you get
>> P> dsync-remote(*user2 at domain2.com <mailto:user2 at
domain2.com>*): Error: Mailbox INBOX sync:
>> P> mailbox_delete failed: INBOX can't be deleted
>> P> ... you'll need to clear the remote directory first! Or maybe
try sync instead of backup?
>> P> I found that running as root on the remote server was necessary
in
>> P> my case, due to the permissions on the remote directory. You
might
>> P> want to check permissions on remote directory are writable by the
user in your ssh command.
>> P> If still no joy you can run
>> P>         doveadm user *user at domain.com <mailto:user at
domain.com>* on your local server and
>> P>         doveadm user *user2 at domain2.com <mailto:user2 at
domain2.com>* on your remote server
>> P> and dovecot will tell you where its synching from and to.
>> P> Finally, this solution is a 'push' from source server to
target.
>> P> You may 'pull' the other way if that makes more sense in
your environment with -R
>> P> Those pieces were enough for me to get it to work ...
>> P> P.
>> P> PS. For the record, on the original job I was under time
pressure,
>> P> and yes, I did use imapsync to get it done in the end. I got this
>> P> to work later on when I had more time to tinker.
>> P> On 14/05/2020 09.09, Gregory Sloop wrote:
>>>> So I've done quite a lot of searching on the list and on
the web - and perhaps my google-fu is really bad - but I can't find any real
recipes on how to sync mail from the old server to the new.
>>>> As an FYI - the old server is a CPanel/WHM setup on a VPS.
>>>> The new is mailcow - which uses docker.
>>>> However, I don't think either of these platforms is
what's causing the issue - but I'm certainly not sure of that.
>>>> ---
>>>> I've tried several things - but have lost track of all the
things I've tried.
>>>> This seemed like the best of all the things I've tried.
>>>> This particular mailbox/user+domain is setup on both servers.
>>>> doveadm backup -D -u *mc-user at abc.net <mailto:mc-user at
abc.net>* ssh *root at abc.net <mailto:root at abc.net>* -p2200 doveadm
dsync-server -u *cp-user at abc.net <mailto:cp-user at abc.net>
>>>> mc-user at abc.net <mailto:mc-user at abc.net>* is the
MC/NEW mailbox/domain
>> *>> cp-user at abc.net <mailto:cp-user at abc.net>* is the
CPanel/OLD user/domain account
>>>> The SSH server of the remote system is running on port 2200.
>>>> However when I try this, I get:
>>>> WARNING: The WATCHDOG_NOTIFY_EMAIL variable is not set.
Defaulting to a blank string.
>>>> dsync-local(*user at abc.net <mailto:user at
abc.net>*)<v44NFyRWvF5efgAAb6OASA>: Error: read(remote) failed: EOF
(version not received)
>>>> doveadm(*user at abc.net <mailto:user at abc.net>*):
Fatal: execvp(ssh) failed: No such file or directory
>>>> The process appears to hang, and a Ctrl+C stops it.
>>>> I'd love to get pointed at a reasonable recipe on how to
make this work.
>>>> I don't really get/understand the docs much at all.
>>>> [And either everyone else understands it just fine, and never
thinks to write a document on how to do it - or, and I think this is a lot more
likely - they're using something like imapsync to do it. I found numerous
places where others were, essentially, "Dovecot's tool is way too
complicated and I can't get it to work right, so I used imapsync." I
suppose I should probably just do that too, but it does seem a shame to do that
when the dovecot tool is almost certainly the best tool for the job, but I
can't figure out how to use it.]
>>>> If someone can help me grok what's going on, I'm glad
to write it up for the list and or a blog entry so it's more accessible.
>>>> TIA
>>>> -Greg
>> */--
>> Gregory Sloop, Principal: Sloop Network & Computer Consulting
>> Voice: 503.251.0452 x121
>> EMail: /gregs at sloop.net <mailto:gregs at sloop.net>
>> http://www.sloop.net
>> /--- /
-- 
Gregory Sloop, Principal: Sloop Network & Computer Consulting
Voice: 503.251.0452 x121
EMail: gregs at sloop.net
http://www.sloop.net
---
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20200514/9928a837/attachment-0001.html>

On 15/05/2020 10.56, Gregory Sloop wrote:
> IMAP sync looks better and better. I'm a little puzzled why I don't
just give up walking on the broken glass and use it - but it just bugs the heck
out of me.
If you can get the dovecot native sync to work, it seems to be a better option,
as it understands all the dovecot index formats etc, and will transfer those
correctly. Imapsync doesn't, and when I did that I had a few problems with
POP email clients re-downloading their entire Inbox, which produced a few
groans. Your mileage may vary.

Another option for you, if you have the time and energy would be to set up
proper replication between the two servers. Then any changes made would be
synced in real time, until the cutoff, when you just shut down the old server
and the new one is already up to date. But if you can't get a simple sync
happening then that might be a bit of over-reach.

My suspicions on your current situation would center around the complications
introduced by docker, and on file/container permissions, environment variables,
paths etc.

How about if you just try a simple doveadm command over ssh from the remote
server to make sure that works. eg. from "Sending" server do

ssh root at po.abc.net \
 docker exec b3093cxxxxxx doveadm user user at abc.net

That should execute the doveadm user command on the remote (receiving) server.
If that works, then at least you can satisfy yourself that the docker bit is
working correctly. Not sure why I'm getting involved in someone else's
troubleshooting nightmare! ;-)

P.

> On 15/05/2020 07:10 Plutocrat <plutocrat at gmail.com> wrote:
> 
>  
> On 15/05/2020 10.56, Gregory Sloop wrote:
> > IMAP sync looks better and better. I'm a little puzzled why I
don't just give up walking on the broken glass and use it - but it just bugs
the heck out of me.
> 
> If you can get the dovecot native sync to work, it seems to be a better
option, as it understands all the dovecot index formats etc, and will transfer
those correctly. Imapsync doesn't, and when I did that I had a few problems
with POP email clients re-downloading their entire Inbox, which produced a few
groans. Your mileage may vary.
> 
> Another option for you, if you have the time and energy would be to set up
proper replication between the two servers. Then any changes made would be
synced in real time, until the cutoff, when you just shut down the old server
and the new one is already up to date. But if you can't get a simple sync
happening then that might be a bit of over-reach.
> 
> My suspicions on your current situation would center around the
complications introduced by docker, and on file/container permissions,
environment variables, paths etc.
> 
> How about if you just try a simple doveadm command over ssh from the remote
server to make sure that works. eg. from "Sending" server do
> 
> ssh root at po.abc.net \
>  docker exec b3093cxxxxxx doveadm user user at abc.net
> 
> That should execute the doveadm user command on the remote (receiving)
server. If that works, then at least you can satisfy yourself that the docker
bit is working correctly. Not sure why I'm getting involved in someone
else's troubleshooting nightmare! ;-)
> 
> P.
You need to have ssh installed. If you want to do sync over ssh. The docker
image hasn't got ssh client installed by default.

Also see https://wiki.dovecot.org/Replication for more configuration options.

Aki

P> On 15/05/2020 10.56, Gregory Sloop wrote:
>> IMAP sync looks better and better. I'm a little puzzled why I
don't just give up walking on the broken glass and use it - but it just bugs
the heck out of me.
P> If you can get the dovecot native sync to work, it seems to be a
P> better option, as it understands all the dovecot index formats etc,
P> and will transfer those correctly. Imapsync doesn't, and when I did
P> that I had a few problems with POP email clients re-downloading
P> their entire Inbox, which produced a few groans. Your mileage may vary.

P> Another option for you, if you have the time and energy would be
P> to set up proper replication between the two servers. Then any
P> changes made would be synced in real time, until the cutoff, when
P> you just shut down the old server and the new one is already up to
P> date. But if you can't get a simple sync happening then that might be
a bit of over-reach.

P> My suspicions on your current situation would center around the
P> complications introduced by docker, and on file/container
P> permissions, environment variables, paths etc. 

P> How about if you just try a simple doveadm command over ssh from
P> the remote server to make sure that works. eg. from "Sending"
server do

P> ssh root at po.abc.net \
P>  docker exec b3093cxxxxxx doveadm user user at abc.net

P> That should execute the doveadm user command on the remote
P> (receiving) server. If that works, then at least you can satisfy
P> yourself that the docker bit is working correctly. Not sure why I'm
P> getting involved in someone else's troubleshooting nightmare! ;-) 

doveadm user does work.
i.e.
ssh root at abc.net docker exec b3093c6200a9 doveadm user user at abc.net

I get the uid/gid/home/maildir of that account, etc.

I do appreciate the help you've given!
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20200514/068ce7d0/attachment.html>