So I've done quite a lot of searching on the list and on the web - and perhaps my google-fu is really bad - but I can't find any real recipes on how to sync mail from the old server to the new. As an FYI - the old server is a CPanel/WHM setup on a VPS. The new is mailcow - which uses docker. However, I don't think either of these platforms is what's causing the issue - but I'm certainly not sure of that. --- I've tried several things - but have lost track of all the things I've tried. This seemed like the best of all the things I've tried. This particular mailbox/user+domain is setup on both servers. doveadm backup -D -u mc-user at abc.net ssh root at abc.net -p2200 doveadm dsync-server -u cp-user at abc.net mc-user at abc.net is the MC/NEW mailbox/domain cp-user at abc.net is the CPanel/OLD user/domain account The SSH server of the remote system is running on port 2200. However when I try this, I get: WARNING: The WATCHDOG_NOTIFY_EMAIL variable is not set. Defaulting to a blank string. dsync-local(user at abc.net)<v44NFyRWvF5efgAAb6OASA>: Error: read(remote) failed: EOF (version not received) doveadm(user at abc.net): Fatal: execvp(ssh) failed: No such file or directory The process appears to hang, and a Ctrl+C stops it. I'd love to get pointed at a reasonable recipe on how to make this work. I don't really get/understand the docs much at all. [And either everyone else understands it just fine, and never thinks to write a document on how to do it - or, and I think this is a lot more likely - they're using something like imapsync to do it. I found numerous places where others were, essentially, "Dovecot's tool is way too complicated and I can't get it to work right, so I used imapsync." I suppose I should probably just do that too, but it does seem a shame to do that when the dovecot tool is almost certainly the best tool for the job, but I can't figure out how to use it.] If someone can help me grok what's going on, I'm glad to write it up for the list and or a blog entry so it's more accessible. TIA -Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20200513/bbd6a65d/attachment.html>
I struggled with this at one time, and with the lack of examples around the internet. In fact, I might have written this very post myself at one point! In the end, the penny dropped when I saw that the format was (command on local server) + (stuff you need to connect over ssh) + (command on remote server) So what worked for me was: doveadm backup -u user at domain.com \ ssh -p 2222 (+ other ssh options) root at remoteserver.com \ doveadm dsync-server -u user2 at domain2.com If you get dsync-remote(user2 at domain2.com): Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted ... you'll need to clear the remote directory first! Or maybe try sync instead of backup? I found that running as root on the remote server was necessary in my case, due to the permissions on the remote directory. You might want to check permissions on remote directory are writable by the user in your ssh command. If still no joy you can run doveadm user user at domain.com on your local server and doveadm user user2 at domain2.com on your remote server and dovecot will tell you where its synching from and to. Finally, this solution is a 'push' from source server to target. You may 'pull' the other way if that makes more sense in your environment with -R Those pieces were enough for me to get it to work ... P. PS. For the record, on the original job I was under time pressure, and yes, I did use imapsync to get it done in the end. I got this to work later on when I had more time to tinker. On 14/05/2020 09.09, Gregory Sloop wrote:> So I've done quite a lot of searching on the list and on the web - and perhaps my google-fu is really bad - but I can't find any real recipes on how to sync mail from the old server to the new. > > As an FYI - the old server is a CPanel/WHM setup on a VPS. > The new is mailcow - which uses docker. > > However, I don't think either of these platforms is what's causing the issue - but I'm certainly not sure of that. > > --- > I've tried several things - but have lost track of all the things I've tried. > This seemed like the best of all the things I've tried. > > This particular mailbox/user+domain is setup on both servers. > > doveadm backup -D -u mc-user at abc.net ssh root at abc.net -p2200 doveadm dsync-server -u cp-user at abc.net > > mc-user at abc.net is the MC/NEW mailbox/domain > cp-user at abc.net is the CPanel/OLD user/domain account > The SSH server of the remote system is running on port 2200. > > However when I try this, I get: > WARNING: The WATCHDOG_NOTIFY_EMAIL variable is not set. Defaulting to a blank string. > dsync-local(user at abc.net)<v44NFyRWvF5efgAAb6OASA>: Error: read(remote) failed: EOF (version not received) > doveadm(user at abc.net): Fatal: execvp(ssh) failed: No such file or directory > > The process appears to hang, and a Ctrl+C stops it. > > I'd love to get pointed at a reasonable recipe on how to make this work. > > I don't really get/understand the docs much at all. > [And either everyone else understands it just fine, and never thinks to write a document on how to do it - or, and I think this is a lot more likely - they're using something like imapsync to do it. I found numerous places where others were, essentially, "Dovecot's tool is way too complicated and I can't get it to work right, so I used imapsync." I suppose I should probably just do that too, but it does seem a shame to do that when the dovecot tool is almost certainly the best tool for the job, but I can't figure out how to use it.] > > If someone can help me grok what's going on, I'm glad to write it up for the list and or a blog entry so it's more accessible. > > TIA > -Greg
So specifically in your case: On 14/05/2020 09.09, Gregory Sloop wrote:> doveadm backup -D -u mc-user at abc.net ssh root at abc.net -p2200 doveadm dsync-server -u cp-user at abc.net > > mc-user at abc.net is the MC/NEW mailbox/domain > cp-user at abc.net is the CPanel/OLD user/domain account > The SSH server of the remote system is running on port 2200.If you're running this on your source server, I think you have the accounts the wrong way around. Try doveadm backup -u cp-user at abc.net \ ssh -p 2200 root at abc.net \ doveadm dsync-server -u mc-user at abc.net Or, if you're running it on your target server, maybe you'll have to pull, using the -R switch. P.
> On 14/05/2020 04:09 Gregory Sloop <gregs at sloop.net> wrote: > > > So I've done quite a lot of searching on the list and on the web - and perhaps my google-fu is really bad - but I can't find any real recipes on how to sync mail from the old server to the new. > > As an FYI - the old server is a CPanel/WHM setup on a VPS. > The new is mailcow - which uses docker. > > However, I don't think either of these platforms is what's causing the issue - but I'm certainly not sure of that. > > --- > I've tried several things - but have lost track of all the things I've tried. > This seemed like the best of all the things I've tried. > > This particular mailbox/user+domain is setup on both servers. > > doveadm backup -D -u mc-user at abc.net ssh root at abc.net -p2200 doveadm dsync-server -u cp-user at abc.net > > mc-user at abc.net is the MC/NEW mailbox/domain > cp-user at abc.net is the CPanel/OLD user/domain account > The SSH server of the remote system is running on port 2200. > > However when I try this, I get: > WARNING: The WATCHDOG_NOTIFY_EMAIL variable is not set. Defaulting to a blank string. > dsync-local(user at abc.net)<v44NFyRWvF5efgAAb6OASA>: Error: read(remote) failed: EOF (version not received) > doveadm(user at abc.net): Fatal: execvp(ssh) failed: No such file or directory > > The process appears to hang, and a Ctrl+C stops it. > > I'd love to get pointed at a reasonable recipe on how to make this work. > > I don't really get/understand the docs much at all. > [And either everyone else understands it just fine, and never thinks to write a document on how to do it - or, and I think this is a lot more likely - they're using something like imapsync to do it. I found numerous places where others were, essentially, "Dovecot's tool is way too complicated and I can't get it to work right, so I used imapsync." I suppose I should probably just do that too, but it does seem a shame to do that when the dovecot tool is almost certainly the best tool for the job, but I can't figure out how to use it.] > > If someone can help me grok what's going on, I'm glad to write it up for the list and or a blog entry so it's more accessible. > > TIA > -Greg >Just thought I'd point out that if you want to use ssh, it should be installed on both ends: "doveadm(user at abc.net): Fatal: execvp(ssh) failed: No such file or directory" If you don't want to, or can't install ssh, you can also use doveadm server: service doveadm { inet_listener { port = 12534 } } doveadm_port = 12534 doveadm_password = secretpass Then you can use tcp:otherhost:12534 as target/source. Aki
> On 14/05/2020 09:02 Adi Pircalabu <adi at ddns.com.au> wrote: > > > On 14-05-2020 15:16, Aki Tuomi wrote: > >> On 14/05/2020 04:09 Gregory Sloop <gregs at sloop.net> wrote: > >> > >> > >> So I've done quite a lot of searching on the list and on the web - and > >> perhaps my google-fu is really bad - but I can't find any real recipes > >> on how to sync mail from the old server to the new. > >> > >> As an FYI - the old server is a CPanel/WHM setup on a VPS. > >> The new is mailcow - which uses docker. > >> > >> However, I don't think either of these platforms is what's causing > >> the issue - but I'm certainly not sure of that. > >> > >> --- > >> I've tried several things - but have lost track of all the things > >> I've tried. > >> This seemed like the best of all the things I've tried. > >> > >> This particular mailbox/user+domain is setup on both servers. > >> > >> doveadm backup -D -u mc-user at abc.net ssh root at abc.net -p2200 doveadm > >> dsync-server -u cp-user at abc.net > >> > >> mc-user at abc.net is the MC/NEW mailbox/domain > >> cp-user at abc.net is the CPanel/OLD user/domain account > >> The SSH server of the remote system is running on port 2200. > >> > >> However when I try this, I get: > >> WARNING: The WATCHDOG_NOTIFY_EMAIL variable is not set. Defaulting to > >> a blank string. > >> dsync-local(user at abc.net)<v44NFyRWvF5efgAAb6OASA>: Error: > >> read(remote) failed: EOF (version not received) > >> doveadm(user at abc.net): Fatal: execvp(ssh) failed: No such file or > >> directory > >> > >> The process appears to hang, and a Ctrl+C stops it. > >> > >> I'd love to get pointed at a reasonable recipe on how to make this > >> work. > >> > >> I don't really get/understand the docs much at all. > >> [And either everyone else understands it just fine, and never thinks > >> to write a document on how to do it - or, and I think this is a lot > >> more likely - they're using something like imapsync to do it. I found > >> numerous places where others were, essentially, "Dovecot's tool is way > >> too complicated and I can't get it to work right, so I used imapsync." > >> I suppose I should probably just do that too, but it does seem a shame > >> to do that when the dovecot tool is almost certainly the best tool for > >> the job, but I can't figure out how to use it.] > >> > >> If someone can help me grok what's going on, I'm glad to write it up > >> for the list and or a blog entry so it's more accessible. > >> > >> TIA > >> -Greg > >> > > > > Just thought I'd point out that if you want to use ssh, it should be > > installed on both ends: > > > > "doveadm(user at abc.net): Fatal: execvp(ssh) failed: No such file or > > directory" > > > > If you don't want to, or can't install ssh, you can also use doveadm > > server: > > > > service doveadm { > > inet_listener { > > port = 12534 > > } > > } > > > > doveadm_port = 12534 > > doveadm_password = secretpass > > > > Then you can use tcp:otherhost:12534 as target/source. > > Hi Aki, > Does this service provide encryption in transit? > > -- > Adi PircalabuYou can use tcps if you enable SSL for doveadm listener. Add ssl=yes after port. Please keep replies on list. Aki
So that was really helpful for me to understand that a lot more clearly. Thanks! [Many, many thanks @Plutocrat!!] But I'm still getting a similar failure. Let me give the command I'm using. doveadm backup -D -u user at abc.net \ ssh root at po.abc.net \ docker exec b3093cxxxxxx doveadm sync -D -u user at abc.net The "local" server is where the data/mail currently is. The remote or docker container/volume is where dovecot is installed. (The data is in a different docker container.) So, I think I'm "backing up" the data from the local machine and pushing that "backed up" data via SSH to the dovecot install in docker and attempting to "sync" that data to the remote dovecot install. However, I immediately get "Error: read(remote) failed: EOF (version not received)" The connection hangs. If I wait a while - perhaps 30-60s, it kills the SSH connection and aborts the process. Is there a way I can test the remote end's ability to accept the data. [i.e. Can I do something like ssh root at po.abc.net \ docker exec b3093cxxxxxx doveadm sync -D -u user at abc.net And see if it would accept the data. [In short, do I have a local end problem or a remote end problem - and being able to test both parts individually would probably help me figure out what's broken. That still won't fix it, but at least I'll know which end I need to concentrate on.] I'd guess this will all seem pretty obvious in retrospect, but for the life of me, I'm completely lost and really don't have any idea where to start to break it down better so I can see how each piece is working or not. TIA -Greg P> I struggled with this at one time, and with the lack of examples P> around the internet. In fact, I might have written this very post P> myself at one point! In the end, the penny dropped when I saw that the format was P> (command on local server) + (stuff you need to connect over ssh) + (command on remote server) P> So what worked for me was: P> doveadm backup -u user at domain.com \ P> ssh -p 2222 (+ other ssh options) root at remoteserver.com \ P> doveadm dsync-server -u user2 at domain2.com P> If you get P> dsync-remote(user2 at domain2.com): Error: Mailbox INBOX sync: P> mailbox_delete failed: INBOX can't be deleted P> ... you'll need to clear the remote directory first! Or maybe try sync instead of backup? P> I found that running as root on the remote server was necessary in P> my case, due to the permissions on the remote directory. You might P> want to check permissions on remote directory are writable by the user in your ssh command. P> If still no joy you can run P> doveadm user user at domain.com on your local server and P> doveadm user user2 at domain2.com on your remote server P> and dovecot will tell you where its synching from and to. P> Finally, this solution is a 'push' from source server to target. P> You may 'pull' the other way if that makes more sense in your environment with -R P> Those pieces were enough for me to get it to work ... P> P. P> PS. For the record, on the original job I was under time pressure, P> and yes, I did use imapsync to get it done in the end. I got this P> to work later on when I had more time to tinker. P> On 14/05/2020 09.09, Gregory Sloop wrote:>> So I've done quite a lot of searching on the list and on the web - and perhaps my google-fu is really bad - but I can't find any real recipes on how to sync mail from the old server to the new.>> As an FYI - the old server is a CPanel/WHM setup on a VPS. >> The new is mailcow - which uses docker.>> However, I don't think either of these platforms is what's causing the issue - but I'm certainly not sure of that.>> --- >> I've tried several things - but have lost track of all the things I've tried. >> This seemed like the best of all the things I've tried.>> This particular mailbox/user+domain is setup on both servers.>> doveadm backup -D -u mc-user at abc.net ssh root at abc.net -p2200 doveadm dsync-server -u cp-user at abc.net>> mc-user at abc.net is the MC/NEW mailbox/domain >> cp-user at abc.net is the CPanel/OLD user/domain account >> The SSH server of the remote system is running on port 2200.>> However when I try this, I get: >> WARNING: The WATCHDOG_NOTIFY_EMAIL variable is not set. Defaulting to a blank string. >> dsync-local(user at abc.net)<v44NFyRWvF5efgAAb6OASA>: Error: read(remote) failed: EOF (version not received) >> doveadm(user at abc.net): Fatal: execvp(ssh) failed: No such file or directory>> The process appears to hang, and a Ctrl+C stops it.>> I'd love to get pointed at a reasonable recipe on how to make this work.>> I don't really get/understand the docs much at all. >> [And either everyone else understands it just fine, and never thinks to write a document on how to do it - or, and I think this is a lot more likely - they're using something like imapsync to do it. I found numerous places where others were, essentially, "Dovecot's tool is way too complicated and I can't get it to work right, so I used imapsync." I suppose I should probably just do that too, but it does seem a shame to do that when the dovecot tool is almost certainly the best tool for the job, but I can't figure out how to use it.]>> If someone can help me grok what's going on, I'm glad to write it up for the list and or a blog entry so it's more accessible.>> TIA >> -Greg-- Gregory Sloop, Principal: Sloop Network & Computer Consulting Voice: 503.251.0452 x121 EMail: gregs at sloop.net http://www.sloop.net --- -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20200514/a484ebe0/attachment-0001.html>