Run the command on the target host, and change
user_query = SELECT '/vmail/%Ld/%Ln' AS home,
'mbox:~/mail:INBOX=/var/mail/%u' AS mail, 10000 AS uid, 10000 AS gid
FROM users WHERE email = '%Lu'
Aki
> On 20/08/2021 13:39 Christian K?ppers <c.kueppers at onoffice.de>
wrote:
>
>
> Please explain in more detail how I can do this.
>
> > Try targeting your director instead.
> in cmd of director
> /usr/bin/doveadm -Dv -o mail_location=imapc: -o
imapc_host=<director-dns> -o imapc_user="source_user_shard1" -o
imapc_password="source_user_shard1_password" copy -u
"dest_user_shard2" "dest_folder" user
"source_user_shard1" mailbox "source_folder" ALL
> does no change, like i said.
>
> in cmd of backend shard2
> /usr/bin/doveadm -Dv -o mail_location=imapc: -o
imapc_host=<director-dns> -o imapc_user="source_user_shard1" -o
imapc_password="source_user_shard1_password" copy -u
"dest_user_shard2" "dest_folder" user
"source_user_shard1" mailbox "source_folder" ALL
> results in same output than targeting backend shard1 direct.
>
> > The problem actually is that you are now targeting the source user
into the source user as well. You need to, somehow, make dovecot return
mail=imapc: for the target user. Or you can try to do the copying on the target
backend instead, so that you can return `mail=whatever your mail location is`
from userdb lookup easier.
>
> mysql dovecot configuration part:
> user_query = SELECT '/vmail/%Ld/%Ln' AS home, 10000 AS uid, 10000
AS gid FROM users WHERE email = '%Lu'
> password_query = SELECT email AS user, password, 10000 AS userdb_uid, 10000
AS userdb_gid, '/vmail/%Ld/%Ln' AS userdb_home FROM users WHERE email =
'%Lu' AND active = '1'
>
> Isn't my described attempt on backend of dest_user_shard2 exactly what
you described as possible next try? If not what has to be changed?
>
> Christian
>
>
>
> ----- Urspr?ngliche Nachricht -----
> Von: Aki Tuomi aki.tuomi at open-xchange.com
> Gesendet: Freitag, 20. August 2021 12:14:54
> An: ckueppers at onoffice.de, dovecot at dovecot.org
> Betreff: Re: AW: Problem with copy e-mails via doveadm
>
> The problem actually is that you are now targeting the source user into the
> source user as well. You need to, somehow, make dovecot return mail=imapc:
> for the target user. Or you can try to do the copying on the target backend
> instead, so that you can return `mail=whatever your mail location is` from
> userdb lookup easier.
>
> Aki
>
> > On 20/08/2021 13:09 Aki Tuomi aki.tuomi at open-xchange.com wrote:
> >
> >
> > Try targeting your director instead.
> >
> > Aki
> >
> > > On 20/08/2021 12:45 Christian K?ppers c.kueppers at onoffice.de
wrote:
> > >
> > >
> > > Okay, i need some further help.?
> > >
> > > What i've tried with your hint:
> > >
> > > Executing on cmd of director&proxy server:
> > > /usr/bin/doveadm -Dv -o mail_location=imapc: copy -u
"dest_user_shard2"
> "dest_folder" user "source_user_shard1" mailbox
"source_folder" ALL
> > > /usr/bin/doveadm -Dv -o mail_location=imapc: -o
> imapc_host=<director-dns> -o
imapc_user="source_user_shard1" -o
> imapc_password="source_user_shard1_password" copy -u
"dest_user_shard2"
> "dest_folder" user "source_user_shard1" mailbox
"source_folder" ALL
> > >
> > > I've also tried to execute "doveadm -c" with copied
and modified
> configuration on director&proxy server without "'y' AS
proxy" in sql
> password_query configuration part - without luck.
> > > This leads all to same debug output and result as command in
first post.
> > >
> > > After that i changed to cmd of backend server of dest_user_shard2
and
> tried:
> > > /usr/bin/doveadm -Dv -o mail_location=imapc: -o
> imapc_host=<backend-shard1-dns> -o
imapc_user="source_user_shard1" -o
> imapc_password="source_user_shard1_password" copy -u
"dest_user_shard2"
> "dest_folder" user "source_user_shard1" mailbox
"source_folder" ALL
> > > [..]
> > > Aug 20 08:52:38
> doveadm(source_user_shard1)<2442101><qAs8KVZtH2F1QyUA5QDIzw>:
Debug:
> imapc(<backend-shard1-dns>:143): Authenticated successfully
> > > Aug 20 08:52:38
> doveadm(source_user_shard1)<2442101><qAs8KVZtH2F1QyUA5QDIzw>:
Debug: imapc:
> root=, index=, indexpvt=, control=, inbox=, alt> > > Aug 20
08:52:38
> doveadm(source_user_shard1)<2442101><qAs8KVZtH2F1QyUA5QDIzw>:
Debug: quota:
> quota_over_flag check: quota_over_script unset - skipping
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Debug: Mailbox
dest_folder:
> Mailbox opened because: copy
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Debug: Mailbox
source_folder:
> Mailbox opened because: copy
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Debug: Mailbox
source_folder:
> UID 1: Opened mail because: copying
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Error: Copying message
UID 1
> from 'source_folder' failed: Mailbox doesn't exist: dest_folder
(0.001 +
> 0.000 secs).
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Debug: Mailbox
source_folder:
> UID 2: Opened mail because: copying
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Error: Copying message
UID 2
> from 'source_folder' failed: Mailbox doesn't exist: dest_folder
(0.001 +
> 0.000 secs).
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Debug: Mailbox
source_folder:
> UID 3: Opened mail because: copying
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Error: Copying message
UID 3
> from 'source_folder' failed: Mailbox doesn't exist: dest_folder
(0.001 +
> 0.000 secs).
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Debug: Mailbox
source_folder:
> UID 4: Opened mail because: copying
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Error: Copying message
UID 4
> from 'source_folder' failed: Mailbox doesn't exist: dest_folder
(0.001 +
> 0.000 secs).
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Debug: Mailbox
source_folder:
> UID 5: Opened mail because: copying
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Error: Copying message
UID 5
> from 'source_folder' failed: Mailbox doesn't exist: dest_folder
(0.001 +
> 0.000 secs).
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Error: Syncing mailbox
> 'dest_folder' failed: Mailbox doesn't exist: dest_folder (0.001
+ 0.000
> secs).
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Debug:
> imapc(<backend-shard1-dns>:143): Disconnected
> > > Aug 20 08:52:38 doveadm(dest_user_shard2): Debug:
> imapc(<backend-shard1-dns>:143): Disconnected
> > > Aug 20 08:52:38 doveadm(2442101): Debug: auth-master: conn
> unix:/var/run/dovecot/auth-userdb (pid=2630417,uid=0): Disconnected:
> Connection closed (fd=9)
> > >
> > > after creating dest_folder in source_user_shard1 mailbox output
changed
> to:
> > > [..]
> > > Aug 20 08:56:11
> doveadm(source_user_shard1)<2446702><0AmaCStuH2FuVSUA5QDIzw>:
Debug:
> imapc(<backend-shard1-dns>:143): Authenticated successfully
> > > Aug 20 08:56:11
> doveadm(source_user_shard1)<2446702><0AmaCStuH2FuVSUA5QDIzw>:
Debug: imapc:
> root=, index=, indexpvt=, control=, inbox=, alt> > > Aug 20
08:56:11
> doveadm(source_user_shard1)<2446702><0AmaCStuH2FuVSUA5QDIzw>:
Debug: quota:
> quota_over_flag check: quota_over_script unset - skipping
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug: Mailbox
dest_folder:
> Mailbox opened because: copy
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug: Mailbox
source_folder:
> Mailbox opened because: copy
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug: Mailbox
source_folder:
> UID 1: Opened mail because: copying
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug: Mailbox
dest_folder:
> saving UID 1: Opened mail
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug: Mailbox
source_folder:
> UID 2: Opened mail because: copying
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug: Mailbox
dest_folder:
> saving UID 2: Opened mail
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug: Mailbox
source_folder:
> UID 3: Opened mail because: copying
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug: Mailbox
dest_folder:
> saving UID 3: Opened mail
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug: Mailbox
source_folder:
> UID 4: Opened mail because: copying
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug: Mailbox
dest_folder:
> saving UID 4: Opened mail
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug: Mailbox
source_folder:
> UID 5: Opened mail because: copying
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug: Mailbox
dest_folder:
> saving UID 5: Opened mail
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug: replication:
> Replication requested by 'cmd_copy_box', priority=2
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Error: Syncing mailbox
> 'dest_folder' failed: BUG: Unknown internal error
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug:
> imapc(<backend-shard1-dns>:143): Disconnected
> > > Aug 20 08:56:11 doveadm(dest_user_shard2): Debug:
> imapc(<backend-shard1-dns>:143): Disconnected
> > > Aug 20 08:56:11 doveadm(2446702): Debug: auth-master: conn
> unix:/var/run/dovecot/auth-userdb (pid=2630417,uid=0): Disconnected:
> Connection closed (fd=9)
> > >
> > > This time e-mails get copied in source_user_shard1 from
source_folder to
> dest_folder although dest_user_shard2 is given in doveadm command and is
> different from source_user_shard1.
> > > It is not what I expected and desired.
> > >
> > > So yeah i haven't get the whole picture right now.
> > > Maybe you can give me some details.
> > >
> > > Christian
> > >
> > >
> > >
> > > ----- Urspr?ngliche Nachricht -----
> > > Von: Aki Tuomi aki.tuomi at open-xchange.com
> > > Gesendet: Freitag, 20. August 2021 09:06:31
> > > An: c.kueppers at onoffice.de
> > > Cc: dovecot at dovecot.org
> > > Betreff: Re: AW: Problem with copy e-mails via doveadm
> > >
> > > The copy command gets proxied to the remote server because you
used
> proxy=y.
> > > To do this kind of copying, you need to specify
mail_location=imapc: and
> > > target to the director.
> > >
> > > Aki
> > >
> > > > On 20/08/2021 09:53 Christian K?ppers c.kueppers at
onoffice.de wrote:
> > > >
> > > >
> > > > Maybe i missunderstood this message. For me this
indicates?that
> doveadm
> > > is searching on the wrong server (backend)
for?"source_user_shard1"
> 's?
> > > and
> > > > folder "source_folder".?From my understanding
doveadm has?to look
> > > on?192.168.0.11 or 192.168.0.12 (according to provided
configuration),
> but
> > > the
> > > > message says?192.168.0.21 which is backend
for?"dest_user_shard2".
> > > >
> > > >
> > > >
> > > > ----- Urspr?ngliche Nachricht -----
> > > > Von: Aki Tuomi aki.tuomi at open-xchange.com
> > > > Gesendet: Freitag, 20. August 2021 08:39:41
> > > > An: c.kueppers at onoffice.de, dovecot at dovecot.org
> > > > Betreff: Re: AW: Problem with copy e-mails via doveadm
> > > >
> > > > Aug 17 10:22:05 doveadm(dest_user_shard2): Debug:
auth-master: passdb
> > > > lookup(dest_user_shard2): Finished passdb lookup
> (user=dest_user_shard2
> > > > proxy=y director_tag=shard2 proxy_refresh=450
host=192.168.0.21)
> > > > Aug 17 10:22:05 doveadm(dest_user_shard2): Error:
> > > > remote(192.168.0.21:24245): Mailbox source_folder: Mailbox
sync
> failed:
> > > > Mailbox doesn't exist: source_folder
> > > >
> > > > Does this help?
> > > >
> > > > Aki
> > > >
> > > > > On 20/08/2021 09:12 Christian K?ppers c.kueppers at
onoffice.de wrote:
> > > > >
> > > > >
> > > > > Try to republish due to unreadable message (in html
format) in
> mailing
> > > > list archive
> > > > > maybe someone can take a better look at it now
> > > > >
> > > > >
> > > >
> > >
>
-------------------------------------------------------------------------------------------
> > > > >
> > > > > Hi,
> > > > >
> > > > > i want to copy e-mails from user
"source_user_shard1" and folder
> > > > "source_folder" lying on backend servers (shard1)
to another
> > > > > user "dest_user_shard2" and folder
"dest_folder" lying on different
> > > > backend servers (shard2) via command line tool doveadm copy
> > > > > executed on dovecot proxy&director servers. But
doveadm seems to
> look
> > > only
> > > > on "dest_user_shard2"'s backend for folder
"source_folder"
> > > > > of user "source_user_shard1" and can not find
it. To confirm this,
> i've
> > > > checked directories on filesystem on backend of
"dest_user_shard2"
> > > > > and a folder for "source_user_shard1" was
created including
> one?folder
> > > > "mdbox" and only one file
"dovecot.list.index.log" in it. Folder and
> file
> > > > > timestamps match the command debug output time.
> > > > > "doveadm list" executed for both users
show?the right folders
> > > > (source_folder and dest_folder exist in respective mailbox).
> > > > >
> > > > > Is it a bug or wrong usage of tool? Please advice.
> > > > >
> > > > > command executed on dovecot proxy&director server
with debug output:
> > > > > /usr/bin/doveadm -Dv copy -u
"dest_user_shard2" "dest_folder" user
> > > > "source_user_shard1" mailbox
"source_folder" ALL
> > > > > Aug 17 10:22:05 Debug: Loading modules from directory:
> > > > /usr/lib/dovecot/modules
> > > > > Aug 17 10:22:05 Debug: Module loaded:
> > > > /usr/lib/dovecot/modules/lib10_quota_plugin.so
> > > > > Aug 17 10:22:05 Debug: Loading modules from directory:
> > > > /usr/lib/dovecot/modules/doveadm
> > > > > Aug 17 10:22:05 Debug: Skipping module
doveadm_acl_plugin, because
> > > > dlopen() failed:
> > > >
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so:
> > > > > undefined symbol: acl_user_module (this is usually
intentional, so
> just
> > > > ignore this message)
> > > > > Aug 17 10:22:05 Debug: Module loaded:
> > > >
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so
> > > > > Aug 17 10:22:05 Debug: Module loaded:
> > > >
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so
> > > > > Aug 17 10:22:05 Debug: Skipping module
doveadm_fts_plugin, because
> > > > dlopen() failed:
> > > >
/usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so:
> > > > > undefined symbol: fts_user_get_language_list (this is
usually
> > > intentional,
> > > > so just ignore this message)
> > > > > Aug 17 10:22:05 doveadm(dest_user_shard2): Debug:
auth-master:
> passdb
> > > > lookup(dest_user_shard2): Started passdb lookup
> > > > > Aug 17 10:22:05 doveadm(dest_user_shard2): Debug:
auth-master: conn
> > > > unix:/var/run/dovecot/director-userdb: Connecting
> > > > > Aug 17 10:22:05 doveadm(dest_user_shard2): Debug:
auth-master: conn
> > > > unix:/var/run/dovecot/director-userdb (pid=647,uid=0):
Client
> connected
> > > > > (fd=9)
> > > > > Aug 17 10:22:05 doveadm(dest_user_shard2): Debug:
auth-master:
> passdb
> > > > lookup(dest_user_shard2): auth PASS input:
user=dest_user_shard2
> proxy=y
> > > > > director_tag=shard2 proxy_refresh=450 host=192.168.0.21
> > > > > Aug 17 10:22:05 doveadm(dest_user_shard2): Debug:
auth-master:
> passdb
> > > > lookup(dest_user_shard2): Finished passdb lookup
> (user=dest_user_shard2
> > > > > proxy=y director_tag=shard2 proxy_refresh=450
host=192.168.0.21)
> > > > > Aug 17 10:22:05 doveadm(dest_user_shard2): Error:
> > > > remote(192.168.0.21:24245): Mailbox source_folder: Mailbox
sync
> failed:
> > > > > Mailbox doesn't exist: source_folder
> > > > > Aug 17 10:22:05 doveadm(dest_user_shard2): Debug:
auth-master: conn
> > > > unix:/var/run/dovecot/director-userdb (pid=647,uid=0):
Disconnected:
> > > > > Connection closed (fd=9)
> > > > >
> > > <snip/>
> > >
> > > > >
> > > > > Kind regards,
> > > > >
> > > > > ?Christian K?ppers
> > > > > Expert Administrator
> > > > >
> > > > > ?
> > > > >
> > > > > onOffice GmbH
> > > > > Charlottenburger Allee 5 | 52068 Aachen
> > > > > Tel. +49 (0)241 446 86-0 | Fax. +49 (0)241 446 86-250
> > > > > E-Mail:c.kueppers at onoffice.de| Web:www.onOffice.com
> > > > >
> > > > > ?
> > > > > Registergericht: Amtsgericht Aachen, HRB 21420
> > > > > Gesch?ftsf?hrer: Dipl.-Kfm. Stefan Mantl
> > > > > Prokuristen: Janosch Reuschenbach, Kristina Andresen,
Christian
> > > M?hringer