Darren Mobley
2022-May-10 22:17 UTC
Can sync/migrate all mail from remote imap account except the main "INBOX"
Greetings!
I've been trying to sync my email to a server, however it only syncs the
mail in folders other than INBOX.
For testing purposes, I've created a fake domain, "letest.tld" and
a new empty user account "mailto:saywhatnow at letest.tld" to receive
the copy of the mail being migrated.
What I've been trying are iterations of:
/usr/bin/doveadm -D? -o mail_fsync=never -o
dsync_features=empty-header-workaround -o imapc_ssl=imaps -o
imapc_features=rfc822.size,fetch-headers -o dsync_commit_msgs_interval=100 -o
imapc_port=993 -o imapc_host=imap.zoho.com -o imapc_password=NoNoTRe4LLyMyP4$$
-o imapc_user=mailto:decker at n3t.net -o mail_prefetch_count=20? sync? -1Ru
mailto:saywhatnow at letest.tld? imapc:
Regardless of running sync -1 or backup (I want to keep original mailbox as-is
for now) I get errors related to INBOX.
With sync -1 ;
May 10 16:53:29 dsync(mailto:saywhatnow at letest.tld): Debug: brain S: Ignore
nonexistent mailbox GUID c92f64f79f0d1ed01e6d5b314f04886c with -1 sync
May 10 16:53:29 dsync(mailto:saywhatnow at letest.tld): Debug: brain S: We
don't have mailbox c92f64f79f0d1ed01e6d5b314f04886c
May 10 16:53:29 dsync(mailto:saywhatnow at letest.tld): Debug: brain M: Ignoring
missing remote box GUID c92f64f79f0d1ed01e6d5b314f04886c
May 10 16:53:29 dsync(mailto:saywhatnow at letest.tld): Debug: Mailbox
INBOX.Archive: Mailbox opened
May 10 16:53:29 dsync(mailto:saywhatnow at letest.tld): Debug: Mailbox
INBOX.Archive: Mailbox opened
May 10 16:53:29 dsync(mailto:saywhatnow at letest.tld): Debug: brain S: Skipping
unchanged mailbox 51a58e1d1d5c491aac71488647b20224
May 10 16:53:29 dsync(mailto:saywhatnow at letest.tld): Debug: Mailbox
INBOX.Commercial Spam: Mailbox opened
May 10 16:53:29 dsync(mailto:saywhatnow at letest.tld): Debug: Mailbox
INBOX.Commercial Spam: Mailbox opened
With backup :
May 10 16:54:55 doveadm(mailto:saywhatnow at letest.tld): Debug: brain M:
Namespace INBOX. has location
maildir:/home/letest/mail/letest.tld/saywhatnow:UTF-8
May 10 16:54:55 doveadm(mailto:saywhatnow at letest.tld): Debug: Mailbox INBOX:
Mailbox opened
May 10 16:54:55 doveadm(mailto:saywhatnow at letest.tld): Debug: Mailbox INBOX:
Mailbox opened
May 10 16:54:55 doveadm(mailto:saywhatnow at letest.tld): Debug: Namespace
INBOX.: Using permissions from /home/letest/mail/letest.tld/saywhatnow:
mode=0700 gid=default
...
May 10 16:54:55 dsync(mailto:saywhatnow at letest.tld): Debug: brain S:
Namespace INBOX. has location imapc:
May 10 16:54:57 dsync(mailto:saywhatnow at letest.tld): Debug: brain M: Local
mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1
uid_next=1 subs=yes last_change=0 last_subs=1652217475
May 10 16:54:57 dsync(mailto:saywhatnow at letest.tld): Debug: brain S: Local
mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1
uid_next=8230 subs=yes last_change=0 last_subs=0
...
May 10 16:54:57 dsync(mailto:saywhatnow at letest.tld): Debug: brain M: Remote
mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1
uid_next=8230 subs=yes last_change=0 last_subs=0
...
May 10 16:54:57 dsync(mailto:saywhatnow at letest.tld): Debug: brain M: Mailbox
INBOX: local=c92f64f79f0d1ed01e6d5b314f04886c/0/1,
remote=c92f64f79f0d1ed01e6d5b314f04886c/0/1: Mailboxes are equal
...
May 10 16:54:57 dsync(mailto:saywhatnow at letest.tld): Debug: Mailbox INBOX:
Mailbox opened
May 10 16:54:57 dsync(mailto:saywhatnow at letest.tld): Debug: brain S: Mailbox
INBOX: local=c92f64f79f0d1ed01e6d5b314f04886c/0/1,
remote=c92f64f79f0d1ed01e6d5b314f04886c/0/1: Mailboxes are equal
...
May 10 16:54:57 dsync(mailto:saywhatnow at letest.tld): Debug: Mailbox INBOX:
Mailbox opened
May 10 16:54:57 dsync(mailto:saywhatnow at letest.tld): Debug: brain S: Change
during sync: Mailbox GUID c92f64f79f0d1ed01e6d5b314f04886c was lost
May 10 16:54:57 dsync(mailto:saywhatnow at letest.tld): Debug: brain S: We
don't have mailbox c92f64f79f0d1ed01e6d5b314f04886c
May 10 16:54:57 dsync(mailto:saywhatnow at letest.tld): Debug: brain M: Change
during sync: Remote lost mailbox GUID c92f64f79f0d1ed01e6d5b314f04886c (maybe it
was just deleted?)
...
May 10 16:55:33 dsync(mailto:saywhatnow at letest.tld): Warning: Mailbox changes
caused a desync. You may want to run dsync again: Remote lost mailbox GUID
c92f64f79f0d1ed01e6d5b314f04886c (maybe it was just deleted?)
This is odd, since that very GUID is in the uidlist file for the base folder:
# grep -R Gc92f64f79f0d1ed01e6d5b314f04886c?
/home/letest/mail/letest.tld/saywhatnow
/home/letest/mail/letest.tld/saywhatnow/dovecot-uidlist:3 V1 N1
Gc92f64f79f0d1ed01e6d5b314f04886c
I've tried all sorts of things to get this to work, including removing the
entire /home/letest/mail/letest.tld/saywhatnow/ directory so there could be no
conflicts with existing mailbox, set mailbox_list_index=no , set
lda_mailbox_autocreate=no , etc.
The errors appear to originate from src/doveadm/dsync/dsync-brain-mailbox.c but
it's not really telling me a lot.
I'm using dovecot version 2.3.18 (9dd8408c18) on a cPanel server.
`dovecot -n` output attached.
Any help would be greatly appreciated!
Thanks,
Darren
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20220510/d460aa47/attachment.htm>
-------------- next part --------------
# 2.3.18 (9dd8408c18): /etc/dovecot/dovecot.conf
# OS: Linux 3.10.0-1160.45.1.el7.x86_64 x86_64 CentOS Linux release 7.9.2009
(Core)
# Hostname: cent-7.test.bed
auth_cache_size = 1 M
auth_mechanisms = plain login
auth_username_chars =
"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!#$-=?^_{}~./@+%"
disable_plaintext_auth = no
first_valid_uid = 201
lmtp_rcpt_check_quota = yes
lmtp_save_to_detail_mailbox = yes
lmtp_user_concurrency_limit = 4
mail_access_groups = dovecot
mail_plugins = quota quota_clone zlib
mail_prefetch_count = 20
maildir_broken_filename_sizes = yes
maildir_very_dirty_syncs = yes
namespace inbox {
inbox = yes
location =
mailbox Archive {
auto = create
special_use = \Archive
}
mailbox Archives {
auto = no
special_use = \Archive
}
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox "Sent Messages" {
auto = no
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
mailbox spam {
auto = subscribe
special_use = \Junk
}
prefix = INBOX.
separator = .
type = private
}
passdb {
args = /usr/local/cpanel/etc/dovecot/cpauthd-dict.conf
driver = dict
result_failure = return-fail
}
plugin {
acl = vfile:cache_secs=86400
quota_exceeded_message = Mailbox is full / Blocks limit exceeded / Inode limit
exceeded
}
protocols = lmtp imap pop3
service auth {
unix_listener auth-client {
mode = 0666
}
}
service config {
vsz_limit = 2 G
}
service dict {
unix_listener dict {
group = dovecot
mode = 0660
}
}
service imap-hibernate {
unix_listener imap-hibernate {
group = $default_internal_group
mode = 0660
}
}
service imap-login {
client_limit = 500
inet_listener imap {
address = *,::
}
inet_listener imaps {
address = *,::
}
process_limit = 50
process_min_avail = 2
service_count = 0
vsz_limit = 128 M
}
service imap {
extra_groups = $default_internal_group
process_limit = 512
unix_listener imap-master {
user = $default_internal_user
}
vsz_limit = 512 M
}
service lmtp {
client_limit = 1
process_limit = 500
unix_listener lmtp {
group = mail
mode = 0660
user = mailnull
}
vsz_limit = 512 M
}
service managesieve-login {
client_limit = 500
process_limit = 50
process_min_avail = 2
service_count = 0
vsz_limit = 128 M
}
service managesieve {
process_limit = 512
vsz_limit = 512 M
}
service pop3-login {
client_limit = 500
inet_listener pop3 {
address = *,::
}
inet_listener pop3s {
address = *,::
}
process_limit = 50
process_min_avail = 2
service_count = 0
vsz_limit = 128 M
}
service pop3 {
process_limit = 512
vsz_limit = 512 M
}
service quota-status {
executable = quota-status -p postfix
unix_listener quota-status {
mode = 0666
}
}
service stats {
unix_listener stats-writer {
mode = 0666
}
}
ssl_cert = </etc/dovecot/ssl/dovecot.crt
ssl_cipher_list =
ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
ssl_dh = # hidden, use -P to show it
ssl_key = # hidden, use -P to show it
userdb {
driver = prefetch
}
userdb {
args = /usr/local/cpanel/etc/dovecot/cpauthd-dict.conf
driver = dict
}
protocol imap {
imap_capability = +NAMESPACE
imap_hibernate_timeout = 30 secs
imap_idle_notify_interval = 24 mins
imap_logout_format = in=%i, out=%o, bytes=%i/%o
mail_max_userip_connections = 20
mail_plugins = acl quota imap_quota zlib imap_zlib quota_clone virtual
namespace sent {
hidden = yes
list = no
location =
virtual:/usr/local/cpanel/etc/dovecot/virtual/sent:INDEX=~/mail/virtual/%u/sent
prefix = sent
separator = .
}
namespace spam {
hidden = yes
list = no
location =
virtual:/usr/local/cpanel/etc/dovecot/virtual/spam:INDEX=~/mail/virtual/%u/spam
prefix = spam
separator = .
}
}
protocol pop3 {
mail_max_userip_connections = 3
mail_plugins = quota quota quota_clone virtual zlib
namespace sent {
hidden = yes
list = no
location =
virtual:/usr/local/cpanel/etc/dovecot/virtual/sent:INDEX=~/mail/virtual/%u/sent
prefix = sent
separator = .
}
namespace spam {
hidden = yes
list = no
location =
virtual:/usr/local/cpanel/etc/dovecot/virtual/spam:INDEX=~/mail/virtual/%u/spam
prefix = spam
separator = .
}
pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s, bytes=%i/%o
pop3_uidl_format = UID%u-%v
}
protocol lmtp {
mail_plugins = quota quota_clone zlib
postmaster_address = root
quota_full_tempfail = no
}
protocol lda {
mail_plugins = quota quota_clone zlib
postmaster_address = root
quota_full_tempfail = no
}
local_name cent-7.test.bed {
ssl_cert = </etc/dovecot/ssl/dovecot.crt
ssl_key = # hidden, use -P to show it
}
Sami Ketola
2022-May-13 13:57 UTC
Can sync/migrate all mail from remote imap account except the main "INBOX"
> On 11. May 2022, at 1.17, Darren Mobley <decker at n3t.net <mailto:decker at n3t.net>> wrote: > > Greetings! > I've been trying to sync my email to a server, however it only syncs the mail in folders other than INBOX. > > For testing purposes, I've created a fake domain, "letest.tld" and a new empty user account "saywhatnow at letest.tld <mailto:saywhatnow at letest.tld>" to receive the copy of the mail being migrated. > > What I've been trying are iterations of: > > /usr/bin/doveadm -D -o mail_fsync=never -o dsync_features=empty-header-workaround -o imapc_ssl=imaps -o imapc_features=rfc822.size,fetch-headers -o dsync_commit_msgs_interval=100 -o imapc_port=993 -o imapc_host=imap.zoho.com <http://imap.zoho.com/> -o imapc_password=NoNoTRe4LLyMyP4$$ -o imapc_user=decker at n3t.net <mailto:decker at n3t.net> -o mail_prefetch_count=20 sync -1Ru saywhatnow at letest.tld <mailto:saywhatnow at letest.tld> imapc:can you record imapc rawlogs to check what the remote responds for some imap commands: doveadm -o <your_imapc_options...> -o imapc_rawlog_dir=/writeable_directory backup -u saywhatnow at letest.tld <mailto:saywhatnow at letest.tld> -R imapc: Then check the saved rawlogs for hints if the remote even lists all subfolders in response. Sami -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20220513/6633fa3a/attachment.htm>