thr3ads.net - dovecot - Unseen field reported by imap status command returns wrong count for shared mailboxes on dovecot cluster [Oct 2022]

If this information is useful, please help other people find it:
Share via:

Nikolaos Pyrgiotis

2022-Sep-08 14:47 UTC

Unseen field reported by imap status command returns wrong count for shared mailboxes on dovecot cluster

Hello,
We recently migrated our mail server to a dovecot cluster of 5nodes, a 
dovecot proxy, 2directors and 2dovecot backends.
All dovecot nodes run version 2.19.1. We use a glusterfs mounted volume 
on the backends forthe mail storage.
We noticed that when issuing the IMAP command to check forUNSEEN 
messages formails on the shared namespace instead of seeing the value of 
the unseen messages of the user that the mailbox has been shared to,
the value of the unseen messages of the owner of the mailbox is returned.
This behavior causes thunderbird when getting new messages forshared 
mailboxes, to show briefly all messages of the mailbox as unseen forthe 
user before showing the correct unseen count forthe shared mailbox.
The mail location of the shared mailboxes is defined with imapc storage 
type as described inthe documentation.
Is this a dovecot bug? Can we configure a different dovecot 
configuration setting so that imap status command reports the correct 
unseen field count when mail location is an imapc storage location?
An examples is given below when running doveadm command from one the 2 
dovecot directors:
When issuing the status command we see that the unseen count is 2:
root at doved0-rmt0-cn1:/etc/cron.d# doveadm mailbox status -u npyrgiotis 
all shared.sysadmins doveadm(npyrgiotis): Info: 
remote(10.101.0.71:8080): 
doveadm(npyrgiotis at domie02.com)<19078><9l8IJTb+GWOGSgAAEU9A+w>: 
imapc(10.101.0.75:143): Connected to 10.101.0.75:143 (local 
10.101.0.71:33476) shared.sysadmins messages=2 recent=0 uidnext=3 
uidvalidity=1662640492 unseen=2 highestmodseq=3 vsize=3950 
guid=c92f64f79f0d1ed01e6d5b314f04886c firstsaved=1662643853
shared.sysadmins 
messages=2recent=0uidnext=3uidvalidity=1662640492unseen=2highestmodseq=3vsize=3950guid=c92f64f79f0d1ed01e6d5b314f04886c
firstsaved=1662643853
But when fetching the emails of the mailboxes we can see that the \Seen 
flag is set for both emails
root at doved0-rmt0-cn1:/etc/cron.d# doveadm fetch -u npyrgiotis flags 
mailbox shared.sysadmins ALL doveadm(npyrgiotis): Info: 
remote(10.101.0.71:8080): 
doveadm(npyrgiotis at domie02.com)<19074><e7rWDgj+GWOCSgAAEU9A+w>: 
imapc(10.101.0.75:143): Connected to 10.101.0.75:143 (local 
10.101.0.71:38750) flags: \Seen flags: \Seen
Below i post the dovecot configuration of one of the two dovecot backends:
# 2.3.19.1 (9b53102964): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.19 (4eae2f79)
# OS: Linux 5.10.0-17-amd64 x86_64 Debian 11.4
# Hostname: doveb0-rmt0-cn1
auth_cache_negative_ttl = 5mins
auth_cache_size = 50M
auth_debug = yes
auth_default_realm = example.com
auth_master_user_separator = *
auth_mechanisms = plain login
auth_verbose = yes
auth_worker_max_count = 16
disable_plaintext_auth = no
first_valid_uid = 499
hostname = smtp.example.com
imapc_features = fetch-bodystructure fetch-headers rfc822.size search 
modseq acl delay-login
imapc_host = 10.101.0.75
imapc_password = # hidden, use -P to show it
imapc_sasl_mechanisms = plain login
imapc_ssl = starttls
imapc_ssl_verify = no
last_valid_uid = 499
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
login_greeting = You have successfully logged into example.com IMAP server
login_trusted_networks = 10.101.0.73 10.101.0.74
mail_always_cache_fields = flags hdr.* date.received date.sent
mail_cache_fields = flags date.received guid size.physical size.virtual 
imap.bodystructure body.snippet
mail_debug = yes
mail_fsync = always
mail_plugins = " notify mail_log zlib acl"
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart extracttext
mdbox_rotate_size = 200M
mmap_disable = yes
namespace example {
list = children
location = imapc:~/shared/%%n:INDEXPVT=~/shared-pvt/%%n
prefix = shared.%%n.
separator = .
subscriptions = no
type = shared
}
namespace inbox {
inbox = yes
location mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Spam.Virus {
auto = subscribe
}
mailbox Spam {
auto = subscribe
}
mailbox Trash {
special_use = \Trash
}
prefix separator = .
subscriptions = yes
type = private
}
passdb {
args = password=#hidden_use-P_to_show#
driver = static
master = yes
override_fields = 
allow_real_nets=127.0.0.0/8,local,10.101.0.73,10.101.0.74 
userdb_acl_user=%{auth_user} userdb_namespace/example/disabled=yes
result_success = continue
}
passdb {
args = password=#hidden_use-P_to_show#
driver = static
override_fields = allow_real_nets=127.0.0.0/8,local,10.101.0.73,10.101.0.74
result_success = continue
}
passdb {
args = /etc/dovecot/dovecot-ldap-users.conf.ext
driver = ldap
}
passdb {
args = /etc/dovecot/dovecot-ldap-groups.conf.ext
driver = ldap
}
plugin {
acl = vfile:/etc/dovecot/global-acls/%d/master-dovecot-acl:cache_secs=300
acl_defaults_from_inbox = yes
acl_globals_only = no
acl_shared_dict = file:/var/dovecot-acl/%d/shared-mailboxes.db
imap_compress_deflate_level = 6
mail_log_events = flag_change
mail_log_fields = uid box flags
sieve = file:~/sieve;active=~/sieve/%n.sieve
sieve_before = /var/vmail/global-sieve/global.sieve
sieve_env_email_address = %u
sieve_env_mailbox_prefix = %{userdb:mailbox_prefix}
sieve_global = /var/vmail/sieve
sieve_global_extensions = +editheader +vnd.dovecot.environment
zlib_save = zstd
zlib_save_level = 5
}
protocols = " imap lmtp sieve submission"
service auth-worker {
user = dovecot
}
service auth {
unix_listener auth-userdb {
group = mail
mode = 0666
user = vmail
}
}
service doveadm {
inet_listener dovadm-dom-mgmt {
address = 10.100.1.71
port = 8080
ssl = no
}
inet_listener dovadm-dom {
address = 10.101.0.71
port = 8080
ssl = no
}
vsz_limit = 256M
}
service imap-login {
inet_listener imap-dom {
address = 10.101.0.71
port = 143
ssl = no
}
inet_listener imap {
port = 0
}
inet_listener imaps-dom {
address = 10.101.0.71
port = 993
ssl = yes
}
inet_listener imaps {
port = 0
}
process_limit = 400
process_min_avail = 2
service_count = 20
vsz_limit = 512M
}
service imap {
process_limit = 2048
process_min_avail = 0
service_count = 1
vsz_limit = 4G
}
service lmtp {
inet_listener lmtp-dom {
address = 10.101.0.71
port = 24
ssl = no
}
process_limit = 0
process_min_avail = 0
service_count = 0
unix_listener lmtp {
mode = 0666
}
vsz_limit = 256M
}
service managesieve-login {
inet_listener sieve-dom {
address = 10.101.0.71
port = 4190
ssl = no
}
inet_listener sieve {
port = 0
}
process_limit = 0
process_min_avail = 0
service_count = 0
vsz_limit = 64M
}
service managesieve {
process_limit = 1024
process_min_avail = 0
service_count = 1
vsz_limit = 256M
}
service submission-login {
inet_listener submission-dom {
address = 10.101.0.71
port = 587
ssl = no
}
inet_listener submission {
port = 0
}
inet_listener submissions-dom {
address = 10.101.0.71
port = 465
ssl = yes
}
process_limit = 250
process_min_avail = 2
service_count = 1
vsz_limit = 256M
}
service submission {
process_limit = 1024
process_min_avail = 0
service_count = 1
vsz_limit = 256M
}
ssl = required
ssl_cert = </etc/ssl/certs/doveb0-rmt0-cn1.mail.rmt.example.com.pem
ssl_client_ca_dir = /etc/ssl/certs
ssl_key = # hidden, use -P to show it
submission_max_mail_size = 30000k
submission_relay_host = 10.101.0.7
submission_relay_port = 587
submission_relay_ssl = starttls
submission_relay_ssl_verify = no
submission_relay_trusted = yes
userdb {
driver = prefetch
}
userdb {
args = /etc/dovecot/dovecot-ldap-users.conf.ext
driver = ldap
}
userdb {
args = /etc/dovecot/dovecot-ldap-groups.conf.ext
driver = ldap
}
protocol lmtp {
mail_plugins = " notify mail_log zlib acl sieve acl"
}
protocol lda {
mail_plugins = " notify mail_log zlib acl sieve acl"
}
protocol imap {
mail_max_userip_connections = 100
mail_plugins = " notify mail_log zlib acl"
}
protocol sieve {
mail_max_userip_connections = 40
}
protocol submission {
mail_max_userip_connections = 70
}
local10.101.0.71 {
doveadm_allowed_commands doveadm_password = # hidden, use -P to show it
}
local10.100.1.71 {
doveadm_allowed_commands doveadm_password = # hidden, use -P to show it
}
Thank you,
Nikolaos Pyrgiotis
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20220908/fe27bdfb/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dovecot-sysreport-doveb0-rmt0-cn1-1662646499.tar.gz
Type: application/gzip
Size: 7268 bytes
Desc: not available
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20220908/fe27bdfb/attachment-0001.gz>

Nikolaos Pyrgiotis

2022-Oct-07 12:38 UTC

head link

Unseen field reported by imap status command returns wrong count for shared mailboxes on dovecot cluster

Hello,

I want to make a correction on my first post. We are using version 
2.3.19.1 for all our dovecot servers. Is there any update regarding this 
issue? I would be happy to provide more details if necessary.

Best Regards,

Nikos Pyrgiotis

On 8/9/22 17:47, Nikolaos Pyrgiotis wrote:> Hello,
> We recently migrated our mail server to a dovecot cluster of 5nodes, a 
> dovecot proxy, 2directors and 2dovecot backends.
> All dovecot nodes run version 2.19.1. We use a glusterfs mounted 
> volume on the backends forthe mail storage.
> We noticed that when issuing the IMAP command to check forUNSEEN 
> messages formails on the shared namespace instead of seeing the value 
> of the unseen messages of the user that the mailbox has been shared to,
> the value of the unseen messages of the owner of the mailbox is returned.
> This behavior causes thunderbird when getting new messages forshared 
> mailboxes, to show briefly all messages of the mailbox as unseen 
> forthe user before showing the correct unseen count forthe shared 
> mailbox.
> The mail location of the shared mailboxes is defined with imapc 
> storage type as described inthe documentation.
> Is this a dovecot bug? Can we configure a different dovecot 
> configuration setting so that imap status command reports the correct 
> unseen field count when mail location is an imapc storage location?
> An examples is given below when running doveadm command from one the 2 
> dovecot directors:
> When issuing the status command we see that the unseen count is 2:
> root at doved0-rmt0-cn1:/etc/cron.d# doveadm mailbox status -u npyrgiotis 
> all shared.sysadmins doveadm(npyrgiotis): Info: 
> remote(10.101.0.71:8080): 
> doveadm(npyrgiotis at
domie02.com)<19078><9l8IJTb+GWOGSgAAEU9A+w>:
> imapc(10.101.0.75:143): Connected to 10.101.0.75:143 (local 
> 10.101.0.71:33476) shared.sysadmins messages=2 recent=0 uidnext=3 
> uidvalidity=1662640492 unseen=2 highestmodseq=3 vsize=3950 
> guid=c92f64f79f0d1ed01e6d5b314f04886c firstsaved=1662643853
> shared.sysadmins 
>
messages=2recent=0uidnext=3uidvalidity=1662640492unseen=2highestmodseq=3vsize=3950guid=c92f64f79f0d1ed01e6d5b314f04886c
> firstsaved=1662643853
> But when fetching the emails of the mailboxes we can see that the 
> \Seen flag is set for both emails
> root at doved0-rmt0-cn1:/etc/cron.d# doveadm fetch -u npyrgiotis flags 
> mailbox shared.sysadmins ALL doveadm(npyrgiotis): Info: 
> remote(10.101.0.71:8080): 
> doveadm(npyrgiotis at
domie02.com)<19074><e7rWDgj+GWOCSgAAEU9A+w>:
> imapc(10.101.0.75:143): Connected to 10.101.0.75:143 (local 
> 10.101.0.71:38750) flags: \Seen flags: \Seen
> Below i post the dovecot configuration of one of the two dovecot backends:
> # 2.3.19.1 (9b53102964): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.19 (4eae2f79)
> # OS: Linux 5.10.0-17-amd64 x86_64 Debian 11.4
> # Hostname: doveb0-rmt0-cn1
> auth_cache_negative_ttl = 5mins
> auth_cache_size = 50M
> auth_debug = yes
> auth_default_realm = example.com
> auth_master_user_separator = *
> auth_mechanisms = plain login
> auth_verbose = yes
> auth_worker_max_count = 16
> disable_plaintext_auth = no
> first_valid_uid = 499
> hostname = smtp.example.com
> imapc_features = fetch-bodystructure fetch-headers rfc822.size search 
> modseq acl delay-login
> imapc_host = 10.101.0.75
> imapc_password = # hidden, use -P to show it
> imapc_sasl_mechanisms = plain login
> imapc_ssl = starttls
> imapc_ssl_verify = no
> last_valid_uid = 499
> lda_mailbox_autocreate = yes
> lda_mailbox_autosubscribe = yes
> login_greeting = You have successfully logged into example.com IMAP server
> login_trusted_networks = 10.101.0.73 10.101.0.74
> mail_always_cache_fields = flags hdr.* date.received date.sent
> mail_cache_fields = flags date.received guid size.physical 
> size.virtual imap.bodystructure body.snippet
> mail_debug = yes
> mail_fsync = always
> mail_plugins = " notify mail_log zlib acl"
> mail_privileged_group = mail
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope 
> encoded-character vacation subaddress comparator-i;ascii-numeric 
> relational regex imap4flags copy include variables body enotify 
> environment mailbox date index ihave duplicate mime foreverypart 
> extracttext
> mdbox_rotate_size = 200M
> mmap_disable = yes
> namespace example {
> list = children
> location = imapc:~/shared/%%n:INDEXPVT=~/shared-pvt/%%n
> prefix = shared.%%n.
> separator = .
> subscriptions = no
> type = shared
> }
> namespace inbox {
> inbox = yes
> location > mailbox Drafts {
> special_use = \Drafts
> }
> mailbox Junk {
> special_use = \Junk
> }
> mailbox Sent {
> special_use = \Sent
> }
> mailbox "Sent Messages" {
> special_use = \Sent
> }
> mailbox Spam.Virus {
> auto = subscribe
> }
> mailbox Spam {
> auto = subscribe
> }
> mailbox Trash {
> special_use = \Trash
> }
> prefix > separator = .
> subscriptions = yes
> type = private
> }
> passdb {
> args = password=#hidden_use-P_to_show#
> driver = static
> master = yes
> override_fields = 
> allow_real_nets=127.0.0.0/8,local,10.101.0.73,10.101.0.74 
> userdb_acl_user=%{auth_user} userdb_namespace/example/disabled=yes
> result_success = continue
> }
> passdb {
> args = password=#hidden_use-P_to_show#
> driver = static
> override_fields = 
> allow_real_nets=127.0.0.0/8,local,10.101.0.73,10.101.0.74
> result_success = continue
> }
> passdb {
> args = /etc/dovecot/dovecot-ldap-users.conf.ext
> driver = ldap
> }
> passdb {
> args = /etc/dovecot/dovecot-ldap-groups.conf.ext
> driver = ldap
> }
> plugin {
> acl = vfile:/etc/dovecot/global-acls/%d/master-dovecot-acl:cache_secs=300
> acl_defaults_from_inbox = yes
> acl_globals_only = no
> acl_shared_dict = file:/var/dovecot-acl/%d/shared-mailboxes.db
> imap_compress_deflate_level = 6
> mail_log_events = flag_change
> mail_log_fields = uid box flags
> sieve = file:~/sieve;active=~/sieve/%n.sieve
> sieve_before = /var/vmail/global-sieve/global.sieve
> sieve_env_email_address = %u
> sieve_env_mailbox_prefix = %{userdb:mailbox_prefix}
> sieve_global = /var/vmail/sieve
> sieve_global_extensions = +editheader +vnd.dovecot.environment
> zlib_save = zstd
> zlib_save_level = 5
> }
> protocols = " imap lmtp sieve submission"
> service auth-worker {
> user = dovecot
> }
> service auth {
> unix_listener auth-userdb {
> group = mail
> mode = 0666
> user = vmail
> }
> }
> service doveadm {
> inet_listener dovadm-dom-mgmt {
> address = 10.100.1.71
> port = 8080
> ssl = no
> }
> inet_listener dovadm-dom {
> address = 10.101.0.71
> port = 8080
> ssl = no
> }
> vsz_limit = 256M
> }
> service imap-login {
> inet_listener imap-dom {
> address = 10.101.0.71
> port = 143
> ssl = no
> }
> inet_listener imap {
> port = 0
> }
> inet_listener imaps-dom {
> address = 10.101.0.71
> port = 993
> ssl = yes
> }
> inet_listener imaps {
> port = 0
> }
> process_limit = 400
> process_min_avail = 2
> service_count = 20
> vsz_limit = 512M
> }
> service imap {
> process_limit = 2048
> process_min_avail = 0
> service_count = 1
> vsz_limit = 4G
> }
> service lmtp {
> inet_listener lmtp-dom {
> address = 10.101.0.71
> port = 24
> ssl = no
> }
> process_limit = 0
> process_min_avail = 0
> service_count = 0
> unix_listener lmtp {
> mode = 0666
> }
> vsz_limit = 256M
> }
> service managesieve-login {
> inet_listener sieve-dom {
> address = 10.101.0.71
> port = 4190
> ssl = no
> }
> inet_listener sieve {
> port = 0
> }
> process_limit = 0
> process_min_avail = 0
> service_count = 0
> vsz_limit = 64M
> }
> service managesieve {
> process_limit = 1024
> process_min_avail = 0
> service_count = 1
> vsz_limit = 256M
> }
> service submission-login {
> inet_listener submission-dom {
> address = 10.101.0.71
> port = 587
> ssl = no
> }
> inet_listener submission {
> port = 0
> }
> inet_listener submissions-dom {
> address = 10.101.0.71
> port = 465
> ssl = yes
> }
> process_limit = 250
> process_min_avail = 2
> service_count = 1
> vsz_limit = 256M
> }
> service submission {
> process_limit = 1024
> process_min_avail = 0
> service_count = 1
> vsz_limit = 256M
> }
> ssl = required
> ssl_cert = </etc/ssl/certs/doveb0-rmt0-cn1.mail.rmt.example.com.pem
> ssl_client_ca_dir = /etc/ssl/certs
> ssl_key = # hidden, use -P to show it
> submission_max_mail_size = 30000k
> submission_relay_host = 10.101.0.7
> submission_relay_port = 587
> submission_relay_ssl = starttls
> submission_relay_ssl_verify = no
> submission_relay_trusted = yes
> userdb {
> driver = prefetch
> }
> userdb {
> args = /etc/dovecot/dovecot-ldap-users.conf.ext
> driver = ldap
> }
> userdb {
> args = /etc/dovecot/dovecot-ldap-groups.conf.ext
> driver = ldap
> }
> protocol lmtp {
> mail_plugins = " notify mail_log zlib acl sieve acl"
> }
> protocol lda {
> mail_plugins = " notify mail_log zlib acl sieve acl"
> }
> protocol imap {
> mail_max_userip_connections = 100
> mail_plugins = " notify mail_log zlib acl"
> }
> protocol sieve {
> mail_max_userip_connections = 40
> }
> protocol submission {
> mail_max_userip_connections = 70
> }
> local10.101.0.71 {
> doveadm_allowed_commands > doveadm_password = # hidden, use -P to show
it
> }
> local10.100.1.71 {
> doveadm_allowed_commands > doveadm_password = # hidden, use -P to show
it
> }
> Thank you,
> Nikolaos Pyrgiotis-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20221007/76ce2458/attachment-0001.htm>

dovecot - Oct 2022 - Unseen field reported by imap status command returns wrong count for shared mailboxes on dovecot cluster

Unseen field reported by imap status command returns wrong count for shared mailboxes on dovecot cluster

Unseen field reported by imap status command returns wrong count for shared mailboxes on dovecot cluster