Hello again,
I've found out that some mailboxes are actually duplicated. Doveadm
replicator status on the production server gives this:
~# doveadm replicator status 'dummy-c-1*'
username priority fast sync full sync
success sync failed
dummy-c-1 none 01:13:19 01:13:19
01:13:19 -
dummy-c-1 at univ-nantes.fr none 00:15:28 00:15:28
00:15:28 -
That'd explain why mails are counted twice when replicated on the new
server but where does this come from since I don't have this quota
problem on the production server?
From the logs, it seems that postfix uses username at univ-nantes.fr when
calling dovecot's LDA so I suppose that the non-'@univ-nantes.fr'
mailboxes are the erroneous ones.
The users aren't duplicated in our ldap user backend and aren't using
the @univ-nantes.fr part.
~# doveadm user 'dummy-c-1*'
dummy-c-1
Any ideas?
Thanks,
Arnaud
On 05/11/2021 16:21, Arnaud Ab?lard wrote:> Hello,
>
> We are very long time happy dovecot users (since 2008 at least). We have
> around 90k mailboxes and since we had to move away from our NAS storage
> to a ceph storage I jumped on the opportunity to enable compression with
> the zlib plugin and dovecot's replication mecanism. We are using
> debian's dovecot 2.2.27 packages on production and our new server is
> running dovecot's own ce-2.3.17 packages.
>
> On the production server everything works fine but on the new server,
> replicated mailboxes' quota is all wrong:
>
> on production:
> # doveadm quota get -u dummy-c-1
> Quota name??????? Type???? Value? Limit???????????????????????????? %
> Quota Utilisateur STORAGE 660026 976563??????????????????????????? 67
> Quota Utilisateur MESSAGE?? 8651????? -???????????????????????????? 0
>
> on new server:
> doveadm quota get -u dummy-c-1
> Quota name??????? Type????? Value? Limit?????????????????????????? %
> Quota Utilisateur STORAGE 1125251 976563???????????????????????? 115
> Quota Utilisateur MESSAGE?? 16646????? -?????????????????????????? 0
>
> If I add all the S flag from the filenames n both servers I get exactly
> the same usage, which is coherent with the quota on the production server:
>
> # find . -type f | grep 'S=' | awk -F'S=' '{print
$2}' | awk -F','
> '{print $1}' | awk -F':' '{print $1}' | paste -sd+
| bc -l
> 675865938
>
> And I have exactly the same amountof mails on the two server, the
> replication works as expected, no unwanted duplication of mails occurs.
>
> Of course, I've tried to ask dovecot to recalculate quotas with doveadm
> quota recalc -u <username>, but it doesn't fix the problem.
>
> What am I missing?
>
> Thanks,
>
> Arnaud
>
> PS: Here is my doveconf -n output:
>
> # 2.3.17 (e2aa53df5b): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.17 (054dddfa)
> # OS: Linux 4.19.0-18-amd64 x86_64 Debian 10.11
> # Hostname: gromel-test
> auth_cache_size = 10 k
> auth_verbose = yes
> disable_plaintext_auth = no
> doveadm_password = # hidden, use -P to show it
> hostname = gromel1.univ-nantes.prive
> lda_mailbox_autosubscribe = yes
> listen = *
> log_timestamp = "%Y-%m-%d %H:%M:%S "
> login_trusted_networks = (...)
> mail_gid = 5000
> mail_location = maildir:%h
> mail_plugins = quota zlib notify replication
> mail_privileged_group = vmail
> mail_uid = 5000
> maildir_stat_dirs = yes
> maildir_very_dirty_syncs = yes
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope
> encoded-character subaddress comparator-i;ascii-numeric relational regex
> imap4flags copy include variables body enotify environment mailbox date
> index ihave duplicate mime foreverypart extracttext
> namespace {
> ? inbox = yes
> ? location > ? prefix = INBOX.
> ? separator = .
> ? type = private
> }
> passdb {
> ? args = /etc/dovecot/dovecot-ldap.conf
> ? driver = ldap
> }
> plugin {
> ? quota = maildir:Quota Utilisateur
> ? quota_exceeded_message = Cet utilisateur a d?pass? son quota, votre
> message n'a pu lui ?tre livr?.
> ? quota_full_tempfail = yes
> ? quota_rule = *:storage=1000M
> ? quota_rule2 = INBOX.Trash:storage=+100M
> ? sieve = ~/dovecot.sieve
> ? sieve_dir = ~/sieve
> ? sieve_extensions = -vacation
> ? sieve_global_dir = /var/lib/dovecot/sieve/global/
> ? sieve_max_redirects = 1
> ? zlib_save = gz
> ? zlib_save_level = 6
> }
> postmaster_address = postmaster@<snip>
> protocols = imap pop3 sieve
> replication_max_conns = 50
> service auth {
> ? client_limit = 49452
> ? unix_listener auth-userdb {
> ??? group = vmail
> ??? mode = 0600
> ??? user = vmail
> ? }
> ? user = root
> }
> service dict {
> ? unix_listener dict {
> ??? mode = 0600
> ??? user = vmail
> ? }
> }
> service doveadm {
> ? inet_listener {
> ??? port = 12345
> ? }
> }
> service imap-login {
> ? process_min_avail = 8
> ? service_count = 0
> ? user = mail
> }
> service imap {
> ? executable = imap
> ? process_limit = 16384
> ? vsz_limit = 2 G
> }
> service managesieve-login {
> ? inet_listener sieve {
> ??? port = 4190
> ? }
> ? process_min_avail = 8
> ? service_count = 0
> ? user = mail
> ? vsz_limit = 2 G
> }
> service managesieve {
> ? drop_priv_before_exec = yes
> ? process_limit = 16384
> }
> service pop3-login {
> ? process_min_avail = 8
> ? service_count = 0
> ? user = mail
> }
> service pop3 {
> ? drop_priv_before_exec = yes
> ? process_limit = 16384
> }
> ssl = no
> userdb {
> ? args = /etc/dovecot/dovecot-ldap.conf
> ? driver = ldap
> }
> verbose_proctitle = yes
> protocol imap {
> ? mail_max_userip_connections = 50
> ? mail_plugins = quota zlib notify replication imap_quota zlib
> }
> protocol pop3 {
> ? mail_plugins = quota zlib notify replication
> }
> protocol sieve {
> ? mail_max_userip_connections = 10
> }
> protocol lda {
> ? mail_plugins = quota zlib notify replication sieve zlib
> }
>
>
>
>
--
Arnaud Ab?lard
Responsable p?le Syst?me et Stockage
Service Infrastructures
DSIN Universit? de Nantes
-