Hello Aki,
Thanks for the tip. I changed postfix configuration so it will not use
the domain part anymore and added "auth_username_format = %Ln" just in
case (and reloaded dovecot). So I shouldn't have any more new
user at domain users but for all the existing users, doveadm replicator
status still show user at domain clone and on the new server mails are
still counted twice.
Our userdb and passdb don't have the @domain defined anywhere so I don't
think it's coming from there.
Is there a local cache that still has a list of those user at domain
usernames that I could reset or something?
Thanks,
Arnaud
On 08/11/2021 09:48, Aki Tuomi wrote:> This sounds like you are not normalizing usernames properly.
>
> Either use
>
> auth_username_format = %Ln
>
> as global setting, or return `user` attribute in both userdb and passdb
lookups.
>
> Aki
>
>> On 07/11/2021 20:31 Arnaud Ab?lard <arnaud.abelard at
univ-nantes.fr> wrote:
>>
>>
>> Hello again,
>>
>> I've found out that some mailboxes are actually duplicated.
Doveadm
>> replicator status on the production server gives this:
>>
>> ~# doveadm replicator status 'dummy-c-1*'
>> username priority fast sync full sync
>> success sync failed
>> dummy-c-1 none 01:13:19 01:13:19
>> 01:13:19 -
>> dummy-c-1 at univ-nantes.fr none 00:15:28 00:15:28
>> 00:15:28 -
>>
>> That'd explain why mails are counted twice when replicated on the
new
>> server but where does this come from since I don't have this quota
>> problem on the production server?
>>
>> From the logs, it seems that postfix uses username at univ-nantes.fr
when
>> calling dovecot's LDA so I suppose that the
non-'@univ-nantes.fr'
>> mailboxes are the erroneous ones.
>>
>> The users aren't duplicated in our ldap user backend and aren't
using
>> the @univ-nantes.fr part.
>>
>> ~# doveadm user 'dummy-c-1*'
>> dummy-c-1
>>
>> Any ideas?
>>
>> Thanks,
>>
>> Arnaud
>>
>>
>>
>>
>>
>> On 05/11/2021 16:21, Arnaud Ab?lard wrote:
>>> Hello,
>>>
>>> We are very long time happy dovecot users (since 2008 at least). We
have
>>> around 90k mailboxes and since we had to move away from our NAS
storage
>>> to a ceph storage I jumped on the opportunity to enable compression
with
>>> the zlib plugin and dovecot's replication mecanism. We are
using
>>> debian's dovecot 2.2.27 packages on production and our new
server is
>>> running dovecot's own ce-2.3.17 packages.
>>>
>>> On the production server everything works fine but on the new
server,
>>> replicated mailboxes' quota is all wrong:
>>>
>>> on production:
>>> # doveadm quota get -u dummy-c-1
>>> Quota name??????? Type???? Value? Limit????????????????????????????
%
>>> Quota Utilisateur STORAGE 660026 976563???????????????????????????
67
>>> Quota Utilisateur MESSAGE?? 8651????? -????????????????????????????
0
>>>
>>> on new server:
>>> doveadm quota get -u dummy-c-1
>>> Quota name??????? Type????? Value? Limit??????????????????????????
%
>>> Quota Utilisateur STORAGE 1125251 976563????????????????????????
115
>>> Quota Utilisateur MESSAGE?? 16646????? -??????????????????????????
0
>>>
>>> If I add all the S flag from the filenames n both servers I get
exactly
>>> the same usage, which is coherent with the quota on the production
server:
>>>
>>> # find . -type f | grep 'S=' | awk -F'S='
'{print $2}' | awk -F','
>>> '{print $1}' | awk -F':' '{print $1}' |
paste -sd+ | bc -l
>>> 675865938
>>>
>>> And I have exactly the same amountof mails on the two server, the
>>> replication works as expected, no unwanted duplication of mails
occurs.
>>>
>>> Of course, I've tried to ask dovecot to recalculate quotas with
doveadm
>>> quota recalc -u <username>, but it doesn't fix the
problem.
>>>
>>> What am I missing?
>>>
>>> Thanks,
>>>
>>> Arnaud
>>>
>>> PS: Here is my doveconf -n output:
>>>
>>> # 2.3.17 (e2aa53df5b): /etc/dovecot/dovecot.conf
>>> # Pigeonhole version 0.5.17 (054dddfa)
>>> # OS: Linux 4.19.0-18-amd64 x86_64 Debian 10.11
>>> # Hostname: gromel-test
>>> auth_cache_size = 10 k
>>> auth_verbose = yes
>>> disable_plaintext_auth = no
>>> doveadm_password = # hidden, use -P to show it
>>> hostname = gromel1.univ-nantes.prive
>>> lda_mailbox_autosubscribe = yes
>>> listen = *
>>> log_timestamp = "%Y-%m-%d %H:%M:%S "
>>> login_trusted_networks = (...)
>>> mail_gid = 5000
>>> mail_location = maildir:%h
>>> mail_plugins = quota zlib notify replication
>>> mail_privileged_group = vmail
>>> mail_uid = 5000
>>> maildir_stat_dirs = yes
>>> maildir_very_dirty_syncs = yes
>>> managesieve_notify_capability = mailto
>>> managesieve_sieve_capability = fileinto reject envelope
>>> encoded-character subaddress comparator-i;ascii-numeric relational
regex
>>> imap4flags copy include variables body enotify environment mailbox
date
>>> index ihave duplicate mime foreverypart extracttext
>>> namespace {
>>> ? inbox = yes
>>> ? location >>> ? prefix = INBOX.
>>> ? separator = .
>>> ? type = private
>>> }
>>> passdb {
>>> ? args = /etc/dovecot/dovecot-ldap.conf
>>> ? driver = ldap
>>> }
>>> plugin {
>>> ? quota = maildir:Quota Utilisateur
>>> ? quota_exceeded_message = Cet utilisateur a d?pass? son quota,
votre
>>> message n'a pu lui ?tre livr?.
>>> ? quota_full_tempfail = yes
>>> ? quota_rule = *:storage=1000M
>>> ? quota_rule2 = INBOX.Trash:storage=+100M
>>> ? sieve = ~/dovecot.sieve
>>> ? sieve_dir = ~/sieve
>>> ? sieve_extensions = -vacation
>>> ? sieve_global_dir = /var/lib/dovecot/sieve/global/
>>> ? sieve_max_redirects = 1
>>> ? zlib_save = gz
>>> ? zlib_save_level = 6
>>> }
>>> postmaster_address = postmaster@<snip>
>>> protocols = imap pop3 sieve
>>> replication_max_conns = 50
>>> service auth {
>>> ? client_limit = 49452
>>> ? unix_listener auth-userdb {
>>> ??? group = vmail
>>> ??? mode = 0600
>>> ??? user = vmail
>>> ? }
>>> ? user = root
>>> }
>>> service dict {
>>> ? unix_listener dict {
>>> ??? mode = 0600
>>> ??? user = vmail
>>> ? }
>>> }
>>> service doveadm {
>>> ? inet_listener {
>>> ??? port = 12345
>>> ? }
>>> }
>>> service imap-login {
>>> ? process_min_avail = 8
>>> ? service_count = 0
>>> ? user = mail
>>> }
>>> service imap {
>>> ? executable = imap
>>> ? process_limit = 16384
>>> ? vsz_limit = 2 G
>>> }
>>> service managesieve-login {
>>> ? inet_listener sieve {
>>> ??? port = 4190
>>> ? }
>>> ? process_min_avail = 8
>>> ? service_count = 0
>>> ? user = mail
>>> ? vsz_limit = 2 G
>>> }
>>> service managesieve {
>>> ? drop_priv_before_exec = yes
>>> ? process_limit = 16384
>>> }
>>> service pop3-login {
>>> ? process_min_avail = 8
>>> ? service_count = 0
>>> ? user = mail
>>> }
>>> service pop3 {
>>> ? drop_priv_before_exec = yes
>>> ? process_limit = 16384
>>> }
>>> ssl = no
>>> userdb {
>>> ? args = /etc/dovecot/dovecot-ldap.conf
>>> ? driver = ldap
>>> }
>>> verbose_proctitle = yes
>>> protocol imap {
>>> ? mail_max_userip_connections = 50
>>> ? mail_plugins = quota zlib notify replication imap_quota zlib
>>> }
>>> protocol pop3 {
>>> ? mail_plugins = quota zlib notify replication
>>> }
>>> protocol sieve {
>>> ? mail_max_userip_connections = 10
>>> }
>>> protocol lda {
>>> ? mail_plugins = quota zlib notify replication sieve zlib
>>> }
>>>
>>>
>>>
>>>
>>
>> --
>> Arnaud Ab?lard
>> Responsable p?le Syst?me et Stockage
>> Service Infrastructures
>> DSIN Universit? de Nantes
>> -
--
Arnaud Ab?lard
Responsable p?le Syst?me et Stockage
Service Infrastructures
DSIN Universit? de Nantes
-