On 08/11/2021 10:39, Aki Tuomi wrote:> Try doveadm replicator remove dummy-c-1 at univ-nantes.fr
I did so on the source server, checked indeed that
dummy-c-1 at univ-nantes.fr wasn't showing up in the doveadm replicator
status list and it was indeed the case, so far so good. On the
destination server, dummy-c-1's mailbox was still showing a 115% quota
so I tried to make dovecot recalculate the quota but it did not change
anything. I ended up deleting the user's mailbox and forced a
replication and, to my surprise, the new mailbox still has a 115% quota.
Arnaud
>
> Aki
>
>> On 08/11/2021 11:32 Arnaud Ab?lard <arnaud.abelard at
univ-nantes.fr> wrote:
>>
>>
>> Hello Aki,
>>
>> Thanks for the tip. I changed postfix configuration so it will not use
>> the domain part anymore and added "auth_username_format =
%Ln" just in
>> case (and reloaded dovecot). So I shouldn't have any more new
>> user at domain users but for all the existing users, doveadm replicator
>> status still show user at domain clone and on the new server mails are
>> still counted twice.
>>
>> Our userdb and passdb don't have the @domain defined anywhere so I
don't
>> think it's coming from there.
>>
>> Is there a local cache that still has a list of those user at domain
>> usernames that I could reset or something?
>>
>> Thanks,
>>
>> Arnaud
>>
>>
>>
>> On 08/11/2021 09:48, Aki Tuomi wrote:
>>> This sounds like you are not normalizing usernames properly.
>>>
>>> Either use
>>>
>>> auth_username_format = %Ln
>>>
>>> as global setting, or return `user` attribute in both userdb and
passdb lookups.
>>>
>>> Aki
>>>
>>>> On 07/11/2021 20:31 Arnaud Ab?lard <arnaud.abelard at
univ-nantes.fr> wrote:
>>>>
>>>>
>>>> Hello again,
>>>>
>>>> I've found out that some mailboxes are actually
duplicated. Doveadm
>>>> replicator status on the production server gives this:
>>>>
>>>> ~# doveadm replicator status 'dummy-c-1*'
>>>> username priority fast sync full
sync
>>>> success sync failed
>>>> dummy-c-1 none 01:13:19
01:13:19
>>>> 01:13:19 -
>>>> dummy-c-1 at univ-nantes.fr none 00:15:28
00:15:28
>>>> 00:15:28 -
>>>>
>>>> That'd explain why mails are counted twice when replicated
on the new
>>>> server but where does this come from since I don't have
this quota
>>>> problem on the production server?
>>>>
>>>> From the logs, it seems that postfix uses username at
univ-nantes.fr when
>>>> calling dovecot's LDA so I suppose that the
non-'@univ-nantes.fr'
>>>> mailboxes are the erroneous ones.
>>>>
>>>> The users aren't duplicated in our ldap user backend and
aren't using
>>>> the @univ-nantes.fr part.
>>>>
>>>> ~# doveadm user 'dummy-c-1*'
>>>> dummy-c-1
>>>>
>>>> Any ideas?
>>>>
>>>> Thanks,
>>>>
>>>> Arnaud
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 05/11/2021 16:21, Arnaud Ab?lard wrote:
>>>>> Hello,
>>>>>
>>>>> We are very long time happy dovecot users (since 2008 at
least). We have
>>>>> around 90k mailboxes and since we had to move away from our
NAS storage
>>>>> to a ceph storage I jumped on the opportunity to enable
compression with
>>>>> the zlib plugin and dovecot's replication mecanism. We
are using
>>>>> debian's dovecot 2.2.27 packages on production and our
new server is
>>>>> running dovecot's own ce-2.3.17 packages.
>>>>>
>>>>> On the production server everything works fine but on the
new server,
>>>>> replicated mailboxes' quota is all wrong:
>>>>>
>>>>> on production:
>>>>> # doveadm quota get -u dummy-c-1
>>>>> Quota name??????? Type???? Value?
Limit???????????????????????????? %
>>>>> Quota Utilisateur STORAGE 660026
976563??????????????????????????? 67
>>>>> Quota Utilisateur MESSAGE?? 8651?????
-???????????????????????????? 0
>>>>>
>>>>> on new server:
>>>>> doveadm quota get -u dummy-c-1
>>>>> Quota name??????? Type????? Value?
Limit?????????????????????????? %
>>>>> Quota Utilisateur STORAGE 1125251
976563???????????????????????? 115
>>>>> Quota Utilisateur MESSAGE?? 16646?????
-?????????????????????????? 0
>>>>>
>>>>> If I add all the S flag from the filenames n both servers I
get exactly
>>>>> the same usage, which is coherent with the quota on the
production server:
>>>>>
>>>>> # find . -type f | grep 'S=' | awk -F'S='
'{print $2}' | awk -F','
>>>>> '{print $1}' | awk -F':' '{print
$1}' | paste -sd+ | bc -l
>>>>> 675865938
>>>>>
>>>>> And I have exactly the same amountof mails on the two
server, the
>>>>> replication works as expected, no unwanted duplication of
mails occurs.
>>>>>
>>>>> Of course, I've tried to ask dovecot to recalculate
quotas with doveadm
>>>>> quota recalc -u <username>, but it doesn't fix
the problem.
>>>>>
>>>>> What am I missing?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Arnaud
>>>>>
>>>>> PS: Here is my doveconf -n output:
>>>>>
>>>>> # 2.3.17 (e2aa53df5b): /etc/dovecot/dovecot.conf
>>>>> # Pigeonhole version 0.5.17 (054dddfa)
>>>>> # OS: Linux 4.19.0-18-amd64 x86_64 Debian 10.11
>>>>> # Hostname: gromel-test
>>>>> auth_cache_size = 10 k
>>>>> auth_verbose = yes
>>>>> disable_plaintext_auth = no
>>>>> doveadm_password = # hidden, use -P to show it
>>>>> hostname = gromel1.univ-nantes.prive
>>>>> lda_mailbox_autosubscribe = yes
>>>>> listen = *
>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S "
>>>>> login_trusted_networks = (...)
>>>>> mail_gid = 5000
>>>>> mail_location = maildir:%h
>>>>> mail_plugins = quota zlib notify replication
>>>>> mail_privileged_group = vmail
>>>>> mail_uid = 5000
>>>>> maildir_stat_dirs = yes
>>>>> maildir_very_dirty_syncs = yes
>>>>> managesieve_notify_capability = mailto
>>>>> managesieve_sieve_capability = fileinto reject envelope
>>>>> encoded-character subaddress comparator-i;ascii-numeric
relational regex
>>>>> imap4flags copy include variables body enotify environment
mailbox date
>>>>> index ihave duplicate mime foreverypart extracttext
>>>>> namespace {
>>>>> ? inbox = yes
>>>>> ? location >>>>> ? prefix = INBOX.
>>>>> ? separator = .
>>>>> ? type = private
>>>>> }
>>>>> passdb {
>>>>> ? args = /etc/dovecot/dovecot-ldap.conf
>>>>> ? driver = ldap
>>>>> }
>>>>> plugin {
>>>>> ? quota = maildir:Quota Utilisateur
>>>>> ? quota_exceeded_message = Cet utilisateur a d?pass? son
quota, votre
>>>>> message n'a pu lui ?tre livr?.
>>>>> ? quota_full_tempfail = yes
>>>>> ? quota_rule = *:storage=1000M
>>>>> ? quota_rule2 = INBOX.Trash:storage=+100M
>>>>> ? sieve = ~/dovecot.sieve
>>>>> ? sieve_dir = ~/sieve
>>>>> ? sieve_extensions = -vacation
>>>>> ? sieve_global_dir = /var/lib/dovecot/sieve/global/
>>>>> ? sieve_max_redirects = 1
>>>>> ? zlib_save = gz
>>>>> ? zlib_save_level = 6
>>>>> }
>>>>> postmaster_address = postmaster@<snip>
>>>>> protocols = imap pop3 sieve
>>>>> replication_max_conns = 50
>>>>> service auth {
>>>>> ? client_limit = 49452
>>>>> ? unix_listener auth-userdb {
>>>>> ??? group = vmail
>>>>> ??? mode = 0600
>>>>> ??? user = vmail
>>>>> ? }
>>>>> ? user = root
>>>>> }
>>>>> service dict {
>>>>> ? unix_listener dict {
>>>>> ??? mode = 0600
>>>>> ??? user = vmail
>>>>> ? }
>>>>> }
>>>>> service doveadm {
>>>>> ? inet_listener {
>>>>> ??? port = 12345
>>>>> ? }
>>>>> }
>>>>> service imap-login {
>>>>> ? process_min_avail = 8
>>>>> ? service_count = 0
>>>>> ? user = mail
>>>>> }
>>>>> service imap {
>>>>> ? executable = imap
>>>>> ? process_limit = 16384
>>>>> ? vsz_limit = 2 G
>>>>> }
>>>>> service managesieve-login {
>>>>> ? inet_listener sieve {
>>>>> ??? port = 4190
>>>>> ? }
>>>>> ? process_min_avail = 8
>>>>> ? service_count = 0
>>>>> ? user = mail
>>>>> ? vsz_limit = 2 G
>>>>> }
>>>>> service managesieve {
>>>>> ? drop_priv_before_exec = yes
>>>>> ? process_limit = 16384
>>>>> }
>>>>> service pop3-login {
>>>>> ? process_min_avail = 8
>>>>> ? service_count = 0
>>>>> ? user = mail
>>>>> }
>>>>> service pop3 {
>>>>> ? drop_priv_before_exec = yes
>>>>> ? process_limit = 16384
>>>>> }
>>>>> ssl = no
>>>>> userdb {
>>>>> ? args = /etc/dovecot/dovecot-ldap.conf
>>>>> ? driver = ldap
>>>>> }
>>>>> verbose_proctitle = yes
>>>>> protocol imap {
>>>>> ? mail_max_userip_connections = 50
>>>>> ? mail_plugins = quota zlib notify replication
imap_quota zlib
>>>>> }
>>>>> protocol pop3 {
>>>>> ? mail_plugins = quota zlib notify replication
>>>>> }
>>>>> protocol sieve {
>>>>> ? mail_max_userip_connections = 10
>>>>> }
>>>>> protocol lda {
>>>>> ? mail_plugins = quota zlib notify replication sieve
zlib
>>>>> }
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> Arnaud Ab?lard
>>>> Responsable p?le Syst?me et Stockage
>>>> Service Infrastructures
>>>> DSIN Universit? de Nantes
>>>> -
>>
>> --
>> Arnaud Ab?lard
>> Responsable p?le Syst?me et Stockage
>> Service Infrastructures
>> DSIN Universit? de Nantes
>> -
--
Arnaud Ab?lard
Responsable p?le Syst?me et Stockage
Service Infrastructures
DSIN Universit? de Nantes
-