Am 06.03.23 um 16:45 schrieb Aki Tuomi:>> On 06/03/2023 16:52 EET Tim Evers <te-ml-ext at artfiles.de>
wrote:
>>
>>
>> Am 06.03.23 um 15:29 schrieb Aki Tuomi:
>>>> On 06/03/2023 15:56 EET Tim Evers <te-ml-ext at
artfiles.de> wrote:
>>>>
>>>>
>>>> Am 06.03.23 um 14:49 schrieb Aki Tuomi:
>>>>>> On 06/03/2023 15:45 EET Tim Evers <te-ml-ext at
artfiles.de> wrote:
>>>>>>
>>>>>>
>>>>>> Am 06.03.23 um 14:00 schrieb Aki Tuomi:
>>>>>>>> On 06/03/2023 13:06 EET Tim Evers <te-ml-ext
at artfiles.de> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> Am 06.03.23 um 11:59 schrieb Aki Tuomi:
>>>>>>>>
>>>>>>>>>> On 06/03/2023 12:44 EET Tim Evers
<te-ml-ext at artfiles.de> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Am 06.03.23 um 11:42 schrieb Aki Tuomi:
>>>>>>>>>>
>>>>>>>>>>>> On 06/03/2023 12:32 EET Tim
Evers <te-ml-ext at artfiles.de> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> since I did not get any
feedback on my bug report post regarding
>>>>>>>>>>>> corrupted sizes while using
zlib
>>>>>>>>>>>>
(https://dovecot.org/pipermail/dovecot/2023-February/126105.html) I
>>>>>>>>>>>> would like to confirm that this
bug report reached someone in charge.
>>>>>>>>>>>>
>>>>>>>>>>>> Or if not - I would kindly ask
for directions on how to post it in a way
>>>>>>>>>>>> that spawns some action.
>>>>>>>>>>>>
>>>>>>>>>>>> Thx
>>>>>>>>>>>>
>>>>>>>>>>>> Tim
>>>>>>>>>>> Can you try erasing
dovecot.index.cache and see if the problem fixes itself? It's possible your
cache contains invalid values from old errors.
>>>>>>>>>>>
>>>>>>>>>>> Aki
>>>>>>>>>> Did that already several times (other
dovecot.index.* files too). The
>>>>>>>>>> only thing that helps is actually
deleting the dovecot-uidlist file but
>>>>>>>>>> the problem returns.
>>>>>>>>>>
>>>>>>>>>> Tim
>>>>>>>>> I think it's because the value on the
filename is wrong. The filename contains the virtual and physical size.
>>>>>>>>>
>>>>>>>>> Can you try if
https://github.com/dovecot/tools/blob/main/maildir-size-fix.pl fixes your issue?
>>>>>>>>>
>>>>>>>>> Aki
>>>>>>>> Quoting
>>>>>>>>
>>>>>>>>
https://doc.dovecot.org/configuration_manual/zlib_plugin/ :
>>>>>>>>
>>>>>>>> "All mails must have ,|S=<size>|in
their filename where <size> contains
>>>>>>>> the original uncompressed mail size, otherwise
there will be problems
>>>>>>>> with quota calculation as well as other
potential random failures."
>>>>>>>>
>>>>>>>> And the script you send:
>>>>>>>>
>>>>>>>> "# Check if the files are compressed. Use
the uncompressed size for S=size."
>>>>>>>>
>>>>>>>> Which it does in my example. S= contains the
uncompressed size, not the
>>>>>>>> size on disk. Is this not correct? All Files
are written exclusively by
>>>>>>>> dovecot, no other software is involved.
>>>>>>>>
>>>>>>>> Tim
>>>>>>> Can you send output from `doveconf -n`? I am having
suspicion that zlib plugin isn't always involved in your config, which
causes the compressed file to be cached wrongly. Especially it feels like your
IMAP protocol does not have compression on while LMTP does.
>>>>>>>
>>>>>>> Aki
>>>>>> Actually it is the other way round (I removed zlib from
lmtp to reduce
>>>>>> the impact of the problem):
>>>>>>
>>>>>> # 2.3.20 (80a5ac675d): /etc/dovecot/dovecot.conf
>>>>>> # Pigeonhole version 0.5.20 (149edcf2)
>>>>>> # OS: Linux 4.15.0-204-generic x86_64 Ubuntu 18.04.6
LTS
>>>>>> # Hostname: node2.mst27.artfiles.de
>>>>>> auth_verbose = yes
>>>>>> auth_worker_max_count = 1000
>>>>>> default_client_limit = 5000
>>>>>> dict {
>>>>>> ? sqlquota =
mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
>>>>>> }
>>>>>> disable_plaintext_auth = no
>>>>>> import_environment = TZ CLUSTERNAME
>>>>>> lda_mailbox_autocreate = yes
>>>>>> lda_mailbox_autosubscribe = yes
>>>>>> mail_location = maildir:~/Maildir
>>>>>> mail_plugins = " mail_log notify old_stats"
>>>>>> mailbox_idle_check_interval = 10 mins
>>>>>> maildir_broken_filename_sizes = yes
>>>>>> managesieve_notify_capability = mailto
>>>>>> managesieve_sieve_capability = fileinto reject envelope
>>>>>> encoded-character vacation subaddress
comparator-i;ascii-numeric
>>>>>> relational regex imap4flags copy include variables body
enotify
>>>>>> environment mailbox date index ihave duplicate mime
foreverypart
>>>>>> extracttext editheader imapsieve vnd.dovecot.imapsieve
>>>>>> namespace inbox {
>>>>>> ? inbox = yes
>>>>>> ? location >>>>>> ? mailbox
Drafts {
>>>>>> ??? auto = subscribe
>>>>>> ??? special_use = \Drafts
>>>>>> ? }
>>>>>> ? mailbox Junk {
>>>>>> ??? auto = subscribe
>>>>>> ??? special_use = \Junk
>>>>>> ? }
>>>>>> ? mailbox Sent {
>>>>>> ??? auto = subscribe
>>>>>> ??? special_use = \Sent
>>>>>> ? }
>>>>>> ? mailbox "Sent Messages" {
>>>>>> ??? special_use = \Sent
>>>>>> ? }
>>>>>> ? mailbox Trash {
>>>>>> ??? auto = subscribe
>>>>>> ??? special_use = \Trash
>>>>>> ? }
>>>>>> ? prefix = INBOX.
>>>>>> ? separator = .
>>>>>> }
>>>>>> passdb {
>>>>>> ? args = /etc/dovecot/dovecot-sql.conf.ext
>>>>>> ? driver = sql
>>>>>> }
>>>>>> plugin {
>>>>>> ? imapsieve_mailbox1_before =
file:~/af_report-spam.sieve
>>>>>> ? imapsieve_mailbox1_causes = COPY
>>>>>> ? imapsieve_mailbox1_name = INBOX.Junk
>>>>>> ? imapsieve_mailbox2_before =
file:~/af_report-ham.sieve
>>>>>> ? imapsieve_mailbox2_causes = COPY
>>>>>> ? imapsieve_mailbox2_from = INBOX.Junk
>>>>>> ? imapsieve_mailbox2_name = *
>>>>>> ? quota = dict:User quota::proxy::sqlquota
>>>>>> ? quota_rule = *:storage=0:messages=0
>>>>>> ? sieve = file:~/sieve;active=~/.dovecot.sieve
>>>>>> ? sieve_before = file:~/af_master.sieve
>>>>>> ? sieve_before2 = file:~/af_konto.sieve
>>>>>> ? sieve_before3 = file:~/af_domain.sieve
>>>>>> ? sieve_before4 = file:~/af_spamfilter.sieve
>>>>>> ? sieve_extensions = +editheader
>>>>>> ? sieve_filter_bin_dir = /usr/local/bin/sieve
>>>>>> ? sieve_global_extensions = +vnd.dovecot.pipe
+vnd.dovecot.filter
>>>>>> +vnd.dovecot.environment
>>>>>> ? sieve_max_redirects = 100
>>>>>> ? sieve_pipe_bin_dir = /usr/local/bin/sieve
>>>>>> ? sieve_plugins = sieve_imapsieve sieve_extprograms
>>>>>> ? stats_refresh = 30 secs
>>>>>> ? zlib_save = gz
>>>>>> ? zlib_save_level = 6
>>>>>> }
>>>>>> pop3_no_flag_updates = yes
>>>>>> protocols = " imap lmtp sieve pop3"
>>>>>> service anvil {
>>>>>> ? client_limit = 6203
>>>>>> }
>>>>>> service auth-worker {
>>>>>> ? process_limit = 1000
>>>>>> }
>>>>>> service auth {
>>>>>> ? client_limit = 6300
>>>>>> }
>>>>>> service dict {
>>>>>> ? process_limit = 500
>>>>>> ? unix_listener dict {
>>>>>> ??? group = vmail
>>>>>> ??? mode = 0600
>>>>>> ??? user = vmail
>>>>>> ? }
>>>>>> }
>>>>>> service imap-login {
>>>>>> ? process_limit = 4000
>>>>>> ? process_min_avail = 0
>>>>>> ? service_count = 0
>>>>>> ? vsz_limit = 512 M
>>>>>> }
>>>>>> service imap {
>>>>>> ? client_limit = 1
>>>>>> ? process_limit = 4000
>>>>>> ? vsz_limit = 1 G
>>>>>> }
>>>>>> service lmtp {
>>>>>> ? inet_listener lmtp {
>>>>>> ??? address = 127.0.0.1
>>>>>> ??? port = 10024
>>>>>> ? }
>>>>>> ? process_limit = 80
>>>>>> ? process_min_avail = 20
>>>>>> ? vsz_limit = 2 G
>>>>>> }
>>>>>> service pop3-login {
>>>>>> ? process_limit = 2000
>>>>>> ? service_count = 0
>>>>>> ? vsz_limit = 512 M
>>>>>> }
>>>>>> service pop3 {
>>>>>> ? client_limit = 1
>>>>>> ? process_limit = 2000
>>>>>> ? vsz_limit = 1 G
>>>>>> }
>>>>>> service stats {
>>>>>> ? fifo_listener stats-mail {
>>>>>> ??? mode = 0600
>>>>>> ??? user = vmail
>>>>>> ? }
>>>>>> }
>>>>>> ssl_cert = </etc/dovecot/private/dovecot.pem
>>>>>> ssl_client_ca_dir = /etc/ssl/certs
>>>>>> ssl_key = # hidden, use -P to show it
>>>>>> submission_host = smtp4.artfiles.de
>>>>>> syslog_facility = local5
>>>>>> userdb {
>>>>>> ? args = /etc/dovecot/dovecot-sql.conf.ext
>>>>>> ? driver = sql
>>>>>> }
>>>>>> protocol lmtp {
>>>>>> ? mail_plugins = " mail_log notify old_stats
sieve quota"
>>>>>> }
>>>>>> protocol imap {
>>>>>> ? mail_max_userip_connections = 50
>>>>>> ? mail_plugins = " mail_log notify old_stats
quota imap_quota
>>>>>> imap_sieve zlib imap_old_stats"
>>>>>> ? rawlog_dir = /var/log/rawlog/%u
>>>>>> }
>>>>>> protocol pop3 {
>>>>>> ? mail_max_userip_connections = 50
>>>>>> ? mail_plugins = " mail_log notify old_stats
zlib"
>>>>>> }
>>>>> It will probably compound on the issue though, especially
when rebuilding caches and indexes, as it will record the sizes wrong. I'm
not sure if this is *really* the issue, but it's most likely culprit. The
8192 is very odd number to have there, which either indicates that the zlib
plugin is not consistently used, or you have some issues with the gz library
itself.
>>>>>
>>>>> Aki
>>>> Not sure if I understand this - shouldn't dovecot always
note the
>>>> decompressed size in cache and filename regardless if it's
compressed or
>>>> not?
>>>>
>>>> I just removed the zlib plugin from lmtp to reduce the
footprint of the
>>>> problem. The problem itself showed up first while lmtp zlib was
on.
>>>>
>>>> Tim
>>> It is complaining about the **physical** size though, that is, the
amount stored on disk. If you have a dovecot process which is unaware of
compression, it can record the wrong size for the file.
>>>
>>> Aki
>> OK, I see. Another thing I noticed: The uidlist entry for a suspicious
>> number of these problematic files contains what looks like a GUID:
>>
>> G1674674677.M641373P33277.node2
>>
>> If I remove the uidlist and let dovecot recreate it, it is usually
gone.
>> Might that be an indicator of some other process writing to the uidlist
>> file? the GUID is not mentioned in the dovecot docs as far as I can
see.
>>
>> Tim
> It sure sounds like that.
>
> Aki
OK, for future reference: I think I found the issue.
There was a periodic doveadm quotacheck job that was unaware of the zlib
compression.
I fixed that and no new corruption occured (though I still see errors
from already corrupted cache data - that will hopefully dry up over time).
However: I have still no idea why dovecot is complaining about the
index.* files. It seems that the source of the wrong data is the
dovecot-uidlist.
Also I don't see why the quotacheck cmd (doveadm quota recalc -u <>)
**sometimes** updates the dovecot-uidlist file. The quota itself is
stored in a sql dict.
Thanks to all who provided their input and especially Aki for the support!
Tim
