This has now been fixed in main with https://github.com/dovecot/core/commit/2780f106e3b185981dd7aaf5cbf2e88daa2f7c64.patch Aki> On 13/06/2022 10:43 gravitini <dovecot at gravitini.com> wrote: > > > Please consider as critical (data loss) and recommend a warning is > issued for 2.3.19 users. > > > On 13/06/22 5:25 pm, Aki Tuomi wrote: > >> On 13/06/2022 02:09 gravitini <dovecot at gravitini.com> wrote: > >> > >> > >> Replying to: https://dovecot.org/pipermail/dovecot/2022-May/124816.html > >> > >> > >> Hi, > >> > >> Looking at the code (and tested via local build from source) it looks > >> like doveadm deduplicate in 2.3.19 can cause significant data loss. > >> > >> A 2022-02-11 commit removed key duplication resulting in undefined > >> behaviour which is often truncation of a mailbox to 67 entries. > >> (HASH_TABLE_MIN_SIZE) > >> > >> https://github.com/dovecot/core/commit/320844f50cd669b602d30210e2e5216f65d2050f?diff=split#diff-5842cf9d4248dc515d80ebb45575341b7d76832f979a8ac5f602784cb5b03f2cL121 > >> > >> diff --git a/src/doveadm/doveadm-mail-deduplicate.c > >> b/src/doveadm/doveadm-mail-deduplicate.c > >> > >> index caec758112..2152482876 100644 > >> --- a/src/doveadm/doveadm-mail-deduplicate.c > >> +++ b/src/doveadm/doveadm-mail-deduplicate.c > >> @@ -63,8 +63,10 @@ cmd_deduplicate_box(struct doveadm_mail_cmd_context > >> *_ctx, > >> ??????????????? if (key != NULL && *key != '\0') { > >> ??????????????????????? if (hash_table_lookup(hash, key) != NULL) > >> ??????????????????????????????? mail_expunge(mail); > >> -?????????????????????? else > >> +?????????????????????? else { > >> +?????????????????????????????? key = p_strdup(pool, key); > >> ??????????????????????????????? hash_table_insert(hash, key, > >> POINTER_CAST(1)); > >> +?????????????????????? } > >> ??????????????? } > >> ??????? } > > Thank you both for the report, we'll look into this! > > > > Aki
>>>>> "Aki" == Aki Tuomi <aki.tuomi at open-xchange.com> writes:Will 2.3.20 be released ASAP with this fix? Aki> This has now been fixed in main with Aki> https://github.com/dovecot/core/commit/2780f106e3b185981dd7aaf5cbf2e88daa2f7c64.patch Aki> Aki>> On 13/06/2022 10:43 gravitini <dovecot at gravitini.com> wrote: >> >> >> Please consider as critical (data loss) and recommend a warning is >> issued for 2.3.19 users. >> >> >> On 13/06/22 5:25 pm, Aki Tuomi wrote: >> >> On 13/06/2022 02:09 gravitini <dovecot at gravitini.com> wrote: >> >> >> >> >> >> Replying to: https://dovecot.org/pipermail/dovecot/2022-May/124816.html >> >> >> >> >> >> Hi, >> >> >> >> Looking at the code (and tested via local build from source) it looks >> >> like doveadm deduplicate in 2.3.19 can cause significant data loss. >> >> >> >> A 2022-02-11 commit removed key duplication resulting in undefined >> >> behaviour which is often truncation of a mailbox to 67 entries. >> >> (HASH_TABLE_MIN_SIZE) >> >> >> >> https://github.com/dovecot/core/commit/320844f50cd669b602d30210e2e5216f65d2050f?diff=split#diff-5842cf9d4248dc515d80ebb45575341b7d76832f979a8ac5f602784cb5b03f2cL121 >> >> >> >> diff --git a/src/doveadm/doveadm-mail-deduplicate.c >> >> b/src/doveadm/doveadm-mail-deduplicate.c >> >> >> >> index caec758112..2152482876 100644 >> >> --- a/src/doveadm/doveadm-mail-deduplicate.c >> >> +++ b/src/doveadm/doveadm-mail-deduplicate.c >> >> @@ -63,8 +63,10 @@ cmd_deduplicate_box(struct doveadm_mail_cmd_context >> >> *_ctx, >> >> ??????????????? if (key != NULL && *key != '\0') { >> >> ??????????????????????? if (hash_table_lookup(hash, key) != NULL) >> >> ??????????????????????????????? mail_expunge(mail); >> >> -?????????????????????? else >> >> +?????????????????????? else { >> >> +?????????????????????????????? key = p_strdup(pool, key); >> >> ??????????????????????????????? hash_table_insert(hash, key, >> >> POINTER_CAST(1)); >> >> +?????????????????????? } >> >> ??????????????? } >> >> ??????? } >> > Thank you both for the report, we'll look into this! >> > >> > Aki