dovecot - Feb 2023 - Dovecot gzip u.files on sdboxes

Hi,

I was wondering if it's possible to lighten the process of converting
sdbox mailboxes to zlib by, instead of using dsync (that brings much more io, 
copying from alt - and copying of unneeded data as we have many mails already 
compressed from activating zlib a couble of years ago) just compressing each
of the u files.

By compression of the u files I mean opening the uncompressed u file for 
reading, a u.temp file for writting and:

1. copy the dbox message header from u to u.tmp

2. compress the plain message and write it to u.tmp

3. fix the message_size_hex on the u.tmp dbox message header to the new, 
   compressed size

4. copy the dbox metadata from u to u.tmp with Zphysical_size added

5. copy the u stat times to u.tmp

6. replace u with u.tmp

Is the message_size_hex stored anywhere else (indexes) that may cause this to
not work?

Thank you.

> On 01/02/2023 14:07 EET Jose Celestino <japc at co.sapo.pt> wrote:
> 
>  
> Hi,
> 
> I was wondering if it's possible to lighten the process of converting
> sdbox mailboxes to zlib by, instead of using dsync (that brings much more
io,
> copying from alt - and copying of unneeded data as we have many mails
already
> compressed from activating zlib a couble of years ago) just compressing
each
> of the u files.
> 
> By compression of the u files I mean opening the uncompressed u file for 
> reading, a u.temp file for writting and:
> 
> 1. copy the dbox message header from u to u.tmp
> 
> 2. compress the plain message and write it to u.tmp
> 
> 3. fix the message_size_hex on the u.tmp dbox message header to the new, 
>    compressed size
> 
> 4. copy the dbox metadata from u to u.tmp with Zphysical_size added
> 
> 5. copy the u stat times to u.tmp
> 
> 6. replace u with u.tmp
> 
> Is the message_size_hex stored anywhere else (indexes) that may cause this
to
> not work?
> 
> Thank you.
The header around the u file is about the physical contents only, you can
rewrite that header as long as you preserve any other fields.

You can use `doveadm dump u.1` to see how it works.

Aki