Not sure what you mean. I?m using lmtp to send messages to Dovecot from Postfix.> On 10 Jun 2017, at 6:08 pm, Aki Tuomi <aki.tuomi at dovecot.fi> wrote: > > What's your LDA? > > Aki > >> On June 10, 2017 at 11:01 AM Peter West <lists at pbw.id.au> wrote: >> >> >> Thanks for that Aki. >> >> Follow-up question. I tried to initiate compression by adding >> >> mail_plugins = $mail_plugins zlib >> >> plugin { >> zlib_save_level = 6 >> zlib_save = xz >> } >> >> >> to dovecot.conf. I restarted dovecot and sent one message to the server, and one message from the server. Neither was compressed. I changed the save type to >> >> zlib_save = bz2 >> >> and repeated. This time the message received (in /var/vmail/<host>/<username>/cur) was not compressed, but the message in /var/vmail/<host>/<username>/.Sent/cur was bzip2 compressed. >> >> Why is the received mail not being compressed? Is this the point of the discussion about compressing old mails? >> >> >>> On 10 Jun 2017, at 4:43 pm, Aki Tuomi <aki.tuomi at dovecot.fi> wrote: >>> >>> >>>> On June 10, 2017 at 5:58 AM Peter West <lists at pbw.id.au> wrote: >>>> >>>> >>>> Concerning Maildir, the wiki page on compression has this: >>>> >>>> All mails must have ,S=<size> in their filename where <size> contains the original uncompressed mail size, otherwise there will be problems with quota calculation as well as other potential random failures. Note that if the filename doesn?t contain the ,S=<size> before compression, adding it afterwards changes the base filename and thus the message UID. The safest thing to do is simply to not compress such files. >>>> >>>> Further down on the same page is this: >>>> >>>> If the file does exist, rename() (mv) the compressed file over the original file. >>>> ? Dovecot can now read the file, but to avoid compressing it again on the next run, you'll probably want to rename it again to include e.g. a "Z" flag in the file name to mark that it was compressed (e.g. 1223212411.M907959P17184.host,S=3271:2,SZ). >>>> >>>> These comments seem to contradict each. Or is there a difference between adding the size specifier to the filename and adding a Z flag to the end of the file name? >>>> >>>> -- >>>> Peter West >>>> pbw at pbw.id.au >>>> And the great throng heard him gladly. >>>> >>> >>> Keyword is 'base filename'. From the wiki, "The standard filename definition is: "<base filename>:2,<flags>".". Z is a flag. >>> >>> Aki >>-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 235 bytes Desc: Message signed with OpenPGP URL: <http://dovecot.org/pipermail/dovecot/attachments/20170610/1a653fb0/attachment.sig>
Please check that you are not overwriting mail plugins for lmtp. Or post your doveconf -n. Aki> On June 10, 2017 at 11:10 AM Peter West <lists at pbw.id.au> wrote: > > > Not sure what you mean. I?m using lmtp to send messages to Dovecot from Postfix. > > > On 10 Jun 2017, at 6:08 pm, Aki Tuomi <aki.tuomi at dovecot.fi> wrote: > > > > What's your LDA? > > > > Aki > > > >> On June 10, 2017 at 11:01 AM Peter West <lists at pbw.id.au> wrote: > >> > >> > >> Thanks for that Aki. > >> > >> Follow-up question. I tried to initiate compression by adding > >> > >> mail_plugins = $mail_plugins zlib > >> > >> plugin { > >> zlib_save_level = 6 > >> zlib_save = xz > >> } > >> > >> > >> to dovecot.conf. I restarted dovecot and sent one message to the server, and one message from the server. Neither was compressed. I changed the save type to > >> > >> zlib_save = bz2 > >> > >> and repeated. This time the message received (in /var/vmail/<host>/<username>/cur) was not compressed, but the message in /var/vmail/<host>/<username>/.Sent/cur was bzip2 compressed. > >> > >> Why is the received mail not being compressed? Is this the point of the discussion about compressing old mails? > >> > >> > >>> On 10 Jun 2017, at 4:43 pm, Aki Tuomi <aki.tuomi at dovecot.fi> wrote: > >>> > >>> > >>>> On June 10, 2017 at 5:58 AM Peter West <lists at pbw.id.au> wrote: > >>>> > >>>> > >>>> Concerning Maildir, the wiki page on compression has this: > >>>> > >>>> All mails must have ,S=<size> in their filename where <size> contains the original uncompressed mail size, otherwise there will be problems with quota calculation as well as other potential random failures. Note that if the filename doesn?t contain the ,S=<size> before compression, adding it afterwards changes the base filename and thus the message UID. The safest thing to do is simply to not compress such files. > >>>> > >>>> Further down on the same page is this: > >>>> > >>>> If the file does exist, rename() (mv) the compressed file over the original file. > >>>> ? Dovecot can now read the file, but to avoid compressing it again on the next run, you'll probably want to rename it again to include e.g. a "Z" flag in the file name to mark that it was compressed (e.g. 1223212411.M907959P17184.host,S=3271:2,SZ). > >>>> > >>>> These comments seem to contradict each. Or is there a difference between adding the size specifier to the filename and adding a Z flag to the end of the file name? > >>>> > >>>> -- > >>>> Peter West > >>>> pbw at pbw.id.au > >>>> And the great throng heard him gladly. > >>>> > >>> > >>> Keyword is 'base filename'. From the wiki, "The standard filename definition is: "<base filename>:2,<flags>".". Z is a flag. > >>> > >>> Aki > >> >
Well spotted.
In my first attempt, the configuration I originally posted was at the end of my
dovecot.conf file, and I was not including the conf.d configuration files.
Preceding that was my lmtp protocol cong.
protocol lmtp {
postmaster_address = postmaster
# Space separated list of plugins to load (default is global mail_plugins).
mail_plugins = $mail_plugins sieve
}
>>>>
>>>> mail_plugins = $mail_plugins zlib
>>>>
>>>> plugin {
>>>> zlib_save_level = 6
>>>> zlib_save = xz
>>>> }
>>>>
The doveconf -n associated with this is as follows:
01 # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
02 # Pigeonhole version 0.4.13 (7b14904)
03 # OS: Linux 4.9.15-x86_64-linode81 x86_64 Ubuntu 16.04.2 LTS
04 auth_mechanisms = plain login
05 log_timestamp = "%Y-%m-%d %H:%M:%S "
06 mail_gid = vmail
07 mail_home = maildir:/var/vmail/%d/%n
08 mail_location = maildir:~/Maildir
09
10 mail_privileged_group = vmail
11 mail_uid = vmail
12 managesieve_notify_capability = mailto
13 managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date ihave
14 namespace inbox {
15 inbox = yes
16 location 17 mailbox Archive {
18 auto = subscribe
19 special_use = \Archive
20 }
21 mailbox Drafts {
22 auto = subscribe
23 special_use = \Drafts
24 }
25 mailbox Sent {
26 auto = subscribe
27 special_use = \Sent
28 }
29 mailbox Spam {
30 auto = subscribe
31 special_use = \Junk
32 }
33 mailbox Trash {
34 auto = subscribe
35 special_use = \Trash
36 }
37 prefix 38 }
39 passdb {
40 args = /etc/dovecot/dovecot-sql.conf.ext
41 driver = sql
42 }
43 plugin {
44 sieve = ~/.dovecot.sieve
45 sieve_dir = ~/sieve
46 zlib_save = bz2
47 zlib_save_level = 6
48 }
49 protocols = " imap lmtp sieve"
50 service auth {
51 unix_listener /var/spool/postfix/private/dovecot-auth {
52 group = postfix
53 mode = 0660
54 user = postfix
55 }
56 user = root
57 }
58 service lmtp {
59 unix_listener /var/spool/postfix/private/dovecot-lmtp {
60 group = postfix
61 mode = 0600
62 user = postfix
63 }
64 }
65 ssl_ca = </etc/ssl/repeal18c.today/fullchain.pem
66 ssl_cert = </etc/ssl/repeal18c.today/cert.pem
67 ssl_cipher_list =
ALL:!LOW:!SSLv2:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM
68 ssl_key = </etc/ssl/repeal18c.today/privkey.pem
69 userdb {
70 args = /etc/dovecot/dovecot-sql.conf.ext
71 driver = sql
72 }
73 protocol imap {
74 imap_client_workarounds = delay-newmail
75 mail_max_userip_connections = 100
76 }
77 protocol lmtp {
78 mail_plugins = " sieve zlib"
79 postmaster_address = postmaster
80 }
(The blank line at 09 I have inserted.)
Note the mail_plugins for protocol lmtp at 77.
I changed the protocol lmtp settings in dovecot.conf as follows:
protocol lmtp {
postmaster_address = postmaster
mail_plugins = $mail_plugins sieve zlib
}
and I removed this line:
>>>> mail_plugins = $mail_plugins zlib
The resulting doveconf -n is:
01 # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
02 # Pigeonhole version 0.4.13 (7b14904)
03 # OS: Linux 4.9.15-x86_64-linode81 x86_64 Ubuntu 16.04.2 LTS
04 auth_mechanisms = plain login
05 log_timestamp = "%Y-%m-%d %H:%M:%S "
06 mail_gid = vmail
07 mail_home = maildir:/var/vmail/%d/%n
08 mail_location = maildir:~/Maildir
09 mail_plugins = " zlib"
10 mail_privileged_group = vmail
11 mail_uid = vmail
12 managesieve_notify_capability = mailto
13 managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date ihave
14 namespace inbox {
15 inbox = yes
16 location 17 mailbox Archive {
18 auto = subscribe
19 special_use = \Archive
20 }
21 mailbox Drafts {
22 auto = subscribe
23 special_use = \Drafts
24 }
25 mailbox Sent {
26 auto = subscribe
27 special_use = \Sent
28 }
29 mailbox Spam {
30 auto = subscribe
31 special_use = \Junk
32 }
33 mailbox Trash {
34 auto = subscribe
35 special_use = \Trash
36 }
37 prefix 38 }
39 passdb {
40 args = /etc/dovecot/dovecot-sql.conf.ext
41 driver = sql
42 }
43 plugin {
44 sieve = ~/.dovecot.sieve
45 sieve_dir = ~/sieve
46 zlib_save = bz2
47 zlib_save_level = 6
48 }
49 protocols = " imap lmtp sieve"
50 service auth {
51 unix_listener /var/spool/postfix/private/dovecot-auth {
52 group = postfix
53 mode = 0660
54 user = postfix
55 }
56 user = root
57 }
58 service lmtp {
59 unix_listener /var/spool/postfix/private/dovecot-lmtp {
60 group = postfix
61 mode = 0600
62 user = postfix
63 }
64 }
65 ssl_ca = </etc/ssl/repeal18c.today/fullchain.pem
66 ssl_cert = </etc/ssl/repeal18c.today/cert.pem
67 ssl_cipher_list =
ALL:!LOW:!SSLv2:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM
68 ssl_key = </etc/ssl/repeal18c.today/privkey.pem
69 userdb {
70 args = /etc/dovecot/dovecot-sql.conf.ext
71 driver = sql
72 }
73 protocol imap {
74 imap_client_workarounds = delay-newmail
75 mail_max_userip_connections = 100
76 }
77 protocol lmtp {
78 mail_plugins = " sieve"
79 postmaster_address = postmaster
80 }
Note line 09, and the modified value of mail_plugins at 78. These are the only
changes in doveconf -n.
The result is a reversal of the previous situation.
Received messages in /var/vmail/<host>/<username>/cur are
compressed;
sent messages in /var/vmail/<host>/<username>/.Sent/cur are not
compressed.
I assume that I need both specifications.
--
Peter West
pbw at pbw.id.au
And the great throng heard him gladly.
> On 10 Jun 2017, at 6:50 pm, Aki Tuomi <aki.tuomi at dovecot.fi>
wrote:
>
> Please check that you are not overwriting mail plugins for lmtp. Or post
your doveconf -n.
>
> Aki
>
>> On June 10, 2017 at 11:10 AM Peter West <lists at pbw.id.au>
wrote:
>>
>>
>> Not sure what you mean. I?m using lmtp to send messages to Dovecot from
Postfix.
>>
>>> On 10 Jun 2017, at 6:08 pm, Aki Tuomi <aki.tuomi at
dovecot.fi> wrote:
>>>
>>> What's your LDA?
>>>
>>> Aki
>>>
>>>> On June 10, 2017 at 11:01 AM Peter West <lists at
pbw.id.au> wrote:
>>>>
>>>>
>>>> Thanks for that Aki.
>>>>
>>>> Follow-up question. I tried to initiate compression by adding
>>>>
>>>> mail_plugins = $mail_plugins zlib
>>>>
>>>> plugin {
>>>> zlib_save_level = 6
>>>> zlib_save = xz
>>>> }
>>>>
>>>>
>>>> to dovecot.conf. I restarted dovecot and sent one message to
the server, and one message from the server. Neither was compressed. I changed
the save type to
>>>>
>>>> zlib_save = bz2
>>>>
>>>> and repeated. This time the message received (in
/var/vmail/<host>/<username>/cur) was not compressed, but the
message in /var/vmail/<host>/<username>/.Sent/cur was bzip2
compressed.
>>>>
>>>> Why is the received mail not being compressed? Is this the
point of the discussion about compressing old mails?
>>>>
>>>>
>>>>> On 10 Jun 2017, at 4:43 pm, Aki Tuomi <aki.tuomi at
dovecot.fi> wrote:
>>>>>
>>>>>
>>>>>> On June 10, 2017 at 5:58 AM Peter West <lists at
pbw.id.au> wrote:
>>>>>>
>>>>>>
>>>>>> Concerning Maildir, the wiki page on compression has
this:
>>>>>>
>>>>>> All mails must have ,S=<size> in their filename
where <size> contains the original uncompressed mail size, otherwise there
will be problems with quota calculation as well as other potential random
failures. Note that if the filename doesn?t contain the ,S=<size> before
compression, adding it afterwards changes the base filename and thus the message
UID. The safest thing to do is simply to not compress such files.
>>>>>>
>>>>>> Further down on the same page is this:
>>>>>>
>>>>>> If the file does exist, rename() (mv) the compressed
file over the original file.
>>>>>> ? Dovecot can now read the file, but to avoid
compressing it again on the next run, you'll probably want to rename it
again to include e.g. a "Z" flag in the file name to mark that it was
compressed (e.g. 1223212411.M907959P17184.host,S=3271:2,SZ).
>>>>>>
>>>>>> These comments seem to contradict each. Or is there a
difference between adding the size specifier to the filename and adding a Z flag
to the end of the file name?
>>>>>>
>>>>>> --
>>>>>> Peter West
>>>>>> pbw at pbw.id.au
>>>>>> And the great throng heard him gladly.
>>>>>>
>>>>>
>>>>> Keyword is 'base filename'. From the wiki,
"The standard filename definition is: "<base
filename>:2,<flags>".". Z is a flag.
>>>>>
>>>>> Aki
>>>>
>>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20170610/5c3de469/attachment.sig>
Ok, I added zlib to imap protocol.
protocol imap {
?
mail_plugins = $mail_plugins zlib
}
Now both imap and lmtp protocols have zlib plugin enabled, and both send and
receive mail is compressed.
Peter
> On 10 Jun 2017, at 6:50 pm, Aki Tuomi <aki.tuomi at dovecot.fi>
wrote:
>
> Please check that you are not overwriting mail plugins for lmtp. Or post
your doveconf -n.
>
> Aki
>
>> On June 10, 2017 at 11:10 AM Peter West <lists at pbw.id.au>
wrote:
>>
>>
>> Not sure what you mean. I?m using lmtp to send messages to Dovecot from
Postfix.
>>
>>> On 10 Jun 2017, at 6:08 pm, Aki Tuomi <aki.tuomi at
dovecot.fi> wrote:
>>>
>>> What's your LDA?
>>>
>>> Aki
>>>
>>>> On June 10, 2017 at 11:01 AM Peter West <lists at
pbw.id.au> wrote:
>>>>
>>>>
>>>> Thanks for that Aki.
>>>>
>>>> Follow-up question. I tried to initiate compression by adding
>>>>
>>>> mail_plugins = $mail_plugins zlib
>>>>
>>>> plugin {
>>>> zlib_save_level = 6
>>>> zlib_save = xz
>>>> }
>>>>
>>>>
>>>> to dovecot.conf. I restarted dovecot and sent one message to
the server, and one message from the server. Neither was compressed. I changed
the save type to
>>>>
>>>> zlib_save = bz2
>>>>
>>>> and repeated. This time the message received (in
/var/vmail/<host>/<username>/cur) was not compressed, but the
message in /var/vmail/<host>/<username>/.Sent/cur was bzip2
compressed.
>>>>
>>>> Why is the received mail not being compressed? Is this the
point of the discussion about compressing old mails?
>>>>
>>>>
>>>>> On 10 Jun 2017, at 4:43 pm, Aki Tuomi <aki.tuomi at
dovecot.fi> wrote:
>>>>>
>>>>>
>>>>>> On June 10, 2017 at 5:58 AM Peter West <lists at
pbw.id.au> wrote:
>>>>>>
>>>>>>
>>>>>> Concerning Maildir, the wiki page on compression has
this:
>>>>>>
>>>>>> All mails must have ,S=<size> in their filename
where <size> contains the original uncompressed mail size, otherwise there
will be problems with quota calculation as well as other potential random
failures. Note that if the filename doesn?t contain the ,S=<size> before
compression, adding it afterwards changes the base filename and thus the message
UID. The safest thing to do is simply to not compress such files.
>>>>>>
>>>>>> Further down on the same page is this:
>>>>>>
>>>>>> If the file does exist, rename() (mv) the compressed
file over the original file.
>>>>>> ? Dovecot can now read the file, but to avoid
compressing it again on the next run, you'll probably want to rename it
again to include e.g. a "Z" flag in the file name to mark that it was
compressed (e.g. 1223212411.M907959P17184.host,S=3271:2,SZ).
>>>>>>
>>>>>> These comments seem to contradict each. Or is there a
difference between adding the size specifier to the filename and adding a Z flag
to the end of the file name?
>>>>>>
>>>>>> --
>>>>>> Peter West
>>>>>> pbw at pbw.id.au
>>>>>> And the great throng heard him gladly.
>>>>>>
>>>>>
>>>>> Keyword is 'base filename'. From the wiki,
"The standard filename definition is: "<base
filename>:2,<flags>".". Z is a flag.
>>>>>
>>>>> Aki
>>>>
>>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20170610/6c84b1e5/attachment.sig>