dovecot - Nov 2016 - lazy-load SNI?

On 11.11.2016 12:22, Arkadiusz Mi?kiewicz wrote:
> On Friday 11 of November 2016, Felipe Gasper wrote:
>> Hello,
>>
>> 	We?re rolling out large SNI deployments for our mail servers. Each
domain
>> gets an entry like this in the config:
>>
>> local_name mail.foo.com {
>>      ssl_cert = </ssl/domain_tls/*.foo.com/combined
>>      ssl_key = </ssl/domain_tls/*.foo.com/combined
>> }
> Lack of glob/regexp support here is also a problem (for me). I could have
50%
> smaller config if local_name supported regexp matching, so it would be
> possible to do:
>
> local_name ^(pop3|imap)\.foo\.com {
> ...
> }
>
> or even with glob like *.foo.com matching.
>
>> 	There are a couple problems we?re finding with this approach:
>>
>> 1) Dovecot wants to load everything at once, which has some machines
taking
>> up many GiB of memory just for Dovecot. Is there any way to defer
loading
>> of an SSL cert until a client actually requests it?
> No - thread here
http://www.dovecot.org/list/dovecot/2016-October/105855.html
>
> Memory is one thing.
>
> The other is that dovecot stops accepting clients when huge config reload
> happens (I guess it's a design problem since it makes no sense to do
that in
> any case. Clients should be processed without gap using old config until
new
> config is loaded and ready to go).
>
> And third problem is that there is hardcoded 10s limit for reloading which
in
> case thousands of certificates is way too short limit. Anyway if you hit
that
> limit it's already lost case due to earlier problem.
>
>> 2) Any time we add or remove a domain, Dovecot?s SNI config matrix
needs to
>> be rebuilt. Is there a way to handle SNI requests dynamically via some
>> sort of configuration plugin, so we wouldn?t need to rebuild the config
on
>> domain add/remove? I looked through the docs but couldn?t see a way to
do
>> this.
> That's unavoidable for now :-(
>
> Here we started analyzing maillog and put into dovecot config only these
ssl
> certs for domains that are actually used with TLS. It's very ugly and
short-
> sighted approach but hopefuly proper solution will be implemented by
dovecot
> team before all people start to use TLS.
>   
>> 	Thank you in advance!
>>
>> -Felipe Gasper
>> Mississauga, ON
>
If you are interested in testing, please find patch attached that allows 
you to specify

local_name *.foo.bar {
}

or

local_name *.*.foo.bar {
}

so basically you can now use certificate name matching rules for 
local_name. It made most sense.

This should apply cleanly to 2.2.26.0.

---
Aki Tuomi
Dovecot oy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-lib-dns-Add-DNS-specific-matching-algorithms.patch
Type: text/x-patch
Size: 4450 bytes
Desc: not available
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20161111/e8b9ee41/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-lib-dns-Add-tests-for-dns-util.patch
Type: text/x-patch
Size: 4020 bytes
Desc: not available
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20161111/e8b9ee41/attachment-0004.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0003-config-Match-local_name-using-dns-util.patch
Type: text/x-patch
Size: 1487 bytes
Desc: not available
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20161111/e8b9ee41/attachment-0005.bin>

On Friday 11 of November 2016, Aki Tuomi wrote:
> If you are interested in testing, please find patch attached that allows
> you to specify
> 
> local_name *.foo.bar {
> }
> 
> or
> 
> local_name *.*.foo.bar {
> }
> 
> so basically you can now use certificate name matching rules for
> local_name. It made most sense.
Great! Seems to be working fine for my usage and makes my configs 50% smaller 
(which is gigantic improvement). Will do more testing though.

Thanks!



What about dovecot stopping processing new clients when reload is in progress 
problem -  is it possible to make it behave better? To minimize (or avoid) 
"downtime".

How to reproduce - just create config file with 20 000 - 50 000 entries

local_name hostX....example.com {
  ssl_cert = </etc/certs/cert.pem
  ssl_key = </etc/certs/cert.pem
}

where cert.pem contains some full chain (CA cert + intermediate + cert + key).

Start dovecot and then doveadm reload should take long time. Enough for 
noticing that dovecot stops processing clients.
> Aki Tuomi
> Dovecot oy
-- 
Arkadiusz Mi?kiewicz, arekm / ( maven.pl | pld-linux.org )

On 11.11.2016 19:17, Arkadiusz Mi?kiewicz wrote:
> On Friday 11 of November 2016, Aki Tuomi wrote:
>
>> If you are interested in testing, please find patch attached that
allows
>> you to specify
>>
>> local_name *.foo.bar {
>> }
>>
>> or
>>
>> local_name *.*.foo.bar {
>> }
>>
>> so basically you can now use certificate name matching rules for
>> local_name. It made most sense.
> Great! Seems to be working fine for my usage and makes my configs 50%
smaller
> (which is gigantic improvement). Will do more testing though.
>
> Thanks!
>
>
>
> What about dovecot stopping processing new clients when reload is in
progress
> problem -  is it possible to make it behave better? To minimize (or avoid)
> "downtime".
>
> How to reproduce - just create config file with 20 000 - 50 000 entries
>
> local_name hostX....example.com {
>    ssl_cert = </etc/certs/cert.pem
>    ssl_key = </etc/certs/cert.pem
> }
>
> where cert.pem contains some full chain (CA cert + intermediate + cert +
key).
>
> Start dovecot and then doveadm reload should take long time. Enough for
> noticing that dovecot stops processing clients.
>
>> Aki Tuomi
>> Dovecot oy
That is something that will happen later. Can't give any date, but it's 
in our internal tasklist.

Aki

> On Nov 11, 2016, at 9:06 AM, Aki Tuomi <aki.tuomi at dovecot.fi>
wrote:
> 
> If you are interested in testing, please find patch attached that allows
you to specify
> 
> local_name *.foo.bar {
> }
> 
> or
> 
> local_name *.*.foo.bar {
> }
> 
Dear Aki et al.,

	How straightforward would it be to implement the following or similar syntax:

local_name foo.tld www.foo.tld mail.foo.tld bar.tld {
    ...
}

??

	Thank you!

-FG