Hi, As some of you know, I started implementing the SMTP submission proxy a few years ago. It acts as a front-end for any MTA, adding the necessary functionality for an SMTP submission service, also known as a Mail Submission Agent (MSA) (https://tools.ietf.org/html/rfc6409). The main reason I created this, back then, was implementing the BURL capability (https://tools.ietf.org/html/rfc4468). The main application of that capability -- together with IMAP URLAUTH -- is avoiding a duplicate upload of submitted e-mail messages; normally the message is both sent through SMTP and uploaded to the "Sent" folder through IMAP. Using BURL, the client can first upload the message to IMAP and then use BURL to make the SMTP server fetch the message from IMAP for submission, thereby avoiding a second upload. Apart from BURL, the submission proxy service? also adds the required AUTH support, avoiding the need to configure the MTA for SASL authentication. More SMTP capabilities like CHUNKING and SIZE are supported, without requiring the backend MTA supporting these extensions. Other capabilities like DSN currently require support from the backend/relay MTA. At this point, the submission proxy is still pretty basic. However, it will provide a basis for adding all kinds of functionality in the (not so distant) future. For the first time, it will be possible to act upon message submission, rather than only message retrieval; e.g. plugins can be devised that process outgoing messages somehow. Examples of the things we could do are adding Sieve filtering support for outgoing messages, or implicitly storing submitted messages to the Sent folder. Once a plugin API is devised, you can create your own plugins. The reason I send this message now, is that this code is finally merged into the Dovecot master repository. This means that it is part of the upcoming 2.3 release. Now that it is merged, you can install and test it from Github if you like. Feedback is of course appreciated. The documentation is still pretty sparse, but there is currently not much to configure. Just add "submission" to the protocols and configure the relay MTA server. The configuration is currently only documented in the example configuration in doc/example-config/conf.d/20-submission.conf. The submission service is a login service, just like IMAP, POP3 and ManageSieve, so clients are required to authenticate. The same authentication configuration will also apply to submission, unless you're doing protocol-specific things, in which case you may need to amend your configuration for the new protocol. BURL support requires a working IMAP URLAUTH implementation. I've updated the automated Xi Debian package builder to create an additional dovecot-submissiond package. So, if you're using the Xi packages, you only need to install that package and configure the relay MTA. Regards, Stephan.
This is fantastic Stephan! Especially since I'll soon be rolling a new Dovecot server to act as a backup for our current Office 365 mail, as well as to be prepared in case I can ever talk the boss into migrating back to dovecot (we were using Dovecot for a really long time until he was convinced by others that we 'had' to be on Office 365). I'll also be setting up a shiny new VPS for my own private mail server, to provide better service that I currently get from my shared Dreamhost account. But can you confirm... Would this also be called 'BURL' support? And will this initial implementation work with current Postfix to provide the basic Save-To-Sent feature? I seem to recall there was some minor code required on the Postfix side, and Wietse seemed to not have a problem implementing it, but had asked about any IMAP Clients supporting BURL... If this won't work with current versions of Postfix, maybe you or Timo or someone could go poke him now that Dovecot supports it? Anyway, thanks again, I'm really looking forward to being able to do away with the 'Save-To-Sent' wastage. On 12/11/2017, 6:14:26 PM, Stephan Bosch <stephan at rename-it.nl> wrote:> Hi, > > As some of you know, I started implementing the SMTP submission proxy a > few years ago. It acts as a front-end for any MTA, adding the necessary > functionality for an SMTP submission service, also known as a Mail > Submission Agent (MSA) (https://tools.ietf.org/html/rfc6409). The main > reason I created this, back then, was implementing the BURL capability > (https://tools.ietf.org/html/rfc4468). The main application of that > capability -- together with IMAP URLAUTH -- is avoiding a duplicate > upload of submitted e-mail messages; normally the message is both sent > through SMTP and uploaded to the "Sent" folder through IMAP. Using BURL, > the client can first upload the message to IMAP and then use BURL to > make the SMTP server fetch the message from IMAP for submission, thereby > avoiding a second upload. Apart from BURL, the submission proxy service? > also adds the required AUTH support, avoiding the need to configure the > MTA for SASL authentication. More SMTP capabilities like CHUNKING and > SIZE are supported, without requiring the backend MTA supporting these > extensions. Other capabilities like DSN currently require support from > the backend/relay MTA. > > At this point, the submission proxy is still pretty basic. However, it > will provide a basis for adding all kinds of functionality in the (not > so distant) future. For the first time, it will be possible to act upon > message submission, rather than only message retrieval; e.g. plugins can > be devised that process outgoing messages somehow. Examples of the > things we could do are adding Sieve filtering support for outgoing > messages, or implicitly storing submitted messages to the Sent folder. > Once a plugin API is devised, you can create your own plugins. > > The reason I send this message now, is that this code is finally merged > into the Dovecot master repository. This means that it is part of the > upcoming 2.3 release. Now that it is merged, you can install and test it > from Github if you like. Feedback is of course appreciated. The > documentation is still pretty sparse, but there is currently not much to > configure. Just add "submission" to the protocols and configure the > relay MTA server. The configuration is currently only documented in the > example configuration in doc/example-config/conf.d/20-submission.conf. > The submission service is a login service, just like IMAP, POP3 and > ManageSieve, so clients are required to authenticate. The same > authentication configuration will also apply to submission, unless > you're doing protocol-specific things, in which case you may need to > amend your configuration for the new protocol. BURL support requires a > working IMAP URLAUTH implementation. > > I've updated the automated Xi Debian package builder to create an > additional dovecot-submissiond package. So, if you're using the Xi > packages, you only need to install that package and configure the relay MTA.
Op 12-12-2017 om 15:28 schreef Tanstaafl:> This is fantastic Stephan! Especially since I'll soon be rolling a new > Dovecot server to act as a backup for our current Office 365 mail, as > well as to be prepared in case I can ever talk the boss into migrating > back to dovecot (we were using Dovecot for a really long time until he > was convinced by others that we 'had' to be on Office 365). > > I'll also be setting up a shiny new VPS for my own private mail server, > to provide better service that I currently get from my shared Dreamhost > account. > > But can you confirm... > > Would this also be called 'BURL' support? > > And will this initial implementation work with current Postfix to > provide the basic Save-To-Sent feature? > > I seem to recall there was some minor code required on the Postfix side, > and Wietse seemed to not have a problem implementing it, but had asked > about any IMAP Clients supporting BURL...The thing is: with the Dovecot submission proxy, the availability of BURL support at Postfix/Exim/Sendmail is irrelevant. Dovecot handles the full SMTP BURL/IMAP URLAUTH functionality without MTA involvement. That should make configuration in most cases quite simple. The current implementation does require that the proxy has direct access to the user's mailbox for BURL (e.g. by running it on the same host as imap), but that restriction should be resolved soon, allowing for more complex setups. However, keep in mind that for this particular feature we're just providing the "chicken" as it were. The "egg", i.e. client support, is still to come. Apart from Trojita (which I think is still not widely used), I know of no IMAP client supporting BURL/URLAUTH for message submission. I'd expect to see it first for clients that can truly benefit; i.e., mobile clients such as K9. Regards, Stephan.> On 12/11/2017, 6:14:26 PM, Stephan Bosch <stephan at rename-it.nl> wrote: >> Hi, >> >> As some of you know, I started implementing the SMTP submission proxy a >> few years ago. It acts as a front-end for any MTA, adding the necessary >> functionality for an SMTP submission service, also known as a Mail >> Submission Agent (MSA) (https://tools.ietf.org/html/rfc6409). The main >> reason I created this, back then, was implementing the BURL capability >> (https://tools.ietf.org/html/rfc4468). The main application of that >> capability -- together with IMAP URLAUTH -- is avoiding a duplicate >> upload of submitted e-mail messages; normally the message is both sent >> through SMTP and uploaded to the "Sent" folder through IMAP. Using BURL, >> the client can first upload the message to IMAP and then use BURL to >> make the SMTP server fetch the message from IMAP for submission, thereby >> avoiding a second upload. Apart from BURL, the submission proxy service >> also adds the required AUTH support, avoiding the need to configure the >> MTA for SASL authentication. More SMTP capabilities like CHUNKING and >> SIZE are supported, without requiring the backend MTA supporting these >> extensions. Other capabilities like DSN currently require support from >> the backend/relay MTA. >> >> At this point, the submission proxy is still pretty basic. However, it >> will provide a basis for adding all kinds of functionality in the (not >> so distant) future. For the first time, it will be possible to act upon >> message submission, rather than only message retrieval; e.g. plugins can >> be devised that process outgoing messages somehow. Examples of the >> things we could do are adding Sieve filtering support for outgoing >> messages, or implicitly storing submitted messages to the Sent folder. >> Once a plugin API is devised, you can create your own plugins. >> >> The reason I send this message now, is that this code is finally merged >> into the Dovecot master repository. This means that it is part of the >> upcoming 2.3 release. Now that it is merged, you can install and test it >> from Github if you like. Feedback is of course appreciated. The >> documentation is still pretty sparse, but there is currently not much to >> configure. Just add "submission" to the protocols and configure the >> relay MTA server. The configuration is currently only documented in the >> example configuration in doc/example-config/conf.d/20-submission.conf. >> The submission service is a login service, just like IMAP, POP3 and >> ManageSieve, so clients are required to authenticate. The same >> authentication configuration will also apply to submission, unless >> you're doing protocol-specific things, in which case you may need to >> amend your configuration for the new protocol. BURL support requires a >> working IMAP URLAUTH implementation. >> >> I've updated the automated Xi Debian package builder to create an >> additional dovecot-submissiond package. So, if you're using the Xi >> packages, you only need to install that package and configure the relay MTA.
Am 12.12.2017 um 00:14 schrieb Stephan Bosch:> Hi, > > As some of you know, I started implementing the SMTP submission proxy a > few years ago. It acts as a front-end for any MTA, adding the necessary > functionality for an SMTP submission service, also known as a Mail > Submission Agent (MSA) (https://tools.ietf.org/html/rfc6409). The main > reason I created this, back then, was implementing the BURL capability > (https://tools.ietf.org/html/rfc4468). The main application of that > capability -- together with IMAP URLAUTH -- is avoiding a duplicate > upload of submitted e-mail messages; normally the message is both sent > through SMTP and uploaded to the "Sent" folder through IMAP. Using BURL, > the client can first upload the message to IMAP and then use BURL to > make the SMTP server fetch the message from IMAP for submission, thereby > avoiding a second upload. Apart from BURL, the submission proxy service? > also adds the required AUTH support, avoiding the need to configure the > MTA for SASL authentication. More SMTP capabilities like CHUNKING and > SIZE are supported, without requiring the backend MTA supporting these > extensions. Other capabilities like DSN currently require support from > the backend/relay MTA. > > At this point, the submission proxy is still pretty basic. However, it > will provide a basis for adding all kinds of functionality in the (not > so distant) future. For the first time, it will be possible to act upon > message submission, rather than only message retrieval; e.g. plugins can > be devised that process outgoing messages somehow. Examples of the > things we could do are adding Sieve filtering support for outgoing > messages, or implicitly storing submitted messages to the Sent folder. > Once a plugin API is devised, you can create your own plugins. > > The reason I send this message now, is that this code is finally merged > into the Dovecot master repository. This means that it is part of the > upcoming 2.3 release. Now that it is merged, you can install and test it > from Github if you like. Feedback is of course appreciated. The > documentation is still pretty sparse, but there is currently not much to > configure. Just add "submission" to the protocols and configure the > relay MTA server. The configuration is currently only documented in the > example configuration in doc/example-config/conf.d/20-submission.conf. > The submission service is a login service, just like IMAP, POP3 and > ManageSieve, so clients are required to authenticate. The same > authentication configuration will also apply to submission, unless > you're doing protocol-specific things, in which case you may need to > amend your configuration for the new protocol. BURL support requires a > working IMAP URLAUTH implementation. > > I've updated the automated Xi Debian package builder to create an > additional dovecot-submissiond package. So, if you're using the Xi > packages, you only need to install that package and configure the relay MTA. > > Regards, > > Stephan. > > > > >Hi Stephan, this is extreme cool ! Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Schlei?heimer Stra?e 26/MG, 80333 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
??? Stephan, thank you very much for your hard work. I want to ask your opinion about jmap ( http://jmap.io/ ) , do you think is a viable alternative to current IMAP + MSA ? ??? Regards ??? Mar?a El 12/12/17 a las 00:14, Stephan Bosch escribi?:> Hi, > > As some of you know, I started implementing the SMTP submission proxy a > few years ago. It acts as a front-end for any MTA, adding the necessary > functionality for an SMTP submission service, also known as a Mail > Submission Agent (MSA) (https://tools.ietf.org/html/rfc6409). The main > reason I created this, back then, was implementing the BURL capability > (https://tools.ietf.org/html/rfc4468). The main application of that > capability -- together with IMAP URLAUTH -- is avoiding a duplicate > upload of submitted e-mail messages; normally the message is both sent > through SMTP and uploaded to the "Sent" folder through IMAP. Using BURL, > the client can first upload the message to IMAP and then use BURL to > make the SMTP server fetch the message from IMAP for submission, thereby > avoiding a second upload. Apart from BURL, the submission proxy service > also adds the required AUTH support, avoiding the need to configure the > MTA for SASL authentication. More SMTP capabilities like CHUNKING and > SIZE are supported, without requiring the backend MTA supporting these > extensions. Other capabilities like DSN currently require support from > the backend/relay MTA. > > At this point, the submission proxy is still pretty basic. However, it > will provide a basis for adding all kinds of functionality in the (not > so distant) future. For the first time, it will be possible to act upon > message submission, rather than only message retrieval; e.g. plugins can > be devised that process outgoing messages somehow. Examples of the > things we could do are adding Sieve filtering support for outgoing > messages, or implicitly storing submitted messages to the Sent folder. > Once a plugin API is devised, you can create your own plugins. > > The reason I send this message now, is that this code is finally merged > into the Dovecot master repository. This means that it is part of the > upcoming 2.3 release. Now that it is merged, you can install and test it > from Github if you like. Feedback is of course appreciated. The > documentation is still pretty sparse, but there is currently not much to > configure. Just add "submission" to the protocols and configure the > relay MTA server. The configuration is currently only documented in the > example configuration in doc/example-config/conf.d/20-submission.conf. > The submission service is a login service, just like IMAP, POP3 and > ManageSieve, so clients are required to authenticate. The same > authentication configuration will also apply to submission, unless > you're doing protocol-specific things, in which case you may need to > amend your configuration for the new protocol. BURL support requires a > working IMAP URLAUTH implementation. > > I've updated the automated Xi Debian package builder to create an > additional dovecot-submissiond package. So, if you're using the Xi > packages, you only need to install that package and configure the relay MTA. > > Regards, > > Stephan. > > > > >
Op 14-12-2017 om 8:26 schreef Mar?a Arrea:> > ??? Stephan, thank you very much for your hard work. I want to ask > your opinion about jmap ( http://jmap.io/ ) , do you think is a viable > alternative to current IMAP + MSA ? >Difficult to tell at this point, as It is not finished yet. It all depends on whether people will want to implement it. Still, technology-wise I think it is solid and, as far as I know now, we'll be implementing it at some point. Regards, Stephan.> Regards > > ??? Mar?a > > El 12/12/17 a las 00:14, Stephan Bosch escribi?: >> Hi, >> >> As some of you know, I started implementing the SMTP submission proxy a >> few years ago. It acts as a front-end for any MTA, adding the necessary >> functionality for an SMTP submission service, also known as a Mail >> Submission Agent (MSA) (https://tools.ietf.org/html/rfc6409). The main >> reason I created this, back then, was implementing the BURL capability >> (https://tools.ietf.org/html/rfc4468). The main application of that >> capability -- together with IMAP URLAUTH -- is avoiding a duplicate >> upload of submitted e-mail messages; normally the message is both sent >> through SMTP and uploaded to the "Sent" folder through IMAP. Using BURL, >> the client can first upload the message to IMAP and then use BURL to >> make the SMTP server fetch the message from IMAP for submission, thereby >> avoiding a second upload. Apart from BURL, the submission proxy service >> also adds the required AUTH support, avoiding the need to configure the >> MTA for SASL authentication. More SMTP capabilities like CHUNKING and >> SIZE are supported, without requiring the backend MTA supporting these >> extensions. Other capabilities like DSN currently require support from >> the backend/relay MTA. >> >> At this point, the submission proxy is still pretty basic. However, it >> will provide a basis for adding all kinds of functionality in the (not >> so distant) future. For the first time, it will be possible to act upon >> message submission, rather than only message retrieval; e.g. plugins can >> be devised that process outgoing messages somehow. Examples of the >> things we could do are adding Sieve filtering support for outgoing >> messages, or implicitly storing submitted messages to the Sent folder. >> Once a plugin API is devised, you can create your own plugins. >> >> The reason I send this message now, is that this code is finally merged >> into the Dovecot master repository. This means that it is part of the >> upcoming 2.3 release. Now that it is merged, you can install and test it >> from Github if you like. Feedback is of course appreciated. The >> documentation is still pretty sparse, but there is currently not much to >> configure. Just add "submission" to the protocols and configure the >> relay MTA server. The configuration is currently only documented in the >> example configuration in doc/example-config/conf.d/20-submission.conf. >> The submission service is a login service, just like IMAP, POP3 and >> ManageSieve, so clients are required to authenticate. The same >> authentication configuration will also apply to submission, unless >> you're doing protocol-specific things, in which case you may need to >> amend your configuration for the new protocol. BURL support requires a >> working IMAP URLAUTH implementation. >> >> I've updated the automated Xi Debian package builder to create an >> additional dovecot-submissiond package. So, if you're using the Xi >> packages, you only need to install that package and configure the >> relay MTA. >> >> Regards, >> >> Stephan. >> >> >> >> >> >