Hi, Nearly every popular programming language has an LMTP/POP/IMAP implementation, most of them suck in many different ways. I don't know any server or library which provides a well-established, compatible protocol frontend with an open backend API, which could be used to easily make a custom storage backend for the LMTP/POP/IMAP frontend in any language, in any programming paradigm, irregardless of the frontend implementation. Dovecot is very close to this: it speaks IMAP to the backend, but you still have to make a nearly complete IMAP implementation, which is a PITA. Therefore I am curious about your opinion: what about an HTTP-based open backend (lib-storage) API to Dovecot? Some thoughts on this: 1. it should be a simple REST API 2. it should support efficient operations on both sides (client, server), for example listing a multi-million folder shouldn't be a single JSON response, but a JSON stream, which could be produced and parsed individually or in smaller batches efficiently 3. it should support capabilities, somewhat like the current imapc backend, so if the backend service suppports search, it should announce this capability and dovecot could offload these requests to it, but if it doesn't, dovecot could build its indices. 4. only the needed amount of POP/IMAP legacy should appear in the protocol 5. the operations should be asynchronous on Dovecot's side (to be scalable) 6. maybe even Dovecot's index-queries could be offloaded to the backend service (this way the local file system wouldn't be used at all and all caches could be stored in a distributed dictionary), this could also be a capability 7. it could support some kind of push (for IDLE-ing and the like) 8. it should be stateless If Dovecot could make this right, it could be the FUSE of the mail protocols, acting as a glue for experimental or production ready mail storage backends, without the need to write these backends in "Dovecot C" (I mean it's process and API structure). Of course HTTP here is not a strict requirement. It could be anything, which can be easy to implement and understand and is able to fulfill the needs. An example could be Dovecot's own dictionary protocol. What do you think about this?
On 08 Feb 2016, at 11:01, Nagy, Attila <bra at fsn.hu> wrote:> > Hi, > > Nearly every popular programming language has an LMTP/POP/IMAP implementation, most of them suck in many different ways. > I don't know any server or library which provides a well-established, compatible protocol frontend with an open backend API, which could be used to easily make a custom storage backend for the LMTP/POP/IMAP frontend in any language, in any programming paradigm, irregardless of the frontend implementation. > > Dovecot is very close to this: it speaks IMAP to the backend, but you still have to make a nearly complete IMAP implementation, which is a PITA. > > Therefore I am curious about your opinion: what about an HTTP-based open backend (lib-storage) API to Dovecot?We're planning on implementing http://jmap.io/ to Dovecot v2.3. It's close enough to how Dovecot works internally (although we'd still need the threading support) and also at least attempting to become a standard. We haven't really started it yet though, so now would be a good time to give any alternative suggestions or complaints about jmap :)
> On 08 Feb 2016, at 11:59, Timo Sirainen <tss at iki.fi> wrote: > > On 08 Feb 2016, at 11:01, Nagy, Attila <bra at fsn.hu> wrote: >> >> Hi, >> >> Nearly every popular programming language has an LMTP/POP/IMAP implementation, most of them suck in many different ways. >> I don't know any server or library which provides a well-established, compatible protocol frontend with an open backend API, which could be used to easily make a custom storage backend for the LMTP/POP/IMAP frontend in any language, in any programming paradigm, irregardless of the frontend implementation. >> >> Dovecot is very close to this: it speaks IMAP to the backend, but you still have to make a nearly complete IMAP implementation, which is a PITA. >> >> Therefore I am curious about your opinion: what about an HTTP-based open backend (lib-storage) API to Dovecot? > > We're planning on implementing http://jmap.io/ to Dovecot v2.3. It's close enough to how Dovecot works internally (although we'd still need the threading support) and also at least attempting to become a standard. We haven't really started it yet though, so now would be a good time to give any alternative suggestions or complaints about jmap :)Oh, you were thinking about ability to provide IMAP/etc support for other random servers, and have Dovecot act as kind of a middleware and translate the requests. Maybe the answer is still jmap though? It would require jmap lib-storage backend similar to imapc, which would be doable, although not really something we're right now planning to implement.
that sounds like s3 API based backend.
also note that dovecot comes with some "non local mail storage
plugins"
in the commercial part:
```
In this release, we support Windows Azure, Amazon S3, Scality and
Dropbox systems.
```
so that should come close to what you want. for a local version you
could use e.g. radosdb/ceph or riak. both can provide the s3 API for
you.
hth
darix
--
openSUSE - SUSE Linux is my linux
openSUSE is good for you
www.opensuse.org
On 02/08/16 11:44, Marcus Rueckert wrote:> that sounds like s3 API based backend. > > also note that dovecot comes with some "non local mail storage plugins" > in the commercial part: > > ``` > In this release, we support Windows Azure, Amazon S3, Scality and > Dropbox systems. > ``` > > so that should come close to what you want. for a local version you > could use e.g. radosdb/ceph or riak. both can provide the s3 API for > you. >Close, but this may be too low level, for example it may hide whose e-mail do you store (I don't know the exact implementation, it's not open source AFAIK).
Zitat von Timo Sirainen <tss at iki.fi>:> On 08 Feb 2016, at 11:01, Nagy, Attila <bra at fsn.hu> wrote: >> >> Hi, >> >> Nearly every popular programming language has an LMTP/POP/IMAP >> implementation, most of them suck in many different ways. >> I don't know any server or library which provides a >> well-established, compatible protocol frontend with an open backend >> API, which could be used to easily make a custom storage backend >> for the LMTP/POP/IMAP frontend in any language, in any programming >> paradigm, irregardless of the frontend implementation. >> >> Dovecot is very close to this: it speaks IMAP to the backend, but >> you still have to make a nearly complete IMAP implementation, which >> is a PITA. >> >> Therefore I am curious about your opinion: what about an HTTP-based >> open backend (lib-storage) API to Dovecot? > > We're planning on implementing http://jmap.io/ to Dovecot v2.3. It's > close enough to how Dovecot works internally (although we'd still > need the threading support) and also at least attempting to become a > standard. We haven't really started it yet though, so now would be a > good time to give any alternative suggestions or complaints about > jmap :)That would be a cool feature, Thanks. Andreas -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5840 bytes Desc: S/MIME Cryptographic Signature URL: <http://dovecot.org/pipermail/dovecot/attachments/20160209/e0678747/attachment-0001.p7s>