I just had a thought. It's probably going to be too big of a change for v2.0, so something to think about for v3.0: Supporting multiple UIDs is difficult. Currently expire-tool and v2.0's LMTP server are implemented in a way that it's always running as root and only temporarily dropping privileges. So any security hole in there gives the attacker root privileges. Not very nice. With a single UID it's possible (or will be) to just use that one UID for the process without running as root. User sharing mailboxes to another user with different UID is even more difficult. Currently the only practical way to implement this is to assign read-write permissions to some group and have all mail files owned by that group. Not very nice. Sharing mailboxes between users located in different servers is impossible if the disk isn't somehow shared. All of these can be solved the same way: 1. imap, pop3, lmtp, etc. processes will be running without any privileges, very much like imap/pop3-login nowadays. Nobody-chrooted to empty directory. 2. Create storage processes that are started as needed. They can be configured in different security/performance settings: One storage process per user, one storage process per UID, setuid-root style shared storage process for multiple UIDs. 3. Mail processes talk to storage processes via some protocol. They can talk via UNIX socket or TCP/IP. If an existing connection can't handle the target UID, a new connection is made that either reuses an existing storage proces or creates a new one. 4. IMAP can finally implement truly simultaneously running queries (where it makes sense) by creating more storage processes if existing ones are busy handling longer-running commands. This also allows using the same storage process for multiple simultaneous IMAP connections and mail deliveries (and etc). The storage processes can also hang around and keep mailboxes open after all connections have closed it (dynamically figured out when it makes sense) to handle webmail clients that create lots of short-living connections or POP3 clients who poll for new mail every minute. The big problem is what the protocol should be. Use some existing RPC protocol? It should be something extensible so that a plugin in imap process can talk to a plugin in storage process, without the base processes knowing anything about the details (e.g. imap-quota plugin asking quota usage from storage's quota plugin). In any case the client side API should be asynchronous. That can make it annoyingly difficult to use though. Wonder if I could switch to erlang or something for the imap/pop3 processes :)
Timo Sirainen schrieb:> > > 3. Mail processes talk to storage processes via some protocol. They > can talk via UNIX socket or TCP/IP. If an existing connection can't > handle the target UID, a new connection is made that either reuses an > existing storage proces or creates a new one. >This sounds really, really cool! I am thinking of the possibilities of separating storage to frontend. That way one could build dynamically distributing mailboxes. Use local ressources of many storage boxes instead of using FC storages all the way. Or just "replicate" mails twice to have them redundantly stored (hadn't you the idea of a replication part, anyway?).> > The big problem is what the protocol should be. Use some existing RPC > protocol? It should be something extensible so that a plugin in imap > process can talk to a plugin in storage process, without the base > processes knowing anything about the details (e.g. imap-quota plugin > asking quota usage from storage's quota plugin). In any case the > client side API should be asynchronous. That can make it annoyingly > difficult to use though. Wonder if I could switch to erlang or > something for the imap/pop3 processes :) >Maybe one could "lend" some parts of erlang based database http://couchdb.apache.org/ ... :) Regards, Sebastian
> The big problem is what the protocol should be. Use some existing RPC > protocol? It should be something extensible so that a plugin in imap process > can talk to a plugin in storage process, without the base processes knowing > anything about the details (e.g. imap-quota plugin asking quota usage from > storage's quota plugin).Googles Protocol Buffers offer both flexibility and speed.
> > The big problem is what the protocol should be. Use some existing > RPC protocol? It should be something extensible so that a plugin in > imap process can talk to a plugin in storage process, without the > base processes knowing anything about the details (e.g. imap-quota > plugin asking quota usage from storage's quota plugin). In any case > the client side API should be asynchronous. That can make it > annoyingly difficult to use though. Wonder if I could switch to > erlang or something for the imap/pop3 processes :)process-to-process D-Bus! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2318 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20090527/0c2c4408/attachment-0002.bin>
On Wednesday 27 May 2009 20.14.01 Aria Stewart wrote:> > The big problem is what the protocol should be. Use some existing > > RPC protocol? It should be something extensible so that a plugin in > > imap process can talk to a plugin in storage process, without the > > base processes knowing anything about the details (e.g. imap-quota > > plugin asking quota usage from storage's quota plugin). In any case > > the client side API should be asynchronous. That can make it > > annoyingly difficult to use though. Wonder if I could switch to > > erlang or something for the imap/pop3 processes :) > > process-to-process D-Bus!Pleeeaaaasseeee. Do not need to bloat dovecot with heavy lib-depends, AT ALL. Daniel -- L?VAI D?niel PGP key ID = 0x4AC0A4B1 Key fingerprint = D037 03B9 C12D D338 4412 2D83 1373 917A 4AC0 A4B1
Timo Sirainen skrev:> The big problem is what the protocol should be. Use some existing RPC > protocol? It should be something extensible so that a plugin in imap > process can talk to a plugin in storage process, without the base > processes knowing anything about the details (e.g. imap-quota plugin > asking quota usage from storage's quota plugin). In any case the client > side API should be asynchronous. That can make it annoyingly difficult > to use though. Wonder if I could switch to erlang or something for the > imap/pop3 processes :)SOAP / BEEP: http://www.faqs.org/rfcs/rfc3288.html You will need to thread calls, waiting for long-running commands. Unless you switch to Erlang, of course :-) If you don't overdo the XML document part, you can use your own implementation or use saxon (sp?) or another serial XML processor. /Peter -- Peter Lindgren
Timo Sirainen <tss at iki.fi> writes:> The big problem is what the protocol should be. Use some existing RPC > protocol? It should be something extensible so that a plugin in imap > process can talk to a plugin in storage process, without the base > processes knowing anything about the details (e.g. imap-quota plugin > asking quota usage from storage's quota plugin). In any case the client > side API should be asynchronous. That can make it annoyingly difficult to > use though. Wonder if I could switch to erlang or something for the > imap/pop3 processes :)I realise I'm coming a little late to this conversation. Various people have suggested various RPC mechanisms. Might I humbly suggest also looking at 9P (or 9P2000) the Plan9 filesystem protocol. [1] The general design philosophy in Plan9 is not to use an increasing number of application specific RPC mechanisms but instead to implement everything in a synthetic filesystem tree. The 9P protocol handles the basic operations required to navigate the filesystem tree and read/write to the files within. The semantics of "RPC" are then relegated to the specification for the layout of the filesystem and meaning assigned to reading or writing to files within that tree. This allows for extensibility (and incidently discoverability) - new features or "plugins" appear as a new file or directory hierarchy in the presented filesystem layout. As a concrete example the so called upas/fs [2] presents a view of a mailbox independent of the actual storage mechanism used for the data contained in the mailbox. From the manpage The mailbox itself becomes a directory under /mail/fs. Each message in the mailbox becomes a numbered directory in the mailbox directory, and each attachment becomes a numbered directory in the message directory. Since an attachment may itself be a mail message, this structure can recurse ad nauseam. See [2] for more details. I'm not suggesting this layout would be exactly right for the Dovecot storage protocol but it may provide a starting point. It's probably worth noting the irony, given this is a maillist about Dovecot, in that this approach almost obsoletes the need for an IMAP protocol in the first place (the upas/fs style layout as documented doesn't really provide sufficient support for server side search for example). Footnotes: [1] http://plan9.bell-labs.com/magic/man2html/5/0intro [2] http://plan9.bell-labs.com/magic/man2html/4/upasfs
> On Jul 14, 2009, at 8:25 AM, Marc Dequ?nes (Duck) wrote: > >> Restarting IMAP/POP3 mail server: dovecotILoading modules from >> directory: /usr/lib/dovecot/modules/imap >> Edlopen(/usr/lib/dovecot/modules/imap/lib10_quota_plugin.so) >> failed: /usr/lib/dovecot/modules/imap/lib10_quota_plugin.so: >> undefined symbol: xdr_ext_getquota_args > > This is because the Debian package is built with automake 1.9. 1.10 > would work, or you could apply this patch: > > http://hg.dovecot.org/dovecot-1.2/rev/66040bffc1a5 > > -- > To UNSUBSCRIBE, email to debian-bugs-dist-REQU... at lists.debian.org > with a subject of "unsubscribe". Trouble? Contact listmas... at lists.debian.orgThe problem is also in CentOS 5.3, the mentioned patch also solves the problem, but applying the new ones http://hg.dovecot.org/dovecot-1.2/rev/1b5a1ac430a7 http://hg.dovecot.org/dovecot-1.2/rev/b6aaca59301a raised the error again. Regards -- +----------------------------------------------------------------+ | Juan C. Blanco | | | | Centro de Calculo | | | Facultad de Informatica U.P.M. | E-mail: jcblanco at fi.upm.es | | Campus de Montegancedo | | | Boadilla del Monte | Tel.: (+34) 91 336 7466 | | 28660 MADRID (Spain) | Fax : (+34) 91 336 6913 | +----------------------------------------------------------------+