dovecot - Jul 2012 - Dovecot shared library to replace libc-client

Hi all Dovecotters,

Lots of programs (e.g, PHP) link to a library called c-client, which is 
a derivative of the original IMAP implementation, UW IMAP.

(For those new to UW IMAP, read here: http://www.washington.edu/imap/ )

UW IMAP is not in as much of active development as it used to be, so I 
am curious:

I see there is a dovecot shared library. I haven't looked into the 
details, but here are things I'm interested in:

1. Replacing libc-client's use as a client library

In Debian, for example, you can search for packages that depend on this 
library as a client. It looks like this:

$  ~  apt-cache rdepends libc-client2007e

libc-client2007e
Reverse Depends:
  uw-mailutils
  uw-imapd
  libc-client2007e-dev
  ipopd
  prayer
  php5-imap
  mailsync
  libmail-cclient-perl
  asterisk
  aolserver4-nsimap
  uw-mailutils
  libc-client2007e-dev
  prayer
  php5-imap
  mailsync
  libmail-cclient-perl
  asterisk-voicemail-imapstorage

Let's say, hypothetically, we wanted to switch php5-imap to using 
Dovecot where it currently uses libc-client.

I envision creating libdovecot-c-client-alike that is a set of headers 
and a library that is API-compatible with (at least a subset of) 
c-client. You can call that a "compatibility shim." Then e.g.
php5-imap
could be given the path to those headers and the corresponding 
libdovecot-c-client-alike library, and when it thinks it is linking to 
c-client, it could instead link to the libdovecot-c-client-alike.

This might be convenient if you want to limit how much of a public API 
is presented by the current "dovecot.so" that gets installed in e.g. 
/usr/lib/dovecot/. The compatiblity shim could have a small API, and if 
you don't want provide ABI guarantees within dovecot.so, the shim could 
dlopen() dovecot.so rather than link to it.

2. Use of Dovecot shared library within alpine, embedding the imapd

Right now, the mail client "alpine" embeds a copy of the UW IMAP 
source. It uses this when accessing local mail spools, for example.

If Dovecot's IMAPd were available as a shared library, perhaps with a 
c-client-like API, (although not necessarily -- it would be feasible to 
upgrade alpine to a different API), then alpine could use Dovecot's 
mail drivers directly.


So, those two are the dream. Timo and others, what are your thoughts?

Thanks for reading this far!

-- Asheesh.

P.S. The radical request #2 would be an excellent, decisive way to end 
a sad thread in the Debian bug tracker about Alpine + Maildir: 
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=405762 .

On 3.7.2012, at 8.53, Asheesh Laroia wrote:
> I see there is a dovecot shared library. I haven't looked into the 
> details, but here are things I'm interested in:
> 
> 1. Replacing libc-client's use as a client library
..
> I envision creating libdovecot-c-client-alike that is a set of headers 
> and a library that is API-compatible with (at least a subset of) 
> c-client. You can call that a "compatibility shim." Then e.g.
php5-imap
> could be given the path to those headers and the corresponding 
> libdovecot-c-client-alike library, and when it thinks it is linking to 
> c-client, it could instead link to the libdovecot-c-client-alike.
Yeah, that's a possibility. Although Dovecot's libraries are still more
about the server side stuff than client side stuff, so it's possible that
there are many important missing things. Also libc-client is commonly used to do
IMAP access and Dovecot's imapc backend is still lacking quite a lot of that
functionality.
> This might be convenient if you want to limit how much of a public API 
> is presented by the current "dovecot.so" that gets installed in
e.g.
> /usr/lib/dovecot/. The compatiblity shim could have a small API, and if 
> you don't want provide ABI guarantees within dovecot.so, the shim could
> dlopen() dovecot.so rather than link to it.
I'm still not ready to give ABI or even API guarantees to libdovecot.. There
are still several important large changes to do and I don't really want to
keep a ton of ugly backwards compatibility stuff just for external users of the
library. Also another potential problem is that libdovecot.so doesn't use a
global namespace prefix for all of its functions, so linking it with php could
cause symbol name conflicts (especially md5_*, sha1_* and such could cause
trouble, like they already have caused with libmysql).

Once Dovecot becomes more "finished" (a few years?) I could consider
API/ABI guarantees.. Of course nothing prevents anyone else from distributing a
(patched) libdovecot already that actually does give some ABI guarantees. I just
don't want to spend time on it. And v2.1 -> v2.2 -> v2.3 etc.
transitions are going to be large changes.
> 2. Use of Dovecot shared library within alpine, embedding the imapd
> 
> Right now, the mail client "alpine" embeds a copy of the UW IMAP 
> source. It uses this when accessing local mail spools, for example.
> 
> If Dovecot's IMAPd were available as a shared library, perhaps with a 
> c-client-like API, (although not necessarily -- it would be feasible to 
> upgrade alpine to a different API), then alpine could use Dovecot's 
> mail drivers directly.
I wonder if it would make any sense to for Alpine not use libdovecot API
directly but rather talk IMAP protocol to Dovecot code (maybe running in a
separate process)? The Dovecot configuration could be passed pretty easily from
Alpine code without requiring any extra config files.