Hi,
I've been working on a patch for dovecot 1.2 from the Kolab branch
(http://hg.intevation.org/kolab/dovecot-1.2_kolab-branch/) that
implements listing of shared namespaces. I've got something that works
in some basic way but is still missing some pieces. See the attached
patch, which also contains some installation and configuration notes.
Implementation notes:
One of the main problems the patch addresses is getting a list of all
users that have mailboxes the logged in user can see. The patch uses a
dict to cache information about which users have at least one mailbox
that is visible to other users. The dict doesn't cache which other
users, though. The cache entry for a given user is updated whenever the
dovecot-acl-list file in the maildir root directory is updated. This
ties the implementation to a specific acl backend to an extent, but that
shouldn't be a problem at the moment.
Another problem is that namespaces for all those users have to be
created. The patch does that in shared-storage.c when the shared
storage is created. At this stage of development of the patch that
works well enough, I think, but it might be better to update the
namespaces whenever a list iterator is created.
To avoid unnecessary coupling between the shared namespace code and the
ACL plugin, the shared namespace code has a hook that it calls when it
needs a list of all the users who may have mailboxes visible to the
current user. The ACL plugin sets that hook and uses the dict to
produce that list. This way, the ACL plugin depends on the shared
namespace code but not the other way round and all the dict handling is
in the ACL plugin.
I'm not sure the new hook is really needed. The patch could perhaps
just as well extend the acl_next_hook_mail_storage_created and
acl_next_hook_mailbox_list_created functions to do the namespace
creation when they're called for a shared storage or mailbox list.
Problems:
All of my tests so far involved a shared namespace of the form
namespace shared {
separator = /
prefix = users/%%u/
location = maildir:.../var/mail/%%u:...
subscriptions = no
list = yes
hidden = no
}
Also, let's assume two users, ford and arthur with ford's
"INBOX/hhgttg"
available to arthur as "users/ford/INBOX/hhgttg". Arthur may not list
ford's INBOX, though. In the following the current user is always
arthur.
I found the following problems:
- LIST response includes namespaces the user doesn't really have access
to. E.g. if there's another user, zaphod who's made some mailbox
available to somebody else, but not arthur, arthur still sees
* LIST (\Noselect \HasChildren) "/" "users/zaphod"
Not sure it's worth fixing this, though.
- List with "%" doesn't list all intermediate mailboxes.
On the one hand arthur sees this:
x LIST "" "*"
...
* LIST (\Noselect \HasChildren) "/" "users/ford"
* LIST (\HasNoChildren) "/" "users/ford/INBOX/hhgttg"
x OK List completed.
OTOH, with "%" only this:
x LIST "" "users/ford/%"
x OK List completed.
cyrus shows
x LIST "" "users/ford/%"
* LIST (\Noselect \HasChildren) "/" "users/ford/INBOX"
x OK List completed.
At least Kontact resp. KMail rely on this.
- The dovecot-acl-list is not always rebuilt, even when it should have
been, AFAICT. In particular, if the file exists but is empty, it's
never updated, even when ACL later change. Maybe this is a bug in
the Kolab branch.
Cheers,
Bernhard
--
Bernhard Herzog | ++49-541-335 08 30 | http://www.intevation.de/
Intevation GmbH, Neuer Graben 17, 49074 Osnabr?ck | AG Osnabr?ck, HR B 18998
Gesch?ftsf?hrer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner
-------------- next part --------------
A non-text attachment was scrubbed...
Name: list-shared-namespaces-20081028.diff
Type: text/x-diff
Size: 9845 bytes
Desc: not available
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20081028/035cd760/attachment-0004.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20081028/035cd760/attachment-0005.bin>
On 28.10.2008, Bernhard Herzog wrote:> I've been working on a patch for dovecot 1.2 from the Kolab branch > (http://hg.intevation.org/kolab/dovecot-1.2_kolab-branch/) that > implements listing of shared namespaces. I've got something that works > in some basic way but is still missing some pieces. See the attached > patch, which also contains some installation and configuration notes.The patch is now available in the kolab branch hg repository: http://hg.intevation.org/kolab/dovecot-1.2_kolab-branch/rev/c2396923cd2f Bernhard -- Bernhard Herzog | ++49-541-335 08 30 | http://www.intevation.de/ Intevation GmbH, Neuer Graben 17, 49074 Osnabr?ck | AG Osnabr?ck, HR B 18998 Gesch?ftsf?hrer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: <http://dovecot.org/pipermail/dovecot/attachments/20081029/33933c94/attachment-0002.bin>
On 28.10.2008, Bernhard Herzog wrote:> - List with "%" doesn't list all intermediate mailboxes. > > On the one hand arthur sees this: > > x LIST "" "*" > ... > * LIST (\Noselect \HasChildren) "/" "users/ford" > * LIST (\HasNoChildren) "/" "users/ford/INBOX/hhgttg" > x OK List completed. > > OTOH, with "%" only this: > > x LIST "" "users/ford/%" > x OK List completed.I've looked into this. The problem is that acl_mailbox_list_info_is_visible in src/plugins/acl/acl-mailbox-list.c considers nonexistent mailboxes as not visible. The attached patch fixes that for me. I'm not sure it really is the right fix, though. Maybe it would cause some mailboxes to be listed even though they shouldn't. There's one thing about acl_mailbox_list_info_is_visible and struct acl_mailbox_list_iterate_context that I don't understand. What's the member struct mailbox_info info; used for? Regards, Bernhard -- Bernhard Herzog | ++49-541-335 08 30 | http://www.intevation.de/ Intevation GmbH, Neuer Graben 17, 49074 Osnabr?ck | AG Osnabr?ck, HR B 18998 Gesch?ftsf?hrer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: acl-plugin-list-nonexistent-parent-mailboxes.diff Type: text/x-diff Size: 498 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20081031/4d06ced3/attachment-0004.bin> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: <http://dovecot.org/pipermail/dovecot/attachments/20081031/4d06ced3/attachment-0005.bin>
On Tue, 2008-10-28 at 21:07 +0100, Bernhard Herzog wrote:> One of the main problems the patch addresses is getting a list of all > users that have mailboxes the logged in user can see. The patch uses a > dict to cache information about which users have at least one mailbox > that is visible to other users. The dict doesn't cache which other > users, though. The cache entry for a given user is updated whenever the > dovecot-acl-list file in the maildir root directory is updated. This > ties the implementation to a specific acl backend to an extent, but that > shouldn't be a problem at the moment.What I'm worrying here is if all users have shared something. For example how does this work with global ACLs? IIRC if there's a global ACL for e.g. "Spam", Dovecot will create dovecot-acl-list with "Spam" in it for all users. Even without global ACLs the number of users sharing something might be a bit too large. So the performance would be a lot better if the dict stored something like source_user -> dest, where dest would be all the user names and group names that are included in the user's ACLs (for any mailbox). Then you'd look only at those users' mailboxes where you or your groups are mentioned. Negative user/group rules perhaps shouldn't be in the dict.> I'm not sure the new hook is really needed. The patch could perhaps > just as well extend the acl_next_hook_mail_storage_created and > acl_next_hook_mailbox_list_created functions to do the namespace > creation when they're called for a shared storage or mailbox list.Perhaps hook_mail_namespaces_created? Some other things: v1.2 code supports multiple users per process. That means you can't really use getenv("USER") and you can't store per-user objects as static variables. Rather you should hook into hook_mail_user_created and add the per-user variables to the mail_user structure. See for example lazy-expunge plugin (struct lazy_expunge_mail_user) or quota plugin how that works. i_info("no acl_shared_dict specified; shared namespaces will not be listed") could be written only if getenv("DEBUG") != NULL. Is acl_shared_debug stuff only a temporary developer debugging thing, or will it be useful also for sysadmins?> All of my tests so far involved a shared namespace of the form > > namespace shared { > separator = / > prefix = users/%%u/ > location = maildir:.../var/mail/%%u:... > subscriptions = no > list = yes > hidden = no > } > > Also, let's assume two users, ford and arthur with ford's "INBOX/hhgttg" > available to arthur as "users/ford/INBOX/hhgttg". Arthur may not list > ford's INBOX, though. In the following the current user is always > arthur. > > I found the following problems: > > - LIST response includes namespaces the user doesn't really have access > to. E.g. if there's another user, zaphod who's made some mailbox > available to somebody else, but not arthur, arthur still sees > > * LIST (\Noselect \HasChildren) "/" "users/zaphod" > > Not sure it's worth fixing this, though.It'll expose other users' names, which isn't good. It needs to be fixed before I can make a release. Probably not too difficult though. I think the ACL plugin's mailbox listing code could detect when a shared namespace prefix is listed and not show it if it doesn't have visible mailboxes under it. And that's related to the next problem:> - List with "%" doesn't list all intermediate mailboxes.Yes, this is actually in my TODO list already. I'll read and aswer to your next mail about this..> - The dovecot-acl-list is not always rebuilt, even when it should have > been, AFAICT. In particular, if the file exists but is empty, it's > never updated, even when ACL later change. Maybe this is a bug in > the Kolab branch.I haven't actually tested dovecot-acl-list all that much, so it's possible that there are bugs in it. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20081031/f22b08f4/attachment-0002.bin>
On 28.10.2008, Bernhard Herzog wrote:> - The dovecot-acl-list is not always rebuilt, even when it should have > been, AFAICT. In particular, if the file exists but is empty, it's > never updated, even when ACL later change. Maybe this is a bug in > the Kolab branch.It happens with vanilla dovecot 1.2 too. Here's my analysis so far: The dovecot-acl-list is rebuilt by acl_backend_vfile_acllist_rebuild which called in two places, acl_backend_vfile_acllist_refresh and acl_backend_vfile_acllist_verify. The _refresh-funtion rereads the file if it has changed and rebuilds it if it cannot be read. The _verify-function checks whether any of the acl files referenced by the dovecot-acl-list file has been changed and rebuilds the dovecot-acl-list file if that's the case. The behavior of the _verify funtion is the problem. It only rebuilds if any of the already referenced acl files has changed but not if new acl files have been created. I'm not sure yet what the best way is to determine whether new mailbox whose acllist is being verified has a new acl file. Cheers, Bernhard -- Bernhard Herzog | ++49-541-335 08 30 | http://www.intevation.de/ Intevation GmbH, Neuer Graben 17, 49074 Osnabr?ck | AG Osnabr?ck, HR B 18998 Gesch?ftsf?hrer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: <http://dovecot.org/pipermail/dovecot/attachments/20081031/e39435a7/attachment-0002.bin>