Adam McDougall
2007-May-07 01:34 UTC
[Dovecot] Dovecot not handling r/o mailboxes completely, and problem with ACL as a workaround
First of all, I think dovecot is really fantastic and I have thanks for all the hard work. I think it will be the best fit for my ~5000 users when I have it setup completely. We normally have approx 500 concurrent IMAP connections during the day. I am trying to convert from courier-imap to dovecot, but I have an issue with public namespace folders that are not writable by certain users. Please let me know if I need to provide more information or how I can help solve this issue. In courier-imap, we were able to take advantage of the maildir structure and standard unix users/groups to allow 'decsstaff' members to have full write access while 'decsall' members only have r/o unless also a member of 'decsstaff': -rw-rw-r-- 1 postlocal decsstaff 37597 May 5 23:37 /egr/mail/shared/decs/.support.In/cur/1178422658.M533373P54269.ice drwxrwxr-x 2 postlocal decsstaff 24576 May 5 23:37 /egr/mail/shared/decs/.support.In/cur drwxrws--- 6 postlocal decsall 4096 Apr 22 18:08 /egr/mail/shared/decs/.support.In drwxrwsr-x 34 postlocal wheel 4096 May 1 07:23 /egr/mail/shared/decs In short, upper directories are used to limit world access to just one group 'decsall', and once inside, r/o world access could be used as an anonymous substitution for 'decsall' and the group permissions would allow writing by 'decsstaff'. I don't know how courier stores and checks mail directory permissions, but it was aware when a user would not be able to delete or change messages and would return a READ-ONLY imap error when an attempt to change a message occurred (courier returns this in the IMAP response to update mail flags to reflect that the client wants to move to delete a mail). I understand that imap clients will usually only set status as read or deleted first and only try to delete an email for real on expunge, but it seems dovecot does not try to detect if a message is modifyable before it is too late for the user to find out. What happens is a user appears to be able to mark messages read, move them to another folder, or delete them (another way of moving to another folder eg. Trash). The user is clueless that the changes will not occur, and only finds out later when they reload the folder to find the messages back, or tries to expunge and get an unhelpful 'internal error' from dovecot (I can help trace this situation if it helps, but I think that stage should not be reached if things were working right). My configuration has the public folder control and indexes inside a user's home directory, so it will always be possible for dovecot to record message status changes, however I do not think dovecot should update the control and indexes to reflect an IMAP operation that is not allowed to complete due to restricted permissions on the actual email file. I suppose dovecot would have to check the permissions on mail message files. As a workaround, I wondered if I could apply dovecot ACLs to the public folders to reinforce my intentions of only allowing readonly access to certain users. If ACLs worked, I could even use it as a viable alternative to the unix permissions on the maildir contents and avoid using more unix groups than necessary. For most of our shared mailboxes, there will only be a few persons with write access, and up to a dozen people with read access, so even if I have to list every user seperately I could. However, I have not been able to get ACLs to do anything useful for me, and I hesitate to use the feature to work around might be a configuration error or bug. It is also more convenient for my migration procedure if I do not have to modify file permissions or structure during downtime to switch to dovecot. I am still not sure how dovecot ACLs affect the actual permissons of dovecot processes. I have mostly tried placing a dovecot-acl file in the maildir directory (alongside cur, tmp, new) and dovecot seems to be reading it according to the log, however I am not getting expected results from the ACLs. If the unix permissions allow a client to enter 'cur' and read mails, the mail client thinks it can delete or move them (until expunge) no matter even if I put acl entries with only 'r' permission. ACL questions (which might help others understand better as well): Is a 'user=foo rl' entry supposed to remove any possible write permissions in a maildir covered by such an ACL for user foo? If not, shouldn't it at least enforce restrictions on the available mail metadata which is more likely to be writable by the user? Is it even possible for an ACL to grant additional permissions above what is granted to a user by standard unix permissions? Does an empty ACL equate to the exact permissions granted by unix, and should an admin try to design an ACL to modify as little as possible, or should the admin try to use enough acl entries to form a complete description of the desired resulting permissions? I get the impression that broad ACLs should be listed first, and more specific ACLS listed below can add or remove ACL permissions applied earlier in the file as a 'last match' case. Should the acl plugin be able to print a summary of the resulting permissions in the log (with mail_debug=yes) when accessing a folder with an ACL? A snip from logs, I generated alot today and I'm not sure which portions would be most useful so I'll start with this: May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): acl: initializing backend with data: vfile:/usr/local/etc/dovecot-acls May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): acl: acl username = mcdouga9 May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): acl: owner username May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): acl vfile: Global ACL directory: /usr/local/etc/dovecot-acls May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): Namespace: type=public, prefix=#shared/decs/, sep=/, inbox=no, hidden=no, subscriptions=no May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): maildir: data=/egr/mail/shared-dovecot2/decs:CONTROL=/home/mcdouga9/Maildir/dovecot/public/control/decs:INDEX=/home/mcdouga9/Maildir/dovecot/public/indexes/decs May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): maildir: root=/egr/mail/shared-dovecot2/decs, index=/home/mcdouga9/Maildir/dovecot/public/indexes/decs, control=/home/mcdouga9/Maildir/dovecot/public/control/decs, inboxMay 6 15:20:08 gribble dovecot: IMAP(mcdouga9): opendir(/egr/mail/shared-dovecot2/decs/tmp) failed: Permission denied # dovecot --version 1.0.0 The reason I have so many public spaces below, is to match the existing directory structure we setup for courier, and also to help us keep the directory listing more tidy since otherwise we would end up with many folders all under one directory. Some of them have monthly rotation and it could get out of hand if we did not split them up. I understand it makes things more difficult for the global acl list at present, if I were to use it, but I may not have any conflicting folder names at this time. # dovecot -n # /usr/local/etc/dovecot.conf ssl_cert_file: /usr/local/etc/apache2/ssl/mail.egr.msu.edu.pem ssl_key_file: /usr/local/etc/apache2/ssl/mail.egr.msu.edu.pem login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 0 mail_location: maildir:~/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes mail_debug: yes mmap_disable: yes maildir_copy_with_hardlinks: yes mail_plugins: acl mail_log_max_lines_per_sec: 0 imap_client_workarounds: delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep namespace: type: public separator: / prefix: #shared/be/ location: maildir:/egr/mail/shared-dovecot2/be:CONTROL=%h/Maildir/dovecot/public/control/be:INDEX=%h/Maildir/dovecot/public/indexes/be namespace: type: public separator: / prefix: #shared/cee/ location: maildir:/egr/mail/shared-dovecot2/cee:CONTROL=%h/Maildir/dovecot/public/control/cee:INDEX=%h/Maildir/dovecot/public/indexes/cee namespace: type: public separator: / prefix: #shared/chems/ location: maildir:/egr/mail/shared-dovecot2/chems:CONTROL=%h/Maildir/dovecot/public/control/chems:INDEX=%h/Maildir/dovecot/public/indexes/chems namespace: type: public separator: / prefix: #shared/decs/ location: maildir:/egr/mail/shared-dovecot2/decs:CONTROL=%h/Maildir/dovecot/public/control/decs:INDEX=%h/Maildir/dovecot/public/indexes/decs namespace: type: public separator: / prefix: #shared/egr/ location: maildir:/egr/mail/shared-dovecot2/egr:CONTROL=%h/Maildir/dovecot/public/control/egr:INDEX=%h/Maildir/dovecot/public/indexes/egr namespace: type: public separator: / prefix: #shared/me/ location: maildir:/egr/mail/shared-dovecot2/me:CONTROL=%h/Maildir/dovecot/public/control/me:INDEX=%h/Maildir/dovecot/public/indexes/me namespace: type: public separator: / prefix: #shared/org/ location: maildir:/egr/mail/shared-dovecot2/org:CONTROL=%h/Maildir/dovecot/public/control/org:INDEX=%h/Maildir/dovecot/public/indexes/org namespace: type: public separator: / prefix: #shared/toptest/ location: maildir:/egr/mail/shared-dovecot2/toptest:CONTROL=%h/Maildir/dovecot/public/control/toptest:INDEX=%h/Maildir/dovecot/public/indexes/toptest namespace: type: public separator: / prefix: #shared/ugs/ location: maildir:/egr/mail/shared-dovecot2/ugs:CONTROL=%h/Maildir/dovecot/public/control/ugs:INDEX=%h/Maildir/dovecot/public/indexes/ugs namespace: type: public separator: / prefix: #shared/vprgs/ location: maildir:/egr/mail/shared-dovecot2/vprgs:CONTROL=%h/Maildir/dovecot/public/control/vprgs:INDEX=%h/Maildir/dovecot/public/indexes/vprgs namespace: type: private separator: / prefix: mail/ hidden: yes namespace: type: private separator: / location: maildir:~/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes inbox: yes auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth user: postfix group: postfix master: plugin: acl: vfile:/usr/local/etc/dovecot-acls Thanks.
Matt Zukowski
2007-May-07 14:16 UTC
[Dovecot] Dovecot not handling r/o mailboxes completely, and problem with ACL as a workaround
Adam, unfortunately I don't have an answer for you, but I'd just like to add that we've had no luck getting dovecot's ACL working properly here either (see my previous message: "some clarification re: ACL"). No one responded to my original message, which leads me to suspect that the ACL feature isn't widely used (if at all) and is therefore not fully implemented. P.S. Sorry Adam, that should have gone out to the list rather than to you personally. Adam McDougall wrote:> First of all, I think dovecot is really fantastic and I have thanks for all > the hard work. I think it will be the best fit for my ~5000 users when I > have it setup completely. We normally have approx 500 concurrent IMAP > connections during the day. > > I am trying to convert from courier-imap to dovecot, but I have an issue with > public namespace folders that are not writable by certain users. Please let me > know if I need to provide more information or how I can help solve this issue. > > In courier-imap, we were able to take advantage of the maildir structure > and standard unix users/groups to allow 'decsstaff' members to have full write > access while 'decsall' members only have r/o unless also a member of > 'decsstaff': > > -rw-rw-r-- 1 postlocal decsstaff 37597 May 5 23:37 > /egr/mail/shared/decs/.support.In/cur/1178422658.M533373P54269.ice > drwxrwxr-x 2 postlocal decsstaff 24576 May 5 23:37 > /egr/mail/shared/decs/.support.In/cur > drwxrws--- 6 postlocal decsall 4096 Apr 22 18:08 /egr/mail/shared/decs/.support.In > drwxrwsr-x 34 postlocal wheel 4096 May 1 07:23 /egr/mail/shared/decs > > In short, upper directories are used to limit world access to just one group > 'decsall', and once inside, r/o world access could be used as an anonymous > substitution for 'decsall' and the group permissions would allow writing by > 'decsstaff'. I don't know how courier stores and checks mail directory > permissions, but it was aware when a user would not be able to delete or change > messages and would return a READ-ONLY imap error when an attempt to change a > message occurred (courier returns this in the IMAP response to update mail flags > to reflect that the client wants to move to delete a mail). I understand that > imap clients will usually only set status as read or deleted first and only try > to delete an email for real on expunge, but it seems dovecot does not try to > detect if a message is modifyable before it is too late for the user to find > out. What happens is a user appears to be able to mark messages read, move them > to another folder, or delete them (another way of moving to another folder eg. > Trash). The user is clueless that the changes will not occur, and only finds > out later when they reload the folder to find the messages back, or tries to > expunge and get an unhelpful 'internal error' from dovecot (I can help trace > this situation if it helps, but I think that stage should not be reached if > things were working right). My configuration has the public folder control and > indexes inside a user's home directory, so it will always be possible for > dovecot to record message status changes, however I do not think dovecot should > update the control and indexes to reflect an IMAP operation that is not allowed > to complete due to restricted permissions on the actual email file. I suppose > dovecot would have to check the permissions on mail message files. > > As a workaround, I wondered if I could apply dovecot ACLs to the public folders > to reinforce my intentions of only allowing readonly access to certain users. If > ACLs worked, I could even use it as a viable alternative to the unix permissions > on the maildir contents and avoid using more unix groups than necessary. For > most of our shared mailboxes, there will only be a few persons with write > access, and up to a dozen people with read access, so even if I have to list > every user seperately I could. However, I have not been able to get ACLs to do > anything useful for me, and I hesitate to use the feature to work around might > be a configuration error or bug. It is also more convenient for my migration > procedure if I do not have to modify file permissions or structure during > downtime to switch to dovecot. I am still not sure how dovecot ACLs affect the > actual permissons of dovecot processes. I have mostly tried placing a > dovecot-acl file in the maildir directory (alongside cur, tmp, new) and dovecot > seems to be reading it according to the log, however I am not getting expected > results from the ACLs. If the unix permissions allow a client to enter 'cur' > and read mails, the mail client thinks it can delete or move them (until > expunge) no matter even if I put acl entries with only 'r' permission. > > ACL questions (which might help others understand better as well): > Is a 'user=foo rl' entry supposed to remove any possible write permissions in a > maildir covered by such an ACL for user foo? If not, shouldn't it at least > enforce restrictions on the available mail metadata which is more likely to be > writable by the user? > > Is it even possible for an ACL to grant additional permissions above what is > granted to a user by standard unix permissions? > > Does an empty ACL equate to the exact permissions granted by unix, and should an > admin try to design an ACL to modify as little as possible, or should the admin > try to use enough acl entries to form a complete description of the desired > resulting permissions? I get the impression that broad ACLs should be listed > first, and more specific ACLS listed below can add or remove ACL permissions > applied earlier in the file as a 'last match' case. > > Should the acl plugin be able to print a summary of the resulting permissions in > the log (with mail_debug=yes) when accessing a folder with an ACL? > > > A snip from logs, I generated alot today and I'm not sure which portions > would be most useful so I'll start with this: > May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): acl: initializing backend with data: vfile:/usr/local/etc/dovecot-acls > May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): acl: acl username = mcdouga9 > May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): acl: owner username > May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): acl vfile: Global ACL directory: /usr/local/etc/dovecot-acls > May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): Namespace: type=public, prefix=#shared/decs/, sep=/, inbox=no, hidden=no, subscriptions=no > May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): maildir: data=/egr/mail/shared-dovecot2/decs:CONTROL=/home/mcdouga9/Maildir/dovecot/public/control/decs:INDEX=/home/mcdouga9/Maildir/dovecot/public/indexes/decs > May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): maildir: root=/egr/mail/shared-dovecot2/decs, index=/home/mcdouga9/Maildir/dovecot/public/indexes/decs, control=/home/mcdouga9/Maildir/dovecot/public/control/decs, inbox> May 6 15:20:08 gribble dovecot: IMAP(mcdouga9): opendir(/egr/mail/shared-dovecot2/decs/tmp) failed: Permission denied > > > # dovecot --version > 1.0.0 > > The reason I have so many public spaces below, is to match the existing > directory structure we setup for courier, and also to help us keep the > directory listing more tidy since otherwise we would end up with many folders > all under one directory. Some of them have monthly rotation and it could get > out of hand if we did not split them up. I understand it makes things more > difficult for the global acl list at present, if I were to use it, but > I may not have any conflicting folder names at this time. > > # dovecot -n > # /usr/local/etc/dovecot.conf > ssl_cert_file: /usr/local/etc/apache2/ssl/mail.egr.msu.edu.pem > ssl_key_file: /usr/local/etc/apache2/ssl/mail.egr.msu.edu.pem > login_dir: /var/run/dovecot/login > login_executable: /usr/local/libexec/dovecot/imap-login > verbose_proctitle: yes > first_valid_uid: 1000 > first_valid_gid: 0 > mail_location: > maildir:~/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes > mail_debug: yes > mmap_disable: yes > maildir_copy_with_hardlinks: yes > mail_plugins: acl > mail_log_max_lines_per_sec: 0 > imap_client_workarounds: delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep > namespace: > type: public > separator: / > prefix: #shared/be/ > location: > maildir:/egr/mail/shared-dovecot2/be:CONTROL=%h/Maildir/dovecot/public/control/be:INDEX=%h/Maildir/dovecot/public/indexes/be > namespace: > type: public > separator: / > prefix: #shared/cee/ > location: > maildir:/egr/mail/shared-dovecot2/cee:CONTROL=%h/Maildir/dovecot/public/control/cee:INDEX=%h/Maildir/dovecot/public/indexes/cee > namespace: > type: public > separator: / > prefix: #shared/chems/ > location: > maildir:/egr/mail/shared-dovecot2/chems:CONTROL=%h/Maildir/dovecot/public/control/chems:INDEX=%h/Maildir/dovecot/public/indexes/chems > namespace: > type: public > separator: / > prefix: #shared/decs/ > location: > maildir:/egr/mail/shared-dovecot2/decs:CONTROL=%h/Maildir/dovecot/public/control/decs:INDEX=%h/Maildir/dovecot/public/indexes/decs > namespace: > type: public > separator: / > prefix: #shared/egr/ > location: > maildir:/egr/mail/shared-dovecot2/egr:CONTROL=%h/Maildir/dovecot/public/control/egr:INDEX=%h/Maildir/dovecot/public/indexes/egr > namespace: > type: public > separator: / > prefix: #shared/me/ > location: > maildir:/egr/mail/shared-dovecot2/me:CONTROL=%h/Maildir/dovecot/public/control/me:INDEX=%h/Maildir/dovecot/public/indexes/me > namespace: > type: public > separator: / > prefix: #shared/org/ > location: > maildir:/egr/mail/shared-dovecot2/org:CONTROL=%h/Maildir/dovecot/public/control/org:INDEX=%h/Maildir/dovecot/public/indexes/org > namespace: > type: public > separator: / > prefix: #shared/toptest/ > location: > maildir:/egr/mail/shared-dovecot2/toptest:CONTROL=%h/Maildir/dovecot/public/control/toptest:INDEX=%h/Maildir/dovecot/public/indexes/toptest > namespace: > type: public > separator: / > prefix: #shared/ugs/ > location: > maildir:/egr/mail/shared-dovecot2/ugs:CONTROL=%h/Maildir/dovecot/public/control/ugs:INDEX=%h/Maildir/dovecot/public/indexes/ugs > namespace: > type: public > separator: / > prefix: #shared/vprgs/ > location: > maildir:/egr/mail/shared-dovecot2/vprgs:CONTROL=%h/Maildir/dovecot/public/control/vprgs:INDEX=%h/Maildir/dovecot/public/indexes/vprgs > namespace: > type: private > separator: / > prefix: mail/ > hidden: yes > namespace: > type: private > separator: / > location: > maildir:~/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes > inbox: yes > auth default: > mechanisms: plain login > passdb: > driver: pam > userdb: > driver: passwd > socket: > type: listen > client: > path: /var/spool/postfix/private/auth > user: postfix > group: postfix > master: > plugin: > acl: vfile:/usr/local/etc/dovecot-acls > > Thanks. >This e-mail message is privileged, confidential and subject to copyright. Any unauthorized use or disclosure is prohibited. Le contenu du pr'esent courriel est privil'egi'e, confidentiel et soumis `a des droits d'auteur. Il est interdit de l'utiliser ou de le divulguer sans autorisation.
Jim Horner
2007-May-07 16:46 UTC
[Dovecot] Dovecot not handling r/o mailboxes completely, and problem with ACL as a workaround
> In courier-imap, we were able to take advantage of the maildir structure > and standard unix users/groups to allow 'decsstaff' members to have full > write access while 'decsall' members only have r/o unless also a member of > 'decsstaff': > > -rw-rw-r-- 1 postlocal decsstaff 37597 May 5 23:37 > /egr/mail/shared/decs/.support.In/cur/1178422658.M533373P54269.ice > drwxrwxr-x 2 postlocal decsstaff 24576 May 5 23:37 > /egr/mail/shared/decs/.support.In/cur > drwxrws--- 6 postlocal decsall 4096 Apr 22 18:08 > /egr/mail/shared/decs/.support.In drwxrwsr-x 34 postlocal wheel 4096 May > 1 07:23 /egr/mail/shared/decs > location: > maildir:/egr/mail/shared-dovecot2/vprgs:CONTROL=%h/Maildir/dovecot/public/c >ontrol/vprgs:INDEX=%h/Maildir/dovecot/public/indexes/vprgs namespace: > type: private > separator: / > prefix: mail/ > hidden: yes> plugin: > acl: vfile:/usr/local/etc/dovecot-aclsI use shared folders. I posted a while back about my setup. There have been a few changelogs since then concerning ACLs. My setup might be whacked but it still continues to work. The simplest example I have is root mail. I have mail folders drwxrwx--- 4 rootmail users /home/services/mail/rootmail/Maildir drwxrwx--- 4 rootmail users ../.RootmailFolder drwxrwx--- 4 rootmail users ../.RootmailFolder.general To get around ACL plugins downside of being unaware of namespaces I create a "RootmailFolder" underneath Maildir. No one else probably (hopefully) will have a folder named that. If they did then the permissions in the ACL plug-in directory would override "owner permissions". Were that to happen then you could just put a dovecot-acl file in the user's directory to compensate though this is a fuzzy part... this used to work but I haven't needed to test it so I don't know if it does still. I then created a general folder under that. I have a sieve script which pumps all mail into the general folder. So this is rootmail's "inbox". I did this as a workaround. <might not be needed nor work anymore> If you actually want a user 'rootmail' to use an imap client and log into their mailbox then you would create a file /home/services/mail/rootmail/Maildir/dovecot-acl /home/services/mail/rootmail/Maildir/.RootmailFolder/dovecot-acl /home/services/mail/rootmail/Maildir/.RootmailFolder.general/dovecot-acl all the files contain: user=rootmail lrwstie </might not be needed nor work anymore> To use the ACL plug-in files must be create in this directory:> plugin: > acl: vfile:/usr/local/etc/dovecot-aclsso I have (using your path) files: /usr/local/etc/dovecot-acls/RootmailFolder /usr/local/etc/dovecot-acls/RootmailFolder.general These files contain user=jhorner lrwstie My namespace is setup as: namespace public { separator = . prefix = ROOTMAIL. location = maildir:/home/services/mail/rootmail/Maildir:CONTROL=%h/shared-settings/rootmail/control:INDEX=%h/shared-settings/rootmail/index hidden = no inbox = no } Everyone can see the namespace but no one but me can access the namespace because RootmailFolder is only accessible by me. Those who do try to access a forbidden folder get a curt techie error. However, most clients do not show the namespace because there aren't any folders underneath the namespace that are accessible so this is not a problem for me. I also have a COMPANY share setup similarly. However there are many many folders underneath this share and different people can access different folders and I accomplish that using the ACL plug-in similar to above. I used to use Courier and I was able to duplicate shared folders via the ACL plug-in though the folders are now one level deeper, i.e. ROOTMAIL/RootmailFolders/general as opposed to ROOTMAIL/general (namespace/foldername). Some users did complain. Oh well... most are still breathing. Jim
Charles Marcus
2007-May-07 21:06 UTC
[Dovecot] Dovecot not handling r/o mailboxes completely, and problem with ACL as a workaround
On 5/7/2007 Charles Marcus (CMarcus at Media-Brokers.com) wrote:> You can at least set yours up so that when someone replies to one of > your messages, it goes to the list.Wups, meant to include a link: http://dovecot.org/cgi-bin/mailman/listinfo/dovecot -- Best regards, Charles
Adam McDougall
2007-May-11 03:28 UTC
[Dovecot] Dovecot not handling r/o mailboxes completely, and problem with ACL as a workaround
I have an update, I realize now that my ACL is working, however applying the ACL still does nothing to improve the original problem that dovecot doesn't seem to communicate permission failures to the client, which allows the client to become out of sync with reality on the server. I'm really not sure how dovecot might currently determine if a folder is read only on disk, because there seems to be some handling for that case in the code. Perhaps if I knew what part of the code (or an explanation) of why it thinks a folder is not writable, I could use that to my advantage and try to make folders look more "read only" on disk. On Sun, May 06, 2007 at 09:34:28PM -0400, Adam McDougall wrote: First of all, I think dovecot is really fantastic and I have thanks for all the hard work. I think it will be the best fit for my ~5000 users when I have it setup completely. We normally have approx 500 concurrent IMAP connections during the day. I am trying to convert from courier-imap to dovecot, but I have an issue with public namespace folders that are not writable by certain users. Please let me know if I need to provide more information or how I can help solve this issue. <snip> I don't know how courier stores and checks mail directory permissions, but it was aware when a user would not be able to delete or change messages and would return a READ-ONLY imap error when an attempt to change a message occurred (courier returns this in the IMAP response to update mail flags to reflect that the client wants to move to delete a mail). I understand that imap clients will usually only set status as read or deleted first and only try to delete an email for real on expunge, but it seems dovecot does not try to detect if a message is modifyable before it is too late for the user to find out. What happens is a user appears to be able to mark messages read, move them to another folder, or delete them (another way of moving to another folder eg. Trash). The user is clueless that the changes will not occur, and only finds out later when they reload the folder to find the messages back, or tries to expunge and get an unhelpful 'internal error' from dovecot (I can help trace this situation if it helps, but I think that stage should not be reached if things were working right). My configuration has the public folder control and indexes inside a user's home directory, so it will always be possible for dovecot to record message status changes, however I do not think dovecot should update the control and indexes to reflect an IMAP operation that is not allowed to complete due to restricted permissions on the actual email file. I suppose dovecot would have to check the permissions on mail message files. <snip portion about ACLs> # dovecot --version 1.0.0 The reason I have so many public spaces below, is to match the existing directory structure we setup for courier, and also to help us keep the directory listing more tidy since otherwise we would end up with many folders all under one directory. Some of them have monthly rotation and it could get out of hand if we did not split them up. I understand it makes things more difficult for the global acl list at present, if I were to use it, but I may not have any conflicting folder names at this time. # dovecot -n # /usr/local/etc/dovecot.conf ssl_cert_file: /usr/local/etc/apache2/ssl/mail.egr.msu.edu.pem ssl_key_file: /usr/local/etc/apache2/ssl/mail.egr.msu.edu.pem login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 0 mail_location: maildir:~/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes mail_debug: yes mmap_disable: yes maildir_copy_with_hardlinks: yes mail_plugins: acl mail_log_max_lines_per_sec: 0 imap_client_workarounds: delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep namespace: type: public separator: / prefix: #shared/be/ location: maildir:/egr/mail/shared-dovecot2/be:CONTROL=%h/Maildir/dovecot/public/control/be:INDEX=%h/Maildir/dovecot/public/indexes/be namespace: type: public separator: / prefix: #shared/cee/ location: maildir:/egr/mail/shared-dovecot2/cee:CONTROL=%h/Maildir/dovecot/public/control/cee:INDEX=%h/Maildir/dovecot/public/indexes/cee namespace: type: public separator: / prefix: #shared/chems/ location: maildir:/egr/mail/shared-dovecot2/chems:CONTROL=%h/Maildir/dovecot/public/control/chems:INDEX=%h/Maildir/dovecot/public/indexes/chems namespace: type: public separator: / prefix: #shared/decs/ location: maildir:/egr/mail/shared-dovecot2/decs:CONTROL=%h/Maildir/dovecot/public/control/decs:INDEX=%h/Maildir/dovecot/public/indexes/decs namespace: type: public separator: / prefix: #shared/egr/ location: maildir:/egr/mail/shared-dovecot2/egr:CONTROL=%h/Maildir/dovecot/public/control/egr:INDEX=%h/Maildir/dovecot/public/indexes/egr namespace: type: public separator: / prefix: #shared/me/ location: maildir:/egr/mail/shared-dovecot2/me:CONTROL=%h/Maildir/dovecot/public/control/me:INDEX=%h/Maildir/dovecot/public/indexes/me namespace: type: public separator: / prefix: #shared/org/ location: maildir:/egr/mail/shared-dovecot2/org:CONTROL=%h/Maildir/dovecot/public/control/org:INDEX=%h/Maildir/dovecot/public/indexes/org namespace: type: public separator: / prefix: #shared/toptest/ location: maildir:/egr/mail/shared-dovecot2/toptest:CONTROL=%h/Maildir/dovecot/public/control/toptest:INDEX=%h/Maildir/dovecot/public/indexes/toptest namespace: type: public separator: / prefix: #shared/ugs/ location: maildir:/egr/mail/shared-dovecot2/ugs:CONTROL=%h/Maildir/dovecot/public/control/ugs:INDEX=%h/Maildir/dovecot/public/indexes/ugs namespace: type: public separator: / prefix: #shared/vprgs/ location: maildir:/egr/mail/shared-dovecot2/vprgs:CONTROL=%h/Maildir/dovecot/public/control/vprgs:INDEX=%h/Maildir/dovecot/public/indexes/vprgs namespace: type: private separator: / prefix: mail/ hidden: yes namespace: type: private separator: / location: maildir:~/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes inbox: yes auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth user: postfix group: postfix master: plugin: acl: vfile:/usr/local/etc/dovecot-acls Thanks.
Timo Sirainen
2007-May-11 12:41 UTC
[Dovecot] Dovecot not handling r/o mailboxes completely, and problem with ACL as a workaround
On Sun, 2007-05-06 at 21:34 -0400, Adam McDougall wrote:> In short, upper directories are used to limit world access to just one group > 'decsall', and once inside, r/o world access could be used as an anonymous > substitution for 'decsall' and the group permissions would allow writing by > 'decsstaff'. I don't know how courier stores and checks mail directory > permissions, but it was aware when a user would not be able to delete or change > messages and would return a READ-ONLY imap error when an attempt to change a > message occurredLooks like Dovecot never returns maildir mailbox as READ-ONLY. So, your issues are: 1. Dovecot doesn't give an error when a flag is changed for mails. This is intentional, because otherwise you'd keep seeing "mailbox is read-only" errors all the time when reading mails (because clients keep setting the \Seen flag). Besides you're using a per-user index directory so the flag changes actually are saved. 2. EXPUNGE gives an internal error because it discovers too late that it can't unlink() the maildir file. This is a problem.. Even worse, you can't recover from it without deleting dovecot.index.log file. This should have two fixes: - If Dovecot ever notices that it can't unlink() a file, it should recover itself from it without giving internal errors. That would probably mean that the expunged message shows up as a new message. I'll see if I can get this fixed for v1.1. - It should treat the maildir as READ-ONLY so the EXPUNGE command is never written into dovecot.index.log in the first place. Fixed: http://dovecot.org/list/dovecot-cvs/2007-May/008734.html 3. EXPUNGE silently ignores no-permissions error. I'm not sure if this is good or not. In any case it shouldn't give an error if there are no messages with \Deleted flag, but perhaps it should give an error if there are.. Some clients do automatic expunges, so it would be annoying to see error messages constantly about it. I won't change this in v1.0 branch, but perhaps in v1.1.> As a workaround, I wondered if I could apply dovecot ACLs to the public folders > to reinforce my intentions of only allowing readonly access to certain users.ACL plugin makes flag changes and expunges also fail silently.> ACL questions (which might help others understand better as well): > Is a 'user=foo rl' entry supposed to remove any possible write permissions in a > maildir covered by such an ACL for user foo?Yes.> Is it even possible for an ACL to grant additional permissions above what is > granted to a user by standard unix permissions?No.> Does an empty ACL equate to the exact permissions granted by unix,No. It means "no permissions".> and should an > admin try to design an ACL to modify as little as possible, or should the admin > try to use enough acl entries to form a complete description of the desired > resulting permissions? I get the impression that broad ACLs should be listed > first, and more specific ACLS listed below can add or remove ACL permissions > applied earlier in the file as a 'last match' case.No comments about that, except that user=x always overrides group=y in the file. I'm not sure if the order matters within group= lines, probably yes. Then again groups aren't actually working at all currently (the code supports them, but there's no way to list user's groups currently).> Should the acl plugin be able to print a summary of the resulting permissions in > the log (with mail_debug=yes) when accessing a folder with an ACL?Maybe. Not that high priority and I'm not sure if it's that easy to implement. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20070511/050b30b3/attachment.bin>
Adam McDougall
2007-May-11 15:08 UTC
[Dovecot] Dovecot not handling r/o mailboxes completely, and problem with ACL as a workaround
On Fri, May 11, 2007 at 03:41:50PM +0300, Timo Sirainen wrote: On Sun, 2007-05-06 at 21:34 -0400, Adam McDougall wrote: > In short, upper directories are used to limit world access to just one group > 'decsall', and once inside, r/o world access could be used as an anonymous > substitution for 'decsall' and the group permissions would allow writing by > 'decsstaff'. I don't know how courier stores and checks mail directory > permissions, but it was aware when a user would not be able to delete or change > messages and would return a READ-ONLY imap error when an attempt to change a > message occurred Looks like Dovecot never returns maildir mailbox as READ-ONLY. So, your issues are: 1. Dovecot doesn't give an error when a flag is changed for mails. This is intentional, because otherwise you'd keep seeing "mailbox is read-only" errors all the time when reading mails (because clients keep setting the \Seen flag). Besides you're using a per-user index directory so the flag changes actually are saved. 2. EXPUNGE gives an internal error because it discovers too late that it can't unlink() the maildir file. This is a problem.. Even worse, you can't recover from it without deleting dovecot.index.log file. This should have two fixes: - If Dovecot ever notices that it can't unlink() a file, it should recover itself from it without giving internal errors. That would probably mean that the expunged message shows up as a new message. I'll see if I can get this fixed for v1.1. - It should treat the maildir as READ-ONLY so the EXPUNGE command is never written into dovecot.index.log in the first place. Fixed: http://dovecot.org/list/dovecot-cvs/2007-May/008734.html Thanks for addressing these issues. I applied this patch and it is a good improvement; I no longer get internal errors, and it has the effect of making thunderbird _appear_ to be able to completely delete an email because dovecot records the deleted flag and ignores the expunge, but mutt can see the messages marked deleted and does print to my screen that the mailbox is readonly when I try to expunge. So thunderbird with dovecot doesn't reflect the readonly warning that courier gives when trying to mark a message deleted, but that is probably acceptable to me since it looks like it is more beneficial for me to use ACLs. I would rather use ACLs so that "deleted" messages reappear on their own, rather than worry about users "deleting" shared mail and requiring me to erase their dovecot metadata incase they changed their mind (and don't want to use something besides thunderbird to fix it). I've also noticed that as soon as I enable the acl plugin, clients no longer have any access at all to shared mailboxes unless a dovecot-acl file is used to give them access previously granted by the file permissions. This might be worth noting in the documentation (if it is expected) so a person doesn't turn on the acl plugin without being aware of the consequences. I do appreciate that it works that way, so I will not forget to apply an acl. 3. EXPUNGE silently ignores no-permissions error. I'm not sure if this is good or not. In any case it shouldn't give an error if there are no messages with \Deleted flag, but perhaps it should give an error if there are.. Some clients do automatic expunges, so it would be annoying to see error messages constantly about it. I won't change this in v1.0 branch, but perhaps in v1.1. > As a workaround, I wondered if I could apply dovecot ACLs to the public folders > to reinforce my intentions of only allowing readonly access to certain users. ACL plugin makes flag changes and expunges also fail silently. > ACL questions (which might help others understand better as well): > Is a 'user=foo rl' entry supposed to remove any possible write permissions in a > maildir covered by such an ACL for user foo? Yes. > Is it even possible for an ACL to grant additional permissions above what is > granted to a user by standard unix permissions? No. > Does an empty ACL equate to the exact permissions granted by unix, No. It means "no permissions". > and should an > admin try to design an ACL to modify as little as possible, or should the admin > try to use enough acl entries to form a complete description of the desired > resulting permissions? I get the impression that broad ACLs should be listed > first, and more specific ACLS listed below can add or remove ACL permissions > applied earlier in the file as a 'last match' case. No comments about that, except that user=x always overrides group=y in the file. I'm not sure if the order matters within group= lines, probably yes. Then again groups aren't actually working at all currently (the code supports them, but there's no way to list user's groups currently). > Should the acl plugin be able to print a summary of the resulting permissions in > the log (with mail_debug=yes) when accessing a folder with an ACL? Maybe. Not that high priority and I'm not sure if it's that easy to implement.
Reasonably Related Threads
- assertion failed: (mailbox_list_is_valid_existing_name(_list, name))
- imap fs quota (rpc) won't work?
- Segfault when opening a public folder, dovecot 1.1 beta4
- Various uidlist and index errors with 1.1 on NFS
- unlink_directory(....Trash.NewFolder) failed: Directory not empty