I think the most common Dovecot newbie problems are: - Not looking at the (correct) logs for errors - Authentication problems - Mail location problems - Mail permission problems Below are some of my ideas how I could stop people from asking these questions with future Dovecot (v1.1+) releases. Suggestions welcome. Distribution people especially should say if they're against some change. * Logging The log file problem is the most annoying one, because a lot of the other problems can be solved once the admin figures out that Dovecot is actually giving useful error messages. Often the admin is only looking at the log file where "info" messages go (eg mail.log) because Dovecot logs its startup message and login messages there, but not where the errors go (eg mail.err). This has happened even with people who in general are experienced sysadmins. Logging to eg. /var/log/dovecot.log by default would be helpful here, but it's probably better to log via syslog by default. Cyrus logs to "local6" by default. Perhaps for Dovecot v1.1 I should make that the default too? Looks like in my Debian system the info messages then go to both /var/log/messages and /var/log/syslog. Error messages only go to /var/log/syslog. So there still isn't by default a single log file where the errors are logged, but it might help a bit. * Authentication Authentication problems can usually be solved by telling the user to set auth_debug_passwords=yes and looking at the logs. I'm not sure if there's anything that can be helped in here. Except the logging message could be updated a bit: "Aborted login: user=<asdga>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, 1 failed authentications" So the last "n failed authentications" could be added, where n could also be 0. Another possibility would be to make Dovecot remember if there have been any successful logins (/var/lib/dovecot/success file) and if not, give a bit more helpful error messages: - Client gets: "NO Authentication failed. Refer to server log for more information." instead of the normal "NO Authentication failed." - Log contains: "Aborted login: user=<asdga>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, 1 failed authentications (set auth_debug_passwords=yes to debug the problem)" I'm not sure if this is a good idea. * Mail location It seems to be difficult for some people to set mail_location. I don't know if anything can be done before Dovecot v2.0 where I'll split it to multiple settings, such as: driver = maildir root_dir = ~/Maildir index_dir = /var/indexes/%u Another problem that seems to be difficult to understand is why the mail directory contains "HOME_DIRECTORY_USED_BUT_NOT_GIVEN_BY_USERDB". The whole userdb concept appears to be weird. This could anyway be fixed by giving an error message earlier and failing the login with internal error. Probably the best place to give the error message would be already in the userdb lookup in dovecot-auth, but that would require that doveoct-auth knows if the home directory is really needed, and to give a useful error message it would also need to tell where it's tried to be used (mail_location, or some namespace's location, or ..). Probably too much trouble to be worth it. So the next best thing is to give the error when it's used: "Home directory is used in mail_location, but userdb didn't return a home directory" It would be nice if it didn't say userdb, but rather the userdb's name. I guess that would be possible if dovecot-auth told master (or deliver) which userdb was used, but that would normally be just extra overhead. * Mail permissions If mail location is difficult for some, then the concept of UIDs are how they're used in Dovecot is pretty much impossible for some to understand. One of the problems is that there exists "dovecot" user. So people think that their mails should be owned by the dovecot user. Although I've mentioned in everywhere I can think of that this should not be done, it won't help because either people won't read the pages or even if they do, they somehow still fail to ignore it even though it's written in bold. So unless people (and most importantly, distributions) are against it, I think the "dovecot" user should be renamed. "dovelogin" perhaps. In future I might split dovecot-auth even more, and then I would like to create a "doveauth" user as well. Another possibility would be to drop the dovecot user completely and instead use "nobody". That would mean that other nobody processes could kill Dovecot's login processes, but that's pretty much it. Once people understand that "dovecot" isn't the right user, they hit the next problem: "How do I tell Dovecot to run as vmail user?" I can paste links to wiki pages or tell them to "make userdb return uid=vmail", but that just doesn't seem to get through. There needs to be an easier way, and I think I figured out what it is: Add new "mail_uid" and "mail_gid" settings to dovecot.conf. Deprecate user_global_uid/gid in dovecot-ldap.conf and make all the userdbs mention that the uid/gid returned by userdb can be used to override the global mail_uid/gid. Perhaps also add "mail_home" template. This change makes it unnecessary to have a userdb configured at all. - "How do I tell Dovecot to run as vmail user?" - "Set mail_uid = vmail" in dovecot.conf - "Thanks" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20070515/165a874c/attachment.bin>
Timo, In general, the better the docs are and the better the error messages are the less support they will need. If the log message said "Failure XXX, check YYY to see if it is set" giving the user as much help as possible. You could have a number of sanity checks at startup where dovecot checks out the environment verifying that directories exist and perhaps do some write testing to make sure areas that need to be writable actually are.
Am Dienstag, den 15.05.2007, 19:40 +0300 schrieb Timo Sirainen:> I think the most common Dovecot newbie problems are: > > - Not looking at the (correct) logs for errors > - Authentication problems > - Mail location problems > - Mail permission problems > > Below are some of my ideas how I could stop people from asking these > questions with future Dovecot (v1.1+) releases. Suggestions welcome. > Distribution people especially should say if they're against some > change. > > * Logging > > The log file problem is the most annoying one, because a lot of the > other problems can be solved once the admin figures out that Dovecot is > actually giving useful error messages. Often the admin is only looking > at the log file where "info" messages go (eg mail.log) because Dovecot > logs its startup message and login messages there, but not where the > errors go (eg mail.err). This has happened even with people who in > general are experienced sysadmins. > > Logging to eg. /var/log/dovecot.log by default would be helpful here, > but it's probably better to log via syslog by default. Cyrus logs to > "local6" by default. Perhaps for Dovecot v1.1 I should make that the > default too? Looks like in my Debian system the info messages then go to > both /var/log/messages and /var/log/syslog. Error messages only go > to /var/log/syslog. So there still isn't by default a single log file > where the errors are logged, but it might help a bit.Logging is always a special issue. No matter where and how much is logged, you can bet that either not all is read or it is misinterpreted. I say this as a first hand prove for that, there are many times when I saw an error message but did not actually read it :-) Getting people to actually read _and_ understand what a log message means is very difficult on a very psychological level :-) But of course, good logging is vital in any circumstance and logging to an own facility would be a good idea anyhow.> * Authentication > > Authentication problems can usually be solved by telling the user to set > auth_debug_passwords=yes and looking at the logs. I'm not sure if > there's anything that can be helped in here. Except the logging message > could be updated a bit: > > "Aborted login: user=<asdga>, method=PLAIN, rip=127.0.0.1, > lip=127.0.0.1, secured, 1 failed authentications" > > So the last "n failed authentications" could be added, where n could > also be 0. > > Another possibility would be to make Dovecot remember if there have been > any successful logins (/var/lib/dovecot/success file) and if not, give a > bit more helpful error messages: > > - Client gets: "NO Authentication failed. Refer to server log for more > information." instead of the normal "NO Authentication failed." > - Log contains: "Aborted login: user=<asdga>, method=PLAIN, > rip=127.0.0.1, lip=127.0.0.1, secured, 1 failed authentications (set > auth_debug_passwords=yes to debug the problem)" > > I'm not sure if this is a good idea.The overhead for this on heavily loaded systems would be quite significant, IMHO. This feature should only be active when explicitly activated in the configuration, so that would not be much of a change to the current situation :-)> * Mail location > > It seems to be difficult for some people to set mail_location. I don't > know if anything can be done before Dovecot v2.0 where I'll split it to > multiple settings, such as: > > driver = maildir > root_dir = ~/Maildir > index_dir = /var/indexes/%u > > Another problem that seems to be difficult to understand is why the mail > directory contains "HOME_DIRECTORY_USED_BUT_NOT_GIVEN_BY_USERDB". The > whole userdb concept appears to be weird. This could anyway be fixed by > giving an error message earlier and failing the login with internal > error. > > Probably the best place to give the error message would be already in > the userdb lookup in dovecot-auth, but that would require that > doveoct-auth knows if the home directory is really needed, and to give a > useful error message it would also need to tell where it's tried to be > used (mail_location, or some namespace's location, or ..). Probably too > much trouble to be worth it. So the next best thing is to give the error > when it's used: > > "Home directory is used in mail_location, but userdb didn't return a > home directory" > > It would be nice if it didn't say userdb, but rather the userdb's name. > I guess that would be possible if dovecot-auth told master (or deliver) > which userdb was used, but that would normally be just extra overhead. > > * Mail permissions > > If mail location is difficult for some, then the concept of UIDs are how > they're used in Dovecot is pretty much impossible for some to > understand. > > One of the problems is that there exists "dovecot" user. So people think > that their mails should be owned by the dovecot user. Although I've > mentioned in everywhere I can think of that this should not be done, it > won't help because either people won't read the pages or even if they > do, they somehow still fail to ignore it even though it's written in > bold. > > So unless people (and most importantly, distributions) are against it, I > think the "dovecot" user should be renamed. "dovelogin" perhaps. In > future I might split dovecot-auth even more, and then I would like to > create a "doveauth" user as well. > > Another possibility would be to drop the dovecot user completely and > instead use "nobody". That would mean that other nobody processes could > kill Dovecot's login processes, but that's pretty much it. > > Once people understand that "dovecot" isn't the right user, they hit the > next problem: "How do I tell Dovecot to run as vmail user?" I can paste > links to wiki pages or tell them to "make userdb return uid=vmail", but > that just doesn't seem to get through. There needs to be an easier way, > and I think I figured out what it is: > > Add new "mail_uid" and "mail_gid" settings to dovecot.conf. Deprecate > user_global_uid/gid in dovecot-ldap.conf and make all the userdbs > mention that the uid/gid returned by userdb can be used to override the > global mail_uid/gid. Perhaps also add "mail_home" template. This change > makes it unnecessary to have a userdb configured at all. > > - "How do I tell Dovecot to run as vmail user?" > - "Set mail_uid = vmail" in dovecot.conf > - "Thanks"IMO the best way to prevent basic errors and basic questions is to provide as many sample configurations as possible (eg. in the wiki), maybe organized as some kind of recipes. In the end most setups boil down to say 4 or 5 basic types, probably differing mostly between authentication settings. I've upgraded our and come clients' dovecot installations multiple times, and to be honest I did not like the extensive documentation provided in the default configuration files. Saying 'I did not like it' does not mean that the information was useless, on the contrary, the amount was just overwhelming ... Sometimes too much information is ... too much :-) Major drawback for sample configurations is that they need to be maintained in order to keep up with the latest and greatest settings. But talking about documentation, what I really miss are manual or info pages for dovecot/dovecot.conf but of course, somebody has to write them ... -- Udo Rader bestsolution.at EDV Systemhaus GmbH http://www.bestsolution.at -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Dies ist ein digital signierter Nachrichtenteil URL: <http://dovecot.org/pipermail/dovecot/attachments/20070516/7fe12f69/attachment.bin>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 15 May 2007, Timo Sirainen wrote:> * Logging > > Logging to eg. /var/log/dovecot.log by default would be helpful here,Personally, I found logging to a separate file helpful during setting up Dovecot, because you can focus on Dovecot that way.> * Authentication > > So the last "n failed authentications" could be added, where n could > also be 0.This message is user-specific, isn't it? How about a login-statistics of the last 10 or so attempts? Maybe one can use the utmp/wtmp service.> rip=127.0.0.1, lip=127.0.0.1, secured, 1 failed authentications (set > auth_debug_passwords=yes to debug the problem)"It usually is not enough to set the option, I think. Shouldn't it be: "read doc/debug_auth_problems.txt" with one of the first text therein "set auth_debug_passwords=yes"> * Mail location > > It seems to be difficult for some people to set mail_location. I don'tIMHO: It looks wierd first, but the doc is OK. The most problem for myself was that it's spelled maildir:%h/maildir rather than maildir=%h/maildir, I mean anything else is in key=value format.> "Home directory is used in mail_location, but userdb didn't return a > home directory" > > It would be nice if it didn't say userdb, but rather the userdb's name.When the userdb is logged, you could add a note that the particular userdb is user-specific, but mentioned in the logs.> * Mail permissions > > Another possibility would be to drop the dovecot user completely and > instead use "nobody". That would mean that other nobody processes could > kill Dovecot's login processes, but that's pretty much it.I would say no :-) Stick with a specific user. Otherwise, if it is still possible to run Dovecot with a specific user, ... . I like to see in ps or top, which process belongs to which "package" at easy glancing.> Add new "mail_uid" and "mail_gid" settings to dovecot.conf. Deprecate > user_global_uid/gid in dovecot-ldap.conf and make all the userdbs > mention that the uid/gid returned by userdb can be used to override the > global mail_uid/gid. Perhaps also add "mail_home" template. This change > makes it unnecessary to have a userdb configured at all. > > - "How do I tell Dovecot to run as vmail user?" > - "Set mail_uid = vmail" in dovecot.conf > - "Thanks"As long as this is transparent, I mean, one need no "mail_uid" in the LDAP backends, this might solve lots of vmail support requests. If I see it correct, this effectively makes userdb static the default - without explizit userdb { } block. BTW: Is mail_home the same as mail_location then? Bye, - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iQEVAwUBRkqxGC9SORjhbDpvAQJFJggAs9M4K9V/cYHccUlW76WWOSgmQVoMYVDV EyVPUQelNcOo0FLYt+OxBmMhbtr1Hd4DxgkiHL1PIofpDlusSQ30D+kGVwBLIvj4 ZQJk8AzhWaiqVfHk6BXzDE+MA4Bi3f6dl8lQBntDCbJ9THtX2wOoGcWoQLIngjPp gBHgzHmffJQAQLwbUrgrI5BPhaGa4fXTXOd1ZtON+2Key4K6bqAvMfvElgdWk4Z1 7tdVB2MciIrUvSe7qzlA3dgQlpYCIWrYPAq9Dswnogj/0ApQbxccpa0+YZ4OdGe5 FlM6a1PaOI2oNXwafhjwD0+TOH3YmDRVeqPAasS66jBsddYGa78/uw==5ZOg -----END PGP SIGNATURE-----
Hi, there's no easy way to get rid of newbie problems: right now dovecot is quite a complex software, so configuration is complex. Even more, all the userdb/passdb thing is really powerfull, but also a bit diferent of anything I was used to, so it's normal that people is a bit confused in the begining. Anyway, I'm quite sure that in apache's, postfix's or sendmail's mailing lists there are far more newbie questions than here, and that isn't necesary bad. For me, the easier solution would be an entry in the wiki with well documented common configs: - system users (using pam/passwd) - ldap (with a common conf) - mysql - postgresql There is something similar in http://wiki.dovecot.org/DovecotServerInstallations but there the main goal seems to be the hardware. I have to document my current config in spanish for my job, and it wouldn't be a hard job to translate it to english a upload it somewhere. Aaaaaaaaaagur. El Martes, 15 de Mayo de 2007 18:40, Timo Sirainen escribi?:> I think the most common Dovecot newbie problems are: > > - Not looking at the (correct) logs for errors > - Authentication problems > - Mail location problems > - Mail permission problems > > Below are some of my ideas how I could stop people from asking these > questions with future Dovecot (v1.1+) releases. Suggestions welcome. > Distribution people especially should say if they're against some > change.-- Joseba Torre. CIDIR Bizkaia. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20070516/c9ab4687/attachment.bin>
On 5/16/2007 Joseba Torre (joseba.torre at ehu.es) wrote:> For me, the easier solution would be an entry in the wiki with well > documented common configs: > > - system users (using pam/passwd) > - ldap (with a common conf) > - mysql > - postgresqlI agree completely... also, explanations for what is required to convert from, say, cyrus-sasl to dovecot-sasl (I want to do that someday soon), from common LDA (ie postfix's) to the dovecot LDA (want to do that as well), etc... Wish I was fluent in all of these, I'd do it myself... if no one has by the time I get around to it, I'll document what I do and add it to the wiki... -- Best regards, Charles
Udo Rader wrote:> I've upgraded our and come clients' dovecot installations multiple times, and > to be honest I did not like the extensive documentation provided in the default > configuration files. Saying 'I did not like it' does not mean that the information > was useless, on the contrary, the amount was just overwhelming ... > > Sometimes too much information is ... too much :-) >I see what you're getting at, but personally i really like having plenty of documentation in the config files. That way when you're editing the config file, you can know exactly what you're doing and have good descriptions right in front of you.... Of course this is no substitute for other documentation like manual/wiki etc MT
Timo Sirainen writes:> actually giving useful error messages. Often the admin is only looking > at the log file where "info" messages go (eg mail.log) because Dovecot > logs its startup message and login messages there, but not where the > errors go (eg mail.err). This has happened even with people who in > general are experienced sysadmins.I had problems with Dovecot because of it's logs when I first started. think that syslog should be default and to point it to "mail.info" level. If the user tries to setup the log variables errors should be sent to syslog "mail.crit". I don't recall exact details, but I recall that when I first tried to use the log files I made a mistake and dovecot did not help me find the error.> Logging to eg. /var/log/dovecot.log by default would be helpful here, > but it's probably better to log via syslog by default.I agree on syslog as default.> Cyrus logs to "local6" by default.I think "mail.info" may be a better choice, although this may be somewhat OS dependant. Many people may not even know where their syslog configuration file is or even how to configure it. Because of this whatever is the most commonly used syslog call may be the best choice. In FreeBSD there is a default "mail.info". Anyone familiar with Linux Distros and/or other BSDs know how common is an entry for ""mail.info"" in syslog.conf.
Joseba Torre writes:> there's no easy way to get rid of newbie problems: right now dovecot is quite > a complex softwareI tried Cyrus once, and have been working with Courier for nearly 2 years before I tried Dovecot. In my opinion Dovecot is much simpler than Cyrus, but perhaps more complex than Courier. However I found the dovecot documentation to be very helpfull. The problem is how to make the documentation friendlier to people with little or no exprience managing a pop/imap server.> common configs: > - system users (using pam/passwd) > - ldap (with a common conf) > - mysql > - postgresqlAgree. Having samples and sections for those different configurations will go a long way to help new users.