> On 14/07/2022 04:56 EEST Brendan Braybrook <brendan at tucows.com> wrote: > > > hi all, > > i'm having a strange issue. i have a few dovecot instances running: > > 1- debian 10 with debian's dovecot-core 1:2.3.4.1-5+deb10u5 > 2- debian 10 with debian's dovecot-core 1:2.3.4.1-5+deb10u6 > 3- debian 11 with dovecot's dovecot-core 2:2.3.19.1-2+debian11 > > for all these, i've just been testing with a static userdb (ignore if > the login user was different; they all had the same userdb data.. > various different passdb configs were tried (none do any userdb > prefetching): > > userdb { > driver = static > args > mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ > home=/mail/mailstore03/492/177/b at b.com/ namespace=inbox,share > namespace/inbox/separator=/ namespace/share/separator=/ > namespace/share/prefix=boop/ > } > > instance #2 and #3 don't seem to pass all the namespace directives i'm > passing through to the imap process. > > (i originally found it happening with a lua userdb script then noticed > it with a dict userdb, but the simpler static userdb still shows it > happening so i don't think it's related to the different userdb > mechanisms themselves) > > debug looks like: > > auth: Debug: master userdb out: USER 3466461185 b at b.com > mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ > home=/mail/mailstore03/492/177/b at b.com/ namespace=inbox,share > namespace/inbox/separator=/ namespace/share/separator=/ > namespace/share/prefix=boop/ auth_mech=PLAIN > auth_token=bcdb366d56f4fceee83775a1b28ca89ccdf881fd > script-login(b at b.com)<56895><>: Debug: Added userdb setting: > mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ > script-login(b at b.com)<56895><>: Debug: Added userdb setting: > namespace=inbox,share > script-login(b at b.com)<56895><>: Debug: Added userdb setting: > namespace/inbox/separator=/ > script-login(b at b.com)<56895><>: Debug: Added userdb setting: > namespace/share/prefix=boop/ > script-login(b at b.com)<56895><>: Debug: Added userdb setting: > namespace/share/separator=/ > imap(b at b.com)<56894><9mFyornjaMcKyAUI>: Debug: Added userdb setting: > mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ > imap(b at b.com)<56894><9mFyornjaMcKyAUI>: Debug: Added userdb setting: > namespace=inbox,share > imap(b at b.com)<56894><9mFyornjaMcKyAUI>: Debug: Namespace inbox: > type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, > subscriptions=yes > location=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ > imap(b at b.com)<56894><9mFyornjaMcKyAUI>: Debug: Namespace : type=private, > prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes > location=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ > > the script-login task notes that these userdb settings are being added: > namespace=inbox,share > namespace/inbox/separator=/ > namespace/share/prefix=boop/ > namespace/share/separator=/ > > but the imap task only notes one: > namespace=inbox,share > > both of these two instances end up getting an error because the imap > task didn't pick up the prefix: > Error: namespace configuration error: Duplicate namespace prefix: "" > > instance #1 behaves differently. it's script-login task debug is the > same, but the imap debug notes: > imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: Added > userdb setting: > mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ > imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: Added > userdb setting: namespace=inbox,share > imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: Added > userdb setting: namespace/inbox/separator=/ > imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: > tc_empty_notify_plugin init > imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: > Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, > list=yes, subscriptions=yes > location=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ > imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: > Namespace : type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, > subscriptions=yes > location=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ > > on this one, the imap task notes that it added ALL the namespace > directives, and consequently doesn't have any namespace prefix collisions. > > the difference between 1:2.3.4.1-5+deb10u5 and 1:2.3.4.1-5+deb10u6 > doesn't seem that great, and the bug tracker for it doesn't indicate > anything around userdb would be patched > (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970386) > > but the problem still seems to exist in 2.3.19 - can anyone else see the > same thing happening with the current builds? > > thanks! > -brendanHi! Can you also provide doveconf -n? You are missing location for namespace/share the least, so it ends up in same location... Other than that, this config seems to work for me at least, with my simple config. Aki
Brendan Braybrook
2022-Jul-14 17:22 UTC
namespace directives from userdb not being applied
On 2022-07-13 23:37, Aki Tuomi wrote:> >> On 14/07/2022 04:56 EEST Brendan Braybrook <brendan at tucows.com> wrote: >> >> >> hi all, >> >> i'm having a strange issue. i have a few dovecot instances running: >> >> 1- debian 10 with debian's dovecot-core 1:2.3.4.1-5+deb10u5 >> 2- debian 10 with debian's dovecot-core 1:2.3.4.1-5+deb10u6 >> 3- debian 11 with dovecot's dovecot-core 2:2.3.19.1-2+debian11 >> >> for all these, i've just been testing with a static userdb (ignore if >> the login user was different; they all had the same userdb data.. >> various different passdb configs were tried (none do any userdb >> prefetching): >> >> userdb { >> driver = static >> args >> mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ >> home=/mail/mailstore03/492/177/b at b.com/ namespace=inbox,share >> namespace/inbox/separator=/ namespace/share/separator=/ >> namespace/share/prefix=boop/ >> } >> >> instance #2 and #3 don't seem to pass all the namespace directives i'm >> passing through to the imap process. >> >> (i originally found it happening with a lua userdb script then noticed >> it with a dict userdb, but the simpler static userdb still shows it >> happening so i don't think it's related to the different userdb >> mechanisms themselves) >> >> debug looks like: >> >> auth: Debug: master userdb out: USER 3466461185 b at b.com >> mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ >> home=/mail/mailstore03/492/177/b at b.com/ namespace=inbox,share >> namespace/inbox/separator=/ namespace/share/separator=/ >> namespace/share/prefix=boop/ auth_mech=PLAIN >> auth_token=bcdb366d56f4fceee83775a1b28ca89ccdf881fd >> script-login(b at b.com)<56895><>: Debug: Added userdb setting: >> mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ >> script-login(b at b.com)<56895><>: Debug: Added userdb setting: >> namespace=inbox,share >> script-login(b at b.com)<56895><>: Debug: Added userdb setting: >> namespace/inbox/separator=/ >> script-login(b at b.com)<56895><>: Debug: Added userdb setting: >> namespace/share/prefix=boop/ >> script-login(b at b.com)<56895><>: Debug: Added userdb setting: >> namespace/share/separator=/ >> imap(b at b.com)<56894><9mFyornjaMcKyAUI>: Debug: Added userdb setting: >> mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ >> imap(b at b.com)<56894><9mFyornjaMcKyAUI>: Debug: Added userdb setting: >> namespace=inbox,share >> imap(b at b.com)<56894><9mFyornjaMcKyAUI>: Debug: Namespace inbox: >> type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, >> subscriptions=yes >> location=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ >> imap(b at b.com)<56894><9mFyornjaMcKyAUI>: Debug: Namespace : type=private, >> prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes >> location=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ >> >> the script-login task notes that these userdb settings are being added: >> namespace=inbox,share >> namespace/inbox/separator=/ >> namespace/share/prefix=boop/ >> namespace/share/separator=/ >> >> but the imap task only notes one: >> namespace=inbox,share >> >> both of these two instances end up getting an error because the imap >> task didn't pick up the prefix: >> Error: namespace configuration error: Duplicate namespace prefix: "" >> >> instance #1 behaves differently. it's script-login task debug is the >> same, but the imap debug notes: >> imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: Added >> userdb setting: >> mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ >> imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: Added >> userdb setting: namespace=inbox,share >> imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: Added >> userdb setting: namespace/inbox/separator=/ >> imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: >> tc_empty_notify_plugin init >> imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: >> Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, >> list=yes, subscriptions=yes >> location=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ >> imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: >> Namespace : type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, >> subscriptions=yes >> location=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ >> >> on this one, the imap task notes that it added ALL the namespace >> directives, and consequently doesn't have any namespace prefix collisions. >> >> the difference between 1:2.3.4.1-5+deb10u5 and 1:2.3.4.1-5+deb10u6 >> doesn't seem that great, and the bug tracker for it doesn't indicate >> anything around userdb would be patched >> (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970386) >> >> but the problem still seems to exist in 2.3.19 - can anyone else see the >> same thing happening with the current builds? >> >> thanks! >> -brendan > > Hi! > > Can you also provide doveconf -n? You are missing location for namespace/share the least, so it ends up in same location...ah, yes. i stripped down the bits i was sending in the userdb result to make the case simpler. if i include namespace/share/location, the same problem seems to occur, as the imap process doesn't include that either.> Other than that, this config seems to work for me at least, with my simple config.here's the config. i'll try it again with fewer plugins and a more minimal config as well. update: it actually seems like some issue with the config - i tried the userdb with the stock config files and it seemed to work fine. i'll report back when i figure out what is going on. # 2.3.19.1 (9b53102964): /var/run/dovecot/dovecot.conf.real # Pigeonhole version 0.5.19 (4eae2f79) # OS: Linux 5.10.0-16-amd64 x86_64 Debian 11.4 # Hostname: imap-p2-01 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes debug_log_path = /var/log/dovecot-debug.log first_valid_uid = 8 imap_id_retain = yes info_log_path = syslog lock_method = dotlock log_timestamp login_trusted_networks = 10.200.0.0/16 10.5.0.0/16 10.0.0.0/8 mail_debug = yes mail_fsync = always mail_gid = mail mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = zlib quota tc_mail_log notify tc_proc tc_max_msgs tc_empty_notify lazy_expunge mail_temp_dir = /var/run/dovecot_tmp mail_uid = mail maildir_very_dirty_syncs = yes managesieve_sieve_capability = fileinto reject mmap_disable = yes namespace inbox { inbox = yes location mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox "Sent Items" { auto = subscribe special_use = \Sent } mailbox Spam { auto = no } mailbox Trash { auto = subscribe special_use = \Trash } prefix } passdb { args = file=/home/brendan/dovecot_auth.lua blocking=yes driver = lua } plugin { acl = vfile:/he/dovecot/conf/lazyexpunge.acl acl_shared_dict = file:/mail/users/cluster/dovecot/shared-mailboxes.db antispam_allow_append_to_spam = yes antispam_backend = pipe antispam_pipe_program = /he/dovecot/utils/he_spamtrain.pl antispam_pipe_program_args = --user=%u antispam_pipe_program_notspam_arg = --falsepositive antispam_pipe_program_spam_arg = --missed antispam_pipe_tmpdir = /var/run/dovecot_as_tmp antispam_signature_missing = move antispam_spam antispam_trash_pattern_ignorecase = trash;OSRSSoftDelete lazy_expunge = OSRSSoftDelete lazy_expunge_only_last_instance = yes mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid flags hetag memcached_servers = mcache01-stub sieve_extensions = fileinto reject zlib_save = gz zlib_save_level = 6 } protocols = imap pop3 service anvil { unix_listener anvil-auth-penalty { mode = 00 } } service auth { client_limit = 3500 } service imap-login { inet_listener imap { address = 0 port = 143 } inet_listener imaps { address = 0 port = 0 } process_limit = 182 process_min_avail = 91 service_count = 0 } service imap-postlogin { executable = script-login -d /he/dovecot/utils/post_login.sh } service imap { executable = imap imap-postlogin process_limit = 7855 vsz_limit = 0 } service managesieve-login { inet_listener sieve { address = 0 port = 4190 } } service managesieve-postlogin { executable = script-login -d /he/dovecot/utils/managesieve_postlogin.pl } service managesieve { executable = managesieve managesieve-postlogin } service pop3-login { inet_listener pop3 { address = 0 port = 110 } inet_listener pop3s { address = 0 port = 0 } process_limit = 182 process_min_avail = 91 service_count = 0 } service pop3-postlogin { executable = script-login -d /he/dovecot/utils/post_login.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 1278 vsz_limit = 512 M } service stats { client_limit = 12000 } userdb { args mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ home=/mail/mailstore03/492/177/b at b.com/ namespace=inbox,share namespace/inbox/separator=/ namespace/share/separator=/ namespace/share/prefix=boop/ driver = static } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 30 mail_plugins = zlib quota tc_mail_log notify tc_proc tc_max_msgs tc_empty_notify lazy_expunge imap_quota acl imap_acl } protocol pop3 { mail_max_userip_connections = 30 pop3_reuse_xuidl = yes pop3_uidl_format = %08Xv%08Xu } the same config, but with a more stripped down userdb response shows the problem in action too: auth: Debug: master userdb out: USER 1535246337 b at x.com mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ home=/mail/mailstore03/492/177/b at b.com/ namespace=inbox namespace/inbox/separator=/ auth_mech=PLAIN auth_token=08c7b084f58d76b028a96eed55f16c57ce3c7ff8 script-login(b at x.com)<59977><>: Debug: Added userdb setting: mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ script-login(b at x.com)<59977><>: Debug: Added userdb setting: namespace=inbox script-login(b at x.com)<59977><>: Debug: Added userdb setting: namespace/inbox/separator=/ imap(b at x.com)<59976><Kc80nMbjXoQKyAUI>: Debug: Added userdb setting: mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ imap(b at x.com)<59976><Kc80nMbjXoQKyAUI>: Debug: Added userdb setting: namespace=inbox imap(b at x.com)<59976><Kc80nMbjXoQKyAUI>: Debug: tc_empty_notify_plugin init imap(b at x.com)<59976><Kc80nMbjXoQKyAUI>: Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/ i don't get an error here since i'm only passing the one namespace, but the same thing occurs: the script-login event logs both namespace settings, but the imap event only logs the "namespace=inbox" one - the "namespace/inbox/separator=/" one never gets applied and the inbox namespace still has the default separator the same thing occurs if i try and send prefix/hidden/list/subscriptions - the imap process just ignores them