Hello, I'm experiencing slow flags changes over IMAP IDLE: - If I start an IDLE session / command - Change the flags of some messages via another email app - Dovecot can take a minute or more to notify the IDLE connection about flags changes If I use another email app to add or remove a message, Dovecot sends that (EXISTS / EXPUNGE) instantly and also flushes the (so far delayed) flags changes. So it does "know" already. The system is Debian Testing, 64 bit, file system is ext4. Dovecot is 2.3.4.1 (f79e8e7e4) - I believe from Debian repos (I had Dovecot's own repo added at some point, disabled now). My mail is stored under ~/mail/.imap (not sure what this format is called), I mean not "single file mbox". I have not changed any IDLE related config settings: doveconf | grep -i idle default_idle_kill = 1 mins director_ping_idle_timeout = 30 secs imap_idle_notify_interval = 2 mins imapc_max_idle_time = 29 mins mailbox_idle_check_interval = 30 secs What can I do to make Dovecot notify IDLE clients about flags changes - more quickly? Preferably near-instant? -- Kostya Vasilyev kman at fastmail.com
Just tried running with mail_debug=yes and unfortunately can't see anything useful. It seems that flags changes (when set from clients) are not logged. Any way to debug this? Maybe it's intended behavior (to buffer up flags changes until later time, unless flushed with adding / removing messages)? If it is working as intended, is there a setting to tune down the delay? -- Kostya Vasilyev kman at fastmail.com On Sun, Mar 10, 2019, at 11:15 AM, Kostya Vasilyev via dovecot wrote:> Hello, > > I'm experiencing slow flags changes over IMAP IDLE: > > - If I start an IDLE session / command > - Change the flags of some messages via another email app > - Dovecot can take a minute or more to notify the IDLE connection about > flags changes > > If I use another email app to add or remove a message, Dovecot sends > that (EXISTS / EXPUNGE) instantly and also flushes the (so far delayed) > flags changes. So it does "know" already. > > The system is Debian Testing, 64 bit, file system is ext4. > > Dovecot is 2.3.4.1 (f79e8e7e4) - I believe from Debian repos (I had > Dovecot's own repo added at some point, disabled now). > > My mail is stored under ~/mail/.imap (not sure what this format is > called), I mean not "single file mbox". > > I have not changed any IDLE related config settings: > > doveconf | grep -i idle > default_idle_kill = 1 mins > director_ping_idle_timeout = 30 secs > imap_idle_notify_interval = 2 mins > imapc_max_idle_time = 29 mins > mailbox_idle_check_interval = 30 secs > > What can I do to make Dovecot notify IDLE clients about flags changes - > more quickly? Preferably near-instant? > > -- > Kostya Vasilyev > kman at fastmail.com >
On 10.3.2019 10.14, Kostya Vasilyev via dovecot wrote:> Hello, > > I'm experiencing slow flags changes over IMAP IDLE: > > - If I start an IDLE session / command > - Change the flags of some messages via another email app > - Dovecot can take a minute or more to notify the IDLE connection about flags changes > > If I use another email app to add or remove a message, Dovecot sends that (EXISTS / EXPUNGE) instantly and also flushes the (so far delayed) flags changes. So it does "know" already. > > The system is Debian Testing, 64 bit, file system is ext4. > > Dovecot is 2.3.4.1 (f79e8e7e4) - I believe from Debian repos (I had Dovecot's own repo added at some point, disabled now). > > My mail is stored under ~/mail/.imap (not sure what this format is called), I mean not "single file mbox". > > I have not changed any IDLE related config settings: > > doveconf | grep -i idle > default_idle_kill = 1 mins > director_ping_idle_timeout = 30 secs > imap_idle_notify_interval = 2 mins > imapc_max_idle_time = 29 mins > mailbox_idle_check_interval = 30 secs > > What can I do to make Dovecot notify IDLE clients about flags changes - more quickly? Preferably near-instant? >Can you send doveconf -n? Aki
Aki, Please see below. On Mon, Mar 11, 2019, at 12:58 PM, Aki Tuomi via dovecot wrote:> > On 10.3.2019 10.14, Kostya Vasilyev via dovecot wrote: > > Hello, > > > > I'm experiencing slow flags changes over IMAP IDLE: > > > > - If I start an IDLE session / command > > - Change the flags of some messages via another email app > > - Dovecot can take a minute or more to notify the IDLE connection about flags changes > > > > If I use another email app to add or remove a message, Dovecot sends that (EXISTS / EXPUNGE) instantly and also flushes the (so far delayed) flags changes. So it does "know" already. > > > > The system is Debian Testing, 64 bit, file system is ext4. > > > > Dovecot is 2.3.4.1 (f79e8e7e4) - I believe from Debian repos (I had Dovecot's own repo added at some point, disabled now). > > > > My mail is stored under ~/mail/.imap (not sure what this format is called), I mean not "single file mbox". > > > > I have not changed any IDLE related config settings: > > > > doveconf | grep -i idle > > default_idle_kill = 1 mins > > director_ping_idle_timeout = 30 secs > > imap_idle_notify_interval = 2 mins > > imapc_max_idle_time = 29 mins > > mailbox_idle_check_interval = 30 secs > > > > What can I do to make Dovecot notify IDLE clients about flags changes - more quickly? Preferably near-instant? > > > Can you send doveconf -n?---------------------------- doveconf -n # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.18.16-x86_64-linode118 x86_64 Debian buster/sid # Hostname: kman.mobi auth_default_realm = kman.mobi auth_mechanisms = plain login login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_location = mbox:~/mail:INBOX=/var/mail/%n mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { auto = create special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = create special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = create special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } protocols = " imap" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service imap-login { inet_listener imap { port = 0 } } service pop3-login { inet_listener pop3 { port = 0 } } ssl_cert = </etc/letsencrypt/live/kman.mobi/fullchain.pem ssl_cipher_list = kECDHE+CHACHA20:kECDHE+AESGCM:kECDHE+AES:!AESCCM:!aNULL ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol imap { mail_max_userip_connections = 20 } ---------------------------- -- K
On 10 Mar 2019, at 10.14, Kostya Vasilyev via dovecot <dovecot at dovecot.org> wrote:> > My mail is stored under ~/mail/.imap (not sure what this format is called), I mean not "single file mbox". > > I have not changed any IDLE related config settings: > > doveconf | grep -i idle > default_idle_kill = 1 mins > director_ping_idle_timeout = 30 secs > imap_idle_notify_interval = 2 mins > imapc_max_idle_time = 29 mins > mailbox_idle_check_interval = 30 secs > > What can I do to make Dovecot notify IDLE clients about flags changes - more quickly? Preferably near-instant?It should simply just work, assuming there aren't any weird inotify limits, but you should get errors logged about reaching those. You could see if it makes any difference to set mailbox_idle_check_interval=1s -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20190312/73d00557/attachment-0001.html>
Timo, On Tue, Mar 12, 2019, at 3:56 AM, Timo Sirainen via dovecot wrote:> On 10 Mar 2019, at 10.14, Kostya Vasilyev via dovecot <dovecot at dovecot.org> wrote: >> >> My mail is stored under ~/mail/.imap (not sure what this format is called), I mean not "single file mbox". >> >> I have not changed any IDLE related config settings: >> >> doveconf | grep -i idle >> default_idle_kill = 1 mins >> director_ping_idle_timeout = 30 secs >> imap_idle_notify_interval = 2 mins >> imapc_max_idle_time = 29 mins >> mailbox_idle_check_interval = 30 secs >> >> What can I do to make Dovecot notify IDLE clients about flags changes - more quickly? Preferably near-instant? > > It should simply just work, assuming there aren't any weird inotify limits, but you should get errors logged about reaching those. You could see if it makes any difference to set mailbox_idle_check_interval=1s >mailbox_idle_check_interval = 1 secs did not help Here is an interesting case - Let's say I have my IDLE connection. - Connection "A" - an email app where I set or clear \Flagged on two messages. The IDLE connection is silent (no unsolicited notifications about these flags). - Connection "B" - an email app where I do a refresh - it does a SELECT (CONDSTORE) followed by FETCH UID FLAGS CHANGEDSINCE (because it sees that there are only flags changes). ---> the IDLE connection is notified about flags changes immediately right as Connection "B" is pulling its changes. I tried a direct network connection (netcat) instead of Connection "B" and the actual trigger is SELECT (CONDSTORE). - Start up IDLE connection - Use Connection "A" to make flags changes ( IDLE connection is silent ) - Use a netcat connection to SELECT (CONDSTORE) Dovecot flushes flags to the IDLE connection immediately - Doing SELECT (without CONDSTORE) does not Looks like some sort of bug in Dovecot related to CONDSTORE? This can probably be reproduced with several direct network connections using netcat / openssl s_client - and CONDSTORE seems to be an important part of the scenario. Ideas? -- K -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20190312/5b532d43/attachment-0001.html>