dovecot - Mar 2005 - aha! (was apple mail vs. v0.99)

it seems that when you say "check for new mail" in apple mail, it
sends
a NOOP to the imap server to get the response.   in uw, the conversation
is something like:

25 NOOP
25 OK NOOP completed
26 NOOP
* 2796 EXISTS
* 7 RECENT
26 OK NOOP completed

however, with dovecot, if it has already notified the client that new
mail has arrived, it will look something like:

* 24 EXISTS
* 2 RECENT
22 NOOP
22 OK NOOP completed.
23 NOOP
23 OK NOOP completed.

(the counts are different because these are seperate accounts)

so evidently, apple mail is too stupid to realize to read info from the
server unless it asked for it...

so now that i think i have a short-term fix (disabling notification for 
new mail server-side), i can breathe a sigh of relief.  for a long-term
fix, would it be possible to have the server always notify the client
of new mail after a NOOP?


thanks,
	sean
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20050314/1e8f440b/attachment-0001.bin>

Hello,

On Mon, Mar 14, 2005 at 12:03:24PM -0500, sean finney
wrote:
> it seems that when you say "check for new mail" in apple mail, it
sends
> a NOOP to the imap server to get the response.   in uw, the conversation
> is something like:
> 
> 25 NOOP
> 25 OK NOOP completed
> 26 NOOP
> * 2796 EXISTS
> * 7 RECENT
> 26 OK NOOP completed
> 
> however, with dovecot, if it has already notified the client that new
> mail has arrived, it will look something like:
> 
> * 24 EXISTS
> * 2 RECENT
> 22 NOOP
> 22 OK NOOP completed.
> 23 NOOP
> 23 OK NOOP completed.
> 
> (the counts are different because these are seperate accounts)
> 
> so evidently, apple mail is too stupid to realize to read info from the
> server unless it asked for it...
> 
> so now that i think i have a short-term fix (disabling notification for 
> new mail server-side), i can breathe a sigh of relief.  for a long-term
> fix, would it be possible to have the server always notify the client
> of new mail after a NOOP?
According to RFC3501 (is this the right one?), NOOP can be used to check
for new mail, but it doesn't say that the server should return status
data every time NOOP is issued.

[...]
      The NOOP command always succeeds.  It does nothing.

      Since any command can return a status update as untagged data, the
      NOOP command can be used as a periodic poll for new messages or
      message status updates during a period of inactivity (this is the
      preferred method to do this).  The NOOP command can also be used
      to reset any inactivity autologout timer on the server.
[...]

The RFC states that the IMAP server MUST send status data to the client
if and when mailbox size changes (and I assume dovecot does that) but
again, once is assumed enough and the client is REQUIRED to record any status
updates (5.2):
[...]
   Regardless of what implementation decisions a client makes on
   remembering data from the server, a client implementation MUST record
   mailbox size updates.  It MUST NOT assume that any command after the
   initial mailbox selection will return the size of the mailbox.
[...]

Clearly, Apple Mail breaks the RFC if it assumes that the server will
report the same data over and over again after each NOOP.
There is a standard way to poll for new messages, i.e. using RECENT
and EXISTS commands (7.3.1 and 7.3.2). If Apple Mail did that instead of
using NOOP, there'd be no problem.

Please, don't hesitate to correct me if I'm wrong.

Regards,

-- 
Dominik 'Rathann' Mierzejewski <rathann*at*icm.edu.pl>
Interdisciplinary Centre for Mathematical and Computational Modelling
Warsaw University  |  http://www.icm.edu.pl  |  tel. +48 (22) 5540810

On 14.3.2005, at 19:03, sean finney wrote:
> so now that i think i have a short-term fix (disabling notification for
> new mail server-side), i can breathe a sigh of relief.  for a long-term
> fix, would it be possible to have the server always notify the client
> of new mail after a NOOP?
How did you disable them? I think enabling oe6-fetch-no-newmail 
workaround should help.

I think I'm also going to change that workaround name for next 1.0-test 
release, and change it to be more generic. Those clients will most 
likely ignore the new mail notifications for STORE and other commands 
as well, not just FETCH.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20050314/3870b324/attachment-0001.bin>

hello!

On Mon, Mar 14, 2005 at 11:08:39PM +0200, dovecot-request at dovecot.org
wrote:
> From: "Dominik 'Rathann' Mierzejewski" <D.Mierzejewski
at icm.edu.pl>
> 
> According to RFC3501 (is this the right one?), NOOP can be used to check
> for new mail, but it doesn't say that the server should return status
> data every time NOOP is issued.
i think the problem is that when the server asynchronously reports to
the client that there's new mail apple mail gets confused.  then noop
doesn't report new mail.  also, it seems that this is also the cause
of apple mail having to re-download the entire mailbox every time
it checks for new mail.
> From: Timo Sirainen <tss at iki.fi>
> 
> How did you disable them? I think enabling oe6-fetch-no-newmail 
> workaround should help.
i set:

mailbox_check_interval = 0
mailbox_idle_check_interval = 0

which did the trick (i already had all the client workarounds enabled).


thanks,
	sean
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20050315/91674418/attachment-0001.bin>

On Tue, Mar 15, 2005 at 07:36:16AM -0500, sean finney
wrote:
> hello!
> 
> On Mon, Mar 14, 2005 at 11:08:39PM +0200, dovecot-request at dovecot.org
wrote:
> > From: "Dominik 'Rathann' Mierzejewski"
<D.Mierzejewski at icm.edu.pl>
> > 
> > According to RFC3501 (is this the right one?), NOOP can be used to
check
> > for new mail, but it doesn't say that the server should return
status
> > data every time NOOP is issued.
> 
> i think the problem is that when the server asynchronously reports to
> the client that there's new mail apple mail gets confused.  then noop
> doesn't report new mail.  also, it seems that this is also the cause
> of apple mail having to re-download the entire mailbox every time
> it checks for new mail.
Well, then it's broken and should be fixed. I don't think Timo should be
required to implement hacks to work around broken clients, but that's his
call.

Regards,
-- 
Dominik 'Rathann' Mierzejewski <rathann*at*icm.edu.pl>
Interdisciplinary Centre for Mathematical and Computational Modelling
Warsaw University  |  http://www.icm.edu.pl  |  tel. +48 (22) 5540810

hey timo,

On Tue, Mar 15, 2005 at 07:32:35PM +0200, dovecot-request at dovecot.org
wrote:
> From: Timo Sirainen <tss at iki.fi>
> > i set:
> > 
> > mailbox_check_interval = 0
> 
> This has been the default since 0.99.5 which was released over 2 years
> ago. How old version are you using? :) Or had you changed the default?
i had indeed changed the default, thinking that it would be a nice
feature to turn on.  when the problem first surfaced, i immediately
thought of this option. however, since the problem was that applemail
wasn't seing the new mail, and this was an option to notify the client
that there *was* new mail, i guess common sense got the better of me
and i forgot this was something i had changed...
> > mailbox_idle_check_interval = 0
> 
> This shouldn't matter because OSX Mail doesn't use IDLE. Probably
better
> idea to turn it back on or clients using IDLE don't see new mails (I
> think?)
aha.  well then i'll have to try turning that back on :)


thanks,
	sean
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20050315/508165d4/attachment-0001.bin>