dovecot - Nov 2012 - Issues with VANISHED CHANGEDSINCE

I've been seeing the following wonky behavior with Dovecot. Currently  
this is with Dovecot 2.0.19, but I was also seeing it in earlier  
versions as well, including versions from the 1.x series.  Some  
background, this is from Horde's ActiveSync library, when it is trying  
to determine what UIDs have recently been expunged.  I can guarantee  
that when this command was sent, there was only one or two emails that  
were removed since the provided MODSEQ value.

C: 8 UID FETCH 1:* UID (VANISHED CHANGEDSINCE 587742)
S: * VANISHED (EARLIER)  
1:119944,119947:123482,123484:128043,128045:130388,130390:130456,130458:132127,132129:135832,135834:136201,136203:136301,136303:136316,136318:138502,138504:140099,140101:141908,141910:142607,142609:142744,142746:142865,142867:143160,143164:143173,143176:143179,143181:143187,143189:143190,143192:143195,143197:143270,143272:143273,143275:143280,143282:143302,143305:143307,143309:143318,143320:143333,143335:143337,143339:143341,143343:143375,143377:143400,143402:143551,143553:143637,143640:143706,143708:144069,144071:144178,144180:144307,144309:144316,144318:144329,144331:144931,144933:144983,144985:145000,145002:145069,145071:145072,145074:145132,145136:145178,145180:145191,145193


Also, to verify it wasn't something screwy with my server, Michael  
Slusarz provided me with this from his server:

<quote>
There's definitely something wonky going on in the code.  There's a  
certain tipping point of modseqs where the values become incorrect.   
For a mailbox with HIGHESTMODSEQ of 54004, I see this:

a uid fetch 1:* UID (VANISHED CHANGEDSINCE 53881)
* VANISHED (EARLIER) 1:37308,37310:40788,40791:41032,41034:41083
a OK Fetch completed.
b uid fetch 1:* UID (VANISHED CHANGEDSINCE 53882)
* VANISHED (EARLIER) 41029:41030,41047:41083
b OK Fetch completed.
</quote>


Thanks for any guidance.
-- 
mike

The Horde Project (www.horde.org)
mrubinsk at horde.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6062 bytes
Desc: S/MIME Cryptographic Signature
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20121105/c62a8af1/attachment-0004.bin>

Quoting Michael J Rubinsky <mrubinsk at horde.org>:
> Also, to verify it wasn't something screwy with my server, Michael  
> Slusarz provided me with this from his server:
>
> <quote>
> There's definitely something wonky going on in the code.  There's a
> certain tipping point of modseqs where the values become incorrect.   
> For a mailbox with HIGHESTMODSEQ of 54004, I see this:
>
> a uid fetch 1:* UID (VANISHED CHANGEDSINCE 53881)
> * VANISHED (EARLIER) 1:37308,37310:40788,40791:41032,41034:41083
> a OK Fetch completed.
> b uid fetch 1:* UID (VANISHED CHANGEDSINCE 53882)
> * VANISHED (EARLIER) 41029:41030,41047:41083
> b OK Fetch completed.
> </quote>
FYI, this is output from Dovecot 2.1.10.

michael

On 5.11.2012, at 18.13, Michael J Rubinsky wrote:
> I've been seeing the following wonky behavior with Dovecot. Currently
this is with Dovecot 2.0.19, but I was also seeing it in earlier versions as
well, including versions from the 1.x series.  Some background, this is from
Horde's ActiveSync library, when it is trying to determine what UIDs have
recently been expunged.  I can guarantee that when this command was sent, there
was only one or two emails that were removed since the provided MODSEQ value.
..
> <quote>
> There's definitely something wonky going on in the code.  There's a
certain tipping point of modseqs where the values become incorrect.
I don't think they become incorrect, just that there are more of them than
really necessary? Yes, there's a tipping point. It's when the modseq no
longer exists in the dovecot.index.log* files, which get rotated once in a
while. This shouldn't happen very often.
> For a mailbox with HIGHESTMODSEQ of 54004, I see this:
> 
> a uid fetch 1:* UID (VANISHED CHANGEDSINCE 53881)
> * VANISHED (EARLIER) 1:37308,37310:40788,40791:41032,41034:41083
> a OK Fetch completed.
> b uid fetch 1:* UID (VANISHED CHANGEDSINCE 53882)
> * VANISHED (EARLIER) 41029:41030,41047:41083
> b OK Fetch completed.
> </quote>
So in b) case Dovecot read the dovecot.index.log and returned the expunges found
from there. In a) case it didn't know what was expunged since the modseq, so
it returned UIDs of all the expunged mails.