dovecot - Jan 2010 - virtual mailbox / INTHREAD use case, issues, questions

For over a year now, I've been trying to acquire the ability to review
the thread history of a message in my INBOX while still deleting
messages as soon as I believe I'm done with them.  Suddenly I'm
gratified to see that several approaches have been right under my nose
all along!

One is the INTHREAD imap extension, which I see is implemented by
dovecot.  Another (related) is dovecot's virtual folders.  When I saw
the

 GMail-style "conversation view"

described at http://wiki.dovecot.org/Plugins/Virtual I was elated, so
I tried it out, with exactly that configuration (almost -- my
separator is '.').

I ran into several problems (using Dovecot 1.2.8 with
http://hg.dovecot.org/dovecot-1.2/rev/9cccb8617820 applied):

1. I had to manually create the virtual folder for all my users or
   they couldn't access their mail at all.  Is this fixed in 2.0?

2. My users were getting errors when accessing via POP3.  The possible
   meaning of such a virtual folder under POP3 is lost on me, but I
   had to enable the virtual plugin for that protocol too.  I think
   that should at least be in the documentation, or better yet the
   requirement should that it be enabled for POP3 should be dropped.

3. I couldn't get the virtual "conversation view" inbox to show
any
   messages.  But maybe that's because of a possible issue I describe
   below with x-mailbox

4. At some point I ended up with an empty mailbox called
   "private.virtual" --- I have no idea how.  Probably user error.

5. The "all" virtual mailbox would only show 89 messages.  I switched
   it to be a mirror of my "archive" mailbox and the number started to
   look more reasonable.  This could be the "lots of mailboxes" bug
   describe in
   http://dovecot.org/list/dovecot-news/2009-December/000145.html

6. I got the impression that looking at the virtual INBOX didn't cause
   the "all" mailbox to be updated, which I'd guess that it should
be.

7. I tried to create an IMAP search, rather than a virtual folder,
   that looked for x-mailbox INBOX header like the virtual folder
   does.  It too came up empty.  It doesn't exactly surprise me
   because I don't see an x-mailbox header in any of these messages.
   In fact, I can't find one in /any/ of my messages.  When I look for
   a description of the x-mailbox header, I don't find much
   information on where it comes from or what it represents.
   http://www.dovecot.org/list/dovecot/2009-June/040619.html seems to
   indicate it's synthesized by the server.  If it's really supposed
   to represent the message's "original" mailbox as implied by
   http://wiki.dovecot.org/Plugins/Virtual then I'm not sure what that
   means for messages that are filed twice by sieve and never moved.

Lastly, I want to report that for me, the speed of these IMAP searches
doesn't begin to approach what I can do with mairix if I want a
message's entire thread.  I'm happy to use mairix, although it would
be nicer not to duplicate that capability if it's already in the
server.  But it needs to be fast.  Is there something I can do to make
it go faster?

Thanks in advance for any advice, hints, fixes, workarounds, etc.

--
Dave Abrahams           Meet me at BoostCon: http://www.boostcon.com
BoostPro Computing
http://www.boostpro.com

On 04/01/2010 18:47, David Abrahams wrote:
> For over a year now, I've been trying to acquire the ability to review
> the thread history of a message in my INBOX while still deleting
> messages as soon as I believe I'm done with them.  Suddenly I'm
> gratified to see that several approaches have been right under my nose
> all along!
>
> One is the INTHREAD imap extension, which I see is implemented by
> dovecot.  Another (related) is dovecot's virtual folders.  When I saw
> the
>
>   GMail-style "conversation view"
>
> described at http://wiki.dovecot.org/Plugins/Virtual I was elated, so
> I tried it out, with exactly that configuration (almost -- my
> separator is '.').
>    
I have recently just switched my Thunderbird to have sent items in the 
"Inbox" and "in the same folder as the message you are replying
to".
The end result is a fairly workable equivalent of the gmail style thing, 
although you also see Sent messages which haven't had a reply (which may 
or may not be what you desire, but it's actually my preference for my needs)

No need for virtual folders, but support for this to work well may not 
be in every mail client...

Ed W

On 4.1.2010, at 20.47, David Abrahams wrote:
> 1. I had to manually create the virtual folder for all my users or
>   they couldn't access their mail at all.  Is this fixed in 2.0?
No.
> 2. My users were getting errors when accessing via POP3.  The possible
>   meaning of such a virtual folder under POP3 is lost on me, but I
>   had to enable the virtual plugin for that protocol too.  I think
>   that should at least be in the documentation, or better yet the
>   requirement should that it be enabled for POP3 should be dropped.
I think you could put the virtual namespace inside protocol imap {} section.
> 3. I couldn't get the virtual "conversation view" inbox to
show any
>   messages.  But maybe that's because of a possible issue I describe
>   below with x-mailbox
> 
> 4. At some point I ended up with an empty mailbox called
>   "private.virtual" --- I have no idea how.  Probably user error.
Dunno.
> 5. The "all" virtual mailbox would only show 89 messages.  I
switched
>   it to be a mirror of my "archive" mailbox and the number
started to
>   look more reasonable.  This could be the "lots of mailboxes"
bug
>   describe in
>   http://dovecot.org/list/dovecot-news/2009-December/000145.html
"lots of mailboxes" should cause error to be logged.
> 6. I got the impression that looking at the virtual INBOX didn't cause
>   the "all" mailbox to be updated, which I'd guess that it
should be.
It should.
> 7. I tried to create an IMAP search, rather than a virtual folder,
>   that looked for x-mailbox INBOX header like the virtual folder
>   does.  It too came up empty.  It doesn't exactly surprise me
>   because I don't see an x-mailbox header in any of these messages.
x-mailbox doesn't use a header, it uses the actual mailbox name where the
message exists.
> Lastly, I want to report that for me, the speed of these IMAP searches
> doesn't begin to approach what I can do with mairix if I want a
> message's entire thread.  I'm happy to use mairix, although it
would
> be nicer not to duplicate that capability if it's already in the
> server.  But it needs to be fast.  Is there something I can do to make
> it go faster?
How slow is it? How many messages do you have? I think it should be pretty fast.

Anyway, dovecot -n output and contents of dovecot-virtual file could help
figuring out what your problem is.