dovecot - Jan 2017 - GMail-like user labels in IMAP/dovecot, situation in 2017?

Hello,

I am in the process of updating my IMAP dovecot installation and I was
wondering what is the current state of the art for GMail-like
user-defined labels.

Are GMail-like user-defined labels now somehow achievable with dovecot?

If now, what is missing? Standardized metadata formats? Support in clients?

I searched the web a bit but I could not find any activity in the
relevant IETF working groups. That seems strange to me, because the
user-defined labels looks like a feature many users would like to have.
I suppose I haven't found the relevant discussion group. Is this kind of
enhancement to IMAP being discussed somewhere?

Regards,

-- 
Gioele Barabucci <gioele at svario.it>

On 10 Jan 2017, at 21.00, Gioele Barabucci <gioele at svario.it>
wrote:
> 
> Hello,
> 
> I am in the process of updating my IMAP dovecot installation and I was
> wondering what is the current state of the art for GMail-like
> user-defined labels.
> 
> Are GMail-like user-defined labels now somehow achievable with dovecot?
I think the main difference between folders and labels is that with folders you
have separate flags for a mail in each folder, while with labels there's
just one global flags/labels status for the mail. Every other operation can be
mapped 1:1 pretty nicely. So the big question is how should this be implemented
in a way that allows both regular IMAP access and also label-like access. There
are two choices:

a) Implement a new storage format where all the mails are in a single physical
folder. Treat flags as labels. When accessing with IMAP, create virtual folders
based on the flags. This is how GMail works.

b) Implement a way to export the current folder-based storage using labels. This
could be done with a virtual "all mails" folder, which combines both
message flags and folder names into labels for the mails. It also likely means
that changing a label changes the flag in each physical folder where the mail
exists in. This is what we were planning to do for the Dovecot GMail API
support.

There are some initial patches for b) that would make it efficient to implement,
but they don't work right with replication or with obox format, so they
didn't get merged. I think I had some thoughts on how to make them work with
a different design, which I probably still have somewhere written down.

In any case, priority for GMail API support has dropped pretty low. Most people
didn't really care about actually having GMail compatible API, but more
about having some kind of a HTTP API. So the plan right now is to add support
for JMAP, which also requires support for thread IDs like GMail does, but
doesn't require support for labels.
> If now, what is missing? Standardized metadata formats? Support in clients?
> 
> I searched the web a bit but I could not find any activity in the
> relevant IETF working groups. That seems strange to me, because the
> user-defined labels looks like a feature many users would like to have.
> I suppose I haven't found the relevant discussion group. Is this kind
of
> enhancement to IMAP being discussed somewhere?
I have a feeling you're thinking about something completely different from
what I'm thinking though. You know that we have user-defined flags already,
right? What exactly do you want to be different?

On 10/01/2017 22:54, Timo Sirainen wrote:
> On 10 Jan 2017, at 21.00, Gioele Barabucci <gioele at svario.it>
wrote:
>>
>> I am in the process of updating my IMAP dovecot installation and I was
>> wondering what is the current state of the art for GMail-like
>> user-defined labels.
>>
>> Are GMail-like user-defined labels now somehow achievable with dovecot?
> 
> a) Implement a new storage format where all the mails are in a single
> physical folder. Treat flags as labels. When accessing with IMAP,
> create virtual folders based on the flags. This is how GMail works.
Hello Timo, thank you for the answers.

To my uneducated eyes this solution seems the most straightforward and
less error-prone. But I suppose that the devil is in the details.
> b) Implement a way to export the current folder-based storage using
> labels. This could be done with a virtual "all mails" folder,
which
> combines both message flags and folder names into labels for the
> mails. It also likely means that changing a label changes the flag in
> each physical folder where the mail exists in. This is what we were
> planning to do for the Dovecot GMail API support.
It is great to see that things are still being worked on.

What about flags like \Seen or \Answered? Will they also be kept in sync
or will each copy of a message have its own set of flags?
> In any case, priority for GMail API support has dropped pretty low.
> Most people didn't really care about actually having GMail
> compatible API, but more about having some kind of a HTTP API.
I do not know about how much a "GMail API" is requested, but all
(_all_,
not most) users I have met lament the lack of GMail-like labels and
conversations in IMAP clients (desktop, mobile and web based).
>> If now, what is missing? Standardized metadata formats? Support in
clients?
> 
> I have a feeling you're thinking about something completely
> different from what I'm thinking though. You know that we have
> user-defined flags already, right? What exactly do you want to be
> different?
My idea of "GMail-like user labels" is composed of two parts:

1. Tags/labels that the user can freely attach to
messages/conversations. User-defined flags can be attached to messages,
but there is no way to store metadata like the human-readable name of
the label (e.g. "Travels in Italy") or its color. This metadata must
be
configured manually in each client.

2. the ability to see a tree-like structure based on these tags/labels,
with the same message/conversation appearing in multiple "branches" at
the same time and where deletion means "removal of tag".

IIUC both point 1 and 2 are still missing from dovecot.

Regards,

--
Gioele Barabucci <gioele at svario.it>