I've converted most of our users from Outlook to Thunderbird. One important feature of Thunderbird is that it pays attention to the IMAP non-standard message flags via the 'tags' feature (see http://wiki2.dovecot.org/MailboxFormat/Maildir). This is important because 2 users make extensive use of Outlook categories (Tags, in Thunderbird). I found that when I set a message to the 1st Tbird tag 0 (Important), the corresponding IMAP file got an 'a' suffix. Likewise, if I manually added an 'a' suffix to an IMAP file the corresponding tag was set displayed in Thunderbird. This made it easy for me to exports Outlook Categories and set IMAP messages flags accordingly. This worked perfectly on the user I was experimenting with (me). Recently, I converted one of these Outlook 'categories' users to Thunderbird. I added the appropriate flag letters to their IMAP messages, changed their tag names in Thunderbird, and created the following dovecot-keywords file as: 0 Board_and_Committee 1 Completed 2 Health_Care_meetings 3 Notifications 4 OSHP-DAS 5 personal_or_To_Do 6 Retirement_exits 7 $label5 8 Junk 9 $Forwarded 10 $MDNSent 11 $label2 However, none of the tags show up correctly in Thunderbird. If I manually set a message to have a tag of 0, the corresponding IMAP file gets a flag of 'm', not 'a' and the following is added to the dovecot-keywords files: 12 $label1 How can I fix this? Where is "$label1" text defined? Why did Thunderbird not snag the text for '0' from the dovecot-keywords file and give the IMAP file a tag of 'a'? My current theory is that the "Default" Thunderbird Tags corresponding to IMAP flags are not changeable and if new tags are create in Tbird, they get new flag letters. That would, of course, mean that if a user changes Thunderbird tag name, they would lose all tag settings on existing message. That doesn't seem right and I hope my theory is wrong. Any insight would be appreciated. --Mark
Peter Chiochetti
2016-Jul-30 09:26 UTC
IMAP flags and dovecot-keywords not working as expected
Am 2016-07-30 um 08:00 schrieb Mark Foley: ?> > However, none of the tags show up correctly in Thunderbird. If I manually set a message to > have a tag of 0, the corresponding IMAP file gets a flag of 'm', not 'a' and the following is > added to the dovecot-keywords files: > > 12 $label1 > > How can I fix this? Where is "$label1" text defined? Why did Thunderbird not snag the text for > '0' from the dovecot-keywords file and give the IMAP file a tag of 'a'?Thunderbird flags are stored in the users prefs.js, eg: - user_pref("mailnews.tags.$label1.tag", "Important"); - user_pref("mailnews.tags.$label1.color", "#FF0000"); A kind of key->value assignment. The "$label[1-9]" keys are special, where the number magically corresponds to the keyboard shorcut to tag messages, 0 meaning clear all tags. There can be more than nine tags, but they wont have a shortcut then: - user_pref("mailnews.tags.ten.tag", "ten"); 1) The server will only ever see the key. The user will only ever see the value. 2) If you rename a label in TB, then only the value will change and the server will still see the same key as before. 3) If you rename a key in dovecot, TB will not create a label for it and the affected messages will no longer appear tagged, if TB does not know about the key. 4) Dovecot adds to the keywords as it receives requests from clients: Very likely there is a limit of 26 (letters of the alphabet) per account; a-d=0-3 are already taken for internal use, so 22 remain.> My current theory is that the "Default" Thunderbird Tags corresponding to IMAP flags are not > changeable and if new tags are create in Tbird, they get new flag letters. That would, of > course, mean that if a user changes Thunderbird tag name, they would lose all tag settings on > existing message. That doesn't seem right and I hope my theory is wrong.I think you are mostly wrong: as long as you only use TB to work and as long as you do not exceed the limit you should be save. Notice that tags are a scarce resource: any key you ever created counts toward the limit - reusing old tags requires you to text-edit both dovecot-keywords and TB prefs.js. -- peter
I think I've partially sorted out my issues. First off, as the wiki:
http://wiki2.dovecot.org/MailboxFormat/Maildir points out: "The file
[dovecot-keywords] must
not be directly modified ...", which is what I tried. So the
dovecot-keywords file I manually
created (entries 0-6):
0 Board_and_Committee
1 Completed
2 Health_Care_meetings
3 Notifications
4 OSHP-DAS
5 personal_or_To_Do
6 Retirement_exits
7 $label5
8 Junk
9 $Forwarded
10 $MDNSent
11 $label2
Was not used by Thunderbird. So, I removed that file, went into Thunderbird, and
manually set a
random message to tag 1 which I had renamed as "Board and Committee".
And behold! a new
dovecot-keyword file was created with the single entry:
0 $label1
As you (Peter) point out, Thunderbird retrieves the text for these from
prefs.js. The reason,
therefore, I have e.g. "11 $label2" in my original hand-edited file
is that Thunderbird caused
what I had designated as "Health Care Meeting" to be added to #11
since the previous slots were
already used by other things which it did not recognize. Hence, they ended up
with a IMAP flag
of 'l'.
I repeated this for a total of 4 more tags, resulting in dovecot-keyword
entries:
0 $label1
1 $label2
2 $label3
3 $label4
4 $label5
Now, setting the tag in Tbird results in the correct text and color being shown
for the message
and the resulting IMAP tag being set correspondingly to 'a' thru
'e'. Flush with victory, I
then foolish violated the "don't modify directly principle" by
adding:
5 $label6
6 $label7
Which I thought would correspond to my remaining TB tags. Of course, wrong. When
I then set a
message to the 6th tag (keyword entry 5, $label6), it did not do it. Instead,
for my last 2
tags it created 2 new entries:
7 personal_or_to_do
8 retirement_exits
So, IMAP flags 'f' and 'g' are skipped and "Personal or To
Do" tags get an 'h' flag and
"Retirement Exits" get an 'i' flag. What was I thinking! Not
worth starting over as this user
had no Outlook categories set to these values, so so-what if that user's
files get flagged with
'h', 'i'.
Also interesting to note that the actual text of these last 2 tags is stored in
the
dovecot-keywords file. I supposed this means that Thunderbird has a limit of 5
(or 7?) $label's
and any added beyond that get stored differently.
But, I'm not finished. Of course, Thunderbird expects to set tags/IMAPflas
on virgin messages
in the user's mail folders. However, my need is to import Outlook
categories and have
Thunderbird interpret the IMAP flags correctly having never set them directly.
So, I repeated the same procedure in other folders with messages having Outlook
categories.
The problem here is that setting a message to "Board and Committee"
(Tbird tag 1) did not
necessarily resulting in an IMAP flag of 'a'. In some folders it was
'b', some 'c' and some
'd'. I think my problem here was that I had manually set the IMAP flags
on these message
beforehand. So (I think) when Thunderbird, coordinating with Dovecot, went to
set Tag 1, it
saw that flag 'a' was already used and it picked the next free flag.
Some folders had IMAP
files with 2 and 3 flags set, hence Tag 1 getting set to 'c' or
'd' in those folders. I
believe if I had not initially pre-set the IMAP flags on these files I would
have seen the
correct correspondance: Tag 1 = flag 'a', Tag 2 = flag 'b', etc.
What I then did in these cases was first, find out what IMAP flag a Tbird tagged
file would be
set to (e.g. 'c'), then renamed all my *a files to e.g. *c. That
worked. Suddenly, folders
with 100 flags thusly set showed up with correct tags and colors in Thunderbird.
To summarize, and this what I will do next time:
1. DO NOT pre-add IMAP flags to mail files.
2. For each folder in which there are Outlook categorized message, set an
arbitrary message in
that folder to each of the Thunderbird flags corresponding to Outlook categories
to determine
the correct flag letter -- which also causes Thunderbird and dovecot to save
their respective
settings.
3. Then, rename each mail file to have the correct IMAP flag(s). That can be
quicky done using a
script and the list of message having Outlook categories.
I believe this procedure will work correctly and I will confirm that when I
process my 2nd
Outlook user (who is, btw, the organization Director and who is a super user of
Outlook
categories! Failure is not an option!).
For anyone needing to do the same thing (migrate Outlook categories), I've
included below a VB
script to run in Outlook which will output a list of all messages having
categories. The IMAP
mail file's Message ID can be used to locate the categorized message file in
the IMAP folder
hierarchy and the category name (following the "~") indicates which
Thunderbird tag to map it
to. I'll not include the bash script to mass-append IMAP flags to these
files as that script
will need some revising based on my recent experimentation, but should be a
rather simple bash
exercise in any case.
Note that the Outlook messages are also the same MAPI files, only the client
used (Outlook
versus Thunderbird) are different. Outlook does not set IMAP flags to designate
categories.
Categories are apparently stored in the user's .pst file.
------------CUT----------
Public Sub ListOutlookFolders()
Dim olApp As Outlook.Application
Dim olNamespace As Outlook.Namespace
Dim olFolder As Outlook.MAPIFolder
Set olApp = New Outlook.Application
Set olNamespace = olApp.GetNamespace("MAPI")
For Each olFolder In olNamespace.Folders
Debug.Print olFolder.Name; ":", olFolder.Description
ListFolders olFolder, 1
Next
Set olFolder = Nothing
Set olNamespace = Nothing
Set olApp = Nothing
End Sub
Sub ListFolders(myFolder As Outlook.MAPIFolder, Level As Integer)
Dim olFolder As Outlook.MAPIFolder
' go through each email
scanFolder myFolder
' Now we'll check for subfolders
For Each olFolder In myFolder.Folders
' Debug.Print ":"; String(Level * 2, "-");
olFolder.Name
' go through each email
scanFolder olFolder
If olFolder.Folders.Count > 0 Then
ListFolders olFolder, Level + 1
End If
Next
End Sub
Sub scanFolder(sFolder As Outlook.MAPIFolder)
Dim src As Folder
Dim oItem As Object
Dim propertyAccessor As Outlook.propertyAccessor
Set src = sFolder
Dim strHeader As String
For Each oItem In src.Items
If TypeOf oItem Is Outlook.MailItem And oItem.Categories <>
"" Then
' Debug.Print "Cat: " + oItem.Categories
Set propertyAccessor = oItem.propertyAccessor
header
propertyAccessor.GetProperty("http://schemas.microsoft.com/mapi/proptag/0x007D001E")
Dim headerLines() As String
headerLines() = Split(header, vbCrLf)
Dim thisHeader As Variant
For Each thisHeader In headerLines
If InStr(thisHeader, "Message-ID:") > 0 Then
Debug.Print thisHeader + "~" + oItem.Categories
Exit For
End If
Next
End If
Next
End Sub
----------CUT---------
Example of resulting output:
Message-ID: <201109011105.p81B5666028910 at webserver.ohprs.org>~Red
Category
Hopefully someone finds this useful.
THX --Mark
-----Original Message-----> Subject: Re: IMAP flags and dovecot-keywords not working as expected
> To: dovecot at dovecot.org
> From: Peter Chiochetti <pch at myzel.net>
> Date: Sat, 30 Jul 2016 11:26:09 +0200
>
> Am 2016-07-30 um 08:00 schrieb Mark Foley:
> ?
> >
> > However, none of the tags show up correctly in Thunderbird. If I
manually set a message to
> > have a tag of 0, the corresponding IMAP file gets a flag of
'm', not 'a' and the following is
> > added to the dovecot-keywords files:
> >
> > 12 $label1
> >
> > How can I fix this? Where is "$label1" text defined? Why did
Thunderbird not snag the text for
> > '0' from the dovecot-keywords file and give the IMAP file a
tag of 'a'?
>
> Thunderbird flags are stored in the users prefs.js, eg:
> - user_pref("mailnews.tags.$label1.tag", "Important");
> - user_pref("mailnews.tags.$label1.color", "#FF0000");
>
> A kind of key->value assignment. The "$label[1-9]" keys are
special,
> where the number magically corresponds to the keyboard shorcut to tag
> messages, 0 meaning clear all tags.
>
> There can be more than nine tags, but they wont have a shortcut then:
> - user_pref("mailnews.tags.ten.tag", "ten");
>
> 1) The server will only ever see the key. The user will only ever see
> the value.
>
> 2) If you rename a label in TB, then only the value will change and the
> server will still see the same key as before.
>
> 3) If you rename a key in dovecot, TB will not create a label for it and
> the affected messages will no longer appear tagged, if TB does not know
> about the key.
>
> 4) Dovecot adds to the keywords as it receives requests from clients:
> Very likely there is a limit of 26 (letters of the alphabet) per
> account; a-d=0-3 are already taken for internal use, so 22 remain.
>
>
> > My current theory is that the "Default" Thunderbird Tags
corresponding to IMAP flags are not
> > changeable and if new tags are create in Tbird, they get new flag
letters. That would, of
> > course, mean that if a user changes Thunderbird tag name, they would
lose all tag settings on
> > existing message. That doesn't seem right and I hope my theory is
wrong.
>
> I think you are mostly wrong: as long as you only use TB to work and as
> long as you do not exceed the limit you should be save.
>
> Notice that tags are a scarce resource: any key you ever created counts
> toward the limit - reusing old tags requires you to text-edit both
> dovecot-keywords and TB prefs.js.
>
> --
> peter
>