dovecot - Jul 2007 - problem with subscriptions file version 1.0.1

Hello,

I am currently working on a patch to add IMAPdir support to dovecot. 
(For those who do not know, IMAPdir is exactly like Maildir except that 
instead of having a folder .foo it would just be foo. The advantage is 
that the file system is slightly easier to search through without the 
dot in front of the folders.)

The first thing I did was get the configuration file working for 
Maildir. So I know that the problem has nothing to do with the 
configuration file. Next I looked through the actual source code to find 
any place where a dot might be added to the beginning of a folder. It 
transpired that there is a constant, MAILDIR_FS_SEP_S, which is assigned 
the value ".". I deleted all the occurrences of this constant,
compiled
the source code again, and ran into a rather interesting problem.

When I try to create a new folder through Thunderbird on a separate 
machine, the folder is not visible unless I restart Thunderbird. I 
checked on the machine acting as a server and the folder is created and 
the subscriptions file is updated, and the dot at the beginning is 
indeed absent. So I see no reason why this should not work.

If I had deleted some of the constants that ought not have been deleted, 
the subscriptions file would not be readable at all and I would not be 
able to see the folder when Thunderbird restarts. If there were some 
sort of deadlock going on with the subscriptions file, one would think 
that it would not be caused by deleting the constants. I would have run 
into the problem before editing any of the source code. I know for fact 
that the only thing different about my source code and the code 
available for downloading is the deleted constants.

If anyone has any ideas, please let me know. If you need more 
information, please feel free to prompt me for it because I have no idea 
what might be useful that I have not already provided.

Regards,
Elizabeth Bermel

On Fri, 2007-07-06 at 11:49 -0400, Liz wrote:
> The first thing I did was get the configuration file working for 
> Maildir. So I know that the problem has nothing to do with the 
> configuration file. Next I looked through the actual source code to find 
> any place where a dot might be added to the beginning of a folder. It 
> transpired that there is a constant, MAILDIR_FS_SEP_S, which is assigned 
> the value ".". I deleted all the occurrences of this constant,
compiled
> the source code again, and ran into a rather interesting problem.
I hope you noticed also MAILDIR_FS_SEP?
> When I try to create a new folder through Thunderbird on a separate 
> machine, the folder is not visible unless I restart Thunderbird. I 
> checked on the machine acting as a server and the folder is created and 
> the subscriptions file is updated, and the dot at the beginning is 
> indeed absent. So I see no reason why this should not work.
First you could try manually the same thing.

Check that all mailboxes really are listed:

1 LIST "" *

Try creating + subscribing:

2 CREATE mailbox
3 SUBSCRIBE mailbox

Check that it's found from subscriptions:

4 LSUB "" *

And might as well check that LIST also finds it:

5 LIST "" *

If none of that showed anything broken, you could check what Thunderbird
and Dovecot talk with each others and see if there's something wrong
with that. For example use rawlog.
http://dovecot.org/bugreport.html#sniffing

Also if you do this, it might be a good idea to change Dovecot's
internal files to begin with a dot so that user is allowed to create
e.g. "dovecot" mailbox.

#define SUBSCRIPTION_FILE_NAME "subscriptions"
#define MAILDIR_INDEX_PREFIX "dovecot.index"

You should also check that deleting mailboxes works right. There's this:

#define MAILDIR_UNLINK_DIRNAME MAILDIR_FS_SEP_S"DOVECOT-TRASHED"

I'm not sure how it works if you delete the separators. If LIST sees
such a directory it goes and deletes it.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20070710/9c42446f/attachment-0002.bin>

I had noticed MAILDIR_FS_SEP, but it did not seem (perhaps I am 
mistaken?) as though any of the places it occurred were adding dots to 
the beginnings of directories. When I pulled up the list of possible 
folders to subscribe to in Thunderbird, the new folders were there and 
they were checked, presumably indicating they were subscribed to 
already. So I unchecked them and rechecked them again and then they were 
visible.

I used wireshark to see what dovecot and Thunderbird might be saying to 
each other and nothing appeared broken there. The responses to the list, 
subscribe, lsub and create requests were all "OK", in any case.
Deleting
folders works fine as well.

The suggestion of adding dots to the beginning of the internal files was 
also a good one.

Thanks for your reply,
Elizabeth Bermel

Timo Sirainen wrote:
> On Fri, 2007-07-06 at 11:49 -0400, Liz wrote:
>   
>> The first thing I did was get the configuration file working for 
>> Maildir. So I know that the problem has nothing to do with the 
>> configuration file. Next I looked through the actual source code to
find
>> any place where a dot might be added to the beginning of a folder. It 
>> transpired that there is a constant, MAILDIR_FS_SEP_S, which is
assigned
>> the value ".". I deleted all the occurrences of this
constant, compiled
>> the source code again, and ran into a rather interesting problem.
>>     
>
> I hope you noticed also MAILDIR_FS_SEP?
>
>   
>> When I try to create a new folder through Thunderbird on a separate 
>> machine, the folder is not visible unless I restart Thunderbird. I 
>> checked on the machine acting as a server and the folder is created and
>> the subscriptions file is updated, and the dot at the beginning is 
>> indeed absent. So I see no reason why this should not work.
>>     
>
> First you could try manually the same thing.
>
> Check that all mailboxes really are listed:
>
> 1 LIST "" *
>
> Try creating + subscribing:
>
> 2 CREATE mailbox
> 3 SUBSCRIBE mailbox
>
> Check that it's found from subscriptions:
>
> 4 LSUB "" *
>
> And might as well check that LIST also finds it:
>
> 5 LIST "" *
>
> If none of that showed anything broken, you could check what Thunderbird
> and Dovecot talk with each others and see if there's something wrong
> with that. For example use rawlog.
> http://dovecot.org/bugreport.html#sniffing
>
> Also if you do this, it might be a good idea to change Dovecot's
> internal files to begin with a dot so that user is allowed to create
> e.g. "dovecot" mailbox.
>
> #define SUBSCRIPTION_FILE_NAME "subscriptions"
> #define MAILDIR_INDEX_PREFIX "dovecot.index"
>
> You should also check that deleting mailboxes works right. There's
this:
>
> #define MAILDIR_UNLINK_DIRNAME MAILDIR_FS_SEP_S"DOVECOT-TRASHED"
>
> I'm not sure how it works if you delete the separators. If LIST sees
> such a directory it goes and deletes it.
>