Btrfs devel - Aug 2011 - BTRFS should increase the hard-link in the same directory limit

It seems a priori that there should not be any need for more than 256
names for the same file in the same directory. However, the GNUS
mailreader''s nnmaildir backend uses hardlinks to mark email messages
read, and instead of creating a separate inode for each marked
message, uses a hardlink to a single markfile. This means that there
maybe thousands of hardlinks to the same inode in a single directory.

This is not unreasonable as the obvious workaround is to wastefully
create more inodes. 

This restriction causes btrfs-convert 0.19 to crash out with a segfault and
no helpful message: something like btrfs-convert: segfault at
ffffffffcfb25fb9 ip 000000000040f9f1 sp 00007fffddefb398 error 6 in
btrfs-convert[400000+21000].

Is there any plan to alleviate this unfortunate limit (or at least make
btrfs-convert give the location of the file which causes it to fail?).
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

>>>>> "JF" == John Fremlin <john@fremlin.org>
writes:
JF> instead of creating a separate inode for each marked message, uses a
JF> hardlink to a single markfile. This means that there maybe thousands
JF> of hardlinks to the same inode in a single directory.

And that behaviour is not limited to gnus.  Many workflows use that idiom.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 08/21/2011 11:13 AM, John Fremlin wrote:
> It seems a priori that there should not be any need for more than 256
> names for the same file in the same directory. However, the GNUS
> mailreader''s nnmaildir backend uses hardlinks to mark email
messages
> read, and instead of creating a separate inode for each marked
> message, uses a hardlink to a single markfile. This means that there
> maybe thousands of hardlinks to the same inode in a single directory.
> 
> This is not unreasonable as the obvious workaround is to wastefully
> create more inodes. 
> 
> This restriction causes btrfs-convert 0.19 to crash out with a segfault and
> no helpful message: something like btrfs-convert: segfault at
> ffffffffcfb25fb9 ip 000000000040f9f1 sp 00007fffddefb398 error 6 in
> btrfs-convert[400000+21000].
> 
> Is there any plan to alleviate this unfortunate limit (or at least make
> btrfs-convert give the location of the file which causes it to fail?).
It''s a disk format change, something we don''t do lightly.

Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Josef Bacik <josef@redhat.com> writes:
> On 08/21/2011 11:13 AM, John Fremlin wrote:
[...]
>> This restriction causes btrfs-convert 0.19 to crash out with a segfault
and
>> no helpful message: something like btrfs-convert: segfault at
>> ffffffffcfb25fb9 ip 000000000040f9f1 sp 00007fffddefb398 error 6 in
>> btrfs-convert[400000+21000].
>> 
>> Is there any plan to alleviate this unfortunate limit (or at least make
>> btrfs-convert give the location of the file which causes it to fail?).
>
> It''s a disk format change, something we don''t do lightly.
It would indeed require a disk format change, and hardlinks are always
tiresome for FS designers ;-)

I think however that the format change could be designed to only affect
people who sadly cannot at the moment use BTRFS because of this
limitation, and be more or less unnoticeable to other people.

As James points out there are other applications that benefit from being
able to create many names for the same inode in the same directory, and
256 is a very low limit!

Could this at least be put on the list of things to change? Is there a
way to vote for it?

And the fact that btrfs-convert crashes horribly could be fixed without a
disk-format change. . . 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 08/22/2011 12:05 PM, John Fremlin wrote:
> Josef Bacik <josef@redhat.com> writes:
>> On 08/21/2011 11:13 AM, John Fremlin wrote:
> [...]
>>> This restriction causes btrfs-convert 0.19 to crash out with a
segfault and
>>> no helpful message: something like btrfs-convert: segfault at
>>> ffffffffcfb25fb9 ip 000000000040f9f1 sp 00007fffddefb398 error 6 in
>>> btrfs-convert[400000+21000].
>>>
>>> Is there any plan to alleviate this unfortunate limit (or at least
make
>>> btrfs-convert give the location of the file which causes it to
fail?).
>>
>> It''s a disk format change, something we don''t do
lightly.
> 
> It would indeed require a disk format change, and hardlinks are always
> tiresome for FS designers ;-)
> 
> I think however that the format change could be designed to only affect
> people who sadly cannot at the moment use BTRFS because of this
> limitation, and be more or less unnoticeable to other people.
> 
> As James points out there are other applications that benefit from being
> able to create many names for the same inode in the same directory, and
> 256 is a very low limit!
> 
> Could this at least be put on the list of things to change? Is there a
> way to vote for it?
> 
> And the fact that btrfs-convert crashes horribly could be fixed without a
> disk-format change. . . 
It''s on the list, but there are a lot of other more pressing things
then
to allow weird apps to do strange things with hardlinks.  Thanks,

Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

so there are these hundreds of message files, and when one is read, a
new link to The Markfile appears with a similar name as the read file?
Is that right?
if the point is to save inodes by making a directory entry that''s a
hardlink to something already existing, why not link to the message
file?

On Sun, Aug 21, 2011 at 5:05 PM, James Cloos <cloos@jhcloos.com>
wrote:
>>>>>> "JF" == John Fremlin <john@fremlin.org>
writes:
>
> JF> instead of creating a separate inode for each marked message, uses a
> JF> hardlink to a single markfile. This means that there maybe thousands
> JF> of hardlinks to the same inode in a single directory.
>
> And that behaviour is not limited to gnus.  Many workflows use that idiom.
>
> -JimC
> --
> James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


-- 
"The tools expect that they have full, unlimited control of the
hardware."
 -- Intel Corporation
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html