John Fremlin
2011-Aug-21 15:13 UTC
BTRFS should increase the hard-link in the same directory limit
It seems a priori that there should not be any need for more than 256 names for the same file in the same directory. However, the GNUS mailreader''s nnmaildir backend uses hardlinks to mark email messages read, and instead of creating a separate inode for each marked message, uses a hardlink to a single markfile. This means that there maybe thousands of hardlinks to the same inode in a single directory. This is not unreasonable as the obvious workaround is to wastefully create more inodes. This restriction causes btrfs-convert 0.19 to crash out with a segfault and no helpful message: something like btrfs-convert: segfault at ffffffffcfb25fb9 ip 000000000040f9f1 sp 00007fffddefb398 error 6 in btrfs-convert[400000+21000]. Is there any plan to alleviate this unfortunate limit (or at least make btrfs-convert give the location of the file which causes it to fail?). -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
James Cloos
2011-Aug-21 22:05 UTC
Re: BTRFS should increase the hard-link in the same directory limit
>>>>> "JF" == John Fremlin <john@fremlin.org> writes:JF> instead of creating a separate inode for each marked message, uses a JF> hardlink to a single markfile. This means that there maybe thousands JF> of hardlinks to the same inode in a single directory. And that behaviour is not limited to gnus. Many workflows use that idiom. -JimC -- James Cloos <cloos@jhcloos.com> OpenPGP: 1024D/ED7DAEA6 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Josef Bacik
2011-Aug-22 14:54 UTC
Re: BTRFS should increase the hard-link in the same directory limit
On 08/21/2011 11:13 AM, John Fremlin wrote:> It seems a priori that there should not be any need for more than 256 > names for the same file in the same directory. However, the GNUS > mailreader''s nnmaildir backend uses hardlinks to mark email messages > read, and instead of creating a separate inode for each marked > message, uses a hardlink to a single markfile. This means that there > maybe thousands of hardlinks to the same inode in a single directory. > > This is not unreasonable as the obvious workaround is to wastefully > create more inodes. > > This restriction causes btrfs-convert 0.19 to crash out with a segfault and > no helpful message: something like btrfs-convert: segfault at > ffffffffcfb25fb9 ip 000000000040f9f1 sp 00007fffddefb398 error 6 in > btrfs-convert[400000+21000]. > > Is there any plan to alleviate this unfortunate limit (or at least make > btrfs-convert give the location of the file which causes it to fail?).It''s a disk format change, something we don''t do lightly. Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
John Fremlin
2011-Aug-22 16:05 UTC
Re: BTRFS should increase the hard-link in the same directory limit
Josef Bacik <josef@redhat.com> writes:> On 08/21/2011 11:13 AM, John Fremlin wrote:[...]>> This restriction causes btrfs-convert 0.19 to crash out with a segfault and >> no helpful message: something like btrfs-convert: segfault at >> ffffffffcfb25fb9 ip 000000000040f9f1 sp 00007fffddefb398 error 6 in >> btrfs-convert[400000+21000]. >> >> Is there any plan to alleviate this unfortunate limit (or at least make >> btrfs-convert give the location of the file which causes it to fail?). > > It''s a disk format change, something we don''t do lightly.It would indeed require a disk format change, and hardlinks are always tiresome for FS designers ;-) I think however that the format change could be designed to only affect people who sadly cannot at the moment use BTRFS because of this limitation, and be more or less unnoticeable to other people. As James points out there are other applications that benefit from being able to create many names for the same inode in the same directory, and 256 is a very low limit! Could this at least be put on the list of things to change? Is there a way to vote for it? And the fact that btrfs-convert crashes horribly could be fixed without a disk-format change. . . -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Josef Bacik
2011-Aug-22 16:06 UTC
Re: BTRFS should increase the hard-link in the same directory limit
On 08/22/2011 12:05 PM, John Fremlin wrote:> Josef Bacik <josef@redhat.com> writes: >> On 08/21/2011 11:13 AM, John Fremlin wrote: > [...] >>> This restriction causes btrfs-convert 0.19 to crash out with a segfault and >>> no helpful message: something like btrfs-convert: segfault at >>> ffffffffcfb25fb9 ip 000000000040f9f1 sp 00007fffddefb398 error 6 in >>> btrfs-convert[400000+21000]. >>> >>> Is there any plan to alleviate this unfortunate limit (or at least make >>> btrfs-convert give the location of the file which causes it to fail?). >> >> It''s a disk format change, something we don''t do lightly. > > It would indeed require a disk format change, and hardlinks are always > tiresome for FS designers ;-) > > I think however that the format change could be designed to only affect > people who sadly cannot at the moment use BTRFS because of this > limitation, and be more or less unnoticeable to other people. > > As James points out there are other applications that benefit from being > able to create many names for the same inode in the same directory, and > 256 is a very low limit! > > Could this at least be put on the list of things to change? Is there a > way to vote for it? > > And the fact that btrfs-convert crashes horribly could be fixed without a > disk-format change. . .It''s on the list, but there are a lot of other more pressing things then to allow weird apps to do strange things with hardlinks. Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
David Nicol
2011-Aug-23 15:29 UTC
Re: BTRFS should increase the hard-link in the same directory limit
so there are these hundreds of message files, and when one is read, a new link to The Markfile appears with a similar name as the read file? Is that right? if the point is to save inodes by making a directory entry that''s a hardlink to something already existing, why not link to the message file? On Sun, Aug 21, 2011 at 5:05 PM, James Cloos <cloos@jhcloos.com> wrote:>>>>>> "JF" == John Fremlin <john@fremlin.org> writes: > > JF> instead of creating a separate inode for each marked message, uses a > JF> hardlink to a single markfile. This means that there maybe thousands > JF> of hardlinks to the same inode in a single directory. > > And that behaviour is not limited to gnus. Many workflows use that idiom. > > -JimC > -- > James Cloos <cloos@jhcloos.com> OpenPGP: 1024D/ED7DAEA6 > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html >-- "The tools expect that they have full, unlimited control of the hardware." -- Intel Corporation -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html