Btrfs devel - Jan 2008 - Handling of big volumes in btrfs

Hi,

>From the XFS list I gather that btrfs, being able to mount a >16 TB 
volume on a 32-bit x86 box, either has some special power - or bug.
Which of the two is it?


---------- Forwarded message ----------
Date: Sat, 19 Jan 2008 15:26:26 -0600
From: Eric Sandeen
Subject: Re: Volume too big

Jan Engelhardt wrote:
> On Jan 19 2008 14:41, Eric Sandeen wrote:
>> It's possible that btrfs can cope with this somehow - but also
quite
>> possible that it's just missing the right checks :)
>>
> I am not sure why Linux would be limited to 16 TB. If LBD is on,
> things are 64 bit, so I would expect to have at least access to
> 2 exabyte.
64-bit sector addressing, but there is a 32-bit index into the (4k)
pagecache.

2^32 * 4096 is 16T

So an address space has a 16T limit.

Even mkfs, if it needs to write past 16T (and I think mkfs.btrfs doesn't
need that...) will have trouble, if the device is > 16T - unless mkfs
uses direct IO.

-Eric

On Saturday 19 January 2008, Jan Engelhardt wrote:
> Hi,
>
> >From the XFS list I gather that btrfs, being able to mount a >16 TB
>
> volume on a 32-bit x86 box, either has some special power - or bug.
> Which of the two is it?
mkfs.btrfs doesn't have to write past 16TB, so it isn't an issue from
userland
(btrfsck is a different story).  That would have to shift to O_DIRECT IO, 
which wouldn't be horrible now that Yan has extent_buffers ported.

From the kernel, you'll bump into the unsigned long radix tree, which is
only
32 bits.  Currently a single address_space is used as the backing for the 
block device, which is used for all metadata.

There aren't many functions that care about this, basically just the ones
that
read, write or find pages for extent_buffers.

So, btrfs could chain multiple address spaces together to go beyond 16TB, and 
abstract it out in extent_map.c:extend_buffer_page() and a handful of other 
spots.

-chris