Mike Sampson
2012-May-05 12:37 UTC
Is it possible to reclaim block groups once they are allocated to data or metadata?
Hello list, recently reformatted my home partition from XFS to RAID1 btrfs. I used the default options to mkfs.btrfs except for enabling raid1 for data as well as metadata. Filesystem is made up of two 1TB drives. mike@mercury (0) pts/3 ~ $ sudo btrfs filesystem show Label: none uuid: f08a8896-e03e-4064-9b94-9342fb547e47 Total devices 2 FS bytes used 888.06GB devid 1 size 931.51GB used 931.51GB path /dev/sdb1 devid 2 size 931.51GB used 931.49GB path /dev/sdc1 Btrfs Btrfs v0.19 mike@mercury (0) pts/3 ~ $ btrfs filesystem df /home Data, RAID1: total=893.48GB, used=886.60GB Data: total=8.00MB, used=0.00 System, RAID1: total=8.00MB, used=136.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=38.00GB, used=2.54GB Metadata: total=8.00MB, used=0.00 As can be seen I don''t have a lot of free space left and while I am planning on adding more storage soon I would like to gain a little breathing room until I can do this. While I don''t have a lot of space remaining in Data, RAID1 I do have a good chunk in Metadata, RAID1. 2.5GB used out of 38GB. Does this excess become available automatically to the file system when the block groups in Data, RAID1 are exhausted or, if not, is there a way to manually reallocate them? Regards, Mike -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hugo Mills
2012-May-05 12:52 UTC
Re: Is it possible to reclaim block groups once they are allocated to data or metadata?
On Sat, May 05, 2012 at 10:37:01PM +1000, Mike Sampson wrote:> Hello list, > > recently reformatted my home partition from XFS to RAID1 btrfs. I used > the default options to mkfs.btrfs except for enabling raid1 for data > as well as metadata. Filesystem is made up of two 1TB drives. > > mike@mercury (0) pts/3 ~ $ sudo btrfs filesystem show > Label: none uuid: f08a8896-e03e-4064-9b94-9342fb547e47 > Total devices 2 FS bytes used 888.06GB > devid 1 size 931.51GB used 931.51GB path /dev/sdb1 > devid 2 size 931.51GB used 931.49GB path /dev/sdc1 > > Btrfs Btrfs v0.19 > > mike@mercury (0) pts/3 ~ $ btrfs filesystem df /home > Data, RAID1: total=893.48GB, used=886.60GB > Data: total=8.00MB, used=0.00 > System, RAID1: total=8.00MB, used=136.00KB > System: total=4.00MB, used=0.00 > Metadata, RAID1: total=38.00GB, used=2.54GB > Metadata: total=8.00MB, used=0.00 > > As can be seen I don''t have a lot of free space left and while I am > planning on adding more storage soon I would like to gain a little > breathing room until I can do this. While I don''t have a lot of space > remaining in Data, RAID1 I do have a good chunk in Metadata, RAID1. > 2.5GB used out of 38GB. Does this excess become available > automatically to the file system when the block groups in Data, RAID1 > are exhausted or, if not, is there a way to manually reallocate them?Youre best bet at the moment is to try a partial balance of metadata chunks: # btrfs balance start -m /home That will rewrite all of your metadata, putting it through the allocator again, and removing the original allocated chunks. This should have the effect of reducing the allocation of metadata chunks. You will need a 3.3 kernel, or later, and an up-to-date userspace from cmason''s git repository. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- All hope abandon, Ye who press Enter here. ---
Mike Sampson
2012-May-06 03:07 UTC
Re: Is it possible to reclaim block groups once they are allocated to data or metadata?
On Sat, May 5, 2012 at 10:52 PM, Hugo Mills <hugo@carfax.org.uk> wrote:> On Sat, May 05, 2012 at 10:37:01PM +1000, Mike Sampson wrote: >> Hello list, >> >> recently reformatted my home partition from XFS to RAID1 btrfs. I used >> the default options to mkfs.btrfs except for enabling raid1 for data >> as well as metadata. Filesystem is made up of two 1TB drives. >> >> mike@mercury (0) pts/3 ~ $ sudo btrfs filesystem show >> Label: none uuid: f08a8896-e03e-4064-9b94-9342fb547e47 >> Total devices 2 FS bytes used 888.06GB >> devid 1 size 931.51GB used 931.51GB path /dev/sdb1 >> devid 2 size 931.51GB used 931.49GB path /dev/sdc1 >> >> Btrfs Btrfs v0.19 >> >> mike@mercury (0) pts/3 ~ $ btrfs filesystem df /home >> Data, RAID1: total=893.48GB, used=886.60GB >> Data: total=8.00MB, used=0.00 >> System, RAID1: total=8.00MB, used=136.00KB >> System: total=4.00MB, used=0.00 >> Metadata, RAID1: total=38.00GB, used=2.54GB >> Metadata: total=8.00MB, used=0.00 >> >> As can be seen I don''t have a lot of free space left and while I am >> planning on adding more storage soon I would like to gain a little >> breathing room until I can do this. While I don''t have a lot of space >> remaining in Data, RAID1 I do have a good chunk in Metadata, RAID1. >> 2.5GB used out of 38GB. Does this excess become available >> automatically to the file system when the block groups in Data, RAID1 >> are exhausted or, if not, is there a way to manually reallocate them? > > Youre best bet at the moment is to try a partial balance of > metadata chunks: > > # btrfs balance start -m /home > > That will rewrite all of your metadata, putting it through the > allocator again, and removing the original allocated chunks. This > should have the effect of reducing the allocation of metadata chunks. > > You will need a 3.3 kernel, or later, and an up-to-date userspace > from cmason''s git repository.Gave this a shot and it did help. mike@mercury (0) pts/3 ~/.../btrfs-progs git:master $ uname -r 3.3.4-2-ARCH mike@mercury (0) pts/3 ~/.../btrfs-progs git:master $ sudo ./btrfs balance start -m /home Done, had to relocate 40 out of 934 chunks mike@mercury (0) pts/3 ~/.../btrfs-progs git:master $ ./btrfs filesystem df /home Data, RAID1: total=900.97GB, used=880.06GB Data: total=8.00MB, used=0.00 System, RAID1: total=32.00MB, used=136.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=30.50GB, used=2.54GB There is now 8GB less in Metadata and I was able to delete some files as well to free up space. There is still a lot of wasted space in the metadata block groups. It seems that it allocates more metadata block groups than required for my filesystem. This will do until I am able to add a couple of devices to the system. Is there anyway to adjust the block group allocation strategy at filesystem creation? Thanks Hugo, Mike -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Ilya Dryomov
2012-May-06 10:26 UTC
Re: Is it possible to reclaim block groups once they are allocated to data or metadata?
On Sun, May 06, 2012 at 01:07:06PM +1000, Mike Sampson wrote:> On Sat, May 5, 2012 at 10:52 PM, Hugo Mills <hugo@carfax.org.uk> wrote: > > On Sat, May 05, 2012 at 10:37:01PM +1000, Mike Sampson wrote: > >> Hello list, > >> > >> recently reformatted my home partition from XFS to RAID1 btrfs. I used > >> the default options to mkfs.btrfs except for enabling raid1 for data > >> as well as metadata. Filesystem is made up of two 1TB drives. > >> > >> mike@mercury (0) pts/3 ~ $ sudo btrfs filesystem show > >> Label: none uuid: f08a8896-e03e-4064-9b94-9342fb547e47 > >> Total devices 2 FS bytes used 888.06GB > >> devid 1 size 931.51GB used 931.51GB path /dev/sdb1 > >> devid 2 size 931.51GB used 931.49GB path /dev/sdc1 > >> > >> Btrfs Btrfs v0.19 > >> > >> mike@mercury (0) pts/3 ~ $ btrfs filesystem df /home > >> Data, RAID1: total=893.48GB, used=886.60GB > >> Data: total=8.00MB, used=0.00 > >> System, RAID1: total=8.00MB, used=136.00KB > >> System: total=4.00MB, used=0.00 > >> Metadata, RAID1: total=38.00GB, used=2.54GB > >> Metadata: total=8.00MB, used=0.00 > >> > >> As can be seen I don''t have a lot of free space left and while I am > >> planning on adding more storage soon I would like to gain a little > >> breathing room until I can do this. While I don''t have a lot of space > >> remaining in Data, RAID1 I do have a good chunk in Metadata, RAID1. > >> 2.5GB used out of 38GB. Does this excess become available > >> automatically to the file system when the block groups in Data, RAID1 > >> are exhausted or, if not, is there a way to manually reallocate them? > > > > Youre best bet at the moment is to try a partial balance of > > metadata chunks: > > > > # btrfs balance start -m /home > > > > That will rewrite all of your metadata, putting it through the > > allocator again, and removing the original allocated chunks. This > > should have the effect of reducing the allocation of metadata chunks. > > > > You will need a 3.3 kernel, or later, and an up-to-date userspace > > from cmason''s git repository. > > Gave this a shot and it did help. > > mike@mercury (0) pts/3 ~/.../btrfs-progs git:master $ uname -r > 3.3.4-2-ARCH > > mike@mercury (0) pts/3 ~/.../btrfs-progs git:master $ sudo ./btrfs > balance start -m /home > > Done, had to relocate 40 out of 934 chunks > > mike@mercury (0) pts/3 ~/.../btrfs-progs git:master $ ./btrfs > filesystem df /home > Data, RAID1: total=900.97GB, used=880.06GB > Data: total=8.00MB, used=0.00 > System, RAID1: total=32.00MB, used=136.00KB > System: total=4.00MB, used=0.00 > Metadata, RAID1: total=30.50GB, used=2.54GB > > There is now 8GB less in Metadata and I was able to delete some files > as well to free up space. There is still a lot of wasted space in the > metadata block groups. It seems that it allocates more metadata block > groups than required for my filesystem. This will do until I am able > to add a couple of devices to the system. Is there anyway to adjust > the block group allocation strategy at filesystem creation?No. Chunk allocator currently allocates a lot more chunks than actually needed, and it impacts both balancing and normal operation. Try this: # btrfs balance start -musage=10 /home This is suboptimal, but it should get rid of more chunks. Thanks, Ilya -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hugo Mills
2012-May-06 10:37 UTC
Re: Is it possible to reclaim block groups once they are allocated to data or metadata?
On Sun, May 06, 2012 at 01:26:45PM +0300, Ilya Dryomov wrote:> On Sun, May 06, 2012 at 01:07:06PM +1000, Mike Sampson wrote: > > There is now 8GB less in Metadata and I was able to delete some files > > as well to free up space. There is still a lot of wasted space in the > > metadata block groups. It seems that it allocates more metadata block > > groups than required for my filesystem. This will do until I am able > > to add a couple of devices to the system. Is there anyway to adjust > > the block group allocation strategy at filesystem creation? > > No. Chunk allocator currently allocates a lot more chunks than actually > needed, and it impacts both balancing and normal operation. Try this: > > # btrfs balance start -musage=10 /home > > This is suboptimal, but it should get rid of more chunks.While we''re talking about it, what is the parameter to the "usage" option? I''m assuming it selects chunks which are less than some amount full -- but is the value a percentage, or a quantity in megabytes (power-of-10 or power-of-2), or something else? Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Doughnut furs ache me, Omar Dorlin. ---
Ilya Dryomov
2012-May-06 14:53 UTC
Re: Is it possible to reclaim block groups once they are allocated to data or metadata?
On Sun, May 06, 2012 at 11:37:27AM +0100, Hugo Mills wrote:> On Sun, May 06, 2012 at 01:26:45PM +0300, Ilya Dryomov wrote: > > On Sun, May 06, 2012 at 01:07:06PM +1000, Mike Sampson wrote: > > > There is now 8GB less in Metadata and I was able to delete some files > > > as well to free up space. There is still a lot of wasted space in the > > > metadata block groups. It seems that it allocates more metadata block > > > groups than required for my filesystem. This will do until I am able > > > to add a couple of devices to the system. Is there anyway to adjust > > > the block group allocation strategy at filesystem creation? > > > > No. Chunk allocator currently allocates a lot more chunks than actually > > needed, and it impacts both balancing and normal operation. Try this: > > > > # btrfs balance start -musage=10 /home > > > > This is suboptimal, but it should get rid of more chunks. > > While we''re talking about it, what is the parameter to the "usage" > option? I''m assuming it selects chunks which are less than some amount > full -- but is the value a percentage, or a quantity in megabytes > (power-of-10 or power-of-2), or something else?It''s a percentage, so the command above will balance out chunks that are less than 10 percent full. I''ll update btrfs man page and the wiki page you started as soon as I can. Thanks, Ilya -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html