Artem Russakovskii
2018-Apr-14 18:39 UTC
[Gluster-users] Getting glusterfs to expand volume size to brick size
Hi, I have a 3-brick replicate volume, but for some reason I can't get it to expand to the size of the bricks. The bricks are 25GB, but even after multiple gluster restarts and remounts, the volume is only about 8GB. I believed I could always extend the bricks (we're using Linode block storage, which allows extending block devices after they're created), and gluster would see the newly available space and extend to use it. Multiple Google searches, and I'm still nowhere. Any ideas? df | ack "block|data" Filesystem 1M-blocks Used Available Use% Mounted on /dev/sdd 25071M 1491M 22284M 7% /mnt/pylon_block1 /dev/sdc 26079M 1491M 23241M 7% /mnt/pylon_block2 /dev/sde 25071M 1491M 22315M 7% /mnt/pylon_block3 localhost:/dev_apkmirror_data 8357M 581M 7428M 8% /mnt/dev_apkmirror_data1 localhost:/dev_apkmirror_data 8357M 581M 7428M 8% /mnt/dev_apkmirror_data2 localhost:/dev_apkmirror_data 8357M 581M 7428M 8% /mnt/dev_apkmirror_data3 gluster volume info Volume Name: dev_apkmirror_data Type: Replicate Volume ID: cd5621ee-7fab-401b-b720-08863717ed56 Status: Started Snapshot Count: 0 Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: pylon:/mnt/pylon_block1/dev_apkmirror_data Brick2: pylon:/mnt/pylon_block2/dev_apkmirror_data Brick3: pylon:/mnt/pylon_block3/dev_apkmirror_data Options Reconfigured: disperse.eager-lock: off cluster.lookup-unhashed: auto cluster.read-hash-mode: 0 performance.strict-o-direct: on cluster.shd-max-threads: 12 performance.nl-cache-timeout: 600 performance.nl-cache: on cluster.quorum-count: 1 cluster.quorum-type: fixed network.ping-timeout: 5 network.remote-dio: enable performance.rda-cache-limit: 256MB performance.parallel-readdir: on network.inode-lru-limit: 500000 performance.md-cache-timeout: 600 performance.cache-invalidation: on performance.stat-prefetch: on features.cache-invalidation-timeout: 600 features.cache-invalidation: on performance.io-thread-count: 32 server.event-threads: 4 client.event-threads: 4 performance.read-ahead: off cluster.lookup-optimize: on performance.client-io-threads: on performance.cache-size: 1GB transport.address-family: inet performance.readdir-ahead: on nfs.disable: on cluster.readdir-optimize: on Thank you. Sincerely, Artem -- Founder, Android Police <http://www.androidpolice.com>, APK Mirror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | +ArtemRussakovskii <https://plus.google.com/+ArtemRussakovskii> | @ArtemR <http://twitter.com/ArtemR> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180414/dfa51d4b/attachment.html>
Nithya Balachandran
2018-Apr-16 05:56 UTC
[Gluster-users] Getting glusterfs to expand volume size to brick size
What version of Gluster are you running? Were the bricks smaller earlier? Regards, Nithya On 15 April 2018 at 00:09, Artem Russakovskii <archon810 at gmail.com> wrote:> Hi, > > I have a 3-brick replicate volume, but for some reason I can't get it to > expand to the size of the bricks. The bricks are 25GB, but even after > multiple gluster restarts and remounts, the volume is only about 8GB. > > I believed I could always extend the bricks (we're using Linode block > storage, which allows extending block devices after they're created), and > gluster would see the newly available space and extend to use it. > > Multiple Google searches, and I'm still nowhere. Any ideas? > > df | ack "block|data" > Filesystem 1M-blocks > Used Available Use% Mounted on > /dev/sdd 25071M > 1491M 22284M 7% /mnt/pylon_block1 > /dev/sdc 26079M > 1491M 23241M 7% /mnt/pylon_block2 > /dev/sde 25071M > 1491M 22315M 7% /mnt/pylon_block3 > localhost:/dev_apkmirror_data 8357M > 581M 7428M 8% /mnt/dev_apkmirror_data1 > localhost:/dev_apkmirror_data 8357M > 581M 7428M 8% /mnt/dev_apkmirror_data2 > localhost:/dev_apkmirror_data 8357M > 581M 7428M 8% /mnt/dev_apkmirror_data3 > > > > gluster volume info > > Volume Name: dev_apkmirror_data > Type: Replicate > Volume ID: cd5621ee-7fab-401b-b720-08863717ed56 > Status: Started > Snapshot Count: 0 > Number of Bricks: 1 x 3 = 3 > Transport-type: tcp > Bricks: > Brick1: pylon:/mnt/pylon_block1/dev_apkmirror_data > Brick2: pylon:/mnt/pylon_block2/dev_apkmirror_data > Brick3: pylon:/mnt/pylon_block3/dev_apkmirror_data > Options Reconfigured: > disperse.eager-lock: off > cluster.lookup-unhashed: auto > cluster.read-hash-mode: 0 > performance.strict-o-direct: on > cluster.shd-max-threads: 12 > performance.nl-cache-timeout: 600 > performance.nl-cache: on > cluster.quorum-count: 1 > cluster.quorum-type: fixed > network.ping-timeout: 5 > network.remote-dio: enable > performance.rda-cache-limit: 256MB > performance.parallel-readdir: on > network.inode-lru-limit: 500000 > performance.md-cache-timeout: 600 > performance.cache-invalidation: on > performance.stat-prefetch: on > features.cache-invalidation-timeout: 600 > features.cache-invalidation: on > performance.io-thread-count: 32 > server.event-threads: 4 > client.event-threads: 4 > performance.read-ahead: off > cluster.lookup-optimize: on > performance.client-io-threads: on > performance.cache-size: 1GB > transport.address-family: inet > performance.readdir-ahead: on > nfs.disable: on > cluster.readdir-optimize: on > > > Thank you. > > Sincerely, > Artem > > -- > Founder, Android Police <http://www.androidpolice.com>, APK Mirror > <http://www.apkmirror.com/>, Illogical Robot LLC > beerpla.net | +ArtemRussakovskii > <https://plus.google.com/+ArtemRussakovskii> | @ArtemR > <http://twitter.com/ArtemR> > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180416/1b443c1f/attachment.html>
Artem Russakovskii
2018-Apr-16 23:47 UTC
[Gluster-users] Getting glusterfs to expand volume size to brick size
Hi Nithya, I'm on Gluster 4.0.1. I don't think the bricks were smaller before - if they were, maybe 20GB because Linode's minimum is 20GB, then I extended them to 25GB, resized with resize2fs as instructed, and rebooted many times over since. Yet, gluster refuses to see the full disk size. Here's the status detail output: gluster volume status dev_apkmirror_data detail Status of volume: dev_apkmirror_data ------------------------------------------------------------ ------------------ Brick : Brick pylon:/mnt/pylon_block1/dev_apkmirror_data TCP Port : 49152 RDMA Port : 0 Online : Y Pid : 1263 File System : ext4 Device : /dev/sdd Mount Options : rw,relatime,data=ordered Inode Size : 256 Disk Space Free : 23.0GB Total Disk Space : 24.5GB Inode Count : 1638400 Free Inodes : 1625429 ------------------------------------------------------------ ------------------ Brick : Brick pylon:/mnt/pylon_block2/dev_apkmirror_data TCP Port : 49153 RDMA Port : 0 Online : Y Pid : 1288 File System : ext4 Device : /dev/sdc Mount Options : rw,relatime,data=ordered Inode Size : 256 Disk Space Free : 24.0GB Total Disk Space : 25.5GB Inode Count : 1703936 Free Inodes : 1690965 ------------------------------------------------------------ ------------------ Brick : Brick pylon:/mnt/pylon_block3/dev_apkmirror_data TCP Port : 49154 RDMA Port : 0 Online : Y Pid : 1313 File System : ext4 Device : /dev/sde Mount Options : rw,relatime,data=ordered Inode Size : 256 Disk Space Free : 23.0GB Total Disk Space : 24.5GB Inode Count : 1638400 Free Inodes : 1625433 What's interesting here is that the gluster volume size is exactly 1/3 of the total (8357M * 3 = 25071M). Yet, each block device is separate, and the total storage available is 25071M on each brick. The fstab is as follows: /dev/disk/by-id/scsi-0Linode_Volume_pylon_block1 /mnt/pylon_block1 ext4 defaults 0 2 /dev/disk/by-id/scsi-0Linode_Volume_pylon_block2 /mnt/pylon_block2 ext4 defaults 0 2 /dev/disk/by-id/scsi-0Linode_Volume_pylon_block3 /mnt/pylon_block3 ext4 defaults 0 2 localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data1 glusterfs defaults,_netdev,fopen-keep-cache,direct-io-mode=enable 0 0 localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data2 glusterfs defaults,_netdev,fopen-keep-cache,direct-io-mode=enable 0 0 localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data3 glusterfs defaults,_netdev,fopen-keep-cache,direct-io-mode=enable 0 0 localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data_ganesha nfs4 defaults,_netdev,bg,intr,soft,timeo=5,retrans=5,actimeo=10,retry=5 0 0 The latter entry is for an nfs ganesha test, in case it matters (which, btw, fails miserably with all kinds of stability issues about broken pipes). Note: this is a test server, so all 3 bricks are attached and mounted on the same server. Sincerely, Artem -- Founder, Android Police <http://www.androidpolice.com>, APK Mirror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | +ArtemRussakovskii <https://plus.google.com/+ArtemRussakovskii> | @ArtemR <http://twitter.com/ArtemR> On Sun, Apr 15, 2018 at 10:56 PM, Nithya Balachandran <nbalacha at redhat.com> wrote:> What version of Gluster are you running? Were the bricks smaller earlier? > > Regards, > Nithya > > On 15 April 2018 at 00:09, Artem Russakovskii <archon810 at gmail.com> wrote: > >> Hi, >> >> I have a 3-brick replicate volume, but for some reason I can't get it to >> expand to the size of the bricks. The bricks are 25GB, but even after >> multiple gluster restarts and remounts, the volume is only about 8GB. >> >> I believed I could always extend the bricks (we're using Linode block >> storage, which allows extending block devices after they're created), and >> gluster would see the newly available space and extend to use it. >> >> Multiple Google searches, and I'm still nowhere. Any ideas? >> >> df | ack "block|data" >> Filesystem 1M-blocks >> Used Available Use% Mounted on >> /dev/sdd 25071M >> 1491M 22284M 7% /mnt/pylon_block1 >> /dev/sdc 26079M >> 1491M 23241M 7% /mnt/pylon_block2 >> /dev/sde 25071M >> 1491M 22315M 7% /mnt/pylon_block3 >> localhost:/dev_apkmirror_data 8357M >> 581M 7428M 8% /mnt/dev_apkmirror_data1 >> localhost:/dev_apkmirror_data 8357M >> 581M 7428M 8% /mnt/dev_apkmirror_data2 >> localhost:/dev_apkmirror_data 8357M >> 581M 7428M 8% /mnt/dev_apkmirror_data3 >> >> >> >> gluster volume info >> >> Volume Name: dev_apkmirror_data >> Type: Replicate >> Volume ID: cd5621ee-7fab-401b-b720-08863717ed56 >> Status: Started >> Snapshot Count: 0 >> Number of Bricks: 1 x 3 = 3 >> Transport-type: tcp >> Bricks: >> Brick1: pylon:/mnt/pylon_block1/dev_apkmirror_data >> Brick2: pylon:/mnt/pylon_block2/dev_apkmirror_data >> Brick3: pylon:/mnt/pylon_block3/dev_apkmirror_data >> Options Reconfigured: >> disperse.eager-lock: off >> cluster.lookup-unhashed: auto >> cluster.read-hash-mode: 0 >> performance.strict-o-direct: on >> cluster.shd-max-threads: 12 >> performance.nl-cache-timeout: 600 >> performance.nl-cache: on >> cluster.quorum-count: 1 >> cluster.quorum-type: fixed >> network.ping-timeout: 5 >> network.remote-dio: enable >> performance.rda-cache-limit: 256MB >> performance.parallel-readdir: on >> network.inode-lru-limit: 500000 >> performance.md-cache-timeout: 600 >> performance.cache-invalidation: on >> performance.stat-prefetch: on >> features.cache-invalidation-timeout: 600 >> features.cache-invalidation: on >> performance.io-thread-count: 32 >> server.event-threads: 4 >> client.event-threads: 4 >> performance.read-ahead: off >> cluster.lookup-optimize: on >> performance.client-io-threads: on >> performance.cache-size: 1GB >> transport.address-family: inet >> performance.readdir-ahead: on >> nfs.disable: on >> cluster.readdir-optimize: on >> >> >> Thank you. >> >> Sincerely, >> Artem >> >> -- >> Founder, Android Police <http://www.androidpolice.com>, APK Mirror >> <http://www.apkmirror.com/>, Illogical Robot LLC >> beerpla.net | +ArtemRussakovskii >> <https://plus.google.com/+ArtemRussakovskii> | @ArtemR >> <http://twitter.com/ArtemR> >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-users >> > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180416/b108df76/attachment.html>
Seemingly Similar Threads
- Getting glusterfs to expand volume size to brick size
- Getting glusterfs to expand volume size to brick size
- Getting glusterfs to expand volume size to brick size
- Getting glusterfs to expand volume size to brick size
- Getting glusterfs to expand volume size to brick size