This is gluster 3.6.6. I've attempted to disable and re-enable quota's on the volume, but when I re-apply the quotas on each directory the same 'Used' value is present as before. Where is quotad getting its information from, and how can I clean up/regenerate that info? On Thu, Jan 21, 2016 at 10:07 AM, Steve Dainard <sdainard at spd1.com> wrote:> I have a distributed volume with quota's enabled: > > Volume Name: storage > Type: Distribute > Volume ID: 26d355cb-c486-481f-ac16-e25390e73775 > Status: Started > Number of Bricks: 4 > Transport-type: tcp > Bricks: > Brick1: 10.0.231.50:/mnt/raid6-storage/storage > Brick2: 10.0.231.51:/mnt/raid6-storage/storage > Brick3: 10.0.231.52:/mnt/raid6-storage/storage > Brick4: 10.0.231.53:/mnt/raid6-storage/storage > Options Reconfigured: > performance.cache-size: 1GB > performance.readdir-ahead: on > features.quota: on > diagnostics.brick-log-level: WARNING > > Here is a partial list of quotas: > # /usr/sbin/gluster volume quota storage list > Path Hard-limit Soft-limit Used > Available Soft-limit exceeded? Hard-limit exceeded? > --------------------------------------------------------------------------------------------------------------------------- > ... > /projects-CanSISE 10.0TB 80% 11.9TB > 0Bytes Yes Yes > ... > > If I du on that location I do not get 11.9TB of space used (fuse mount point): > [root at storage projects-CanSISE]# du -hs > 9.5T . > > Can someone provide an explanation for how the quota mechanism tracks > disk usage? How often does the quota mechanism check its accuracy? And > how could it get so far off? > > Can I get gluster to rescan that location and update the quota usage? > > Thanks, > Steve
Manikandan Selvaganesh
2016-Jan-22 05:20 UTC
[Gluster-users] Quota list not reflecting disk usage
Hi Steve,
If you would like disk usage using df utility by taking quota limits into
consideration, then you are expected to run the following command.
'gluster volume set VOLNAME quota-deem-statfs on'
with older versions where quota-deem-statfs is OFF by default. However with
the latest versions, quota-deem-statfs is by default ON. In this case, the total
disk space of the directory is taken as the quota hard limit set on the
directory
of the volume and disk utility would display accordingly. This answers why there
is
a mismatch in disk utility.
Next, answering to quota mechanism and accuracy: There is something called
timeouts
in quota. For performance reasons, quota caches the directory size on client.
You can
set timeout indicating the maximum valid duration of directory sizes in cache,
from the time they are populated. By default the hard-timeout is 5s and soft
timeout
is 60s. Setting a timeout of zero will do a force fetching of directory sizes
from server
for every operation that modifies file data and will effectively disables
directory size
caching on client side. If you do not have a timeout of 0(which we do not
encourage due to
performance reasons), then till you reach soft-limit, soft timeout will be taken
into
consideration, and only for every 60s operations will be synced and that could
cause the
usage to exceed more than the hard-limit specified. If you would like quota to
strictly enforce then please run the following commands,
'gluster v quota VOLNAME hard-timeout 0s'
'gluster v quota VOLNAME soft-timeout 0s'
Appreciate your curiosity in exploring and if you would like to know more about
quota
please refer[1]
[1]
http://gluster.readthedocs.org/en/release-3.7.0-1/Administrator%20Guide/Directory%20Quota/
--
Thanks & Regards,
Manikandan Selvaganesh.
----- Original Message -----
From: "Steve Dainard" <sdainard at spd1.com>
To: "gluster-users at gluster.org List" <gluster-users at
gluster.org>
Sent: Friday, January 22, 2016 1:40:07 AM
Subject: Re: [Gluster-users] Quota list not reflecting disk usage
This is gluster 3.6.6.
I've attempted to disable and re-enable quota's on the volume, but
when I re-apply the quotas on each directory the same 'Used' value is
present as before.
Where is quotad getting its information from, and how can I clean
up/regenerate that info?
On Thu, Jan 21, 2016 at 10:07 AM, Steve Dainard <sdainard at spd1.com>
wrote:> I have a distributed volume with quota's enabled:
>
> Volume Name: storage
> Type: Distribute
> Volume ID: 26d355cb-c486-481f-ac16-e25390e73775
> Status: Started
> Number of Bricks: 4
> Transport-type: tcp
> Bricks:
> Brick1: 10.0.231.50:/mnt/raid6-storage/storage
> Brick2: 10.0.231.51:/mnt/raid6-storage/storage
> Brick3: 10.0.231.52:/mnt/raid6-storage/storage
> Brick4: 10.0.231.53:/mnt/raid6-storage/storage
> Options Reconfigured:
> performance.cache-size: 1GB
> performance.readdir-ahead: on
> features.quota: on
> diagnostics.brick-log-level: WARNING
>
> Here is a partial list of quotas:
> # /usr/sbin/gluster volume quota storage list
> Path Hard-limit Soft-limit Used
> Available Soft-limit exceeded? Hard-limit exceeded?
>
---------------------------------------------------------------------------------------------------------------------------
> ...
> /projects-CanSISE 10.0TB 80% 11.9TB
> 0Bytes Yes Yes
> ...
>
> If I du on that location I do not get 11.9TB of space used (fuse mount
point):
> [root at storage projects-CanSISE]# du -hs
> 9.5T .
>
> Can someone provide an explanation for how the quota mechanism tracks
> disk usage? How often does the quota mechanism check its accuracy? And
> how could it get so far off?
>
> Can I get gluster to rescan that location and update the quota usage?
>
> Thanks,
> Steve
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users