Hari Gowtham
2018-Sep-10 08:51 UTC
[Gluster-users] Gluster 3.10.5: used disk size reported by quota and du mismatch
Hi, Looking at the logs, I can see that the file: /orientgate/ftp/climate/3_urban_adaptation_health/6_budapest_veszprem_hungary/RHMSS_CMCC-CM_NMMB_Balkan_8km_1971-2005 /orientgate/ftp/climate/3_urban_adaptation_health/6_budapest_veszprem_hungary/RHMSS_ERA40_NMMB_Balkan_8km_1971-2000 /orientgate/ftp/climate/3_urban_adaptation_health/6_budapest_veszprem_hungary/RHMSS_CMCC-CM_NMMB-RCP8.5_Balkan_8km_2010-2100 /primavera/cam has mismatch. You can try setting dirty for this and then do a du on it. A few corrections for my above comments. The contri size in the xattr and the aggregated size have to be checked. On Mon, Sep 10, 2018 at 1:16 PM Mauro Tridici <mauro.tridici at cmcc.it> wrote:> > > Hi Hari, > > thank you very much for your help. > I will try to use the latest available version of quota_fsck script and I will provide you a feedback as soon as possible. > > Thank you again for the detailed explanation. > Regards, > Mauro > > Il giorno 10 set 2018, alle ore 09:17, Hari Gowtham <hgowtham at redhat.com> ha scritto: > > Hi Mauro, > > The problem might be at some other place, So setting the xattr and > doing the lookup might not have fixed the issue. > > To resolve this we need to read the log file reported by the fsck > script. In this log file we need to look for the size reported by the > xattr (the value "SIZE:" in the log file) and the size reported by the > stat on the file (the value after "st_size=" ).The contri size in the xattr and the aggregated size have to be checked> These two should be the same. If they mismatch, then we have to find > the top most dir which has the mismatch.Bottom most dir/file has to be found. Replace top with bottom in the following places as well.> On this top most directory you have to do a set dirty xattr and then > do a lookup. > > If there are two different directories without a common top directory, > then both these have to undergo the above process. > > The fsck script should work fine. can you try the "--fix-issue" with > the latest script instead of the 6th patch used above? > > > >-- Regards, Hari Gowtham.
Mauro Tridici
2018-Sep-10 09:42 UTC
[Gluster-users] Gluster 3.10.5: used disk size reported by quota and du mismatch
Dear Hari,
I followed you suggestions, but, unfortunately, nothing is changed.
I tried to execute both the quota-fsck script with ?fix-issues options both the
"setfattr -n trusted.glusterfs.quota.dirty -v 0x3100? command against the
files and directory mentioned by you (on each available brick).
Disk quota assigned to /tier2/ASC directory seems to be partially used (about
2,6 TB used), but the ?real and current? situation is the following one (I
deleted all files in primavera directory):
[root at s03 qc]# du -hsc /tier2/ASC/*
22G /tier2/ASC/orientgate
26K /tier2/ASC/primavera
22G totale
So, I think that the problem should be only in "orientgate? or in
?primavera? directory, right!?
For this reason, in order to collect some fresh logs, I executed again the check
script starting from the top level directory ?ASC? using the following bash
script (named hari-20180910) based on the new version of quota_fsck (rel. 9):
hari-20180910 script:
#!/bin/bash
#set -xv
host=$(hostname)
for i in {1..12}
do
./quota_fsck_r9.py --full-logs --sub-dir ASC /gluster/mnt$i/brick >>
$host.log
done
~
In attachment, you can find the log files generated by the script.
SOME IMPORTANT NOTES:
- in the new log files, ?primavera? directory is no more present
Is there something more that I can do?
Thank you very much for your patience.
Regards,
Mauro
> Il giorno 10 set 2018, alle ore 10:51, Hari Gowtham <hgowtham at
redhat.com> ha scritto:
>
> Hi,
>
> Looking at the logs, I can see that the file:
>
>
/orientgate/ftp/climate/3_urban_adaptation_health/6_budapest_veszprem_hungary/RHMSS_CMCC-CM_NMMB_Balkan_8km_1971-2005
>
/orientgate/ftp/climate/3_urban_adaptation_health/6_budapest_veszprem_hungary/RHMSS_ERA40_NMMB_Balkan_8km_1971-2000
>
/orientgate/ftp/climate/3_urban_adaptation_health/6_budapest_veszprem_hungary/RHMSS_CMCC-CM_NMMB-RCP8.5_Balkan_8km_2010-2100
> /primavera/cam
>
> has mismatch.
>
> You can try setting dirty for this and then do a du on it.
>
> A few corrections for my above comments.
> The contri size in the xattr and the aggregated size have to be checked.
>
> On Mon, Sep 10, 2018 at 1:16 PM Mauro Tridici <mauro.tridici at
cmcc.it> wrote:
>>
>>
>> Hi Hari,
>>
>> thank you very much for your help.
>> I will try to use the latest available version of quota_fsck script and
I will provide you a feedback as soon as possible.
>>
>> Thank you again for the detailed explanation.
>> Regards,
>> Mauro
>>
>> Il giorno 10 set 2018, alle ore 09:17, Hari Gowtham <hgowtham at
redhat.com> ha scritto:
>>
>> Hi Mauro,
>>
>> The problem might be at some other place, So setting the xattr and
>> doing the lookup might not have fixed the issue.
>>
>> To resolve this we need to read the log file reported by the fsck
>> script. In this log file we need to look for the size reported by the
>> xattr (the value "SIZE:" in the log file) and the size
reported by the
>> stat on the file (the value after "st_size=" ).
>
> The contri size in the xattr and the aggregated size have to be checked
>
>> These two should be the same. If they mismatch, then we have to find
>> the top most dir which has the mismatch.
>
> Bottom most dir/file has to be found. Replace top with bottom in the
> following places as well.
>
>> On this top most directory you have to do a set dirty xattr and then
>> do a lookup.
>>
>> If there are two different directories without a common top directory,
>> then both these have to undergo the above process.
>>
>> The fsck script should work fine. can you try the
"--fix-issue" with
>> the latest script instead of the 6th patch used above?
>>
>>
>>
>>
>
>
> --
> Regards,
> Hari Gowtham.
-------------------------
Mauro Tridici
Fondazione CMCC
CMCC Supercomputing Center
presso Complesso Ecotekne - Universit? del Salento -
Strada Prov.le Lecce - Monteroni sn
73100 Lecce IT
http://www.cmcc.it
mobile: (+39) 327 5630841
email: mauro.tridici at cmcc.it
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20180910/aa4a7c73/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: log.20180910.tar.gz
Type: application/x-gzip
Size: 560447 bytes
Desc: not available
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20180910/aa4a7c73/attachment-0001.gz>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20180910/aa4a7c73/attachment-0003.html>