Julie Ashworth
2011-Jun-25 00:17 UTC
[CentOS] Quota (and disk usage) is incorrectly reported on nfs client mounting XFS filesystem
hi all,
I hope you can help me with a strange quota/nfs/XFS behavior...
I'm using Centos 5.6 on both a nfs server and client. Both are
64-bit, and using a recent kernel:
NFS server:
Linux fs2.priv 2.6.18-238.12.1.el5 #1 SMP Tue May 31 13:22:04 EDT 2011 x86_64
x86_64 x86_64 GNU/Linux
NFS client:
Linux nx8.priv 2.6.18-238.12.1.el5 #1 SMP Tue May 31 13:22:04 EDT 2011 x86_64
x86_64 x86_64 GNU/Linux
The NFS server is exporting a XFS filesystem:
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vgXX-lvXX 24T 16T 8.2T 66% /export
User foo (for anonymity) added ~3TB of data in the last 2-3 days.
On the NFS server, her quota is reported as ~5.8 TB:
Disk quotas for user foo (uid 1314):
Filesystem blocks quota limit grace files quota limit grace
/dev/mapper/vgXXX-lvXXX
6129681348 7490000000 0 6991914 0 0
On a NFS client, her quota is reported as ~1.8TB:
Disk quotas for user foo (uid 1314):
Filesystem blocks quota limit grace files quota limit grace
fs2:/XXXXX 1831504512 3195032704 0 6988049 0 0
When I modify her quota on the NFS sever, it's not reflected
on the NFS client.
This problem is specific user foo. There are about 20 other
users with quotas on this NFS server, and if I change one of
their quotas, the quota/usage is correctly (and instantly)
reflected on the NFS client.
rpc.quotad is configured to use port 875, and I can see this
is working with strace. And tcpdump shows an established
connection:
16:50:04.480757 IP nx8.priv.pssc > fs2.priv.875: UDP, length 116
16:50:04.481015 IP fs2.priv.875 > nx8.priv.pssc: UDP, length 68
Any ideas?
Thanks!
Best,
Julie
--
Julie Ashworth <julie.ashworth at berkeley.edu>
http://www.neuro.berkeley.edu
PGP Key ID: 0x17F013D2
Julie Ashworth
2011-Jun-28 20:43 UTC
[CentOS] Quota (and disk usage) is incorrectly reported on nfs client mounting XFS filesystem
hi all,
I've done more investigation into this problem. I think I
uncovered a bug, but I want a sanity check... (and please excuse
me for top posting - I wanted to include the original message
for easier reference).
As a reminder, I have a XFS filesystem exported via nfs4. The
nfs4 clients (all 13) report the quota incorrectly for a user
with >6TB data.
On the NFS server, the 'quota' and 'xfs_quota' commands report
disk usage correctly.
I modified the this user's quota on the server, and then
report the output of the 'quota' command below. I start
with a quota equal to the disk usage (6635021816), and then
decrease it, to the point that the client reports the same
quota value as the server.
Note: the lines separate instances where I adjusted the quota,
where the top line is 'quota' output on the nfs server, the
second line is from the nfs client. (The asterisk just
indicates that the user exceeded the quota block limit).
-----------------------------------------
Filesystem blocks quota
/dev/mapper/fs 6635021816 6635021816
Filesystem blocks quota
server:/fs 2340054520 2340054520
-----------------------------------------
Filesystem blocks quota
/dev/mapper/fs 6635021816* 4294990000
Filesystem blocks quota
server:/fs 2340054520* 22704
-----------------------------------------
Filesystem blocks quota
/dev/mapper/fs 6635021816* 4294970000
Filesystem blocks quota
server:/fs 2340054520* 2704
-----------------------------------------
Filesystem blocks quota
/dev/mapper/fs 6635021816* 4294970000
Filesystem blocks quota
server:/fs 2340054520* 1704
-----------------------------------------
Filesystem blocks quota
/dev/mapper/fs 6635021816* 4294968000
Filesystem blocks quota
server:/fs 2340054520* 704
-----------------------------------------
Filesystem blocks quota
/dev/mapper/fs 6635021816* 4294967900
Filesystem blocks quota
server:/fs 2340054520* 604
-----------------------------------------
Filesystem blocks quota
/dev/mapper/fs 6635021816* 4294967500
Filesystem blocks quota
server:/fs 2340054520* 204
-----------------------------------------
Filesystem blocks quota
/dev/mapper/fs 6635021816* 4294967300
Filesystem blocks quota
server:/fs 2340054520* 4
-----------------------------------------
Filesystem blocks quota
/dev/mapper/fs 6635021816* 4294967296
<NONE>
-----------------------------------------
Filesystem blocks quota
/dev/mapper/fs 6635021816* 4294967292
Filesystem blocks quota
server:/fs 2340054520 4294967292
-----------------------------------------
I think it's interesting that the second to last instance
didn't report any quota for this user on the client. Of
course, increasing or decreasing the quota produced output.
Any insight?
Thanks!
Julie
On 24-06-2011 17.17 -0700, Julie Ashworth wrote:> hi all,
> I hope you can help me with a strange quota/nfs/XFS behavior...
>
> I'm using Centos 5.6 on both a nfs server and client. Both are
> 64-bit, and using a recent kernel:
>
> NFS server:
> Linux fs2.priv 2.6.18-238.12.1.el5 #1 SMP Tue May 31 13:22:04 EDT 2011
x86_64 x86_64 x86_64 GNU/Linux
>
> NFS client:
> Linux nx8.priv 2.6.18-238.12.1.el5 #1 SMP Tue May 31 13:22:04 EDT 2011
x86_64 x86_64 x86_64 GNU/Linux
>
> The NFS server is exporting a XFS filesystem:
> Filesystem Size Used Avail Use% Mounted on
> /dev/mapper/vgXX-lvXX 24T 16T 8.2T 66% /export
>
>
>
> User foo (for anonymity) added ~3TB of data in the last 2-3 days.
> On the NFS server, her quota is reported as ~5.8 TB:
>
> Disk quotas for user foo (uid 1314):
> Filesystem blocks quota limit grace files quota limit
grace
> /dev/mapper/vgXXX-lvXXX
> 6129681348 7490000000 0 6991914 0
0
>
>
> On a NFS client, her quota is reported as ~1.8TB:
>
> Disk quotas for user foo (uid 1314):
> Filesystem blocks quota limit grace files quota limit
grace
> server:/XXXXX 1831504512 3195032704 0 6988049 0
0
>
>
> When I modify her quota on the NFS sever, it's not reflected
> on the NFS client.
>
> This problem is specific user foo. There are about 20 other
> users with quotas on this NFS server, and if I change one of
> their quotas, the quota/usage is correctly (and instantly)
> reflected on the NFS client.
>
> rpc.quotad is configured to use port 875, and I can see this
> is working with strace. And tcpdump shows an established
> connection:
> 16:50:04.480757 IP nx8.priv.pssc > fs2.priv.875: UDP, length 116
> 16:50:04.481015 IP fs2.priv.875 > nx8.priv.pssc: UDP, length 68
>
> Any ideas?
>
> Thanks!
> Best,
> Julie
>
>
> --
> Julie Ashworth <julie.ashworth at berkeley.edu>
> http://www.neuro.berkeley.edu
> PGP Key ID: 0x17F013D2
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
---end quoted text---
--
Julie Ashworth <julie.ashworth at berkeley.edu>
http://www.neuro.berkeley.edu
PGP Key ID: 0x17F013D2