CentOS - May 2012 - File size diff on local disk vs NFS share

On May 3, 2012, at 3:04 PM, Glenn Cooper wrote:
>>>> I never really paid attention to this but a file on an NFS
mount is
>>>> showing 64M in size, but when copying the file to a local
drive, it
>>>> shows 2.5MB in size.
>>>> 
>>>> My NFS server is hardware Raided with a volume stripe size of
128K
>>>> were the volume size is 20TB, my local disk is about 500GB.
>>>> 
>>>> Is this due to my stripe size?
>>>> 
>>>> Nuggets are appreciated.
> 
>> By the way, this is only across NFS as when ssh'd into the server,
the file size shows 2.5M, same as the clients when its local so its seems NFS is
the culprit.
>> 
>> I have r/wsize tweaked but thats it.
> 
> Wild guess - cached file attributes not getting updated properly in the NFS
client?  Has the actual file size changed recently, like in the last few hours
or days?
> 
> Cheers,
> Glenn

Hi,

More info.

So the file in question is showing to be 2.5MB today, were as yesterday it was
~66MB.

A new file generated today is also 2.4MB but via NFS its 66MB, even when logged
in to the server locally its 66MB!

You hinted to cache and so I looked around my Raid controller settings,
specifically write through vs write back.

I will change it from write back to write through later tonight and report back.

The pattern is all files new files look to be 66MB until some time later.

I do have a 4GB mem cache on this Raid controller (Areca).

- aurf

On May 3, 2012, at 3:04 PM, Glenn Cooper wrote:
>>>> I never really paid attention to this but a file on an NFS
mount is
>>>> showing 64M in size, but when copying the file to a local
drive, it
>>>> shows 2.5MB in size.
>>>> 
>>>> My NFS server is hardware Raided with a volume stripe size of
128K
>>>> were the volume size is 20TB, my local disk is about 500GB.
>>>> 
>>>> Is this due to my stripe size?
>>>> 
>>>> Nuggets are appreciated.
> 
>> By the way, this is only across NFS as when ssh'd into the server,
the file size shows 2.5M, same as the clients when its local so its seems NFS is
the culprit.
>> 
>> I have r/wsize tweaked but thats it.
> 
> Wild guess - cached file attributes not getting updated properly in the NFS
client?  Has the actual file size changed recently, like in the last few hours
or days?
> 
> Cheers,
> Glenn
Something more relevant perhaps, mount options via FSTAB on the server are;

/dev/###		/mountpoint       xfs    
defaults,allocsize=64m,relatime,logbufs=8,logbsize=256k		0 0

Notice the allocsize of 64MB, perhaps relatime might be playing a role as well?

I will remove allocsize first and report back.

- aurf