CentOS - Jun 2006 - unable to write files bigger than 4GB over NFS

This may not be a CentOS problem, because I'm in a mixed environment.  
But I put it out in case anybody has any thoughts.

I recently replaced a file server, which was on a Pentium III based  
system running 32-bit CentOS 4.3, with an Opteron based system  
running Fedora 4. The NFS clients are all CentOS 4.3, some are 32 bit  
and some are 64 bit.

Since the replacement, when I try to write a big file from the 32bit  
CentOS  boxes over NFS to the server, I get "filesize exceed  
messages" at 2GB. However this used to work with the old server. And  
I can  write big files if I'm logged on to the server.

One more oddity, the  shared file system on the server is xfs  
formatted; if I instead mount a reiserfs file system I can  
successfully write. This part does not make sense to me.

And one more, I can write large files from a 32-bit Fedora 4 system  
to the server over NFS, just not from 32-bit CentOS.

I'm exporting with (rw,async) and mounting with  
(bg,rsize=8192,wsize=8192)

Is there some interaction between nfs and xfs that I need to know about?

Thanks,
Tony Schreiner
Boston College

On 6/22/06, Tony Schreiner <schreian at bc.edu>
wrote:
> This may not be a CentOS problem, because I'm in a mixed environment.
> But I put it out in case anybody has any thoughts.
>
> I recently replaced a file server, which was on a Pentium III based
> system running 32-bit CentOS 4.3, with an Opteron based system
> running Fedora 4. The NFS clients are all CentOS 4.3, some are 32 bit
> and some are 64 bit.
>
> Since the replacement, when I try to write a big file from the 32bit
> CentOS  boxes over NFS to the server, I get "filesize exceed
> messages" at 2GB. However this used to work with the old server. And
> I can  write big files if I'm logged on to the server.
>
> One more oddity, the  shared file system on the server is xfs
> formatted; if I instead mount a reiserfs file system I can
> successfully write. This part does not make sense to me.
>
> And one more, I can write large files from a 32-bit Fedora 4 system
> to the server over NFS, just not from 32-bit CentOS.
>
> I'm exporting with (rw,async) and mounting with
> (bg,rsize=8192,wsize=8192)
>
> Is there some interaction between nfs and xfs that I need to know about?
>
I don't know whether this is your problem or not, but when we upgraded
our desktops to RHEL4-U3 (boxes were running RH9 or RHEL3), we noticed
weirdness with NFS. Our developers use NFSs between their desktops and
embedded Linux boxes. Worked without a hitch before the latest nfs in
RHEL4. The problem turned out to be that the newer NFS was overrunning
the (very minimal) buffers in the embedded Linux boxes. We had to use
rsize=1024,wsize=1024. Without the transfer size limitations the
transfer would terminate at exactly 16384 bytes.

HTH,

-- 
Collins Richey
     If you fill your heart with regrets of yesterday and the worries
     of tomorrow, you have no today to be thankful for.

Tony Schreiner wrote:
> This may not be a CentOS problem, because I'm in a mixed environment.  
> But I put it out in case anybody has any thoughts.
>
> I recently replaced a file server, which was on a Pentium III based  
> system running 32-bit CentOS 4.3, with an Opteron based system  
> running Fedora 4. The NFS clients are all CentOS 4.3, some are 32 bit  
> and some are 64 bit.
>
> Since the replacement, when I try to write a big file from the 32bit  
> CentOS  boxes over NFS to the server, I get "filesize exceed  
> messages" at 2GB. However this used to work with the old server. And  
> I can  write big files if I'm logged on to the server.
>
I belive that the maximum file size for an NFS partition is 2G, so I am 
surprised that you didn't see the problem before replacing the server.  
I got round this problem by using ssh, but this probably isn't going to 
be of use if you need to use the files frequently.
> One more oddity, the  shared file system on the server is xfs  
> formatted; if I instead mount a reiserfs file system I can  
> successfully write. This part does not make sense to me.
>
Maybe the limit doesn't come from just NFS but a combination of NFS and 
the filesystem.