Hello, we want to migrate to CentOS 7(.1) , Server, Client etc ... Right now we need to suspend that because we have some serious issues with NFS4 shared ZFS ( kernel module - zfs on linux project ) Volumes and CentOS 7 clients. Our current server are CentOS6.5, if we share a zfs volume and do the following on this share: client A reads/access file "z.txt". Now client B replaces (copy, move, unlink+link ) files "z.txt" with a new different version and now client A reads/access the file "z.txt" ( and only the file - do not do anything which does a "stat" on the file e.g. "cat z.txt") the old version/content is read. As long as you do something that issues a "stat" for the file/dir. This is really a problem because a soon you are not browsing through the dirs e.g. updating some scripts or anything automated the changes are not recognized by the client. This does not happen with a ext4 share. This also happen with a centos7.0 NFS Server and ZFS/XFS. We permute pretty much every nfs mount/share option - it always happens. Nothing does help. This does not happen with a Centos7.1 server and a centos7.1 client and xfs (NFS4.1). Of course it works with our current client server Version: Ubuntu 12.04 / CentOS 6.5. It really does not seem to be a zfs issue - it does look more like a nfs problem with certain filessystems. I asked this already on the zfs mailing list. So is there something I can do? Is the problem known? Is there a workaround , patch or fix? I am still unsure if I missed something completely because the problem is just so serious... With kind regards Timo Ballin ____ ESET 11570 (20150504) ____ The message was checked by ESET Mail Security.
John R Pierce
2015-May-04 08:22 UTC
[CentOS] CentOS 7, NFS 4, and a non ext4 fs (like zfs)
On 5/4/2015 1:09 AM, tballin wrote:> Hello, > > we want to migrate to CentOS 7(.1) , Server, Client etc ... > > Right now we need to suspend that because we have some serious issues > with NFS4 shared ZFS ( kernel module - zfs on linux project ) Volumes > and CentOS 7 clients. > > Our current server are CentOS6.5, if we share a zfs volume and do the > following on this share: client A reads/access file "z.txt". Now > client B replaces (copy, move, unlink+link ) files "z.txt" with a new > different version and now client A reads/access the file "z.txt" ( and > only the file - do not do anything which does a "stat" on the file > e.g. "cat z.txt") the old version/content is read. As long as you do > something that issues a "stat" for the file/dir.I've used ZFS file systems on a NFS Server extensively on Solaris, and a fair bit with FreeBSD 9.x, 10.1, and its never had any such issues. AFAIK, ZFS on Linux is still considered experimental. -- john r pierce, recycling bits in santa cruz
Hi Perhaps. But they (ZoL) say its production ready. It worked since CentosOS 6.1 to 6.6 and still working. Problem occurs if we change the NFS version. Not the ZFS. And the problem also exsist with XFS on CentOS 7. Also it works with all old Ubuntu Clients here. The main problem seems to be the client which seems to use NFS features which are not (yet) working! Also with XFS you need to use NFS 4.1 or you will have the old links/nodes. problem too. As you say the Solaris/BSD NFS Server Version seems to work. Thats what the ZFS mailing list said. If it works under FreeBSD its the NFS Server... I could try some other file systems... But I would also like to know what the issue is. I am still wondering what the problem is. because on the server everything looks fine. Timo On 05/04/2015 10:22 AM, John R Pierce wrote:> On 5/4/2015 1:09 AM, tballin wrote: >> Hello, >> >> we want to migrate to CentOS 7(.1) , Server, Client etc ... >> >> Right now we need to suspend that because we have some serious issues >> with NFS4 shared ZFS ( kernel module - zfs on linux project ) Volumes >> and CentOS 7 clients. >> >> Our current server are CentOS6.5, if we share a zfs volume and do the >> following on this share: client A reads/access file "z.txt". Now >> client B replaces (copy, move, unlink+link ) files "z.txt" with a new >> different version and now client A reads/access the file "z.txt" ( >> and only the file - do not do anything which does a "stat" on the >> file e.g. "cat z.txt") the old version/content is read. As long as >> you do something that issues a "stat" for the file/dir. > > I've used ZFS file systems on a NFS Server extensively on Solaris, and > a fair bit with FreeBSD 9.x, 10.1, and its never had any such > issues. AFAIK, ZFS on Linux is still considered experimental. > >____ ESET 11570 (20150504) ____ The message was checked by ESET Mail Security.
Jonathan Billings
2015-May-04 13:27 UTC
[CentOS] CentOS 7, NFS 4, and a non ext4 fs (like zfs)
On Mon, May 04, 2015 at 10:09:04AM +0200, tballin wrote:> It really does not seem to be a zfs issue - it does look more like a nfs > problem with certain filessystems. I asked this already on the zfs mailing > list. > > So is there something I can do? > Is the problem known? > Is there a workaround , patch or fix?There's probably a bug somewhere between the Linux Kernel NFS service and the ZFS code. I suspect that it'll be up to the ZFS on Linux folks to debug it, since the Linux Kernel developers will probably be more concerned with making supported filesystems work with NFS. I know I used to have problems back in the RHEL4 days with exporting XFS filesystems via NFS on Linux, and we had to migrate back to ext3. XFS support is much better now, but it's also something that both upstream (RHEL) and the kernel developers maintain. -- Jonathan Billings <billings at negate.org>
Gordon Messmer
2015-May-05 22:10 UTC
[CentOS] CentOS 7, NFS 4, and a non ext4 fs (like zfs)
I'd like to try to help, but your question is very confusing, and appears to present conflicting information. On 05/04/2015 01:09 AM, tballin wrote:> Right now we need to suspend that because we have some serious issues > with NFS4 shared ZFS ( kernel module - zfs on linux project ) Volumes > and CentOS 7 clients. > > Our current server are CentOS6.5, if we share a zfs volume and do the > following on this share:Are you, here, describing the behavior of a CentOS 6.5 server with a ZFS volume? As written, it sounds that way, but the context sounds like you're describing a CentOS 7 system.> client A reads/access file "z.txt". Now client > B replaces (copy, move, unlink+link ) files "z.txt" with a new different > version and now client A reads/access the file "z.txt" ( and only the > file - do not do anything which does a "stat" on the file e.g. "cat > z.txt") the old version/content is read.I can't tell if you're using "cat" as an example of something that does or does not stat the file. It would be helpful if you could describe the specific steps or commands that you are using to test this condition, what outcome you actually see, and what outcome you expected to see instead.> This does not happen with a ext4 share. This also happen with a > centos7.0 NFS Server and ZFS/XFS.In a later message, you said that the problem affects ZFS and XFS both, and appears to be dependent on the NFS version. Please describe the behavior of the following: CentOS 7 server, ZFS volume, NFS 4 export to CentOS 7 client CentOS 7 server, ext4 volume, NFS 4 export to CentOS 7 client CentOS 7 server, ZFS volume, NFS 3 export to CentOS 7 client CentOS 7 server, ext4 volume, NFS 3 export to CentOS 7 client Script the test on the client side if possible to reduce any other variables.> mount/share option - it always happens. Nothing does help. This does not > happen with a Centos7.1 server and a centos7.1 client and xfs (NFS4.1).Earlier you said that it DOES happen with xfs. This is why your request is confusing.