I've been involved with a project at work doing some fun things with NFS. Recently due to changes in a available hardware we did a complete refresh of the system. New HBAs, new storage boxes, and due to some internal software confusion we updated the OS on the head from 8.1-R with zpool 15 backported to STABLE. Our primary client that we were using against this setup was a ESXi 4.1 machine. In a nutshell, it didn't work. Long description is, ZFS would deadlock and any operation on the pool would hang. The ESXi instance would mark the NFS mount as unavailable. I initially thought this could be due to any number of factors, we have new HBAs in the mix, new storage boxes, a new version of zpool, and one test case. Meanwhile, back at the ranch, I have a somewhat similar setup at home. FreeBSD 8.1 NFS server, ESXi 4.1 box mounting an NFS exported ZFS filesystem from the FreeBSD box as a datastore. Last night I pulled that box up to STABLE, rebooted it, and a minute after it rebooted the ESXi box marked the NFS datastore as unavailable. I checked the FreeBSD machine and sure enough it hung doing an ls on the zpool. I ran a few tests, and as soon as the ESXi box mounts the NFS export it hangs the ZFS filesystem. If I don't mount it up, the NFS server does fine. Thinking it might be a ZFS problem I moved the mount to a UFS filesystem. While this doesn't cause the box to hang on filesystem operations, the mount goes unavailable. The only other client I have on my network is a FreeBSD 8.1 box, and that has no issues All of this is with the standard NFS server, I haven't yet tested with the experimental server. -- Thanks, Josh Paetzel -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 488 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20101123/06cf7ba0/attachment.pgp
On Tue, Nov 23, 2010 at 08:33:52AM -0600, Josh Paetzel wrote:> I've been involved with a project at work doing some fun things with NFS. > Recently due to changes in a available hardware we did a complete refresh of > the system. New HBAs, new storage boxes, and due to some internal software > confusion we updated the OS on the head from 8.1-R with zpool 15 backported to > STABLE. > > Our primary client that we were using against this setup was a ESXi 4.1 > machine. In a nutshell, it didn't work. Long description is, ZFS would > deadlock and any operation on the pool would hang. The ESXi instance would > mark the NFS mount as unavailable. I initially thought this could be due to > any number of factors, we have new HBAs in the mix, new storage boxes, a new > version of zpool, and one test case. > > Meanwhile, back at the ranch, I have a somewhat similar setup at home. > FreeBSD 8.1 NFS server, ESXi 4.1 box mounting an NFS exported ZFS filesystem > from the FreeBSD box as a datastore. > > Last night I pulled that box up to STABLE, rebooted it, and a minute after it > rebooted the ESXi box marked the NFS datastore as unavailable. I checked the > FreeBSD machine and sure enough it hung doing an ls on the zpool. > > I ran a few tests, and as soon as the ESXi box mounts the NFS export it hangs > the ZFS filesystem. If I don't mount it up, the NFS server does fine. > Thinking it might be a ZFS problem I moved the mount to a UFS filesystem. > While this doesn't cause the box to hang on filesystem operations, the mount > goes unavailable. > > The only other client I have on my network is a FreeBSD 8.1 box, and that has > no issues > > All of this is with the standard NFS server, I haven't yet tested with the > experimental server.Sounds relevant, try the patch provided: http://lists.freebsd.org/pipermail/freebsd-stable/2010-November/059853.html Relevant follow-up: http://lists.freebsd.org/pipermail/freebsd-stable/2010-November/059869.html -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |
On Tuesday, November 23, 2010 08:33:52 am Josh Paetzel wrote:> I've been involved with a project at work doing some fun things with NFS. > Recently due to changes in a available hardware we did a complete refresh > of the system. New HBAs, new storage boxes, and due to some internal > software confusion we updated the OS on the head from 8.1-R with zpool 15 > backported to STABLE. > > Our primary client that we were using against this setup was a ESXi 4.1 > machine. In a nutshell, it didn't work. Long description is, ZFS would > deadlock and any operation on the pool would hang. The ESXi instance would > mark the NFS mount as unavailable. I initially thought this could be due > to any number of factors, we have new HBAs in the mix, new storage boxes, > a new version of zpool, and one test case. > > Meanwhile, back at the ranch, I have a somewhat similar setup at home. > FreeBSD 8.1 NFS server, ESXi 4.1 box mounting an NFS exported ZFS > filesystem from the FreeBSD box as a datastore. > > Last night I pulled that box up to STABLE, rebooted it, and a minute after > it rebooted the ESXi box marked the NFS datastore as unavailable. I > checked the FreeBSD machine and sure enough it hung doing an ls on the > zpool. > > I ran a few tests, and as soon as the ESXi box mounts the NFS export it > hangs the ZFS filesystem. If I don't mount it up, the NFS server does > fine. Thinking it might be a ZFS problem I moved the mount to a UFS > filesystem. While this doesn't cause the box to hang on filesystem > operations, the mount goes unavailable. > > The only other client I have on my network is a FreeBSD 8.1 box, and that > has no issues > > All of this is with the standard NFS server, I haven't yet tested with the > experimental server.Thanks to kib for helping resolve this. In a nutshell, at one point there was a problem in STABLE, it was fixed, and my local cvsup mirror fell out of sync and while I thought I was seeing a problem from a system updated last night, I was getting an old rev. Updating the system to a mirror that isn't broken fixed the problem. -- Thanks, Josh Paetzel -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 488 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20101123/1270332e/attachment.pgp