freebsd stable - May 2015 - Status of NFS4.1 FS_RECLAIM in FreeBSD 10.1?

Mahoud Al-Qudsi wrote:
> Hello,
> 
> I have not delved too deeply into either the NFS spec or the FreeBSD
> nfsd
> code, but from my admittedly-limited understanding, it seems that
> reclaim is
> both a mandatory feature and one that is present in the current
> FreeBSD NFS
> v4.1 implementation. Is my understanding of this correct?
> 
Only the global RECLAIM_COMPLETE is implemented. I'll be honest that
I don't even really understand what the "single fs
reclaim_complete"
semantics are and, as such, it isn't implemented.

I think it is meant to be used when a file system is migrated from
one server to another (transferring the locks to the new server) or
something like that.
Migration/replication isn't supported. Maybe someday if I figure out
what the RFC expects the server to do for this case.
> My reason for asking is when attempting to migrate an ESXi server to
> a FreeBSD
> NFSv4.1 datastore, ESXi throws the following error:
> 
> > WARNING: NFS41: NFS41FSCompleteMount:3601: RECLAIM_COMPLETE FS
> > failed: Not
> > supported; forcing read-only operation
> 
This is the first time I've heard of a client using this. The only clients
I've ever had the opportunity to test against are Linux, Solaris and the
FreeBSD one.
> VMware ESXi 6.0 is able to mount NFSv4.1 shares exported from other
> operating systems, so I figured I would ask here on the list before
> digging
> out a copy of tcpdump and going down that rabbit hole.
> 
> I can mount and use NFSv3 shares just fine with ESXi from this same
> server, and
> can mount the same shares as NFSv4 from other clients (e.g. OS X) as
> well.
> 
This is NFSv4.1 specific, so NFSv4.0 should work, I think. Or just use NFSv3.

rick
> Thanks,
> 
> Mahmoud Al-Qudsi
> NeoSmart Technologies
> 
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to
> "freebsd-stable-unsubscribe at freebsd.org"
>

On May 20, 2015, at 8:57 PM, Rick Macklem <rmacklem at uoguelph.ca>
wrote:
> Only the global RECLAIM_COMPLETE is implemented. I'll be honest that
> I don't even really understand what the "single fs
reclaim_complete"
> semantics are and, as such, it isn't implemented.
Thanks for verifying that.
> I think it is meant to be used when a file system is migrated from
> one server to another (transferring the locks to the new server) or
> something like that.
> Migration/replication isn't supported. Maybe someday if I figure out
> what the RFC expects the server to do for this case.
I wasn?t clear on if this was lock reclaiming or block reclaiming. Thanks.
>> I can mount and use NFSv3 shares just fine with ESXi from this same
>> server, and
>> can mount the same shares as NFSv4 from other clients (e.g. OS X) as
>> well.
>> 
> This is NFSv4.1 specific, so NFSv4.0 should work, I think. Or just use
NFSv3.
> 
> rick
For some reason, ESXi doesn?t do ESXi 4.0, only v3 or v4.1.

I am using NFS v3 for now, but unless I?m mistaken, since FreeBSD supports
neither ?nohide? nor ?crossmnt? there is no way for a single export(/import)
to cross ZFS filesystem boundaries. 

I am using ZFS snapshots to manage virtual machine images, each machine
has its own ZFS filesystem so I can snapshot and rollback individually. But 
this means that under NFSv3 (so far as I can tell), each ?folder? (ZFS fs) 
must be mounted separately on the ESXi host. I can get around exporting 
them each individually with the -alldirs parameter, but client-side, there does
not seem to be a way of traversing ZFS filesystem mounts without explicitly
mounting each and every one - a maintenance nightmare if there ever was one.

The only thing I can think of would be unions for the top-level directory, but
I?m
very, very leery of the the nullfs/unionfs modules as they?ve been a source of 
system instability for us in the past (deadlocks, undetected lock inversions,
etc).
That and I really rather a maintenance nightmare than a hack.

Would you have any other suggestions?

Thanks,

Mahmoud