freebsd stable - Apr 2021 - FreeBSD NFSv4.1/4.2 server is broken when Linux mounts use "nconnects"

Hi,

The FreeBSD NFS server is broken when handling multiple
connections for NFSv4.1/4.2.
It incorrectly binds the back channel to a new connection
when is sees an RPC with Sequence in it (almost all RPCs)
and might send a callback on the wrong connection.

I have a fix, but it won't be in a release until 12.3/13.1.
It can be found in Bugzilla PR#254560.

In the meantime, avoid using the "nconnects" option
on Linux client mounts.

Thanks go to Ben Coddington @redhat for diagnosing
this and testing the patch.

Sorry about the breakage, rick
ps: It was done intentionally long ago, when clients
      did not do BindConnectionToSession and then slipped
      through the cracks.

On Mon, Mar 29, 2021 at 3:52 PM Rick Macklem <rmacklem at uoguelph.ca>
wrote:
> Hi,
>
> The FreeBSD NFS server is broken when handling multiple
> connections for NFSv4.1/4.2.
> It incorrectly binds the back channel to a new connection
> when is sees an RPC with Sequence in it (almost all RPCs)
> and might send a callback on the wrong connection.
>
> I have a fix, but it won't be in a release until 12.3/13.1.
> It can be found in Bugzilla PR#254560.
>
> In the meantime, avoid using the "nconnects" option
> on Linux client mounts.
>
> Thanks go to Ben Coddington @redhat for diagnosing
> this and testing the patch.
>
> Sorry about the breakage, rick
> ps: It was done intentionally long ago, when clients
>       did not do BindConnectionToSession and then slipped
>       through the cracks.
>
Since the bug involves callbacks, would disabling delegations be a valid
workaround?

On 30/03/2021 8:52 am, Rick Macklem wrote:
> Hi,
> 
> The FreeBSD NFS server is broken when handling multiple
> connections for NFSv4.1/4.2.
Whats the version scope for this issue? If a regression in 12.x 13.x, 
adding to RELNOTES and ERRATA would be great
> It incorrectly binds the back channel to a new connection
> when is sees an RPC with Sequence in it (almost all RPCs)
> and might send a callback on the wrong connection.
> 
> I have a fix, but it won't be in a release until 12.3/13.1.
> It can be found in Bugzilla PR#254560.
> 
> In the meantime, avoid using the "nconnects" option
> on Linux client mounts.
> 
> Thanks go to Ben Coddington @redhat for diagnosing
> this and testing the patch.
> 
> Sorry about the breakage, rick
> ps: It was done intentionally long ago, when clients
>        did not do BindConnectionToSession and then slipped
>        through the cracks.

On Mon, Mar 29, 2021 at 3:52 PM Rick Macklem <rmacklem at uoguelph.ca>
wrote:
> Hi,
>
> The FreeBSD NFS server is broken when handling multiple
> connections for NFSv4.1/4.2.
> It incorrectly binds the back channel to a new connection
> when is sees an RPC with Sequence in it (almost all RPCs)
> and might send a callback on the wrong connection.
>
> I have a fix, but it won't be in a release until 12.3/13.1.
> It can be found in Bugzilla PR#254560.
>
> In the meantime, avoid using the "nconnects" option
> on Linux client mounts.
>
> Thanks go to Ben Coddington @redhat for diagnosing
> this and testing the patch.
>
> Sorry about the breakage, rick
> ps: It was done intentionally long ago, when clients
>       did not do BindConnectionToSession and then slipped
>       through the cracks.
>
I've tested the patch and it works.  Specifically:

* Without the patch and without nconnect, everything works on both 12.2 and
13.0.
* Without the patch, with nconnect, and without delegations, everything
works on 12.2, but on 13.0 clients lock up when they try file I/O.
* Without the patch, with nconnect, and with delegations, clients lock up
on both 12.2 and 13.0.
* With the patch on 13.0, everything works even with nconnect and
delegations.

I haven't tried backporting the patch to 12.2 yet.
-Alan