freebsd stable - Dec 2019 - nfs lockd errors after NetApp software upgrade.

> On 18 Dec 2019, at 16:55, Rick Macklem <rmacklem at uoguelph.ca>
wrote:
> 
> Daniel Braniss wrote:
> 
>> Hi,
>> The server with the problems is running FreeBSD 11.1 stable, it was
working fine for >several months,
>> but after a software upgrade of our NetAPP server it?s reporting many
lockd errors >and becomes catatonic,
>> ...
>> Dec 18 13:11:02 moo-09 kernel: nfs server fr-06:/web/www: lockd not
responding
>> Dec 18 13:11:45 moo-09 last message repeated 7 times
>> Dec 18 13:12:55 moo-09 last message repeated 8 times
>> Dec 18 13:13:10 moo-09 kernel: nfs server fr-06:/web/www: lockd is
alive again
>> Dec 18 13:13:10 moo-09 last message repeated 8 times
>> Dec 18 13:13:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0:
Listen queue >overflow: 194 already in queue awaiting acceptance (1
occurrences)
>> Dec 18 13:14:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0:
Listen queue >overflow: 193 already in queue awaiting acceptance (3957
occurrences)
>> Dec 18 13:15:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0:
Listen queue >overflow: 193 already in queue awaiting acceptance ?
> Seems like their software upgrade didn't improve handling of NLM RPCs?
> Appears to be handling RPCs slowly and/or intermittently. Note that no one
> tests it with IPv6, so at least make sure you are still using IPv4 for the
mounts and
> try and make sure IP broadcast works between client and Netapp. I think the
NLM
> and NSM (rpc.statd) still use IP broadcast sometimes.
> 
we are ipv4 - we have our own class c :-)
> Maybe the network guys can suggest more w.r.t. why, but as I've stated
before,
> the NLM is a fundamentally broken protocol which was never published by
Sun,
> so I suggest you avoid using it if at all possible.
well, at the moment the ball is on NetAPP court, and switching to NFSv4 at the
moment is out of the question, it?s
a production server used by several thousand students.
> 
> - If the locks don't need to be seen by other clients, you can just use
the "nolockd"
>   mount option.
> or
> - If locks need to be seen by other clients, try NFSv4 mounts. Netapp
filers
>   should support NFSv4.1, which is a much better protocol that NFSv4.0.
> 
> Good luck with it, rick
thanks
	danny
> ?
> any ideas?
> 
> thanks,
>        danny
> 
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at
freebsd.org"

Hi,
What software version is the NetApp using?
Is the exported volume big?
Is the vserver configured for 64bit identifiers?

If you enable NFS V4.0 or 4.1 other NFS clients using defaults might mount
NFSv4.x unexpectedly after a reboot so you need to watch that.

Cheers

Richard
(NetApp admin)

On Wed, 18 Dec 2019 at 15:46, Daniel Braniss <danny at cs.huji.ac.il>
wrote:
>
>
> > On 18 Dec 2019, at 16:55, Rick Macklem <rmacklem at uoguelph.ca>
wrote:
> >
> > Daniel Braniss wrote:
> >
> >> Hi,
> >> The server with the problems is running FreeBSD 11.1 stable, it
was
> working fine for >several months,
> >> but after a software upgrade of our NetAPP server it?s reporting
many
> lockd errors >and becomes catatonic,
> >> ...
> >> Dec 18 13:11:02 moo-09 kernel: nfs server fr-06:/web/www: lockd
not
> responding
> >> Dec 18 13:11:45 moo-09 last message repeated 7 times
> >> Dec 18 13:12:55 moo-09 last message repeated 8 times
> >> Dec 18 13:13:10 moo-09 kernel: nfs server fr-06:/web/www: lockd is
> alive again
> >> Dec 18 13:13:10 moo-09 last message repeated 8 times
> >> Dec 18 13:13:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0:
> Listen queue >overflow: 194 already in queue awaiting acceptance (1
> occurrences)
> >> Dec 18 13:14:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0:
> Listen queue >overflow: 193 already in queue awaiting acceptance (3957
> occurrences)
> >> Dec 18 13:15:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0:
> Listen queue >overflow: 193 already in queue awaiting acceptance ?
> > Seems like their software upgrade didn't improve handling of NLM
RPCs?
> > Appears to be handling RPCs slowly and/or intermittently. Note that no
> one
> > tests it with IPv6, so at least make sure you are still using IPv4 for
> the mounts and
> > try and make sure IP broadcast works between client and Netapp. I
think
> the NLM
> > and NSM (rpc.statd) still use IP broadcast sometimes.
> >
> we are ipv4 - we have our own class c :-)
> > Maybe the network guys can suggest more w.r.t. why, but as I've
stated
> before,
> > the NLM is a fundamentally broken protocol which was never published
by
> Sun,
> > so I suggest you avoid using it if at all possible.
> well, at the moment the ball is on NetAPP court, and switching to NFSv4 at
> the moment is out of the question, it?s
> a production server used by several thousand students.
>
> >
> > - If the locks don't need to be seen by other clients, you can
just use
> the "nolockd"
> >   mount option.
> > or
> > - If locks need to be seen by other clients, try NFSv4 mounts. Netapp
> filers
> >   should support NFSv4.1, which is a much better protocol that
NFSv4.0.
> >
> > Good luck with it, rick
> thanks
>         danny
>
> > ?
> > any ideas?
> >
> > thanks,
> >        danny
> >
> > _______________________________________________
> > freebsd-stable at freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> > To unsubscribe, send any mail to "freebsd-stable-unsubscribe at
freebsd.org
> "
>
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at
freebsd.org"
>