I'm having some problems with NFS serving on a FreeBSD 5.4-RELEASE
machine. The FreeBSD machine is the NFS/NIS server for a group of four
Linux clusters. The network archictecture looks like this:
234/24 234/24
Cluster 1 --- |--------------- Cluster 3
| ---------------
em0| File server | fxp0
| --------------
Cluster 2 --- |--------------- Cluster 4
234/24 230/24
em0 and fxp0 are bridged, and em0 has a 234/24 IP address while fxp0 is
just in promiscuous mode. 234/24 is an 802.1q VLAN on the fxp0 side of
the server, so packets are untagged at the switch just before fxp0, and
are forwarded to em0 through the bridge.
The problem manifests itself in large UDP NFS requests from Clusters 3
and 4. The export can be mounted fine from both those clusters, and
small transfers such as with ls work fine, but the moment any serious
data transfer starts, the entire mount just hangs. Running ethereal on
the file server shows a a lot of fragmented packets, and RPC
retransmissions on just a single request. Reducing the read and write
NFS buffers on the Linux clients to 1kB from the default of 4kB solves
the issue, but kills the transfer rate. The moment I go to 2kB, the
problem reappearss. Clusters 1 and 2 use the default of 4kB buffers, and
have no problems communicating to em0.
Poking through the list archives, I ran across this message
(http://lists.freebsd.org/pipermail/freebsd-stable/2003-May/001007.html)
that reveals a bug in the fxp(4) driver in 4-RELEASE that incorrectly
detects the capabilities of the NIC. Is this still an issue in
5-RELEASE, or am I looking at a different problem? Any ideas on how I
can get the NFS buffers up to a reasonable level?
--
-- Skylar Thompson (skylar@cs.earlham.edu)
-- http://www.cs.earlham.edu/~skylar/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 256 bytes
Desc: OpenPGP digital signature
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20050526/eeb45ca2/signature.bin
On 26 May, Skylar Thompson wrote:> I'm having some problems with NFS serving on a FreeBSD 5.4-RELEASE > machine. The FreeBSD machine is the NFS/NIS server for a group of four > Linux clusters. The network archictecture looks like this: > > 234/24 234/24 > Cluster 1 --- |--------------- Cluster 3 > | --------------- > em0| File server | fxp0 > | -------------- > Cluster 2 --- |--------------- Cluster 4 > 234/24 230/24 > > > em0 and fxp0 are bridged, and em0 has a 234/24 IP address while fxp0 is > just in promiscuous mode. 234/24 is an 802.1q VLAN on the fxp0 side of > the server, so packets are untagged at the switch just before fxp0, and > are forwarded to em0 through the bridge. > > The problem manifests itself in large UDP NFS requests from Clusters 3 > and 4. The export can be mounted fine from both those clusters, and > small transfers such as with ls work fine, but the moment any serious > data transfer starts, the entire mount just hangs. Running ethereal on > the file server shows a a lot of fragmented packets, and RPC > retransmissions on just a single request. Reducing the read and write > NFS buffers on the Linux clients to 1kB from the default of 4kB solves > the issue, but kills the transfer rate. The moment I go to 2kB, the > problem reappearss. Clusters 1 and 2 use the default of 4kB buffers, and > have no problems communicating to em0. > > Poking through the list archives, I ran across this message > (http://lists.freebsd.org/pipermail/freebsd-stable/2003-May/001007.html) > that reveals a bug in the fxp(4) driver in 4-RELEASE that incorrectly > detects the capabilities of the NIC. Is this still an issue in > 5-RELEASE, or am I looking at a different problem? Any ideas on how I > can get the NFS buffers up to a reasonable level?That problem was fixed quite some time ago. Which transfer direction fails? Client writing to server Client reading from server Both? Do you see all the fragments in the retransmitted request?
On 26 May, Skylar Thompson wrote:> I'm having some problems with NFS serving on a FreeBSD 5.4-RELEASE > machine. The FreeBSD machine is the NFS/NIS server for a group of four > Linux clusters. The network archictecture looks like this: > > 234/24 234/24 > Cluster 1 --- |--------------- Cluster 3 > | --------------- > em0| File server | fxp0 > | -------------- > Cluster 2 --- |--------------- Cluster 4 > 234/24 230/24 > > > em0 and fxp0 are bridged, and em0 has a 234/24 IP address while fxp0 is > just in promiscuous mode. 234/24 is an 802.1q VLAN on the fxp0 side of > the server, so packets are untagged at the switch just before fxp0, and > are forwarded to em0 through the bridge. > > The problem manifests itself in large UDP NFS requests from Clusters 3 > and 4. The export can be mounted fine from both those clusters, and > small transfers such as with ls work fine, but the moment any serious > data transfer starts, the entire mount just hangs. Running ethereal on > the file server shows a a lot of fragmented packets, and RPC > retransmissions on just a single request. Reducing the read and write > NFS buffers on the Linux clients to 1kB from the default of 4kB solves > the issue, but kills the transfer rate. The moment I go to 2kB, the > problem reappearss. Clusters 1 and 2 use the default of 4kB buffers, and > have no problems communicating to em0. > > Poking through the list archives, I ran across this message > (http://lists.freebsd.org/pipermail/freebsd-stable/2003-May/001007.html) > that reveals a bug in the fxp(4) driver in 4-RELEASE that incorrectly > detects the capabilities of the NIC. Is this still an issue in > 5-RELEASE, or am I looking at a different problem? Any ideas on how I > can get the NFS buffers up to a reasonable level?That problem was fixed quite some time ago. Which transfer direction fails? Client writing to server Client reading from server Both? Do you see all the fragments in the retransmitted request? _______________________________________________ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org"