freebsd stable - Sep 2010 - Why is NFSv4 so slow?

Hi everyone,

I am experiencing similar issues with newnfs:

1) I have two clients that each get around 0.5MiB/s to 2.6MiB/s reading 
from the NFS4-share on Gbit-Lan

2) Mounting with -t newnfs -o nfsv3 results in no performance gain 
whatsoever.

3) Mounting with -t nfs results in 58MiB/s ! (Netcat has similar 
performance) ? not a hardware/driver issue from my pov

Is there anything I can do to help fix this?

Thank you,
-- 
                         ???????????????????????????????????????????
Best Regards,            ? Free Software Foundation Europe    ??   ?
Hannes Hauswedell        ?           German Team            ?????? ?
                         ? Coordinator for pdfreaders.org     ??   ?
                         ???????????????????????????????????????????

> Hi everyone,
> 
> I am experiencing similar issues with newnfs:
> 
> 1) I have two clients that each get around 0.5MiB/s to 2.6MiB/s
> reading
> from the NFS4-share on Gbit-Lan
> 
> 2) Mounting with -t newnfs -o nfsv3 results in no performance gain
> whatsoever.
> 
> 3) Mounting with -t nfs results in 58MiB/s ! (Netcat has similar
> performance) ? not a hardware/driver issue from my pov
> 
The experimental client does reads in larger MAXBSIZE chunks,
which did cause a similar problem in Mac OS X until
rsize=32768,wsize=32768 was specified. Rick already tried that,
but you might want to try it for your case.
> Is there anything I can do to help fix this?
> 
Ok, so it does sound like an issue in the experimental client and
not NFSv4. For the most part, the read code is the same as
the regular client, but it hasn't been brought up-to-date
with recent changes.

One thing you could try is building a kernel without SMP enabled
and see if that helps? (I only have single core hardware, so I won't
see any SMP races.) If that helps, I can compare the regular vs
experimental client for smp locking in the read stuff.

rick

--On September 12, 2010 11:44:40 -0400 Rick Macklem <rmacklem@uoguelph.ca>
wrote:
>> On Wed, Sep 01, 2010 at 11:46:30AM -0400, Rick Macklem wrote:
	<snip>

My results seems to confirm a factor of two (or 1.5) but it's stable:
new nfs nfsv4
3999969792 bytes transferred in 71.932692 secs (55607119 bytes/sec)
3999969792 bytes transferred in 66.806218 secs (59874214 bytes/sec)
3999969792 bytes transferred in 65.127972 secs (61417079 bytes/sec)
3999969792 bytes transferred in 64.493585 secs (62021204 bytes/sec)

old nfs nfsv3
3999969792 bytes transferred in 42.290365 secs (94583478 bytes/sec)
3999969792 bytes transferred in 42.135682 secs (94930700 bytes/sec)
3999969792 bytes transferred in 41.404841 secs (96606332 bytes/sec)
3999969792 bytes transferred in 41.461210 secs (96474989 bytes/sec)

new nfs nfsv3
3999969792 bytes transferred in 63.172592 secs (63318121 bytes/sec)
3999969792 bytes transferred in 64.149324 secs (62354044 bytes/sec)
3999969792 bytes transferred in 62.447537 secs (64053284 bytes/sec)
3999969792 bytes transferred in 57.203868 secs (69924813 bytes/sec)

Client:
FreeBSD 8.1-STABLE #200: Sun Sep 12 12:03:25 CEST 2010
    root@skade.glz.hidden-powers.com:/usr/obj/usr/src/sys/GENERIC amd64
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 5000+ (2600.26-MHz K8-class 
CPU)
  Origin = "AuthenticAMD"  Id = 0x60fb2  Family = f  Model = 6b 
Stepping =
2

em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
	ether 00:1b:21:2e:7d:3c
	inet 10.255.253.3 netmask 0xffffff00 broadcast 10.255.253.255
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active

Server:
FreeBSD 8.1-STABLE #74: Sun Sep  5 18:47:12 CEST 2010
    root@midgard.glz.hidden-powers.com:/usr/obj/usr/src/sys/SERVER amd64
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Phenom(tm) 9550 Quad-Core Processor (2210.08-MHz K8-class CPU)
  Origin = "AuthenticAMD"  Id = 0x100f23  Family = 10  Model = 2 
Stepping
= 3

re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 
mtu 1500
	options=3898<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC>
	ether 00:1f:d0:59:d8:e2
	inet 10.255.253.1 netmask 0xffffff00 broadcast 10.255.253.255
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active

Network:
Systems connected via two Netgear GS108T, one system to each switch, the 
switches connected via TP cable.

Patchar:
stable-8-v15.patch
zfs_metaslab_v2.patch
zfs_abe_stat_rrwlock.patch
arc.c.9.patch
r211970.patch

Cheers,
	G?ran

---
"There is hopeful symbolism in the fact that flags do not wave in a
vacuum."
                -- Arthur C. Clarke