Gluster users - Jul 2008 - How to boost NFS performance of exported glusterfs?

Hi,

I have set up a standard GlusterFS, glfs, AFR. Now I'm trying to
export my glfs mount point over NFS, but I don't get any performance.

I use:

  - fuse-2.7.3glfs10
  - glusterfs-1.3.10

glfs filesystem is mounted as follows:

/etc/glusterfs/opdata.vol	/glfs glusterfs
direct-io-mode=DISABLE,log-level=CRITICAL,log-file=/var/log/glusterfs.log	0
0


When I make a simple dd test on my glfs mountpoint i get good results
over my busy ethernet:

/glfs/opdata# dd if=/dev/full of=100MB count=100 bs=1M
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 1.25825 seconds, 83.3 MB/s

Then i exports the glfs over NFSv4 with these options:

/glfs/opdata *(rw,async,no_subtree_check,fsid=10,no_root_squash)

I mount the NFS filesystem with the following options. Using the UDP
protocol as recommended when setting up a HA-NFS system.

mount -o remount,udp,rsize=32768,wsize=32768 opdata16:/glfs/opdata /mnt

When running the same test, I had to just copy 10MB - 100MB take ages:

/mnt# dd if=/dev/full of=10MB count=10 bs=1M
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 8.9089 seconds, 1.2 MB/s

When I just export a "normal" filesystem over NFS i get the same
performance as glfs.

I was hoping to use glfs on a HA-NFS setup since I then can have more
than two replica. Is there anyway to tune either glfs or NFS to get
better throughput. Making the glfs clients for about 600-800 nodes
isn't a option since I have various distros spread out on several
locations.

I use balance-alb (6) bonding on my interfaces on the server.

Any help is appreciated.

Arnulf
-- 
"When all else fails, read the instructions." ~ L. Iasellio