Gluster users - Jun 2014 - Performance optimization of glusterfs with samba-glusterfs-vfs plugin

Hi guys,
I have spent many days on debuging read/write performance of glusterfs with
samba-gluster-vfs plugin recently. The write performance is okay, but the read
speed is always dispirited.
The testing environment is like this:
    OS: centos 6.4
    hosts: two hosts, one for samba server which glusterfs with
samba-glusterfs-vfs runs on it, the other for samba client
    glusterfs version:  3.4.2
    samba-glusterfs-vfs: 
git://forge.gluster.org/samba-glusterfs/samba-glusterfs-vfs.git.
    gluster volume type : Distributed
    smb.conf: 
    large readwrite = yes
    aio read size = 262144
    aio write size = 262144
    aio write behind = true
    ;min receivefile size = 262144
    ;write cache size = 268435456
    read raw = yes
    write raw = yes
    max xmit = 262144
    socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=262144
SO_SNDBUF=262144
    kernel oplocks = no
    stat cache = no
 
    [gluster]
    vfs objects = glusterfs
    glusterfs:volume = test
    glusterfs:volfile_server = localhost
    browseable = yes
    guest ok = yes
    read only = no
    writable = yes
    path = /
    public = yes
    comment = For samba export of volume test
    write list     valid users = a2
Firstly, i mount the shared on client with command like this:
# mount -t cifs -o username=a2 //192.168.102.189/gluster /mnt
And then test read performance with dd command like this:
# dd if=/dev/zero of=/mnt/d1 bs=128k count=10240
the write speed : 350 - 420 MB
# dd if=/mnt/d1 of=/dev/null bs=128k count=10240
the read speed : 19 - 20 MB
The read speed is unbelievably slow.
When i modified the smb.conf to exporting glusterfs fuse mount directory
directly without using samba-glusterfs-vfs plugin, I got the performance results
as follows:
the write speed : 150 - 180 MB
the read speed :  62 - 64MB
In contrast with the result with samba-gluster-vfs, it looks like our plugin
reduced the read speed through CIFS.
Why is there such a situation?  Where is the problem? Any reply will be 
appreciated. Thanks very much!
with best regards!
lou.chen 
 
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140609/26917d21/attachment.html>