Dear all,
I am setting up a big scale GlusterFS with tow servers and about 400
clients.
The problems taht I encountered is every time I log in client, it takes 10
to 30 secs as well as ls command(even not in share dirs).
Could any help to increase the performance? Thanks.
Environment:
Servers:
Two RadHat 5 x86_64 machines.
Each shares 2TB storage making RAID 1 for HA.
Clients:
400 machines with Two RadHat 5 x86_64OS and install fuse-2.6.3-3.slc5.
I have tried this guide, it doesn't work. (
http://www.mail-archive.com/gluster-devel at nongnu.org/msg01985.html )
Files:
In gluster01 already have 1T files and gluster02 not really sync yet.
I have tried close gluster02 to prevent sync latency, but it still takes
about 5 to 10 secs.
Most of files are less than 10M but been read really frequently.
Configs:
gluster01-export.vol
## file auto generated by /usr/bin/glusterfs-volgen (export.vol)
# Cmd line:
# $ /usr/bin/glusterfs-volgen --name gluster --raid 1
gluster01:/opt/exp_soft/ gluster02:/opt/exp_soft/
volume posix1
type storage/posix
option directory /opt/exp_soft/
end-volume
volume locks1
type features/locks
subvolumes posix1
end-volume
volume brick1
type performance/io-threads
option thread-count 16
subvolumes locks1
end-volume
volume server-tcp
type protocol/server
option transport-type tcp
option auth.addr.brick1.allow *
option transport.socket.bind-address MY-IP-1
option transport.socket.listen-port 6996
option transport.socket.nodelay on
subvolumes brick1
end-volume
===================================================================================gluster02-export.vol
gluster01-export.vol
## file auto generated by /usr/bin/glusterfs-volgen (export.vol)
# Cmd line:
# $ /usr/bin/glusterfs-volgen --name gluster --raid 1
gluster01:/opt/exp_soft/ gluster02:/opt/exp_soft/
volume posix1
type storage/posix
option directory /opt/exp_soft/
end-volume
volume locks1
type features/locks
subvolumes posix1
end-volume
volume brick1
type performance/io-threads
option thread-count 16
subvolumes locks1
end-volume
volume server-tcp
type protocol/server
option transport-type tcp
option auth.addr.brick1.allow *
option transport.socket.bind-address MY-IP-2
option transport.socket.listen-port 6996
option transport.socket.nodelay on
subvolumes brick1
end-volume
===================================================================================Clients:
glusterfs.vol
## file auto generated by /usr/bin/glusterfs-volgen (mount.vol)
# Cmd line:
# $ /usr/bin/glusterfs-volgen --name gluster --raid 1
gluster01:/opt/exp_soft/ gluster02:/opt/exp_soft/
# RAID 1
# TRANSPORT-TYPE tcp
volume gluster02-1
type protocol/client
option transport-type tcp
option remote-host gluster02
option transport.socket.nodelay on
option transport.remote-port 6996
option remote-subvolume brick1
end-volume
volume gluster01-1
type protocol/client
option transport-type tcp
option remote-host gluster01
option transport.socket.nodelay on
option transport.remote-port 6996
option remote-subvolume brick1
end-volume
volume mirror-0
type cluster/replicate
subvolumes gluster01-1 gluster02-1
end-volume
volume readahead
type performance/read-ahead
option page-count 4
subvolumes mirror-0
end-volume
volume iocache
type performance/io-cache
option cache-size `echo $(( $(grep 'MemTotal' /proc/meminfo | sed
's/[^0-9]//g') / 5120 ))`MB
option cache-timeout 1
subvolumes readahead
end-volume
volume quickread
type performance/quick-read
option cache-timeout 1
option max-file-size 64kB
subvolumes iocache
end-volume
volume writebehind
type performance/write-behind
option cache-size 4MB
subvolumes quickread
end-volume
volume statprefetch
type performance/stat-prefetch
subvolumes writebehind
end-volume
BR,
Todd