i got 1.2 gb/s on seagate sshd?ST1000LX001 raid 5 x3 (but with the dreaded cache
array on) and 1.1 gb/s on samsung pro ssd 1tb x3 raid5 (no array caching on for
it's not compatible on proliant---not enterprise ssd).
On Thursday, August 4, 2016 5:23 AM, Kaamesh Kamalaaharan <kaamesh at
novocraft.com> wrote:
hi,?thanks for the reply. I have hardware raid 5 ?storage servers with 4TB WD
red drives. I think they are capable of 6GB/s transfers so it shouldnt be a
drive speed issue. Just for testing i tried to do a dd test directy into the
brick mounted from the storage server itself and got around 800mb/s transfer
rate which is double what i get when the brick is mounted on the client. Are
there any other options or tests that i can perform to figure out the root cause
of my problem as i have exhaused most google searches and tests.?
Kaamesh
On Wed, Aug 3, 2016 at 10:58 PM, Leno Vo <lenovolastname at yahoo.com>
wrote:
your 10G nic is capable, the problem is the disk speed, fix ur disk speed first,
use ssd or sshd or sas 15k in a raid 0 or raid 5/6 x4 at least.
On Wednesday, August 3, 2016 2:40 AM, Kaamesh Kamalaaharan <kaamesh at
novocraft.com> wrote:
Hi ,?I have gluster 3.6.2 installed on my server network. Due to internal
issues we are not allowed to upgrade the gluster version. All the clients are on
the same version of gluster. When transferring files ?to/from the clients or
between my nodes over the 10gb network, the transfer rate is capped at 450Mb/s
.Is there any way to increase the transfer speeds for gluster mounts??
Our server setup is as following:
2 gluster servers -gfs1 and gfs2?volume name : gfsvolume3 clients - hpc1,
hpc2,hpc3gluster volume mounted on /export/gfsmount/
The following is the average results what i did so far:
1) test bandwith with iperf between all machines - 9.4 GiB/s2) test write speed
with dd?
dd if=/dev/zero of=/export/gfsmount/testfile bs=1G count=1
result=399Mb/s
3) test read speed with dd
dd if=/export/gfsmount/testfile of=/dev/zero bs=1G count=1
result=284MB/s
My gluster volume configuration:?
Volume Name: gfsvolumeType: ReplicateVolume ID:
a29bd2fb-b1ef-4481-be10-c2f4faf4059bStatus: StartedNumber of Bricks: 1 x 2 =
2Transport-type: tcpBricks:Brick1: gfs1:/export/sda/brickBrick2:
gfs2:/export/sda/brickOptions Reconfigured:performance.quick-read:
offnetwork.ping-timeout: 30network.frame-timeout:
90performance.cache-max-file-size: 2MBcluster.server-quorum-type:
nonenfs.addr-namelookup: offnfs.trusted-write:
offperformance.write-behind-window-size: 4MBcluster.data-self-heal-algorithm:
diffperformance.cache-refresh-timeout: 60performance.cache-size:
1GBcluster.quorum-type: fixedauth.allow: 172.*cluster.quorum-count:
1diagnostics.latency-measurement: ondiagnostics.count-fop-hits:
oncluster.server-quorum-ratio: 50%
Any help would be appreciated. Thanks,Kaamesh
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20160804/9e77ec27/attachment.html>
Kaamesh Kamalaaharan
2016-Aug-05 03:56 UTC
[Gluster-users] Gluster not saturating 10gb network
Hi, I was mistaken about my server specs. my actual specs for each server are 12 x 4.0TB 3.5" LFF NL-SAS 6G, 128MB, 7.2K rpm HDDs (as Data Store set as RAID 6 achieve 36.0TB usable storage) not the WD RED as i mentioned earlier. I guess i should have a higher transfer rate with these drives in. 400 MB/s is a bit too slow in my opinion. Any help i can get will be greatly appreciated as im not sure where i should start debugging this issue On Fri, Aug 5, 2016 at 2:44 AM, Leno Vo <lenovolastname at yahoo.com> wrote:> i got 1.2 gb/s on seagate sshd ST1000LX001 raid 5 x3 (but with the > dreaded cache array on) and 1.1 gb/s on samsung pro ssd 1tb x3 raid5 (no > array caching on for it's not compatible on proliant---not enterprise ssd). > > > On Thursday, August 4, 2016 5:23 AM, Kaamesh Kamalaaharan < > kaamesh at novocraft.com> wrote: > > > hi, > thanks for the reply. I have hardware raid 5 storage servers with 4TB WD > red drives. I think they are capable of 6GB/s transfers so it shouldnt be a > drive speed issue. Just for testing i tried to do a dd test directy into > the brick mounted from the storage server itself and got around 800mb/s > transfer rate which is double what i get when the brick is mounted on the > client. Are there any other options or tests that i can perform to figure > out the root cause of my problem as i have exhaused most google searches > and tests. > > Kaamesh > > On Wed, Aug 3, 2016 at 10:58 PM, Leno Vo <lenovolastname at yahoo.com> wrote: > > your 10G nic is capable, the problem is the disk speed, fix ur disk speed > first, use ssd or sshd or sas 15k in a raid 0 or raid 5/6 x4 at least. > > > On Wednesday, August 3, 2016 2:40 AM, Kaamesh Kamalaaharan < > kaamesh at novocraft.com> wrote: > > > Hi , > I have gluster 3.6.2 installed on my server network. Due to internal > issues we are not allowed to upgrade the gluster version. All the clients > are on the same version of gluster. When transferring files to/from the > clients or between my nodes over the 10gb network, the transfer rate is > capped at 450Mb/s .Is there any way to increase the transfer speeds for > gluster mounts? > > Our server setup is as following: > > 2 gluster servers -gfs1 and gfs2 > volume name : gfsvolume > 3 clients - hpc1, hpc2,hpc3 > gluster volume mounted on /export/gfsmount/ > > > > > The following is the average results what i did so far: > > 1) test bandwith with iperf between all machines - 9.4 GiB/s > 2) test write speed with dd > > dd if=/dev/zero of=/export/gfsmount/testfile bs=1G count=1 > > result=399Mb/s > > > 3) test read speed with dd > > dd if=/export/gfsmount/testfile of=/dev/zero bs=1G count=1 > > > result=284MB/s > > > My gluster volume configuration: > > > Volume Name: gfsvolume > > Type: Replicate > > Volume ID: a29bd2fb-b1ef-4481-be10-c2f4faf4059b > > Status: Started > > Number of Bricks: 1 x 2 = 2 > > Transport-type: tcp > > Bricks: > > Brick1: gfs1:/export/sda/brick > > Brick2: gfs2:/export/sda/brick > > Options Reconfigured: > > performance.quick-read: off > > network.ping-timeout: 30 > > network.frame-timeout: 90 > > performance.cache-max-file-size: 2MB > > cluster.server-quorum-type: none > > nfs.addr-namelookup: off > > nfs.trusted-write: off > > performance.write-behind-window-size: 4MB > > cluster.data-self-heal-algorithm: diff > > performance.cache-refresh-timeout: 60 > > performance.cache-size: 1GB > > cluster.quorum-type: fixed > > auth.allow: 172.* > > cluster.quorum-count: 1 > > diagnostics.latency-measurement: on > > diagnostics.count-fop-hits: on > > cluster.server-quorum-ratio: 50% > > > Any help would be appreciated. > > Thanks, > > Kaamesh > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160805/886d50a4/attachment.html>