yang.bin18 at zte.com.cn
2015-Jan-26 03:22 UTC
[Gluster-users] fio testing performance problem
Hi:
environment:
node A :client with fio running
node B :brick
network?1G
Problem is fio with libgfapi getting poor performance than fio whit
fuse,what could be the reason?
[fuse]
/usr/local/bin/fio --filename=/home/gluster/file --direct=1 --rw=randwrite
--bs=32k --size=20G --numjobs=60 --runtime=120 --group_reporting
--name=test-read --ioengine=libaio --iodepth=1
test-read: (g=0): rw=randwrite, bs=32K-32K/32K-32K/32K-32K,
ioengine=libaio, iodepth=1
...
fio-2.2.5
Starting 60 processes
Jobs: 60 (f=60): [w(60)] [100.0% done] [0KB/103.8MB/0KB /s] [0/3318/0
iops] [eta 00m:00s]
test-read: (groupid=0, jobs=60): err= 0: pid=46659: Mon Jan 26 04:40:28
2015
write: io=12804MB, bw=109247KB/s, iops=3413, runt=120016msec
slat (usec): min=46, max=14914, avg=546.08, stdev=1596.17
clat (usec): min=73, max=123085, avg=17017.70, stdev=4057.91
lat (usec): min=208, max=123264, avg=17564.57, stdev=3443.93
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 11], 10.00th=[ 16], 20.00th=[ 17],
| 30.00th=[ 17], 40.00th=[ 17], 50.00th=[ 18], 60.00th=[ 18],
| 70.00th=[ 18], 80.00th=[ 19], 90.00th=[ 19], 95.00th=[ 20],
| 99.00th=[ 35], 99.50th=[ 38], 99.90th=[ 48], 99.95th=[ 56],
| 99.99th=[ 118]
bw (KB /s): min= 1418, max= 2091, per=1.67%, avg=1822.44, stdev=66.16
lat (usec) : 100=0.18%, 250=0.11%, 500=0.04%, 750=0.03%, 1000=0.04%
lat (msec) : 2=0.20%, 4=0.55%, 10=3.41%, 20=92.28%, 50=3.08%
lat (msec) : 100=0.05%, 250=0.03%
cpu : usr=0.11%, sys=0.22%, ctx=955257, majf=0, minf=1696
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
issued : total=r=0/w=409732/d=0, short=r=0/w=0/d=0,
drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=12804MB, aggrb=109247KB/s, minb=109247KB/s, maxb=109247KB/s,
mint=120016msec, maxt=120016msec
[libgfapi]
/usr/local/bin/fio --direct=1 --rw=randwrite --bs=32k --size=20g
--runtime=120 --numjobs=60 --group_reporting --ioengine=gfapi
--volume=gluster_*** --brick=*** --name=/file
/file: (g=0): rw=randwrite, bs=32K-32K/32K-32K/32K-32K, ioengine=gfapi,
iodepth=1
...
fio-2.2.5
Starting 60 processes
Jobs: 42 (f=0):
[w(3),_(5),w(1),_(1),w(1),_(1),w(1),_(1),w(1),_(2),w(4),_(3),w(1),_(1),w(1),_(1),w(1),_(1),w(3),_(1),w(5),_(1),w(20)]
[5.4% done] [0KB/3168KB/0KB /s] [0/99/0 iops] [eta 36m:17s]
/file: (groupid=0, jobs=60): err= 0: pid=46325: Mon Jan 26 04:37:13 2015
write: io=11288MB, bw=96260KB/s, iops=3008, runt=120075msec
clat (usec): min=11, max=488899, avg=19927.46, stdev=15351.15
lat (usec): min=12, max=488901, avg=19928.92, stdev=15351.30
clat percentiles (usec):
| 1.00th=[ 217], 5.00th=[ 1496], 10.00th=[ 2896], 20.00th=[ 6048],
| 30.00th=[10048], 40.00th=[13888], 50.00th=[17792], 60.00th=[21888],
| 70.00th=[27008], 80.00th=[31360], 90.00th=[36096], 95.00th=[46336],
| 99.00th=[66048], 99.50th=[74240], 99.90th=[101888],
99.95th=[126464],
| 99.99th=[189440]
bw (KB /s): min= 101, max=27130, per=1.68%, avg=1615.36,
stdev=900.74
lat (usec) : 20=0.34%, 50=0.19%, 100=0.14%, 250=0.44%, 500=0.65%
lat (usec) : 750=0.77%, 1000=0.87%
lat (msec) : 2=3.49%, 4=6.76%, 10=16.36%, 20=25.76%, 50=40.16%
lat (msec) : 100=3.95%, 250=0.12%, 500=0.01%
cpu : usr=0.39%, sys=0.17%, ctx=362268, majf=0, minf=3608
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
issued : total=r=0/w=361200/d=0, short=r=0/w=0/d=0,
drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=11288MB, aggrb=96259KB/s, minb=96259KB/s, maxb=96259KB/s,
mint=120075msec, maxt=120075msec
--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail (and any
attachment transmitted herewith) is privileged and confidential and is intended
for the exclusive use of the addressee(s). If you are not an intended
recipient, any disclosure, reproduction, distribution or other dissemination or
use of the information contained is strictly prohibited. If you have received
this mail in error, please delete it and notify us immediately.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20150126/cfff8715/attachment.html>