Wanted to run some stats by you guy's to see if this is the sort of IO
expected off the GlusterFS w/ oVirt that I have:
[root at mdskvm-p01 master]# iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.0.39, port 59108
[ 5] local 192.168.0.60 port 5201 connected to 192.168.0.39 port 59110
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 108 MBytes 905 Mbits/sec
[ 5] 1.00-2.00 sec 112 MBytes 939 Mbits/sec
[ 5] 2.00-3.00 sec 112 MBytes 939 Mbits/sec
[ 5] 3.00-4.00 sec 112 MBytes 939 Mbits/sec
[ 5] 4.00-5.00 sec 112 MBytes 939 Mbits/sec
[ 5] 5.00-6.00 sec 112 MBytes 938 Mbits/sec
[ 5] 6.00-7.00 sec 112 MBytes 938 Mbits/sec
[ 5] 7.00-8.00 sec 112 MBytes 939 Mbits/sec
[ 5] 8.00-9.00 sec 112 MBytes 940 Mbits/sec
[ 5] 9.00-10.00 sec 112 MBytes 939 Mbits/sec
[ 5] 10.00-10.04 sec 4.25 MBytes 939 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-10.04 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-10.04 sec 1.09 GBytes 935 Mbits/sec
receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.0.39, port 59126
[ 5] local 192.168.0.60 port 5201 connected to 192.168.0.39 port 59128
iperf3: the client has unexpectedly closed the connection
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.0.39, port 59164
[ 6] local 192.168.0.60 port 5201 connected to 192.168.0.39 port 59166
[ ID] Interval Transfer Bandwidth
[ 6] 0.00-1.00 sec 107 MBytes 898 Mbits/sec
[ 6] 1.00-2.00 sec 102 MBytes 858 Mbits/sec
[ 6] 2.00-3.00 sec 107 MBytes 901 Mbits/sec
[ 6] 3.00-4.00 sec 107 MBytes 896 Mbits/sec
[ 6] 4.00-5.00 sec 94.5 MBytes 793 Mbits/sec
[ 6] 5.00-6.00 sec 93.9 MBytes 788 Mbits/sec
[ 6] 6.00-7.00 sec 78.1 MBytes 655 Mbits/sec
[ 6] 7.00-8.00 sec 65.8 MBytes 552 Mbits/sec
[ 6] 8.00-9.00 sec 50.6 MBytes 425 Mbits/sec
[ 6] 9.00-10.00 sec 52.8 MBytes 443 Mbits/sec
[ 6] 10.00-10.07 sec 4.28 MBytes 491 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 6] 0.00-10.07 sec 0.00 Bytes 0.00 bits/sec sender
[ 6] 0.00-10.07 sec 864 MBytes 719 Mbits/sec
receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
^Ciperf3: interrupt - the server has terminated
[root at mdskvm-p01 master]#
Then the client:
[root at mdskvm-p02 master]# iperf3 -c mdskvm-p01.nix.mds.xyz
Connecting to host mdskvm-p01.nix.mds.xyz, port 5201
[ 4] local 192.168.0.39 port 59110 connected to 192.168.0.60 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 114 MBytes 955 Mbits/sec 0 358 KBytes
[ 4] 1.00-2.00 sec 112 MBytes 938 Mbits/sec 0 358 KBytes
[ 4] 2.00-3.00 sec 112 MBytes 939 Mbits/sec 0 362 KBytes
[ 4] 3.00-4.00 sec 111 MBytes 935 Mbits/sec 0 362 KBytes
[ 4] 4.00-5.00 sec 112 MBytes 940 Mbits/sec 0 362 KBytes
[ 4] 5.00-6.00 sec 112 MBytes 942 Mbits/sec 0 362 KBytes
[ 4] 6.00-7.00 sec 111 MBytes 935 Mbits/sec 0 363 KBytes
[ 4] 7.00-8.00 sec 112 MBytes 941 Mbits/sec 0 363 KBytes
[ 4] 8.00-9.00 sec 112 MBytes 937 Mbits/sec 0 363 KBytes
[ 4] 9.00-10.00 sec 112 MBytes 942 Mbits/sec 0 363 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 1.09 GBytes 940 Mbits/sec 0 sender
[ 4] 0.00-10.00 sec 1.09 GBytes 939 Mbits/sec
receiver
iperf Done.
[root at mdskvm-p02 master]#
[root at mdskvm-p02 28fcc382-f469-4f8b-af0f-6b6e048dfcca]# iperf3 -c
mdskvm-p01.nix.mds.xyz -F e13fca65-65c5-4665-8817-9668f81ad6d4
Connecting to host mdskvm-p01.nix.mds.xyz, port 5201
[ 4] local 192.168.0.39 port 59166 connected to 192.168.0.60 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 113 MBytes 945 Mbits/sec 0 373 KBytes
[ 4] 1.00-2.00 sec 102 MBytes 859 Mbits/sec 0 373 KBytes
[ 4] 2.00-3.00 sec 108 MBytes 905 Mbits/sec 0 373 KBytes
[ 4] 3.00-4.00 sec 107 MBytes 896 Mbits/sec 0 373 KBytes
[ 4] 4.00-5.00 sec 94.5 MBytes 793 Mbits/sec 0 373 KBytes
[ 4] 5.00-6.05 sec 88.4 MBytes 710 Mbits/sec 0 373 KBytes
[ 4] 6.05-7.01 sec 78.6 MBytes 687 Mbits/sec 0 373 KBytes
[ 4] 7.01-8.01 sec 68.3 MBytes 573 Mbits/sec 0 373 KBytes
[ 4] 8.01-9.00 sec 52.4 MBytes 442 Mbits/sec 0 373 KBytes
[ 4] 9.00-10.04 sec 52.1 MBytes 422 Mbits/sec 0 379 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.04 sec 864 MBytes 722 Mbits/sec 0 sender
Sent 864 MByte / 7.24 GByte (11%) of
e13fca65-65c5-4665-8817-9668f81ad6d4
[ 4] 0.00-10.04 sec 864 MBytes 722 Mbits/sec
receiver
iperf Done.
[root at mdskvm-p02 28fcc382-f469-4f8b-af0f-6b6e048dfcca]# ls -altri
total 7588869
13832606062331202506 -rw-rw----. 1 vdsm kvm 1048576 Oct 1 07:11
e13fca65-65c5-4665-8817-9668f81ad6d4.lease
9777876818032195061 drwxr-xr-x. 10 vdsm kvm 4096 Oct 1 07:12 ..
9787392456663466582 -rw-rw----. 1 vdsm kvm 7769948160 Oct 1 07:21
e13fca65-65c5-4665-8817-9668f81ad6d4
12966769610620992411 -rw-r--r--. 1 vdsm kvm 357 Oct 1 07:21
e13fca65-65c5-4665-8817-9668f81ad6d4.meta
10878339854193785066 drwxr-xr-x. 2 vdsm kvm 146 Oct 1 07:21 .
[root at mdskvm-p02 28fcc382-f469-4f8b-af0f-6b6e048dfcca]#
A bit about the storage. Cheap LAB hardware presenting a SAN LUN to the
physicals running the GlusterFS. The raw storage itself can deliver
@300MB/s . Network is 1GB/s.
Guessing the numbers are fine given the raw specs?
--
Thx,
TK.