Gluster users - Jul 2016 - New cluster - first experience

2016-07-07 14:04 GMT+02:00 Lindsay Mathieson <lindsay.mathieson at
gmail.com>:
> The default shard size is 4MB, I'd tend towards a larger one which
improves
> write speed. For my VM cluster I use shardsize = 64MB
>
> nb. To change the shardsize you should recreate the volume.
I did more test.
Is not a shard issue, with shard enabled:

# dd if=/dev/zero of=/mnt/glusterfs/zero1 bs=1M count=1000
1000+0 record dentro
1000+0 record fuori
1048576000 byte (1,0 GB) copiati, 93,8803 s, 11,2 MB/s

With shard disabled:

# dd if=/dev/zero of=/mnt/glusterfs/zero2 bs=1M count=1000
1000+0 record dentro
1000+0 record fuori
1048576000 byte (1,0 GB) copiati, 100,051 s, 10,5 MB/s

mostly the same.



# iperf -c 1.2.3.4
------------------------------------------------------------
Client connecting to 1.2.3.4, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 1.2.3.5 port 34357 connected with 1.2.3.4 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.10 GBytes   943 Mbits/sec

2016-07-07 17:12 GMT+02:00 Gandalf Corvotempesta
<gandalf.corvotempesta at gmail.com>:
> I did more test.
> Is not a shard issue, with shard enabled:
>
> # dd if=/dev/zero of=/mnt/glusterfs/zero1 bs=1M count=1000
> 1000+0 record dentro
> 1000+0 record fuori
> 1048576000 byte (1,0 GB) copiati, 93,8803 s, 11,2 MB/s
>
> With shard disabled:
>
> # dd if=/dev/zero of=/mnt/glusterfs/zero2 bs=1M count=1000
> 1000+0 record dentro
> 1000+0 record fuori
> 1048576000 byte (1,0 GB) copiati, 100,051 s, 10,5 MB/s
>
> mostly the same.
Now i'm using a bonded gigabit (2x1GB) on every server but i'm still
stuck at about 15-30mbit/s when extracting the Linux kernel. Total
extraction is still about 10minutes.
Something strange is going on, on a dual gigabit connection (2000mbit)
I'm expecting to reach hundreds of megabit (200-300), not tens (15-20)

# dd if=/dev/zero of=/mnt/glusterfs/zero4 bs=1M count=1000
1000+0 record dentro
1000+0 record fuori
1048576000 byte (1,0 GB) copiati, 88,2981 s, 11,9 MB/s



# gluster volume info gv0

Volume Name: gv0
Type: Replicate
Volume ID: 2a36dc0f-1d9b-469c-82de-9d8d98321b83
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 77.95.175.112:/export/sdb1/brick
Brick2: 77.95.175.113:/export/sdb1/brick
Brick3: 77.95.175.114:/export/sdb1/brick
Options Reconfigured:
performance.cache-size: 2GB
performance.write-behind-window-size: 2GB
features.shard-block-size: 64MB
features.shard: on
transport.address-family: inet
performance.readdir-ahead: on
nfs.disable: on