Gluster users - Jul 2016 - New cluster - first experience

2016-07-07 17:12 GMT+02:00 Gandalf Corvotempesta
<gandalf.corvotempesta at gmail.com>:
> I did more test.
> Is not a shard issue, with shard enabled:
>
> # dd if=/dev/zero of=/mnt/glusterfs/zero1 bs=1M count=1000
> 1000+0 record dentro
> 1000+0 record fuori
> 1048576000 byte (1,0 GB) copiati, 93,8803 s, 11,2 MB/s
>
> With shard disabled:
>
> # dd if=/dev/zero of=/mnt/glusterfs/zero2 bs=1M count=1000
> 1000+0 record dentro
> 1000+0 record fuori
> 1048576000 byte (1,0 GB) copiati, 100,051 s, 10,5 MB/s
>
> mostly the same.
Now i'm using a bonded gigabit (2x1GB) on every server but i'm still
stuck at about 15-30mbit/s when extracting the Linux kernel. Total
extraction is still about 10minutes.
Something strange is going on, on a dual gigabit connection (2000mbit)
I'm expecting to reach hundreds of megabit (200-300), not tens (15-20)

# dd if=/dev/zero of=/mnt/glusterfs/zero4 bs=1M count=1000
1000+0 record dentro
1000+0 record fuori
1048576000 byte (1,0 GB) copiati, 88,2981 s, 11,9 MB/s



# gluster volume info gv0

Volume Name: gv0
Type: Replicate
Volume ID: 2a36dc0f-1d9b-469c-82de-9d8d98321b83
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 77.95.175.112:/export/sdb1/brick
Brick2: 77.95.175.113:/export/sdb1/brick
Brick3: 77.95.175.114:/export/sdb1/brick
Options Reconfigured:
performance.cache-size: 2GB
performance.write-behind-window-size: 2GB
features.shard-block-size: 64MB
features.shard: on
transport.address-family: inet
performance.readdir-ahead: on
nfs.disable: on

2016-07-08 10:55 GMT+02:00 Gandalf Corvotempesta
<gandalf.corvotempesta at gmail.com>:
> # gluster volume info gv0
>
> Volume Name: gv0
> Type: Replicate
> Volume ID: 2a36dc0f-1d9b-469c-82de-9d8d98321b83
> Status: Started
> Number of Bricks: 1 x 3 = 3
> Transport-type: tcp
> Bricks:
> Brick1: 77.95.175.112:/export/sdb1/brick
> Brick2: 77.95.175.113:/export/sdb1/brick
> Brick3: 77.95.175.114:/export/sdb1/brick
> Options Reconfigured:
> performance.cache-size: 2GB
> performance.write-behind-window-size: 2GB
> features.shard-block-size: 64MB
> features.shard: on
> transport.address-family: inet
> performance.readdir-ahead: on
> nfs.disable: on

Bond configuration on each server:

# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 200
Down Delay (ms): 200

Slave Interface: eth0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:25:90:c8:a0:6c
Slave queue ID: 0

Slave Interface: eth1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:25:90:c8:a0:6d
Slave queue ID: 0

2016-07-08 10:55 GMT+02:00 Gandalf Corvotempesta
<gandalf.corvotempesta at gmail.com>:
> Now i'm using a bonded gigabit (2x1GB) on every server but i'm
still
> stuck at about 15-30mbit/s when extracting the Linux kernel. Total
> extraction is still about 10minutes.
> Something strange is going on, on a dual gigabit connection (2000mbit)
> I'm expecting to reach hundreds of megabit (200-300), not tens (15-20)
>
> # dd if=/dev/zero of=/mnt/glusterfs/zero4 bs=1M count=1000
> 1000+0 record dentro
> 1000+0 record fuori
> 1048576000 byte (1,0 GB) copiati, 88,2981 s, 11,9 MB/s
No suggestions?