Nico Schottelius
2015-Feb-11 23:07 UTC
[Gluster-users] Performance loss from 3.4.2 to 3.6.2
Hello,
switching from 3.4.2 to 3.6.2 reduces our average test performance dramatically.
Our test setup: directly connected 1 GBit/s hosts setup with:
rm -rf /home/gluster/.glusterfs/
rm /home/gluster/*
setfattr -x trusted.glusterfs.volume-id /home/gluster/
setfattr -x trusted.gfid /home/gluster/
gluster volume create xfs-plain replica 2 transport tcp
vmhost1-cluster1:/home/gluster vmhost2-cluster1:/home/gluster
gluster volume start xfs-plain
Afterwards we run our near real world test
mount -t glusterfs vmhost1-cluster1:/xfs-plain /mnt/gluster/
while true; do dd if=redmine-from-ceph-20150204 of=/mnt/gluster/testvm
bs=1M; rm /mnt/gluster/testvm; done
The results are
3.4.2: ~71-72 MiB/s [ubuntu 14.04]
3.6.2: ~59-64 MiB/s [gluster-3.6 ppa]
We have removed 3.6.2 and re-installed 3.4.2 and can consistently reproduce
these numbers over hours of testing.
Is there any configuration change that we need to incorporate to run
3.6.2 faster or is this a known problem?
Cheers,
Nico
--
New PGP key: 659B 0D91 E86E 7E24 FD15 69D0 C729 21A1 293F 2D24
Pranith Kumar Karampuri
2015-Feb-12 07:24 UTC
[Gluster-users] Performance loss from 3.4.2 to 3.6.2
On 02/12/2015 04:37 AM, Nico Schottelius wrote:> Hello, > > switching from 3.4.2 to 3.6.2 reduces our average test performance dramatically. > > Our test setup: directly connected 1 GBit/s hosts setup with: > > rm -rf /home/gluster/.glusterfs/ > rm /home/gluster/* > setfattr -x trusted.glusterfs.volume-id /home/gluster/ > setfattr -x trusted.gfid /home/gluster/ > gluster volume create xfs-plain replica 2 transport tcp vmhost1-cluster1:/home/gluster vmhost2-cluster1:/home/gluster > gluster volume start xfs-plain > > Afterwards we run our near real world test > > mount -t glusterfs vmhost1-cluster1:/xfs-plain /mnt/gluster/ > while true; do dd if=redmine-from-ceph-20150204 of=/mnt/gluster/testvm bs=1M; rm /mnt/gluster/testvm; done > > The results are > > 3.4.2: ~71-72 MiB/s [ubuntu 14.04] > 3.6.2: ~59-64 MiB/s [gluster-3.6 ppa] > > We have removed 3.6.2 and re-installed 3.4.2 and can consistently reproduce > these numbers over hours of testing.Could you repeat the test on 3.6.2 with 'volume set <volname> cluster.ensure-durability off' option and post the results. Pranith> > Is there any configuration change that we need to incorporate to run > 3.6.2 faster or is this a known problem? > > Cheers, > > Nico >
Nico Schottelius
2015-Feb-16 16:48 UTC
[Gluster-users] Performance loss from 3.4.2 to 3.6.2
Hello,
re-tested, however also changed some more parameters than just the
volume option due to reinstall:
Ubuntu 14.04 -> CentOS 7 (*)
Glusterfs 3.6.2 from RPM glusterfs-epel.repo vs. gluster-3.6 ppa
Otherwise the test setup is the same. The results are:
3.6.2-CentOS7-cluster.ensure-durability-on: ~65-72 MiB/s
3.6.2-CentOS7-cluster.ensure-durability-off: ~86-89 MiB/s
So the question from our side is now, what is exactly the threat that
we have when using cluster.ensure-durability-off?
Also I wonder which other tuning parameters you can recommend for
a workload in which we _only_ have VM images (i.e. big files)?
Cheers,
Nico
(*) Using CentOS 7, as we are interested in using libgfapi with qemu.
Nico Schottelius [Thu, Feb 12, 2015 at 12:07:33AM
+0100]:> Hello,
>
> switching from 3.4.2 to 3.6.2 reduces our average test performance
dramatically.
>
> Our test setup: directly connected 1 GBit/s hosts setup with:
>
> rm -rf /home/gluster/.glusterfs/
> rm /home/gluster/*
> setfattr -x trusted.glusterfs.volume-id /home/gluster/
> setfattr -x trusted.gfid /home/gluster/
> gluster volume create xfs-plain replica 2 transport tcp
vmhost1-cluster1:/home/gluster vmhost2-cluster1:/home/gluster
> gluster volume start xfs-plain
>
> Afterwards we run our near real world test
>
> mount -t glusterfs vmhost1-cluster1:/xfs-plain /mnt/gluster/
> while true; do dd if=redmine-from-ceph-20150204 of=/mnt/gluster/testvm
bs=1M; rm /mnt/gluster/testvm; done
>
> The results are
>
> 3.4.2: ~71-72 MiB/s [ubuntu 14.04]
> 3.6.2: ~59-64 MiB/s [gluster-3.6 ppa]
>
> We have removed 3.6.2 and re-installed 3.4.2 and can consistently reproduce
> these numbers over hours of testing.
>
> Is there any configuration change that we need to incorporate to run
> 3.6.2 faster or is this a known problem?
>
> Cheers,
>
> Nico
>
> --
> New PGP key: 659B 0D91 E86E 7E24 FD15 69D0 C729 21A1 293F 2D24
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
--
New PGP key: 659B 0D91 E86E 7E24 FD15 69D0 C729 21A1 293F 2D24