Gluster users - Jul 2017 - Web python framework under GlusterFS

View Project Issue [1]






Dear Gluster Users group,
We are looking from feedbacks, advise and production experience running
GlusterFS in cloud infrastructure environment.
*--? Main aim ---*
Using GlusterFS as a mechanism to ensure data application sourcecode replication
in order to run a web based Python Framework
(https://u3749694.ct.sendgrid.net/wf/click?upn=HWc8wtYN823mxBvAUMvLQK-2B-2F-2B2Z7mxVEhKyXGFKSORRR20kAvWIy4EzocF-2BgkT6s_pIB5qNyu3krz5eLOmjwFZihAdiw-2B0VPkJEh9zUhMLa7Pm0My-2BVFmUDLlBqJEqreuhu7EDXCsuFZNiU-2BzNCIU12IYLHgB2MvAyOU8alnbTEyCCfuSS146hN-2FLAgdctvVSYCHZkxfY6rZw5RxLuxMDybvMCdTp6pQvdHN-2FEeFXKu-2Bif6T5ipgJglUftp3lDfqgkDKoj2a5sk48EmZno9uKurh8mhZXLtkILmQ3QqJO-2BZs-3D
*See attached cloud infrastructure diagram for more details.*
As a Cloud provider we are using Google Cloud Platform.
Servers Specs
GlusterFS nodes
3 Nodes
each one with
2 Vcore
4 GB ram
100 SSD external disk (Gluster bricks)

We were able to setup and start a GlusterFS volume:
Volume Name: XXXXXXXXXXX
Type: Replicate
Volume ID:XXXXXXXXXXXXXXXXXXXXX
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: node01:/mnt/disks/brick01/XXXXXXXXXXX
Brick2: node02:/mnt/disks/brick02/XXXXXXXXXXX
Brick3: node03:/mnt/disks/brick03/XXXXXXXXXXX
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
diagnostics.latency-measurement: on
diagnostics.count-fop-hits: on

Then install and mount the glusterFS client on one of the application server.
GlusterFS client/application server spec
4 Vcore
8 GB Ram
We mount the glusterFS volume with the standard glusterfs client.
*--- Main Problem ---*
Access to the web application from the browser is very slow and we think we are
experience really poor performance in case glusterFS have to handle small file
(session file, .py sourcecode file).
We have try to read and understand this
https://u3749694.ct.sendgrid.net/wf/click?upn=wkIqdVoiyobZGWx9D6bWPq5g0xgKD07gXF8jx4NhbAnZdJkG-2FNfFp7w28ZyT4CcJIEWfb9h9M8eq-2Fk3fxKKJXcQ7FK-2FyBFIvKnXZcKrOINeSLkrSW1-2FwR0Qho7kU5jl2_pIB5qNyu3krz5eLOmjwFZihAdiw-2B0VPkJEh9zUhMLa7Pm0My-2BVFmUDLlBqJEqreuhu7EDXCsuFZNiU-2BzNCIU1zvMUJfIa-2BVUFsKnV79r55VhVqwO8zybChNZjxNFzlt5zrS8uoNmGehlbcvdchvejw9LCAZkSwNNqCy7u2oDzxpLweA7R32OEfhT9eiWCudBC5K9lk-2Bgeb65df8trm-2B8KtHcmR6B8XlDYK4r-2BBgNptA-3D
But even with the new settings below the application still very slow.
sudo gluster volume set gvol0 features.cache-invalidation on
sudo gluster volume set gvol0 features.cache-invalidation-timeout 600
sudo gluster volume set gvol0 performance.stat-prefetch on
sudo gluster volume set gvol0 performance.cache-samba-metadata on
sudo gluster volume set gvol0 performance.cache-invalidation on
sudo gluster volume set gvol0 performance.md-cache-timeout 600
sudo gluster volume set gvol0 network.inode-lru-limit 90000
sudo gluster volume set gvol0 cluster.lookup-optimize on

If that helps, *please find attach a txt file that collect the
"profile" data after 2 hours of "data sharing".*
*--- Main questions --- *
Are we facing a network speed performance?
Are we looking into the right direction (small file performance)?
Shall we look to implement NFS-Ganesha to gain more performance?
Any ideas on how we can tune and improved the situation?

If you need anything else in terms of logs or anything else, don't hesitate
to contact us.

Thanks in advance to anyone willing to share their ideas/feedbacks.

Regards,  Marino
Marino Buccolini
IT Development Manager
e:? ??? marino.buccolini at bizitas.com  t: 	 020 8253 4450  w: 	
https://u3749694.ct.sendgrid.net/wf/click?upn=HWc8wtYN823mxBvAUMvLQFSroPq1s9BuppquPUdzlzs-3D_pIB5qNyu3krz5eLOmjwFZihAdiw-2B0VPkJEh9zUhMLa7Pm0My-2BVFmUDLlBqJEqreuhu7EDXCsuFZNiU-2BzNCIU10z4o5jWtcEqgDrThKVA-2BStB1HZgir7eCSovYz2GDSo-2BtFboJlsPLFNFEsiEKGiGtf6Zwzqy12PRMhZZkTJkUxvrGz3bicXSQ4ZK5MnvKU-2FxRio7PfXYlBTy3PU25KpYqxBQvc8st1jgZki0kwy51I4-3D
90-92 Southbridge Road  Croydon
CR0 1AF
United Kingdom

*
*
Bizitas Limited*
Company Reg:?**10145796* ?*

*Think about the environment.. Do you?really need to print this e-mail?*
-----------------------------------------------------------------------------------------------------------------------
*CONFIDENTIALITY - PLEASE NOTE:*  The information contained in this e-mail is
confidential and may be privileged. It is intended for the addressee only. If
you are not the intended recipient, please
delete this e-mail immediately and advise the sender accordingly.
The contents of this e-mail must not be disclosed or copied to any third party
without the?   senders express written authority and by opening it, you hereby
irrevocably agree to this?  stipulation. If you do not agree, then?please inform
the sender immediately.?    We cannot accept any responsibility for viruses, so
please scan all attachments. ?
------------------------------------------------------------------------------------------------------------------------

Sent by
Bizitas Limited
[2]
using
Odoo [3] .



[1]
https://u3749694.ct.sendgrid.net/wf/click?upn=HWc8wtYN823mxBvAUMvLQNK2-2BsgaUEIsC1kkQPl519WdwEnSQH7-2B-2FYiilp-2BsdkZci0Q0AArEKrjoFcz9s9LykA-3D-3D_pIB5qNyu3krz5eLOmjwFZihAdiw-2B0VPkJEh9zUhMLa7Pm0My-2BVFmUDLlBqJEqreuhu7EDXCsuFZNiU-2BzNCIU1-2BJS1D-2Bph-2FYgBGGygFSo8zxlotTiN-2BRXEIULFPWHcXohIBOR5SoIcB3X-2FZTfGaRpG7-2B6PCT0bF6kpMSa8zYxxT6fI13QxH0c51w0g2wu40Luq2vZD1TmjxrNAGKs2Kh2BGaQ7j8l5s50u3yHp5vka18-3D
[2]
https://u3749694.ct.sendgrid.net/wf/click?upn=pPUwSTkBFaToQbWXUg4b80OXmsBmt1jDnhlLfaHsh14-3D_pIB5qNyu3krz5eLOmjwFZihAdiw-2B0VPkJEh9zUhMLa7Pm0My-2BVFmUDLlBqJEqreuhu7EDXCsuFZNiU-2BzNCIU12xyggjO5wu-2BPqLiH1gsvZ68DR75aPBE69OlBjY3baBjlNGa6VU5l4-2BwE9q4Iu7IumjWLqHZ01ym6BATDpGioSNetoYaogGV9yjOl9YQAGCDQAlDICuue3U2qBnapvSfLgJDUwkaG-2FGfC9wqcrvBPII-3D
[3]
https://u3749694.ct.sendgrid.net/wf/click?upn=HWc8wtYN823mxBvAUMvLQCKYuTiHF-2BBO5vy-2Bv1WWwYQ-3D_pIB5qNyu3krz5eLOmjwFZihAdiw-2B0VPkJEh9zUhMLa7Pm0My-2BVFmUDLlBqJEqreuhu7EDXCsuFZNiU-2BzNCIU110u08DRfIiK02VXlDr21EPM0Ro-2B0fqAci5yVVZRUoAsG-2FNggfbMB3Mk1N-2Bae3pcHVwUBF6EjhZmycd5-2BeEvFnIc1RtxKly6VDoTO-2FM2V4JtsX8Ir7MP2gNfuqNw5B751SyyjbLLKeO9LvnBe0Cqels-3D
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170726/271fbc3f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: profile_gluster_log.txt
Type: application/octet-stream
Size: 20851 bytes
Desc: not available
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170726/271fbc3f/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Base GCP Architecture.pdf
Type: application/octet-stream
Size: 85984 bytes
Desc: not available
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170726/271fbc3f/attachment-0001.obj>