On Tue, 18 Aug 2020 at 18:50, Yaniv Kaul <ykaul at redhat.com> wrote:> > > > On Tue, Aug 18, 2020 at 3:57 PM Gilberto Nunes <gilberto.nunes32 at gmail.com> wrote: >> >> Hi friends... >> >> I have a 2-nodes GlusterFS, with has the follow configuration: >> gluster vol info >>I'd be interested in the chosen configuration for this deployment - the 2 node set up. Was there a specific requirement which led to this?>> Volume Name: VMS >> Type: Replicate >> Volume ID: a4ec9cfb-1bba-405c-b249-8bd5467e0b91 >> Status: Started >> Snapshot Count: 0 >> Number of Bricks: 1 x 2 = 2 >> Transport-type: tcp >> Bricks: >> Brick1: server02:/DATA/vms >> Brick2: server01:/DATA/vms >> Options Reconfigured: >> performance.read-ahead: off >> performance.io-cache: on >> performance.cache-refresh-timeout: 1 >> performance.cache-size: 1073741824 >> performance.io-thread-count: 64 >> performance.write-behind-window-size: 64MB >> cluster.granular-entry-heal: enable >> cluster.self-heal-daemon: enable >> performance.client-io-threads: on >> cluster.data-self-heal-algorithm: full >> cluster.favorite-child-policy: mtime >> network.ping-timeout: 2 >> cluster.quorum-count: 1 >> cluster.quorum-reads: false >> cluster.heal-timeout: 20 >> storage.fips-mode-rchecksum: on >> transport.address-family: inet >> nfs.disable: on >> >> HDDs are SSD and SAS >> Network connections between the servers are dedicated 1GB (no switch!). > > > You can't get good performance on 1Gb. >> >> Files are 500G 200G 200G 250G 200G 100G size each. >> >> Performance so far so good is ok... > > > What's your workload? Read? Write? sequential? random? many files? > With more bricks and nodes, you should probably use sharding. > > What are your expectations, btw? > Y. > >> >> Any other advice which could point me, let me know! >> >> Thanks >> >> >> >> --- >> Gilberto Nunes Ferreira >> >> ________ >> >> >> >> Community Meeting Calendar: >> >> Schedule - >> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC >> Bridge: https://bluejeans.com/441850968 >> >> Gluster-users mailing list >> Gluster-users at gluster.org >> https://lists.gluster.org/mailman/listinfo/gluster-users > > ________ > > > > Community Meeting Calendar: > > Schedule - > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC > Bridge: https://bluejeans.com/441850968 > > Gluster-users mailing list > Gluster-users at gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users-- sankarshan mukhopadhyay <https://about.me/sankarshan.mukhopadhyay>
Gilberto Nunes
2020-Aug-18 13:47 UTC
[Gluster-users] GlusterFS performance for big files...
>> What's your workload?I have 6 KVM VMs which have Windows and Linux installed on it.>> Read? >> Write?iostat (I am using sdc as the main storage) cavg-cpu: %user %nice %system %iowait %steal %idle 9.15 0.00 1.25 1.38 0.00 88.22 Device r/s w/s rkB/s wkB/s rrqm/s wrqm/s %rrqm %wrqm r_await w_await aqu-sz rareq-sz wareq-sz svctm %util sdc 0.00 1.00 0.00 1.50 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.50>> sequential? random?sequential>> many files?6 files 500G 200G 200G 250G 200G 100G size each. With more bricks and nodes, you should probably use sharding. For now I have only two bricks/nodes.... Plan for more is now out of the question! What are your expectations, btw? I ran many environments with Proxmox Virtual Environment, which use QEMU (not virt) and LXC...But I use majority KVM (QEMU) virtual machines. My goal is to use glusterfs since I think it's more resource demanding such as memory and cpu and nic, when compared to ZFS or CEPH. --- Gilberto Nunes Ferreira (47) 3025-5907 (47) 99676-7530 - Whatsapp / Telegram Skype: gilberto.nunes36 Em ter., 18 de ago. de 2020 ?s 10:29, sankarshan < sankarshan.mukhopadhyay at gmail.com> escreveu:> On Tue, 18 Aug 2020 at 18:50, Yaniv Kaul <ykaul at redhat.com> wrote: > > > > > > > > On Tue, Aug 18, 2020 at 3:57 PM Gilberto Nunes < > gilberto.nunes32 at gmail.com> wrote: > >> > >> Hi friends... > >> > >> I have a 2-nodes GlusterFS, with has the follow configuration: > >> gluster vol info > >> > > I'd be interested in the chosen configuration for this deployment - > the 2 node set up. Was there a specific requirement which led to this? > > >> Volume Name: VMS > >> Type: Replicate > >> Volume ID: a4ec9cfb-1bba-405c-b249-8bd5467e0b91 > >> Status: Started > >> Snapshot Count: 0 > >> Number of Bricks: 1 x 2 = 2 > >> Transport-type: tcp > >> Bricks: > >> Brick1: server02:/DATA/vms > >> Brick2: server01:/DATA/vms > >> Options Reconfigured: > >> performance.read-ahead: off > >> performance.io-cache: on > >> performance.cache-refresh-timeout: 1 > >> performance.cache-size: 1073741824 > >> performance.io-thread-count: 64 > >> performance.write-behind-window-size: 64MB > >> cluster.granular-entry-heal: enable > >> cluster.self-heal-daemon: enable > >> performance.client-io-threads: on > >> cluster.data-self-heal-algorithm: full > >> cluster.favorite-child-policy: mtime > >> network.ping-timeout: 2 > >> cluster.quorum-count: 1 > >> cluster.quorum-reads: false > >> cluster.heal-timeout: 20 > >> storage.fips-mode-rchecksum: on > >> transport.address-family: inet > >> nfs.disable: on > >> > >> HDDs are SSD and SAS > >> Network connections between the servers are dedicated 1GB (no switch!). > > > > > > You can't get good performance on 1Gb. > >> > >> Files are 500G 200G 200G 250G 200G 100G size each. > >> > >> Performance so far so good is ok... > > > > > > What's your workload? Read? Write? sequential? random? many files? > > With more bricks and nodes, you should probably use sharding. > > > > What are your expectations, btw? > > Y. > > > >> > >> Any other advice which could point me, let me know! > >> > >> Thanks > >> > >> > >> > >> --- > >> Gilberto Nunes Ferreira > >> > >> ________ > >> > >> > >> > >> Community Meeting Calendar: > >> > >> Schedule - > >> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC > >> Bridge: https://bluejeans.com/441850968 > >> > >> Gluster-users mailing list > >> Gluster-users at gluster.org > >> https://lists.gluster.org/mailman/listinfo/gluster-users > > > > ________ > > > > > > > > Community Meeting Calendar: > > > > Schedule - > > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC > > Bridge: https://bluejeans.com/441850968 > > > > Gluster-users mailing list > > Gluster-users at gluster.org > > https://lists.gluster.org/mailman/listinfo/gluster-users > > > > -- > sankarshan mukhopadhyay > <https://about.me/sankarshan.mukhopadhyay> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20200818/db913bd6/attachment.html>