Charles Cooke
2013-Jun-07 13:53 UTC
[Gluster-users] Gluster-users Digest, Vol 62, Issue 17
On Fri, Jun 7, 2013 at 5:00 AM, <gluster-users-request at gluster.org> wrote:> > Message: 2 > Date: Thu, 6 Jun 2013 16:07:08 +0200 > From: Stephan von Krawczynski <skraw at ithnet.com> > To: Pablo <paa.listas at gmail.com> > Cc: gluster-users at gluster.org > Subject: Re: [Gluster-users] GlusterFS (3.3.1) - performance issues - > large number of LOOKUP calls & high CPU usage > Message-ID: <20130606160708.898f5049.skraw at ithnet.com> > Content-Type: text/plain; charset=US-ASCII > > On Thu, 06 Jun 2013 10:39:21 -0300 > Pablo <paa.listas at gmail.com> wrote: > > > I have never try this (In fact I'm just learning a bit more how to > > administer a Gluster server.), buy you may find it useful. > > > > > http://download.gluster.org/pub/gluster/glusterfs/doc/HA%20and%20Load%20Balancing%20for%20NFS%20and%20SMB.html > > > > Pablo. > > The thing with this way of failover is though, that you will likely > corrupt a > currently written file. If your NFS-server (gluster) node dies while you > write > your file will be corrupt. If you use native glusterfs mounts it will not > (should not). This is why I consider the NFS server feature nothing more > than > a bad hack. It does not deliver the safety that glusterfs promises, even if > you solve the failover problem somehow. > > -- > Regards, > Stephan > > > ------------------------------ > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://supercolony.gluster.org/mailman/listinfo/gluster-users > > End of Gluster-users Digest, Vol 62, Issue 17 > ********************************************* >Yes - and in fact, we'd rather NOT use NFS if it can be avoided. As another point of interest, the GlusterFS FUSE native appears to be due to the large number of LOOKUP calls as initially indicated (69000s for FUSE vs. 592 for NFS). We also see a tremendous increase in network overhead using the FUSE client which I suppose is related to the LOOKUP calls (about 300-400k sustained over 60s for FUSE, 30-50k over about 10s for NFS). Any idea as to what would be causing this overhead? ===FUSE==Interval 10 Stats: Block Size: 1b+ 2b+ 4b+ No. of Reads: 0 0 0 No. of Writes: 15 12 20 Block Size: 8b+ 16b+ 32b+ No. of Reads: 0 0 0 No. of Writes: 27 17 59 Block Size: 64b+ 128b+ 256b+ No. of Reads: 1 0 0 No. of Writes: 68 27 24 Block Size: 512b+ 1024b+ 2048b+ No. of Reads: 5 20 5 No. of Writes: 60 0 10 Block Size: 4096b+ No. of Reads: 0 No. of Writes: 30 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 58 FORGET 0.00 0.00 us 0.00 us 0.00 us 671 RELEASE 0.00 0.00 us 0.00 us 0.00 us 114 RELEASEDIR 0.01 31.20 us 21.00 us 62.00 us 10 LK 0.02 38.45 us 32.00 us 56.00 us 31 READ 0.03 210.17 us 187.00 us 271.00 us 6 STATFS 0.03 287.20 us 264.00 us 313.00 us 5 MKDIR 0.11 62.73 us 48.00 us 168.00 us 82 TRUNCATE 0.11 38.25 us 23.00 us 620.00 us 135 STAT 0.13 52.50 us 37.00 us 268.00 us 114 OPENDIR 0.15 156.36 us 129.00 us 247.00 us 45 UNLINK 0.21 123.66 us 51.00 us 2014.00 us 82 SETATTR 0.23 132.70 us 90.00 us 254.00 us 82 RENAME 0.44 56.75 us 33.00 us 1096.00 us 369 WRITE 0.44 45.49 us 24.00 us 3014.00 us 461 FSTAT 0.69 292.22 us 216.00 us 1777.00 us 112 CREATE 0.75 64.18 us 36.00 us 3763.00 us 559 OPEN 0.94 46.40 us 10.00 us 2253.00 us 963 FLUSH 3.55 669.45 us 83.00 us 3746.00 us 253 READDIRP 4.10 2385.21 us 762.00 us 5424.00 us 82 FSYNC 88.09 60.94 us 40.00 us 21474.00 us 69004 LOOKUP Duration: 113 seconds Data Read: 47519 bytes Data Written: 224013 bytes ===NFS==Interval 13 Stats: Block Size: 8b+ 16b+ 64b+ No. of Reads: 0 0 0 No. of Writes: 5 10 11 Block Size: 128b+ 256b+ 512b+ No. of Reads: 0 0 0 No. of Writes: 6 10 23 Block Size: 1024b+ 2048b+ 16384b+ No. of Reads: 0 0 0 No. of Writes: 18 12 5 %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop --------- ----------- ----------- ----------- ------------ ---- 0.00 0.00 us 0.00 us 0.00 us 57 FORGET 0.00 0.00 us 0.00 us 0.00 us 113 RELEASE 0.07 225.00 us 199.00 us 251.00 us 2 STATFS 0.09 58.22 us 43.00 us 137.00 us 9 OPEN 0.12 36.65 us 20.00 us 139.00 us 20 LK 0.24 292.20 us 277.00 us 297.00 us 5 MKDIR 1.01 150.39 us 102.00 us 278.00 us 41 FSTAT 1.09 66.31 us 18.00 us 194.00 us 100 FLUSH 1.36 83.16 us 46.00 us 424.00 us 100 WRITE 1.94 289.24 us 189.00 us 599.00 us 41 READDIRP 2.28 29.13 us 14.00 us 181.00 us 478 ACCESS 2.61 39.76 us 18.00 us 376.00 us 401 FINODELK 2.98 81.78 us 52.00 us 383.00 us 222 SETATTR 2.98 40.78 us 18.00 us 531.00 us 446 INODELK 3.92 322.81 us 95.00 us 3197.00 us 74 RENAME 4.71 143.64 us 83.00 us 303.00 us 200 FXATTROP 4.91 287.68 us 218.00 us 679.00 us 104 CREATE 5.48 54.41 us 16.00 us 10294.00 us 614 ENTRYLK 6.10 35.55 us 23.00 us 299.00 us 1046 STAT 10.15 104.55 us 42.00 us 2412.00 us 592 LOOKUP 13.87 1692.02 us 782.00 us 4956.00 us 50 UNLINK 34.09 196.52 us 91.00 us 14576.00 us 1058 XATTROP Duration: 39 seconds Data Read: 0 bytes Data Written: 222075 bytes -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130607/37a7b4bc/attachment.html>