Hey!
Using write-behind translator should improve small writes performance,
but using it in configuration with dht over two afrs shows to me almost
none improvement. When I'm watching networkt traffic (gluster works on
1Gb network) for every send packet there is one recieved:
10:05:20.205346 00:1e:c9:b9:bc:75 (oui Unknown) > 00:1e:c9:b9:bc:39 (oui
Unknown), ethertype IPv4 (0x0800), length 189: n1.1019 > n2.6996: P
138505:138628(123) ack 71706 win 6533 <nop,nop,timestamp 517009620
517010547>
10:05:20.205583 00:1e:c9:b9:bc:39 (oui Unknown) > 00:1e:c9:b9:bc:75 (oui
Unknown), ethertype IPv4 (0x0800), length 170: n2.6996 > n1.1019: P
71706:71810(104) ack 138628 win 3027 <nop,nop,timestamp 517010547
517009620>
10:05:20.205746 00:1e:c9:b9:bc:75 (oui Unknown) > 00:1e:c9:b9:bc:39 (oui
Unknown), ethertype IPv4 (0x0800), length 126: n1.1019 > n2.6996: P
138628:138688(60) ack 71810 win 6533 <nop,nop,timestamp 517009620
517010547>
10:05:20.205801 00:1e:c9:b9:bc:39 (oui Unknown) > 00:1e:c9:b9:bc:75 (oui
Unknown), ethertype IPv4 (0x0800), length 110: n2.6996 > n1.1019: P
71810:71854(44) ack 138688 win 3027 <nop,nop,timestamp 517010547
517009620>
10:05:20.205945 00:1e:c9:b9:bc:75 (oui Unknown) > 00:1e:c9:b9:bc:39 (oui
Unknown), ethertype IPv4 (0x0800), length 158: n1.1019 > n2.6996: P
138688:138780(92) ack 71854 win 6533 <nop,nop,timestamp 517009620
517010547>
10:05:20.205994 00:1e:c9:b9:bc:39 (oui Unknown) > 00:1e:c9:b9:bc:75 (oui
Unknown), ethertype IPv4 (0x0800), length 110: n2.6996 > n1.1019: P
71854:71898(44) ack 138780 win 3027 <nop,nop,timestamp 517010547
517009620>
10:05:20.206145 00:1e:c9:b9:bc:75 (oui Unknown) > 00:1e:c9:b9:bc:39 (oui
Unknown), ethertype IPv4 (0x0800), length 126: n1.1019 > n2.6996: P
138780:138840(60) ack 71898 win 6533 <nop,nop,timestamp 517009620
517010547>
10:05:20.206211 00:1e:c9:b9:bc:39 (oui Unknown) > 00:1e:c9:b9:bc:75 (oui
Unknown), ethertype IPv4 (0x0800), length 110: n2.6996 > n1.1019: P
71898:71942(44) ack 138840 win 3027 <nop,nop,timestamp 517010547
517009620>
10:05:20.207543 00:1e:c9:b9:bc:75 (oui Unknown) > 00:1e:c9:b9:bc:39 (oui
Unknown), ethertype IPv4 (0x0800), length 168: n1.1020 > n2.6996: P
115271:115373(102) ack 61422 win 6161 <nop,nop,timestamp 517009620
517010525>
10:05:20.208136 00:1e:c9:b9:bc:39 (oui Unknown) > 00:1e:c9:b9:bc:75 (oui
Unknown), ethertype IPv4 (0x0800), length 245: n2.6996 > n1.1020: P
61422:61601(179) ack 115373 win 19825 <nop,nop,timestamp 517010547
517009620>
10:05:20.208291 00:1e:c9:b9:bc:75 (oui Unknown) > 00:1e:c9:b9:bc:39 (oui
Unknown), ethertype IPv4 (0x0800), length 66: n1.1020 > n2.6996: . ack 61601
win 6161 <nop,nop,timestamp 517009620 517010547>
In config there is:
volume wb
type performance/write-behind
option block-size 256kB
option cache-size 2MB
option flush-behind on
subvolumes iot
end-volume
So shouldn't write-behind translator agregate these small writes in one
bigger - small packets in larger or larger window?
Regards
--
rash at konto pl
Amar (ಅಮರ್ ತುಂಬಳ್ಳಿ)
2009-Feb-06 16:08 UTC
[Gluster-users] Write-behind translator and network traffic.
Hi Rash, That was a bug with 2.0.0rc1 release, (check https://savannah.nongnu.org/bugs/?25470 ). Fix will be available this week in a release. Regards, Amar 2009/2/6 Rash <rash at konto.pl>> Hey! > > Using write-behind translator should improve small writes performance, > but using it in configuration with dht over two afrs shows to me almost > none improvement. When I'm watching networkt traffic (gluster works on > 1Gb network) for every send packet there is one recieved: >-- Amar Tumballi Gluster/GlusterFS Hacker [bulde on #gluster/irc.gnu.org] http://www.zresearch.com - Commoditizing Super Storage! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20090206/ddb50403/attachment.html>
Cory Meyer
2009-Feb-06 23:14 UTC
[Gluster-users] Write-behind translator and network traffic.
I've got the same config (Client side AFRx2 w/write-behind) that I've been trying to optimize for small file writes and optimization of write-behind is thus very important in my own case. Using DD and Postmark for benchmarking between GlusterFS 1.3.12, 2.0.0rc1 and glusterfs--mainline--3.0--patch-888 though have seen no performance difference with and without AFR and it's tweaks. Here is a link to my test config.. http://gluster.pastebin.com/m46cb6a70 Benchmarking with DD and various block sizes produces a wide range of results.. Optimally a block size of 1MB produces results of 70MB/sec. 4k blocks 4.3 MB/s 8k blocks 8.3 MB/s 16k blocks 15.4 MB/s Is there a better way to tune write-behind since after a month of tinkering I still don't see any performance increase in write-behind? Thanks, Cory On Fri, Feb 6, 2009 at 3:09 AM, Rash <rash at konto.pl> wrote:> Hey! > > Using write-behind translator should improve small writes performance, > but using it in configuration with dht over two afrs shows to me almost > none improvement. When I'm watching networkt traffic (gluster works on > 1Gb network) for every send packet there is one recieved: > > 10:05:20.205346 00:1e:c9:b9:bc:75 (oui Unknown) > 00:1e:c9:b9:bc:39 (oui > Unknown), ethertype IPv4 (0x0800), length 189: n1.1019 > n2.6996: P > 138505:138628(123) ack 71706 win 6533 <nop,nop,timestamp 517009620 > 517010547> > 10:05:20.205583 00:1e:c9:b9:bc:39 (oui Unknown) > 00:1e:c9:b9:bc:75 (oui > Unknown), ethertype IPv4 (0x0800), length 170: n2.6996 > n1.1019: P > 71706:71810(104) ack 138628 win 3027 <nop,nop,timestamp 517010547 517009620> > 10:05:20.205746 00:1e:c9:b9:bc:75 (oui Unknown) > 00:1e:c9:b9:bc:39 (oui > Unknown), ethertype IPv4 (0x0800), length 126: n1.1019 > n2.6996: P > 138628:138688(60) ack 71810 win 6533 <nop,nop,timestamp 517009620 517010547> > 10:05:20.205801 00:1e:c9:b9:bc:39 (oui Unknown) > 00:1e:c9:b9:bc:75 (oui > Unknown), ethertype IPv4 (0x0800), length 110: n2.6996 > n1.1019: P > 71810:71854(44) ack 138688 win 3027 <nop,nop,timestamp 517010547 517009620> > 10:05:20.205945 00:1e:c9:b9:bc:75 (oui Unknown) > 00:1e:c9:b9:bc:39 (oui > Unknown), ethertype IPv4 (0x0800), length 158: n1.1019 > n2.6996: P > 138688:138780(92) ack 71854 win 6533 <nop,nop,timestamp 517009620 517010547> > 10:05:20.205994 00:1e:c9:b9:bc:39 (oui Unknown) > 00:1e:c9:b9:bc:75 (oui > Unknown), ethertype IPv4 (0x0800), length 110: n2.6996 > n1.1019: P > 71854:71898(44) ack 138780 win 3027 <nop,nop,timestamp 517010547 517009620> > 10:05:20.206145 00:1e:c9:b9:bc:75 (oui Unknown) > 00:1e:c9:b9:bc:39 (oui > Unknown), ethertype IPv4 (0x0800), length 126: n1.1019 > n2.6996: P > 138780:138840(60) ack 71898 win 6533 <nop,nop,timestamp 517009620 517010547> > 10:05:20.206211 00:1e:c9:b9:bc:39 (oui Unknown) > 00:1e:c9:b9:bc:75 (oui > Unknown), ethertype IPv4 (0x0800), length 110: n2.6996 > n1.1019: P > 71898:71942(44) ack 138840 win 3027 <nop,nop,timestamp 517010547 517009620> > 10:05:20.207543 00:1e:c9:b9:bc:75 (oui Unknown) > 00:1e:c9:b9:bc:39 (oui > Unknown), ethertype IPv4 (0x0800), length 168: n1.1020 > n2.6996: P > 115271:115373(102) ack 61422 win 6161 <nop,nop,timestamp 517009620 > 517010525> > 10:05:20.208136 00:1e:c9:b9:bc:39 (oui Unknown) > 00:1e:c9:b9:bc:75 (oui > Unknown), ethertype IPv4 (0x0800), length 245: n2.6996 > n1.1020: P > 61422:61601(179) ack 115373 win 19825 <nop,nop,timestamp 517010547 > 517009620> > 10:05:20.208291 00:1e:c9:b9:bc:75 (oui Unknown) > 00:1e:c9:b9:bc:39 (oui > Unknown), ethertype IPv4 (0x0800), length 66: n1.1020 > n2.6996: . ack 61601 > win 6161 <nop,nop,timestamp 517009620 517010547> > > In config there is: > > volume wb > type performance/write-behind > option block-size 256kB > option cache-size 2MB > option flush-behind on > subvolumes iot > end-volume > > So shouldn't write-behind translator agregate these small writes in one > bigger - small packets in larger or larger window? > > Regards > > -- > rash at konto pl > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20090206/1890b3e0/attachment.html>