I'm using an unpatched fuse 2.7.4-1 and glusterfs 2.0.2-1 with the following configs and have this result which surprised me: # dd if=/dev/zero of=foo bs=512k count=1024 1024+0 records in 1024+0 records out 536870912 bytes (537 MB) copied, 14.1538 seconds, 37.9 MB/s # dd if=/dev/zero of=foo bs=512k count=1024 1024+0 records in 1024+0 records out 536870912 bytes (537 MB) copied, 24.4553 seconds, 22.0 MB/s Why is it slower if the file exists? Should it be? ############# Servers ############# volume posix0 type storage/posix option directory /cluster/0 end-volume volume locks0 type features/locks subvolumes posix0 end-volume volume brick0 type performance/io-threads option thread-count 8 subvolumes locks0 end-volume volume posix1 type storage/posix option directory /cluster/1 end-volume volume locks1 type features/locks subvolumes posix1 end-volume volume brick1 type performance/io-threads option thread-count 8 subvolumes locks1 end-volume volume posix2 type storage/posix option directory /cluster/2 end-volume volume locks2 type features/locks subvolumes posix2 end-volume volume brick2 type performance/io-threads option thread-count 8 subvolumes locks2 end-volume volume posix3 type storage/posix option directory /cluster/3 end-volume volume locks3 type features/locks subvolumes posix3 end-volume volume brick3 type performance/io-threads option thread-count 8 subvolumes locks3 end-volume volume server type protocol/server option transport-type tcp subvolumes brick0 brick1 brick2 brick3 option auth.addr.brick0.allow * option auth.addr.brick1.allow * option auth.addr.brick2.allow * option auth.addr.brick3.allow * end-volume ################ Client ################ volume ewcs2_cluster0 type protocol/client option transport-type tcp option remote-host ewcs2.ewcs.com option remote-subvolume brick0 end-volume volume ewcs2_cluster1 type protocol/client option transport-type tcp option remote-host ewcs2.ewcs.com option remote-subvolume brick1 end-volume volume ewcs2_cluster2 type protocol/client option transport-type tcp option remote-host ewcs2.ewcs.com option remote-subvolume brick2 end-volume volume ewcs2_cluster3 type protocol/client option transport-type tcp option remote-host ewcs2.ewcs.com option remote-subvolume brick3 end-volume volume ewcs4_cluster0 type protocol/client option transport-type tcp option remote-host ewcs4.ewcs.com option remote-subvolume brick0 end-volume volume ewcs4_cluster1 type protocol/client option transport-type tcp option remote-host ewcs4.ewcs.com option remote-subvolume brick1 end-volume volume ewcs4_cluster2 type protocol/client option transport-type tcp option remote-host ewcs4.ewcs.com option remote-subvolume brick2 end-volume volume ewcs4_cluster3 type protocol/client option transport-type tcp option remote-host ewcs4.ewcs.com option remote-subvolume brick3 end-volume volume ewcs7_cluster0 type protocol/client option transport-type tcp option remote-host ewcs7.ewcs.com option remote-subvolume brick0 end-volume volume ewcs7_cluster1 type protocol/client option transport-type tcp option remote-host ewcs7.ewcs.com option remote-subvolume brick1 end-volume volume ewcs7_cluster2 type protocol/client option transport-type tcp option remote-host ewcs7.ewcs.com option remote-subvolume brick2 end-volume volume ewcs7_cluster3 type protocol/client option transport-type tcp option remote-host ewcs7.ewcs.com option remote-subvolume brick3 end-volume volume repl1 type cluster/replicate subvolumes ewcs2_cluster0 ewcs4_cluster0 ewcs7_cluster0 end-volume volume repl2 type cluster/replicate subvolumes ewcs2_cluster1 ewcs4_cluster1 ewcs7_cluster1 end-volume volume repl3 type cluster/replicate subvolumes ewcs2_cluster2 ewcs4_cluster2 ewcs7_cluster2 end-volume volume repl4 type cluster/replicate subvolumes ewcs2_cluster3 ewcs4_cluster3 ewcs7_cluster3 end-volume volume distribute type cluster/distribute subvolumes repl1 repl2 repl3 repl4 end-volume volume writebehind type performance/write-behind option aggregate-size 128KB option cache-size 1MB subvolumes distribute end-volume volume ioc type performance/io-cache option cache-size 512MB subvolumes writebehind end-volume ####################### mount -t glusterfs /etc/glusterfs/glusterfs-client.vol /mnt/gluster
I'm using an unpatched fuse 2.7.4-1 and glusterfs 2.0.2-1 with the following configs and have this result which surprised me: # dd if=/dev/zero of=foo bs=512k count=1024 1024+0 records in 1024+0 records out 536870912 bytes (537 MB) copied, 14.1538 seconds, 37.9 MB/s # dd if=/dev/zero of=foo bs=512k count=1024 1024+0 records in 1024+0 records out 536870912 bytes (537 MB) copied, 24.4553 seconds, 22.0 MB/s Why is it slower if the file exists? Should it be? ############# Servers ############# volume posix0 type storage/posix option directory /cluster/0 end-volume volume locks0 type features/locks subvolumes posix0 end-volume volume brick0 type performance/io-threads option thread-count 8 subvolumes locks0 end-volume volume posix1 type storage/posix option directory /cluster/1 end-volume volume locks1 type features/locks subvolumes posix1 end-volume volume brick1 type performance/io-threads option thread-count 8 subvolumes locks1 end-volume volume posix2 type storage/posix option directory /cluster/2 end-volume volume locks2 type features/locks subvolumes posix2 end-volume volume brick2 type performance/io-threads option thread-count 8 subvolumes locks2 end-volume volume posix3 type storage/posix option directory /cluster/3 end-volume volume locks3 type features/locks subvolumes posix3 end-volume volume brick3 type performance/io-threads option thread-count 8 subvolumes locks3 end-volume volume server type protocol/server option transport-type tcp subvolumes brick0 brick1 brick2 brick3 option auth.addr.brick0.allow * option auth.addr.brick1.allow * option auth.addr.brick2.allow * option auth.addr.brick3.allow * end-volume ################ Client ################ volume ewcs2_cluster0 type protocol/client option transport-type tcp option remote-host ewcs2.ewcs.com option remote-subvolume brick0 end-volume volume ewcs2_cluster1 type protocol/client option transport-type tcp option remote-host ewcs2.ewcs.com option remote-subvolume brick1 end-volume volume ewcs2_cluster2 type protocol/client option transport-type tcp option remote-host ewcs2.ewcs.com option remote-subvolume brick2 end-volume volume ewcs2_cluster3 type protocol/client option transport-type tcp option remote-host ewcs2.ewcs.com option remote-subvolume brick3 end-volume volume ewcs4_cluster0 type protocol/client option transport-type tcp option remote-host ewcs4.ewcs.com option remote-subvolume brick0 end-volume volume ewcs4_cluster1 type protocol/client option transport-type tcp option remote-host ewcs4.ewcs.com option remote-subvolume brick1 end-volume volume ewcs4_cluster2 type protocol/client option transport-type tcp option remote-host ewcs4.ewcs.com option remote-subvolume brick2 end-volume volume ewcs4_cluster3 type protocol/client option transport-type tcp option remote-host ewcs4.ewcs.com option remote-subvolume brick3 end-volume volume ewcs7_cluster0 type protocol/client option transport-type tcp option remote-host ewcs7.ewcs.com option remote-subvolume brick0 end-volume volume ewcs7_cluster1 type protocol/client option transport-type tcp option remote-host ewcs7.ewcs.com option remote-subvolume brick1 end-volume volume ewcs7_cluster2 type protocol/client option transport-type tcp option remote-host ewcs7.ewcs.com option remote-subvolume brick2 end-volume volume ewcs7_cluster3 type protocol/client option transport-type tcp option remote-host ewcs7.ewcs.com option remote-subvolume brick3 end-volume volume repl1 type cluster/replicate subvolumes ewcs2_cluster0 ewcs4_cluster0 ewcs7_cluster0 end-volume volume repl2 type cluster/replicate subvolumes ewcs2_cluster1 ewcs4_cluster1 ewcs7_cluster1 end-volume volume repl3 type cluster/replicate subvolumes ewcs2_cluster2 ewcs4_cluster2 ewcs7_cluster2 end-volume volume repl4 type cluster/replicate subvolumes ewcs2_cluster3 ewcs4_cluster3 ewcs7_cluster3 end-volume volume distribute type cluster/distribute subvolumes repl1 repl2 repl3 repl4 end-volume volume writebehind type performance/write-behind option aggregate-size 128KB option cache-size 1MB subvolumes distribute end-volume volume ioc type performance/io-cache option cache-size 512MB subvolumes writebehind end-volume ####################### mount -t glusterfs /etc/glusterfs/glusterfs-client.vol /mnt/gluster