Lindsay Mathieson
2015-Nov-14 11:26 UTC
[Gluster-users] File Corruption with shards - 100% reproducable
On 14 November 2015 at 17:30, Krutika Dhananjay <kdhananj at redhat.com> wrote:> You should be able to find a file named group-virt.example under > /etc/glusterfs/ > Copy that as /var/lib/glusterd/virt. >Doesn't seem to exist in the debian jessie apt repo, but I copied it from here: https://raw.githubusercontent.com/gluster/glusterfs/master/extras/group-virt.example And I think it needed to go here: /var/lib/glusterd/virt/group However the good news is, I applied it and seems to have made the difference. Freely migrating my test VM now with no corruption. Will stress test it a bit more. current settings are now: gluster volume info Volume Name: datastore1 Type: Replicate Volume ID: 238fddd0-a88c-4edb-8ac5-ef87c58682bf Status: Started Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: vnb.proxmox.softlog:/mnt/ext4 Brick2: vng.proxmox.softlog:/mnt/ext4 Brick3: vna.proxmox.softlog:/mnt/ext4 Options Reconfigured: cluster.server-quorum-type: server network.remote-dio: enable cluster.eager-lock: enable performance.stat-prefetch: off performance.io-cache: off performance.read-ahead: off performance.quick-read: off cluster.quorum-type: auto performance.readdir-ahead: off features.shard: on features.shard-block-size: 128MB performance.strict-write-ordering: on Thanks for all the help. If you like I can start unsetting setting until we discover which one does the trick. -- Lindsay -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20151114/c0b899f6/attachment.html>
Krutika Dhananjay
2015-Nov-15 03:32 UTC
[Gluster-users] File Corruption with shards - 100% reproducable
Sorry, that was a typo. :( It is supposed to be moved into /var/lib/glusterd/groups/virt. I am suspecting stat-prefetch to be the culprit. So to start with, just disable performance.stat-prefetch and leave the rest of the options as they were before and run the test case. Thank you very much for your help. :) -Krutika ----- Original Message -----> From: "Lindsay Mathieson" <lindsay.mathieson at gmail.com> > To: "Krutika Dhananjay" <kdhananj at redhat.com> > Cc: "gluster-users" <gluster-users at gluster.org> > Sent: Saturday, November 14, 2015 4:56:39 PM > Subject: Re: [Gluster-users] File Corruption with shards - 100% reproducable> On 14 November 2015 at 17:30, Krutika Dhananjay < kdhananj at redhat.com > > wrote:> > You should be able to find a file named group-virt.example under > > /etc/glusterfs/ > > > Copy that as /var/lib/glusterd/virt. >> Doesn't seem to exist in the debian jessie apt repo, but I copied it from > here:> https://raw.githubusercontent.com/gluster/glusterfs/master/extras/group-virt.example> And I think it needed to go here:> /var/lib/glusterd/virt/group> However the good news is, I applied it and seems to have made the difference. > Freely migrating my test VM now with no corruption. Will stress test it a > bit more.> current settings are now: > gluster volume info> Volume Name: datastore1 > Type: Replicate > Volume ID: 238fddd0-a88c-4edb-8ac5-ef87c58682bf > Status: Started > Number of Bricks: 1 x 3 = 3 > Transport-type: tcp > Bricks: > Brick1: vnb.proxmox.softlog:/mnt/ext4 > Brick2: vng.proxmox.softlog:/mnt/ext4 > Brick3: vna.proxmox.softlog:/mnt/ext4 > Options Reconfigured: > cluster.server-quorum-type: server > network.remote-dio: enable > cluster.eager-lock: enable > performance.stat-prefetch: off > performance.io-cache: off > performance.read-ahead: off > performance.quick-read: off > cluster.quorum-type: auto > performance.readdir-ahead: off > features.shard: on > features.shard-block-size: 128MB > performance.strict-write-ordering: on> Thanks for all the help. If you like I can start unsetting setting until we > discover which one does the trick.> -- > Lindsay-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20151114/a33504e6/attachment.html>