similar to: Rebalance + VM corruption - current status and request for feedback

Hi Mahdi, Did you get a chance to verify this fix again? If this fix works for you, is it OK if we move this bug to CLOSED state and revert the rebalance-cli warning patch? -Krutika On Mon, May 29, 2017 at 6:51 PM, Mahdi Adnan <mahdi.adnan at outlook.com> wrote: > Hello, > > > Yes, i forgot to upgrade the client as well. > > I did the upgrade and created a new volume,

Great news. Is this planned to be published in next release? Il 29 mag 2017 3:27 PM, "Krutika Dhananjay" <kdhananj at redhat.com> ha scritto: > Thanks for that update. Very happy to hear it ran fine without any issues. > :) > > Yeah so you can ignore those 'No such file or directory' errors. They > represent a transient state where DHT in the client process

Great, thanks! Il 5 giu 2017 6:49 AM, "Krutika Dhananjay" <kdhananj at redhat.com> ha scritto: > The fixes are already available in 3.10.2, 3.8.12 and 3.11.0 > > -Krutika > > On Sun, Jun 4, 2017 at 5:30 PM, Gandalf Corvotempesta < > gandalf.corvotempesta at gmail.com> wrote: > >> Great news. >> Is this planned to be published in next

The fixes are already available in 3.10.2, 3.8.12 and 3.11.0 -Krutika On Sun, Jun 4, 2017 at 5:30 PM, Gandalf Corvotempesta < gandalf.corvotempesta at gmail.com> wrote: > Great news. > Is this planned to be published in next release? > > Il 29 mag 2017 3:27 PM, "Krutika Dhananjay" <kdhananj at redhat.com> ha > scritto: > >> Thanks for that update.

Hi, Sorry i did't confirm the results sooner. Yes, it's working fine without issues for me. If anyone else can confirm so we can be sure it's 100% resolved. -- Respectfully Mahdi A. Mahdi ________________________________ From: Krutika Dhananjay <kdhananj at redhat.com> Sent: Tuesday, June 6, 2017 9:17:40 AM To: Mahdi Adnan Cc: gluster-user; Gandalf Corvotempesta; Lindsay

Any additional tests would be great as a similiar bug was detected and fixed some months ago and after that, this bug arose?. Is still unclear to me why two very similiar bug was discovered in two different times for the same operation How this is possible? If you fixed the first bug, why the second one wasn't triggered on your test environment? Il 6 giu 2017 10:35 AM, "Mahdi

Could you disable stat-prefetch on the volume and create another vm off that template and see if it works? -Krutika On Fri, Oct 6, 2017 at 8:28 AM, Lindsay Mathieson < lindsay.mathieson at gmail.com> wrote: > Any chance of a backup you could do bit compare with? > > > > Sent from my Windows 10 phone > > > > *From: *Mahdi Adnan <mahdi.adnan at outlook.com>

OK. Is this problem unique to templates for a particular guest OS type? Or is this something you see for all guest OS? Also, can you get the output of `getfattr -d -m . -e hex <path>` for the following two "paths" from all of the bricks: path to the file representing the vm created off this template wrt the brick. It will usually be $BRICKPATH/xxxx....xx/images/$UUID where $UUID

Hi, Thank you for your reply. Lindsay, Uunfortunately i do not have backup for this template. Krutika, The stat-prefetch is already disabled on the volume. -- Respectfully Mahdi A. Mahdi ________________________________ From: Krutika Dhananjay <kdhananj at redhat.com> Sent: Friday, October 6, 2017 7:39 AM To: Lindsay Mathieson Cc: Mahdi Adnan; gluster-users at gluster.org Subject: Re:

Hi, We're running Gluster 3.8.13 replica 2 (SSDs), it's used as storage domain for oVirt. Today, we found an issue with one of the VMs template, after deploying a VM from this template it will not boot, it stuck at mount the root partition. We've been using this templates for months now and we did not had any issues with it. Both oVirt and Gluster logs is not showing any errors or

Any chance of a backup you could do bit compare with? Sent from my Windows 10 phone From: Mahdi Adnan Sent: Friday, 6 October 2017 12:26 PM To: gluster-users at gluster.org Subject: [Gluster-users] Gluster 3.8.13 data corruption Hi, We're running Gluster 3.8.13 replica 2 (SSDs), it's used as storage domain for oVirt. Today, we found an issue with one of the VMs template, after

On Mon, Jul 10, 2017 at 10:33 PM, Mahdi Adnan <mahdi.adnan at outlook.com> wrote: > I upgraded from 3.8.12 to 3.8.13 without issues. > > Two replicated volumes with online update, upgraded clients first and > followed by servers upgrade, "stop glusterd, pkill gluster*, update > gluster*, start glusterd, monitor healing process and logs, after > completion proceed to

> corruption happens only in this cases: > > - volume with shard enabled > AND > - rebalance operation > I believe so > So, what If I have to replace a failed brick/disks ? Will this trigger > a rebalance and then corruption? > > rebalance, is only needed when you have to expend a volume, ie by > adding more bricks ? That's correct, replacing a brick

On 4 October 2017 at 23:34, WK <wkmail at bneit.com> wrote: > Just so I know. > > Is it correct to assume that this corruption issue is ONLY involved if you > are doing rebalancing with sharding enabled. > > So if I am not doing rebalancing I should be fine? > That is correct. > -bill > > > > On 10/3/2017 10:30 PM, Krutika Dhananjay wrote: > >

Hi, I have an issue with Gluster 3.8.14. The cluster is 4 nodes with replica count 2, on of the nodes went offline for around 15 minutes, when it came back online, self heal triggered and it just did not stop afterward, it's been running for 3 days now, maxing the bricks utilization without actually healing anything. The bricks are all SSDs, and the logs of the source node is spamming with

Hi, is there a recommended way to upgrade Gluster cluster when upgrading to newer revision? I experienced filesystem corruption on several but not all VMs (KVM, FUSE) stored on Gluster during Gluster upgrade. After upgrading one of two nodes, I checked peer status and volume heal info, everything seemed fine so I upgraded second node and then two VMs remounted root as read-only and dmesg

I upgraded from 3.8.12 to 3.8.13 without issues. Two replicated volumes with online update, upgraded clients first and followed by servers upgrade, "stop glusterd, pkill gluster*, update gluster*, start glusterd, monitor healing process and logs, after completion proceed to the other node" check gluster logs for more information. -- Respectfully Mahdi A. Mahdi

Ki Krutika, After that setting: $ dd if=/dev/zero of=/mnt/ddfile bs=1G count=1 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.7351 s, 91.5 MB/s $ dd if=/dev/zero of=/mnt/ddfile2 bs=2G count=1 0+1 records in 0+1 records out 2147479552 bytes (2.1 GB, 2.0 GiB) copied, 23.7351 s, 90.5 MB/s $ dd if=/dev/zero of=/mnt/ddfile3 bs=1G count=1 1+0 records

Hi Krutika, I also did one more test. I re-created another volume (single volume. Old one destroyed-deleted) then do 2 dd tests. One for 1GB other for 2GB. Both are 32MB shard and eager-lock off. Samples: sr:~# gluster volume profile testvol start Starting volume profile on testvol has been successful sr:~# dd if=/dev/zero of=/testvol/dtestfil0xb bs=1G count=1 1+0 records in 1+0

Thanks. I think reusing the same volume was the cause of lack of IO distribution. The latest profile output looks much more realistic and in line with i would expect. Let me analyse the numbers a bit and get back. -Krutika On Tue, Jul 4, 2017 at 12:55 PM, <gencer at gencgiyen.com> wrote: > Hi Krutika, > > > > Thank you so much for myour reply. Let me answer all: > >