Gluster users - Nov 2014 - Sparse Files and Heal

On Sat, 22 Nov 2014 12:14:57 AM you wrote:
> Lindsay,
> 
> Could you clarify your method of healing the vm-images?
> Did you use full heal, and once the heal completed, you switched to diff
and
> let it finish?
> By the end, both bricks would consist of images with identical file sizes?
There's a story there ...

Initially it was diff, whch wasa problem as I had 8 VM images being healed 
which killed the cpu on the 2 nodes.

I moved the VM's to one node, changed it to full and rebooted. That restored
performance.

But all my VM's are sparse allocated, on at 512GB with only 33GB used. I 
noticed the file being healed had increased in real size to 200GB - obviously 
the heal process wasn't catering to sparse files.

I switched back to diff and that stopped the real file size increase and 
eventually the heal completed (overnight). Unfortunately the real file sizes 
remained different - 512GB allocated on both bricks, but du showed ther 
original was still 33GB whereas the healed one was 200GB.


I somewhat naively tried to force a resync by deleting the healed file on disk 
and starting a heal. Heal reported the file as being healed, but it finished 
much to quickly and the file on node 2 was not there. Disconcertingly, the 
file showed on the gluster mount on node 1, but not on the gluster mount on 
node 2. 


In the end I deleted it and restored from backups.

-- 
Lindsay
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20141122/c91ada6c/attachment.sig>

On 11/22/2014 08:42 AM, Lindsay Mathieson wrote:
> On Sat, 22 Nov 2014 12:14:57 AM you wrote:
>> Lindsay,
>>
>> Could you clarify your method of healing the vm-images?
>> Did you use full heal, and once the heal completed, you switched to
diff and
>> let it finish?
>> By the end, both bricks would consist of images with identical file
sizes?
> There's a story there ...
>
> Initially it was diff, whch wasa problem as I had 8 VM images being healed
> which killed the cpu on the 2 nodes.
>
> I moved the VM's to one node, changed it to full and rebooted. That
restored
> performance.
>
> But all my VM's are sparse allocated, on at 512GB with only 33GB used.
I
> noticed the file being healed had increased in real size to 200GB -
obviously
> the heal process wasn't catering to sparse files.
>
> I switched back to diff and that stopped the real file size increase and
> eventually the heal completed (overnight). Unfortunately the real file
sizes
> remained different - 512GB allocated on both bricks, but du showed ther
> original was still 33GB whereas the healed one was 200GB.
>
>
> I somewhat naively tried to force a resync by deleting the healed file on
disk
> and starting a heal. Heal reported the file as being healed, but it
finished
> much to quickly and the file on node 2 was not there. Disconcertingly, the
> file showed on the gluster mount on node 1, but not on the gluster mount on
> node 2.
Lindsay,
     Was the brick running when you deleted this file? Because as long 
as the brick is running the VM image file would still be open, so the 
healing won't happen properly.

Pranith
>
>
> In the end I deleted it and restored from backups.
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20141122/5669d03d/attachment.html>