Gluster users - Sep 2021 - Recovering from remove-brick where shards did not rebalance

The data should be recoverable by concatenating the main file with all shards.
Then you can copy the data back via the FUSE mount point.
I think that some users reported that add-brick with the force option allows to
'undo' the situation and 're-add' the data, but I have never
tried that and I cannot guarantee that it will even work.
The simplest way is to recover from a recent backup , but sometimes this leads
to a data loss.
Best Regards,Strahil Nikolov
 
 
  On Tue, Sep 7, 2021 at 9:29, Anthony Hoppe<anthony at vofr.net> wrote:  
Hello,

I did a bad thing and did a remove-brick on a set of bricks in a
distributed-replicate volume where rebalancing did not successfully rebalance
all files.? In sleuthing around the various bricks on the 3 node pool, it
appears that a number of the files within the volume may have been stored as
shards.? With that, I'm unsure how to proceed with recovery.

Is it possible to re-add the removed bricks somehow and then do a heal?? Or is
there a way to recover data from shards somehow?

Thanks!
________



Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users at gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users
  
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20210907/3215c959/attachment.html>

I was contemplating these options, actually, but not finding anything in my
research showing someone had tried either before gave me pause.

One thing I wasn't sure about when doing a force add-brick was if gluster
would wipe the existing data from the added bricks. Sounds like that may not be
the case?

With regards to concatenating the main file + shards, how would I go about
identifying the shards that pair with the main file? I see the shards have
sequence numbers, but I'm not sure how to match the identifier to the main
file.

Thanks!! 
> From: "Strahil Nikolov" <hunter86_bg at yahoo.com>
> To: "anthony" <anthony at vofr.net>,
"gluster-users" <gluster-users at gluster.org>
> Sent: Tuesday, September 7, 2021 6:02:36 AM
> Subject: Re: [Gluster-users] Recovering from remove-brick where shards did
not
> rebalance
> The data should be recoverable by concatenating the main file with all
shards.
> Then you can copy the data back via the FUSE mount point.
> I think that some users reported that add-brick with the force option
allows to
> 'undo' the situation and 're-add' the data, but I have
never tried that and I
> cannot guarantee that it will even work.
> The simplest way is to recover from a recent backup , but sometimes this
leads
> to a data loss.
> Best Regards,
> Strahil Nikolov
>> On Tue, Sep 7, 2021 at 9:29, Anthony Hoppe
>> <anthony at vofr.net> wrote:
>> Hello,
>> I did a bad thing and did a remove-brick on a set of bricks in a
>> distributed-replicate volume where rebalancing did not successfully
rebalance
>> all files. In sleuthing around the various bricks on the 3 node pool,
it
>> appears that a number of the files within the volume may have been
stored as
>> shards. With that, I'm unsure how to proceed with recovery.
>> Is it possible to re-add the removed bricks somehow and then do a heal?
Or is
>> there a way to recover data from shards somehow?
>> Thanks!
>> ________
>> Community Meeting Calendar:
>> Schedule -
>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>> Bridge: [ https://meet.google.com/cpu-eiue-hvk |
>> https://meet.google.com/cpu-eiue-hvk ]
>> Gluster-users mailing list
>> [ mailto:Gluster-users at gluster.org | Gluster-users at gluster.org ]
>> [ https://lists.gluster.org/mailman/listinfo/gluster-users |
>> https://lists.gluster.org/mailman/listinfo/gluster-users ]
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20210907/5ff5bc8b/attachment.html>