Gluster users - Jan 2015 - ... i was able to produce a split brain...

> On 01/27/2015 11:43 PM, Joe Julian wrote:
> > No, there's not. I've been asking for this for years.
> Hey Joe,
>         Vijay and I were just talking about this today. We were
> wondering if you could give us the inputs to make it a feature to
implement.
> Here are the questions I have:
> Basic requirements if I understand correctly are as follows:
> 1) User should be able to fix the split-brain without any intervention
> from admin as the user knows best about the data.
> 2) He should be able to preview some-how about the data before selecting
> the copy which he/she wants to preserve.
One possibility would be to implement something like DHT's
filter_loc_subvol_key, though perhaps using child indices instead of
translator names.  Another would be a script which can manipulate
volfiles and use GFAPI to fetch a specific version of a file.  I've
written several scripts which can do the necessary volfile manipulation.
If we finally have a commitment to do something like this, actually
implementing it will be the easy part.

----- Original Message -----
> From: "Jeff Darcy" <jdarcy at redhat.com>
> To: "Pranith Kumar Karampuri" <pkarampu at redhat.com>
> Cc: "Joe Julian" <joe at julianfamily.org>, gluster-users
at gluster.org, "Anuradha Talur" <atalur at redhat.com>
> Sent: Wednesday, January 28, 2015 9:01:15 PM
> Subject: Re: [Gluster-users] ... i was able to produce a split brain...
> 
> > On 01/27/2015 11:43 PM, Joe Julian wrote:
> > > No, there's not. I've been asking for this for years.
> > Hey Joe,
> >         Vijay and I were just talking about this today. We were
> > wondering if you could give us the inputs to make it a feature to
> > implement.
> > Here are the questions I have:
> > Basic requirements if I understand correctly are as follows:
> > 1) User should be able to fix the split-brain without any intervention
> > from admin as the user knows best about the data.
> > 2) He should be able to preview some-how about the data before
selecting
> > the copy which he/she wants to preserve.
> 
> One possibility would be to implement something like DHT's
> filter_loc_subvol_key, though perhaps using child indices instead of
> translator names.  Another would be a script which can manipulate
> volfiles and use GFAPI to fetch a specific version of a file.  I've
> written several scripts which can do the necessary volfile manipulation.
> If we finally have a commitment to do something like this, actually
> implementing it will be the easy part.
> 
Hello,

Pranith and I had a discussion regarding this issue and here is what we have in
our mind right now.

We plan to provide the user commands to execute from mount so that he can access
the files in split-brain. This way he can choose which copy is to be used as
source. The user will have to perform a set of getfattrs and setfattrs (on
virtual xattrs) to decide which child to choose as source and inform AFR with
his decision.

A) To know the split-brain status :
getfattr -n trusted.afr.split-brain-status <path-to-file>

This will provide user with the following details -
1) Whether the file is in metadata split-brain
2) Whether the file is in data split-brain

It will also list the name of afr-children to choose from. Something like :
Option0: client-0
Option1: client-1

We also tell the user what the user could do to view metadata/data info; like
stat to get metadata etc.

B) Now the user has to choose one of the options (client-x/client-y..) to
inspect the files.
e.g., setfattr -n trusted.afr.split-brain-choice -v client-0
<path-to-file>
We save the read-child info in inode-ctx in order to provide the user access to
the file in split-brain from that child. Once the user inspects the file, he
proceeds to do the same from the other child of replica pair and makes an
informed decision.

C) Once the above steps are done, AFR is to be informed with the final choice
for source. This is achieved by -
(say the fresh copy is in client-0)
e.g., setfattr -n trusted.afr.split-brain-heal-finalize -v client-0
<path-to-file>
This child will be chosen as source and split-brain resolution will be done.

-- 
Thanks,
Anuradha.