Nithya Balachandran
2017-May-02 15:16 UTC
[Gluster-users] [Gluster-devel] Don't allow data loss via add-brick (was Re: Add single server)
On 2 May 2017 at 16:59, Shyam <srangana at redhat.com> wrote:> Talur, > > Please wait for this fix before releasing 3.10.2. > > We will take in the change to either prevent add-brick in > sharded+distrbuted volumes, or throw a warning and force the use of --force > to execute this. > > IIUC, the problem is less the add brick operation and more therebalance/fix-layout. It is those that need to be prevented (as someone could trigger those without an add-brick). Nithya> Let's get a bug going, and not wait for someone to report it in bugzilla, > and also mark it as blocking 3.10.2 release tracker bug. > > Thanks, > Shyam > > On 05/02/2017 06:20 AM, Pranith Kumar Karampuri wrote: > >> >> >> On Tue, May 2, 2017 at 9:16 AM, Pranith Kumar Karampuri >> <pkarampu at redhat.com <mailto:pkarampu at redhat.com>> wrote: >> >> Yeah it is a good idea. I asked him to raise a bug and we can move >> forward with it. >> >> >> +Raghavendra/Nitya who can help with the fix. >> >> >> >> On Mon, May 1, 2017 at 9:07 PM, Joe Julian <joe at julianfamily.org >> <mailto:joe at julianfamily.org>> wrote: >> >> >> On 04/30/2017 01:13 AM, lemonnierk at ulrar.net >> <mailto:lemonnierk at ulrar.net> wrote: >> >> So I was a little but luck. If I has all the hardware >> part, probably i >> would be firesd after causing data loss by using a >> software marked as stable >> >> Yes, we lost our data last year to this bug, and it wasn't a >> test cluster. >> We still hear from it from our clients to this day. >> >> Is known that this feature is causing data loss and >> there is no evidence or >> no warning in official docs. >> >> I was (I believe) the first one to run into the bug, it >> happens and I knew it >> was a risk when installing gluster. >> But since then I didn't see any warnings anywhere except >> here, I agree >> with you that it should be mentionned in big bold letters on >> the site. >> >> Might even be worth adding a warning directly on the cli >> when trying to >> add bricks if sharding is enabled, to make sure no-one will >> destroy a >> whole cluster for a known bug. >> >> >> I absolutely agree - or, just disable the ability to add-brick >> with sharding enabled. Losing data should never be allowed. >> _______________________________________________ >> Gluster-devel mailing list >> Gluster-devel at gluster.org <mailto:Gluster-devel at gluster.org> >> http://lists.gluster.org/mailman/listinfo/gluster-devel >> <http://lists.gluster.org/mailman/listinfo/gluster-devel> >> >> >> >> >> -- >> Pranith >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org> >> http://lists.gluster.org/mailman/listinfo/gluster-users >> <http://lists.gluster.org/mailman/listinfo/gluster-users> >> >> >> >> >> -- >> Pranith >> >> >> _______________________________________________ >> Gluster-devel mailing list >> Gluster-devel at gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-devel >> >> _______________________________________________ > Gluster-devel mailing list > Gluster-devel at gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-devel >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170502/996afe4a/attachment.html>
Raghavendra Talur
2017-May-03 10:11 UTC
[Gluster-users] [Gluster-devel] Don't allow data loss via add-brick (was Re: Add single server)
On Tue, May 2, 2017 at 8:46 PM, Nithya Balachandran <nbalacha at redhat.com> wrote:> > > On 2 May 2017 at 16:59, Shyam <srangana at redhat.com> wrote: >> >> Talur, >> >> Please wait for this fix before releasing 3.10.2. >> >> We will take in the change to either prevent add-brick in >> sharded+distrbuted volumes, or throw a warning and force the use of --force >> to execute this.Agreed, I have filed bug and marked as blocker for 3.10.2. https://bugzilla.redhat.com/show_bug.cgi?id=1447608>> > IIUC, the problem is less the add brick operation and more the > rebalance/fix-layout. It is those that need to be prevented (as someone > could trigger those without an add-brick).Yes, that problem seems to be with fix-layout/rebalance and not add-brick. However, depending on how users have arranged their dir structure, a add-brick without a fix-layout might be useless for them. I also had a look at the code to see if I can do the cli/glusterd change myself. However, sharding is enabled just as a xlator and not added to glusterd_volinfo_t. If someone from dht team could work with glusterd team here it would fix the issue faster. Action item on Nithya/Atin to assign bug 1447608 to someone. I will wait for the fix for 3.10.2. Thanks, Raghavendra Talur> > Nithya >> >> Let's get a bug going, and not wait for someone to report it in bugzilla, >> and also mark it as blocking 3.10.2 release tracker bug. >> >> Thanks, >> Shyam >> >> On 05/02/2017 06:20 AM, Pranith Kumar Karampuri wrote: >>> >>> >>> >>> On Tue, May 2, 2017 at 9:16 AM, Pranith Kumar Karampuri >>> <pkarampu at redhat.com <mailto:pkarampu at redhat.com>> wrote: >>> >>> Yeah it is a good idea. I asked him to raise a bug and we can move >>> forward with it. >>> >>> >>> +Raghavendra/Nitya who can help with the fix. >>> >>> >>> >>> On Mon, May 1, 2017 at 9:07 PM, Joe Julian <joe at julianfamily.org >>> <mailto:joe at julianfamily.org>> wrote: >>> >>> >>> On 04/30/2017 01:13 AM, lemonnierk at ulrar.net >>> <mailto:lemonnierk at ulrar.net> wrote: >>> >>> So I was a little but luck. If I has all the hardware >>> part, probably i >>> would be firesd after causing data loss by using a >>> software marked as stable >>> >>> Yes, we lost our data last year to this bug, and it wasn't a >>> test cluster. >>> We still hear from it from our clients to this day. >>> >>> Is known that this feature is causing data loss and >>> there is no evidence or >>> no warning in official docs. >>> >>> I was (I believe) the first one to run into the bug, it >>> happens and I knew it >>> was a risk when installing gluster. >>> But since then I didn't see any warnings anywhere except >>> here, I agree >>> with you that it should be mentionned in big bold letters on >>> the site. >>> >>> Might even be worth adding a warning directly on the cli >>> when trying to >>> add bricks if sharding is enabled, to make sure no-one will >>> destroy a >>> whole cluster for a known bug. >>> >>> >>> I absolutely agree - or, just disable the ability to add-brick >>> with sharding enabled. Losing data should never be allowed. >>> _______________________________________________ >>> Gluster-devel mailing list >>> Gluster-devel at gluster.org <mailto:Gluster-devel at gluster.org> >>> http://lists.gluster.org/mailman/listinfo/gluster-devel >>> <http://lists.gluster.org/mailman/listinfo/gluster-devel> >>> >>> >>> >>> >>> -- >>> Pranith >>> >>> _______________________________________________ >>> Gluster-users mailing list >>> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org> >>> http://lists.gluster.org/mailman/listinfo/gluster-users >>> <http://lists.gluster.org/mailman/listinfo/gluster-users> >>> >>> >>> >>> >>> -- >>> Pranith >>> >>> >>> _______________________________________________ >>> Gluster-devel mailing list >>> Gluster-devel at gluster.org >>> http://lists.gluster.org/mailman/listinfo/gluster-devel >>> >> _______________________________________________ >> Gluster-devel mailing list >> Gluster-devel at gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-devel > >