Pranith Kumar Karampuri
2016-Jul-13 04:43 UTC
[Gluster-users] 3.7.13, index healing broken?
On Wed, Jul 13, 2016 at 9:41 AM, Dmitry Melekhov <dm at belkam.com> wrote:> 13.07.2016 07:46, Pranith Kumar Karampuri ?????: > > > > On Tue, Jul 12, 2016 at 9:27 PM, Dmitry Melekhov < <dm at belkam.com> > dm at belkam.com> wrote: > >> >> >> 12.07.2016 17:39, Pranith Kumar Karampuri ?????: >> >> Wow, what are the steps to recreate the problem? >> >> >> just set file length to zero, always reproducible. >> > > Changing things on the brick i.e. not from gluster volume mount is not > something you want to do. In the worst case(I have seen this only once in > the last 5 years though) where you do this it can lead to data loss also. > So please be aware of it. > > > Data replication with gluster is a way to avoid data loss, right? Or no? > If not- why use gluster then? > I though that gluster self-healing will heal or at least report missed > files or files with wrong lenths- i.e. corruptions visible just by reading > brick's directory, > not comparing data as bit rot detection... > If this is not a bug, then gluster is not what I expected :-( >Yes Data replication with gluster is a way to avoid data loss. Changing files directly on the brick is similar to changing internal data structures of a disk filesystem. Or changing the internal files of a database. Things may stop working as you expect it to. All the hardwork done by the stack is nullified if you fiddle with the data on the brick directly. To put it succinctly you enter into the area of undefined behavior if you start fiddling with the data on the brick directly. Unless it is a documented behaviour I suggest you don't do it.> > Thank you! > > > >> >> >> >> On Tue, Jul 12, 2016 at 3:09 PM, Dmitry Melekhov < <dm at belkam.com> >> dm at belkam.com> wrote: >> >>> 12.07.2016 13:33, Pranith Kumar Karampuri ?????: >>> >>> What was "gluster volume heal <volname> info" showing when you saw this >>> issue? >>> >>> >>> just reproduced : >>> >>> >>> [root at father brick]# > gstatus-0.64-3.el7.x86_64.rpm >>> >>> [root at father brick]# gluster volume heal pool >>> Launching heal operation to perform index self heal on volume pool has >>> been successful >>> Use heal info commands to check status >>> [root at father brick]# gluster volume heal pool info >>> Brick father:/wall/pool/brick >>> Status: Connected >>> Number of entries: 0 >>> >>> Brick son:/wall/pool/brick >>> Status: Connected >>> Number of entries: 0 >>> >>> Brick spirit:/wall/pool/brick >>> Status: Connected >>> Number of entries: 0 >>> >>> [root at father brick]# >>> >>> >>> >>> >>> On Mon, Jul 11, 2016 at 3:28 PM, Dmitry Melekhov < <dm at belkam.com> >>> dm at belkam.com> wrote: >>> >>>> Hello! >>>> >>>> 3.7.13, 3 bricks volume. >>>> >>>> inside one of bricks: >>>> >>>> [root at father brick]# ls -l gstatus-0.64-3.el7.x86_64.rpm >>>> -rw-r--r-- 2 root root 52268 ??? 11 13:00 gstatus-0.64-3.el7.x86_64.rpm >>>> [root at father brick]# >>>> >>>> >>>> [root at father brick]# > gstatus-0.64-3.el7.x86_64.rpm >>>> [root at father brick]# ls -l gstatus-0.64-3.el7.x86_64.rpm >>>> -rw-r--r-- 2 root root 0 ??? 11 13:54 gstatus-0.64-3.el7.x86_64.rpm >>>> [root at father brick]# >>>> >>>> so now file has 0 length. >>>> >>>> try to heal: >>>> >>>> >>>> >>>> [root at father brick]# gluster volume heal pool >>>> Launching heal operation to perform index self heal on volume pool has >>>> been successful >>>> Use heal info commands to check status >>>> [root at father brick]# ls -l gstatus-0.64-3.el7.x86_64.rpm >>>> -rw-r--r-- 2 root root 0 ??? 11 13:54 gstatus-0.64-3.el7.x86_64.rpm >>>> [root at father brick]# >>>> >>>> >>>> nothing! >>>> >>>> [root at father brick]# gluster volume heal pool full >>>> Launching heal operation to perform full self heal on volume pool has >>>> been successful >>>> Use heal info commands to check status >>>> [root at father brick]# ls -l gstatus-0.64-3.el7.x86_64.rpm >>>> -rw-r--r-- 2 root root 52268 ??? 11 13:00 gstatus-0.64-3.el7.x86_64.rpm >>>> [root at father brick]# >>>> >>>> >>>> full heal is OK. >>>> >>>> But, self-heal is doing index heal according to >>>> >>>> >>>> http://staged-gluster-docs.readthedocs.io/en/release3.7.0beta1/Developer-guide/afr-self-heal-daemon/ >>>> >>>> Is this bug? >>>> >>>> >>>> As far as I remember it worked in 3.7.10.... >>>> >>>> >>>> _______________________________________________ >>>> Gluster-users mailing list >>>> Gluster-users at gluster.org >>>> http://www.gluster.org/mailman/listinfo/gluster-users >>> >>> >>> >>> >>> -- >>> Pranith >>> >>> >>> >> >> >> -- >> Pranith >> >> >> > > > -- > Pranith > > >-- Pranith -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160713/caab9bdc/attachment.html>
13.07.2016 08:43, Pranith Kumar Karampuri ?????:> > > On Wed, Jul 13, 2016 at 9:41 AM, Dmitry Melekhov <dm at belkam.com > <mailto:dm at belkam.com>> wrote: > > 13.07.2016 07:46, Pranith Kumar Karampuri ?????: >> >> >> On Tue, Jul 12, 2016 at 9:27 PM, Dmitry Melekhov <dm at belkam.com >> <mailto:dm at belkam.com>> wrote: >> >> >> >> 12.07.2016 17:39, Pranith Kumar Karampuri ?????: >>> Wow, what are the steps to recreate the problem? >> >> just set file length to zero, always reproducible. >> >> >> Changing things on the brick i.e. not from gluster volume mount >> is not something you want to do. In the worst case(I have seen >> this only once in the last 5 years though) where you do this it >> can lead to data loss also. So please be aware of it. > > Data replication with gluster is a way to avoid data loss, right? > Or no? If not- why use gluster then? > I though that gluster self-healing will heal or at least report > missed files or files with wrong lenths- i.e. corruptions visible > just by reading brick's directory, > not comparing data as bit rot detection... > If this is not a bug, then gluster is not what I expected :-( > > > Yes Data replication with gluster is a way to avoid data loss. > Changing files directly on the brick is similar to changing internal > data structures of a disk filesystem. Or changing the internal files > of a database. Things may stop working as you expect it to. All the > hardwork done by the stack is nullified if you fiddle with the data on > the brick directly. To put it succinctly you enter into the area of > undefined behavior if you start fiddling with the data on the brick > directly. Unless it is a documented behaviour I suggest you don't do it. >Sorry, I'm talking not about direct data manipulation in bricks as way to use gluster, I'm talking about problems detection and recovery. As I already said- if I for some reason ( real case can be only by accident ) will delete file this will not be detected by self-heal daemon, and, thus, will lead to lower replication level, i.e. lower failure tolerance. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160713/e00ceb4b/attachment.html>