Nithya Balachandran
2018-Oct-04 13:22 UTC
[Gluster-users] Rebalance failed on Distributed Disperse volume based on 3.12.14 version
Hi Mauro, The files on s04 and s05 can be deleted safely as long as those bricks have been removed from the volume and their brick processes are not running. .glusterfs/indices/xattrop/xattrop-* are links to files that need to be healed. .glusterfs/quarantine/stub-00000000-0000-0000-0000-000000000008 links to files that bitrot (if enabled)says are corrupted. (none in this case) I will get back to you on s06. Can you please provide the output of gluster volume info again? Regards, Nithya On 4 October 2018 at 13:47, Mauro Tridici <mauro.tridici at cmcc.it> wrote:> > Dear Ashish, Dear Nithya, > > I?m writing this message only to summarize and simplify the information > about the "not migrated? files left on removed bricks on server s04, s05 > and s06. > In attachment, you can find 3 files (1 file for each server) containing > the ?not migrated? files lists and related brick number. > > In particular: > > - s04 and s05 bricks contain only not migrated files in hidden > directories ?/gluster/mnt#/brick/.glusterfs" (I could delete them, > doesn?t it?) > - s06 bricks contain > - not migrated files in hidden directories ?/gluster/mnt#/ > brick/.glusterfs?; > - not migrated files with size equal to 0; > - not migrated files with size greater than 0. > > > I think it was necessary to collect and summarize information to simplify > your analysis. > Thank you very much, > Mauro > > > > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20181004/3f84bbbb/attachment.html>
Mauro Tridici
2018-Oct-04 13:31 UTC
[Gluster-users] Rebalance failed on Distributed Disperse volume based on 3.12.14 version
Hi Nithya, thank you very much. This is the current ?gluster volume info? output after removing bricks (and after peer detach command). [root at s01 ~]# gluster volume info Volume Name: tier2 Type: Distributed-Disperse Volume ID: a28d88c5-3295-4e35-98d4-210b3af9358c Status: Started Snapshot Count: 0 Number of Bricks: 6 x (4 + 2) = 36 Transport-type: tcp Bricks: Brick1: s01-stg:/gluster/mnt1/brick Brick2: s02-stg:/gluster/mnt1/brick Brick3: s03-stg:/gluster/mnt1/brick Brick4: s01-stg:/gluster/mnt2/brick Brick5: s02-stg:/gluster/mnt2/brick Brick6: s03-stg:/gluster/mnt2/brick Brick7: s01-stg:/gluster/mnt3/brick Brick8: s02-stg:/gluster/mnt3/brick Brick9: s03-stg:/gluster/mnt3/brick Brick10: s01-stg:/gluster/mnt4/brick Brick11: s02-stg:/gluster/mnt4/brick Brick12: s03-stg:/gluster/mnt4/brick Brick13: s01-stg:/gluster/mnt5/brick Brick14: s02-stg:/gluster/mnt5/brick Brick15: s03-stg:/gluster/mnt5/brick Brick16: s01-stg:/gluster/mnt6/brick Brick17: s02-stg:/gluster/mnt6/brick Brick18: s03-stg:/gluster/mnt6/brick Brick19: s01-stg:/gluster/mnt7/brick Brick20: s02-stg:/gluster/mnt7/brick Brick21: s03-stg:/gluster/mnt7/brick Brick22: s01-stg:/gluster/mnt8/brick Brick23: s02-stg:/gluster/mnt8/brick Brick24: s03-stg:/gluster/mnt8/brick Brick25: s01-stg:/gluster/mnt9/brick Brick26: s02-stg:/gluster/mnt9/brick Brick27: s03-stg:/gluster/mnt9/brick Brick28: s01-stg:/gluster/mnt10/brick Brick29: s02-stg:/gluster/mnt10/brick Brick30: s03-stg:/gluster/mnt10/brick Brick31: s01-stg:/gluster/mnt11/brick Brick32: s02-stg:/gluster/mnt11/brick Brick33: s03-stg:/gluster/mnt11/brick Brick34: s01-stg:/gluster/mnt12/brick Brick35: s02-stg:/gluster/mnt12/brick Brick36: s03-stg:/gluster/mnt12/brick Options Reconfigured: network.ping-timeout: 0 features.scrub: Active features.bitrot: on features.inode-quota: on features.quota: on performance.client-io-threads: on cluster.min-free-disk: 10 cluster.quorum-type: auto transport.address-family: inet nfs.disable: on server.event-threads: 4 client.event-threads: 4 cluster.lookup-optimize: on performance.readdir-ahead: on performance.parallel-readdir: off cluster.readdir-optimize: on features.cache-invalidation: on features.cache-invalidation-timeout: 600 performance.stat-prefetch: on performance.cache-invalidation: on performance.md-cache-timeout: 600 network.inode-lru-limit: 50000 performance.io-cache: off disperse.cpu-extensions: auto performance.io-thread-count: 16 features.quota-deem-statfs: on features.default-soft-limit: 90 cluster.server-quorum-type: server diagnostics.latency-measurement: on diagnostics.count-fop-hits: on cluster.brick-multiplex: on cluster.server-quorum-ratio: 51% Regards, Mauro> Il giorno 04 ott 2018, alle ore 15:22, Nithya Balachandran <nbalacha at redhat.com> ha scritto: > > Hi Mauro, > > > The files on s04 and s05 can be deleted safely as long as those bricks have been removed from the volume and their brick processes are not running. > > > .glusterfs/indices/xattrop/xattrop-* are links to files that need to be healed. > .glusterfs/quarantine/stub-00000000-0000-0000-0000-000000000008 links to files that bitrot (if enabled)says are corrupted. (none in this case) > > > I will get back to you on s06. Can you please provide the output of gluster volume info again? > > Regards, > Nithya > > > > On 4 October 2018 at 13:47, Mauro Tridici <mauro.tridici at cmcc.it <mailto:mauro.tridici at cmcc.it>> wrote: > > Dear Ashish, Dear Nithya, > > I?m writing this message only to summarize and simplify the information about the "not migrated? files left on removed bricks on server s04, s05 and s06. > In attachment, you can find 3 files (1 file for each server) containing the ?not migrated? files lists and related brick number. > > In particular: > s04 and s05 bricks contain only not migrated files in hidden directories ?/gluster/mnt#/brick/.glusterfs" (I could delete them, doesn?t it?) > s06 bricks contain > not migrated files in hidden directories ?/gluster/mnt#/brick/.glusterfs?; > not migrated files with size equal to 0; > not migrated files with size greater than 0. > > I think it was necessary to collect and summarize information to simplify your analysis. > Thank you very much, > Mauro > > > > > > > >------------------------- Mauro Tridici Fondazione CMCC CMCC Supercomputing Center presso Complesso Ecotekne - Universit? del Salento - Strada Prov.le Lecce - Monteroni sn 73100 Lecce IT http://www.cmcc.it mobile: (+39) 327 5630841 email: mauro.tridici at cmcc.it <mailto:mauro.tridici at cmcc.it> https://it.linkedin.com/in/mauro-tridici-5977238b -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20181004/4f770e1e/attachment.html>