Gluster users - Jul 2017 - [ovirt-users] ovirt 4.1 hosted engine hyper converged on glusterfs 3.8.10 : "engine" storage domain alway complain about "unsynced" elements

Hi,

Thank you for the answer and sorry for delay:

2017-07-19 16:55 GMT+02:00 Ravishankar N <ravishankar at redhat.com>:

1. What does the glustershd.log say on all 3 nodes when you run
the
> command? Does it complain anything about these files?
>
No, glustershd.log is clean, no extra log after command on all 3 nodes

> 2. Are these 12 files also present in the 3rd data brick?
>
I've checked right now: all files exists in all 3 nodes

> 3. Can you provide the output of `gluster volume info` for the this volume?
>

*Volume Name: engine*
*Type: Replicate*
*Volume ID: d19c19e3-910d-437b-8ba7-4f2a23d17515*
*Status: Started*
*Snapshot Count: 0*
*Number of Bricks: 1 x 3 = 3*
*Transport-type: tcp*
*Bricks:*
*Brick1: node01:/gluster/engine/brick*
*Brick2: node02:/gluster/engine/brick*
*Brick3: node04:/gluster/engine/brick*
*Options Reconfigured:*
*nfs.disable: on*
*performance.readdir-ahead: on*
*transport.address-family: inet*
*storage.owner-uid: 36*
*performance.quick-read: off*
*performance.read-ahead: off*
*performance.io-cache: off*
*performance.stat-prefetch: off*
*performance.low-prio-threads: 32*
*network.remote-dio: off*
*cluster.eager-lock: enable*
*cluster.quorum-type: auto*
*cluster.server-quorum-type: server*
*cluster.data-self-heal-algorithm: full*
*cluster.locking-scheme: granular*
*cluster.shd-max-threads: 8*
*cluster.shd-wait-qlength: 10000*
*features.shard: on*
*user.cifs: off*
*storage.owner-gid: 36*
*features.shard-block-size: 512MB*
*network.ping-timeout: 30*
*performance.strict-o-direct: on*
*cluster.granular-entry-heal: on*
*auth.allow: **

          server.allow-insecure: on




>
> Some extra info:
>>
>> We have recently changed the gluster from: 2 (full repliacated) + 1
>> arbiter to 3 full replicated cluster
>>
>
> Just curious, how did you do this? `remove-brick` of arbiter brick
> followed by an `add-brick` to increase to replica-3?
>
>
Yes


#gluster volume remove-brick engine replica 2 node03:/gluster/data/brick
force *(OK!)*

#gluster volume heal engine info *(no entries!)*

#gluster volume add-brick engine replica 3 node04:/gluster/engine/brick
*(OK!)*

*After some minutes*

[root at node01 ~]#  gluster volume heal engine info
Brick node01:/gluster/engine/brick
Status: Connected
Number of entries: 0

Brick node02:/gluster/engine/brick
Status: Connected
Number of entries: 0

Brick node04:/gluster/engine/brick
Status: Connected
Number of entries: 0


> Thanks,
> Ravi
>
Another extra info (I don't know if this can be the problem): Five days ago
A black out has suddenly shut down the networks switch (also gluster
network) of node 03 and 04 ... But I don't know this problem is in place
after this black out

Thank you!
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170720/74878e11/attachment.html>

On 07/20/2017 02:20 PM, yayo (j) wrote:
> Hi,
>
> Thank you for the answer and sorry for delay:
>
> 2017-07-19 16:55 GMT+02:00 Ravishankar N <ravishankar at redhat.com 
> <mailto:ravishankar at redhat.com>>:
>
>     1. What does the glustershd.log say on all 3 nodes when you run
>     the command? Does it complain anything about these files?
>
>
> No, glustershd.log is clean, no extra log after command on all 3 nodes
Could you check if the self-heal daemon on all nodes is connected to the 
3 bricks? You will need to check the glustershd.log for that.
If it is not connected, try restarting the shd using `gluster volume 
start engine force`, then launch the heal command like you did earlier 
and see if heals happen.

If it doesn't, please provide the getfattr outputs of the 12 files from 
all 3 nodes using `getfattr -d -m . -e hex 
//gluster/engine/brick//path-to-file` ?

Thanks,
Ravi
>     2. Are these 12 files also present in the 3rd data brick?
>
>
> I've checked right now: all files exists in all 3 nodes
>
>     3. Can you provide the output of `gluster volume info` for the
>     this volume?
>
>
>
>     /Volume Name: engine/
>     /Type: Replicate/
>     /Volume ID: d19c19e3-910d-437b-8ba7-4f2a23d17515/
>     /Status: Started/
>     /Snapshot Count: 0/
>     /Number of Bricks: 1 x 3 = 3/
>     /Transport-type: tcp/
>     /Bricks:/
>     /Brick1: node01:/gluster/engine/brick/
>     /Brick2: node02:/gluster/engine/brick/
>     /Brick3: node04:/gluster/engine/brick/
>     /Options Reconfigured:/
>     /nfs.disable: on/
>     /performance.readdir-ahead: on/
>     /transport.address-family: inet/
>     /storage.owner-uid: 36/
>     /performance.quick-read: off/
>     /performance.read-ahead: off/
>     /performance.io-cache: off/
>     /performance.stat-prefetch: off/
>     /performance.low-prio-threads: 32/
>     /network.remote-dio: off/
>     /cluster.eager-lock: enable/
>     /cluster.quorum-type: auto/
>     /cluster.server-quorum-type: server/
>     /cluster.data-self-heal-algorithm: full/
>     /cluster.locking-scheme: granular/
>     /cluster.shd-max-threads: 8/
>     /cluster.shd-wait-qlength: 10000/
>     /features.shard: on/
>     /user.cifs: off/
>     /storage.owner-gid: 36/
>     /features.shard-block-size: 512MB/
>     /network.ping-timeout: 30/
>     /performance.strict-o-direct: on/
>     /cluster.granular-entry-heal: on/
>     /auth.allow: */
>
>           server.allow-insecure: on
>
>
>
>
>>         Some extra info:
>>
>>         We have recently changed the gluster from: 2 (full
>>         repliacated) + 1 arbiter to 3 full replicated cluster
>>
>
>     Just curious, how did you do this? `remove-brick` of arbiter
>     brick  followed by an `add-brick` to increase to replica-3?
>
>
> Yes
>
>
> #gluster volume remove-brick engine replica 2 
> node03:/gluster/data/brick force *(OK!)*
>
> #gluster volume heal engine info *(no entries!)*
>
> #gluster volume add-brick engine replica 3 
> node04:/gluster/engine/brick *(OK!)*
>
> *After some minutes*
>
> [root at node01 ~]#  gluster volume heal engine info
> Brick node01:/gluster/engine/brick
> Status: Connected
> Number of entries: 0
>
> Brick node02:/gluster/engine/brick
> Status: Connected
> Number of entries: 0
>
> Brick node04:/gluster/engine/brick
> Status: Connected
> Number of entries: 0
>
>     Thanks,
>     Ravi
>
>
> Another extra info (I don't know if this can be the problem): Five 
> days ago A black out has suddenly shut down the networks switch (also 
> gluster network) of node 03 and 04 ... But I don't know this problem 
> is in place after this black out
>
> Thank you!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170720/2be61256/attachment.html>

2017-07-20 11:34 GMT+02:00 Ravishankar N <ravishankar at redhat.com>:
>
> Could you check if the self-heal daemon on all nodes is connected to the 3
> bricks? You will need to check the glustershd.log for that.
> If it is not connected, try restarting the shd using `gluster volume start
> engine force`, then launch the heal command like you did earlier and see if
> heals happen.
>
>
I've executed the command on all 3 nodes (Know is enougth only one) , after
that the "heal" command report elements between 6 and 10 ... (sometime
6,
sometime 8, sometime 10)


Log on glustershd.log don't say anything :

*[2017-07-20 09:58:46.573079] I [MSGID: 108026]
[afr-self-heal-common.c:1254:afr_log_selfheal] 0-engine-replicate-0:
Completed data selfheal on e6dfd556-340b-4b76-b47b-7b6f5bd74327.
sources=[0] 1  sinks=2*
*[2017-07-20 09:59:22.995003] I [MSGID: 108026]
[afr-self-heal-metadata.c:51:__afr_selfheal_metadata_do]
0-engine-replicate-0: performing metadata selfheal on
f05b9742-2771-484a-85fc-5b6974bcef81*
*[2017-07-20 09:59:22.999372] I [MSGID: 108026]
[afr-self-heal-common.c:1254:afr_log_selfheal] 0-engine-replicate-0:
Completed metadata selfheal on f05b9742-2771-484a-85fc-5b6974bcef81.
sources=[0] 1  sinks=2*



> If it doesn't, please provide the getfattr outputs of the 12 files from
> all 3 nodes using `getfattr -d -m . -e hex */gluster/engine/brick/*
> path-to-file` ?
>
>
*NODE01:*
*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/.shard/8aa74564-6740-403e-ad51-f56d9ca5d7a7.68*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-1=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x000000120000000000000000*
*trusted.bit-rot.version=0x090000000000000059647d5b000447e9*
*trusted.gfid=0xe3565b5014954e5bae883bceca47b7d9*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/.shard/8aa74564-6740-403e-ad51-f56d9ca5d7a7.48*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-1=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x0000000e0000000000000000*
*trusted.bit-rot.version=0x090000000000000059647d5b000447e9*
*trusted.gfid=0x676067891f344c1586b8c0d05b07f187*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/8f215dd2-8531-4a4f-b6ed-ea789dd8821b/images/19d71267-52a4-42a3-bb1e-e3145361c0c2/7a215635-02f3-47db-80db-8b689c6a8f01*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-1=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x000000550000000000000000*
*trusted.bit-rot.version=0x090000000000000059647d5b000447e9*
*trusted.gfid=0x8aa745646740403ead51f56d9ca5d7a7*
*trusted.glusterfs.shard.block-size=0x0000000020000000*
*trusted.glusterfs.shard.file-size=0x0000000c8000000000000000000000000000000000d4f2290000000000000000*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/.shard/8aa74564-6740-403e-ad51-f56d9ca5d7a7.60*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-1=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x000000070000000000000000*
*trusted.bit-rot.version=0x090000000000000059647d5b000447e9*
*trusted.gfid=0x4e33ac33dddb4e29b4a351770b81166a*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/8f215dd2-8531-4a4f-b6ed-ea789dd8821b/dom_md/ids*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-1=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x000000000000000000000000*
*trusted.bit-rot.version=0x0f0000000000000059647d5b000447e9*
*trusted.gfid=0x2581cb9ac2b74bd9ac17a09bd2f001b3*
*trusted.glusterfs.shard.block-size=0x0000000020000000*
*trusted.glusterfs.shard.file-size=0x0000000000100000000000000000000000000000000008000000000000000000*

*getfattr: Removing leading '/' from absolute path names*
*# file: gluster/engine/brick/__DIRECT_IO_TEST__*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-1=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x000000000000000000000000*
*trusted.gfid=0xf05b97422771484a85fc5b6974bcef81*
*trusted.glusterfs.shard.block-size=0x0000000020000000*
*trusted.glusterfs.shard.file-size=0x0000000000000000000000000000000000000000000000000000000000000000*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/8f215dd2-8531-4a4f-b6ed-ea789dd8821b/images/88d41053-a257-4272-9e2e-2f3de0743b81/6573ed08-d3ed-4d12-9227-2c95941e1ad6*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-1=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x000000010000000000000000*
*trusted.bit-rot.version=0x0f0000000000000059647d5b000447e9*
*trusted.gfid=0xe6dfd556340b4b76b47b7b6f5bd74327*
*trusted.glusterfs.shard.block-size=0x0000000020000000*
*trusted.glusterfs.shard.file-size=0x0000000000100000000000000000000000000000000008000000000000000000*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/.shard/8aa74564-6740-403e-ad51-f56d9ca5d7a7.64*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-1=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x0000000a0000000000000000*
*trusted.bit-rot.version=0x090000000000000059647d5b000447e9*
*trusted.gfid=0x9ef88647cfe64a35a38ca5173c9e8fc0*


*NODE02:*
*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/.shard/8aa74564-6740-403e-ad51-f56d9ca5d7a7.68*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-0=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x0000001a0000000000000000*
*trusted.bit-rot.version=0x08000000000000005965ede0000c352d*
*trusted.gfid=0xe3565b5014954e5bae883bceca47b7d9*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/.shard/8aa74564-6740-403e-ad51-f56d9ca5d7a7.48*
*trusted.afr.dirty=0x000000010000000000000000*
*trusted.afr.engine-client-0=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x0000000c0000000000000000*
*trusted.bit-rot.version=0x08000000000000005965ede0000c352d*
*trusted.gfid=0x676067891f344c1586b8c0d05b07f187*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/8f215dd2-8531-4a4f-b6ed-ea789dd8821b/images/19d71267-52a4-42a3-bb1e-e3145361c0c2/7a215635-02f3-47db-80db-8b689c6a8f01*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-0=0x000000000000000000000000*
*trusted.afr.engine-client-1=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x0000008e0000000000000000*
*trusted.bit-rot.version=0x08000000000000005965ede0000c352d*
*trusted.gfid=0x8aa745646740403ead51f56d9ca5d7a7*
*trusted.glusterfs.shard.block-size=0x0000000020000000*
*trusted.glusterfs.shard.file-size=0x0000000c8000000000000000000000000000000000d4f2290000000000000000*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/.shard/8aa74564-6740-403e-ad51-f56d9ca5d7a7.60*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-0=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x000000090000000000000000*
*trusted.bit-rot.version=0x08000000000000005965ede0000c352d*
*trusted.gfid=0x4e33ac33dddb4e29b4a351770b81166a*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/8f215dd2-8531-4a4f-b6ed-ea789dd8821b/dom_md/ids*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-0=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x000000010000000000000000*
*trusted.bit-rot.version=0x08000000000000005965ede0000c352d*
*trusted.gfid=0x2581cb9ac2b74bd9ac17a09bd2f001b3*
*trusted.glusterfs.shard.block-size=0x0000000020000000*
*trusted.glusterfs.shard.file-size=0x0000000000100000000000000000000000000000000008000000000000000000*

*getfattr: Removing leading '/' from absolute path names*
*# file: gluster/engine/brick/__DIRECT_IO_TEST__*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-0=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x000000000000000000000000*
*trusted.gfid=0xf05b97422771484a85fc5b6974bcef81*
*trusted.glusterfs.shard.block-size=0x0000000020000000*
*trusted.glusterfs.shard.file-size=0x0000000000000000000000000000000000000000000000000000000000000000*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/8f215dd2-8531-4a4f-b6ed-ea789dd8821b/images/88d41053-a257-4272-9e2e-2f3de0743b81/6573ed08-d3ed-4d12-9227-2c95941e1ad6*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-0=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x000000020000000000000000*
*trusted.bit-rot.version=0x08000000000000005965ede0000c352d*
*trusted.gfid=0xe6dfd556340b4b76b47b7b6f5bd74327*
*trusted.glusterfs.shard.block-size=0x0000000020000000*
*trusted.glusterfs.shard.file-size=0x0000000000100000000000000000000000000000000008000000000000000000*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/.shard/8aa74564-6740-403e-ad51-f56d9ca5d7a7.64*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.afr.engine-client-0=0x000000000000000000000000*
*trusted.afr.engine-client-2=0x000000120000000000000000*
*trusted.bit-rot.version=0x08000000000000005965ede0000c352d*
*trusted.gfid=0x9ef88647cfe64a35a38ca5173c9e8fc0*




*NODE04:*
*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/.shard/8aa74564-6740-403e-ad51-f56d9ca5d7a7.68*
*security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000*
*trusted.bit-rot.version=0x050000000000000059662c390006b836*
*trusted.gfid=0xe3565b5014954e5bae883bceca47b7d9*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/.shard/8aa74564-6740-403e-ad51-f56d9ca5d7a7.48*
*security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000*
*trusted.bit-rot.version=0x050000000000000059662c390006b836*
*trusted.gfid=0x676067891f344c1586b8c0d05b07f187*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/8f215dd2-8531-4a4f-b6ed-ea789dd8821b/images/19d71267-52a4-42a3-bb1e-e3145361c0c2/7a215635-02f3-47db-80db-8b689c6a8f01*
*security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000*
*trusted.bit-rot.version=0x050000000000000059662c390006b836*
*trusted.gfid=0x8aa745646740403ead51f56d9ca5d7a7*
*trusted.glusterfs.shard.block-size=0x0000000020000000*
*trusted.glusterfs.shard.file-size=0x0000000c8000000000000000000000000000000000d4f2290000000000000000*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/.shard/8aa74564-6740-403e-ad51-f56d9ca5d7a7.60*
*security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000*
*trusted.bit-rot.version=0x050000000000000059662c390006b836*
*trusted.gfid=0x4e33ac33dddb4e29b4a351770b81166a*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/8f215dd2-8531-4a4f-b6ed-ea789dd8821b/dom_md/ids*
*security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.bit-rot.version=0x050000000000000059662c390006b836*
*trusted.gfid=0x2581cb9ac2b74bd9ac17a09bd2f001b3*
*trusted.glusterfs.shard.block-size=0x0000000020000000*
*trusted.glusterfs.shard.file-size=0x0000000000100000000000000000000000000000000008000000000000000000*

*getfattr: Removing leading '/' from absolute path names*
*# file: gluster/engine/brick/__DIRECT_IO_TEST__*
*security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000*
*trusted.bit-rot.version=0x0200000000000000596484e20006237b*
*trusted.gfid=0xf05b97422771484a85fc5b6974bcef81*
*trusted.glusterfs.shard.block-size=0x0000000020000000*
*trusted.glusterfs.shard.file-size=0x0000000000000000000000000000000000000000000000000000000000000000*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/8f215dd2-8531-4a4f-b6ed-ea789dd8821b/images/88d41053-a257-4272-9e2e-2f3de0743b81/6573ed08-d3ed-4d12-9227-2c95941e1ad6*
*security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000*
*trusted.afr.dirty=0x000000000000000000000000*
*trusted.bit-rot.version=0x050000000000000059662c390006b836*
*trusted.gfid=0xe6dfd556340b4b76b47b7b6f5bd74327*
*trusted.glusterfs.shard.block-size=0x0000000020000000*
*trusted.glusterfs.shard.file-size=0x0000000000100000000000000000000000000000000008000000000000000000*

*getfattr: Removing leading '/' from absolute path names*
*# file:
gluster/engine/brick/.shard/8aa74564-6740-403e-ad51-f56d9ca5d7a7.64*
*security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000*
*trusted.bit-rot.version=0x050000000000000059662c390006b836*
*trusted.gfid=0x9ef88647cfe64a35a38ca5173c9e8fc0*





hum.... Is selinux the problem? but on node04 was disabled (AFTER GLUSTER
JOIN, I hope to remember) ... You think I needs to relabel? how?

*[root at node01 ~]# sestatus*
*SELinux status:                 disabled*

*[root at node02 ~]# sestatus*
*SELinux status:                 disabled*

*[root at node04 ~]# sestatus*
*SELinux status:                 disabled*



Thank you


> Thanks,
> Ravi
>
>
>
>
>> 2. Are these 12 files also present in the 3rd data brick?
>>
>
> I've checked right now: all files exists in all 3 nodes
>
>
>> 3. Can you provide the output of `gluster volume info` for the this
>> volume?
>>
>
>
> *Volume Name: engine*
> *Type: Replicate*
> *Volume ID: d19c19e3-910d-437b-8ba7-4f2a23d17515*
> *Status: Started*
> *Snapshot Count: 0*
> *Number of Bricks: 1 x 3 = 3*
> *Transport-type: tcp*
> *Bricks:*
> *Brick1: node01:/gluster/engine/brick*
> *Brick2: node02:/gluster/engine/brick*
> *Brick3: node04:/gluster/engine/brick*
> *Options Reconfigured:*
> *nfs.disable: on*
> *performance.readdir-ahead: on*
> *transport.address-family: inet*
> *storage.owner-uid: 36*
> *performance.quick-read: off*
> *performance.read-ahead: off*
> *performance.io-cache: off*
> *performance.stat-prefetch: off*
> *performance.low-prio-threads: 32*
> *network.remote-dio: off*
> *cluster.eager-lock: enable*
> *cluster.quorum-type: auto*
> *cluster.server-quorum-type: server*
> *cluster.data-self-heal-algorithm: full*
> *cluster.locking-scheme: granular*
> *cluster.shd-max-threads: 8*
> *cluster.shd-wait-qlength: 10000*
> *features.shard: on*
> *user.cifs: off*
> *storage.owner-gid: 36*
> *features.shard-block-size: 512MB*
> *network.ping-timeout: 30*
> *performance.strict-o-direct: on*
> *cluster.granular-entry-heal: on*
> *auth.allow: **
>
>           server.allow-insecure: on
>
>
>
>
>
>>
>> Some extra info:
>>>
>>> We have recently changed the gluster from: 2 (full repliacated) + 1
>>> arbiter to 3 full replicated cluster
>>>
>>
>> Just curious, how did you do this? `remove-brick` of arbiter brick
>> followed by an `add-brick` to increase to replica-3?
>>
>>
> Yes
>
>
> #gluster volume remove-brick engine replica 2 node03:/gluster/data/brick
> force *(OK!)*
>
> #gluster volume heal engine info *(no entries!)*
>
> #gluster volume add-brick engine replica 3 node04:/gluster/engine/brick
> *(OK!)*
>
> *After some minutes*
>
> [root at node01 ~]#  gluster volume heal engine info
> Brick node01:/gluster/engine/brick
> Status: Connected
> Number of entries: 0
>
> Brick node02:/gluster/engine/brick
> Status: Connected
> Number of entries: 0
>
> Brick node04:/gluster/engine/brick
> Status: Connected
> Number of entries: 0
>
>
>
>> Thanks,
>> Ravi
>>
>
> Another extra info (I don't know if this can be the problem): Five days
> ago A black out has suddenly shut down the networks switch (also gluster
> network) of node 03 and 04 ... But I don't know this problem is in
place
> after this black out
>
> Thank you!
>
>
>

-- 
Linux User: 369739 http://counter.li.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170720/cd975f48/attachment.html>