similar to: Possible split-brain

I put the share into debug mode and then repeated the process from a ppc64 client and an x86 client. Weirdly the client logs were almost identical. Here's the ppc64 gluster client log of attempting to create a folder... ------------- [2017-09-20 13:34:23.344321] D [rpc-clnt-ping.c:93:rpc_clnt_remove_ping_timer_locked] (-->

I started rebalancing my volume after updating from 3.2.7 to 3.3.1. After a few hours, I noticed a large number of failures in the rebalance status: > Node Rebalanced-files size scanned failures > status > --------- ----------- ----------- ----------- ----------- > ------------ > localhost 0 0Bytes 4288805

I am seeing this crashing happening, I am working on the self healing errors as well, not sure if the two are related. I would appreciate any direction on trying to resolve the issue, I have clients dropping connection daily. [2013-10-26 15:35:46.935903] E [afr-self-heal-common.c:2160:afr_self_heal_completion_cbk] 0-ENTV04EP-replicate-9: background meta-data self-heal failed on / [2013-10-26

Hi, I'm running glusterFS 3.3.1 on Centos 6.4. ? Gluster volume status Status of volume: glustervol Gluster process Port Online Pid ------------------------------------------------------------------------------ Brick KWTOCUATGS001:/mnt/cloudbrick 24009 Y 20031 Brick KWTOCUATGS002:/mnt/cloudbrick

Hello, Were having issues with macs writing to our gluster system. Gluster vol info at end. On a mac, if I make a file in the shell I get the following message: smoke:hunter david$ echo hello > test -bash: test: Operation not permitted And the file is made but is zero size. smoke:hunter david$ ls -l test -rw-r--r-- 1 david realise 0 Mar 3 08:44 test glusterfs/nfslog logs thus:

"May you live in interesting times" Is this a curse or a blessing? :) I've just tested a 3.1.5 GlusterFS native client against a 3.1.3 storage pool using this volume: Volume Name: pfs-rw1 Type: Distributed-Replicate Status: Started Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: jc1letgfs16-pfs1:/export/read-write/g01 Brick2: jc1letgfs13-pfs1:/export/read-write/g01

Looks like it is an issue with architecture compatibility in RPC layer (ie, with XDRs and how it is used). Just glance the logs of the client process where you saw the errors, which could give some hints. If you don't understand the logs, share them, so we will try to look into it. -Amar On Wed, Sep 20, 2017 at 2:40 AM, Walter Deignan <WDeignan at uline.com> wrote: > I recently

I recently compiled the 3.10-5 client from source on a few PPC64 systems running RHEL 7.3. They are mounting a Gluster volume which is hosted on more traditional x86 servers. Everything seems to be working properly except for creating new directories from the PPC64 clients. The mkdir command gives a "Input/output error" and for the first few minutes the new directory is

It's a pity I don't know how to re-create the issue. While there are 1-2 crashed clients in total 120 clients every day. Below is gdb result: (gdb) where #0 0x0000003267432885 in raise () from /lib64/libc.so.6 #1 0x0000003267434065 in abort () from /lib64/libc.so.6 #2 0x000000326746f7a7 in __libc_message () from /lib64/libc.so.6 #3 0x00000032674750c6 in malloc_printerr () from

Dear all, Right out of the blue glusterfs is not working fine any more every now end the it stops working telling me, Endpoint not connected and writing core files: [root at tuepdc /]# file core.15288 core.15288: ELF 64-bit LSB core file AMD x86-64, version 1 (SYSV), SVR4-style, from 'glusterfs' My Version: [root at tuepdc /]# glusterfs --version glusterfs 3.2.0 built on Apr 22 2011

On 07/20/2017 02:20 PM, yayo (j) wrote: > Hi, > > Thank you for the answer and sorry for delay: > > 2017-07-19 16:55 GMT+02:00 Ravishankar N <ravishankar at redhat.com > <mailto:ravishankar at redhat.com>>: > > 1. What does the glustershd.log say on all 3 nodes when you run > the command? Does it complain anything about these files? > > >

2017-07-20 11:34 GMT+02:00 Ravishankar N <ravishankar at redhat.com>: > > Could you check if the self-heal daemon on all nodes is connected to the 3 > bricks? You will need to check the glustershd.log for that. > If it is not connected, try restarting the shd using `gluster volume start > engine force`, then launch the heal command like you did earlier and see if > heals

Hi, Thank you for the answer and sorry for delay: 2017-07-19 16:55 GMT+02:00 Ravishankar N <ravishankar at redhat.com>: 1. What does the glustershd.log say on all 3 nodes when you run the > command? Does it complain anything about these files? > No, glustershd.log is clean, no extra log after command on all 3 nodes > 2. Are these 12 files also present in the 3rd data brick?

Hi, I am still trying to get kick-start centos in my vmware5 because pxe cannot find the pxe server. I do not control the dhcp or pxe server. I have both my kickstart file and my iso image for centos6.5 on my vmware datastore, but am trying to run my kickstart file from VMware guest. Can I tell the command line to run from the datastore in VMWare? Or must I convert my kickstart file to a

I should add that in /var/lib/glusterd/vols/gv0/gv0-shd.vol and in all other configs in /var/lib/glusterd/ on all three machines the nodes are consistently named client-2: zephyrosaurus client-3: alvarezsaurus client-4: nanosaurus This is normal. It was the second time that a brick was removed, so client-0 and client-1 are gone. So the problem is the file attibutes themselves. And there I see

Hi All , My first message and I have a little problem with my FC6 box trying to block emule traffic using layer7 . Here my network : Internet --------- ADSL Router ------------------- FC6 Box -------------------- Emule Box external ADSL : Dynamic Internal ADSL : 192.168.254.1 external FC6 : 192.168.254.3 internal FC6 : 192.168.253.1 Emule Box : 192.168.253.3 I guess that everything

Hi, I have a glusterfs volume replicated on three nodes. I am planing to use the volume as storage for vMware ESXi machines using NFS. The reason for using tree nodes is to be able to configure Quorum and avoid split-brains. However, during my initial testing when intentionally and gracefully restart the node "ned", a split-brain/self-heal error occurred. The log on "todd"

Hi all, I've been trying out Gluster Platform, so some info will be from the management interface. If additional data from logs is required, please let me know which logfiles. When using a Gluster NFS volume as a ESXi datastore, creation of a disk file (vmdk) takes a considerable amount of time (with unfsd using 10-50% cpu) and then crashes unfsd. A reboot of the system seems necessary to

I sent a long message yesterday describing a problem I thought had to do with the JIT stubs. After further investigating, the problem seems to be in the code generation. The following basic block seems to have an error in it's code generation: __exp.exit: ; preds = %codeRepl258, %__exp_bb_bb.exit phi double [ 1.000000e+00, %codeRepl258 ], [ %.reload.reload.i,

Hello all I have a mail store on a volume replica 3 with no arbiter. A while ago the disk of one of the bricks failed and I was several days late to notice it. When I did, I removed that brick from the volume, replaced the failed disk, updated the OS on that machine from el8 to el9 and gluster on all three nodes from 10.3 to 11.1, added back the brick and started a heal. Things appeared to work