similar to: Add an arbiter when have multiple bricks at same server.

Yes but I want to add. Is it the same logic? --- Gilberto Nunes Ferreira +55 (47) 99676-7530 Proxmox VE VinChin Backup & Restore Em ter., 5 de nov. de 2024, 14:09, Aravinda <aravinda at kadalu.tech> escreveu: > Hello Gilberto, > > You can create a Arbiter volume using three bricks. Two of them will be > data bricks and one will be Arbiter brick. > > gluster volume

Your add-brick command adds 2 bricks 1 arbiter (even though you name them all arbiter!) The sequence is important: gluster v add-brick VMS replica 2 arbiter 1 gluster1:/gv0 gluster2:/gv0 arbiter1:/arb1 adds two data bricks and a corresponding arbiter from 3 different servers and 3 different disks,? thus you can loose any one server OR any one disk and stay up and consistent. adding more bricks

If you create a volume with replica 2 arbiter 1 you create 2 data bricks that are mirrored (makes 2 file copies) + you create 1 arbiter that holds metadata of all files on these bricks. You "can" create all on the same server, but this makes no sense, because when the server goes down, no files on these disks are accessible anymore, hence why bestpractice is to spread out over 3

After force the add-brick gluster volume add-brick VMS replica 3 arbiter 1 arbiter:/arbiter1 arbiter:/arbiter2 arbiter:/arbiter3 force volume add-brick: success pve01:~# gluster volume info Volume Name: VMS Type: Distributed-Replicate Volume ID: e1a4f787-3f62-441e-a7ce-c0ae6b111ebf Status: Started Snapshot Count: 0 Number of Bricks: 3 x (2 + 1) = 9 Transport-type: tcp Bricks: Brick1:

What's the volume structure right now? Best Regards, Strahil Nikolov On Wed, Nov 6, 2024 at 18:24, Gilberto Ferreira<gilberto.nunes32 at gmail.com> wrote: So I went ahead and do the force (is with you!) gluster volume add-brick VMS replica 3 arbiter 1 arbiter:/arbiter1 arbiter:/arbiter2 arbiter:/arbiter3 volume add-brick: failed: Multiple bricks of a replicate volume are present

Ok. I got confused here! For each brick I will need one arbiter brick, in a different partition/folder? And what if in some point in the future I decide to add a new brick in the main servers? Do I need to provide another partition/folder in the arbiter and then adjust the arbiter brick counter? --- Gilberto Nunes Ferreira Em ter., 5 de nov. de 2024 ?s 13:22, Andreas Schwibbe

So I went ahead and do the force (is with you!) gluster volume add-brick VMS replica 3 arbiter 1 arbiter:/arbiter1 arbiter:/arbiter2 arbiter:/arbiter3 volume add-brick: failed: Multiple bricks of a replicate volume are present on the same server. This setup is not optimal. Bricks should be on different nodes to have best fault tolerant co nfiguration. Use 'force' at the end of the command

But if I change replica 2 arbiter 1 to replica 3 arbiter 1 gluster volume add-brick VMS replica 3 arbiter 1 arbiter:/arbiter1 arbiter:/arbiter2 arbiter:/arbiter3 I got thir error: volume add-brick: failed: Multiple bricks of a replicate volume are present on the same server. This setup is not optimal. Bricks should be on different nodes to have best fault tolerant configuration. Use

Ok. I have a 3rd host with Debian 12 installed and Gluster v11. The name of the host is arbiter! I already add this host into the pool: arbiter:~# gluster pool list UUID Hostname State 0cbbfc27-3876-400a-ac1d-2d73e72a4bfd gluster1.home.local Connected 99ed1f1e-7169-4da8-b630-a712a5b71ccd gluster2 Connected

Still getting error pve01:~# gluster vol info Volume Name: VMS Type: Distributed-Replicate Volume ID: e1a4f787-3f62-441e-a7ce-c0ae6b111ebf Status: Started Snapshot Count: 0 Number of Bricks: 3 x 2 = 6 Transport-type: tcp Bricks: Brick1: gluster1:/disco2TB-0/vms Brick2: gluster2:/disco2TB-0/vms Brick3: gluster1:/disco1TB-0/vms Brick4: gluster2:/disco1TB-0/vms Brick5: gluster1:/disco1TB-1/vms

Right now you have 3 "sets" of replica 2 on 2 hosts.In your case you don't need so much space for arbiters (10-15GB with 95 maxpct is enough for each "set") and you need a 3rd system or when the node that holds the data brick + arbiter brick fails (2 node scenario) - that "set" will be unavailable. If you do have a 3rd host, I think the command would be:gluster

Hi there. I have 2 servers with this number of disks in each side: pve01:~# df | grep disco /dev/sdd 1.0T 9.4G 1015G 1% /disco1TB-0 /dev/sdh 1.0T 9.3G 1015G 1% /disco1TB-3 /dev/sde 1.0T 9.5G 1015G 1% /disco1TB-1 /dev/sdf 1.0T 9.4G 1015G 1% /disco1TB-2 /dev/sdg 2.0T 19G 2.0T 1% /disco2TB-1 /dev/sdc 2.0T 19G 2.0T 1%

If it's replica 2, you can loose up to 1 replica per distribution group.For example, if you have a volume TEST with such setup: server1:/brick1 server2:/brick1 server1:/brick2 server2:/brick2 You can loose any brick of the replica "/brick1" and any brick in the replica "/brick2". So if you loose server1:/brick1 and server2:/brick2 -> no data loss will be experienced.

Ok! I got it about how many disks I can lose and so on. But regard the arbiter isse, I always set this parameters in the gluster volume, in order to avoid split-brain and I might add that work pretty well to me. I already have a Proxmox VE cluster with 2 nodes and about 50 vms, running different Linux distro - and Windows as well - with Cpanel and other stuff, in production. Anyway here the

Hi there. I can't understand why I am having this different values: proxmox01:/vms/images# df Sist. Arq. Tam. Usado Disp. Uso% Montado em udev 252G 0 252G 0% /dev tmpfs 51G 9,4M 51G 1% /run /dev/sda4 433G 20G 413G 5% / tmpfs 252G 63M 252G 1% /dev/shm tmpfs 5,0M 0 5,0M 0% /run/lock efivarfs 496K 335K

Hi Matt, Run these commands on all the bricks of the replica pair to get the attrs set on the backend. On the bricks of first replica set: getfattr -d -e hex -m . <brick path>/.glusterfs/10/86/ 108694db-c039-4b7c-bd3d-ad6a15d811a2 On the fourth replica set: getfattr -d -e hex -m . <brick path>/.glusterfs/ e0/c5/e0c56bf7-8bfe-46ca-bde1-e46b92d33df3 Also run the "gluster volume

Attached is the heal log for the volume as well as the shd log. >> Run these commands on all the bricks of the replica pair to get the attrs set on the backend. [root at tpc-cent-glus1-081017 ~]# getfattr -d -e hex -m . /exp/b1/gv0/.glusterfs/10/86/108694db-c039-4b7c-bd3d-ad6a15d811a2 getfattr: Removing leading '/' from absolute path names # file:

Hi Matt, The files might be in split brain. Could you please send the outputs of these? gluster volume info <volname> gluster volume heal <volname> info And also the getfattr output of the files which are in the heal info output from all the bricks of that replica pair. getfattr -d -e hex -m . <file path on brick> Thanks & Regards Karthik On 16-Oct-2017 8:16 PM,

I've been following this particular thread as I have a similar issue (RAID6 array failed out with 3 dead drives at once while a 12 TB load was being copied into one mounted space - what a mess) I have >700K GFID entries that have no path data:Example:getfattr -d -e hex -m . .glusterfs/00/00/0000a5ef-5af7-401b-84b5-ff2a51c10421# file: .glusterfs/00/00/0000a5ef-5af7-401b-84b5-

In my case I was able to delete the hard links in the .glusterfs folders of the bricks and it seems to have done the trick, thanks! From: Karthik Subrahmanya [mailto:ksubrahm at redhat.com] Sent: Monday, October 23, 2017 1:52 AM To: Jim Kinney <jim.kinney at gmail.com>; Matt Waymack <mwaymack at nsgdv.com> Cc: gluster-users <Gluster-users at gluster.org> Subject: Re: