Mark Staudinger
2018-Apr-26 03:04 UTC
[Gluster-users] FreeBSD problem adding/removing replicated bricks
Hi Folks, I'm trying to debug an issue that I've found while attempting to qualify GlusterFS for potential distributed storage projects on the FreeBSD-11.1 server platform - using the existing package of GlusterFS v3.11.1_4 The main issue I've encountered is that I cannot add new bricks while setting/increasing the replica count. If I create a replicated volume "poc" on two hosts, say s1:/gluster/1/poc and s2:/gluster/1/poc, the volume is created properly and shows replicated status, files are written to both volumes. If I create a single volume: s1:/gluster/1/poc as a single / distributed brick, and then try to run: gluster volume add-brick poc replica 2 s2:/gluster/1/poc it will always fail (sometimes after a pause, sometimes not.)? The only error I'm seeing on the server hosting the new brick, aside from the generic "Unable to add bricks" message, is like so: I [MSGID: 106578] [glusterd-brick-ops.c:1352:glusterd_op_perform_add_bricks] 0-management:? replica-count is set 2 I [MSGID: 106578] [glusterd-brick-ops.c:1362:glusterd_op_perform_add_bricks] 0-management: type is set 2, need to change it E [MSGID: 106054] [glusterd-utils.c:12974:glusterd_handle_replicate_brick_ops] 0-management:? Failed to set extended attribute trusted.add-brick : Operation not?? supported [Operation not supported] E [MSGID: 106074] [glusterd-brick-ops.c:2565:glusterd_op_add_brick] 0-glusterd: Unable to add bricks E [MSGID: 106123] [glusterd-mgmt.c:311:gd_mgmt_v3_commit_fn] 0-management: Add-brick commit failed. I was initially using ZFS and noted that ZFS on FreeBSD does not support xattr, so I reverted to using UFS as the storage type for the brick, and still encounter this behavior. I also recompiled the port (again, GlusterFS v3.11.1) with the patch from https://bugzilla.redhat.com/show_bug.cgi?id=1484246 as this deals specifically with xattr handling in FreeBSD. To recap - I'm able to create any type of volume (2 or 3-way replicated or distributed), but I'm unable to add replicated bricks to a volume. I was, however, able to add a second distributed brick ( gluster volume add-brick poc s2:/gluster/1/poc ) - so the issue seems specific to adding and/or changing the replica count while adding a new brick. Please let me know if there are any other issues in addition to bug #1452961 I should be aware of, or additional log or debug info I can provide. Best Regards, Mark PS apologies if this made it to the list twice - I had this prior to acknowledging the confirmation email so I believe the first message was discarded.