Poornima Gurusiddaiah
2018-Oct-04 14:15 UTC
[Gluster-users] Gluster problems permission denied LOOKUP () /etc/samba/private/msg.sock
On Tue, Oct 2, 2018 at 5:26 PM Diego Remolina <dijuremo at gmail.com> wrote:> Dear all, > > I have a two node setup running on Centos and gluster version > glusterfs-3.10.12-1.el7.x86_64 > > One of my nodes died (motherboard issue). Since I had to continue > being up, I modified the quorum to below 50% to make sure I could > still run on one server. > > The server runs ovirt and 2 VMs on top of a volume called vmstorage. I > also had a third node in the peer list, but never configured it as an > arbiter, so it just comes up in gluster v status. The server also run > a file server with samba to serve files to windows machines. > > The issue is that since starting the server on it's own as the samba > server, I am seeing permission denied errors for the "export" volume > in /var/log/glusterfs/export.log > > The errors look like this and repeat over and over: > > [2018-10-02 11:46:56.327925] I [MSGID: 139001] > [posix-acl.c:269:posix_acl_log_permit_denied] 0-posix-acl-autoload: > client: -, gfid: 5b5bed22-ace0-410d-8623-4f1a31069b81, > req(uid:1051,gid:513,perm:1,ngrps:2), > ctx(uid:0,gid:0,in-groups:0,perm:700,updated-fop:LOOKUP, acl:-) > [Permission denied] > [2018-10-02 11:46:56.328004] W [fuse-bridge.c:490:fuse_entry_cbk] > 0-glusterfs-fuse: 20599112: LOOKUP() /etc/samba/private/msg.sock/15149 > => -1 (Permission denied) > [2018-10-02 11:46:56.328185] W [fuse-bridge.c:490:fuse_entry_cbk] > 0-glusterfs-fuse: 20599113: LOOKUP() /etc/samba/private/msg.sock/15149 > => -1 (Permission denied) > [2018-10-02 11:47:53.766562] W [fuse-bridge.c:490:fuse_entry_cbk] > 0-glusterfs-fuse: 20600590: LOOKUP() /etc/samba/private/msg.sock/15149 > => -1 (Permission denied) > > The gluster volume export is mounted on /export, samba and ctdb are > instructed to use /export/etc/samba/private and /export/lock which is > on the gluster file system for the clustered tdb, etc. However, I keep > getting the log messages that fuse seems to try to access a folder > that does not exist, /etc/samba/private/msg.sock >This is an unconventional setup, the suggested way of clustering samba is as mentioned in [1]. Sharing tdbs using gluster volume can lead to more issues. Has the setup ever worked? Was this setup suggested somewhere? [1] https://access.qa.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html-single/administration_guide/#sect-SMB_CTDB> Why is this, how can I fix it? > > [root at ysmha01 export]# gluster v status export > Status of volume: export > Gluster process TCP Port RDMA Port Online > Pid > > ------------------------------------------------------------------------------ > Brick 10.0.1.6:/bricks/hdds/brick 49153 0 Y > 3516 > Self-heal Daemon on localhost N/A N/A Y > 3710 > Self-heal Daemon on 10.0.1.5 N/A N/A Y > 4380 > > Task Status of Volume export > > ------------------------------------------------------------------------------ > There are no active volume tasks > > These are all the volume options currently set: > > http://termbin.com/1xm5 > > Diego > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20181004/051b5b5f/attachment.html>
Diego Remolina
2018-Oct-05 14:20 UTC
[Gluster-users] Gluster problems permission denied LOOKUP () /etc/samba/private/msg.sock
Hi, Thanks for the reply! This was setup a few years ago and was working OK, even when falling back to this server. We had not failed over to this server recently after the latest samba upgrades, so Not sure if maybe the new samba and ctdb packages had a change that is creating the issue. samba-libs-4.7.1-9.el7_5.x86_64 samba-client-libs-4.7.1-9.el7_5.x86_64 samba-common-tools-4.7.1-9.el7_5.x86_64 samba-common-4.7.1-9.el7_5.noarch samba-common-libs-4.7.1-9.el7_5.x86_64 samba-vfs-glusterfs-4.7.1-9.el7_5.x86_64 samba-4.7.1-9.el7_5.x86_64 It may not be the right way to do it, so I am going to investigate your suggestion and find out if it works for us. I do need your help with answers to some questions below. A bit of an explanation on the current setup. Both servers, ysmha01 and ysmha02 are joined against AD using sssd. We are not using winbindd at all. For each server, we created a machine account in AD, and we also created a computer account for the "Shared" host name. So we have these 3 computer objects in AD ysmha01 10.0.0.6 ysmha02 10.0.0.7 ysmserver 10.0.0.1 (this ip is handled by ctdb) We are not controlling smb with ctdb (doing it manually). Both ysmha01 and ysmha02 were tied to AD using: realm join domain -v unattended Then we modified the sssd.conf file as follows: http://termbin.com/wulh And restarted sssd and everything works fine getting users and groups. We populate uidNumbers and gidNumbers for all users and groups in AD, so the permissions work. Then we configured samba to join the domain using the ysmserver machine account and only password (not keytab). So in order to keep the samba information available to both servers, we used the configuration: private dir = /export/etc/samba/private Since this is an un-conventional setup, could you explain the process of using both sssd and joining the machine to the AD domain? I am not quite sure I understand how to do that after having used SSSD first. In occasions where I set ysmha01 and ysmha02 as the netbios name for smb.conf and then ran net ads join after realm join, it simply updated the keytab and then sssd would not work anymore. This is why we ended up using the setup above. If you could point to a good process including smb.conf and how to join the machines to the domain, that would be appreciated. This is the current config for samba. For the Projects share I had to disable vfs gluster because I had issues with one specific type of files, but it would be really nice if I can clean up all of this and get it to work properly using vfs gluster for all shares. http://termbin.com/2f64 After replacing the motherboard on ysmha02 and bringing it back up last night, things seem to be working fine so far, but I still see the gluster error messages and I want to fix this and run it properly as it should: [2018-10-05 13:41:21.279685] I [MSGID: 139001] [posix-acl.c:269:posix_acl_log_permit_denied] 0-posix-acl-autoload: cli ent: -, gfid: 5b5bed22-ace0-410d-8623-4f1a31069b81, req(uid:1058,gid:513,perm:1,ngrps:3), ctx(uid:0,gid:0,in-groups:0, perm:700,updated-fop:LOOKUP, acl:-) [Permission denied] [2018-10-05 13:41:21.279758] W [fuse-bridge.c:490:fuse_entry_cbk] 0-glusterfs-fuse: 10521075: LOOKUP() /etc/samba/priv ate/msg.sock/6945 => -1 (Permission denied) [2018-10-05 13:41:21.279827] W [fuse-bridge.c:490:fuse_entry_cbk] 0-glusterfs-fuse: 10521076: LOOKUP() /etc/samba/priv ate/msg.sock/6945 => -1 (Permission denied) The link you sent is broken, but I think it should be: https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html-single/administration_guide/#sect-SMB_CTDB Thanks Diego On Thu, Oct 4, 2018, 09:16 Poornima Gurusiddaiah <pgurusid at redhat.com> wrote:> > > On Tue, Oct 2, 2018 at 5:26 PM Diego Remolina <dijuremo at gmail.com> wrote: > >> Dear all, >> >> I have a two node setup running on Centos and gluster version >> glusterfs-3.10.12-1.el7.x86_64 >> >> One of my nodes died (motherboard issue). Since I had to continue >> being up, I modified the quorum to below 50% to make sure I could >> still run on one server. >> >> The server runs ovirt and 2 VMs on top of a volume called vmstorage. I >> also had a third node in the peer list, but never configured it as an >> arbiter, so it just comes up in gluster v status. The server also run >> a file server with samba to serve files to windows machines. >> >> The issue is that since starting the server on it's own as the samba >> server, I am seeing permission denied errors for the "export" volume >> in /var/log/glusterfs/export.log >> >> The errors look like this and repeat over and over: >> >> [2018-10-02 11:46:56.327925] I [MSGID: 139001] >> [posix-acl.c:269:posix_acl_log_permit_denied] 0-posix-acl-autoload: >> client: -, gfid: 5b5bed22-ace0-410d-8623-4f1a31069b81, >> req(uid:1051,gid:513,perm:1,ngrps:2), >> ctx(uid:0,gid:0,in-groups:0,perm:700,updated-fop:LOOKUP, acl:-) >> [Permission denied] >> [2018-10-02 11:46:56.328004] W [fuse-bridge.c:490:fuse_entry_cbk] >> 0-glusterfs-fuse: 20599112: LOOKUP() /etc/samba/private/msg.sock/15149 >> => -1 (Permission denied) >> [2018-10-02 11:46:56.328185] W [fuse-bridge.c:490:fuse_entry_cbk] >> 0-glusterfs-fuse: 20599113: LOOKUP() /etc/samba/private/msg.sock/15149 >> => -1 (Permission denied) >> [2018-10-02 11:47:53.766562] W [fuse-bridge.c:490:fuse_entry_cbk] >> 0-glusterfs-fuse: 20600590: LOOKUP() /etc/samba/private/msg.sock/15149 >> => -1 (Permission denied) >> >> The gluster volume export is mounted on /export, samba and ctdb are >> instructed to use /export/etc/samba/private and /export/lock which is >> on the gluster file system for the clustered tdb, etc. However, I keep >> getting the log messages that fuse seems to try to access a folder >> that does not exist, /etc/samba/private/msg.sock >> > > This is an unconventional setup, the suggested way of clustering samba is > as mentioned in [1]. Sharing tdbs using gluster volume can lead to more > issues. Has the setup ever worked? Was this setup suggested somewhere? > > [1] > https://access.qa.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html-single/administration_guide/#sect-SMB_CTDB > > >> Why is this, how can I fix it? >> >> [root at ysmha01 export]# gluster v status export >> Status of volume: export >> Gluster process TCP Port RDMA Port Online >> Pid >> >> ------------------------------------------------------------------------------ >> Brick 10.0.1.6:/bricks/hdds/brick 49153 0 Y >> 3516 >> Self-heal Daemon on localhost N/A N/A Y >> 3710 >> Self-heal Daemon on 10.0.1.5 N/A N/A Y >> 4380 >> >> Task Status of Volume export >> >> ------------------------------------------------------------------------------ >> There are no active volume tasks >> >> These are all the volume options currently set: >> >> http://termbin.com/1xm5 >> >> Diego >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> https://lists.gluster.org/mailman/listinfo/gluster-users >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20181005/fb20e2b1/attachment.html>