Raghavendra Talur
2017-May-31 16:21 UTC
[Gluster-users] URGENT: Update issues from 3.6.6 to 3.10.2 Accessing files via samba come up with permission denied
Also, please attach your smb.conf. You can directly attach in the list and need not have a google drive link. On Wed, May 31, 2017 at 9:37 PM, Raghavendra Talur <rtalur at redhat.com> wrote:> Diego, > > I see that Samba is still using 3.6.6 Gluster. Is it possible you did > not restart smb after upgrading gluster(if samba is on same machine as > Gluster) or if you forgot to update Gluster client packages on Samba > node? > > On Wed, May 31, 2017 at 9:04 PM, Diego Remolina <dijuremo at gmail.com> wrote: >> Please download the log file from this link: >> >> https://drive.google.com/open?id=0B8EAPWIe4oyKN0h0X1pZVkRWVEU >> >> Let me know if you need any other log files. >> >> Diego >> >> On Wed, May 31, 2017 at 11:19 AM, Raghavendra Talur <rtalur at redhat.com> wrote: >>> If possible please share the glusterfs-* log files from /var/log/samba. >>> >>> This might be because of cluster.lookup-optimize. Adding Poornima and >>> Raghavendra Gowdappa to help with this. >>> >>> >>> On Wed, May 31, 2017 at 1:03 AM, Diego Remolina <dijuremo at gmail.com> wrote: >>>> This is a bit puzzling, not sure what difference it would make, but: >>>> >>>> 1. Try to open file that has a problem, ie. MyRevitFile.rvt >>>> Revit opens and shows a window that says access denied. >>>> >>>> 2. Rename file, i.e from windows explorer right click, rename to >>>> MyRevitFile2.rvt >>>> >>>> 3. Without opening the file, rename file back to the original, i.e >>>> MyRevitFile.rvt >>>> >>>> 4. Double click on file and now it will open just fine without the >>>> Access Denied error. >>>> >>>> Any explanation for this? Could the rename operation be forcing or >>>> updating some attributes that then allow the file to open? >>>> >>>> Diego >>>> >>>> On Tue, May 30, 2017 at 10:57 AM, Diego Remolina <dijuremo at gmail.com> wrote: >>>>> This is what I see in the logs set from smb.conf via line -> >>>>> glusterfs:logfile = /var/log/samba/glusterfs-projects.log >>>>> >>>>> [2017-05-30 14:52:31.051524] E [MSGID: 123001] >>>>> [io-cache.c:564:ioc_open_cbk] 0-export-io-cache: inode context is NULL >>>>> (a97bc9bb-68cf-4a69-aef7-39766b323c14) [Invalid argument] >>>>> [2017-05-30 14:52:31.241850] W [MSGID: 114031] >>>>> [client-rpc-fops.c:1100:client3_3_getxattr_cbk] 0-export-client-0: >>>>> remote operation failed. Path: >>>>> /projects/INACTIVE/WESTCOAST/Automotive/Acura/AS-Acura of Richmond/02 >>>>> DRAWINGS/02 ARCH/CrownAcura-SD02-ArchModel.rvt (a97bc9bb-68cf-4a69- >>>>> aef7-39766b323c14). Key: glusterfs.get_real_filename:desktop.ini [Not >>>>> a directory] >>>>> [2017-05-30 14:52:31.242956] W [MSGID: 114031] >>>>> [client-rpc-fops.c:1100:client3_3_getxattr_cbk] 0-export-client-1: >>>>> remote operation failed. Path: >>>>> /projects/INACTIVE/WESTCOAST/Automotive/Acura/AS-Acura of Richmond/02 >>>>> DRAWINGS/02 ARCH/CrownAcura-SD02-ArchModel.rvt (a97bc9bb-68cf-4a69- >>>>> aef7-39766b323c14). Key: glusterfs.get_real_filename:desktop.ini [Not >>>>> a directory] >>>>> >>>>> >>>>> On Tue, May 30, 2017 at 10:37 AM, Diego Remolina <dijuremo at gmail.com> wrote: >>>>>> Hi, >>>>>> >>>>>> Over the weekend we updated a two server glusterfs 3.6.6 install to >>>>>> 3.10.2 We also updated samba and samba-vfs to the latest in CentOS. I >>>>>> enabled several of the newer caching features from gluster 3.9 for >>>>>> small file performance and samba, and we now seem to have some issues >>>>>> with accessing files from glusterfs. When users try to access some >>>>>> files, they get a Permission denied message. This seems to be only via >>>>>> samba as I am able to su - username and then do strings on the file. >>>>>> >>>>>> [root at ysmha02 gluster-backups]# rpm -qa | grep gluster >>>>>> glusterfs-client-xlators-3.10.2-1.el7.x86_64 >>>>>> glusterfs-server-3.10.2-1.el7.x86_64 >>>>>> glusterfs-api-3.10.2-1.el7.x86_64 >>>>>> glusterfs-3.10.2-1.el7.x86_64 >>>>>> glusterfs-cli-3.10.2-1.el7.x86_64 >>>>>> centos-release-gluster310-1.0-1.el7.centos.noarch >>>>>> samba-vfs-glusterfs-4.4.4-14.el7_3.x86_64 >>>>>> glusterfs-fuse-3.10.2-1.el7.x86_64 >>>>>> glusterfs-libs-3.10.2-1.el7.x86_64 >>>>>> glusterfs-rdma-3.10.2-1.el7.x86_64 >>>>>> >>>>>> [root at ysmha02 gluster-backups]# rpm -qa | grep samba >>>>>> samba-common-libs-4.4.4-14.el7_3.x86_64 >>>>>> samba-common-tools-4.4.4-14.el7_3.x86_64 >>>>>> samba-libs-4.4.4-14.el7_3.x86_64 >>>>>> samba-4.4.4-14.el7_3.x86_64 >>>>>> samba-client-libs-4.4.4-14.el7_3.x86_64 >>>>>> samba-vfs-glusterfs-4.4.4-14.el7_3.x86_64 >>>>>> samba-common-4.4.4-14.el7_3.noarch >>>>>> >>>>>> On the samba logs for the machine I notice something weird, samba >>>>>> seems to be trying to stat the file we are trying as a directory to >>>>>> see if it contains desktop.ini: >>>>>> >>>>>> [2017/05/30 10:13:07.297026, 0] >>>>>> ../source3/modules/vfs_glusterfs.c:870(vfs_gluster_stat) >>>>>> glfs_stat(ACTIVE/Automotive/FORD/AN - Ford East/02 >>>>>> DRAWINGS/CURRENT/AN-FORD EAST_04-05-17_CD_R17.rvt/desktop.ini) failed: >>>>>> Not a directory >>>>>> [2017/05/30 10:13:07.298155, 0] >>>>>> ../source3/modules/vfs_glusterfs.c:870(vfs_gluster_stat) >>>>>> glfs_stat(ACTIVE/Automotive/FORD/AN - Ford East/02 >>>>>> DRAWINGS/CURRENT/AN-FORD EAST_04-05-17_CD_R17.rvt/desktop.ini) failed: >>>>>> Not a directory >>>>>> >>>>>> This seems to be happening only with files with the .rvt extension. >>>>>> Though these files are usually larger in size vs other smaller excel, >>>>>> power point, etc files. >>>>>> >>>>>> Here are the complete options for the volume: >>>>>> >>>>>> https://pastebin.com/ZH2vMsMN >>>>>> >>>>>> I turned off performance.cache-samba-metadata again to see if that >>>>>> would help, but seems it does not help. >>>>>> >>>>>> I really appreciate any help with this. >>>>>> >>>>>> DIego >>>> _______________________________________________ >>>> Gluster-users mailing list >>>> Gluster-users at gluster.org >>>> http://lists.gluster.org/mailman/listinfo/gluster-users
Diego Remolina
2017-May-31 16:39 UTC
[Gluster-users] URGENT: Update issues from 3.6.6 to 3.10.2 Accessing files via samba come up with permission denied
Samba is running in the same machine as glusterd. The machines were rebooted after the upgrades and samba has been restarted a few times. # rpm -qa | grep gluster glusterfs-client-xlators-3.10.2-1.el7.x86_64 glusterfs-server-3.10.2-1.el7.x86_64 glusterfs-api-3.10.2-1.el7.x86_64 glusterfs-3.10.2-1.el7.x86_64 glusterfs-cli-3.10.2-1.el7.x86_64 centos-release-gluster310-1.0-1.el7.centos.noarch samba-vfs-glusterfs-4.4.4-14.el7_3.x86_64 glusterfs-fuse-3.10.2-1.el7.x86_64 glusterfs-libs-3.10.2-1.el7.x86_64 glusterfs-rdma-3.10.2-1.el7.x86_64 # rpm -qa | grep samba samba-common-libs-4.4.4-14.el7_3.x86_64 samba-common-tools-4.4.4-14.el7_3.x86_64 samba-libs-4.4.4-14.el7_3.x86_64 samba-4.4.4-14.el7_3.x86_64 samba-client-libs-4.4.4-14.el7_3.x86_64 samba-vfs-glusterfs-4.4.4-14.el7_3.x86_64 samba-common-4.4.4-14.el7_3.noarch # cat /etc/redhat-release CentOS Linux release 7.3.1611 (Core) I also raised the op version. # gluster volume get all cluster.op-version Option Value ------ ----- cluster.op-version 31000 # gluster volume get all cluster.max-op-version Option Value ------ ----- cluster.max-op-version 31000 On Wed, May 31, 2017 at 12:21 PM, Raghavendra Talur <rtalur at redhat.com> wrote:> Also, please attach your smb.conf. You can directly attach in the list > and need not have a google drive link. > > > On Wed, May 31, 2017 at 9:37 PM, Raghavendra Talur <rtalur at redhat.com> wrote: >> Diego, >> >> I see that Samba is still using 3.6.6 Gluster. Is it possible you did >> not restart smb after upgrading gluster(if samba is on same machine as >> Gluster) or if you forgot to update Gluster client packages on Samba >> node? >> >> On Wed, May 31, 2017 at 9:04 PM, Diego Remolina <dijuremo at gmail.com> wrote: >>> Please download the log file from this link: >>> >>> https://drive.google.com/open?id=0B8EAPWIe4oyKN0h0X1pZVkRWVEU >>> >>> Let me know if you need any other log files. >>> >>> Diego >>> >>> On Wed, May 31, 2017 at 11:19 AM, Raghavendra Talur <rtalur at redhat.com> wrote: >>>> If possible please share the glusterfs-* log files from /var/log/samba. >>>> >>>> This might be because of cluster.lookup-optimize. Adding Poornima and >>>> Raghavendra Gowdappa to help with this. >>>> >>>> >>>> On Wed, May 31, 2017 at 1:03 AM, Diego Remolina <dijuremo at gmail.com> wrote: >>>>> This is a bit puzzling, not sure what difference it would make, but: >>>>> >>>>> 1. Try to open file that has a problem, ie. MyRevitFile.rvt >>>>> Revit opens and shows a window that says access denied. >>>>> >>>>> 2. Rename file, i.e from windows explorer right click, rename to >>>>> MyRevitFile2.rvt >>>>> >>>>> 3. Without opening the file, rename file back to the original, i.e >>>>> MyRevitFile.rvt >>>>> >>>>> 4. Double click on file and now it will open just fine without the >>>>> Access Denied error. >>>>> >>>>> Any explanation for this? Could the rename operation be forcing or >>>>> updating some attributes that then allow the file to open? >>>>> >>>>> Diego >>>>> >>>>> On Tue, May 30, 2017 at 10:57 AM, Diego Remolina <dijuremo at gmail.com> wrote: >>>>>> This is what I see in the logs set from smb.conf via line -> >>>>>> glusterfs:logfile = /var/log/samba/glusterfs-projects.log >>>>>> >>>>>> [2017-05-30 14:52:31.051524] E [MSGID: 123001] >>>>>> [io-cache.c:564:ioc_open_cbk] 0-export-io-cache: inode context is NULL >>>>>> (a97bc9bb-68cf-4a69-aef7-39766b323c14) [Invalid argument] >>>>>> [2017-05-30 14:52:31.241850] W [MSGID: 114031] >>>>>> [client-rpc-fops.c:1100:client3_3_getxattr_cbk] 0-export-client-0: >>>>>> remote operation failed. Path: >>>>>> /projects/INACTIVE/WESTCOAST/Automotive/Acura/AS-Acura of Richmond/02 >>>>>> DRAWINGS/02 ARCH/CrownAcura-SD02-ArchModel.rvt (a97bc9bb-68cf-4a69- >>>>>> aef7-39766b323c14). Key: glusterfs.get_real_filename:desktop.ini [Not >>>>>> a directory] >>>>>> [2017-05-30 14:52:31.242956] W [MSGID: 114031] >>>>>> [client-rpc-fops.c:1100:client3_3_getxattr_cbk] 0-export-client-1: >>>>>> remote operation failed. Path: >>>>>> /projects/INACTIVE/WESTCOAST/Automotive/Acura/AS-Acura of Richmond/02 >>>>>> DRAWINGS/02 ARCH/CrownAcura-SD02-ArchModel.rvt (a97bc9bb-68cf-4a69- >>>>>> aef7-39766b323c14). Key: glusterfs.get_real_filename:desktop.ini [Not >>>>>> a directory] >>>>>> >>>>>> >>>>>> On Tue, May 30, 2017 at 10:37 AM, Diego Remolina <dijuremo at gmail.com> wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Over the weekend we updated a two server glusterfs 3.6.6 install to >>>>>>> 3.10.2 We also updated samba and samba-vfs to the latest in CentOS. I >>>>>>> enabled several of the newer caching features from gluster 3.9 for >>>>>>> small file performance and samba, and we now seem to have some issues >>>>>>> with accessing files from glusterfs. When users try to access some >>>>>>> files, they get a Permission denied message. This seems to be only via >>>>>>> samba as I am able to su - username and then do strings on the file. >>>>>>> >>>>>>> [root at ysmha02 gluster-backups]# rpm -qa | grep gluster >>>>>>> glusterfs-client-xlators-3.10.2-1.el7.x86_64 >>>>>>> glusterfs-server-3.10.2-1.el7.x86_64 >>>>>>> glusterfs-api-3.10.2-1.el7.x86_64 >>>>>>> glusterfs-3.10.2-1.el7.x86_64 >>>>>>> glusterfs-cli-3.10.2-1.el7.x86_64 >>>>>>> centos-release-gluster310-1.0-1.el7.centos.noarch >>>>>>> samba-vfs-glusterfs-4.4.4-14.el7_3.x86_64 >>>>>>> glusterfs-fuse-3.10.2-1.el7.x86_64 >>>>>>> glusterfs-libs-3.10.2-1.el7.x86_64 >>>>>>> glusterfs-rdma-3.10.2-1.el7.x86_64 >>>>>>> >>>>>>> [root at ysmha02 gluster-backups]# rpm -qa | grep samba >>>>>>> samba-common-libs-4.4.4-14.el7_3.x86_64 >>>>>>> samba-common-tools-4.4.4-14.el7_3.x86_64 >>>>>>> samba-libs-4.4.4-14.el7_3.x86_64 >>>>>>> samba-4.4.4-14.el7_3.x86_64 >>>>>>> samba-client-libs-4.4.4-14.el7_3.x86_64 >>>>>>> samba-vfs-glusterfs-4.4.4-14.el7_3.x86_64 >>>>>>> samba-common-4.4.4-14.el7_3.noarch >>>>>>> >>>>>>> On the samba logs for the machine I notice something weird, samba >>>>>>> seems to be trying to stat the file we are trying as a directory to >>>>>>> see if it contains desktop.ini: >>>>>>> >>>>>>> [2017/05/30 10:13:07.297026, 0] >>>>>>> ../source3/modules/vfs_glusterfs.c:870(vfs_gluster_stat) >>>>>>> glfs_stat(ACTIVE/Automotive/FORD/AN - Ford East/02 >>>>>>> DRAWINGS/CURRENT/AN-FORD EAST_04-05-17_CD_R17.rvt/desktop.ini) failed: >>>>>>> Not a directory >>>>>>> [2017/05/30 10:13:07.298155, 0] >>>>>>> ../source3/modules/vfs_glusterfs.c:870(vfs_gluster_stat) >>>>>>> glfs_stat(ACTIVE/Automotive/FORD/AN - Ford East/02 >>>>>>> DRAWINGS/CURRENT/AN-FORD EAST_04-05-17_CD_R17.rvt/desktop.ini) failed: >>>>>>> Not a directory >>>>>>> >>>>>>> This seems to be happening only with files with the .rvt extension. >>>>>>> Though these files are usually larger in size vs other smaller excel, >>>>>>> power point, etc files. >>>>>>> >>>>>>> Here are the complete options for the volume: >>>>>>> >>>>>>> https://pastebin.com/ZH2vMsMN >>>>>>> >>>>>>> I turned off performance.cache-samba-metadata again to see if that >>>>>>> would help, but seems it does not help. >>>>>>> >>>>>>> I really appreciate any help with this. >>>>>>> >>>>>>> DIego >>>>> _______________________________________________ >>>>> Gluster-users mailing list >>>>> Gluster-users at gluster.org >>>>> http://lists.gluster.org/mailman/listinfo/gluster-users-------------- next part -------------- A non-text attachment was scrubbed... Name: smb.conf Type: application/octet-stream Size: 7248 bytes Desc: not available URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170531/67eeb521/attachment.obj>