Felix Kölzow
2019-Oct-20 16:26 UTC
[Gluster-users] dispersed volume + cifs export does not work (replicated + cifs works fine)
Dear Gluster-Users, _ _ _short story:_ _ _ Two volumes are exported via smb/cifs with the (almost) the same configuration with respect to smb.conf. The replicated volume is easily accessible via cifs and fuse. The dispersed volume is accessible via fuse, but not via cifs. Error message from windows client: The parameter is incorrect Maybe the error is somehow related to this: https://gluster-users.gluster.narkive.com/g35gmGj6/vfs-gluster-broken _more information: _ We have created a gluster setup that consists of three servers, and each server provides two bricks. Two volumes are created on these bricks and are going to exported via smb/cifs. * replicated distributed * dispersed The volume settings are given here: [root at node1 ~]# gluster volume info replicated_cifs Volume Name: replicated_cifs Type: Distributed-Replicate Volume ID: 51bb4440-3b8e-48be-a84c-5ea9e1ddd38e Status: Started Snapshot Count: 0 Number of Bricks: 2 x 3 = 6 Transport-type: tcp Bricks: Brick1: node1:/gluster/vg00/replicated_cifs/brick Brick2: node2:/gluster/vg00/replicated_cifs/brick Brick3: node3:/gluster/vg00/replicated_cifs/brick Brick4: node1:/gluster/vg01/replicated_cifs/brick Brick5: node2:/gluster/vg01/replicated_cifs/brick Brick6: node3:/gluster/vg01/replicated_cifs/brick Options Reconfigured: features.show-snapshot-directory: on features.uss: enable features.barrier: disable changelog.changelog: on geo-replication.ignore-pid-check: on geo-replication.indexing: on transport.address-family: inet nfs.disable: on performance.client-io-threads: off diagnostics.latency-measurement: on diagnostics.count-fop-hits: on user.cifs: enable features.cache-invalidation: on features.cache-invalidation-timeout: 600 performance.cache-samba-metadata: on performance.stat-prefetch: disable performance.cache-invalidation: on performance.md-cache-timeout: 600 network.inode-lru-limit: 200000 performance.nl-cache: on performance.nl-cache-timeout: 600 performance.readdir-ahead: on performance.parallel-readdir: on client.event-threads: 4 server.event-threads: 4 server.root-squash: off cluster.lookup-optimize: on features.quota: on features.inode-quota: on features.quota-deem-statfs: on performance.cache-size: 10GB cluster.server-quorum-ratio: 51% cluster.enable-shared-storage: enable [root at node1 ~]# gluster volume info dispersed_cifs Volume Name: dispersed_cifs Type: Disperse Volume ID: 0a291429-1875-41c8-96ff-bce0054ed309 Status: Started Snapshot Count: 0 Number of Bricks: 1 x (4 + 2) = 6 Transport-type: tcp Bricks: Brick1: node1:/gluster/vg00/dispersed_cifs/brick Brick2: node2:/gluster/vg00/dispersed_cifs/brick Brick3: node3:/gluster/vg00/dispersed_cifs/brick Brick4: node1:/gluster/vg01/dispersed_cifs/brick Brick5: node2:/gluster/vg01/dispersed_cifs/brick Brick6: node3:/gluster/vg01/dispersed_cifs/brick Options Reconfigured: changelog.changelog: on geo-replication.ignore-pid-check: on geo-replication.indexing: on transport.address-family: inet nfs.disable: on diagnostics.latency-measurement: on diagnostics.count-fop-hits: on user.cifs: enable features.cache-invalidation: on features.cache-invalidation-timeout: 600 performance.cache-samba-metadata: on performance.stat-prefetch: disable performance.cache-invalidation: on performance.md-cache-timeout: 600 network.inode-lru-limit: 200000 performance.nl-cache: on performance.nl-cache-timeout: 600 performance.readdir-ahead: on performance.parallel-readdir: on server.event-threads: 4 client.event-threads: 4 server.root-squash: off cluster.lookup-optimize: on features.quota: on features.inode-quota: on features.quota-deem-statfs: on performance.cache-size: 10GB cluster.server-quorum-ratio: 51% cluster.enable-shared-storage: enable The export via cifs looks: _distributed replicated:_ [gluster-replicated_cifs] vfs objects = fruit acl_xattr glusterfs acl_xattr:ignore system acls = yes acl_xattr:default acl style = windows glusterfs:volume = replicated_cifs glusterfs:logfile = /var/log/samba/glusterfs-replicated_cifs.%M.log glusterfs:loglevel = 7 kernel share modes = no path = / read only = no guest ok = no browseable = no [replicated_data] vfs objects = fruit acl_xattr shadow_copy2 glusterfs acl_xattr:ignore system acls = yes acl_xattr:default acl style = windows glusterfs:volume = replicated_cifs glusterfs:logfile = /var/log/samba/glusterfs-replicated_data.%M.log glusterfs:loglevel = 7 kernel share modes = no path = /replicated_data read only = no guest ok = no create mask = 0660 directory mask = 0770 map acl inherit = yes inherit permissions = yes inherit acls = true store dos attributes = yes shadow:snapdir = /.snaps shadow:basedir = / shadow:sort = desc shadow:snapprefix = snap_replicated_cifs shadow:format = _GMT-%Y.%m.%d-%H.%M.%S _dispersed volume:_ [gluster-dispersed_cifs] vfs objects = fruit acl_xattr glusterfs acl_xattr:ignore system acls = yes acl_xattr:default acl style = windows glusterfs:volume = dispersed_cifs glusterfs:logfile = /var/log/samba/glusterfs-dispersed_cifs.%M.log glusterfs:loglevel = 7 kernel share modes = no path = / read only = no guest ok = no browseable = no [dispersed_data] vfs objects = fruit acl_xattr shadow_copy2 glusterfs acl_xattr:ignore system acls = yes acl_xattr:default acl style = windows glusterfs:volume = dispersed_cifs glusterfs:logfile = /var/log/samba/glusterfs-dispersed_data.%M.log glusterfs:loglevel = 7 kernel share modes = no path = /dispersed_data read only = no guest ok = no create mask = 0660 directory mask = 0770 map acl inherit = yes inherit permissions = yes inherit acls = true store dos attributes = yes shadow:snapdir = /.snaps shadow:basedir = / shadow:sort = desc shadow:snapprefix = snap_dispersed_cifs shadow:format = _GMT-%Y.%m.%d-%H.%M.%S Any help or hints are appreciated to resolve that issue. Thanks in advance. Felix -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20191020/6079933a/attachment.html>
Felix Kölzow
2020-Mar-10 20:00 UTC
[Gluster-users] dispersed volume + cifs export does not work (replicated + cifs works fine)
To answer my one question and maybe it is helpful for the community: Due to a different (minor) issue with the posix-permissions, we set stat-prefetch to off. This causes the issue mentioned below, i.e. it had nothing to do with the smb.conf setting. gluster volume set volname performance.stat-prefetch on solved that issue. Reference: https://access.redhat.com/solutions/4558341 On 20/10/2019 18:26, Felix K?lzow wrote:> > Dear Gluster-Users, > > _ > _ > > _short story:_ > > _ > _ > > Two volumes are exported via smb/cifs with the > > (almost) the same configuration with respect to smb.conf. The > replicated volume > > is easily accessible via cifs and fuse. The dispersed volume is > accessible via fuse, > > but not via cifs. > > Error message from windows client: > > The parameter is incorrect > > > > Maybe the error is somehow related to this: > > https://gluster-users.gluster.narkive.com/g35gmGj6/vfs-gluster-broken > > > _more information: > _ > > > We have created a gluster setup that consists of three servers, and > each server > > provides two bricks. Two volumes are created on these bricks and are > going to exported via > > smb/cifs. > > > * replicated distributed > * dispersed > > > The volume settings are given here: > > > [root at node1 ~]# gluster volume info replicated_cifs > > Volume Name: replicated_cifs > Type: Distributed-Replicate > Volume ID: 51bb4440-3b8e-48be-a84c-5ea9e1ddd38e > Status: Started > Snapshot Count: 0 > Number of Bricks: 2 x 3 = 6 > Transport-type: tcp > Bricks: > Brick1: node1:/gluster/vg00/replicated_cifs/brick > Brick2: node2:/gluster/vg00/replicated_cifs/brick > Brick3: node3:/gluster/vg00/replicated_cifs/brick > Brick4: node1:/gluster/vg01/replicated_cifs/brick > Brick5: node2:/gluster/vg01/replicated_cifs/brick > Brick6: node3:/gluster/vg01/replicated_cifs/brick > Options Reconfigured: > features.show-snapshot-directory: on > features.uss: enable > features.barrier: disable > changelog.changelog: on > geo-replication.ignore-pid-check: on > geo-replication.indexing: on > transport.address-family: inet > nfs.disable: on > performance.client-io-threads: off > diagnostics.latency-measurement: on > diagnostics.count-fop-hits: on > user.cifs: enable > features.cache-invalidation: on > features.cache-invalidation-timeout: 600 > performance.cache-samba-metadata: on > performance.stat-prefetch: disable > performance.cache-invalidation: on > performance.md-cache-timeout: 600 > network.inode-lru-limit: 200000 > performance.nl-cache: on > performance.nl-cache-timeout: 600 > performance.readdir-ahead: on > performance.parallel-readdir: on > client.event-threads: 4 > server.event-threads: 4 > server.root-squash: off > cluster.lookup-optimize: on > features.quota: on > features.inode-quota: on > features.quota-deem-statfs: on > performance.cache-size: 10GB > cluster.server-quorum-ratio: 51% > cluster.enable-shared-storage: enable > > > [root at node1 ~]# gluster volume info dispersed_cifs > > Volume Name: dispersed_cifs > Type: Disperse > Volume ID: 0a291429-1875-41c8-96ff-bce0054ed309 > Status: Started > Snapshot Count: 0 > Number of Bricks: 1 x (4 + 2) = 6 > Transport-type: tcp > Bricks: > Brick1: node1:/gluster/vg00/dispersed_cifs/brick > Brick2: node2:/gluster/vg00/dispersed_cifs/brick > Brick3: node3:/gluster/vg00/dispersed_cifs/brick > Brick4: node1:/gluster/vg01/dispersed_cifs/brick > Brick5: node2:/gluster/vg01/dispersed_cifs/brick > Brick6: node3:/gluster/vg01/dispersed_cifs/brick > Options Reconfigured: > changelog.changelog: on > geo-replication.ignore-pid-check: on > geo-replication.indexing: on > transport.address-family: inet > nfs.disable: on > diagnostics.latency-measurement: on > diagnostics.count-fop-hits: on > user.cifs: enable > features.cache-invalidation: on > features.cache-invalidation-timeout: 600 > performance.cache-samba-metadata: on > performance.stat-prefetch: disable > performance.cache-invalidation: on > performance.md-cache-timeout: 600 > network.inode-lru-limit: 200000 > performance.nl-cache: on > performance.nl-cache-timeout: 600 > performance.readdir-ahead: on > performance.parallel-readdir: on > server.event-threads: 4 > client.event-threads: 4 > server.root-squash: off > cluster.lookup-optimize: on > features.quota: on > features.inode-quota: on > features.quota-deem-statfs: on > performance.cache-size: 10GB > cluster.server-quorum-ratio: 51% > cluster.enable-shared-storage: enable > > > The export via cifs looks: > > _distributed replicated:_ > > [gluster-replicated_cifs] > vfs objects = fruit acl_xattr glusterfs > acl_xattr:ignore system acls = yes > acl_xattr:default acl style = windows > glusterfs:volume = replicated_cifs > glusterfs:logfile = /var/log/samba/glusterfs-replicated_cifs.%M.log > glusterfs:loglevel = 7 > kernel share modes = no > path = / > read only = no > guest ok = no > browseable = no > > [replicated_data] > vfs objects = fruit acl_xattr shadow_copy2 glusterfs > acl_xattr:ignore system acls = yes > acl_xattr:default acl style = windows > glusterfs:volume = replicated_cifs > glusterfs:logfile = /var/log/samba/glusterfs-replicated_data.%M.log > glusterfs:loglevel = 7 > kernel share modes = no > path = /replicated_data > read only = no > guest ok = no > create mask = 0660 > directory mask = 0770 > map acl inherit = yes > inherit permissions = yes > inherit acls = true > store dos attributes = yes > shadow:snapdir = /.snaps > shadow:basedir = / > shadow:sort = desc > shadow:snapprefix = snap_replicated_cifs > shadow:format = _GMT-%Y.%m.%d-%H.%M.%S > > > _dispersed volume:_ > > [gluster-dispersed_cifs] > vfs objects = fruit acl_xattr glusterfs > acl_xattr:ignore system acls = yes > acl_xattr:default acl style = windows > glusterfs:volume = dispersed_cifs > glusterfs:logfile = /var/log/samba/glusterfs-dispersed_cifs.%M.log > glusterfs:loglevel = 7 > kernel share modes = no > path = / > read only = no > guest ok = no > browseable = no > > [dispersed_data] > vfs objects = fruit acl_xattr shadow_copy2 glusterfs > acl_xattr:ignore system acls = yes > acl_xattr:default acl style = windows > glusterfs:volume = dispersed_cifs > glusterfs:logfile = /var/log/samba/glusterfs-dispersed_data.%M.log > glusterfs:loglevel = 7 > kernel share modes = no > path = /dispersed_data > read only = no > guest ok = no > create mask = 0660 > directory mask = 0770 > map acl inherit = yes > inherit permissions = yes > inherit acls = true > store dos attributes = yes > shadow:snapdir = /.snaps > shadow:basedir = / > shadow:sort = desc > shadow:snapprefix = snap_dispersed_cifs > shadow:format = _GMT-%Y.%m.%d-%H.%M.%S > > Any help or hints are appreciated to resolve that issue. > > Thanks in advance. > > Felix > > > > > > ________ > > Community Meeting Calendar: > > APAC Schedule - > Every 2nd and 4th Tuesday at 11:30 AM IST > Bridge: https://bluejeans.com/118564314 > > NA/EMEA Schedule - > Every 1st and 3rd Tuesday at 01:00 PM EDT > Bridge: https://bluejeans.com/118564314 > > Gluster-users mailing list > Gluster-users at gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20200310/1f29db7a/attachment.html>