Hello,
we are trying to configure a CTDB-Cluster with Glusterfs. We are using
Samba 4.5 together with gluster 3.9. We set up a lvm2 thin-provisioned
volume to use gluster-snapshots.
Then we configured the first share without using shadow_copy2 and
everything was working fine.
Then we added the shadow_copy2 parameters, when we did a "smbclient"
we
got the following message:
root at ng-data1:~# smbclient -m SMB3 //ctdb/gluster -k
smb: \> ls
NT_STATUS_NOT_FOUND listing \*
On a windows-system we got the message "Element not found"
Here is our smb.conf:
-------------------
[global]
workgroup = uni
netbios name = ctdb
security = ads
realm = UNI.DE
idmap config *: range = 10000-19999
idmap config uni: range = 1000000-1999999
idmap config uni: backend = rid
winbind use default domain = yes
winbind refresh tickets = yes
store dos attributes = yes
map acl inherit = yes
vfs objects = acl_xattr
server min protocol = SMB2
[gluster]
comment = Daten im Cluster
guest ok = no
read only = no
vfs objects = acl_xattr glusterfs shadow_copy2
glusterfs:volume = gv1
glusterfs:logfile = /var/log/samba/gluster-gv1.log
glusterfs:loglevel = 10
gluster:volfile_server = localhost
kernel share modes = no
path = /win-share
shadow:snapdir = /win-share/.snaps
shadow:basedir = /win-share
shadow:sort = desc
shadow:snapprefix = ^S[A-Za-z0-9]*p$
shadow:format = _GMT-%Y.%m.%d-%H.%M.%S
-------------
when we try to excess the share we get the following errormessage insite
the logfile:
--------------------
[2017-01-03 13:55:14.757049] T [rpc-clnt.c:1659:rpc_clnt_submit]
0-rpc-clnt: submitted request (XID: 0x4a Program: GlusterFS 3.3,
ProgVers: 330, Proc: 18) to rpc-transport (gv1-client-1
)
[2017-01-03 13:55:14.757291] T [rpc-clnt.c:687:rpc_clnt_reply_init]
0-gv1-client-2: received rpc message (RPC XID: 0x4b Program: GlusterFS
3.3, ProgVers: 330, Proc: 18) from rpc-transpo
rt (gv1-client-2)
[2017-01-03 13:55:14.757320] T [rpc-clnt.c:687:rpc_clnt_reply_init]
0-gv1-client-1: received rpc message (RPC XID: 0x4a Program: GlusterFS
3.3, ProgVers: 330, Proc: 18) from rpc-transpo
rt (gv1-client-1)
[2017-01-03 13:55:14.757347] D [MSGID: 0]
[client-rpc-fops.c:1095:client3_3_getxattr_cbk] 0-gv1-client-2: remote
operation failed: No data available. Path: /win-share/. (0dafa3a2-5928-4
b05-8cb2-bd90ac84d7c4). Key: security.NTACL
[2017-01-03 13:55:14.757355] D [MSGID: 0]
[client-rpc-fops.c:1095:client3_3_getxattr_cbk] 0-gv1-client-1: remote
operation failed: No data available. Path: /win-share/. (0dafa3a2-5928-4
b05-8cb2-bd90ac84d7c4). Key: security.NTACL
[2017-01-03 13:55:14.757378] D [MSGID: 0]
[client-rpc-fops.c:1106:client3_3_getxattr_cbk] 0-stack-trace:
stack-address: 0x7fd99d682310, gv1-client-1 returned -1 error: No data
available
[No data available]
[2017-01-03 13:55:14.757378] D [MSGID: 0]
[client-rpc-fops.c:1106:client3_3_getxattr_cbk] 0-stack-trace:
stack-address: 0x7fd99d682310, gv1-client-2 returned -1 error: No data
available
[No data available]
[2017-01-03 13:55:14.757400] D [MSGID: 0]
[afr-inode-read.c:505:afr_getxattr_cbk] 0-stack-trace: stack-address:
0x7fd99d682310, gv1-replicate-0 returned -1 error: No data available [No
data available]
[2017-01-03 13:55:14.757409] D [MSGID: 0]
[afr-inode-read.c:505:afr_getxattr_cbk] 0-stack-trace: stack-address:
0x7fd99d682310, gv1-replicate-1 returned -1 error: No data available [No
data available]
[2017-01-03 13:55:14.757429] D [MSGID: 0]
[dht-common.c:3032:dht_getxattr_cbk] 0-stack-trace: stack-address:
0x7fd99d682310, gv1-dht returned -1 error: No data available [No data availa
ble]
[2017-01-03 13:55:14.757445] D [MSGID: 0]
[defaults.c:1346:default_getxattr_cbk] 0-stack-trace: stack-address:
0x7fd99d682310, gv1-io-threads returned -1 error: No data available [No da
ta available]
[2017-01-03 13:55:14.757461] D [MSGID: 0]
[io-stats.c:2349:io_stats_getxattr_cbk] 0-stack-trace: stack-address:
0x7fd99d682310, gv1 returned -1 error: No data available [No data availab
le]
--------------------
When we do a "gluster snapshot create Snap1 gv1" we can create and use
the snapshot on the commandline, so the snapshot it self is working.
Problem seams to be the settings for shadow_copy2.
Any help will help ;-)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.samba.org/pipermail/samba/attachments/20170103/593a46e1/signature.sig>
On Tue, 2017-01-03 at 15:16 +0100, Stefan Kania via samba wrote:> Hello, > > we are trying to configure a CTDB-Cluster with Glusterfs. We are using > Samba 4.5 together with gluster 3.9. We set up a lvm2 thin-provisioned > volume to use gluster-snapshots. > Then we configured the first share without using shadow_copy2 and > everything was working fine. > > Then we added the shadow_copy2 parameters, when we did a "smbclient" we > got the following message: > > root at ng-data1:~# smbclient -m SMB3 //ctdb/gluster -k > smb: \> ls > NT_STATUS_NOT_FOUND listing \* > > On a windows-system we got the message "Element not found" > > Here is our smb.conf: > ------------------- > [global] > workgroup = uni > netbios name = ctdb > security = ads > realm = UNI.DE > idmap config *: range = 10000-19999 > idmap config uni: range = 1000000-1999999 > idmap config uni: backend = rid > winbind use default domain = yes > winbind refresh tickets = yes > store dos attributes = yes > map acl inherit = yes > vfs objects = acl_xattr > server min protocol = SMB2 > > > [gluster] > comment = Daten im Cluster > guest ok = no > read only = no > vfs objects = acl_xattr glusterfs shadow_copy2Assuming you have configured CTDB-GlusterFS cluster in the right way, one correction to your smb.conf here is that glusterfs module should be always the last one in vfs objects list. See the following snippet from man page for vfs_glusterfs(8): . . . This module can be combined with other modules, but it should be the last module in the vfs objects list. Modules added to this list to the right of the glusterfs entry may not have any effect at all. . . . Even then we do have an issue when vfs_shadow_copy2 module is used along with vfs_glusterfs as explained in the following thread: https://lists.samba.org/archive/samba-technical/2016-October/116834.html If you encounter this issue you can workaround it by setting mountpoint parameter for shadow_copy2 module to / as mentioned in the above link.> glusterfs:volume = gv1 > glusterfs:logfile = /var/log/samba/gluster-gv1.log > glusterfs:loglevel = 10 > gluster:volfile_server = localhost > kernel share modes = no > path = /win-share > shadow:snapdir = /win-share/.snaps > shadow:basedir = /win-share > shadow:sort = desc > shadow:snapprefix = ^S[A-Za-z0-9]*p$ > shadow:format = _GMT-%Y.%m.%d-%H.%M.%S > ------------- > > when we try to excess the share we get the following errormessage insite > the logfile: > -------------------- > [2017-01-03 13:55:14.757049] T [rpc-clnt.c:1659:rpc_clnt_submit] > 0-rpc-clnt: submitted request (XID: 0x4a Program: GlusterFS 3.3, > ProgVers: 330, Proc: 18) to rpc-transport (gv1-client-1 > ) > [2017-01-03 13:55:14.757291] T [rpc-clnt.c:687:rpc_clnt_reply_init] > 0-gv1-client-2: received rpc message (RPC XID: 0x4b Program: GlusterFS > 3.3, ProgVers: 330, Proc: 18) from rpc-transpo > rt (gv1-client-2) > [2017-01-03 13:55:14.757320] T [rpc-clnt.c:687:rpc_clnt_reply_init] > 0-gv1-client-1: received rpc message (RPC XID: 0x4a Program: GlusterFS > 3.3, ProgVers: 330, Proc: 18) from rpc-transpo > rt (gv1-client-1) > [2017-01-03 13:55:14.757347] D [MSGID: 0] > [client-rpc-fops.c:1095:client3_3_getxattr_cbk] 0-gv1-client-2: remote > operation failed: No data available. Path: /win-share/. (0dafa3a2-5928-4 > b05-8cb2-bd90ac84d7c4). Key: security.NTACL > [2017-01-03 13:55:14.757355] D [MSGID: 0] > [client-rpc-fops.c:1095:client3_3_getxattr_cbk] 0-gv1-client-1: remote > operation failed: No data available. Path: /win-share/. (0dafa3a2-5928-4 > b05-8cb2-bd90ac84d7c4). Key: security.NTACL > [2017-01-03 13:55:14.757378] D [MSGID: 0] > [client-rpc-fops.c:1106:client3_3_getxattr_cbk] 0-stack-trace: > stack-address: 0x7fd99d682310, gv1-client-1 returned -1 error: No data > available > [No data available] > [2017-01-03 13:55:14.757378] D [MSGID: 0] > [client-rpc-fops.c:1106:client3_3_getxattr_cbk] 0-stack-trace: > stack-address: 0x7fd99d682310, gv1-client-2 returned -1 error: No data > available > [No data available] > [2017-01-03 13:55:14.757400] D [MSGID: 0] > [afr-inode-read.c:505:afr_getxattr_cbk] 0-stack-trace: stack-address: > 0x7fd99d682310, gv1-replicate-0 returned -1 error: No data available [No > data available] > [2017-01-03 13:55:14.757409] D [MSGID: 0] > [afr-inode-read.c:505:afr_getxattr_cbk] 0-stack-trace: stack-address: > 0x7fd99d682310, gv1-replicate-1 returned -1 error: No data available [No > data available] > [2017-01-03 13:55:14.757429] D [MSGID: 0] > [dht-common.c:3032:dht_getxattr_cbk] 0-stack-trace: stack-address: > 0x7fd99d682310, gv1-dht returned -1 error: No data available [No data availa > ble] > [2017-01-03 13:55:14.757445] D [MSGID: 0] > [defaults.c:1346:default_getxattr_cbk] 0-stack-trace: stack-address: > 0x7fd99d682310, gv1-io-threads returned -1 error: No data available [No da > ta available] > [2017-01-03 13:55:14.757461] D [MSGID: 0] > [io-stats.c:2349:io_stats_getxattr_cbk] 0-stack-trace: stack-address: > 0x7fd99d682310, gv1 returned -1 error: No data available [No data availab > le] > -------------------- > When we do a "gluster snapshot create Snap1 gv1" we can create and use > the snapshot on the commandline, so the snapshot it self is working. > Problem seams to be the settings for shadow_copy2. > > Any help will help ;-) >-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 299 bytes Desc: This is a digitally signed message part URL: <http://lists.samba.org/pipermail/samba/attachments/20170104/358e3365/signature.sig>
Am 04.01.2017 um 07:47 schrieb Anoop C S via samba:> On Tue, 2017-01-03 at 15:16 +0100, Stefan Kania via samba wrote: >> Hello, >> >> we are trying to configure a CTDB-Cluster with Glusterfs. We are using >> Samba 4.5 together with gluster 3.9. We set up a lvm2 thin-provisioned >> volume to use gluster-snapshots. >> Then we configured the first share without using shadow_copy2 and >> everything was working fine. >> >> Then we added the shadow_copy2 parameters, when we did a "smbclient" we >> got the following message: >> >> root at ng-data1:~# smbclient -m SMB3 //ctdb/gluster -k >> smb: \> ls >> NT_STATUS_NOT_FOUND listing \* >> >> On a windows-system we got the message "Element not found" >> >> Here is our smb.conf: >> ------------------- >> [global] >> workgroup = uni >> netbios name = ctdb >> security = ads >> realm = UNI.DE >> idmap config *: range = 10000-19999 >> idmap config uni: range = 1000000-1999999 >> idmap config uni: backend = rid >> winbind use default domain = yes >> winbind refresh tickets = yes >> store dos attributes = yes >> map acl inherit = yes >> vfs objects = acl_xattr >> server min protocol = SMB2 >> >> >> [gluster] >> comment = Daten im Cluster >> guest ok = no >> read only = no >> vfs objects = acl_xattr glusterfs shadow_copy2 >Thank you, that solved our problem. Sometimes it's good to read a man-page ;-)> Assuming you have configured CTDB-GlusterFS cluster in the right way, one correction to your > smb.conf here is that glusterfs module should be always the last one in vfs objects list. See the > following snippet from man page for vfs_glusterfs(8): > > . . . > This module can be combined with other modules, but it should be the last module in the vfs objects > list. Modules added to this list to the right of the glusterfs entry may not have any effect at all. > . . . > > Even then we do have an issue when vfs_shadow_copy2 module is used along with vfs_glusterfs as > explained in the following thread: > > https://lists.samba.org/archive/samba-technical/2016-October/116834.html > > If you encounter this issue you can workaround it by setting mountpoint parameter for shadow_copy2 > module to / as mentioned in the above link. > >> glusterfs:volume = gv1 >> glusterfs:logfile = /var/log/samba/gluster-gv1.log >> glusterfs:loglevel = 10 >> gluster:volfile_server = localhost >> kernel share modes = no >> path = /win-share >> shadow:snapdir = /win-share/.snaps >> shadow:basedir = /win-share >> shadow:sort = desc >> shadow:snapprefix = ^S[A-Za-z0-9]*p$ >> shadow:format = _GMT-%Y.%m.%d-%H.%M.%S >> ------------- >> >> when we try to excess the share we get the following errormessage insite >> the logfile: >> -------------------- >> [2017-01-03 13:55:14.757049] T [rpc-clnt.c:1659:rpc_clnt_submit] >> 0-rpc-clnt: submitted request (XID: 0x4a Program: GlusterFS 3.3, >> ProgVers: 330, Proc: 18) to rpc-transport (gv1-client-1 >> ) >> [2017-01-03 13:55:14.757291] T [rpc-clnt.c:687:rpc_clnt_reply_init] >> 0-gv1-client-2: received rpc message (RPC XID: 0x4b Program: GlusterFS >> 3.3, ProgVers: 330, Proc: 18) from rpc-transpo >> rt (gv1-client-2) >> [2017-01-03 13:55:14.757320] T [rpc-clnt.c:687:rpc_clnt_reply_init] >> 0-gv1-client-1: received rpc message (RPC XID: 0x4a Program: GlusterFS >> 3.3, ProgVers: 330, Proc: 18) from rpc-transpo >> rt (gv1-client-1) >> [2017-01-03 13:55:14.757347] D [MSGID: 0] >> [client-rpc-fops.c:1095:client3_3_getxattr_cbk] 0-gv1-client-2: remote >> operation failed: No data available. Path: /win-share/. (0dafa3a2-5928-4 >> b05-8cb2-bd90ac84d7c4). Key: security.NTACL >> [2017-01-03 13:55:14.757355] D [MSGID: 0] >> [client-rpc-fops.c:1095:client3_3_getxattr_cbk] 0-gv1-client-1: remote >> operation failed: No data available. Path: /win-share/. (0dafa3a2-5928-4 >> b05-8cb2-bd90ac84d7c4). Key: security.NTACL >> [2017-01-03 13:55:14.757378] D [MSGID: 0] >> [client-rpc-fops.c:1106:client3_3_getxattr_cbk] 0-stack-trace: >> stack-address: 0x7fd99d682310, gv1-client-1 returned -1 error: No data >> available >> [No data available] >> [2017-01-03 13:55:14.757378] D [MSGID: 0] >> [client-rpc-fops.c:1106:client3_3_getxattr_cbk] 0-stack-trace: >> stack-address: 0x7fd99d682310, gv1-client-2 returned -1 error: No data >> available >> [No data available] >> [2017-01-03 13:55:14.757400] D [MSGID: 0] >> [afr-inode-read.c:505:afr_getxattr_cbk] 0-stack-trace: stack-address: >> 0x7fd99d682310, gv1-replicate-0 returned -1 error: No data available [No >> data available] >> [2017-01-03 13:55:14.757409] D [MSGID: 0] >> [afr-inode-read.c:505:afr_getxattr_cbk] 0-stack-trace: stack-address: >> 0x7fd99d682310, gv1-replicate-1 returned -1 error: No data available [No >> data available] >> [2017-01-03 13:55:14.757429] D [MSGID: 0] >> [dht-common.c:3032:dht_getxattr_cbk] 0-stack-trace: stack-address: >> 0x7fd99d682310, gv1-dht returned -1 error: No data available [No data availa >> ble] >> [2017-01-03 13:55:14.757445] D [MSGID: 0] >> [defaults.c:1346:default_getxattr_cbk] 0-stack-trace: stack-address: >> 0x7fd99d682310, gv1-io-threads returned -1 error: No data available [No da >> ta available] >> [2017-01-03 13:55:14.757461] D [MSGID: 0] >> [io-stats.c:2349:io_stats_getxattr_cbk] 0-stack-trace: stack-address: >> 0x7fd99d682310, gv1 returned -1 error: No data available [No data availab >> le] >> -------------------- >> When we do a "gluster snapshot create Snap1 gv1" we can create and use >> the snapshot on the commandline, so the snapshot it self is working. >> Problem seams to be the settings for shadow_copy2. >> >> Any help will help ;-) >> >>-- Stefan Kania Landweg 13 25693 St. Michaelisdonn Signieren jeder E-Mail hilft Spam zu reduzieren. Signieren Sie ihre E-Mail. Weiter Informationen unter http://www.gnupg.org Mein Schlüssel liegt auf hkp://subkeys.pgp.net -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/samba/attachments/20170104/3d7ca213/signature.sig>