> I've now managed to replicate the error on my test setup. I added a > third host and all hell broke loose - no locking at all when opening > Excel files apart from same-host. It seemed fine with two. > > I then downgraded to 4.4.5 and the problem persisted. So it cannot be > a regression from that patch. > > Regards, > > Alex >Here you go: Locked files: Pid Uid DenyMode Access R/W Oplock SharePath Name Time -------------------------------------------------------------------------------------------------- 1:11489 1046 DENY_WRITE 0x12019f RDWR NONE /mfs/samba/drive_v jmi/Corporate Accounting/ISL Ltd Accounts/2016/1609/201609 213_ ISL Balance sheet variance analysis.xls Fri Oct 21 17:27:22 2016 0:9375 1046 DENY_WRITE 0x12019f RDWR NONE /mfs/samba/drive_v jmi/Corporate Accounting/ISL Ltd Accounts/2016/1609/201609 213_ ISL Balance sheet variance analysis.xls Fri Oct 21 17:26:38 2016 This is what I'm seeing. I could open the same file r/w off two different hosts. Both seem to have a DENY_WRITE lock - which is surely supposed to be impossible? This is on 4.4.5 with only two hosts operational. Alex -- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. This email is not intended to, nor should it be taken to, constitute advice. The information provided is correct to our knowledge & belief and must not be used as a substitute for obtaining tax, regulatory, investment, legal or any other appropriate advice. "Transact" is operated by Integrated Financial Arrangements Ltd. 29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608 5300. (Registered office: as above; Registered in England and Wales under number: 3727592). Authorised and regulated by the Financial Conduct Authority (entered on the Financial Services Register; no. 190856).
Ralph Böhme
2016-Oct-21 17:06 UTC
[Samba] CTDB and locking issues in 4.4.6 (Classic domain)
On Fri, Oct 21, 2016 at 05:48:52PM +0100, Alex Crow via samba wrote:> > > I've now managed to replicate the error on my test setup. I added a > > third host and all hell broke loose - no locking at all when opening > > Excel files apart from same-host. It seemed fine with two. > > > > I then downgraded to 4.4.5 and the problem persisted. So it cannot be a > > regression from that patch. > > > > Regards, > > > > Alex > > > Here you go: > > Locked files: > Pid Uid DenyMode Access R/W Oplock > SharePath Name Time > -------------------------------------------------------------------------------------------------- > 1:11489 1046 DENY_WRITE 0x12019f RDWR NONE > /mfs/samba/drive_v jmi/Corporate Accounting/ISL Ltd > Accounts/2016/1609/201609 213_ ISL Balance sheet variance analysis.xls Fri > Oct 21 17:27:22 2016 > > 0:9375 1046 DENY_WRITE 0x12019f RDWR NONE > /mfs/samba/drive_v jmi/Corporate Accounting/ISL Ltd > Accounts/2016/1609/201609 213_ ISL Balance sheet variance analysis.xls Fri > Oct 21 17:26:38 2016 > > This is what I'm seeing. I could open the same file r/w off two different > hosts. Both seem to have a DENY_WRITE lock - which is surely supposed to be > impossible? > > This is on 4.4.5 with only two hosts operational.*phew*! At least it's not a regression by the patches for bug 12005. Can you try to reproduce the issue with a specially patched smbclient? Attached patch modifies the smbclient open command to request deny read and write sharemodes, so any two opens should conflict: smb: \> open test open file \test: for read/write fnum 1 smb: \> open test Failed to open file \test. NT_STATUS_SHARING_VIOLATION Without the patch the opens would succeed: smb: \> open test open file \test: for read/write fnum 1 smb: \> open test open file \test: for read/write fnum 2 If you can reproduce the issue with smbclient, please set loglevel to 10 on both nodes and reproduce the issue with smbclient. Reproducing the issue with smbclient instead of Excel reduces the noise and should make it easier to analyse. Please setup logging in a way so that we get per client logfiles, eg "log file = /var/log/samba/%m.log". Cheerio! -slow -------------- next part --------------
> *phew*! At least it's not a regression by the patches for bug 12005.Hi Ralph, Looking back through the changelogs suggests to me that it's been around a long time.... I did however find it odd that even after disabling the 3rd host, stopping ctdb everywhere and removing *all* the db files (including persistent) I still could reproduce on 4.4.5.> > Can you try to reproduce the issue with a specially patched smbclient? > Attached patch modifies the smbclient open command to request deny > read and write sharemodes, so any two opens should conflict:Luckily I've realised that sernet has source packages so I'll be able to rebuild the rpms and install individually. Can you tell me if the client patch should be applied to 4.4.5 or 4.4.6 (or is valid for both)?> > smb: \> open test > open file \test: for read/write fnum 1 > > smb: \> open test > Failed to open file \test. NT_STATUS_SHARING_VIOLATION > > Without the patch the opens would succeed: > > smb: \> open test > open file \test: for read/write fnum 1 > smb: \> open test > open file \test: for read/write fnum 2 > > If you can reproduce the issue with smbclient, please set loglevel to > 10 on both nodes and reproduce the issue with smbclient. Reproducing > the issue with smbclient instead of Excel reduces the noise and should > make it easier to analyse. > > Please setup logging in a way so that we get per client logfiles, eg > "log file = /var/log/samba/%m.log".That is our existing logging method. We prefer to know which client is misbehaving and it makes it easy to diagnose. I think it should be a default!> Cheerio! > -slowMany thanks for your help so far. I'd guess this might also affect people like Rozo who market appliances with a similar focus to our internal solution's aims (although we're using another distributed FS - MooseFS). I don't think we're doing anything quirky - I fact I've followed the wiki docs very closely. I've been worrying about our FS but I even tested with "posix locking no" and had the same result, so given we pass ping-pong that would rule that out, yes? Cheerio to you too. I may be able to get this test done over the weekend but I have a couple of kids to keep happy ;-) I have pasted our test smb.conf (anonymised) after this in case I've done something idiotic. Cheers Alex [global] workgroup = MY_NET realm = samba.my.net netbios name = S4FILES security = ADS #bind interfaces only = yes #interfaces = eth0, lo #dedicated keytab file = /etc/krb5.keytab #kerberos method = secrets and keytab idmap_ldb:use rfc2307 = yes clustering = yes #private dir = /mfs/ctdb/private idmap config *:backend = tdb idmap config *:range = 200000-299999 idmap config MY_NET:backend = ad idmap config MY_NET:default = yes idmap config MY_NET:schema_mode = rfc2307 idmap config MY_NET:range = 500-199999 idmap config ALEX:backend = rid idmap config ALEX:range = 300000-399999 winbind nss info = rfc2307 winbind trusted domains only = no winbind use default domain = yes winbind enum users = yes winbind enum groups = yes winbind refresh tickets = Yes [homes] comment = Home Directories valid users = %S read only = no browseable = no path = /mfs/samba/home/%S [test] comment = test path = /mfs/samba/test read only = no browseable = yes oplocks = yes .. other shares.. (most have oplocks = yes set, but other shares that don't also suffer the issue -- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. This email is not intended to, nor should it be taken to, constitute advice. The information provided is correct to our knowledge & belief and must not be used as a substitute for obtaining tax, regulatory, investment, legal or any other appropriate advice. "Transact" is operated by Integrated Financial Arrangements Ltd. 29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608 5300. (Registered office: as above; Registered in England and Wales under number: 3727592). Authorised and regulated by the Financial Conduct Authority (entered on the Financial Services Register; no. 190856).
Hi Ralph, I applied the patch to a new build of Sernet 4.4.6 SRPM. I could still open the file from both hosts, and in fact this time I could actually open it on the same host without any problems: host 1: smbclient -U ajc //172.31.0.120/ifa_v ... smb: \jmi\Corporate Accounting\ISL Ltd Accounts\2016\1609\> open "201609 213_ ISL Balance sheet variance analysis.xls" open file \jmi\Corporate Accounting\ISL Ltd Accounts\2016\1609\201609 213_ ISL Balance sheet variance analysis.xls: for read/write fnum 56287 host 2: smbclient -U ajc //172.31.0.120/ifa_v ... smb: \jmi\Corporate Accounting\ISL Ltd Accounts\2016\1609\> open "201609 213_ ISL Balance sheet variance analysis.xls" open file \jmi\Corporate Accounting\ISL Ltd Accounts\2016\1609\201609 213_ ISL Balance sheet variance analysis.xls: for read/write fnum 17669 But this: # smbstatus -L Locked files: Pid Uid DenyMode Access R/W Oplock SharePath Name Time -------------------------------------------------------------------------------------------------- 1:19650 1046 DENY_NONE 0x83 RDWR NONE /mfs/samba/drive_v jmi/Corporate Accounting/ISL Ltd Accounts/2016/1609/201609 213_ ISL Balance sheet variance analysis.xls Sat Oct 22 19:26:06 2016 0:13503 1046 DENY_NONE 0x83 RDWR NONE /mfs/samba/drive_v jmi/Corporate Accounting/ISL Ltd Accounts/2016/1609/201609 213_ ISL Balance sheet variance analysis.xls Sat Oct 22 18:58:06 2016 Is odd as the patch was applied fine during the RPM build process. Is there something I need to do when building from sRPMs to make this stick? Cheers Alex On 21/10/16 18:06, Ralph Böhme wrote:> diff --git a/source3/client/client.c b/source3/client/client.c > index 831b9bc..28e98af 100644 > --- a/source3/client/client.c > +++ b/source3/client/client.c > @@ -2498,12 +2498,12 @@ static int cmd_open(void) > > status = cli_ntcreate(targetcli, targetname, 0, > FILE_READ_DATA|FILE_WRITE_DATA, 0, > - FILE_SHARE_READ|FILE_SHARE_WRITE, FILE_OPEN, > + 0, FILE_OPEN, > 0x0, 0x0, &fnum, NULL); > if (!NT_STATUS_IS_OK(status)) { > status = cli_ntcreate(targetcli, targetname, 0, > FILE_READ_DATA, 0, > - FILE_SHARE_READ|FILE_SHARE_WRITE, FILE_OPEN, > + 0, FILE_OPEN, > 0x0, 0x0, &fnum, NULL); > if (NT_STATUS_IS_OK(status)) { > d_printf("open file %s: for read/write fnum %d\n", targetname, fnum); > -- 2.7.4-- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. This email is not intended to, nor should it be taken to, constitute advice. The information provided is correct to our knowledge & belief and must not be used as a substitute for obtaining tax, regulatory, investment, legal or any other appropriate advice. "Transact" is operated by Integrated Financial Arrangements Ltd. 29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608 5300. (Registered office: as above; Registered in England and Wales under number: 3727592). Authorised and regulated by the Financial Conduct Authority (entered on the Financial Services Register; no. 190856).