samba - Jun 2016 - access denied with "hide dot files = Yes"

Hello,

at our site we're using the revision control software Mercurial. A typical
workflow scenariao is that we have a Mercurial repository on a Samba AD
Member share and the users pull and push commits to this central repository.

This workflow is broken as of Samba 4.3.4. When doing certain operations
(pull or push) on the central repository on the Samba share, Mercurial
complains that it has no access to a file in the repository and aborts the
operation, even if the user actually has full access to the repository (the
repository resides in the directory /.hg).

It seems that the change
* BUG 11645: smbd: Make "hide dot files" option work with "store
dos
attributes = yes".
is related to the problem. When "hide dot files" is set to
"No", the
problem doesn't occur.

The smb.conf is as follows:
[global]
        workgroup = AD
        security = ADS
        realm = xx.xxxx.xx

        idmap config *:backend = tdb
        idmap config *:range = 70001-80000
        idmap config AD:backend = ad
        idmap config AD:schema_mode = rfc2307
        idmap config AD:range = 10000-70000

        vfs objects = acl_xattr
        map acl inherit = Yes
        store dos attributes = Yes

        winbind nss info = rfc2307
        winbind enum users = yes
        winbind enum groups = yes
[ftp]
        path = /home/shares/ftp/
        hide dot files = Yes
        read only = no
        vfs objects = acl_xattr

The smbd log shows the following when the problem occurs:
[2016/05/30 17:06:26.602636,  5, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/files.c:128(file_new)
  allocated file structure fnum 1601163582 (2 used)
[2016/05/30 17:06:26.602652, 10, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/files.c:745(file_name_hash)
  file_name_hash:
/home/shares/ftp/LaserLightSection/TEST/.hg/.dirstate-3r6ocf hash 0xedf88623
[2016/05/30 17:06:26.602667,  5, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/dosmode.c:203(unix_mode)
  unix_mode: unix_mode(LaserLightSection/TEST/.hg/.dirstate-3r6ocf)
returning 0744
[2016/05/30 17:06:26.602679, 10, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/open.c:2479(open_file_ntcreate)
  open_file_ntcreate: fname=LaserLightSection/TEST/.hg/.dirstate-3r6ocf,
dos_attrs=0x80 access_mask=0x12019f share_access=0x7 create_disposition 0x5
create_options=0x40 unix mode=0744 oplock_request=2 private_flags = 0x0
[2016/05/30 17:06:26.602692,  8, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/dosmode.c:566(dos_mode)
  dos_mode: LaserLightSection/TEST/.hg/.dirstate-3r6ocf
[2016/05/30 17:06:26.602717, 10, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/dosmode.c:303(get_ea_dos_attribute)
  get_ea_dos_attribute: LaserLightSection/TEST/.hg/.dirstate-3r6ocf attr 0x20
[2016/05/30 17:06:26.602731, 10, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/dosmode.c:345(get_ea_dos_attribute)
  get_ea_dos_attribute: file LaserLightSection/TEST/.hg/.dirstate-3r6ocf
case 3 set btime Mon May 30 17:06:27 2016

[2016/05/30 17:06:26.602749,  5, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/dosmode.c:70(dos_mode_debug_print)
  dos_mode_debug_print: get_ea_dos_attribute returning (0x22): "ha"
[2016/05/30 17:06:26.602760,  5, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/dosmode.c:70(dos_mode_debug_print)
  dos_mode_debug_print: dos_mode returning (0x22): "ha"
[2016/05/30 17:06:26.602772, 10, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/open.c:2014(open_match_attributes)
  open_match_attributes: old_dos_attr = 0x22, existing_unx_mode = 0100775,
new_dos_attr = 0x0 returned_unx_mode = 00
[2016/05/30 17:06:26.602784,  5, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/open.c:2613(open_file_ntcreate)
  open_file_ntcreate: attributes missmatch for file
LaserLightSection/TEST/.hg/.dirstate-3r6ocf (22 0) (0100775, 0744)
[2016/05/30 17:06:26.602796,  5, pid=21947, effective(10011, 10031),
real(10011, 0)] ../lib/dbwrap/dbwrap.c:178(dbwrap_check_lock_order)
  check lock order 1 for /usr/local/samba/var/lock/smbXsrv_open_global.tdb
[2016/05/30 17:06:26.602808, 10, pid=21947, effective(10011, 10031),
real(10011, 0)] ../lib/dbwrap/dbwrap.c:133(debug_lock_order)
  lock order:  1:/usr/local/samba/var/lock/smbXsrv_open_global.tdb
2:<none>
3:<none>
[2016/05/30 17:06:26.602821, 10, pid=21947, effective(10011, 10031),
real(10011, 0)] ../lib/dbwrap/dbwrap_tdb.c:59(db_tdb_log_key)
  Locking key DBCDDE9B
[2016/05/30 17:06:26.602835, 10, pid=21947, effective(10011, 10031),
real(10011, 0)] ../lib/dbwrap/dbwrap_tdb.c:143(db_tdb_fetch_locked_internal)
  Allocated locked data 0x0x7fc942bba210
[2016/05/30 17:06:26.602854, 10, pid=21947, effective(10011, 10031),
real(10011, 0)] ../lib/dbwrap/dbwrap_tdb.c:59(db_tdb_log_key)
  Unlocking key DBCDDE9B
[2016/05/30 17:06:26.602866,  5, pid=21947, effective(10011, 10031),
real(10011, 0)]
../lib/dbwrap/dbwrap.c:146(dbwrap_lock_order_state_destructor)
  release lock order 1 for /usr/local/samba/var/lock/smbXsrv_open_global.tdb
[2016/05/30 17:06:26.602886, 10, pid=21947, effective(10011, 10031),
real(10011, 0)] ../lib/dbwrap/dbwrap.c:133(debug_lock_order)
  lock order:  1:<none> 2:<none> 3:<none>
[2016/05/30 17:06:26.602901,  5, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/files.c:554(file_free)
  freed files structure 1601163582 (1 used)
[2016/05/30 17:06:26.602913, 10, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/open.c:4806(create_file_unixpath)
  create_file_unixpath: NT_STATUS_ACCESS_DENIED
[2016/05/30 17:06:26.602923, 10, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/open.c:5084(create_file_default)
  create_file: NT_STATUS_ACCESS_DENIED
[2016/05/30 17:06:26.602937,  3, pid=21947, effective(10011, 10031),
real(10011, 0)]
../source3/smbd/smb2_server.c:2935(smbd_smb2_request_error_ex)
  smbd_smb2_request_error_ex: smbd_smb2_request_error_ex: idx[1]
status[NT_STATUS_ACCESS_DENIED] || at ../source3/smbd/smb2_create.c:293
[2016/05/30 17:06:26.602951, 10, pid=21947, effective(10011, 10031),
real(10011, 0)]
../source3/smbd/smb2_server.c:2826(smbd_smb2_request_done_ex)
  smbd_smb2_request_done_ex: idx[1] status[NT_STATUS_ACCESS_DENIED] body[8]
dyn[yes:1] at ../source3/smbd/smb2_server.c:2983
[2016/05/30 17:06:26.602963, 10, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/smb2_server.c:908(smb2_set_operation_credit)
  smb2_set_operation_credit: requested 1, charge 1, granted 1, current
possible/max 482/512, total granted/max/low/range 31/8192/23/31
[2016/05/30 17:06:26.606649, 10, pid=21947, effective(10011, 10031),
real(10011, 0)] ../source3/smbd/smb2_server.c:3686(smbd_smb2_io_handler)
  smbd_smb2_request idx[1] of 5 vectors

Is this a known problem and is there something I can do about this?

Best regards
Andreas

On Wed, Jun 01, 2016 at 10:08:20AM +0200, samba.20.andwin at spamgourmet.com
wrote:
> Hello,
> 
> at our site we're using the revision control software Mercurial. A
typical
> workflow scenariao is that we have a Mercurial repository on a Samba AD
> Member share and the users pull and push commits to this central
repository.
> 
> This workflow is broken as of Samba 4.3.4. When doing certain operations
> (pull or push) on the central repository on the Samba share, Mercurial
> complains that it has no access to a file in the repository and aborts the
> operation, even if the user actually has full access to the repository (the
> repository resides in the directory /.hg).
> 
> It seems that the change
> * BUG 11645: smbd: Make "hide dot files" option work with
"store dos
> attributes = yes".
> is related to the problem. When "hide dot files" is set to
"No", the
> problem doesn't occur.
> 
> The smb.conf is as follows:
> [global]
>         workgroup = AD
>         security = ADS
>         realm = xx.xxxx.xx
> 
>         idmap config *:backend = tdb
>         idmap config *:range = 70001-80000
>         idmap config AD:backend = ad
>         idmap config AD:schema_mode = rfc2307
>         idmap config AD:range = 10000-70000
> 
>         vfs objects = acl_xattr
>         map acl inherit = Yes
>         store dos attributes = Yes
> 
>         winbind nss info = rfc2307
>         winbind enum users = yes
>         winbind enum groups = yes
> [ftp]
>         path = /home/shares/ftp/
>         hide dot files = Yes
>         read only = no
>         vfs objects = acl_xattr
This actually worked before due to a bug in Samba,
which 11645 fixed to make us work the same as
Windows.

When "hide dot files" is true the attribute
returned is H, which restricts operations
the Windows client can do to the file until
it is removed.

Prior to BUG 11645, the stored DOS attributes
would override the H attribute, so you weren't
actually getting correct behavior.

If you depend on accessing dot files without
the H attribute, set "hide dot files = no".

> On Wed, Jun 01, 2016 at 10:08:20AM +0200, samba.20.andwin at
spamgourmet.com
> wrote:
> > Hello,
> >
> > at our site we're using the revision control software Mercurial. A
> typical
> > workflow scenariao is that we have a Mercurial repository on a Samba
AD
> > Member share and the users pull and push commits to this central
> repository.
> >
> > This workflow is broken as of Samba 4.3.4. When doing certain
operations
> > (pull or push) on the central repository on the Samba share, Mercurial
> > complains that it has no access to a file in the repository and aborts
> the
> > operation, even if the user actually has full access to the repository
> (the
> > repository resides in the directory /.hg).
> >
> > It seems that the change
> > * BUG 11645: smbd: Make "hide dot files" option work with
"store dos
> > attributes = yes".
> > is related to the problem. When "hide dot files" is set to
"No", the
> > problem doesn't occur.
> >
> > The smb.conf is as follows:
> > [global]
> >         workgroup = AD
> >         security = ADS
> >         realm = xx.xxxx.xx
> >
> >         idmap config *:backend = tdb
> >         idmap config *:range = 70001-80000
> >         idmap config AD:backend = ad
> >         idmap config AD:schema_mode = rfc2307
> >         idmap config AD:range = 10000-70000
> >
> >         vfs objects = acl_xattr
> >         map acl inherit = Yes
> >         store dos attributes = Yes
> >
> >         winbind nss info = rfc2307
> >         winbind enum users = yes
> >         winbind enum groups = yes
> > [ftp]
> >         path = /home/shares/ftp/
> >         hide dot files = Yes
> >         read only = no
> >         vfs objects = acl_xattr
>
> This actually worked before due to a bug in Samba,
> which 11645 fixed to make us work the same as
> Windows.
>

I'm not familiar with Windows Servers. Do they provide an option similar to
"hide dot files"?

> When "hide dot files" is true the attribute
> returned is H, which restricts operations
> the Windows client can do to the file until
> it is removed.
>
I'd be curious what kind of restrictions these are. I still can't figure
out why Mercurial complains about "access denied" for certain files,
whereby I can edit, delete,... the very same files manually without a
problem.
>
> Prior to BUG 11645, the stored DOS attributes
> would override the H attribute, so you weren't
> actually getting correct behavior.
>
If I understand this correctly, the stored DOS attributes didn't contain
the H attribute and this did override (prior to the bug fix) the forced H
attribute from "hide dot files = yes". But if this was the case, why
did
the Windows Explorer show these files then as hidden?
>
> If you depend on accessing dot files without
> the H attribute, set "hide dot files = no".
>
Many thanks for your answers, this is what I will do. However, I'd also
like to understand the problem at hand.