samba - Aug 2018 - samba 4.7.7 shares on FreeBSD 11.1-p11 started to ignore ACL

Hi,

This morning three of our FreeBSD-11.1-p11 servers with Samba 4.7.7 
installations started to ignore ACL settings and reject user access to 
shares.  All three servers are members of DC running on Windows Server 
2008R2.  Everything has been running ok for last few year.  I have been 
upgrading Samba and FreeBSD installations and on last Friday upgraded to 
the latest packages from samba47-4.7.6 to samba47-4.7.7 and after 
restarting the services everything worked as expected.

Samba shares are on ZFS volume with ACL settings set to passthrough and 
inherited.  If I open Properties->Security then I do not see any of ACL 
settings rather Everyone, root and Administrators set to special 
permissions.  ZFS ACLs on files/dirs are just fine according to 
getfacl.  wbinfo -u and -g, getent passwd returns users and groups as 
expected.  Listing with getfacl shows actual/resolved names.  I may 
modify ACL with setfacl of course.

Still If I open shares from Windows7/10 hosts it shows share, give me 
access as an admin however all other users do not have access to those 
shares.

I have tried to remove ACLs with setfacl for some shares and set ACL 
from Windows7/10 Properties from the scratch however the problem 
remains.  If I try to modify Security settings I receive a error message 
"The parameter is incorrect" for all files I try to update on.

One of the shares running as virtual server so I did made a snapshot and 
tried to clean up /var/db/samba4/ so to start from scratch however it 
did not help.  It still rejects to update ACL/Security from Windows7/10 
and whatever getfacl shows on server the client sees shares and 
Eveybody, Administators (local on server) and root.

Here is an example of smb4.conf from one of the servers.  It is 
explicitly set to master (the others are set to master=no).  That 
configuration worked just fine for last 2 years or so with Samba 4.6.* 
and recently 4.7.6 version and worked just fine on 4.7.7 after upgrade.

[global]
        security = ADS
        workgroup = DOMAIN.LO
        realm = DOMAIN.LO
        password server = 10.54.148.9

        os level = 66
        preferred master = yes

        bind interfaces only = yes
        interfaces = 10.54.148.51

        log file = /var/log/samba4/%m.log
        log level = 5

        veto files = /Thumbs.db/.DS_Store/._.DS_Store/.apdisk/
        delete veto files = yes

        idmap config * : backend = tdb
        idmap config * : range = 3000-79999
        idmap config DOMAIN-LO : backend = rid
        idmap config DOMAIN-LO : range = 80000-3000000

        winbind enum users = yes
        winbind enum groups = yes
        winbind cache time = 64800
        winbind max domain connections = 1
        winbind normalize names = no
        winbind offline logon = true

        use sendfile = no
        use mmap = yes
        aio read size = 2048
        aio write size = 2048
        min receivefile size = 2048
        write cache size = 2048
        socket options = TCP_NODELAY IPTOS_LOWDELAY
        large readwrite = yes
        strict locking = no
        strict sync = no
        getwd cache = yes
        read raw = yes
        write raw = yes
        unix extensions = no

        map acl inherit = yes
        nt acl support = yes
        store dos attributes = yes
        inherit acls = yes
        inherit owner = yes
        inherit permissions = yes
        map archive = no
        map readonly = no
        vfs objects = zfsacl streams_xattr
        nfs4:mode = special
        nfs4:acedup = merge
        nfs4:chown = no

        browseable = yes
        guest ok = no
        writable = yes
        create mask = 0775
        directory mask = 0775
        csc policy = disable

        access based share enum = yes
        hide unreadable = yes

        vfs objects = full_audit
        full_audit:prefix = %u|%m|%S
        full_audit:success = mkdir rmdir write pwrite rename unlink
        full_audit:failure = mkdir rmdir write pwrite rename unlink
        full_audit:facility = local5
        full_audit:priority = info

[ShareA]
        path = /data/sharea
        admin users = @"DOMAIN-LO\LocalAdmins"
        valid users = @"DOMAIN-LO\Domain Users"

[ShareB]
        path = /data/shareb
        admin users = @"DOMAIN-LO\LocalAdmins"
        valid users = @"DOMAIN-LO\Domain Users"


Does anyone had similar issues?

It seems the problem is not with samba 4.7.7 upgrade because one of test 
virtual hosts with almost identical configuration works just fine 
still.  Three other samba hosts lost ACL settings ...


Thanks!
Oleg

On 06. aug. 2018 15:15, Oleg Cherkasov via samba wrote:
> 
> This morning three of our FreeBSD-11.1-p11 servers with Samba 4.7.7 
> installations started to ignore ACL settings and reject user access to 
> shares.  All three servers are members of DC running on Windows Server 
> 2008R2.  Everything has been running ok for last few year.  I have been 
> upgrading Samba and FreeBSD installations and on last Friday upgraded to 
> the latest packages from samba47-4.7.6 to samba47-4.7.7 and after 
> restarting the services everything worked as expected.
> 
> Samba shares are on ZFS volume with ACL settings set to passthrough and 
> inherited.  If I open Properties->Security then I do not see any of ACL 
> settings rather Everyone, root and Administrators set to special 
> permissions.  ZFS ACLs on files/dirs are just fine according to 
> getfacl.  wbinfo -u and -g, getent passwd returns users and groups as 
> expected.  Listing with getfacl shows actual/resolved names.  I may 
> modify ACL with setfacl of course.
> 
> Still If I open shares from Windows7/10 hosts it shows share, give me 
> access as an admin however all other users do not have access to those 
> shares.
> 
> I have tried to remove ACLs with setfacl for some shares and set ACL 
> from Windows7/10 Properties from the scratch however the problem 
> remains.  If I try to modify Security settings I receive a error message 
> "The parameter is incorrect" for all files I try to update on.
> 
> One of the shares running as virtual server so I did made a snapshot and 
> tried to clean up /var/db/samba4/ so to start from scratch however it 
> did not help.  It still rejects to update ACL/Security from Windows7/10 
> and whatever getfacl shows on server the client sees shares and 
> Eveybody, Administators (local on server) and root.
> 
> Here is an example of smb4.conf from one of the servers.  It is 
> explicitly set to master (the others are set to master=no).  That 
> configuration worked just fine for last 2 years or so with Samba 4.6.* 
> and recently 4.7.6 version and worked just fine on 4.7.7 after upgrade.
> 
> [global]
>         security = ADS
>         workgroup = DOMAIN.LO
>         realm = DOMAIN.LO
>         password server = 10.54.148.9
> 
>         os level = 66
>         preferred master = yes
> 
>         bind interfaces only = yes
>         interfaces = 10.54.148.51
> 
>         log file = /var/log/samba4/%m.log
>         log level = 5
> 
>         veto files = /Thumbs.db/.DS_Store/._.DS_Store/.apdisk/
>         delete veto files = yes
> 
>         idmap config * : backend = tdb
>         idmap config * : range = 3000-79999
>         idmap config DOMAIN-LO : backend = rid
>         idmap config DOMAIN-LO : range = 80000-3000000
> 
>         winbind enum users = yes
>         winbind enum groups = yes
>         winbind cache time = 64800
>         winbind max domain connections = 1
>         winbind normalize names = no
>         winbind offline logon = true
> 
>         use sendfile = no
>         use mmap = yes
>         aio read size = 2048
>         aio write size = 2048
>         min receivefile size = 2048
>         write cache size = 2048
>         socket options = TCP_NODELAY IPTOS_LOWDELAY
>         large readwrite = yes
>         strict locking = no
>         strict sync = no
>         getwd cache = yes
>         read raw = yes
>         write raw = yes
>         unix extensions = no
> 
>         map acl inherit = yes
>         nt acl support = yes
>         store dos attributes = yes
>         inherit acls = yes
>         inherit owner = yes
>         inherit permissions = yes
>         map archive = no
>         map readonly = no
>         vfs objects = zfsacl streams_xattr
>         nfs4:mode = special
>         nfs4:acedup = merge
>         nfs4:chown = no
> 
>         browseable = yes
>         guest ok = no
>         writable = yes
>         create mask = 0775
>         directory mask = 0775
>         csc policy = disable
> 
>         access based share enum = yes
>         hide unreadable = yes
> 
>         vfs objects = full_audit
>         full_audit:prefix = %u|%m|%S
>         full_audit:success = mkdir rmdir write pwrite rename unlink
>         full_audit:failure = mkdir rmdir write pwrite rename unlink
>         full_audit:facility = local5
>         full_audit:priority = info
> 
> [ShareA]
>         path = /data/sharea
>         admin users = @"DOMAIN-LO\LocalAdmins"
>         valid users = @"DOMAIN-LO\Domain Users"
> 
> [ShareB]
>         path = /data/shareb
>         admin users = @"DOMAIN-LO\LocalAdmins"
>         valid users = @"DOMAIN-LO\Domain Users"
> 

Eventually log.wb-LO reports:

[2018/08/06 16:30:36.602929, 50, pid=1218, effective(0, 0), real(0, 0), 
class=tevent] ../lib/util/tevent_debug.c:66(samba_tevent_debug)
   samba_tevent: Run immediate event "tevent_req_trigger": 0x813478d60
[2018/08/06 16:30:36.603163,  3, pid=1218, effective(0, 0), real(0, 0)] 
../source3/libsmb/cliconnect.c:1678(cli_session_setup_creds_done_spnego)
   SPNEGO login failed: The attempted logon is invalid. This is either 
due to a bad username or authentication information.
[2018/08/06 16:30:36.603366,  1, pid=1218, effective(0, 0), real(0, 0), 
class=winbind] ../source3/winbindd/winbindd_cm.c:1118(cm_prepare_connection)
   authenticated session setup to DM-LO-DC01.lo using 
DOMAIN-LO\TEST02NO$ failed with NT_STATUS_LOGON_FAILURE
[2018/08/06 16:30:36.603596,  3, pid=1218, effective(0, 0), real(0, 0), 
class=winbind] ../source3/winbindd/winbindd_cm.c:665(cm_get_ipc_userpass)
   cm_get_ipc_userpass: No auth-user defined
[2018/08/06 16:30:36.603813,  3, pid=1218, effective(0, 0), real(0, 0), 
class=winbind] ../source3/winbindd/winbindd_cm.c:665(cm_get_ipc_userpass)
   cm_get_ipc_userpass: No auth-user defined
[2018/08/06 16:30:36.604181,  1, pid=1218, effective(0, 0), real(0, 0), 
class=winbind] ../source3/winbindd/winbindd_cm.c:1258(cm_prepare_connection)
   Failed to prepare SMB connection to DM-LO-DC01.lo: 
NT_STATUS_LOGON_FAILURE
[2018/08/06 16:30:36.604365, 10, pid=1218, effective(0, 0), real(0, 0), 
class=tdb] ../source3/lib/gencache.c:304(gencache_set_data_blob)
   Did not store value for NEG_CONN_CACHE/LO,DM-LO-DC01.lo, we already 
got it
[2018/08/06 16:30:36.604532,  9, pid=1218, effective(0, 0), real(0, 0)] 
../source3/libsmb/conncache.c:189(add_failed_connection_entry)
   add_failed_connection_entry: added domain LO (DM-LO-DC01.lo) to 
failed conn cache
[2018/08/06 16:30:36.604726, 10, pid=1218, effective(0, 0), real(0, 0), 
class=tdb] ../source3/lib/gencache.c:397(gencache_del)
   Deleting cache entry (key=[SAFJOIN/DOMAIN/LO])
[2018/08/06 16:30:36.605180, 10, pid=1218, effective(0, 0), real(0, 0), 
class=tdb] ../source3/lib/gencache.c:397(gencache_del)
   Deleting cache entry (key=[SAF/DOMAIN/LO])
[2018/08/06 16:30:36.605372, 10, pid=1218, effective(0, 0), real(0, 0), 
class=tdb] ../source3/lib/gencache.c:304(gencache_set_data_blob)
   Did not store value for NEG_CONN_CACHE/lo,DM-LO-DC01.lo, we already 
got it
[2018/08/06 16:30:36.605558,  9, pid=1218, effective(0, 0), real(0, 0)] 
../source3/libsmb/conncache.c:189(add_failed_connection_entry)
   add_failed_connection_entry: added domain lo (DM-LO-DC01.lo) to 
failed conn cache
[2018/08/06 16:30:36.605744, 10, pid=1218, effective(0, 0), real(0, 0), 
class=tdb] ../source3/lib/gencache.c:397(gencache_del)
   Deleting cache entry (key=[SAFJOIN/DOMAIN/LO])
[2018/08/06 16:30:36.605939, 10, pid=1218, effective(0, 0), real(0, 0), 
class=tdb] ../source3/lib/gencache.c:397(gencache_del)
   Deleting cache entry (key=[SAF/DOMAIN/LO])
[2018/08/06 16:30:36.606168, 10, pid=1218, effective(0, 0), real(0, 0), 
class=winbind] ../source3/winbindd/winbindd_cm.c:399(set_domain_offline)
   set_domain_offline: called for domain LO
[2018/08/06 16:30:36.606373, 50, pid=1218, effective(0, 0), real(0, 0), 
class=tevent] ../lib/util/tevent_debug.c:66(samba_tevent_debug)
   samba_tevent: Added timed event "check_domain_online_handler": 
0x81344e820
[2018/08/06 16:30:36.606545, 10, pid=1218, effective(0, 0), real(0, 0), 
class=winbind] ../source3/winbindd/winbindd_cm.c:443(set_domain_offline)
   set_domain_offline: added event handler for domain LO
[2018/08/06 16:30:36.606823, 50, pid=1218, effective(0, 0), real(0, 0), 
class=tevent] ../lib/util/tevent_debug.c:66(samba_tevent_debug)
   samba_tevent: Added timed event "messaging_dgm_out_idle_handler": 
0x81344cc60
[2018/08/06 16:30:36.606997, 10, pid=1218, effective(0, 0), real(0, 0)] 
../source3/lib/messages_dgm.c:1344(messaging_dgm_send)
   messaging_dgm_send: Sending message to 1210
[2018/08/06 16:30:37.678915, 50, pid=1218, effective(0, 0), real(0, 0), 
class=tevent] ../lib/util/tevent_debug.c:66(samba_tevent_debug)
   samba_tevent: Running timer event 0x81344cc60 
"messaging_dgm_out_idle_handler"
[2018/08/06 16:30:37.679191, 50, pid=1218, effective(0, 0), real(0, 0), 
class=tevent] ../lib/util/tevent_debug.c:66(samba_tevent_debug)
   samba_tevent: Ending timer event 0x81344cc60 
"messaging_dgm_out_idle_handler"


Not sure if "SPNEGO login failed: The attempted logon is invalid. This 
is either due to a bad username or authentication information." related 
to my issue but I do not see any suspicious messages in logs.

I made a quick test and deployed new virtual host similar to existing 
servers but based on FreeBSD 11.2 and Samba47-4.7.7 and the problem may 
be reproduced indeed.  So I wonder if it is something wrong with our DC :(

On Mon, 6 Aug 2018 16:37:46 +0200
Oleg Cherkasov via samba <samba at lists.samba.org> wrote:
> On 06. aug. 2018 15:15, Oleg Cherkasov via samba wrote:
> > 
> > 
> > [global]
> >         security = ADS
> >         workgroup = DOMAIN.LO
> >         realm = DOMAIN.LO
Is the above a typo ?
I refer to workgroup & realm being identical ?

Rowland

On 06. aug. 2018 16:37, Oleg Cherkasov via samba wrote:
> On 06. aug. 2018 15:15, Oleg Cherkasov via samba wrote:
>>
>> This morning three of our FreeBSD-11.1-p11 servers with Samba 4.7.7 
>> installations started to ignore ACL settings and reject user access to 
>> shares.  All three servers are members of DC running on Windows Server 
>> 2008R2.  Everything has been running ok for last few year.  I have 
>> been upgrading Samba and FreeBSD installations and on last Friday 
>> upgraded to the latest packages from samba47-4.7.6 to samba47-4.7.7 
>> and after restarting the services everything worked as expected.
>>
Have found the issue, it is audit or full_audit vfs.  It seems if I 
remove 'vfs objects = full_audit' or 'vfs objects = audit'
everything
works as expected.

So the next question security and vfs_full_audit have some issue :(
>> [global]
>>         security = ADS
>>         workgroup = DOMAIN.LO
>>         realm = DOMAIN.LO
>>         password server = 10.54.148.9
>>
...
>>
>>         vfs objects = full_audit
>>         full_audit:prefix = %u|%m|%S
>>         full_audit:success = mkdir rmdir write pwrite rename unlink
>>         full_audit:failure = mkdir rmdir write pwrite rename unlink
>>         full_audit:facility = local5
>>         full_audit:priority = info
Does full_audit/audit works with ADS?

With 'vfs objects = full_audit' shares report root, wheels and Everyone 
in Security Permissions rather actual ACL.  Disabling full_audit 
immediately shows actual ACLs and I may update it as well.