samba - May 2020 - Multi-homed Samba 4 file server on Samba 4 AD domain - cross network authentication

Hi all,

I have a question about a multi-homed Samba file server and interoperability
with AD.  It's a bit complicated, so please bear with me.

I've been running Samba 4.11.6 as an AD server (two DCs) for a while (in
RFC2307 mode) in a mixed Windows/Linux environment.  I have a server running
Proxmox (Debian) with Samba 4.9.5 and it is sharing my huge ZFS volume via
Samba to Windows clients.  Windows 7 clients on the same LAN log on to the
Domain and can easily map a network drive to the Proxmox file server and
everything works fine - they connect automatically using the domain
credentials.  After some pain, all of the permissions work ok.  This regular
gigabit Ethernet network is 192.168.42.0/24.

Now I work with some huge video files on my Windows 7 workstation and I want
to send them to my Proxmox file server using 10 Gigabit (10G BASE-T) cards,
but I don't have a 10 Gig switch.  So I put the cards in the server and
workstation and I configured ISC DHCP Server on the Proxmox server for the
10G adapters only, so the special workstation client gets an IP address from
it.  They can ping each other, and I have set jumbo packets and all the
usual optimisations.  No Gateway or DNS is configured for this network.
This mini 10G network is 192.168.84.0/24.

I configured Samba on the file server to listen on both the normal 1G and
the 10G networks.  File sharing on the normal network continues to work
fine.  The special Windows 7 client continues to access the file share over
the normal network.  

The problem is I cannot map a network drive using the 10G IP address,
because it asks for a username/password and authentication fails.  I have
tried the domain username/password, and I have tried local Linux accounts
(even root!) but I always get "The specified network password is not
correct", which shows as access denied in the Samba logs (see below).

	C:\Users\lomaxd>net use x: \\192.168.84.253\fs$ /user:NSA\lomaxd
	Enter the password for 'NSA\lomaxd' to connect to
'192.168.84.253':
	System error 86 has occurred.

	The specified network password is not correct.

I think what is happening is that the file server for some reason cannot
authenticate the username/password because the request comes from a
different network to the one having the Domain Controllers.  I tried
changing the provider order (network routing priority) on the Windows 7
client, but it makes no difference.   Does anyone have any ideas how to get
Samba to authenticate the request from the 2nd network?

Things I have tried:
	. Mapping the drive by IP address (192.168.84.253) on the Windows 7
client
	. I tried authenticating with the domain admin username/password, as
well as local Linux accounts (even root!) but I always get access denied.
	. I tried by listing both adapters explicitly in
smb.conf/interfaces, and also by putting a wider subnet (192.168.0.0/16)
instead for interfaces.
	.Using static IP addresses (instead of a DHCP server)
	.Ping and SSH work on the 10G network

Below you can see the topology and configuration:

The normal network looks like this, which includes the following machines:

1 Gb 'normal' LAN:  192.168.42.0/24:
	. 192.168.42.253   pfSense, with internal AD domain DNS delegated to
DC1
	. 192.168.42.60   DC1, running Samba 4
	. 192.168.42.61   DC2, running Samba 4
	. 192.168.42.70   Proxmox, also used as my monster file server
running the default version of Samba (3.x).  This machine also has a 10G
card.
	. 192.168.42.111   Windows 7 client, with mapped network drives to
the Proxmox machine.  This machine also has a 10G card.

10 Gb 'fast' LANL: 192.168.84.0/24:
	. 192.168.84.253  Proxmox file server (same machine as
192.168.42.70).
	. 192.168.84.101  Windows 7 client trying to access files from above
server.  (same machine as 192.168.42.111)

I should mention that 3 Bridges are also defined manually on the Proxmox
server:
	. vmbr0: This unifies the Gigabit Ethernet ports (normal network)
	. vmbr1: This unifies the 10 Gigabit Ethernet ports (fast network)
	. vmbr2: This is a private host-only subnet for the VMs on the box -
ignore

Below, here are my logs and configuration files.  It's all very long, so
I'll close here.

I would very much appreciate some advice on whether it is possible to
authenticate against a Domain Controller on a different network to the
client.  I'm sure I had it working once, but I don't understand the bad
password error I get now.

Thank you all very much in advance, for reading this far! :-)

Cheers,
Dave



Logs:

In /var/log/samba/wb-NSA:

	[2020/05/13 13:44:49.104704,  2]
../source3/winbindd/winbindd_pam.c:2395(winbind_dual_SamLogon)
	  NTLM CRAP authentication for user [NSA]\[lomaxd] returned
NT_STATUS_WRONG_PASSWORD

In /var/log/samba/wb-VULCAN:

	[2020/05/13 13:46:46.802888,  2]
../source3/winbindd/winbindd_rpc.c:291(rpc_name_to_sid)
	  name_to_sid: failed to lookup name: NT_STATUS_NONE_MAPPED

In /var/log/samba/log.192.168.84.101:

	[2020/05/13 16:28:04.650026,  2]
../source3/param/loadparm.c:2803(lp_do_section)
	  Processing section "[fs$]"
	 [2020/05/13 16:28:04.650112,  1]
../lib/param/loadparm.c:1022(lpcfg_service_ok)
	  NOTE: Service test is flagged unavailable.
	[2020/05/13 16:28:04.654259,  2]
../source3/auth/auth.c:334(auth_check_ntlm_password)
	  check_ntlm_password:  Authentication for user [lomaxd] -> [lomaxd]
FAILED with error NT_STATUS_WRONG_PASSWORD, authoritative=1
	[2020/05/13 16:28:04.654299,  2]
../auth/auth_log.c:610(log_authentication_event_human_readable)
	  Auth: [SMB2,(null)] user [NSA]\[lomaxd] at [Wed, 13 May 2020
16:28:04.654290 BST] with [NTLMv1] status [NT_STATUS_WRONG_PASSWORD]
workstation [ROMULUS] remote host [ipv4:192.168.84.101:49382] mapped to
[NSA]\[lomaxd]. local host [ipv4:192.168.84.253:445]
	  {"timestamp": "2020-05-13T16:28:04.654375+0100",
"type":
"Authentication", "Authentication": {"version":
{"major": 1, "minor": 0},
"status": "NT_STATUS_WRONG_PASSWORD",
"localAddress":
"ipv4:192.168.84.253:445", 	"remoteAddress":
"ipv4:192.168.84.101:49382", "serviceDescription":
"SMB2",
"authDescription": null, "clientDomain": "NSA",
"clientAccount": "lomaxd",
"workstation": "ROMULUS", "becameAccount": null,
"becameDomain": null,
"becameSid": null, "mappedAccount": "lomaxd",
"mappedDomain": "NSA",
"netlogonComputer": null, "netlogonTrustAccount": null,
"netlogonNegotiateFlags": "0x00000000",
"netlogonSecureChannelType": 0,
"netlogonTrustAccountSid": 	null, "passwordType":
"NTLMv1", "duration":
5196}}

In /var/log/samba/log.smbd.1:

	[2020/05/13 16:28:17.733824,  2]
../lib/util/tevent_debug.c:66(samba_tevent_debug)
	  samba_tevent: EPOLL_CTL_DEL EBADF for fde[0x55e2e5e02900]
mpx_fde[(nil)] fd[14] - disabling


Now I'll share my configuration files:


My /etc/samba/smb.conf:
(My file share is fs$)

#======================= Global Settings ======================
[global]

## Browsing/Identification ###

   netbios name = VULCAN
   workgroup = NSA
   realm = NSA.INT
   #server role = member server
   security = ads
   vfs objects = acl_xattr
   map acl inherit = yes
   store dos attributes = yes
   #map archive = no
   #map hidden = no
   #map read only = no
   #map system = no
   inherit permissions = yes
   nt acl support = yes
   inherit acls = yes
   server string = %h server (Samba, Ubuntu)

   # DL: Including any of the below overrides the defaults.  Comment them
out for the defaults.  Dont change the values!
   lanman auth = yes
   client lanman auth = yes
   allow trusted domains = yes
   follow symlinks = no
   wide links = no
   unix extensions = yes

   winbind offline logon = false
   winbind nss info = rfc2307 # In samba >4.6.0 this has been replaced by
idmap config HOME
   winbind enum users = yes
   winbind enum groups = yes
   winbind cache time = 10
   winbind nested groups = yes
   winbind refresh tickets = yes

   dns forwarder = 192.168.42.253
   dns proxy = no


#### Networking ####

   interfaces = lo vmbr0 vmbr1 vmbr2
   ;interfaces = 192.168.0.0/16
   ;bind interfaces only = yes


#### Debugging/Accounting ####

   log file = /var/log/samba/log.%m
   log level = 2
   max log size = 1000000
   logging = file
   panic action = /usr/share/samba/panic-action %d


####### Authentication #######

   #passdb backend = tdbsam
   obey pam restrictions = yes
   unix password sync = yes
   passwd program = /usr/bin/passwd %u
   passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:*
%n\n *password\supdated\ssuccessfully* .
   pam password change = yes
   map to guest = bad user
   guest account = nobody

   dedicated keytab file = /etc/krb5.keytab
   kerberos method = secrets and keytab


########## Domains ###########

#   logon path = \\%N\profiles\%U
#   logon path = \\%N\%U\profile
#   logon drive = H:
#   logon home = \\%N\%U
#   logon script = logon.cmd
#   add user script = /usr/sbin/adduser --quiet --disabled-password --gecos
"" %u
#   add machine script  = /usr/sbin/useradd -g machines -c "%u machine
account" -d /var/lib/samba -s /bin/false %u
#   add group script = /usr/sbin/addgroup --force-badname %g


############ Misc ############

;   include = /home/samba/etc/smb.conf.%m
   idmap_ldb:use rfc2307 = yes
   idmap config * : backend = tdb
   idmap config * : range = 20000 - 40000
   idmap config NSA : backend = ad
   idmap config NSA : schema_mode = rfc2307
   idmap config NSA : range = 2000 - 4000
   idmap config NSA : unix_nss_info = yes
   idmap config NSA : unix_primary_group = yes
   #username map = /etc/samba/user.map
   username map script = /bin/echo
   #map untrusted to domain = yes
   template shell = /bin/bash
   template homedir = /home/%U
#   usershare max shares = 100
   usershare allow guests = yes

# DL: Experimental - boost performance of Samba file shares
#socket options = TCP_NODELAY

#======================= Share Definitions ======================
# This exports every folder under /tank/fs/usr/ by username
[homes]
   comment = Home Directories
   path = /tank/fs/usr/%U
   browseable = yes
   writeable = yes
   create mask = 0700
   directory mask = 0700
   #valid users = %S
   #write list = root, NSA.INT\Domain Users

#[sysvol]
#       path = /usr/local/samba/var/locks/sysvol
#       read only = no

[netlogon]
   comment = Network Logon Service
   path = /home/samba/netlogon
   #guest ok = yes
   writeable = yes
   #valid users = %S, NSA.INT\%S
   write list = root, NSA.INT\Domain Users

[profiles]
   comment = Users profiles
   path = /tank/fs/usr
   #guest ok = no
   browseable = no
   #valid users = %S, NSA.INT\%S
   write list = root, NSA.INT\Domain Users
   create mask = 0600
   directory mask = 0700

[printers]
   comment = All Printers
   browseable = no
   path = /var/spool/samba
   printable = yes
   guest ok = yes
   writeable = yes
   #valid users = %S, NSA.INT\%S
   write list = root, NSA.INT\Domain Users
   create mask = 0700

[print$]
   comment = Printer Drivers
   path = /var/lib/samba/printers
   browseable = yes
   writeable = no
   guest ok = yes
   #valid users = %S, NSA.INT\%S
   write list = root, NSA.INT\Domain Users

[fs$]
   comment = ZPool FS
   browseable = yes
   path = /tank/fs
   writeable = yes
   #valid users = %S, NSA.INT\%S
   write list = root, NSA.INT\Domain Users
   create mask = 0700
   directory mask = 0700


My /etc/nsswitch.conf:

# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed,
try:
# `info libc "Name Service Switch"' for information about this
file.

passwd:         files systemd winbind
group:          files systemd winbind
shadow:         files
gshadow:        files

hosts:          files dns winbind
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

netgroup:       nis


My /etc/resolv.conf:

search nsa.int
domain nsa.int
nameserver 192.168.42.253


My /etc/network/interfaces:

auto lo
iface lo inet loopback

# 10 Gigabit Port 1
allow-hotplug ens3f0
auto ens3f0
iface ens3f0 inet manual
        address  192.168.84.71
        netmask  255.255.255.0
        mtu 9014

# 10 Gigabit Port 2
allow-hotplug ens3f1
auto ens3f1
iface ens3f1 inet manual
        address  192.168.84.72
        netmask  255.255.255.0
        mtu 9014

# 10 Gigabit Port 3
allow-hotplug ens4f0
auto ens4f0
iface ens4f0 inet manual
        address 192.168.84.73
        netmask 255.255.255.0
        mtu 9014

# 10 Gigabit Port 4
allow-hotplug ens4f1
auto ens4f1
iface ens4f1 inet manual
        address 192.168.84.74
        netmask 255.255.255.0
        mtu 9014

# 10 Gigabit Port 5
allow-hotplug enp4s0f0
auto enp4s0f0
iface enp4s0f0 inet manual
        address 192.168.84.75
        netmask 255.255.255.0
        mtu 9014

# 10 Gigabit Port 6
allow-hotplug enp4s0f1
auto enp4s0f1
iface enp4s0f1 inet manual
        address 192.168.84.76
        netmask 255.255.255.0
        mtu 9014

# 1 Gig Bridge (normal network)
auto vmbr0
iface vmbr0 inet static
    address  192.168.42.70
    netmask  255.255.255.0
    network  192.168.42.0
    broadcast 192.168.42.255
    gateway  192.168.42.253
    bridge-ports eno2
    bridge-stp off
    bridge-fd 0
    mtu 1500
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr0/proxy_arp

# 10 Gigabit Bridge (fast network)
auto vmbr1
iface vmbr1 inet static
    address  192.168.84.253
    netmask  255.255.255.0
    network  192.168.84.0
    broadcast 192.168.84.255
    bridge-ports ens3f0 ens3f1 ens4f0 ens4f1 enp4s0f0 enp4s0f1
    bridge-stp off
    bridge-fd 0
    mtu 9014
    pre-up ifconfig ens3f0 mtu 9014
    pre-up ifconfig ens3f1 mtu 9014
    pre-up ifconfig ens4f0 mtu 9014
    pre-up ifconfig ens4f1 mtu 9014
    pre-up ifconfig enp4s0f0 mtu 9014
    pre-up ifconfig enp4s0f1 mtu 9014

# Bridge network for Proxmox (a private host-only subnet you can ignore)
auto vmbr2
iface vmbr2 inet static
    address  192.168.30.253
    netmask  255.255.255.0
    bridge-ports vmbr0
    bridge-stp off
    bridge-fd 0

On 13/05/2020 18:52, David Lomax via samba wrote:
> Hi all,
>
> I have a question about a multi-homed Samba file server and
interoperability
> with AD.  It's a bit complicated, so please bear with me.
Your problem is probably because your DC knows your Samba ADS client by 
its 192.168.42.0/24 Ipaddress.

Also, why only use 10G on part of your network, surely the network speed 
will be dictated by the slowest part of your network, if your clients 
only have 1G, then that is what the network speed will be, or have I got 
it wrong ?
> The problem is I cannot map a network drive using the 10G IP address,
> because it asks for a username/password and authentication fails.
Do the DC's know about the 192.168.84.0/24 network, have you created a 
reverse zone ?
> 	. 192.168.42.70   Proxmox, also used as my monster file server
> running the default version of Samba (3.x).  This machine also has a 10G
> card.
You do know that Samba 3.x.x is dead, this probably means that your 
Proxmox needs updating.
> In /var/log/samba/log.192.168.84.101:
>
> 	[2020/05/13 16:28:04.654299,  2]
> ../auth/auth_log.c:610(log_authentication_event_human_readable)
> 	  Auth: [SMB2,(null)] user [NSA]\[lomaxd] at [Wed, 13 May 2020
> 16:28:04.654290 BST] with [NTLMv1] status [NT_STATUS_WRONG_PASSWORD]
'NTLMv1' ? You do know that this is insecure.
> My /etc/samba/smb.conf:
> (My file share is fs$)
>
> [global]
>
> ## Browsing/Identification ###
>
>     vfs objects = acl_xattr
'acl_xattr' doesn't work with ZFS
>
>
>
>     lanman auth = yes
>     client lanman auth = yes
Why lanman ? do you have any Win 95/98 clients ?
>     dns forwarder = 192.168.42.253
'dns forwarder' is only used on a DC
>     unix password sync = yes
This isn't allowed on a domain member, you cannot have the same user in 
AD and /etc/passwd
>     idmap_ldb:use rfc2307 = yes
That is only used on a DC
>     #username map = /etc/samba/user.map
>     username map script = /bin/echo
you need the one you commented out and you don't need the one below
it.
> [netlogon]
>     comment = Network Logon Service
>     path = /home/samba/netlogon
>     #guest ok = yes
>     writeable = yes
>     #valid users = %S, NSA.INT\%S
>     write list = root, NSA.INT\Domain Users
'netlogon' on a domain member ?
> [fs$]
>     comment = ZPool FS
>     browseable = yes
>     path = /tank/fs
>     writeable = yes
>     #valid users = %S, NSA.INT\%S
>     write list = root, NSA.INT\Domain Users
>     create mask = 0700
>     directory mask = 0700
>
>
> My /etc/nsswitch.conf:
>
> hosts:          files dns winbind
Remove 'winbind' from the hosts line
> My /etc/resolv.conf:
>
> search nsa.int
> domain nsa.int
> nameserver 192.168.42.253
Remove the 'domain' line and point the nameserver to one of your
DC's

Rowland

Hi Rowland,

Thank you very much, you were spot on.  I had changed the Windows 7 client to LM
compatibility level, and now that I reverted it back to 5 (use NTLMv2) it works.
It was this registry key that made it start working:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa]
"LmCompatibilityLevel"=dword:00000005

I was ignoring the old LM options and the "NTLMv1" in the logs, until
you pointed it out, so thanks.

Your questions are good points, and I'll answer them for the benefit of
anyone else reading this thread.

* There are two networks, and both the server and the client are connected to
both of them.  To override the default 1G network, I map the network drive by IP
address.
* Good point about the reverse lookup zone.  I forgot about that, I will create
it.
* I take your point about vfs objects and ZFS.  I've had a lot of problems; 
it's working at the moment, but still trying to understand how permissions
are stored...
* Sorry, I initially wrote Samba 3 but when I checked my versions Proxmox is
Samba 4.9.5; just forgot to replace it in the email :-)
* I removed the lanman and client lanman options from the file server.  That was
an earlier act of desperation!
* I removed the dns forwarder clause - wasn't sure if it gets used on a
domain member
* I removed the unix password sync clause - I was never sure about that
* I removed the idmap_ldb:use rfc2307 clause - again, wasn't sure if the
client uses it
* I changed the username map as you suggested:
>     #username map = /etc/samba/user.map
>     username map script = /bin/echo
you need the one you commented out and you don't need the one below it.

(Should it be "username map" or "username map script"?)

* I removed the [netlogon] share.  Should I also remove [profiles] from the
client?  I have user directories on the file server, but not sure if it is the
role of the DC or not to host that.
* I removed winbind from nsswitch.conf as you suggest, but isn't it needed
to look up computer names from the DC?  Or it uses regular DNS nowadays?
* I removed the domain line from resolv.conf, although I'm still not sure
what it does  :-)
* I removed the nameserver entry for the gateway, and added 2 nameserver entries
with each of the DCs IPs.

Question ... I configured my gateway (pfsense) to delegate DNS lookups for
nsa.int to the DCs.  Does that mean I can keep all machines pointing their DNS
lookups to the gateway?
Or do domain members need to make the DCs their first port-of-call for DNS
lookups?

I've always scratched my head over trying to understand what are the samba
options applicable to the latest version.  What resources can you recommend I
look at?

Switching off LM & NTLM has really nailed it - thank you - I just hope my
"trust relationship failed" issues don't come back!!!

Cheers Rowland.

Thanks,
Dave



-----Original Message-----
From: Rowland penny [mailto:rpenny at samba.org] 
Sent: 13 May 2020 21:13
To: samba at lists.samba.org
Subject: Re: [Samba] Multi-homed Samba 4 file server on Samba 4 AD domain -
cross network authentication

On 13/05/2020 18:52, David Lomax via samba wrote:
> Hi all,
>
> I have a question about a multi-homed Samba file server and
interoperability
> with AD.  It's a bit complicated, so please bear with me.
Your problem is probably because your DC knows your Samba ADS client by 
its 192.168.42.0/24 Ipaddress.

Also, why only use 10G on part of your network, surely the network speed 
will be dictated by the slowest part of your network, if your clients 
only have 1G, then that is what the network speed will be, or have I got 
it wrong ?
> The problem is I cannot map a network drive using the 10G IP address,
> because it asks for a username/password and authentication fails.
Do the DC's know about the 192.168.84.0/24 network, have you created a 
reverse zone ?
> 	. 192.168.42.70   Proxmox, also used as my monster file server
> running the default version of Samba (3.x).  This machine also has a 10G
> card.
You do know that Samba 3.x.x is dead, this probably means that your 
Proxmox needs updating.
> In /var/log/samba/log.192.168.84.101:
>
> 	[2020/05/13 16:28:04.654299,  2]
> ../auth/auth_log.c:610(log_authentication_event_human_readable)
> 	  Auth: [SMB2,(null)] user [NSA]\[lomaxd] at [Wed, 13 May 2020
> 16:28:04.654290 BST] with [NTLMv1] status [NT_STATUS_WRONG_PASSWORD]
'NTLMv1' ? You do know that this is insecure.
> My /etc/samba/smb.conf:
> (My file share is fs$)
>
> [global]
>
> ## Browsing/Identification ###
>
>     vfs objects = acl_xattr
'acl_xattr' doesn't work with ZFS
>
>
>
>     lanman auth = yes
>     client lanman auth = yes
Why lanman ? do you have any Win 95/98 clients ?
>     dns forwarder = 192.168.42.253
'dns forwarder' is only used on a DC
>     unix password sync = yes
This isn't allowed on a domain member, you cannot have the same user in 
AD and /etc/passwd
>     idmap_ldb:use rfc2307 = yes
That is only used on a DC
>     #username map = /etc/samba/user.map
>     username map script = /bin/echo
you need the one you commented out and you don't need the one below
it.
> [netlogon]
>     comment = Network Logon Service
>     path = /home/samba/netlogon
>     #guest ok = yes
>     writeable = yes
>     #valid users = %S, NSA.INT\%S
>     write list = root, NSA.INT\Domain Users
'netlogon' on a domain member ?
> [fs$]
>     comment = ZPool FS
>     browseable = yes
>     path = /tank/fs
>     writeable = yes
>     #valid users = %S, NSA.INT\%S
>     write list = root, NSA.INT\Domain Users
>     create mask = 0700
>     directory mask = 0700
>
>
> My /etc/nsswitch.conf:
>
> hosts:          files dns winbind
Remove 'winbind' from the hosts line
> My /etc/resolv.conf:
>
> search nsa.int
> domain nsa.int
> nameserver 192.168.42.253
Remove the 'domain' line and point the nameserver to one of your
DC's

Rowland