samba - Feb 2020 - Provisioning fails - Codebase question

I'm trying to configure an SoC system as an AD DC on Ubuntu 18.04.  I 
was successful on Ubuntu 16.04 a couple of years ago, and the script I 
used to configure the system from first boot to provisioned fails on 
18.04 and 16.04.

The failure is thus:
Setting up self join
set_nt_acl_no_snum: fset_nt_acl returned NT_STATUS_NOT_SUPPORTED.
ERROR(runtime): uncaught exception - (-1073741637, 'The request is not 
supported.')
   File "/usr/lib/python2.7/dist-packages/samba/netcmd/__init__.py",
line
176, in _run
     return self.run(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/samba/netcmd/domain.py",
line
474, in run
     nosync=ldap_backend_nosync, ldap_dryrun_mode=ldap_dryrun_mode)
   File
"/usr/lib/python2.7/dist-packages/samba/provision/__init__.py",
line 2175, in provision
     skip_sysvolacl=skip_sysvolacl)
   File
"/usr/lib/python2.7/dist-packages/samba/provision/__init__.py",
line 1806, in provision_fill
     names.domaindn, lp, use_ntvfs)
   File
"/usr/lib/python2.7/dist-packages/samba/provision/__init__.py",
line 1593, in setsysvolacl
     service=SYSVOL_SERVICE)
   File "/usr/lib/python2.7/dist-packages/samba/ntacls.py", line 162,
in
setntacl
     smbd.set_nt_acl(file, security.SECINFO_OWNER | 
security.SECINFO_GROUP | security.SECINFO_DACL | security.SECINFO_SACL, 
sd, service=service)

A couple of things stand out as possible sources, but I've been unable 
to get these questions answered with five weeks of searching:

Is Samba still using Python 2.7, or am I missing something to force it 
to use 3.x? Python 2.7 isn't even installed on the system until after 
Samba is installed.

I have traced the code that fails, and have identified that ntacls.py, 
the script which is failing the self join is using the function 
samba.samba3.smbd.set_nt_acl, but "apt install samba" installs samba 
version 4.7.6-Ubuntu. Is the use of samba3 python code related to the 
previous question (solve the former and it'll solve the latter)?

On 09/02/2020 02:27, Michael Hendricks via samba wrote:
> I'm trying to configure an SoC system as an AD DC on Ubuntu 18.04.? I 
> was successful on Ubuntu 16.04 a couple of years ago, and the script I 
> used to configure the system from first boot to provisioned fails on 
> 18.04 and 16.04.
>
> The failure is thus:
> Setting up self join
> set_nt_acl_no_snum: fset_nt_acl returned NT_STATUS_NOT_SUPPORTED.
> ERROR(runtime): uncaught exception - (-1073741637, 'The request is not 
> supported.')
> ? File
"/usr/lib/python2.7/dist-packages/samba/netcmd/__init__.py",
> line 176, in _run
> ??? return self.run(*args, **kwargs)
> ? File "/usr/lib/python2.7/dist-packages/samba/netcmd/domain.py",
line
> 474, in run
> ??? nosync=ldap_backend_nosync, ldap_dryrun_mode=ldap_dryrun_mode)
> ? File
"/usr/lib/python2.7/dist-packages/samba/provision/__init__.py",
> line 2175, in provision
> ??? skip_sysvolacl=skip_sysvolacl)
> ? File
"/usr/lib/python2.7/dist-packages/samba/provision/__init__.py",
> line 1806, in provision_fill
> ??? names.domaindn, lp, use_ntvfs)
> ? File
"/usr/lib/python2.7/dist-packages/samba/provision/__init__.py",
> line 1593, in setsysvolacl
> ??? service=SYSVOL_SERVICE)
> ? File "/usr/lib/python2.7/dist-packages/samba/ntacls.py", line
162,
> in setntacl
> ??? smbd.set_nt_acl(file, security.SECINFO_OWNER | 
> security.SECINFO_GROUP | security.SECINFO_DACL | 
> security.SECINFO_SACL, sd, service=service)
>
> A couple of things stand out as possible sources, but I've been unable 
> to get these questions answered with five weeks of searching:
>
> Is Samba still using Python 2.7, or am I missing something to force it 
> to use 3.x? Python 2.7 isn't even installed on the system until after 
> Samba is installed.
>
> I have traced the code that fails, and have identified that ntacls.py, 
> the script which is failing the self join is using the function 
> samba.samba3.smbd.set_nt_acl, but "apt install samba" installs
samba
> version 4.7.6-Ubuntu. Is the use of samba3 python code related to the 
> previous question (solve the former and it'll solve the latter)?
>
>
This sounds like a problem with your SoC, what is it ? and what is the 
filesystem ?

Samba 4.7.6 uses python2.7, it is later versions that use python3 
(4.10.0 onwards) and there have been numerous reports of Samba running 
on Ubuntu 18.04 as an AD DC.

Rowland

On 10/02/2020 01:24, michael at chameleonoriginals.com
wrote:
> It is an EspressoBin v7, ext4 filesystem, Ubuntu 18.04.? I encountered 
> previous failures saying I needed POSIX ALCs enabled, so am using a 
> 4.4.52 kernel built from source with the EspressoBin required updates 
> and POSIX ACLs (ext3 and ext4) enabled.
>
> tune2fs reports
> Filesystem features:????? has_journal ext_attr resize_inode dir_index 
> filetype needs_recovery extent 64bit flex_bg sparse_super large_file 
> huge_file dir_nlink extra_isize metadata_csum
> Default mount options:??? user_xattr acl
Much the same as my computer.
>
>
> I've seen the reports that this works on Ubuntu 18.04, which is why I 
> have spent 5 weeks trying.? I've been tracing every 
> report/tutorial/blog/video I can find trying to identify what I'm 
> missing.? I have followed each step by step and am still against a 
> brick wall.? Each how-to either has no room for comments or questions, 
> or the author has moved on to other projects, so has stopped paying 
> attention.? The one person who did respond says he did absolutely 
> nothing before starting the tutorial, except that he says to use nano 
> (which isn't installed by default on Ubuntu 18.04) to edit a file, and 
> he already set the static IP for the system...I pointed this out and 
> get nothing but crickets.
It definitely works on an Ubuntu 18.04 distro, it just seems to be a 
problem with your SoC.

Have you read this: 
https://github.com/thctlo/samba4/blob/master/full-howto-Ubuntu18.04-samba-AD_DC.txt

And this: https://wiki.samba.org/index.php/File_System_Support
>
> Thank you for pointing out that Samba 4.7.6 uses Python 2.7.? I have 
> specifically tried to find this through innumerable searches and found 
> nothing that answers this. - Is it still using Samba 3 python code 
> (samba.samba3.smbd.set_nt_acl), or is there a way to make it use Samba 
> 4 modules?? If there is a way to force this, I'll gladly try but need 
> a pointer in the right direction.
I personally think that python isn't the problem, it looks like your 
filesystem isn't allowing the ACL's to be set.
>
> I could be considered an advanced user on Ubuntu, but am quite new at 
> building custom OS images and kernels or building software from 
> source, but learn very fast.
>
> The EspressoBin has really sad documentation and support.? As a piece 
> of hardware, it's pretty sweet, especially for the price. I'm very 
> critical of my fellow geeks, and how poorly most of us document our 
> projects (like every tutorial I've found on this subject).? We're 
> especially bad at documenting those hard-learned lessons which we now 
> see as common sense.? I built a heavily annotated script for 16.04 
> (which now fails) and am revising it for 18.04.? I have started a repo 
> on GitHub specifically to outfit a first-boot EspressoBin as a 
> functional DC, walking a day-one Linux user through the process of 
> building the OS image and kernel and installing/configuring necessary 
> and optional software, assuming I can pull this off.
I can assure that it works on ARM,well it does on my Rpi4, problem is, 
you seem to be using a niche product.


Where is your github page, I couldn't find it with a google search.

Rowland

On 10/02/2020 22:34, michael at chameleonoriginals.com
wrote:
> I should have included this in the last response.? The ExpressoBin v7 
> has a Marvell Armada 3700LP (88F3720) dual core ARM Cortex A53 (arm64) 
> CPU, 1GB RAM.
>
> Seeing that Samba 4.7.6 isn't working, I have stepped into building 
> Samba from source (first attempt was 4.11.6).? After identifying quite 
> a few dependencies not listed on Samba's own page identifying Ubuntu 
> 18.04 dependencies, the make process seemed to go fairly well (but 
> with quite a few warnings).? Note that I'm giving Samba the benefit of 
> the doubt since I'm doing this on a SoC rather than a full PC or
server.
Which set of dependencies did you follow ?

On our wiki page: 
https://wiki.samba.org/index.php/Package_Dependencies_Required_to_Build_Samba

There are two sets, an older set (that may or may not be correct) or the 
ones that you should follow, these are under the heading 'Verified 
Package Dependencies'
>
> After nearly two hours of apparent success in the compile process, it 
> completely blows up with a huge number of "libsamba-passdb.so: 
> <different function names> has been removed - please update major 
> version"
No real idea about this, perhaps missing dependencies leading to 
something not being built.
>
> I will be stepping back a few releases for the next attempt, but am 
> wondering if you could point me to a "build from source for
dummies"
> that doesn't fit the standard mold by speaking as if the user has done 
> so for years?
Not really, I haven't built Samba for sometime, but it was very simple, 
install the correct dependencies, run (as a normal user) './configure', 
followed by 'make', followed by (as root) 'make install'. This
should
lead to all of Samba being installed in /usr/local/samba

If you want to build Samba packages, then perhaps Louis can help here 
(when he gets time), he builds packages for the raspberrypi.

Rowland

> On 2020-02-11 01:38, Rowland penny via samba wrote:
>> On 10/02/2020 22:34, michael at chameleonoriginals.com wrote:
>> I should have included this in the last response.? The ExpressoBin v7 
>> has a Marvell Armada 3700LP (88F3720) dual core ARM Cortex A53 (arm64) 
>> CPU, 1GB RAM.
>> 
>> Seeing that Samba 4.7.6 isn't working, I have stepped into building
>> Samba from source (first attempt was 4.11.6).? After identifying quite 
>> a few dependencies not listed on Samba's own page identifying
Ubuntu
>> 18.04 dependencies, the make process seemed to go fairly well (but 
>> with quite a few warnings).? Note that I'm giving Samba the benefit
of
>> the doubt since I'm doing this on a SoC rather than a full PC or 
>> server.
> 
> Which set of dependencies did you follow ?
> On our wiki page: 
>
https://wiki.samba.org/index.php/Package_Dependencies_Required_to_Build_Samba
> There are two sets, an older set (that may or may not be correct) or 
> the ones that you should follow, these are under the heading 'Verified 
> Package Dependencies'
I've been following the whole process outlined at 
https://wiki.samba.org/index.php/Build_Samba_from_Source.  Of course, 
I've run update/upgrade.  I have already downloaded and executed the 
bootstrap script for Ubuntu 18.04 from the page you mention.  After 
this, ./configure starts with a large number of "NOT FOUND" warnings. 
I
traced those items down and installed them after the bootstrap.  Those 
unmentioned dependencies are included in the script I'm using to create 
and repeat the process when it fails.
>> After nearly two hours of apparent success in the compile process, it 
>> completely blows up with a huge number of "libsamba-passdb.so: 
>> <different function names> has been removed - please update major
>> version"
> No real idea about this, perhaps missing dependencies leading to 
> something not being built.
libsamba-passdb.so is part of the samba source downloaded for the build 
process.  The error implies (to me) that one or more of the source files 
wasn't properly updated (I'd assume libsamba-passdb.so or files it 
depends upon) in the source package.  But I acknowledge that I'm a total 
noob at building from source.
>> I will be stepping back a few releases for the next attempt, but am 
>> wondering if you could point me to a "build from source for
dummies"
>> that doesn't fit the standard mold by speaking as if the user has
done
>> so for years?
> 
> Not really, I haven't built Samba for sometime, but it was very simple,
> install the correct dependencies, run (as a normal user)
'./configure',
> followed by 'make', followed by (as root) 'make install'.
This should
> lead to all of Samba being installed in /usr/local/samba
Everything is run as root (the only user on a first-boot EspressoBin).  
This is part of a bash script-package repo I'm building on GitHub, which 
will allow a total Linux noob to take an EspressoBin straight out of the 
box and then configure it as one of multiple configurations.  The main 
script which installs Samba asks all of the necessary questions at the 
very beginning of the process.  It then allows the user to step away for 
a while (two-plus hours in the case of an AD DC, as it appears) as it 
installs and configures everything via silent install, altering 
configuration files and debconf-set-selections.  The script then assigns 
the user-entered password for root and creates a non-root sudo user with 
password.  A primary goal is to not have a single point of user 
interaction between the initial questions and the "all done" 
notification.

One of my goals with this script is to share my love of the EspressoBin 
and Linux with others.  I want this to allow more experienced users 
(like me) to take advantage of the hardware.  I also want to give 
less-advanced users the ability to poke around with the EspressoBin and 
Linux without feeling hopeless seeing all of the background information 
that is essential for installing and configuring a system and software 
like this.  The whole repo is built upon teaching users.  The scripts 
include almost as much commentary as executable code, explaining what 
nearly every command issued is doing and why it is
needed.
> 
> If you want to build Samba packages, then perhaps Louis can help here 
> (when he gets time), he builds packages for the raspberrypi.
I'd love to hear from Louis.  I love the RasPi and currently have eight 
of them performing a number of roles in my home computing environment.  
I switched to the EspressoBin for this project since it has USB3 
support, USB2 support, a SATA3 port, 3 gigabit network ports, with 
ethernet and USB NOT dramatically limiting each other.  Drive <-> 
network throughput is easily 6x on the EspressoBin vs. the RasPi, while 
still less than $100 US including SD card.  As a Samba file server, it 
dramatically outperforms my QNAP, which costs more than four times as 
much.
> 
> Rowland

> -----Oorspronkelijk bericht-----
> Van: samba [mailto:samba-bounces at lists.samba.org] Namens 
> Michael Hendricks via samba
> Verzonden: woensdag 12 februari 2020 2:36
> Aan: Rowland penny
> CC: sambalist
> Onderwerp: Re: [Samba] Provisioning fails - Codebase question
> 
> > On 2020-02-11 01:38, Rowland penny via samba wrote:
> >> On 10/02/2020 22:34, michael at chameleonoriginals.com wrote:
> >> I should have included this in the last response.? The 
> ExpressoBin v7 
> >> has a Marvell Armada 3700LP (88F3720) dual core ARM Cortex 
> A53 (arm64) 
> >> CPU, 1GB RAM.
> >> 
> >> Seeing that Samba 4.7.6 isn't working, I have stepped into 
> building 
> >> Samba from source (first attempt was 4.11.6).? After 
> identifying quite 
> >> a few dependencies not listed on Samba's own page 
> identifying Ubuntu 
> >> 18.04 dependencies, the make process seemed to go fairly well (but
> >> with quite a few warnings).? Note that I'm giving Samba 
> the benefit of 
> >> the doubt since I'm doing this on a SoC rather than a full PC
or
> >> server.
> > 
> > Which set of dependencies did you follow ?
> > On our wiki page: 
> > 
> https://wiki.samba.org/index.php/Package_Dependencies_Required
> _to_Build_Samba
> > There are two sets, an older set (that may or may not be 
> correct) or 
> > the ones that you should follow, these are under the 
> heading 'Verified 
> > Package Dependencies'
> 
> I've been following the whole process outlined at 
> https://wiki.samba.org/index.php/Build_Samba_from_Source.  Of course, 
> I've run update/upgrade.  I have already downloaded and executed the 
> bootstrap script for Ubuntu 18.04 from the page you mention.  After 
> this, ./configure starts with a large number of "NOT FOUND" 
> warnings.  I 
> traced those items down and installed them after the 
> bootstrap.  Those 
> unmentioned dependencies are included in the script I'm using 
> to create 
> and repeat the process when it fails.
> 
> >> After nearly two hours of apparent success in the compile 
> process, it 
> >> completely blows up with a huge number of
"libsamba-passdb.so:
> >> <different function names> has been removed - please update
major
> >> version"
> > No real idea about this, perhaps missing dependencies leading to 
> > something not being built.
Most probely, ( and not, i dont know everything here.).. 
But, some libs like : libsamba-passdb.so dont have numbers at the end.
Like libsamba-passdb.so.1 i see these warnings also while building. 
> 
> libsamba-passdb.so is part of the samba source downloaded for 
> the build 
> process.  The error implies (to me) that one or more of the 
> source files 
> wasn't properly updated (I'd assume libsamba-passdb.so or files it 
> depends upon) in the source package.  But I acknowledge that 
> I'm a total 
> noob at building from source.
Same here. Im a noob in buildin from sources.. 
> 
> >> I will be stepping back a few releases for the next 
> attempt, but am 
> >> wondering if you could point me to a "build from source 
> for dummies" 
> >> that doesn't fit the standard mold by speaking as if the 
> user has done 
> >> so for years?
> > 
> > Not really, I haven't built Samba for sometime, but it was 
> very simple, 
> > install the correct dependencies, run (as a normal user) 
> './configure', 
> > followed by 'make', followed by (as root) 'make
install'.
> This should 
> > lead to all of Samba being installed in /usr/local/samba
> 
> Everything is run as root (the only user on a first-boot 
> EspressoBin).  
> This is part of a bash script-package repo I'm building on 
> GitHub, which 
> will allow a total Linux noob to take an EspressoBin straight 
> out of the 
> box and then configure it as one of multiple configurations.  
> The main 
> script which installs Samba asks all of the necessary 
> questions at the 
> very beginning of the process.  It then allows the user to 
> step away for 
> a while (two-plus hours in the case of an AD DC, as it appears) as it 
> installs and configures everything via silent install, altering 
> configuration files and debconf-set-selections.  The script 
> then assigns 
> the user-entered password for root and creates a non-root 
> sudo user with 
> password.  A primary goal is to not have a single point of user 
> interaction between the initial questions and the "all done" 
> notification.
> 
> One of my goals with this script is to share my love of the 
> EspressoBin 
> and Linux with others.  I want this to allow more experienced users 
> (like me) to take advantage of the hardware.  I also want to give 
> less-advanced users the ability to poke around with the 
> EspressoBin and 
> Linux without feeling hopeless seeing all of the background 
> information 
> that is essential for installing and configuring a system and 
> software 
> like this.  The whole repo is built upon teaching users.  The scripts 
> include almost as much commentary as executable code, explaining what 
> nearly every command issued is doing and why it is needed.
> > 
> > If you want to build Samba packages, then perhaps Louis can 
> help here 
> > (when he gets time), he builds packages for the raspberrypi.
> 
> I'd love to hear from Louis.  I love the RasPi and currently 
> have eight 
#waving hi.. 

What i suggest to try, add the source line in apt also. 
Now, what might be a bit tricky.  The samba depends, IF you need to rebuild it.
And i suggest try first if it runs. 
Yes, even if you run ubuntu..  And if it runs, then start rebuilding. 

Now, i just added this to my repo, 
https://raw.githubusercontent.com/thctlo/samba4/master/get-samba-sources.sh 

The content explains enough, short version.

Pull the sources and add them in the order of building.
And show the minimal samba requirement to build it with OS supplied packages. 
or mine, even if you remove mine this still works. 

If you just : 
cd 01-talloc
apt-get build-dep talloc
apt-get source talloc -b

It already rebuilds. 

Repeat for the others.. 
And, for every *.deb you builded, add these to a folder and create a local repo
Include this and repaet it every step. 

Thats.. About this. 
mkdir $HOME/local-repo
cp *.deb $HOME/local-repo/

cd $HOME/local-repo/
apt-ftparchive packages ./ > Packages
echo "deb [trusted=yes] file://$HOME/local-repo/ ./" >
/etc/apt/sources.list.d/local.list
apt-get update

Next. 

Good luck..

Greetz, 

Louis