samba - Sep 2018 - Migrating from Samba 3: no groups/users are imported ("listed, but then not found", "does not belong to our domain")

Rowland Penny via samba wrote 2018-09-04 14:24:
> On Tue, 04 Sep 2018 10:26:38 +0700
> Konstantin Boyandin via samba <samba at lists.samba.org> wrote:
> 
>> Rowland Penny via samba wrote 2018-09-03 17:12:
>> > On Mon, 03 Sep 2018 04:27:07 +0000
>> > "Konstantin Boyandin \(lists\) via samba" <samba at
lists.samba.org>
>> > wrote:
>> >
>> >> Hello,
>> >>
>> >> Going further with migrating NT4 domain (Samba 3) to Samba 4.
>> >> Thanks for the previous suggestions.
>> >>
>> >> When doing
>> >>
>> >> # samba-tool domain classicupgrade
--dbdir=/usr/local/samba.LAN/
>> >> --realm=ad-lan.com
>> >> --dns-backend=BIND9_DLZ /usr/local/samba.LAN/smb.conf
>> >> --option="interfaces=lo ens3" --option="bind
interfaces only=yes"
>> >>
>> >> I see in stderr the below:
>> >>
>> >> Ignoring group 'ossi'
>> >> S-1-5-21-1411277624-4092985889-3405756581-3001 listed but then
not
>> >> found: Unable to enumerate group members, (-1073741722,The
>> >> specified group does not exist.)
>> >>
>> >> for every group from existing LDAP backend of Samba 3, and
>> >>
>> >> sid S-1-5-21-1411277624-4092985889-3405756581-2062 does not
belong
>> >> to our domain
>> >>
>> >
>> > Okay, I take it your PDC was called pdclan and the domain was
called
>> > 'LAN', I have no idea what the dns domain was.
>> >
>> > You have now created a new AD DC using the dns domain
'ad-lan.com'
>> > and the new AD DC is called 'dc'
>> >
>> > So from my reading there are three Samba workgroup names in play:
>> >
>> > PDCLAN
>> > LAN
>> > AD-LAN
>> >
>> > I think this, (along with using '--realm=ad-lan.com'
instead of
>> > 'realm = ad-lan' in smb.conf) is your problem. You are
trying to
>> > change the domain from 'LAN' to 'AD-LAN', Samba is
undoubtedly
>> > treating this as a new domain and creating a new SID for it.
>> 
>> That's intentional.
>> 
>> LAN is NT4 (Samba 3) domain, and I may not just upgrade it without
>> thorough testing - too many resources are using it, and breaking down
>> network is not an option.
>> 
>> So yes, I create a new domain, under real-life domain name (I own
>> ad-lan.com) and, after transferring everything into it, testing in
>> sandbox environment, I will begin transferring everything from Samba
>> 3 into the Samba 4 domain (i.e., both LAN and AD-LAN will co-exist in
>> the same network for some time).
>> 
>> So the question, how do I do the upgrade to Samba 4 while importing
>> the users/groups from Samba 3 domain in this case? Alternately, how
>> can I import Samba 3 entities from Samba 3LDAP backend *after*
>> creating a separate Samba 4 domain?
>> 
>> Also, what's wrong with '--realm=ad-lan.com' ?
> 
> The main thing is that the upgrade code ignores it!
> 
> The classic upgrade is built upon doing just that, upgrading an
> NT4-style domain to an AD domain using the same workgroup name.
> 
> You seem to be trying to do some hybrid method and might as well
> create a new domain. You cannot have a domain called 'LAN' and a
> domain called 'AD-LAN' with the same SID.
> 
> What most people do is to create a test domain in a sandbox, carry
> out the upgrade multiple times, correcting errors, until they know
> just what they have to do to get a new AD domain. Once they are sure
> it will work, they do it for real. You should also be aware that once
> your clients see your new AD domain, they will not go back to the
> NT4-style domain.
> If the upgrade is carried out correctly, your clients shouldn't
> notice.
> 
> Your method (which is creating a new domain) will mean you will have
> to rejoin the computers to the domain.
Exactly that. I need to create a separate domain; after all the checks 
are done that switching to it works, the computers will rejoin the new 
domain. Our Samba 3 domain is used for years; since Window 10 is unable 
to join it any more, we are finally migrating everything to Samba 4.

Actually, I did the following:
- loaded the dump of LDAP backend of existing Samba 3
- replaced domain SID part in the dump; replaced domain controller 
NetBIOS name as well (I cose the same SID Smaba 4 was creating when 
trying to do classic upgrade with existing remote LDAP backend)
- imported the resulting LDAP dump into local sandbox OpenLDAP server
- re-ran the classic upgrade using the above local LDAP installation

After some cursing and fixing minor typos, I received the Samba 4 domain 
in viable state

My only remaining problem I couldn't solve is that source groups/users 
are still not recognized, i.e. I see multiple

Ignoring group 'project' S-1-5-21-2473926874-590573496-2946143095-3001 
listed but then not found: Unable to enumerate group members, 
(-1073741722,The specified group does not exist.)

records in stderr of classic upgrade command.

It isn't blocker, since both users and groups are actually added to the 
new domain and I can re-add users to groups manually - but I am still 
unsure why that happens. The entire output of upgrade command is like 
this:

---------------- output of classic upgrade below
Reading smb.conf
WARNING: The "syslog" option is deprecated
WARNING: The "idmap backend" option is deprecated
WARNING: The "idmap uid" option is deprecated
WARNING: The "idmap gid" option is deprecated
Unknown parameter encountered: "printer admin"
Ignoring unknown parameter "printer admin"
Provisioning
Exporting account policy
Exporting groups
Ignoring group 'Domain Admins' 
S-1-5-21-2473926874-590573496-2946143095-512 listed but then not found: 
Unable to enumerate group members, (-1073741722,The specified group does 
not exist.)
[...and 18 more records like above...]
Exporting users
   Skipping wellknown rid=500 (for username=root)
Ignoring group memberships of 'user' 
S-1-5-21-2473926874-590573496-2946143095-3020: Unable to enumerate group 
memberships, (-1073741724,The specified account does not exist.)
[...same line for the rest of existing users...]
Next rid = 3323
Exporting posix attributes
Reading WINS database
Cannot open wins database, Ignoring: [Errno 2] No such file or 
directory: '/usr/local/samba.LAN/wins.dat'
WARNING: The "syslog" option is deprecated
Looking up IPv4 addresses
Looking up IPv6 addresses
No IPv6 address will be assigned
Setting up secrets.ldb
Setting up the registry
Setting up the privileges database
Setting up idmap db
Setting up SAM db
Setting up sam.ldb partitions and settings
Setting up sam.ldb rootDSE
Pre-loading the Samba 4 and AD schema
Adding DomainDN: DC=ad-lan,DC=com
Adding configuration container
Setting up sam.ldb schema
Setting up sam.ldb configuration data
Setting up display specifiers
Modifying display specifiers
Adding users container
Modifying users container
Adding computers container
Modifying computers container
Setting up sam.ldb data
Setting up well known security principals
Setting up sam.ldb users and groups
Setting up self join
Setting acl on sysvol skipped
Adding DNS accounts
Creating CN=MicrosoftDNS,CN=System,DC=ad-lan,DC=com
Creating DomainDnsZones and ForestDnsZones partitions
Populating DomainDnsZones and ForestDnsZones partitions
See /var/lib/samba/private/named.conf for an example configuration 
include file for BIND
and /var/lib/samba/private/named.txt for further documentation required 
for secure DNS updates
Setting up sam.ldb rootDSE marking as synchronized
Fixing provision GUIDs
A Kerberos configuration suitable for Samba AD has been generated at 
/var/lib/samba/private/krb5.conf
Setting up fake yp server settings
Once the above files are installed, your Samba AD server will be ready 
to use
Admin password:        [replaced]
Server Role:           active directory domain controller
Hostname:              dc
NetBIOS Domain:        AD-LAN
DNS Domain:            ad-lan.com
DOMAIN SID:            S-1-5-21-2473926874-590573496-2946143095
Importing WINS database
Importing Account policy
Importing idmap database
Cannot open idmap database, Ignoring: [Errno 2] No such file or 
directory
WARNING: The "syslog" option is deprecated
Adding groups
Importing groups
Group already exists sid=S-1-5-21-2473926874-590573496-2946143095-512, 
groupname=Domain Admins existing_groupname=Domain Admins, Ignoring.
Group already exists sid=S-1-5-21-2473926874-590573496-2946143095-513, 
groupname=Domain Users existing_groupname=Domain Users, Ignoring.
Group already exists sid=S-1-5-21-2473926874-590573496-2946143095-514, 
groupname=Domain Guests existing_groupname=Domain Guests, Ignoring.
Group already exists sid=S-1-5-21-2473926874-590573496-2946143095-515, 
groupname=Domain Computers existing_groupname=Domain Computers, 
Ignoring.
Group already exists sid=S-1-5-32-544, groupname=Administrators 
existing_groupname=Administrators, Ignoring.
Group already exists sid=S-1-5-32-548, groupname=Account Operators 
existing_groupname=Account Operators, Ignoring.
Group already exists sid=S-1-5-32-550, groupname=Print Operators 
existing_groupname=Print Operators, Ignoring.
Group already exists sid=S-1-5-32-551, groupname=Backup Operators 
existing_groupname=Backup Operators, Ignoring.
Group already exists sid=S-1-5-32-552, groupname=Replicators 
existing_groupname=Replicator, Ignoring.
Committing 'add groups' transaction to disk
Adding users
Importing users
Committing 'add users' transaction to disk
Adding users to groups
Committing 'add users to groups' transaction to disk
WARNING: The "syslog" option is deprecated
WARNING: The "syslog" option is deprecated
---------------- output of classic upgrade above

Note: every user belongs to "Domain Users" group, other group 
memberships are lost.

I would appreciate assistance with above, if possible.

Sincerely,
Konstantin

On Wed, 05 Sep 2018 15:26:30 +0700
Konstantin Boyandin via samba <samba at lists.samba.org>
wrote:
> 
> Exactly that. I need to create a separate domain; after all the
> checks are done that switching to it works, the computers will rejoin
> the new domain. Our Samba 3 domain is used for years; since Window 10
> is unable to join it any more, we are finally migrating everything to
> Samba 4.
Then you might as well just provision a new domain, dump your users,
groups etc to a file. Write a script to parse the file and then add
them to your new AD.
 
> Note: every user belongs to "Domain Users" group, other group 
> memberships are lost.
Yes, every AD users primary group is Domain Users, your other problem
is very probably being caused by the way you are trying to bend the
classicupgrade upgrade script 

Rowland

Rowland Penny via samba писал 2018-09-05 15:56:
> On Wed, 05 Sep 2018 15:26:30 +0700
> Konstantin Boyandin via samba <samba at lists.samba.org> wrote:
>> 
>> Exactly that. I need to create a separate domain; after all the
>> checks are done that switching to it works, the computers will rejoin
>> the new domain. Our Samba 3 domain is used for years; since Window 10
>> is unable to join it any more, we are finally migrating everything to
>> Samba 4.
> 
> Then you might as well just provision a new domain, dump your users,
> groups etc to a file. Write a script to parse the file and then add
> them to your new AD.
Current approach does import users and groups; it only fails to assign 
users to groups properly. It can do already, but I would prefer less 
manual interaction.
>> Note: every user belongs to "Domain Users" group, other group
>> memberships are lost.
> 
> Yes, every AD users primary group is Domain Users, your other problem
> is very probably being caused by the way you are trying to bend the
> classicupgrade upgrade script
I am not sure what I am "bending".

The classic upgrade did fail in exactly the same way even when I tried 
to do it literally as the corresponding guide tells:

https://wiki.samba.org/index.php/Migrating_a_Samba_NT4_Domain_to_Samba_AD_(Classic_Upgrade)

(i.e. while keeping same workgroup name)

All I did was to ensure the new domain with unique SID is generated. 
 From the viewpoint of LDAP database, domain SID matches groups/users 
SID, so a) why the above problem b) why classic upgrade *does* copy 
users/groups anyway?

Thanks.

Sincerely,
Konstantin