samba - Jul 2022 - ldbmodify sometimes fails when changing attribute "unicodePwd" depending on line order in LDIF file

Dear list members,

In our Samba-DC setup we have a shell-script which regularly adds /
modifies / deletes user accounts by importing LDIF files with changes.
The script runs directly on the Samba-DC, and has been serving our use-
case well and without problems for the last 5+ years. Originally, this
was deployed on an Ubuntu-18.04 system with Samba-4.7.6-Ubuntu, but we
are currently preparing the migration to Debian (stable / Bullseye)
which comes with Samba-4.13.13-Debian.?

While the initial deployment on Bullseye (done via Puppet) ran without
modifications, the script that imports changes did not run anymore, if
the LDIF file contains changes of the 'unicodePwd' attribute - but, as
we later found out, ONLY if the lines in the LDIF file have a certain
order. 

The below LDIF file can be imported without errors on Ubuntu-18.04 with
Samba-4.7.6:

$ cat password.ldif

# passwords
dn:CN=username,OU=group,DC=domain,DC=example,DC=com
changetype: modify
replace: unicodePwd
replace: pwdLastSet
unicodepwd::B4ek4PI4I1IExnPVfZz+Ag=pwdLastSet:132868855091315424

The actual command invocation in the script is as follows:
cat password.ldif | /usr/bin/ldbmodify -H
/var/lib/samba/private/sam.ldb --verbose --controls=relax:0 --
controls=local_oid:1.3.6.1.4.1.7165.4.3.7:0 --
controls=local_oid:1.3.6.1.4.1.7165.4.3.12:0

With the new setup (on Debian with Samba-4.13.13), the import of this
exact same LDIF file gives one of the following 2 results (usually the
first, but in some cases we've seen the second one as well):

- ERR: (Constraint violation) "count > max at
../../source4/dsdb/samdb/ldb_modules/password_hash.c:261" on DN
CN=username,OU=group,DC=domain,DC=example,DC=com at block before line 8
Modify failed after processing 0 records

- ERR: (Constraint violation) "length % 16 != 0 at
../../source4/dsdb/samdb/ldb_modules/password_hash.c:261" on DN
CN=username,OU=group,DC=domain,DC=example,DC=com at block before line 8
Modify failed after processing 0 records

We weren't able to determine why we sometimes see the "count >
max"
error, and sometimes the "length % 16 != 0" error though, but we think
it might have to do with the other attributes in the change-set
(unconfirmed).


Anyway, after much trial and error we were able to "solve" this by
changing the order of the lines in the LDIF like so:

$ cat password.ldif

# passwords
dn:CN=username,OU=group,DC=domain,DC=example,DC=com
changetype: modify
replace: unicodePwd
unicodepwd::B4ek4PI4I1IExnPVfZz+Ag=replace: pwdLastSet
pwdLastSet:132868855091315424

With this re-ordered LDIF file, the ldbmodify command runs without
errors:
Modified CN=username,CN=group,DC=domain,DC=example,DC=com
Modified 1 records successfully



Some further observations that might or might not help:

1. We also tried the import on another Debian system with Samba-4.15.7-
Debian => same error as with Samba-4.13.13

2. We played around a bit with other attributes, but it seems this
problem only happens when "unicodePwd" attribute changes are included,
AND if the lines for "replace: unicodePwd" and "unicodePwd:
xxxxxxxxxxxxxxxxxxxxxx==" are NOT consecutive.?

Examples:

# testcase1
changetype: modify
replace: countryCode
replace: unicodePwd
countryCode: 276
unicodePwd:: OF6bXmbaYVv4who3PG1JXQ=
Result NOT OK:
ERR: (Constraint violation) "count > max at
../../source4/dsdb/samdb/ldb_modules/password_hash.c:261" on DN
CN=username,CN=group,DC=domain,DC=example,DC=com at block before line 6
Modify failed after processing 0 records


# testcase2
changetype: modify
replace: countryCode
replace: codePage
replace: pwdLastSet
countryCode: 276
codePage: 850
pwdLastSet: 133011596276122110
replace: unicodePwd
unicodePwd:: OF6bXmbaYVv4who3PG1JXQ=
Result OK:
Modified CN=username,CN=group,DC=domain,DC=example,DC=com
Modified 1 records successfully
 

Also working (add / replace / delete in same LDIF):

# testcase3
changetype: modify
replace: countryCode
countryCode: 276
delete: unicodePwd
add: unicodePwd
unicodePwd:: OF6bXmbaXRv4who3PG1JXQ=
Result OK:
Modified CN=username,CN=group,DC=domain,DC=example,DC=com
Modified 1 records successfully


Since the error message is a bit counter-intuitive - the problem is not
with the content / complexity / length of the actual unicodePwd - we
thought it makes sense to ask on the mailing list if this is intended
behaviour or a bug??

The LDIF specification is a bit unclear / does not define order of
modify commands, but as far as we can tell it should not be invalid?

Either way, this import script and the underlying business process are
important to us, so we just want to make sure that this won't become a
problem again in the future, and of course we'd be interested to
understand better what is going on here.

Thanks for any help / insights / leads!

Best,
Stefan

On Mon, 2022-07-04 at 12:38 +0000, Heil, Stefan via samba
wrote:
> Dear list members,
> 
> In our Samba-DC setup we have a shell-script which regularly adds /
> modifies / deletes user accounts by importing LDIF files with
> changes.
> The script runs directly on the Samba-DC, and has been serving our
> use-
> case well and without problems for the last 5+ years. Originally,
> this
> was deployed on an Ubuntu-18.04 system with Samba-4.7.6-Ubuntu, but
> we
> are currently preparing the migration to Debian (stable / Bullseye)
> which comes with Samba-4.13.13-Debian. 
> 
> While the initial deployment on Bullseye (done via Puppet) ran
> without
> modifications, the script that imports changes did not run anymore,
> if
> the LDIF file contains changes of the 'unicodePwd' attribute - but,
> as
> we later found out, ONLY if the lines in the LDIF file have a certain
> order. 
> 
> The below LDIF file can be imported without errors on Ubuntu-18.04
> with
> Samba-4.7.6:
> 
> $ cat password.ldif
> 
> # passwords
> dn:CN=username,OU=group,DC=domain,DC=example,DC=com
> changetype: modify
> replace: unicodePwd
> replace: pwdLastSet
> unicodepwd::B4ek4PI4I1IExnPVfZz+Ag=> pwdLastSet:132868855091315424
You shouldn't have to set 'pwdLastSet', it should be done for you
when
the password is changed, having said that, why are you doing it this
way ? Wouldn't it be better to use samba-tool ?

Rowland

On Mon, 2022-07-04 at 12:38 +0000, Heil, Stefan via samba
wrote:
> Dear list members,
> 
> In our Samba-DC setup we have a shell-script which regularly adds /
> modifies / deletes user accounts by importing LDIF files with
> changes.
> The script runs directly on the Samba-DC, and has been serving our
> use-
> case well and without problems for the last 5+ years. Originally,
> this
> was deployed on an Ubuntu-18.04 system with Samba-4.7.6-Ubuntu, but
> we
> are currently preparing the migration to Debian (stable / Bullseye)
> which comes with Samba-4.13.13-Debian. 
So what has happened is that since Samba 4.13 we have become much more
strict in the handling of password updates, as we work to avoid
database corruption, even when users like yourself set passwords
'directly' in the DB with the internal controls.

By very strictly restricting the values, we try not to have more
security issues in this code (we have been bitten in the past). 
> While the initial deployment on Bullseye (done via Puppet) ran
> without
> modifications, the script that imports changes did not run anymore,
> if
> the LDIF file contains changes of the 'unicodePwd' attribute - but,
> as
> we later found out, ONLY if the lines in the LDIF file have a certain
> order. 
> 
> The below LDIF file can be imported without errors on Ubuntu-18.04
> with
> Samba-4.7.6:
> 
> $ cat password.ldif
> 
> # passwords
> dn:CN=username,OU=group,DC=domain,DC=example,DC=com
> changetype: modify
> replace: unicodePwd
> replace: pwdLastSet
> unicodepwd::B4ek4PI4I1IExnPVfZz+Ag=> pwdLastSet:132868855091315424
This is not valid LDIF for thie operation.  This LDIF becomes:

dn:CN=username,OU=group,DC=domain,DC=example,DC=com
changetype: modify
replace: unicodePwd
unicodePwd: <no value>
replace: pwdLastSet
pwdLastSet: <no value>
add: unicodePwd
unicodepwd::B4ek4PI4I1IExnPVfZz+Ag=add: pwdLastSet
pwdLastSet:132868855091315424
> The actual command invocation in the script is as follows:
> cat password.ldif | /usr/bin/ldbmodify -H
> /var/lib/samba/private/sam.ldb --verbose --controls=relax:0 --
> controls=local_oid:1.3.6.1.4.1.7165.4.3.7:0 --
> controls=local_oid:1.3.6.1.4.1.7165.4.3.12:0
> 
> With the new setup (on Debian with Samba-4.13.13), the import of this
> exact same LDIF file gives one of the following 2 results (usually
> the
> first, but in some cases we've seen the second one as well):
> 
> - ERR: (Constraint violation) "count > max at
> ../../source4/dsdb/samdb/ldb_modules/password_hash.c:261" on DN
> CN=username,OU=group,DC=domain,DC=example,DC=com at block before line
> 8
> Modify failed after processing 0 records
> 
> - ERR: (Constraint violation) "length % 16 != 0 at
> ../../source4/dsdb/samdb/ldb_modules/password_hash.c:261" on DN
> CN=username,OU=group,DC=domain,DC=example,DC=com at block before line
> 8
> Modify failed after processing 0 records
> 
> We weren't able to determine why we sometimes see the "count >
max"
> error, and sometimes the "length % 16 != 0" error though, but we
> think
> it might have to do with the other attributes in the change-set
> (unconfirmed).
We don't like seeing the unicodePwd more than once, it makes us
worried, and as specifying an internal control is a local system only
operation (so not a network behaviour we have to preserve) we restrict
it. 
> Anyway, after much trial and error we were able to "solve" this
by
> changing the order of the lines in the LDIF like so:
> 
> $ cat password.ldif
> 
> # passwords
> dn:CN=username,OU=group,DC=domain,DC=example,DC=com
> changetype: modify
> replace: unicodePwd
> unicodepwd::B4ek4PI4I1IExnPVfZz+Ag=> replace: pwdLastSet
> pwdLastSet:132868855091315424
This looks reasonable and matches what you were wanting to do. 
> With this re-ordered LDIF file, the ldbmodify command runs without
> errors:
> Modified CN=username,CN=group,DC=domain,DC=example,DC=com
> Modified 1 records successfully
> 
> 
> 
> Some further observations that might or might not help:
> 
> 1. We also tried the import on another Debian system with Samba-
> 4.15.7-
> Debian => same error as with Samba-4.13.13
> 
> 2. We played around a bit with other attributes, but it seems this
> problem only happens when "unicodePwd" attribute changes are
> included,
> AND if the lines for "replace: unicodePwd" and "unicodePwd:
> xxxxxxxxxxxxxxxxxxxxxx==" are NOT consecutive. 
Correct. 
> 
> 
> Since the error message is a bit counter-intuitive - the problem is
> not
> with the content / complexity / length of the actual unicodePwd - we
> thought it makes sense to ask on the mailing list if this is intended
> behaviour or a bug? 
Intended, just not intended that users would see it.  Patches to add a
better error message are welcome. 
> The LDIF specification is a bit unclear / does not define order of
> modify commands, but as far as we can tell it should not be invalid?
> 
> Either way, this import script and the underlying business process
> are
> important to us, so we just want to make sure that this won't become
> a
> problem again in the future, and of course we'd be interested to
> understand better what is going on here.
> 
> Thanks for any help / insights / leads!
Just specify the attributes exactly once and things should be fine. 

However I have concerns.  The bypass method you use was set up for
migrations from Samba3 into an empty domain.

If your domain is already in operation there may be a password history
(ntPwdHistory, lmPwdHistory) and Kerberos passwords
(supplementalCrednetials).  If these are not reset, you could end up
with both synced and unsynced passwords available.

Andrew Bartlett

-- 
Andrew Bartlett (he/him)       https://samba.org/~abartlet/
Samba Team Member (since 2001) https://samba.org
Samba Team Lead, Catalyst IT   https://catalyst.net.nz/services/samba

Samba Development and Support, Catalyst IT - Expert Open Source
Solutions