samba - Oct 2024 - How to set `unicodePwd`? "it's not allowed to set the NT hash password directly"

Op 27-10-2024 om 21:11 schreef William David Edwards:
> Kees van Vloten schreef op 2024-10-27 20:45:
>> Op 27-10-2024 om 19:58 schreef William David Edwards:
>>> Kees van Vloten via samba schreef op 2024-10-27 15:37:
>>>> Op 27-10-2024 om 15:31 schreef Rowland Penny via samba:
>>>>> On Sun, 27 Oct 2024 15:08:14 +0100
>>>>> William Edwards <wedwards at cyberfusion.nl> wrote:
>>>>>
>>>>>>> Op 27 okt 2024 om 14:50 heeft Rowland Penny via
samba
>>>>>>> <samba at lists.samba.org> het volgende
geschreven:
>>>>>>>
>>>>>>> ?On Sun, 27 Oct 2024 13:58:56 +0100
>>>>>>> William David Edwards via samba <samba at
lists.samba.org> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I'm tying to set user's LDAP passwords
using LDAP.
>>>>>>>>
>>>>>>>> `samba-tool user setpassword` does so by
setting the write-only
>>>>>>>> `unicodePwd` attribute, but turning it into
binary and
>>>>>>>> Base64-encoding it first:
>>>>>>>>
>>>>>>>> ```
>>>>>>>> if not isinstance(password, str):
>>>>>>>> ???? pw = password.decode('utf-8')
>>>>>>>> else:
>>>>>>>> ???? pw = password
>>>>>>>> pw = ('"' + pw +
'"').encode('utf-16-le')
>>>>>>>>
>>>>>>>> setpw = """
>>>>>>>> dn: %s
>>>>>>>> changetype: modify
>>>>>>>> replace: unicodePwd
>>>>>>>> unicodePwd:: %s
>>>>>>>> """ % (user_dn,
base64.b64encode(pw).decode('utf-8'))
>>>>>>>> ```
>>>>>>>>
>>>>>>>> When doing the same, Samba returns:
>>>>>>>>
>>>>>>>> ???? 00002035: setup_io: it's not allowed
to set the NT hash
>>>>>>>> password directly' Code: 0x35
>>>>>>>>
>>>>>>>> This happens both when 1) passing `unicodePwd`
during entry
>>>>>>>> creation, and 2) when modifying it for an
existing entry (like
>>>>>>>> `samba-tool` does).
>>>>>>>>
>>>>>>>> This is the (Wireshark-interpreted)
`ModifyRequest`:
>>>>>>>>
>>>>>>>> ```
>>>>>>>> Lightweight Directory Access Protocol
>>>>>>>> ???? LDAPMessage modifyRequest(3)
>>>>>>>>
"CN=williamedwards1730031523476,CN=Users,DC=ldaptest,DC=nl"
>>>>>>>> ???????? messageID: 3
>>>>>>>> ???????? protocolOp: modifyRequest (6)
>>>>>>>> ???????????? modifyRequest
>>>>>>>> ???????????????? object:
>>>>>>>>
CN=williamedwards1730031523476,CN=Users,DC=ldaptest,DC=nl
>>>>>>>> ???????????????? modification: 1 item
>>>>>>>> ???????????????????? modification item
>>>>>>>> ???????????????????????? operation: replace (2)
>>>>>>>> ???????????????????????? modification
unicodePwd
>>>>>>>> ???????????????????????????? type: unicodePwd
>>>>>>>> ???????????????????????????? vals: 1 item
>>>>>>>> ????????????????????????????????
AttributeValue:
>>>>>>>>
IgARADwACAAiAHwATwBZAFAAdAB0AGQARQBKAE8AawBVAHoAfwBqAFwANAAtAGYAXAA4ADoADgBcAEsAKwBGAFoAIgA=
>>>>>>>> ```
>>>>>>>>
>>>>>>>> (The Base64-encoded password was randomly
generated.)
>>>>>>>>
>>>>>>>> The use of the word 'directly' in
"Not allowed to [...] directly"
>>>>>>>> could be interpreted as: changing `unicodePwd`
non-locally is
>>>>>>>> disallowed (although Samba can't detect
that), and that is indeed
>>>>>>>> not the case: LDAP Account Manager
>>>>>>>> (https://ldap-account-manager.org/lamcms/) uses
the same approach
>>>>>>>> of modifying `unicodePwd`, which works on the
same DC.
>>>>>>>>
>>>>>>>> Looking this error up, it seems like it can be
caused by simply
>>>>>>>> passing an incorrectly formatted password:
>>>>>>>>
>>>>>>>> -
https://lists.samba.org/archive/samba/2015-December/196890.html
>>>>>>>> -
>>>>>>>>
https://lsc-users.lsc-project.narkive.com/3Ltw5zOZ/pushing-a-password-to-samba-4#post1
>>>>>>>> ... but as I'm using the code from
`samba-tool`, I don't consider
>>>>>>>> that to be likely.
>>>>>>>>
>>>>>>>> Finally, I tried passing a plaintext string,
hoping Samba would do
>>>>>>>> the transformation to binary + Base64 itself
(although neither
>>>>>>>> `samba-tool` nor LAM do so). As expected, that
changes nothing.
>>>>>>>>
>>>>>>>> What am I doing wrong or misunderstanding here?
>>>>>>> You are missing that to change an active directory
password over
>>>>>>> ldap, you don't use ldap, you use ldaps.
>>>>>> That?s not the issue, just tested LDAPS. I?m using LDAP
for
>>>>>> debugging
>>>>>> purposes (no need for a MITM to look at the payload).
>>>>>>
>>>> Did you enable password change via ldap? :
>>>>
>>>> samba-tool forest directory_service dsheuristics
'000000001'
>>>
>>> According to 
>>>
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-adts/e5899be4-862e-496f-9a38-33950617d2c5,
>>> a dSHeuristic is required only for changing passwords over 
>>> unencrypted LDAP
(`fAllowPasswordOperationsOverNonSecureConnection`).
>> Above link talks about AD DS vs. AD LDS (where the latter refers to 
>> ldap, unclear what the first is). At the same that ldap must be over 
>> ssl/tls, as is mentioned 
>>
here:https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-adts/f3adda9f-89e1-4340-a3f2-1f0a6249f1f8?redirectedfrom=MSDN.
>> This also states:
>>
>> The special encoding required for updating the unicodePwd attribute 
>> is not used with the userPassword attribute; that is, Vpassword = V. 
>> The same restrictions on SSL/TLS- or SASL-protected connections are 
>> enforced. The password values are sent to the server as UTF-8 
>> strings, and surrounding quotation marks are not used.
>>
>> It looks like 'userPassword' is easier to use. It is the
attribute
>> that e.g. NextCloud and Self-Service-Password use for password changes.
>
> Tested `userPassword`. Logging in with the set password leads to an 
> authentication failure. Also, the attribute is shown/stored as 
> plaintext, which is obviously unacceptable (and probably the need for 
> a heuristic to set this attribute an unencrypted connection).
>
> I found 
> https://docs.pingidentity.com/pingds/7.3/config-guide/samba.html, 
> which says:
>
> "When you store Samba accounts in a directory server, Samba stores its
> own attributes as defined in the Samba schema. Samba does not use the 
> LDAP standard userPassword attribute to store users' Samba passwords. 
> You can configure Samba to apply changes to Samba passwords to LDAP 
> passwords as well. Yet, if a user modifies their LDAP password 
> directly without updating the Samba password, the LDAP and Samba 
> passwords get out of sync."
>
> Not sure how authoritative this source is, but the Samba docs don't 
> seem to mention `userPassword` in any relevant context.
>
> Additionally, the examples you mentioned (NextCloud and 
> Self-Service-Password) don't seem to use `userPassword` per se. For 
> Self-Service-Password, it's configurable: 
>
https://github.com/ltb-project/self-service-password/blob/c89946a51396d0f2c27142cec7708540a088ba08/lib/functions.inc.php#L88.

That's nice, you have a working example here :-)

> And NextCloud expects `userPassword` to be an alias of `unicodePwd` 
> (mentioned on 
>
https://docs.nextcloud.com/server/latest/admin_manual/configuration_user/user_auth_ldap.html
> under "Additional requirements for Active Directory").
>
>>
>> - Kees
>>
>>>
>>> As mentioned, modifying `unicodePwd` does not work over LDAPS
either
>>> in my specific case, so a heuristic should not be needed. Also, 
>>> changing passwords the same way *does* work from samba-tool and
LAM.
>>>
>>>>
>>>> - Kees.
>>>>
>>>>
>>>>> Try reading this:
>>>>>
>>>>>
https://learn.microsoft.com/en-us/troubleshoot/windows-server/active-directory/change-windows-active-directory-user-password
>>>>> Rowland
>>>>>
>>>
>>> Met vriendelijke groeten,
>>>
>>> William David Edwards
>>>
>
> Met vriendelijke groeten,
>
> William David Edwards
>

Hi Kees,

Kees van Vloten schreef op 2024-10-27 22:17:
> Op 27-10-2024 om 21:11 schreef William David Edwards:
>> Kees van Vloten schreef op 2024-10-27 20:45:
>>> Op 27-10-2024 om 19:58 schreef William David Edwards:
>>>> Kees van Vloten via samba schreef op 2024-10-27 15:37:
>>>>> Op 27-10-2024 om 15:31 schreef Rowland Penny via samba:
>>>>>> On Sun, 27 Oct 2024 15:08:14 +0100
>>>>>> William Edwards <wedwards at cyberfusion.nl>
wrote:
>>>>>> 
>>>>>>>> Op 27 okt 2024 om 14:50 heeft Rowland Penny via
samba
>>>>>>>> <samba at lists.samba.org> het volgende
geschreven:
>>>>>>>> 
>>>>>>>> ?On Sun, 27 Oct 2024 13:58:56 +0100
>>>>>>>> William David Edwards via samba <samba at
lists.samba.org> wrote:
>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> I'm tying to set user's LDAP
passwords using LDAP.
>>>>>>>>> 
>>>>>>>>> `samba-tool user setpassword` does so by
setting the write-only
>>>>>>>>> `unicodePwd` attribute, but turning it into
binary and
>>>>>>>>> Base64-encoding it first:
>>>>>>>>> 
>>>>>>>>> ```
>>>>>>>>> if not isinstance(password, str):
>>>>>>>>> ???? pw = password.decode('utf-8')
>>>>>>>>> else:
>>>>>>>>> ???? pw = password
>>>>>>>>> pw = ('"' + pw +
'"').encode('utf-16-le')
>>>>>>>>> 
>>>>>>>>> setpw = """
>>>>>>>>> dn: %s
>>>>>>>>> changetype: modify
>>>>>>>>> replace: unicodePwd
>>>>>>>>> unicodePwd:: %s
>>>>>>>>> """ % (user_dn,
base64.b64encode(pw).decode('utf-8'))
>>>>>>>>> ```
>>>>>>>>> 
>>>>>>>>> When doing the same, Samba returns:
>>>>>>>>> 
>>>>>>>>> ???? 00002035: setup_io: it's not
allowed to set the NT hash
>>>>>>>>> password directly' Code: 0x35
>>>>>>>>> 
>>>>>>>>> This happens both when 1) passing
`unicodePwd` during entry
>>>>>>>>> creation, and 2) when modifying it for an
existing entry (like
>>>>>>>>> `samba-tool` does).
>>>>>>>>> 
>>>>>>>>> This is the (Wireshark-interpreted)
`ModifyRequest`:
>>>>>>>>> 
>>>>>>>>> ```
>>>>>>>>> Lightweight Directory Access Protocol
>>>>>>>>> ???? LDAPMessage modifyRequest(3)
>>>>>>>>>
"CN=williamedwards1730031523476,CN=Users,DC=ldaptest,DC=nl"
>>>>>>>>> ???????? messageID: 3
>>>>>>>>> ???????? protocolOp: modifyRequest (6)
>>>>>>>>> ???????????? modifyRequest
>>>>>>>>> ???????????????? object:
>>>>>>>>>
CN=williamedwards1730031523476,CN=Users,DC=ldaptest,DC=nl
>>>>>>>>> ???????????????? modification: 1 item
>>>>>>>>> ???????????????????? modification item
>>>>>>>>> ???????????????????????? operation: replace
(2)
>>>>>>>>> ???????????????????????? modification
unicodePwd
>>>>>>>>> ???????????????????????????? type:
unicodePwd
>>>>>>>>> ???????????????????????????? vals: 1 item
>>>>>>>>> ????????????????????????????????
AttributeValue:
>>>>>>>>>
IgARADwACAAiAHwATwBZAFAAdAB0AGQARQBKAE8AawBVAHoAfwBqAFwANAAtAGYAXAA4ADoADgBcAEsAKwBGAFoAIgA=
>>>>>>>>> ```
>>>>>>>>> 
>>>>>>>>> (The Base64-encoded password was randomly
generated.)
>>>>>>>>> 
>>>>>>>>> The use of the word 'directly' in
"Not allowed to [...]
>>>>>>>>> directly"
>>>>>>>>> could be interpreted as: changing
`unicodePwd` non-locally is
>>>>>>>>> disallowed (although Samba can't detect
that), and that is
>>>>>>>>> indeed
>>>>>>>>> not the case: LDAP Account Manager
>>>>>>>>> (https://ldap-account-manager.org/lamcms/)
uses the same
>>>>>>>>> approach
>>>>>>>>> of modifying `unicodePwd`, which works on
the same DC.
>>>>>>>>> 
>>>>>>>>> Looking this error up, it seems like it can
be caused by simply
>>>>>>>>> passing an incorrectly formatted password:
>>>>>>>>> 
>>>>>>>>> - 
>>>>>>>>>
https://lists.samba.org/archive/samba/2015-December/196890.html
>>>>>>>>> -
>>>>>>>>>
https://lsc-users.lsc-project.narkive.com/3Ltw5zOZ/pushing-a-password-to-samba-4#post1
>>>>>>>>> ... but as I'm using the code from
`samba-tool`, I don't
>>>>>>>>> consider
>>>>>>>>> that to be likely.
>>>>>>>>> 
>>>>>>>>> Finally, I tried passing a plaintext
string, hoping Samba would
>>>>>>>>> do
>>>>>>>>> the transformation to binary + Base64
itself (although neither
>>>>>>>>> `samba-tool` nor LAM do so). As expected,
that changes nothing.
>>>>>>>>> 
>>>>>>>>> What am I doing wrong or misunderstanding
here?
>>>>>>>> You are missing that to change an active
directory password over
>>>>>>>> ldap, you don't use ldap, you use ldaps.
>>>>>>> That?s not the issue, just tested LDAPS. I?m using
LDAP for
>>>>>>> debugging
>>>>>>> purposes (no need for a MITM to look at the
payload).
>>>>>>> 
>>>>> Did you enable password change via ldap? :
>>>>> 
>>>>> samba-tool forest directory_service dsheuristics
'000000001'
>>>> 
>>>> According to 
>>>>
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-adts/e5899be4-862e-496f-9a38-33950617d2c5,
>>>> a dSHeuristic is required only for changing passwords over 
>>>> unencrypted LDAP 
>>>> (`fAllowPasswordOperationsOverNonSecureConnection`).
>>> Above link talks about AD DS vs. AD LDS (where the latter refers to
>>> ldap, unclear what the first is). At the same that ldap must be
over
>>> ssl/tls, as is mentioned 
>>>
here:https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-adts/f3adda9f-89e1-4340-a3f2-1f0a6249f1f8?redirectedfrom=MSDN.
>>> This also states:
>>> 
>>> The special encoding required for updating the unicodePwd attribute
>>> is not used with the userPassword attribute; that is, Vpassword =
V.
>>> The same restrictions on SSL/TLS- or SASL-protected connections are
>>> enforced. The password values are sent to the server as UTF-8 
>>> strings, and surrounding quotation marks are not used.
>>> 
>>> It looks like 'userPassword' is easier to use. It is the
attribute
>>> that e.g. NextCloud and Self-Service-Password use for password 
>>> changes.
>> 
>> Tested `userPassword`. Logging in with the set password leads to an 
>> authentication failure. Also, the attribute is shown/stored as 
>> plaintext, which is obviously unacceptable (and probably the need for 
>> a heuristic to set this attribute an unencrypted connection).
>> 
>> I found 
>> https://docs.pingidentity.com/pingds/7.3/config-guide/samba.html, 
>> which says:
>> 
>> "When you store Samba accounts in a directory server, Samba stores
its
>> own attributes as defined in the Samba schema. Samba does not use the 
>> LDAP standard userPassword attribute to store users' Samba
passwords.
>> You can configure Samba to apply changes to Samba passwords to LDAP 
>> passwords as well. Yet, if a user modifies their LDAP password 
>> directly without updating the Samba password, the LDAP and Samba 
>> passwords get out of sync."
>> 
>> Not sure how authoritative this source is, but the Samba docs don't
>> seem to mention `userPassword` in any relevant context.
>> 
>> Additionally, the examples you mentioned (NextCloud and 
>> Self-Service-Password) don't seem to use `userPassword` per se. For
>> Self-Service-Password, it's configurable: 
>>
https://github.com/ltb-project/self-service-password/blob/c89946a51396d0f2c27142cec7708540a088ba08/lib/functions.inc.php#L88.
> 
> 
> That's nice, you have a working example here :-)
I think I might've found a solution while debugging.

To understand what I'm doing wrong with `unicodePwd`, I'm trying to get 
the LDAP request that LAM does, and compare it to mine.

As I temporarily switched to an unencrypted connection to be able to 
dump the payload without a MTIM, Samba -rightfully- says:

"Password modification over LDAP must be over an encrypted connection"

To mitigate this, I set 
`fAllowPasswordOperationsOverNonSecureConnection` (`dSHeuristic` 13):

`root at addc-test:~# samba-tool forest directory_service dsheuristics 
0000000011001`

Note that I also set fUserPwdSupport to 1, which I don't believe to be 
needed (as I'm using `unicodePwd`, not `userPassword`), which means TRUE 
according to 
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-adts/e5899be4-862e-496f-9a38-33950617d2c5:

"If this character is neither "0" nor "2", then the
fUserPwdSupport
heuristic is TRUE. If this character is "2", then the fUserPwdSupport 
heuristic is FALSE. If this character is "0", then the fUserPwdSupport
heuristic is FALSE for AD DS and TRUE for AD LDS."

However, after enabling this heuristic, `userPassword` works. You 
previously adviced using it instead of `unicodePwd`. This didn't work, 
and the attribute was stored plaintext. I now believe this was the case 
simply because `userPassword` wasn't enabled (I didn't realise it 
requires a heuristic).

Which begs the question: why does samba-tool go through the trouble of 
transforming the user-specified password into something that's 
acceptable to `unicodePwd`? Is this a historical artifact 
(`userPassword` doesn't look new)? And why would software like NextCloud 
expect one to be an alias of the other?

I'm not expecting any concrete answers, but it's the state my search is 
in.
> 
> 
>> And NextCloud expects `userPassword` to be an alias of `unicodePwd` 
>> (mentioned on 
>>
https://docs.nextcloud.com/server/latest/admin_manual/configuration_user/user_auth_ldap.html
>> under "Additional requirements for Active Directory").
>> 
>>> 
>>> - Kees
>>> 
>>>> 
>>>> As mentioned, modifying `unicodePwd` does not work over LDAPS
either
>>>> in my specific case, so a heuristic should not be needed. Also,
>>>> changing passwords the same way *does* work from samba-tool and
LAM.
>>>> 
>>>>> 
>>>>> - Kees.
>>>>> 
>>>>> 
>>>>>> Try reading this:
>>>>>> 
>>>>>>
https://learn.microsoft.com/en-us/troubleshoot/windows-server/active-directory/change-windows-active-directory-user-password
>>>>>> Rowland
>>>>>> 
>>>> 
>>>> Met vriendelijke groeten,
>>>> 
>>>> William David Edwards
>>>> 
>> 
>> Met vriendelijke groeten,
>> 
>> William David Edwards
>> 
Met vriendelijke groeten,

William David Edwards