samba - Oct 2024 - How to set `unicodePwd`? "it's not allowed to set the NT hash password directly"

On Mon, 28 Oct 2024 12:17:02 +0100
William David Edwards via samba <samba at lists.samba.org> wrote:
> I think I might've found a solution while debugging.
> 
> To understand what I'm doing wrong with `unicodePwd`, I'm trying to
> get the LDAP request that LAM does, and compare it to mine.
> 
> As I temporarily switched to an unencrypted connection to be able to 
> dump the payload without a MTIM, Samba -rightfully- says:
> 
> "Password modification over LDAP must be over an encrypted
connection"
> 
> To mitigate this, I set 
> `fAllowPasswordOperationsOverNonSecureConnection` (`dSHeuristic` 13):
> 
> `root at addc-test:~# samba-tool forest directory_service dsheuristics 
> 0000000011001`
> 
> Note that I also set fUserPwdSupport to 1, which I don't believe to
> be needed (as I'm using `unicodePwd`, not `userPassword`), which
> means TRUE according to 
>
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-adts/e5899be4-862e-496f-9a38-33950617d2c5:
> 
> "If this character is neither "0" nor "2", then
the fUserPwdSupport
> heuristic is TRUE. If this character is "2", then the
fUserPwdSupport
> heuristic is FALSE. If this character is "0", then the
> fUserPwdSupport heuristic is FALSE for AD DS and TRUE for AD LDS."
> 
> However, after enabling this heuristic, `userPassword` works. You 
> previously adviced using it instead of `unicodePwd`. This didn't
> work, and the attribute was stored plaintext. I now believe this was
> the case simply because `userPassword` wasn't enabled (I didn't
> realise it requires a heuristic).
> 
> Which begs the question: why does samba-tool go through the trouble
> of transforming the user-specified password into something that's 
> acceptable to `unicodePwd`? 
Because the unicodePwd attribute is used to store the encoded AD
password.
> Is this a historical artifact 
No, it is very much still in use.
> (`userPassword` doesn't look new)? 
It isn't, it comes from rfc2256
> And why would software like
> NextCloud expect one to be an alias of the other?
I have no idea, something they do ? userPassword is not used by AD.
> 
> I'm not expecting any concrete answers, but it's the state my
search
> is in.
It might help us to see just what is going on if you post the entire
code that you are trying to use to set the users password (note setting
and changing a users password are done in different ways).

Rowland

Rowland Penny via samba schreef op 2024-10-28 12:50:
> On Mon, 28 Oct 2024 12:17:02 +0100
> William David Edwards via samba <samba at lists.samba.org> wrote:
> 
>> I think I might've found a solution while debugging.
>> 
>> To understand what I'm doing wrong with `unicodePwd`, I'm
trying to
>> get the LDAP request that LAM does, and compare it to mine.
>> 
>> As I temporarily switched to an unencrypted connection to be able to
>> dump the payload without a MTIM, Samba -rightfully- says:
>> 
>> "Password modification over LDAP must be over an encrypted
connection"
>> 
>> To mitigate this, I set
>> `fAllowPasswordOperationsOverNonSecureConnection` (`dSHeuristic` 13):
>> 
>> `root at addc-test:~# samba-tool forest directory_service dsheuristics
>> 0000000011001`
>> 
>> Note that I also set fUserPwdSupport to 1, which I don't believe to
>> be needed (as I'm using `unicodePwd`, not `userPassword`), which
>> means TRUE according to
>>
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-adts/e5899be4-862e-496f-9a38-33950617d2c5:
>> 
>> "If this character is neither "0" nor "2",
then the fUserPwdSupport
>> heuristic is TRUE. If this character is "2", then the
fUserPwdSupport
>> heuristic is FALSE. If this character is "0", then the
>> fUserPwdSupport heuristic is FALSE for AD DS and TRUE for AD LDS."
>> 
>> However, after enabling this heuristic, `userPassword` works. You
>> previously adviced using it instead of `unicodePwd`. This didn't
>> work, and the attribute was stored plaintext. I now believe this was
>> the case simply because `userPassword` wasn't enabled (I didn't
>> realise it requires a heuristic).
>> 
>> Which begs the question: why does samba-tool go through the trouble
>> of transforming the user-specified password into something that's
>> acceptable to `unicodePwd`?
> 
> Because the unicodePwd attribute is used to store the encoded AD
> password.
According to 
https://microsoft.public.windows.server.active-directory.narkive.com/Vo4nv0wF/difference-between-userpassword-and-unicodepwd:

"unicodePwd is the "real password attribute" [...] userPassword
is
"switchable". It can be turned into a regular attribute, or it can be 
turned into a write-alias for unicodePwd. AD by default has it as a 
regular attribute. ADAM by default has it as a unicodePwd alias. This is 
controlled by the 9th char of dsHeuristics. 0 is the default (different 
in AD w2k3 and ADAM). 1 means "userPassword is write-alias for 
unicodePwd", 2 means "userPassword is a regular attribute". [...]
When
userPassword is a write-alias for unicodePwd, it is written as a regular 
value, no unicode, no double-quotes. However, passwords can never be 
read."

In other words: if `userPassword` is a write-alias for `unicodePwd`, a 
non-encrypted password can be passed, but it can't be read. So, how is 
it relevant that "the unicodePwd attribute is used to store the encoded 
AD password"?

Side-note: 
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-adts/e5899be4-862e-496f-9a38-33950617d2c5
doesn't mention the heuristic having an effect on being a write-alias or 
not, which just confuses me more.
> 
>> Is this a historical artifact
> 
> No, it is very much still in use.
> 
>> (`userPassword` doesn't look new)?
> 
> It isn't, it comes from rfc2256
> 
>> And why would software like
>> NextCloud expect one to be an alias of the other?
> 
> I have no idea, something they do ? userPassword is not used by AD.
What do you mean by "userPassword is not used by AD"? From 
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-adts/f3adda9f-89e1-4340-a3f2-1f0a6249f1f8:

"Active Directory supports modifying passwords on objects via the 
userPassword attribute"
> 
>> 
>> I'm not expecting any concrete answers, but it's the state my
search
>> is in.
> 
> It might help us to see just what is going on if you post the entire
> code that you are trying to use to set the users password (note setting
> and changing a users password are done in different ways).
> 
> Rowland
Met vriendelijke groeten,

William David Edwards

Rowland Penny via samba schreef op 2024-10-28 12:50:
> On Mon, 28 Oct 2024 12:17:02 +0100
> William David Edwards via samba <samba at lists.samba.org> wrote:
> 
>> I think I might've found a solution while debugging.
>> 
>> To understand what I'm doing wrong with `unicodePwd`, I'm
trying to
>> get the LDAP request that LAM does, and compare it to mine.
>> 
>> As I temporarily switched to an unencrypted connection to be able to
>> dump the payload without a MTIM, Samba -rightfully- says:
>> 
>> "Password modification over LDAP must be over an encrypted
connection"
>> 
>> To mitigate this, I set
>> `fAllowPasswordOperationsOverNonSecureConnection` (`dSHeuristic` 13):
>> 
>> `root at addc-test:~# samba-tool forest directory_service dsheuristics
>> 0000000011001`
>> 
>> Note that I also set fUserPwdSupport to 1, which I don't believe to
>> be needed (as I'm using `unicodePwd`, not `userPassword`), which
>> means TRUE according to
>>
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-adts/e5899be4-862e-496f-9a38-33950617d2c5:
>> 
>> "If this character is neither "0" nor "2",
then the fUserPwdSupport
>> heuristic is TRUE. If this character is "2", then the
fUserPwdSupport
>> heuristic is FALSE. If this character is "0", then the
>> fUserPwdSupport heuristic is FALSE for AD DS and TRUE for AD LDS."
>> 
>> However, after enabling this heuristic, `userPassword` works. You
>> previously adviced using it instead of `unicodePwd`. This didn't
>> work, and the attribute was stored plaintext. I now believe this was
>> the case simply because `userPassword` wasn't enabled (I didn't
>> realise it requires a heuristic).
>> 
>> Which begs the question: why does samba-tool go through the trouble
>> of transforming the user-specified password into something that's
>> acceptable to `unicodePwd`?
> 
> Because the unicodePwd attribute is used to store the encoded AD
> password.
> 
>> Is this a historical artifact
> 
> No, it is very much still in use.
> 
>> (`userPassword` doesn't look new)?
> 
> It isn't, it comes from rfc2256
> 
>> And why would software like
>> NextCloud expect one to be an alias of the other?
> 
> I have no idea, something they do ? userPassword is not used by AD.
> 
>> 
>> I'm not expecting any concrete answers, but it's the state my
search
>> is in.
> 
> It might help us to see just what is going on if you post the entire
> code that you are trying to use to set the users password (note setting
> and changing a users password are done in different ways).
> 
I missed this question in the previous reply, sorry.

First of all, I'm not aware of a difference between setting and changing 
passwords. Can you explain? I *am* aware that 'resetting' (as a 
highly-privileged user) and changing a password (as the user itself) are 
different things. Number one is going on here.

Regarding code: this is how I'm generating `unicodePwd`:

```
import base64

password = 'foobar'

password = ('"' + password +
'"').encode('utf-16-le')

password = base64.b64encode(password).decode('utf-8')

return {'ldap_formatted_password': password}
```

(Taken from samba-tool; see OP.)

There is no code for the request itself to speak of, as I'm using an 
LDAP client that I have little control over. The resulting LDAP request 
was shared in one of my earlier messages, though.
> Rowland
Met vriendelijke groeten,

William David Edwards