samba - Oct 2019 - samba-tool user syncpasswords crashes with python3

the script works...




root at probe28:~# cat test.dat 
dn: CN=userxxxx,OU=Users,OU=xxxxx,OU=Organizations,DC=xxxxx,DC=xxxxx
proxyAddresses: SMTP:xxxxx.xxxxx at xxxxx.org
objectGUID: 637f4e70-8c1e-4e89-a6fc-82d525e584f2
pwdLastSet: 0
objectSid: S-1-5-21-1608159440-4144762864-1017073214-27184
sAMAccountName: userxxxx
mail: xxxxx.xxxxx at xxxxx.xxxxx
userAccountControl: 514
virtualClearTextUTF8:: S2VubndvcnQx

root at probe28:~# 
root at probe28:~# 
root at probe28:~# 
root at probe28:~# cat test.dat | /usr/local/bin/syncpw.py
DONE-EXIT: change password to userxxxx
root at probe28:~# 


Am Freitag, den 04.10.2019, 13:53 +0100 schrieb Rowland penny via
samba:
> On 04/10/2019 12:03, Heinz H?lzl via samba wrote:
> > Hi,
> > 
> > i sync the passwords from samba to other backends using
"samba-tool
> > user syncpasswords"
> > 
> > On my operative system (samba 4.10 and python2) all works fine.
> > I upgraded my test-DC to samba 4.11 and python3 and now the samba-
> > tool
> > user syncpasswords --daemon crashes.
> > 
> > Fri Oct  4 12:29:47 2019: pid[983]: Attached to
> > logfile[/usr/local/samba/var/log.syncpw]
> > Fri Oct  4 12:29:47 2019: pid[983]: Using
> > cache_ldb[/usr/local/samba/private/user-syncpasswords-cache.ldb]
> > Fri Oct  4 12:29:47 2019: pid[985]: Daemonized as pid 985 (from
> > 983)
> > Fri Oct  4 12:29:47 2019: pid[985]: Using
> > cache_ldb[/usr/local/samba/private/user-syncpasswords-cache.ldb]
> > Fri Oct  4 12:29:47 2019: pid[985]: currentPid: 985
> > Fri Oct  4 12:29:47 2019: pid[985]: Wait before connect - sleep(1)
> > Fri Oct  4 12:29:48 2019: pid[985]: Connecting to
> > 'ldapi:///usr/local/samba/private/ldap_priv/ldapi'
> > Fri Oct  4 12:29:48 2019: pid[985]: Resuming monitoring
> > dirsyncFilter:
> >
(&(objectClass=user)(userAccountControl:1.2.840.113556.1.4.803:=512
> > )(!(
> > sAMAccountName=krbtgt*)))
> > dirsyncControls: ['dirsync:1:0:0', 'extended_dn:1:0']
> > syncCommand: /usr/local/bin/syncpw.py
> > Fri Oct  4 12:29:52 2019: pid[985]: dirsync_loop(): results 15220
> > Fri Oct  4 12:29:52 2019: pid[985]: # Dirsync[0] 7d1b0000-b7f7-
> > 4fda-
> > 8479-b5cb70a01030 S-1-5-21-1608159440-4144762864-1017073214-15729
> > dn:
<GUID=7d1b0000-b7f7-4fda-8479-b5cb70a01030>;<SID=S-1-5-21-
> > 1608159440-4144762864-1017073214-
> >
15729>;CN=gorkon_klingons,OU=Users,OU=klingons,OU=Organizations,DC> >
xxx,
> > xx=net
> > pwdLastSet: 132112663830494760
> > userPrincipalName: gorkon_klingons at klingons.imp
> > sAMAccountName: gorkon_klingons
> > userAccountControl: 512
> > objectGUID: 7d1b0000-b7f7-4fda-8479-b5cb70a01030
> > instanceType: 4
> > # unicodePwd::: REDACTED SECRET ATTRIBUTE
> > # supplementalCredentials::: REDACTED SECRET ATTRIBUTE
> > 
> > Fri Oct  4 12:29:52 2019: pid[985]: # Passwords[0] 7d1b0000-b7f7-
> > 4fda-
> > 8479-b5cb70a01030 S-1-5-21-1608159440-4144762864-1017073214-15729
> > # attrs=['dn', 'mail', 'objectGUID',
'objectSid', 'proxyAddresses',
> > 'pwdLastSet', 'sAMAccountName',
'userAccountControl',
> > 'virtualClearTextUTF8']
> > Fri Oct  4 12:29:52 2019: pid[985]: Call
> > Popen[/usr/local/bin/syncpw.py] for
> > CN=gorkon_klingons,OU=Users,OU=klingons,OU=Organizations,DC=xxxx,DC
> > =xxx
> > ERROR(<class 'TypeError'>): uncaught exception -
memoryview: a
> > bytes-
> > like object is required, not 'str'
> >    File "/usr/local/samba/lib/python3.6/site-
> > packages/samba/netcmd/__init__.py", line 186, in _run
> >      return self.run(*args, **kwargs)
> >    File "/usr/local/samba/lib/python3.6/site-
> > packages/samba/netcmd/user.py", line 2368, in run
> >      sync_loop(wait)
> >    File "/usr/local/samba/lib/python3.6/site-
> > packages/samba/netcmd/user.py", line 2257, in sync_loop
> >      dirsync_loop()
> >    File "/usr/local/samba/lib/python3.6/site-
> > packages/samba/netcmd/user.py", line 2234, in dirsync_loop
> >      handle_object(ri, r)
> >    File "/usr/local/samba/lib/python3.6/site-
> > packages/samba/netcmd/user.py", line 2043, in handle_object
> >      run_sync_command(obj.dn, ldif)
> >    File "/usr/local/samba/lib/python3.6/site-
> > packages/samba/netcmd/user.py", line 2001, in run_sync_command
> >      reply = bytes(sync_command_p.communicate(input)[0], encoding >
> 'utf-8')
> It seems to working up to here
> >    File "/usr/lib/python3.6/subprocess.py", line 863, in
> > communicate
> >      stdout, stderr = self._communicate(input, endtime, timeout)
> >    File "/usr/lib/python3.6/subprocess.py", line 1519, in
> > _communicate
> >      input_view = memoryview(self._input)
> 
> Then it stops working, probably because your python script isn't
> python3 
> compatible.
> 
> Rowland
> 
> 
> 
>

On 04/10/2019 15:28, Heinz H?lzl via samba wrote:
> the script works...
The script may work when you run it manually, but the error samba-tool 
throws is this:

ERROR(<class 'TypeError'>): uncaught exception - memoryview: a 
bytes-like object is required, not 'str'

If you examine the error, samba-tool runs your script with this:

 ? File 
"/usr/local/samba/lib/python3.6/site-packages/samba/netcmd/user.py", 
line 2001, in run_sync_command
 ??? reply = bytes(sync_command_p.communicate(input)[0], encoding =
'utf-8')

After that 'subprocess.py' is called and the error seems to coming from 
there. Samba doesn't import all of 'subprocess', it only does this:
from
subprocess import Popen, PIPE, STDOUT, check_call, CalledProcessError

The error output goes on to do this:

 ? File "/usr/lib/python3.6/subprocess.py", line 863, in communicate
 ??? stdout, stderr = self._communicate(input, endtime, timeout)
 ? File "/usr/lib/python3.6/subprocess.py", line 1519, in _communicate
 ??? input_view = memoryview(self._input)

And it is 'memoryview' that throws the error and it looks like it is 
something in subprocess.py that is wrong, or something in your script. 
Samba, as far as I can see, does not use 'memoryview' directly.

Rowland

I think, i found the error:

in /usr/local/samba/lib/python3.6/site-packages/samba/netcmd/user.py on
line 2001:

...snip
       def run_sync_command(dn, ldif):
            log_msg("Call Popen[%s] for %s\n" % (self.sync_command,
dn))
            sync_command_p = Popen(self.sync_command,
                                   stdin=PIPE,
                                   stdout=PIPE,
                                   stderr=STDOUT)

            res = sync_command_p.poll()
            assert res is None

            input = "%s" % (ldif)
            reply = sync_command_p.communicate(input)[0]
...snip


input and ldif are Strings, but the type of input in
Popen.communicate(input) must by bytes.
The python3.6 documentation : 

coroutine communicate(input=None)
Interact with process: Send data to stdin. Read data from stdout and
stderr, until end-of-file is reached. Wait for process to terminate.
The optional input argument should be data to be sent to the child
process, or None, if no data should be sent to the child. The type of
input must be bytes.
https://docs.python.org/3.6/library/asyncio-subprocess.html


so, converting input to bytes and replay to string solves the problem. 

#reply = sync_command_p.communicate(input)[0]
reply = sync_command_p.communicate(input.encode())[0].decode()






Am Freitag, den 04.10.2019, 21:45 +0100 schrieb Rowland penny via
samba:
> On 04/10/2019 15:28, Heinz H?lzl via samba wrote:
> > the script works...
> 
> The script may work when you run it manually, but the error samba-
> tool 
> throws is this:
> 
> ERROR(<class 'TypeError'>): uncaught exception - memoryview:
a
> bytes-like object is required, not 'str'
> 
> If you examine the error, samba-tool runs your script with this:
> 
>    File 
>
"/usr/local/samba/lib/python3.6/site-packages/samba/netcmd/user.py",
> line 2001, in run_sync_command
>      reply = bytes(sync_command_p.communicate(input)[0], encoding >
'utf-8')
> 
> After that 'subprocess.py' is called and the error seems to coming
> from 
> there. Samba doesn't import all of 'subprocess', it only does
this:
> from 
> subprocess import Popen, PIPE, STDOUT, check_call, CalledProcessError
> 
> The error output goes on to do this:
> 
>    File "/usr/lib/python3.6/subprocess.py", line 863, in
communicate
>      stdout, stderr = self._communicate(input, endtime, timeout)
>    File "/usr/lib/python3.6/subprocess.py", line 1519, in
> _communicate
>      input_view = memoryview(self._input)
> 
> And it is 'memoryview' that throws the error and it looks like it
is
> something in subprocess.py that is wrong, or something in your
> script. 
> Samba, as far as I can see, does not use 'memoryview' directly.
> 
> Rowland
> 
> 
> 
>