similar to: [PATCH] python: Support buffer protocol for pread() result

It's more efficient if we pass the C buffer directly to Python code. In some cases the Python code will be able to write directly into the C buffer using functions like file.readinto and socket.recv_into. This avoids an extra copy. Thanks: Nir Soffer --- plugins/python/example.py | 8 ++++--- plugins/python/nbdkit-python-plugin.pod | 16 +++++-------- plugins/python/python.c

As suggested by Nir, here: https://www.redhat.com/archives/libguestfs/2019-November/thread.html#00220

Passing a memoryview we avoid unneeded copy of the original buffer. On the python side memoryview object can be used for slicing, writing to file, or sending to socket. This may break plugins assuming that the they get a bytearray, but good python code should not care about the type of the buffer, only about the behaviour. Testing with a plugin writing to /dev/null shows 2.7x speedup. Real

Changes since last time: - split auto-generated code in to c and h files - un-break the build system - fix ocaml binding due to libxl API change - lot''s of tidy-ups too numerous to mention Please consider and apply :) -----8<--------------------------------------------------------------- Introduce python binding for libxl. The binding is not yet complete but list_domains,

I think, i found the error: in /usr/local/samba/lib/python3.6/site-packages/samba/netcmd/user.py on line 2001: ...snip def run_sync_command(dn, ldif): log_msg("Call Popen[%s] for %s\n" % (self.sync_command, dn)) sync_command_p = Popen(self.sync_command, stdin=PIPE, stdout=PIPE,

Replace python 2 only "buffer" with "memoryview". Falling back to emulated zero would fail with: NameError: name 'buffer' is not defined I did not test the changed code but it was not tested before so it is unlikely to be worse. Detected by pylint. --- v2v/rhv-upload-plugin.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

the script works... root at probe28:~# cat test.dat dn: CN=userxxxx,OU=Users,OU=xxxxx,OU=Organizations,DC=xxxxx,DC=xxxxx proxyAddresses: SMTP:xxxxx.xxxxx at xxxxx.org objectGUID: 637f4e70-8c1e-4e89-a6fc-82d525e584f2 pwdLastSet: 0 objectSid: S-1-5-21-1608159440-4144762864-1017073214-27184 sAMAccountName: userxxxx mail: xxxxx.xxxxx at xxxxx.xxxxx userAccountControl: 514 virtualClearTextUTF8::

PyObject_CallObject is powerful, but awkward - we have to wrap all arguments into a temporary tuple before using it. Let python do more of the work by using PyObject_CallFunction anywhere that all arguments can be described by a Py_BuildValue() format string, instead of creating one-shot arguments ourselves. In fact, for our py_config(), this makes it easier to not worry about python 2 String vs.

On Fri, Dec 07, 2018 at 07:55:28PM +0200, Nir Soffer wrote: > Replace python 2 only "buffer" with "memoryview". > > Falling back to emulated zero would fail with: > > NameError: name 'buffer' is not defined > > I did not test the changed code but it was not tested before so it is > unlikely to be worse. > > Detected by pylint.

Now that we have FUA support, the C code can call can_fua as frequently as on every write. If the python script has a can_fua, we can avoid doubling the calls into python by caching the per-connection results, done by wrapping the python handle in a C struct. This commit is marked RFC because it might be nicer if the C code implemented the caching for ALL plugins (TODO already mentions that).

To avoid breaking existing plugins, Python plugins wishing to use version 2 of the API must opt in by declaring: def api_version(): return 2 (Plugins which do not do this are assumed to want API version 1). --- plugins/python/example.py | 15 +++- plugins/python/nbdkit-python-plugin.pod | 61 ++++++++++----- plugins/python/python.c | 100

On 04/10/2019 12:03, Heinz H?lzl via samba wrote: > Hi, > > i sync the passwords from samba to other backends using "samba-tool > user syncpasswords" > > On my operative system (samba 4.10 and python2) all works fine. > I upgraded my test-DC to samba 4.11 and python3 and now the samba-tool > user syncpasswords --daemon crashes. > > Fri Oct 4 12:29:47 2019:

Expose support for the FUA flag to pwrite, zero, and trim, as well as a can_fua callback, for use in python plugins. There are some slight semantic differences: the C plugin had to switch to a new API with a single uint32_t flags argument (so we don't have to keep adding new ABI when new flags are added), but in so doing, there is no way to probe whether a C plugin supports FUA flags, so the

On Wed, Apr 11, 2018 at 12:03:42AM -0500, Eric Blake wrote: > Now that we have FUA support, the C code can call can_fua as > frequently as on every write. If the python script has a > can_fua, we can avoid doubling the calls into python by > caching the per-connection results, done by wrapping the > python handle in a C struct. > > This commit is marked RFC because it might

Fairly straightforward. .list_exports uses the same idiom as .extents for returning an iterable of tuples, with additional support for a bare name rather than a name/desc tuple. .default_export and .export_description are rather easy clients of nbdkit_strdup_intern. Signed-off-by: Eric Blake <eblake at redhat.com> --- plugins/python/nbdkit-python-plugin.pod | 25 ++++ tests/Makefile.am

Hi, i sync the passwords from samba to other backends using "samba-tool user syncpasswords" On my operative system (samba 4.10 and python2) all works fine. I upgraded my test-DC to samba 4.11 and python3 and now the samba-tool user syncpasswords --daemon crashes. Fri Oct 4 12:29:47 2019: pid[983]: Attached to logfile[/usr/local/samba/var/log.syncpw] Fri Oct 4 12:29:47 2019:

To avoid breaking existing plugins, Python plugins wishing to use version 2 of the API must opt in by declaring: API_VERSION = 2 (Plugins which do not do this are assumed to want API version 1). --- plugins/python/example.py | 14 ++- plugins/python/nbdkit-python-plugin.pod | 56 +++++++----- plugins/python/python.c | 114 ++++++++++++++++++++---- tests/test.py

This change is pure refactoring and should have no effect. --- plugins/python/nbdkit-python-plugin.pod | 12 ++-- plugins/python/python.c | 90 +++++-------------------- 2 files changed, 24 insertions(+), 78 deletions(-) diff --git a/plugins/python/nbdkit-python-plugin.pod b/plugins/python/nbdkit-python-plugin.pod index 51e0f57..0fd4dcb 100644 ---

Fairly straightforward, as the code in plugins.c already takes care of necessary .pwrite fallbacks. Simplify similar can_FOO callbacks that were copy-and-pasted while implementing py_can_zero. Signed-off-by: Eric Blake <eblake@redhat.com> --- plugins/python/nbdkit-python-plugin.pod | 7 +++++ plugins/python/python.c | 51 +++++++++++++++++++-------------- tests/test.py

On Fri, Dec 7, 2018 at 8:50 PM Richard W.M. Jones <rjones@redhat.com> wrote: > On Fri, Dec 07, 2018 at 07:55:28PM +0200, Nir Soffer wrote: > > Replace python 2 only "buffer" with "memoryview". > > > > Falling back to emulated zero would fail with: > > > > NameError: name 'buffer' is not defined > > > > I did not