similar to: [nbdkit PATCH] python: Let zero's may_trim parameter be optional

In preparation for adding other optional flag arguments to the python bindings, start by making the existing 'may_trim' flag to 'zero' be optional. That is, the plugin need not define the parameter if it does not make any semantic difference (ie. if the plugin ignores the hint and never trims); while if the parameter exists, we now pass it as a keyword argument rather than as a

First out of our four language bindings to add FUA support (for reference, I added 'zero' support for python, perl, and ruby back in 1.1.13, then Rich had to add it for ocaml in 1.1.20). I tested this heavily under python 2, but for now only compile tested under python 3; I plan to do further testing there and make any tweaks if necessary. I wrote patch 5 early on, but then realized I

PyObject_CallObject is powerful, but awkward - we have to wrap all arguments into a temporary tuple before using it. Let python do more of the work by using PyObject_CallFunction anywhere that all arguments can be described by a Py_BuildValue() format string, instead of creating one-shot arguments ourselves. In fact, for our py_config(), this makes it easier to not worry about python 2 String vs.

Expose support for the FUA flag to pwrite, zero, and trim, as well as a can_fua callback, for use in python plugins. There are some slight semantic differences: the C plugin had to switch to a new API with a single uint32_t flags argument (so we don't have to keep adding new ABI when new flags are added), but in so doing, there is no way to probe whether a C plugin supports FUA flags, so the

Add a python language binding for the .zero callback, used for implementing NBD_CMD_WRITE_ZEROES. The caller doesn't have to return anything, but should use nbdkit.set_error(errno.EOPNOTSUPP) to get an automatic fallback to pwrite. Enhance the example to show the use of the fallback mechanism, and to serve as a test of nbdkit.set_error(). Signed-off-by: Eric Blake <eblake@redhat.com>

Add a python language binding for the .zero callback, used for implementing NBD_CMD_WRITE_ZEROES. The caller doesn't have to return anything, but should use nbdkit.set_error(errno.EOPNOTSUPP) to get an automatic fallback to pwrite. Enhance the example to show the use of the fallback mechanism, and to serve as a test of nbdkit.set_error(). Signed-off-by: Eric Blake <eblake@redhat.com>

It is unclear why we were constructing this by hand, but using the following tip we can use PyObject_CallFunction: https://stackoverflow.com/a/21221335 --- plugins/python/python.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/plugins/python/python.c b/plugins/python/python.c index 148097f..d65ac45 100644 --- a/plugins/python/python.c +++

It is unclear why we were constructing this by hand, but using the following tip we can use PyObject_CallFunction: https://stackoverflow.com/a/21221335 --- plugins/python/python.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/plugins/python/python.c b/plugins/python/python.c index 148097f..d65ac45 100644 --- a/plugins/python/python.c +++

As mentioned in my reviews, I wonder if we should make our python callbacks look a bit more Pythonic by having kwargs added for each new flag that we want to expose. The idea was first floated here: https://www.redhat.com/archives/libguestfs/2018-April/msg00108.html Note that with my proposal, there is no need for a python script to expose a global API_VERSION variable; new flags are added

I noticed these while working on adding fua support into python, these are independent enough to push now (and I'll have to rebase my 'optional may_trim' patch on top of this). Eric Blake (2): python: Use Py_XDEFREF() python: Simplify calling into plugin plugins/python/python.c | 106 ++++++++---------------------------------------- 1 file changed, 18 insertions(+), 88

To avoid breaking existing plugins, Python plugins wishing to use version 2 of the API must opt in by declaring: def api_version(): return 2 (Plugins which do not do this are assumed to want API version 1). --- plugins/python/example.py | 15 +++- plugins/python/nbdkit-python-plugin.pod | 61 ++++++++++----- plugins/python/python.c | 100

To avoid breaking existing plugins, Python plugins wishing to use version 2 of the API must opt in by declaring: API_VERSION = 2 (Plugins which do not do this are assumed to want API version 1). --- plugins/python/example.py | 14 ++- plugins/python/nbdkit-python-plugin.pod | 56 +++++++----- plugins/python/python.c | 114 ++++++++++++++++++++---- tests/test.py

The current need for #ifdef's based on the presence of PyString_FromString makes both the OCaml code of the generator, and the generated C code a mess to read. Hence, add three simple wrappers to make both the OCaml, and C code more readable, and easier to tweak in the future. --- generator/python.ml | 72 ++++++++++++----------------------------------------- python/handle.c | 52

The current need for #ifdef's based on the presence of PyString_FromString makes both the OCaml code of the generator, and the generated C code a mess to read. Hence, add three simple wrappers to make both the OCaml, and C code more readable, and easier to tweak in the future. --- generator/python.ml | 72 ++++++++++++----------------------------------------- python/handle.c | 65

Adjust internal functions in preparation for FUA support; although at this point, the default plugins.c can_fua implementation correctly reports python as needing emulation, and we can assert that we aren't seeing a FUA flag. Signed-off-by: Eric Blake <eblake@redhat.com> --- plugins/python/python.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git

The tracebacks are compressed into a single line because we're using PyObject_Str, but they are just about usable if not very readable. For example you would see an error like this: nbdkit: error: ./python-exception.py: config_complete: error: ['Traceback (most recent call last):\n', ' File "./python-exception.py", line 54, in config_complete\n raise_error1()\n',

On Wed, Aug 8, 2018 at 4:07 PM Richard W.M. Jones <rjones@redhat.com> wrote: > The tracebacks are compressed into a single line because we're using > PyObject_Str, but they are just about usable if not very readable. > For example you would see an error like this: > > nbdkit: error: ./python-exception.py: config_complete: error: ['Traceback > (most recent call

--- generator/generator.ml | 23 ++++------------------- 1 file changed, 4 insertions(+), 19 deletions(-) diff --git a/generator/generator.ml b/generator/generator.ml index 1f2690d..908c5f3 100755 --- a/generator/generator.ml +++ b/generator/generator.ml @@ -2925,11 +2925,7 @@ put_string_list (char * const * const argv) list = PyList_New (argc); for (i = 0; i < argc; ++i) { -#ifdef

Starting with nbdkit 1.16, Python >= 3.3 will be required. Python 2 reaches end of life on 2020-01-01: https://python3statement.org/ https://pythonclock.org/ Debian oldoldstable and RHEL 7 have Python 3.4 and 3.6 respectively, so it seems pointless to try to support Python < 3.3 which lacked support for PyUnicode_AsUTF8. --- README | 13 +++++++------ configure.ac

Now that we have FUA support, the C code can call can_fua as frequently as on every write. If the python script has a can_fua, we can avoid doubling the calls into python by caching the per-connection results, done by wrapping the python handle in a C struct. This commit is marked RFC because it might be nicer if the C code implemented the caching for ALL plugins (TODO already mentions that).