similar to: [PATCH v2 0/2] python: fixes for Python 3

A couple of fixes for Python 3 to the Python binding. Unfortunately a behaviour change is needed, although it fixes broken types used. Pino Toscano (2): python: fix call of Python handlers of events python: change return type for RBufferOut with Python 3 (RHBZ#1661871) generator/python.ml | 4 ++++ python/handle.c | 3 ++- 2 files changed, 6 insertions(+), 1 deletion(-) -- 2.20.1

The current need for #ifdef's based on the presence of PyString_FromString makes both the OCaml code of the generator, and the generated C code a mess to read. Hence, add three simple wrappers to make both the OCaml, and C code more readable, and easier to tweak in the future. --- generator/python.ml | 72 ++++++++++++----------------------------------------- python/handle.c | 65

This series addresses the issue where non UTF8 file names in a guest image lead to libguestfs segfault with Python 3 APIs. The core issue is the APIs are not checking the return value when constructing a new PyObject. Therefore NULL pointers are added to Python collections (lists and dictionaries) crashing the application. Few notes regarding the comments on the previous patch. - Added a

Patch #7 depends on: https://www.redhat.com/archives/libguestfs/2020-January/msg00035.html No, Python < 3 support is not dropped yet, however it will be easier after this series. Pino Toscano (7): build: enforce a minimum Python version python: drop code for Python < 2.5 python: assume support for Capsules python: remove compile time check for PyString_AsString python: replace

In the case that building the parameters to the Python event callback fails, args was returned as NULL. We immediately tried to call Py_INCREF on this which crashed. Returning NULL means the Python function threw an exception, so print the exception and return (there is no way to return an error here - the event is lost). Reported-by: Yonatan Shtarkman See:

The Python 3 bindings currently are unable to deal with non UTF8 characters. This series continues what proposed in RHBZ#1406906. A new function 'set_decode_error_handler' allows the User to specify how to deal with decoding errors. The default behaviour will be raising a UnicodeDecodeError. If the handler is changed to 'surrogateescape', non UTF8 characters will be escaped in a

--- configure.ac | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/configure.ac b/configure.ac index 6785037..203f34f 100644 --- a/configure.ac +++ b/configure.ac @@ -329,8 +329,8 @@ AS_IF([test "x$enable_python" != "xno"], AC_MSG_CHECKING([for Python extension suffix (PEP-3149)]) if test -z "$PYTHON_EXT_SUFFIX"; then

Version 1 was here: https://listman.redhat.com/archives/libguestfs/2023-February/030732.html (Ignore version 2 which had a mistake, this is version 3) Following Eric's suggestion here: https://listman.redhat.com/archives/libguestfs/2023-February/030746.html let's decrement the reference of py_array right after adding it to the args. (This works even if args fails to be built).

So far RBufferOut return values, and FBuffer struct fields are 'str' on all the versions of Python. Python 3 distinguishes between 'str' (unicode strings), and 'bytes', with 'str' no more able to hold arbitrary data. For this reason, switch the return value of RBufferOut functions, and FBuffer struct fields to bytes on Python 3: while this is a potentially

The current need for #ifdef's based on the presence of PyString_FromString makes both the OCaml code of the generator, and the generated C code a mess to read. Hence, add three simple wrappers to make both the OCaml, and C code more readable, and easier to tweak in the future. --- generator/python.ml | 72 ++++++++++++----------------------------------------- python/handle.c | 52

Addressing issue #1406906. When constructing the returned objects, check the return value of Python APIs. This is especially relevant when dealing with non UTF-8 strings. A RuntimeError will be raised on failure pointing to the problematic entry and the field name. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 143

When constructing the returned objects, check the return value of Python APIs. A RuntimeError will be raised on failure pointing to the problematic entry and the field name. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 143 +++++++++++++++++++++++++++++++++++----------------- 1 file changed, 97 insertions(+), 46 deletions(-) diff --git

The python3 bindings create PyUnicode objects from application strings on the guest (i.e. installed rpm, deb packages). It is documented that rpm package fields such as description should be utf8 encoded - however in some cases they are not a valid unicode string, on SLES11 SP4 the encoding of the description of the following packages is latin1 and they fail to be converted to unicode using

The python3 bindings create unicode objects from application strings on the guest (i.e. installed rpm, deb packages). It is documented that rpm package fields such as description should be utf8 encoded - however in some cases they are not a valid unicode string, on SLES11 SP4 the following packages fail to be converted to unicode using guestfs_int_py_fromstring() (which invokes

On Windows, there exist at least two APIs for dealing with the Registry: The Win32 API (RegCreateKeyA, RegCreateKeyW, etc.) works with null-terminated ASCII or UTF-16 strings. The native API (ZwCreateKey, etc.), on the other hand works with UTF-16 strings that are stored as buffers+length and may contain null characters. Malware authors have been relying on the Win32 API's inability to

All Py_* functions should be protected by the GIL. Otherwise internal python data structures can get corrupted. Move PyGILState_Ensure to the beginning of the block and PyGILState_Release to the bottom. Signed-off-by: Sam Eiderman <sameid@google.com> --- python/handle.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/python/handle.c b/python/handle.c

In Python 2 code branches use PyString_FromStringAndSize directly instead of the guestfs_int_py_fromstringsize wrapper (which calls PyString_FromStringAndSize anyway on Python < 3). --- generator/python.ml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/generator/python.ml b/generator/python.ml index 2a1a48fc0..0e1ed20d8 100644 --- a/generator/python.ml +++

--- generator/generator.ml | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/generator/generator.ml b/generator/generator.ml index 31478cd..de911f1 100755 --- a/generator/generator.ml +++ b/generator/generator.ml @@ -1771,7 +1771,7 @@ static void raise_closed (const char *) Noreturn; pr " rv = copy_type_value (r, len, t);\n"; pr "

I've summarised how the GObject bindings work below, which should hopefully help reviewing the generator/generated code. There are a couple of points in here I'm still not 100% happy with. Specifically the handling of FBuffer and the Cancellable flag. Both are explained below. I'm interested in suggestions. Return values: ************** All functions which can return an error

On Wed, May 13, 2020 at 10:06 PM Richard W.M. Jones <rjones@redhat.com> wrote: > > On Sun, Apr 26, 2020 at 09:14:03PM +0300, Sam Eiderman wrote: > > The python3 bindings create PyUnicode objects from application strings > > on the guest (i.e. installed rpm, deb packages). > > It is documented that rpm package fields such as description should be > > utf8 encoded