similar to: [PATCH 1/2] python: Avoid crash if callback parameters cannot be built

On Thu, Feb 16, 2023 at 03:09:02PM +0100, Laszlo Ersek wrote: > On 2/14/23 19:51, Richard W.M. Jones wrote: > > In the case that building the parameters to the Python event callback > > fails, args was returned as NULL. We immediately tried to call > > Py_INCREF on this which crashed. Returning NULL means the Python > > function threw an exception, so print the

The event callback gets a buffer parameter which is usually something like a log message. However as it comes from C it is not necessarily well-formed (eg) UTF-8 but could contain any old byte sequence. In the test case provided by the reported, we failed to encode the buffer as str with this error: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 137: unexpected

Version 1 was here: https://listman.redhat.com/archives/libguestfs/2023-February/030732.html (Ignore version 2 which had a mistake, this is version 3) Following Eric's suggestion here: https://listman.redhat.com/archives/libguestfs/2023-February/030746.html let's decrement the reference of py_array right after adding it to the args. (This works even if args fails to be built).

A couple of fixes for Python 3 to the Python binding. Unfortunately a behaviour change is needed, although it fixes broken types used. Changes from v1: - handle also FBuffer in structs Pino Toscano (2): python: fix call of Python handlers of events python: change types for RBufferOut/FBuffer with Python 3 (RHBZ#1661871) generator/python.ml | 9 +++++++++ python/handle.c | 3 ++- 2

A couple of fixes for Python 3 to the Python binding. Unfortunately a behaviour change is needed, although it fixes broken types used. Pino Toscano (2): python: fix call of Python handlers of events python: change return type for RBufferOut with Python 3 (RHBZ#1661871) generator/python.ml | 4 ++++ python/handle.c | 3 ++- 2 files changed, 6 insertions(+), 1 deletion(-) -- 2.20.1

Version 1 was here: https://listman.redhat.com/archives/libguestfs/2023-February/030732.html Following Eric's suggestion here: https://listman.redhat.com/archives/libguestfs/2023-February/030746.html let's decrement the reference of py_array right after adding it to the args. (This works even if args fails to be built). However the other part of Eric's suggestion is wrong as

All Py_* functions should be protected by the GIL. Otherwise internal python data structures can get corrupted. Move PyGILState_Ensure to the beginning of the block and PyGILState_Release to the bottom. Signed-off-by: Sam Eiderman <sameid@google.com> --- python/handle.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/python/handle.c b/python/handle.c

On Thu, Feb 16, 2023 at 03:23:51PM +0000, Richard W.M. Jones wrote: > See-also: https://listman.redhat.com/archives/libguestfs/2023-February/030730.html > Fixes: commit 6ef5837e2d8c5d4d83eff51c0201eb2e08f719de > Thanks: Laszlo Ersek > --- > python/handle.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/python/handle.c b/python/handle.c > index

On Tue, Feb 14, 2023 at 08:02:59PM +0200, Yonatan Shtarkman wrote: > The attached code snippet?reproduces the issue when running: > python3 libugestfs_segfault_repro.py > > With: > guestfs version: 1.48.6 > guestfs-python version: 1.48.6 Can confirm with {python3-,}libguestfs-1.51.1-1.fc39.x86_64 The stack trace is roughly the same as yours. Rich. > Stacktrace: > #0

On 2/17/23 17:52, Eric Blake wrote: > On Thu, Feb 16, 2023 at 03:09:02PM +0100, Laszlo Ersek wrote: >> - Py_BuildValue with the "O" format specifier transfers the new list's >> *sole* reference (= ownership) to the just-built higher-level object "args" > > Reference transfer is done with "N", not "O". That would be an >

On Mon, Feb 20, 2023 at 10:45 AM Laszlo Ersek <lersek at redhat.com> wrote: > > On 2/17/23 17:52, Eric Blake wrote: > > On Thu, Feb 16, 2023 at 03:09:02PM +0100, Laszlo Ersek wrote: > > >> - Py_BuildValue with the "O" format specifier transfers the new list's > >> *sole* reference (= ownership) to the just-built higher-level object

On Sun, Feb 12, 2023 at 03:31:08PM +0200, Yonatan Shtarkman wrote: > Hey, > When downloading a file whose path contains multi-byte utf-8, libguestfs > sometimes crashes. > This reproduces when using python, and not when using guestfish. > > Code to reproduce: > for i in range(2000): > ? ? g.download ('/xxx?', '/tmp/1') 'i' is not used inside the

Tested by reverting bbf396fc5562b4f so that the error path is used, and re-running the reproducer supplied by Google. Rich.

The current need for #ifdef's based on the presence of PyString_FromString makes both the OCaml code of the generator, and the generated C code a mess to read. Hence, add three simple wrappers to make both the OCaml, and C code more readable, and easier to tweak in the future. --- generator/python.ml | 72 ++++++++++++----------------------------------------- python/handle.c | 65

Hey, When downloading a file whose path contains multi-byte utf-8, libguestfs sometimes crashes. This reproduces when using python, and not when using guestfish. Code to reproduce: for i in range(2000): g.download ('/xxx?', '/tmp/1') #0 raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x00007ffff7fac140 in <signal handler called> () at

The current need for #ifdef's based on the presence of PyString_FromString makes both the OCaml code of the generator, and the generated C code a mess to read. Hence, add three simple wrappers to make both the OCaml, and C code more readable, and easier to tweak in the future. --- generator/python.ml | 72 ++++++++++++----------------------------------------- python/handle.c | 52

We also need to be more careful about PyString_FromString and similar functions returning NULL on failure. Currently we don't check this every time. This commit adds more checks, but is still not complete. --- TODO | 6 ------ generator/python.ml | 19 +++++++++++++++---- 2 files changed, 15 insertions(+), 10 deletions(-) diff --git a/TODO b/TODO index 075bdc6..666400e

--- generator/generator.ml | 23 ++++------------------- 1 file changed, 4 insertions(+), 19 deletions(-) diff --git a/generator/generator.ml b/generator/generator.ml index 1f2690d..908c5f3 100755 --- a/generator/generator.ml +++ b/generator/generator.ml @@ -2925,11 +2925,7 @@ put_string_list (char * const * const argv) list = PyList_New (argc); for (i = 0; i < argc; ++i) { -#ifdef

Addressing issue #1406906. When constructing the returned objects, check the return value of Python APIs. This is especially relevant when dealing with non UTF-8 strings. A RuntimeError will be raised on failure pointing to the problematic entry and the field name. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 143

--- configure.ac | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/configure.ac b/configure.ac index 6785037..203f34f 100644 --- a/configure.ac +++ b/configure.ac @@ -329,8 +329,8 @@ AS_IF([test "x$enable_python" != "xno"], AC_MSG_CHECKING([for Python extension suffix (PEP-3149)]) if test -z "$PYTHON_EXT_SUFFIX"; then