similar to: [PATCH] python: Call PyErr_Clear() on non-error paths out of the Python bindings.

The current need for #ifdef's based on the presence of PyString_FromString makes both the OCaml code of the generator, and the generated C code a mess to read. Hence, add three simple wrappers to make both the OCaml, and C code more readable, and easier to tweak in the future. --- generator/python.ml | 72 ++++++++++++----------------------------------------- python/handle.c | 52

The current need for #ifdef's based on the presence of PyString_FromString makes both the OCaml code of the generator, and the generated C code a mess to read. Hence, add three simple wrappers to make both the OCaml, and C code more readable, and easier to tweak in the future. --- generator/python.ml | 72 ++++++++++++----------------------------------------- python/handle.c | 65

Addressing issue #1406906. When constructing the returned objects, check the return value of Python APIs. This is especially relevant when dealing with non UTF-8 strings. A RuntimeError will be raised on failure pointing to the problematic entry and the field name. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 143

--- generator/generator.ml | 23 ++++------------------- 1 file changed, 4 insertions(+), 19 deletions(-) diff --git a/generator/generator.ml b/generator/generator.ml index 1f2690d..908c5f3 100755 --- a/generator/generator.ml +++ b/generator/generator.ml @@ -2925,11 +2925,7 @@ put_string_list (char * const * const argv) list = PyList_New (argc); for (i = 0; i < argc; ++i) { -#ifdef

When constructing the returned objects, check the return value of Python APIs. A RuntimeError will be raised on failure pointing to the problematic entry and the field name. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 143 +++++++++++++++++++++++++++++++++++----------------- 1 file changed, 97 insertions(+), 46 deletions(-) diff --git

Starting with nbdkit 1.16, Python >= 3.3 will be required. Python 2 reaches end of life on 2020-01-01: https://python3statement.org/ https://pythonclock.org/ Debian oldoldstable and RHEL 7 have Python 3.4 and 3.6 respectively, so it seems pointless to try to support Python < 3.3 which lacked support for PyUnicode_AsUTF8. --- README | 13 +++++++------ configure.ac

When using optional arguments of type OStringList, the code free'ing the member in the optargs_s struct corresponding to that optional argument would just check for a non-PyNone PyObject for that argument. If before that optional argument there are other arguments which can cause an earlier error return from that binding function, the free'ing code will then act on garbage values. Enhance

Verify the returned values of Python Object constructor functions are not NULL before adding them to a collection. This is particularly relevant when constructing Unicode strings in Python 3 as they will return NULL if non UTF-8 characters are present. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 102 ++++++++++++++++++++++++++---------------

This series addresses the issue where non UTF8 file names in a guest image lead to libguestfs segfault with Python 3 APIs. The core issue is the APIs are not checking the return value when constructing a new PyObject. Therefore NULL pointers are added to Python collections (lists and dictionaries) crashing the application. Few notes regarding the comments on the previous patch. - Added a

Verify the returned values of Python Object constructor functions are not NULL before adding them to a collection. This is particularly relevant when constructing Unicode strings in Python 3 as they will return NULL if non UTF-8 characters are present. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 102 ++++++++++++++++++++++++++---------------

When a struct containing FChar fields is defined in generator/structs.ml, the Python code generator will generate code referencing a dirent struct instead of the correct one thus breaking at compile time. This bug was left unnoticed because the dirent struct is the only one using FChar fields. Yet it prevents any other struct from using such fields. Signed-off-by: Matteo Cafasso <noxdafox at

The set_decode_error_handler function allows the User to set the decoding error scheme to be used when non UTF8 characters are encountered in Python 3. The function has no effect in Python 2. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 16 ++++++++++++++++ python/handle.c | 18 ++++++++++++++++-- python/t/test830RHBZ1406906.py |

EDIT: forgot to add the list to the subject, sorry for the spam. When a struct containing FChar fields is defined in generator/structs.ml, the Python code generator will generate code referencing a dirent struct instead of the correct one thus breaking at compile time. This bug was left unnoticed because the dirent struct is the only one using FChar fields. Yet it prevents any other struct from

Patch to fix: https://bugzilla.redhat.com/show_bug.cgi?id=1262983 Note this won't help until the first two patches get backported to the stable branches, since <guestfs.h> won't define the necessary GUESTFS_HAVE_* macros. Rich.

--- configure.ac | 8 +++++++ plugins/python/python.c | 51 +++++++++++++++++++++++++++++++++++++++--- tests/Makefile.am | 1 + tests/python-exception.py | 45 +++++++++++++++++++++++++++++++++++++ tests/test-python-exception.sh | 42 ++++++++++++++++++++++++++++++++++ 5 files changed, 144 insertions(+), 3 deletions(-) diff --git a/configure.ac

Patch #7 depends on: https://www.redhat.com/archives/libguestfs/2020-January/msg00035.html No, Python < 3 support is not dropped yet, however it will be easier after this series. Pino Toscano (7): build: enforce a minimum Python version python: drop code for Python < 2.5 python: assume support for Capsules python: remove compile time check for PyString_AsString python: replace

--- configure.ac | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/configure.ac b/configure.ac index 6785037..203f34f 100644 --- a/configure.ac +++ b/configure.ac @@ -329,8 +329,8 @@ AS_IF([test "x$enable_python" != "xno"], AC_MSG_CHECKING([for Python extension suffix (PEP-3149)]) if test -z "$PYTHON_EXT_SUFFIX"; then

Per discussion on IRC, here are two changes to convert all TABs-as-indentation to spaces. The first one is the fully-automated conversion. However, note that the command mentioned uses a file (the .x-sc* one) that is added only in the following patch. The second patch adds rules to help keep things that way: Document and enforce the new spaces-only indentation policy. * cfg.mk

PyObject_CallObject is powerful, but awkward - we have to wrap all arguments into a temporary tuple before using it. Let python do more of the work by using PyObject_CallFunction anywhere that all arguments can be described by a Py_BuildValue() format string, instead of creating one-shot arguments ourselves. In fact, for our py_config(), this makes it easier to not worry about python 2 String vs.

This patch adds byte run reporters for node and value metadata in the hivexml program. Each byte run represents the offset and length of a data structure within the hive, one per node, and one or two per value depending on the length of the value data. In order to add this metadata reporting, the following changes were put in place: * Split value_key function into value_key and value_key_len. *