similar to: [PATCH] python: Fix bug in code generator

EDIT: forgot to add the list to the subject, sorry for the spam. When a struct containing FChar fields is defined in generator/structs.ml, the Python code generator will generate code referencing a dirent struct instead of the correct one thus breaking at compile time. This bug was left unnoticed because the dirent struct is the only one using FChar fields. Yet it prevents any other struct from

The current need for #ifdef's based on the presence of PyString_FromString makes both the OCaml code of the generator, and the generated C code a mess to read. Hence, add three simple wrappers to make both the OCaml, and C code more readable, and easier to tweak in the future. --- generator/python.ml | 72 ++++++++++++----------------------------------------- python/handle.c | 65

Addressing issue #1406906. When constructing the returned objects, check the return value of Python APIs. This is especially relevant when dealing with non UTF-8 strings. A RuntimeError will be raised on failure pointing to the problematic entry and the field name. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 143

When constructing the returned objects, check the return value of Python APIs. A RuntimeError will be raised on failure pointing to the problematic entry and the field name. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 143 +++++++++++++++++++++++++++++++++++----------------- 1 file changed, 97 insertions(+), 46 deletions(-) diff --git

The current need for #ifdef's based on the presence of PyString_FromString makes both the OCaml code of the generator, and the generated C code a mess to read. Hence, add three simple wrappers to make both the OCaml, and C code more readable, and easier to tweak in the future. --- generator/python.ml | 72 ++++++++++++----------------------------------------- python/handle.c | 52

This series addresses the issue where non UTF8 file names in a guest image lead to libguestfs segfault with Python 3 APIs. The core issue is the APIs are not checking the return value when constructing a new PyObject. Therefore NULL pointers are added to Python collections (lists and dictionaries) crashing the application. Few notes regarding the comments on the previous patch. - Added a

The interesting line is this one ... checking for PyString_AsString in -lc... no and if you compare that to the code, eg in python/guestfs-py.c: #ifdef HAVE_PYSTRING_ASSTRING PyDict_SetItemString (dict, "ftyp", PyString_FromStringAndSize (&dirent->ftyp, 1)); #else PyDict_SetItemString (dict, "ftyp",

The python3 bindings create PyUnicode objects from application strings on the guest (i.e. installed rpm, deb packages). It is documented that rpm package fields such as description should be utf8 encoded - however in some cases they are not a valid unicode string, on SLES11 SP4 the encoding of the description of the following packages is latin1 and they fail to be converted to unicode using

The python3 bindings create unicode objects from application strings on the guest (i.e. installed rpm, deb packages). It is documented that rpm package fields such as description should be utf8 encoded - however in some cases they are not a valid unicode string, on SLES11 SP4 the following packages fail to be converted to unicode using guestfs_int_py_fromstring() (which invokes

Hey Pino, Can you search for the previous patches I submitted? I had some discussions regarding this with Daniel and Nir. Thanks! On Tue, Jun 30, 2020 at 11:43 AM Pino Toscano <ptoscano@redhat.com> wrote: > On Sunday, 26 April 2020 20:14:03 CEST Sam Eiderman wrote: > > The python3 bindings create PyUnicode objects from application strings > > on the guest (i.e. installed

--- generator/generator.ml | 23 ++++------------------- 1 file changed, 4 insertions(+), 19 deletions(-) diff --git a/generator/generator.ml b/generator/generator.ml index 1f2690d..908c5f3 100755 --- a/generator/generator.ml +++ b/generator/generator.ml @@ -2925,11 +2925,7 @@ put_string_list (char * const * const argv) list = PyList_New (argc); for (i = 0; i < argc; ++i) { -#ifdef

On Wed, May 13, 2020 at 10:06 PM Richard W.M. Jones <rjones@redhat.com> wrote: > > On Sun, Apr 26, 2020 at 09:14:03PM +0300, Sam Eiderman wrote: > > The python3 bindings create PyUnicode objects from application strings > > on the guest (i.e. installed rpm, deb packages). > > It is documented that rpm package fields such as description should be > > utf8 encoded

Patch #7 depends on: https://www.redhat.com/archives/libguestfs/2020-January/msg00035.html No, Python < 3 support is not dropped yet, however it will be easier after this series. Pino Toscano (7): build: enforce a minimum Python version python: drop code for Python < 2.5 python: assume support for Capsules python: remove compile time check for PyString_AsString python: replace

--- configure.ac | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/configure.ac b/configure.ac index 6785037..203f34f 100644 --- a/configure.ac +++ b/configure.ac @@ -329,8 +329,8 @@ AS_IF([test "x$enable_python" != "xno"], AC_MSG_CHECKING([for Python extension suffix (PEP-3149)]) if test -z "$PYTHON_EXT_SUFFIX"; then

The set_decode_error_handler function allows the User to set the decoding error scheme to be used when non UTF8 characters are encountered in Python 3. The function has no effect in Python 2. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 16 ++++++++++++++++ python/handle.c | 18 ++++++++++++++++-- python/t/test830RHBZ1406906.py |

The Python 3 bindings currently are unable to deal with non UTF8 characters. This series continues what proposed in RHBZ#1406906. A new function 'set_decode_error_handler' allows the User to specify how to deal with decoding errors. The default behaviour will be raising a UnicodeDecodeError. If the handler is changed to 'surrogateescape', non UTF8 characters will be escaped in a

Verify the returned values of Python Object constructor functions are not NULL before adding them to a collection. This is particularly relevant when constructing Unicode strings in Python 3 as they will return NULL if non UTF-8 characters are present. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 102 ++++++++++++++++++++++++++---------------

Verify the returned values of Python Object constructor functions are not NULL before adding them to a collection. This is particularly relevant when constructing Unicode strings in Python 3 as they will return NULL if non UTF-8 characters are present. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 102 ++++++++++++++++++++++++++---------------

On Mon, Apr 20, 2020 at 03:37:16PM +0300, Sam Eiderman wrote: > The python3 bindings create unicode objects from application strings > on the guest (i.e. installed rpm, deb packages). > It is documented that rpm package fields such as description should be > utf8 encoded - however in some cases they are not a valid unicode > string, on SLES11 SP4 the following packages fail to be

On Mon, Apr 20, 2020 at 3:38 PM Sam Eiderman <sameid@google.com> wrote: > > The python3 bindings create unicode objects from application strings > on the guest (i.e. installed rpm, deb packages). > It is documented that rpm package fields such as description should be > utf8 encoded - however in some cases they are not a valid unicode > string, So what are they? latin1