similar to: [PATCH] python: Fix bug in code generator

When a struct containing FChar fields is defined in generator/structs.ml, the Python code generator will generate code referencing a dirent struct instead of the correct one thus breaking at compile time. This bug was left unnoticed because the dirent struct is the only one using FChar fields. Yet it prevents any other struct from using such fields. Signed-off-by: Matteo Cafasso <noxdafox at

Addressing issue #1406906. When constructing the returned objects, check the return value of Python APIs. This is especially relevant when dealing with non UTF-8 strings. A RuntimeError will be raised on failure pointing to the problematic entry and the field name. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 143

The current need for #ifdef's based on the presence of PyString_FromString makes both the OCaml code of the generator, and the generated C code a mess to read. Hence, add three simple wrappers to make both the OCaml, and C code more readable, and easier to tweak in the future. --- generator/python.ml | 72 ++++++++++++----------------------------------------- python/handle.c | 52

When constructing the returned objects, check the return value of Python APIs. A RuntimeError will be raised on failure pointing to the problematic entry and the field name. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 143 +++++++++++++++++++++++++++++++++++----------------- 1 file changed, 97 insertions(+), 46 deletions(-) diff --git

The current need for #ifdef's based on the presence of PyString_FromString makes both the OCaml code of the generator, and the generated C code a mess to read. Hence, add three simple wrappers to make both the OCaml, and C code more readable, and easier to tweak in the future. --- generator/python.ml | 72 ++++++++++++----------------------------------------- python/handle.c | 65

The interesting line is this one ... checking for PyString_AsString in -lc... no and if you compare that to the code, eg in python/guestfs-py.c: #ifdef HAVE_PYSTRING_ASSTRING PyDict_SetItemString (dict, "ftyp", PyString_FromStringAndSize (&dirent->ftyp, 1)); #else PyDict_SetItemString (dict, "ftyp",

--- generator/generator.ml | 23 ++++------------------- 1 file changed, 4 insertions(+), 19 deletions(-) diff --git a/generator/generator.ml b/generator/generator.ml index 1f2690d..908c5f3 100755 --- a/generator/generator.ml +++ b/generator/generator.ml @@ -2925,11 +2925,7 @@ put_string_list (char * const * const argv) list = PyList_New (argc); for (i = 0; i < argc; ++i) { -#ifdef

The set_decode_error_handler function allows the User to set the decoding error scheme to be used when non UTF8 characters are encountered in Python 3. The function has no effect in Python 2. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 16 ++++++++++++++++ python/handle.c | 18 ++++++++++++++++-- python/t/test830RHBZ1406906.py |

Verify the returned values of Python Object constructor functions are not NULL before adding them to a collection. This is particularly relevant when constructing Unicode strings in Python 3 as they will return NULL if non UTF-8 characters are present. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 102 ++++++++++++++++++++++++++---------------

Verify the returned values of Python Object constructor functions are not NULL before adding them to a collection. This is particularly relevant when constructing Unicode strings in Python 3 as they will return NULL if non UTF-8 characters are present. Signed-off-by: Matteo Cafasso <noxdafox@gmail.com> --- generator/python.ml | 102 ++++++++++++++++++++++++++---------------

This series addresses the issue where non UTF8 file names in a guest image lead to libguestfs segfault with Python 3 APIs. The core issue is the APIs are not checking the return value when constructing a new PyObject. Therefore NULL pointers are added to Python collections (lists and dictionaries) crashing the application. Few notes regarding the comments on the previous patch. - Added a

We also need to be more careful about PyString_FromString and similar functions returning NULL on failure. Currently we don't check this every time. This commit adds more checks, but is still not complete. --- TODO | 6 ------ generator/python.ml | 19 +++++++++++++++---- 2 files changed, 15 insertions(+), 10 deletions(-) diff --git a/TODO b/TODO index 075bdc6..666400e

Per discussion on IRC, here are two changes to convert all TABs-as-indentation to spaces. The first one is the fully-automated conversion. However, note that the command mentioned uses a file (the .x-sc* one) that is added only in the following patch. The second patch adds rules to help keep things that way: Document and enforce the new spaces-only indentation policy. * cfg.mk

The python3 bindings create PyUnicode objects from application strings on the guest (i.e. installed rpm, deb packages). It is documented that rpm package fields such as description should be utf8 encoded - however in some cases they are not a valid unicode string, on SLES11 SP4 the encoding of the description of the following packages is latin1 and they fail to be converted to unicode using

Hey Pino, Can you search for the previous patches I submitted? I had some discussions regarding this with Daniel and Nir. Thanks! On Tue, Jun 30, 2020 at 11:43 AM Pino Toscano <ptoscano@redhat.com> wrote: > On Sunday, 26 April 2020 20:14:03 CEST Sam Eiderman wrote: > > The python3 bindings create PyUnicode objects from application strings > > on the guest (i.e. installed

In Python 2 code branches use PyString_FromStringAndSize directly instead of the guestfs_int_py_fromstringsize wrapper (which calls PyString_FromStringAndSize anyway on Python < 3). --- generator/python.ml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/generator/python.ml b/generator/python.ml index 2a1a48fc0..0e1ed20d8 100644 --- a/generator/python.ml +++

The python3 bindings create unicode objects from application strings on the guest (i.e. installed rpm, deb packages). It is documented that rpm package fields such as description should be utf8 encoded - however in some cases they are not a valid unicode string, on SLES11 SP4 the following packages fail to be converted to unicode using guestfs_int_py_fromstring() (which invokes

On Mon, Apr 20, 2020 at 03:37:16PM +0300, Sam Eiderman wrote: > The python3 bindings create unicode objects from application strings > on the guest (i.e. installed rpm, deb packages). > It is documented that rpm package fields such as description should be > utf8 encoded - however in some cases they are not a valid unicode > string, on SLES11 SP4 the following packages fail to be

On Mon, Apr 20, 2020 at 3:38 PM Sam Eiderman <sameid@google.com> wrote: > > The python3 bindings create unicode objects from application strings > on the guest (i.e. installed rpm, deb packages). > It is documented that rpm package fields such as description should be > utf8 encoded - however in some cases they are not a valid unicode > string, So what are they? latin1

On Sun, Apr 26, 2020 at 09:14:03PM +0300, Sam Eiderman wrote: > The python3 bindings create PyUnicode objects from application strings > on the guest (i.e. installed rpm, deb packages). > It is documented that rpm package fields such as description should be > utf8 encoded - however in some cases they are not a valid unicode > string, on SLES11 SP4 the encoding of the description of