search for: pystring_fromstringandsize

...guestfs_int_py_fromstring (%s->%s));\n" + typ name | name, FBuffer -> pr " PyDict_SetItemString (dict, \"%s\",\n" name; - pr "#ifdef HAVE_PYSTRING_ASSTRING\n"; - pr " PyString_FromStringAndSize (%s->%s, %s->%s_len));\n" - typ name typ name; - pr "#else\n"; - pr " PyBytes_FromStringAndSize (%s->%s, %s->%s_len));\n" - typ name typ name; - pr "#endif\n" +...

...r " PyDict_SetItemString (dict, \"%s\", value);\n" name; | name, FBuffer -> - pr " PyDict_SetItemString (dict, \"%s\",\n" name; pr "#ifdef HAVE_PYSTRING_ASSTRING\n"; - pr " PyString_FromStringAndSize (%s->%s, %s->%s_len));\n" + pr " value = PyString_FromStringAndSize (%s->%s, %s->%s_len);\n" typ name typ name; pr "#else\n"; - pr " PyBytes_FromStringAndSize (%s->%s, %s->%s_le...

...guestfs_int_py_fromstring (%s->%s));\n" + typ name | name, FBuffer -> pr " PyDict_SetItemString (dict, \"%s\",\n" name; - pr "#ifdef HAVE_PYSTRING_ASSTRING\n"; - pr " PyString_FromStringAndSize (%s->%s, %s->%s_len));\n" - typ name typ name; - pr "#else\n"; - pr " PyBytes_FromStringAndSize (%s->%s, %s->%s_len));\n" - typ name typ name; - pr "#endif\n" +...

In Python 2 code branches use PyString_FromStringAndSize directly instead of the guestfs_int_py_fromstringsize wrapper (which calls PyString_FromStringAndSize anyway on Python < 3). --- generator/python.ml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/generator/python.ml b/generator/python.ml index 2a1a48fc0..0e1ed20d8 10064...

...generator/python.ml @@ -225,10 +225,12 @@ put_table (char * const * const argv) | name, FChar -> pr "#ifdef HAVE_PYSTRING_ASSTRING\n"; pr " PyDict_SetItemString (dict, \"%s\",\n" name; - pr " PyString_FromStringAndSize (&dirent->%s, 1));\n" name; + pr " PyString_FromStringAndSize (&%s->%s, 1));\n" + typ name; pr "#else\n"; pr " PyDict_SetItemString (dict, \"%s\",\n" name; -...

...r " PyDict_SetItemString (dict, \"%s\", value);\n" name; | name, FBuffer -> - pr " PyDict_SetItemString (dict, \"%s\",\n" name; pr "#ifdef HAVE_PYSTRING_ASSTRING\n"; - pr " PyString_FromStringAndSize (%s->%s, %s->%s_len));\n" + pr " value = PyString_FromStringAndSize (%s->%s, %s->%s_len);\n" typ name typ name; pr "#else\n"; - pr " PyBytes_FromStringAndSize (%s->%s, %s->%s_le...

This series addresses the issue where non UTF8 file names in a guest image lead to libguestfs segfault with Python 3 APIs. The core issue is the APIs are not checking the return value when constructing a new PyObject. Therefore NULL pointers are added to Python collections (lists and dictionaries) crashing the application. Few notes regarding the comments on the previous patch. - Added a

...Y_MAJOR_VERSION < 3 return PyString_FromString (str); #else - return PyUnicode_FromString (str); + return guestfs_int_py_fromstringsize (str, strlen (str)); #endif } @@ -397,7 +397,12 @@ guestfs_int_py_fromstringsize (const char *str, size_t size) #if PY_MAJOR_VERSION < 3 return PyString_FromStringAndSize (str, size); #else - return PyUnicode_FromStringAndSize (str, size); + PyObject *s = PyUnicode_FromString (str); + if (s == NULL) { + PyErr_Clear (); + s = PyUnicode_Decode (str, strlen(str), "latin1", "strict"); + } + return s; #endif } -- 2.26.2.303.gf8c07b1a7...

Patch #7 depends on: https://www.redhat.com/archives/libguestfs/2020-January/msg00035.html No, Python < 3 support is not dropped yet, however it will be easier after this series. Pino Toscano (7): build: enforce a minimum Python version python: drop code for Python < 2.5 python: assume support for Capsules python: remove compile time check for PyString_AsString python: replace

...return PyString_FromString (str); #else - return PyUnicode_FromString (str); + return PyUnicode_Decode(str, strlen(str), "utf-8", "replace"); #endif } @@ -397,7 +397,7 @@ guestfs_int_py_fromstringsize (const char *str, size_t size) #if PY_MAJOR_VERSION < 3 return PyString_FromStringAndSize (str, size); #else - return PyUnicode_FromStringAndSize (str, size); + return PyUnicode_Decode(str, size, "utf-8", "replace"); #endif } -- 2.26.1.301.g55bc3eb7cb9-goog

...generator/python.ml @@ -225,10 +225,12 @@ put_table (char * const * const argv) | name, FChar -> pr "#ifdef HAVE_PYSTRING_ASSTRING\n"; pr " PyDict_SetItemString (dict, \"%s\",\n" name; - pr " PyString_FromStringAndSize (&dirent->%s, 1));\n" name; + pr " PyString_FromStringAndSize (&%s->%s, 1));\n" + typ name; pr "#else\n"; pr " PyDict_SetItemString (dict, \"%s\",\n" name; -...

...e> > File "/opt/libguestfs/x86_64/1.32.10/lib/python3.6/site-packages/guestfs.py", line 73, in <module> > import libguestfsmod > ImportError: /opt/libguestfs/x86_64/1.32.10/lib/python3.6/site-packages/libguestfsmod.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PyString_FromStringAndSize Somehow during ./configure it thought PyString_FromStringAndSize is defined, but when running it's not. I believe that symbol is defined by Python 2 and was removed in Python 3. Most likely the ./configure test is running against the wrong version of Python, but I can't tell without the...

The interesting line is this one ... checking for PyString_AsString in -lc... no and if you compare that to the code, eg in python/guestfs-py.c: #ifdef HAVE_PYSTRING_ASSTRING PyDict_SetItemString (dict, "ftyp", PyString_FromStringAndSize (&dirent->ftyp, 1)); #else PyDict_SetItemString (dict, "ftyp", PyUnicode_FromStringAndSize (&dirent->ftyp, 1)); #endif So it shouldn't be using that function (PyString_FromStringAndSize) at all. I think most likely you've either attached t...

...Unicode_DecodeUTF8 (argv[i], strlen (argv[i]), NULL)); } return list; @@ -2985,11 +2981,7 @@ put_val_type (char *val, size_t len, hive_type t) { PyObject *r = PyTuple_New (2); PyTuple_SetItem (r, 0, PyLong_FromLong ((long) t)); -#ifdef HAVE_PYSTRING_ASSTRING - PyTuple_SetItem (r, 1, PyString_FromStringAndSize (val, len)); -#else - PyTuple_SetItem (r, 1, PyBytes_FromStringAndSize (val, len)); -#endif + PyTuple_SetItem (r, 1, PyUnicode_DecodeUTF8 (val, len, NULL)); return r; } @@ -3194,17 +3186,10 @@ put_val_type (char *val, size_t len, hive_type t) if f_len_exists name then...

...urn PyUnicode_FromString (str); > > + return guestfs_int_py_fromstringsize (str, strlen (str)); > > #endif > > } > > > > @@ -397,7 +397,12 @@ guestfs_int_py_fromstringsize (const char *str, > size_t size) > > #if PY_MAJOR_VERSION < 3 > > return PyString_FromStringAndSize (str, size); > > #else > > - return PyUnicode_FromStringAndSize (str, size); > > + PyObject *s = PyUnicode_FromString (str); > > + if (s == NULL) { > > + PyErr_Clear (); > > + s = PyUnicode_Decode (str, strlen(str), "latin1", "strict&quot...

--- configure.ac | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/configure.ac b/configure.ac index 6785037..203f34f 100644 --- a/configure.ac +++ b/configure.ac @@ -329,8 +329,8 @@ AS_IF([test "x$enable_python" != "xno"], AC_MSG_CHECKING([for Python extension suffix (PEP-3149)]) if test -z "$PYTHON_EXT_SUFFIX"; then

...return PyUnicode_FromString (str); > > + return guestfs_int_py_fromstringsize (str, strlen (str)); > > #endif > > } > > > > @@ -397,7 +397,12 @@ guestfs_int_py_fromstringsize (const char *str, size_t size) > > #if PY_MAJOR_VERSION < 3 > > return PyString_FromStringAndSize (str, size); > > #else > > - return PyUnicode_FromStringAndSize (str, size); > > + PyObject *s = PyUnicode_FromString (str); > > + if (s == NULL) { > > + PyErr_Clear (); > > + s = PyUnicode_Decode (str, strlen(str), "latin1", "strict&quot...

The Python 3 bindings currently are unable to deal with non UTF8 characters. This series continues what proposed in RHBZ#1406906. A new function 'set_decode_error_handler' allows the User to specify how to deal with decoding errors. The default behaviour will be raising a UnicodeDecodeError. If the handler is changed to 'surrogateescape', non UTF8 characters will be escaped in a

...se > - return PyUnicode_FromString (str); > + return PyUnicode_Decode(str, strlen(str), "utf-8", "replace"); > #endif > } > > @@ -397,7 +397,7 @@ guestfs_int_py_fromstringsize (const char *str, size_t size) > #if PY_MAJOR_VERSION < 3 > return PyString_FromStringAndSize (str, size); > #else > - return PyUnicode_FromStringAndSize (str, size); > + return PyUnicode_Decode(str, size, "utf-8", "replace"); > #endif > } Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o...

...else > - return PyUnicode_FromString (str); > + return PyUnicode_Decode(str, strlen(str), "utf-8", "replace"); > #endif > } > > @@ -397,7 +397,7 @@ guestfs_int_py_fromstringsize (const char *str, size_t size) > #if PY_MAJOR_VERSION < 3 > return PyString_FromStringAndSize (str, size); > #else > - return PyUnicode_FromStringAndSize (str, size); > + return PyUnicode_Decode(str, size, "utf-8", "replace"); > #endif > } > > -- > 2.26.1.301.g55bc3eb7cb9-goog > > > _______________________________________________ &gt...