search for: pyunicode_fromstr

...that rpm package fields such as description should be utf8 encoded - however in some cases they are not a valid unicode string, on SLES11 SP4 the encoding of the description of the following packages is latin1 and they fail to be converted to unicode using guestfs_int_py_fromstring() (which invokes PyUnicode_FromString()): PackageKit aaa_base coreutils dejavu desktop-data-SLED gnome-utils hunspell hunspell-32bit hunspell-tools libblocxx6 libexif libgphoto2 libgtksourceview-2_0-0 libmpfr1 libopensc2 libopensc2-32bit liborc-0_4-0 libpackagekit-glib10 libpixman-1-0 libpixman-1-0-32bit libpop...

...ing (dict, \"%s\",\n" name; - pr "#ifdef HAVE_PYSTRING_ASSTRING\n"; - pr " PyString_FromString (%s->%s));\n" - typ name; - pr "#else\n"; - pr " PyUnicode_FromString (%s->%s));\n" - typ name; - pr "#endif\n" + pr " guestfs_int_py_fromstring (%s->%s));\n" + typ name | name, FBuffer -> pr " PyDict_SetItemString (dict, \"%s\&...

...ing (dict, \"%s\",\n" name; - pr "#ifdef HAVE_PYSTRING_ASSTRING\n"; - pr " PyString_FromString (%s->%s));\n" - typ name; - pr "#else\n"; - pr " PyUnicode_FromString (%s->%s));\n" - typ name; - pr "#endif\n" + pr " guestfs_int_py_fromstring (%s->%s));\n" + typ name | name, FBuffer -> pr " PyDict_SetItemString (dict, \"%s\&...

...(i.e. installed rpm, deb packages). It is documented that rpm package fields such as description should be utf8 encoded - however in some cases they are not a valid unicode string, on SLES11 SP4 the following packages fail to be converted to unicode using guestfs_int_py_fromstring() (which invokes PyUnicode_FromString()): PackageKit aaa_base coreutils dejavu desktop-data-SLED gnome-utils hunspell hunspell-32bit hunspell-tools libblocxx6 libexif libgphoto2 libgtksourceview-2_0-0 libmpfr1 libopensc2 libopensc2-32bit liborc-0_4-0 libpackagekit-glib10 libpixman-1-0 libpixman-1-0-32bit libpop...

...ion should be > > utf8 encoded - however in some cases they are not a valid unicode > > string, on SLES11 SP4 the encoding of the description of the following > > packages is latin1 and they fail to be converted to unicode using > > guestfs_int_py_fromstring() (which invokes PyUnicode_FromString()): > > Sorry, I wanted to reach our resident Python maintainers to get their > feedback, and so far had no time for it. Will do it shortly. > > BTW do you have a reproducer I can actually try freely? > > > diff --git a/python/handle.c b/python/handle.c > > index 2f...

...ion should be > > utf8 encoded - however in some cases they are not a valid unicode > > string, on SLES11 SP4 the encoding of the description of the following > > packages is latin1 and they fail to be converted to unicode using > > guestfs_int_py_fromstring() (which invokes PyUnicode_FromString()): > > > > PackageKit > > aaa_base > > coreutils > > dejavu > > desktop-data-SLED > > gnome-utils > > hunspell > > hunspell-32bit > > hunspell-tools > > libblocxx6 > > libexif > > libgphoto2 > > lib...

This series addresses the issue where non UTF8 file names in a guest image lead to libguestfs segfault with Python 3 APIs. The core issue is the APIs are not checking the return value when constructing a new PyObject. Therefore NULL pointers are added to Python collections (lists and dictionaries) crashing the application. Few notes regarding the comments on the previous patch. - Added a

...lds such as description should be > utf8 encoded - however in some cases they are not a valid unicode > string, on SLES11 SP4 the encoding of the description of the following > packages is latin1 and they fail to be converted to unicode using > guestfs_int_py_fromstring() (which invokes PyUnicode_FromString()): > > PackageKit > aaa_base > coreutils > dejavu > desktop-data-SLED > gnome-utils > hunspell > hunspell-32bit > hunspell-tools > libblocxx6 > libexif > libgphoto2 > libgtksourceview-2_0-0 > libmpfr1 > libopensc2 > libopensc2-...

...lds such as description should be > utf8 encoded - however in some cases they are not a valid unicode > string, on SLES11 SP4 the encoding of the description of the following > packages is latin1 and they fail to be converted to unicode using > guestfs_int_py_fromstring() (which invokes PyUnicode_FromString()): Sorry, I wanted to reach our resident Python maintainers to get their feedback, and so far had no time for it. Will do it shortly. BTW do you have a reproducer I can actually try freely? > diff --git a/python/handle.c b/python/handle.c > index 2fb8c18f0..fe89dc58a 100644 > --- a/...

...gt; > utf8 encoded - however in some cases they are not a valid unicode > > > string, on SLES11 SP4 the encoding of the description of the following > > > packages is latin1 and they fail to be converted to unicode using > > > guestfs_int_py_fromstring() (which invokes PyUnicode_FromString()): > > > > > > PackageKit > > > aaa_base > > > coreutils > > > dejavu > > > desktop-data-SLED > > > gnome-utils > > > hunspell > > > hunspell-32bit > > > hunspell-tools > > > libblocxx6...

...(i.e. installed rpm, deb packages). It is documented that rpm package fields such as description should be utf8 encoded - however in some cases they are not a valid unicode string, on SLES11 SP4 the following packages fail to be converted to unicode using guestfs_int_py_fromstring() (which invokes PyUnicode_FromString()): PackageKit aaa_base coreutils dejavu desktop-data-SLED gnome-utils hunspell hunspell-32bit hunspell-tools libblocxx6 libexif libgphoto2 libgtksourceview-2_0-0 libmpfr1 libopensc2 libopensc2-32bit liborc-0_4-0 libpackagekit-glib10 libpixman-1-0 libpixman-1-0-32bit libpop...

...packages). > It is documented that rpm package fields such as description should be > utf8 encoded - however in some cases they are not a valid unicode > string, on SLES11 SP4 the following packages fail to be converted to > unicode using guestfs_int_py_fromstring() (which invokes > PyUnicode_FromString()): > > PackageKit > aaa_base > coreutils > dejavu > desktop-data-SLED > gnome-utils > hunspell > hunspell-32bit > hunspell-tools > libblocxx6 > libexif > libgphoto2 > libgtksourceview-2_0-0 > libmpfr1 > libopensc2 > libopensc2-...

...except UnicodeDecodeError: value.decode("latin1") This will always succeed, producing possibly garbage output but so is errors='replace'. > on SLES11 SP4 the following packages fail to be converted to > unicode using guestfs_int_py_fromstring() (which invokes > PyUnicode_FromString()): > > PackageKit > aaa_base > coreutils > dejavu > desktop-data-SLED > gnome-utils > hunspell > hunspell-32bit > hunspell-tools > libblocxx6 > libexif > libgphoto2 > libgtksourceview-2_0-0 > libmpfr1 > libopensc2 > libopensc2-3...

...ator.ml +++ b/generator/generator.ml @@ -2925,11 +2925,7 @@ put_string_list (char * const * const argv) list = PyList_New (argc); for (i = 0; i < argc; ++i) { -#ifdef HAVE_PYSTRING_ASSTRING - PyList_SetItem (list, i, PyString_FromString (argv[i])); -#else - PyList_SetItem (list, i, PyUnicode_FromString (argv[i])); -#endif + PyList_SetItem (list, i, PyUnicode_DecodeUTF8 (argv[i], strlen (argv[i]), NULL)); } return list; @@ -2985,11 +2981,7 @@ put_val_type (char *val, size_t len, hive_type t) { PyObject *r = PyTuple_New (2); PyTuple_SetItem (r, 0, PyLong_FromLong ((long) t)); -...

...thon.ml b/generator/python.ml index 4b823e6..637f22d 100644 --- a/generator/python.ml +++ b/generator/python.ml @@ -455,7 +455,8 @@ put_table (char * const * const argv) pr " py_r = PyString_FromString (r);\n"; pr "#else\n"; pr " py_r = PyUnicode_FromString (r);\n"; - pr "#endif\n" + pr "#endif\n"; + pr " if (py_r == NULL) goto out;\n"; | RConstOptString _ -> pr " if (r) {\n"; pr "#ifdef HAVE_PYSTRING_ASSTRING\n"; @@ -466,14 +46...

I noticed these while working on adding fua support into python, these are independent enough to push now (and I'll have to rebase my 'optional may_trim' patch on top of this). Eric Blake (2): python: Use Py_XDEFREF() python: Simplify calling into plugin plugins/python/python.c | 106 ++++++++---------------------------------------- 1 file changed, 18 insertions(+), 88

...value.decode("latin1") > > This will always succeed, producing possibly garbage output but so is > errors='replace'. > > > on SLES11 SP4 the following packages fail to be converted to > > unicode using guestfs_int_py_fromstring() (which invokes > > PyUnicode_FromString()): > > > > PackageKit > > aaa_base > > coreutils > > dejavu > > desktop-data-SLED > > gnome-utils > > hunspell > > hunspell-32bit > > hunspell-tools > > libblocxx6 > > libexif > > libgphoto2 > > lib...

...yBytes_AS_STRING (str)); } return NULL; @@ -159,11 +151,7 @@ print_python_traceback (const char *callback, *traceback_str; CLEANUP_FREE char *traceback_cstr = NULL; -#ifdef HAVE_PYSTRING_FROMSTRING - module_name = PyString_FromString ("traceback"); -#else module_name = PyUnicode_FromString ("traceback"); -#endif traceback_module = PyImport_Import (module_name); Py_DECREF (module_name); @@ -222,7 +210,6 @@ check_python_failure (const char *callback) return 0; } -#if PY_MAJOR_VERSION >= 3 static struct PyModuleDef moduledef = { PyModuleDef_HEAD_INIT,...

--- configure.ac | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/configure.ac b/configure.ac index 6785037..203f34f 100644 --- a/configure.ac +++ b/configure.ac @@ -329,8 +329,8 @@ AS_IF([test "x$enable_python" != "xno"], AC_MSG_CHECKING([for Python extension suffix (PEP-3149)]) if test -z "$PYTHON_EXT_SUFFIX"; then

...parameters are passed to the Python .config callback. */ PyErr_Clear (); - args = PyTuple_New (2); -#ifdef HAVE_PYSTRING_FROMSTRING - PyTuple_SetItem (args, 0, PyString_FromString (key)); - PyTuple_SetItem (args, 1, PyString_FromString (value)); -#else - PyTuple_SetItem (args, 0, PyUnicode_FromString (key)); - PyTuple_SetItem (args, 1, PyUnicode_FromString (value)); -#endif - r = PyObject_CallObject (fn, args); + r = PyObject_CallFunction (fn, "ss", key, value); Py_DECREF (fn); - Py_DECREF (args); if (check_python_failure ("config") == -1) ret...