search for: guestfs_int_py_fromstrings

...args) extern char **guestfs_int_py_get_string_list (PyObject *obj); extern PyObject *guestfs_int_py_put_string_list (char * const * const argv); extern PyObject *guestfs_int_py_put_table (char * const * const argv); +extern PyObject *guestfs_int_py_fromstring (const char *str); +extern PyObject *guestfs_int_py_fromstringsize (const char *str, size_t size); +extern char *guestfs_int_py_asstring (PyObject *obj); "; @@ -178,31 +181,16 @@ and generate_python_structs () = function | name, FString -> pr " PyDict_SetItemString (dict, \"%s\",\n" name; -...

...an-1-0-32bit libpoppler-glib4 libpoppler5 libsensors3 libtelepathy-glib0 m4 opensc opensc-32bit permissions pinentry poppler-tools python-gtksourceview splashy syslog-ng tar tightvnc xorg-x11 xorg-x11-xauth yast2-mouse Fix this by globally changing guestfs_int_py_fromstring() and guestfs_int_py_fromstringsize() to fallback to latin1 decoding if utf-8 decoding fails. Using the "strict" error handler doesn't matter in the case of latin1 and has the same effect of "replace": https://docs.python.org/3/library/codecs.html#error-handlers Signed-off-by: Sam Eiderman <sameid@go...

...an-1-0-32bit libpoppler-glib4 libpoppler5 libsensors3 libtelepathy-glib0 m4 opensc opensc-32bit permissions pinentry poppler-tools python-gtksourceview splashy syslog-ng tar tightvnc xorg-x11 xorg-x11-xauth yast2-mouse Fix this by globally changing guestfs_int_py_fromstring() and guestfs_int_py_fromstringsize() to decode utf-8 with the "replace" error handler: https://docs.python.org/3/library/codecs.html#error-handlers For example, this will decode PackageKit's description on SLES4 the following way: Backend: pisi S.�a&#287;lar Onur <caglar@pardus.org.tr> Sig...

...args) extern char **guestfs_int_py_get_string_list (PyObject *obj); extern PyObject *guestfs_int_py_put_string_list (char * const * const argv); extern PyObject *guestfs_int_py_put_table (char * const * const argv); +extern PyObject *guestfs_int_py_fromstring (const char *str); +extern PyObject *guestfs_int_py_fromstringsize (const char *str, size_t size); +extern char *guestfs_int_py_asstring (PyObject *obj); "; @@ -178,31 +181,16 @@ and generate_python_structs () = function | name, FString -> pr " PyDict_SetItemString (dict, \"%s\",\n" name; -...

The python3 bindings create unicode objects from application strings on the guest (i.e. installed rpm, deb packages). It is documented that rpm package fields such as description should be utf8 encoded - however in some cases they are not a valid unicode string, on SLES11 SP4 the following packages fail to be converted to unicode using guestfs_int_py_fromstring() (which invokes

...; + pr " goto err;\n"; + pr " PyDict_SetItemString (dict, \"%s\", value);\n" name; | name, FBuffer -> - pr " PyDict_SetItemString (dict, \"%s\",\n" name; - pr " guestfs_int_py_fromstringsize (%s->%s, %s->%s_len));\n" - typ name typ name + pr " value = guestfs_int_py_fromstringsize (%s->%s, %s->%s_len);\n" + typ name typ name; + pr " if (value == NULL)\n"; + pr " goto err;\n&quot...

...; + pr " goto err;\n"; + pr " PyDict_SetItemString (dict, \"%s\", value);\n" name; | name, FBuffer -> - pr " PyDict_SetItemString (dict, \"%s\",\n" name; - pr " guestfs_int_py_fromstringsize (%s->%s, %s->%s_len));\n" - typ name typ name + pr " value = guestfs_int_py_fromstringsize (%s->%s, %s->%s_len);\n" + typ name typ name; + pr " if (value == NULL)\n"; + pr " goto err;\n&quot...

...nentry > > poppler-tools > > python-gtksourceview > > splashy > > syslog-ng > > tar > > tightvnc > > xorg-x11 > > xorg-x11-xauth > > yast2-mouse > > > > Fix this by globally changing guestfs_int_py_fromstring() > > and guestfs_int_py_fromstringsize() to fallback to latin1 decoding if > > utf-8 decoding fails. > > > > Using the "strict" error handler doesn't matter in the case of latin1 > > and has the same effect of "replace": > > > > https://docs.python.org/3/library/codecs.html...

...--- a/python/handle.c > > +++ b/python/handle.c > > @@ -387,7 +387,7 @@ guestfs_int_py_fromstring (const char *str) > > #if PY_MAJOR_VERSION < 3 > > return PyString_FromString (str); > > #else > > - return PyUnicode_FromString (str); > > + return guestfs_int_py_fromstringsize (str, strlen (str)); > > #endif > > } > > > > @@ -397,7 +397,12 @@ guestfs_int_py_fromstringsize (const char *str, > size_t size) > > #if PY_MAJOR_VERSION < 3 > > return PyString_FromStringAndSize (str, size); > > #else > > - return P...

The python3 bindings create unicode objects from application strings on the guest (i.e. installed rpm, deb packages). It is documented that rpm package fields such as description should be utf8 encoded - however in some cases they are not a valid unicode string, on SLES11 SP4 the following packages fail to be converted to unicode using guestfs_int_py_fromstring() (which invokes

On Mon, Apr 20, 2020 at 01:17:35PM +0300, Sam Eiderman wrote: > The python3 bindings create unicode objects from application strings > on the guest (i.e. installed rpm, deb packages). > It is documented that rpm package fields such as description should be > utf8 encoded - however in some cases they are not a valid unicode > string, on SLES11 SP4 the following packages fail to be

...> opensc > opensc-32bit > permissions > pinentry > poppler-tools > python-gtksourceview > splashy > syslog-ng > tar > tightvnc > xorg-x11 > xorg-x11-xauth > yast2-mouse > > Fix this by globally changing guestfs_int_py_fromstring() > and guestfs_int_py_fromstringsize() to decode utf-8 with the "replace" > error handler: > > https://docs.python.org/3/library/codecs.html#error-handlers > > For example, this will decode PackageKit's description on SLES4 the > following way: > > Backend: pisi > S.�a&#2...

...4 > opensc > opensc-32bit > permissions > pinentry > poppler-tools > python-gtksourceview > splashy > syslog-ng > tar > tightvnc > xorg-x11 > xorg-x11-xauth > yast2-mouse > > Fix this by globally changing guestfs_int_py_fromstring() > and guestfs_int_py_fromstringsize() to decode utf-8 with the "replace" > error handler: > > https://docs.python.org/3/library/codecs.html#error-handlers > > For example, this will decode PackageKit's description on SLES4 the > following way: > > Backend: pisi > S.�a&#287;...

...> opensc > opensc-32bit > permissions > pinentry > poppler-tools > python-gtksourceview > splashy > syslog-ng > tar > tightvnc > xorg-x11 > xorg-x11-xauth > yast2-mouse > > Fix this by globally changing guestfs_int_py_fromstring() > and guestfs_int_py_fromstringsize() to fallback to latin1 decoding if > utf-8 decoding fails. > > Using the "strict" error handler doesn't matter in the case of latin1 > and has the same effect of "replace": > > https://docs.python.org/3/library/codecs.html#error-handlers > > S...

...gtksourceview > > > splashy > > > syslog-ng > > > tar > > > tightvnc > > > xorg-x11 > > > xorg-x11-xauth > > > yast2-mouse > > > > > > Fix this by globally changing guestfs_int_py_fromstring() > > > and guestfs_int_py_fromstringsize() to fallback to latin1 decoding if > > > utf-8 decoding fails. > > > > > > Using the "strict" error handler doesn't matter in the case of latin1 > > > and has the same effect of "replace": > > > > > > https://docs.py...

...nentry > > poppler-tools > > python-gtksourceview > > splashy > > syslog-ng > > tar > > tightvnc > > xorg-x11 > > xorg-x11-xauth > > yast2-mouse > > > > Fix this by globally changing guestfs_int_py_fromstring() > > and guestfs_int_py_fromstringsize() to decode utf-8 with the "replace" > > error handler: > > > > https://docs.python.org/3/library/codecs.html#error-handlers > > > > For example, this will decode PackageKit's description on SLES4 the > > following way: > > > > B...

...ex 2fb8c18f0..fe89dc58a 100644 > --- a/python/handle.c > +++ b/python/handle.c > @@ -387,7 +387,7 @@ guestfs_int_py_fromstring (const char *str) > #if PY_MAJOR_VERSION < 3 > return PyString_FromString (str); > #else > - return PyUnicode_FromString (str); > + return guestfs_int_py_fromstringsize (str, strlen (str)); > #endif > } > > @@ -397,7 +397,12 @@ guestfs_int_py_fromstringsize (const char *str, size_t size) > #if PY_MAJOR_VERSION < 3 > return PyString_FromStringAndSize (str, size); > #else > - return PyUnicode_FromStringAndSize (str, size); &gt...

...+399,8 @@ guestfs_int_py_fromstring (const char *str) #ifdef HAVE_PYSTRING_ASSTRING return PyString_FromString (str); #else - return PyUnicode_FromString (str); + Py_ssize_t size = strlen(str); + return PyUnicode_DecodeUTF8 (str, size, decode_error_handler); #endif } @@ -396,7 +410,7 @@ guestfs_int_py_fromstringsize (const char *str, size_t size) #ifdef HAVE_PYSTRING_ASSTRING return PyString_FromStringAndSize (str, size); #else - return PyUnicode_FromStringAndSize (str, size); + return PyUnicode_DecodeUTF8 (str, size, decode_error_handler); #endif } diff --git a/python/t/test830RHBZ1406906.py b/py...

The Python 3 bindings currently are unable to deal with non UTF8 characters. This series continues what proposed in RHBZ#1406906. A new function 'set_decode_error_handler' allows the User to specify how to deal with decoding errors. The default behaviour will be raising a UnicodeDecodeError. If the handler is changed to 'surrogateescape', non UTF8 characters will be escaped in a

...thon < 3 support is not dropped yet, however it will be easier after this series. Pino Toscano (7): build: enforce a minimum Python version python: drop code for Python < 2.5 python: assume support for Capsules python: remove compile time check for PyString_AsString python: replace guestfs_int_py_fromstringsize in Py2 branches python: stop including config.h python: stop bundling config.h in dist archives .gitignore | 1 - generator/python.ml | 32 ++------------------------------ m4/guestfs-python.m4 | 21 +++++---------------- python/MANIFEST.in | 1 - python/Makefile.am | 5 -...