Libguestfs - Jan 2019 - [PATCH v2 2/2] python: change types for RBufferOut/FBuffer with Python 3 (RHBZ#1661871)

A couple of fixes for Python 3 to the Python binding.

Unfortunately a behaviour change is needed, although it fixes broken
types used.

Changes from v1:
- handle also FBuffer in structs

Pino Toscano (2):
  python: fix call of Python handlers of events
  python: change types for RBufferOut/FBuffer with Python 3
    (RHBZ#1661871)

 generator/python.ml | 9 +++++++++
 python/handle.c     | 3 ++-
 2 files changed, 11 insertions(+), 1 deletion(-)

-- 
2.20.1

Make sure to reference the arguments, to make sure they are kept alive
during the function call; this is visible when setting an event handler
for the CLOSE event, and testing it with Python 3.

This does not seem to create a memory leak e.g. with Python 2.

Also, switch away from the quasi-internal PyEval_CallObject to the
public PyObject_CallObject, which takes care of doing safety checks.
---
 python/handle.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/python/handle.c b/python/handle.c
index e84a04195..adba1b823 100644
--- a/python/handle.c
+++ b/python/handle.c
@@ -136,11 +136,12 @@ guestfs_int_py_event_callback_wrapper (guestfs_h *g,
   args = Py_BuildValue ("(Kis#O)",
                         (unsigned PY_LONG_LONG) event, event_handle,
                         buf, buf_len, py_array);
+  Py_INCREF (args);
 
   if (PyEval_ThreadsInitialized ())
     py_save = PyGILState_Ensure ();
 
-  py_r = PyEval_CallObject (py_callback, args);
+  py_r = PyObject_CallObject (py_callback, args);
 
   if (PyEval_ThreadsInitialized ())
     PyGILState_Release (py_save);
-- 
2.20.1

So far RBufferOut return values, and FBuffer struct fields are 'str' on
all the versions of Python.  Python 3 distinguishes between 'str'
(unicode strings), and 'bytes', with 'str' no more able to hold
arbitrary data.

For this reason, switch the return value of RBufferOut functions, and
FBuffer struct fields to bytes on Python 3: while this is a potentially
incompatibile change, this is the only way to handle safely sequences
of arbitrary bytes.
---
 generator/python.ml | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/generator/python.ml b/generator/python.ml
index a70faec8c..bc5af528c 100644
--- a/generator/python.ml
+++ b/generator/python.ml
@@ -195,8 +195,13 @@ and generate_python_structs ()              pr "   
goto err;\n";
             pr "  PyDict_SetItemString (dict, \"%s\",
value);\n" name;
         | name, FBuffer ->
+            pr "#if PY_MAJOR_VERSION >= 3\n";
+            pr "  value = PyBytes_FromStringAndSize (%s->%s,
%s->%s_len);\n"
+              typ name typ name;
+            pr "#else\n";
             pr "  value = guestfs_int_py_fromstringsize (%s->%s,
%s->%s_len);\n"
               typ name typ name;
+            pr "#endif\n";
             pr "  if (value == NULL)\n";
             pr "    goto err;\n";
             pr "  PyDict_SetItemString (dict, \"%s\",
value);\n" name;
@@ -511,7 +516,11 @@ and generate_python_actions actions ()             pr
"  guestfs_int_free_string_list (r);\n";
            pr "  if (py_r == NULL) goto out;\n";
        | RBufferOut _ ->
+           pr "#if PY_MAJOR_VERSION >= 3\n";
+           pr "  py_r = PyBytes_FromStringAndSize (r, size);\n";
+           pr "#else\n";
            pr "  py_r = guestfs_int_py_fromstringsize (r, size);\n";
+           pr "#endif\n";
            pr "  free (r);\n";
            pr "  if (py_r == NULL) goto out;\n";
       );
-- 
2.20.1

ACK series.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html