search for: pygilstate_ensure

All Py_* functions should be protected by the GIL. Otherwise internal python data structures can get corrupted. Move PyGILState_Ensure to the beginning of the block and PyGILState_Release to the bottom. Signed-off-by: Sam Eiderman <sameid@google.com> --- python/handle.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/python/handle.c b/python/handle.c index ffd0794d8..659bb5b06 100644 ---...

...on close is not locked against the requests within that connection, and any garbage collection triggered by the last request may still be running while plugin_close runs. Boom. In general, when working with python any call into python should be wrapped like this: PyGILState_STATE gstate; gstate = PyGILState_Ensure(); /* Perform Python actions here. */ result = CallSomeFunction(); /* evaluate result or handle exception */ /* Release the thread. No Python API allowed beyond this point. */ PyGILState_Release(gstate); That would even allow us to run everything in parallel. I have now added a call to plugin_...

...| CBUInt n | CBUInt64 n -> pr ", %s" n + | CBArrayAndLen _ | CBMutable _ -> assert false + ) cbargs; + pr ");\n"; + pr " Py_INCREF (py_args);\n"; + pr "\n"; + pr " if (PyEval_ThreadsInitialized ())\n"; + pr " py_save = PyGILState_Ensure ();\n"; + pr "\n"; + pr " py_ret = PyObject_CallObject ((PyObject *)user_data, py_args);\n"; + pr "\n"; + pr " if (PyEval_ThreadsInitialized ())\n"; + pr " PyGILState_Release (py_save);\n"; + pr "\n"; + pr "...

A couple of fixes for Python 3 to the Python binding. Unfortunately a behaviour change is needed, although it fixes broken types used. Changes from v1: - handle also FBuffer in structs Pino Toscano (2): python: fix call of Python handlers of events python: change types for RBufferOut/FBuffer with Python 3 (RHBZ#1661871) generator/python.ml | 9 +++++++++ python/handle.c | 3 ++- 2

A couple of fixes for Python 3 to the Python binding. Unfortunately a behaviour change is needed, although it fixes broken types used. Pino Toscano (2): python: fix call of Python handlers of events python: change return type for RBufferOut with Python 3 (RHBZ#1661871) generator/python.ml | 4 ++++ python/handle.c | 3 ++- 2 files changed, 6 insertions(+), 1 deletion(-) -- 2.20.1

Patch 2 certainly allows you to set the thread model. However patch 3 shows that if you set it to nbdkit.THREAD_MODEL_PARALLEL it will crash. If you look closely at the stack trace (attached below) you can see that ignoring threads which are in parts of nbdkit unrelated to Python: Thread 4: In pread, waiting in time.sleep(). This thread has released the GIL. Thread 2: Started to

..._ + | Path _ | SockAddrAndLen _ | StringList _ + | UInt _ | UInt32 _ -> assert false + ) cb.cbargs; + pr ");\n"; + pr " Py_INCREF (py_args);\n"; + pr "\n"; + pr " if (PyEval_ThreadsInitialized ())\n"; + pr " py_save = PyGILState_Ensure ();\n"; + pr "\n"; + pr " py_ret = PyObject_CallObject (data->%s, py_args);\n" cb.name; + pr "\n"; + pr " if (PyEval_ThreadsInitialized ())\n"; + pr " PyGILState_Release (py_save);\n"; + pr "\n"; + pr &quo...

...Mutable _ -> assert false ) cbargs; pr ");\n"; - pr " Py_INCREF (py_args);\n"; + pr " Py_INCREF (py_args);\n"; pr "\n"; - pr " if (PyEval_ThreadsInitialized ())\n"; - pr " py_save = PyGILState_Ensure ();\n"; + pr " if (PyEval_ThreadsInitialized ())\n"; + pr " py_save = PyGILState_Ensure ();\n"; pr "\n"; - pr " py_ret = PyObject_CallObject ((PyObject *)user_data, py_args);\n"; + pr " py_ret = PyObject_CallO...

Posting now that I got something to compile (at the expense of breaking OCaml bindings), but I'm open to ideas on how to improve it. Eric Blake (2): generator: Tweak print_c_arg_list to take alternate first arg RFC: generator: Handle shared callbacks in Python generator/generator | 556 ++++++++++++++++++++++---------------------- 1 file changed, 280 insertions(+), 276 deletions(-) --

...rgs } = | CBArrayAndLen _ | CBMutable _ -> assert false ) cbargs; pr ");\n"; - pr " Py_INCREF (py_args);\n"; + pr " Py_INCREF (py_args);\n"; pr "\n"; - pr " if (PyEval_ThreadsInitialized ())\n"; - pr " py_save = PyGILState_Ensure ();\n"; + pr " if (PyEval_ThreadsInitialized ())\n"; + pr " py_save = PyGILState_Ensure ();\n"; pr "\n"; - pr " py_ret = PyObject_CallObject ((PyObject *)user_data, py_args);\n"; + pr " py_ret = PyObject_CallObject ((PyObject *)user_d...

Basically the same as this patch series, but for Python: https://www.redhat.com/archives/libguestfs/2019-August/msg00235.html plus adding the 590 asynch test at the end. Rich.

...| UInt32 _ -> assert false ) cbargs; pr ");\n"; - pr " Py_INCREF (py_args);\n"; + pr " Py_INCREF (py_args);\n"; pr "\n"; - pr " if (PyEval_ThreadsInitialized ())\n"; - pr " py_save = PyGILState_Ensure ();\n"; + pr " if (PyEval_ThreadsInitialized ())\n"; + pr " py_save = PyGILState_Ensure ();\n"; pr "\n"; - pr " py_ret = PyObject_CallObject ((PyObject *)user_data, py_args);\n"; + pr " py_ret = PyObject_C...

This is working for me now, although possibly only on Python 3.9. Dan suggested PyEval_InitThreads but that was deprecated in Python 3.7. Rich.

...List _ - | UInt _ | UInt32 _ -> assert false - ) cbargs; - pr ");\n"; - pr " Py_INCREF (py_args);\n"; - pr "\n"; - pr " if (PyEval_ThreadsInitialized ())\n"; - pr " py_save = PyGILState_Ensure ();\n"; - pr "\n"; - pr " py_ret = PyObject_CallObject (((struct %s_user_data *)user_data)->%s, py_args);\n" name cbname; - pr "\n"; - pr " if (PyEval_ThreadsInitialized ())\n"; - pr " PyGILS...

v1 was here: https://www.redhat.com/archives/libguestfs/2019-July/thread.html#00231 The changes address everything that Eric picked up in his review of the first two patches. I have also added two more patches (4 and 5) which respectively fix docs and change int status -> unsigned status, as discussed. Passes make, check, check-valgrind. Rich.

On 26.09.2016 14:29, Richard W.M. Jones wrote: > On Mon, Sep 26, 2016 at 02:18:02PM +0200, Carl-Daniel Hailfinger wrote: >> Hi, >> >> has anyone ever run "make check" from nbd against nbdkit with a python >> plugin? I usually get segfaults during such a run, and sometimes various >> other errors happen before the segfault, suggesting that some memory

Patches 1-4 are basically uncontroversial, straightforward refactoring and IMHO we should just push them. Possibly 1-3 should be squashed together, but I posted them separately so they are easier to review. Patches 5 and 6 together implement OClosure. Patch 5 adds the feature and is simple to understand. Patch 6 changes the Closure completion callbacks into OClosure, but because it doesn't

..._ | SockAddrAndLen _ | StringList _ - | UInt _ | UInt32 _ -> assert false - ) args; - pr ");\n"; - pr " Py_INCREF (py_args);\n"; - pr "\n"; - pr " if (PyEval_ThreadsInitialized ())\n"; - pr " py_save = PyGILState_Ensure ();\n"; - pr "\n"; - pr " py_ret = PyObject_CallObject (_data->fn, py_args);\n"; - pr "\n"; - pr " if (PyEval_ThreadsInitialized ())\n"; - pr " PyGILState_Release (py_save);\n"; - pr "\n"; -...

As before, but this one has working Python bindings. OCaml still TBD. Rich.

As proposed here: https://www.redhat.com/archives/libguestfs/2019-August/msg00130.html I didn't actually read Eric's replies to that yet because I've been concentrating on writing these patches all day. Anyway here they are and I'll look at what Eric said about the proposal next. Rich.