search for: py_close

...urn NULL; + + name = python_to_string (r); + Py_DECREF (r); + if (!name) { + nbdkit_error ("default_export method did not return a string"); + return NULL; + } + + return nbdkit_strdup_intern (name); +} + struct handle { int can_zero; PyObject *py_h; @@ -595,6 +688,35 @@ py_close (void *handle) free (h); } +static const char * +py_export_description (void *handle) +{ + ACQUIRE_PYTHON_GIL_FOR_CURRENT_SCOPE; + struct handle *h = handle; + PyObject *fn; + PyObject *r; + CLEANUP_FREE char *desc = NULL; + + if (!callback_defined ("export_description", &f...

...ff704e6d0 in ?? () from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 #7 0x00007ffff6fbad43 in PyObject_Call () from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 #8 0x00007ffff7033577 in PyEval_CallObjectWithKeywords () from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 #9 0x00007ffff73f0af0 in py_close (handle=0x625338) at python.c:234 #10 0x0000000000405803 in plugin_close (conn=conn@entry=0x7fffe8000910) at plugins.c:377 #11 0x00000000004037ec in free_connection (conn=0x7fffe8000910) at connections.c:147 #12 0x0000000000404476 in _handle_single_connection (sockout=<optimized out>, sockin=...

...lose function pickles all python objects which > were accessed in the plugin. Maybe that's what exposes the memory > corruption on close. > [...] > Will try instrumenting the close function next. The results are in. I dumped the reference counters for various python objects inside py_close and often the reference counters are off-by-one or off-by-two in the crashing case. AFAICS the python API says that once you have threads, you have to do the GIL dance: https://docs.python.org/2/c-api/init.html#thread-state-and-the-global-interpreter-lock . All the crashes I saw can be explained b...

...() from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 > #7 0x00007ffff6fbad43 in PyObject_Call () from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 > #8 0x00007ffff7033577 in PyEval_CallObjectWithKeywords () from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 > #9 0x00007ffff73f0af0 in py_close (handle=0x625338) at python.c:234 > #10 0x0000000000405803 in plugin_close (conn=conn@entry=0x7fffe8000910) at plugins.c:377 > #11 0x00000000004037ec in free_connection (conn=0x7fffe8000910) at connections.c:147 > #12 0x0000000000404476 in _handle_single_connection (sockout=<optimized o...

...ython.c b/plugins/python/python.c index 35e8df2..c5cc4bd 100644 --- a/plugins/python/python.c +++ b/plugins/python/python.c @@ -183,8 +183,7 @@ py_load (void) static void py_unload (void) { - if (module) - Py_DECREF (module); + Py_XDECREF (module); Py_Finalize (); } @@ -346,8 +345,7 @@ py_close (void *handle) Py_DECREF (fn); Py_DECREF (args); check_python_failure ("close"); - if (r) - Py_DECREF (r); + Py_XDECREF (r); } Py_DECREF (obj); @@ -550,8 +548,7 @@ py_zero (void *handle, uint32_t count, uint64_t offset, int may_trim) gracefully f...

This picks up python and ocaml. Some of our languages are lacking a number of bindings (for example, lua and perl lack .extents, so I didn't have anything to copy from), and I felt less comfortable with golang and rust. But for python and ocaml, I was able to test a working implementation. Eric Blake (2): python: Implement .list_exports and friends ocaml: Implement .list_exports and

I noticed these while working on adding fua support into python, these are independent enough to push now (and I'll have to rebase my 'optional may_trim' patch on top of this). Eric Blake (2): python: Use Py_XDEFREF() python: Simplify calling into plugin plugins/python/python.c | 106 ++++++++---------------------------------------- 1 file changed, 18 insertions(+), 88

...t the end of the world. Otherwise, nice reduction in lines of code (it may get trickier with can_FOO that return tristate, but for this patch you really did simplify true bool/error return functions). > @@ -812,9 +758,9 @@ static struct nbdkit_plugin plugin = { > .close = py_close, > > .get_size = py_get_size, > + .is_rotational = py_is_rotational, > .can_write = py_can_write, > .can_flush = py_can_flush, > - .is_rotational = py_is_rotational, > .can_trim = py_can_trim, > > .prea...

...o this). - */ - else if (callback_defined ("trim", NULL)) - return 1; - else - return 0; + return boolean_callback (handle, "can_trim", "trim"); } #define py_config_help \ @@ -812,9 +758,9 @@ static struct nbdkit_plugin plugin = { .close = py_close, .get_size = py_get_size, + .is_rotational = py_is_rotational, .can_write = py_can_write, .can_flush = py_can_flush, - .is_rotational = py_is_rotational, .can_trim = py_can_trim, .pread = py_pread, -- 2.23.0

This is working for me now, although possibly only on Python 3.9. Dan suggested PyEval_InitThreads but that was deprecated in Python 3.7. Rich.

...ndle = PyObject_CallObject (fn, args); + handle = PyObject_CallFunctionObjArgs (fn, readonly ? Py_True : Py_False, + NULL); Py_DECREF (fn); - Py_DECREF (args); if (check_python_failure ("open") == -1) return NULL; @@ -332,18 +319,13 @@ py_close (void *handle) { PyObject *obj = handle; PyObject *fn; - PyObject *args; PyObject *r; if (callback_defined ("close", &fn)) { PyErr_Clear (); - args = PyTuple_New (1); - Py_INCREF (obj); /* decremented by Py_DECREF (args) */ - PyTuple_SetItem (args, 0, obj...

...ue : Py_False, NULL); Py_DECREF (fn); - if (check_python_failure ("open") == -1) + if (check_python_failure ("open") == -1) { + free (h); return NULL; + } - return handle; + h->fua = -1; + return h; } static void py_close (void *handle) { - PyObject *obj = handle; + ConnHandle *h = handle; PyObject *fn; PyObject *r; if (callback_defined ("close", &fn)) { PyErr_Clear (); - r = PyObject_CallFunctionObjArgs (fn, obj, NULL); + r = PyObject_CallFunctionObjArgs (fn, h->obj, NULL);...

...Py_DECREF (fn); > - if (check_python_failure ("open") == -1) > + if (check_python_failure ("open") == -1) { > + free (h); > return NULL; > + } > > - return handle; > + h->fua = -1; > + return h; > } > > static void > py_close (void *handle) > { > - PyObject *obj = handle; > + ConnHandle *h = handle; > PyObject *fn; > PyObject *r; > > if (callback_defined ("close", &fn)) { > PyErr_Clear (); > > - r = PyObject_CallFunctionObjArgs (fn, obj, NULL); > +...

Hi, has anyone ever run "make check" from nbd against nbdkit with a python plugin? I usually get segfaults during such a run, and sometimes various other errors happen before the segfault, suggesting that some memory corruption is underway. AFAICS a pure python plugin should not be able to cause memory corruption. Examples of nbdkit logs for running "make check" or subsets of

First out of our four language bindings to add FUA support (for reference, I added 'zero' support for python, perl, and ruby back in 1.1.13, then Rich had to add it for ocaml in 1.1.20). I tested this heavily under python 2, but for now only compile tested under python 3; I plan to do further testing there and make any tweaks if necessary. I wrote patch 5 early on, but then realized I

And fill out most of the missing bits of the API. Rich.

I like this change. I think we were overloading the config_complete method before to do two different things (complete configuration; do any allocation/housekeeping necessary before we can start serving). The only questions in my mind are whether we want this before 1.18, and whether the name ("get_ready") is a good one. Rich.

When I added .can_FOO caching in 1.16, I missed the case that the sh plugin itself was calling .can_flush twice in some situations (in order to default .can_fua). Then right after, I regressed it to call .can_zero twice (in order to default .can_fast_zero). I also missed that .thread_model could use better caching, because at the time, I did not add testsuite coverage. Fix that now. Eric Blake

v1: https://www.redhat.com/archives/libguestfs/2019-November/msg00153.html v2: - Fix implementation of can_cache. - Add implementation of can_fua. - Add a very thorough test suite which tests every command + flag combination.

It's been several weeks since I posted v2 (I got distracted by improving libnbd to better test things, which in turn surfaced some major memory leak problems in nbdsh that are now fixed). Many of the patches are minor rebases from v2, with the biggest changes being fallout from: - patch 2: rename nbdkit_add_default_export to nbdkit_use_default_export - overall: this missed 1.22, so update