similar to: [nbdkit] can't import mmap

On Thu, Oct 29, 2015 at 09:19:03AM -0500, Jason Pepas wrote: > Hi, > > I'm new to nbdkit and am trying to run a simple "single file block > device using mmap" setup, using this python script: > > import mmap > > def config(key, value): > print ("ignored parameter %s=%s" % (key, value)) > > def open(readonly): > print

Hi, the nbdkit python plugin example has suboptimal memory management: - it creates the disk image as a string on init - it casts the string to bytearray on every read - it copies the string before and the string after the written region, then reassembles those pieces together with the written region to a new disk image string This is not a problem as long as the image is small, but in my tests

Hi, sorry for the resend. My mailer mangled the previous attempt. On 26.09.2016 17:19, Pino Toscano wrote: > On Monday, 26 September 2016 17:07:41 CEST Carl-Daniel Hailfinger wrote: >> the nbdkit python plugin example has suboptimal memory management: >> - it creates the disk image as a string on init >> - it casts the string to bytearray on every read >> - it copies

On Sat, Nov 23, 2019 at 3:10 PM Richard W.M. Jones <rjones@redhat.com> wrote: > > On Sat, Nov 23, 2019 at 01:42:15AM +0200, Nir Soffer wrote: > > On Fri, Nov 22, 2019 at 9:55 PM Richard W.M. Jones <rjones@redhat.com> wrote: > > > +def pread(h, count, offset, flags): > > > + assert flags == 0 > > > + return

On Fri, Nov 22, 2019 at 9:55 PM Richard W.M. Jones <rjones@redhat.com> wrote: > > This tests the Python plugin thoroughly by issuing client commands > through libnbd and checking we get the expected results. > --- > tests/Makefile.am | 13 +-- > tests/test-python-plugin.py | 134 ++++++++++++++++++++++++++++ > tests/test-python.py | 172

Passing a memoryview we avoid unneeded copy of the original buffer. On the python side memoryview object can be used for slicing, writing to file, or sending to socket. This may break plugins assuming that the they get a bytearray, but good python code should not care about the type of the buffer, only about the behaviour. Testing with a plugin writing to /dev/null shows 2.7x speedup. Real

On Sun, Nov 24, 2019, 12:42 Richard W.M. Jones <rjones@redhat.com> wrote: > On Sat, Nov 23, 2019 at 06:11:47PM +0200, Nir Soffer wrote: > > On Sat, Nov 23, 2019 at 3:10 PM Richard W.M. Jones <rjones@redhat.com> > wrote: > > > > > > On Sat, Nov 23, 2019 at 01:42:15AM +0200, Nir Soffer wrote: > > > > On Fri, Nov 22, 2019 at 9:55 PM Richard W.M.

v2 was here: https://www.redhat.com/archives/libguestfs/2019-November/msg00163.html I pushed patch 1 (with spelling fix), patch 4 and patch 5 since those were previously ACKed on the list. Differences in v3: - Add error checking to PyModule_AddIntConstant. - Use API_VERSION constant instead of function. - Add max API version supported to --dump-plugin output. - Print API_VERSION selected by

v3 was here: https://www.redhat.com/archives/libguestfs/2019-November/msg00209.html In v4: - Rebase on top of current master. Includes various fixes and updates required because of Nir's patches that went into master. - Fix api_version() -> API_VERSION in patch 2 noted previously on the mailing list. Rich.

As suggested by Nir, here: https://www.redhat.com/archives/libguestfs/2019-November/thread.html#00220

v1: https://www.redhat.com/archives/libguestfs/2019-November/msg00153.html v2: - Fix implementation of can_cache. - Add implementation of can_fua. - Add a very thorough test suite which tests every command + flag combination.

On Sat, Aug 4, 2018 at 10:58 PM Nir Soffer <nsoffer@redhat.com> wrote: > On Sat, Aug 4, 2018 at 4:04 PM Richard W.M. Jones <rjones@redhat.com> > wrote: > >> This is only lightly tested (against just qemu NBD client), and the >> code might be structured a little better as the >> _negotiate_handshake_newstyle_options function has now grown to be >> huge.

And fill out most of the missing bits of the API. Rich.

Hi, So I've been hacking together an nbdkit plugin (similar to the "file" plugin, but it splits the file up into chunks): https://github.com/pepaslabs/nbdkit-chunks-plugin I got it to the point of being a working prototype. Then I threw it onto a raspberry pi, which it turns out only has a 50/50 shot of fallocate() working correctly. I'm checking the return code of

See comment for explanation and https://listman.redhat.com/archives/libguestfs/2023-August/032468.html --- tests/Makefile.am | 2 + plugins/sh/call.c | 33 +++++++----- tests/test-sh-pwrite-ignore-stdin.sh | 77 ++++++++++++++++++++++++++++ 3 files changed, 100 insertions(+), 12 deletions(-) diff --git a/tests/Makefile.am b/tests/Makefile.am index

This doesn't yet work. However it does make me more convinced than ever that we really need to sort out persistent buffer lifetimes in the library (similar to what we did for closures). Rich.

This is only lightly tested (against just qemu NBD client), and the code might be structured a little better as the _negotiate_handshake_newstyle_options function has now grown to be huge. Anyway works for me. Rich.

Very much a work in progress as there are still many tests using qemu-io which are candidates for conversion. You'll notice at the end of test-full.sh that the new test has some duplicated code which looks as if it ought to be refactored into a Python function. When I tried to do that, I got loads of strange Python problems which may indicate bugs in nbdsh itself or problems with my

These patches improve error handling when PUT request fail, including the error response from oVirt server. This will make it easier to debug issue when oVirt server logs have been rotated. Nir Soffer (2): v2v: rhv-upload-plugin: Handle send send failures v2v: rhv-upload-plugin: Fix error formatting v2v/rhv-upload-plugin.py | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7

pread() can return now any object supporting the buffer protocol[1] such as bytearray, bytes, or memoryview. [1] https://docs.python.org/3/c-api/buffer.html --- When the new python tests will be in master we can add some automated tests. For now I tested this manually using the example plugin as is and with these changes:  def pread(h, count, offset):      global disk     return