Libguestfs - Jan 2022 - [PATCH nbdkit] plugins: python: Add error example

On Mon, Jan 31, 2022 at 12:14 AM Nir Soffer <nsoffer at redhat.com>
wrote:
>
> This plugin simulates errors in pread, pwrite, and extents. This is
> useful for testing error handling in NBD clients, and understanding how
> plugin exceptions are reported to the NBD client.
> ---
>  plugins/python/Makefile.am       |  1 +
>  plugins/python/examples/error.py | 77 ++++++++++++++++++++++++++++++++
>  2 files changed, 78 insertions(+)
>  create mode 100644 plugins/python/examples/error.py
>
> diff --git a/plugins/python/Makefile.am b/plugins/python/Makefile.am
> index eecd7e89..e6e6c9e6 100644
> --- a/plugins/python/Makefile.am
> +++ b/plugins/python/Makefile.am
> @@ -27,20 +27,21 @@
>  # ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
>  # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
>  # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
>  # SUCH DAMAGE.
>
>  include $(top_srcdir)/common-rules.mk
>
>  EXTRA_DIST = \
>         nbdkit-python-plugin.pod \
>         examples/file.py \
> +       examples/error.py \
>         examples/imageio.py \
>         examples/ramdisk.py \
>         examples/url.py \
>         $(NULL)
>
>  if HAVE_PYTHON
>
>  plugin_LTLIBRARIES = nbdkit-python-plugin.la
>
>  nbdkit_python_plugin_la_SOURCES = \
> diff --git a/plugins/python/examples/error.py
b/plugins/python/examples/error.py
> new file mode 100644
> index 00000000..0331a83e
> --- /dev/null
> +++ b/plugins/python/examples/error.py
> @@ -0,0 +1,77 @@
> +# Example Python plugin.
> +#
> +# This plugin simulates errors for testing NBD client error hanlding.
> +# Every odd call will fail, and every even call will succeed, unless
> +# there a real error accesing the specified file.
> +#
> +# This example can be freely used for any purpose.
> +
> +# Run it from the build directory like this:
> +#
> +#   ./nbdkit -f -v python ./plugins/python/examples/error.py file=test.img
> +#
> +# Or run it after installing nbdkit like this:
> +#
> +#   nbdkit -f -v python ./plugins/python/examples/error.py file=test.img
> +#
> +# The -f -v arguments are optional.  They cause the server to stay in
> +# the foreground and print debugging, which is useful when testing.
> +
> +import os
> +
> +API_VERSION = 2
> +
> +filename = None
> +calls = 0
> +
> +
> +def config(key, value):
> +    global filename
> +    assert key == "file"
> +    filename = value
> +
> +
> +def open(readonly):
> +    flags = os.O_RDONLY if readonly else os.O_RDWR
> +    return {"fd": os.open(filename, flags)}
> +
> +
> +def can_extents(h):
> +    return True
> +
> +
> +def get_size(h):
> +    return os.stat(h["fd"]).st_size
> +
> +
> +def extents(h, count, offset, flags):
> +    global calls
> +    calls += 1
> +    if calls % 2:
> +        raise RuntimeError(f"extents error offset={offset}
count={count}")
> +
> +    # We don't really support extents, so we report the entire file as
> +    # data.
> +    return [(offset, count, 0)]
> +
> +
> +def pread(h, buf, offset, flags):
> +    global calls
> +    calls += 1
> +    if calls % 2:
> +        raise RuntimeError(f"pread error offset={offset}
count={len(buf)}")
> +
> +    os.lseek(h["fd"], offset, os.SEEK_SET)
> +    n = os.readv(h['fd'], [buf])
We can use os.preadv in current python, but I'm trying to keep this plugin
useful on rhel 8 systems (python 3.6). os.preadv added in python 3.7.
> +    assert n == len(buf)
> +
> +
> +def pwrite(h, buf, offset, flags):
> +    global calls
> +    calls += 1
> +    if calls % 2:
> +        raise RuntimeError(f"pwrite error offset={offset}
count={len(buf)}")
> +
> +    os.lseek(h["fd"], offset, os.SEEK_SET)
> +    n = os.writev(h['fd'], [buf])
> +    assert n == len(buf)
> --
> 2.34.1
>

On Mon, Jan 31, 2022 at 12:17:36AM +0200, Nir Soffer
wrote:
> On Mon, Jan 31, 2022 at 12:14 AM Nir Soffer <nsoffer at redhat.com>
wrote:
> >
> > This plugin simulates errors in pread, pwrite, and extents. This is
> > useful for testing error handling in NBD clients, and understanding
how
> > plugin exceptions are reported to the NBD client.
> > ---
> >  plugins/python/Makefile.am       |  1 +
> >  plugins/python/examples/error.py | 77
++++++++++++++++++++++++++++++++
> >  2 files changed, 78 insertions(+)
> >  create mode 100644 plugins/python/examples/error.py
> >
> > diff --git a/plugins/python/Makefile.am b/plugins/python/Makefile.am
> > index eecd7e89..e6e6c9e6 100644
> > --- a/plugins/python/Makefile.am
> > +++ b/plugins/python/Makefile.am
> > @@ -27,20 +27,21 @@
> >  # ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
> >  # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
> >  # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> >  # SUCH DAMAGE.
> >
> >  include $(top_srcdir)/common-rules.mk
> >
> >  EXTRA_DIST = \
> >         nbdkit-python-plugin.pod \
> >         examples/file.py \
> > +       examples/error.py \
> >         examples/imageio.py \
> >         examples/ramdisk.py \
> >         examples/url.py \
> >         $(NULL)
> >
> >  if HAVE_PYTHON
> >
> >  plugin_LTLIBRARIES = nbdkit-python-plugin.la
> >
> >  nbdkit_python_plugin_la_SOURCES = \
> > diff --git a/plugins/python/examples/error.py
b/plugins/python/examples/error.py
> > new file mode 100644
> > index 00000000..0331a83e
> > --- /dev/null
> > +++ b/plugins/python/examples/error.py
> > @@ -0,0 +1,77 @@
> > +# Example Python plugin.
> > +#
> > +# This plugin simulates errors for testing NBD client error hanlding.
> > +# Every odd call will fail, and every even call will succeed, unless
> > +# there a real error accesing the specified file.
> > +#
> > +# This example can be freely used for any purpose.
> > +
> > +# Run it from the build directory like this:
> > +#
> > +#   ./nbdkit -f -v python ./plugins/python/examples/error.py
file=test.img
> > +#
> > +# Or run it after installing nbdkit like this:
> > +#
> > +#   nbdkit -f -v python ./plugins/python/examples/error.py
file=test.img
> > +#
> > +# The -f -v arguments are optional.  They cause the server to stay in
> > +# the foreground and print debugging, which is useful when testing.
> > +
> > +import os
> > +
> > +API_VERSION = 2
> > +
> > +filename = None
> > +calls = 0
> > +
> > +
> > +def config(key, value):
> > +    global filename
> > +    assert key == "file"
> > +    filename = value
> > +
> > +
> > +def open(readonly):
> > +    flags = os.O_RDONLY if readonly else os.O_RDWR
> > +    return {"fd": os.open(filename, flags)}
> > +
> > +
> > +def can_extents(h):
> > +    return True
> > +
> > +
> > +def get_size(h):
> > +    return os.stat(h["fd"]).st_size
> > +
> > +
> > +def extents(h, count, offset, flags):
> > +    global calls
> > +    calls += 1
> > +    if calls % 2:
> > +        raise RuntimeError(f"extents error offset={offset}
count={count}")
> > +
> > +    # We don't really support extents, so we report the entire
file as
> > +    # data.
> > +    return [(offset, count, 0)]
> > +
> > +
> > +def pread(h, buf, offset, flags):
> > +    global calls
> > +    calls += 1
> > +    if calls % 2:
> > +        raise RuntimeError(f"pread error offset={offset}
count={len(buf)}")
> > +
> > +    os.lseek(h["fd"], offset, os.SEEK_SET)
> > +    n = os.readv(h['fd'], [buf])
> 
> We can use os.preadv in current python, but I'm trying to keep this
plugin
> useful on rhel 8 systems (python 3.6). os.preadv added in python 3.7.
The default thread model for Python plugins is
nbdkit.THREAD_MODEL_SERIALIZE_ALL_REQUESTS, ie. you cannot have two
pread() functions racing to do lseek + readon the same file
descriptor, so this should be safe.

Reviewed-by: Richard W.M. Jones <rjones at redhat.com>

Rich.
> > +    assert n == len(buf)
> > +
> > +
> > +def pwrite(h, buf, offset, flags):
> > +    global calls
> > +    calls += 1
> > +    if calls % 2:
> > +        raise RuntimeError(f"pwrite error offset={offset}
count={len(buf)}")
> > +
> > +    os.lseek(h["fd"], offset, os.SEEK_SET)
> > +    n = os.writev(h['fd'], [buf])
> > +    assert n == len(buf)
> > --
> > 2.34.1
> >
-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

On Mon, Jan 31, 2022 at 3:10 PM Richard W.M. Jones <rjones at redhat.com>
wrote:
>
> On Mon, Jan 31, 2022 at 12:17:36AM +0200, Nir Soffer wrote:
> > On Mon, Jan 31, 2022 at 12:14 AM Nir Soffer <nsoffer at
redhat.com> wrote:
> > >
> > > This plugin simulates errors in pread, pwrite, and extents. This
is
> > > useful for testing error handling in NBD clients, and
understanding how
> > > plugin exceptions are reported to the NBD client.
> > > ---
> > >  plugins/python/Makefile.am       |  1 +
> > >  plugins/python/examples/error.py | 77
++++++++++++++++++++++++++++++++
> > >  2 files changed, 78 insertions(+)
> > >  create mode 100644 plugins/python/examples/error.py
> > >
> > > diff --git a/plugins/python/Makefile.am
b/plugins/python/Makefile.am
> > > index eecd7e89..e6e6c9e6 100644
> > > --- a/plugins/python/Makefile.am
> > > +++ b/plugins/python/Makefile.am
> > > @@ -27,20 +27,21 @@
> > >  # ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY,
> > >  # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT
> > >  # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF
> > >  # SUCH DAMAGE.
> > >
> > >  include $(top_srcdir)/common-rules.mk
> > >
> > >  EXTRA_DIST = \
> > >         nbdkit-python-plugin.pod \
> > >         examples/file.py \
> > > +       examples/error.py \
> > >         examples/imageio.py \
> > >         examples/ramdisk.py \
> > >         examples/url.py \
> > >         $(NULL)
> > >
> > >  if HAVE_PYTHON
> > >
> > >  plugin_LTLIBRARIES = nbdkit-python-plugin.la
> > >
> > >  nbdkit_python_plugin_la_SOURCES = \
> > > diff --git a/plugins/python/examples/error.py
b/plugins/python/examples/error.py
> > > new file mode 100644
> > > index 00000000..0331a83e
> > > --- /dev/null
> > > +++ b/plugins/python/examples/error.py
> > > @@ -0,0 +1,77 @@
> > > +# Example Python plugin.
> > > +#
> > > +# This plugin simulates errors for testing NBD client error
hanlding.
> > > +# Every odd call will fail, and every even call will succeed,
unless
> > > +# there a real error accesing the specified file.
> > > +#
> > > +# This example can be freely used for any purpose.
> > > +
> > > +# Run it from the build directory like this:
> > > +#
> > > +#   ./nbdkit -f -v python ./plugins/python/examples/error.py
file=test.img
> > > +#
> > > +# Or run it after installing nbdkit like this:
> > > +#
> > > +#   nbdkit -f -v python ./plugins/python/examples/error.py
file=test.img
> > > +#
> > > +# The -f -v arguments are optional.  They cause the server to
stay in
> > > +# the foreground and print debugging, which is useful when
testing.
> > > +
> > > +import os
> > > +
> > > +API_VERSION = 2
> > > +
> > > +filename = None
> > > +calls = 0
> > > +
> > > +
> > > +def config(key, value):
> > > +    global filename
> > > +    assert key == "file"
> > > +    filename = value
> > > +
> > > +
> > > +def open(readonly):
> > > +    flags = os.O_RDONLY if readonly else os.O_RDWR
> > > +    return {"fd": os.open(filename, flags)}
> > > +
> > > +
> > > +def can_extents(h):
> > > +    return True
> > > +
> > > +
> > > +def get_size(h):
> > > +    return os.stat(h["fd"]).st_size
> > > +
> > > +
> > > +def extents(h, count, offset, flags):
> > > +    global calls
> > > +    calls += 1
> > > +    if calls % 2:
> > > +        raise RuntimeError(f"extents error offset={offset}
count={count}")
> > > +
> > > +    # We don't really support extents, so we report the
entire file as
> > > +    # data.
> > > +    return [(offset, count, 0)]
> > > +
> > > +
> > > +def pread(h, buf, offset, flags):
> > > +    global calls
> > > +    calls += 1
> > > +    if calls % 2:
> > > +        raise RuntimeError(f"pread error offset={offset}
count={len(buf)}")
> > > +
> > > +    os.lseek(h["fd"], offset, os.SEEK_SET)
> > > +    n = os.readv(h['fd'], [buf])
> >
> > We can use os.preadv in current python, but I'm trying to keep
this plugin
> > useful on rhel 8 systems (python 3.6). os.preadv added in python 3.7.
>
> The default thread model for Python plugins is
> nbdkit.THREAD_MODEL_SERIALIZE_ALL_REQUESTS, ie. you cannot have two
> pread() functions racing to do lseek + readon the same file
> descriptor, so this should be safe.
>
> Reviewed-by: Richard W.M. Jones <rjones at redhat.com>
Pushed as 096a5dd07c560439ca7d2544d9df33062df2801e.