Libguestfs - Jan 2021 - [PATCH nbdkit] python: Simulate slow extents

In some situations getting image extents can be slow. Add configuration
value to simulate slow extents. This can be useful for testing and
optimizing nbd clients.

Example usage - simulating 1 seconds delay:

qemu-img create -f raw test.img 10g

./nbdkit -f -v python ./plugins/python/examples/file.py \
    file=test.img extents_delay=1 2>&1 | grep 'python: extents'
nbdkit: python.1: debug: python: extents count=2147483136 offset=0 req_one=1
nbdkit: python.0: debug: python: extents count=2147483136 offset=2147483136
req_one=1
nbdkit: python.2: debug: python: extents count=2147483136 offset=4294966272
req_one=1
nbdkit: python.3: debug: python: extents count=2147483136 offset=6442449408
req_one=1
nbdkit: python.5: debug: python: extents count=2147483136 offset=8589932544
req_one=1
nbdkit: python.6: debug: python: extents count=2560 offset=10737415680 req_one=1
nbdkit: python.4: debug: python: extents count=2147483136 offset=0 req_one=1
nbdkit: python.9: debug: python: extents count=2147483136 offset=2147483136
req_one=1
nbdkit: python.14: debug: python: extents count=2147483136 offset=4294966272
req_one=1
nbdkit: python.11: debug: python: extents count=2147483136 offset=6442449408
req_one=1
nbdkit: python.9: debug: python: extents count=2147483136 offset=8589932544
req_one=1
nbdkit: python.9: debug: python: extents count=2560 offset=10737415680 req_one=1

In another shell:

$ time qemu-img convert -p -f raw -O raw nbd://localhost/ test.img
(100.00/100%)

real	0m18.029s
user	0m2.314s
sys	0m3.581s

Without extent delay this takes about 5.6 seconds.

Signed-off-by: Nir Soffer <nsoffer at redhat.com>
---
 plugins/python/examples/file.py | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/plugins/python/examples/file.py b/plugins/python/examples/file.py
index 8501b8eb..e607b338 100644
--- a/plugins/python/examples/file.py
+++ b/plugins/python/examples/file.py
@@ -6,6 +6,11 @@
 #
 #   ./nbdkit -f -v python ./plugins/python/examples/file.py file=test.img
 #
+# To simulate slow extents operation, specify a delay:
+#
+#   ./nbdkit -f -v python ./plugins/python/examples/file.py file=test.img \
+#       extents_delay=70
+#
 # Or run it after installing nbdkit like this:
 #
 #   nbdkit -f -v python ./plugins/python/examples/file.py file=test.img
@@ -14,6 +19,7 @@
 # the foreground and print debugging, which is useful when testing.
 
 import os
+import time
 
 import nbdkit
 
@@ -25,13 +31,18 @@ API_VERSION = 2
 # The file we want to serve.
 filename = None
 
+# Simulate slow extents.
+extents_delay = 0.0
+
 
 # Parse the file parameter which contains the name of the file that we
 # want to serve.
 def config(key, value):
-    global filename
+    global filename, extents_delay
     if key == "file":
         filename = os.path.abspath(value)
+    elif key == "extents_delay":
+        extents_delay = float(value)
     else:
         raise RuntimeError("unknown parameter: " + key)
 
@@ -64,6 +75,15 @@ def get_size(h):
     return sb.st_size
 
 
+def can_extents(h):
+    return True
+
+
+def extents(h, count, offset, flags):
+    time.sleep(extents_delay)
+    yield (offset, count, 0)
+
+
 def pread(h, buf, offset, flags):
     n = os.preadv(h['fd'], [buf], offset)
     if n != len(buf):
-- 
2.26.2

On Wed, Jan 20, 2021 at 09:12:58PM +0200, Nir Soffer
wrote:
> In some situations getting image extents can be slow. Add configuration
> value to simulate slow extents. This can be useful for testing and
> optimizing nbd clients.
> 
> Example usage - simulating 1 seconds delay:
> 
> qemu-img create -f raw test.img 10g
> 
> ./nbdkit -f -v python ./plugins/python/examples/file.py \
>     file=test.img extents_delay=1 2>&1 | grep 'python:
extents'
Sorry, I meant to send you a follow-up email about using the delay
filter for this testing, and didn't get around to it.  It is possible
to do this already by putting the delay filter over and controlling
the extents delay.  Something like:

  ./nbdkit -f -v --filter=delay \
      python ./plugins/python/examples/file.py \
      file=test.img delay-extents=1

---

What is the purpose of yield() here?
> +def extents(h, count, offset, flags):
> +    time.sleep(extents_delay)
> +    yield (offset, count, 0)
Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW