Libguestfs - Aug 2022 - Benchmarks of asynch nbdublk (was: Re: [PATCH libnbd] ublk: Add new nbdublk program)

FYI here is the asynch version:

https://gitlab.com/rwmjones/libnbd/-/commits/nbdublk/

I kept it as two commits so you can see the differences but obviously
it would be folded into one commit upstream.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit

I have an interesting benchmark to share.  I wanted to find out the
answers to a couple of questions: (1) Is nbdublk stable, and
(2) Is the performance terrible / good / excellent?  I only wanted a
rough impression for these, we can work on optimising it more later.

Since I am developing nbdublk (the client) in a disposable Rawhide VM,
and because I already need to recompile the kernel in that VM to
enable ublk, I have the Linux kernel tree from git around, so I
decided to compare regular VM disk:

  +------------+
  | Rawhide VM |
  |            |
  | compile    |
  | Linux on   |
  | VM disk    |
  +------------+
     |
  +--|--------------------------+
  |  |                   Host   |
  |  V                          |
  | raw format disk             |
  +-----------------------------+

versus nbdublk + ublk:

  +------------+
  | Rawhide VM |
  |            |
  | compile    |
  | Linux on   |
  | /dev/ublkb0|
  | nbdublk    |
  +------------+
     | NBD connection
  +--|--------------------------+
  |  |                   Host   |
  |  V                          |
  | qemu-nbd + qcow2 disk       |
  +-----------------------------+

This is with qemu-7.0.0 and the same version of qemu-nbd.  I did
drop_caches == 3 before both tests.  The .config was identical in both
builds.  Both filesystems use XFS.

With the regular VM disk:

  real 21m12.563s
  user 136m25.367s
  sys  24m11.165s

With nbdublk disk:

  real 26m3.990s
  user 137m16.734s
  sys  24m37.143s

So it's interesting that although nbdublk is a bit slower, it's not
terrible.  In fact this is much better performance than I expected for
a first pass.  (I was honestly expecting it'd be like half the speed).
I'm pretty happy with this performance for now.

Also the compile completed without any errors both times, indicating
that at least for compiling, nbdublk is stable.

However while I was doing these tests I did notice that nbdublk isn't
stable for some operations.  In particular 'git clone' cloning the
Linux git tree from the VM disk to /dev/ublkb0 would sometimes
deadlock or have other problems.  I'm not entirely sure why that is,
given that I didn't get any problems while compiling.  I guess that
git uses the filesystem very aggressively compared to just compiling.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html