Libguestfs - May 2021 - [PATCH libnbd 3/3] copy: Update nbdcopy(1) about multiple connections

Fix the test using qemu-nbd to allow multiple connections and update the
man page about the new semantics.

Unfortunately we have no way to tell if a NBD server supports multiple
connetions, so our default of using 4 connetions is fragile when using
remote qemu-nbd. I think the default should change to use one connection
so it always works.

Nir Soffer (3):
  copy: copy-file-to-qcow2.sh: Enable multiple connections
  Revert "Revert "copy: Always allow multiple connections""
  copy: Update nbdcopy(1) about multiple connections

 copy/copy-file-to-qcow2.sh |  2 +-
 copy/main.c                | 25 ++++++++++++++++---------
 copy/nbdcopy.pod           | 25 +++++++------------------
 3 files changed, 24 insertions(+), 28 deletions(-)

-- 
2.26.3

With --shared=4 qemu-nbd allows 4 concurrent connections. Without this
only the first connection succeed and the rest will block forever.

Signed-off-by: Nir Soffer <nsoffer at redhat.com>
---
 copy/copy-file-to-qcow2.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/copy/copy-file-to-qcow2.sh b/copy/copy-file-to-qcow2.sh
index 57c18b3..ae41ed0 100755
--- a/copy/copy-file-to-qcow2.sh
+++ b/copy/copy-file-to-qcow2.sh
@@ -54,7 +54,7 @@ qemu-img create -f qcow2 $qcow2 $size
 
 # Run qemu-nbd as a separate process so that we can copy to and from
 # the single process in two separate operations.
-qemu-nbd -f qcow2 -t --socket=$sock --pid-file=$pidfile $qcow2 &
+qemu-nbd -f qcow2 -t --shared=4 --socket=$sock --pid-file=$pidfile $qcow2 &
 cleanup_fn kill $!
 
 # Wait for qemu-nbd to start up.
-- 
2.26.3

Enable again multiple connections. When using qemu-nbd users need to
use:

    qemu-nbd --shared=4 ...

Otherwise nbdcopy will deadlock trying to open 4 connections to
qemu-nbd.

This reverts commit fb42056929e9177e0de2603e2f430a8111ae7a5f.

Signed-off-by: Nir Soffer <nsoffer at redhat.com>
---
 copy/main.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/copy/main.c b/copy/main.c
index b9dbe1d..70a4e11 100644
--- a/copy/main.c
+++ b/copy/main.c
@@ -321,10 +321,6 @@ main (int argc, char *argv[])
     exit (EXIT_FAILURE);
   }
 
-  /* If multi-conn is not supported, force connections to 1. */
-  if (! src->ops->can_multi_conn (src) || !
dst->ops->can_multi_conn (dst))
-    connections = 1;
-
   /* Calculate the number of threads from the number of connections. */
   if (threads == 0) {
     long t;
@@ -379,15 +375,26 @@ main (int argc, char *argv[])
              synchronous ? "true" : "false");
   }
 
-  /* If multi-conn is enabled on either side, then at this point we
+  /* If using multiple connections, then at this point we
    * need to ask the backend to open the extra connections.
+   *
+   * Using multiple connections when the server does not report
+   * can_multi_conn is not safe in general, but is safe for the special
+   * access pattern in nbdcopy.
+   *
+   * - We split the image to 128m segments, and every segment is written
+   *   by single worker.
+   *
+   * - Writes do not overlap, and are never partial block that another
+   *   worker may modify at the same time.
+   *
+   * - We never read from the destination so we don't have caching
+   *   synchronization issue between clients.
    */
   if (connections > 1) {
     assert (threads == connections);
-    if (src->ops->can_multi_conn (src))
-      src->ops->start_multi_conn (src);
-    if (dst->ops->can_multi_conn (dst))
-      dst->ops->start_multi_conn (dst);
+    src->ops->start_multi_conn (src);
+    dst->ops->start_multi_conn (dst);
   }
 
   /* If the source is NBD and we couldn't negotiate meta
-- 
2.26.3

Using multiple connections does not depend now on NBD multi-conn flag.

Add a note about using --shared=4 in qemu-nbd to allow multiple
connections.

Signed-off-by: Nir Soffer <nsoffer at redhat.com>
---
 copy/nbdcopy.pod | 25 +++++++------------------
 1 file changed, 7 insertions(+), 18 deletions(-)

diff --git a/copy/nbdcopy.pod b/copy/nbdcopy.pod
index f710093..781bb10 100644
--- a/copy/nbdcopy.pod
+++ b/copy/nbdcopy.pod
@@ -103,10 +103,9 @@ output on the destination.
 
 =item B<--connections=>N
 
-Set the maximum number of NBD connections ("multi-conn").  By default
-nbdcopy will try to use multi-conn with up to 4 connections if the NBD
-server supports it.  If copying between NBD servers then nbdcopy
-cannot use multi-conn if either of the servers does not support it.
+Set the maximum number of NBD connections.  By default nbdcopy will try
+to use multiple connections with up to 4 connections. Some servers
+like L<qemu-nbd(8)> support only one connection by default.
 
 =item B<--destination-is-zero>
 
@@ -216,23 +215,13 @@ pipe, or a socket, or if you use the
I<--synchronous> option, then
 nbdcopy works in synchronous mode with no parallelism, and nothing
 else in this section applies.
 
-The I<--connections=N> option controls NBD multi-conn (see
-L<libnbd(3)/Multi-conn>), opening up to N connections to the NBD
+The I<--connections=N> option open up to N connections to the NBD
 server (or to both NBD servers if copying between NBD servers).  This
-defaults to 4.  The NBD servers must support and advertise multi-conn.
-For L<nbdkit(1)> availability of multi-conn can depend on the plugin.
-You can use L<nbdinfo(1)> to find out if a particular NBD server is
-advertising multi-conn.  If the NBD server doesn?t advertise
-multi-conn then only one connection will be opened regardless of the
-I<--connections> flag.
-
-When copying between two NBD servers, the number of connections is
-limited to the minimum multi-conn supported on both sides.  For the
-purposes of this calculation, you can consider local files and block
-devices as supporting infinite multi-conn.
+defaults to 4. When using L<qemu-nbd(8)> you need to enable multiple
+conections using the I<--shared=4> command line option.
 
 When you run an NBD server as a subprocess (using the S<C<[ ... ]>>
-syntax) multi-conn cannot be used.
+syntax) multiple connections cannot be used.
 
 The I<--threads=N> option allows nbdcopy to start up to N threads
 (defaulting to the number of cores).  However nbdcopy cannot use more
-- 
2.26.3