Libguestfs - Jul 2021 - nbdcpy: from scratch nbdcopy using io_uring

On Sat, Jul 10, 2021 at 03:03:19AM +0530, Abhay Raj Singh wrote:
> I managed to get the reading from the server part of nbdcpy working
> (queuing new requests after a socket gets free etc.), the writing to
> destination part is not being executed due to some sneaky bug I hope
> to find? it soon.
Cool!  I was following your git repo and watching some of the changes
you'd made.

Just to let you know I'm on holiday until Thursday, but you can keep
looking at the write bug and sending emails and I'll catch up as soon
as I get back.
> This zero-copy TCP receive API looks promising should I pursue it further
> https://lwn.net/Articles/752188/
I wonder how this interacts (if at all?) with io_uring.

I'm not committed to any particular way to make nbdcopy faster.  Any
method that makes it faster on Linux is good as far as I'm concerned.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW

I was able to resolve the issue caused by multiplexing where
HEADER1+DATA1...
was read as
HEADER1+HEADER2+HEADER3...

So nbdcpy is functional! and copies completely.

Though I have been trying to resolve another issue

The command below works fine (16MB)
nbdkit -v -o data ' ( 0x41 0x42 )*16777216 ' --run 'bin/nbdcpy $port
$port'

But when I use random
`nbdkit -v -o random size=16M --run 'bin/nbdcpy $port $port'`

I get
error: data written does not match expected

So, I assume my code is affected by data somehow which I am not able to find.

As it's functional now I will update the documentation to make it more
understandable and true to code and then ask for reviews from io_uring
people. I will also resume work on documenting how libnbd works and on
how to integrate io_uring with libnbd or nbdcopy.

Regards,
Abhay