Libguestfs - Jun 2021 - nbdcpy: from scratch nbdcopy using io_uring

On Fri, Jun 25, 2021 at 02:16:51PM +0530, Abhay Raj Singh
wrote:
> > However 6 is probably too low - in nbdcopy we use 64.
> I kept it low just to verify the system works in the way I intended it to.
> 
> > So you can
> > open multiple TCP connections on each side and issue multiple
> > commands in flight on each of those connections.
> I will look into this, so we open multiple sockets hence get multiple
> socket_fds which we can read from and write to?
Yes.
> > The code looks very minimal at the moment.? I'm not very familiar
with
> > the fmt:: class.
> It's just for formatting and output like printf.
> 
> >Does it successfully make a handshake to ?nbdkit -o?
> > yet?
> I read in the protocol document
https://github.com/NetworkBlockDevice/nbd/blob/
> master/doc/proto.md#oldstyle-negotiation
> that handshake is just about the server sending data to the client on
accept,
> I open a socket and connect then read the handshake, and with the data,
> then populate NbdConnection variables. All this is done currently in the
> constructor of NbdConnection synchronously. So I guess the answer
> is yes.
[...]
> command to run nbkit server
> nbdkit data ' ( 0x55 0xAA )*2048 ' ?-o -f -p 1234
Yes this is good.

Another way to test it might be something like this:

  nbdkit -v -o random 1G --run './src/nbdcpy --source \$port --destination
\$port'

(You could use other plugins instead of random, such as data,
sparse-random, memory, etc.)

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

I ran into a problem while working on receiving data from nbd source
(Reply to NBD_CMD_READ)

As you know we need to parse the error in Reply Header before we can
proceed reading the data.
Let's say an error occurred so instead of
HEADER1_DATA1..._HEADER2_DATA2... we will
get HEADER1_HEADER2_ DATA2... (as far as I know) so submitting a recv
request to io_uring
with length = sizeof(HEADER1+DATA1) would cause problem as it won't
detect NBD packet boundaries
and will give us as many bytes we ask it (I may be wrong here that's
what I read till now).

A remedy to this would be just submit 'header reads' to io_uring when
we get a read and if header says there were no errors
we can be sure there is length bytes ready to be read in the
buffer(rest of the NBD packet) and read won't block.
Now, as far as I can tell this would work as I expect but our main
concern is avoiding copy_user_enhanced_fast_string
so this won't be nice.

Also attaching metadata (Operation) to read SQE doesn't make sense
because As far as I know io_uring won't be able to tell
the difference the read is for which io_uring request, Reply Header's
handle will tell us which operation in operations vector
does this NBD packet belong to.

Another solution would be opening multiple sockets one for each slot
in operations vector, only one NBD operation runs on a socket
i.e. only one inflight request per socket, that too sounds like a bad idea.

Thanks and regards,
Abhay