Libguestfs - Jun 2021 - nbdcpy: from scratch nbdcopy using io_uring

> However 6 is probably too low - in nbdcopy we use 64.
I kept it low just to verify the system works in the way I intended it to.
> So you can
> open multiple TCP connections on each side and issue multiple
> commands in flight on each of those connections.
I will look into this, so we open multiple sockets hence get multiple
socket_fds which we can read from and write to?
> The code looks very minimal at the moment.  I'm not very familiar with
> the fmt:: class.
It's just for formatting and output like printf.
>Does it successfully make a handshake to ?nbdkit -o?
> yet?
I read in the protocol document
https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md#oldstyle-negotiation
that handshake is just about the server sending data to the client on
accept,
I open a socket and connect then read the handshake, and with the data,
then populate NbdConnection variables. All this is done currently in the
constructor of NbdConnection synchronously. So I guess the answer
is yes.

Current data I receive after handshake from source and destination ports
Copying from port 1234 to port 2345.
????????????????????????????????
?            source            ?
????????????????????????????????
NBDMAGIC (0x4e42444d41474943)
0x420281861253
4096
0b0000000000000000
0b0000110101101101
????????????????????????????????
?         destination          ?
????????????????????????????????
NBDMAGIC (0x4e42444d41474943)
0x420281861253
8192
0b0000000000000000
0b0000110101101101

and then the program ends, destructors call disconnecting code.

command to run nbkit server
nbdkit data ' ( 0x55 0xAA )*2048 '  -o -f -p 1234
> I notice that although you create an io_uring you're not using it.
> But I guess that's still to come.
I am making it now done with the NBD read request submission
should be ready soon.

Thanks and regards,
Abhay
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://listman.redhat.com/archives/libguestfs/attachments/20210625/a3a59bef/attachment.htm>

On Fri, Jun 25, 2021 at 02:16:51PM +0530, Abhay Raj Singh
wrote:
> > However 6 is probably too low - in nbdcopy we use 64.
> I kept it low just to verify the system works in the way I intended it to.
> 
> > So you can
> > open multiple TCP connections on each side and issue multiple
> > commands in flight on each of those connections.
> I will look into this, so we open multiple sockets hence get multiple
> socket_fds which we can read from and write to?
Yes.
> > The code looks very minimal at the moment.? I'm not very familiar
with
> > the fmt:: class.
> It's just for formatting and output like printf.
> 
> >Does it successfully make a handshake to ?nbdkit -o?
> > yet?
> I read in the protocol document
https://github.com/NetworkBlockDevice/nbd/blob/
> master/doc/proto.md#oldstyle-negotiation
> that handshake is just about the server sending data to the client on
accept,
> I open a socket and connect then read the handshake, and with the data,
> then populate NbdConnection variables. All this is done currently in the
> constructor of NbdConnection synchronously. So I guess the answer
> is yes.
[...]
> command to run nbkit server
> nbdkit data ' ( 0x55 0xAA )*2048 ' ?-o -f -p 1234
Yes this is good.

Another way to test it might be something like this:

  nbdkit -v -o random 1G --run './src/nbdcpy --source \$port --destination
\$port'

(You could use other plugins instead of random, such as data,
sparse-random, memory, etc.)

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

Sent the message by reply instead of reply all so wasn't delivered to the
list sending it again, sorry ^-^


------
I managed to get the reading from the server part of nbdcpy working
(queuing new requests after a socket gets free etc.), the writing to
destination part is not being executed due to some sneaky bug I hope to
find  it soon.

This zero-copy TCP receive API looks promising should I pursue it further
https://lwn.net/Articles/752188/

Thanks and regards,
Abhay
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://listman.redhat.com/archives/libguestfs/attachments/20210710/ad80387d/attachment.htm>