Libguestfs - Jun 2021 - nbdcpy: from scratch nbdcopy using io_uring

On Thu, Jun 24, 2021 at 07:16:10PM +0100, Richard W.M. Jones
wrote:
> On Thu, Jun 24, 2021 at 09:30:00PM +0530, Abhay Raj Singh wrote:
> > 
> > In a previous email, Mr. Jones suggested, writing a test
implementation of
> > nbdcopy from scratch. First of all, thank you, Mr. Jones, as I got a
better
> > perspective of the problem and the solution.
> > 
> > I have almost written the bare-bones implementation for handling
NBD-related
> > stuff on which I will base the io_uring core piece by piece. It is
written in
> > C++, as I am more comfortable with it. But, made it easily
translatable to C.
> > 
> > I have been working on a solution and have documented the current
approach, it
> > would be great if you have a look at it and let me know what you
think.
> > 
> > Approach document:
> > https://gitlab.com/rathod-sahaab/nbdcpy/-/blob/dev/docs/SOLUTION.md
> 
> So the document is fine.  Since this is throw-away code I wouldn't
> worry about handling errors except to abort the program.
> 
> You can issue the read requests out of order, although perhaps it
> isn't helpful.  And you're correct that you should try to limit the
> number of requests in flight (otherwise you'll run out of memory when
> you try to copy very large disks).  However 6 is probably too low - in
> nbdcopy we use 64.
> 
> NBD supports multi-conn (well, technically speaking you should check a
> flag on the server but that doesn't matter for testing).  So you can
> open multiple TCP connections on each side, and issue multiple
> commands in flight on each of those connections.  We found this
> improves performance a lot.
> 
> The code looks very minimal at the moment.  I'm not very familiar with
> the fmt:: class.  Does it successfully make a handshake to ?nbdkit -o?
> yet?
Another thing is that real nbdcopy handshakes with the servers
synchronously, and only does the data transfer part asynchronously.
This simplifies the current code quite a bit.
(https://gitlab.com/nbdkit/libnbd/-/blob/master/copy/nbd-ops.c)
> I notice that although you create an io_uring you're not using it.
> But I guess that's still to come.
> 
> Rich.
> 
> -- 
> Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> Fedora Windows cross-compiler. Compile Windows programs, test, and
> build Windows installers. Over 100 libraries supported.
> http://fedoraproject.org/wiki/MinGW
> 
> _______________________________________________
> Libguestfs mailing list
> Libguestfs at redhat.com
> https://listman.redhat.com/mailman/listinfo/libguestfs
-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

> However 6 is probably too low - in nbdcopy we use 64.
I kept it low just to verify the system works in the way I intended it to.
> So you can
> open multiple TCP connections on each side and issue multiple
> commands in flight on each of those connections.
I will look into this, so we open multiple sockets hence get multiple
socket_fds which we can read from and write to?
> The code looks very minimal at the moment.  I'm not very familiar with
> the fmt:: class.
It's just for formatting and output like printf.
>Does it successfully make a handshake to ?nbdkit -o?
> yet?
I read in the protocol document
https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md#oldstyle-negotiation
that handshake is just about the server sending data to the client on
accept,
I open a socket and connect then read the handshake, and with the data,
then populate NbdConnection variables. All this is done currently in the
constructor of NbdConnection synchronously. So I guess the answer
is yes.

Current data I receive after handshake from source and destination ports
Copying from port 1234 to port 2345.
????????????????????????????????
?            source            ?
????????????????????????????????
NBDMAGIC (0x4e42444d41474943)
0x420281861253
4096
0b0000000000000000
0b0000110101101101
????????????????????????????????
?         destination          ?
????????????????????????????????
NBDMAGIC (0x4e42444d41474943)
0x420281861253
8192
0b0000000000000000
0b0000110101101101

and then the program ends, destructors call disconnecting code.

command to run nbkit server
nbdkit data ' ( 0x55 0xAA )*2048 '  -o -f -p 1234
> I notice that although you create an io_uring you're not using it.
> But I guess that's still to come.
I am making it now done with the NBD read request submission
should be ready soon.

Thanks and regards,
Abhay
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://listman.redhat.com/archives/libguestfs/attachments/20210625/a3a59bef/attachment.htm>