Libguestfs - Aug 2021 - nbdcpy: from scratch nbdcopy using io_uring

On Sat, Aug 07, 2021 at 09:53:10PM +0530, Abhay Raj Singh
wrote:
> Fixed the issue, I needed to learn to use Wireshark so took some time.
> Turns out I was not updating the offset for an 'operation' /
operation
> slot in a rare case.
> 
> As an operation owns and NBD handle (the number) say 7
> 
> Initially, it read from offset 512 then 2048 then 4096, etc.
> But was always writing to offset 512 (in rare cases) because I didn't
> update the offset field of operation slot.
> 
> There's a bug where it gets stuck after reading 300KB or so, should be
> done by tomorrow.
OK, let me know how it goes :-)

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html

I had an idea for optimizing my current approach, it's good in some
ways but can be faster with some breaking changes to the protocol.

Currently, we read (from socket connected to source) one request at a time
the simple flow looks like `read_header(io_uring) ---- success --->
recv(data) --- success ---> send(data) & queue another read header`
but it's not as efficient as it could be at best it's a hack.

Another approach I am thinking about is a large buffer
where we can read all of the socket's data and process packets from
that buffer as all the I/O is handled.
this minimizes the number of read requests to the kernel as we do 1
read for multiple NBD packets.

Further optimization requires changing the NBD protocol a bit
Current protocol
1. Memory representation of a response (20-byte header + data)
2. Memory representation of a request (28-byte header + data)

HHHHH_DDDDDDDDD...
HHHHHHH_DDDDDDDDD...

H and D represent 4 bytes, _ represents 0 bytes

With the large buffer approach, we read data into a large buffer, then
copy the NBD packet's data to a new buffer, strap a new header to it
and send it.
This copying is what we wanted to avoid in the first place.

If the response header was 28 bytes or the first 8-bytes of data were
useless we could have just overwritten the header part and sent data
directly from the large buffer, therefore avoiding the copy.

What are your thoughts?

Thanks and Regards.
Abhay