Libguestfs - Aug 2021 - nbdcpy: from scratch nbdcopy using io_uring

I was able to resolve the issue caused by multiplexing where
HEADER1+DATA1...
was read as
HEADER1+HEADER2+HEADER3...

So nbdcpy is functional! and copies completely.

Though I have been trying to resolve another issue

The command below works fine (16MB)
nbdkit -v -o data ' ( 0x41 0x42 )*16777216 ' --run 'bin/nbdcpy $port
$port'

But when I use random
`nbdkit -v -o random size=16M --run 'bin/nbdcpy $port $port'`

I get
error: data written does not match expected

So, I assume my code is affected by data somehow which I am not able to find.

As it's functional now I will update the documentation to make it more
understandable and true to code and then ask for reviews from io_uring
people. I will also resume work on documenting how libnbd works and on
how to integrate io_uring with libnbd or nbdcopy.

Regards,
Abhay

On Sat, Jul 31, 2021 at 06:12:16PM +0530, Abhay Raj Singh
wrote:
> I was able to resolve the issue caused by multiplexing where
> HEADER1+DATA1...
> was read as
> HEADER1+HEADER2+HEADER3...
> 
> So nbdcpy is functional! and copies completely.
> 
> Though I have been trying to resolve another issue
> 
> The command below works fine (16MB)
> nbdkit -v -o data ' ( 0x41 0x42 )*16777216 ' --run 'bin/nbdcpy
$port $port'
> 
> But when I use random
> `nbdkit -v -o random size=16M --run 'bin/nbdcpy $port $port'`
> 
> I get
> error: data written does not match expected
This is a problem :-(

It comes from here and indicates that the data you're writing doesn't
exactly match what was originally read:

https://gitlab.com/nbdkit/nbdkit/-/blob/de8dcd3a34a38b088a0f9a6f8ca754702ad1f598/plugins/random/random.c#L180

I would suggest starting with a very small disk and printing out
everything that you read and write.  Should help to track down the
problem.
> So, I assume my code is affected by data somehow which I am not able to
find.
> 
> As it's functional now I will update the documentation to make it more
> understandable and true to code and then ask for reviews from io_uring
> people. I will also resume work on documenting how libnbd works and on
> how to integrate io_uring with libnbd or nbdcopy.
Thanks,

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW

Fixed the issue, I needed to learn to use Wireshark so took some time.
Turns out I was not updating the offset for an 'operation' / operation
slot in a rare case.

As an operation owns and NBD handle (the number) say 7

Initially, it read from offset 512 then 2048 then 4096, etc.
But was always writing to offset 512 (in rare cases) because I didn't
update the offset field of operation slot.

There's a bug where it gets stuck after reading 300KB or so, should be
done by tomorrow.

Thanks & Regards,
Abhay