Libguestfs - Mar 2021 - [libnbd] GSoC21: IO_uring optimization questions

Greetings everyone! I am Abhay a 3rd-year student at the
National Institute of Technology, Hamirpur, Bharat(India).

I am interested in utilizing the recently introduced feature
io_uring of the Linux kernel to speed up libnbd async I/O.

I have been in contact with Richard regarding this and
thanks to his guidance I have some ideas and approaches.

The rough approaches I can think of are:
1. Replacing the poll mechanism(nbd_poll) with the io_uring one.
2. A socket wrapper using io_uring

The latter requires less work as far as I can see and is easily
reusable. Though I have only recently read about the socket
wrapper method so can't tell for sure. I will try making a
wrapper to check feasibility.

A general suggestion for io_uring is to not use it, directly:)
But use a convenient wrapper liburing instead.

The doubts I have regarding this are:
1.
Should I create a new API or replace the old aio API.
On one end I see no problem in doing so but I am
not familiar with uses
2.
For testing wouldn't libnbd limited by nbdkit if try
to test on my pc and would be only marginally better
unless nbdkit is ported too.

As the task is not too difficult once I get my head around
the code base, I might be able to port nbdkit too(incase of socket wrapper)
and write some tests in the 180-200hr period of GSoC.

Thanks!

--
Humble Regards,
Abhay
(rathod-sahaab)

[Adding Stefan: Have we retro-fitted io_uring into any other libraries?]

Welcome Abhay,

On Tue, Mar 30, 2021 at 09:33:17PM +0530, Abhay Raj Singh
wrote:
> Greetings everyone! I am Abhay a 3rd-year student at the
> National Institute of Technology, Hamirpur, Bharat(India).
> 
> I am interested in utilizing the recently introduced feature
> io_uring of the Linux kernel to speed up libnbd async I/O.
> 
> I have been in contact with Richard regarding this and
> thanks to his guidance I have some ideas and approaches.
> 
> The rough approaches I can think of are:
> 1. Replacing the poll mechanism(nbd_poll) with the io_uring one.
This would be nice and self-contained.
> 2. A socket wrapper using io_uring
> 
> The latter requires less work as far as I can see and is easily
> reusable. Though I have only recently read about the socket
> wrapper method so can't tell for sure. I will try making a
> wrapper to check feasibility.
Does "socket wrapper" mean adding another struct socket_ops defined in
lib/internal.h and implemented so far in lib/socket.c (for plain
sockets) and lib/crypto.c (for TLS sockets)?

As an aside, libnbd only makes socket calls in a few places:

$ git grep -- '->recv'
generator/states-reply.c:  r = h->sock->ops->recv (h, h->sock,
h->rbuf, h->rlen);
generator/states-reply.c:    /* sock->ops->recv called set_error already.
*/
generator/states.c:  r = h->sock->ops->recv (h, h->sock, rbuf,
rlen);
generator/states.c:    /* sock->ops->recv called set_error already. */

$ git grep -- '->send'
generator/states.c:  r = h->sock->ops->send (h, h->sock, h->wbuf,
h->wlen, h->wflags);
generator/states.c:    /* sock->ops->send called set_error already. */

(Also you'll want to look for '->pending', '->get_fd',
'->shut_writes', '->close').
> A general suggestion for io_uring is to not use it, directly:)
> But use a convenient wrapper liburing instead.
> 
> The doubts I have regarding this are:
> 1.
> Should I create a new API or replace the old aio API.
> On one end I see no problem in doing so but I am
> not familiar with uses
If there's a new API then that's going to require every existing user
to opt in, which isn't nice for those users.  They would need to
choose io_uring (and risk not working on many platforms) or try to
support both APIs.  So the bar to doing this way must be very high -
only if it's not possible any other way.
> 2.
> For testing wouldn't libnbd limited by nbdkit if try
> to test on my pc and would be only marginally better
> unless nbdkit is ported too.
You could also test against qemu-nbd, although it is also not using
io_uring AFAIK.

Also you could test against a simple hand-written NBD server
implemented using io_uring.

Or maybe there will be enough of a performance difference against
unmodified nbdkit.
> As the task is not too difficult once I get my head around
> the code base, I might be able to port nbdkit too(incase of socket wrapper)
> and write some tests in the 180-200hr period of GSoC.
> 
> Thanks!
> Humble Regards,
> Abhay
> (rathod-sahaab)
Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org