On Tue, May 31, 2022 at 09:13:38AM +0200, Eugenio Perez Martin
wrote:> On Tue, May 31, 2022 at 7:42 AM Michael S. Tsirkin <mst at
redhat.com> wrote:
> >
> > On Thu, May 26, 2022 at 02:43:34PM +0200, Eugenio P?rez wrote:
> > > Implement stop operation for vdpa_sim devices, so vhost-vdpa will
offer
> > > that backend feature and userspace can effectively stop the
device.
> > >
> > > This is a must before get virtqueue indexes (base) for live
migration,
> > > since the device could modify them after userland gets them.
There are
> > > individual ways to perform that action for some devices
> > > (VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there
was no
> > > way to perform it for any vhost device (and, in particular,
vhost-vdpa).
> > >
> > > After the return of ioctl with stop != 0, the device MUST finish
any
> > > pending operations like in flight requests. It must also preserve
all
> > > the necessary state (the virtqueue vring base plus the possible
device
> > > specific states) that is required for restoring in the future.
The
> > > device must not change its configuration after that point.
> > >
> > > After the return of ioctl with stop == 0, the device can continue
> > > processing buffers as long as typical conditions are met (vq is
enabled,
> > > DRIVER_OK status bit is enabled, etc).
> > >
> > > In the future, we will provide features similar to
VHOST_USER_GET_INFLIGHT_FD
> > > so the device can save pending operations.
> > >
> > > Comments are welcome.
> >
> >
> > So given this is just for simulator and affects UAPI I think it's
fine
> > to make it wait for the next merge window, until there's a
consensus.
> > Right?
> >
>
> While the change is only implemented in the simulator at this moment,
> it's just the very last missing piece in the kernel to implement
> complete live migration for net devices with cvq :). All vendor
> drivers can implement this call with current code, just a little bit
> of plumbing is needed. And it was accepted in previous meetings.
>
> If it proves it works for every configuration (nested, etc), the
> implementation can forward the call to the admin vq for example. At
> the moment, it follows the proposed stop status bit sematic to stop
> the device, which POC has been tested in these circumstances.
>
> Thanks!
Oh absolutely, but I am guessing this plumbing won't
be ready for this merge window.
> > > v4:
> > > * Replace VHOST_STOP to VHOST_VDPA_STOP in vhost ioctl switch
case too.
> > >
> > > v3:
> > > * s/VHOST_STOP/VHOST_VDPA_STOP/
> > > * Add documentation and requirements of the ioctl above its
definition.
> > >
> > > v2:
> > > * Replace raw _F_STOP with BIT_ULL(_F_STOP).
> > > * Fix obtaining of stop ioctl arg (it was not obtained but
written).
> > > * Add stop to vdpa_sim_blk.
> > >
> > > Eugenio P?rez (4):
> > > vdpa: Add stop operation
> > > vhost-vdpa: introduce STOP backend feature bit
> > > vhost-vdpa: uAPI to stop the device
> > > vdpa_sim: Implement stop vdpa op
> > >
> > > drivers/vdpa/vdpa_sim/vdpa_sim.c | 21 +++++++++++++++++
> > > drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 +
> > > drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 3 +++
> > > drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 3 +++
> > > drivers/vhost/vdpa.c | 34
+++++++++++++++++++++++++++-
> > > include/linux/vdpa.h | 6 +++++
> > > include/uapi/linux/vhost.h | 14 ++++++++++++
> > > include/uapi/linux/vhost_types.h | 2 ++
> > > 8 files changed, 83 insertions(+), 1 deletion(-)
> > >
> > > --
> > > 2.31.1
> > >
> >