Michael S. Tsirkin
2023-Jan-22 08:34 UTC
[PATCH 0/2] vhost: improve livepatch switching for heavily loaded vhost worker kthreads
On Fri, Jan 20, 2023 at 04:12:20PM -0600, Seth Forshee (DigitalOcean) wrote:> We've fairly regularaly seen liveptches which cannot transition within kpatch's > timeout period due to busy vhost worker kthreads. In looking for a solution the > only answer I found was to call klp_update_patch_state() from a safe location. > I tried adding this call to vhost_worker(), and it works, but this creates the > potential for problems if a livepatch attempted to patch vhost_worker(). > Without a call to klp_update_patch_state() fully loaded vhost kthreads can > never switch because vhost_worker() will always appear on the stack, but with > the call these kthreads can switch but will still be running the old version of > vhost_worker(). > > To avoid this situation I've added a new function, klp_switch_current(), which > switches the current task only if its stack does not include any function being > patched. This allows kthreads to safely attempt switching themselves if a patch > is pending. There is at least one downside, however. Since there's no way for > the kthread to track whether it has already tried to switch for a pending patch > it can end up calling klp_switch_current() repeatedly when it can never be > safely switched. > > I don't know whether this is the right solution, and I'm happy to try out other > suggestions. But in my testing these patches proved effective in consistently > switching heavily loaded vhost kthreads almost immediately. > > To: Josh Poimboeuf <jpoimboe at kernel.org> > To: Jiri Kosina <jikos at kernel.org> > To: Miroslav Benes <mbenes at suse.cz> > To: Petr Mladek <pmladek at suse.com> > To: Joe Lawrence <joe.lawrence at redhat.com> > To: "Michael S. Tsirkin" <mst at redhat.com> > To: Jason Wang <jasowang at redhat.com> > Cc: live-patching at vger.kernel.org > Cc: linux-kernel at vger.kernel.org > Cc: kvm at vger.kernel.org > Cc: virtualization at lists.linux-foundation.org > Cc: netdev at vger.kernel.org > Signed-off-by: Seth Forshee (DigitalOcean) <sforshee at kernel.org>Don't know enough about live patching to judge this. I'll let livepatch maintainers judge this, and merge through the livepatch tree if appropriate. For that: Acked-by: Michael S. Tsirkin <mst at redhat.com> but pls underestand this is more a 'looks ok superficially and I don't have better ideas' than 'I have reviewed this thoroughly'.> > --- > Seth Forshee (DigitalOcean) (2): > livepatch: add an interface for safely switching kthreads > vhost: check for pending livepatches from vhost worker kthreads > > drivers/vhost/vhost.c | 4 ++++ > include/linux/livepatch.h | 2 ++ > kernel/livepatch/transition.c | 11 +++++++++++ > 3 files changed, 17 insertions(+) > --- > base-commit: 5dc4c995db9eb45f6373a956eb1f69460e69e6d4 > change-id: 20230120-vhost-klp-switching-ba9a3ae38b8a > > Best regards, > -- > Seth Forshee (DigitalOcean) <sforshee at kernel.org>
Maybe Matching Threads
- [PATCH 2/2] vhost: check for pending livepatches from vhost worker kthreads
- [PATCH 0/2] vhost: improve livepatch switching for heavily loaded vhost worker kthreads
- [PATCH 0/2] vhost: improve livepatch switching for heavily loaded vhost worker kthreads
- [PATCH 0/2] vhost: improve livepatch switching for heavily loaded vhost worker kthreads
- Livepatch: Linux kernel updates without rebooting