Raghavendra K T
2013-Aug-06 18:21 UTC
[PATCH V12 0/5] Paravirtualized ticket spinlocks for KVM host
This series forms the kvm host part of paravirtual spinlock based against kvm tree. Please refer https://lkml.org/lkml/2013/8/6/178 for kvm guest part of the series. Please note that: kvm uapi: Add KICK_CPU and PV_UNHALT definition to uapi is a common patch for both guest and host. Srivatsa Vaddagiri (1): kvm hypervisor : Add a hypercall to KVM hypervisor to support pv-ticketlocks Raghavendra K T (4): kvm uapi: Add KICK_CPU and PV_UNHALT definition to uapi kvm : Fold pv_unhalt flag into GET_MP_STATE ioctl to aid migration kvm hypervisor: Simplify kvm_for_each_vcpu with kvm_irq_delivery_to_apic Documentation/kvm : Add documentation on Hypercalls and features used for PV spinlock --- Documentation/virtual/kvm/cpuid.txt | 4 ++++ Documentation/virtual/kvm/hypercalls.txt | 14 ++++++++++++++ arch/x86/include/asm/kvm_host.h | 5 +++++ arch/x86/include/uapi/asm/kvm_para.h | 1 + arch/x86/kvm/cpuid.c | 3 ++- arch/x86/kvm/lapic.c | 5 ++++- arch/x86/kvm/x86.c | 31 ++++++++++++++++++++++++++++++- include/uapi/linux/kvm_para.h | 1 + 8 files changed, 61 insertions(+), 3 deletions(-)
Raghavendra K T
2013-Aug-06 18:22 UTC
[PATCH V12 1/5] kvm uapi: Add KICK_CPU and PV_UNHALT definition to uapi
kvm uapi: Add KICK_CPU and PV_UNHALT definition to uapi From: Raghavendra K T <raghavendra.kt at linux.vnet.ibm.com> this is needed by both guest and host. Originally-from: Srivatsa Vaddagiri <vatsa at linux.vnet.ibm.com> Signed-off-by: Raghavendra K T <raghavendra.kt at linux.vnet.ibm.com> Acked-by: Gleb Natapov <gleb at redhat.com> Acked-by: Ingo Molnar <mingo at kernel.org> --- arch/x86/include/uapi/asm/kvm_para.h | 1 + include/uapi/linux/kvm_para.h | 1 + 2 files changed, 2 insertions(+) diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h index 06fdbd9..94dc8ca 100644 --- a/arch/x86/include/uapi/asm/kvm_para.h +++ b/arch/x86/include/uapi/asm/kvm_para.h @@ -23,6 +23,7 @@ #define KVM_FEATURE_ASYNC_PF 4 #define KVM_FEATURE_STEAL_TIME 5 #define KVM_FEATURE_PV_EOI 6 +#define KVM_FEATURE_PV_UNHALT 7 /* The last 8 bits are used to indicate how to interpret the flags field * in pvclock structure. If no bits are set, all flags are ignored. diff --git a/include/uapi/linux/kvm_para.h b/include/uapi/linux/kvm_para.h index cea2c5c..2841f86 100644 --- a/include/uapi/linux/kvm_para.h +++ b/include/uapi/linux/kvm_para.h @@ -19,6 +19,7 @@ #define KVM_HC_MMU_OP 2 #define KVM_HC_FEATURES 3 #define KVM_HC_PPC_MAP_MAGIC_PAGE 4 +#define KVM_HC_KICK_CPU 5 /* * hypercalls use architecture specific
Raghavendra K T
2013-Aug-06 18:22 UTC
[PATCH V12 2/5] kvm hypervisor : Add a hypercall to KVM hypervisor to support pv-ticketlocks
kvm hypervisor : Add a hypercall to KVM hypervisor to support pv-ticketlocks From: Srivatsa Vaddagiri <vatsa at linux.vnet.ibm.com> kvm_hc_kick_cpu allows the calling vcpu to kick another vcpu out of halt state. the presence of these hypercalls is indicated to guest via kvm_feature_pv_unhalt. Signed-off-by: Srivatsa Vaddagiri <vatsa at linux.vnet.ibm.com> Signed-off-by: Suzuki Poulose <suzuki at in.ibm.com> [Raghu: Apic related changes, folding pvunhalted into vcpu_runnable Added flags for future use (suggested by Gleb)] Signed-off-by: Raghavendra K T <raghavendra.kt at linux.vnet.ibm.com> Acked-by: Gleb Natapov <gleb at redhat.com> Acked-by: Ingo Molnar <mingo at kernel.org> --- arch/x86/include/asm/kvm_host.h | 5 +++++ arch/x86/kvm/cpuid.c | 3 ++- arch/x86/kvm/x86.c | 37 +++++++++++++++++++++++++++++++++++++ 3 files changed, 44 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index f87f7fc..1d1f711 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -511,6 +511,11 @@ struct kvm_vcpu_arch { * instruction. */ bool write_fault_to_shadow_pgtable; + + /* pv related host specific info */ + struct { + bool pv_unhalted; + } pv; }; struct kvm_lpage_info { diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index a20ecb5..b110fe6 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -413,7 +413,8 @@ static int do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function, (1 << KVM_FEATURE_CLOCKSOURCE2) | (1 << KVM_FEATURE_ASYNC_PF) | (1 << KVM_FEATURE_PV_EOI) | - (1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT); + (1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT) | + (1 << KVM_FEATURE_PV_UNHALT); if (sched_info_on()) entry->eax |= (1 << KVM_FEATURE_STEAL_TIME); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index d21bce5..dae4575 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5495,6 +5495,36 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) return 1; } +/* + * kvm_pv_kick_cpu_op: Kick a vcpu. + * + * @apicid - apicid of vcpu to be kicked. + */ +static void kvm_pv_kick_cpu_op(struct kvm *kvm, unsigned long flags, int apicid) +{ + struct kvm_vcpu *vcpu = NULL; + int i; + + kvm_for_each_vcpu(i, vcpu, kvm) { + if (!kvm_apic_present(vcpu)) + continue; + + if (kvm_apic_match_dest(vcpu, 0, 0, apicid, 0)) + break; + } + if (vcpu) { + /* + * Setting unhalt flag here can result in spurious runnable + * state when unhalt reset does not happen in vcpu_block. + * But that is harmless since that should soon result in halt. + */ + vcpu->arch.pv.pv_unhalted = true; + /* We need everybody see unhalt before vcpu unblocks */ + smp_wmb(); + kvm_vcpu_kick(vcpu); + } +} + int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) { unsigned long nr, a0, a1, a2, a3, ret; @@ -5528,6 +5558,10 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) case KVM_HC_VAPIC_POLL_IRQ: ret = 0; break; + case KVM_HC_KICK_CPU: + kvm_pv_kick_cpu_op(vcpu->kvm, a0, a1); + ret = 0; + break; default: ret = -KVM_ENOSYS; break; @@ -5950,6 +5984,7 @@ static int __vcpu_run(struct kvm_vcpu *vcpu) kvm_apic_accept_events(vcpu); switch(vcpu->arch.mp_state) { case KVM_MP_STATE_HALTED: + vcpu->arch.pv.pv_unhalted = false; vcpu->arch.mp_state KVM_MP_STATE_RUNNABLE; case KVM_MP_STATE_RUNNABLE: @@ -6770,6 +6805,7 @@ int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu) BUG_ON(vcpu->kvm == NULL); kvm = vcpu->kvm; + vcpu->arch.pv.pv_unhalted = false; vcpu->arch.emulate_ctxt.ops = &emulate_ops; if (!irqchip_in_kernel(kvm) || kvm_vcpu_is_bsp(vcpu)) vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; @@ -7103,6 +7139,7 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu) !vcpu->arch.apf.halted) || !list_empty_careful(&vcpu->async_pf.done) || kvm_apic_has_events(vcpu) + || vcpu->arch.pv.pv_unhalted || atomic_read(&vcpu->arch.nmi_queued) || (kvm_arch_interrupt_allowed(vcpu) && kvm_cpu_has_interrupt(vcpu));
Raghavendra K T
2013-Aug-06 18:23 UTC
[PATCH V12 3/5] kvm : Fold pv_unhalt flag into GET_MP_STATE ioctl to aid migration
kvm : Fold pv_unhalt flag into GET_MP_STATE ioctl to aid migration From: Raghavendra K T <raghavendra.kt at linux.vnet.ibm.com> During migration, any vcpu that got kicked but did not become runnable (still in halted state) should be runnable after migration. Signed-off-by: Raghavendra K T <raghavendra.kt at linux.vnet.ibm.com> Acked-by: Gleb Natapov <gleb at redhat.com> Acked-by: Ingo Molnar <mingo at kernel.org> --- arch/x86/kvm/x86.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index dae4575..1e73dab 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -6284,7 +6284,12 @@ int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu, struct kvm_mp_state *mp_state) { kvm_apic_accept_events(vcpu); - mp_state->mp_state = vcpu->arch.mp_state; + if (vcpu->arch.mp_state == KVM_MP_STATE_HALTED && + vcpu->arch.pv.pv_unhalted) + mp_state->mp_state = KVM_MP_STATE_RUNNABLE; + else + mp_state->mp_state = vcpu->arch.mp_state; + return 0; }
Raghavendra K T
2013-Aug-06 18:23 UTC
[PATCH V12 4/5] kvm hypervisor : Simplify kvm_for_each_vcpu with kvm_irq_delivery_to_apic
kvm hypervisor: Simplify kvm_for_each_vcpu with kvm_irq_delivery_to_apic From: Raghavendra K T <raghavendra.kt at linux.vnet.ibm.com> Note that we are using APIC_DM_REMRD which has reserved usage. In future if APIC_DM_REMRD usage is standardized, then we should find some other way or go back to old method. Suggested-by: Gleb Natapov <gleb at redhat.com> Signed-off-by: Raghavendra K T <raghavendra.kt at linux.vnet.ibm.com> Acked-by: Gleb Natapov <gleb at redhat.com> Acked-by: Ingo Molnar <mingo at kernel.org> --- arch/x86/kvm/lapic.c | 5 ++++- arch/x86/kvm/x86.c | 25 ++++++------------------- 2 files changed, 10 insertions(+), 20 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index afc1124..48c13c9 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -706,7 +706,10 @@ out: break; case APIC_DM_REMRD: - apic_debug("Ignoring delivery mode 3\n"); + result = 1; + vcpu->arch.pv.pv_unhalted = 1; + kvm_make_request(KVM_REQ_EVENT, vcpu); + kvm_vcpu_kick(vcpu); break; case APIC_DM_SMI: diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 1e73dab..640d112 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5502,27 +5502,14 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) */ static void kvm_pv_kick_cpu_op(struct kvm *kvm, unsigned long flags, int apicid) { - struct kvm_vcpu *vcpu = NULL; - int i; + struct kvm_lapic_irq lapic_irq; - kvm_for_each_vcpu(i, vcpu, kvm) { - if (!kvm_apic_present(vcpu)) - continue; + lapic_irq.shorthand = 0; + lapic_irq.dest_mode = 0; + lapic_irq.dest_id = apicid; - if (kvm_apic_match_dest(vcpu, 0, 0, apicid, 0)) - break; - } - if (vcpu) { - /* - * Setting unhalt flag here can result in spurious runnable - * state when unhalt reset does not happen in vcpu_block. - * But that is harmless since that should soon result in halt. - */ - vcpu->arch.pv.pv_unhalted = true; - /* We need everybody see unhalt before vcpu unblocks */ - smp_wmb(); - kvm_vcpu_kick(vcpu); - } + lapic_irq.delivery_mode = APIC_DM_REMRD; + kvm_irq_delivery_to_apic(kvm, 0, &lapic_irq, NULL); } int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
Raghavendra K T
2013-Aug-06 18:23 UTC
[PATCH V12 5/5] Documentation/kvm : Add documentation on Hypercalls and features used for PV spinlock
Documentation/kvm : Add documentation on Hypercalls and features used for PV spinlock From: Raghavendra K T <raghavendra.kt at linux.vnet.ibm.com> KVM_HC_KICK_CPU hypercall added to wakeup halted vcpu in paravirtual spinlock enabled guest. KVM_FEATURE_PV_UNHALT enables guest to check whether pv spinlock can be enabled in guest. Thanks Vatsa for rewriting KVM_HC_KICK_CPU Signed-off-by: Srivatsa Vaddagiri <vatsa at linux.vnet.ibm.com> Signed-off-by: Raghavendra K T <raghavendra.kt at linux.vnet.ibm.com> Acked-by: Gleb Natapov <gleb at redhat.com> Acked-by: Ingo Molnar <mingo at kernel.org> --- Documentation/virtual/kvm/cpuid.txt | 4 ++++ Documentation/virtual/kvm/hypercalls.txt | 14 ++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/Documentation/virtual/kvm/cpuid.txt b/Documentation/virtual/kvm/cpuid.txt index 83afe65..22ff659 100644 --- a/Documentation/virtual/kvm/cpuid.txt +++ b/Documentation/virtual/kvm/cpuid.txt @@ -43,6 +43,10 @@ KVM_FEATURE_CLOCKSOURCE2 || 3 || kvmclock available at msrs KVM_FEATURE_ASYNC_PF || 4 || async pf can be enabled by || || writing to msr 0x4b564d02 ------------------------------------------------------------------------------ +KVM_FEATURE_PV_UNHALT || 7 || guest checks this feature bit + || || before enabling paravirtualized + || || spinlock support. +------------------------------------------------------------------------------ KVM_FEATURE_CLOCKSOURCE_STABLE_BIT || 24 || host will warn if no guest-side || || per-cpu warps are expected in || || kvmclock. diff --git a/Documentation/virtual/kvm/hypercalls.txt b/Documentation/virtual/kvm/hypercalls.txt index ea113b5..022198e 100644 --- a/Documentation/virtual/kvm/hypercalls.txt +++ b/Documentation/virtual/kvm/hypercalls.txt @@ -64,3 +64,17 @@ Purpose: To enable communication between the hypervisor and guest there is a shared page that contains parts of supervisor visible register state. The guest can map this shared page to access its supervisor register through memory using this hypercall. + +5. KVM_HC_KICK_CPU +------------------------ +Architecture: x86 +Status: active +Purpose: Hypercall used to wakeup a vcpu from HLT state +Usage example : A vcpu of a paravirtualized guest that is busywaiting in guest +kernel mode for an event to occur (ex: a spinlock to become available) can +execute HLT instruction once it has busy-waited for more than a threshold +time-interval. Execution of HLT instruction would cause the hypervisor to put +the vcpu to sleep until occurence of an appropriate event. Another vcpu of the +same guest can wakeup the sleeping vcpu by issuing KVM_HC_KICK_CPU hypercall, +specifying APIC ID (a1) of the vcpu to be woken up. An additional argument (a0) +is used in the hypercall for future use.
Eric Northup
2013-Aug-06 18:32 UTC
[PATCH V12 3/5] kvm : Fold pv_unhalt flag into GET_MP_STATE ioctl to aid migration
On Tue, Aug 6, 2013 at 11:23 AM, Raghavendra K T <raghavendra.kt at linux.vnet.ibm.com> wrote:> kvm : Fold pv_unhalt flag into GET_MP_STATE ioctl to aid migration > > From: Raghavendra K T <raghavendra.kt at linux.vnet.ibm.com> > > During migration, any vcpu that got kicked but did not become runnable > (still in halted state) should be runnable after migration.If this is about migration correctness, could it get folded into the previous patch 2/5, so that there's not a broken commit which could hurt bisection?> > Signed-off-by: Raghavendra K T <raghavendra.kt at linux.vnet.ibm.com> > Acked-by: Gleb Natapov <gleb at redhat.com> > Acked-by: Ingo Molnar <mingo at kernel.org> > --- > arch/x86/kvm/x86.c | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > index dae4575..1e73dab 100644 > --- a/arch/x86/kvm/x86.c > +++ b/arch/x86/kvm/x86.c > @@ -6284,7 +6284,12 @@ int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu, > struct kvm_mp_state *mp_state) > { > kvm_apic_accept_events(vcpu); > - mp_state->mp_state = vcpu->arch.mp_state; > + if (vcpu->arch.mp_state == KVM_MP_STATE_HALTED && > + vcpu->arch.pv.pv_unhalted) > + mp_state->mp_state = KVM_MP_STATE_RUNNABLE; > + else > + mp_state->mp_state = vcpu->arch.mp_state; > + > return 0; > } > > > _______________________________________________ > Virtualization mailing list > Virtualization at lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Seemingly Similar Threads
- [PATCH V12 3/5] kvm : Fold pv_unhalt flag into GET_MP_STATE ioctl to aid migration
- [PATCH V12 3/5] kvm : Fold pv_unhalt flag into GET_MP_STATE ioctl to aid migration
- [PATCH V12 3/5] kvm : Fold pv_unhalt flag into GET_MP_STATE ioctl to aid migration
- [PATCH V13 0/4] Paravirtualized ticket spinlocks for KVM host
- [PATCH V13 0/4] Paravirtualized ticket spinlocks for KVM host