Hi all, this is the fixed, updated and rebased version of the PV on HVM series: the series is based on 2.6.34 now and supports Xen PV frontends running in a HVM domain, including netfront, blkfront and the VIRQ_TIMER. The list of bugs fixed in this update includes: xenbus drivers crashes when xenbus is not properly initialized, a memory corruption bug in suspend/resume and testing for the xen platform pci version and protocol has been moved to enlighten.c (before unplugging emulated devices). In order to be able to use VIRQ_TIMER and to improve performances you need a patch to Xen to implement the vector callback mechanism for event channel delivery. A git tree is also available here: git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git branch name 2.6.34-pvhvm. Cheers, Stefano _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
From: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Signed-off-by: Sheng Yang <sheng@linux.intel.com> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- arch/x86/include/asm/xen/hypercall.h | 6 ++ include/xen/hvm.h | 24 +++++++++ include/xen/interface/hvm/hvm_op.h | 35 ++++++++++++ include/xen/interface/hvm/params.h | 95 ++++++++++++++++++++++++++++++++++ 4 files changed, 160 insertions(+), 0 deletions(-) create mode 100644 include/xen/hvm.h create mode 100644 include/xen/interface/hvm/hvm_op.h create mode 100644 include/xen/interface/hvm/params.h diff --git a/arch/x86/include/asm/xen/hypercall.h b/arch/x86/include/asm/xen/hypercall.h index 9c371e4..7fda040 100644 --- a/arch/x86/include/asm/xen/hypercall.h +++ b/arch/x86/include/asm/xen/hypercall.h @@ -417,6 +417,12 @@ HYPERVISOR_nmi_op(unsigned long op, unsigned long arg) return _hypercall2(int, nmi_op, op, arg); } +static inline unsigned long __must_check +HYPERVISOR_hvm_op(int op, void *arg) +{ + return _hypercall2(unsigned long, hvm_op, op, arg); +} + static inline void MULTI_fpu_taskswitch(struct multicall_entry *mcl, int set) { diff --git a/include/xen/hvm.h b/include/xen/hvm.h new file mode 100644 index 0000000..6b0d418 --- /dev/null +++ b/include/xen/hvm.h @@ -0,0 +1,24 @@ +/* Simple wrappers around HVM functions */ +#ifndef XEN_HVM_H__ +#define XEN_HVM_H__ + +#include <xen/interface/hvm/params.h> + +static inline int hvm_get_parameter(int idx, uint64_t *value) +{ + struct xen_hvm_param xhv; + int r; + + xhv.domid = DOMID_SELF; + xhv.index = idx; + r = HYPERVISOR_hvm_op(HVMOP_get_param, &xhv); + if (r < 0) { + printk(KERN_ERR "cannot get hvm parameter %d: %d.\n", + idx, r); + return r; + } + *value = xhv.value; + return r; +} + +#endif /* XEN_HVM_H__ */ diff --git a/include/xen/interface/hvm/hvm_op.h b/include/xen/interface/hvm/hvm_op.h new file mode 100644 index 0000000..73c8c7e --- /dev/null +++ b/include/xen/interface/hvm/hvm_op.h @@ -0,0 +1,35 @@ +/* + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to + * deal in the Software without restriction, including without limitation the + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or + * sell copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. + */ + +#ifndef __XEN_PUBLIC_HVM_HVM_OP_H__ +#define __XEN_PUBLIC_HVM_HVM_OP_H__ + +/* Get/set subcommands: the second argument of the hypercall is a + * pointer to a xen_hvm_param struct. */ +#define HVMOP_set_param 0 +#define HVMOP_get_param 1 +struct xen_hvm_param { + domid_t domid; /* IN */ + uint32_t index; /* IN */ + uint64_t value; /* IN/OUT */ +}; +DEFINE_GUEST_HANDLE_STRUCT(xen_hvm_param); + +#endif /* __XEN_PUBLIC_HVM_HVM_OP_H__ */ diff --git a/include/xen/interface/hvm/params.h b/include/xen/interface/hvm/params.h new file mode 100644 index 0000000..1888d8c --- /dev/null +++ b/include/xen/interface/hvm/params.h @@ -0,0 +1,95 @@ +/* + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to + * deal in the Software without restriction, including without limitation the + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or + * sell copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. + */ + +#ifndef __XEN_PUBLIC_HVM_PARAMS_H__ +#define __XEN_PUBLIC_HVM_PARAMS_H__ + +#include "hvm_op.h" + +/* + * Parameter space for HVMOP_{set,get}_param. + */ + +/* + * How should CPU0 event-channel notifications be delivered? + * val[63:56] == 0: val[55:0] is a delivery GSI (Global System Interrupt). + * val[63:56] == 1: val[55:0] is a delivery PCI INTx line, as follows: + * Domain = val[47:32], Bus = val[31:16], + * DevFn = val[15: 8], IntX = val[ 1: 0] + * val[63:56] == 2: val[7:0] is a vector number. + * If val == 0 then CPU0 event-channel notifications are not delivered. + */ +#define HVM_PARAM_CALLBACK_IRQ 0 + +#define HVM_PARAM_STORE_PFN 1 +#define HVM_PARAM_STORE_EVTCHN 2 + +#define HVM_PARAM_PAE_ENABLED 4 + +#define HVM_PARAM_IOREQ_PFN 5 + +#define HVM_PARAM_BUFIOREQ_PFN 6 + +/* + * Set mode for virtual timers (currently x86 only): + * delay_for_missed_ticks (default): + * Do not advance a vcpu''s time beyond the correct delivery time for + * interrupts that have been missed due to preemption. Deliver missed + * interrupts when the vcpu is rescheduled and advance the vcpu''s virtual + * time stepwise for each one. + * no_delay_for_missed_ticks: + * As above, missed interrupts are delivered, but guest time always tracks + * wallclock (i.e., real) time while doing so. + * no_missed_ticks_pending: + * No missed interrupts are held pending. Instead, to ensure ticks are + * delivered at some non-zero rate, if we detect missed ticks then the + * internal tick alarm is not disabled if the VCPU is preempted during the + * next tick period. + * one_missed_tick_pending: + * Missed interrupts are collapsed together and delivered as one ''late tick''. + * Guest time always tracks wallclock (i.e., real) time. + */ +#define HVM_PARAM_TIMER_MODE 10 +#define HVMPTM_delay_for_missed_ticks 0 +#define HVMPTM_no_delay_for_missed_ticks 1 +#define HVMPTM_no_missed_ticks_pending 2 +#define HVMPTM_one_missed_tick_pending 3 + +/* Boolean: Enable virtual HPET (high-precision event timer)? (x86-only) */ +#define HVM_PARAM_HPET_ENABLED 11 + +/* Identity-map page directory used by Intel EPT when CR0.PG=0. */ +#define HVM_PARAM_IDENT_PT 12 + +/* Device Model domain, defaults to 0. */ +#define HVM_PARAM_DM_DOMAIN 13 + +/* ACPI S state: currently support S0 and S3 on x86. */ +#define HVM_PARAM_ACPI_S_STATE 14 + +/* TSS used on Intel when CR0.PE=0. */ +#define HVM_PARAM_VM86_TSS 15 + +/* Boolean: Enable aligning all periodic vpts to reduce interrupts */ +#define HVM_PARAM_VPT_ALIGN 16 + +#define HVM_NR_PARAMS 17 + +#endif /* __XEN_PUBLIC_HVM_PARAMS_H__ */ -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
From: Sheng Yang <sheng@linux.intel.com> Initialize basic pv on hvm features in xen_guest_init. Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> Signed-off-by: Sheng Yang <sheng@linux.intel.com> Signed-off-by: Yaozu (Eddie) Dong <eddie.dong@intel.com> --- arch/x86/kernel/setup.c | 2 + arch/x86/xen/enlighten.c | 87 +++++++++++++++++++++++++++++++++++++ drivers/input/xen-kbdfront.c | 2 +- drivers/video/xen-fbfront.c | 2 +- drivers/xen/xenbus/xenbus_probe.c | 21 ++++++++- include/xen/xen.h | 2 + 6 files changed, 111 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index c4851ef..ae9b6cb 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -69,6 +69,7 @@ #include <linux/tboot.h> #include <video/edid.h> +#include <xen/xen.h> #include <asm/mtrr.h> #include <asm/apic.h> @@ -1032,6 +1033,7 @@ void __init setup_arch(char **cmdline_p) probe_nr_irqs_gsi(); kvm_guest_init(); + xen_guest_init(); e820_reserve_resources(); e820_mark_nosave_regions(max_low_pfn); diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index 65d8d79..87a3b10 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -35,6 +35,8 @@ #include <xen/interface/version.h> #include <xen/interface/physdev.h> #include <xen/interface/vcpu.h> +#include <xen/interface/memory.h> +#include <xen/interface/hvm/hvm_op.h> #include <xen/features.h> #include <xen/page.h> #include <xen/hvc-console.h> @@ -56,6 +58,7 @@ #include <asm/tlbflush.h> #include <asm/reboot.h> #include <asm/stackprotector.h> +#include <asm/hypervisor.h> #include "xen-ops.h" #include "mmu.h" @@ -1206,3 +1209,87 @@ asmlinkage void __init xen_start_kernel(void) x86_64_start_reservations((char *)__pa_symbol(&boot_params)); #endif } + +static uint32_t xen_cpuid_base(void) +{ + uint32_t base, eax, ebx, ecx, edx; + char signature[13]; + + for (base = 0x40000000; base < 0x40010000; base += 0x100) { + cpuid(base, &eax, &ebx, &ecx, &edx); + *(uint32_t*)(signature + 0) = ebx; + *(uint32_t*)(signature + 4) = ecx; + *(uint32_t*)(signature + 8) = edx; + signature[12] = 0; + + if (!strcmp("XenVMMXenVMM", signature) && ((eax - base) >= 2)) + return base; + } + + return 0; +} + +static int init_hvm_pv_info(int *major, int *minor) +{ + uint32_t eax, ebx, ecx, edx, pages, msr, base; + u64 pfn; + + base = xen_cpuid_base(); + if (!base) + return -EINVAL; + + cpuid(base + 1, &eax, &ebx, &ecx, &edx); + + *major = eax >> 16; + *minor = eax & 0xffff; + printk(KERN_INFO "Xen version %d.%d.\n", *major, *minor); + + cpuid(base + 2, &pages, &msr, &ecx, &edx); + + pfn = __pa(hypercall_page); + wrmsr_safe(msr, (u32)pfn, (u32)(pfn >> 32)); + + xen_setup_features(); + + pv_info = xen_info; + pv_info.kernel_rpl = 0; + + xen_domain_type = XEN_HVM_DOMAIN; + + return 0; +} + +static void __init init_shared_info(void) +{ + struct xen_add_to_physmap xatp; + struct shared_info *shared_info_page; + + shared_info_page = (struct shared_info *) alloc_bootmem_pages(PAGE_SIZE); + xatp.domid = DOMID_SELF; + xatp.idx = 0; + xatp.space = XENMAPSPACE_shared_info; + xatp.gpfn = __pa(shared_info_page) >> PAGE_SHIFT; + if (HYPERVISOR_memory_op(XENMEM_add_to_physmap, &xatp)) + BUG(); + + HYPERVISOR_shared_info = (struct shared_info *)shared_info_page; + + /* Don''t do the full vcpu_info placement stuff until we have a + possible map and a non-dummy shared_info. */ + per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0]; +} + +void __init xen_guest_init(void) +{ + int r; + int major, minor; + + if (xen_pv_domain()) + return; + + r = init_hvm_pv_info(&major, &minor); + if (r < 0) + return; + + init_shared_info(); +} diff --git a/drivers/input/xen-kbdfront.c b/drivers/input/xen-kbdfront.c index e140816..7451d78 100644 --- a/drivers/input/xen-kbdfront.c +++ b/drivers/input/xen-kbdfront.c @@ -339,7 +339,7 @@ static struct xenbus_driver xenkbd_driver = { static int __init xenkbd_init(void) { - if (!xen_domain()) + if (!xen_domain() || xen_hvm_domain()) return -ENODEV; /* Nothing to do if running in dom0. */ diff --git a/drivers/video/xen-fbfront.c b/drivers/video/xen-fbfront.c index fa97d3e..a105a19 100644 --- a/drivers/video/xen-fbfront.c +++ b/drivers/video/xen-fbfront.c @@ -684,7 +684,7 @@ static struct xenbus_driver xenfb_driver = { static int __init xenfb_init(void) { - if (!xen_domain()) + if (!xen_domain() || xen_hvm_domain()) return -ENODEV; /* Nothing to do if running in dom0. */ diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index 3479332..0b05b62 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -56,6 +56,8 @@ #include <xen/events.h> #include <xen/page.h> +#include <xen/hvm.h> + #include "xenbus_comms.h" #include "xenbus_probe.h" @@ -806,10 +808,23 @@ static int __init xenbus_probe_init(void) /* dom0 not yet supported */ } else { xenstored_ready = 1; - xen_store_evtchn = xen_start_info->store_evtchn; - xen_store_mfn = xen_start_info->store_mfn; + if (xen_hvm_domain()) { + uint64_t v = 0; + err = hvm_get_parameter(HVM_PARAM_STORE_EVTCHN, &v); + if (err) + goto out_error; + xen_store_evtchn = (int)v; + err = hvm_get_parameter(HVM_PARAM_STORE_PFN, &v); + if (err) + goto out_error; + xen_store_mfn = (unsigned long)v; + xen_store_interface = ioremap(xen_store_mfn << PAGE_SHIFT, PAGE_SIZE); + } else { + xen_store_evtchn = xen_start_info->store_evtchn; + xen_store_mfn = xen_start_info->store_mfn; + xen_store_interface = mfn_to_virt(xen_store_mfn); + } } - xen_store_interface = mfn_to_virt(xen_store_mfn); /* Initialize the interface to xenstore. */ err = xs_init(); diff --git a/include/xen/xen.h b/include/xen/xen.h index a164024..cb8c48b 100644 --- a/include/xen/xen.h +++ b/include/xen/xen.h @@ -9,8 +9,10 @@ enum xen_domain_type { #ifdef CONFIG_XEN extern enum xen_domain_type xen_domain_type; +extern void xen_guest_init(void); #else #define xen_domain_type XEN_NATIVE +#define xen_guest_init() do { } while (0) #endif #define xen_domain() (xen_domain_type != XEN_NATIVE) -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
From: Sheng Yang <sheng@linux.intel.com> Set the callback to receive evtchns from Xen, using the callback vector delivery mechanism. Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> Signed-off-by: Sheng Yang <sheng@linux.intel.com> --- arch/x86/xen/enlighten.c | 35 +++++++++++++++++++++++++++++++++++ drivers/xen/events.c | 31 ++++++++++++++++++++++++------- include/xen/events.h | 3 +++ include/xen/hvm.h | 9 +++++++++ include/xen/interface/features.h | 3 +++ 5 files changed, 74 insertions(+), 7 deletions(-) diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index 87a3b10..502c4f8 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -37,8 +37,11 @@ #include <xen/interface/vcpu.h> #include <xen/interface/memory.h> #include <xen/interface/hvm/hvm_op.h> +#include <xen/interface/hvm/params.h> #include <xen/features.h> #include <xen/page.h> +#include <xen/hvm.h> +#include <xen/events.h> #include <xen/hvc-console.h> #include <asm/paravirt.h> @@ -79,6 +82,8 @@ struct shared_info xen_dummy_shared_info; void *xen_initial_gdt; +int xen_have_vector_callback; + /* * Point at some empty memory to start with. We map the real shared_info * page as soon as fixmap is up and running. @@ -1279,6 +1284,31 @@ static void __init init_shared_info(void) per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0]; } +int xen_set_callback_via(uint64_t via) +{ + struct xen_hvm_param a; + a.domid = DOMID_SELF; + a.index = HVM_PARAM_CALLBACK_IRQ; + a.value = via; + return HYPERVISOR_hvm_op(HVMOP_set_param, &a); +} + +void do_hvm_pv_evtchn_intr(void) +{ + xen_hvm_evtchn_do_upcall(get_irq_regs()); +} + +static void xen_callback_vector(void) +{ + uint64_t callback_via; + if (xen_feature(XENFEAT_hvm_callback_vector)) { + callback_via = HVM_CALLBACK_VECTOR(X86_PLATFORM_IPI_VECTOR); + xen_set_callback_via(callback_via); + x86_platform_ipi_callback = do_hvm_pv_evtchn_intr; + xen_have_vector_callback = 1; + } +} + void __init xen_guest_init(void) { int r; @@ -1292,4 +1322,9 @@ void __init xen_guest_init(void) return; init_shared_info(); + + xen_callback_vector(); + + have_vcpu_info_placement = 0; + x86_init.irqs.intr_init = xen_init_IRQ; } diff --git a/drivers/xen/events.c b/drivers/xen/events.c index db8f506..3523dbb 100644 --- a/drivers/xen/events.c +++ b/drivers/xen/events.c @@ -36,6 +36,8 @@ #include <asm/xen/hypercall.h> #include <asm/xen/hypervisor.h> +#include <xen/xen.h> +#include <xen/hvm.h> #include <xen/xen-ops.h> #include <xen/events.h> #include <xen/interface/xen.h> @@ -617,17 +619,13 @@ static DEFINE_PER_CPU(unsigned, xed_nesting_count); * a bitset of words which contain pending event bits. The second * level is a bitset of pending events themselves. */ -void xen_evtchn_do_upcall(struct pt_regs *regs) +void __xen_evtchn_do_upcall(struct pt_regs *regs) { int cpu = get_cpu(); - struct pt_regs *old_regs = set_irq_regs(regs); struct shared_info *s = HYPERVISOR_shared_info; struct vcpu_info *vcpu_info = __get_cpu_var(xen_vcpu); unsigned count; - exit_idle(); - irq_enter(); - do { unsigned long pending_words; @@ -667,10 +665,26 @@ void xen_evtchn_do_upcall(struct pt_regs *regs) } while(count != 1); out: + + put_cpu(); +} + +void xen_evtchn_do_upcall(struct pt_regs *regs) +{ + struct pt_regs *old_regs = set_irq_regs(regs); + + exit_idle(); + irq_enter(); + + __xen_evtchn_do_upcall(regs); + irq_exit(); set_irq_regs(old_regs); +} - put_cpu(); +void xen_hvm_evtchn_do_upcall(struct pt_regs *regs) +{ + __xen_evtchn_do_upcall(regs); } /* Rebind a new event channel to an existing irq. */ @@ -947,5 +961,8 @@ void __init xen_init_IRQ(void) for (i = 0; i < NR_EVENT_CHANNELS; i++) mask_evtchn(i); - irq_ctx_init(smp_processor_id()); + if (xen_hvm_domain()) + native_init_IRQ(); + else + irq_ctx_init(smp_processor_id()); } diff --git a/include/xen/events.h b/include/xen/events.h index e68d59a..868e5d6 100644 --- a/include/xen/events.h +++ b/include/xen/events.h @@ -56,4 +56,7 @@ void xen_poll_irq(int irq); /* Determine the IRQ which is bound to an event channel */ unsigned irq_from_evtchn(unsigned int evtchn); +void xen_evtchn_do_upcall(struct pt_regs *regs); +void xen_hvm_evtchn_do_upcall(struct pt_regs *regs); + #endif /* _XEN_EVENTS_H */ diff --git a/include/xen/hvm.h b/include/xen/hvm.h index 6b0d418..5940ee5 100644 --- a/include/xen/hvm.h +++ b/include/xen/hvm.h @@ -3,6 +3,7 @@ #define XEN_HVM_H__ #include <xen/interface/hvm/params.h> +#include <asm/xen/hypercall.h> static inline int hvm_get_parameter(int idx, uint64_t *value) { @@ -21,4 +22,12 @@ static inline int hvm_get_parameter(int idx, uint64_t *value) return r; } +int xen_set_callback_via(uint64_t via); +extern int xen_have_vector_callback; + +#define HVM_CALLBACK_VIA_TYPE_VECTOR 0x2 +#define HVM_CALLBACK_VIA_TYPE_SHIFT 56 +#define HVM_CALLBACK_VECTOR(x) (((uint64_t)HVM_CALLBACK_VIA_TYPE_VECTOR)<<\ + HVM_CALLBACK_VIA_TYPE_SHIFT | (x)) + #endif /* XEN_HVM_H__ */ diff --git a/include/xen/interface/features.h b/include/xen/interface/features.h index f51b641..8ab08b9 100644 --- a/include/xen/interface/features.h +++ b/include/xen/interface/features.h @@ -41,6 +41,9 @@ /* x86: Does this Xen host support the MMU_PT_UPDATE_PRESERVE_AD hypercall? */ #define XENFEAT_mmu_pt_update_preserve_ad 5 +/* x86: Does this Xen host support the HVM callback vector type? */ +#define XENFEAT_hvm_callback_vector 8 + #define XENFEAT_NR_SUBMAPS 1 #endif /* __XEN_PUBLIC_FEATURES_H__ */ -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-18 10:22 UTC
[Xen-devel] [PATCH 04/12] Fix find_unbound_irq in presence of ioapic irqs.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- drivers/xen/events.c | 11 ++++++++++- 1 files changed, 10 insertions(+), 1 deletions(-) diff --git a/drivers/xen/events.c b/drivers/xen/events.c index 3523dbb..a137a2f 100644 --- a/drivers/xen/events.c +++ b/drivers/xen/events.c @@ -337,9 +337,18 @@ static int find_unbound_irq(void) int irq; struct irq_desc *desc; - for (irq = 0; irq < nr_irqs; irq++) + for (irq = 0; irq < nr_irqs; irq++) { + desc = irq_to_desc(irq); + /* only 0->15 have init''d desc; handle irq > 16 */ + if (desc == NULL) + break; + if (desc->chip == &no_irq_chip) + break; + if (desc->chip != &xen_dynamic_chip) + continue; if (irq_info[irq].type == IRQT_UNBOUND) break; + } if (irq == nr_irqs) panic("No available IRQ to bind to: increase nr_irqs!\n"); -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-18 10:23 UTC
[Xen-devel] [PATCH 05/12] unplug emulated disks and nics
add a xen_unplug command line option to the kernel to unplug xen emulated disks and nics. Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- arch/x86/xen/enlighten.c | 66 ++++++++++++++++++++++++++++++++++ include/xen/hvm.h | 2 + include/xen/interface/platform_pci.h | 46 +++++++++++++++++++++++ include/xen/platform_pci.h | 32 ++++++++++++++++ 4 files changed, 146 insertions(+), 0 deletions(-) create mode 100644 include/xen/interface/platform_pci.h create mode 100644 include/xen/platform_pci.h diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index 502c4f8..aac47b0 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -31,6 +31,7 @@ #include <linux/gfp.h> #include <xen/xen.h> +#include <xen/platform_pci.h> #include <xen/interface/xen.h> #include <xen/interface/version.h> #include <xen/interface/physdev.h> @@ -38,6 +39,7 @@ #include <xen/interface/memory.h> #include <xen/interface/hvm/hvm_op.h> #include <xen/interface/hvm/params.h> +#include <xen/interface/platform_pci.h> #include <xen/features.h> #include <xen/page.h> #include <xen/hvm.h> @@ -83,6 +85,8 @@ struct shared_info xen_dummy_shared_info; void *xen_initial_gdt; int xen_have_vector_callback; +int xen_platform_pci; +static int unplug; /* * Point at some empty memory to start with. We map the real shared_info @@ -1309,6 +1313,39 @@ static void xen_callback_vector(void) } } +static int __init check_platform_magic(void) +{ + short magic; + char protocol; + + magic = inw(XEN_IOPORT_MAGIC); + if (magic != XEN_IOPORT_MAGIC_VAL) { + printk(KERN_ERR "Xen Platform Pci: unrecognised magic value\n"); + return -1; + } + + protocol = inb(XEN_IOPORT_PROTOVER); + + printk(KERN_DEBUG "Xen Platform Pci: I/O protocol version %d\n", + protocol); + + switch (protocol) { + case 1: + outw(XEN_IOPORT_LINUX_PRODNUM, XEN_IOPORT_PRODNUM); + outl(XEN_IOPORT_LINUX_DRVVER, XEN_IOPORT_DRVVER); + if (inw(XEN_IOPORT_MAGIC) != XEN_IOPORT_MAGIC_VAL) { + printk(KERN_ERR "Xen Platform: blacklisted by host\n"); + return -3; + } + break; + default: + printk(KERN_WARNING "Xen Platform Pci: unknown I/O protocol version"); + return -2; + } + + return 0; +} + void __init xen_guest_init(void) { int r; @@ -1325,6 +1362,35 @@ void __init xen_guest_init(void) xen_callback_vector(); + r = check_platform_magic(); + if (!r || (r == -1 && (unplug & UNPLUG_IGNORE))) + xen_platform_pci = 1; + if (xen_platform_pci && !(unplug & UNPLUG_IGNORE)) + outw(unplug, XEN_IOPORT_UNPLUG); have_vcpu_info_placement = 0; x86_init.irqs.intr_init = xen_init_IRQ; } + +static int __init parse_unplug(char *arg) +{ + char *p, *q; + + for (p = arg; p; p = q) { + q = strchr(arg, '',''); + if (q) + *q++ = ''\0''; + if (!strcmp(p, "all")) + unplug |= UNPLUG_ALL; + else if (!strcmp(p, "ide-disks")) + unplug |= UNPLUG_ALL_IDE_DISKS; + else if (!strcmp(p, "aux-ide-disks")) + unplug |= UNPLUG_AUX_IDE_DISKS; + else if (!strcmp(p, "nics")) + unplug |= UNPLUG_ALL_NICS; + else + printk(KERN_WARNING "unrecognised option ''%s'' " + "in module parameter ''dev_unplug''\n", p); + } + return 0; +} +early_param("xen_unplug", parse_unplug); diff --git a/include/xen/hvm.h b/include/xen/hvm.h index 5940ee5..777d2ce 100644 --- a/include/xen/hvm.h +++ b/include/xen/hvm.h @@ -30,4 +30,6 @@ extern int xen_have_vector_callback; #define HVM_CALLBACK_VECTOR(x) (((uint64_t)HVM_CALLBACK_VIA_TYPE_VECTOR)<<\ HVM_CALLBACK_VIA_TYPE_SHIFT | (x)) +extern int xen_platform_pci; + #endif /* XEN_HVM_H__ */ diff --git a/include/xen/interface/platform_pci.h b/include/xen/interface/platform_pci.h new file mode 100644 index 0000000..720eaf5 --- /dev/null +++ b/include/xen/interface/platform_pci.h @@ -0,0 +1,46 @@ +/****************************************************************************** + * platform_pci.h + * + * Interface for granting foreign access to page frames, and receiving + * page-ownership transfers. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to + * deal in the Software without restriction, including without limitation the + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or + * sell copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. + */ + +#ifndef __XEN_PUBLIC_PLATFORM_PCI_H__ +#define __XEN_PUBLIC_PLATFORM_PCI_H__ + +#define XEN_IOPORT_BASE 0x10 + +#define XEN_IOPORT_PLATFLAGS (XEN_IOPORT_BASE + 0) /* 1 byte access (R/W) */ +#define XEN_IOPORT_MAGIC (XEN_IOPORT_BASE + 0) /* 2 byte access (R) */ +#define XEN_IOPORT_UNPLUG (XEN_IOPORT_BASE + 0) /* 2 byte access (W) */ +#define XEN_IOPORT_DRVVER (XEN_IOPORT_BASE + 0) /* 4 byte access (W) */ + +#define XEN_IOPORT_SYSLOG (XEN_IOPORT_BASE + 2) /* 1 byte access (W) */ +#define XEN_IOPORT_PROTOVER (XEN_IOPORT_BASE + 2) /* 1 byte access (R) */ +#define XEN_IOPORT_PRODNUM (XEN_IOPORT_BASE + 2) /* 2 byte access (W) */ + +#define UNPLUG_ALL_IDE_DISKS 1 +#define UNPLUG_ALL_NICS 2 +#define UNPLUG_AUX_IDE_DISKS 4 +#define UNPLUG_ALL 7 +#define UNPLUG_IGNORE 8 + +#endif /* __XEN_PUBLIC_PLATFORM_PCI_H__ */ diff --git a/include/xen/platform_pci.h b/include/xen/platform_pci.h new file mode 100644 index 0000000..f39f4d3 --- /dev/null +++ b/include/xen/platform_pci.h @@ -0,0 +1,32 @@ +/****************************************************************************** + * platform-pci.h + * + * Xen platform PCI device driver + * Copyright (c) 2004, Intel Corporation. <xiaofeng.ling@intel.com> + * Copyright (c) 2007, XenSource Inc. + * Copyright (c) 2010, Citrix + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 59 Temple + * Place - Suite 330, Boston, MA 02111-1307 USA. + */ + +#ifndef _XEN_PLATFORM_PCI_H +#define _XEN_PLATFORM_PCI_H + +#include <linux/version.h> + +#define XEN_IOPORT_MAGIC_VAL 0x49d2 +#define XEN_IOPORT_LINUX_PRODNUM 0xffff +#define XEN_IOPORT_LINUX_DRVVER ((LINUX_VERSION_CODE << 8) + 0x0) + +#endif /* _XEN_PLATFORM_PCI_H */ -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-18 10:23 UTC
[Xen-devel] [PATCH 06/12] xen pci platform device driver
Add the xen pci platform device driver that is responsible for initializing the grant table and xenbus in PV on HVM mode. Few changes to xenbus and grant table are necessary to allow the delayed initialization in HVM mode. Grant table needs few additional modifications to work in HVM mode. When running on HVM the event channel upcall is never called while in progress because it is a normal Linux irq handler, therefore we cannot be sure that evtchn_upcall_pending is 0 when returning. For this reason if evtchn_upcall_pending is set by Xen we need to loop again on the event channels set pending otherwise we might loose some event channel deliveries. Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> Signed-off-by: Sheng Yang <sheng@linux.intel.com> --- drivers/xen/Kconfig | 8 ++ drivers/xen/Makefile | 3 +- drivers/xen/events.c | 5 +- drivers/xen/grant-table.c | 70 +++++++++++-- drivers/xen/platform-pci.c | 198 +++++++++++++++++++++++++++++++++++ drivers/xen/xenbus/xenbus_probe.c | 20 +++- include/xen/grant_table.h | 1 + include/xen/interface/grant_table.h | 1 + include/xen/platform_pci.h | 9 ++ include/xen/xenbus.h | 1 + 10 files changed, 300 insertions(+), 16 deletions(-) create mode 100644 drivers/xen/platform-pci.c diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig index fad3df2..da312e2 100644 --- a/drivers/xen/Kconfig +++ b/drivers/xen/Kconfig @@ -62,4 +62,12 @@ config XEN_SYS_HYPERVISOR virtual environment, /sys/hypervisor will still be present, but will have no xen contents. +config XEN_PLATFORM_PCI + tristate "xen platform pci device driver" + depends on XEN + help + Driver for the Xen PCI Platform device: it is responsible for + initializing xenbus and grant_table when running in a Xen HVM + domain. As a consequence this driver is required to run any Xen PV + frontend on Xen HVM. endmenu diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile index 7c28434..e392fb7 100644 --- a/drivers/xen/Makefile +++ b/drivers/xen/Makefile @@ -9,4 +9,5 @@ obj-$(CONFIG_XEN_XENCOMM) += xencomm.o obj-$(CONFIG_XEN_BALLOON) += balloon.o obj-$(CONFIG_XEN_DEV_EVTCHN) += evtchn.o obj-$(CONFIG_XENFS) += xenfs/ -obj-$(CONFIG_XEN_SYS_HYPERVISOR) += sys-hypervisor.o \ No newline at end of file +obj-$(CONFIG_XEN_SYS_HYPERVISOR) += sys-hypervisor.o +obj-$(CONFIG_XEN_PLATFORM_PCI) += platform-pci.o diff --git a/drivers/xen/events.c b/drivers/xen/events.c index a137a2f..cfc6d96 100644 --- a/drivers/xen/events.c +++ b/drivers/xen/events.c @@ -671,7 +671,7 @@ void __xen_evtchn_do_upcall(struct pt_regs *regs) count = __get_cpu_var(xed_nesting_count); __get_cpu_var(xed_nesting_count) = 0; - } while(count != 1); + } while(count != 1 || vcpu_info->evtchn_upcall_pending); out: @@ -731,7 +731,8 @@ static int rebind_irq_to_cpu(unsigned irq, unsigned tcpu) struct evtchn_bind_vcpu bind_vcpu; int evtchn = evtchn_from_irq(irq); - if (!VALID_EVTCHN(evtchn)) + if (!VALID_EVTCHN(evtchn) || + (xen_hvm_domain() && !xen_have_vector_callback)) return -1; /* Send future instances of this interrupt to other vcpu. */ diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c index f66db3b..6f5f3ba 100644 --- a/drivers/xen/grant-table.c +++ b/drivers/xen/grant-table.c @@ -37,11 +37,14 @@ #include <linux/slab.h> #include <linux/vmalloc.h> #include <linux/uaccess.h> +#include <linux/io.h> #include <xen/xen.h> #include <xen/interface/xen.h> #include <xen/page.h> #include <xen/grant_table.h> +#include <xen/platform_pci.h> +#include <xen/interface/memory.h> #include <asm/xen/hypercall.h> #include <asm/pgtable.h> @@ -59,6 +62,7 @@ static unsigned int boot_max_nr_grant_frames; static int gnttab_free_count; static grant_ref_t gnttab_free_head; static DEFINE_SPINLOCK(gnttab_list_lock); +static unsigned long hvm_pv_resume_frames; static struct grant_entry *shared; @@ -449,6 +453,30 @@ static int gnttab_map(unsigned int start_idx, unsigned int end_idx) unsigned int nr_gframes = end_idx + 1; int rc; + if (xen_hvm_domain()) { + struct xen_add_to_physmap xatp; + unsigned int i = end_idx; + rc = 0; + /* + * Loop backwards, so that the first hypercall has the largest + * index, ensuring that the table will grow only once. + */ + do { + xatp.domid = DOMID_SELF; + xatp.idx = i; + xatp.space = XENMAPSPACE_grant_table; + xatp.gpfn = (hvm_pv_resume_frames >> PAGE_SHIFT) + i; + rc = HYPERVISOR_memory_op(XENMEM_add_to_physmap, &xatp); + if (rc != 0) { + printk(KERN_WARNING + "grant table add_to_physmap failed, err=%d\n", rc); + break; + } + } while (i-- > start_idx); + + return rc; + } + frames = kmalloc(nr_gframes * sizeof(unsigned long), GFP_ATOMIC); if (!frames) return -ENOMEM; @@ -476,9 +504,28 @@ static int gnttab_map(unsigned int start_idx, unsigned int end_idx) int gnttab_resume(void) { - if (max_nr_grant_frames() < nr_grant_frames) + unsigned int max_nr_gframes; + + max_nr_gframes = max_nr_grant_frames(); + if (max_nr_gframes < nr_grant_frames) return -ENOSYS; - return gnttab_map(0, nr_grant_frames - 1); + + if (xen_pv_domain()) + return gnttab_map(0, nr_grant_frames - 1); + + if (!hvm_pv_resume_frames) { + hvm_pv_resume_frames = alloc_xen_mmio(PAGE_SIZE * max_nr_gframes); + shared = ioremap(hvm_pv_resume_frames, PAGE_SIZE * max_nr_gframes); + if (shared == NULL) { + printk(KERN_WARNING + "Fail to ioremap gnttab share frames\n"); + return -ENOMEM; + } + } + + gnttab_map(0, nr_grant_frames - 1); + + return 0; } int gnttab_suspend(void) @@ -505,15 +552,12 @@ static int gnttab_expand(unsigned int req_entries) return rc; } -static int __devinit gnttab_init(void) +int gnttab_init(void) { int i; unsigned int max_nr_glist_frames, nr_glist_frames; unsigned int nr_init_grefs; - if (!xen_domain()) - return -ENODEV; - nr_grant_frames = 1; boot_max_nr_grant_frames = __max_nr_grant_frames(); @@ -557,4 +601,16 @@ static int __devinit gnttab_init(void) return -ENOMEM; } -core_initcall(gnttab_init); +static int __devinit __gnttab_init(void) +{ + /* Delay grant-table initialization in the PV on HVM case */ + if (xen_hvm_domain()) + return 0; + + if (!xen_pv_domain()) + return -ENODEV; + + return gnttab_init(); +} + +core_initcall(__gnttab_init); diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c new file mode 100644 index 0000000..7a8da66 --- /dev/null +++ b/drivers/xen/platform-pci.c @@ -0,0 +1,198 @@ +/****************************************************************************** + * platform-pci.c + * + * Xen platform PCI device driver + * Copyright (c) 2005, Intel Corporation. + * Copyright (c) 2007, XenSource Inc. + * Copyright (c) 2010, Citrix + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 59 Temple + * Place - Suite 330, Boston, MA 02111-1307 USA. + * + */ + +#include <asm/io.h> + +#include <linux/interrupt.h> +#include <linux/module.h> +#include <linux/pci.h> + +#include <xen/grant_table.h> +#include <xen/platform_pci.h> +#include <xen/interface/platform_pci.h> +#include <xen/xenbus.h> +#include <xen/events.h> +#include <xen/hvm.h> + +#define DRV_NAME "xen-platform-pci" + +MODULE_AUTHOR("ssmith@xensource.com and stefano.stabellini@eu.citrix.com"); +MODULE_DESCRIPTION("Xen platform PCI device"); +MODULE_LICENSE("GPL"); + +static unsigned long platform_mmio; +static unsigned long platform_mmio_alloc; +static unsigned long platform_mmiolen; + +unsigned long alloc_xen_mmio(unsigned long len) +{ + unsigned long addr; + + addr = platform_mmio + platform_mmio_alloc; + platform_mmio_alloc += len; + BUG_ON(platform_mmio_alloc > platform_mmiolen); + + return addr; +} + +static uint64_t get_callback_via(struct pci_dev *pdev) +{ + u8 pin; + int irq; + + irq = pdev->irq; + if (irq < 16) + return irq; /* ISA IRQ */ + + pin = pdev->pin; + + /* We don''t know the GSI. Specify the PCI INTx line instead. */ + return ((uint64_t)0x01 << 56) | /* PCI INTx identifier */ + ((uint64_t)pci_domain_nr(pdev->bus) << 32) | + ((uint64_t)pdev->bus->number << 16) | + ((uint64_t)(pdev->devfn & 0xff) << 8) | + ((uint64_t)(pin - 1) & 3); +} + +static irqreturn_t do_hvm_evtchn_intr(int irq, void *dev_id) +{ + xen_hvm_evtchn_do_upcall(get_irq_regs()); + return IRQ_HANDLED; +} + +static int xen_allocate_irq(struct pci_dev *pdev) +{ + return request_irq(pdev->irq, do_hvm_evtchn_intr, + IRQF_DISABLED | IRQF_NOBALANCING | IRQF_TRIGGER_RISING, + "xen-platform-pci", pdev); +} + +static int __devinit platform_pci_init(struct pci_dev *pdev, + const struct pci_device_id *ent) +{ + int i, ret; + long ioaddr, iolen; + long mmio_addr, mmio_len; + uint64_t callback_via; + + i = pci_enable_device(pdev); + if (i) + return i; + + ioaddr = pci_resource_start(pdev, 0); + iolen = pci_resource_len(pdev, 0); + + mmio_addr = pci_resource_start(pdev, 1); + mmio_len = pci_resource_len(pdev, 1); + + if (mmio_addr == 0 || ioaddr == 0) { + dev_err(&pdev->dev, "no resources found\n"); + ret = -ENOENT; + } + + if (request_mem_region(mmio_addr, mmio_len, DRV_NAME) == NULL) { + dev_err(&pdev->dev, "MEM I/O resource 0x%lx @ 0x%lx busy\n", + mmio_addr, mmio_len); + ret = -EBUSY; + } + + if (request_region(ioaddr, iolen, DRV_NAME) == NULL) { + dev_err(&pdev->dev, "I/O resource 0x%lx @ 0x%lx busy\n", + iolen, ioaddr); + ret = -EBUSY; + goto out; + } + + platform_mmio = mmio_addr; + platform_mmiolen = mmio_len; + + if (!xen_have_vector_callback) { + ret = xen_allocate_irq(pdev); + if (ret) { + printk(KERN_WARNING "request_irq failed err=%d\n", ret); + goto out; + } + callback_via = get_callback_via(pdev); + ret = xen_set_callback_via(callback_via); + if (ret) { + printk(KERN_WARNING + "Unable to set the evtchn callback err=%d\n", ret); + goto out; + } + } + + alloc_xen_mmio_hook = alloc_xen_mmio; + platform_pci_resume_hook = platform_pci_resume; + platform_pci_disable_irq_hook = platform_pci_disable_irq; + platform_pci_enable_irq_hook = platform_pci_enable_irq; + + ret = gnttab_init(); + if (ret) + goto out; + ret = xenbus_probe_init(); + if (ret) + goto out; + +out: + if (ret) { + release_mem_region(mmio_addr, mmio_len); + release_region(ioaddr, iolen); + pci_disable_device(pdev); + } + + return ret; +} + +#define XEN_PLATFORM_VENDOR_ID 0x5853 +#define XEN_PLATFORM_DEVICE_ID 0x0001 +static struct pci_device_id platform_pci_tbl[] __devinitdata = { + {XEN_PLATFORM_VENDOR_ID, XEN_PLATFORM_DEVICE_ID, + PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0}, + {0,} +}; + +MODULE_DEVICE_TABLE(pci, platform_pci_tbl); + +static struct pci_driver platform_driver = { + name: DRV_NAME, + probe : platform_pci_init, + id_table : platform_pci_tbl, +}; + +static int __init platform_pci_module_init(void) +{ + int rc; + + if (!xen_platform_pci) + return -ENODEV; + + rc = pci_register_driver(&platform_driver); + if (rc) { + printk(KERN_INFO DRV_NAME + ": No platform pci device model found\n"); + return rc; + } + return 0; +} + +module_init(platform_pci_module_init); diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index 0b05b62..dc6ed06 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -782,16 +782,24 @@ void xenbus_probe(struct work_struct *unused) blocking_notifier_call_chain(&xenstore_chain, 0, NULL); } -static int __init xenbus_probe_init(void) +static int __init __xenbus_probe_init(void) +{ + /* Delay initialization in the PV on HVM case */ + if (xen_hvm_domain()) + return 0; + + if (!xen_pv_domain()) + return -ENODEV; + + return xenbus_probe_init(); +} + +int xenbus_probe_init(void) { int err = 0; DPRINTK(""); - err = -ENODEV; - if (!xen_domain()) - goto out_error; - /* Register ourselves with the kernel bus subsystem */ err = bus_register(&xenbus_frontend.bus); if (err) @@ -857,7 +865,7 @@ static int __init xenbus_probe_init(void) return err; } -postcore_initcall(xenbus_probe_init); +postcore_initcall(__xenbus_probe_init); MODULE_LICENSE("GPL"); diff --git a/include/xen/grant_table.h b/include/xen/grant_table.h index a40f1cd..811cda5 100644 --- a/include/xen/grant_table.h +++ b/include/xen/grant_table.h @@ -51,6 +51,7 @@ struct gnttab_free_callback { u16 count; }; +int gnttab_init(void); int gnttab_suspend(void); int gnttab_resume(void); diff --git a/include/xen/interface/grant_table.h b/include/xen/interface/grant_table.h index 39da93c..39e5717 100644 --- a/include/xen/interface/grant_table.h +++ b/include/xen/interface/grant_table.h @@ -28,6 +28,7 @@ #ifndef __XEN_PUBLIC_GRANT_TABLE_H__ #define __XEN_PUBLIC_GRANT_TABLE_H__ +#include <xen/interface/xen.h> /*********************************** * GRANT TABLE REPRESENTATION diff --git a/include/xen/platform_pci.h b/include/xen/platform_pci.h index f39f4d3..59a120c 100644 --- a/include/xen/platform_pci.h +++ b/include/xen/platform_pci.h @@ -29,4 +29,13 @@ #define XEN_IOPORT_LINUX_PRODNUM 0xffff #define XEN_IOPORT_LINUX_DRVVER ((LINUX_VERSION_CODE << 8) + 0x0) +#ifdef CONFIG_XEN_PLATFORM_PCI +unsigned long alloc_xen_mmio(unsigned long len); +#else +static inline unsigned long alloc_xen_mmio(unsigned long len) +{ + return ~0UL; +} +#endif + #endif /* _XEN_PLATFORM_PCI_H */ diff --git a/include/xen/xenbus.h b/include/xen/xenbus.h index 43e2d7d..ffa97de 100644 --- a/include/xen/xenbus.h +++ b/include/xen/xenbus.h @@ -174,6 +174,7 @@ void unregister_xenbus_watch(struct xenbus_watch *watch); void xs_suspend(void); void xs_resume(void); void xs_suspend_cancel(void); +int xenbus_probe_init(void); /* Used by xenbus_dev to borrow kernel''s store connection. */ void *xenbus_dev_request_and_reply(struct xsd_sockmsg *msg); -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-18 10:23 UTC
[Xen-devel] [PATCH 07/12] Add suspend\resume support for PV on HVM guests.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- arch/x86/xen/enlighten.c | 9 ++-- arch/x86/xen/suspend.c | 6 ++ arch/x86/xen/xen-ops.h | 3 + drivers/xen/manage.c | 95 +++++++++++++++++++++++++++++++++++-- drivers/xen/platform-pci.c | 29 +++++++++++- drivers/xen/xenbus/xenbus_probe.c | 28 +++++++++++ include/xen/platform_pci.h | 6 ++ include/xen/xen-ops.h | 3 + 8 files changed, 170 insertions(+), 9 deletions(-) diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index aac47b0..23b8200 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -1268,12 +1268,13 @@ static int init_hvm_pv_info(int *major, int *minor) return 0; } -static void __init init_shared_info(void) +void init_shared_info(void) { struct xen_add_to_physmap xatp; - struct shared_info *shared_info_page; + static struct shared_info *shared_info_page = 0; - shared_info_page = (struct shared_info *) alloc_bootmem_pages(PAGE_SIZE); + if (!shared_info_page) + shared_info_page = (struct shared_info *) alloc_bootmem_pages(PAGE_SIZE); xatp.domid = DOMID_SELF; xatp.idx = 0; xatp.space = XENMAPSPACE_shared_info; @@ -1302,7 +1303,7 @@ void do_hvm_pv_evtchn_intr(void) xen_hvm_evtchn_do_upcall(get_irq_regs()); } -static void xen_callback_vector(void) +void xen_callback_vector(void) { uint64_t callback_via; if (xen_feature(XENFEAT_hvm_callback_vector)) { diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c index 987267f..86f3b45 100644 --- a/arch/x86/xen/suspend.c +++ b/arch/x86/xen/suspend.c @@ -26,6 +26,12 @@ void xen_pre_suspend(void) BUG(); } +void xen_hvm_post_suspend(int suspend_cancelled) +{ + init_shared_info(); + xen_callback_vector(); +} + void xen_post_suspend(int suspend_cancelled) { xen_build_mfn_list_list(); diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h index f9153a3..caf89ee 100644 --- a/arch/x86/xen/xen-ops.h +++ b/arch/x86/xen/xen-ops.h @@ -38,6 +38,9 @@ void xen_enable_sysenter(void); void xen_enable_syscall(void); void xen_vcpu_restore(void); +void xen_callback_vector(void); +void init_shared_info(void); + void __init xen_build_dynamic_phys_to_machine(void); void xen_init_irq_ops(void); diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c index 2ac4440..a73edd8 100644 --- a/drivers/xen/manage.c +++ b/drivers/xen/manage.c @@ -8,15 +8,20 @@ #include <linux/sysrq.h> #include <linux/stop_machine.h> #include <linux/freezer.h> +#include <linux/pci.h> +#include <linux/cpumask.h> +#include <xen/xen.h> #include <xen/xenbus.h> #include <xen/grant_table.h> #include <xen/events.h> #include <xen/hvc-console.h> #include <xen/xen-ops.h> +#include <xen/platform_pci.h> #include <asm/xen/hypercall.h> #include <asm/xen/page.h> +#include <asm/xen/hypervisor.h> enum shutdown_state { SHUTDOWN_INVALID = -1, @@ -33,10 +38,30 @@ enum shutdown_state { static enum shutdown_state shutting_down = SHUTDOWN_INVALID; #ifdef CONFIG_PM_SLEEP -static int xen_suspend(void *data) +static int xen_hvm_suspend(void *data) { + struct sched_shutdown r = { .reason = SHUTDOWN_suspend }; int *cancelled = data; + + BUG_ON(!irqs_disabled()); + + *cancelled = HYPERVISOR_sched_op(SCHEDOP_shutdown, &r); + + xen_hvm_post_suspend(*cancelled); + gnttab_resume(); + + if (!*cancelled) { + xen_irq_resume(); + platform_pci_resume(); + } + + return 0; +} + +static int xen_suspend(void *data) +{ int err; + int *cancelled = data; BUG_ON(!irqs_disabled()); @@ -73,6 +98,53 @@ static int xen_suspend(void *data) return 0; } +static void do_hvm_suspend(void) +{ + int err; + int cancelled = 1; + + shutting_down = SHUTDOWN_SUSPEND; + +#ifdef CONFIG_PREEMPT + /* If the kernel is preemptible, we need to freeze all the processes + to prevent them from being in the middle of a pagetable update + during suspend. */ + err = freeze_processes(); + if (err) { + printk(KERN_ERR "xen suspend: freeze failed %d\n", err); + goto out; + } +#endif + + printk(KERN_DEBUG "suspending xenstore... "); + xenbus_suspend(); + printk(KERN_DEBUG "xenstore suspended\n"); + platform_pci_disable_irq(); + + err = stop_machine(xen_hvm_suspend, &cancelled, cpumask_of(0)); + if (err) { + printk(KERN_ERR "failed to start xen_suspend: %d\n", err); + cancelled = 1; + } + + platform_pci_enable_irq(); + + if (!cancelled) { + xen_arch_resume(); + xenbus_resume(); + } else + xs_suspend_cancel(); + + /* Make sure timer events get retriggered on all CPUs */ + clock_was_set(); + +#ifdef CONFIG_PREEMPT + thaw_processes(); +out: +#endif + shutting_down = SHUTDOWN_INVALID; +} + static void do_suspend(void) { int err; @@ -185,7 +257,10 @@ static void shutdown_handler(struct xenbus_watch *watch, ctrl_alt_del(); #ifdef CONFIG_PM_SLEEP } else if (strcmp(str, "suspend") == 0) { - do_suspend(); + if (xen_hvm_domain()) + do_hvm_suspend(); + else + do_suspend(); #endif } else { printk(KERN_INFO "Ignoring shutdown request: %s\n", str); @@ -261,7 +336,19 @@ static int shutdown_event(struct notifier_block *notifier, return NOTIFY_DONE; } -static int __init setup_shutdown_event(void) +static int __init __setup_shutdown_event(void) +{ + /* Delay initialization in the PV on HVM case */ + if (xen_hvm_domain()) + return 0; + + if (!xen_pv_domain()) + return -ENODEV; + + return xen_setup_shutdown_event(); +} + +int xen_setup_shutdown_event(void) { static struct notifier_block xenstore_notifier = { .notifier_call = shutdown_event @@ -271,4 +358,4 @@ static int __init setup_shutdown_event(void) return 0; } -subsys_initcall(setup_shutdown_event); +subsys_initcall(__setup_shutdown_event); diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c index 7a8da66..b15f809 100644 --- a/drivers/xen/platform-pci.c +++ b/drivers/xen/platform-pci.c @@ -33,6 +33,7 @@ #include <xen/xenbus.h> #include <xen/events.h> #include <xen/hvm.h> +#include <xen/xen-ops.h> #define DRV_NAME "xen-platform-pci" @@ -43,6 +44,8 @@ MODULE_LICENSE("GPL"); static unsigned long platform_mmio; static unsigned long platform_mmio_alloc; static unsigned long platform_mmiolen; +static uint64_t callback_via; +struct pci_dev *xen_platform_pdev; unsigned long alloc_xen_mmio(unsigned long len) { @@ -87,13 +90,33 @@ static int xen_allocate_irq(struct pci_dev *pdev) "xen-platform-pci", pdev); } +void platform_pci_disable_irq(void) +{ + printk(KERN_DEBUG "platform_pci_disable_irq\n"); + disable_irq(xen_platform_pdev->irq); +} + +void platform_pci_enable_irq(void) +{ + printk(KERN_DEBUG "platform_pci_enable_irq\n"); + enable_irq(xen_platform_pdev->irq); +} + +void platform_pci_resume(void) +{ + if (!xen_have_vector_callback && xen_set_callback_via(callback_via)) { + printk("platform_pci_resume failure!\n"); + return; + } +} + static int __devinit platform_pci_init(struct pci_dev *pdev, const struct pci_device_id *ent) { int i, ret; long ioaddr, iolen; long mmio_addr, mmio_len; - uint64_t callback_via; + xen_platform_pdev = pdev; i = pci_enable_device(pdev); if (i) @@ -152,6 +175,10 @@ static int __devinit platform_pci_init(struct pci_dev *pdev, ret = xenbus_probe_init(); if (ret) goto out; + ret = xen_setup_shutdown_event(); + if (ret) + goto out; + out: if (ret) { diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index dc6ed06..a679205 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -746,6 +746,34 @@ static int xenbus_dev_resume(struct device *dev) return 0; } +static int dev_suspend(struct device *dev, void *data) +{ + return xenbus_dev_suspend(dev, PMSG_SUSPEND); +} + +void xenbus_suspend(void) +{ + DPRINTK(""); + + bus_for_each_dev(&xenbus_frontend.bus, NULL, NULL, dev_suspend); + xs_suspend(); +} +EXPORT_SYMBOL_GPL(xenbus_suspend); + +static int dev_resume(struct device *dev, void *data) +{ + return xenbus_dev_resume(dev); +} + +void xenbus_resume(void) +{ + DPRINTK(""); + + xs_resume(); + bus_for_each_dev(&xenbus_frontend.bus, NULL, NULL, dev_resume); +} +EXPORT_SYMBOL_GPL(xenbus_resume); + /* A flag to determine if xenstored is ''ready'' (i.e. has started) */ int xenstored_ready = 0; diff --git a/include/xen/platform_pci.h b/include/xen/platform_pci.h index 59a120c..ced434d 100644 --- a/include/xen/platform_pci.h +++ b/include/xen/platform_pci.h @@ -31,11 +31,17 @@ #ifdef CONFIG_XEN_PLATFORM_PCI unsigned long alloc_xen_mmio(unsigned long len); +void platform_pci_resume(void); +void platform_pci_disable_irq(void); +void platform_pci_enable_irq(void); #else static inline unsigned long alloc_xen_mmio(unsigned long len) { return ~0UL; } +static inline void platform_pci_resume(void) {} +static inline void platform_pci_disable_irq(void) {} +static inline void platform_pci_enable_irq(void) {} #endif #endif /* _XEN_PLATFORM_PCI_H */ diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h index 883a21b..46bc81e 100644 --- a/include/xen/xen-ops.h +++ b/include/xen/xen-ops.h @@ -7,6 +7,7 @@ DECLARE_PER_CPU(struct vcpu_info *, xen_vcpu); void xen_pre_suspend(void); void xen_post_suspend(int suspend_cancelled); +void xen_hvm_post_suspend(int suspend_cancelled); void xen_mm_pin_all(void); void xen_mm_unpin_all(void); @@ -14,4 +15,6 @@ void xen_mm_unpin_all(void); void xen_timer_resume(void); void xen_arch_resume(void); +int xen_setup_shutdown_event(void); + #endif /* INCLUDE_XEN_OPS_H */ -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-18 10:23 UTC
[Xen-devel] [PATCH 08/12] Allow xen platform pci device to be compiled as a module
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- arch/x86/xen/enlighten.c | 3 +++ drivers/xen/events.c | 1 + drivers/xen/grant-table.c | 6 +++++- drivers/xen/manage.c | 14 +++++++++++--- drivers/xen/xenbus/xenbus_probe.c | 1 + include/xen/platform_pci.h | 18 ++++-------------- 6 files changed, 25 insertions(+), 18 deletions(-) diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index 23b8200..77ba321 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -85,7 +85,9 @@ struct shared_info xen_dummy_shared_info; void *xen_initial_gdt; int xen_have_vector_callback; +EXPORT_SYMBOL_GPL(xen_have_vector_callback); int xen_platform_pci; +EXPORT_SYMBOL_GPL(xen_platform_pci); static int unplug; /* @@ -1297,6 +1299,7 @@ int xen_set_callback_via(uint64_t via) a.value = via; return HYPERVISOR_hvm_op(HVMOP_set_param, &a); } +EXPORT_SYMBOL_GPL(xen_set_callback_via); void do_hvm_pv_evtchn_intr(void) { diff --git a/drivers/xen/events.c b/drivers/xen/events.c index cfc6d96..4840a03 100644 --- a/drivers/xen/events.c +++ b/drivers/xen/events.c @@ -695,6 +695,7 @@ void xen_hvm_evtchn_do_upcall(struct pt_regs *regs) { __xen_evtchn_do_upcall(regs); } +EXPORT_SYMBOL_GPL(xen_hvm_evtchn_do_upcall); /* Rebind a new event channel to an existing irq. */ void rebind_evtchn_irq(int evtchn, int irq) diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c index 6f5f3ba..f936d30 100644 --- a/drivers/xen/grant-table.c +++ b/drivers/xen/grant-table.c @@ -56,6 +56,9 @@ #define GNTTAB_LIST_END 0xffffffff #define GREFS_PER_GRANT_FRAME (PAGE_SIZE / sizeof(struct grant_entry)) +unsigned long (*alloc_xen_mmio_hook)(unsigned long len); +EXPORT_SYMBOL_GPL(alloc_xen_mmio_hook); + static grant_ref_t **gnttab_list; static unsigned int nr_grant_frames; static unsigned int boot_max_nr_grant_frames; @@ -514,7 +517,7 @@ int gnttab_resume(void) return gnttab_map(0, nr_grant_frames - 1); if (!hvm_pv_resume_frames) { - hvm_pv_resume_frames = alloc_xen_mmio(PAGE_SIZE * max_nr_gframes); + hvm_pv_resume_frames = alloc_xen_mmio_hook(PAGE_SIZE * max_nr_gframes); shared = ioremap(hvm_pv_resume_frames, PAGE_SIZE * max_nr_gframes); if (shared == NULL) { printk(KERN_WARNING @@ -600,6 +603,7 @@ int gnttab_init(void) kfree(gnttab_list); return -ENOMEM; } +EXPORT_SYMBOL_GPL(gnttab_init); static int __devinit __gnttab_init(void) { diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c index a73edd8..49ee52d 100644 --- a/drivers/xen/manage.c +++ b/drivers/xen/manage.c @@ -34,6 +34,13 @@ enum shutdown_state { SHUTDOWN_HALT = 4, }; +void (*platform_pci_resume_hook)(void); +EXPORT_SYMBOL_GPL(platform_pci_resume_hook); +void (*platform_pci_disable_irq_hook)(void); +EXPORT_SYMBOL_GPL(platform_pci_disable_irq_hook); +void (*platform_pci_enable_irq_hook)(void); +EXPORT_SYMBOL_GPL(platform_pci_enable_irq_hook); + /* Ignore multiple shutdown requests. */ static enum shutdown_state shutting_down = SHUTDOWN_INVALID; @@ -52,7 +59,7 @@ static int xen_hvm_suspend(void *data) if (!*cancelled) { xen_irq_resume(); - platform_pci_resume(); + platform_pci_resume_hook(); } return 0; @@ -119,7 +126,7 @@ static void do_hvm_suspend(void) printk(KERN_DEBUG "suspending xenstore... "); xenbus_suspend(); printk(KERN_DEBUG "xenstore suspended\n"); - platform_pci_disable_irq(); + platform_pci_disable_irq_hook(); err = stop_machine(xen_hvm_suspend, &cancelled, cpumask_of(0)); if (err) { @@ -127,7 +134,7 @@ static void do_hvm_suspend(void) cancelled = 1; } - platform_pci_enable_irq(); + platform_pci_enable_irq_hook(); if (!cancelled) { xen_arch_resume(); @@ -357,5 +364,6 @@ int xen_setup_shutdown_event(void) return 0; } +EXPORT_SYMBOL_GPL(xen_setup_shutdown_event); subsys_initcall(__setup_shutdown_event); diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index a679205..f83e083 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -892,6 +892,7 @@ int xenbus_probe_init(void) out_error: return err; } +EXPORT_SYMBOL_GPL(xenbus_probe_init); postcore_initcall(__xenbus_probe_init); diff --git a/include/xen/platform_pci.h b/include/xen/platform_pci.h index ced434d..c3c2527 100644 --- a/include/xen/platform_pci.h +++ b/include/xen/platform_pci.h @@ -29,19 +29,9 @@ #define XEN_IOPORT_LINUX_PRODNUM 0xffff #define XEN_IOPORT_LINUX_DRVVER ((LINUX_VERSION_CODE << 8) + 0x0) -#ifdef CONFIG_XEN_PLATFORM_PCI -unsigned long alloc_xen_mmio(unsigned long len); -void platform_pci_resume(void); -void platform_pci_disable_irq(void); -void platform_pci_enable_irq(void); -#else -static inline unsigned long alloc_xen_mmio(unsigned long len) -{ - return ~0UL; -} -static inline void platform_pci_resume(void) {} -static inline void platform_pci_disable_irq(void) {} -static inline void platform_pci_enable_irq(void) {} -#endif +extern unsigned long (*alloc_xen_mmio_hook)(unsigned long len); +extern void (*platform_pci_resume_hook)(void); +extern void (*platform_pci_disable_irq_hook)(void); +extern void (*platform_pci_enable_irq_hook)(void); #endif /* _XEN_PLATFORM_PCI_H */ -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-18 10:23 UTC
[Xen-devel] [PATCH 09/12] Fix possible NULL pointer dereference in print_IO_APIC
Make sure chip_data is not NULL before accessing it. Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- arch/x86/kernel/apic/io_apic.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c index eb2789c..c64499c 100644 --- a/arch/x86/kernel/apic/io_apic.c +++ b/arch/x86/kernel/apic/io_apic.c @@ -1732,6 +1732,8 @@ __apicdebuginit(void) print_IO_APIC(void) struct irq_pin_list *entry; cfg = desc->chip_data; + if (!cfg) + continue; entry = cfg->irq_2_pin; if (!entry) continue; -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-18 10:23 UTC
[Xen-devel] [PATCH 10/12] __setup_vector_irq: handle NULL chip_data
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- arch/x86/kernel/apic/io_apic.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c index c64499c..4d3d391 100644 --- a/arch/x86/kernel/apic/io_apic.c +++ b/arch/x86/kernel/apic/io_apic.c @@ -1269,6 +1269,8 @@ void __setup_vector_irq(int cpu) /* Mark the inuse vectors */ for_each_irq_desc(irq, desc) { cfg = desc->chip_data; + if (!cfg) + continue; /* * If it is a legacy IRQ handled by the legacy PIC, this cpu -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-18 10:23 UTC
[Xen-devel] [PATCH 11/12] Support VIRQ_TIMER and pvclock on HVM
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- arch/x86/xen/enlighten.c | 39 +++++++++++++++++++++++++++++++++++++- arch/x86/xen/time.c | 3 ++ drivers/xen/manage.c | 1 + include/xen/interface/features.h | 3 ++ 4 files changed, 45 insertions(+), 1 deletions(-) diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index 77ba321..41677fe 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -1274,6 +1274,7 @@ void init_shared_info(void) { struct xen_add_to_physmap xatp; static struct shared_info *shared_info_page = 0; + int cpu; if (!shared_info_page) shared_info_page = (struct shared_info *) alloc_bootmem_pages(PAGE_SIZE); @@ -1288,7 +1289,42 @@ void init_shared_info(void) /* Don''t do the full vcpu_info placement stuff until we have a possible map and a non-dummy shared_info. */ - per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0]; + /* This code is run at resume time so make sure all the online cpus + * have xen_vcpu properly set */ + for_each_online_cpu(cpu) + per_cpu(xen_vcpu, cpu) = &HYPERVISOR_shared_info->vcpu_info[cpu]; +} + +static void xen_hvm_setup_cpu_clockevents(void) +{ + int cpu = smp_processor_id(); + xen_setup_timer(cpu); + per_cpu(xen_vcpu, cpu) = &HYPERVISOR_shared_info->vcpu_info[cpu]; + xen_setup_cpu_clockevents(); +} + +static void init_hvm_time(void) +{ +#ifdef CONFIG_SMP + /* vector callback is needed otherwise we cannot receive interrupts + * on cpu > 0 */ + if (!xen_have_vector_callback) + return; +#endif + if (!xen_feature(XENFEAT_hvm_safe_pvclock)) { + printk(KERN_WARNING "Xen doesn''t support pvclock on HVM," + "disable pv timer\n"); + return; + } + + pv_time_ops = xen_time_ops; + x86_init.timers.timer_init = xen_time_init; + x86_init.timers.setup_percpu_clockev = x86_init_noop; + x86_cpuinit.setup_percpu_clockev = xen_hvm_setup_cpu_clockevents; + + x86_platform.calibrate_tsc = xen_tsc_khz; + x86_platform.get_wallclock = xen_get_wallclock; + x86_platform.set_wallclock = xen_set_wallclock; } int xen_set_callback_via(uint64_t via) @@ -1373,6 +1409,7 @@ void __init xen_guest_init(void) outw(unplug, XEN_IOPORT_UNPLUG); have_vcpu_info_placement = 0; x86_init.irqs.intr_init = xen_init_IRQ; + init_hvm_time(); } static int __init parse_unplug(char *arg) diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 32764b8..620e68f 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -19,6 +19,7 @@ #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <xen/xen.h> #include <xen/events.h> #include <xen/interface/xen.h> #include <xen/interface/vcpu.h> @@ -470,6 +471,8 @@ void xen_timer_resume(void) for_each_online_cpu(cpu) { if (HYPERVISOR_vcpu_op(VCPUOP_stop_periodic_timer, cpu, NULL)) BUG(); + if (xen_hvm_domain()) + xen_setup_runstate_info(cpu); } } diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c index 49ee52d..4a8af22 100644 --- a/drivers/xen/manage.c +++ b/drivers/xen/manage.c @@ -60,6 +60,7 @@ static int xen_hvm_suspend(void *data) if (!*cancelled) { xen_irq_resume(); platform_pci_resume_hook(); + xen_timer_resume(); } return 0; diff --git a/include/xen/interface/features.h b/include/xen/interface/features.h index 8ab08b9..70d2563 100644 --- a/include/xen/interface/features.h +++ b/include/xen/interface/features.h @@ -44,6 +44,9 @@ /* x86: Does this Xen host support the HVM callback vector type? */ #define XENFEAT_hvm_callback_vector 8 +/* x86: pvclock algorithm is safe to use on HVM */ +#define XENFEAT_hvm_safe_pvclock 9 + #define XENFEAT_NR_SUBMAPS 1 #endif /* __XEN_PUBLIC_FEATURES_H__ */ -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-18 10:23 UTC
[Xen-devel] [PATCH 12/12] Initialize xenbus device structs with ENODEV as default state
From: Don Dutile <ddutile@redhat.com> this way if xenbus isn''t configured in a FV xen guest, loading pv drivers (like netfront) won''t crash the guest. Signed-off-by: Don Dutile <ddutile@redhat.com> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- drivers/xen/xenbus/xenbus_probe.c | 29 +++++++++++++++++++++++++---- drivers/xen/xenbus/xenbus_probe.h | 1 + 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index f83e083..5e8dae6 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -188,6 +188,11 @@ static struct xen_bus_type xenbus_frontend = { .levels = 2, /* device/type/<id> */ .get_bus_id = frontend_bus_id, .probe = xenbus_probe_frontend, + /* + * to ensure loading pv-on-hvm drivers on FV guest + * doesn''t blow up trying to use uninit''d xenbus. + */ + .error = -ENODEV, .bus = { .name = "xen", .match = xenbus_match, @@ -352,6 +357,9 @@ int xenbus_register_driver_common(struct xenbus_driver *drv, struct module *owner, const char *mod_name) { + if (bus->error) + return bus->error; + drv->driver.name = drv->name; drv->driver.bus = &bus->bus; drv->driver.owner = owner; @@ -484,8 +492,12 @@ int xenbus_probe_node(struct xen_bus_type *bus, struct xenbus_device *xendev; size_t stringlen; char *tmpstring; + enum xenbus_state state; + + if (bus->error) + return bus->error; - enum xenbus_state state = xenbus_read_driver_state(nodename); + state = xenbus_read_driver_state(nodename); if (state != XenbusStateInitialising) { /* Device is not new, so ignore it. This can happen if a @@ -593,6 +605,9 @@ int xenbus_probe_devices(struct xen_bus_type *bus) char **dir; unsigned int i, dir_n; + if (bus->error) + return bus->error; + dir = xenbus_directory(XBT_NIL, bus->root, "", &dir_n); if (IS_ERR(dir)) return PTR_ERR(dir); @@ -636,7 +651,7 @@ void xenbus_dev_changed(const char *node, struct xen_bus_type *bus) char type[XEN_BUS_ID_SIZE]; const char *p, *root; - if (char_count(node, ''/'') < 2) + if (bus->error || char_count(node, ''/'') < 2) return; exists = xenbus_exists(XBT_NIL, node, ""); @@ -829,8 +844,8 @@ int xenbus_probe_init(void) DPRINTK(""); /* Register ourselves with the kernel bus subsystem */ - err = bus_register(&xenbus_frontend.bus); - if (err) + xenbus_frontend.error = bus_register(&xenbus_frontend.bus); + if (xenbus_frontend.error) goto out_error; err = xenbus_backend_bus_register(); @@ -923,6 +938,9 @@ static int is_device_connecting(struct device *dev, void *data) static int exists_connecting_device(struct device_driver *drv) { + if (xenbus_frontend.error) + return xenbus_frontend.error; + return bus_for_each_dev(&xenbus_frontend.bus, NULL, drv, is_device_connecting); } @@ -1002,6 +1020,9 @@ static void wait_for_devices(struct xenbus_driver *xendrv) #ifndef MODULE static int __init boot_wait_for_devices(void) { + if (!xenbus_frontend.error) + return xenbus_frontend.error; + ready_to_wait_for_devices = 1; wait_for_devices(NULL); return 0; diff --git a/drivers/xen/xenbus/xenbus_probe.h b/drivers/xen/xenbus/xenbus_probe.h index 6c5e318..15febe4 100644 --- a/drivers/xen/xenbus/xenbus_probe.h +++ b/drivers/xen/xenbus/xenbus_probe.h @@ -53,6 +53,7 @@ static inline void xenbus_backend_bus_unregister(void) {} struct xen_bus_type { char *root; + int error; unsigned int levels; int (*get_bus_id)(char bus_id[XEN_BUS_ID_SIZE], const char *nodename); int (*probe)(const char *type, const char *dir); -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Stefano, what are the particular advantages of running PVonHVM vs. traditional PV (vs pure HVM)? I''d like to update the wiki with some info about it... Thanks, Christian Am 18.05.2010 12:22, schrieb Stefano Stabellini:> Hi all, > this is the fixed, updated and rebased version of the PV on HVM series: > the series is based on 2.6.34 now and supports Xen PV frontends running > in a HVM domain, including netfront, blkfront and the VIRQ_TIMER. > > The list of bugs fixed in this update includes: xenbus drivers crashes > when xenbus is not properly initialized, a memory corruption bug in > suspend/resume and testing for the xen platform pci version and protocol > has been moved to enlighten.c (before unplugging emulated devices). > > In order to be able to use VIRQ_TIMER and to improve performances you > need a patch to Xen to implement the vector callback mechanism > for event channel delivery. > > A git tree is also available here: > > git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git > > branch name 2.6.34-pvhvm. > > Cheers, > > Stefano_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-18 17:17 UTC
[Xen-devel] Re: [PATCH 03/12] evtchn delivery on HVM
On 05/18/2010 03:22 AM, Stefano Stabellini wrote:> From: Sheng Yang <sheng@linux.intel.com> > > Set the callback to receive evtchns from Xen, using the > callback vector delivery mechanism. >Could you expand on this a little? Like, why is this desireable? What functional difference does it make? Is this patch useful in its own right, or is it just laying the groundwork for something else? Thanks, J> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > Signed-off-by: Sheng Yang <sheng@linux.intel.com> > --- > arch/x86/xen/enlighten.c | 35 +++++++++++++++++++++++++++++++++++ > drivers/xen/events.c | 31 ++++++++++++++++++++++++------- > include/xen/events.h | 3 +++ > include/xen/hvm.h | 9 +++++++++ > include/xen/interface/features.h | 3 +++ > 5 files changed, 74 insertions(+), 7 deletions(-) > > diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c > index 87a3b10..502c4f8 100644 > --- a/arch/x86/xen/enlighten.c > +++ b/arch/x86/xen/enlighten.c > @@ -37,8 +37,11 @@ > #include <xen/interface/vcpu.h> > #include <xen/interface/memory.h> > #include <xen/interface/hvm/hvm_op.h> > +#include <xen/interface/hvm/params.h> > #include <xen/features.h> > #include <xen/page.h> > +#include <xen/hvm.h> > +#include <xen/events.h> > #include <xen/hvc-console.h> > > #include <asm/paravirt.h> > @@ -79,6 +82,8 @@ struct shared_info xen_dummy_shared_info; > > void *xen_initial_gdt; > > +int xen_have_vector_callback; > + > /* > * Point at some empty memory to start with. We map the real shared_info > * page as soon as fixmap is up and running. > @@ -1279,6 +1284,31 @@ static void __init init_shared_info(void) > per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0]; > } > > +int xen_set_callback_via(uint64_t via) > +{ > + struct xen_hvm_param a; > + a.domid = DOMID_SELF; > + a.index = HVM_PARAM_CALLBACK_IRQ; > + a.value = via; > + return HYPERVISOR_hvm_op(HVMOP_set_param, &a); > +} > + > +void do_hvm_pv_evtchn_intr(void) > +{ > + xen_hvm_evtchn_do_upcall(get_irq_regs()); > +} > + > +static void xen_callback_vector(void) > +{ > + uint64_t callback_via; > + if (xen_feature(XENFEAT_hvm_callback_vector)) { > + callback_via = HVM_CALLBACK_VECTOR(X86_PLATFORM_IPI_VECTOR); > + xen_set_callback_via(callback_via); > + x86_platform_ipi_callback = do_hvm_pv_evtchn_intr; > + xen_have_vector_callback = 1; > + } > +} > + > void __init xen_guest_init(void) > { > int r; > @@ -1292,4 +1322,9 @@ void __init xen_guest_init(void) > return; > > init_shared_info(); > + > + xen_callback_vector(); > + > + have_vcpu_info_placement = 0; > + x86_init.irqs.intr_init = xen_init_IRQ; > } > diff --git a/drivers/xen/events.c b/drivers/xen/events.c > index db8f506..3523dbb 100644 > --- a/drivers/xen/events.c > +++ b/drivers/xen/events.c > @@ -36,6 +36,8 @@ > #include <asm/xen/hypercall.h> > #include <asm/xen/hypervisor.h> > > +#include <xen/xen.h> > +#include <xen/hvm.h> > #include <xen/xen-ops.h> > #include <xen/events.h> > #include <xen/interface/xen.h> > @@ -617,17 +619,13 @@ static DEFINE_PER_CPU(unsigned, xed_nesting_count); > * a bitset of words which contain pending event bits. The second > * level is a bitset of pending events themselves. > */ > -void xen_evtchn_do_upcall(struct pt_regs *regs) > +void __xen_evtchn_do_upcall(struct pt_regs *regs) > { > int cpu = get_cpu(); > - struct pt_regs *old_regs = set_irq_regs(regs); > struct shared_info *s = HYPERVISOR_shared_info; > struct vcpu_info *vcpu_info = __get_cpu_var(xen_vcpu); > unsigned count; > > - exit_idle(); > - irq_enter(); > - > do { > unsigned long pending_words; > > @@ -667,10 +665,26 @@ void xen_evtchn_do_upcall(struct pt_regs *regs) > } while(count != 1); > > out: > + > + put_cpu(); > +} > + > +void xen_evtchn_do_upcall(struct pt_regs *regs) > +{ > + struct pt_regs *old_regs = set_irq_regs(regs); > + > + exit_idle(); > + irq_enter(); > + > + __xen_evtchn_do_upcall(regs); > + > irq_exit(); > set_irq_regs(old_regs); > +} > > - put_cpu(); > +void xen_hvm_evtchn_do_upcall(struct pt_regs *regs) > +{ > + __xen_evtchn_do_upcall(regs); > } > > /* Rebind a new event channel to an existing irq. */ > @@ -947,5 +961,8 @@ void __init xen_init_IRQ(void) > for (i = 0; i < NR_EVENT_CHANNELS; i++) > mask_evtchn(i); > > - irq_ctx_init(smp_processor_id()); > + if (xen_hvm_domain()) > + native_init_IRQ(); > + else > + irq_ctx_init(smp_processor_id()); > } > diff --git a/include/xen/events.h b/include/xen/events.h > index e68d59a..868e5d6 100644 > --- a/include/xen/events.h > +++ b/include/xen/events.h > @@ -56,4 +56,7 @@ void xen_poll_irq(int irq); > /* Determine the IRQ which is bound to an event channel */ > unsigned irq_from_evtchn(unsigned int evtchn); > > +void xen_evtchn_do_upcall(struct pt_regs *regs); > +void xen_hvm_evtchn_do_upcall(struct pt_regs *regs); > + > #endif /* _XEN_EVENTS_H */ > diff --git a/include/xen/hvm.h b/include/xen/hvm.h > index 6b0d418..5940ee5 100644 > --- a/include/xen/hvm.h > +++ b/include/xen/hvm.h > @@ -3,6 +3,7 @@ > #define XEN_HVM_H__ > > #include <xen/interface/hvm/params.h> > +#include <asm/xen/hypercall.h> > > static inline int hvm_get_parameter(int idx, uint64_t *value) > { > @@ -21,4 +22,12 @@ static inline int hvm_get_parameter(int idx, uint64_t *value) > return r; > } > > +int xen_set_callback_via(uint64_t via); > +extern int xen_have_vector_callback; > + > +#define HVM_CALLBACK_VIA_TYPE_VECTOR 0x2 > +#define HVM_CALLBACK_VIA_TYPE_SHIFT 56 > +#define HVM_CALLBACK_VECTOR(x) (((uint64_t)HVM_CALLBACK_VIA_TYPE_VECTOR)<<\ > + HVM_CALLBACK_VIA_TYPE_SHIFT | (x)) > + > #endif /* XEN_HVM_H__ */ > diff --git a/include/xen/interface/features.h b/include/xen/interface/features.h > index f51b641..8ab08b9 100644 > --- a/include/xen/interface/features.h > +++ b/include/xen/interface/features.h > @@ -41,6 +41,9 @@ > /* x86: Does this Xen host support the MMU_PT_UPDATE_PRESERVE_AD hypercall? */ > #define XENFEAT_mmu_pt_update_preserve_ad 5 > > +/* x86: Does this Xen host support the HVM callback vector type? */ > +#define XENFEAT_hvm_callback_vector 8 > + > #define XENFEAT_NR_SUBMAPS 1 > > #endif /* __XEN_PUBLIC_FEATURES_H__ */ >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-18 17:27 UTC
[Xen-devel] Re: [PATCH 05/12] unplug emulated disks and nics
On 05/18/2010 03:23 AM, Stefano Stabellini wrote:> add a xen_unplug command line option to the kernel to unplug > xen emulated disks and nics. >I think it would be nice to call it something like "xen_emul_unplug" to clarify what this actually means. And is it really necessary to make it a command-line option? Can''t we unplug these things once the pv drivers are brought up? What happens if the user doesn''t specify this?> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > --- > arch/x86/xen/enlighten.c | 66 ++++++++++++++++++++++++++++++++++ > include/xen/hvm.h | 2 + > include/xen/interface/platform_pci.h | 46 +++++++++++++++++++++++ > include/xen/platform_pci.h | 32 ++++++++++++++++ > 4 files changed, 146 insertions(+), 0 deletions(-) > create mode 100644 include/xen/interface/platform_pci.h > create mode 100644 include/xen/platform_pci.h > > diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c > index 502c4f8..aac47b0 100644 > --- a/arch/x86/xen/enlighten.c > +++ b/arch/x86/xen/enlighten.c > @@ -31,6 +31,7 @@ > #include <linux/gfp.h> > > #include <xen/xen.h> > +#include <xen/platform_pci.h> > #include <xen/interface/xen.h> > #include <xen/interface/version.h> > #include <xen/interface/physdev.h> > @@ -38,6 +39,7 @@ > #include <xen/interface/memory.h> > #include <xen/interface/hvm/hvm_op.h> > #include <xen/interface/hvm/params.h> > +#include <xen/interface/platform_pci.h> > #include <xen/features.h> > #include <xen/page.h> > #include <xen/hvm.h> > @@ -83,6 +85,8 @@ struct shared_info xen_dummy_shared_info; > void *xen_initial_gdt; > > int xen_have_vector_callback; > +int xen_platform_pci; > +static int unplug; > > /* > * Point at some empty memory to start with. We map the real shared_info > @@ -1309,6 +1313,39 @@ static void xen_callback_vector(void) > } > } > > +static int __init check_platform_magic(void) >I''d prefer not to put all this in enlighten.c unless it really needs to be here. Given that all this is dependent on the Xen platform PCI device being enabled, it would probably be happy in a separate conditionally compiled file.> +{ > + short magic; > + char protocol; > + > + magic = inw(XEN_IOPORT_MAGIC); >Does this get run only once we''ve established we''re running on Xen, or could this be run in an arbitrary environment?> + if (magic != XEN_IOPORT_MAGIC_VAL) { > + printk(KERN_ERR "Xen Platform Pci: unrecognised magic value\n"); > + return -1; > + } > + > + protocol = inb(XEN_IOPORT_PROTOVER); > + > + printk(KERN_DEBUG "Xen Platform Pci: I/O protocol version %d\n", >"PCI" please. Also, is that really accurate since we''re doing random IO port stuff with no obvious connection to PCI? "Xen Platform Device" perhaps? (Though given that we have a proper fake PCI device, why all this random IO port hackery anyway?)> + protocol); > + > + switch (protocol) { > + case 1: > + outw(XEN_IOPORT_LINUX_PRODNUM, XEN_IOPORT_PRODNUM); > + outl(XEN_IOPORT_LINUX_DRVVER, XEN_IOPORT_DRVVER); > + if (inw(XEN_IOPORT_MAGIC) != XEN_IOPORT_MAGIC_VAL) { > + printk(KERN_ERR "Xen Platform: blacklisted by host\n"); > + return -3; > + } > + break; > + default: > + printk(KERN_WARNING "Xen Platform Pci: unknown I/O protocol version"); > + return -2; > + } > + > + return 0; > +} > + > void __init xen_guest_init(void) > { > int r; > @@ -1325,6 +1362,35 @@ void __init xen_guest_init(void) > > xen_callback_vector(); > > + r = check_platform_magic(); > + if (!r || (r == -1 && (unplug & UNPLUG_IGNORE))) > + xen_platform_pci = 1; > + if (xen_platform_pci && !(unplug & UNPLUG_IGNORE)) > + outw(unplug, XEN_IOPORT_UNPLUG); >What does all this do? A comment would be nice.> have_vcpu_info_placement = 0; > x86_init.irqs.intr_init = xen_init_IRQ; > } > + > +static int __init parse_unplug(char *arg) > +{ > + char *p, *q; > + > + for (p = arg; p; p = q) { > + q = strchr(arg, '',''); > + if (q) > + *q++ = ''\0''; > + if (!strcmp(p, "all")) > + unplug |= UNPLUG_ALL; > + else if (!strcmp(p, "ide-disks")) > + unplug |= UNPLUG_ALL_IDE_DISKS; > + else if (!strcmp(p, "aux-ide-disks")) > + unplug |= UNPLUG_AUX_IDE_DISKS; > + else if (!strcmp(p, "nics")) > + unplug |= UNPLUG_ALL_NICS; > + else > + printk(KERN_WARNING "unrecognised option ''%s'' " > + "in module parameter ''dev_unplug''\n", p); >"xen_unplug" (or whatever it becomes).> + } > + return 0; > +} > +early_param("xen_unplug", parse_unplug); >If we must have this kernel command line parameter, make sure you update Documentation/kernel-parameters.txt.> diff --git a/include/xen/hvm.h b/include/xen/hvm.h > index 5940ee5..777d2ce 100644 > --- a/include/xen/hvm.h > +++ b/include/xen/hvm.h > @@ -30,4 +30,6 @@ extern int xen_have_vector_callback; > #define HVM_CALLBACK_VECTOR(x) (((uint64_t)HVM_CALLBACK_VIA_TYPE_VECTOR)<<\ > HVM_CALLBACK_VIA_TYPE_SHIFT | (x)) > > +extern int xen_platform_pci; > + > #endif /* XEN_HVM_H__ */ > diff --git a/include/xen/interface/platform_pci.h b/include/xen/interface/platform_pci.h > new file mode 100644 > index 0000000..720eaf5 > --- /dev/null > +++ b/include/xen/interface/platform_pci.h > @@ -0,0 +1,46 @@ > +/****************************************************************************** > + * platform_pci.h > + * > + * Interface for granting foreign access to page frames, and receiving > + * page-ownership transfers. > + * > + * Permission is hereby granted, free of charge, to any person obtaining a copy > + * of this software and associated documentation files (the "Software"), to > + * deal in the Software without restriction, including without limitation the > + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or > + * sell copies of the Software, and to permit persons to whom the Software is > + * furnished to do so, subject to the following conditions: > + * > + * The above copyright notice and this permission notice shall be included in > + * all copies or substantial portions of the Software. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE > + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING > + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER > + * DEALINGS IN THE SOFTWARE. > + */ > + > +#ifndef __XEN_PUBLIC_PLATFORM_PCI_H__ > +#define __XEN_PUBLIC_PLATFORM_PCI_H__ > + > +#define XEN_IOPORT_BASE 0x10 > + > +#define XEN_IOPORT_PLATFLAGS (XEN_IOPORT_BASE + 0) /* 1 byte access (R/W) */ > +#define XEN_IOPORT_MAGIC (XEN_IOPORT_BASE + 0) /* 2 byte access (R) */ > +#define XEN_IOPORT_UNPLUG (XEN_IOPORT_BASE + 0) /* 2 byte access (W) */ > +#define XEN_IOPORT_DRVVER (XEN_IOPORT_BASE + 0) /* 4 byte access (W) */ > + > +#define XEN_IOPORT_SYSLOG (XEN_IOPORT_BASE + 2) /* 1 byte access (W) */ > +#define XEN_IOPORT_PROTOVER (XEN_IOPORT_BASE + 2) /* 1 byte access (R) */ > +#define XEN_IOPORT_PRODNUM (XEN_IOPORT_BASE + 2) /* 2 byte access (W) */ > + > +#define UNPLUG_ALL_IDE_DISKS 1 > +#define UNPLUG_ALL_NICS 2 > +#define UNPLUG_AUX_IDE_DISKS 4 > +#define UNPLUG_ALL 7 > +#define UNPLUG_IGNORE 8 > + > +#endif /* __XEN_PUBLIC_PLATFORM_PCI_H__ */ > diff --git a/include/xen/platform_pci.h b/include/xen/platform_pci.h > new file mode 100644 > index 0000000..f39f4d3 > --- /dev/null > +++ b/include/xen/platform_pci.h > @@ -0,0 +1,32 @@ > +/****************************************************************************** > + * platform-pci.h > + * > + * Xen platform PCI device driver > + * Copyright (c) 2004, Intel Corporation. <xiaofeng.ling@intel.com> > + * Copyright (c) 2007, XenSource Inc. > + * Copyright (c) 2010, Citrix > + * > + * This program is free software; you can redistribute it and/or modify it > + * under the terms and conditions of the GNU General Public License, > + * version 2, as published by the Free Software Foundation. > + * > + * This program is distributed in the hope it will be useful, but WITHOUT > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for > + * more details. > + * > + * You should have received a copy of the GNU General Public License along with > + * this program; if not, write to the Free Software Foundation, Inc., 59 Temple > + * Place - Suite 330, Boston, MA 02111-1307 USA. > + */ > + > +#ifndef _XEN_PLATFORM_PCI_H > +#define _XEN_PLATFORM_PCI_H > + > +#include <linux/version.h> > + > +#define XEN_IOPORT_MAGIC_VAL 0x49d2 > +#define XEN_IOPORT_LINUX_PRODNUM 0xffff > +#define XEN_IOPORT_LINUX_DRVVER ((LINUX_VERSION_CODE << 8) + 0x0) >Can''t these two headers be folded together? There doesn''t seem much point in splitting these XEN_IOPORT definitions across two files. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-18 17:43 UTC
[Xen-devel] Re: [PATCH 03/12] evtchn delivery on HVM
On 05/18/2010 03:22 AM, Stefano Stabellini wrote:> From: Sheng Yang <sheng@linux.intel.com> > > Set the callback to receive evtchns from Xen, using the > callback vector delivery mechanism. > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > Signed-off-by: Sheng Yang <sheng@linux.intel.com> > --- > arch/x86/xen/enlighten.c | 35 +++++++++++++++++++++++++++++++++++ > drivers/xen/events.c | 31 ++++++++++++++++++++++++------- > include/xen/events.h | 3 +++ > include/xen/hvm.h | 9 +++++++++ > include/xen/interface/features.h | 3 +++ > 5 files changed, 74 insertions(+), 7 deletions(-) > > diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c > index 87a3b10..502c4f8 100644 > --- a/arch/x86/xen/enlighten.c > +++ b/arch/x86/xen/enlighten.c > @@ -37,8 +37,11 @@ > #include <xen/interface/vcpu.h> > #include <xen/interface/memory.h> > #include <xen/interface/hvm/hvm_op.h> > +#include <xen/interface/hvm/params.h> > #include <xen/features.h> > #include <xen/page.h> > +#include <xen/hvm.h> > +#include <xen/events.h> > #include <xen/hvc-console.h> > > #include <asm/paravirt.h> > @@ -79,6 +82,8 @@ struct shared_info xen_dummy_shared_info; > > void *xen_initial_gdt; > > +int xen_have_vector_callback; >BTW, this can be a __read_mostly. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-18 18:10 UTC
[Xen-devel] Re: [PATCH 03/12] evtchn delivery on HVM
On 05/18/2010 03:22 AM, Stefano Stabellini wrote:> From: Sheng Yang <sheng@linux.intel.com> > > Set the callback to receive evtchns from Xen, using the > callback vector delivery mechanism. > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > Signed-off-by: Sheng Yang <sheng@linux.intel.com> > --- > arch/x86/xen/enlighten.c | 35 +++++++++++++++++++++++++++++++++++ > drivers/xen/events.c | 31 ++++++++++++++++++++++++------- > include/xen/events.h | 3 +++ > include/xen/hvm.h | 9 +++++++++ > include/xen/interface/features.h | 3 +++ > 5 files changed, 74 insertions(+), 7 deletions(-) > > diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c > index 87a3b10..502c4f8 100644 > --- a/arch/x86/xen/enlighten.c > +++ b/arch/x86/xen/enlighten.c > @@ -37,8 +37,11 @@ > #include <xen/interface/vcpu.h> > #include <xen/interface/memory.h> > #include <xen/interface/hvm/hvm_op.h> > +#include <xen/interface/hvm/params.h> > #include <xen/features.h> > #include <xen/page.h> > +#include <xen/hvm.h> > +#include <xen/events.h> > #include <xen/hvc-console.h> > > #include <asm/paravirt.h> > @@ -79,6 +82,8 @@ struct shared_info xen_dummy_shared_info; > > void *xen_initial_gdt; > > +int xen_have_vector_callback; > + > /* > * Point at some empty memory to start with. We map the real shared_info > * page as soon as fixmap is up and running. > @@ -1279,6 +1284,31 @@ static void __init init_shared_info(void) > per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0]; > } > > +int xen_set_callback_via(uint64_t via) > +{ > + struct xen_hvm_param a; > + a.domid = DOMID_SELF; > + a.index = HVM_PARAM_CALLBACK_IRQ; > + a.value = via; > + return HYPERVISOR_hvm_op(HVMOP_set_param, &a); >Does this implicitly set the vector delivery on all vcpus, current and future?> +} > + > +void do_hvm_pv_evtchn_intr(void) > +{ > + xen_hvm_evtchn_do_upcall(get_irq_regs()); > +} > + > +static void xen_callback_vector(void) >All this callback vector stuff should be in drivers/xen/events.c. It would also be good to give it a more descriptive name ("xen_set_callback_vector"?), and make it an init function.> +{ > + uint64_t callback_via; > + if (xen_feature(XENFEAT_hvm_callback_vector)) { > + callback_via = HVM_CALLBACK_VECTOR(X86_PLATFORM_IPI_VECTOR); > + xen_set_callback_via(callback_via); >Do you need to check the return value here? Can it possibly fail?> + x86_platform_ipi_callback = do_hvm_pv_evtchn_intr; > + xen_have_vector_callback = 1; > + } > +} > + > void __init xen_guest_init(void) > { > int r; > @@ -1292,4 +1322,9 @@ void __init xen_guest_init(void) > return; > > init_shared_info(); > + > + xen_callback_vector(); > + > + have_vcpu_info_placement = 0; > + x86_init.irqs.intr_init = xen_init_IRQ; > } > diff --git a/drivers/xen/events.c b/drivers/xen/events.c > index db8f506..3523dbb 100644 > --- a/drivers/xen/events.c > +++ b/drivers/xen/events.c > @@ -36,6 +36,8 @@ > #include <asm/xen/hypercall.h> > #include <asm/xen/hypervisor.h> > > +#include <xen/xen.h> > +#include <xen/hvm.h> > #include <xen/xen-ops.h> > #include <xen/events.h> > #include <xen/interface/xen.h> > @@ -617,17 +619,13 @@ static DEFINE_PER_CPU(unsigned, xed_nesting_count); > * a bitset of words which contain pending event bits. The second > * level is a bitset of pending events themselves. > */ > -void xen_evtchn_do_upcall(struct pt_regs *regs) > +void __xen_evtchn_do_upcall(struct pt_regs *regs) >Given that the regs arg is completely unused, you should drop it.> { > int cpu = get_cpu(); > - struct pt_regs *old_regs = set_irq_regs(regs); > struct shared_info *s = HYPERVISOR_shared_info; > struct vcpu_info *vcpu_info = __get_cpu_var(xen_vcpu); > unsigned count; > > - exit_idle(); > - irq_enter(); > - > do { > unsigned long pending_words; > > @@ -667,10 +665,26 @@ void xen_evtchn_do_upcall(struct pt_regs *regs) > } while(count != 1); > > out: > + > + put_cpu(); > +} > + > +void xen_evtchn_do_upcall(struct pt_regs *regs) > +{ > + struct pt_regs *old_regs = set_irq_regs(regs); > + > + exit_idle(); > + irq_enter(); > + > + __xen_evtchn_do_upcall(regs); > + > irq_exit(); > set_irq_regs(old_regs); > +} > > - put_cpu(); > +void xen_hvm_evtchn_do_upcall(struct pt_regs *regs) > +{ > + __xen_evtchn_do_upcall(regs); >Don''t you need to set_irq_regs here?> } > > /* Rebind a new event channel to an existing irq. */ > @@ -947,5 +961,8 @@ void __init xen_init_IRQ(void) > for (i = 0; i < NR_EVENT_CHANNELS; i++) > mask_evtchn(i); > > - irq_ctx_init(smp_processor_id()); > + if (xen_hvm_domain()) > + native_init_IRQ(); > + else > + irq_ctx_init(smp_processor_id()); > } > diff --git a/include/xen/events.h b/include/xen/events.h > index e68d59a..868e5d6 100644 > --- a/include/xen/events.h > +++ b/include/xen/events.h > @@ -56,4 +56,7 @@ void xen_poll_irq(int irq); > /* Determine the IRQ which is bound to an event channel */ > unsigned irq_from_evtchn(unsigned int evtchn); > > +void xen_evtchn_do_upcall(struct pt_regs *regs); > +void xen_hvm_evtchn_do_upcall(struct pt_regs *regs); > + > #endif /* _XEN_EVENTS_H */ > diff --git a/include/xen/hvm.h b/include/xen/hvm.h > index 6b0d418..5940ee5 100644 > --- a/include/xen/hvm.h > +++ b/include/xen/hvm.h > @@ -3,6 +3,7 @@ > #define XEN_HVM_H__ > > #include <xen/interface/hvm/params.h> > +#include <asm/xen/hypercall.h> > > static inline int hvm_get_parameter(int idx, uint64_t *value) > { > @@ -21,4 +22,12 @@ static inline int hvm_get_parameter(int idx, uint64_t *value) > return r; > } > > +int xen_set_callback_via(uint64_t via); > +extern int xen_have_vector_callback; > + > +#define HVM_CALLBACK_VIA_TYPE_VECTOR 0x2 > +#define HVM_CALLBACK_VIA_TYPE_SHIFT 56 > +#define HVM_CALLBACK_VECTOR(x) (((uint64_t)HVM_CALLBACK_VIA_TYPE_VECTOR)<<\ > + HVM_CALLBACK_VIA_TYPE_SHIFT | (x)) > + > #endif /* XEN_HVM_H__ */ > diff --git a/include/xen/interface/features.h b/include/xen/interface/features.h > index f51b641..8ab08b9 100644 > --- a/include/xen/interface/features.h > +++ b/include/xen/interface/features.h > @@ -41,6 +41,9 @@ > /* x86: Does this Xen host support the MMU_PT_UPDATE_PRESERVE_AD hypercall? */ > #define XENFEAT_mmu_pt_update_preserve_ad 5 > > +/* x86: Does this Xen host support the HVM callback vector type? */ > +#define XENFEAT_hvm_callback_vector 8 > + > #define XENFEAT_NR_SUBMAPS 1 > > #endif /* __XEN_PUBLIC_FEATURES_H__ */ >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-18 18:11 UTC
[Xen-devel] Re: [PATCH 06/12] xen pci platform device driver
On 05/18/2010 03:23 AM, Stefano Stabellini wrote:> Add the xen pci platform device driver that is responsible > for initializing the grant table and xenbus in PV on HVM mode. > Few changes to xenbus and grant table are necessary to allow the delayed > initialization in HVM mode. > Grant table needs few additional modifications to work in HVM mode. >This needs a description of how event and interrupt handling work in this environment.> When running on HVM the event channel upcall is never called while in > progress because it is a normal Linux irq handler, therefore we cannot > be sure that evtchn_upcall_pending is 0 when returning. >Is that because the interrupt raised by a pending event is edge-triggered, so that even if the event is still pending on return, the corresponding interrupt isn''t still asserted?> For this reason if evtchn_upcall_pending is set by Xen we need to loop > again on the event channels set pending otherwise we might loose some > event channel deliveries. >So if the event is raised after the event processing loop but before the handler returns, the corresponding interrupt is still asserted so the interrupt handler will be re-entered immediately? But if that''s true, then why is an event occurring during the loop liable to get missed?> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > Signed-off-by: Sheng Yang <sheng@linux.intel.com> > --- > drivers/xen/Kconfig | 8 ++ > drivers/xen/Makefile | 3 +- > drivers/xen/events.c | 5 +- > drivers/xen/grant-table.c | 70 +++++++++++-- > drivers/xen/platform-pci.c | 198 +++++++++++++++++++++++++++++++++++ > drivers/xen/xenbus/xenbus_probe.c | 20 +++- > include/xen/grant_table.h | 1 + > include/xen/interface/grant_table.h | 1 + > include/xen/platform_pci.h | 9 ++ > include/xen/xenbus.h | 1 + > 10 files changed, 300 insertions(+), 16 deletions(-) > create mode 100644 drivers/xen/platform-pci.c > > diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig > index fad3df2..da312e2 100644 > --- a/drivers/xen/Kconfig > +++ b/drivers/xen/Kconfig > @@ -62,4 +62,12 @@ config XEN_SYS_HYPERVISOR > virtual environment, /sys/hypervisor will still be present, > but will have no xen contents. > > +config XEN_PLATFORM_PCI > + tristate "xen platform pci device driver" > + depends on XEN > + help > + Driver for the Xen PCI Platform device: it is responsible for > + initializing xenbus and grant_table when running in a Xen HVM > + domain. As a consequence this driver is required to run any Xen PV > + frontend on Xen HVM. > endmenu > diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile > index 7c28434..e392fb7 100644 > --- a/drivers/xen/Makefile > +++ b/drivers/xen/Makefile > @@ -9,4 +9,5 @@ obj-$(CONFIG_XEN_XENCOMM) += xencomm.o > obj-$(CONFIG_XEN_BALLOON) += balloon.o > obj-$(CONFIG_XEN_DEV_EVTCHN) += evtchn.o > obj-$(CONFIG_XENFS) += xenfs/ > -obj-$(CONFIG_XEN_SYS_HYPERVISOR) += sys-hypervisor.o > \ No newline at end of file > +obj-$(CONFIG_XEN_SYS_HYPERVISOR) += sys-hypervisor.o > +obj-$(CONFIG_XEN_PLATFORM_PCI) += platform-pci.o > diff --git a/drivers/xen/events.c b/drivers/xen/events.c > index a137a2f..cfc6d96 100644 > --- a/drivers/xen/events.c > +++ b/drivers/xen/events.c > @@ -671,7 +671,7 @@ void __xen_evtchn_do_upcall(struct pt_regs *regs) > > count = __get_cpu_var(xed_nesting_count); > __get_cpu_var(xed_nesting_count) = 0; > - } while(count != 1); > + } while(count != 1 || vcpu_info->evtchn_upcall_pending); >I still don''t think I understand the need for this (or if its needed, why its correct).> > out: > > @@ -731,7 +731,8 @@ static int rebind_irq_to_cpu(unsigned irq, unsigned tcpu) > struct evtchn_bind_vcpu bind_vcpu; > int evtchn = evtchn_from_irq(irq); > > - if (!VALID_EVTCHN(evtchn)) > + if (!VALID_EVTCHN(evtchn) || > + (xen_hvm_domain() && !xen_have_vector_callback)) >A comment would be useful here. Is it that events delivered via IO APIC are always routed to vcpu 0, but PV events and vectored HVM events can be delivered to any vcpu?> return -1; > > /* Send future instances of this interrupt to other vcpu. */ > diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c > index f66db3b..6f5f3ba 100644 > --- a/drivers/xen/grant-table.c > +++ b/drivers/xen/grant-table.c > @@ -37,11 +37,14 @@ > #include <linux/slab.h> > #include <linux/vmalloc.h> > #include <linux/uaccess.h> > +#include <linux/io.h> > > #include <xen/xen.h> > #include <xen/interface/xen.h> > #include <xen/page.h> > #include <xen/grant_table.h> > +#include <xen/platform_pci.h> > +#include <xen/interface/memory.h> > #include <asm/xen/hypercall.h> > > #include <asm/pgtable.h> > @@ -59,6 +62,7 @@ static unsigned int boot_max_nr_grant_frames; > static int gnttab_free_count; > static grant_ref_t gnttab_free_head; > static DEFINE_SPINLOCK(gnttab_list_lock); > +static unsigned long hvm_pv_resume_frames; > > static struct grant_entry *shared; > > @@ -449,6 +453,30 @@ static int gnttab_map(unsigned int start_idx, unsigned int end_idx) > unsigned int nr_gframes = end_idx + 1; > int rc; > > + if (xen_hvm_domain()) { > + struct xen_add_to_physmap xatp; > + unsigned int i = end_idx; > + rc = 0; > + /* > + * Loop backwards, so that the first hypercall has the largest > + * index, ensuring that the table will grow only once. > + */ > + do { > + xatp.domid = DOMID_SELF; > + xatp.idx = i; > + xatp.space = XENMAPSPACE_grant_table; > + xatp.gpfn = (hvm_pv_resume_frames >> PAGE_SHIFT) + i; > + rc = HYPERVISOR_memory_op(XENMEM_add_to_physmap, &xatp); > + if (rc != 0) { > + printk(KERN_WARNING > + "grant table add_to_physmap failed, err=%d\n", rc); > + break; > + } > + } while (i-- > start_idx); > + > + return rc; > + } > + > frames = kmalloc(nr_gframes * sizeof(unsigned long), GFP_ATOMIC); > if (!frames) > return -ENOMEM; > @@ -476,9 +504,28 @@ static int gnttab_map(unsigned int start_idx, unsigned int end_idx) > > int gnttab_resume(void) > { > - if (max_nr_grant_frames() < nr_grant_frames) > + unsigned int max_nr_gframes; > + > + max_nr_gframes = max_nr_grant_frames(); > + if (max_nr_gframes < nr_grant_frames) > return -ENOSYS; > - return gnttab_map(0, nr_grant_frames - 1); > + > + if (xen_pv_domain()) > + return gnttab_map(0, nr_grant_frames - 1); > + > + if (!hvm_pv_resume_frames) { > + hvm_pv_resume_frames = alloc_xen_mmio(PAGE_SIZE * max_nr_gframes); >Can alloc_xen_mmio fail? Can this possibly get called with the stub version in place (returning ~0UL), and what does ioremap do if you pass that into it?> + shared = ioremap(hvm_pv_resume_frames, PAGE_SIZE * max_nr_gframes); > + if (shared == NULL) { > + printk(KERN_WARNING > + "Fail to ioremap gnttab share frames\n"); > + return -ENOMEM; > + } > + } > + > + gnttab_map(0, nr_grant_frames - 1); > + > + return 0; > } > > int gnttab_suspend(void) > @@ -505,15 +552,12 @@ static int gnttab_expand(unsigned int req_entries) > return rc; > } > > -static int __devinit gnttab_init(void) > +int gnttab_init(void) > { > int i; > unsigned int max_nr_glist_frames, nr_glist_frames; > unsigned int nr_init_grefs; > > - if (!xen_domain()) > - return -ENODEV; > - > nr_grant_frames = 1; > boot_max_nr_grant_frames = __max_nr_grant_frames(); > > @@ -557,4 +601,16 @@ static int __devinit gnttab_init(void) > return -ENOMEM; > } > > -core_initcall(gnttab_init); > +static int __devinit __gnttab_init(void) > +{ > + /* Delay grant-table initialization in the PV on HVM case */ > + if (xen_hvm_domain()) > + return 0; > + > + if (!xen_pv_domain()) > + return -ENODEV; > + > + return gnttab_init(); > +} > + > +core_initcall(__gnttab_init); > diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c > new file mode 100644 > index 0000000..7a8da66 > --- /dev/null > +++ b/drivers/xen/platform-pci.c > @@ -0,0 +1,198 @@ > +/****************************************************************************** > + * platform-pci.c > + * > + * Xen platform PCI device driver > + * Copyright (c) 2005, Intel Corporation. > + * Copyright (c) 2007, XenSource Inc. > + * Copyright (c) 2010, Citrix > + * > + * This program is free software; you can redistribute it and/or modify it > + * under the terms and conditions of the GNU General Public License, > + * version 2, as published by the Free Software Foundation. > + * > + * This program is distributed in the hope it will be useful, but WITHOUT > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for > + * more details. > + * > + * You should have received a copy of the GNU General Public License along with > + * this program; if not, write to the Free Software Foundation, Inc., 59 Temple > + * Place - Suite 330, Boston, MA 02111-1307 USA. > + * > + */ > + > +#include <asm/io.h> > + > +#include <linux/interrupt.h> > +#include <linux/module.h> > +#include <linux/pci.h> > + > +#include <xen/grant_table.h> > +#include <xen/platform_pci.h> > +#include <xen/interface/platform_pci.h> > +#include <xen/xenbus.h> > +#include <xen/events.h> > +#include <xen/hvm.h> > + > +#define DRV_NAME "xen-platform-pci" > + > +MODULE_AUTHOR("ssmith@xensource.com and stefano.stabellini@eu.citrix.com"); > +MODULE_DESCRIPTION("Xen platform PCI device"); > +MODULE_LICENSE("GPL"); > + > +static unsigned long platform_mmio; > +static unsigned long platform_mmio_alloc; > +static unsigned long platform_mmiolen; > + > +unsigned long alloc_xen_mmio(unsigned long len) > +{ > + unsigned long addr; > + > + addr = platform_mmio + platform_mmio_alloc; > + platform_mmio_alloc += len; > + BUG_ON(platform_mmio_alloc > platform_mmiolen); > + > + return addr; > +} > + > +static uint64_t get_callback_via(struct pci_dev *pdev) > +{ > + u8 pin; > + int irq; > + > + irq = pdev->irq; > + if (irq < 16) > + return irq; /* ISA IRQ */ > + > + pin = pdev->pin; > + > + /* We don''t know the GSI. Specify the PCI INTx line instead. */ > + return ((uint64_t)0x01 << 56) | /* PCI INTx identifier */ > + ((uint64_t)pci_domain_nr(pdev->bus) << 32) | > + ((uint64_t)pdev->bus->number << 16) | > + ((uint64_t)(pdev->devfn & 0xff) << 8) | > + ((uint64_t)(pin - 1) & 3); > +} > + > +static irqreturn_t do_hvm_evtchn_intr(int irq, void *dev_id) > +{ > + xen_hvm_evtchn_do_upcall(get_irq_regs()); > + return IRQ_HANDLED; > +} > + > +static int xen_allocate_irq(struct pci_dev *pdev) > +{ > + return request_irq(pdev->irq, do_hvm_evtchn_intr, > + IRQF_DISABLED | IRQF_NOBALANCING | IRQF_TRIGGER_RISING, > + "xen-platform-pci", pdev); > +} > + > +static int __devinit platform_pci_init(struct pci_dev *pdev, > + const struct pci_device_id *ent) > +{ > + int i, ret; > + long ioaddr, iolen; > + long mmio_addr, mmio_len; > + uint64_t callback_via; > + > + i = pci_enable_device(pdev); > + if (i) > + return i; > + > + ioaddr = pci_resource_start(pdev, 0); > + iolen = pci_resource_len(pdev, 0); > + > + mmio_addr = pci_resource_start(pdev, 1); > + mmio_len = pci_resource_len(pdev, 1); > + > + if (mmio_addr == 0 || ioaddr == 0) { > + dev_err(&pdev->dev, "no resources found\n"); > + ret = -ENOENT; > + } > + > + if (request_mem_region(mmio_addr, mmio_len, DRV_NAME) == NULL) { > + dev_err(&pdev->dev, "MEM I/O resource 0x%lx @ 0x%lx busy\n", > + mmio_addr, mmio_len); > + ret = -EBUSY; > + } > + > + if (request_region(ioaddr, iolen, DRV_NAME) == NULL) { > + dev_err(&pdev->dev, "I/O resource 0x%lx @ 0x%lx busy\n", > + iolen, ioaddr); > + ret = -EBUSY; > + goto out; > + } > + > + platform_mmio = mmio_addr; > + platform_mmiolen = mmio_len; > + > + if (!xen_have_vector_callback) { > + ret = xen_allocate_irq(pdev); > + if (ret) { > + printk(KERN_WARNING "request_irq failed err=%d\n", ret); > + goto out; > + } > + callback_via = get_callback_via(pdev); > + ret = xen_set_callback_via(callback_via); > + if (ret) { > + printk(KERN_WARNING > + "Unable to set the evtchn callback err=%d\n", ret); > + goto out; > + } > + } > + > + alloc_xen_mmio_hook = alloc_xen_mmio; > + platform_pci_resume_hook = platform_pci_resume; > + platform_pci_disable_irq_hook = platform_pci_disable_irq; > + platform_pci_enable_irq_hook = platform_pci_enable_irq; >What''s this _hook stuff for?> + > + ret = gnttab_init(); > + if (ret) > + goto out; > + ret = xenbus_probe_init(); > + if (ret) > + goto out; > + > +out: > + if (ret) { > + release_mem_region(mmio_addr, mmio_len); > + release_region(ioaddr, iolen); > + pci_disable_device(pdev); > + } > + > + return ret; > +} > + > +#define XEN_PLATFORM_VENDOR_ID 0x5853 > +#define XEN_PLATFORM_DEVICE_ID 0x0001 > +static struct pci_device_id platform_pci_tbl[] __devinitdata = { > + {XEN_PLATFORM_VENDOR_ID, XEN_PLATFORM_DEVICE_ID, > + PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0}, > + {0,} > +}; > + > +MODULE_DEVICE_TABLE(pci, platform_pci_tbl); > + > +static struct pci_driver platform_driver = { > + name: DRV_NAME, > + probe : platform_pci_init, > + id_table : platform_pci_tbl, > +}; > + > +static int __init platform_pci_module_init(void) > +{ > + int rc; > + > + if (!xen_platform_pci) > + return -ENODEV; > + > + rc = pci_register_driver(&platform_driver); > + if (rc) { > + printk(KERN_INFO DRV_NAME > + ": No platform pci device model found\n"); > + return rc; > + } > + return 0; > +} > + > +module_init(platform_pci_module_init); > diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c > index 0b05b62..dc6ed06 100644 > --- a/drivers/xen/xenbus/xenbus_probe.c > +++ b/drivers/xen/xenbus/xenbus_probe.c > @@ -782,16 +782,24 @@ void xenbus_probe(struct work_struct *unused) > blocking_notifier_call_chain(&xenstore_chain, 0, NULL); > } > > -static int __init xenbus_probe_init(void) > +static int __init __xenbus_probe_init(void) > +{ > + /* Delay initialization in the PV on HVM case */ > + if (xen_hvm_domain()) > + return 0; > + > + if (!xen_pv_domain()) > + return -ENODEV; > + > + return xenbus_probe_init(); > +} > + > +int xenbus_probe_init(void) > { > int err = 0; > > DPRINTK(""); > > - err = -ENODEV; > - if (!xen_domain()) > - goto out_error; > - > /* Register ourselves with the kernel bus subsystem */ > err = bus_register(&xenbus_frontend.bus); > if (err) > @@ -857,7 +865,7 @@ static int __init xenbus_probe_init(void) > return err; > } > > -postcore_initcall(xenbus_probe_init); > +postcore_initcall(__xenbus_probe_init); > > MODULE_LICENSE("GPL"); > > diff --git a/include/xen/grant_table.h b/include/xen/grant_table.h > index a40f1cd..811cda5 100644 > --- a/include/xen/grant_table.h > +++ b/include/xen/grant_table.h > @@ -51,6 +51,7 @@ struct gnttab_free_callback { > u16 count; > }; > > +int gnttab_init(void); > int gnttab_suspend(void); > int gnttab_resume(void); > > diff --git a/include/xen/interface/grant_table.h b/include/xen/interface/grant_table.h > index 39da93c..39e5717 100644 > --- a/include/xen/interface/grant_table.h > +++ b/include/xen/interface/grant_table.h > @@ -28,6 +28,7 @@ > #ifndef __XEN_PUBLIC_GRANT_TABLE_H__ > #define __XEN_PUBLIC_GRANT_TABLE_H__ > > +#include <xen/interface/xen.h> > > /*********************************** > * GRANT TABLE REPRESENTATION > diff --git a/include/xen/platform_pci.h b/include/xen/platform_pci.h > index f39f4d3..59a120c 100644 > --- a/include/xen/platform_pci.h > +++ b/include/xen/platform_pci.h > @@ -29,4 +29,13 @@ > #define XEN_IOPORT_LINUX_PRODNUM 0xffff > #define XEN_IOPORT_LINUX_DRVVER ((LINUX_VERSION_CODE << 8) + 0x0) > > +#ifdef CONFIG_XEN_PLATFORM_PCI > +unsigned long alloc_xen_mmio(unsigned long len); > +#else > +static inline unsigned long alloc_xen_mmio(unsigned long len) > +{ > + return ~0UL; > +} >Why is this stub needed?> +#endif > + > #endif /* _XEN_PLATFORM_PCI_H */ > diff --git a/include/xen/xenbus.h b/include/xen/xenbus.h > index 43e2d7d..ffa97de 100644 > --- a/include/xen/xenbus.h > +++ b/include/xen/xenbus.h > @@ -174,6 +174,7 @@ void unregister_xenbus_watch(struct xenbus_watch *watch); > void xs_suspend(void); > void xs_resume(void); > void xs_suspend_cancel(void); > +int xenbus_probe_init(void); > > /* Used by xenbus_dev to borrow kernel''s store connection. */ > void *xenbus_dev_request_and_reply(struct xsd_sockmsg *msg); >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-18 18:11 UTC
[Xen-devel] Re: [PATCH 07/12] Add suspend\resume support for PV on HVM guests.
On 05/18/2010 03:23 AM, Stefano Stabellini wrote: "/" Please describe what''s needed to support suspend/resume. Is this a normal x86 ACPI suspend/resume, or a Xen save/restore?> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > --- > arch/x86/xen/enlighten.c | 9 ++-- > arch/x86/xen/suspend.c | 6 ++ > arch/x86/xen/xen-ops.h | 3 + > drivers/xen/manage.c | 95 +++++++++++++++++++++++++++++++++++-- > drivers/xen/platform-pci.c | 29 +++++++++++- > drivers/xen/xenbus/xenbus_probe.c | 28 +++++++++++ > include/xen/platform_pci.h | 6 ++ > include/xen/xen-ops.h | 3 + > 8 files changed, 170 insertions(+), 9 deletions(-) > > diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c > index aac47b0..23b8200 100644 > --- a/arch/x86/xen/enlighten.c > +++ b/arch/x86/xen/enlighten.c > @@ -1268,12 +1268,13 @@ static int init_hvm_pv_info(int *major, int *minor) > return 0; > } > > -static void __init init_shared_info(void) > +void init_shared_info(void) > { > struct xen_add_to_physmap xatp; > - struct shared_info *shared_info_page; > + static struct shared_info *shared_info_page = 0; > > - shared_info_page = (struct shared_info *) alloc_bootmem_pages(PAGE_SIZE); > + if (!shared_info_page) > + shared_info_page = (struct shared_info *) alloc_bootmem_pages(PAGE_SIZE); > xatp.domid = DOMID_SELF; > xatp.idx = 0; > xatp.space = XENMAPSPACE_shared_info; > @@ -1302,7 +1303,7 @@ void do_hvm_pv_evtchn_intr(void) > xen_hvm_evtchn_do_upcall(get_irq_regs()); > } > > -static void xen_callback_vector(void) > +void xen_callback_vector(void) > { > uint64_t callback_via; > if (xen_feature(XENFEAT_hvm_callback_vector)) { > diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c > index 987267f..86f3b45 100644 > --- a/arch/x86/xen/suspend.c > +++ b/arch/x86/xen/suspend.c > @@ -26,6 +26,12 @@ void xen_pre_suspend(void) > BUG(); > } > > +void xen_hvm_post_suspend(int suspend_cancelled) > +{ > + init_shared_info(); > + xen_callback_vector(); > +} > + > void xen_post_suspend(int suspend_cancelled) > { > xen_build_mfn_list_list(); > diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h > index f9153a3..caf89ee 100644 > --- a/arch/x86/xen/xen-ops.h > +++ b/arch/x86/xen/xen-ops.h > @@ -38,6 +38,9 @@ void xen_enable_sysenter(void); > void xen_enable_syscall(void); > void xen_vcpu_restore(void); > > +void xen_callback_vector(void); > +void init_shared_info(void); > + > void __init xen_build_dynamic_phys_to_machine(void); > > void xen_init_irq_ops(void); > diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c > index 2ac4440..a73edd8 100644 > --- a/drivers/xen/manage.c > +++ b/drivers/xen/manage.c > @@ -8,15 +8,20 @@ > #include <linux/sysrq.h> > #include <linux/stop_machine.h> > #include <linux/freezer.h> > +#include <linux/pci.h> > +#include <linux/cpumask.h> > > +#include <xen/xen.h> > #include <xen/xenbus.h> > #include <xen/grant_table.h> > #include <xen/events.h> > #include <xen/hvc-console.h> > #include <xen/xen-ops.h> > +#include <xen/platform_pci.h> > > #include <asm/xen/hypercall.h> > #include <asm/xen/page.h> > +#include <asm/xen/hypervisor.h> > > enum shutdown_state { > SHUTDOWN_INVALID = -1, > @@ -33,10 +38,30 @@ enum shutdown_state { > static enum shutdown_state shutting_down = SHUTDOWN_INVALID; > > #ifdef CONFIG_PM_SLEEP > -static int xen_suspend(void *data) > +static int xen_hvm_suspend(void *data) > { > + struct sched_shutdown r = { .reason = SHUTDOWN_suspend }; > int *cancelled = data; > + > + BUG_ON(!irqs_disabled()); > + > + *cancelled = HYPERVISOR_sched_op(SCHEDOP_shutdown, &r); > + > + xen_hvm_post_suspend(*cancelled); > + gnttab_resume(); > + > + if (!*cancelled) { > + xen_irq_resume(); > + platform_pci_resume(); > + } > + > + return 0; > +} > + > +static int xen_suspend(void *data) > +{ > int err; > + int *cancelled = data; > > BUG_ON(!irqs_disabled()); > > @@ -73,6 +98,53 @@ static int xen_suspend(void *data) > return 0; > } > > +static void do_hvm_suspend(void) > +{ > + int err; > + int cancelled = 1; > + > + shutting_down = SHUTDOWN_SUSPEND; > + > +#ifdef CONFIG_PREEMPT > + /* If the kernel is preemptible, we need to freeze all the processes > + to prevent them from being in the middle of a pagetable update > + during suspend. */ > + err = freeze_processes(); > + if (err) { > + printk(KERN_ERR "xen suspend: freeze failed %d\n", err); > + goto out; > + } > +#endif > + > + printk(KERN_DEBUG "suspending xenstore... "); > + xenbus_suspend(); > + printk(KERN_DEBUG "xenstore suspended\n"); > + platform_pci_disable_irq(); > + > + err = stop_machine(xen_hvm_suspend, &cancelled, cpumask_of(0)); > + if (err) { > + printk(KERN_ERR "failed to start xen_suspend: %d\n", err); > + cancelled = 1; > + } > + > + platform_pci_enable_irq(); > + > + if (!cancelled) { > + xen_arch_resume(); > + xenbus_resume(); > + } else > + xs_suspend_cancel(); > + > + /* Make sure timer events get retriggered on all CPUs */ > + clock_was_set(); > + > +#ifdef CONFIG_PREEMPT > + thaw_processes(); > +out: > +#endif > + shutting_down = SHUTDOWN_INVALID; > +} > + > static void do_suspend(void) > { > int err; > @@ -185,7 +257,10 @@ static void shutdown_handler(struct xenbus_watch *watch, > ctrl_alt_del(); > #ifdef CONFIG_PM_SLEEP > } else if (strcmp(str, "suspend") == 0) { > - do_suspend(); > + if (xen_hvm_domain()) > + do_hvm_suspend(); >Why does HVM come via this path? Wouldn''t ACPI S3 be a better match for HVM? Does this make sure the full device model suspend/resume callbacks get called? Previously I think we cut corners because we knew there wouldn''t be any PCI devices in the system... And if the full device model is being used properly, then can''t all this hvm-specific stuff be done in the platform pci driver itself, rather than here? Is checkpoint the issue? (Is checkpointing hvm domains supported?)> + else > + do_suspend(); > #endif > } else { > printk(KERN_INFO "Ignoring shutdown request: %s\n", str); > @@ -261,7 +336,19 @@ static int shutdown_event(struct notifier_block *notifier, > return NOTIFY_DONE; > } > > -static int __init setup_shutdown_event(void) > +static int __init __setup_shutdown_event(void) > +{ > + /* Delay initialization in the PV on HVM case */ > + if (xen_hvm_domain()) > + return 0; > + > + if (!xen_pv_domain()) > + return -ENODEV; > + > + return xen_setup_shutdown_event(); > +} > + > +int xen_setup_shutdown_event(void) > { > static struct notifier_block xenstore_notifier = { > .notifier_call = shutdown_event > @@ -271,4 +358,4 @@ static int __init setup_shutdown_event(void) > return 0; > } > > -subsys_initcall(setup_shutdown_event); > +subsys_initcall(__setup_shutdown_event); > diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c > index 7a8da66..b15f809 100644 > --- a/drivers/xen/platform-pci.c > +++ b/drivers/xen/platform-pci.c > @@ -33,6 +33,7 @@ > #include <xen/xenbus.h> > #include <xen/events.h> > #include <xen/hvm.h> > +#include <xen/xen-ops.h> > > #define DRV_NAME "xen-platform-pci" > > @@ -43,6 +44,8 @@ MODULE_LICENSE("GPL"); > static unsigned long platform_mmio; > static unsigned long platform_mmio_alloc; > static unsigned long platform_mmiolen; > +static uint64_t callback_via; > +struct pci_dev *xen_platform_pdev; > > unsigned long alloc_xen_mmio(unsigned long len) > { > @@ -87,13 +90,33 @@ static int xen_allocate_irq(struct pci_dev *pdev) > "xen-platform-pci", pdev); > } > > +void platform_pci_disable_irq(void) >If these are non-static they need a xen_ prefix. In fact "platform_pci_" is too generic anyway, and they should all have xen_ prefixes. Aside from that, why do they need to be externally callable? Can''t the pci device''s own suspend/resume handlers do this?> +{ > + printk(KERN_DEBUG "platform_pci_disable_irq\n"); > + disable_irq(xen_platform_pdev->irq); > +} > + > +void platform_pci_enable_irq(void) > +{ > + printk(KERN_DEBUG "platform_pci_enable_irq\n"); > + enable_irq(xen_platform_pdev->irq); > +} > + > +void platform_pci_resume(void) > +{ > + if (!xen_have_vector_callback && xen_set_callback_via(callback_via)) { > + printk("platform_pci_resume failure!\n"); > + return; > + } > +} > + > static int __devinit platform_pci_init(struct pci_dev *pdev, > const struct pci_device_id *ent) > { > int i, ret; > long ioaddr, iolen; > long mmio_addr, mmio_len; > - uint64_t callback_via; > + xen_platform_pdev = pdev; > > i = pci_enable_device(pdev); > if (i) > @@ -152,6 +175,10 @@ static int __devinit platform_pci_init(struct pci_dev *pdev, > ret = xenbus_probe_init(); > if (ret) > goto out; > + ret = xen_setup_shutdown_event(); > + if (ret) > + goto out; > + > > out: > if (ret) { > diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c > index dc6ed06..a679205 100644 > --- a/drivers/xen/xenbus/xenbus_probe.c > +++ b/drivers/xen/xenbus/xenbus_probe.c > @@ -746,6 +746,34 @@ static int xenbus_dev_resume(struct device *dev) > return 0; > } > > +static int dev_suspend(struct device *dev, void *data) > +{ > + return xenbus_dev_suspend(dev, PMSG_SUSPEND); > +} > + > +void xenbus_suspend(void) > +{ > + DPRINTK(""); > + > + bus_for_each_dev(&xenbus_frontend.bus, NULL, NULL, dev_suspend); > + xs_suspend(); > +} > +EXPORT_SYMBOL_GPL(xenbus_suspend); > + > +static int dev_resume(struct device *dev, void *data) > +{ > + return xenbus_dev_resume(dev); > +} > + > +void xenbus_resume(void) > +{ > + DPRINTK(""); > + > + xs_resume(); > + bus_for_each_dev(&xenbus_frontend.bus, NULL, NULL, dev_resume); > +} > +EXPORT_SYMBOL_GPL(xenbus_resume); > + > /* A flag to determine if xenstored is ''ready'' (i.e. has started) */ > int xenstored_ready = 0; > > diff --git a/include/xen/platform_pci.h b/include/xen/platform_pci.h > index 59a120c..ced434d 100644 > --- a/include/xen/platform_pci.h > +++ b/include/xen/platform_pci.h > @@ -31,11 +31,17 @@ > > #ifdef CONFIG_XEN_PLATFORM_PCI > unsigned long alloc_xen_mmio(unsigned long len); > +void platform_pci_resume(void); > +void platform_pci_disable_irq(void); > +void platform_pci_enable_irq(void); > #else > static inline unsigned long alloc_xen_mmio(unsigned long len) > { > return ~0UL; > } > +static inline void platform_pci_resume(void) {} > +static inline void platform_pci_disable_irq(void) {} > +static inline void platform_pci_enable_irq(void) {} > #endif > > #endif /* _XEN_PLATFORM_PCI_H */ > diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h > index 883a21b..46bc81e 100644 > --- a/include/xen/xen-ops.h > +++ b/include/xen/xen-ops.h > @@ -7,6 +7,7 @@ DECLARE_PER_CPU(struct vcpu_info *, xen_vcpu); > > void xen_pre_suspend(void); > void xen_post_suspend(int suspend_cancelled); > +void xen_hvm_post_suspend(int suspend_cancelled); > > void xen_mm_pin_all(void); > void xen_mm_unpin_all(void); > @@ -14,4 +15,6 @@ void xen_mm_unpin_all(void); > void xen_timer_resume(void); > void xen_arch_resume(void); > > +int xen_setup_shutdown_event(void); > + > #endif /* INCLUDE_XEN_OPS_H */ >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-18 18:15 UTC
[Xen-devel] Re: [PATCH 08/12] Allow xen platform pci device to be compiled as a module
On 05/18/2010 03:23 AM, Stefano Stabellini wrote: All the _hook stuff looks wrong. (below)> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > --- > arch/x86/xen/enlighten.c | 3 +++ > drivers/xen/events.c | 1 + > drivers/xen/grant-table.c | 6 +++++- > drivers/xen/manage.c | 14 +++++++++++--- > drivers/xen/xenbus/xenbus_probe.c | 1 + > include/xen/platform_pci.h | 18 ++++-------------- > 6 files changed, 25 insertions(+), 18 deletions(-) > > diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c > index 23b8200..77ba321 100644 > --- a/arch/x86/xen/enlighten.c > +++ b/arch/x86/xen/enlighten.c > @@ -85,7 +85,9 @@ struct shared_info xen_dummy_shared_info; > void *xen_initial_gdt; > > int xen_have_vector_callback; > +EXPORT_SYMBOL_GPL(xen_have_vector_callback); > int xen_platform_pci; > +EXPORT_SYMBOL_GPL(xen_platform_pci); > static int unplug; > > /* > @@ -1297,6 +1299,7 @@ int xen_set_callback_via(uint64_t via) > a.value = via; > return HYPERVISOR_hvm_op(HVMOP_set_param, &a); > } > +EXPORT_SYMBOL_GPL(xen_set_callback_via); > > void do_hvm_pv_evtchn_intr(void) > { > diff --git a/drivers/xen/events.c b/drivers/xen/events.c > index cfc6d96..4840a03 100644 > --- a/drivers/xen/events.c > +++ b/drivers/xen/events.c > @@ -695,6 +695,7 @@ void xen_hvm_evtchn_do_upcall(struct pt_regs *regs) > { > __xen_evtchn_do_upcall(regs); > } > +EXPORT_SYMBOL_GPL(xen_hvm_evtchn_do_upcall); > > /* Rebind a new event channel to an existing irq. */ > void rebind_evtchn_irq(int evtchn, int irq) > diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c > index 6f5f3ba..f936d30 100644 > --- a/drivers/xen/grant-table.c > +++ b/drivers/xen/grant-table.c > @@ -56,6 +56,9 @@ > #define GNTTAB_LIST_END 0xffffffff > #define GREFS_PER_GRANT_FRAME (PAGE_SIZE / sizeof(struct grant_entry)) > > +unsigned long (*alloc_xen_mmio_hook)(unsigned long len); > +EXPORT_SYMBOL_GPL(alloc_xen_mmio_hook); > + > static grant_ref_t **gnttab_list; > static unsigned int nr_grant_frames; > static unsigned int boot_max_nr_grant_frames; > @@ -514,7 +517,7 @@ int gnttab_resume(void) > return gnttab_map(0, nr_grant_frames - 1); > > if (!hvm_pv_resume_frames) { > - hvm_pv_resume_frames = alloc_xen_mmio(PAGE_SIZE * max_nr_gframes); > + hvm_pv_resume_frames = alloc_xen_mmio_hook(PAGE_SIZE * max_nr_gframes); >This looks like it should be restructured so the pci device driver itself is doing this mapping, then calling into the grant subsystem to tell it where the mapping is.> shared = ioremap(hvm_pv_resume_frames, PAGE_SIZE * max_nr_gframes); > if (shared == NULL) { > printk(KERN_WARNING > @@ -600,6 +603,7 @@ int gnttab_init(void) > kfree(gnttab_list); > return -ENOMEM; > } > +EXPORT_SYMBOL_GPL(gnttab_init); > > static int __devinit __gnttab_init(void) > { > diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c > index a73edd8..49ee52d 100644 > --- a/drivers/xen/manage.c > +++ b/drivers/xen/manage.c > @@ -34,6 +34,13 @@ enum shutdown_state { > SHUTDOWN_HALT = 4, > }; > > +void (*platform_pci_resume_hook)(void); > +EXPORT_SYMBOL_GPL(platform_pci_resume_hook); > +void (*platform_pci_disable_irq_hook)(void); > +EXPORT_SYMBOL_GPL(platform_pci_disable_irq_hook); > +void (*platform_pci_enable_irq_hook)(void); > +EXPORT_SYMBOL_GPL(platform_pci_enable_irq_hook); >If all this _hook stuff is here to support a modular xen platform pci device, then something has gone wrong. The device should be doing this via its own suspend/resume handlers.> + > /* Ignore multiple shutdown requests. */ > static enum shutdown_state shutting_down = SHUTDOWN_INVALID; > > @@ -52,7 +59,7 @@ static int xen_hvm_suspend(void *data) > > if (!*cancelled) { > xen_irq_resume(); > - platform_pci_resume(); > + platform_pci_resume_hook(); > } > > return 0; > @@ -119,7 +126,7 @@ static void do_hvm_suspend(void) > printk(KERN_DEBUG "suspending xenstore... "); > xenbus_suspend(); > printk(KERN_DEBUG "xenstore suspended\n"); > - platform_pci_disable_irq(); > + platform_pci_disable_irq_hook(); > > err = stop_machine(xen_hvm_suspend, &cancelled, cpumask_of(0)); > if (err) { > @@ -127,7 +134,7 @@ static void do_hvm_suspend(void) > cancelled = 1; > } > > - platform_pci_enable_irq(); > + platform_pci_enable_irq_hook(); > > if (!cancelled) { > xen_arch_resume(); > @@ -357,5 +364,6 @@ int xen_setup_shutdown_event(void) > > return 0; > } > +EXPORT_SYMBOL_GPL(xen_setup_shutdown_event); > > subsys_initcall(__setup_shutdown_event); > diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c > index a679205..f83e083 100644 > --- a/drivers/xen/xenbus/xenbus_probe.c > +++ b/drivers/xen/xenbus/xenbus_probe.c > @@ -892,6 +892,7 @@ int xenbus_probe_init(void) > out_error: > return err; > } > +EXPORT_SYMBOL_GPL(xenbus_probe_init); > > postcore_initcall(__xenbus_probe_init); > > diff --git a/include/xen/platform_pci.h b/include/xen/platform_pci.h > index ced434d..c3c2527 100644 > --- a/include/xen/platform_pci.h > +++ b/include/xen/platform_pci.h > @@ -29,19 +29,9 @@ > #define XEN_IOPORT_LINUX_PRODNUM 0xffff > #define XEN_IOPORT_LINUX_DRVVER ((LINUX_VERSION_CODE << 8) + 0x0) > > -#ifdef CONFIG_XEN_PLATFORM_PCI > -unsigned long alloc_xen_mmio(unsigned long len); > -void platform_pci_resume(void); > -void platform_pci_disable_irq(void); > -void platform_pci_enable_irq(void); > -#else > -static inline unsigned long alloc_xen_mmio(unsigned long len) > -{ > - return ~0UL; > -} > -static inline void platform_pci_resume(void) {} > -static inline void platform_pci_disable_irq(void) {} > -static inline void platform_pci_enable_irq(void) {} > -#endif > +extern unsigned long (*alloc_xen_mmio_hook)(unsigned long len); > +extern void (*platform_pci_resume_hook)(void); > +extern void (*platform_pci_disable_irq_hook)(void); > +extern void (*platform_pci_enable_irq_hook)(void); > > #endif /* _XEN_PLATFORM_PCI_H */ >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-18 18:15 UTC
[Xen-devel] Re: [PATCH 09/12] Fix possible NULL pointer dereference in print_IO_APIC
On 05/18/2010 03:23 AM, Stefano Stabellini wrote:> Make sure chip_data is not NULL before accessing it. >You should clarify under what circumstances it can be legitimately NULL. J> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > --- > arch/x86/kernel/apic/io_apic.c | 2 ++ > 1 files changed, 2 insertions(+), 0 deletions(-) > > diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c > index eb2789c..c64499c 100644 > --- a/arch/x86/kernel/apic/io_apic.c > +++ b/arch/x86/kernel/apic/io_apic.c > @@ -1732,6 +1732,8 @@ __apicdebuginit(void) print_IO_APIC(void) > struct irq_pin_list *entry; > > cfg = desc->chip_data; > + if (!cfg) > + continue; > entry = cfg->irq_2_pin; > if (!entry) > continue; >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-18 18:23 UTC
[Xen-devel] Re: [PATCH 11/12] Support VIRQ_TIMER and pvclock on HVM
On 05/18/2010 03:23 AM, Stefano Stabellini wrote: Please describe what you''re doing, why its useful and how it works.> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > --- > arch/x86/xen/enlighten.c | 39 +++++++++++++++++++++++++++++++++++++- > arch/x86/xen/time.c | 3 ++ > drivers/xen/manage.c | 1 + > include/xen/interface/features.h | 3 ++ > 4 files changed, 45 insertions(+), 1 deletions(-) > > diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c > index 77ba321..41677fe 100644 > --- a/arch/x86/xen/enlighten.c > +++ b/arch/x86/xen/enlighten.c > @@ -1274,6 +1274,7 @@ void init_shared_info(void) > { > struct xen_add_to_physmap xatp; > static struct shared_info *shared_info_page = 0; > + int cpu; > > if (!shared_info_page) > shared_info_page = (struct shared_info *) alloc_bootmem_pages(PAGE_SIZE); > @@ -1288,7 +1289,42 @@ void init_shared_info(void) > > /* Don''t do the full vcpu_info placement stuff until we have a > possible map and a non-dummy shared_info. */ > - per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0]; > + /* This code is run at resume time so make sure all the online cpus > + * have xen_vcpu properly set */ >Why is this necessary? Can the vcpu structures move on resume?> + for_each_online_cpu(cpu) > + per_cpu(xen_vcpu, cpu) = &HYPERVISOR_shared_info->vcpu_info[cpu]; > +} > + > +static void xen_hvm_setup_cpu_clockevents(void) > +{ > + int cpu = smp_processor_id(); > + xen_setup_timer(cpu); > + per_cpu(xen_vcpu, cpu) = &HYPERVISOR_shared_info->vcpu_info[cpu]; > + xen_setup_cpu_clockevents(); > +} > + > +static void init_hvm_time(void) > +{ > +#ifdef CONFIG_SMP > + /* vector callback is needed otherwise we cannot receive interrupts > + * on cpu > 0 */ > + if (!xen_have_vector_callback) > + return; >Putting this check in CONFIG_SMP is more or less pointless, since effectively every kernel is built with SMP enabled now. Can you either check whether the max CPUs is 1, or just make it always depend on vector callback, even on a UP kernel/domain?> +#endif > + if (!xen_feature(XENFEAT_hvm_safe_pvclock)) { > + printk(KERN_WARNING "Xen doesn''t support pvclock on HVM," > + "disable pv timer\n"); >What does this mean? Is it asking the user to do something?> + return; > + } > + > + pv_time_ops = xen_time_ops; > + x86_init.timers.timer_init = xen_time_init; > + x86_init.timers.setup_percpu_clockev = x86_init_noop; > + x86_cpuinit.setup_percpu_clockev = xen_hvm_setup_cpu_clockevents; > + > + x86_platform.calibrate_tsc = xen_tsc_khz; > + x86_platform.get_wallclock = xen_get_wallclock; > + x86_platform.set_wallclock = xen_set_wallclock; > } > > int xen_set_callback_via(uint64_t via) > @@ -1373,6 +1409,7 @@ void __init xen_guest_init(void) > outw(unplug, XEN_IOPORT_UNPLUG); > have_vcpu_info_placement = 0; > x86_init.irqs.intr_init = xen_init_IRQ; > + init_hvm_time(); > } > > static int __init parse_unplug(char *arg) > diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c > index 32764b8..620e68f 100644 > --- a/arch/x86/xen/time.c > +++ b/arch/x86/xen/time.c > @@ -19,6 +19,7 @@ > #include <asm/xen/hypervisor.h> > #include <asm/xen/hypercall.h> > > +#include <xen/xen.h> > #include <xen/events.h> > #include <xen/interface/xen.h> > #include <xen/interface/vcpu.h> > @@ -470,6 +471,8 @@ void xen_timer_resume(void) > for_each_online_cpu(cpu) { > if (HYPERVISOR_vcpu_op(VCPUOP_stop_periodic_timer, cpu, NULL)) > BUG(); > + if (xen_hvm_domain()) > + xen_setup_runstate_info(cpu); > } > } > > diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c > index 49ee52d..4a8af22 100644 > --- a/drivers/xen/manage.c > +++ b/drivers/xen/manage.c > @@ -60,6 +60,7 @@ static int xen_hvm_suspend(void *data) > if (!*cancelled) { > xen_irq_resume(); > platform_pci_resume_hook(); > + xen_timer_resume(); > } > > return 0; > diff --git a/include/xen/interface/features.h b/include/xen/interface/features.h > index 8ab08b9..70d2563 100644 > --- a/include/xen/interface/features.h > +++ b/include/xen/interface/features.h > @@ -44,6 +44,9 @@ > /* x86: Does this Xen host support the HVM callback vector type? */ > #define XENFEAT_hvm_callback_vector 8 > > +/* x86: pvclock algorithm is safe to use on HVM */ > +#define XENFEAT_hvm_safe_pvclock 9 >Why is this needed? When is it not safe?> + > #define XENFEAT_NR_SUBMAPS 1 > > #endif /* __XEN_PUBLIC_FEATURES_H__ */ >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-18 18:28 UTC
[Xen-devel] Re: [PATCH 12/12] Initialize xenbus device structs with ENODEV as default state
On 05/18/2010 03:23 AM, Stefano Stabellini wrote:> From: Don Dutile <ddutile@redhat.com> > > this way if xenbus isn''t configured in a FV xen guest, >What does "FV" mean here? Do you mean HVM?> loading pv drivers (like netfront) won''t crash the guest. >No, this is way too hacky. Is the issue that xenbus can''t handle drivers registering with it before it has been initialized? How does pci, usb, etc bus implementations deal with this? J> Signed-off-by: Don Dutile <ddutile@redhat.com> > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > --- > drivers/xen/xenbus/xenbus_probe.c | 29 +++++++++++++++++++++++++---- > drivers/xen/xenbus/xenbus_probe.h | 1 + > 2 files changed, 26 insertions(+), 4 deletions(-) > > diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c > index f83e083..5e8dae6 100644 > --- a/drivers/xen/xenbus/xenbus_probe.c > +++ b/drivers/xen/xenbus/xenbus_probe.c > @@ -188,6 +188,11 @@ static struct xen_bus_type xenbus_frontend = { > .levels = 2, /* device/type/<id> */ > .get_bus_id = frontend_bus_id, > .probe = xenbus_probe_frontend, > + /* > + * to ensure loading pv-on-hvm drivers on FV guest > + * doesn''t blow up trying to use uninit''d xenbus. > + */ > + .error = -ENODEV, > .bus = { > .name = "xen", > .match = xenbus_match, > @@ -352,6 +357,9 @@ int xenbus_register_driver_common(struct xenbus_driver *drv, > struct module *owner, > const char *mod_name) > { > + if (bus->error) > + return bus->error; > + > drv->driver.name = drv->name; > drv->driver.bus = &bus->bus; > drv->driver.owner = owner; > @@ -484,8 +492,12 @@ int xenbus_probe_node(struct xen_bus_type *bus, > struct xenbus_device *xendev; > size_t stringlen; > char *tmpstring; > + enum xenbus_state state; > + > + if (bus->error) > + return bus->error; > > - enum xenbus_state state = xenbus_read_driver_state(nodename); > + state = xenbus_read_driver_state(nodename); > > if (state != XenbusStateInitialising) { > /* Device is not new, so ignore it. This can happen if a > @@ -593,6 +605,9 @@ int xenbus_probe_devices(struct xen_bus_type *bus) > char **dir; > unsigned int i, dir_n; > > + if (bus->error) > + return bus->error; > + > dir = xenbus_directory(XBT_NIL, bus->root, "", &dir_n); > if (IS_ERR(dir)) > return PTR_ERR(dir); > @@ -636,7 +651,7 @@ void xenbus_dev_changed(const char *node, struct xen_bus_type *bus) > char type[XEN_BUS_ID_SIZE]; > const char *p, *root; > > - if (char_count(node, ''/'') < 2) > + if (bus->error || char_count(node, ''/'') < 2) > return; > > exists = xenbus_exists(XBT_NIL, node, ""); > @@ -829,8 +844,8 @@ int xenbus_probe_init(void) > DPRINTK(""); > > /* Register ourselves with the kernel bus subsystem */ > - err = bus_register(&xenbus_frontend.bus); > - if (err) > + xenbus_frontend.error = bus_register(&xenbus_frontend.bus); > + if (xenbus_frontend.error) > goto out_error; > > err = xenbus_backend_bus_register(); > @@ -923,6 +938,9 @@ static int is_device_connecting(struct device *dev, void *data) > > static int exists_connecting_device(struct device_driver *drv) > { > + if (xenbus_frontend.error) > + return xenbus_frontend.error; > + > return bus_for_each_dev(&xenbus_frontend.bus, NULL, drv, > is_device_connecting); > } > @@ -1002,6 +1020,9 @@ static void wait_for_devices(struct xenbus_driver *xendrv) > #ifndef MODULE > static int __init boot_wait_for_devices(void) > { > + if (!xenbus_frontend.error) > + return xenbus_frontend.error; > + > ready_to_wait_for_devices = 1; > wait_for_devices(NULL); > return 0; > diff --git a/drivers/xen/xenbus/xenbus_probe.h b/drivers/xen/xenbus/xenbus_probe.h > index 6c5e318..15febe4 100644 > --- a/drivers/xen/xenbus/xenbus_probe.h > +++ b/drivers/xen/xenbus/xenbus_probe.h > @@ -53,6 +53,7 @@ static inline void xenbus_backend_bus_unregister(void) {} > struct xen_bus_type > { > char *root; > + int error; > unsigned int levels; > int (*get_bus_id)(char bus_id[XEN_BUS_ID_SIZE], const char *nodename); > int (*probe)(const char *type, const char *dir); >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-18 18:41 UTC
Re: [Xen-devel] Re: [PATCH 0 of 12] PV on HVM Xen
On 05/18/2010 03:55 AM, Christian Tramnitz wrote:> Hi Stefano, > > what are the particular advantages of running PVonHVM vs. traditional > PV (vs pure HVM)? > I''d like to update the wiki with some info about it...That''s an interesting question. It probably has two dimensions: ease of deployment and performance. Some users may just find it easier to deploy a guest image in pure HVM mode since its most similar to the native environment (though with pvgrub these differences are fairly small). Performance is more complex, as it depends on both your workload and the capabilities of the processor you''re running on. So, try it and see ;) J> > > > Thanks, > Christian > > Am 18.05.2010 12:22, schrieb Stefano Stabellini: >> Hi all, >> this is the fixed, updated and rebased version of the PV on HVM series: >> the series is based on 2.6.34 now and supports Xen PV frontends running >> in a HVM domain, including netfront, blkfront and the VIRQ_TIMER. >> >> The list of bugs fixed in this update includes: xenbus drivers crashes >> when xenbus is not properly initialized, a memory corruption bug in >> suspend/resume and testing for the xen platform pci version and protocol >> has been moved to enlighten.c (before unplugging emulated devices). >> >> In order to be able to use VIRQ_TIMER and to improve performances you >> need a patch to Xen to implement the vector callback mechanism >> for event channel delivery. >> >> A git tree is also available here: >> >> git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git >> >> branch name 2.6.34-pvhvm. >> >> Cheers, >> >> Stefano > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-19 12:24 UTC
[Xen-devel] Re: [PATCH 03/12] evtchn delivery on HVM
On Tue, 18 May 2010, Jeremy Fitzhardinge wrote:> On 05/18/2010 03:22 AM, Stefano Stabellini wrote: > > From: Sheng Yang <sheng@linux.intel.com> > > > > Set the callback to receive evtchns from Xen, using the > > callback vector delivery mechanism. > > > > Could you expand on this a little? Like, why is this desireable? What > functional difference does it make? Is this patch useful in its own > right, or is it just laying the groundwork for something else? >In order to use PV frontends on HVM we need to receive notifications on event channel deliveries somehow. Using the callback vector is the preferred way, because it is available independently from any (emulated) PCI device, all the vcpus can receive these callbacks and theoretically there is no need to interact with the emulated lapic (even though at the moment we are doing it anyway because we are using the IPI vector). The other way is to receive interrupts from the xen platform pci device, but in that case interaction with the emulated lapic is unavoidable and we are limited to receive interrupts on vcpu 0. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-19 13:00 UTC
[Xen-devel] Re: [PATCH 05/12] unplug emulated disks and nics
On Tue, 18 May 2010, Jeremy Fitzhardinge wrote:> On 05/18/2010 03:23 AM, Stefano Stabellini wrote: > > add a xen_unplug command line option to the kernel to unplug > > xen emulated disks and nics. > > > > I think it would be nice to call it something like "xen_emul_unplug" to > clarify what this actually means. And is it really necessary to make it > a command-line option? Can''t we unplug these things once the pv drivers > are brought up? What happens if the user doesn''t specify this? >We have to do it before PCI is initialized to make sure the unplug doesn''t create any issues, for this reason we cannot wait for the PV frontends to come up. Also the PV frontends might be compiled as modules or even missing so we cannot just unplug any emulated device as soon as we detect that we are running on Xen. As a conclusion we thought that deferring the decision to the user might be the safest way of doing it. If the user doesn''t specify this, both the PV frontends and the drivers for the emulated network card and disk could come up, and this might lead to data corruptions (especially in the disk case). Probably I should prevent the xen platform pci driver from loading (that in turn would prevent the PV frontends from loading) if the option is not specified. s/xen_unplug/xen_emul_unplug is fine by me.> > +static int __init check_platform_magic(void) > > > > I''d prefer not to put all this in enlighten.c unless it really needs to > be here. Given that all this is dependent on the Xen platform PCI > device being enabled, it would probably be happy in a separate > conditionally compiled file.OK> > > +{ > > + short magic; > > + char protocol; > > + > > + magic = inw(XEN_IOPORT_MAGIC); > > > > Does this get run only once we''ve established we''re running on Xen, or > could this be run in an arbitrary environment? >only once we know we are running on Xen> > + if (magic != XEN_IOPORT_MAGIC_VAL) { > > + printk(KERN_ERR "Xen Platform Pci: unrecognised magic value\n"); > > + return -1; > > + } > > + > > + protocol = inb(XEN_IOPORT_PROTOVER); > > + > > + printk(KERN_DEBUG "Xen Platform Pci: I/O protocol version %d\n", > > > > "PCI" please. Also, is that really accurate since we''re doing random IO > port stuff with no obvious connection to PCI? "Xen Platform Device" > perhaps? (Though given that we have a proper fake PCI device, why all > this random IO port hackery anyway?) > >That''s because the ioports even though they are static, they still belong to the Xen platform PCI device. The ioports are static so that they can be probed before the PCI bus is even initialized.> > + protocol); > > + > > + switch (protocol) { > > + case 1: > > + outw(XEN_IOPORT_LINUX_PRODNUM, XEN_IOPORT_PRODNUM); > > + outl(XEN_IOPORT_LINUX_DRVVER, XEN_IOPORT_DRVVER); > > + if (inw(XEN_IOPORT_MAGIC) != XEN_IOPORT_MAGIC_VAL) { > > + printk(KERN_ERR "Xen Platform: blacklisted by host\n"); > > + return -3; > > + } > > + break; > > + default: > > + printk(KERN_WARNING "Xen Platform Pci: unknown I/O protocol version"); > > + return -2; > > + } > > + > > + return 0; > > +} > > + > > void __init xen_guest_init(void) > > { > > int r; > > @@ -1325,6 +1362,35 @@ void __init xen_guest_init(void) > > > > xen_callback_vector(); > > > > + r = check_platform_magic(); > > + if (!r || (r == -1 && (unplug & UNPLUG_IGNORE))) > > + xen_platform_pci = 1; > > + if (xen_platform_pci && !(unplug & UNPLUG_IGNORE)) > > + outw(unplug, XEN_IOPORT_UNPLUG); > > > > What does all this do? A comment would be nice. >OK. The idea is that xen_platform_pci is set to 1 (meaning: the xen platform pci device driver is enabled) if the xen platform protocol version matches our check or if it doesn''t match because we are running on a very old version of Xen and the user told us to continue anyway. After my comment above on preventing the PV frontends from loading if no unplug is done, I think the whole thing should be changed to: /* check the version of the xen platform PCI device */ r = check_platform_magic(); /* If the version matches and the user told us to unplug the emulated * devices, enable the Xen platform PCI driver. * Also enable the Xen platform PCI driver if the version is really old * and the user told us to ignore it. */ if (!r && unplug || (r == -1 && (unplug & UNPLUG_IGNORE))) xen_platform_pci = 1; /* Now unplug the emulated devices */ if (xen_platform_pci && !(unplug & UNPLUG_IGNORE)) outw(unplug, XEN_IOPORT_UNPLUG);> > have_vcpu_info_placement = 0; > > x86_init.irqs.intr_init = xen_init_IRQ; > > } > > + > > +static int __init parse_unplug(char *arg) > > +{ > > + char *p, *q; > > + > > + for (p = arg; p; p = q) { > > + q = strchr(arg, '',''); > > + if (q) > > + *q++ = ''\0''; > > + if (!strcmp(p, "all")) > > + unplug |= UNPLUG_ALL; > > + else if (!strcmp(p, "ide-disks")) > > + unplug |= UNPLUG_ALL_IDE_DISKS; > > + else if (!strcmp(p, "aux-ide-disks")) > > + unplug |= UNPLUG_AUX_IDE_DISKS; > > + else if (!strcmp(p, "nics")) > > + unplug |= UNPLUG_ALL_NICS; > > + else > > + printk(KERN_WARNING "unrecognised option ''%s'' " > > + "in module parameter ''dev_unplug''\n", p); > > > > "xen_unplug" (or whatever it becomes).OK> > > + } > > + return 0; > > +} > > +early_param("xen_unplug", parse_unplug); > > > > If we must have this kernel command line parameter, make sure you update > Documentation/kernel-parameters.txt. >Sure> > diff --git a/include/xen/hvm.h b/include/xen/hvm.h > > index 5940ee5..777d2ce 100644 > > --- a/include/xen/hvm.h > > +++ b/include/xen/hvm.h > > @@ -30,4 +30,6 @@ extern int xen_have_vector_callback; > > #define HVM_CALLBACK_VECTOR(x) (((uint64_t)HVM_CALLBACK_VIA_TYPE_VECTOR)<<\ > > HVM_CALLBACK_VIA_TYPE_SHIFT | (x)) > > > > +extern int xen_platform_pci; > > + > > #endif /* XEN_HVM_H__ */ > > diff --git a/include/xen/interface/platform_pci.h b/include/xen/interface/platform_pci.h > > new file mode 100644 > > index 0000000..720eaf5 > > --- /dev/null > > +++ b/include/xen/interface/platform_pci.h > > @@ -0,0 +1,46 @@ > > +/****************************************************************************** > > + * platform_pci.h > > + * > > + * Interface for granting foreign access to page frames, and receiving > > + * page-ownership transfers. > > + * > > + * Permission is hereby granted, free of charge, to any person obtaining a copy > > + * of this software and associated documentation files (the "Software"), to > > + * deal in the Software without restriction, including without limitation the > > + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or > > + * sell copies of the Software, and to permit persons to whom the Software is > > + * furnished to do so, subject to the following conditions: > > + * > > + * The above copyright notice and this permission notice shall be included in > > + * all copies or substantial portions of the Software. > > + * > > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > > + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > > + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE > > + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > > + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING > > + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER > > + * DEALINGS IN THE SOFTWARE. > > + */ > > + > > +#ifndef __XEN_PUBLIC_PLATFORM_PCI_H__ > > +#define __XEN_PUBLIC_PLATFORM_PCI_H__ > > + > > +#define XEN_IOPORT_BASE 0x10 > > + > > +#define XEN_IOPORT_PLATFLAGS (XEN_IOPORT_BASE + 0) /* 1 byte access (R/W) */ > > +#define XEN_IOPORT_MAGIC (XEN_IOPORT_BASE + 0) /* 2 byte access (R) */ > > +#define XEN_IOPORT_UNPLUG (XEN_IOPORT_BASE + 0) /* 2 byte access (W) */ > > +#define XEN_IOPORT_DRVVER (XEN_IOPORT_BASE + 0) /* 4 byte access (W) */ > > + > > +#define XEN_IOPORT_SYSLOG (XEN_IOPORT_BASE + 2) /* 1 byte access (W) */ > > +#define XEN_IOPORT_PROTOVER (XEN_IOPORT_BASE + 2) /* 1 byte access (R) */ > > +#define XEN_IOPORT_PRODNUM (XEN_IOPORT_BASE + 2) /* 2 byte access (W) */ > > + > > +#define UNPLUG_ALL_IDE_DISKS 1 > > +#define UNPLUG_ALL_NICS 2 > > +#define UNPLUG_AUX_IDE_DISKS 4 > > +#define UNPLUG_ALL 7 > > +#define UNPLUG_IGNORE 8 > > + > > +#endif /* __XEN_PUBLIC_PLATFORM_PCI_H__ */ > > diff --git a/include/xen/platform_pci.h b/include/xen/platform_pci.h > > new file mode 100644 > > index 0000000..f39f4d3 > > --- /dev/null > > +++ b/include/xen/platform_pci.h > > @@ -0,0 +1,32 @@ > > +/****************************************************************************** > > + * platform-pci.h > > + * > > + * Xen platform PCI device driver > > + * Copyright (c) 2004, Intel Corporation. <xiaofeng.ling@intel.com> > > + * Copyright (c) 2007, XenSource Inc. > > + * Copyright (c) 2010, Citrix > > + * > > + * This program is free software; you can redistribute it and/or modify it > > + * under the terms and conditions of the GNU General Public License, > > + * version 2, as published by the Free Software Foundation. > > + * > > + * This program is distributed in the hope it will be useful, but WITHOUT > > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > > + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for > > + * more details. > > + * > > + * You should have received a copy of the GNU General Public License along with > > + * this program; if not, write to the Free Software Foundation, Inc., 59 Temple > > + * Place - Suite 330, Boston, MA 02111-1307 USA. > > + */ > > + > > +#ifndef _XEN_PLATFORM_PCI_H > > +#define _XEN_PLATFORM_PCI_H > > + > > +#include <linux/version.h> > > + > > +#define XEN_IOPORT_MAGIC_VAL 0x49d2 > > +#define XEN_IOPORT_LINUX_PRODNUM 0xffff > > +#define XEN_IOPORT_LINUX_DRVVER ((LINUX_VERSION_CODE << 8) + 0x0) > > > > Can''t these two headers be folded together? There doesn''t seem much > point in splitting these XEN_IOPORT definitions across two files. >One is supposed to be a common header file shared with the hypervisor (actually the device model in this case), the other is the Xen platform PCI driver header file. I don''t think they should be merged. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-19 13:01 UTC
[Xen-devel] Re: [PATCH 03/12] evtchn delivery on HVM
On Tue, 18 May 2010, Jeremy Fitzhardinge wrote: oid *xen_initial_gdt;> > > > +int xen_have_vector_callback; > > > > BTW, this can be a __read_mostly. >I''ll do that, thanks. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-19 13:08 UTC
[Xen-devel] Re: [PATCH 03/12] evtchn delivery on HVM
On Tue, 18 May 2010, Jeremy Fitzhardinge wrote:> On 05/18/2010 03:22 AM, Stefano Stabellini wrote: > > From: Sheng Yang <sheng@linux.intel.com> > > > > Set the callback to receive evtchns from Xen, using the > > callback vector delivery mechanism. > > > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > > Signed-off-by: Sheng Yang <sheng@linux.intel.com> > > --- > > arch/x86/xen/enlighten.c | 35 +++++++++++++++++++++++++++++++++++ > > drivers/xen/events.c | 31 ++++++++++++++++++++++++------- > > include/xen/events.h | 3 +++ > > include/xen/hvm.h | 9 +++++++++ > > include/xen/interface/features.h | 3 +++ > > 5 files changed, 74 insertions(+), 7 deletions(-) > > > > diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c > > index 87a3b10..502c4f8 100644 > > --- a/arch/x86/xen/enlighten.c > > +++ b/arch/x86/xen/enlighten.c > > @@ -37,8 +37,11 @@ > > #include <xen/interface/vcpu.h> > > #include <xen/interface/memory.h> > > #include <xen/interface/hvm/hvm_op.h> > > +#include <xen/interface/hvm/params.h> > > #include <xen/features.h> > > #include <xen/page.h> > > +#include <xen/hvm.h> > > +#include <xen/events.h> > > #include <xen/hvc-console.h> > > > > #include <asm/paravirt.h> > > @@ -79,6 +82,8 @@ struct shared_info xen_dummy_shared_info; > > > > void *xen_initial_gdt; > > > > +int xen_have_vector_callback; > > + > > /* > > * Point at some empty memory to start with. We map the real shared_info > > * page as soon as fixmap is up and running. > > @@ -1279,6 +1284,31 @@ static void __init init_shared_info(void) > > per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0]; > > } > > > > +int xen_set_callback_via(uint64_t via) > > +{ > > + struct xen_hvm_param a; > > + a.domid = DOMID_SELF; > > + a.index = HVM_PARAM_CALLBACK_IRQ; > > + a.value = via; > > + return HYPERVISOR_hvm_op(HVMOP_set_param, &a); > > > > Does this implicitly set the vector delivery on all vcpus, current and > future? >Yes.> > +} > > + > > +void do_hvm_pv_evtchn_intr(void) > > +{ > > + xen_hvm_evtchn_do_upcall(get_irq_regs()); > > +} > > + > > +static void xen_callback_vector(void) > > > > All this callback vector stuff should be in drivers/xen/events.c. It > would also be good to give it a more descriptive name > ("xen_set_callback_vector"?), and make it an init function. >I could move it events.c and call it at the beginning of xen_init_IRQ, is that OK?> > +{ > > + uint64_t callback_via; > > + if (xen_feature(XENFEAT_hvm_callback_vector)) { > > + callback_via = HVM_CALLBACK_VECTOR(X86_PLATFORM_IPI_VECTOR); > > + xen_set_callback_via(callback_via); > > > > Do you need to check the return value here? Can it possibly fail? >Yes, it can fail. The vector delivery mechanism hasn''t been checked in Xen yet (I sent the patch right after this patch series).> > + x86_platform_ipi_callback = do_hvm_pv_evtchn_intr; > > + xen_have_vector_callback = 1; > > + } > > +} > > + > > void __init xen_guest_init(void) > > { > > int r; > > @@ -1292,4 +1322,9 @@ void __init xen_guest_init(void) > > return; > > > > init_shared_info(); > > + > > + xen_callback_vector(); > > + > > + have_vcpu_info_placement = 0; > > + x86_init.irqs.intr_init = xen_init_IRQ; > > } > > diff --git a/drivers/xen/events.c b/drivers/xen/events.c > > index db8f506..3523dbb 100644 > > --- a/drivers/xen/events.c > > +++ b/drivers/xen/events.c > > @@ -36,6 +36,8 @@ > > #include <asm/xen/hypercall.h> > > #include <asm/xen/hypervisor.h> > > > > +#include <xen/xen.h> > > +#include <xen/hvm.h> > > #include <xen/xen-ops.h> > > #include <xen/events.h> > > #include <xen/interface/xen.h> > > @@ -617,17 +619,13 @@ static DEFINE_PER_CPU(unsigned, xed_nesting_count); > > * a bitset of words which contain pending event bits. The second > > * level is a bitset of pending events themselves. > > */ > > -void xen_evtchn_do_upcall(struct pt_regs *regs) > > +void __xen_evtchn_do_upcall(struct pt_regs *regs) > > > > Given that the regs arg is completely unused, you should drop it. >OK> > { > > int cpu = get_cpu(); > > - struct pt_regs *old_regs = set_irq_regs(regs); > > struct shared_info *s = HYPERVISOR_shared_info; > > struct vcpu_info *vcpu_info = __get_cpu_var(xen_vcpu); > > unsigned count; > > > > - exit_idle(); > > - irq_enter(); > > - > > do { > > unsigned long pending_words; > > > > @@ -667,10 +665,26 @@ void xen_evtchn_do_upcall(struct pt_regs *regs) > > } while(count != 1); > > > > out: > > + > > + put_cpu(); > > +} > > + > > +void xen_evtchn_do_upcall(struct pt_regs *regs) > > +{ > > + struct pt_regs *old_regs = set_irq_regs(regs); > > + > > + exit_idle(); > > + irq_enter(); > > + > > + __xen_evtchn_do_upcall(regs); > > + > > irq_exit(); > > set_irq_regs(old_regs); > > +} > > > > - put_cpu(); > > +void xen_hvm_evtchn_do_upcall(struct pt_regs *regs) > > +{ > > + __xen_evtchn_do_upcall(regs); > > > > Don''t you need to set_irq_regs here?No, that was done by smp_x86_platform_ipi or do_IRQ. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-19 13:50 UTC
[Xen-devel] Re: [PATCH 06/12] xen pci platform device driver
On Tue, 18 May 2010, Jeremy Fitzhardinge wrote:> On 05/18/2010 03:23 AM, Stefano Stabellini wrote: > > Add the xen pci platform device driver that is responsible > > for initializing the grant table and xenbus in PV on HVM mode. > > Few changes to xenbus and grant table are necessary to allow the delayed > > initialization in HVM mode. > > Grant table needs few additional modifications to work in HVM mode. > > > > This needs a description of how event and interrupt handling work in > this environment. >event channel handling is still done by __xen_evtchn_do_upcall, however every time Xen raises an event channel Xen also notifies us injecting an interrupt (from the Xen platform PCI device) or using the callback vector. Both the callback handler and the Xen platform PCI interrupt handler calls __xen_evtchn_do_upcall.> > When running on HVM the event channel upcall is never called while in > > progress because it is a normal Linux irq handler, therefore we cannot > > be sure that evtchn_upcall_pending is 0 when returning. > > > > Is that because the interrupt raised by a pending event is > edge-triggered, so that even if the event is still pending on return, > the corresponding interrupt isn''t still asserted? >The interrupt is level triggered and it is handled by the fasteoi handler in Linux, but edge or level don''t mean much here because it is just an emulated interrupt. If you look at handle_fasteoi_irq you''ll notice that if the same interrupt is IRQ_INPROGRESS it just goes out.> > For this reason if evtchn_upcall_pending is set by Xen we need to loop > > again on the event channels set pending otherwise we might loose some > > event channel deliveries. > > > > So if the event is raised after the event processing loop but before the > handler returns, the corresponding interrupt is still asserted so the > interrupt handler will be re-entered immediately? But if that''s true, > then why is an event occurring during the loop liable to get missed? >If an event is raised after the event processing loop but before the handler returns, we have interrupts disabled at the time so we never even get to handle_fasteoi_irq. Once we renable interrupts we''ll process the interrupt and the event as expected. The problem is when we receive interrupts while we are processing an event with interrupts enabled, in that case we would receive anther interrupt, we''ll go to handle_fasteoi_irq and out without resetting evtchn_upcall_pending. In this case Xen will not raise other interrupts, and we will not read again evtchn_upcall_pending (because we expect it to be 0), so events would be lost.> > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > > Signed-off-by: Sheng Yang <sheng@linux.intel.com> > > --- > > drivers/xen/Kconfig | 8 ++ > > drivers/xen/Makefile | 3 +- > > drivers/xen/events.c | 5 +- > > drivers/xen/grant-table.c | 70 +++++++++++-- > > drivers/xen/platform-pci.c | 198 +++++++++++++++++++++++++++++++++++ > > drivers/xen/xenbus/xenbus_probe.c | 20 +++- > > include/xen/grant_table.h | 1 + > > include/xen/interface/grant_table.h | 1 + > > include/xen/platform_pci.h | 9 ++ > > include/xen/xenbus.h | 1 + > > 10 files changed, 300 insertions(+), 16 deletions(-) > > create mode 100644 drivers/xen/platform-pci.c > > > > diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig > > index fad3df2..da312e2 100644 > > --- a/drivers/xen/Kconfig > > +++ b/drivers/xen/Kconfig > > @@ -62,4 +62,12 @@ config XEN_SYS_HYPERVISOR > > virtual environment, /sys/hypervisor will still be present, > > but will have no xen contents. > > > > +config XEN_PLATFORM_PCI > > + tristate "xen platform pci device driver" > > + depends on XEN > > + help > > + Driver for the Xen PCI Platform device: it is responsible for > > + initializing xenbus and grant_table when running in a Xen HVM > > + domain. As a consequence this driver is required to run any Xen PV > > + frontend on Xen HVM. > > endmenu > > diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile > > index 7c28434..e392fb7 100644 > > --- a/drivers/xen/Makefile > > +++ b/drivers/xen/Makefile > > @@ -9,4 +9,5 @@ obj-$(CONFIG_XEN_XENCOMM) += xencomm.o > > obj-$(CONFIG_XEN_BALLOON) += balloon.o > > obj-$(CONFIG_XEN_DEV_EVTCHN) += evtchn.o > > obj-$(CONFIG_XENFS) += xenfs/ > > -obj-$(CONFIG_XEN_SYS_HYPERVISOR) += sys-hypervisor.o > > \ No newline at end of file > > +obj-$(CONFIG_XEN_SYS_HYPERVISOR) += sys-hypervisor.o > > +obj-$(CONFIG_XEN_PLATFORM_PCI) += platform-pci.o > > diff --git a/drivers/xen/events.c b/drivers/xen/events.c > > index a137a2f..cfc6d96 100644 > > --- a/drivers/xen/events.c > > +++ b/drivers/xen/events.c > > @@ -671,7 +671,7 @@ void __xen_evtchn_do_upcall(struct pt_regs *regs) > > > > count = __get_cpu_var(xed_nesting_count); > > __get_cpu_var(xed_nesting_count) = 0; > > - } while(count != 1); > > + } while(count != 1 || vcpu_info->evtchn_upcall_pending); > > > > I still don''t think I understand the need for this (or if its needed, > why its correct). >see above> > > > out: > > > > @@ -731,7 +731,8 @@ static int rebind_irq_to_cpu(unsigned irq, unsigned tcpu) > > struct evtchn_bind_vcpu bind_vcpu; > > int evtchn = evtchn_from_irq(irq); > > > > - if (!VALID_EVTCHN(evtchn)) > > + if (!VALID_EVTCHN(evtchn) || > > + (xen_hvm_domain() && !xen_have_vector_callback)) > > > > A comment would be useful here. Is it that events delivered via IO APIC > are always routed to vcpu 0, but PV events and vectored HVM events can > be delivered to any vcpu? >Yes, exactly.> > return -1; > > > > /* Send future instances of this interrupt to other vcpu. */ > > diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c > > index f66db3b..6f5f3ba 100644 > > --- a/drivers/xen/grant-table.c > > +++ b/drivers/xen/grant-table.c > > @@ -37,11 +37,14 @@ > > #include <linux/slab.h> > > #include <linux/vmalloc.h> > > #include <linux/uaccess.h> > > +#include <linux/io.h> > > > > #include <xen/xen.h> > > #include <xen/interface/xen.h> > > #include <xen/page.h> > > #include <xen/grant_table.h> > > +#include <xen/platform_pci.h> > > +#include <xen/interface/memory.h> > > #include <asm/xen/hypercall.h> > > > > #include <asm/pgtable.h> > > @@ -59,6 +62,7 @@ static unsigned int boot_max_nr_grant_frames; > > static int gnttab_free_count; > > static grant_ref_t gnttab_free_head; > > static DEFINE_SPINLOCK(gnttab_list_lock); > > +static unsigned long hvm_pv_resume_frames; > > > > static struct grant_entry *shared; > > > > @@ -449,6 +453,30 @@ static int gnttab_map(unsigned int start_idx, unsigned int end_idx) > > unsigned int nr_gframes = end_idx + 1; > > int rc; > > > > + if (xen_hvm_domain()) { > > + struct xen_add_to_physmap xatp; > > + unsigned int i = end_idx; > > + rc = 0; > > + /* > > + * Loop backwards, so that the first hypercall has the largest > > + * index, ensuring that the table will grow only once. > > + */ > > + do { > > + xatp.domid = DOMID_SELF; > > + xatp.idx = i; > > + xatp.space = XENMAPSPACE_grant_table; > > + xatp.gpfn = (hvm_pv_resume_frames >> PAGE_SHIFT) + i; > > + rc = HYPERVISOR_memory_op(XENMEM_add_to_physmap, &xatp); > > + if (rc != 0) { > > + printk(KERN_WARNING > > + "grant table add_to_physmap failed, err=%d\n", rc); > > + break; > > + } > > + } while (i-- > start_idx); > > + > > + return rc; > > + } > > + > > frames = kmalloc(nr_gframes * sizeof(unsigned long), GFP_ATOMIC); > > if (!frames) > > return -ENOMEM; > > @@ -476,9 +504,28 @@ static int gnttab_map(unsigned int start_idx, unsigned int end_idx) > > > > int gnttab_resume(void) > > { > > - if (max_nr_grant_frames() < nr_grant_frames) > > + unsigned int max_nr_gframes; > > + > > + max_nr_gframes = max_nr_grant_frames(); > > + if (max_nr_gframes < nr_grant_frames) > > return -ENOSYS; > > - return gnttab_map(0, nr_grant_frames - 1); > > + > > + if (xen_pv_domain()) > > + return gnttab_map(0, nr_grant_frames - 1); > > + > > + if (!hvm_pv_resume_frames) { > > + hvm_pv_resume_frames = alloc_xen_mmio(PAGE_SIZE * max_nr_gframes); > > > > Can alloc_xen_mmio fail? > Can this possibly get called with the stub > version in place (returning ~0UL), and what does ioremap do if you pass > that into it? >No, alloc_xen_mmio cannot fail if the Xen platform PCI device driver loaded successfully, and we wouldn''t be here if it didn''t. For this reason this cannot get called with the stub version in place.> > + shared = ioremap(hvm_pv_resume_frames, PAGE_SIZE * max_nr_gframes); > > + if (shared == NULL) { > > + printk(KERN_WARNING > > + "Fail to ioremap gnttab share frames\n"); > > + return -ENOMEM; > > + } > > + } > > + > > + gnttab_map(0, nr_grant_frames - 1); > > + > > + return 0; > > } > > > > int gnttab_suspend(void) > > @@ -505,15 +552,12 @@ static int gnttab_expand(unsigned int req_entries) > > return rc; > > } > > > > -static int __devinit gnttab_init(void) > > +int gnttab_init(void) > > { > > int i; > > unsigned int max_nr_glist_frames, nr_glist_frames; > > unsigned int nr_init_grefs; > > > > - if (!xen_domain()) > > - return -ENODEV; > > - > > nr_grant_frames = 1; > > boot_max_nr_grant_frames = __max_nr_grant_frames(); > > > > @@ -557,4 +601,16 @@ static int __devinit gnttab_init(void) > > return -ENOMEM; > > } > > > > -core_initcall(gnttab_init); > > +static int __devinit __gnttab_init(void) > > +{ > > + /* Delay grant-table initialization in the PV on HVM case */ > > + if (xen_hvm_domain()) > > + return 0; > > + > > + if (!xen_pv_domain()) > > + return -ENODEV; > > + > > + return gnttab_init(); > > +} > > + > > +core_initcall(__gnttab_init); > > diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c > > new file mode 100644 > > index 0000000..7a8da66 > > --- /dev/null > > +++ b/drivers/xen/platform-pci.c > > @@ -0,0 +1,198 @@ > > +/****************************************************************************** > > + * platform-pci.c > > + * > > + * Xen platform PCI device driver > > + * Copyright (c) 2005, Intel Corporation. > > + * Copyright (c) 2007, XenSource Inc. > > + * Copyright (c) 2010, Citrix > > + * > > + * This program is free software; you can redistribute it and/or modify it > > + * under the terms and conditions of the GNU General Public License, > > + * version 2, as published by the Free Software Foundation. > > + * > > + * This program is distributed in the hope it will be useful, but WITHOUT > > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > > + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for > > + * more details. > > + * > > + * You should have received a copy of the GNU General Public License along with > > + * this program; if not, write to the Free Software Foundation, Inc., 59 Temple > > + * Place - Suite 330, Boston, MA 02111-1307 USA. > > + * > > + */ > > + > > +#include <asm/io.h> > > + > > +#include <linux/interrupt.h> > > +#include <linux/module.h> > > +#include <linux/pci.h> > > + > > +#include <xen/grant_table.h> > > +#include <xen/platform_pci.h> > > +#include <xen/interface/platform_pci.h> > > +#include <xen/xenbus.h> > > +#include <xen/events.h> > > +#include <xen/hvm.h> > > + > > +#define DRV_NAME "xen-platform-pci" > > + > > +MODULE_AUTHOR("ssmith@xensource.com and stefano.stabellini@eu.citrix.com"); > > +MODULE_DESCRIPTION("Xen platform PCI device"); > > +MODULE_LICENSE("GPL"); > > + > > +static unsigned long platform_mmio; > > +static unsigned long platform_mmio_alloc; > > +static unsigned long platform_mmiolen; > > + > > +unsigned long alloc_xen_mmio(unsigned long len) > > +{ > > + unsigned long addr; > > + > > + addr = platform_mmio + platform_mmio_alloc; > > + platform_mmio_alloc += len; > > + BUG_ON(platform_mmio_alloc > platform_mmiolen); > > + > > + return addr; > > +} > > + > > +static uint64_t get_callback_via(struct pci_dev *pdev) > > +{ > > + u8 pin; > > + int irq; > > + > > + irq = pdev->irq; > > + if (irq < 16) > > + return irq; /* ISA IRQ */ > > + > > + pin = pdev->pin; > > + > > + /* We don''t know the GSI. Specify the PCI INTx line instead. */ > > + return ((uint64_t)0x01 << 56) | /* PCI INTx identifier */ > > + ((uint64_t)pci_domain_nr(pdev->bus) << 32) | > > + ((uint64_t)pdev->bus->number << 16) | > > + ((uint64_t)(pdev->devfn & 0xff) << 8) | > > + ((uint64_t)(pin - 1) & 3); > > +} > > + > > +static irqreturn_t do_hvm_evtchn_intr(int irq, void *dev_id) > > +{ > > + xen_hvm_evtchn_do_upcall(get_irq_regs()); > > + return IRQ_HANDLED; > > +} > > + > > +static int xen_allocate_irq(struct pci_dev *pdev) > > +{ > > + return request_irq(pdev->irq, do_hvm_evtchn_intr, > > + IRQF_DISABLED | IRQF_NOBALANCING | IRQF_TRIGGER_RISING, > > + "xen-platform-pci", pdev); > > +} > > + > > +static int __devinit platform_pci_init(struct pci_dev *pdev, > > + const struct pci_device_id *ent) > > +{ > > + int i, ret; > > + long ioaddr, iolen; > > + long mmio_addr, mmio_len; > > + uint64_t callback_via; > > + > > + i = pci_enable_device(pdev); > > + if (i) > > + return i; > > + > > + ioaddr = pci_resource_start(pdev, 0); > > + iolen = pci_resource_len(pdev, 0); > > + > > + mmio_addr = pci_resource_start(pdev, 1); > > + mmio_len = pci_resource_len(pdev, 1); > > + > > + if (mmio_addr == 0 || ioaddr == 0) { > > + dev_err(&pdev->dev, "no resources found\n"); > > + ret = -ENOENT; > > + } > > + > > + if (request_mem_region(mmio_addr, mmio_len, DRV_NAME) == NULL) { > > + dev_err(&pdev->dev, "MEM I/O resource 0x%lx @ 0x%lx busy\n", > > + mmio_addr, mmio_len); > > + ret = -EBUSY; > > + } > > + > > + if (request_region(ioaddr, iolen, DRV_NAME) == NULL) { > > + dev_err(&pdev->dev, "I/O resource 0x%lx @ 0x%lx busy\n", > > + iolen, ioaddr); > > + ret = -EBUSY; > > + goto out; > > + } > > + > > + platform_mmio = mmio_addr; > > + platform_mmiolen = mmio_len; > > + > > + if (!xen_have_vector_callback) { > > + ret = xen_allocate_irq(pdev); > > + if (ret) { > > + printk(KERN_WARNING "request_irq failed err=%d\n", ret); > > + goto out; > > + } > > + callback_via = get_callback_via(pdev); > > + ret = xen_set_callback_via(callback_via); > > + if (ret) { > > + printk(KERN_WARNING > > + "Unable to set the evtchn callback err=%d\n", ret); > > + goto out; > > + } > > + } > > + > > + alloc_xen_mmio_hook = alloc_xen_mmio; > > + platform_pci_resume_hook = platform_pci_resume; > > + platform_pci_disable_irq_hook = platform_pci_disable_irq; > > + platform_pci_enable_irq_hook = platform_pci_enable_irq; > > > > What''s this _hook stuff for? >The hooks are needed to allow the xen platform device to be compiled and used as a module successfully. See also PATCH number 8. The problem is that grant_table.c and manage.c depend on functions provided by the Xen platform PCI driver if running on HVM, and this could be compiled as a module. In that case we need to provide stubs until the Xen platform PCI module is loaded.> > + > > + ret = gnttab_init(); > > + if (ret) > > + goto out; > > + ret = xenbus_probe_init(); > > + if (ret) > > + goto out; > > + > > +out: > > + if (ret) { > > + release_mem_region(mmio_addr, mmio_len); > > + release_region(ioaddr, iolen); > > + pci_disable_device(pdev); > > + } > > + > > + return ret; > > +} > > + > > +#define XEN_PLATFORM_VENDOR_ID 0x5853 > > +#define XEN_PLATFORM_DEVICE_ID 0x0001 > > +static struct pci_device_id platform_pci_tbl[] __devinitdata = { > > + {XEN_PLATFORM_VENDOR_ID, XEN_PLATFORM_DEVICE_ID, > > + PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0}, > > + {0,} > > +}; > > + > > +MODULE_DEVICE_TABLE(pci, platform_pci_tbl); > > + > > +static struct pci_driver platform_driver = { > > + name: DRV_NAME, > > + probe : platform_pci_init, > > + id_table : platform_pci_tbl, > > +}; > > + > > +static int __init platform_pci_module_init(void) > > +{ > > + int rc; > > + > > + if (!xen_platform_pci) > > + return -ENODEV; > > + > > + rc = pci_register_driver(&platform_driver); > > + if (rc) { > > + printk(KERN_INFO DRV_NAME > > + ": No platform pci device model found\n"); > > + return rc; > > + } > > + return 0; > > +} > > + > > +module_init(platform_pci_module_init); > > diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c > > index 0b05b62..dc6ed06 100644 > > --- a/drivers/xen/xenbus/xenbus_probe.c > > +++ b/drivers/xen/xenbus/xenbus_probe.c > > @@ -782,16 +782,24 @@ void xenbus_probe(struct work_struct *unused) > > blocking_notifier_call_chain(&xenstore_chain, 0, NULL); > > } > > > > -static int __init xenbus_probe_init(void) > > +static int __init __xenbus_probe_init(void) > > +{ > > + /* Delay initialization in the PV on HVM case */ > > + if (xen_hvm_domain()) > > + return 0; > > + > > + if (!xen_pv_domain()) > > + return -ENODEV; > > + > > + return xenbus_probe_init(); > > +} > > + > > +int xenbus_probe_init(void) > > { > > int err = 0; > > > > DPRINTK(""); > > > > - err = -ENODEV; > > - if (!xen_domain()) > > - goto out_error; > > - > > /* Register ourselves with the kernel bus subsystem */ > > err = bus_register(&xenbus_frontend.bus); > > if (err) > > @@ -857,7 +865,7 @@ static int __init xenbus_probe_init(void) > > return err; > > } > > > > -postcore_initcall(xenbus_probe_init); > > +postcore_initcall(__xenbus_probe_init); > > > > MODULE_LICENSE("GPL"); > > > > diff --git a/include/xen/grant_table.h b/include/xen/grant_table.h > > index a40f1cd..811cda5 100644 > > --- a/include/xen/grant_table.h > > +++ b/include/xen/grant_table.h > > @@ -51,6 +51,7 @@ struct gnttab_free_callback { > > u16 count; > > }; > > > > +int gnttab_init(void); > > int gnttab_suspend(void); > > int gnttab_resume(void); > > > > diff --git a/include/xen/interface/grant_table.h b/include/xen/interface/grant_table.h > > index 39da93c..39e5717 100644 > > --- a/include/xen/interface/grant_table.h > > +++ b/include/xen/interface/grant_table.h > > @@ -28,6 +28,7 @@ > > #ifndef __XEN_PUBLIC_GRANT_TABLE_H__ > > #define __XEN_PUBLIC_GRANT_TABLE_H__ > > > > +#include <xen/interface/xen.h> > > > > /*********************************** > > * GRANT TABLE REPRESENTATION > > diff --git a/include/xen/platform_pci.h b/include/xen/platform_pci.h > > index f39f4d3..59a120c 100644 > > --- a/include/xen/platform_pci.h > > +++ b/include/xen/platform_pci.h > > @@ -29,4 +29,13 @@ > > #define XEN_IOPORT_LINUX_PRODNUM 0xffff > > #define XEN_IOPORT_LINUX_DRVVER ((LINUX_VERSION_CODE << 8) + 0x0) > > > > +#ifdef CONFIG_XEN_PLATFORM_PCI > > +unsigned long alloc_xen_mmio(unsigned long len); > > +#else > > +static inline unsigned long alloc_xen_mmio(unsigned long len) > > +{ > > + return ~0UL; > > +} > > > > Why is this stub needed? >Actually after introducing the corresponding hook it is not needed anymore. PATCH 8 removes it.> > +#endif > > + > > #endif /* _XEN_PLATFORM_PCI_H */ > > diff --git a/include/xen/xenbus.h b/include/xen/xenbus.h > > index 43e2d7d..ffa97de 100644 > > --- a/include/xen/xenbus.h > > +++ b/include/xen/xenbus.h > > @@ -174,6 +174,7 @@ void unregister_xenbus_watch(struct xenbus_watch *watch); > > void xs_suspend(void); > > void xs_resume(void); > > void xs_suspend_cancel(void); > > +int xenbus_probe_init(void); > > > > /* Used by xenbus_dev to borrow kernel''s store connection. */ > > void *xenbus_dev_request_and_reply(struct xsd_sockmsg *msg); > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-19 14:18 UTC
[Xen-devel] Re: [PATCH 07/12] Add suspend\resume support for PV on HVM guests.
On Tue, 18 May 2010, Jeremy Fitzhardinge wrote:> On 05/18/2010 03:23 AM, Stefano Stabellini wrote: > > "/" > > Please describe what''s needed to support suspend/resume. Is this a > normal x86 ACPI suspend/resume, or a Xen save/restore? >This is Xen save/restore, it doesn''t have anything to do with ACPI. In order to support it, we need to listen to xenbus for the suspend event, freeze all the processes, suspend the PV frontends and call an hypercall.> > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > > --- > > arch/x86/xen/enlighten.c | 9 ++-- > > arch/x86/xen/suspend.c | 6 ++ > > arch/x86/xen/xen-ops.h | 3 + > > drivers/xen/manage.c | 95 +++++++++++++++++++++++++++++++++++-- > > drivers/xen/platform-pci.c | 29 +++++++++++- > > drivers/xen/xenbus/xenbus_probe.c | 28 +++++++++++ > > include/xen/platform_pci.h | 6 ++ > > include/xen/xen-ops.h | 3 + > > 8 files changed, 170 insertions(+), 9 deletions(-) > > > > diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c > > index aac47b0..23b8200 100644 > > --- a/arch/x86/xen/enlighten.c > > +++ b/arch/x86/xen/enlighten.c > > @@ -1268,12 +1268,13 @@ static int init_hvm_pv_info(int *major, int *minor) > > return 0; > > } > > > > -static void __init init_shared_info(void) > > +void init_shared_info(void) > > { > > struct xen_add_to_physmap xatp; > > - struct shared_info *shared_info_page; > > + static struct shared_info *shared_info_page = 0; > > > > - shared_info_page = (struct shared_info *) alloc_bootmem_pages(PAGE_SIZE); > > + if (!shared_info_page) > > + shared_info_page = (struct shared_info *) alloc_bootmem_pages(PAGE_SIZE); > > xatp.domid = DOMID_SELF; > > xatp.idx = 0; > > xatp.space = XENMAPSPACE_shared_info; > > @@ -1302,7 +1303,7 @@ void do_hvm_pv_evtchn_intr(void) > > xen_hvm_evtchn_do_upcall(get_irq_regs()); > > } > > > > -static void xen_callback_vector(void) > > +void xen_callback_vector(void) > > { > > uint64_t callback_via; > > if (xen_feature(XENFEAT_hvm_callback_vector)) { > > diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c > > index 987267f..86f3b45 100644 > > --- a/arch/x86/xen/suspend.c > > +++ b/arch/x86/xen/suspend.c > > @@ -26,6 +26,12 @@ void xen_pre_suspend(void) > > BUG(); > > } > > > > +void xen_hvm_post_suspend(int suspend_cancelled) > > +{ > > + init_shared_info(); > > + xen_callback_vector(); > > +} > > + > > void xen_post_suspend(int suspend_cancelled) > > { > > xen_build_mfn_list_list(); > > diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h > > index f9153a3..caf89ee 100644 > > --- a/arch/x86/xen/xen-ops.h > > +++ b/arch/x86/xen/xen-ops.h > > @@ -38,6 +38,9 @@ void xen_enable_sysenter(void); > > void xen_enable_syscall(void); > > void xen_vcpu_restore(void); > > > > +void xen_callback_vector(void); > > +void init_shared_info(void); > > + > > void __init xen_build_dynamic_phys_to_machine(void); > > > > void xen_init_irq_ops(void); > > diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c > > index 2ac4440..a73edd8 100644 > > --- a/drivers/xen/manage.c > > +++ b/drivers/xen/manage.c > > @@ -8,15 +8,20 @@ > > #include <linux/sysrq.h> > > #include <linux/stop_machine.h> > > #include <linux/freezer.h> > > +#include <linux/pci.h> > > +#include <linux/cpumask.h> > > > > +#include <xen/xen.h> > > #include <xen/xenbus.h> > > #include <xen/grant_table.h> > > #include <xen/events.h> > > #include <xen/hvc-console.h> > > #include <xen/xen-ops.h> > > +#include <xen/platform_pci.h> > > > > #include <asm/xen/hypercall.h> > > #include <asm/xen/page.h> > > +#include <asm/xen/hypervisor.h> > > > > enum shutdown_state { > > SHUTDOWN_INVALID = -1, > > @@ -33,10 +38,30 @@ enum shutdown_state { > > static enum shutdown_state shutting_down = SHUTDOWN_INVALID; > > > > #ifdef CONFIG_PM_SLEEP > > -static int xen_suspend(void *data) > > +static int xen_hvm_suspend(void *data) > > { > > + struct sched_shutdown r = { .reason = SHUTDOWN_suspend }; > > int *cancelled = data; > > + > > + BUG_ON(!irqs_disabled()); > > + > > + *cancelled = HYPERVISOR_sched_op(SCHEDOP_shutdown, &r); > > + > > + xen_hvm_post_suspend(*cancelled); > > + gnttab_resume(); > > + > > + if (!*cancelled) { > > + xen_irq_resume(); > > + platform_pci_resume(); > > + } > > + > > + return 0; > > +} > > + > > +static int xen_suspend(void *data) > > +{ > > int err; > > + int *cancelled = data; > > > > BUG_ON(!irqs_disabled()); > > > > @@ -73,6 +98,53 @@ static int xen_suspend(void *data) > > return 0; > > } > > > > +static void do_hvm_suspend(void) > > +{ > > + int err; > > + int cancelled = 1; > > + > > + shutting_down = SHUTDOWN_SUSPEND; > > + > > +#ifdef CONFIG_PREEMPT > > + /* If the kernel is preemptible, we need to freeze all the processes > > + to prevent them from being in the middle of a pagetable update > > + during suspend. */ > > + err = freeze_processes(); > > + if (err) { > > + printk(KERN_ERR "xen suspend: freeze failed %d\n", err); > > + goto out; > > + } > > +#endif > > + > > + printk(KERN_DEBUG "suspending xenstore... "); > > + xenbus_suspend(); > > + printk(KERN_DEBUG "xenstore suspended\n"); > > + platform_pci_disable_irq(); > > + > > + err = stop_machine(xen_hvm_suspend, &cancelled, cpumask_of(0)); > > + if (err) { > > + printk(KERN_ERR "failed to start xen_suspend: %d\n", err); > > + cancelled = 1; > > + } > > + > > + platform_pci_enable_irq(); > > + > > + if (!cancelled) { > > + xen_arch_resume(); > > + xenbus_resume(); > > + } else > > + xs_suspend_cancel(); > > + > > + /* Make sure timer events get retriggered on all CPUs */ > > + clock_was_set(); > > + > > +#ifdef CONFIG_PREEMPT > > + thaw_processes(); > > +out: > > +#endif > > + shutting_down = SHUTDOWN_INVALID; > > +} > > + > > static void do_suspend(void) > > { > > int err; > > @@ -185,7 +257,10 @@ static void shutdown_handler(struct xenbus_watch *watch, > > ctrl_alt_del(); > > #ifdef CONFIG_PM_SLEEP > > } else if (strcmp(str, "suspend") == 0) { > > - do_suspend(); > > + if (xen_hvm_domain()) > > + do_hvm_suspend(); > > > > Why does HVM come via this path? Wouldn''t ACPI S3 be a better match for > HVM? Does this make sure the full device model suspend/resume callbacks > get called? Previously I think we cut corners because we knew there > wouldn''t be any PCI devices in the system... > > And if the full device model is being used properly, then can''t all this > hvm-specific stuff be done in the platform pci driver itself, rather > than here? Is checkpoint the issue? (Is checkpointing hvm domains > supported?) >I don''t think we want to handle this with ACPI S3. Xen is capable of issuing ACPI S3 requests if it wants to. This is a different case. My first attempt was to use the PV suspend handler but I end up with too many if (xen_hvm_domain()) so I decided to write a new one. Now that the code is much more stable and it doesn''t have any known bugs anymore I might be able to refactor it in a better way either moving it to the platform pci driver or using the PV suspend handler.> > + else > > + do_suspend(); > > #endif > > } else { > > printk(KERN_INFO "Ignoring shutdown request: %s\n", str); > > @@ -261,7 +336,19 @@ static int shutdown_event(struct notifier_block *notifier, > > return NOTIFY_DONE; > > } > > > > -static int __init setup_shutdown_event(void) > > +static int __init __setup_shutdown_event(void) > > +{ > > + /* Delay initialization in the PV on HVM case */ > > + if (xen_hvm_domain()) > > + return 0; > > + > > + if (!xen_pv_domain()) > > + return -ENODEV; > > + > > + return xen_setup_shutdown_event(); > > +} > > + > > +int xen_setup_shutdown_event(void) > > { > > static struct notifier_block xenstore_notifier = { > > .notifier_call = shutdown_event > > @@ -271,4 +358,4 @@ static int __init setup_shutdown_event(void) > > return 0; > > } > > > > -subsys_initcall(setup_shutdown_event); > > +subsys_initcall(__setup_shutdown_event); > > diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c > > index 7a8da66..b15f809 100644 > > --- a/drivers/xen/platform-pci.c > > +++ b/drivers/xen/platform-pci.c > > @@ -33,6 +33,7 @@ > > #include <xen/xenbus.h> > > #include <xen/events.h> > > #include <xen/hvm.h> > > +#include <xen/xen-ops.h> > > > > #define DRV_NAME "xen-platform-pci" > > > > @@ -43,6 +44,8 @@ MODULE_LICENSE("GPL"); > > static unsigned long platform_mmio; > > static unsigned long platform_mmio_alloc; > > static unsigned long platform_mmiolen; > > +static uint64_t callback_via; > > +struct pci_dev *xen_platform_pdev; > > > > unsigned long alloc_xen_mmio(unsigned long len) > > { > > @@ -87,13 +90,33 @@ static int xen_allocate_irq(struct pci_dev *pdev) > > "xen-platform-pci", pdev); > > } > > > > +void platform_pci_disable_irq(void) > > > > If these are non-static they need a xen_ prefix. In fact > "platform_pci_" is too generic anyway, and they should all have xen_ > prefixes. > > Aside from that, why do they need to be externally callable? Can''t the > pci device''s own suspend/resume handlers do this?I had serious problems the first time I tried to do that, but it is true that at that point I had many other serious bugs in the hvm suspend\resume code. I''ll give it another try now that is stable.> > > +{ > > + printk(KERN_DEBUG "platform_pci_disable_irq\n"); > > + disable_irq(xen_platform_pdev->irq); > > +} > > + > > +void platform_pci_enable_irq(void) > > +{ > > + printk(KERN_DEBUG "platform_pci_enable_irq\n"); > > + enable_irq(xen_platform_pdev->irq); > > +} > > + > > +void platform_pci_resume(void) > > +{ > > + if (!xen_have_vector_callback && xen_set_callback_via(callback_via)) { > > + printk("platform_pci_resume failure!\n"); > > + return; > > + } > > +} > > + > > static int __devinit platform_pci_init(struct pci_dev *pdev, > > const struct pci_device_id *ent) > > { > > int i, ret; > > long ioaddr, iolen; > > long mmio_addr, mmio_len; > > - uint64_t callback_via; > > + xen_platform_pdev = pdev; > > > > i = pci_enable_device(pdev); > > if (i) > > @@ -152,6 +175,10 @@ static int __devinit platform_pci_init(struct pci_dev *pdev, > > ret = xenbus_probe_init(); > > if (ret) > > goto out; > > + ret = xen_setup_shutdown_event(); > > + if (ret) > > + goto out; > > + > > > > out: > > if (ret) { > > diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c > > index dc6ed06..a679205 100644 > > --- a/drivers/xen/xenbus/xenbus_probe.c > > +++ b/drivers/xen/xenbus/xenbus_probe.c > > @@ -746,6 +746,34 @@ static int xenbus_dev_resume(struct device *dev) > > return 0; > > } > > > > +static int dev_suspend(struct device *dev, void *data) > > +{ > > + return xenbus_dev_suspend(dev, PMSG_SUSPEND); > > +} > > + > > +void xenbus_suspend(void) > > +{ > > + DPRINTK(""); > > + > > + bus_for_each_dev(&xenbus_frontend.bus, NULL, NULL, dev_suspend); > > + xs_suspend(); > > +} > > +EXPORT_SYMBOL_GPL(xenbus_suspend); > > + > > +static int dev_resume(struct device *dev, void *data) > > +{ > > + return xenbus_dev_resume(dev); > > +} > > + > > +void xenbus_resume(void) > > +{ > > + DPRINTK(""); > > + > > + xs_resume(); > > + bus_for_each_dev(&xenbus_frontend.bus, NULL, NULL, dev_resume); > > +} > > +EXPORT_SYMBOL_GPL(xenbus_resume); > > + > > /* A flag to determine if xenstored is ''ready'' (i.e. has started) */ > > int xenstored_ready = 0; > > > > diff --git a/include/xen/platform_pci.h b/include/xen/platform_pci.h > > index 59a120c..ced434d 100644 > > --- a/include/xen/platform_pci.h > > +++ b/include/xen/platform_pci.h > > @@ -31,11 +31,17 @@ > > > > #ifdef CONFIG_XEN_PLATFORM_PCI > > unsigned long alloc_xen_mmio(unsigned long len); > > +void platform_pci_resume(void); > > +void platform_pci_disable_irq(void); > > +void platform_pci_enable_irq(void); > > #else > > static inline unsigned long alloc_xen_mmio(unsigned long len) > > { > > return ~0UL; > > } > > +static inline void platform_pci_resume(void) {} > > +static inline void platform_pci_disable_irq(void) {} > > +static inline void platform_pci_enable_irq(void) {} > > #endif > > > > #endif /* _XEN_PLATFORM_PCI_H */ > > diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h > > index 883a21b..46bc81e 100644 > > --- a/include/xen/xen-ops.h > > +++ b/include/xen/xen-ops.h > > @@ -7,6 +7,7 @@ DECLARE_PER_CPU(struct vcpu_info *, xen_vcpu); > > > > void xen_pre_suspend(void); > > void xen_post_suspend(int suspend_cancelled); > > +void xen_hvm_post_suspend(int suspend_cancelled); > > > > void xen_mm_pin_all(void); > > void xen_mm_unpin_all(void); > > @@ -14,4 +15,6 @@ void xen_mm_unpin_all(void); > > void xen_timer_resume(void); > > void xen_arch_resume(void); > > > > +int xen_setup_shutdown_event(void); > > + > > #endif /* INCLUDE_XEN_OPS_H */ > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-19 14:19 UTC
[Xen-devel] Re: [PATCH 08/12] Allow xen platform pci device to be compiled as a module
On Tue, 18 May 2010, Jeremy Fitzhardinge wrote:> > diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c > > index 6f5f3ba..f936d30 100644 > > --- a/drivers/xen/grant-table.c > > +++ b/drivers/xen/grant-table.c > > @@ -56,6 +56,9 @@ > > #define GNTTAB_LIST_END 0xffffffff > > #define GREFS_PER_GRANT_FRAME (PAGE_SIZE / sizeof(struct grant_entry)) > > > > +unsigned long (*alloc_xen_mmio_hook)(unsigned long len); > > +EXPORT_SYMBOL_GPL(alloc_xen_mmio_hook); > > + > > static grant_ref_t **gnttab_list; > > static unsigned int nr_grant_frames; > > static unsigned int boot_max_nr_grant_frames; > > @@ -514,7 +517,7 @@ int gnttab_resume(void) > > return gnttab_map(0, nr_grant_frames - 1); > > > > if (!hvm_pv_resume_frames) { > > - hvm_pv_resume_frames = alloc_xen_mmio(PAGE_SIZE * max_nr_gframes); > > + hvm_pv_resume_frames = alloc_xen_mmio_hook(PAGE_SIZE * max_nr_gframes); > > > > This looks like it should be restructured so the pci device driver > itself is doing this mapping, then calling into the grant subsystem to > tell it where the mapping is. >OK> > shared = ioremap(hvm_pv_resume_frames, PAGE_SIZE * max_nr_gframes); > > if (shared == NULL) { > > printk(KERN_WARNING > > @@ -600,6 +603,7 @@ int gnttab_init(void) > > kfree(gnttab_list); > > return -ENOMEM; > > } > > +EXPORT_SYMBOL_GPL(gnttab_init); > > > > static int __devinit __gnttab_init(void) > > { > > diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c > > index a73edd8..49ee52d 100644 > > --- a/drivers/xen/manage.c > > +++ b/drivers/xen/manage.c > > @@ -34,6 +34,13 @@ enum shutdown_state { > > SHUTDOWN_HALT = 4, > > }; > > > > +void (*platform_pci_resume_hook)(void); > > +EXPORT_SYMBOL_GPL(platform_pci_resume_hook); > > +void (*platform_pci_disable_irq_hook)(void); > > +EXPORT_SYMBOL_GPL(platform_pci_disable_irq_hook); > > +void (*platform_pci_enable_irq_hook)(void); > > +EXPORT_SYMBOL_GPL(platform_pci_enable_irq_hook); > > > > If all this _hook stuff is here to support a modular xen platform pci > device, then something has gone wrong. The device should be doing this > via its own suspend/resume handlers. >I''ll give it another try. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-19 14:25 UTC
[Xen-devel] Re: [PATCH 09/12] Fix possible NULL pointer dereference in print_IO_APIC
On Tue, 18 May 2010, Jeremy Fitzhardinge wrote:> On 05/18/2010 03:23 AM, Stefano Stabellini wrote: > > Make sure chip_data is not NULL before accessing it. > > > > You should clarify under what circumstances it can be legitimately NULL. > >The VIRQ_TIMER handler and virq handlers in general don''t have any chip_data.> > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > > --- > > arch/x86/kernel/apic/io_apic.c | 2 ++ > > 1 files changed, 2 insertions(+), 0 deletions(-) > > > > diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c > > index eb2789c..c64499c 100644 > > --- a/arch/x86/kernel/apic/io_apic.c > > +++ b/arch/x86/kernel/apic/io_apic.c > > @@ -1732,6 +1732,8 @@ __apicdebuginit(void) print_IO_APIC(void) > > struct irq_pin_list *entry; > > > > cfg = desc->chip_data; > > + if (!cfg) > > + continue; > > entry = cfg->irq_2_pin; > > if (!entry) > > continue; > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-19 18:19 UTC
[Xen-devel] Re: [PATCH 03/12] evtchn delivery on HVM
On 05/19/2010 05:24 AM, Stefano Stabellini wrote:> On Tue, 18 May 2010, Jeremy Fitzhardinge wrote: > >> On 05/18/2010 03:22 AM, Stefano Stabellini wrote: >> >>> From: Sheng Yang <sheng@linux.intel.com> >>> >>> Set the callback to receive evtchns from Xen, using the >>> callback vector delivery mechanism. >>> >>> >> Could you expand on this a little? Like, why is this desireable? What >> functional difference does it make? Is this patch useful in its own >> right, or is it just laying the groundwork for something else? >> >> > In order to use PV frontends on HVM we need to receive notifications on > event channel deliveries somehow. >(OK, but I just meant update the commit comment on the patch itself.)> Using the callback vector is the preferred way, because it is available > independently from any (emulated) PCI device, all the vcpus can receive > these callbacks and theoretically there is no need to interact with the > emulated lapic (even though at the moment we are doing it anyway because > we are using the IPI vector). > > The other way is to receive interrupts from the xen platform pci device, > but in that case interaction with the emulated lapic is unavoidable and > we are limited to receive interrupts on vcpu 0. >Perhaps you should mention this first, since it is the historical way of doing it, and then talk about its limitations, and then talk about the replacement to address those limitations. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-24 17:28 UTC
Re: [Xen-devel] Re: [PATCH 0 of 12] PV on HVM Xen
On Tue, 18 May 2010, Christian Tramnitz wrote:> Hi Stefano, > > what are the particular advantages of running PVonHVM vs. traditional PV > (vs pure HVM)? > I''d like to update the wiki with some info about it... >Sorry for the late reply, but I wanted to have some numbers to compare the three cases... PVonHVM guests are as easy to install, use and maintain as pure HVM guests but provide performances comparable or even better than traditional PV guests. In particular you''ll find that PVonHVM guests are always faster than pure HVM guests, most of the times faster than 64 bit PV guests but probably slower than 32 bit PV guests. The following are the results of kernbench run in optimal mode on a 4 vcpu linux guest with 512MB of ram. The host is a Dell Precision T3500 (HAP enabled): testbox:~# cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Xeon(R) CPU E5520 @ 2.27GHz stepping : 5 cpu MHz : 2266.806 cache size : 8192 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht nx constant_tsc pni est ssse3 sse4_1 sse4_2 popcnt ida bogomips : 4538.00 clflush size : 64 power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Xeon(R) CPU E5520 @ 2.27GHz stepping : 5 cpu MHz : 2266.806 cache size : 8192 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht nx constant_tsc pni est ssse3 sse4_1 sse4_2 popcnt ida bogomips : 4538.00 clflush size : 64 power management: processor : 2 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Xeon(R) CPU E5520 @ 2.27GHz stepping : 5 cpu MHz : 2266.806 cache size : 8192 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht nx constant_tsc pni est ssse3 sse4_1 sse4_2 popcnt ida bogomips : 4538.00 clflush size : 64 power management: processor : 3 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Xeon(R) CPU E5520 @ 2.27GHz stepping : 5 cpu MHz : 2266.806 cache size : 8192 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht nx constant_tsc pni est ssse3 sse4_1 sse4_2 popcnt ida bogomips : 4538.00 clflush size : 64 power management: 64 bit PV on HVM ---------------- Average Optimal load -j 4 Run (std deviation): Elapsed Time 215.307 (10.1294) User Time 632.503 (6.4785) System Time 115.497 (4.53905) Percent CPU 347.333 (15.885) Context Switches 43319.7 (2088.39) Sleeps 48950 (3140.18) 64 bit pure HVM --------------- Average Optimal load -j 4 Run (std deviation): Elapsed Time 235.25 (4.10598) User Time 512.48 (1.69714) System Time 73.5967 (0.65041) Percent CPU 248.667 (4.93288) Context Switches 35930.3 (342.837) Sleeps 56249.7 (2784.42) 64 bit traditional PV --------------------- Average Optimal load -j 4 Run (std deviation): Elapsed Time 248.187 (12.3954) User Time 535.283 (0.818189) System Time 127.497 (0.342685) Percent CPU 266.667 (12.8582) Context Switches 32978 (2968.78) Sleeps 54391.3 (3141.44) the results show that the 64 bit PV on HVM case is the fastest but it is also the one that uses most CPU and User time. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi all, this is another update of the PV on HVM Xen series that addresses Jeremy''s comments. The platform_pci hooks have been removed, suspend/resume for HVM domains is now much more similar to the PV case and shares the same do_suspend function. Alloc_xen_mmio_hook has been removed has well, now the memory allocation for the grant table is done by the xen platform pci driver directly. The per_cpu xen_vcpu variable is set by a cpu_notifier function so that secondary vcpus have the variable set correctly no matter what the xen features are on the host. The kernel command line option xen_unplug has been renamed to xen_emul_unplug and the code that makes use of it has been moved to a separate file (arch/x86/xen/platform-pci-unplug.c). Xen_unplug_emulated_devices is now able to detect if blkfront, netfront and the Xen platform PCI driver have been compiled, and set the default value of xen_emul_unplug accordingly. The patch "Initialize xenbus device structs with ENODEV as default" has been removed from the series and it will be sent separately. Finally the comments on most of the patches have been improved. The series is based on 2.6.34 and supports Xen PV frontends running in a HVM domain, including netfront, blkfront and the VIRQ_TIMER. In order to be able to use VIRQ_TIMER and to improve performances you need a patch to Xen to implement the vector callback mechanism for event channel delivery. A git tree is also available here: git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git branch name 2.6.34-pvhvm-v2. Cheers, Stefano _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
From: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Signed-off-by: Sheng Yang <sheng@linux.intel.com> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- arch/x86/include/asm/xen/hypercall.h | 6 ++ include/xen/hvm.h | 24 +++++++++ include/xen/interface/hvm/hvm_op.h | 35 ++++++++++++ include/xen/interface/hvm/params.h | 95 ++++++++++++++++++++++++++++++++++ 4 files changed, 160 insertions(+), 0 deletions(-) create mode 100644 include/xen/hvm.h create mode 100644 include/xen/interface/hvm/hvm_op.h create mode 100644 include/xen/interface/hvm/params.h diff --git a/arch/x86/include/asm/xen/hypercall.h b/arch/x86/include/asm/xen/hypercall.h index 9c371e4..7fda040 100644 --- a/arch/x86/include/asm/xen/hypercall.h +++ b/arch/x86/include/asm/xen/hypercall.h @@ -417,6 +417,12 @@ HYPERVISOR_nmi_op(unsigned long op, unsigned long arg) return _hypercall2(int, nmi_op, op, arg); } +static inline unsigned long __must_check +HYPERVISOR_hvm_op(int op, void *arg) +{ + return _hypercall2(unsigned long, hvm_op, op, arg); +} + static inline void MULTI_fpu_taskswitch(struct multicall_entry *mcl, int set) { diff --git a/include/xen/hvm.h b/include/xen/hvm.h new file mode 100644 index 0000000..6b0d418 --- /dev/null +++ b/include/xen/hvm.h @@ -0,0 +1,24 @@ +/* Simple wrappers around HVM functions */ +#ifndef XEN_HVM_H__ +#define XEN_HVM_H__ + +#include <xen/interface/hvm/params.h> + +static inline int hvm_get_parameter(int idx, uint64_t *value) +{ + struct xen_hvm_param xhv; + int r; + + xhv.domid = DOMID_SELF; + xhv.index = idx; + r = HYPERVISOR_hvm_op(HVMOP_get_param, &xhv); + if (r < 0) { + printk(KERN_ERR "cannot get hvm parameter %d: %d.\n", + idx, r); + return r; + } + *value = xhv.value; + return r; +} + +#endif /* XEN_HVM_H__ */ diff --git a/include/xen/interface/hvm/hvm_op.h b/include/xen/interface/hvm/hvm_op.h new file mode 100644 index 0000000..73c8c7e --- /dev/null +++ b/include/xen/interface/hvm/hvm_op.h @@ -0,0 +1,35 @@ +/* + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to + * deal in the Software without restriction, including without limitation the + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or + * sell copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. + */ + +#ifndef __XEN_PUBLIC_HVM_HVM_OP_H__ +#define __XEN_PUBLIC_HVM_HVM_OP_H__ + +/* Get/set subcommands: the second argument of the hypercall is a + * pointer to a xen_hvm_param struct. */ +#define HVMOP_set_param 0 +#define HVMOP_get_param 1 +struct xen_hvm_param { + domid_t domid; /* IN */ + uint32_t index; /* IN */ + uint64_t value; /* IN/OUT */ +}; +DEFINE_GUEST_HANDLE_STRUCT(xen_hvm_param); + +#endif /* __XEN_PUBLIC_HVM_HVM_OP_H__ */ diff --git a/include/xen/interface/hvm/params.h b/include/xen/interface/hvm/params.h new file mode 100644 index 0000000..1888d8c --- /dev/null +++ b/include/xen/interface/hvm/params.h @@ -0,0 +1,95 @@ +/* + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to + * deal in the Software without restriction, including without limitation the + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or + * sell copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. + */ + +#ifndef __XEN_PUBLIC_HVM_PARAMS_H__ +#define __XEN_PUBLIC_HVM_PARAMS_H__ + +#include "hvm_op.h" + +/* + * Parameter space for HVMOP_{set,get}_param. + */ + +/* + * How should CPU0 event-channel notifications be delivered? + * val[63:56] == 0: val[55:0] is a delivery GSI (Global System Interrupt). + * val[63:56] == 1: val[55:0] is a delivery PCI INTx line, as follows: + * Domain = val[47:32], Bus = val[31:16], + * DevFn = val[15: 8], IntX = val[ 1: 0] + * val[63:56] == 2: val[7:0] is a vector number. + * If val == 0 then CPU0 event-channel notifications are not delivered. + */ +#define HVM_PARAM_CALLBACK_IRQ 0 + +#define HVM_PARAM_STORE_PFN 1 +#define HVM_PARAM_STORE_EVTCHN 2 + +#define HVM_PARAM_PAE_ENABLED 4 + +#define HVM_PARAM_IOREQ_PFN 5 + +#define HVM_PARAM_BUFIOREQ_PFN 6 + +/* + * Set mode for virtual timers (currently x86 only): + * delay_for_missed_ticks (default): + * Do not advance a vcpu''s time beyond the correct delivery time for + * interrupts that have been missed due to preemption. Deliver missed + * interrupts when the vcpu is rescheduled and advance the vcpu''s virtual + * time stepwise for each one. + * no_delay_for_missed_ticks: + * As above, missed interrupts are delivered, but guest time always tracks + * wallclock (i.e., real) time while doing so. + * no_missed_ticks_pending: + * No missed interrupts are held pending. Instead, to ensure ticks are + * delivered at some non-zero rate, if we detect missed ticks then the + * internal tick alarm is not disabled if the VCPU is preempted during the + * next tick period. + * one_missed_tick_pending: + * Missed interrupts are collapsed together and delivered as one ''late tick''. + * Guest time always tracks wallclock (i.e., real) time. + */ +#define HVM_PARAM_TIMER_MODE 10 +#define HVMPTM_delay_for_missed_ticks 0 +#define HVMPTM_no_delay_for_missed_ticks 1 +#define HVMPTM_no_missed_ticks_pending 2 +#define HVMPTM_one_missed_tick_pending 3 + +/* Boolean: Enable virtual HPET (high-precision event timer)? (x86-only) */ +#define HVM_PARAM_HPET_ENABLED 11 + +/* Identity-map page directory used by Intel EPT when CR0.PG=0. */ +#define HVM_PARAM_IDENT_PT 12 + +/* Device Model domain, defaults to 0. */ +#define HVM_PARAM_DM_DOMAIN 13 + +/* ACPI S state: currently support S0 and S3 on x86. */ +#define HVM_PARAM_ACPI_S_STATE 14 + +/* TSS used on Intel when CR0.PE=0. */ +#define HVM_PARAM_VM86_TSS 15 + +/* Boolean: Enable aligning all periodic vpts to reduce interrupts */ +#define HVM_PARAM_VPT_ALIGN 16 + +#define HVM_NR_PARAMS 17 + +#endif /* __XEN_PUBLIC_HVM_PARAMS_H__ */ -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
From: Sheng Yang <sheng@linux.intel.com> Initialize basic pv on hvm features in xen_guest_init. The hook in arch/x86/kernel/setup.c can easily be removed using the new generic hypervisor independent initialization infrastructure present in the linux-2.6-tip tree. Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> Signed-off-by: Sheng Yang <sheng@linux.intel.com> Signed-off-by: Yaozu (Eddie) Dong <eddie.dong@intel.com> --- arch/x86/kernel/setup.c | 2 + arch/x86/xen/enlighten.c | 86 +++++++++++++++++++++++++++++++++++++ drivers/input/xen-kbdfront.c | 2 +- drivers/video/xen-fbfront.c | 2 +- drivers/xen/xenbus/xenbus_probe.c | 21 ++++++++- include/xen/xen.h | 2 + 6 files changed, 110 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index c4851ef..ae9b6cb 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -69,6 +69,7 @@ #include <linux/tboot.h> #include <video/edid.h> +#include <xen/xen.h> #include <asm/mtrr.h> #include <asm/apic.h> @@ -1032,6 +1033,7 @@ void __init setup_arch(char **cmdline_p) probe_nr_irqs_gsi(); kvm_guest_init(); + xen_guest_init(); e820_reserve_resources(); e820_mark_nosave_regions(max_low_pfn); diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index 65d8d79..c1f6545 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -35,6 +35,7 @@ #include <xen/interface/version.h> #include <xen/interface/physdev.h> #include <xen/interface/vcpu.h> +#include <xen/interface/memory.h> #include <xen/features.h> #include <xen/page.h> #include <xen/hvc-console.h> @@ -56,6 +57,7 @@ #include <asm/tlbflush.h> #include <asm/reboot.h> #include <asm/stackprotector.h> +#include <asm/hypervisor.h> #include "xen-ops.h" #include "mmu.h" @@ -1206,3 +1208,87 @@ asmlinkage void __init xen_start_kernel(void) x86_64_start_reservations((char *)__pa_symbol(&boot_params)); #endif } + +static uint32_t xen_cpuid_base(void) +{ + uint32_t base, eax, ebx, ecx, edx; + char signature[13]; + + for (base = 0x40000000; base < 0x40010000; base += 0x100) { + cpuid(base, &eax, &ebx, &ecx, &edx); + *(uint32_t*)(signature + 0) = ebx; + *(uint32_t*)(signature + 4) = ecx; + *(uint32_t*)(signature + 8) = edx; + signature[12] = 0; + + if (!strcmp("XenVMMXenVMM", signature) && ((eax - base) >= 2)) + return base; + } + + return 0; +} + +static int init_hvm_pv_info(int *major, int *minor) +{ + uint32_t eax, ebx, ecx, edx, pages, msr, base; + u64 pfn; + + base = xen_cpuid_base(); + if (!base) + return -EINVAL; + + cpuid(base + 1, &eax, &ebx, &ecx, &edx); + + *major = eax >> 16; + *minor = eax & 0xffff; + printk(KERN_INFO "Xen version %d.%d.\n", *major, *minor); + + cpuid(base + 2, &pages, &msr, &ecx, &edx); + + pfn = __pa(hypercall_page); + wrmsr_safe(msr, (u32)pfn, (u32)(pfn >> 32)); + + xen_setup_features(); + + pv_info = xen_info; + pv_info.kernel_rpl = 0; + + xen_domain_type = XEN_HVM_DOMAIN; + + return 0; +} + +static void __init init_shared_info(void) +{ + struct xen_add_to_physmap xatp; + struct shared_info *shared_info_page; + + shared_info_page = (struct shared_info *) alloc_bootmem_pages(PAGE_SIZE); + xatp.domid = DOMID_SELF; + xatp.idx = 0; + xatp.space = XENMAPSPACE_shared_info; + xatp.gpfn = __pa(shared_info_page) >> PAGE_SHIFT; + if (HYPERVISOR_memory_op(XENMEM_add_to_physmap, &xatp)) + BUG(); + + HYPERVISOR_shared_info = (struct shared_info *)shared_info_page; + + /* Don''t do the full vcpu_info placement stuff until we have a + possible map and a non-dummy shared_info. */ + per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0]; +} + +void __init xen_guest_init(void) +{ + int r; + int major, minor; + + if (xen_pv_domain()) + return; + + r = init_hvm_pv_info(&major, &minor); + if (r < 0) + return; + + init_shared_info(); +} diff --git a/drivers/input/xen-kbdfront.c b/drivers/input/xen-kbdfront.c index e140816..7451d78 100644 --- a/drivers/input/xen-kbdfront.c +++ b/drivers/input/xen-kbdfront.c @@ -339,7 +339,7 @@ static struct xenbus_driver xenkbd_driver = { static int __init xenkbd_init(void) { - if (!xen_domain()) + if (!xen_domain() || xen_hvm_domain()) return -ENODEV; /* Nothing to do if running in dom0. */ diff --git a/drivers/video/xen-fbfront.c b/drivers/video/xen-fbfront.c index fa97d3e..a105a19 100644 --- a/drivers/video/xen-fbfront.c +++ b/drivers/video/xen-fbfront.c @@ -684,7 +684,7 @@ static struct xenbus_driver xenfb_driver = { static int __init xenfb_init(void) { - if (!xen_domain()) + if (!xen_domain() || xen_hvm_domain()) return -ENODEV; /* Nothing to do if running in dom0. */ diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index 3479332..0b05b62 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -56,6 +56,8 @@ #include <xen/events.h> #include <xen/page.h> +#include <xen/hvm.h> + #include "xenbus_comms.h" #include "xenbus_probe.h" @@ -806,10 +808,23 @@ static int __init xenbus_probe_init(void) /* dom0 not yet supported */ } else { xenstored_ready = 1; - xen_store_evtchn = xen_start_info->store_evtchn; - xen_store_mfn = xen_start_info->store_mfn; + if (xen_hvm_domain()) { + uint64_t v = 0; + err = hvm_get_parameter(HVM_PARAM_STORE_EVTCHN, &v); + if (err) + goto out_error; + xen_store_evtchn = (int)v; + err = hvm_get_parameter(HVM_PARAM_STORE_PFN, &v); + if (err) + goto out_error; + xen_store_mfn = (unsigned long)v; + xen_store_interface = ioremap(xen_store_mfn << PAGE_SHIFT, PAGE_SIZE); + } else { + xen_store_evtchn = xen_start_info->store_evtchn; + xen_store_mfn = xen_start_info->store_mfn; + xen_store_interface = mfn_to_virt(xen_store_mfn); + } } - xen_store_interface = mfn_to_virt(xen_store_mfn); /* Initialize the interface to xenstore. */ err = xs_init(); diff --git a/include/xen/xen.h b/include/xen/xen.h index a164024..cb8c48b 100644 --- a/include/xen/xen.h +++ b/include/xen/xen.h @@ -9,8 +9,10 @@ enum xen_domain_type { #ifdef CONFIG_XEN extern enum xen_domain_type xen_domain_type; +extern void xen_guest_init(void); #else #define xen_domain_type XEN_NATIVE +#define xen_guest_init() do { } while (0) #endif #define xen_domain() (xen_domain_type != XEN_NATIVE) -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
From: Sheng Yang <sheng@linux.intel.com> Set the callback to receive evtchns from Xen, using the callback vector delivery mechanism. The traditional way for receiving event channel notifications from Xen is interrupts from the platform PCI device. The callback vector is a newer alternative that allow us to receive notifications on any vcpu and doesn''t need any PCI support. Besides it is possible to avoid any interaction with the emulated lapic even though at the moment we are still going through the IPI vector. Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> Signed-off-by: Sheng Yang <sheng@linux.intel.com> --- arch/x86/xen/enlighten.c | 29 ++++++++++++++++ arch/x86/xen/xen-ops.h | 2 + drivers/xen/events.c | 69 ++++++++++++++++++++++++++++++++++---- include/xen/events.h | 5 +++ include/xen/hvm.h | 6 +++ include/xen/interface/features.h | 3 ++ 6 files changed, 107 insertions(+), 7 deletions(-) diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index c1f6545..ed34b30 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -11,6 +11,7 @@ * Jeremy Fitzhardinge <jeremy@xensource.com>, XenSource Inc, 2007 */ +#include <linux/cpu.h> #include <linux/kernel.h> #include <linux/init.h> #include <linux/smp.h> @@ -38,6 +39,7 @@ #include <xen/interface/memory.h> #include <xen/features.h> #include <xen/page.h> +#include <xen/hvm.h> #include <xen/hvc-console.h> #include <asm/paravirt.h> @@ -78,6 +80,9 @@ struct shared_info xen_dummy_shared_info; void *xen_initial_gdt; +__read_mostly int xen_have_vector_callback; +EXPORT_SYMBOL_GPL(xen_have_vector_callback); + /* * Point at some empty memory to start with. We map the real shared_info * page as soon as fixmap is up and running. @@ -1278,6 +1283,24 @@ static void __init init_shared_info(void) per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0]; } +static int __cpuinit xen_hvm_cpu_notify(struct notifier_block *self, + unsigned long action, void *hcpu) +{ + int cpu = (long)hcpu; + switch (action) { + case CPU_UP_PREPARE: + per_cpu(xen_vcpu, cpu) = &HYPERVISOR_shared_info->vcpu_info[cpu]; + break; + default: + break; + } + return NOTIFY_OK; +} + +static struct notifier_block __cpuinitdata xen_hvm_cpu_notifier = { + .notifier_call = xen_hvm_cpu_notify, +}; + void __init xen_guest_init(void) { int r; @@ -1291,4 +1314,10 @@ void __init xen_guest_init(void) return; init_shared_info(); + + if (xen_feature(XENFEAT_hvm_callback_vector)) + xen_have_vector_callback = 1; + register_cpu_notifier(&xen_hvm_cpu_notifier); + have_vcpu_info_placement = 0; + x86_init.irqs.intr_init = xen_init_IRQ; } diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h index f9153a3..0d0e0e6 100644 --- a/arch/x86/xen/xen-ops.h +++ b/arch/x86/xen/xen-ops.h @@ -38,6 +38,8 @@ void xen_enable_sysenter(void); void xen_enable_syscall(void); void xen_vcpu_restore(void); +void xen_callback_vector(void); + void __init xen_build_dynamic_phys_to_machine(void); void xen_init_irq_ops(void); diff --git a/drivers/xen/events.c b/drivers/xen/events.c index db8f506..8b68fc4 100644 --- a/drivers/xen/events.c +++ b/drivers/xen/events.c @@ -36,10 +36,14 @@ #include <asm/xen/hypercall.h> #include <asm/xen/hypervisor.h> +#include <xen/xen.h> +#include <xen/hvm.h> #include <xen/xen-ops.h> #include <xen/events.h> #include <xen/interface/xen.h> #include <xen/interface/event_channel.h> +#include <xen/interface/hvm/hvm_op.h> +#include <xen/interface/hvm/params.h> /* * This lock protects updates to the following mapping and reference-count @@ -617,17 +621,13 @@ static DEFINE_PER_CPU(unsigned, xed_nesting_count); * a bitset of words which contain pending event bits. The second * level is a bitset of pending events themselves. */ -void xen_evtchn_do_upcall(struct pt_regs *regs) +static void __xen_evtchn_do_upcall(void) { int cpu = get_cpu(); - struct pt_regs *old_regs = set_irq_regs(regs); struct shared_info *s = HYPERVISOR_shared_info; struct vcpu_info *vcpu_info = __get_cpu_var(xen_vcpu); unsigned count; - exit_idle(); - irq_enter(); - do { unsigned long pending_words; @@ -667,10 +667,26 @@ void xen_evtchn_do_upcall(struct pt_regs *regs) } while(count != 1); out: + + put_cpu(); +} + +void xen_evtchn_do_upcall(struct pt_regs *regs) +{ + struct pt_regs *old_regs = set_irq_regs(regs); + + exit_idle(); + irq_enter(); + + __xen_evtchn_do_upcall(); + irq_exit(); set_irq_regs(old_regs); +} - put_cpu(); +void xen_hvm_evtchn_do_upcall(struct pt_regs *regs) +{ + __xen_evtchn_do_upcall(); } /* Rebind a new event channel to an existing irq. */ @@ -933,6 +949,40 @@ static struct irq_chip xen_dynamic_chip __read_mostly = { .retrigger = retrigger_dynirq, }; +int xen_set_callback_via(uint64_t via) +{ + struct xen_hvm_param a; + a.domid = DOMID_SELF; + a.index = HVM_PARAM_CALLBACK_IRQ; + a.value = via; + return HYPERVISOR_hvm_op(HVMOP_set_param, &a); +} +EXPORT_SYMBOL_GPL(xen_set_callback_via); + +void do_hvm_pv_evtchn_intr(void) +{ + xen_hvm_evtchn_do_upcall(get_irq_regs()); +} + +/* the callback vector mechanism is a newer alternative way of receiving + * event channel notifications from Xen: we can receive vector callbacks + * on any vcpus and we don''t need any PCI or IO APIC support */ +void xen_callback_vector(void) +{ + int rc; + uint64_t callback_via; + if (xen_have_vector_callback) { + callback_via = HVM_CALLBACK_VECTOR(X86_PLATFORM_IPI_VECTOR); + rc = xen_set_callback_via(callback_via); + if (rc) { + printk(KERN_ERR "request for callback vector failed\n"); + xen_have_vector_callback = 0; + return; + } + x86_platform_ipi_callback = do_hvm_pv_evtchn_intr; + } +} + void __init xen_init_IRQ(void) { int i; @@ -947,5 +997,10 @@ void __init xen_init_IRQ(void) for (i = 0; i < NR_EVENT_CHANNELS; i++) mask_evtchn(i); - irq_ctx_init(smp_processor_id()); + if (xen_hvm_domain()) { + xen_callback_vector(); + native_init_IRQ(); + } else { + irq_ctx_init(smp_processor_id()); + } } diff --git a/include/xen/events.h b/include/xen/events.h index e68d59a..a3375f7 100644 --- a/include/xen/events.h +++ b/include/xen/events.h @@ -56,4 +56,9 @@ void xen_poll_irq(int irq); /* Determine the IRQ which is bound to an event channel */ unsigned irq_from_evtchn(unsigned int evtchn); +extern int xen_have_vector_callback; +int xen_set_callback_via(uint64_t via); +void xen_evtchn_do_upcall(struct pt_regs *regs); +void xen_hvm_evtchn_do_upcall(struct pt_regs *regs); + #endif /* _XEN_EVENTS_H */ diff --git a/include/xen/hvm.h b/include/xen/hvm.h index 6b0d418..d0cf7a6 100644 --- a/include/xen/hvm.h +++ b/include/xen/hvm.h @@ -3,6 +3,7 @@ #define XEN_HVM_H__ #include <xen/interface/hvm/params.h> +#include <asm/xen/hypercall.h> static inline int hvm_get_parameter(int idx, uint64_t *value) { @@ -21,4 +22,9 @@ static inline int hvm_get_parameter(int idx, uint64_t *value) return r; } +#define HVM_CALLBACK_VIA_TYPE_VECTOR 0x2 +#define HVM_CALLBACK_VIA_TYPE_SHIFT 56 +#define HVM_CALLBACK_VECTOR(x) (((uint64_t)HVM_CALLBACK_VIA_TYPE_VECTOR)<<\ + HVM_CALLBACK_VIA_TYPE_SHIFT | (x)) + #endif /* XEN_HVM_H__ */ diff --git a/include/xen/interface/features.h b/include/xen/interface/features.h index f51b641..8ab08b9 100644 --- a/include/xen/interface/features.h +++ b/include/xen/interface/features.h @@ -41,6 +41,9 @@ /* x86: Does this Xen host support the MMU_PT_UPDATE_PRESERVE_AD hypercall? */ #define XENFEAT_mmu_pt_update_preserve_ad 5 +/* x86: Does this Xen host support the HVM callback vector type? */ +#define XENFEAT_hvm_callback_vector 8 + #define XENFEAT_NR_SUBMAPS 1 #endif /* __XEN_PUBLIC_FEATURES_H__ */ -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-24 18:27 UTC
[Xen-devel] [PATCH 04/11] Fix find_unbound_irq in presence of ioapic irqs.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- drivers/xen/events.c | 11 ++++++++++- 1 files changed, 10 insertions(+), 1 deletions(-) diff --git a/drivers/xen/events.c b/drivers/xen/events.c index 8b68fc4..1356fa2 100644 --- a/drivers/xen/events.c +++ b/drivers/xen/events.c @@ -339,9 +339,18 @@ static int find_unbound_irq(void) int irq; struct irq_desc *desc; - for (irq = 0; irq < nr_irqs; irq++) + for (irq = 0; irq < nr_irqs; irq++) { + desc = irq_to_desc(irq); + /* only 0->15 have init''d desc; handle irq > 16 */ + if (desc == NULL) + break; + if (desc->chip == &no_irq_chip) + break; + if (desc->chip != &xen_dynamic_chip) + continue; if (irq_info[irq].type == IRQT_UNBOUND) break; + } if (irq == nr_irqs) panic("No available IRQ to bind to: increase nr_irqs!\n"); -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-24 18:27 UTC
[Xen-devel] [PATCH 05/11] Xen PCI platform device driver
Add the xen pci platform device driver that is responsible for initializing the grant table and xenbus in PV on HVM mode. Few changes to xenbus and grant table are necessary to allow the delayed initialization in HVM mode. Grant table needs few additional modifications to work in HVM mode. The Xen PCI platform device raises an irq every time an event has been delivered to us. However these interrupts are only delivered to vcpu 0. The Xen PCI platform interrupt handler calls xen_hvm_evtchn_do_upcall that is a little wrapper around __xen_evtchn_do_upcall, the traditional Xen upcall handler, the very same used with traditional PV guests. When running on HVM the event channel upcall is never called while in progress because it is a normal Linux irq handler, therefore we cannot be sure that evtchn_upcall_pending is 0 when returning. For this reason if evtchn_upcall_pending is set by Xen we need to loop again on the event channels set pending otherwise we might loose some event channel deliveries. Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> Signed-off-by: Sheng Yang <sheng@linux.intel.com> --- drivers/xen/Kconfig | 8 ++ drivers/xen/Makefile | 3 +- drivers/xen/events.c | 7 +- drivers/xen/grant-table.c | 69 ++++++++++++-- drivers/xen/platform-pci.c | 186 +++++++++++++++++++++++++++++++++++ drivers/xen/xenbus/xenbus_probe.c | 20 +++- include/linux/pci_ids.h | 3 + include/xen/grant_table.h | 1 + include/xen/interface/grant_table.h | 1 + include/xen/xenbus.h | 1 + 10 files changed, 283 insertions(+), 16 deletions(-) create mode 100644 drivers/xen/platform-pci.c diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig index fad3df2..da312e2 100644 --- a/drivers/xen/Kconfig +++ b/drivers/xen/Kconfig @@ -62,4 +62,12 @@ config XEN_SYS_HYPERVISOR virtual environment, /sys/hypervisor will still be present, but will have no xen contents. +config XEN_PLATFORM_PCI + tristate "xen platform pci device driver" + depends on XEN + help + Driver for the Xen PCI Platform device: it is responsible for + initializing xenbus and grant_table when running in a Xen HVM + domain. As a consequence this driver is required to run any Xen PV + frontend on Xen HVM. endmenu diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile index 7c28434..e392fb7 100644 --- a/drivers/xen/Makefile +++ b/drivers/xen/Makefile @@ -9,4 +9,5 @@ obj-$(CONFIG_XEN_XENCOMM) += xencomm.o obj-$(CONFIG_XEN_BALLOON) += balloon.o obj-$(CONFIG_XEN_DEV_EVTCHN) += evtchn.o obj-$(CONFIG_XENFS) += xenfs/ -obj-$(CONFIG_XEN_SYS_HYPERVISOR) += sys-hypervisor.o \ No newline at end of file +obj-$(CONFIG_XEN_SYS_HYPERVISOR) += sys-hypervisor.o +obj-$(CONFIG_XEN_PLATFORM_PCI) += platform-pci.o diff --git a/drivers/xen/events.c b/drivers/xen/events.c index 1356fa2..c6f561d 100644 --- a/drivers/xen/events.c +++ b/drivers/xen/events.c @@ -673,7 +673,7 @@ static void __xen_evtchn_do_upcall(void) count = __get_cpu_var(xed_nesting_count); __get_cpu_var(xed_nesting_count) = 0; - } while(count != 1); + } while(count != 1 || vcpu_info->evtchn_upcall_pending); out: @@ -733,7 +733,10 @@ static int rebind_irq_to_cpu(unsigned irq, unsigned tcpu) struct evtchn_bind_vcpu bind_vcpu; int evtchn = evtchn_from_irq(irq); - if (!VALID_EVTCHN(evtchn)) + /* events delivered via platform PCI interrupts are always + * routed to vcpu 0 */ + if (!VALID_EVTCHN(evtchn) || + (xen_hvm_domain() && !xen_have_vector_callback)) return -1; /* Send future instances of this interrupt to other vcpu. */ diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c index f66db3b..4c959a5 100644 --- a/drivers/xen/grant-table.c +++ b/drivers/xen/grant-table.c @@ -37,11 +37,13 @@ #include <linux/slab.h> #include <linux/vmalloc.h> #include <linux/uaccess.h> +#include <linux/io.h> #include <xen/xen.h> #include <xen/interface/xen.h> #include <xen/page.h> #include <xen/grant_table.h> +#include <xen/interface/memory.h> #include <asm/xen/hypercall.h> #include <asm/pgtable.h> @@ -59,6 +61,7 @@ static unsigned int boot_max_nr_grant_frames; static int gnttab_free_count; static grant_ref_t gnttab_free_head; static DEFINE_SPINLOCK(gnttab_list_lock); +static unsigned long hvm_pv_resume_frames; static struct grant_entry *shared; @@ -449,6 +452,30 @@ static int gnttab_map(unsigned int start_idx, unsigned int end_idx) unsigned int nr_gframes = end_idx + 1; int rc; + if (xen_hvm_domain()) { + struct xen_add_to_physmap xatp; + unsigned int i = end_idx; + rc = 0; + /* + * Loop backwards, so that the first hypercall has the largest + * index, ensuring that the table will grow only once. + */ + do { + xatp.domid = DOMID_SELF; + xatp.idx = i; + xatp.space = XENMAPSPACE_grant_table; + xatp.gpfn = (hvm_pv_resume_frames >> PAGE_SHIFT) + i; + rc = HYPERVISOR_memory_op(XENMEM_add_to_physmap, &xatp); + if (rc != 0) { + printk(KERN_WARNING + "grant table add_to_physmap failed, err=%d\n", rc); + break; + } + } while (i-- > start_idx); + + return rc; + } + frames = kmalloc(nr_gframes * sizeof(unsigned long), GFP_ATOMIC); if (!frames) return -ENOMEM; @@ -476,9 +503,28 @@ static int gnttab_map(unsigned int start_idx, unsigned int end_idx) int gnttab_resume(void) { - if (max_nr_grant_frames() < nr_grant_frames) + unsigned int max_nr_gframes; + + max_nr_gframes = max_nr_grant_frames(); + if (max_nr_gframes < nr_grant_frames) return -ENOSYS; - return gnttab_map(0, nr_grant_frames - 1); + + if (xen_pv_domain()) + return gnttab_map(0, nr_grant_frames - 1); + + if (!hvm_pv_resume_frames) { + hvm_pv_resume_frames = alloc_xen_mmio(PAGE_SIZE * max_nr_gframes); + shared = ioremap(hvm_pv_resume_frames, PAGE_SIZE * max_nr_gframes); + if (shared == NULL) { + printk(KERN_WARNING + "Fail to ioremap gnttab share frames\n"); + return -ENOMEM; + } + } + + gnttab_map(0, nr_grant_frames - 1); + + return 0; } int gnttab_suspend(void) @@ -505,15 +551,12 @@ static int gnttab_expand(unsigned int req_entries) return rc; } -static int __devinit gnttab_init(void) +int gnttab_init(void) { int i; unsigned int max_nr_glist_frames, nr_glist_frames; unsigned int nr_init_grefs; - if (!xen_domain()) - return -ENODEV; - nr_grant_frames = 1; boot_max_nr_grant_frames = __max_nr_grant_frames(); @@ -557,4 +600,16 @@ static int __devinit gnttab_init(void) return -ENOMEM; } -core_initcall(gnttab_init); +static int __devinit __gnttab_init(void) +{ + /* Delay grant-table initialization in the PV on HVM case */ + if (xen_hvm_domain()) + return 0; + + if (!xen_pv_domain()) + return -ENODEV; + + return gnttab_init(); +} + +core_initcall(__gnttab_init); diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c new file mode 100644 index 0000000..0f78318 --- /dev/null +++ b/drivers/xen/platform-pci.c @@ -0,0 +1,186 @@ +/****************************************************************************** + * platform-pci.c + * + * Xen platform PCI device driver + * Copyright (c) 2005, Intel Corporation. + * Copyright (c) 2007, XenSource Inc. + * Copyright (c) 2010, Citrix + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 59 Temple + * Place - Suite 330, Boston, MA 02111-1307 USA. + * + */ + +#include <asm/io.h> + +#include <linux/interrupt.h> +#include <linux/module.h> +#include <linux/pci.h> + +#include <xen/grant_table.h> +#include <xen/xenbus.h> +#include <xen/events.h> +#include <xen/hvm.h> + +#define DRV_NAME "xen-platform-pci" + +MODULE_AUTHOR("ssmith@xensource.com and stefano.stabellini@eu.citrix.com"); +MODULE_DESCRIPTION("Xen platform PCI device"); +MODULE_LICENSE("GPL"); + +static unsigned long platform_mmio; +static unsigned long platform_mmio_alloc; +static unsigned long platform_mmiolen; + +unsigned long alloc_xen_mmio(unsigned long len) +{ + unsigned long addr; + + addr = platform_mmio + platform_mmio_alloc; + platform_mmio_alloc += len; + BUG_ON(platform_mmio_alloc > platform_mmiolen); + + return addr; +} + +static uint64_t get_callback_via(struct pci_dev *pdev) +{ + u8 pin; + int irq; + + irq = pdev->irq; + if (irq < 16) + return irq; /* ISA IRQ */ + + pin = pdev->pin; + + /* We don''t know the GSI. Specify the PCI INTx line instead. */ + return ((uint64_t)0x01 << 56) | /* PCI INTx identifier */ + ((uint64_t)pci_domain_nr(pdev->bus) << 32) | + ((uint64_t)pdev->bus->number << 16) | + ((uint64_t)(pdev->devfn & 0xff) << 8) | + ((uint64_t)(pin - 1) & 3); +} + +static irqreturn_t do_hvm_evtchn_intr(int irq, void *dev_id) +{ + xen_hvm_evtchn_do_upcall(get_irq_regs()); + return IRQ_HANDLED; +} + +static int xen_allocate_irq(struct pci_dev *pdev) +{ + return request_irq(pdev->irq, do_hvm_evtchn_intr, + IRQF_DISABLED | IRQF_NOBALANCING | IRQF_TRIGGER_RISING, + "xen-platform-pci", pdev); +} + +static int __devinit platform_pci_init(struct pci_dev *pdev, + const struct pci_device_id *ent) +{ + int i, ret; + long ioaddr, iolen; + long mmio_addr, mmio_len; + uint64_t callback_via; + + i = pci_enable_device(pdev); + if (i) + return i; + + ioaddr = pci_resource_start(pdev, 0); + iolen = pci_resource_len(pdev, 0); + + mmio_addr = pci_resource_start(pdev, 1); + mmio_len = pci_resource_len(pdev, 1); + + if (mmio_addr == 0 || ioaddr == 0) { + dev_err(&pdev->dev, "no resources found\n"); + ret = -ENOENT; + } + + if (request_mem_region(mmio_addr, mmio_len, DRV_NAME) == NULL) { + dev_err(&pdev->dev, "MEM I/O resource 0x%lx @ 0x%lx busy\n", + mmio_addr, mmio_len); + ret = -EBUSY; + } + + if (request_region(ioaddr, iolen, DRV_NAME) == NULL) { + dev_err(&pdev->dev, "I/O resource 0x%lx @ 0x%lx busy\n", + iolen, ioaddr); + ret = -EBUSY; + goto out; + } + + platform_mmio = mmio_addr; + platform_mmiolen = mmio_len; + + if (!xen_have_vector_callback) { + ret = xen_allocate_irq(pdev); + if (ret) { + printk(KERN_WARNING "request_irq failed err=%d\n", ret); + goto out; + } + callback_via = get_callback_via(pdev); + ret = xen_set_callback_via(callback_via); + if (ret) { + printk(KERN_WARNING + "Unable to set the evtchn callback err=%d\n", ret); + goto out; + } + } + + ret = gnttab_init(); + if (ret) + goto out; + ret = xenbus_probe_init(); + if (ret) + goto out; + +out: + if (ret) { + release_mem_region(mmio_addr, mmio_len); + release_region(ioaddr, iolen); + pci_disable_device(pdev); + } + + return ret; +} + +static struct pci_device_id platform_pci_tbl[] __devinitdata = { + {PCI_VENDOR_ID_XEN, PCI_DEVICE_ID_XEN_PLATFORM, + PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0}, + {0,} +}; + +MODULE_DEVICE_TABLE(pci, platform_pci_tbl); + +static struct pci_driver platform_driver = { + .name = DRV_NAME, + .probe = platform_pci_init, + .id_table = platform_pci_tbl, +}; + +static int __init platform_pci_module_init(void) +{ + int rc; + + rc = pci_register_driver(&platform_driver); + if (rc) { + printk(KERN_INFO DRV_NAME + ": No platform pci device model found\n"); + return rc; + } + return 0; +} + +module_init(platform_pci_module_init); diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index 0b05b62..dc6ed06 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -782,16 +782,24 @@ void xenbus_probe(struct work_struct *unused) blocking_notifier_call_chain(&xenstore_chain, 0, NULL); } -static int __init xenbus_probe_init(void) +static int __init __xenbus_probe_init(void) +{ + /* Delay initialization in the PV on HVM case */ + if (xen_hvm_domain()) + return 0; + + if (!xen_pv_domain()) + return -ENODEV; + + return xenbus_probe_init(); +} + +int xenbus_probe_init(void) { int err = 0; DPRINTK(""); - err = -ENODEV; - if (!xen_domain()) - goto out_error; - /* Register ourselves with the kernel bus subsystem */ err = bus_register(&xenbus_frontend.bus); if (err) @@ -857,7 +865,7 @@ static int __init xenbus_probe_init(void) return err; } -postcore_initcall(xenbus_probe_init); +postcore_initcall(__xenbus_probe_init); MODULE_LICENSE("GPL"); diff --git a/include/linux/pci_ids.h b/include/linux/pci_ids.h index 9f688d2..64b528d 100644 --- a/include/linux/pci_ids.h +++ b/include/linux/pci_ids.h @@ -2718,3 +2718,6 @@ #define PCI_DEVICE_ID_RME_DIGI32 0x9896 #define PCI_DEVICE_ID_RME_DIGI32_PRO 0x9897 #define PCI_DEVICE_ID_RME_DIGI32_8 0x9898 + +#define PCI_VENDOR_ID_XEN 0x5853 +#define PCI_DEVICE_ID_XEN_PLATFORM 0x0001 diff --git a/include/xen/grant_table.h b/include/xen/grant_table.h index a40f1cd..811cda5 100644 --- a/include/xen/grant_table.h +++ b/include/xen/grant_table.h @@ -51,6 +51,7 @@ struct gnttab_free_callback { u16 count; }; +int gnttab_init(void); int gnttab_suspend(void); int gnttab_resume(void); diff --git a/include/xen/interface/grant_table.h b/include/xen/interface/grant_table.h index 39da93c..39e5717 100644 --- a/include/xen/interface/grant_table.h +++ b/include/xen/interface/grant_table.h @@ -28,6 +28,7 @@ #ifndef __XEN_PUBLIC_GRANT_TABLE_H__ #define __XEN_PUBLIC_GRANT_TABLE_H__ +#include <xen/interface/xen.h> /*********************************** * GRANT TABLE REPRESENTATION diff --git a/include/xen/xenbus.h b/include/xen/xenbus.h index 43e2d7d..ffa97de 100644 --- a/include/xen/xenbus.h +++ b/include/xen/xenbus.h @@ -174,6 +174,7 @@ void unregister_xenbus_watch(struct xenbus_watch *watch); void xs_suspend(void); void xs_resume(void); void xs_suspend_cancel(void); +int xenbus_probe_init(void); /* Used by xenbus_dev to borrow kernel''s store connection. */ void *xenbus_dev_request_and_reply(struct xsd_sockmsg *msg); -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-24 18:27 UTC
[Xen-devel] [PATCH 06/11] Add suspend\resume support for PV on HVM guests.
Suspend\resume requires few different thing on HVM: the suspend hypercall is different, we don''t need to save\restore any memory related setting, but we need to reinitialize the shared info page and the callback mechanism. Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- arch/x86/xen/enlighten.c | 14 +++++++++--- arch/x86/xen/suspend.c | 6 +++++ arch/x86/xen/xen-ops.h | 1 + drivers/xen/manage.c | 45 ++++++++++++++++++++++++++++++++++++++++--- drivers/xen/platform-pci.c | 23 +++++++++++++++++++++- include/xen/xen-ops.h | 3 ++ 6 files changed, 83 insertions(+), 9 deletions(-) diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index ed34b30..cb316b5 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -1263,12 +1263,14 @@ static int init_hvm_pv_info(int *major, int *minor) return 0; } -static void __init init_shared_info(void) +void init_shared_info(void) { + int cpu; struct xen_add_to_physmap xatp; - struct shared_info *shared_info_page; + static struct shared_info *shared_info_page = 0; - shared_info_page = (struct shared_info *) alloc_bootmem_pages(PAGE_SIZE); + if (!shared_info_page) + shared_info_page = (struct shared_info *) alloc_bootmem_pages(PAGE_SIZE); xatp.domid = DOMID_SELF; xatp.idx = 0; xatp.space = XENMAPSPACE_shared_info; @@ -1280,7 +1282,11 @@ static void __init init_shared_info(void) /* Don''t do the full vcpu_info placement stuff until we have a possible map and a non-dummy shared_info. */ - per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0]; + /* init_shared_info is run at resume time too, in that case multiple + * vcpus might be online */ + for_each_online_cpu(cpu) { + per_cpu(xen_vcpu, cpu) = &HYPERVISOR_shared_info->vcpu_info[cpu]; + } } static int __cpuinit xen_hvm_cpu_notify(struct notifier_block *self, diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c index 987267f..ef517ee 100644 --- a/arch/x86/xen/suspend.c +++ b/arch/x86/xen/suspend.c @@ -26,6 +26,12 @@ void xen_pre_suspend(void) BUG(); } +void xen_hvm_post_suspend(int suspend_cancelled) +{ + init_shared_info(); + xen_callback_vector(); +} + void xen_post_suspend(int suspend_cancelled) { xen_build_mfn_list_list(); diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h index 0d0e0e6..caf89ee 100644 --- a/arch/x86/xen/xen-ops.h +++ b/arch/x86/xen/xen-ops.h @@ -39,6 +39,7 @@ void xen_enable_syscall(void); void xen_vcpu_restore(void); void xen_callback_vector(void); +void init_shared_info(void); void __init xen_build_dynamic_phys_to_machine(void); diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c index 2ac4440..0716ba6 100644 --- a/drivers/xen/manage.c +++ b/drivers/xen/manage.c @@ -9,6 +9,7 @@ #include <linux/stop_machine.h> #include <linux/freezer.h> +#include <xen/xen.h> #include <xen/xenbus.h> #include <xen/grant_table.h> #include <xen/events.h> @@ -17,6 +18,7 @@ #include <asm/xen/hypercall.h> #include <asm/xen/page.h> +#include <asm/xen/hypervisor.h> enum shutdown_state { SHUTDOWN_INVALID = -1, @@ -33,10 +35,30 @@ enum shutdown_state { static enum shutdown_state shutting_down = SHUTDOWN_INVALID; #ifdef CONFIG_PM_SLEEP -static int xen_suspend(void *data) +static int xen_hvm_suspend(void *data) { + struct sched_shutdown r = { .reason = SHUTDOWN_suspend }; int *cancelled = data; + + BUG_ON(!irqs_disabled()); + + *cancelled = HYPERVISOR_sched_op(SCHEDOP_shutdown, &r); + + xen_hvm_post_suspend(*cancelled); + gnttab_resume(); + + if (!*cancelled) { + xen_irq_resume(); + xen_timer_resume(); + } + + return 0; +} + +static int xen_suspend(void *data) +{ int err; + int *cancelled = data; BUG_ON(!irqs_disabled()); @@ -112,7 +134,10 @@ static void do_suspend(void) goto out_resume; } - err = stop_machine(xen_suspend, &cancelled, cpumask_of(0)); + if (xen_hvm_domain()) + err = stop_machine(xen_hvm_suspend, &cancelled, cpumask_of(0)); + else + err = stop_machine(xen_suspend, &cancelled, cpumask_of(0)); dpm_resume_noirq(PMSG_RESUME); @@ -261,7 +286,19 @@ static int shutdown_event(struct notifier_block *notifier, return NOTIFY_DONE; } -static int __init setup_shutdown_event(void) +static int __init __setup_shutdown_event(void) +{ + /* Delay initialization in the PV on HVM case */ + if (xen_hvm_domain()) + return 0; + + if (!xen_pv_domain()) + return -ENODEV; + + return xen_setup_shutdown_event(); +} + +int xen_setup_shutdown_event(void) { static struct notifier_block xenstore_notifier = { .notifier_call = shutdown_event @@ -271,4 +308,4 @@ static int __init setup_shutdown_event(void) return 0; } -subsys_initcall(setup_shutdown_event); +subsys_initcall(__setup_shutdown_event); diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c index 0f78318..70395c9 100644 --- a/drivers/xen/platform-pci.c +++ b/drivers/xen/platform-pci.c @@ -31,6 +31,7 @@ #include <xen/xenbus.h> #include <xen/events.h> #include <xen/hvm.h> +#include <xen/xen-ops.h> #define DRV_NAME "xen-platform-pci" @@ -41,6 +42,7 @@ MODULE_LICENSE("GPL"); static unsigned long platform_mmio; static unsigned long platform_mmio_alloc; static unsigned long platform_mmiolen; +static uint64_t callback_via; unsigned long alloc_xen_mmio(unsigned long len) { @@ -85,13 +87,25 @@ static int xen_allocate_irq(struct pci_dev *pdev) "xen-platform-pci", pdev); } +static int platform_pci_resume(struct pci_dev *pdev) +{ + int err; + if (xen_have_vector_callback) + return 0; + err = xen_set_callback_via(callback_via); + if (err) { + printk("platform_pci_resume failure!\n"); + return err; + } + return 0; +} + static int __devinit platform_pci_init(struct pci_dev *pdev, const struct pci_device_id *ent) { int i, ret; long ioaddr, iolen; long mmio_addr, mmio_len; - uint64_t callback_via; i = pci_enable_device(pdev); if (i) @@ -145,6 +159,10 @@ static int __devinit platform_pci_init(struct pci_dev *pdev, ret = xenbus_probe_init(); if (ret) goto out; + ret = xen_setup_shutdown_event(); + if (ret) + goto out; + out: if (ret) { @@ -168,6 +186,9 @@ static struct pci_driver platform_driver = { .name = DRV_NAME, .probe = platform_pci_init, .id_table = platform_pci_tbl, +#ifdef CONFIG_PM + .resume_early = platform_pci_resume, +#endif }; static int __init platform_pci_module_init(void) diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h index 883a21b..46bc81e 100644 --- a/include/xen/xen-ops.h +++ b/include/xen/xen-ops.h @@ -7,6 +7,7 @@ DECLARE_PER_CPU(struct vcpu_info *, xen_vcpu); void xen_pre_suspend(void); void xen_post_suspend(int suspend_cancelled); +void xen_hvm_post_suspend(int suspend_cancelled); void xen_mm_pin_all(void); void xen_mm_unpin_all(void); @@ -14,4 +15,6 @@ void xen_mm_unpin_all(void); void xen_timer_resume(void); void xen_arch_resume(void); +int xen_setup_shutdown_event(void); + #endif /* INCLUDE_XEN_OPS_H */ -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-24 18:27 UTC
[Xen-devel] [PATCH 07/11] Allow xen platform pci device to be compiled as a module
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- drivers/xen/events.c | 1 + drivers/xen/grant-table.c | 20 +++++++++++--------- drivers/xen/manage.c | 1 + drivers/xen/platform-pci.c | 3 +++ drivers/xen/xenbus/xenbus_probe.c | 1 + include/xen/grant_table.h | 3 +++ 6 files changed, 20 insertions(+), 9 deletions(-) diff --git a/drivers/xen/events.c b/drivers/xen/events.c index c6f561d..e6dd8ba 100644 --- a/drivers/xen/events.c +++ b/drivers/xen/events.c @@ -697,6 +697,7 @@ void xen_hvm_evtchn_do_upcall(struct pt_regs *regs) { __xen_evtchn_do_upcall(); } +EXPORT_SYMBOL_GPL(xen_hvm_evtchn_do_upcall); /* Rebind a new event channel to an existing irq. */ void rebind_evtchn_irq(int evtchn, int irq) diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c index 4c959a5..64bcad1 100644 --- a/drivers/xen/grant-table.c +++ b/drivers/xen/grant-table.c @@ -61,7 +61,8 @@ static unsigned int boot_max_nr_grant_frames; static int gnttab_free_count; static grant_ref_t gnttab_free_head; static DEFINE_SPINLOCK(gnttab_list_lock); -static unsigned long hvm_pv_resume_frames; +unsigned long xen_hvm_resume_frames; +EXPORT_SYMBOL_GPL(xen_hvm_resume_frames); static struct grant_entry *shared; @@ -436,7 +437,7 @@ static unsigned int __max_nr_grant_frames(void) return query.max_nr_frames; } -static inline unsigned int max_nr_grant_frames(void) +unsigned int gnttab_max_nr_grant_frames(void) { unsigned int xen_max = __max_nr_grant_frames(); @@ -444,6 +445,7 @@ static inline unsigned int max_nr_grant_frames(void) return boot_max_nr_grant_frames; return xen_max; } +EXPORT_SYMBOL_GPL(gnttab_max_nr_grant_frames); static int gnttab_map(unsigned int start_idx, unsigned int end_idx) { @@ -464,7 +466,7 @@ static int gnttab_map(unsigned int start_idx, unsigned int end_idx) xatp.domid = DOMID_SELF; xatp.idx = i; xatp.space = XENMAPSPACE_grant_table; - xatp.gpfn = (hvm_pv_resume_frames >> PAGE_SHIFT) + i; + xatp.gpfn = (xen_hvm_resume_frames >> PAGE_SHIFT) + i; rc = HYPERVISOR_memory_op(XENMEM_add_to_physmap, &xatp); if (rc != 0) { printk(KERN_WARNING @@ -492,7 +494,7 @@ static int gnttab_map(unsigned int start_idx, unsigned int end_idx) BUG_ON(rc || setup.status); - rc = arch_gnttab_map_shared(frames, nr_gframes, max_nr_grant_frames(), + rc = arch_gnttab_map_shared(frames, nr_gframes, gnttab_max_nr_grant_frames(), &shared); BUG_ON(rc); @@ -505,16 +507,15 @@ int gnttab_resume(void) { unsigned int max_nr_gframes; - max_nr_gframes = max_nr_grant_frames(); + max_nr_gframes = gnttab_max_nr_grant_frames(); if (max_nr_gframes < nr_grant_frames) return -ENOSYS; if (xen_pv_domain()) return gnttab_map(0, nr_grant_frames - 1); - if (!hvm_pv_resume_frames) { - hvm_pv_resume_frames = alloc_xen_mmio(PAGE_SIZE * max_nr_gframes); - shared = ioremap(hvm_pv_resume_frames, PAGE_SIZE * max_nr_gframes); + if (!shared) { + shared = ioremap(xen_hvm_resume_frames, PAGE_SIZE * max_nr_gframes); if (shared == NULL) { printk(KERN_WARNING "Fail to ioremap gnttab share frames\n"); @@ -541,7 +542,7 @@ static int gnttab_expand(unsigned int req_entries) cur = nr_grant_frames; extra = ((req_entries + (GREFS_PER_GRANT_FRAME-1)) / GREFS_PER_GRANT_FRAME); - if (cur + extra > max_nr_grant_frames()) + if (cur + extra > gnttab_max_nr_grant_frames()) return -ENOSPC; rc = gnttab_map(cur, cur + extra - 1); @@ -599,6 +600,7 @@ int gnttab_init(void) kfree(gnttab_list); return -ENOMEM; } +EXPORT_SYMBOL_GPL(gnttab_init); static int __devinit __gnttab_init(void) { diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c index 0716ba6..f5162e4 100644 --- a/drivers/xen/manage.c +++ b/drivers/xen/manage.c @@ -307,5 +307,6 @@ int xen_setup_shutdown_event(void) return 0; } +EXPORT_SYMBOL_GPL(xen_setup_shutdown_event); subsys_initcall(__setup_shutdown_event); diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c index 70395c9..07c821c 100644 --- a/drivers/xen/platform-pci.c +++ b/drivers/xen/platform-pci.c @@ -106,6 +106,7 @@ static int __devinit platform_pci_init(struct pci_dev *pdev, int i, ret; long ioaddr, iolen; long mmio_addr, mmio_len; + unsigned int max_nr_gframes; i = pci_enable_device(pdev); if (i) @@ -153,6 +154,8 @@ static int __devinit platform_pci_init(struct pci_dev *pdev, } } + max_nr_gframes = gnttab_max_nr_grant_frames(); + xen_hvm_resume_frames = alloc_xen_mmio(PAGE_SIZE * max_nr_gframes); ret = gnttab_init(); if (ret) goto out; diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index dc6ed06..cf971da 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -864,6 +864,7 @@ int xenbus_probe_init(void) out_error: return err; } +EXPORT_SYMBOL_GPL(xenbus_probe_init); postcore_initcall(__xenbus_probe_init); diff --git a/include/xen/grant_table.h b/include/xen/grant_table.h index 811cda5..2a58ee0 100644 --- a/include/xen/grant_table.h +++ b/include/xen/grant_table.h @@ -113,6 +113,9 @@ int arch_gnttab_map_shared(unsigned long *frames, unsigned long nr_gframes, void arch_gnttab_unmap_shared(struct grant_entry *shared, unsigned long nr_gframes); +extern unsigned long xen_hvm_resume_frames; +unsigned int gnttab_max_nr_grant_frames(void); + #define gnttab_map_vaddr(map) ((void *)(map.host_virt_addr)) #endif /* __ASM_GNTTAB_H__ */ -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-24 18:27 UTC
[Xen-devel] [PATCH 08/11] Fix possible NULL pointer dereference in print_IO_APIC
Make sure chip_data is not NULL before accessing it (the VIRQ_TIMER handler and virq handlers in general don''t have any chip_data). Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- arch/x86/kernel/apic/io_apic.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c index eb2789c..c64499c 100644 --- a/arch/x86/kernel/apic/io_apic.c +++ b/arch/x86/kernel/apic/io_apic.c @@ -1732,6 +1732,8 @@ __apicdebuginit(void) print_IO_APIC(void) struct irq_pin_list *entry; cfg = desc->chip_data; + if (!cfg) + continue; entry = cfg->irq_2_pin; if (!entry) continue; -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-24 18:27 UTC
[Xen-devel] [PATCH 09/11] __setup_vector_irq: handle NULL chip_data
the VIRQ_TIMER handler and virq handlers in general don''t have any chip_data. Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- arch/x86/kernel/apic/io_apic.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c index c64499c..4d3d391 100644 --- a/arch/x86/kernel/apic/io_apic.c +++ b/arch/x86/kernel/apic/io_apic.c @@ -1269,6 +1269,8 @@ void __setup_vector_irq(int cpu) /* Mark the inuse vectors */ for_each_irq_desc(irq, desc) { cfg = desc->chip_data; + if (!cfg) + continue; /* * If it is a legacy IRQ handled by the legacy PIC, this cpu -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-24 18:27 UTC
[Xen-devel] [PATCH 10/11] Support VIRQ_TIMER and pvclock on HVM
Use the paravirtualized timer (VIRQ_TIMER) as timer and the paravirtualized clock (pvclock) as wallclock on HVM domains too. Any interaction with emulated hardware is not very fast, so using the paravirtualized timer instead of the emulated hpet is defenitely a performance improvement. Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- arch/x86/xen/enlighten.c | 31 +++++++++++++++++++++++++++++++ arch/x86/xen/suspend.c | 4 ++++ include/xen/interface/features.h | 3 +++ 3 files changed, 38 insertions(+), 0 deletions(-) diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index cb316b5..88e8a09 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -1307,6 +1307,36 @@ static struct notifier_block __cpuinitdata xen_hvm_cpu_notifier = { .notifier_call = xen_hvm_cpu_notify, }; +static void xen_hvm_setup_cpu_clockevents(void) +{ + int cpu = smp_processor_id(); + xen_setup_runstate_info(cpu); + xen_setup_timer(cpu); + xen_setup_cpu_clockevents(); +} + +static void init_hvm_time(void) +{ + /* vector callback is needed otherwise we cannot receive interrupts + * on cpu > 0 */ + if (!xen_have_vector_callback && num_present_cpus() > 1) + return; + if (!xen_feature(XENFEAT_hvm_safe_pvclock)) { + printk(KERN_INFO "Xen doesn''t support pvclock on HVM," + "disable pv timer\n"); + return; + } + + pv_time_ops = xen_time_ops; + x86_init.timers.timer_init = xen_time_init; + x86_init.timers.setup_percpu_clockev = x86_init_noop; + x86_cpuinit.setup_percpu_clockev = xen_hvm_setup_cpu_clockevents; + + x86_platform.calibrate_tsc = xen_tsc_khz; + x86_platform.get_wallclock = xen_get_wallclock; + x86_platform.set_wallclock = xen_set_wallclock; +} + void __init xen_guest_init(void) { int r; @@ -1326,4 +1356,5 @@ void __init xen_guest_init(void) register_cpu_notifier(&xen_hvm_cpu_notifier); have_vcpu_info_placement = 0; x86_init.irqs.intr_init = xen_init_IRQ; + init_hvm_time(); } diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c index ef517ee..02c3253 100644 --- a/arch/x86/xen/suspend.c +++ b/arch/x86/xen/suspend.c @@ -28,8 +28,12 @@ void xen_pre_suspend(void) void xen_hvm_post_suspend(int suspend_cancelled) { + int cpu; init_shared_info(); xen_callback_vector(); + for_each_online_cpu(cpu) { + xen_setup_runstate_info(cpu); + } } void xen_post_suspend(int suspend_cancelled) diff --git a/include/xen/interface/features.h b/include/xen/interface/features.h index 8ab08b9..70d2563 100644 --- a/include/xen/interface/features.h +++ b/include/xen/interface/features.h @@ -44,6 +44,9 @@ /* x86: Does this Xen host support the HVM callback vector type? */ #define XENFEAT_hvm_callback_vector 8 +/* x86: pvclock algorithm is safe to use on HVM */ +#define XENFEAT_hvm_safe_pvclock 9 + #define XENFEAT_NR_SUBMAPS 1 #endif /* __XEN_PUBLIC_FEATURES_H__ */ -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-24 18:27 UTC
[Xen-devel] [PATCH 11/11] Unplug emulated disks and nics
Add a xen_emul_unplug command line option to the kernel to unplug xen emulated disks and nics. Set the default value of xen_emul_unplug depending on whether or not the Xen PV frontends and the Xen platform PCI driver have been compiled for this kernel (modules or built-in are both OK). Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> --- Documentation/kernel-parameters.txt | 11 +++ arch/x86/xen/Makefile | 2 +- arch/x86/xen/enlighten.c | 1 + arch/x86/xen/platform-pci-unplug.c | 131 ++++++++++++++++++++++++++++++++++ arch/x86/xen/xen-ops.h | 1 + drivers/xen/platform-pci.c | 4 + include/xen/interface/platform_pci.h | 46 ++++++++++++ include/xen/platform_pci.h | 34 +++++++++ 8 files changed, 229 insertions(+), 1 deletions(-) create mode 100644 arch/x86/xen/platform-pci-unplug.c create mode 100644 include/xen/interface/platform_pci.h create mode 100644 include/xen/platform_pci.h diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index 839b21b..716eea8 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -113,6 +113,7 @@ parameter is applicable: More X86-64 boot options can be found in Documentation/x86/x86_64/boot-options.txt . X86 Either 32bit or 64bit x86 (same as X86-32+X86-64) + XEN Xen support is enabled In addition, the following text indicates that the option: @@ -2834,6 +2835,16 @@ and is between 256 and 4096 characters. It is defined in the file xd= [HW,XT] Original XT pre-IDE (RLL encoded) disks. xd_geo= See header of drivers/block/xd.c. + xen_emul_unplug= [HW,X86,XEN] + Unplug Xen emulated devices + Format: [unplug0,][unplug1] + ide-disks -- unplug primary master IDE devices + aux-ide-disks -- unplug non-primary-master IDE devices + nics -- unplug network devices + all -- unplug all emulated devices (NICs and IDE disks) + ignore -- continue loading the Xen platform PCI driver even + if the version check failed + xirc2ps_cs= [NET,PCMCIA] Format: <irq>,<irq_mask>,<io>,<full_duplex>,<do_sound>,<lockup_hack>[,<irq2>[,<irq3>[,<irq4>]]] diff --git a/arch/x86/xen/Makefile b/arch/x86/xen/Makefile index 3bb4fc2..9309546 100644 --- a/arch/x86/xen/Makefile +++ b/arch/x86/xen/Makefile @@ -12,7 +12,7 @@ CFLAGS_mmu.o := $(nostackp) obj-y := enlighten.o setup.o multicalls.o mmu.o irq.o \ time.o xen-asm.o xen-asm_$(BITS).o \ - grant-table.o suspend.o + grant-table.o suspend.o platform-pci-unplug.o obj-$(CONFIG_SMP) += smp.o obj-$(CONFIG_PARAVIRT_SPINLOCKS)+= spinlock.o diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index 88e8a09..fbf7c43 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -1354,6 +1354,7 @@ void __init xen_guest_init(void) if (xen_feature(XENFEAT_hvm_callback_vector)) xen_have_vector_callback = 1; register_cpu_notifier(&xen_hvm_cpu_notifier); + xen_unplug_emulated_devices(); have_vcpu_info_placement = 0; x86_init.irqs.intr_init = xen_init_IRQ; init_hvm_time(); diff --git a/arch/x86/xen/platform-pci-unplug.c b/arch/x86/xen/platform-pci-unplug.c new file mode 100644 index 0000000..4094f29 --- /dev/null +++ b/arch/x86/xen/platform-pci-unplug.c @@ -0,0 +1,131 @@ +/****************************************************************************** + * platform-pci-unplug.c + * + * Xen platform PCI device driver + * Copyright (c) 2010, Citrix + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 59 Temple + * Place - Suite 330, Boston, MA 02111-1307 USA. + * + */ + +#include <asm/io.h> + +#include <linux/init.h> +#include <linux/module.h> + +#include <xen/platform_pci.h> +#include <xen/interface/platform_pci.h> + +int xen_platform_pci; +EXPORT_SYMBOL_GPL(xen_platform_pci); +static int xen_emul_unplug; + +static int __init check_platform_magic(void) +{ + short magic; + char protocol; + + magic = inw(XEN_IOPORT_MAGIC); + if (magic != XEN_IOPORT_MAGIC_VAL) { + printk(KERN_ERR "Xen Platform PCI: unrecognised magic value\n"); + return -1; + } + + protocol = inb(XEN_IOPORT_PROTOVER); + + printk(KERN_DEBUG "Xen Platform PCI: I/O protocol version %d\n", + protocol); + + switch (protocol) { + case 1: + outw(XEN_IOPORT_LINUX_PRODNUM, XEN_IOPORT_PRODNUM); + outl(XEN_IOPORT_LINUX_DRVVER, XEN_IOPORT_DRVVER); + if (inw(XEN_IOPORT_MAGIC) != XEN_IOPORT_MAGIC_VAL) { + printk(KERN_ERR "Xen Platform: blacklisted by host\n"); + return -3; + } + break; + default: + printk(KERN_WARNING "Xen Platform PCI: unknown I/O protocol version"); + return -2; + } + + return 0; +} + +void __init xen_unplug_emulated_devices(void) +{ + int r; + + /* check the version of the xen platform PCI device */ + r = check_platform_magic(); + /* If the version matches enable the Xen platform PCI driver. + * Also enable the Xen platform PCI driver if the version is really old + * and the user told us to ignore it. */ + if (!r || (r == -1 && (xen_emul_unplug & UNPLUG_IGNORE))) + xen_platform_pci = 1; + /* Set the default value of xen_emul_unplug depending on whether or + * not the Xen PV frontends and the Xen platform PCI driver have + * been compiled for this kernel (modules or built-in are both OK). */ + if (xen_platform_pci && !xen_emul_unplug) { +#if (defined(CONFIG_XEN_NETDEV_FRONTEND) || \ + defined(CONFIG_XEN_NETDEV_FRONTEND_MODULE)) && \ + (defined(CONFIG_XEN_PLATFORM_PCI) || \ + defined(CONFIG_XEN_PLATFORM_PCI_MODULE)) + printk(KERN_INFO "Netfront and the Xen platform PCI driver have " + "been compiled for this kernel: unplug emulated NICs.\n"); + xen_emul_unplug |= UNPLUG_ALL_NICS; +#endif +#if (defined(CONFIG_XEN_BLKDEV_FRONTEND) || \ + defined(CONFIG_XEN_BLKDEV_FRONTEND_MODULE)) && \ + (defined(CONFIG_XEN_PLATFORM_PCI) || \ + defined(CONFIG_XEN_PLATFORM_PCI_MODULE)) + printk(KERN_INFO "Blkfront and the Xen platform PCI driver have " + "been compiled for this kernel: unplug emulated disks.\n" + "You might have to change the root device\n" + "from /dev/hd[a-d] to /dev/xvd[a-d]\n" + "in your root= kernel command line option\n"); + xen_emul_unplug |= UNPLUG_ALL_IDE_DISKS; +#endif + } + /* Now unplug the emulated devices */ + if (xen_platform_pci && !(xen_emul_unplug & UNPLUG_IGNORE)) + outw(xen_emul_unplug, XEN_IOPORT_UNPLUG); +} + +static int __init parse_xen_emul_unplug(char *arg) +{ + char *p, *q; + + for (p = arg; p; p = q) { + q = strchr(arg, '',''); + if (q) + *q++ = ''\0''; + if (!strcmp(p, "all")) + xen_emul_unplug |= UNPLUG_ALL; + else if (!strcmp(p, "ide-disks")) + xen_emul_unplug |= UNPLUG_ALL_IDE_DISKS; + else if (!strcmp(p, "aux-ide-disks")) + xen_emul_unplug |= UNPLUG_AUX_IDE_DISKS; + else if (!strcmp(p, "nics")) + xen_emul_unplug |= UNPLUG_ALL_NICS; + else if (!strcmp(p, "ignore")) + xen_emul_unplug |= UNPLUG_IGNORE; + else + printk(KERN_WARNING "unrecognised option ''%s'' " + "in module parameter ''dev_unplug''\n", p); + } + return 0; +} +early_param("xen_emul_unplug", parse_xen_emul_unplug); diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h index caf89ee..e146f5f 100644 --- a/arch/x86/xen/xen-ops.h +++ b/arch/x86/xen/xen-ops.h @@ -40,6 +40,7 @@ void xen_vcpu_restore(void); void xen_callback_vector(void); void init_shared_info(void); +void __init xen_unplug_emulated_devices(void); void __init xen_build_dynamic_phys_to_machine(void); diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c index 07c821c..de3ce1e 100644 --- a/drivers/xen/platform-pci.c +++ b/drivers/xen/platform-pci.c @@ -27,6 +27,7 @@ #include <linux/module.h> #include <linux/pci.h> +#include <xen/platform_pci.h> #include <xen/grant_table.h> #include <xen/xenbus.h> #include <xen/events.h> @@ -198,6 +199,9 @@ static int __init platform_pci_module_init(void) { int rc; + if (!xen_platform_pci) + return -ENODEV; + rc = pci_register_driver(&platform_driver); if (rc) { printk(KERN_INFO DRV_NAME diff --git a/include/xen/interface/platform_pci.h b/include/xen/interface/platform_pci.h new file mode 100644 index 0000000..720eaf5 --- /dev/null +++ b/include/xen/interface/platform_pci.h @@ -0,0 +1,46 @@ +/****************************************************************************** + * platform_pci.h + * + * Interface for granting foreign access to page frames, and receiving + * page-ownership transfers. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to + * deal in the Software without restriction, including without limitation the + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or + * sell copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS IN THE SOFTWARE. + */ + +#ifndef __XEN_PUBLIC_PLATFORM_PCI_H__ +#define __XEN_PUBLIC_PLATFORM_PCI_H__ + +#define XEN_IOPORT_BASE 0x10 + +#define XEN_IOPORT_PLATFLAGS (XEN_IOPORT_BASE + 0) /* 1 byte access (R/W) */ +#define XEN_IOPORT_MAGIC (XEN_IOPORT_BASE + 0) /* 2 byte access (R) */ +#define XEN_IOPORT_UNPLUG (XEN_IOPORT_BASE + 0) /* 2 byte access (W) */ +#define XEN_IOPORT_DRVVER (XEN_IOPORT_BASE + 0) /* 4 byte access (W) */ + +#define XEN_IOPORT_SYSLOG (XEN_IOPORT_BASE + 2) /* 1 byte access (W) */ +#define XEN_IOPORT_PROTOVER (XEN_IOPORT_BASE + 2) /* 1 byte access (R) */ +#define XEN_IOPORT_PRODNUM (XEN_IOPORT_BASE + 2) /* 2 byte access (W) */ + +#define UNPLUG_ALL_IDE_DISKS 1 +#define UNPLUG_ALL_NICS 2 +#define UNPLUG_AUX_IDE_DISKS 4 +#define UNPLUG_ALL 7 +#define UNPLUG_IGNORE 8 + +#endif /* __XEN_PUBLIC_PLATFORM_PCI_H__ */ diff --git a/include/xen/platform_pci.h b/include/xen/platform_pci.h new file mode 100644 index 0000000..ae04d8a --- /dev/null +++ b/include/xen/platform_pci.h @@ -0,0 +1,34 @@ +/****************************************************************************** + * platform-pci.h + * + * Xen platform PCI device driver + * Copyright (c) 2004, Intel Corporation. <xiaofeng.ling@intel.com> + * Copyright (c) 2007, XenSource Inc. + * Copyright (c) 2010, Citrix + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 59 Temple + * Place - Suite 330, Boston, MA 02111-1307 USA. + */ + +#ifndef _XEN_PLATFORM_PCI_H +#define _XEN_PLATFORM_PCI_H + +#include <linux/version.h> + +#define XEN_IOPORT_MAGIC_VAL 0x49d2 +#define XEN_IOPORT_LINUX_PRODNUM 0xffff +#define XEN_IOPORT_LINUX_DRVVER ((LINUX_VERSION_CODE << 8) + 0x0) + +extern int xen_platform_pci; + +#endif /* _XEN_PLATFORM_PCI_H */ -- 1.5.4.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
The subject should be [PATCH 0 of 11] PV on HVM Xen, sorry about that. On Mon, 24 May 2010, Stefano Stabellini wrote:> Hi all, > this is another update of the PV on HVM Xen series that addresses > Jeremy''s comments. > The platform_pci hooks have been removed, suspend/resume for HVM > domains is now much more similar to the PV case and shares the same > do_suspend function. > Alloc_xen_mmio_hook has been removed has well, now the memory allocation for > the grant table is done by the xen platform pci driver directly. > The per_cpu xen_vcpu variable is set by a cpu_notifier function so that > secondary vcpus have the variable set correctly no matter what the xen > features are on the host. > The kernel command line option xen_unplug has been renamed to > xen_emul_unplug and the code that makes use of it has been moved to a > separate file (arch/x86/xen/platform-pci-unplug.c). > Xen_unplug_emulated_devices is now able to detect if blkfront, netfront > and the Xen platform PCI driver have been compiled, and set the default > value of xen_emul_unplug accordingly. > The patch "Initialize xenbus device structs with ENODEV as > default" has been removed from the series and it will be sent > separately. > Finally the comments on most of the patches have been improved. > > The series is based on 2.6.34 and supports Xen PV frontends running > in a HVM domain, including netfront, blkfront and the VIRQ_TIMER. > > In order to be able to use VIRQ_TIMER and to improve performances you > need a patch to Xen to implement the vector callback mechanism > for event channel delivery. > > A git tree is also available here: > > git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git > > branch name 2.6.34-pvhvm-v2. > > Cheers, > > Stefano >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> In order to be able to use VIRQ_TIMER and to improve performances you > needa patch to Xen to implement the vector callback mechanism> for eventchannel delivery. Where to get it ? Boris. --- On Mon, 5/24/10, Stefano Stabellini <Stefano.Stabellini@eu.citrix.com> wrote: From: Stefano Stabellini <Stefano.Stabellini@eu.citrix.com> Subject: [Xen-devel] [PATCH 0 of 12] PV on HVM Xen To: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org> Cc: "Stefano Stabellini" <Stefano.Stabellini@eu.citrix.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>, "Don Dutile" <ddutile@redhat.com>, "Sheng Yang" <sheng@linux.intel.com> Date: Monday, May 24, 2010, 2:25 PM Hi all, this is another update of the PV on HVM Xen series that addresses Jeremy''s comments. The platform_pci hooks have been removed, suspend/resume for HVM domains is now much more similar to the PV case and shares the same do_suspend function. Alloc_xen_mmio_hook has been removed has well, now the memory allocation for the grant table is done by the xen platform pci driver directly. The per_cpu xen_vcpu variable is set by a cpu_notifier function so that secondary vcpus have the variable set correctly no matter what the xen features are on the host. The kernel command line option xen_unplug has been renamed to xen_emul_unplug and the code that makes use of it has been moved to a separate file (arch/x86/xen/platform-pci-unplug.c). Xen_unplug_emulated_devices is now able to detect if blkfront, netfront and the Xen platform PCI driver have been compiled, and set the default value of xen_emul_unplug accordingly. The patch "Initialize xenbus device structs with ENODEV as default" has been removed from the series and it will be sent separately. Finally the comments on most of the patches have been improved. The series is based on 2.6.34 and supports Xen PV frontends running in a HVM domain, including netfront, blkfront and the VIRQ_TIMER. In order to be able to use VIRQ_TIMER and to improve performances you need a patch to Xen to implement the vector callback mechanism for event channel delivery. A git tree is also available here: git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git branch name 2.6.34-pvhvm-v2. Cheers, Stefano _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> In order to be able to use VIRQ_TIMER and to improve performancesyou> needa patch to Xen to implement the vector callback mechanism> forevent channel delivery. Where to get it ? Boris. --- On Mon, 5/24/10, Stefano Stabellini <Stefano.Stabellini@eu.citrix.com> wrote: From: Stefano Stabellini <Stefano.Stabellini@eu.citrix.com> Subject: [Xen-devel] [PATCH 0 of 12] PV on HVM Xen To: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org> Cc: "Stefano Stabellini" <Stefano.Stabellini@eu.citrix.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>, "Don Dutile" <ddutile@redhat.com>, "Sheng Yang" <sheng@linux.intel.com> Date: Monday, May 24, 2010, 2:25 PM Hi all, this is another update of the PV on HVM Xen series that addresses Jeremy''s comments. The platform_pci hooks have been removed, suspend/resume for HVM domains is now much more similar to the PV case and shares the same do_suspend function. Alloc_xen_mmio_hook has been removed has well, now the memory allocation for the grant table is done by the xen platform pci driver directly. The per_cpu xen_vcpu variable is set by a cpu_notifier function so that secondary vcpus have the variable set correctly no matter what the xen features are on the host. The kernel command line option xen_unplug has been renamed to xen_emul_unplug and the code that makes use of it has been moved to a separate file (arch/x86/xen/platform-pci-unplug.c). Xen_unplug_emulated_devices is now able to detect if blkfront, netfront and the Xen platform PCI driver have been compiled, and set the default value of xen_emul_unplug accordingly. The patch "Initialize xenbus device structs with ENODEV as default" has been removed from the series and it will be sent separately. Finally the comments on most of the patches have been improved. The series is based on 2.6.34 and supports Xen PV frontends running in a HVM domain, including netfront, blkfront and the VIRQ_TIMER. In order to be able to use VIRQ_TIMER and to improve performances you need a patch to Xen to implement the vector callback mechanism for event channel delivery. A git tree is also available here: git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git branch name 2.6.34-pvhvm-v2. Cheers, Stefano _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, 24 May 2010, Boris Derzhavets wrote:> > In order to be able to use VIRQ_TIMER and to improve performances you > > need a patch to Xen to implement the vector callback mechanism > > for event channel delivery. > > Where to get it ? >It is this one: http://lists.xensource.com/archives/html/xen-devel/2010-05/msg00875.html _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, May 24, 2010 at 06:28:42PM +0100, Stefano Stabellini wrote:> On Tue, 18 May 2010, Christian Tramnitz wrote: > > Hi Stefano, > > > > what are the particular advantages of running PVonHVM vs. traditional PV > > (vs pure HVM)? > > I''d like to update the wiki with some info about it... > > > > Sorry for the late reply, but I wanted to have some numbers to compare > the three cases... > > PVonHVM guests are as easy to install, use and maintain as pure HVM > guests but provide performances comparable or even better than > traditional PV guests. > > In particular you''ll find that PVonHVM guests are always faster than > pure HVM guests, most of the times faster than 64 bit PV guests but > probably slower than 32 bit PV guests. >Hmm.. are you sure kernbench is a proper benchmark for disk IO? kernel compilations fire up a lot of new processes, and that favours HVM guests. Basicly it seems really weird to me that in the results HVM (without PV drivers) is faster than pure PV.. If you compare pure disk-IO PV vs. HVM the difference is usually big.. PV beating HVM hands down. -- Pasi> > The following are the results of kernbench run in optimal mode on a 4 > vcpu linux guest with 512MB of ram. > The host is a Dell Precision T3500 (HAP enabled): > testbox:~# cat /proc/cpuinfo > processor : 0 > vendor_id : GenuineIntel > cpu family : 6 > model : 26 > model name : Intel(R) Xeon(R) CPU E5520 @ 2.27GHz > stepping : 5 > cpu MHz : 2266.806 > cache size : 8192 KB > fdiv_bug : no > hlt_bug : no > f00f_bug : no > coma_bug : no > fpu : yes > fpu_exception : yes > cpuid level : 11 > wp : yes > flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht nx constant_tsc pni est ssse3 sse4_1 sse4_2 popcnt ida > bogomips : 4538.00 > clflush size : 64 > power management: > > processor : 1 > vendor_id : GenuineIntel > cpu family : 6 > model : 26 > model name : Intel(R) Xeon(R) CPU E5520 @ 2.27GHz > stepping : 5 > cpu MHz : 2266.806 > cache size : 8192 KB > fdiv_bug : no > hlt_bug : no > f00f_bug : no > coma_bug : no > fpu : yes > fpu_exception : yes > cpuid level : 11 > wp : yes > flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht nx constant_tsc pni est ssse3 sse4_1 sse4_2 popcnt ida > bogomips : 4538.00 > clflush size : 64 > power management: > > processor : 2 > vendor_id : GenuineIntel > cpu family : 6 > model : 26 > model name : Intel(R) Xeon(R) CPU E5520 @ 2.27GHz > stepping : 5 > cpu MHz : 2266.806 > cache size : 8192 KB > fdiv_bug : no > hlt_bug : no > f00f_bug : no > coma_bug : no > fpu : yes > fpu_exception : yes > cpuid level : 11 > wp : yes > flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht nx constant_tsc pni est ssse3 sse4_1 sse4_2 popcnt ida > bogomips : 4538.00 > clflush size : 64 > power management: > > processor : 3 > vendor_id : GenuineIntel > cpu family : 6 > model : 26 > model name : Intel(R) Xeon(R) CPU E5520 @ 2.27GHz > stepping : 5 > cpu MHz : 2266.806 > cache size : 8192 KB > fdiv_bug : no > hlt_bug : no > f00f_bug : no > coma_bug : no > fpu : yes > fpu_exception : yes > cpuid level : 11 > wp : yes > flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht nx constant_tsc pni est ssse3 sse4_1 sse4_2 popcnt ida > bogomips : 4538.00 > clflush size : 64 > power management: > > > > 64 bit PV on HVM > ---------------- > Average Optimal load -j 4 Run (std deviation): > Elapsed Time 215.307 (10.1294) > User Time 632.503 (6.4785) > System Time 115.497 (4.53905) > Percent CPU 347.333 (15.885) > Context Switches 43319.7 (2088.39) > Sleeps 48950 (3140.18) > > 64 bit pure HVM > --------------- > Average Optimal load -j 4 Run (std deviation): > Elapsed Time 235.25 (4.10598) > User Time 512.48 (1.69714) > System Time 73.5967 (0.65041) > Percent CPU 248.667 (4.93288) > Context Switches 35930.3 (342.837) > Sleeps 56249.7 (2784.42) > > 64 bit traditional PV > --------------------- > Average Optimal load -j 4 Run (std deviation): > Elapsed Time 248.187 (12.3954) > User Time 535.283 (0.818189) > System Time 127.497 (0.342685) > Percent CPU 266.667 (12.8582) > Context Switches 32978 (2968.78) > Sleeps 54391.3 (3141.44) > > > the results show that the 64 bit PV on HVM case is the fastest but it is > also the one that uses most CPU and User time. > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-24 19:51 UTC
Re: [Xen-devel] Re: [PATCH 0 of 12] PV on HVM Xen
On Mon, 24 May 2010, Pasi Kärkkäinen wrote:> Hmm.. are you sure kernbench is a proper benchmark for disk IO? > kernel compilations fire up a lot of new processes, and that favours HVM guests. > > Basicly it seems really weird to me that in the results HVM (without PV drivers) > is faster than pure PV.. > > If you compare pure disk-IO PV vs. HVM the difference is usually big.. > PV beating HVM hands down. >32 bit PV certainly would, even on a kernbench, but keep in mind that 64 bit PV is not as fast. In any case, like Jeremy previously said, it all depends on the workload... _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
What Xen Version this patch is supposed to be applied ? Seems like not 4.0. Boris. --- On Mon, 5/24/10, Stefano Stabellini <stefano.stabellini@eu.citrix.com> wrote: From: Stefano Stabellini <stefano.stabellini@eu.citrix.com> Subject: Re: [Xen-devel] [PATCH 0 of 12] PV on HVM Xen To: "Boris Derzhavets" <bderzhavets@yahoo.com> Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>, "Stabellini" <Stefano.Stabellini@eu.citrix.com>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Stefano@yahoo.com, "Don Dutile" <ddutile@redhat.com>, "Sheng Yang" <sheng@linux.intel.com> Date: Monday, May 24, 2010, 3:06 PM On Mon, 24 May 2010, Boris Derzhavets wrote:> > In order to be able to use VIRQ_TIMER and to improve performances you > > need a patch to Xen to implement the vector callback mechanism > > for event channel delivery. > > Where to get it ? >It is this one: http://lists.xensource.com/archives/html/xen-devel/2010-05/msg00875.html _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, 25 May 2010, Boris Derzhavets wrote:> What Xen Version this patch is supposed to be applied ? > Seems like not 4.0. >xen-unstable, but it shouldn''t be difficult to port to 4.0. BTW I just sent an updated version of the patch to the list. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Could you,please, resubmit it as raw attachment. Boris. --- On Tue, 5/25/10, Stefano Stabellini <stefano.stabellini@eu.citrix.com> wrote: From: Stefano Stabellini <stefano.stabellini@eu.citrix.com> Subject: Re: [Xen-devel] [PATCH 0 of 12] PV on HVM Xen To: "Boris Derzhavets" <bderzhavets@yahoo.com> Cc: "Stefano Stabellini" <Stefano.Stabellini@eu.citrix.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "Stefano@yahoo.com" <Stefano@yahoo.com>, "Don Dutile" <ddutile@redhat.com>, "Sheng Yang" <sheng@linux.intel.com> Date: Tuesday, May 25, 2010, 5:55 AM On Tue, 25 May 2010, Boris Derzhavets wrote:> What Xen Version this patch is supposed to be applied ? > Seems like not 4.0. >xen-unstable, but it shouldn''t be difficult to port to 4.0. BTW I just sent an updated version of the patch to the list. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-25 20:20 UTC
[Xen-devel] Re: [PATCH 06/11] Add suspend\resume support for PV on HVM guests.
On 05/24/2010 11:27 AM, Stefano Stabellini wrote:> Suspend\resume requires few different thing on HVM: the suspend > hypercall is different, we don''t need to save\restore any memory related > setting, but we need to reinitialize the shared info page and the > callback mechanism. > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > --- > arch/x86/xen/enlighten.c | 14 +++++++++--- > arch/x86/xen/suspend.c | 6 +++++ > arch/x86/xen/xen-ops.h | 1 + > drivers/xen/manage.c | 45 ++++++++++++++++++++++++++++++++++++++++--- > drivers/xen/platform-pci.c | 23 +++++++++++++++++++++- > include/xen/xen-ops.h | 3 ++ > 6 files changed, 83 insertions(+), 9 deletions(-) > > diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c > index ed34b30..cb316b5 100644 > --- a/arch/x86/xen/enlighten.c > +++ b/arch/x86/xen/enlighten.c > @@ -1263,12 +1263,14 @@ static int init_hvm_pv_info(int *major, int *minor) > return 0; > } > > -static void __init init_shared_info(void) > +void init_shared_info(void) >This needs a xen_ prefix. J> { > + int cpu; > struct xen_add_to_physmap xatp; > - struct shared_info *shared_info_page; > + static struct shared_info *shared_info_page = 0; > > - shared_info_page = (struct shared_info *) alloc_bootmem_pages(PAGE_SIZE); > + if (!shared_info_page) > + shared_info_page = (struct shared_info *) alloc_bootmem_pages(PAGE_SIZE); > xatp.domid = DOMID_SELF; > xatp.idx = 0; > xatp.space = XENMAPSPACE_shared_info; > @@ -1280,7 +1282,11 @@ static void __init init_shared_info(void) > > /* Don''t do the full vcpu_info placement stuff until we have a > possible map and a non-dummy shared_info. */ > - per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0]; > + /* init_shared_info is run at resume time too, in that case multiple > + * vcpus might be online */ > + for_each_online_cpu(cpu) { > + per_cpu(xen_vcpu, cpu) = &HYPERVISOR_shared_info->vcpu_info[cpu]; > + } > } > > static int __cpuinit xen_hvm_cpu_notify(struct notifier_block *self, > diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c > index 987267f..ef517ee 100644 > --- a/arch/x86/xen/suspend.c > +++ b/arch/x86/xen/suspend.c > @@ -26,6 +26,12 @@ void xen_pre_suspend(void) > BUG(); > } > > +void xen_hvm_post_suspend(int suspend_cancelled) > +{ > + init_shared_info(); > + xen_callback_vector(); > +} > + > void xen_post_suspend(int suspend_cancelled) > { > xen_build_mfn_list_list(); > diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h > index 0d0e0e6..caf89ee 100644 > --- a/arch/x86/xen/xen-ops.h > +++ b/arch/x86/xen/xen-ops.h > @@ -39,6 +39,7 @@ void xen_enable_syscall(void); > void xen_vcpu_restore(void); > > void xen_callback_vector(void); > +void init_shared_info(void); > > void __init xen_build_dynamic_phys_to_machine(void); > > diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c > index 2ac4440..0716ba6 100644 > --- a/drivers/xen/manage.c > +++ b/drivers/xen/manage.c > @@ -9,6 +9,7 @@ > #include <linux/stop_machine.h> > #include <linux/freezer.h> > > +#include <xen/xen.h> > #include <xen/xenbus.h> > #include <xen/grant_table.h> > #include <xen/events.h> > @@ -17,6 +18,7 @@ > > #include <asm/xen/hypercall.h> > #include <asm/xen/page.h> > +#include <asm/xen/hypervisor.h> > > enum shutdown_state { > SHUTDOWN_INVALID = -1, > @@ -33,10 +35,30 @@ enum shutdown_state { > static enum shutdown_state shutting_down = SHUTDOWN_INVALID; > > #ifdef CONFIG_PM_SLEEP > -static int xen_suspend(void *data) > +static int xen_hvm_suspend(void *data) > { > + struct sched_shutdown r = { .reason = SHUTDOWN_suspend }; > int *cancelled = data; > + > + BUG_ON(!irqs_disabled()); > + > + *cancelled = HYPERVISOR_sched_op(SCHEDOP_shutdown, &r); > + > + xen_hvm_post_suspend(*cancelled); > + gnttab_resume(); > + > + if (!*cancelled) { > + xen_irq_resume(); > + xen_timer_resume(); > + } > + > + return 0; > +} > + > +static int xen_suspend(void *data) > +{ > int err; > + int *cancelled = data; > > BUG_ON(!irqs_disabled()); > > @@ -112,7 +134,10 @@ static void do_suspend(void) > goto out_resume; > } > > - err = stop_machine(xen_suspend, &cancelled, cpumask_of(0)); > + if (xen_hvm_domain()) > + err = stop_machine(xen_hvm_suspend, &cancelled, cpumask_of(0)); > + else > + err = stop_machine(xen_suspend, &cancelled, cpumask_of(0)); > > dpm_resume_noirq(PMSG_RESUME); > > @@ -261,7 +286,19 @@ static int shutdown_event(struct notifier_block *notifier, > return NOTIFY_DONE; > } > > -static int __init setup_shutdown_event(void) > +static int __init __setup_shutdown_event(void) > +{ > + /* Delay initialization in the PV on HVM case */ > + if (xen_hvm_domain()) > + return 0; > + > + if (!xen_pv_domain()) > + return -ENODEV; > + > + return xen_setup_shutdown_event(); > +} > + > +int xen_setup_shutdown_event(void) > { > static struct notifier_block xenstore_notifier = { > .notifier_call = shutdown_event > @@ -271,4 +308,4 @@ static int __init setup_shutdown_event(void) > return 0; > } > > -subsys_initcall(setup_shutdown_event); > +subsys_initcall(__setup_shutdown_event); > diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c > index 0f78318..70395c9 100644 > --- a/drivers/xen/platform-pci.c > +++ b/drivers/xen/platform-pci.c > @@ -31,6 +31,7 @@ > #include <xen/xenbus.h> > #include <xen/events.h> > #include <xen/hvm.h> > +#include <xen/xen-ops.h> > > #define DRV_NAME "xen-platform-pci" > > @@ -41,6 +42,7 @@ MODULE_LICENSE("GPL"); > static unsigned long platform_mmio; > static unsigned long platform_mmio_alloc; > static unsigned long platform_mmiolen; > +static uint64_t callback_via; > > unsigned long alloc_xen_mmio(unsigned long len) > { > @@ -85,13 +87,25 @@ static int xen_allocate_irq(struct pci_dev *pdev) > "xen-platform-pci", pdev); > } > > +static int platform_pci_resume(struct pci_dev *pdev) > +{ > + int err; > + if (xen_have_vector_callback) > + return 0; > + err = xen_set_callback_via(callback_via); > + if (err) { > + printk("platform_pci_resume failure!\n"); > + return err; > + } > + return 0; > +} > + > static int __devinit platform_pci_init(struct pci_dev *pdev, > const struct pci_device_id *ent) > { > int i, ret; > long ioaddr, iolen; > long mmio_addr, mmio_len; > - uint64_t callback_via; > > i = pci_enable_device(pdev); > if (i) > @@ -145,6 +159,10 @@ static int __devinit platform_pci_init(struct pci_dev *pdev, > ret = xenbus_probe_init(); > if (ret) > goto out; > + ret = xen_setup_shutdown_event(); > + if (ret) > + goto out; > + > > out: > if (ret) { > @@ -168,6 +186,9 @@ static struct pci_driver platform_driver = { > .name = DRV_NAME, > .probe = platform_pci_init, > .id_table = platform_pci_tbl, > +#ifdef CONFIG_PM > + .resume_early = platform_pci_resume, > +#endif > }; > > static int __init platform_pci_module_init(void) > diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h > index 883a21b..46bc81e 100644 > --- a/include/xen/xen-ops.h > +++ b/include/xen/xen-ops.h > @@ -7,6 +7,7 @@ DECLARE_PER_CPU(struct vcpu_info *, xen_vcpu); > > void xen_pre_suspend(void); > void xen_post_suspend(int suspend_cancelled); > +void xen_hvm_post_suspend(int suspend_cancelled); > > void xen_mm_pin_all(void); > void xen_mm_unpin_all(void); > @@ -14,4 +15,6 @@ void xen_mm_unpin_all(void); > void xen_timer_resume(void); > void xen_arch_resume(void); > > +int xen_setup_shutdown_event(void); > + > #endif /* INCLUDE_XEN_OPS_H */ >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-25 20:24 UTC
[Xen-devel] Re: [PATCH 10/11] Support VIRQ_TIMER and pvclock on HVM
On 05/24/2010 11:27 AM, Stefano Stabellini wrote:> Use the paravirtualized timer (VIRQ_TIMER) as timer and the > paravirtualized clock (pvclock) as wallclock on HVM domains too. >That''s not quite correct. There''s the Xen clockevent driver, the Xen clocksource, and a distinct-but-related Xen notion of wallclock time. In general, clocksources and wallclock don''t have much direct relationship to each other.> Any interaction with emulated hardware is not very fast, so using the > paravirtualized timer instead of the emulated hpet is defenitely a >"definitely"> performance improvement. >Are there accuracy/stability/precision improvements too? J> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > --- > arch/x86/xen/enlighten.c | 31 +++++++++++++++++++++++++++++++ > arch/x86/xen/suspend.c | 4 ++++ > include/xen/interface/features.h | 3 +++ > 3 files changed, 38 insertions(+), 0 deletions(-) > > diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c > index cb316b5..88e8a09 100644 > --- a/arch/x86/xen/enlighten.c > +++ b/arch/x86/xen/enlighten.c > @@ -1307,6 +1307,36 @@ static struct notifier_block __cpuinitdata xen_hvm_cpu_notifier = { > .notifier_call = xen_hvm_cpu_notify, > }; > > +static void xen_hvm_setup_cpu_clockevents(void) > +{ > + int cpu = smp_processor_id(); > + xen_setup_runstate_info(cpu); > + xen_setup_timer(cpu); > + xen_setup_cpu_clockevents(); > +} > + > +static void init_hvm_time(void) > +{ > + /* vector callback is needed otherwise we cannot receive interrupts > + * on cpu > 0 */ > + if (!xen_have_vector_callback && num_present_cpus() > 1) > + return; > + if (!xen_feature(XENFEAT_hvm_safe_pvclock)) { > + printk(KERN_INFO "Xen doesn''t support pvclock on HVM," > + "disable pv timer\n"); > + return; > + } > + > + pv_time_ops = xen_time_ops; > + x86_init.timers.timer_init = xen_time_init; > + x86_init.timers.setup_percpu_clockev = x86_init_noop; > + x86_cpuinit.setup_percpu_clockev = xen_hvm_setup_cpu_clockevents; > + > + x86_platform.calibrate_tsc = xen_tsc_khz; > + x86_platform.get_wallclock = xen_get_wallclock; > + x86_platform.set_wallclock = xen_set_wallclock; > +} > + > void __init xen_guest_init(void) > { > int r; > @@ -1326,4 +1356,5 @@ void __init xen_guest_init(void) > register_cpu_notifier(&xen_hvm_cpu_notifier); > have_vcpu_info_placement = 0; > x86_init.irqs.intr_init = xen_init_IRQ; > + init_hvm_time(); > } > diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c > index ef517ee..02c3253 100644 > --- a/arch/x86/xen/suspend.c > +++ b/arch/x86/xen/suspend.c > @@ -28,8 +28,12 @@ void xen_pre_suspend(void) > > void xen_hvm_post_suspend(int suspend_cancelled) > { > + int cpu; > init_shared_info(); > xen_callback_vector(); > + for_each_online_cpu(cpu) { > + xen_setup_runstate_info(cpu); > + } > } > > void xen_post_suspend(int suspend_cancelled) > diff --git a/include/xen/interface/features.h b/include/xen/interface/features.h > index 8ab08b9..70d2563 100644 > --- a/include/xen/interface/features.h > +++ b/include/xen/interface/features.h > @@ -44,6 +44,9 @@ > /* x86: Does this Xen host support the HVM callback vector type? */ > #define XENFEAT_hvm_callback_vector 8 > > +/* x86: pvclock algorithm is safe to use on HVM */ > +#define XENFEAT_hvm_safe_pvclock 9 > + > #define XENFEAT_NR_SUBMAPS 1 > > #endif /* __XEN_PUBLIC_FEATURES_H__ */ >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-25 20:31 UTC
[Xen-devel] Re: [PATCH 11/11] Unplug emulated disks and nics
On 05/24/2010 11:27 AM, Stefano Stabellini wrote:> Add a xen_emul_unplug command line option to the kernel to unplug > xen emulated disks and nics. > > Set the default value of xen_emul_unplug depending on whether or > not the Xen PV frontends and the Xen platform PCI driver have > been compiled for this kernel (modules or built-in are both OK). > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > --- > Documentation/kernel-parameters.txt | 11 +++ > arch/x86/xen/Makefile | 2 +- > arch/x86/xen/enlighten.c | 1 + > arch/x86/xen/platform-pci-unplug.c | 131 ++++++++++++++++++++++++++++++++++ > arch/x86/xen/xen-ops.h | 1 + > drivers/xen/platform-pci.c | 4 + > include/xen/interface/platform_pci.h | 46 ++++++++++++ > include/xen/platform_pci.h | 34 +++++++++ > 8 files changed, 229 insertions(+), 1 deletions(-) > create mode 100644 arch/x86/xen/platform-pci-unplug.c > create mode 100644 include/xen/interface/platform_pci.h > create mode 100644 include/xen/platform_pci.h > > diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt > index 839b21b..716eea8 100644 > --- a/Documentation/kernel-parameters.txt > +++ b/Documentation/kernel-parameters.txt > @@ -113,6 +113,7 @@ parameter is applicable: > More X86-64 boot options can be found in > Documentation/x86/x86_64/boot-options.txt . > X86 Either 32bit or 64bit x86 (same as X86-32+X86-64) > + XEN Xen support is enabled > > In addition, the following text indicates that the option: > > @@ -2834,6 +2835,16 @@ and is between 256 and 4096 characters. It is defined in the file > xd= [HW,XT] Original XT pre-IDE (RLL encoded) disks. > xd_geo= See header of drivers/block/xd.c. > > + xen_emul_unplug= [HW,X86,XEN] > + Unplug Xen emulated devices > + Format: [unplug0,][unplug1] > + ide-disks -- unplug primary master IDE devices > + aux-ide-disks -- unplug non-primary-master IDE devices > + nics -- unplug network devices > + all -- unplug all emulated devices (NICs and IDE disks) > + ignore -- continue loading the Xen platform PCI driver even > + if the version check failed > + > xirc2ps_cs= [NET,PCMCIA] > Format: > <irq>,<irq_mask>,<io>,<full_duplex>,<do_sound>,<lockup_hack>[,<irq2>[,<irq3>[,<irq4>]]] > diff --git a/arch/x86/xen/Makefile b/arch/x86/xen/Makefile > index 3bb4fc2..9309546 100644 > --- a/arch/x86/xen/Makefile > +++ b/arch/x86/xen/Makefile > @@ -12,7 +12,7 @@ CFLAGS_mmu.o := $(nostackp) > > obj-y := enlighten.o setup.o multicalls.o mmu.o irq.o \ > time.o xen-asm.o xen-asm_$(BITS).o \ > - grant-table.o suspend.o > + grant-table.o suspend.o platform-pci-unplug.o > > obj-$(CONFIG_SMP) += smp.o > obj-$(CONFIG_PARAVIRT_SPINLOCKS)+= spinlock.o > diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c > index 88e8a09..fbf7c43 100644 > --- a/arch/x86/xen/enlighten.c > +++ b/arch/x86/xen/enlighten.c > @@ -1354,6 +1354,7 @@ void __init xen_guest_init(void) > if (xen_feature(XENFEAT_hvm_callback_vector)) > xen_have_vector_callback = 1; > register_cpu_notifier(&xen_hvm_cpu_notifier); > + xen_unplug_emulated_devices(); > have_vcpu_info_placement = 0; > x86_init.irqs.intr_init = xen_init_IRQ; > init_hvm_time(); > diff --git a/arch/x86/xen/platform-pci-unplug.c b/arch/x86/xen/platform-pci-unplug.c > new file mode 100644 > index 0000000..4094f29 > --- /dev/null > +++ b/arch/x86/xen/platform-pci-unplug.c > @@ -0,0 +1,131 @@ > +/****************************************************************************** > + * platform-pci-unplug.c > + * > + * Xen platform PCI device driver > + * Copyright (c) 2010, Citrix > + * > + * This program is free software; you can redistribute it and/or modify it > + * under the terms and conditions of the GNU General Public License, > + * version 2, as published by the Free Software Foundation. > + * > + * This program is distributed in the hope it will be useful, but WITHOUT > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for > + * more details. > + * > + * You should have received a copy of the GNU General Public License along with > + * this program; if not, write to the Free Software Foundation, Inc., 59 Temple > + * Place - Suite 330, Boston, MA 02111-1307 USA. > + * > + */ > + > +#include <asm/io.h> > + > +#include <linux/init.h> > +#include <linux/module.h> > + > +#include <xen/platform_pci.h> > +#include <xen/interface/platform_pci.h> > + > +int xen_platform_pci; > +EXPORT_SYMBOL_GPL(xen_platform_pci); >Could you give this a bit more of a descriptive name, type and/or comment? What is it indicating about the xen_platform_pci driver? That it has been detected? That it is in use? That it can be used? Is it boolean, or can it have any value?> +static int xen_emul_unplug; > + > +static int __init check_platform_magic(void) > +{ > + short magic; > + char protocol; > + > + magic = inw(XEN_IOPORT_MAGIC); > + if (magic != XEN_IOPORT_MAGIC_VAL) { > + printk(KERN_ERR "Xen Platform PCI: unrecognised magic value\n"); > + return -1; > + } > + > + protocol = inb(XEN_IOPORT_PROTOVER); > + > + printk(KERN_DEBUG "Xen Platform PCI: I/O protocol version %d\n", > + protocol); > + > + switch (protocol) { > + case 1: > + outw(XEN_IOPORT_LINUX_PRODNUM, XEN_IOPORT_PRODNUM); > + outl(XEN_IOPORT_LINUX_DRVVER, XEN_IOPORT_DRVVER); > + if (inw(XEN_IOPORT_MAGIC) != XEN_IOPORT_MAGIC_VAL) { > + printk(KERN_ERR "Xen Platform: blacklisted by host\n"); > + return -3; > + } > + break; > + default: > + printk(KERN_WARNING "Xen Platform PCI: unknown I/O protocol version"); > + return -2; > + } > + > + return 0; > +} > + > +void __init xen_unplug_emulated_devices(void) > +{ > + int r; > + > + /* check the version of the xen platform PCI device */ > + r = check_platform_magic(); > + /* If the version matches enable the Xen platform PCI driver. > + * Also enable the Xen platform PCI driver if the version is really old > + * and the user told us to ignore it. */ > + if (!r || (r == -1 && (xen_emul_unplug & UNPLUG_IGNORE))) > + xen_platform_pci = 1; > + /* Set the default value of xen_emul_unplug depending on whether or > + * not the Xen PV frontends and the Xen platform PCI driver have > + * been compiled for this kernel (modules or built-in are both OK). */ > + if (xen_platform_pci && !xen_emul_unplug) { > +#if (defined(CONFIG_XEN_NETDEV_FRONTEND) || \ > + defined(CONFIG_XEN_NETDEV_FRONTEND_MODULE)) && \ > + (defined(CONFIG_XEN_PLATFORM_PCI) || \ > + defined(CONFIG_XEN_PLATFORM_PCI_MODULE)) > + printk(KERN_INFO "Netfront and the Xen platform PCI driver have " > + "been compiled for this kernel: unplug emulated NICs.\n"); > + xen_emul_unplug |= UNPLUG_ALL_NICS; > +#endif > +#if (defined(CONFIG_XEN_BLKDEV_FRONTEND) || \ > + defined(CONFIG_XEN_BLKDEV_FRONTEND_MODULE)) && \ > + (defined(CONFIG_XEN_PLATFORM_PCI) || \ > + defined(CONFIG_XEN_PLATFORM_PCI_MODULE)) > + printk(KERN_INFO "Blkfront and the Xen platform PCI driver have " > + "been compiled for this kernel: unplug emulated disks.\n" > + "You might have to change the root device\n" > + "from /dev/hd[a-d] to /dev/xvd[a-d]\n" > + "in your root= kernel command line option\n"); > + xen_emul_unplug |= UNPLUG_ALL_IDE_DISKS; > +#endif > + } > + /* Now unplug the emulated devices */ > + if (xen_platform_pci && !(xen_emul_unplug & UNPLUG_IGNORE)) > + outw(xen_emul_unplug, XEN_IOPORT_UNPLUG); > +} > + > +static int __init parse_xen_emul_unplug(char *arg) > +{ > + char *p, *q; > + > + for (p = arg; p; p = q) { > + q = strchr(arg, '',''); > + if (q) > + *q++ = ''\0''; > + if (!strcmp(p, "all")) > + xen_emul_unplug |= UNPLUG_ALL; > + else if (!strcmp(p, "ide-disks")) > + xen_emul_unplug |= UNPLUG_ALL_IDE_DISKS; > + else if (!strcmp(p, "aux-ide-disks")) > + xen_emul_unplug |= UNPLUG_AUX_IDE_DISKS; > + else if (!strcmp(p, "nics")) > + xen_emul_unplug |= UNPLUG_ALL_NICS; > + else if (!strcmp(p, "ignore")) > + xen_emul_unplug |= UNPLUG_IGNORE; > + else > + printk(KERN_WARNING "unrecognised option ''%s'' " > + "in module parameter ''dev_unplug''\n", p); > + } > + return 0; > +} > +early_param("xen_emul_unplug", parse_xen_emul_unplug); > diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h > index caf89ee..e146f5f 100644 > --- a/arch/x86/xen/xen-ops.h > +++ b/arch/x86/xen/xen-ops.h > @@ -40,6 +40,7 @@ void xen_vcpu_restore(void); > > void xen_callback_vector(void); > void init_shared_info(void); > +void __init xen_unplug_emulated_devices(void); > > void __init xen_build_dynamic_phys_to_machine(void); > > diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c > index 07c821c..de3ce1e 100644 > --- a/drivers/xen/platform-pci.c > +++ b/drivers/xen/platform-pci.c > @@ -27,6 +27,7 @@ > #include <linux/module.h> > #include <linux/pci.h> > > +#include <xen/platform_pci.h> > #include <xen/grant_table.h> > #include <xen/xenbus.h> > #include <xen/events.h> > @@ -198,6 +199,9 @@ static int __init platform_pci_module_init(void) > { > int rc; > > + if (!xen_platform_pci) > + return -ENODEV; > + > rc = pci_register_driver(&platform_driver); > if (rc) { > printk(KERN_INFO DRV_NAME > diff --git a/include/xen/interface/platform_pci.h b/include/xen/interface/platform_pci.h > new file mode 100644 > index 0000000..720eaf5 > --- /dev/null > +++ b/include/xen/interface/platform_pci.h > @@ -0,0 +1,46 @@ > +/****************************************************************************** > + * platform_pci.h > + * > + * Interface for granting foreign access to page frames, and receiving > + * page-ownership transfers. >That doesn''t look very accurate.> + * > + * Permission is hereby granted, free of charge, to any person obtaining a copy > + * of this software and associated documentation files (the "Software"), to > + * deal in the Software without restriction, including without limitation the > + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or > + * sell copies of the Software, and to permit persons to whom the Software is > + * furnished to do so, subject to the following conditions: > + * > + * The above copyright notice and this permission notice shall be included in > + * all copies or substantial portions of the Software. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE > + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING > + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER > + * DEALINGS IN THE SOFTWARE. >Drop all this copyright goo; these kinds of definitions can''t be copyright in the US anyway.> + */ > + > +#ifndef __XEN_PUBLIC_PLATFORM_PCI_H__ > +#define __XEN_PUBLIC_PLATFORM_PCI_H__ > + > +#define XEN_IOPORT_BASE 0x10 > + > +#define XEN_IOPORT_PLATFLAGS (XEN_IOPORT_BASE + 0) /* 1 byte access (R/W) */ > +#define XEN_IOPORT_MAGIC (XEN_IOPORT_BASE + 0) /* 2 byte access (R) */ > +#define XEN_IOPORT_UNPLUG (XEN_IOPORT_BASE + 0) /* 2 byte access (W) */ > +#define XEN_IOPORT_DRVVER (XEN_IOPORT_BASE + 0) /* 4 byte access (W) */ > + > +#define XEN_IOPORT_SYSLOG (XEN_IOPORT_BASE + 2) /* 1 byte access (W) */ > +#define XEN_IOPORT_PROTOVER (XEN_IOPORT_BASE + 2) /* 1 byte access (R) */ > +#define XEN_IOPORT_PRODNUM (XEN_IOPORT_BASE + 2) /* 2 byte access (W) */ > + > +#define UNPLUG_ALL_IDE_DISKS 1 > +#define UNPLUG_ALL_NICS 2 > +#define UNPLUG_AUX_IDE_DISKS 4 > +#define UNPLUG_ALL 7 > +#define UNPLUG_IGNORE 8 >These need XEN_ prefixes, otherwise they look too generic.> + > +#endif /* __XEN_PUBLIC_PLATFORM_PCI_H__ */ > diff --git a/include/xen/platform_pci.h b/include/xen/platform_pci.h > new file mode 100644 > index 0000000..ae04d8a > --- /dev/null > +++ b/include/xen/platform_pci.h > @@ -0,0 +1,34 @@ > +/****************************************************************************** > + * platform-pci.h > + * > + * Xen platform PCI device driver > + * Copyright (c) 2004, Intel Corporation. <xiaofeng.ling@intel.com> > + * Copyright (c) 2007, XenSource Inc. > + * Copyright (c) 2010, Citrix > + * > + * This program is free software; you can redistribute it and/or modify it > + * under the terms and conditions of the GNU General Public License, > + * version 2, as published by the Free Software Foundation. > + * > + * This program is distributed in the hope it will be useful, but WITHOUT > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for > + * more details. > + * > + * You should have received a copy of the GNU General Public License along with > + * this program; if not, write to the Free Software Foundation, Inc., 59 Temple > + * Place - Suite 330, Boston, MA 02111-1307 USA. >Ditto; there''s no copyrightable content in this file.> + */ > + > +#ifndef _XEN_PLATFORM_PCI_H > +#define _XEN_PLATFORM_PCI_H > + > +#include <linux/version.h> >Why?> + > +#define XEN_IOPORT_MAGIC_VAL 0x49d2 > +#define XEN_IOPORT_LINUX_PRODNUM 0xffff > +#define XEN_IOPORT_LINUX_DRVVER ((LINUX_VERSION_CODE << 8) + 0x0) > + > +extern int xen_platform_pci; >I still think these headers should be folded together. There''s no point in having them separate. They describe an ABI, so there''s no need to keep the files in sync with the master Xen sources (since the ABI will never change). J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-26 12:27 UTC
[Xen-devel] Re: [PATCH 11/11] Unplug emulated disks and nics
On Tue, 25 May 2010, Jeremy Fitzhardinge wrote:> Could you give this a bit more of a descriptive name, type and/or > comment? What is it indicating about the xen_platform_pci driver? That > it has been detected? That it is in use? That it can be used? Is it > boolean, or can it have any value? > >It is boolean and it signals that the xen platform pci device can be used. I''ll add a comment and rename it to xen_platform_pci_enabled.> > + */ > > + > > +#ifndef _XEN_PLATFORM_PCI_H > > +#define _XEN_PLATFORM_PCI_H > > + > > +#include <linux/version.h> > > > Why? >It is needed for LINUX_VERSION_CODE, see below:> > + > > +#define XEN_IOPORT_MAGIC_VAL 0x49d2 > > +#define XEN_IOPORT_LINUX_PRODNUM 0xffff > > +#define XEN_IOPORT_LINUX_DRVVER ((LINUX_VERSION_CODE << 8) + 0x0) > > + > > +extern int xen_platform_pci; > > > > I still think these headers should be folded together. There''s no point > in having them separate. They describe an ABI, so there''s no need to > keep the files in sync with the master Xen sources (since the ABI will > never change).All right, I also agree with the other comments. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-26 13:08 UTC
[Xen-devel] Re: [PATCH 10/11] Support VIRQ_TIMER and pvclock on HVM
On Tue, 25 May 2010, Jeremy Fitzhardinge wrote:> On 05/24/2010 11:27 AM, Stefano Stabellini wrote: > > Use the paravirtualized timer (VIRQ_TIMER) as timer and the > > paravirtualized clock (pvclock) as wallclock on HVM domains too. > > > > That''s not quite correct. There''s the Xen clockevent driver, the Xen > clocksource, and a distinct-but-related Xen notion of wallclock time. > In general, clocksources and wallclock don''t have much direct > relationship to each other.Let me rephrase it: use xen_vcpuop_clockevent instead of hpet and APIC timers as main clockevent device on all vcpus, use the xen wallclock time as wallclock instead of rtc and use xen_clocksource as clocksource. The pv clock algorithm needs to work correctly for the xen_clocksource and xen wallclock to be usable, only modern Xen versions offer a reliable pv clock in HVM guests (XENFEAT_hvm_safe_pvclock).> > > Any interaction with emulated hardware is not very fast, so using the > > paravirtualized timer instead of the emulated hpet is defenitely a > > > "definitely" > > > performance improvement. > > > > Are there accuracy/stability/precision improvements too? >Using the hpet as clocksource means a VMEXIT every time we read/write to the hpet mmio addresses, pvclock give us a better rating without VMEXITs. Same goes for the xen wallclock and xen_vcpuop_clockevent. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-May-26 18:26 UTC
Re: [Xen-devel] [PATCH 04/11] Fix find_unbound_irq in presence of ioapic irqs.
On Mon, May 24, 2010 at 07:27:30PM +0100, Stefano Stabellini wrote:> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>Can you provide a more detailed description of why this is needed?> --- > drivers/xen/events.c | 11 ++++++++++- > 1 files changed, 10 insertions(+), 1 deletions(-) > > diff --git a/drivers/xen/events.c b/drivers/xen/events.c > index 8b68fc4..1356fa2 100644 > --- a/drivers/xen/events.c > +++ b/drivers/xen/events.c > @@ -339,9 +339,18 @@ static int find_unbound_irq(void) > int irq; > struct irq_desc *desc; > > - for (irq = 0; irq < nr_irqs; irq++) > + for (irq = 0; irq < nr_irqs; irq++) { > + desc = irq_to_desc(irq); > + /* only 0->15 have init''d desc; handle irq > 16 */ > + if (desc == NULL) > + break; > + if (desc->chip == &no_irq_chip) > + break; > + if (desc->chip != &xen_dynamic_chip) > + continue; > if (irq_info[irq].type == IRQT_UNBOUND) > break; > + } > > if (irq == nr_irqs) > panic("No available IRQ to bind to: increase nr_irqs!\n"); > -- > 1.5.4.3 > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-26 20:59 UTC
[Xen-devel] Re: [PATCH 11/11] Unplug emulated disks and nics
On 05/26/2010 05:27 AM, Stefano Stabellini wrote:> It is needed for LINUX_VERSION_CODE, see below: > > >>> + >>> +#define XEN_IOPORT_MAGIC_VAL 0x49d2 >>> +#define XEN_IOPORT_LINUX_PRODNUM 0xffff >>> +#define XEN_IOPORT_LINUX_DRVVER ((LINUX_VERSION_CODE << 8) + 0x0) >>>Ah. What''s that for? Does qemu change its behaviour depending on the kernel version? Ah, something to do with blacklisting. But is the Linux kernel version the most useful value here? Shouldn''t there be some kind of separate version number? And what''s the blacklisting stuff for anyway? Wow, this interface is perverse. It reuses the same IO port but changes function depending on the size of the IO? Again, wow. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-27 12:29 UTC
[Xen-devel] Re: [PATCH 11/11] Unplug emulated disks and nics
On Wed, 26 May 2010, Jeremy Fitzhardinge wrote:> On 05/26/2010 05:27 AM, Stefano Stabellini wrote: > > It is needed for LINUX_VERSION_CODE, see below: > > > > > >>> + > >>> +#define XEN_IOPORT_MAGIC_VAL 0x49d2 > >>> +#define XEN_IOPORT_LINUX_PRODNUM 0xffff > >>> +#define XEN_IOPORT_LINUX_DRVVER ((LINUX_VERSION_CODE << 8) + 0x0) > >>> > > Ah. What''s that for? Does qemu change its behaviour depending on the > kernel version? Ah, something to do with blacklisting. But is the > Linux kernel version the most useful value here? Shouldn''t there be > some kind of separate version number? And what''s the blacklisting stuff > for anyway? >Well at the moment qemu doesn''t actually change behaviour but as part of the protocol negotiation the pv drivers are supposed to communicate their version number to qemu. Also it is possible to blacklist a particular pv driver version in qemu even though I don''t think it is actually used at the moment. Regarding what version number should we give to qemu, I guess we could have our own separate version number here, independent fron the linux version. I''ll start from 1 then.> Wow, this interface is perverse. It reuses the same IO port but changes > function depending on the size of the IO? Again, wow.Yeah, before you ask, I didn''t write it :) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-27 14:49 UTC
Re: [Xen-devel] [PATCH 04/11] Fix find_unbound_irq in presence of ioapic irqs.
On Wed, 26 May 2010, Konrad Rzeszutek Wilk wrote:> On Mon, May 24, 2010 at 07:27:30PM +0100, Stefano Stabellini wrote: > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > > Can you provide a more detailed description of why this is needed? >We need it because we don''t want to break the assumption that the first 16 irqs are ISA irqs, and among the other irqs we actually want to be sure they are available before using them.> > --- > > drivers/xen/events.c | 11 ++++++++++- > > 1 files changed, 10 insertions(+), 1 deletions(-) > > > > diff --git a/drivers/xen/events.c b/drivers/xen/events.c > > index 8b68fc4..1356fa2 100644 > > --- a/drivers/xen/events.c > > +++ b/drivers/xen/events.c > > @@ -339,9 +339,18 @@ static int find_unbound_irq(void) > > int irq; > > struct irq_desc *desc; > > > > - for (irq = 0; irq < nr_irqs; irq++) > > + for (irq = 0; irq < nr_irqs; irq++) { > > + desc = irq_to_desc(irq); > > + /* only 0->15 have init''d desc; handle irq > 16 */ > > + if (desc == NULL) > > + break; > > + if (desc->chip == &no_irq_chip) > > + break; > > + if (desc->chip != &xen_dynamic_chip) > > + continue; > > if (irq_info[irq].type == IRQT_UNBOUND) > > break; > > + } > > > > if (irq == nr_irqs) > > panic("No available IRQ to bind to: increase nr_irqs!\n"); > > -- > > 1.5.4.3 > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2010-May-27 14:49 UTC
[Xen-devel] Re: [PATCH 11/11] Unplug emulated disks and nics
Stefano Stabellini writes ("[Xen-devel] Re: [PATCH 11/11] Unplug emulated disks and nics"):> On Wed, 26 May 2010, Jeremy Fitzhardinge wrote: > > Wow, this interface is perverse. It reuses the same IO port but changes > > function depending on the size of the IO? Again, wow. > > Yeah, before you ask, I didn''t write it :)Yes, neither did I :-). However, I did document it and now I also maintain the "product number" registry. Did you find the interface spec ? Enclosed below in case not. I hereby allocate you ("pvops PV-on-HVM Linux, upstream") product number 3. Does the kernel have a way to distinguish between upstream and other versions ? Eg, there''s the kernel version name suffix thingy if I remember rightly. Perhaps we should allocate a different number for "some pvops pv-on-hvm Linux with a nonempty kernel version name suffix". Please advise. You are welcome to use whatever you like for the "build number". Perhaps the best thing would a two-byte encoding of the kernel version number if that is possible. As the purpose is logging and blacklisting, it''s not that critical although it''s better to reuse the same number for excessively similar builds than to use a random scheme which might generate accidental clashes between unrelated versions. Ian. MAGIC IOPORT 0x10 PROTOCOL The protocol covers three basic things: -- Disconnecting emulated devices. -- Getting log messages out of the drivers and into dom0. -- Allowing dom0 to block the loading of specific drivers. This is intended as a backwards-compatibility thing: if we discover a bug in some old version of the drivers, then rather than working around it in Xen, we have the option of just making those drivers fall back to emulated mode. The current protocol works like this (from the point of view of drivers): 1) When the drivers first come up, they check whether the unplug logic is available by reading a two-byte magic number from IO port 0x10. These should be 0x49d2. If the magic number doesn''t match, the drivers don''t do anything. 2) The drivers read a one-byte protocol version from IO port 0x12. If this is 0, skip to 6. 3) The drivers write a two-byte product number to IO port 0x12. At the moment, the only drivers using this protocol are our closed-source ones, which use product number 1. 4) The drivers write a four-byte build number to IO port 0x10. 5) The drivers check the magic number by reading two bytes from 0x10 again. If it''s changed from 0x49d2 to 0xd249, the drivers are blacklisted and should not load. 6) The drivers write a two-byte bitmask of devices to unplug to IO port 0x10. The defined fields are: 1 -- All IDE disks (not including CD drives) 2 -- All emulated NICs 4 -- All IDE disks except for the primary master (not including CD drives) The relevant emulated devices then disappear from the relevant buses. For most guest operating systems, you want to do this before device enumeration happens. ...) Once the drivers have checked the magic number, they can send log messages to qemu which will be logged to wherever qemu''s logs go (/var/log/xen/qemu-dm.log on normal Xen, dom0 syslog on XenServer). These messages are written to IO port 0x12 a byte at a time, and are terminated by newlines. There''s a fairly aggressive rate limiter on these messages, so they shouldn''t be used for anything even vaguely high-volume, but they''re rather useful for debugging and support. It is still permitted for a driver to use this logging feature if it is blacklisted, but ONLY if it has checked the magic number and found it to be 0x49d2 or 0xd249. This isn''t exactly a pretty protocol, but it does solve the problem. The blacklist is, from qemu''s point of view, handled mostly through xenstore. A driver version is considered to be blacklisted if /mh/driver-blacklist/{product_name}/{build_number} exists and is readable, where {build_number} is the build number from step 4 as a decimal number. {product_name} is a string corresponding to the product number in step 3. The master registry of product names and numbers is in qemu-xen-unstable''s xenstore.c. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-May-27 15:36 UTC
Re: [Xen-devel] [PATCH 04/11] Fix find_unbound_irq in presence of ioapic irqs.
On Thu, May 27, 2010 at 03:49:12PM +0100, Stefano Stabellini wrote:> On Wed, 26 May 2010, Konrad Rzeszutek Wilk wrote: > > On Mon, May 24, 2010 at 07:27:30PM +0100, Stefano Stabellini wrote: > > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > > > > Can you provide a more detailed description of why this is needed? > > > > We need it because we don''t want to break the assumption that the first > 16 irqs are ISA irqs, and among the other irqs we actually want to be > sure they are available before using them.Interesting. So with this commit: 482839e7 and bfbc226d where we try our best to allocate IRQs for Xen starting from high and then going down (so that when we are running in PV context with Xen PCI passthrough we can allocate IRQs for physical devices above IRQ 16 and not collide with Xen event channels), this would still work. But we actually wouldn''t test the availability of the first sixteen IRQ for a long time (or never).> > > > > --- > > > drivers/xen/events.c | 11 ++++++++++- > > > 1 files changed, 10 insertions(+), 1 deletions(-) > > > > > > diff --git a/drivers/xen/events.c b/drivers/xen/events.c > > > index 8b68fc4..1356fa2 100644 > > > --- a/drivers/xen/events.c > > > +++ b/drivers/xen/events.c > > > @@ -339,9 +339,18 @@ static int find_unbound_irq(void) > > > int irq; > > > struct irq_desc *desc; > > > > > > - for (irq = 0; irq < nr_irqs; irq++) > > > + for (irq = 0; irq < nr_irqs; irq++) { > > > + desc = irq_to_desc(irq); > > > + /* only 0->15 have init''d desc; handle irq > 16 */ > > > + if (desc == NULL) > > > + break; > > > + if (desc->chip == &no_irq_chip) > > > + break; > > > + if (desc->chip != &xen_dynamic_chip) > > > + continue; > > > if (irq_info[irq].type == IRQT_UNBOUND) > > > break; > > > + } > > > > > > if (irq == nr_irqs) > > > panic("No available IRQ to bind to: increase nr_irqs!\n"); > > > -- > > > 1.5.4.3 > > > > > > > > > _______________________________________________ > > > Xen-devel mailing list > > > Xen-devel@lists.xensource.com > > > http://lists.xensource.com/xen-devel > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-27 17:25 UTC
Re: [Xen-devel] Re: [PATCH 11/11] Unplug emulated disks and nics
On Thu, 27 May 2010, Ian Jackson wrote:> Stefano Stabellini writes ("[Xen-devel] Re: [PATCH 11/11] Unplug emulated disks and nics"): > > On Wed, 26 May 2010, Jeremy Fitzhardinge wrote: > > > Wow, this interface is perverse. It reuses the same IO port but changes > > > function depending on the size of the IO? Again, wow. > > > > Yeah, before you ask, I didn''t write it :) > > Yes, neither did I :-). However, I did document it and now I also > maintain the "product number" registry. Did you find the interface > spec ? Enclosed below in case not. > > I hereby allocate you ("pvops PV-on-HVM Linux, upstream") product > number 3. Does the kernel have a way to distinguish between upstream > and other versions ? Eg, there''s the kernel version name suffix > thingy if I remember rightly. Perhaps we should allocate a different > number for "some pvops pv-on-hvm Linux with a nonempty kernel version > name suffix". Please advise. > > You are welcome to use whatever you like for the "build number". > Perhaps the best thing would a two-byte encoding of the kernel version > number if that is possible. As the purpose is logging and > blacklisting, it''s not that critical although it''s better to reuse the > same number for excessively similar builds than to use a random scheme > which might generate accidental clashes between unrelated versions. >It seems reasonable to me. Jeremy, what do you think? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-27 17:51 UTC
Re: [Xen-devel] Re: [PATCH 11/11] Unplug emulated disks and nics
On 05/27/2010 07:49 AM, Ian Jackson wrote:> Stefano Stabellini writes ("[Xen-devel] Re: [PATCH 11/11] Unplug emulated disks and nics"): > >> On Wed, 26 May 2010, Jeremy Fitzhardinge wrote: >> >>> Wow, this interface is perverse. It reuses the same IO port but changes >>> function depending on the size of the IO? Again, wow. >>> >> >> Yeah, before you ask, I didn''t write it :) >> > Yes, neither did I :-). However, I did document it and now I also > maintain the "product number" registry. Did you find the interface > spec ? Enclosed below in case not. >Thanks. We should probably start a Documentation/xen/ and put this in there as part of the patch.> I hereby allocate you ("pvops PV-on-HVM Linux, upstream") product > number 3. Does the kernel have a way to distinguish between upstream > and other versions ? Eg, there''s the kernel version name suffix > thingy if I remember rightly. Perhaps we should allocate a different > number for "some pvops pv-on-hvm Linux with a nonempty kernel version > name suffix". Please advise. > > You are welcome to use whatever you like for the "build number". > Perhaps the best thing would a two-byte encoding of the kernel version > number if that is possible. As the purpose is logging and > blacklisting, it''s not that critical although it''s better to reuse the > same number for excessively similar builds than to use a random scheme > which might generate accidental clashes between unrelated versions. >We could include 2 bytes of the HEAD changeset or something, with some risk of collision. Or just choose a constant and stick with it until some interesting qualitative driver change makes it worthwhile bumping the version. I guess I can see some value in this info for recording in a log to do some diagnostics, but the whole blacklist concept seems highly dubious to me. Some kind of feature negotiation makes a lot more sense to me... J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-27 18:10 UTC
Re: [Xen-devel] Re: [PATCH 11/11] Unplug emulated disks and nics
On Thu, 27 May 2010, Jeremy Fitzhardinge wrote:> We could include 2 bytes of the HEAD changeset or something, with some > risk of collision. Or just choose a constant and stick with it until > some interesting qualitative driver change makes it worthwhile bumping > the version. >the constant would be more useful than a CS I think _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Zhigang Wang
2010-May-28 08:55 UTC
Re: [Xen-devel] [PATCH 05/11] Xen PCI platform device driver
On 05/25/2010 02:27 AM, Stefano Stabellini wrote:> Add the xen pci platform device driver that is responsible > for initializing the grant table and xenbus in PV on HVM mode. > Few changes to xenbus and grant table are necessary to allow the delayed > initialization in HVM mode. > Grant table needs few additional modifications to work in HVM mode. > > The Xen PCI platform device raises an irq every time an event has been > delivered to us. However these interrupts are only delivered to vcpu 0. > The Xen PCI platform interrupt handler calls xen_hvm_evtchn_do_upcall > that is a little wrapper around __xen_evtchn_do_upcall, the traditional > Xen upcall handler, the very same used with traditional PV guests. > > When running on HVM the event channel upcall is never called while in > progress because it is a normal Linux irq handler, therefore we cannot > be sure that evtchn_upcall_pending is 0 when returning. > For this reason if evtchn_upcall_pending is set by Xen we need to loop > again on the event channels set pending otherwise we might loose some > event channel deliveries. > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > Signed-off-by: Sheng Yang <sheng@linux.intel.com> > --- > drivers/xen/Kconfig | 8 ++ > drivers/xen/Makefile | 3 +- > drivers/xen/events.c | 7 +- > drivers/xen/grant-table.c | 69 ++++++++++++-- > drivers/xen/platform-pci.c | 186 +++++++++++++++++++++++++++++++++++ > drivers/xen/xenbus/xenbus_probe.c | 20 +++- > include/linux/pci_ids.h | 3 + > include/xen/grant_table.h | 1 + > include/xen/interface/grant_table.h | 1 + > include/xen/xenbus.h | 1 + > 10 files changed, 283 insertions(+), 16 deletions(-) > create mode 100644 drivers/xen/platform-pci.c > > diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig > index fad3df2..da312e2 100644 > --- a/drivers/xen/Kconfig > +++ b/drivers/xen/Kconfig > @@ -62,4 +62,12 @@ config XEN_SYS_HYPERVISOR > virtual environment, /sys/hypervisor will still be present, > but will have no xen contents. > > +config XEN_PLATFORM_PCI > + tristate "xen platform pci device driver" > + depends on XENdefault m ?> + help > + Driver for the Xen PCI Platform device: it is responsible for > + initializing xenbus and grant_table when running in a Xen HVM > + domain. As a consequence this driver is required to run any Xen PV > + frontend on Xen HVM. > endmenu > diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile > index 7c28434..e392fb7 100644 > --- a/drivers/xen/Makefile > +++ b/drivers/xen/Makefile > @@ -9,4 +9,5 @@ obj-$(CONFIG_XEN_XENCOMM) += xencomm.o > obj-$(CONFIG_XEN_BALLOON) += balloon.o > obj-$(CONFIG_XEN_DEV_EVTCHN) += evtchn.o > obj-$(CONFIG_XENFS) += xenfs/ > -obj-$(CONFIG_XEN_SYS_HYPERVISOR) += sys-hypervisor.o > \ No newline at end of file > +obj-$(CONFIG_XEN_SYS_HYPERVISOR) += sys-hypervisor.o > +obj-$(CONFIG_XEN_PLATFORM_PCI) += platform-pci.o > diff --git a/drivers/xen/events.c b/drivers/xen/events.c > index 1356fa2..c6f561d 100644 > --- a/drivers/xen/events.c > +++ b/drivers/xen/events.c > @@ -673,7 +673,7 @@ static void __xen_evtchn_do_upcall(void) > > count = __get_cpu_var(xed_nesting_count); > __get_cpu_var(xed_nesting_count) = 0; > - } while(count != 1); > + } while(count != 1 || vcpu_info->evtchn_upcall_pending); > > out: > > @@ -733,7 +733,10 @@ static int rebind_irq_to_cpu(unsigned irq, unsigned tcpu) > struct evtchn_bind_vcpu bind_vcpu; > int evtchn = evtchn_from_irq(irq); > > - if (!VALID_EVTCHN(evtchn)) > + /* events delivered via platform PCI interrupts are always > + * routed to vcpu 0 */ > + if (!VALID_EVTCHN(evtchn) || > + (xen_hvm_domain() && !xen_have_vector_callback)) > return -1; > > /* Send future instances of this interrupt to other vcpu. */ > diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c > index f66db3b..4c959a5 100644 > --- a/drivers/xen/grant-table.c > +++ b/drivers/xen/grant-table.c > @@ -37,11 +37,13 @@ > #include <linux/slab.h> > #include <linux/vmalloc.h> > #include <linux/uaccess.h> > +#include <linux/io.h> > > #include <xen/xen.h> > #include <xen/interface/xen.h> > #include <xen/page.h> > #include <xen/grant_table.h> > +#include <xen/interface/memory.h> > #include <asm/xen/hypercall.h> > > #include <asm/pgtable.h> > @@ -59,6 +61,7 @@ static unsigned int boot_max_nr_grant_frames; > static int gnttab_free_count; > static grant_ref_t gnttab_free_head; > static DEFINE_SPINLOCK(gnttab_list_lock); > +static unsigned long hvm_pv_resume_frames; > > static struct grant_entry *shared; > > @@ -449,6 +452,30 @@ static int gnttab_map(unsigned int start_idx, unsigned int end_idx) > unsigned int nr_gframes = end_idx + 1; > int rc; > > + if (xen_hvm_domain()) { > + struct xen_add_to_physmap xatp; > + unsigned int i = end_idx; > + rc = 0; > + /* > + * Loop backwards, so that the first hypercall has the largest > + * index, ensuring that the table will grow only once. > + */ > + do { > + xatp.domid = DOMID_SELF; > + xatp.idx = i; > + xatp.space = XENMAPSPACE_grant_table; > + xatp.gpfn = (hvm_pv_resume_frames >> PAGE_SHIFT) + i; > + rc = HYPERVISOR_memory_op(XENMEM_add_to_physmap, &xatp); > + if (rc != 0) { > + printk(KERN_WARNING > + "grant table add_to_physmap failed, err=%d\n", rc); > + break; > + } > + } while (i-- > start_idx); > + > + return rc; > + } > + > frames = kmalloc(nr_gframes * sizeof(unsigned long), GFP_ATOMIC); > if (!frames) > return -ENOMEM; > @@ -476,9 +503,28 @@ static int gnttab_map(unsigned int start_idx, unsigned int end_idx) > > int gnttab_resume(void) > { > - if (max_nr_grant_frames() < nr_grant_frames) > + unsigned int max_nr_gframes; > + > + max_nr_gframes = max_nr_grant_frames(); > + if (max_nr_gframes < nr_grant_frames) > return -ENOSYS; > - return gnttab_map(0, nr_grant_frames - 1); > + > + if (xen_pv_domain()) > + return gnttab_map(0, nr_grant_frames - 1); > + > + if (!hvm_pv_resume_frames) { > + hvm_pv_resume_frames = alloc_xen_mmio(PAGE_SIZE * max_nr_gframes); > + shared = ioremap(hvm_pv_resume_frames, PAGE_SIZE * max_nr_gframes); > + if (shared == NULL) { > + printk(KERN_WARNING > + "Fail to ioremap gnttab share frames\n"); > + return -ENOMEM; > + } > + } > + > + gnttab_map(0, nr_grant_frames - 1); > + > + return 0; > } > > int gnttab_suspend(void) > @@ -505,15 +551,12 @@ static int gnttab_expand(unsigned int req_entries) > return rc; > } > > -static int __devinit gnttab_init(void) > +int gnttab_init(void) > { > int i; > unsigned int max_nr_glist_frames, nr_glist_frames; > unsigned int nr_init_grefs; > > - if (!xen_domain()) > - return -ENODEV; > - > nr_grant_frames = 1; > boot_max_nr_grant_frames = __max_nr_grant_frames(); > > @@ -557,4 +600,16 @@ static int __devinit gnttab_init(void) > return -ENOMEM; > } > > -core_initcall(gnttab_init); > +static int __devinit __gnttab_init(void) > +{ > + /* Delay grant-table initialization in the PV on HVM case */ > + if (xen_hvm_domain()) > + return 0; > + > + if (!xen_pv_domain()) > + return -ENODEV; > + > + return gnttab_init(); > +} > + > +core_initcall(__gnttab_init); > diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c > new file mode 100644 > index 0000000..0f78318 > --- /dev/null > +++ b/drivers/xen/platform-pci.c > @@ -0,0 +1,186 @@ > +/****************************************************************************** > + * platform-pci.c > + * > + * Xen platform PCI device driver > + * Copyright (c) 2005, Intel Corporation. > + * Copyright (c) 2007, XenSource Inc. > + * Copyright (c) 2010, Citrix > + * > + * This program is free software; you can redistribute it and/or modify it > + * under the terms and conditions of the GNU General Public License, > + * version 2, as published by the Free Software Foundation. > + * > + * This program is distributed in the hope it will be useful, but WITHOUT > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for > + * more details. > + * > + * You should have received a copy of the GNU General Public License along with > + * this program; if not, write to the Free Software Foundation, Inc., 59 Temple > + * Place - Suite 330, Boston, MA 02111-1307 USA. > + * > + */ > + > +#include <asm/io.h> > + > +#include <linux/interrupt.h> > +#include <linux/module.h> > +#include <linux/pci.h> > + > +#include <xen/grant_table.h> > +#include <xen/xenbus.h> > +#include <xen/events.h> > +#include <xen/hvm.h> > + > +#define DRV_NAME "xen-platform-pci" > + > +MODULE_AUTHOR("ssmith@xensource.com and stefano.stabellini@eu.citrix.com"); > +MODULE_DESCRIPTION("Xen platform PCI device"); > +MODULE_LICENSE("GPL"); > + > +static unsigned long platform_mmio; > +static unsigned long platform_mmio_alloc; > +static unsigned long platform_mmiolen; > + > +unsigned long alloc_xen_mmio(unsigned long len) > +{ > + unsigned long addr; > + > + addr = platform_mmio + platform_mmio_alloc; > + platform_mmio_alloc += len; > + BUG_ON(platform_mmio_alloc > platform_mmiolen); > + > + return addr; > +} > + > +static uint64_t get_callback_via(struct pci_dev *pdev) > +{ > + u8 pin; > + int irq; > + > + irq = pdev->irq; > + if (irq < 16) > + return irq; /* ISA IRQ */ > + > + pin = pdev->pin; > + > + /* We don''t know the GSI. Specify the PCI INTx line instead. */ > + return ((uint64_t)0x01 << 56) | /* PCI INTx identifier */ > + ((uint64_t)pci_domain_nr(pdev->bus) << 32) | > + ((uint64_t)pdev->bus->number << 16) | > + ((uint64_t)(pdev->devfn & 0xff) << 8) | > + ((uint64_t)(pin - 1) & 3); > +} > + > +static irqreturn_t do_hvm_evtchn_intr(int irq, void *dev_id) > +{ > + xen_hvm_evtchn_do_upcall(get_irq_regs()); > + return IRQ_HANDLED; > +} > + > +static int xen_allocate_irq(struct pci_dev *pdev) > +{ > + return request_irq(pdev->irq, do_hvm_evtchn_intr, > + IRQF_DISABLED | IRQF_NOBALANCING | IRQF_TRIGGER_RISING, > + "xen-platform-pci", pdev); > +} > + > +static int __devinit platform_pci_init(struct pci_dev *pdev, > + const struct pci_device_id *ent) > +{ > + int i, ret; > + long ioaddr, iolen; > + long mmio_addr, mmio_len; > + uint64_t callback_via; > + > + i = pci_enable_device(pdev); > + if (i) > + return i; > + > + ioaddr = pci_resource_start(pdev, 0); > + iolen = pci_resource_len(pdev, 0); > + > + mmio_addr = pci_resource_start(pdev, 1); > + mmio_len = pci_resource_len(pdev, 1); > + > + if (mmio_addr == 0 || ioaddr == 0) { > + dev_err(&pdev->dev, "no resources found\n"); > + ret = -ENOENT; > + } > + > + if (request_mem_region(mmio_addr, mmio_len, DRV_NAME) == NULL) { > + dev_err(&pdev->dev, "MEM I/O resource 0x%lx @ 0x%lx busy\n", > + mmio_addr, mmio_len); > + ret = -EBUSY; > + } > + > + if (request_region(ioaddr, iolen, DRV_NAME) == NULL) { > + dev_err(&pdev->dev, "I/O resource 0x%lx @ 0x%lx busy\n", > + iolen, ioaddr); > + ret = -EBUSY; > + goto out; > + } > + > + platform_mmio = mmio_addr; > + platform_mmiolen = mmio_len; > + > + if (!xen_have_vector_callback) { > + ret = xen_allocate_irq(pdev); > + if (ret) { > + printk(KERN_WARNING "request_irq failed err=%d\n", ret); > + goto out; > + } > + callback_via = get_callback_via(pdev); > + ret = xen_set_callback_via(callback_via); > + if (ret) { > + printk(KERN_WARNING > + "Unable to set the evtchn callback err=%d\n", ret); > + goto out; > + } > + } > + > + ret = gnttab_init(); > + if (ret) > + goto out; > + ret = xenbus_probe_init(); > + if (ret) > + goto out; > + > +out: > + if (ret) { > + release_mem_region(mmio_addr, mmio_len); > + release_region(ioaddr, iolen); > + pci_disable_device(pdev); > + } > + > + return ret; > +} > + > +static struct pci_device_id platform_pci_tbl[] __devinitdata = { > + {PCI_VENDOR_ID_XEN, PCI_DEVICE_ID_XEN_PLATFORM, > + PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0}, > + {0,} > +}; > + > +MODULE_DEVICE_TABLE(pci, platform_pci_tbl); > + > +static struct pci_driver platform_driver = { > + .name = DRV_NAME, > + .probe = platform_pci_init, > + .id_table = platform_pci_tbl, > +}; > + > +static int __init platform_pci_module_init(void) > +{ > + int rc; > + > + rc = pci_register_driver(&platform_driver); > + if (rc) { > + printk(KERN_INFO DRV_NAME > + ": No platform pci device model found\n"); > + return rc; > + } > + return 0; > +} > + > +module_init(platform_pci_module_init); > diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c > index 0b05b62..dc6ed06 100644 > --- a/drivers/xen/xenbus/xenbus_probe.c > +++ b/drivers/xen/xenbus/xenbus_probe.c > @@ -782,16 +782,24 @@ void xenbus_probe(struct work_struct *unused) > blocking_notifier_call_chain(&xenstore_chain, 0, NULL); > } > > -static int __init xenbus_probe_init(void) > +static int __init __xenbus_probe_init(void) > +{ > + /* Delay initialization in the PV on HVM case */ > + if (xen_hvm_domain()) > + return 0; > + > + if (!xen_pv_domain()) > + return -ENODEV; > + > + return xenbus_probe_init(); > +} > + > +int xenbus_probe_init(void) > { > int err = 0; > > DPRINTK(""); > > - err = -ENODEV; > - if (!xen_domain()) > - goto out_error; > - > /* Register ourselves with the kernel bus subsystem */ > err = bus_register(&xenbus_frontend.bus); > if (err) > @@ -857,7 +865,7 @@ static int __init xenbus_probe_init(void) > return err; > } > > -postcore_initcall(xenbus_probe_init); > +postcore_initcall(__xenbus_probe_init); > > MODULE_LICENSE("GPL"); > > diff --git a/include/linux/pci_ids.h b/include/linux/pci_ids.h > index 9f688d2..64b528d 100644 > --- a/include/linux/pci_ids.h > +++ b/include/linux/pci_ids.h > @@ -2718,3 +2718,6 @@ > #define PCI_DEVICE_ID_RME_DIGI32 0x9896 > #define PCI_DEVICE_ID_RME_DIGI32_PRO 0x9897 > #define PCI_DEVICE_ID_RME_DIGI32_8 0x9898 > + > +#define PCI_VENDOR_ID_XEN 0x5853 > +#define PCI_DEVICE_ID_XEN_PLATFORM 0x0001 > diff --git a/include/xen/grant_table.h b/include/xen/grant_table.h > index a40f1cd..811cda5 100644 > --- a/include/xen/grant_table.h > +++ b/include/xen/grant_table.h > @@ -51,6 +51,7 @@ struct gnttab_free_callback { > u16 count; > }; > > +int gnttab_init(void); > int gnttab_suspend(void); > int gnttab_resume(void); > > diff --git a/include/xen/interface/grant_table.h b/include/xen/interface/grant_table.h > index 39da93c..39e5717 100644 > --- a/include/xen/interface/grant_table.h > +++ b/include/xen/interface/grant_table.h > @@ -28,6 +28,7 @@ > #ifndef __XEN_PUBLIC_GRANT_TABLE_H__ > #define __XEN_PUBLIC_GRANT_TABLE_H__ > > +#include <xen/interface/xen.h> > > /*********************************** > * GRANT TABLE REPRESENTATION > diff --git a/include/xen/xenbus.h b/include/xen/xenbus.h > index 43e2d7d..ffa97de 100644 > --- a/include/xen/xenbus.h > +++ b/include/xen/xenbus.h > @@ -174,6 +174,7 @@ void unregister_xenbus_watch(struct xenbus_watch *watch); > void xs_suspend(void); > void xs_resume(void); > void xs_suspend_cancel(void); > +int xenbus_probe_init(void); > > /* Used by xenbus_dev to borrow kernel''s store connection. */ > void *xenbus_dev_request_and_reply(struct xsd_sockmsg *msg);_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
What is an advantage of PV on HVM ? Kernel 2.6.34 with Stefano''s patches may be built i believe only on Linux HVM DomU. At the same time any recent Linux ( >=24 or >=26) supports PV guest install ( it''s in mainline for a while). What i am missing here ? Boris. --- On Mon, 5/24/10, Stefano Stabellini <Stefano.Stabellini@eu.citrix.com> wrote: From: Stefano Stabellini <Stefano.Stabellini@eu.citrix.com> Subject: [Xen-devel] [PATCH 0 of 12] PV on HVM Xen To: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org> Cc: "Stefano Stabellini" <Stefano.Stabellini@eu.citrix.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>, "Don Dutile" <ddutile@redhat.com>, "Sheng Yang" <sheng@linux.intel.com> Date: Monday, May 24, 2010, 2:25 PM Hi all, this is another update of the PV on HVM Xen series that addresses Jeremy''s comments. The platform_pci hooks have been removed, suspend/resume for HVM domains is now much more similar to the PV case and shares the same do_suspend function. Alloc_xen_mmio_hook has been removed has well, now the memory allocation for the grant table is done by the xen platform pci driver directly. The per_cpu xen_vcpu variable is set by a cpu_notifier function so that secondary vcpus have the variable set correctly no matter what the xen features are on the host. The kernel command line option xen_unplug has been renamed to xen_emul_unplug and the code that makes use of it has been moved to a separate file (arch/x86/xen/platform-pci-unplug.c). Xen_unplug_emulated_devices is now able to detect if blkfront, netfront and the Xen platform PCI driver have been compiled, and set the default value of xen_emul_unplug accordingly. The patch "Initialize xenbus device structs with ENODEV as default" has been removed from the series and it will be sent separately. Finally the comments on most of the patches have been improved. The series is based on 2.6.34 and supports Xen PV frontends running in a HVM domain, including netfront, blkfront and the VIRQ_TIMER. In order to be able to use VIRQ_TIMER and to improve performances you need a patch to Xen to implement the vector callback mechanism for event channel delivery. A git tree is also available here: git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git branch name 2.6.34-pvhvm-v2. Cheers, Stefano _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, May 28, 2010 at 03:25:34AM -0700, Boris Derzhavets wrote:> What is an advantage of PV on HVM ? >Pure HVM guests using the Qemu emulated disk/network devices are slow. PV-on-HVM drivers make disk- and network IO fast for HVM guests.> Kernel 2.6.34 with Stefano''s patches may be built i believe only on Linux > HVM DomU. >Exactly. They''re meant for an upstream kernel, running as Xen HVM guest.> At the same time any recent Linux ( >=24 or >=26) supports PV guest > install ( it''s in > mainline for a while). > What i am missing here ? >HVM guests might be faster for some workloads compared to PV guests. Kernel compilation could be one example..ie. workloads spawning a lot of new processes all the time. -- Pasi> Boris. > > --- On Mon, 5/24/10, Stefano Stabellini <Stefano.Stabellini@eu.citrix.com> > wrote: > > From: Stefano Stabellini <Stefano.Stabellini@eu.citrix.com> > Subject: [Xen-devel] [PATCH 0 of 12] PV on HVM Xen > To: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org> > Cc: "Stefano Stabellini" <Stefano.Stabellini@eu.citrix.com>, "Jeremy > Fitzhardinge" <jeremy@goop.org>, "xen-devel@lists.xensource.com" > <xen-devel@lists.xensource.com>, "Don Dutile" <ddutile@redhat.com>, > "Sheng Yang" <sheng@linux.intel.com> > Date: Monday, May 24, 2010, 2:25 PM > > Hi all, > this is another update of the PV on HVM Xen series that addresses > Jeremy''s comments. > The platform_pci hooks have been removed, suspend/resume for HVM > domains is now much more similar to the PV case and shares the same > do_suspend function. > Alloc_xen_mmio_hook has been removed has well, now the memory allocation > for > the grant table is done by the xen platform pci driver directly. > The per_cpu xen_vcpu variable is set by a cpu_notifier function so that > secondary vcpus have the variable set correctly no matter what the xen > features are on the host. > The kernel command line option xen_unplug has been renamed to > xen_emul_unplug and the code that makes use of it has been moved to a > separate file (arch/x86/xen/platform-pci-unplug.c). > Xen_unplug_emulated_devices is now able to detect if blkfront, netfront > and the Xen platform PCI driver have been compiled, and set the default > value of xen_emul_unplug accordingly. > The patch "Initialize xenbus device structs with ENODEV as > default" has been removed from the series and it will be sent > separately. > Finally the comments on most of the patches have been improved. > > The series is based on 2.6.34 and supports Xen PV frontends running > in a HVM domain, including netfront, blkfront and the VIRQ_TIMER. > > In order to be able to use VIRQ_TIMER and to improve performances you > need a patch to Xen to implement the vector callback mechanism > for event channel delivery. > > A git tree is also available here: > > git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git > > branch name 2.6.34-pvhvm-v2. > > Cheers, > > Stefano > > _______________________________________________ > Xen-devel mailing list > [1]Xen-devel@lists.xensource.com > [2]http://lists.xensource.com/xen-devel > > References > > Visible links > 1. file:///mc/compose?to=Xen-devel@lists.xensource.com > 2. http://lists.xensource.com/xen-devel> _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, 28 May 2010, Boris Derzhavets wrote:> What is an advantage of PV on HVM ? > Kernel 2.6.34 with Stefano''s patches may be built i believe only on Linux HVM DomU. > At the same time any recent Linux ( >=24 or >=26) supports PV guest install ( it''s in > mainline for a while). > What i am missing here ? >A PV on HVM kernel doesn''t need to be a special kernel: your pvops kernel that can be used as native kernel or PV kernel can also be used as PV on HVM kernel just adding CONFIG_XEN_PLATFORM_PCI. PV on HVM support is good mainly for performances of 64 bit guests and simplicity of installation. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2010-May-28 14:18 UTC
Re: [Xen-devel] [PATCH 05/11] Xen PCI platform device driver
On Fri, 28 May 2010, Zhigang Wang wrote:> > +config XEN_PLATFORM_PCI > > + tristate "xen platform pci device driver" > > + depends on XEN > default m > > ?given the size of the platform pci driver and the fact that blkfront and netfront are both ''default y'' I am going to go with ''default y'' here too _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel