Jan Kiszka
2018-Jan-22 06:12 UTC
[PATCH 0/6] jailhouse: Enhance secondary Jailhouse guest support /wrt PCI
Basic x86 support [1] for running Linux as secondary Jailhouse [2] guest is currently pending in the tip tree. This builds on top and enhances the PCI support for x86 and also ARM guests (ARM[64] does not require platform patches and works already). Key elements of this series are: - detection of Jailhouse via device tree hypervisor node - function-level PCI scan if Jailhouse is detected - MMCONFIG support for x86 guests As most changes affect x86, I would suggest to route the series also via tip after the necessary acks are collected. Jan [1] https://lkml.org/lkml/2017/11/27/125 [2] http://jailhouse-project.org CC: Benedikt Spranger <b.spranger at linutronix.de> CC: Mark Rutland <mark.rutland at arm.com> CC: Otavio Pontes <otavio.pontes at intel.com> CC: Rob Herring <robh+dt at kernel.org> Jan Kiszka (5): jailhouse: Provide detection for non-x86 systems pci: Scan all functions when probing while running over Jailhouse x86: Consolidate PCI_MMCONFIG configs x86/jailhouse: Allow to use PCI_MMCONFIG without ACPI MAINTAINERS: Add entry for Jailhouse Otavio Pontes (1): x86/jailhouse: Enable PCI mmconfig access in inmates Documentation/devicetree/bindings/jailhouse.txt | 8 ++++++++ MAINTAINERS | 7 +++++++ arch/x86/Kconfig | 11 ++++++----- arch/x86/include/asm/jailhouse_para.h | 2 +- arch/x86/include/asm/pci_x86.h | 2 ++ arch/x86/kernel/Makefile | 2 +- arch/x86/kernel/cpu/amd.c | 2 +- arch/x86/kernel/jailhouse.c | 7 +++++++ arch/x86/pci/legacy.c | 4 +++- arch/x86/pci/mmconfig-shared.c | 4 ++-- drivers/pci/probe.c | 4 +++- include/linux/hypervisor.h | 17 +++++++++++++++-- 12 files changed, 56 insertions(+), 14 deletions(-) create mode 100644 Documentation/devicetree/bindings/jailhouse.txt -- 2.13.6
Jan Kiszka
2018-Jan-22 06:12 UTC
[PATCH 1/6] jailhouse: Provide detection for non-x86 systems
From: Jan Kiszka <jan.kiszka at siemens.com> Implement jailhouse_paravirt() via device tree probing on architectures != x86. Will be used by the PCI core. CC: Rob Herring <robh+dt at kernel.org> CC: Mark Rutland <mark.rutland at arm.com> Signed-off-by: Jan Kiszka <jan.kiszka at siemens.com> --- Documentation/devicetree/bindings/jailhouse.txt | 8 ++++++++ arch/x86/include/asm/jailhouse_para.h | 2 +- include/linux/hypervisor.h | 17 +++++++++++++++-- 3 files changed, 24 insertions(+), 3 deletions(-) create mode 100644 Documentation/devicetree/bindings/jailhouse.txt diff --git a/Documentation/devicetree/bindings/jailhouse.txt b/Documentation/devicetree/bindings/jailhouse.txt new file mode 100644 index 000000000000..2901c25ff340 --- /dev/null +++ b/Documentation/devicetree/bindings/jailhouse.txt @@ -0,0 +1,8 @@ +Jailhouse non-root cell device tree bindings +-------------------------------------------- + +When running in a non-root Jailhouse cell (partition), the device tree of this +platform shall have a top-level "hypervisor" node with the following +properties: + +- compatible = "jailhouse,cell" diff --git a/arch/x86/include/asm/jailhouse_para.h b/arch/x86/include/asm/jailhouse_para.h index 875b54376689..b885a961a150 100644 --- a/arch/x86/include/asm/jailhouse_para.h +++ b/arch/x86/include/asm/jailhouse_para.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL2.0 */ /* - * Jailhouse paravirt_ops implementation + * Jailhouse paravirt detection * * Copyright (c) Siemens AG, 2015-2017 * diff --git a/include/linux/hypervisor.h b/include/linux/hypervisor.h index b19563f9a8eb..fc08b433c856 100644 --- a/include/linux/hypervisor.h +++ b/include/linux/hypervisor.h @@ -8,15 +8,28 @@ */ #ifdef CONFIG_X86 + +#include <asm/jailhouse_para.h> #include <asm/x86_init.h> + static inline void hypervisor_pin_vcpu(int cpu) { x86_platform.hyper.pin_vcpu(cpu); } -#else + +#else /* !CONFIG_X86 */ + +#include <linux/of.h> + static inline void hypervisor_pin_vcpu(int cpu) { } -#endif + +static inline bool jailhouse_paravirt(void) +{ + return of_find_compatible_node(NULL, NULL, "jailhouse,cell"); +} + +#endif /* !CONFIG_X86 */ #endif /* __LINUX_HYPEVISOR_H */ -- 2.13.6
Jan Kiszka
2018-Jan-22 06:12 UTC
[PATCH 2/6] pci: Scan all functions when probing while running over Jailhouse
From: Jan Kiszka <jan.kiszka at siemens.com> PCI and PCIBIOS probing only scans devices at function number 0/8/16/... Subdevices (e.g. multiqueue) have function numbers which are not a multiple of 8. The simple hypervisor Jailhouse passes subdevices directly w/o providing a virtual PCI topology like KVM. As a consequence a PCI passthrough from Jailhouse to a guest will not be detected by Linux. Based on patch by Benedikt Spranger, adding Jailhouse probing to avoid changing the behavior in the absence of the hypervisor. CC: Benedikt Spranger <b.spranger at linutronix.de> Signed-off-by: Jan Kiszka <jan.kiszka at siemens.com> --- arch/x86/pci/legacy.c | 4 +++- drivers/pci/probe.c | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/x86/pci/legacy.c b/arch/x86/pci/legacy.c index 1cb01abcb1be..a7b0476b4f44 100644 --- a/arch/x86/pci/legacy.c +++ b/arch/x86/pci/legacy.c @@ -5,6 +5,7 @@ #include <linux/export.h> #include <linux/pci.h> #include <asm/pci_x86.h> +#include <asm/jailhouse_para.h> /* * Discover remaining PCI buses in case there are peer host bridges. @@ -34,13 +35,14 @@ int __init pci_legacy_init(void) void pcibios_scan_specific_bus(int busn) { + int stride = jailhouse_paravirt() ? 1 : 8; int devfn; u32 l; if (pci_find_bus(0, busn)) return; - for (devfn = 0; devfn < 256; devfn += 8) { + for (devfn = 0; devfn < 256; devfn += stride) { if (!raw_pci_read(0, busn, devfn, PCI_VENDOR_ID, 2, &l) && l != 0x0000 && l != 0xffff) { DBG("Found device at %02x:%02x [%04x]\n", busn, devfn, l); diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 14e0ea1ff38b..60ad14c8245f 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -17,6 +17,7 @@ #include <linux/acpi.h> #include <linux/irqdomain.h> #include <linux/pm_runtime.h> +#include <linux/hypervisor.h> #include "pci.h" #define CARDBUS_LATENCY_TIMER 176 /* secondary latency timer */ @@ -2454,6 +2455,7 @@ static unsigned int pci_scan_child_bus_extend(struct pci_bus *bus, unsigned int available_buses) { unsigned int used_buses, normal_bridges = 0, hotplug_bridges = 0; + unsigned int stride = jailhouse_paravirt() ? 1 : 8; unsigned int start = bus->busn_res.start; unsigned int devfn, cmax, max = start; struct pci_dev *dev; @@ -2461,7 +2463,7 @@ static unsigned int pci_scan_child_bus_extend(struct pci_bus *bus, dev_dbg(&bus->dev, "scanning bus\n"); /* Go find them, Rover! */ - for (devfn = 0; devfn < 0x100; devfn += 8) + for (devfn = 0; devfn < 0x100; devfn += stride) pci_scan_slot(bus, devfn); /* Reserve buses for SR-IOV capability. */ -- 2.13.6
Jan Kiszka
2018-Jan-22 06:12 UTC
[PATCH 3/6] x86/jailhouse: Enable PCI mmconfig access in inmates
From: Otavio Pontes <otavio.pontes at intel.com> Use the PCI mmconfig base address exported by jailhouse in boot parameters in order to access the memory mapped PCI configuration space. Signed-off-by: Otavio Pontes <otavio.pontes at intel.com> [Jan: rebased, fixed !CONFIG_PCI_MMCONFIG] Signed-off-by: Jan Kiszka <jan.kiszka at siemens.com> --- arch/x86/include/asm/pci_x86.h | 2 ++ arch/x86/kernel/jailhouse.c | 7 +++++++ arch/x86/pci/mmconfig-shared.c | 4 ++-- 3 files changed, 11 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h index eb66fa9cd0fc..959d618dbb17 100644 --- a/arch/x86/include/asm/pci_x86.h +++ b/arch/x86/include/asm/pci_x86.h @@ -151,6 +151,8 @@ extern int pci_mmconfig_insert(struct device *dev, u16 seg, u8 start, u8 end, phys_addr_t addr); extern int pci_mmconfig_delete(u16 seg, u8 start, u8 end); extern struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus); +extern struct pci_mmcfg_region *__init pci_mmconfig_add(int segment, int start, + int end, u64 addr); extern struct list_head pci_mmcfg_list; diff --git a/arch/x86/kernel/jailhouse.c b/arch/x86/kernel/jailhouse.c index b68fd895235a..7fe2a73da0b3 100644 --- a/arch/x86/kernel/jailhouse.c +++ b/arch/x86/kernel/jailhouse.c @@ -124,6 +124,13 @@ static int __init jailhouse_pci_arch_init(void) if (pcibios_last_bus < 0) pcibios_last_bus = 0xff; +#ifdef CONFIG_PCI_MMCONFIG + if (setup_data.pci_mmconfig_base) { + pci_mmconfig_add(0, 0, 0xff, setup_data.pci_mmconfig_base); + pci_mmcfg_arch_init(); + } +#endif + return 0; } diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c index 96684d0adcf9..0e590272366b 100644 --- a/arch/x86/pci/mmconfig-shared.c +++ b/arch/x86/pci/mmconfig-shared.c @@ -94,8 +94,8 @@ static struct pci_mmcfg_region *pci_mmconfig_alloc(int segment, int start, return new; } -static struct pci_mmcfg_region *__init pci_mmconfig_add(int segment, int start, - int end, u64 addr) +struct pci_mmcfg_region *__init pci_mmconfig_add(int segment, int start, + int end, u64 addr) { struct pci_mmcfg_region *new; -- 2.13.6
From: Jan Kiszka <jan.kiszka at siemens.com> Not sure if those two worked by design or just by chance so far. In any case, it's at least cleaner and clearer to express this in a single config statement. Signed-off-by: Jan Kiszka <jan.kiszka at siemens.com> --- arch/x86/Kconfig | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 423e4b64e683..f2038417a590 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -2595,8 +2595,9 @@ config PCI_DIRECT depends on PCI && (X86_64 || (PCI_GODIRECT || PCI_GOANY || PCI_GOOLPC || PCI_GOMMCONFIG)) config PCI_MMCONFIG - def_bool y - depends on X86_32 && PCI && (ACPI || SFI) && (PCI_GOMMCONFIG || PCI_GOANY) + bool "Support mmconfig PCI config space access" if X86_64 + default y + depends on PCI && (ACPI || SFI) && (PCI_GOMMCONFIG || PCI_GOANY || X86_64) config PCI_OLPC def_bool y @@ -2611,10 +2612,6 @@ config PCI_DOMAINS def_bool y depends on PCI -config PCI_MMCONFIG - bool "Support mmconfig PCI config space access" - depends on X86_64 && PCI && ACPI - config PCI_CNB20LE_QUIRK bool "Read CNB20LE Host Bridge Windows" if EXPERT depends on PCI -- 2.13.6
Jan Kiszka
2018-Jan-22 06:12 UTC
[PATCH 5/6] x86/jailhouse: Allow to use PCI_MMCONFIG without ACPI
From: Jan Kiszka <jan.kiszka at siemens.com> Jailhouse does not use ACPI, but it does support MMCONFIG. Make sure the latter can be built without having to enable ACPI as well. Primarily, we need to make the AMD mmconf-fam10h_64 depend upon MMCONFIG and ACPI, instead of just the former. Saves some bytes in the Jailhouse non-root kernel. Signed-off-by: Jan Kiszka <jan.kiszka at siemens.com> --- arch/x86/Kconfig | 6 +++++- arch/x86/kernel/Makefile | 2 +- arch/x86/kernel/cpu/amd.c | 2 +- 3 files changed, 7 insertions(+), 3 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index f2038417a590..77ba0eb0a258 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -2597,7 +2597,7 @@ config PCI_DIRECT config PCI_MMCONFIG bool "Support mmconfig PCI config space access" if X86_64 default y - depends on PCI && (ACPI || SFI) && (PCI_GOMMCONFIG || PCI_GOANY || X86_64) + depends on PCI && (ACPI || SFI || JAILHOUSE_GUEST) && (PCI_GOMMCONFIG || PCI_GOANY || X86_64) config PCI_OLPC def_bool y @@ -2612,6 +2612,10 @@ config PCI_DOMAINS def_bool y depends on PCI +config MMCONF_FAM10H + def_bool y + depends on PCI_MMCONFIG && ACPI + config PCI_CNB20LE_QUIRK bool "Read CNB20LE Host Bridge Windows" if EXPERT depends on PCI diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index aed9296dccd3..b2c9e230e2fe 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -143,6 +143,6 @@ ifeq ($(CONFIG_X86_64),y) obj-$(CONFIG_GART_IOMMU) += amd_gart_64.o aperture_64.o obj-$(CONFIG_CALGARY_IOMMU) += pci-calgary_64.o tce_64.o - obj-$(CONFIG_PCI_MMCONFIG) += mmconf-fam10h_64.o + obj-$(CONFIG_MMCONF_FAM10H) += mmconf-fam10h_64.o obj-y += vsmp_64.o endif diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c index ea831c858195..47edf599f6fd 100644 --- a/arch/x86/kernel/cpu/amd.c +++ b/arch/x86/kernel/cpu/amd.c @@ -690,7 +690,7 @@ static void init_amd_k8(struct cpuinfo_x86 *c) static void init_amd_gh(struct cpuinfo_x86 *c) { -#ifdef CONFIG_X86_64 +#ifdef CONFIG_MMCONF_FAM10H /* do this for boot cpu */ if (c == &boot_cpu_data) check_enable_amd_mmconf_dmi(); -- 2.13.6
From: Jan Kiszka <jan.kiszka at siemens.com> Signed-off-by: Jan Kiszka <jan.kiszka at siemens.com> --- MAINTAINERS | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 426ba037d943..dd51a2012b36 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -7468,6 +7468,13 @@ Q: http://patchwork.linuxtv.org/project/linux-media/list/ S: Maintained F: drivers/media/dvb-frontends/ix2505v* +JAILHOUSE HYPERVISOR INTERFACE +M: Jan Kiszka <jan.kiszka at siemens.com> +L: jailhouse-dev at googlegroups.com +S: Maintained +F: arch/x86/kernel/jailhouse.c +F: arch/x86/include/asm/jailhouse_para.h + JC42.4 TEMPERATURE SENSOR DRIVER M: Guenter Roeck <linux at roeck-us.net> L: linux-hwmon at vger.kernel.org -- 2.13.6
On Mon, Jan 22, 2018 at 8:12 AM, Jan Kiszka <jan.kiszka at siemens.com> wrote:> From: Jan Kiszka <jan.kiszka at siemens.com> > > Not sure if those two worked by design or just by chance so far. In any > case, it's at least cleaner and clearer to express this in a single > config statement.Congrats! You found by the way a bug in commit e279b6c1d329e50b766bce96aacc197eae8a053b Author: Sam Ravnborg <sam at ravnborg.org> Date: Tue Nov 6 20:41:05 2007 +0100 x86: start unification of arch/x86/Kconfig.* ...and proper fix seems to split PCI stuff to common + X86_32 only + X86_64 only -- With Best Regards, Andy Shevchenko
Andy Shevchenko
2018-Feb-23 13:23 UTC
[PATCH 2/6] pci: Scan all functions when probing while running over Jailhouse
On Mon, Jan 22, 2018 at 8:12 AM, Jan Kiszka <jan.kiszka at siemens.com> wrote:> #include <linux/export.h> > #include <linux/pci.h> > #include <asm/pci_x86.h> > +#include <asm/jailhouse_para.h>Keep it in order?> #include <linux/acpi.h> > #include <linux/irqdomain.h> > #include <linux/pm_runtime.h> > +#include <linux/hypervisor.h>Ditto. -- With Best Regards, Andy Shevchenko
Jan Kiszka
2018-Feb-27 07:25 UTC
[PATCH 2/6] pci: Scan all functions when probing while running over Jailhouse
On 2018-02-22 21:57, Bjorn Helgaas wrote:> On Mon, Jan 22, 2018 at 07:12:46AM +0100, Jan Kiszka wrote: >> From: Jan Kiszka <jan.kiszka at siemens.com> >> >> PCI and PCIBIOS probing only scans devices at function number 0/8/16/... >> Subdevices (e.g. multiqueue) have function numbers which are not a >> multiple of 8. > > Suggested text: > > Per PCIe r4.0, sec 7.5.1.1.9, multi-function devices are required to > have a function 0. Therefore, Linux scans for devices at function 0 > (devfn 0/8/16/...) and only scans for other functions if function 0 > has its Multi-Function Device bit set or ARI or SR-IOV indicate > there are more functions. > > The Jailhouse hypervisor may pass individual functions of a > multi-function device to a guest without passing function 0, which > means a Linux guest won't find them. > > Change Linux PCI probing so it scans all function numbers when > running as a guest over Jailhouse. > > This is technically prohibited by the spec, so it is possible that > PCI devices without the Multi-Function Device bit set may have > unexpected behavior in response to this probe. > >> The simple hypervisor Jailhouse passes subdevices directly w/o providing >> a virtual PCI topology like KVM. As a consequence a PCI passthrough from >> Jailhouse to a guest will not be detected by Linux. >> >> Based on patch by Benedikt Spranger, adding Jailhouse probing to avoid >> changing the behavior in the absence of the hypervisor. >> >> CC: Benedikt Spranger <b.spranger at linutronix.de> >> Signed-off-by: Jan Kiszka <jan.kiszka at siemens.com> > > With subject change to: > > PCI: Scan all functions when running over Jailhouse > > Acked-by: Bjorn Helgaas <bhelgaas at google.com> >Thanks, all suggestions picked up for next round. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
Maybe Matching Threads
- [PATCH v4 3/7] x86/jailhouse: Enable PCI mmconfig access in inmates
- [PATCH v2 3/6] x86/jailhouse: Enable PCI mmconfig access in inmates
- [PATCH v3 3/6] x86/jailhouse: Enable PCI mmconfig access in inmates
- [PATCH v3 3/6] x86/jailhouse: Enable PCI mmconfig access in inmates
- [PATCH v4 0/7] jailhouse: Enhance secondary Jailhouse guest support /wrt PCI