thr3ads.net - Xen devel - [PATCH RFC 0/6] Introduce Xen support to ARM64 [May 2013]

If this information is useful, please help other people find it:
Share via:

Stefano Stabellini

2013-May-30 16:18 UTC

[PATCH RFC 0/6] Introduce Xen support to ARM64

Hi all,
this patch series introduces Xen support to arch/arm64.

As you can see from the following patches, there is very little arm64
specific code here, basically only one assembly file
(arch/arm64/xen/hypercall.S).

Everything else is common with Xen for ARMv7, in particular the code
under arch/arm/xen (enlighten.c and the stubs in grant-table.c) and all
the header files under arch/arm/include/asm/xen (events.h, hypercall.h,
hypervisor.h, interface.h, page.h) can be reused (almost) as-is.

Not knowning what is the best way forward I have just created symlinks
to the original files under arch/arm, see the first patch.

Please advice on what you think is the best way forward.



Stefano Stabellini (6):
      [HACK!] arm64/xen: create links to arch/arm include files and Xen code
      arm64/xen: arm header changes to compile on arm64
      arm64/xen: use XEN_IO_PROTO_ABI_ARM on ARM64
      arm64/xen: introduce asm/hypervisor.h and sys_bitops.h
      arm64/xen: implement xen_remap on arm64
      arm64/xen: introduce CONFIG_XEN and hypercall.S on ARM64

 arch/arm/include/asm/xen/events.h    |    4 +
 arch/arm/include/asm/xen/page.h      |    6 ++
 arch/arm64/Kconfig                   |   11 ++++
 arch/arm64/Makefile                  |    1 +
 arch/arm64/include/asm/hypervisor.h  |    6 ++
 arch/arm64/include/asm/io.h          |    1 +
 arch/arm64/include/asm/sync_bitops.h |   26 ++++++++
 arch/arm64/include/asm/xen           |    1 +
 arch/arm64/xen/Makefile              |    1 +
 arch/arm64/xen/enlighten.c           |    1 +
 arch/arm64/xen/grant-table.c         |    1 +
 arch/arm64/xen/hypercall.S           |  105 ++++++++++++++++++++++++++++++++++
 include/xen/interface/io/protocols.h |    2 +-
 13 files changed, 165 insertions(+), 1 deletions(-)
 create mode 100644 arch/arm64/include/asm/hypervisor.h
 create mode 100644 arch/arm64/include/asm/sync_bitops.h
 create mode 120000 arch/arm64/include/asm/xen
 create mode 100644 arch/arm64/xen/Makefile
 create mode 120000 arch/arm64/xen/enlighten.c
 create mode 120000 arch/arm64/xen/grant-table.c
 create mode 100644 arch/arm64/xen/hypercall.S

- Stefano

Stefano Stabellini

2013-May-30 16:18 UTC

head link

[PATCH RFC 1/6] [HACK!] arm64/xen: create links to arch/arm include files and Xen code

Most of Xen support for ARM is common between ARMv7 and ARMv8.
Create links to the code under arch/arm (bleah).

Other, probably better alternatives:

- move the code to a different location, maybe the header files to
include/xen/arm and the code to drivers/xen/arm (still pretty ugly)?

- create a copy of the code to arch/arm64 (even worse);

Please advice on how you would like me to move forward on this.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/arm64/include/asm/xen   |    1 +
 arch/arm64/xen/enlighten.c   |    1 +
 arch/arm64/xen/grant-table.c |    1 +
 3 files changed, 3 insertions(+), 0 deletions(-)
 create mode 120000 arch/arm64/include/asm/xen
 create mode 120000 arch/arm64/xen/enlighten.c
 create mode 120000 arch/arm64/xen/grant-table.c

diff --git a/arch/arm64/include/asm/xen b/arch/arm64/include/asm/xen
new file mode 120000
index 0000000..c27d6980
--- /dev/null
+++ b/arch/arm64/include/asm/xen
@@ -0,0 +1 @@
+../../../arm/include/asm/xen
\ No newline at end of file
diff --git a/arch/arm64/xen/enlighten.c b/arch/arm64/xen/enlighten.c
new file mode 120000
index 0000000..cb31b7c
--- /dev/null
+++ b/arch/arm64/xen/enlighten.c
@@ -0,0 +1 @@
+../../arm/xen/enlighten.c
\ No newline at end of file
diff --git a/arch/arm64/xen/grant-table.c b/arch/arm64/xen/grant-table.c
new file mode 120000
index 0000000..5494f6c
--- /dev/null
+++ b/arch/arm64/xen/grant-table.c
@@ -0,0 +1 @@
+../../arm/xen/grant-table.c
\ No newline at end of file
-- 
1.7.2.5

Stefano Stabellini

2013-May-30 16:18 UTC

head link

[PATCH RFC 2/6] arm64/xen: arm/xen header changes to compile on arm64

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/arm/include/asm/xen/events.h |    4 ++++
 arch/arm/include/asm/xen/page.h   |    2 ++
 2 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/arch/arm/include/asm/xen/events.h
b/arch/arm/include/asm/xen/events.h
index 8b1f37b..12e4e0b 100644
--- a/arch/arm/include/asm/xen/events.h
+++ b/arch/arm/include/asm/xen/events.h
@@ -13,7 +13,11 @@ enum ipi_vector {
 
 static inline int xen_irqs_disabled(struct pt_regs *regs)
 {
+#ifdef CONFIG_ARM
 	return raw_irqs_disabled_flags(regs->ARM_cpsr);
+#else
+	return raw_irqs_disabled_flags((unsigned long) regs->pstate);
+#endif
 }
 
 #define xchg_xen_ulong(ptr, val) atomic64_xchg(container_of((ptr),	\
diff --git a/arch/arm/include/asm/xen/page.h b/arch/arm/include/asm/xen/page.h
index 30cdacb..cb2fa15 100644
--- a/arch/arm/include/asm/xen/page.h
+++ b/arch/arm/include/asm/xen/page.h
@@ -1,7 +1,9 @@
 #ifndef _ASM_ARM_XEN_PAGE_H
 #define _ASM_ARM_XEN_PAGE_H
 
+#ifdef ARM
 #include <asm/mach/map.h>
+#endif
 #include <asm/page.h>
 #include <asm/pgtable.h>
 
-- 
1.7.2.5

Stefano Stabellini

2013-May-30 16:18 UTC

head link

[PATCH RFC 3/6] arm64/xen: use XEN_IO_PROTO_ABI_ARM on ARM64

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 include/xen/interface/io/protocols.h |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/include/xen/interface/io/protocols.h
b/include/xen/interface/io/protocols.h
index 0eafaf2..056744b 100644
--- a/include/xen/interface/io/protocols.h
+++ b/include/xen/interface/io/protocols.h
@@ -15,7 +15,7 @@
 # define XEN_IO_PROTO_ABI_NATIVE XEN_IO_PROTO_ABI_IA64
 #elif defined(__powerpc64__)
 # define XEN_IO_PROTO_ABI_NATIVE XEN_IO_PROTO_ABI_POWERPC64
-#elif defined(__arm__)
+#elif defined(__arm__) || defined(__aarch64__)
 # define XEN_IO_PROTO_ABI_NATIVE XEN_IO_PROTO_ABI_ARM
 #else
 # error arch fixup needed here
-- 
1.7.2.5

Stefano Stabellini

2013-May-30 16:18 UTC

head link

[PATCH RFC 4/6] arm64/xen: introduce asm/hypervisor.h and sys_bitops.h

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/arm64/include/asm/hypervisor.h  |    6 ++++++
 arch/arm64/include/asm/sync_bitops.h |   26 ++++++++++++++++++++++++++
 2 files changed, 32 insertions(+), 0 deletions(-)
 create mode 100644 arch/arm64/include/asm/hypervisor.h
 create mode 100644 arch/arm64/include/asm/sync_bitops.h

diff --git a/arch/arm64/include/asm/hypervisor.h
b/arch/arm64/include/asm/hypervisor.h
new file mode 100644
index 0000000..b90d9e5
--- /dev/null
+++ b/arch/arm64/include/asm/hypervisor.h
@@ -0,0 +1,6 @@
+#ifndef _ASM_ARM_HYPERVISOR_H
+#define _ASM_ARM_HYPERVISOR_H
+
+#include <asm/xen/hypervisor.h>
+
+#endif
diff --git a/arch/arm64/include/asm/sync_bitops.h
b/arch/arm64/include/asm/sync_bitops.h
new file mode 100644
index 0000000..8da0bf4
--- /dev/null
+++ b/arch/arm64/include/asm/sync_bitops.h
@@ -0,0 +1,26 @@
+#ifndef __ASM_SYNC_BITOPS_H__
+#define __ASM_SYNC_BITOPS_H__
+
+#include <asm/bitops.h>
+#include <asm/cmpxchg.h>
+
+/* sync_bitops functions are equivalent to the SMP implementation of the
+ * original functions, independently from CONFIG_SMP being defined.
+ *
+ * We need them because _set_bit etc are not SMP safe if !CONFIG_SMP. But
+ * under Xen you might be communicating with a completely external entity
+ * who might be on another CPU (e.g. two uniprocessor guests communicating
+ * via event channels and grant tables). So we need a variant of the bit
+ * ops which are SMP safe even on a UP kernel.
+ */
+
+#define sync_set_bit(nr, p)            set_bit(nr, p)
+#define sync_clear_bit(nr, p)          clear_bit(nr, p)
+#define sync_change_bit(nr, p)         change_bit(nr, p)
+#define sync_test_and_set_bit(nr, p)   test_and_set_bit(nr, p)
+#define sync_test_and_clear_bit(nr, p) test_and_clear_bit(nr, p)
+#define sync_test_and_change_bit(nr, p)        test_and_change_bit(nr, p)
+#define sync_test_bit(nr, addr)                test_bit(nr, addr)
+#define sync_cmpxchg                   cmpxchg
+
+#endif
-- 
1.7.2.5

Stefano Stabellini

2013-May-30 16:18 UTC

head link

[PATCH RFC 5/6] arm64/xen: implement xen_remap on arm64

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/arm/include/asm/xen/page.h |    4 ++++
 arch/arm64/include/asm/io.h     |    1 +
 2 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/arch/arm/include/asm/xen/page.h b/arch/arm/include/asm/xen/page.h
index cb2fa15..d995ece 100644
--- a/arch/arm/include/asm/xen/page.h
+++ b/arch/arm/include/asm/xen/page.h
@@ -90,6 +90,10 @@ static inline bool set_phys_to_machine(unsigned long pfn,
unsigned long mfn)
 	return __set_phys_to_machine(pfn, mfn);
 }
 
+#ifdef CONFIG_ARM64
+#define xen_remap(cookie, size) __ioremap((cookie), (size),
__pgprot(PROT_NORMAL))
+#else
 #define xen_remap(cookie, size) __arm_ioremap((cookie), (size), MT_MEMORY);
+#endif
 
 #endif /* _ASM_ARM_XEN_PAGE_H */
diff --git a/arch/arm64/include/asm/io.h b/arch/arm64/include/asm/io.h
index 2e12258..0e9c9ac 100644
--- a/arch/arm64/include/asm/io.h
+++ b/arch/arm64/include/asm/io.h
@@ -228,6 +228,7 @@ extern void __iounmap(volatile void __iomem *addr);
 #define PROT_DEFAULT		(PTE_TYPE_PAGE | PTE_AF | PTE_DIRTY)
 #define PROT_DEVICE_nGnRE	(PROT_DEFAULT | PTE_PXN | PTE_UXN |
PTE_ATTRINDX(MT_DEVICE_nGnRE))
 #define PROT_NORMAL_NC		(PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL_NC))
+#define PROT_NORMAL		(PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL))
 
 #define ioremap(addr, size)		__ioremap((addr), (size),
__pgprot(PROT_DEVICE_nGnRE))
 #define ioremap_nocache(addr, size)	__ioremap((addr), (size),
__pgprot(PROT_DEVICE_nGnRE))
-- 
1.7.2.5

Stefano Stabellini

2013-May-30 16:18 UTC

head link

[PATCH RFC 6/6] arm64/xen: introduce CONFIG_XEN and hypercall.S on ARM64

Introduce CONFIG_XEN and the implementation of hypercall.S (that is
the only ARMv8 specific code in Xen support for ARM).

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/arm64/Kconfig         |   11 +++++
 arch/arm64/Makefile        |    1 +
 arch/arm64/xen/Makefile    |    1 +
 arch/arm64/xen/hypercall.S |  105 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 118 insertions(+), 0 deletions(-)
 create mode 100644 arch/arm64/xen/Makefile
 create mode 100644 arch/arm64/xen/hypercall.S

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 56b3f6d..f35751f 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -182,6 +182,17 @@ config HW_PERF_EVENTS
 
 source "mm/Kconfig"
 
+config XEN_DOM0
+	def_bool y
+	depends on XEN
+
+config XEN
+	bool "Xen guest support on ARM64 (EXPERIMENTAL)"
+	depends on ARM64 && OF
+	depends on !GENERIC_ATOMIC64
+	help
+	  Say Y if you want to run Linux in a Virtual Machine on Xen on ARM64.
+
 endmenu
 
 menu "Boot options"
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index c95c5cb..79dd13d 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -37,6 +37,7 @@ TEXT_OFFSET := 0x00080000
 export	TEXT_OFFSET GZFLAGS
 
 core-y		+= arch/arm64/kernel/ arch/arm64/mm/
+core-$(CONFIG_XEN)		+= arch/arm64/xen/
 libs-y		:= arch/arm64/lib/ $(libs-y)
 libs-y		+= $(LIBGCC)
 
diff --git a/arch/arm64/xen/Makefile b/arch/arm64/xen/Makefile
new file mode 100644
index 0000000..4384103
--- /dev/null
+++ b/arch/arm64/xen/Makefile
@@ -0,0 +1 @@
+obj-y		:= enlighten.o hypercall.o grant-table.o
diff --git a/arch/arm64/xen/hypercall.S b/arch/arm64/xen/hypercall.S
new file mode 100644
index 0000000..131e4a6
--- /dev/null
+++ b/arch/arm64/xen/hypercall.S
@@ -0,0 +1,105 @@
+/******************************************************************************
+ * hypercall.S
+ *
+ * Xen hypercall wrappers
+ *
+ * Stefano Stabellini <stefano.stabellini@eu.citrix.com>, Citrix, 2012
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version 2
+ * as published by the Free Software Foundation; or, when distributed
+ * separately from the Linux kernel or incorporated into other
+ * software packages, subject to the following license:
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this source file (the "Software"), to deal in the Software
without
+ * restriction, including without limitation the rights to use, copy, modify,
+ * merge, publish, distribute, sublicense, and/or sell copies of the Software,
+ * and to permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+
+/*
+ * The Xen hypercall calling convention is very similar to the ARM AEBI
+ * procedure calling convention: the first paramter is passed in x0, the
+ * second in x1, the third in x2 and the fourth in x3. Considering that
+ * Xen hypercalls have 5 arguments at most, the fifth paramter is passed
+ * in rx, differently from the procedure calling convention of using the
+ * stack for that case.
+ *
+ * The hypercall number is passed in x16.
+ *
+ * The return value is in x0.
+ *
+ * The hvc ISS is required to be 0xEA1, that is the Xen specific ARM
+ * hypercall tag.
+ *
+ * Parameter structs passed to hypercalls are laid out according to
+ * the ARM EABI standard.
+ */
+
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+#include <xen/interface/xen.h>
+
+
+#define XEN_IMM 0xEA1
+
+#define HYPERCALL_SIMPLE(hypercall)		\
+ENTRY(HYPERVISOR_##hypercall)			\
+	mov x16, #__HYPERVISOR_##hypercall;	\
+	hvc XEN_IMM;						\
+	ret;								\
+ENDPROC(HYPERVISOR_##hypercall)
+
+#define HYPERCALL0 HYPERCALL_SIMPLE
+#define HYPERCALL1 HYPERCALL_SIMPLE
+#define HYPERCALL2 HYPERCALL_SIMPLE
+#define HYPERCALL3 HYPERCALL_SIMPLE
+#define HYPERCALL4 HYPERCALL_SIMPLE
+
+#define HYPERCALL5(hypercall)			\
+ENTRY(HYPERVISOR_##hypercall)			\
+	stmdb sp!, {x4}						\
+	ldr x4, [sp, #4]					\
+	mov x16, #__HYPERVISOR_##hypercall;	\
+	hvc XEN_IMM;						\
+	ldm sp!, {r4}						\
+	ret									\
+ENDPROC(HYPERVISOR_##hypercall)
+
+                .text
+
+HYPERCALL2(xen_version);
+HYPERCALL3(console_io);
+HYPERCALL3(grant_table_op);
+HYPERCALL2(sched_op);
+HYPERCALL2(event_channel_op);
+HYPERCALL2(hvm_op);
+HYPERCALL2(memory_op);
+HYPERCALL2(physdev_op);
+HYPERCALL3(vcpu_op);
+
+ENTRY(privcmd_call)
+	push x4, xzr
+	mov x16, x0
+	mov x0, x1
+	mov x1, x2
+	mov x2, x3
+	ldr x3, [sp, #8]
+	ldr x4, [sp, #4]
+	hvc XEN_IMM
+	pop x5, xzr
+	ret
+ENDPROC(privcmd_call);
-- 
1.7.2.5

Catalin Marinas

2013-May-31 10:36 UTC

head link

Re: [PATCH RFC 1/6] [HACK!] arm64/xen: create links to arch/arm include files and Xen code

On Thu, May 30, 2013 at 05:18:28PM +0100, Stefano Stabellini
wrote:> Most of Xen support for ARM is common between ARMv7 and ARMv8.
> Create links to the code under arch/arm (bleah).
> 
> Other, probably better alternatives:
> 
> - move the code to a different location, maybe the header files to
> include/xen/arm and the code to drivers/xen/arm (still pretty ugly)?
> 
> - create a copy of the code to arch/arm64 (even worse);
KVM handles this in the Makefile by referencing back to arch/arm or even
the generic kvm directory. I think that''s the
''cleanest'' ;)

-- 
Catalin

Catalin Marinas

2013-May-31 10:43 UTC

head link

Re: [PATCH RFC 2/6] arm64/xen: arm/xen header changes to compile on arm64

On Thu, May 30, 2013 at 05:18:29PM +0100, Stefano Stabellini
wrote:> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> ---
>  arch/arm/include/asm/xen/events.h |    4 ++++
>  arch/arm/include/asm/xen/page.h   |    2 ++
>  2 files changed, 6 insertions(+), 0 deletions(-)
> 
> diff --git a/arch/arm/include/asm/xen/events.h
b/arch/arm/include/asm/xen/events.h
> index 8b1f37b..12e4e0b 100644
> --- a/arch/arm/include/asm/xen/events.h
> +++ b/arch/arm/include/asm/xen/events.h
> @@ -13,7 +13,11 @@ enum ipi_vector {
>  
>  static inline int xen_irqs_disabled(struct pt_regs *regs)
>  {
> +#ifdef CONFIG_ARM
>  	return raw_irqs_disabled_flags(regs->ARM_cpsr);
> +#else
> +	return raw_irqs_disabled_flags((unsigned long) regs->pstate);
> +#endif
>  }
At least for this part, it makes sense to live in the arch/arm64
directory.
>  #define xchg_xen_ulong(ptr, val) atomic64_xchg(container_of((ptr),	\
> diff --git a/arch/arm/include/asm/xen/page.h
b/arch/arm/include/asm/xen/page.h
> index 30cdacb..cb2fa15 100644
> --- a/arch/arm/include/asm/xen/page.h
> +++ b/arch/arm/include/asm/xen/page.h
> @@ -1,7 +1,9 @@
>  #ifndef _ASM_ARM_XEN_PAGE_H
>  #define _ASM_ARM_XEN_PAGE_H
>  
> +#ifdef ARM
>  #include <asm/mach/map.h>
> +#endif
>  #include <asm/page.h>
>  #include <asm/pgtable.h>
Is there anything ARM-specific in this file?

-- 
Catalin

Catalin Marinas

2013-May-31 10:59 UTC

head link

Re: [PATCH RFC 5/6] arm64/xen: implement xen_remap on arm64

On Thu, May 30, 2013 at 05:18:32PM +0100, Stefano Stabellini
wrote:> --- a/arch/arm/include/asm/xen/page.h
> +++ b/arch/arm/include/asm/xen/page.h
> @@ -90,6 +90,10 @@ static inline bool set_phys_to_machine(unsigned long
pfn, unsigned long mfn)
>  	return __set_phys_to_machine(pfn, mfn);
>  }
>  
> +#ifdef CONFIG_ARM64
> +#define xen_remap(cookie, size) __ioremap((cookie), (size),
__pgprot(PROT_NORMAL))
> +#else
>  #define xen_remap(cookie, size) __arm_ioremap((cookie), (size),
MT_MEMORY);
> +#endif
Now I saw the ARM-specific part. Can you not use something like
ioremap_cached() which would give normal cacheable memory (at least on
ARMv7).
>  #endif /* _ASM_ARM_XEN_PAGE_H */
> diff --git a/arch/arm64/include/asm/io.h b/arch/arm64/include/asm/io.h
> index 2e12258..0e9c9ac 100644
> --- a/arch/arm64/include/asm/io.h
> +++ b/arch/arm64/include/asm/io.h
> @@ -228,6 +228,7 @@ extern void __iounmap(volatile void __iomem *addr);
>  #define PROT_DEFAULT		(PTE_TYPE_PAGE | PTE_AF | PTE_DIRTY)
>  #define PROT_DEVICE_nGnRE	(PROT_DEFAULT | PTE_PXN | PTE_UXN |
PTE_ATTRINDX(MT_DEVICE_nGnRE))
>  #define PROT_NORMAL_NC		(PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL_NC))
> +#define PROT_NORMAL		(PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL))
>  
>  #define ioremap(addr, size)		__ioremap((addr), (size),
__pgprot(PROT_DEVICE_nGnRE))
>  #define ioremap_nocache(addr, size)	__ioremap((addr), (size),
__pgprot(PROT_DEVICE_nGnRE))
Of course we need to add ioremap_cached() for arm64 but until now we
didn''t need it.

-- 
Catalin

Catalin Marinas

2013-May-31 11:02 UTC

head link

Re: [PATCH RFC 6/6] arm64/xen: introduce CONFIG_XEN and hypercall.S on ARM64

On Thu, May 30, 2013 at 05:18:33PM +0100, Stefano Stabellini
wrote:> Introduce CONFIG_XEN and the implementation of hypercall.S (that is
> the only ARMv8 specific code in Xen support for ARM).
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> ---
>  arch/arm64/Kconfig         |   11 +++++
>  arch/arm64/Makefile        |    1 +
>  arch/arm64/xen/Makefile    |    1 +
>  arch/arm64/xen/hypercall.S |  105
++++++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 118 insertions(+), 0 deletions(-)
>  create mode 100644 arch/arm64/xen/Makefile
>  create mode 100644 arch/arm64/xen/hypercall.S
> 
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 56b3f6d..f35751f 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -182,6 +182,17 @@ config HW_PERF_EVENTS
>  
>  source "mm/Kconfig"
>  
> +config XEN_DOM0
> +	def_bool y
> +	depends on XEN
> +
> +config XEN
> +	bool "Xen guest support on ARM64 (EXPERIMENTAL)"
> +	depends on ARM64 && OF
> +	depends on !GENERIC_ATOMIC64
I don''t think we need this depends on !GENERIC_ATOMIC64, we
don''t enable
it on arm64.

-- 
Catalin

Stefano Stabellini

2013-May-31 12:02 UTC

head link

Re: [PATCH RFC 1/6] [HACK!] arm64/xen: create links to arch/arm include files and Xen code

On Fri, 31 May 2013, Catalin Marinas wrote:> On Thu, May 30, 2013 at 05:18:28PM +0100, Stefano Stabellini wrote:
> > Most of Xen support for ARM is common between ARMv7 and ARMv8.
> > Create links to the code under arch/arm (bleah).
> > 
> > Other, probably better alternatives:
> > 
> > - move the code to a different location, maybe the header files to
> > include/xen/arm and the code to drivers/xen/arm (still pretty ugly)?
> > 
> > - create a copy of the code to arch/arm64 (even worse);
> 
> KVM handles this in the Makefile by referencing back to arch/arm or even
> the generic kvm directory. I think that''s the
''cleanest'' ;)
Do you mean creating links in the Makefile or generating header file
copies from the Makefile?
Or do you mean using something like "-I arch/arm/include/asm/xen" in
the
arch/arm64 Makefile?

I tried to find exactly what KVM is doing but I couldn''t find the code
you are talking about..

Catalin Marinas

2013-May-31 13:10 UTC

head link

Re: [PATCH RFC 1/6] [HACK!] arm64/xen: create links to arch/arm include files and Xen code

On Fri, May 31, 2013 at 01:02:04PM +0100, Stefano Stabellini
wrote:> On Fri, 31 May 2013, Catalin Marinas wrote:
> > On Thu, May 30, 2013 at 05:18:28PM +0100, Stefano Stabellini wrote:
> > > Most of Xen support for ARM is common between ARMv7 and ARMv8.
> > > Create links to the code under arch/arm (bleah).
> > > 
> > > Other, probably better alternatives:
> > > 
> > > - move the code to a different location, maybe the header files
to
> > > include/xen/arm and the code to drivers/xen/arm (still pretty
ugly)?
> > > 
> > > - create a copy of the code to arch/arm64 (even worse);
> > 
> > KVM handles this in the Makefile by referencing back to arch/arm or
even
> > the generic kvm directory. I think that''s the
''cleanest'' ;)
> 
> Do you mean creating links in the Makefile or generating header file
> copies from the Makefile?
> Or do you mean using something like "-I arch/arm/include/asm/xen"
in the
> arch/arm64 Makefile?
I meant C files being compiled directly from arch/arm (no links).

For headers, if they are specific to arm or arm64, just copy the header
in each place (e.g. not using regs->pstate in the arch/arm code with
#ifdef''s). For the rest, if they cannot be made generic, one way is to
have a dummy file including the arm equivalent:

#include <../../arm/include/asm/xen/events.h>

Passing -I is dangerous as you actually need "-I arch/arm/include"
which
could bring other files.

-- 
Catalin

Stefano Stabellini

2013-May-31 13:21 UTC

head link

Re: [PATCH RFC 5/6] arm64/xen: implement xen_remap on arm64

On Fri, 31 May 2013, Catalin Marinas wrote:> On Thu, May 30, 2013 at 05:18:32PM +0100, Stefano Stabellini wrote:
> > --- a/arch/arm/include/asm/xen/page.h
> > +++ b/arch/arm/include/asm/xen/page.h
> > @@ -90,6 +90,10 @@ static inline bool set_phys_to_machine(unsigned
long pfn, unsigned long mfn)
> >  	return __set_phys_to_machine(pfn, mfn);
> >  }
> >  
> > +#ifdef CONFIG_ARM64
> > +#define xen_remap(cookie, size) __ioremap((cookie), (size),
__pgprot(PROT_NORMAL))
> > +#else
> >  #define xen_remap(cookie, size) __arm_ioremap((cookie), (size),
MT_MEMORY);
> > +#endif
> 
> Now I saw the ARM-specific part. Can you not use something like
> ioremap_cached() which would give normal cacheable memory (at least on
> ARMv7).
No, I cannot because ioremap_cached uses MT_DEVICE_CACHED, while this
needs to be MT_MEMORY. It is used for normal memory pages, not device
memory.

Stefano Stabellini

2013-May-31 13:25 UTC

head link

Re: [PATCH RFC 2/6] arm64/xen: arm/xen header changes to compile on arm64

On Fri, 31 May 2013, Catalin Marinas wrote:> On Thu, May 30, 2013 at 05:18:29PM +0100, Stefano Stabellini wrote:
> > Signed-off-by: Stefano Stabellini
<stefano.stabellini@eu.citrix.com>
> > ---
> >  arch/arm/include/asm/xen/events.h |    4 ++++
> >  arch/arm/include/asm/xen/page.h   |    2 ++
> >  2 files changed, 6 insertions(+), 0 deletions(-)
> > 
> > diff --git a/arch/arm/include/asm/xen/events.h
b/arch/arm/include/asm/xen/events.h
> > index 8b1f37b..12e4e0b 100644
> > --- a/arch/arm/include/asm/xen/events.h
> > +++ b/arch/arm/include/asm/xen/events.h
> > @@ -13,7 +13,11 @@ enum ipi_vector {
> >  
> >  static inline int xen_irqs_disabled(struct pt_regs *regs)
> >  {
> > +#ifdef CONFIG_ARM
> >  	return raw_irqs_disabled_flags(regs->ARM_cpsr);
> > +#else
> > +	return raw_irqs_disabled_flags((unsigned long) regs->pstate);
> > +#endif
> >  }
> 
> At least for this part, it makes sense to live in the arch/arm64
> directory.
Yes, good idea, there isn''t much else in that header file anyway.

> >  #define xchg_xen_ulong(ptr, val) atomic64_xchg(container_of((ptr),	\
> > diff --git a/arch/arm/include/asm/xen/page.h
b/arch/arm/include/asm/xen/page.h
> > index 30cdacb..cb2fa15 100644
> > --- a/arch/arm/include/asm/xen/page.h
> > +++ b/arch/arm/include/asm/xen/page.h
> > @@ -1,7 +1,9 @@
> >  #ifndef _ASM_ARM_XEN_PAGE_H
> >  #define _ASM_ARM_XEN_PAGE_H
> >  
> > +#ifdef ARM
> >  #include <asm/mach/map.h>
> > +#endif
> >  #include <asm/page.h>
> >  #include <asm/pgtable.h>
> 
> Is there anything ARM-specific in this file?
The only ARM-specific stuff here is the definition of xen_remap, like
you have spotted in the other email. The inclusion of asm/mach/map.h is
needed because it containes the definition of MT_MEMORY.

Catalin Marinas

2013-May-31 14:16 UTC

head link

Re: [PATCH RFC 5/6] arm64/xen: implement xen_remap on arm64

On Fri, May 31, 2013 at 02:21:16PM +0100, Stefano Stabellini
wrote:> On Fri, 31 May 2013, Catalin Marinas wrote:
> > On Thu, May 30, 2013 at 05:18:32PM +0100, Stefano Stabellini wrote:
> > > --- a/arch/arm/include/asm/xen/page.h
> > > +++ b/arch/arm/include/asm/xen/page.h
> > > @@ -90,6 +90,10 @@ static inline bool
set_phys_to_machine(unsigned long pfn, unsigned long mfn)
> > >  	return __set_phys_to_machine(pfn, mfn);
> > >  }
> > >  
> > > +#ifdef CONFIG_ARM64
> > > +#define xen_remap(cookie, size) __ioremap((cookie), (size),
__pgprot(PROT_NORMAL))
> > > +#else
> > >  #define xen_remap(cookie, size) __arm_ioremap((cookie), (size),
MT_MEMORY);
> > > +#endif
> > 
> > Now I saw the ARM-specific part. Can you not use something like
> > ioremap_cached() which would give normal cacheable memory (at least on
> > ARMv7).
> 
> No, I cannot because ioremap_cached uses MT_DEVICE_CACHED, while this
> needs to be MT_MEMORY. It is used for normal memory pages, not device
> memory.
MT_DEVICE_CACHED is Normal memory for ARMv7.

-- 
Catalin

Stefano Stabellini

2013-May-31 14:50 UTC

head link

Re: [PATCH RFC 5/6] arm64/xen: implement xen_remap on arm64

On Fri, 31 May 2013, Catalin Marinas wrote:> On Fri, May 31, 2013 at 02:21:16PM +0100, Stefano Stabellini wrote:
> > On Fri, 31 May 2013, Catalin Marinas wrote:
> > > On Thu, May 30, 2013 at 05:18:32PM +0100, Stefano Stabellini
wrote:
> > > > --- a/arch/arm/include/asm/xen/page.h
> > > > +++ b/arch/arm/include/asm/xen/page.h
> > > > @@ -90,6 +90,10 @@ static inline bool
set_phys_to_machine(unsigned long pfn, unsigned long mfn)
> > > >  	return __set_phys_to_machine(pfn, mfn);
> > > >  }
> > > >  
> > > > +#ifdef CONFIG_ARM64
> > > > +#define xen_remap(cookie, size) __ioremap((cookie), (size),
__pgprot(PROT_NORMAL))
> > > > +#else
> > > >  #define xen_remap(cookie, size) __arm_ioremap((cookie),
(size), MT_MEMORY);
> > > > +#endif
> > > 
> > > Now I saw the ARM-specific part. Can you not use something like
> > > ioremap_cached() which would give normal cacheable memory (at
least on
> > > ARMv7).
> > 
> > No, I cannot because ioremap_cached uses MT_DEVICE_CACHED, while this
> > needs to be MT_MEMORY. It is used for normal memory pages, not device
> > memory.
> 
> MT_DEVICE_CACHED is Normal memory for ARMv7.
I didn''t realize that MT_DEVICE_CACHED and MT_MEMORY end up having the
same AttrIndx encoding!
In that case yes, I should be able to use it.
I take that I should just implement ioremap_cached on arm64 as well?

Stefano Stabellini

2013-May-31 15:20 UTC

head link

Re: [PATCH RFC 1/6] [HACK!] arm64/xen: create links to arch/arm include files and Xen code

On Fri, 31 May 2013, Catalin Marinas wrote:> On Fri, May 31, 2013 at 01:02:04PM +0100, Stefano Stabellini wrote:
> > On Fri, 31 May 2013, Catalin Marinas wrote:
> > > On Thu, May 30, 2013 at 05:18:28PM +0100, Stefano Stabellini
wrote:
> > > > Most of Xen support for ARM is common between ARMv7 and
ARMv8.
> > > > Create links to the code under arch/arm (bleah).
> > > > 
> > > > Other, probably better alternatives:
> > > > 
> > > > - move the code to a different location, maybe the header
files to
> > > > include/xen/arm and the code to drivers/xen/arm (still
pretty ugly)?
> > > > 
> > > > - create a copy of the code to arch/arm64 (even worse);
> > > 
> > > KVM handles this in the Makefile by referencing back to arch/arm
or even
> > > the generic kvm directory. I think that''s the
''cleanest'' ;)
> > 
> > Do you mean creating links in the Makefile or generating header file
> > copies from the Makefile?
> > Or do you mean using something like "-I
arch/arm/include/asm/xen" in the
> > arch/arm64 Makefile?
> 
> I meant C files being compiled directly from arch/arm (no links).
OK.

> For headers, if they are specific to arm or arm64, just copy the header
> in each place (e.g. not using regs->pstate in the arch/arm code with
> #ifdef''s). For the rest, if they cannot be made generic, one way
is to
> have a dummy file including the arm equivalent:
> 
> #include <../../arm/include/asm/xen/events.h>
> 
> Passing -I is dangerous as you actually need "-I
arch/arm/include" which
> could bring other files.
Thanks, that is a good idea, I''ll do that.

Stefano Stabellini

2013-May-31 15:23 UTC

head link

Re: [PATCH RFC 5/6] arm64/xen: implement xen_remap on arm64

On Fri, 31 May 2013, Stefano Stabellini wrote:> On Fri, 31 May 2013, Catalin Marinas wrote:
> > On Fri, May 31, 2013 at 02:21:16PM +0100, Stefano Stabellini wrote:
> > > On Fri, 31 May 2013, Catalin Marinas wrote:
> > > > On Thu, May 30, 2013 at 05:18:32PM +0100, Stefano Stabellini
wrote:
> > > > > --- a/arch/arm/include/asm/xen/page.h
> > > > > +++ b/arch/arm/include/asm/xen/page.h
> > > > > @@ -90,6 +90,10 @@ static inline bool
set_phys_to_machine(unsigned long pfn, unsigned long mfn)
> > > > >  	return __set_phys_to_machine(pfn, mfn);
> > > > >  }
> > > > >  
> > > > > +#ifdef CONFIG_ARM64
> > > > > +#define xen_remap(cookie, size) __ioremap((cookie),
(size), __pgprot(PROT_NORMAL))
> > > > > +#else
> > > > >  #define xen_remap(cookie, size)
__arm_ioremap((cookie), (size), MT_MEMORY);
> > > > > +#endif
> > > > 
> > > > Now I saw the ARM-specific part. Can you not use something
like
> > > > ioremap_cached() which would give normal cacheable memory
(at least on
> > > > ARMv7).
> > > 
> > > No, I cannot because ioremap_cached uses MT_DEVICE_CACHED, while
this
> > > needs to be MT_MEMORY. It is used for normal memory pages, not
device
> > > memory.
> > 
> > MT_DEVICE_CACHED is Normal memory for ARMv7.
> 
> I didn''t realize that MT_DEVICE_CACHED and MT_MEMORY end up having
the
> same AttrIndx encoding!
> In that case yes, I should be able to use it.
> I take that I should just implement ioremap_cached on arm64 as well?
> 
Should just use MT_MEMORY and call it PROT_NORMAL?

#define PROT_NORMAL     (PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL))
#define ioremap_cached(addr, size)     __ioremap((addr), (size),
__pgprot(PROT_NORMAL))

Catalin Marinas

2013-May-31 15:56 UTC

head link

Re: [PATCH RFC 5/6] arm64/xen: implement xen_remap on arm64

On Fri, May 31, 2013 at 04:23:20PM +0100, Stefano Stabellini
wrote:> On Fri, 31 May 2013, Stefano Stabellini wrote:
> > On Fri, 31 May 2013, Catalin Marinas wrote:
> > > On Fri, May 31, 2013 at 02:21:16PM +0100, Stefano Stabellini
wrote:
> > > > On Fri, 31 May 2013, Catalin Marinas wrote:
> > > > > On Thu, May 30, 2013 at 05:18:32PM +0100, Stefano
Stabellini wrote:
> > > > > > --- a/arch/arm/include/asm/xen/page.h
> > > > > > +++ b/arch/arm/include/asm/xen/page.h
> > > > > > @@ -90,6 +90,10 @@ static inline bool
set_phys_to_machine(unsigned long pfn, unsigned long mfn)
> > > > > >  	return __set_phys_to_machine(pfn, mfn);
> > > > > >  }
> > > > > >  
> > > > > > +#ifdef CONFIG_ARM64
> > > > > > +#define xen_remap(cookie, size)
__ioremap((cookie), (size), __pgprot(PROT_NORMAL))
> > > > > > +#else
> > > > > >  #define xen_remap(cookie, size)
__arm_ioremap((cookie), (size), MT_MEMORY);
> > > > > > +#endif
> > > > > 
> > > > > Now I saw the ARM-specific part. Can you not use
something like
> > > > > ioremap_cached() which would give normal cacheable
memory (at least on
> > > > > ARMv7).
> > > > 
> > > > No, I cannot because ioremap_cached uses MT_DEVICE_CACHED,
while this
> > > > needs to be MT_MEMORY. It is used for normal memory pages,
not device
> > > > memory.
> > > 
> > > MT_DEVICE_CACHED is Normal memory for ARMv7.
> > 
> > I didn''t realize that MT_DEVICE_CACHED and MT_MEMORY end up
having the
> > same AttrIndx encoding!
> > In that case yes, I should be able to use it.
> > I take that I should just implement ioremap_cached on arm64 as well?
> > 
> 
> Should just use MT_MEMORY and call it PROT_NORMAL?
> 
> #define PROT_NORMAL     (PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL))
> #define ioremap_cached(addr, size)     __ioremap((addr), (size),
__pgprot(PROT_NORMAL))
Looks fine.

-- 
Catalin

Xen devel - May 2013 - [PATCH RFC 0/6] Introduce Xen support to ARM64

[PATCH RFC 0/6] Introduce Xen support to ARM64

[PATCH RFC 1/6] [HACK!] arm64/xen: create links to arch/arm include files and Xen code

[PATCH RFC 2/6] arm64/xen: arm/xen header changes to compile on arm64

[PATCH RFC 3/6] arm64/xen: use XEN_IO_PROTO_ABI_ARM on ARM64

[PATCH RFC 4/6] arm64/xen: introduce asm/hypervisor.h and sys_bitops.h

[PATCH RFC 5/6] arm64/xen: implement xen_remap on arm64

[PATCH RFC 6/6] arm64/xen: introduce CONFIG_XEN and hypercall.S on ARM64

Re: [PATCH RFC 1/6] [HACK!] arm64/xen: create links to arch/arm include files and Xen code

Re: [PATCH RFC 2/6] arm64/xen: arm/xen header changes to compile on arm64

Re: [PATCH RFC 5/6] arm64/xen: implement xen_remap on arm64

Re: [PATCH RFC 6/6] arm64/xen: introduce CONFIG_XEN and hypercall.S on ARM64

Re: [PATCH RFC 1/6] [HACK!] arm64/xen: create links to arch/arm include files and Xen code

Re: [PATCH RFC 1/6] [HACK!] arm64/xen: create links to arch/arm include files and Xen code

Re: [PATCH RFC 5/6] arm64/xen: implement xen_remap on arm64

Re: [PATCH RFC 2/6] arm64/xen: arm/xen header changes to compile on arm64

Re: [PATCH RFC 5/6] arm64/xen: implement xen_remap on arm64

Re: [PATCH RFC 5/6] arm64/xen: implement xen_remap on arm64

Re: [PATCH RFC 1/6] [HACK!] arm64/xen: create links to arch/arm include files and Xen code

Re: [PATCH RFC 5/6] arm64/xen: implement xen_remap on arm64

Re: [PATCH RFC 5/6] arm64/xen: implement xen_remap on arm64