Xen devel - Jul 2013 - [PATCH] xen: arm: document which hypercalls (and subops) are supported on ARM

From: Ian Campbell <ian.campbell@citrix.com>

There are many hypercalls which make no sense or which are not supported on ARM
systems but it''s not all that obvious which ones we do support. So lets
try and
document the hypercalls which are useful on ARM.

I''m not sure this is the best way to go about this, I''m open
to other ideas.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 xen/include/public/arch-arm.h | 66 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 64 insertions(+), 2 deletions(-)

diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 8aa62d3..b4a20bb 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -27,8 +27,8 @@
 #ifndef __XEN_PUBLIC_ARCH_ARM_H__
 #define __XEN_PUBLIC_ARCH_ARM_H__
 
-/* hypercall calling convention
- * ----------------------------
+/*
+ * `incontents 50 arm_abi Hypercall Calling Convention
  *
  * A hypercall is issued using the ARM HVC instruction.
  *
@@ -60,6 +60,68 @@
  * hypercall arguments are always little endian.
  */
 
+/*
+ * `incontents 55 arm_hcall Supported Hypercalls
+ *
+ * Xen on ARM makes extensive use of hardware facilities and therefore
+ * only a subset of the potential hypercalls are required.
+ *
+ * Since ARM uses second stage paging any machine/physical addresses
+ * passed to hypercalls are Guest Physical Addresses (Intermediate
+ * Physical Addresses) unless otherwise noted.
+ *
+ * The following hypercalls (and sub operations) are supported on the
+ * ARM platform. Other hypercalls should be considered
+ * unavailable/unsupported.
+ *
+ *  HYPERVISOR_memory_op
+ *   All generic sub-operations.
+ *
+ *   In addition the following arch specific sub-ops:
+ *    * XENMEM_add_to_physmap
+ *    * XENMEM_add_to_physmap_range
+ *
+ *  HYPERVISOR_domctl
+ *   All generic sub-operations, with the exception of:
+ *    * XEN_DOMCTL_iomem_permission (not yet implemented)
+ *    * XEN_DOMCTL_irq_permission (not yet implemented)
+ *
+ *  HYPERVISOR_sched_op
+ *   All generic sub-operations, with the exception of:
+ *    * SCHEDOP_block -- prefer wfi hardware instruction
+ *
+ *  HYPERVISOR_console_io
+ *   All generic sub-operations
+ *
+ *  HYPERVISOR_xen_version
+ *   All generic sub-operations
+ *
+ *  HYPERVISOR_event_channel_op
+ *   All generic sub-operations
+ *
+ *  HYPERVISOR_physdev_op
+ *   No sub-operations are currenty supported
+ *
+ *  HYPERVISOR_sysctl
+ *   All generic sub-operations, with the exception of:
+ *    * XEN_SYSCTL_page_offline_op
+ *    * XEN_SYSCTL_get_pmstat
+ *    * XEN_SYSCTL_pm_op
+ *
+ *  HYPERVISOR_hvm_op
+ *   Exactly these sub-operations are supported:
+ *    * HVMOP_set_param
+ *    * HVMOP_get_param
+ *
+ *  HYPERVISOR_grant_table_op
+ *   All generic sub-operations
+ *
+ *  HYPERVISOR_vcpu_op
+ *   Exactly these sub-operations are supported:
+ *    * VCPUOP_register_vcpu_info
+ *    * VCPUOP_register_runstate_memory_area
+ */
+
 #define XEN_HYPERCALL_TAG   0XEA1
 
 #define uint64_aligned_t uint64_t __attribute__((aligned(8)))
-- 
1.8.3.2

On Mon, 22 Jul 2013, Ian Campbell wrote:
> From: Ian Campbell <ian.campbell@citrix.com>
> 
> There are many hypercalls which make no sense or which are not supported on
ARM
> systems but it''s not all that obvious which ones we do support. So
lets try and
> document the hypercalls which are useful on ARM.
> 
> I''m not sure this is the best way to go about this, I''m
open to other ideas.
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> ---
>  xen/include/public/arch-arm.h | 66
+++++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 64 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 8aa62d3..b4a20bb 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -27,8 +27,8 @@
>  #ifndef __XEN_PUBLIC_ARCH_ARM_H__
>  #define __XEN_PUBLIC_ARCH_ARM_H__
>  
> -/* hypercall calling convention
> - * ----------------------------
> +/*
> + * `incontents 50 arm_abi Hypercall Calling Convention
>   *
>   * A hypercall is issued using the ARM HVC instruction.
>   *
> @@ -60,6 +60,68 @@
>   * hypercall arguments are always little endian.
>   */
>  
> +/*
> + * `incontents 55 arm_hcall Supported Hypercalls
> + *
> + * Xen on ARM makes extensive use of hardware facilities and therefore
> + * only a subset of the potential hypercalls are required.
> + *
> + * Since ARM uses second stage paging any machine/physical addresses
> + * passed to hypercalls are Guest Physical Addresses (Intermediate
> + * Physical Addresses) unless otherwise noted.
> + *
> + * The following hypercalls (and sub operations) are supported on the
> + * ARM platform. Other hypercalls should be considered
> + * unavailable/unsupported.
> + *
> + *  HYPERVISOR_memory_op
> + *   All generic sub-operations.
> + *
> + *   In addition the following arch specific sub-ops:
> + *    * XENMEM_add_to_physmap
> + *    * XENMEM_add_to_physmap_range
> + *
> + *  HYPERVISOR_domctl
> + *   All generic sub-operations, with the exception of:
> + *    * XEN_DOMCTL_iomem_permission (not yet implemented)
> + *    * XEN_DOMCTL_irq_permission (not yet implemented)
> + *
> + *  HYPERVISOR_sched_op
> + *   All generic sub-operations, with the exception of:
> + *    * SCHEDOP_block -- prefer wfi hardware instruction
> + *
> + *  HYPERVISOR_console_io
> + *   All generic sub-operations
> + *
> + *  HYPERVISOR_xen_version
> + *   All generic sub-operations
> + *
> + *  HYPERVISOR_event_channel_op
> + *   All generic sub-operations
> + *
> + *  HYPERVISOR_physdev_op
> + *   No sub-operations are currenty supported
> + *
> + *  HYPERVISOR_sysctl
> + *   All generic sub-operations, with the exception of:
> + *    * XEN_SYSCTL_page_offline_op
> + *    * XEN_SYSCTL_get_pmstat
> + *    * XEN_SYSCTL_pm_op
> + *
> + *  HYPERVISOR_hvm_op
> + *   Exactly these sub-operations are supported:
> + *    * HVMOP_set_param
> + *    * HVMOP_get_param
> + *
> + *  HYPERVISOR_grant_table_op
> + *   All generic sub-operations
> + *
> + *  HYPERVISOR_vcpu_op
> + *   Exactly these sub-operations are supported:
> + *    * VCPUOP_register_vcpu_info
> + *    * VCPUOP_register_runstate_memory_area
> + */
> +
You are missing all the common tmem ops.

On Tue, 2013-07-23 at 12:33 +0100, Stefano Stabellini wrote:
> You are missing all the common tmem ops.
tmem_ops are not hooked up in arm_hypercall_table, so they are currently
not supported. We should add them here when we add the necessary
support. ISTR stubbing out some functions which are used by tmem
(steal_page and donate_page perhaps?) so it''s not just a case or wiring
it up either.

Ian.

On Tue, 23 Jul 2013, Ian Campbell wrote:
> On Tue, 2013-07-23 at 12:33 +0100, Stefano Stabellini wrote:
> 
> > You are missing all the common tmem ops.
> 
> tmem_ops are not hooked up in arm_hypercall_table, so they are currently
> not supported. We should add them here when we add the necessary
> support. ISTR stubbing out some functions which are used by tmem
> (steal_page and donate_page perhaps?) so it''s not just a case or
wiring
> it up either.
Right.. I was confused by the fact that we do compile tmem_op in Linux,
even though XEN_TMEM is actually not available on ARM or ARM64

On Tue, Jul 23, 2013 at 04:34:18PM +0100, Stefano Stabellini
wrote:
> On Tue, 23 Jul 2013, Ian Campbell wrote:
> > On Tue, 2013-07-23 at 12:33 +0100, Stefano Stabellini wrote:
> > 
> > > You are missing all the common tmem ops.
> > 
> > tmem_ops are not hooked up in arm_hypercall_table, so they are
currently
> > not supported. We should add them here when we add the necessary
> > support. ISTR stubbing out some functions which are used by tmem
> > (steal_page and donate_page perhaps?) so it''s not just a case
or wiring
> > it up either.
> 
> Right.. I was confused by the fact that we do compile tmem_op in Linux,
> even though XEN_TMEM is actually not available on ARM or ARM64
It is an attractive option of using TMEM on those platforms - but I
hadn''t had
a chance yet to get my Chromebook setup to work with Xen :-(

(It also did not help that I stepped on it and crushed the LCD).

On Tue, 2013-07-23 at 16:34 +0100, Stefano Stabellini
wrote:
> On Tue, 23 Jul 2013, Ian Campbell wrote:
> > On Tue, 2013-07-23 at 12:33 +0100, Stefano Stabellini wrote:
> > 
> > > You are missing all the common tmem ops.
> > 
> > tmem_ops are not hooked up in arm_hypercall_table, so they are
currently
> > not supported. We should add them here when we add the necessary
> > support. ISTR stubbing out some functions which are used by tmem
> > (steal_page and donate_page perhaps?) so it''s not just a case
or wiring
> > it up either.
> 
> Right.. I was confused by the fact that we do compile tmem_op in Linux,
> even though XEN_TMEM is actually not available on ARM or ARM64
I started from the other end and looked at what the hypervisor actually
implements rather than what Linux calls. It''s useful to validate from
both angles though. For now I think we should stick with documenting
what the hypervisor provides even if the kernel might try and
optimistically call other stuff too.

Ian.

On Tue, 2013-07-23 at 17:59 +0100, Ian Campbell wrote:
> On Tue, 2013-07-23 at 16:34 +0100, Stefano Stabellini wrote:
> > On Tue, 23 Jul 2013, Ian Campbell wrote:
> > > On Tue, 2013-07-23 at 12:33 +0100, Stefano Stabellini wrote:
> > > 
> > > > You are missing all the common tmem ops.
> > > 
> > > tmem_ops are not hooked up in arm_hypercall_table, so they are
currently
> > > not supported. We should add them here when we add the necessary
> > > support. ISTR stubbing out some functions which are used by tmem
> > > (steal_page and donate_page perhaps?) so it''s not just a
case or wiring
> > > it up either.
> > 
> > Right.. I was confused by the fact that we do compile tmem_op in
Linux,
> > even though XEN_TMEM is actually not available on ARM or ARM64
> 
> I started from the other end and looked at what the hypervisor actually
> implements rather than what Linux calls. It''s useful to validate
from
> both angles though. For now I think we should stick with documenting
> what the hypervisor provides even if the kernel might try and
> optimistically call other stuff too.
So did we decide whether to run with this patch or not?

Ian.

On Mon, Jul 29, 2013 at 03:56:51PM +0100, Ian Campbell
wrote:
> On Tue, 2013-07-23 at 17:59 +0100, Ian Campbell wrote:
> > On Tue, 2013-07-23 at 16:34 +0100, Stefano Stabellini wrote:
> > > On Tue, 23 Jul 2013, Ian Campbell wrote:
> > > > On Tue, 2013-07-23 at 12:33 +0100, Stefano Stabellini wrote:
> > > > 
> > > > > You are missing all the common tmem ops.
> > > > 
> > > > tmem_ops are not hooked up in arm_hypercall_table, so they
are currently
> > > > not supported. We should add them here when we add the
necessary
> > > > support. ISTR stubbing out some functions which are used by
tmem
> > > > (steal_page and donate_page perhaps?) so it''s not
just a case or wiring
> > > > it up either.
> > > 
> > > Right.. I was confused by the fact that we do compile tmem_op in
Linux,
> > > even though XEN_TMEM is actually not available on ARM or ARM64
> > 
> > I started from the other end and looked at what the hypervisor
actually
> > implements rather than what Linux calls. It''s useful to
validate from
> > both angles though. For now I think we should stick with documenting
> > what the hypervisor provides even if the kernel might try and
> > optimistically call other stuff too.
> 
> So did we decide whether to run with this patch or not?
I would say a better option is to just disable by default tmem in Xen hypervisor
for ARM. So even if you boot with ''tmem=1'' nothing will
happen.

That way when I get an ARM box up and I can test tmem we can just revert that
simple patch without much trouble.

On Mon, 2013-07-29 at 14:09 -0400, Konrad Rzeszutek Wilk
wrote:
> On Mon, Jul 29, 2013 at 03:56:51PM +0100, Ian Campbell wrote:
> > On Tue, 2013-07-23 at 17:59 +0100, Ian Campbell wrote:
> > > On Tue, 2013-07-23 at 16:34 +0100, Stefano Stabellini wrote:
> > > > On Tue, 23 Jul 2013, Ian Campbell wrote:
> > > > > On Tue, 2013-07-23 at 12:33 +0100, Stefano Stabellini
wrote:
> > > > > 
> > > > > > You are missing all the common tmem ops.
> > > > > 
> > > > > tmem_ops are not hooked up in arm_hypercall_table, so
they are currently
> > > > > not supported. We should add them here when we add the
necessary
> > > > > support. ISTR stubbing out some functions which are
used by tmem
> > > > > (steal_page and donate_page perhaps?) so it''s
not just a case or wiring
> > > > > it up either.
> > > > 
> > > > Right.. I was confused by the fact that we do compile
tmem_op in Linux,
> > > > even though XEN_TMEM is actually not available on ARM or
ARM64
> > > 
> > > I started from the other end and looked at what the hypervisor
actually
> > > implements rather than what Linux calls. It''s useful to
validate from
> > > both angles though. For now I think we should stick with
documenting
> > > what the hypervisor provides even if the kernel might try and
> > > optimistically call other stuff too.
> > 
> > So did we decide whether to run with this patch or not?
> 
> I would say a better option is to just disable by default tmem in Xen
hypervisor
> for ARM. So even if you boot with ''tmem=1'' nothing will
happen.
> 
> That way when I get an ARM box up and I can test tmem we can just revert
that
> simple patch without much trouble.
The "this patch" I was talking about was the original documentation
only
patch at the root of this thread which has nothing to do with tmem.

What we want to do with tmem on ARM is a different question. If you want
to start a separate thread on that topic please feel free.

Ian.

ping?

There was some discussion but I think the conclusion was to go with this
version, but no acks yet.

On Mon, 2013-07-22 at 19:17 +0100, Ian Campbell wrote:
> From: Ian Campbell <ian.campbell@citrix.com>
> 
> There are many hypercalls which make no sense or which are not supported on
ARM
> systems but it''s not all that obvious which ones we do support. So
lets try and
> document the hypercalls which are useful on ARM.
> 
> I''m not sure this is the best way to go about this, I''m
open to other ideas.
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> ---
>  xen/include/public/arch-arm.h | 66
+++++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 64 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index 8aa62d3..b4a20bb 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -27,8 +27,8 @@
>  #ifndef __XEN_PUBLIC_ARCH_ARM_H__
>  #define __XEN_PUBLIC_ARCH_ARM_H__
>  
> -/* hypercall calling convention
> - * ----------------------------
> +/*
> + * `incontents 50 arm_abi Hypercall Calling Convention
>   *
>   * A hypercall is issued using the ARM HVC instruction.
>   *
> @@ -60,6 +60,68 @@
>   * hypercall arguments are always little endian.
>   */
>  
> +/*
> + * `incontents 55 arm_hcall Supported Hypercalls
> + *
> + * Xen on ARM makes extensive use of hardware facilities and therefore
> + * only a subset of the potential hypercalls are required.
> + *
> + * Since ARM uses second stage paging any machine/physical addresses
> + * passed to hypercalls are Guest Physical Addresses (Intermediate
> + * Physical Addresses) unless otherwise noted.
> + *
> + * The following hypercalls (and sub operations) are supported on the
> + * ARM platform. Other hypercalls should be considered
> + * unavailable/unsupported.
> + *
> + *  HYPERVISOR_memory_op
> + *   All generic sub-operations.
> + *
> + *   In addition the following arch specific sub-ops:
> + *    * XENMEM_add_to_physmap
> + *    * XENMEM_add_to_physmap_range
> + *
> + *  HYPERVISOR_domctl
> + *   All generic sub-operations, with the exception of:
> + *    * XEN_DOMCTL_iomem_permission (not yet implemented)
> + *    * XEN_DOMCTL_irq_permission (not yet implemented)
> + *
> + *  HYPERVISOR_sched_op
> + *   All generic sub-operations, with the exception of:
> + *    * SCHEDOP_block -- prefer wfi hardware instruction
> + *
> + *  HYPERVISOR_console_io
> + *   All generic sub-operations
> + *
> + *  HYPERVISOR_xen_version
> + *   All generic sub-operations
> + *
> + *  HYPERVISOR_event_channel_op
> + *   All generic sub-operations
> + *
> + *  HYPERVISOR_physdev_op
> + *   No sub-operations are currenty supported
> + *
> + *  HYPERVISOR_sysctl
> + *   All generic sub-operations, with the exception of:
> + *    * XEN_SYSCTL_page_offline_op
> + *    * XEN_SYSCTL_get_pmstat
> + *    * XEN_SYSCTL_pm_op
> + *
> + *  HYPERVISOR_hvm_op
> + *   Exactly these sub-operations are supported:
> + *    * HVMOP_set_param
> + *    * HVMOP_get_param
> + *
> + *  HYPERVISOR_grant_table_op
> + *   All generic sub-operations
> + *
> + *  HYPERVISOR_vcpu_op
> + *   Exactly these sub-operations are supported:
> + *    * VCPUOP_register_vcpu_info
> + *    * VCPUOP_register_runstate_memory_area
> + */
> +
>  #define XEN_HYPERCALL_TAG   0XEA1
>  
>  #define uint64_aligned_t uint64_t __attribute__((aligned(8)))

At 11:59 +0100 on 08 Aug (1375963190), Ian Campbell
wrote:
> ping?
Acked-by: Tim Deegan <tim@xen.org>?
> There was some discussion but I think the conclusion was to go with this
> version, but no acks yet.
> 
> On Mon, 2013-07-22 at 19:17 +0100, Ian Campbell wrote:
> > From: Ian Campbell <ian.campbell@citrix.com>
> > 
> > There are many hypercalls which make no sense or which are not
supported on ARM
> > systems but it''s not all that obvious which ones we do
support. So lets try and
> > document the hypercalls which are useful on ARM.
> > 
> > I''m not sure this is the best way to go about this,
I''m open to other ideas.
> > 
> > Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> > ---
> >  xen/include/public/arch-arm.h | 66
+++++++++++++++++++++++++++++++++++++++++--
> >  1 file changed, 64 insertions(+), 2 deletions(-)
> > 
> > diff --git a/xen/include/public/arch-arm.h
b/xen/include/public/arch-arm.h
> > index 8aa62d3..b4a20bb 100644
> > --- a/xen/include/public/arch-arm.h
> > +++ b/xen/include/public/arch-arm.h
> > @@ -27,8 +27,8 @@
> >  #ifndef __XEN_PUBLIC_ARCH_ARM_H__
> >  #define __XEN_PUBLIC_ARCH_ARM_H__
> >  
> > -/* hypercall calling convention
> > - * ----------------------------
> > +/*
> > + * `incontents 50 arm_abi Hypercall Calling Convention
> >   *
> >   * A hypercall is issued using the ARM HVC instruction.
> >   *
> > @@ -60,6 +60,68 @@
> >   * hypercall arguments are always little endian.
> >   */
> >  
> > +/*
> > + * `incontents 55 arm_hcall Supported Hypercalls
> > + *
> > + * Xen on ARM makes extensive use of hardware facilities and
therefore
> > + * only a subset of the potential hypercalls are required.
> > + *
> > + * Since ARM uses second stage paging any machine/physical addresses
> > + * passed to hypercalls are Guest Physical Addresses (Intermediate
> > + * Physical Addresses) unless otherwise noted.
> > + *
> > + * The following hypercalls (and sub operations) are supported on the
> > + * ARM platform. Other hypercalls should be considered
> > + * unavailable/unsupported.
> > + *
> > + *  HYPERVISOR_memory_op
> > + *   All generic sub-operations.
> > + *
> > + *   In addition the following arch specific sub-ops:
> > + *    * XENMEM_add_to_physmap
> > + *    * XENMEM_add_to_physmap_range
> > + *
> > + *  HYPERVISOR_domctl
> > + *   All generic sub-operations, with the exception of:
> > + *    * XEN_DOMCTL_iomem_permission (not yet implemented)
> > + *    * XEN_DOMCTL_irq_permission (not yet implemented)
> > + *
> > + *  HYPERVISOR_sched_op
> > + *   All generic sub-operations, with the exception of:
> > + *    * SCHEDOP_block -- prefer wfi hardware instruction
> > + *
> > + *  HYPERVISOR_console_io
> > + *   All generic sub-operations
> > + *
> > + *  HYPERVISOR_xen_version
> > + *   All generic sub-operations
> > + *
> > + *  HYPERVISOR_event_channel_op
> > + *   All generic sub-operations
> > + *
> > + *  HYPERVISOR_physdev_op
> > + *   No sub-operations are currenty supported
> > + *
> > + *  HYPERVISOR_sysctl
> > + *   All generic sub-operations, with the exception of:
> > + *    * XEN_SYSCTL_page_offline_op
> > + *    * XEN_SYSCTL_get_pmstat
> > + *    * XEN_SYSCTL_pm_op
> > + *
> > + *  HYPERVISOR_hvm_op
> > + *   Exactly these sub-operations are supported:
> > + *    * HVMOP_set_param
> > + *    * HVMOP_get_param
> > + *
> > + *  HYPERVISOR_grant_table_op
> > + *   All generic sub-operations
> > + *
> > + *  HYPERVISOR_vcpu_op
> > + *   Exactly these sub-operations are supported:
> > + *    * VCPUOP_register_vcpu_info
> > + *    * VCPUOP_register_runstate_memory_area
> > + */
> > +
> >  #define XEN_HYPERCALL_TAG   0XEA1
> >  
> >  #define uint64_aligned_t uint64_t __attribute__((aligned(8)))
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

On Thu, 2013-08-08 at 15:05 +0100, Tim Deegan wrote:
> At 11:59 +0100 on 08 Aug (1375963190), Ian Campbell wrote:
> > ping?
> 
> Acked-by: Tim Deegan <tim@xen.org>?
Thanks?

I''ve gone ahead and applied.

Ian.