Being in the process of trying to enable sysenter/syscall use from compat mode guests and compat mode apps in native guests (for performance to a certain degree, but more importantly - at least for the compat mode app case - to get closer to native behavior, i.e. mid/long term requiring less kernel modifications) I''m facing a few backwards compatibility issues that I''m not really sure how to deal with: 1) Obviously I need to extend the guest_context structure (to store the additional callback addresses), but there do not seem to be provisions to do so without breaking the dom0 interface. I''m currently considering adding a flag indicating use of the larger structure, but this certainly doesn''t scale well considering future additions. An alternative might be to add a single flag covering all future additions, and using the first field past the current size to store the overall or add-on size, so that the hypervisor has a way to know how much of the structure to copy. 2) While the x86-32 hv can''t support syscall and is unlikely to support sysenter, save/restore/migration (which hopefully will work at least from 32-bit hv to 64-bit hv in the future) imposes an issue here in that native wouldn''t need these extra fields, but a compat mode guest would have to have a way to store them (in compat mode format), implying that guest_context would also need to be extended for 32-bits. 3) Currently, the 32-bit kernel check X86_FEATURE_SEP and the return status of setting the sysenter hypercall, to detect its availability when run in supervisor mode. This, however, is being done only on the boot CPU, which works thanks to a quirk in how Xen handles the hypercall - MSRs for all CPUs get set by this single call, which clearly doesn''t match native behavior (where in theory all CPUs could have distinct settings and have to establish them as they come up). The problem with this is that if things turn out to work as intended, X86_FEATURE_SEP will be seen enabled by the guest when run on a 64-bit hv, and setting the sysenter callback will also succeed, but existing guests will fail to set the callback on all CPUs. I have to admit that I''m rather reluctant to add the same kind of quirk for !supervisor_mode_kernel in the hypervisor''s handling of CALLBACK_sysenter. Thanks for suggestions/opinions, Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
(1) and (2): I want to kill off use of vcpu_guest_context in dom0 tools, make hvm save/restore a generic state load/save interface, and define extensible structures at that interface (pass a stream of state chunks back and forth at the interface, each chunk having a size in its header, and so increasing the size of a chunk allows it to be naturally appended to and hence extended). (3) Bump the XEN_INTERFACE_VERSION, rename current CALLBACKTYPE_sysenter to a compat name, and introduce new define for CALLBACKTYPE_sysenter with proper native semantics. -- Keir On 21/6/07 09:10, "Jan Beulich" <jbeulich@novell.com> wrote:> Being in the process of trying to enable sysenter/syscall use from compat mode > guests and compat mode apps in native guests (for performance to a certain > degree, but more importantly - at least for the compat mode app case - to get > closer to native behavior, i.e. mid/long term requiring less kernel > modifications) > I''m facing a few backwards compatibility issues that I''m not really sure how > to > deal with: > > 1) Obviously I need to extend the guest_context structure (to store the > additional callback addresses), but there do not seem to be provisions to do > so without breaking the dom0 interface. I''m currently considering adding a > flag indicating use of the larger structure, but this certainly doesn''t scale > well > considering future additions. An alternative might be to add a single flag > covering all future additions, and using the first field past the current size > to > store the overall or add-on size, so that the hypervisor has a way to know > how much of the structure to copy. > > 2) While the x86-32 hv can''t support syscall and is unlikely to support > sysenter, > save/restore/migration (which hopefully will work at least from 32-bit hv to > 64-bit hv in the future) imposes an issue here in that native wouldn''t need > these extra fields, but a compat mode guest would have to have a way to > store them (in compat mode format), implying that guest_context would also > need to be extended for 32-bits. > > 3) Currently, the 32-bit kernel check X86_FEATURE_SEP and the return > status of setting the sysenter hypercall, to detect its availability when run > in supervisor mode. This, however, is being done only on the boot CPU, > which works thanks to a quirk in how Xen handles the hypercall - MSRs for > all CPUs get set by this single call, which clearly doesn''t match native > behavior (where in theory all CPUs could have distinct settings and have > to establish them as they come up). The problem with this is that if things > turn out to work as intended, X86_FEATURE_SEP will be seen enabled by > the guest when run on a 64-bit hv, and setting the sysenter callback will > also succeed, but existing guests will fail to set the callback on all CPUs. > I have to admit that I''m rather reluctant to add the same kind of quirk for > !supervisor_mode_kernel in the hypervisor''s handling of CALLBACK_sysenter. > > Thanks for suggestions/opinions, > Jan > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>>> Keir Fraser <keir@xensource.com> 21.06.07 10:37 >>> >(1) and (2): I want to kill off use of vcpu_guest_context in dom0 tools, >make hvm save/restore a generic state load/save interface, and define >extensible structures at that interface (pass a stream of state chunks back >and forth at the interface, each chunk having a size in its header, and so >increasing the size of a chunk allows it to be naturally appended to and >hence extended).I wasn''t concerned about the tools interface. The real compatibility problem is VCPUOP_initialize. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 21/6/07 10:53, "Jan Beulich" <jbeulich@novell.com> wrote:>> (1) and (2): I want to kill off use of vcpu_guest_context in dom0 tools, >> make hvm save/restore a generic state load/save interface, and define >> extensible structures at that interface (pass a stream of state chunks back >> and forth at the interface, each chunk having a size in its header, and so >> increasing the size of a chunk allows it to be naturally appended to and >> hence extended). > > I wasn''t concerned about the tools interface. The real compatibility problem > is VCPUOP_initialize.I don''t see the problem. The guest will not be able to initialise secondary VCPU''s sysenter/syscall state via this interface. So what? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>>> Keir Fraser <keir@xensource.com> 21.06.07 11:58 >>> >On 21/6/07 10:53, "Jan Beulich" <jbeulich@novell.com> wrote: > >>> (1) and (2): I want to kill off use of vcpu_guest_context in dom0 tools, >>> make hvm save/restore a generic state load/save interface, and define >>> extensible structures at that interface (pass a stream of state chunks back >>> and forth at the interface, each chunk having a size in its header, and so >>> increasing the size of a chunk allows it to be naturally appended to and >>> hence extended). >> >> I wasn''t concerned about the tools interface. The real compatibility problem >> is VCPUOP_initialize. > >I don''t see the problem. The guest will not be able to initialise secondary >VCPU''s sysenter/syscall state via this interface. So what?For (1), the guest will supply a too short guest_context structure, and currently Xen has no way of detecting this. I was proposing two possible solutions, neither of which seemed ideal to me. For (2), I am just not certain whether there isn''t an alternative not breaking the interface for pure 32-bits. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 21/6/07 11:16, "Jan Beulich" <jbeulich@novell.com> wrote:>> I don''t see the problem. The guest will not be able to initialise secondary >> VCPU''s sysenter/syscall state via this interface. So what? > > For (1), the guest will supply a too short guest_context structure, and > currently Xen has no way of detecting this. I was proposing two possible > solutions, neither of which seemed ideal to me. > > For (2), I am just not certain whether there isn''t an alternative not breaking > the interface for pure 32-bits.I''m proposing we do not change the guest_context structure at all. Instead the tools will get at this extra state via a generalised form of the current hvm save/restore interface. The guest itself will always have to set the callback via the callback_op hypercall. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel