Hi, Booting latest upstream pv linux as dom0, I noticed dm_mapper, user level process, is running with XEN_EMULATE_PREFIX cpuid. I am wondering if the prefix is statically put there, or how its put there? For hybrid, the hvm container traps cpuid, so we don''t need to prefix cpuid. Less code if I don''t have to worry about trapping the invalid op. There seems places where the cpuid is run without the xen prefix in user mode. BTW, for cpuid vmexit, if kernel mode I call pv_cpuid, if user mode, I just return cpuid values, as that''s what would happen without the HVM container. thanks, Mukesh
Konrad Rzeszutek Wilk
2012-Jun-29 01:09 UTC
Re: [HYBRID]: XEN_EMULATE_PREFIX in user process
On Thu, Jun 28, 2012 at 06:00:07PM -0700, Mukesh Rathor wrote:> Hi, > > Booting latest upstream pv linux as dom0, I noticed dm_mapper, user > level process, is running with XEN_EMULATE_PREFIX cpuid. I am > wondering if the prefix is statically put there, or how its put there?That is impressive. It should be only be in the kernel via the pvops cpuid call which does this: 252 asm(XEN_EMULATE_PREFIX "cpuid" 253 : "=a" (*ax), 254 "=b" (*bx), 255 "=c" (*cx), 256 "=d" (*dx) 257 : "0" (*ax), "2" (*cx)); 258> > For hybrid, the hvm container traps cpuid, so we don''t need to prefix > cpuid. Less code if I don''t have to worry about trapping the invalid > op. There seems places where the cpuid is run without the xen prefix in > user mode.Right, you can make the .cpuid pvops call point to the native.> > BTW, for cpuid vmexit, if kernel mode I call pv_cpuid, if user mode, I > just return cpuid values, as that''s what would happen without the HVM > container.You mean in PV mode in user-space you would get the unfiltered cpuid value? Yes, that is true. Which is why I am surprised that dm_mapper (are you sure it is not a kernel thread?) is doing that.> > thanks, > Mukesh > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On Thu, 28 Jun 2012 21:09:03 -0400 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:> On Thu, Jun 28, 2012 at 06:00:07PM -0700, Mukesh Rathor wrote: > > Hi, > > > > Booting latest upstream pv linux as dom0, I noticed dm_mapper, user > > level process, is running with XEN_EMULATE_PREFIX cpuid. I am > > wondering if the prefix is statically put there, or how its put > > there? > > That is impressive. It should be only be in the kernel via the pvops > cpuid call which does this: > > 252 asm(XEN_EMULATE_PREFIX "cpuid" > 253 : "=a" (*ax), > 254 "=b" (*bx), > 255 "=c" (*cx), > 256 "=d" (*dx) > 257 : "0" (*ax), "2" (*cx)); > 258 > > > > > > For hybrid, the hvm container traps cpuid, so we don''t need to > > prefix cpuid. Less code if I don''t have to worry about trapping the > > invalid op. There seems places where the cpuid is run without the > > xen prefix in user mode. > > Right, you can make the .cpuid pvops call point to the native.Yup. In the kernel I do that already.> > > > BTW, for cpuid vmexit, if kernel mode I call pv_cpuid, if user > > mode, I just return cpuid values, as that''s what would happen > > without the HVM container. > > You mean in PV mode in user-space you would get the unfiltered cpuid > value? Yes, that is true. Which is why I am surprised that dm_mapper > (are you sure it is not a kernel thread?) is doing that.User mode: RIP: 0x0000000000400692 CS: 0x0033 [0]xkdb> dd 0x0000000000400680 32 0 0000000000400680: 89f88953e5894855 db8500000000b9d3 0000000000400690: 0f6e65780b0f0574 4e89045e890689a2 thanks, Mukesh
>>> On 29.06.12 at 03:00, Mukesh Rathor <mukesh.rathor@oracle.com> wrote: > BTW, for cpuid vmexit, if kernel mode I call pv_cpuid, if user mode, I > just return cpuid values, as that''s what would happen without the HVM > container.Which means you''re not leveraging one of the things you could gain from hybrid (after all, returning the native value to user mode is one of the weaknesses of PV). Jan
On Fri, 2012-06-29 at 08:56 +0100, Jan Beulich wrote:> >>> On 29.06.12 at 03:00, Mukesh Rathor <mukesh.rathor@oracle.com> wrote: > > BTW, for cpuid vmexit, if kernel mode I call pv_cpuid, if user mode, I > > just return cpuid values, as that''s what would happen without the HVM > > container. > > Which means you''re not leveraging one of the things you could > gain from hybrid (after all, returning the native value to user > mode is one of the weaknesses of PV).cpuid masking could happen in the HVM layer too, couldn''t it? Probably "XEN_EMULATE_PREFIX cpuid" and raw "cpuid" ought to return the same set of PV values in hybrid? Ian.
On Fri, 2012-06-29 at 02:26 +0100, Mukesh Rathor wrote:> On Thu, 28 Jun 2012 21:09:03 -0400 > Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote: > > > On Thu, Jun 28, 2012 at 06:00:07PM -0700, Mukesh Rathor wrote: > > > Hi, > > > > > > Booting latest upstream pv linux as dom0, I noticed dm_mapper, user > > > level process, is running with XEN_EMULATE_PREFIX cpuid. I am > > > wondering if the prefix is statically put there, or how its put > > > there? > > > > That is impressive. It should be only be in the kernel via the pvops > > cpuid call which does this: > > > > 252 asm(XEN_EMULATE_PREFIX "cpuid" > > 253 : "=a" (*ax), > > 254 "=b" (*bx), > > 255 "=c" (*cx), > > 256 "=d" (*dx) > > 257 : "0" (*ax), "2" (*cx)); > > 258 > > > > > > > > > > For hybrid, the hvm container traps cpuid, so we don''t need to > > > prefix cpuid. Less code if I don''t have to worry about trapping the > > > invalid op. There seems places where the cpuid is run without the > > > xen prefix in user mode. > > > > Right, you can make the .cpuid pvops call point to the native. > > Yup. In the kernel I do that already. > > > > > > > BTW, for cpuid vmexit, if kernel mode I call pv_cpuid, if user > > > mode, I just return cpuid values, as that''s what would happen > > > without the HVM container. > > > > You mean in PV mode in user-space you would get the unfiltered cpuid > > value? Yes, that is true. Which is why I am surprised that dm_mapper > > (are you sure it is not a kernel thread?) is doing that. > > User mode: > > RIP: 0x0000000000400692 CS: 0x0033CS 0x033 could still be kernel mode on regular PV 64 bit, but I suppose not on hybrid? It''s unlikely but not impossible for a userspace developer to have done some "Xen magic" and used the prefix in userspace. What version of dm_mapper do you have? I checked the version in Debian Sid and it doesn''t do this (at least not directly). Are you able to run the dm_mapper process under gdb and inspect it to find the prefix? Ian.> > [0]xkdb> dd 0x0000000000400680 32 0 > 0000000000400680: 89f88953e5894855 db8500000000b9d3 > 0000000000400690: 0f6e65780b0f0574 4e89045e890689a2 > > thanks, > Mukesh > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
>>> On 29.06.12 at 10:07, Ian Campbell <Ian.Campbell@citrix.com> wrote: > On Fri, 2012-06-29 at 08:56 +0100, Jan Beulich wrote: >> >>> On 29.06.12 at 03:00, Mukesh Rathor <mukesh.rathor@oracle.com> wrote: >> > BTW, for cpuid vmexit, if kernel mode I call pv_cpuid, if user mode, I >> > just return cpuid values, as that''s what would happen without the HVM >> > container. >> >> Which means you''re not leveraging one of the things you could >> gain from hybrid (after all, returning the native value to user >> mode is one of the weaknesses of PV). > > cpuid masking could happen in the HVM layer too, couldn''t it?But CPUID masking is rather limited (after all that''s why CPUID faulting got added later).> Probably "XEN_EMULATE_PREFIX cpuid" and raw "cpuid" ought to return the > same set of PV values in hybrid?Yes, that''s what I would think. Jan
On Fri, 29 Jun 2012 09:09:29 +0100 Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Fri, 2012-06-29 at 02:26 +0100, Mukesh Rathor wrote: > > On Thu, 28 Jun 2012 21:09:03 -0400 > > CS 0x033 could still be kernel mode on regular PV 64 bit, but I > suppose not on hybrid?correct, not on hybrid, unless some bug! But then the RIP also appears user mode.> It''s unlikely but not impossible for a userspace developer to have > done some "Xen magic" and used the prefix in userspace. > > What version of dm_mapper do you have? I checked the version in Debian > Sid and it doesn''t do this (at least not directly). > > Are you able to run the dm_mapper process under gdb and inspect it to > find the prefix?Well, let me take that back. dm_mapper is the last printk, but it could be anything after that or that. Unfortunately, it''s dom0 and during boot, so can''t run gdb on it. I am hacking the kernel now to print every user process name in schedule. thanks, Mukesh
On Fri, 29 Jun 2012 11:15:46 -0700 Mukesh Rathor <mukesh.rathor@oracle.com> wrote:> > It''s unlikely but not impossible for a userspace developer to have > > done some "Xen magic" and used the prefix in userspace. > > > > What version of dm_mapper do you have? I checked the version in > > Debian Sid and it doesn''t do this (at least not directly). > > > > Are you able to run the dm_mapper process under gdb and inspect it > > to find the prefix? > > Well, let me take that back. dm_mapper is the last printk, but it > could be anything after that or that. > > Unfortunately, it''s dom0 and during boot, so can''t run gdb on it. I am > hacking the kernel now to print every user process name in schedule.Ah, it''s "xen-detect" coming in right after dm_mapper. I see the xen prefix in it. Hmm... let me see if I can add some run time check for hybrid dom0, then won''t have to worry about invalid_op trap. Less code that way. That''s the end goal anyways... Thanks, Mukesh
On Fri, 29 Jun 2012 08:56:39 +0100 "Jan Beulich" <JBeulich@suse.com> wrote:> >>> On 29.06.12 at 03:00, Mukesh Rathor <mukesh.rathor@oracle.com> > >>> wrote: > > BTW, for cpuid vmexit, if kernel mode I call pv_cpuid, if user > > mode, I just return cpuid values, as that''s what would happen > > without the HVM container. > > Which means you''re not leveraging one of the things you could > gain from hybrid (after all, returning the native value to user > mode is one of the weaknesses of PV). > > Jan >Easy, fixed. So, now, for both kernel/user I return pv_cpuid.
On Fri, 2012-06-29 at 20:07 +0100, Mukesh Rathor wrote:> On Fri, 29 Jun 2012 11:15:46 -0700 > Mukesh Rathor <mukesh.rathor@oracle.com> wrote: > > > It''s unlikely but not impossible for a userspace developer to have > > > done some "Xen magic" and used the prefix in userspace. > > > > > > What version of dm_mapper do you have? I checked the version in > > > Debian Sid and it doesn''t do this (at least not directly). > > > > > > Are you able to run the dm_mapper process under gdb and inspect it > > > to find the prefix? > > > > Well, let me take that back. dm_mapper is the last printk, but it > > could be anything after that or that. > > > > Unfortunately, it''s dom0 and during boot, so can''t run gdb on it. I am > > hacking the kernel now to print every user process name in schedule. > > Ah, it''s "xen-detect" coming in right after dm_mapper. I see the > xen prefix in it. Hmm... let me see if I can add some run time check > for hybrid dom0, then won''t have to worry about invalid_op trap. Less > code that way. That''s the end goal anyways...I don''t think reducing code should come at the expense of adding special cases for hybrid to userspace programs... I think you''ll have to handle the invalid op, it can''t be that much code, can it? Ian
On Fri, 29 Jun 2012 15:04:14 -0700 (PDT) Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Fri, 2012-06-29 at 20:07 +0100, Mukesh Rathor wrote: > > I don''t think reducing code should come at the expense of adding > special cases for hybrid to userspace programs... > > I think you''ll have to handle the invalid op, it can''t be that much > code, can it?Not a whole lot of code, but other than xen-detect, just curious, what is the possibility of any user level using XEN_EMULATE_PREFIX cpuid? The kernel won''t, since it''s only the hybrid running in hvm container, and it''s modified to use native cpuid. The user level can just run cpuid and will be trapped in vmexit handler to call pv_cpuid(). thanks, Mukesh
>>> On 30.06.12 at 00:50, Mukesh Rathor <mukesh.rathor@oracle.com> wrote: > On Fri, 29 Jun 2012 15:04:14 -0700 (PDT) > Ian Campbell <Ian.Campbell@citrix.com> wrote: > >> On Fri, 2012-06-29 at 20:07 +0100, Mukesh Rathor wrote: >> >> I don''t think reducing code should come at the expense of adding >> special cases for hybrid to userspace programs... >> >> I think you''ll have to handle the invalid op, it can''t be that much >> code, can it? > > Not a whole lot of code, but other than xen-detect, just curious, > what is the possibility of any user level using XEN_EMULATE_PREFIX cpuid?Obviously any code cloned from xen-detect could, i.e. anything that tries to be Xen-PV-aware. Jan
On Mon, 2012-07-02 at 09:27 +0100, Jan Beulich wrote:> >>> On 30.06.12 at 00:50, Mukesh Rathor <mukesh.rathor@oracle.com> wrote: > > On Fri, 29 Jun 2012 15:04:14 -0700 (PDT) > > Ian Campbell <Ian.Campbell@citrix.com> wrote: > > > >> On Fri, 2012-06-29 at 20:07 +0100, Mukesh Rathor wrote: > >> > >> I don''t think reducing code should come at the expense of adding > >> special cases for hybrid to userspace programs... > >> > >> I think you''ll have to handle the invalid op, it can''t be that much > >> code, can it? > > > > Not a whole lot of code, but other than xen-detect, just curious, > > what is the possibility of any user level using XEN_EMULATE_PREFIX cpuid? > > Obviously any code cloned from xen-detect could, i.e. anything > that tries to be Xen-PV-aware.Which shouldn''t be all that much code but we can''t really just declare it to be wrong... Ian.
On Mon, 02 Jul 2012 09:27:47 +0100 "Jan Beulich" <JBeulich@suse.com> wrote:> >>> On 30.06.12 at 00:50, Mukesh Rathor <mukesh.rathor@oracle.com> > >>> wrote: > > On Fri, 29 Jun 2012 15:04:14 -0700 (PDT) > > Ian Campbell <Ian.Campbell@citrix.com> wrote: > > > >> On Fri, 2012-06-29 at 20:07 +0100, Mukesh Rathor wrote: > >> > >> I don''t think reducing code should come at the expense of adding > >> special cases for hybrid to userspace programs... > >> > >> I think you''ll have to handle the invalid op, it can''t be that much > >> code, can it? > > > > Not a whole lot of code, but other than xen-detect, just curious, > > what is the possibility of any user level using XEN_EMULATE_PREFIX > > cpuid? > > Obviously any code cloned from xen-detect could, i.e. anything > that tries to be Xen-PV-aware. > > Jan >Ok, so it is official to do that from user level. I will add support for it. thanks, Mukesh