In the tools there is a declaration: #if defined(__i386__) #define rmb() __asm__ __volatile__ ( "lock; addl $0,0(%%esp)" : : : "memory" ) #define wmb() __asm__ __volatile__ ( "" : : : "memory" ) #else #error "Define barriers" #endif located in: xcs/xcs.h tools/python/xen/lowlevel/xu/xu.c I''m assuming this has a convenient side-effect that it prevents read reordering. Otherwise I can''t figure out why this is being done at all. Now I''m guessing that that using rsp instead of esp since we are in 64bit mode will give the same effect needed. #elif defined(__x86_64__) #define rmb() __asm__ __volatile__ ( "lock; addl $0,0(%%rsp)" : : : "memory" ) #define wmb() __asm__ __volatile__ ( "" : : : "memory" ) I would like to discuss is this correct, dead wrong, or even needed at all? -- Jerone Young Open Virtualization IBM Linux Technology Center jyoung5@us.ibm.com 512-838-1157 (T/L: 678-1157) ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On 1 Mar 2005, at 19:56, Jerone Young wrote:> I''m assuming this has a convenient side-effect that it prevents read > reordering. Otherwise I can''t figure out why this is being done at all. > Now I''m guessing that that using rsp instead of esp since we are in > 64bit mode will give the same effect needed. > > #elif defined(__x86_64__) > #define rmb() __asm__ __volatile__ ( "lock; addl $0,0(%%rsp)" : : : > "memory" ) > #define wmb() __asm__ __volatile__ ( "" : : : "memory" ) > > > I would like to discuss is this correct, dead wrong, or even needed at > all?x86/64 has proper barrier instructions -- see include/asm-x86_64/system.h in Linux. It is from there that we should pull our definitions. Barrier macros are defined in a few places in the tools -- we ought to pull them all into one single header incorporated by all tools that need it. -- Keir ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Keir Fraser wrote:> > On 1 Mar 2005, at 19:56, Jerone Young wrote: > >> I would like to discuss is this correct, dead wrong, or even >> needed at >> all? > > > x86/64 has proper barrier instructions -- see > include/asm-x86_64/system.h in Linux. It is from there that we should > pull our definitions. Barrier macros are defined in a few places in > the tools -- we ought to pull them all into one single header > incorporated by all tools that need it.I was just about to send this out myself :-) I think the consensus was to put them all in xc.h. I posted a patch recently that made everything include asm/system.h. All it should take is replacing asm/system.h with xc.h in the patch and then making the necessary changes to xc.h. For reference, the proper x86-64 barriers are: #define mb() asm volatile("mfence":::"memory") #define rmb() asm volatile("lfence":::"memory") #define wmb() asm volatile("sfence":::"memory") Regards, Anthony Liguori -- Regards, Anthony Liguori ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Jerone Young
2005-Mar-01 22:24 UTC
[Xen-devel] [PATCH] cpu barriers moved and x86-64 barriers add
We actually should have wmb() be defined as "asm
volatile("":::"memory")
since we are not using out of order io.
The patch attached removes all the instances and places the definitions
in libxc/xc.h as per the discussion last week between Anthony & Keir.
Signed-off-by: Jerone Young <jyoung5@us.ibm.com>
On Tue, 2005-03-01 at 15:23 -0600, Anthony Liguori
wrote:> Keir Fraser wrote:
>
> >
> > On 1 Mar 2005, at 19:56, Jerone Young wrote:
> >
> >> I would like to discuss is this correct, dead wrong, or even
> >> needed at
> >> all?
> >
> >
> > x86/64 has proper barrier instructions -- see
> > include/asm-x86_64/system.h in Linux. It is from there that we should
> > pull our definitions. Barrier macros are defined in a few places in
> > the tools -- we ought to pull them all into one single header
> > incorporated by all tools that need it.
>
> I was just about to send this out myself :-) I think the consensus was
> to put them all in xc.h. I posted a patch recently that made everything
> include asm/system.h. All it should take is replacing asm/system.h with
> xc.h in the patch and then making the necessary changes to xc.h.
>
> For reference, the proper x86-64 barriers are:
>
> #define mb() asm volatile("mfence":::"memory")
> #define rmb() asm volatile("lfence":::"memory")
> #define wmb() asm volatile("sfence":::"memory")
>
> Regards,
> Anthony Liguori
>
--
Jerone Young
Open Virtualization
IBM Linux Technology Center
jyoung5@us.ibm.com
512-838-1157 (T/L: 678-1157)
Jerone Young wrote:> In the tools there is a declaration: > > #if defined(__i386__) > #define rmb() __asm__ __volatile__ ( "lock; addl $0,0(%%esp)" : : : > "memory" ) > #define wmb() __asm__ __volatile__ ( "" : : : "memory" ) > #else > #error "Define barriers" > #endif > > located in: > xcs/xcs.h > tools/python/xen/lowlevel/xu/xu.c > > I''m assuming this has a convenient side-effect that it prevents read > reordering. Otherwise I can''t figure out why this is being done at all. > Now I''m guessing that that using rsp instead of esp since we are in > 64bit mode will give the same effect needed. > > #elif defined(__x86_64__) > #define rmb() __asm__ __volatile__ ( "lock; addl $0,0(%%rsp)" : : : > "memory" ) > #define wmb() __asm__ __volatile__ ( "" : : : "memory" ) > > I would like to discuss is this correct, dead wrong, or even needed at > all?See volume 1, section 3.9.2 of the AMD64 architecture manual (http://www.amd.com/us-en/Processors/DevelopWithAMD/0,,30_2252_875_7044,00.html), quoted below for convenience. This should be guaranteed to work: #define rmb() __asm__ __volatile__ ( "mfence" : : : "memory" ) #define wmb() __asm__ __volatile__ ( "mfence" : : : "memory" ) but may be overkill. Are these macros used in any performance-critical situations? If not then the overkill wouldn''t matter. # 3.9.2 Forcing Memory Order # # Special instructions are provided for application software to force # memory ordering in situations where such ordering is important. These # instructions are: # * Load Fence -- The LFENCE instruction forces ordering of # memory loads (reads). All memory loads preceding the # LFENCE (in program order) are completed prior to # completing memory loads following the LFENCE. Memory # loads cannot be reordered around an LFENCE instruction, # but other non-serializing instructions (such as memory # writes) can be reordered around the LFENCE. # * Store Fence -- The SFENCE instruction forces ordering of # memory stores (writes). All memory stores preceding the # SFENCE (in program order) are completed prior to # completing memory stores following the SFENCE. Memory # stores cannot be reordered around an SFENCE instruction, # but other non-serializing instructions (such as memory # loads) can be reordered around the SFENCE. # * Memory Fence -- The MFENCE instruction forces ordering of # all memory accesses (reads and writes). All memory accesses # preceding the MFENCE (in program order) are completed # prior to completing any memory access following the # MFENCE. Memory accesses cannot be reordered around an # MFENCE instruction, but other non-serializing instructions # that do not access memory can be reordered around the # MFENCE. -- David Hopwood <david.nospam.hopwood@blueyonder.co.uk> ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
David Hopwood wrote:> Jerone Young wrote: > See volume 1, section 3.9.2 of the AMD64 architecture manual > (http://www.amd.com/us-en/Processors/DevelopWithAMD/0,,30_2252_875_7044,00.html), > > quoted below for convenience. > > This should be guaranteed to work: > > #define rmb() __asm__ __volatile__ ( "mfence" : : : "memory" ) > #define wmb() __asm__ __volatile__ ( "mfence" : : : "memory" )sfence and lfence would seem to be more appropriate. mfence would seem to be appropriate for an mb() macro (which is usually also defined--but in this case not necessary).> but may be overkill. Are these macros used in any performance-critical > situations? If not then the overkill wouldn''t matter.Not at all. They''re currently used to prevent race conditions within the domain control channel ring queue. This queue is relatively low-traffic. The confusion stemmed from the x86-32 code that doesn''t actually implement a wmb() since no relevant architecture does out-of-order stores. x86-64 is another story. The linux includes still have the same comments indicating that wmb() is not necessary (an obvious cut-and-paste job) even though wmb() is defined to sfence. To make matters worse, the wmb() is wrapped in the same CONFIG_UNORDERED_IO guarded (that''s enabled by default in x86-64 but disabled by default in x86-32). It''s understandably confusing. I''m hoping we can do a much better job in Xen in supporting both architectures to avoid these sort of confusions. Regards, Anthony Liguori ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Jerone Young
2005-Mar-02 17:27 UTC
Re: [Xen-devel] [PATCH] cpu barriers moved and x86-64 barriers add
Ok pay this email no mind. This was the first one I sent yesterday. Something must have happened with the IBM smtp server and caused my email to be held and outragously long time then sent. Sorry about this guys. On Tue, 01 Mar 2005 16:24:37 -0600, Jerone Young <jyoung5@us.ibm.com> wrote:> We actually should have wmb() be defined as "asm volatile("":::"memory") > since we are not using out of order io. > > The patch attached removes all the instances and places the definitions > in libxc/xc.h as per the discussion last week between Anthony & Keir. > > Signed-off-by: Jerone Young <jyoung5@us.ibm.com> > > On Tue, 2005-03-01 at 15:23 -0600, Anthony Liguori wrote: > > Keir Fraser wrote: > > > > > > > > On 1 Mar 2005, at 19:56, Jerone Young wrote: > > > > > >> I would like to discuss is this correct, dead wrong, or even > > >> needed at > > >> all? > > > > > > > > > x86/64 has proper barrier instructions -- see > > > include/asm-x86_64/system.h in Linux. It is from there that we should > > > pull our definitions. Barrier macros are defined in a few places in > > > the tools -- we ought to pull them all into one single header > > > incorporated by all tools that need it. > > > > I was just about to send this out myself :-) I think the consensus was > > to put them all in xc.h. I posted a patch recently that made everything > > include asm/system.h. All it should take is replacing asm/system.h with > > xc.h in the patch and then making the necessary changes to xc.h. > > > > For reference, the proper x86-64 barriers are: > > > > #define mb() asm volatile("mfence":::"memory") > > #define rmb() asm volatile("lfence":::"memory") > > #define wmb() asm volatile("sfence":::"memory") > > > > Regards, > > Anthony Liguori > > > -- > Jerone Young > Open Virtualization > IBM Linux Technology Center > jyoung5@us.ibm.com > 512-838-1157 (T/L: 678-1157) > > >------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel