Xen users - Aug 2006 - Xen and TLS, -mno-tls-direct-seg-refs needed?

Hi,

The Gentoo-Wiki about Xen suggests, to add "-mno-tls-direct-seg-refs"
to
CFLAGS and to recompile the whole system. So it seems, NPTL will work
without problems (and without performance drawback) after that.

Sweet!

But i carefully read http://wiki.xensource.com/xenwiki/XenSpecificGlibc,
and it says:

"On 32-bit x86 platforms, Xen uses segmentation to provide protection of
the memory used for the hypervisor. This results in some performance
issues since wrap-around segments as used by glibc need expensive extra
handling. For more information on how Xen uses segmentation, see
XenSegments."

On 32-bit platforms?

Does it mean, that i don''t need to use that
"-mno-tls-direct-seg-refs"
on 64-Bit platforms?


That''s actually all i''m concerned about.


Thanks,
  Sven



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Fri, Aug 11, 2006 at 12:26:52AM +0200, Sven Köhler
wrote:
> Does it mean, that i don''t need to use that
"-mno-tls-direct-seg-refs"
> on 64-Bit platforms?
That is correct.

- Matt

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Sven Köhler wrote:
> "On 32-bit x86 platforms, ..."
Woo!
I save someone''s ass from doing the same stupidity that I did :-).
*pat self on back, feel proud*

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thursday 10 August 2006 5:26 pm, Sven Köhler wrote:
> On 32-bit platforms?
>
> Does it mean, that i don''t need to use that
"-mno-tls-direct-seg-refs"
> on 64-Bit platforms?
right.  the problem is that on 32bit x86 NPTL uses a weird trick to store TLS 
on negative segment references.  in other platforms that trick isn''t
used,
and there''s no problem with Xen.

the "-mno-tls-direct-seg-refs" flag turns off the trick for 32bit x86,
so it''s
forced to use a saner approach.  in theory it''s slower than the default
way,
but much faster than being emulated by Xen

-- 
Javier


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

>> "On 32-bit x86 platforms, ..."
> 
> Woo!
> I save someone''s ass from doing the same stupidity that I did :-).
> *pat self on back, feel proud*
LOL ... though ...

If i have a multilib x86_64 system (that means: 64-bit glibc and 32-bit
glibc are present), then the 32-bit does still needs this
-mno-tls-direct-seg-refs neede - or does it not?



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Sven Köhler wrote:
> If i have a multilib x86_64 system (that means: 64-bit glibc and
> 32-bit glibc are present), then the 32-bit does still needs this
> -mno-tls-direct-seg-refs neede - or does it not?
I honestly have NO idea.

I would imagine that it works like this:

Since the CPU is running in 64-bit mode by virtue of you running a
64-bit OS, any 32-bit code is executed in a fashion where the
operating system has first told the CPU to set up a virtual 4GB 32-bit
environment.  Any kernel calls from 32-bit code goes into a
translation layer the kernel has, and the translation layer knows how
to switch the CPU to 64-bit mode to actually perform the kernel call
onto the 64-bit kernel.  Xen would live entirely outside of the 4 GB
virtual 32-bit area, being used only from the 64-bit kernel, and being
protected by whatever mechanism it is that Xen normally uses on
x86_64.

But I''m just imagining these things, I haven''t actually read a
thing
about CPUs since the switch from 80286 to 80386.  Hope someone who
actually knows comes along :-).

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

>> If i have a multilib x86_64 system (that means: 64-bit glibc and
>> 32-bit glibc are present), then the 32-bit does still needs this
>> -mno-tls-direct-seg-refs neede - or does it not?
> 
> I honestly have NO idea.
> 
> I would imagine that it works like this:
> 
> Since the CPU is running in 64-bit mode by virtue of you running a
> 64-bit OS, any 32-bit code is executed in a fashion where the
> operating system has first told the CPU to set up a virtual 4GB 32-bit
> environment.  Any kernel calls from 32-bit code goes into a
> translation layer the kernel has, and the translation layer knows how
> to switch the CPU to 64-bit mode to actually perform the kernel call
> onto the 64-bit kernel.  Xen would live entirely outside of the 4 GB
> virtual 32-bit area, being used only from the 64-bit kernel, and being
> protected by whatever mechanism it is that Xen normally uses on
> x86_64.
Yes, but what you describe is kernel-internal. But the
"-mno-tls-direct-seg-refs" is glibc-specific and changes
glibc-internal
things, not kernel-internal things.

So if the 64-bit kernel allows the 32-bit glibc to access the TLS-data
the usual way, and if the 64-Bit Xen also has the problems with that
usual way, then the "-mno-tls-direct-seg-refs" might be needed.

But who knows? :-)



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Sven Köhler wrote:
> Yes, but what you describe is kernel-internal. But the
> "-mno-tls-direct-seg-refs" is glibc-specific and changes
glibc-internal
> things, not kernel-internal things.
Yes, but what I was implying (but not writing - sorry), was that since
the 4 GB address space is virtual, and Xen does not live within it,
there''s no need for Xen to fiddle with segments for protection
purposes, and thus it probably doesn''t.  And so there wouldn''t
be a
performance penalty.
> But who knows? :-)
Not me, I just keep ranting :-).

I hope Mats can answer it when he comes around; he seems to have quite
a lot of knowledge on all things CPU and I''ve seen him help out people
in xen-users quite a lot.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> -----Original Message-----
> From: xen-users-bounces@lists.xensource.com 
> [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of 
> Molle Bestefich
> Sent: 12 August 2006 11:39
> To: Sven Köhler
> Cc: xen-users@lists.xensource.com
> Subject: [Xen-users] Re: Xen and TLS, -mno-tls-direct-seg-refs needed?
> 
> Sven Köhler wrote:
> > Yes, but what you describe is kernel-internal. But the
> > "-mno-tls-direct-seg-refs" is glibc-specific and changes 
> glibc-internal
> > things, not kernel-internal things.
> 
> Yes, but what I was implying (but not writing - sorry), was that since
> the 4 GB address space is virtual, and Xen does not live within it,
> there''s no need for Xen to fiddle with segments for protection
> purposes, and thus it probably doesn''t.  And so there
wouldn''t be a
> performance penalty.
> 
> > But who knows? :-)
> 
> Not me, I just keep ranting :-).
> 
> I hope Mats can answer it when he comes around; he seems to have quite
> a lot of knowledge on all things CPU and I''ve seen him help out
people
> in xen-users quite a lot.
Seeing as I was mentioned directly, I suppose I have to try to reply to this. 

What is being said above seems to make sense to me (I actually don''t
REALLY know how the TLS stuff works, other than what I''ve read here,
which says that it uses negative segment offsets, which makes funny things
happen if you use limits other than 4GB on the segment!).

In fact, the segment limits aren''t being used in the 64-bit port at all
- it uses some tricky page-table stuff instead. This is because when we
initially made the 64-bit mode for Opteron/Athlon64 processors, we
didn''t foresee anyone using segments for protection any longer - most
32-bit code doesn''t, the exception being some of the VMMs... So it was
said that the limit of a segment is "unused".

However, there is a fix in the later models of Opteron/Athlon64 processors that
allows segments to be used. Intel, I think, hasn''t got this feature
(yet?)...

So for backwards compatibility reasons and such, the 64-bit version of Xen uses
a different method for protecting itself from being overwritten, and therfore
negative segment references work fine.

--
Mats
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> 
> 
> 


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> However, there is a fix in the later models of Opteron/Athlon64 processors
that allows segments to be used. Intel, I think, hasn''t got this
feature (yet?)...
And 64-Bit VMWare doesn''t run without it, and my Athlon64-CPU
doesn''t
have segmentation-support yet :-(

(And so i guess, Intel CPUs have that feature)
> So for backwards compatibility reasons and such, the 64-bit version of Xen
uses a different method for protecting itself from being overwritten, and
therfore negative segment references work fine.
I see. Thanks for the clarification.



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Sven Köhler wrote:
>> However, there is a fix in the later models of Opteron/Athlon64
processors that allows segments to be used. Intel, I think, hasn''t got
this feature (yet?)...
> 
> And 64-Bit VMWare doesn''t run without it, and my Athlon64-CPU
doesn''t
> have segmentation-support yet :-(
> 
> (And so i guess, Intel CPUs have that feature)
no they don''t - if you want to read up a bit you could take a look at
the following vmware knowledgebase article:

http://kb.vmware.com/vmtnkb/search.do?cmd=displayKC&docType=kc&externalId=1901&sliceId=SAL_Public

Stefan

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users