llvm dev - Aug 2013 - [LLVMdev] [RFC PATCH] X32 ABI support for Clang/compiler-rt (compiler-rt patch)

On Thu, 2013-08-22 at 11:27 +0400, Alexey Samsonov
wrote:
> Hi Steven,
> 
> This looks interesting and raises a number of questions :)
> 
> 1) Does applying this patch actually bring working sanitizers to x32
> platform?
> That is, after you build the clang, does "clang -fsanitize=whatever
foo.c"
> compile/link/run with expected results?
> I doubt that, as there is some platform-specific code in all the
> sanitizers, and many of them heavily depend
> on the address space layout. Porting TSan and MSan to 32-bit address space
> is especially hard, and we don't plan
> to do this anytime soon. I think it makes sense to build only the libraries
> that are expected to work on a given arch.
> 
I should have made clear this is very much a WIP. I expect to have make
an effort to port compiler-rt, but first I need to be able to generate
elf32_x86_64 objects with Clang.  I confess, my method of porting code
to x32 so far has consisted of trying to build existing code and
modifying what breaks, or disabling features where inapplicable.  I
haven't studied the code well enough to understand and predict where I'm
going to have make modification in advance.  It's worked so far with
most things I've attempted but LLVM/Clang/compiler-rt is an order of
magnitude more complex than anything else I've poked at.  The biggest
success I've had so far with this method is porting of Mozilla
Spidermonkey/js17.
> 2) If we're stepping on the path of porting some sanitizers to x32, it
> would be great to
> setup a buildbot, make our tests pass cleanly, and make this bot public to
> catch regressions.
> Do you have plans for that? Running sanitizer test suites also leads us
> to...
> 
I'm new to llvm/clang, I'm going to have to take guidance from others.
> 3) Do you plan to add support for building sanitizers on x32 to CMake build
> system? It would
> make sense (and, in fact, I would start from there), as our testsuite can
> be run only in CMake builds.
> 
I can do so, (locally switch to cmake) I have a little experience with
cmake so I should be able to figure it out.  From my distro point of
view, the plan on Gentoo is to switch to CMake for the LLVM/Clang
ebuild, but it hasn't happened yet.  Perhaps this could be expedited, I
can contact the package maintainer.

On Thu, Aug 22, 2013 at 11:39 AM, Steven Newbury <steve at
snewbury.org.uk>wrote:
> On Thu, 2013-08-22 at 11:27 +0400, Alexey Samsonov wrote:
> > Hi Steven,
> >
> > This looks interesting and raises a number of questions :)
> >
> > 1) Does applying this patch actually bring working sanitizers to x32
> > platform?
> > That is, after you build the clang, does "clang
-fsanitize=whatever
> foo.c"
> > compile/link/run with expected results?
> > I doubt that, as there is some platform-specific code in all the
> > sanitizers, and many of them heavily depend
> > on the address space layout. Porting TSan and MSan to 32-bit address
> space
> > is especially hard, and we don't plan
> > to do this anytime soon. I think it makes sense to build only the
> libraries
> > that are expected to work on a given arch.
> >
> I should have made clear this is very much a WIP. I expect to have make
> an effort to port compiler-rt, but first I need to be able to generate
> elf32_x86_64 objects with Clang.  I confess, my method of porting code
> to x32 so far has consisted of trying to build existing code and
> modifying what breaks, or disabling features where inapplicable.  I
> haven't studied the code well enough to understand and predict where
I'm
> going to have make modification in advance.  It's worked so far with
> most things I've attempted but LLVM/Clang/compiler-rt is an order of
> magnitude more complex than anything else I've poked at.  The biggest
> success I've had so far with this method is porting of Mozilla
> Spidermonkey/js17.
>
So, IIUC, if you're working on making LLVM/Clang/compiler-rt source tree
build on x32 host, I suggest your first step should be: disable building
compiler-rt
on x32 host :) One of the reasons: in makefile-based build compiler-rt
sources
are compiled with just-built Clang (I honestly don't know if Clang can
target x32
at the moment). Another reason: compiler-rt code is really arch-specific,
and even
if it compiles, it wouldn't work.

>
> > 2) If we're stepping on the path of porting some sanitizers to
x32, it
> > would be great to
> > setup a buildbot, make our tests pass cleanly, and make this bot
public
> to
> > catch regressions.
> > Do you have plans for that? Running sanitizer test suites also leads
us
> > to...
> >
> I'm new to llvm/clang, I'm going to have to take guidance from
others.
>
> > 3) Do you plan to add support for building sanitizers on x32 to CMake
> build
> > system? It would
> > make sense (and, in fact, I would start from there), as our testsuite
can
> > be run only in CMake builds.
> >
> I can do so, (locally switch to cmake) I have a little experience with
> cmake so I should be able to figure it out.  From my distro point of
> view, the plan on Gentoo is to switch to CMake for the LLVM/Clang
> ebuild, but it hasn't happened yet.  Perhaps this could be expedited, I
> can contact the package maintainer.

FTR, CMake would probably see that we our host is neither x86_64 nor i386,
and won't build anything in compiler-rt. If it wouldn't, we should go
and
fix this.

-- 
Alexey Samsonov, MSK
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130822/191df00f/attachment.html>

On Thu, 2013-08-22 at 11:50 +0400, Alexey Samsonov
wrote:
> On Thu, Aug 22, 2013 at 11:39 AM, Steven Newbury <steve at
snewbury.org.uk>wrote:
> 
> > On Thu, 2013-08-22 at 11:27 +0400, Alexey Samsonov wrote:
> > > Hi Steven,
> > >
> > > This looks interesting and raises a number of questions :)
> > >
> > > 1) Does applying this patch actually bring working sanitizers to
x32
> > > platform?
> > > That is, after you build the clang, does "clang
-fsanitize=whatever
> > foo.c"
> > > compile/link/run with expected results?
> > > I doubt that, as there is some platform-specific code in all the
> > > sanitizers, and many of them heavily depend
> > > on the address space layout. Porting TSan and MSan to 32-bit
address
> > space
> > > is especially hard, and we don't plan
> > > to do this anytime soon. I think it makes sense to build only the
> > libraries
> > > that are expected to work on a given arch.
> > >
> > I should have made clear this is very much a WIP. I expect to have
make
> > an effort to port compiler-rt, but first I need to be able to generate
> > elf32_x86_64 objects with Clang.  I confess, my method of porting code
> > to x32 so far has consisted of trying to build existing code and
> > modifying what breaks, or disabling features where inapplicable.  I
> > haven't studied the code well enough to understand and predict
where I'm
> > going to have make modification in advance.  It's worked so far
with
> > most things I've attempted but LLVM/Clang/compiler-rt is an order
of
> > magnitude more complex than anything else I've poked at.  The
biggest
> > success I've had so far with this method is porting of Mozilla
> > Spidermonkey/js17.
> >
> 
> So, IIUC, if you're working on making LLVM/Clang/compiler-rt source
tree
> build on x32 host, I suggest your first step should be: disable building
> compiler-rt
> on x32 host :) One of the reasons: in makefile-based build compiler-rt
> sources
> are compiled with just-built Clang (I honestly don't know if Clang can
> target x32
> at the moment). Another reason: compiler-rt code is really arch-specific,
> and even
> if it compiles, it wouldn't work.
> 
See accompanied Clang patch in other email.  I've attempted to get Clang
to target x32, I'm also suspicious that x32 built clang isn't correctly
working to produce 64-bit code, but that's a somewhat different bug that
I haven't yet attempted to address.  Without modifying Clang to target
x32, the x32 *built* Clang fails to compile compiler-rt with -m64, I
need help decoding the crash backtrace to figure out where that bug
lies, but it's not related to (or helped by) my
patch.
> 
> >
> > > 2) If we're stepping on the path of porting some sanitizers
to x32, it
> > > would be great to
> > > setup a buildbot, make our tests pass cleanly, and make this bot
public
> > to
> > > catch regressions.
> > > Do you have plans for that? Running sanitizer test suites also
leads us
> > > to...
> > >
> > I'm new to llvm/clang, I'm going to have to take guidance from
others.
> >
> > > 3) Do you plan to add support for building sanitizers on x32 to
CMake
> > build
> > > system? It would
> > > make sense (and, in fact, I would start from there), as our
testsuite can
> > > be run only in CMake builds.
> > >
> > I can do so, (locally switch to cmake) I have a little experience with
> > cmake so I should be able to figure it out.  From my distro point of
> > view, the plan on Gentoo is to switch to CMake for the LLVM/Clang
> > ebuild, but it hasn't happened yet.  Perhaps this could be
expedited, I
> > can contact the package maintainer.
> 
> 
> FTR, CMake would probably see that we our host is neither x86_64 nor i386,
> and won't build anything in compiler-rt. If it wouldn't, we should
go and
> fix this.
> 
Maybe.  Although, technically it *is* x86_64, but with a different ABI.
I don't know how that's handled, although it works for ARM/MIPS, right??