llvm dev - Jul 2012 - [LLVMdev] RFC: Staging area proposal for new backends

On Sat, Jul 21, 2012 at 8:08 PM, Chris Lattner <clattner at apple.com>
wrote:
> On Jul 20, 2012, at 8:51 AM, Tom Stellard wrote:
> > The goals of the staging area will be:
> >  1. Facilitate communication between the LLVM project and backend
> >     developers
> >  2. Ensure that new backends meet LLVM standards
> >  3. Give the backend more exposure to users and prospective developers
>
> FWIW, I really like this idea or concept, but we have to be careful for it
> to be done right. This is also more general than just backends:
> experimental optimizers and runtime libraries can also benefit from
> something like this.
>
> > ++ Staging Area:
> >
> > Similar to the Linux kernel, the staging area for new backends will
> > be in the main LLVM tree, with building of the backend being disabled
> > by default.
>
> Makes sense.  Them being in the main tree isn't a problem as long as
they
> aren't built by default, tested in buildbots, etc.
>
So will new back-ends be allowed to begin development in the "staging
area?"  Let's say I want to develop a new back-end.  Would I be able to
start writing it in the LLVM main tree, or does it need to progress to a
certain point in an external repository first?

>
> > 1. It will be easier for LLVM developers to become familiar with the
> >   new backend and identify areas for improvement.
> >
> >  If the new backend is in the main tree, LLVM developers are more
> >  likely to encounter it in their day to day development.  Imagine a
> >  scenario where a developer makes a change to LLVM core that impacts
> >  several backends.  The developer may grep the code looking for
> >  backends that make use of the feature that they have added or
> >  changed.
>
> This makes sense, but it should not be a *requirement* that API changes
> don't break experimental backends.  This would be the responsibility of
the
> contributors/owner of that backend and/or steward to make sure it keeps
> building.  Of course it's great for someone to update all the
experimental
> backends if they want, but it shouldn't be a requirement.
>
> >  If the backend were staged in a separate tree, this kind of
> >  simple review would not be possible, and I would be concerned that
> >  developers would be too busy to ever get around to checking out
> >  the staging tree.
>
> Yep.
>
> > 2. It will allow the backend developers to always develop against TOT.
> >
> >  Developing against TOT is the recommended development procedure for
> >  anyone working on LLVM, and this is regularly reiterated on the
> >  mailing list.  If the new backend is included in the main tree,
> >  the backend developers will have no choice but to work against TOT.
>
> +1!!!
>
> > 3. It will make it easier for end users and distributions to test and
> >   also make it easier for new contributors.
> >
> >  New backends will be more visible to the public if they are in the
> >  main tree.  This will mean more users, an expanded testing base, and
> >  more potential developers which will lead to a higher quality
backend.
>
> I'd also add:
>
> 4. Infrastructure enhancements that are only required for an experimental
> backend can be implemented in the main tree, even if that infrastructure
> isn't needed by other targets.  Of course, these changes need to meet
the
> standard quality bar for general code in the compiler.
>
> In the past, we've had some general infrastructure features get denied
> because they didn't relate to any targets in-tree.
>
This is definitely a great point.  Part of the problem we had with the
NVPTX back-end was that we needed some LLVM core changes, but only the
NVPTX back-end would use these changes.  It was a chicken-and-egg problem;
this would solve that problem.

>
> > ++ What is needed from the LLVM developers:
> >
> >  In order to make this staging program successful, the LLVM project
> >  will need to appoint a "code owner" for the staging
process, who
> >  backend developers can contact when they are interested in getting
> >  the backend included in the main tree.  An LLVM developer will also
> >  be needed to act as a steward for the new backend and help guide
> >  the backend developers through the process.
>
> I think we also need to define the minimum quality bar for a backend to be
> included.
>
This could be the trickiest part overall. In this sense, X86 and ARM are
easy because reference assemblers/hardware/simulators are so easily
available.  These back-ends can also be verified through the LLVM test
suite.  My understanding of Tom's R600 back-end is that it only works in
conjunction with Mesa/Gallium, so verification on real hardware cannot
easily be done (please correct me if I am wrong here).

On one hand, we could establish a minimum set of LLVM IR or SDAG that must
be handled by the back-end and verified through unit test cases.  Having
100% LLVM IR/SDAG coverage is not reasonable simply because some concepts
(e.g. exception handling, all intrinsics, etc.) are not present in the
target hardware, VM, etc..

My feeling is that this should remain somewhat generic and be established
on a back-end by back-end basis, based on the target hardware and what can
feasibly be supported.

>
> Also, if we do this, can we demote CellSPU? :)
>
> -Chris
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>


-- 

Thanks,

Justin Holewinski
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120723/8fc2b205/attachment.html>

On Mon, Jul 23, 2012 at 10:21:57AM -0400, Justin Holewinski
wrote:
> On Sat, Jul 21, 2012 at 8:08 PM, Chris Lattner <clattner at
apple.com> wrote:
> 
> > On Jul 20, 2012, at 8:51 AM, Tom Stellard wrote:
> > > The goals of the staging area will be:
> > >  1. Facilitate communication between the LLVM project and backend
> > >     developers
> > >  2. Ensure that new backends meet LLVM standards
> > >  3. Give the backend more exposure to users and prospective
developers
> >
> > FWIW, I really like this idea or concept, but we have to be careful
for it
> > to be done right. This is also more general than just backends:
> > experimental optimizers and runtime libraries can also benefit from
> > something like this.
> >
> > > ++ Staging Area:
> > >
> > > Similar to the Linux kernel, the staging area for new backends
will
> > > be in the main LLVM tree, with building of the backend being
disabled
> > > by default.
> >
> > Makes sense.  Them being in the main tree isn't a problem as long
as they
> > aren't built by default, tested in buildbots, etc.
> >
> 
> So will new back-ends be allowed to begin development in the "staging
> area?"  Let's say I want to develop a new back-end.  Would I be
able to
> start writing it in the LLVM main tree, or does it need to progress to a
> certain point in an external repository first?
> 
> 
> >
> > > 1. It will be easier for LLVM developers to become familiar with
the
> > >   new backend and identify areas for improvement.
> > >
> > >  If the new backend is in the main tree, LLVM developers are more
> > >  likely to encounter it in their day to day development.  Imagine
a
> > >  scenario where a developer makes a change to LLVM core that
impacts
> > >  several backends.  The developer may grep the code looking for
> > >  backends that make use of the feature that they have added or
> > >  changed.
> >
> > This makes sense, but it should not be a *requirement* that API
changes
> > don't break experimental backends.  This would be the
responsibility of the
> > contributors/owner of that backend and/or steward to make sure it
keeps
> > building.  Of course it's great for someone to update all the
experimental
> > backends if they want, but it shouldn't be a requirement.
> >
> > >  If the backend were staged in a separate tree, this kind of
> > >  simple review would not be possible, and I would be concerned
that
> > >  developers would be too busy to ever get around to checking out
> > >  the staging tree.
> >
> > Yep.
> >
> > > 2. It will allow the backend developers to always develop against
TOT.
> > >
> > >  Developing against TOT is the recommended development procedure
for
> > >  anyone working on LLVM, and this is regularly reiterated on the
> > >  mailing list.  If the new backend is included in the main tree,
> > >  the backend developers will have no choice but to work against
TOT.
> >
> > +1!!!
> >
> > > 3. It will make it easier for end users and distributions to test
and
> > >   also make it easier for new contributors.
> > >
> > >  New backends will be more visible to the public if they are in
the
> > >  main tree.  This will mean more users, an expanded testing base,
and
> > >  more potential developers which will lead to a higher quality
backend.
> >
> > I'd also add:
> >
> > 4. Infrastructure enhancements that are only required for an
experimental
> > backend can be implemented in the main tree, even if that
infrastructure
> > isn't needed by other targets.  Of course, these changes need to
meet the
> > standard quality bar for general code in the compiler.
> >
> > In the past, we've had some general infrastructure features get
denied
> > because they didn't relate to any targets in-tree.
> >
> 
> This is definitely a great point.  Part of the problem we had with the
> NVPTX back-end was that we needed some LLVM core changes, but only the
> NVPTX back-end would use these changes.  It was a chicken-and-egg problem;
> this would solve that problem.
> 
> 
> >
> > > ++ What is needed from the LLVM developers:
> > >
> > >  In order to make this staging program successful, the LLVM
project
> > >  will need to appoint a "code owner" for the staging
process, who
> > >  backend developers can contact when they are interested in
getting
> > >  the backend included in the main tree.  An LLVM developer will
also
> > >  be needed to act as a steward for the new backend and help guide
> > >  the backend developers through the process.
> >
> > I think we also need to define the minimum quality bar for a backend
to be
> > included.
> >
> 
> This could be the trickiest part overall. In this sense, X86 and ARM are
> easy because reference assemblers/hardware/simulators are so easily
> available.  These back-ends can also be verified through the LLVM test
> suite.  My understanding of Tom's R600 back-end is that it only works
in
> conjunction with Mesa/Gallium, so verification on real hardware cannot
> easily be done (please correct me if I am wrong here).
>
Assuming you have supported hardware (AMD HD2XXX - HD6XXX GPUs), all you
need to do to verify the R600 compiler is download the latest development
version of Mesa[1], configure with --enable-r600-llvm-compiler (if you
want to use it for 3D apps) and/or --enable-opencl (if you want to use
it for OpenCL programs, this requires a patched clang/llvm at the moment
see the build instructions[2]), and then build and install.

We have a test suite called piglit[3] that we use for regression testing.
Not all the piglit tests stress the compiler, but a majority of them do,
so that would probably be the best way to verify the compiler.

A more fun way to verify the compiler would be to try playing your
favorite 3D game and check to see if the rendering matches a reference
OpenGL implementation.

While this isn't really a substitute for tests in the LLVM test suite,
if you have the hardware and are comfortable compiling your Mesa from
source then it is relatively easy to verify on real hardware.

[1] http://cgit.freedesktop.org/mesa/mesa/
[2] http://dri.freedesktop.org/wiki/GalliumCompute#How_to_Install
[3] http://cgit.freedesktop.org/piglit

-Tom

Hi Justin,
> On Mon, Jul 23, 2012 at 10:21:57AM -0400, Justin Holewinski wrote:
>> On Sat, Jul 21, 2012 at 8:08 PM, Chris Lattner <clattner at
apple.com> wrote:
>>
>>> On Jul 20, 2012, at 8:51 AM, Tom Stellard wrote:
>>>> The goals of the staging area will be:
>>>>   1. Facilitate communication between the LLVM project and
backend
>>>>      developers
>>>>   2. Ensure that new backends meet LLVM standards
>>>>   3. Give the backend more exposure to users and prospective
developers
>>>
>>> FWIW, I really like this idea or concept, but we have to be careful
for it
>>> to be done right. This is also more general than just backends:
>>> experimental optimizers and runtime libraries can also benefit from
>>> something like this.
>>>
>>>> ++ Staging Area:
>>>>
>>>> Similar to the Linux kernel, the staging area for new backends
will
>>>> be in the main LLVM tree, with building of the backend being
disabled
>>>> by default.
>>>
>>> Makes sense.  Them being in the main tree isn't a problem as
long as they
>>> aren't built by default, tested in buildbots, etc.
>>>
>>
>> So will new back-ends be allowed to begin development in the
"staging
>> area?"  Let's say I want to develop a new back-end.  Would I
be able to
>> start writing it in the LLVM main tree, or does it need to progress to
a
>> certain point in an external repository first?
personally I would say it should be mostly developed out of tree, and the
staging area is only for helping get it into the main branch once it is
looking good (just like the linux kernel's staging tree).

Ciao, Duncan.