llvm dev - Jun 2015 - [LLVMdev] Long-Term Support for LLVM Projects Extension to Build System?

----- Original Message -----
> From: "Eric Christopher" <echristo at gmail.com>
> To: "John Criswell" <jtcriswel at gmail.com>, LLVMdev at
cs.uiuc.edu, "Chris Bieneman" <beanz at apple.com>
> Sent: Thursday, June 18, 2015 7:14:06 PM
> Subject: Re: [LLVMdev] Long-Term Support for LLVM Projects Extension to
Build System?
> 
> 
> On Thu, Jun 18, 2015 at 5:07 PM John Criswell < jtcriswel at gmail.com
>
> wrote:
> 
> 
> On 6/18/15 6:49 PM, Eric Christopher wrote:
> 
> 
> 
> Hi John,
> 
> 
> Long term we don't want to keep the burden of two build systems in
> tree. CMake is turning out to be the build system we want because of
> its multi-platform support, etc and as soon as the CMake system can
> do everything we can do with the autoconf/makefile build I plan on
> turning down the support for that and removing the code.
> 
> Thanks, but I wasn't asking about autoconf vs. cmake.
> 
> 
> 
> 
> Well, you are, but only sorta.
> 
> 
> 
> I'm wanting to know whether the ability to "plug into" the
LLVM build
> system by external projects will be maintained. For example,
> SAFECode doesn't have its own build system; it uses LLVM's, and it
> does so without being a "patch" to the LLVM source tree. You can
put
> the SAFECode source code anywhere you like, use LLVM-style Makefiles
> in its source code, and build it. All of the PROJ_SRC_ROOT,
> PROJ_OBJ_ROOT magic in the autoconf build system is what permits
> that feature to work.
> 
> Do you intend to keep this functionality when you replace autoconf
> with cmake, or will all projects that want to use the LLVM build
> system need to place their source code in the LLVM source tree? To
> put it another way, do you intend to deprecate the
> PROJ_SRC_ROOT/PROJ_OBJ_ROOT stuff?
> 
> While I've enjoyed the PROJ_* magic, I can see why you'd want to
get
> rid of it. I just want to ensure that you are not planning on
> replacing it.
> 
> 
> Right. So this is a "autoconf/makefile build system vs cmake build
> system" question as the Makefiles would, theoretically, all go away
> at the end. I don't think anyone has plans of replacing this
> functionality (Chris?), but migrating out of tree projects to hook
> in via modifying the cmake build locally shouldn't be too bad.
> 
This is also related to the metapoint that the cmake build system doesn't
glob for source files (or anything else), unlike the makefile build system. This
is understandable given the constraints, but we might want to restore some of
that functionality for the purpose of supporting out-of-tree projects.

What I mean specifically, is that, for example, we could have a normally-empty
directory tools/extensions, and tools/CMakeLists.txt could glob for directories
in tools/extensions and call add_llvm_tool_subdirectory on all of them. It is
true that you'd need to re-run cmake whenever you added something there (we
could have a README in that directory to remind people of that), but that seems
like a much smaller price than having to maintain a patched version of the
upstream sources.

 -Hal
> 
> -eric
> 
> 
> 
> 
> 
> Regards,
> 
> John Criswell
> 
> 
> 
> 
> 
> 
> Thanks!
> 
> 
> -eric
> 
> 
> 
> On Thu, Jun 18, 2015 at 4:47 PM John Criswell < jtcriswel at gmail.com
>
> wrote:
> 
> 
> Dear All,
> 
> Will the LLVM project system (the extension to the build system that
> allows sub-projects to reuse the LLVM Makefiles) be maintained long
> term, or is the slow push to CMake intending to deprecate this
> functionality?
> 
> We used this feature a lot for research projects at UIUC, but if the
> current maintainers of the build system are planning on deprecating
> it,
> I'll have my students avoid using it.
> 
> Regards,
> 
> John Criswell
> 
> --
> John Criswell
> Assistant Professor
> Department of Computer Science, University of Rochester
> http://www.cs.rochester.edu/u/criswell
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
> 
> --
> John Criswell
> Assistant Professor
> Department of Computer Science, University of Rochester
> http://www.cs.rochester.edu/u/criswell
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

On Thu, Jun 18, 2015 at 5:46 PM Hal Finkel <hfinkel at anl.gov> wrote:
> ----- Original Message -----
> > From: "Eric Christopher" <echristo at gmail.com>
> > To: "John Criswell" <jtcriswel at gmail.com>, LLVMdev
at cs.uiuc.edu, "Chris
> Bieneman" <beanz at apple.com>
> > Sent: Thursday, June 18, 2015 7:14:06 PM
> > Subject: Re: [LLVMdev] Long-Term Support for LLVM Projects Extension
to
>      Build System?
> >
> >
> > On Thu, Jun 18, 2015 at 5:07 PM John Criswell < jtcriswel at
gmail.com >
> > wrote:
> >
> >
> > On 6/18/15 6:49 PM, Eric Christopher wrote:
> >
> >
> >
> > Hi John,
> >
> >
> > Long term we don't want to keep the burden of two build systems in
> > tree. CMake is turning out to be the build system we want because of
> > its multi-platform support, etc and as soon as the CMake system can
> > do everything we can do with the autoconf/makefile build I plan on
> > turning down the support for that and removing the code.
> >
> > Thanks, but I wasn't asking about autoconf vs. cmake.
> >
> >
> >
> >
> > Well, you are, but only sorta.
> >
> >
> >
> > I'm wanting to know whether the ability to "plug into"
the LLVM build
> > system by external projects will be maintained. For example,
> > SAFECode doesn't have its own build system; it uses LLVM's,
and it
> > does so without being a "patch" to the LLVM source tree. You
can put
> > the SAFECode source code anywhere you like, use LLVM-style Makefiles
> > in its source code, and build it. All of the PROJ_SRC_ROOT,
> > PROJ_OBJ_ROOT magic in the autoconf build system is what permits
> > that feature to work.
> >
> > Do you intend to keep this functionality when you replace autoconf
> > with cmake, or will all projects that want to use the LLVM build
> > system need to place their source code in the LLVM source tree? To
> > put it another way, do you intend to deprecate the
> > PROJ_SRC_ROOT/PROJ_OBJ_ROOT stuff?
> >
> > While I've enjoyed the PROJ_* magic, I can see why you'd want
to get
> > rid of it. I just want to ensure that you are not planning on
> > replacing it.
> >
> >
> > Right. So this is a "autoconf/makefile build system vs cmake
build
> > system" question as the Makefiles would, theoretically, all go
away
> > at the end. I don't think anyone has plans of replacing this
> > functionality (Chris?), but migrating out of tree projects to hook
> > in via modifying the cmake build locally shouldn't be too bad.
> >
>
> This is also related to the metapoint that the cmake build system
doesn't
> glob for source files (or anything else), unlike the makefile build system.
> This is understandable given the constraints, but we might want to restore
> some of that functionality for the purpose of supporting out-of-tree
> projects.
>
> What I mean specifically, is that, for example, we could have a
> normally-empty directory tools/extensions, and tools/CMakeLists.txt could
> glob for directories in tools/extensions and call
> add_llvm_tool_subdirectory on all of them. It is true that you'd need
to
> re-run cmake whenever you added something there (we could have a README in
> that directory to remind people of that), but that seems like a much
> smaller price than having to maintain a patched version of the upstream
> sources.
>
>
Sure, seems reasonable to me. I'm not working on the cmake transition so
I'm just answering as far as "things I plan on deleting when I can
delete"
:)

-eric

>  -Hal
>
> >
> > -eric
> >
> >
> >
> >
> >
> > Regards,
> >
> > John Criswell
> >
> >
> >
> >
> >
> >
> > Thanks!
> >
> >
> > -eric
> >
> >
> >
> > On Thu, Jun 18, 2015 at 4:47 PM John Criswell < jtcriswel at
gmail.com >
> > wrote:
> >
> >
> > Dear All,
> >
> > Will the LLVM project system (the extension to the build system that
> > allows sub-projects to reuse the LLVM Makefiles) be maintained long
> > term, or is the slow push to CMake intending to deprecate this
> > functionality?
> >
> > We used this feature a lot for research projects at UIUC, but if the
> > current maintainers of the build system are planning on deprecating
> > it,
> > I'll have my students avoid using it.
> >
> > Regards,
> >
> > John Criswell
> >
> > --
> > John Criswell
> > Assistant Professor
> > Department of Computer Science, University of Rochester
> > http://www.cs.rochester.edu/u/criswell
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> >
> >
> > --
> > John Criswell
> > Assistant Professor
> > Department of Computer Science, University of Rochester
> > http://www.cs.rochester.edu/u/criswell
> > _______________________________________________
> > LLVM Developers mailing list
> > LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> >
>
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150619/d6db5615/attachment.html>

----- Original Message -----
> From: "Eric Christopher" <echristo at gmail.com>
> To: "Hal Finkel" <hfinkel at anl.gov>
> Cc: "John Criswell" <jtcriswel at gmail.com>, LLVMdev at
cs.uiuc.edu, "Chris Bieneman" <beanz at apple.com>
> Sent: Thursday, June 18, 2015 7:57:40 PM
> Subject: Re: [LLVMdev] Long-Term Support for LLVM Projects Extension to
Build System?
> 
> 
> On Thu, Jun 18, 2015 at 5:46 PM Hal Finkel < hfinkel at anl.gov >
wrote:
> 
> 
> ----- Original Message -----
> > From: "Eric Christopher" < echristo at gmail.com >
> > To: "John Criswell" < jtcriswel at gmail.com >,
LLVMdev at cs.uiuc.edu ,
> > "Chris Bieneman" < beanz at apple.com >
> > Sent: Thursday, June 18, 2015 7:14:06 PM
> > Subject: Re: [LLVMdev] Long-Term Support for LLVM Projects
> > Extension to Build System?
> > 
> > 
> > On Thu, Jun 18, 2015 at 5:07 PM John Criswell < jtcriswel at
gmail.com
> > >
> > wrote:
> > 
> > 
> > On 6/18/15 6:49 PM, Eric Christopher wrote:
> > 
> > 
> > 
> > Hi John,
> > 
> > 
> > Long term we don't want to keep the burden of two build systems in
> > tree. CMake is turning out to be the build system we want because
> > of
> > its multi-platform support, etc and as soon as the CMake system can
> > do everything we can do with the autoconf/makefile build I plan on
> > turning down the support for that and removing the code.
> > 
> > Thanks, but I wasn't asking about autoconf vs. cmake.
> > 
> > 
> > 
> > 
> > Well, you are, but only sorta.
> > 
> > 
> > 
> > I'm wanting to know whether the ability to "plug into"
the LLVM
> > build
> > system by external projects will be maintained. For example,
> > SAFECode doesn't have its own build system; it uses LLVM's,
and it
> > does so without being a "patch" to the LLVM source tree. You
can
> > put
> > the SAFECode source code anywhere you like, use LLVM-style
> > Makefiles
> > in its source code, and build it. All of the PROJ_SRC_ROOT,
> > PROJ_OBJ_ROOT magic in the autoconf build system is what permits
> > that feature to work.
> > 
> > Do you intend to keep this functionality when you replace autoconf
> > with cmake, or will all projects that want to use the LLVM build
> > system need to place their source code in the LLVM source tree? To
> > put it another way, do you intend to deprecate the
> > PROJ_SRC_ROOT/PROJ_OBJ_ROOT stuff?
> > 
> > While I've enjoyed the PROJ_* magic, I can see why you'd want
to
> > get
> > rid of it. I just want to ensure that you are not planning on
> > replacing it.
> > 
> > 
> > Right. So this is a "autoconf/makefile build system vs cmake
build
> > system" question as the Makefiles would, theoretically, all go
away
> > at the end. I don't think anyone has plans of replacing this
> > functionality (Chris?), but migrating out of tree projects to hook
> > in via modifying the cmake build locally shouldn't be too bad.
> > 
> 
> This is also related to the metapoint that the cmake build system
> doesn't glob for source files (or anything else), unlike the
> makefile build system. This is understandable given the constraints,
> but we might want to restore some of that functionality for the
> purpose of supporting out-of-tree projects.
> 
> What I mean specifically, is that, for example, we could have a
> normally-empty directory tools/extensions, and tools/CMakeLists.txt
> could glob for directories in tools/extensions and call
> add_llvm_tool_subdirectory on all of them. It is true that you'd
> need to re-run cmake whenever you added something there (we could
> have a README in that directory to remind people of that), but that
> seems like a much smaller price than having to maintain a patched
> version of the upstream sources.
> 
> Sure, seems reasonable to me. I'm not working on the cmake transition
> so I'm just answering as far as "things I plan on deleting when I
> can delete" :)
Sounds good ;)

John, I think you're probably the best person to send a patch for this kind
of thing since that way you can make sure the solution actually makes sense for
you.

 -Hal
> 
> 
> -eric
> 
> 
> -Hal
> 
> > 
> > -eric
> > 
> > 
> > 
> > 
> > 
> > Regards,
> > 
> > John Criswell
> > 
> > 
> > 
> > 
> > 
> > 
> > Thanks!
> > 
> > 
> > -eric
> > 
> > 
> > 
> > On Thu, Jun 18, 2015 at 4:47 PM John Criswell < jtcriswel at
gmail.com
> > >
> > wrote:
> > 
> > 
> > Dear All,
> > 
> > Will the LLVM project system (the extension to the build system
> > that
> > allows sub-projects to reuse the LLVM Makefiles) be maintained long
> > term, or is the slow push to CMake intending to deprecate this
> > functionality?
> > 
> > We used this feature a lot for research projects at UIUC, but if
> > the
> > current maintainers of the build system are planning on deprecating
> > it,
> > I'll have my students avoid using it.
> > 
> > Regards,
> > 
> > John Criswell
> > 
> > --
> > John Criswell
> > Assistant Professor
> > Department of Computer Science, University of Rochester
> > http://www.cs.rochester.edu/u/criswell
> > 
> > _______________________________________________
> > LLVM Developers mailing list
> > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> > 
> > 
> > --
> > John Criswell
> > Assistant Professor
> > Department of Computer Science, University of Rochester
> > http://www.cs.rochester.edu/u/criswell
> > _______________________________________________
> > LLVM Developers mailing list
> > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> > 
> 
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
> 
-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

On Thu, Jun 18, 2015 at 5:50 PM Hal Finkel <hfinkel at anl.gov> wrote:
> This is also related to the metapoint that the cmake build system
doesn't
> glob for source files (or anything else), unlike the makefile build system.
> This is understandable given the constraints, but we might want to restore
> some of that functionality for the purpose of supporting out-of-tree
> projects.
>
I actually think these are completely separate. See below.

>
> What I mean specifically, is that, for example, we could have a
> normally-empty directory tools/extensions, and tools/CMakeLists.txt could
> glob for directories in tools/extensions and call
> add_llvm_tool_subdirectory on all of them. It is true that you'd need
to
> re-run cmake whenever you added something there (we could have a README in
> that directory to remind people of that), but that seems like a much
> smaller price than having to maintain a patched version of the upstream
> sources.
>
We already do this in numerous places. See projects/CMakeLists.txt which
does exactly this.

We don't automatically support external projects *in different locations*
through globbing, but that's because that doesn't make any sense.

We can add globbing-based recursion to subproject CMake files anywhere it
is useful.

There is no problem with globbing to add subdirectories IMO because
subdirectories *with new CMake files* clearly have to be introduced and
maintained at the points you run the 'cmake' tool. Only globbing for
*source files* seems problematic to me at all.

-Chandler
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150619/48e26eea/attachment.html>

Dear All,

Thanks for the feedback.

To clarify, this is for LLVM-related projects that are linking against 
the LLVM libraries.

The feature for keeping our source code in arbitrary locations isn't 
something we need.  If we can just drop our source code into 
llvm/projects (or some other designated LLVM subdirectory) and write 
LLVM-esque CMake files, that should be enough for projects that don't 
need to patch the existing LLVM tools.

Regards,

John Criswell


On 6/18/15 9:06 PM, Chandler Carruth wrote:
> On Thu, Jun 18, 2015 at 5:50 PM Hal Finkel <hfinkel at anl.gov 
> <mailto:hfinkel at anl.gov>> wrote:
>
>     This is also related to the metapoint that the cmake build system
>     doesn't glob for source files (or anything else), unlike the
>     makefile build system. This is understandable given the
>     constraints, but we might want to restore some of that
>     functionality for the purpose of supporting out-of-tree projects.
>
>
> I actually think these are completely separate. See below.
>
>
>     What I mean specifically, is that, for example, we could have a
>     normally-empty directory tools/extensions, and
>     tools/CMakeLists.txt could glob for directories in
>     tools/extensions and call add_llvm_tool_subdirectory on all of
>     them. It is true that you'd need to re-run cmake whenever you
>     added something there (we could have a README in that directory to
>     remind people of that), but that seems like a much smaller price
>     than having to maintain a patched version of the upstream sources.
>
>
> We already do this in numerous places. See projects/CMakeLists.txt 
> which does exactly this.
>
> We don't automatically support external projects *in different 
> locations* through globbing, but that's because that doesn't make
any
> sense.
>
> We can add globbing-based recursion to subproject CMake files anywhere 
> it is useful.
>
> There is no problem with globbing to add subdirectories IMO because 
> subdirectories *with new CMake files* clearly have to be introduced 
> and maintained at the points you run the 'cmake' tool. Only
globbing
> for *source files* seems problematic to me at all.
>
> -Chandler
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

-- 
John Criswell
Assistant Professor
Department of Computer Science, University of Rochester
http://www.cs.rochester.edu/u/criswell

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150619/852f01e8/attachment.html>