llvm dev - Jan 2021 - [RFC] Modernize CMake LLVM "Components"/libLLVM Facility

Hi folks, happy new year!

*Proposal:*

   - See comments at the top of LLVMComponents.cmake
  
<https://github.com/stellaraccident/llvm-project/blob/newcomponents/llvm/cmake/modules/LLVMComponents.cmake>
   in my fork
   <https://github.com/stellaraccident/llvm-project/tree/newcomponents>.
   - Draft phab: https://reviews.llvm.org/D94000


*Background:*
As I've been working on NPCOMP <https://github.com/llvm/mlir-npcomp>
trying
to come up with a release flow for MLIR derived Python projects (see
py-mlir-release <https://github.com/stellaraccident/mlir-py-release>),
I've
repeatedly run into issues with how the LLVM build system generates shared
libraries. While the problems have been varied, I pattern match most of
them to a certain "pragmatic" nature to how components/libLLVM/libMLIR
have
come to be: in my experience, you can fix most individual dynamic linkage
issues with another work-around, but the need for this tends to be rooted
in a lack of definition and structure to the libraries themselves, causing
various kinds of problems and scenarios that don't arise if developed to
stricter standards. (This isn't a knock on anyone -- I know how these
things tend to grow. My main observation is that I think we have outgrown
the ad-hoc nature of shared libraries in the LLVM build now).

I think I'm hitting this because reasonable Python projects and releases
pre-supposes a robust dynamic linkage story. Also, I use Windows and am
very aware that LLVM basically does not support dynamic linking on Windows
-- and cannot without more structure (and in my experience, this structure
would also benefit the robustness of dynamic linking on the others).

Several of us got together to discuss this in November
<https://llvm.discourse.group/t/meeting-notes-mlir-build-install-and-shared-libraries/2257>.
We generally agreed that BUILD_SHARED_LIBS was closer to what we wanted vs
libLLVM/libMLIR, but the result is really only factored for development
(i.e. not every add_library should result in a shared object -- the shared
library surface should mirror public interface boundaries and add_library
mirrors private boundaries). The primary difference between the two is:

   - BUILD_SHARED_LIBS preserves the invariant that every translation unit
   will be "homed" in one library at link time (either .so/.dll or .a)
and the
   system will never try to link together shared and static dependencies of
   the same thing (which is what libLLVM/libMLIR do today). It turns out that
   this is merely a good idea on most platforms but is the core requirement on
   native Windows (leaving out mingw, which uses some clever and dirty tricks
   to try to blend the worlds).
   - LLVM_BUILD_LLVM_DYLIB treats libLLVM.so as a "bucket" to throw
things
   that might benefit from shared linkage, but end binaries end up also
   needing to link against the static libraries in case if what you want
isn't
   in libLLVM.so. When this is done just right, it can work (on Unix) but it
   is very fragile and prone to multiple definition and other linkage issues
   that can be extremely hard to track down.

*What I did:*

   1. Well, first, I tried looking the other way for a few months and
   hoping someone else would fix it :)
   2. When I started trying to generalize some of the shared library
   handling for MLIR and NPCOMP, I noted that the LLVM_LINK_COMPONENTS (as in
   named groups of things) are in the right direction of having a structure to
   the libraries, and I found that I could actually rebase all of what the
   LLVM_LINK_COMPONENTS was trying to do on the same facility, relegating the
   existing LLVM_LINK_COMPONENTS to a name normalization layer on top of a
   more generic "LLVM Components" facility that enforces stricter
layering and
   more control than the old libLLVM.so facility did.
   3. I rewrote it twice to progressively more modern CMake and was able to
   eliminate all of the ad-hoc dependency tracking in favor of
   straight-forward use of INTERFACE libraries and $<TARGET_PROPERTY>
   generator expressions for selecting static or dynamic component trees based
   on global flags and the presence (or absence) of per-executable
   LLVM_LINK_STATIC properties
      1. Note that since this is rooted only in CMake features and not LLVM
      macros, out of tree, non-LLVM projects should be able to depend on LLVM
      components in their own targets.
   4. I hacked up AddLLVM/LLVM-Build/LLVM-Config to (mostly) use the new
   facility (leaving out a few things that can be fixed but aren't
conceptual
   issues), applied a bunch of fixes to the tree that were revealed by
   stricter checks and got all related tests passing for LLVM and MLIR (on X86
   -- some mechanical changes need to be made to other targets) for both
   dynamic and static builds.

*What I'd like to do:*

   - Get some consensus that we'd like to improve things in this area and
   that the approach I'm taking makes sense. I can do a lot of the work, but
I
   don't want to waste my time, and this stuff is fragile if we keep it in
an
   intermediate state for too long (I'm already paying this price
downstream).
   - Land LLVMComponents.cmake
  
<https://github.com/stellaraccident/llvm-project/blob/newcomponents/llvm/cmake/modules/LLVMComponents.cmake>
   as the basis of the new facility.
   - Finish implementing the "Redirection" feature that would allow us
to
   emulate an aggregate libLLVM as it is today.
   - Start pre-staging the various stricter constraints to the build tree
   that will be needed to swap AddLLVM to use the new facility.
   - Rewrite component-related AddLLVM/LLVM-Build/LLVM-Config bits in a
   more principled way to use the new facility (or remove features entirely
   that are no longer needed) -- what I did in the above patch was just a
   minimal amount of working around for a POC.
   - Agree on whether we should try to have the two co-exist for a time or
   do a more clean break with the old.
   - Start applying the facility to downstream projects like MLIR and
   NPCOMP.

*What I would need:*

   - Help, testing and expertise. I am reasonably confident in my
   understanding of how to make shared libraries work and how to use CMake,
   but the legacy in LLVM here is deep -- I likely pattern matched some old
   features as no longer needed when they actually are (I am not clear at all
   on how much of LLVM-Config is still relevant).
   - Pointers to who the stakeholders are that I should be coordinating
   with.

Comments?

Thanks!
- Stella
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210103/314f51a1/attachment.html>

On Sun, Jan 3, 2021 at 1:50 PM Stella Laurenzo via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi folks, happy new year!
>
> *Proposal:*
>
>    - See comments at the top of LLVMComponents.cmake
>   
<https://github.com/stellaraccident/llvm-project/blob/newcomponents/llvm/cmake/modules/LLVMComponents.cmake>
>    in my fork
>   
<https://github.com/stellaraccident/llvm-project/tree/newcomponents>.
>    - Draft phab: https://reviews.llvm.org/D94000
>
>
> *Background:*
> As I've been working on NPCOMP
<https://github.com/llvm/mlir-npcomp> trying
> to come up with a release flow for MLIR derived Python projects (see
> py-mlir-release
<https://github.com/stellaraccident/mlir-py-release>),
> I've repeatedly run into issues with how the LLVM build system
generates
> shared libraries. While the problems have been varied, I pattern match most
> of them to a certain "pragmatic" nature to how
components/libLLVM/libMLIR
> have come to be: in my experience, you can fix most individual dynamic
> linkage issues with another work-around, but the need for this tends to be
> rooted in a lack of definition and structure to the libraries themselves,
> causing various kinds of problems and scenarios that don't arise if
> developed to stricter standards. (This isn't a knock on anyone -- I
know
> how these things tend to grow. My main observation is that I think we have
> outgrown the ad-hoc nature of shared libraries in the LLVM build now).
>
> I think I'm hitting this because reasonable Python projects and
releases
> pre-supposes a robust dynamic linkage story. Also, I use Windows and am
> very aware that LLVM basically does not support dynamic linking on Windows
> -- and cannot without more structure (and in my experience, this structure
> would also benefit the robustness of dynamic linking on the others).
>
> Several of us got together to discuss this in November
>
<https://llvm.discourse.group/t/meeting-notes-mlir-build-install-and-shared-libraries/2257>.
> We generally agreed that BUILD_SHARED_LIBS was closer to what we wanted vs
> libLLVM/libMLIR, but the result is really only factored for development
> (i.e. not every add_library should result in a shared object -- the shared
> library surface should mirror public interface boundaries and add_library
> mirrors private boundaries). The primary difference between the two is:
>
>    - BUILD_SHARED_LIBS preserves the invariant that every translation
>    unit will be "homed" in one library at link time (either
.so/.dll or .a)
>    and the system will never try to link together shared and static
>    dependencies of the same thing (which is what libLLVM/libMLIR do today).
It
>    turns out that this is merely a good idea on most platforms but is the
core
>    requirement on native Windows (leaving out mingw, which uses some clever
>    and dirty tricks to try to blend the worlds).
>    - LLVM_BUILD_LLVM_DYLIB treats libLLVM.so as a "bucket" to
throw
>    things that might benefit from shared linkage, but end binaries end up
also
>    needing to link against the static libraries in case if what you want
isn't
>    in libLLVM.so. When this is done just right, it can work (on Unix) but
it
>    is very fragile and prone to multiple definition and other linkage
issues
>    that can be extremely hard to track down.
>
> *What I did:*
>
>    1. Well, first, I tried looking the other way for a few months and
>    hoping someone else would fix it :)
>    2. When I started trying to generalize some of the shared library
>    handling for MLIR and NPCOMP, I noted that the LLVM_LINK_COMPONENTS (as
in
>    named groups of things) are in the right direction of having a structure
to
>    the libraries, and I found that I could actually rebase all of what the
>    LLVM_LINK_COMPONENTS was trying to do on the same facility, relegating
the
>    existing LLVM_LINK_COMPONENTS to a name normalization layer on top of a
>    more generic "LLVM Components" facility that enforces stricter
layering and
>    more control than the old libLLVM.so facility did.
>    3. I rewrote it twice to progressively more modern CMake and was able
>    to eliminate all of the ad-hoc dependency tracking in favor of
>    straight-forward use of INTERFACE libraries and $<TARGET_PROPERTY>
>    generator expressions for selecting static or dynamic component trees
based
>    on global flags and the presence (or absence) of per-executable
>    LLVM_LINK_STATIC properties
>       1. Note that since this is rooted only in CMake features and not
>       LLVM macros, out of tree, non-LLVM projects should be able to depend
on
>       LLVM components in their own targets.
>    4. I hacked up AddLLVM/LLVM-Build/LLVM-Config to (mostly) use the new
>    facility (leaving out a few things that can be fixed but aren't
conceptual
>    issues), applied a bunch of fixes to the tree that were revealed by
>    stricter checks and got all related tests passing for LLVM and MLIR (on
X86
>    -- some mechanical changes need to be made to other targets) for both
>    dynamic and static builds.
>
> *What I'd like to do:*
>
>    - Get some consensus that we'd like to improve things in this area
and
>    that the approach I'm taking makes sense. I can do a lot of the
work, but I
>    don't want to waste my time, and this stuff is fragile if we keep it
in an
>    intermediate state for too long (I'm already paying this price
downstream).
>    - Land LLVMComponents.cmake
>   
<https://github.com/stellaraccident/llvm-project/blob/newcomponents/llvm/cmake/modules/LLVMComponents.cmake>
>    as the basis of the new facility.
>    - Finish implementing the "Redirection" feature that would
allow us to
>    emulate an aggregate libLLVM as it is today.
>    - Start pre-staging the various stricter constraints to the build tree
>    that will be needed to swap AddLLVM to use the new facility.
>    - Rewrite component-related AddLLVM/LLVM-Build/LLVM-Config bits in a
>    more principled way to use the new facility (or remove features entirely
>    that are no longer needed) -- what I did in the above patch was just a
>    minimal amount of working around for a POC.
>    - Agree on whether we should try to have the two co-exist for a time
>    or do a more clean break with the old.
>    - Start applying the facility to downstream projects like MLIR and
>    NPCOMP.
>
> *What I would need:*
>
>    - Help, testing and expertise. I am reasonably confident in my
>    understanding of how to make shared libraries work and how to use CMake,
>    but the legacy in LLVM here is deep -- I likely pattern matched some old
>    features as no longer needed when they actually are (I am not clear at
all
>    on how much of LLVM-Config is still relevant).
>    - Pointers to who the stakeholders are that I should be coordinating
>    with.
>
> Comments?
>
Looks great! In particular it is interesting to see how more modern CMake
features could replace some of the custom-LLVM CMake macros that are likely
almost a decade old now.

One thing I wonder about trying to see BUILD_SHARED_LIBS as some desirable
for a production environment: I seem to remember that there were
non-trivial performance regression when using many .so instead of a single
libLLVM.so (even a single libLLVM.so was showing a measurable performance
impact for clang IIRC).

-- 
Mehdi
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210103/424875ca/attachment.html>

I'm curious if you've prototyped how this would affect llvm-dependent
projects?  I'm thinking in particular about CIRCT and NPCOMP where properly
dealing with libMLIR and libCIRCT requires duplicating alot of
boilerplate.  Would this simplify things?  Generally, I like the idea and
would like to see this move forward.  (But I'm hardly one of those people
with deep experience in the legacy of LLVM here...)

Steve

On Sun, Jan 3, 2021 at 1:50 PM Stella Laurenzo via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi folks, happy new year!
>
> *Proposal:*
>
>    - See comments at the top of LLVMComponents.cmake
>   
<https://github.com/stellaraccident/llvm-project/blob/newcomponents/llvm/cmake/modules/LLVMComponents.cmake>
>    in my fork
>   
<https://github.com/stellaraccident/llvm-project/tree/newcomponents>.
>    - Draft phab: https://reviews.llvm.org/D94000
>
>
> *Background:*
> As I've been working on NPCOMP
<https://github.com/llvm/mlir-npcomp> trying
> to come up with a release flow for MLIR derived Python projects (see
> py-mlir-release
<https://github.com/stellaraccident/mlir-py-release>),
> I've repeatedly run into issues with how the LLVM build system
generates
> shared libraries. While the problems have been varied, I pattern match most
> of them to a certain "pragmatic" nature to how
components/libLLVM/libMLIR
> have come to be: in my experience, you can fix most individual dynamic
> linkage issues with another work-around, but the need for this tends to be
> rooted in a lack of definition and structure to the libraries themselves,
> causing various kinds of problems and scenarios that don't arise if
> developed to stricter standards. (This isn't a knock on anyone -- I
know
> how these things tend to grow. My main observation is that I think we have
> outgrown the ad-hoc nature of shared libraries in the LLVM build now).
>
> I think I'm hitting this because reasonable Python projects and
releases
> pre-supposes a robust dynamic linkage story. Also, I use Windows and am
> very aware that LLVM basically does not support dynamic linking on Windows
> -- and cannot without more structure (and in my experience, this structure
> would also benefit the robustness of dynamic linking on the others).
>
> Several of us got together to discuss this in November
>
<https://llvm.discourse.group/t/meeting-notes-mlir-build-install-and-shared-libraries/2257>.
> We generally agreed that BUILD_SHARED_LIBS was closer to what we wanted vs
> libLLVM/libMLIR, but the result is really only factored for development
> (i.e. not every add_library should result in a shared object -- the shared
> library surface should mirror public interface boundaries and add_library
> mirrors private boundaries). The primary difference between the two is:
>
>    - BUILD_SHARED_LIBS preserves the invariant that every translation
>    unit will be "homed" in one library at link time (either
.so/.dll or .a)
>    and the system will never try to link together shared and static
>    dependencies of the same thing (which is what libLLVM/libMLIR do today).
It
>    turns out that this is merely a good idea on most platforms but is the
core
>    requirement on native Windows (leaving out mingw, which uses some clever
>    and dirty tricks to try to blend the worlds).
>    - LLVM_BUILD_LLVM_DYLIB treats libLLVM.so as a "bucket" to
throw
>    things that might benefit from shared linkage, but end binaries end up
also
>    needing to link against the static libraries in case if what you want
isn't
>    in libLLVM.so. When this is done just right, it can work (on Unix) but
it
>    is very fragile and prone to multiple definition and other linkage
issues
>    that can be extremely hard to track down.
>
> *What I did:*
>
>    1. Well, first, I tried looking the other way for a few months and
>    hoping someone else would fix it :)
>    2. When I started trying to generalize some of the shared library
>    handling for MLIR and NPCOMP, I noted that the LLVM_LINK_COMPONENTS (as
in
>    named groups of things) are in the right direction of having a structure
to
>    the libraries, and I found that I could actually rebase all of what the
>    LLVM_LINK_COMPONENTS was trying to do on the same facility, relegating
the
>    existing LLVM_LINK_COMPONENTS to a name normalization layer on top of a
>    more generic "LLVM Components" facility that enforces stricter
layering and
>    more control than the old libLLVM.so facility did.
>    3. I rewrote it twice to progressively more modern CMake and was able
>    to eliminate all of the ad-hoc dependency tracking in favor of
>    straight-forward use of INTERFACE libraries and $<TARGET_PROPERTY>
>    generator expressions for selecting static or dynamic component trees
based
>    on global flags and the presence (or absence) of per-executable
>    LLVM_LINK_STATIC properties
>       1. Note that since this is rooted only in CMake features and not
>       LLVM macros, out of tree, non-LLVM projects should be able to depend
on
>       LLVM components in their own targets.
>    4. I hacked up AddLLVM/LLVM-Build/LLVM-Config to (mostly) use the new
>    facility (leaving out a few things that can be fixed but aren't
conceptual
>    issues), applied a bunch of fixes to the tree that were revealed by
>    stricter checks and got all related tests passing for LLVM and MLIR (on
X86
>    -- some mechanical changes need to be made to other targets) for both
>    dynamic and static builds.
>
> *What I'd like to do:*
>
>    - Get some consensus that we'd like to improve things in this area
and
>    that the approach I'm taking makes sense. I can do a lot of the
work, but I
>    don't want to waste my time, and this stuff is fragile if we keep it
in an
>    intermediate state for too long (I'm already paying this price
downstream).
>    - Land LLVMComponents.cmake
>   
<https://github.com/stellaraccident/llvm-project/blob/newcomponents/llvm/cmake/modules/LLVMComponents.cmake>
>    as the basis of the new facility.
>    - Finish implementing the "Redirection" feature that would
allow us to
>    emulate an aggregate libLLVM as it is today.
>    - Start pre-staging the various stricter constraints to the build tree
>    that will be needed to swap AddLLVM to use the new facility.
>    - Rewrite component-related AddLLVM/LLVM-Build/LLVM-Config bits in a
>    more principled way to use the new facility (or remove features entirely
>    that are no longer needed) -- what I did in the above patch was just a
>    minimal amount of working around for a POC.
>    - Agree on whether we should try to have the two co-exist for a time
>    or do a more clean break with the old.
>    - Start applying the facility to downstream projects like MLIR and
>    NPCOMP.
>
> *What I would need:*
>
>    - Help, testing and expertise. I am reasonably confident in my
>    understanding of how to make shared libraries work and how to use CMake,
>    but the legacy in LLVM here is deep -- I likely pattern matched some old
>    features as no longer needed when they actually are (I am not clear at
all
>    on how much of LLVM-Config is still relevant).
>    - Pointers to who the stakeholders are that I should be coordinating
>    with.
>
> Comments?
>
> Thanks!
> - Stella
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210103/b6facb10/attachment-0001.html>

On 1/3/21 1:49 PM, Stella Laurenzo via llvm-dev wrote:
> Hi folks, happy new year!
> 
> *Proposal:*
> 
>   * See comments at the top of LLVMComponents.cmake
>    
<https://github.com/stellaraccident/llvm-project/blob/newcomponents/llvm/cmake/modules/LLVMComponents.cmake>
>     in my fork
>    
<https://github.com/stellaraccident/llvm-project/tree/newcomponents>.
>   * Draft phab: https://reviews.llvm.org/D94000
> 
> 
> *Background:*
> As I've been working on NPCOMP 
> <https://github.com/llvm/mlir-npcomp> trying to come up with a
release
> flow for MLIR derived Python projects (see py-mlir-release 
> <https://github.com/stellaraccident/mlir-py-release>), I've
repeatedly
> run into issues with how the LLVM build system generates shared 
> libraries. While the problems have been varied, I pattern match most of 
> them to a certain "pragmatic" nature to how
components/libLLVM/libMLIR
> have come to be: in my experience, you can fix most individual dynamic 
> linkage issues with another work-around, but the need for this tends to 
> be rooted in a lack of definition and structure to the libraries 
> themselves, causing various kinds of problems and scenarios that don't 
> arise if developed to stricter standards. (This isn't a knock on anyone
> -- I know how these things tend to grow. My main observation is that I 
> think we have outgrown the ad-hoc nature of shared libraries in the LLVM 
> build now).
> 
> I think I'm hitting this because reasonable Python projects and
releases
> pre-supposes a robust dynamic linkage story. Also, I use Windows and am 
> very aware that LLVM basically does not support dynamic linking on 
> Windows -- and cannot without more structure (and in my experience, this 
> structure would also benefit the robustness of dynamic linking on the 
> others).
> 
> Several of us got together to discuss this in November 
>
<https://llvm.discourse.group/t/meeting-notes-mlir-build-install-and-shared-libraries/2257>.
> We generally agreed that BUILD_SHARED_LIBS was closer to what we wanted 
> vs libLLVM/libMLIR, but the result is really only factored for 
> development (i.e. not every add_library should result in a shared object 
> -- the shared library surface should mirror public interface boundaries 
> and add_library mirrors private boundaries). The primary difference 
> between the two is:
> 
>   * BUILD_SHARED_LIBS preserves the invariant that every translation
>     unit will be "homed" in one library at link time (either
.so/.dll or
>     .a) and the system will never try to link together shared and static
>     dependencies of the same thing (which is what libLLVM/libMLIR do
>     today). It turns out that this is merely a good idea on most
>     platforms but is the core requirement on native Windows (leaving out
>     mingw, which uses some clever and dirty tricks to try to blend the
>     worlds).
>   * LLVM_BUILD_LLVM_DYLIB treats libLLVM.so as a "bucket" to
throw
>     things that might benefit from shared linkage, but end binaries end
>     up also needing to link against the static libraries in case if what
>     you want isn't in libLLVM.so. When this is done just right, it can
>     work (on Unix) but it is very fragile and prone to multiple
>     definition and other linkage issues that can be extremely hard to
>     track down.
> 
> *What I did:*
> 
>  1. Well, first, I tried looking the other way for a few months and
>     hoping someone else would fix it :)
>  2. When I started trying to generalize some of the shared library
>     handling for MLIR and NPCOMP, I noted that the LLVM_LINK_COMPONENTS
>     (as in named groups of things) are in the right direction of having
>     a structure to the libraries, and I found that I could actually
>     rebase all of what the LLVM_LINK_COMPONENTS was trying to do on the
>     same facility, relegating the existing LLVM_LINK_COMPONENTS to a
>     name normalization layer on top of a more generic "LLVM
Components"
>     facility that enforces stricter layering and more control than the
>     old libLLVM.so facility did.
>  3. I rewrote it twice to progressively more modern CMake and was able
>     to eliminate all of the ad-hoc dependency tracking in favor of
>     straight-forward use of INTERFACE libraries and
$<TARGET_PROPERTY>
>     generator expressions for selecting static or dynamic component
>     trees based on global flags and the presence (or absence) of
>     per-executable LLVM_LINK_STATIC properties
>      1. Note that since this is rooted only in CMake features and not
>         LLVM macros, out of tree, non-LLVM projects should be able to
>         depend on LLVM components in their own targets.
>  4. I hacked up AddLLVM/LLVM-Build/LLVM-Config to (mostly) use the new
>     facility (leaving out a few things that can be fixed but aren't
>     conceptual issues), applied a bunch of fixes to the tree that were
>     revealed by stricter checks and got all related tests passing for
>     LLVM and MLIR (on X86 -- some mechanical changes need to be made to
>     other targets) for both dynamic and static builds.
> 
> *What I'd like to do:*
> 
>   * Get some consensus that we'd like to improve things in this area
and
>     that the approach I'm taking makes sense. I can do a lot of the
>     work, but I don't want to waste my time, and this stuff is fragile
>     if we keep it in an intermediate state for too long (I'm already
>     paying this price downstream).
>   * Land LLVMComponents.cmake
>    
<https://github.com/stellaraccident/llvm-project/blob/newcomponents/llvm/cmake/modules/LLVMComponents.cmake>
>     as the basis of the new facility.
Do you have a proposed list of components yet for LLVM?
>   * Finish implementing the "Redirection" feature that would
allow us to
>     emulate an aggregate libLLVM as it is today.
>   * Start pre-staging the various stricter constraints to the build tree
>     that will be needed to swap AddLLVM to use the new facility.
>   * Rewrite component-related AddLLVM/LLVM-Build/LLVM-Config bits in a
>     more principled way to use the new facility (or remove features
>     entirely that are no longer needed) -- what I did in the above patch
>     was just a minimal amount of working around for a POC.
>   * Agree on whether we should try to have the two co-exist for a time
>     or do a more clean break with the old.
>   * Start applying the facility to downstream projects like MLIR and
NPCOMP.
> 
It sounds like what you are proposing is BUILD_SHARED_LIBS=ON but with 
fewer total libraries, is this an accurate summary?

I would prefer for any large change like this that we do not add any net 
new configuration options (meaning if we add a new option we should 
remove an old one)to LLVM as we already have too many.  Would this be 
able to replace BUILD_SHARED_LIBS=ON?

- Tom
> *What I would need:*
> 
>   * Help, testing and expertise. I am reasonably confident in my
>     understanding of how to make shared libraries work and how to use
>     CMake, but the legacy in LLVM here is deep -- I likely pattern
>     matched some old features as no longer needed when they actually are
>     (I am not clear at all on how much of LLVM-Config is still relevant).
>   * Pointers to who the stakeholders are that I should be coordinating
with.
> 
> Comments?
> 
> Thanks!
> - Stella
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

Thank you for the proposal. Would you also consider unifying the
library handling of LLVM and clang (and potentially other subprojects)
as well? I found the differences always annoying, e.g. there is no
equivalent to llvm-config in clang, or clangXYZ/MLIRxyz  libraries
creating object libraries under some conditions (e.g. to create static
as well as shared libraries in the same build), but LLVM itself not
doing that.

Michael

Am So., 3. Jan. 2021 um 15:50 Uhr schrieb Stella Laurenzo via llvm-dev
<llvm-dev at lists.llvm.org>:
>
> Hi folks, happy new year!
>
> Proposal:
>
> See comments at the top of LLVMComponents.cmake in my fork.
> Draft phab: https://reviews.llvm.org/D94000
>
>
> Background:
> As I've been working on NPCOMP trying to come up with a release flow
for MLIR derived Python projects (see py-mlir-release), I've repeatedly run
into issues with how the LLVM build system generates shared libraries. While the
problems have been varied, I pattern match most of them to a certain
"pragmatic" nature to how components/libLLVM/libMLIR have come to be:
in my experience, you can fix most individual dynamic linkage issues with
another work-around, but the need for this tends to be rooted in a lack of
definition and structure to the libraries themselves, causing various kinds of
problems and scenarios that don't arise if developed to stricter standards.
(This isn't a knock on anyone -- I know how these things tend to grow. My
main observation is that I think we have outgrown the ad-hoc nature of shared
libraries in the LLVM build now).
>
> I think I'm hitting this because reasonable Python projects and
releases pre-supposes a robust dynamic linkage story. Also, I use Windows and am
very aware that LLVM basically does not support dynamic linking on Windows --
and cannot without more structure (and in my experience, this structure would
also benefit the robustness of dynamic linking on the others).
>
> Several of us got together to discuss this in November. We generally agreed
that BUILD_SHARED_LIBS was closer to what we wanted vs libLLVM/libMLIR, but the
result is really only factored for development (i.e. not every add_library
should result in a shared object -- the shared library surface should mirror
public interface boundaries and add_library mirrors private boundaries). The
primary difference between the two is:
>
> BUILD_SHARED_LIBS preserves the invariant that every translation unit will
be "homed" in one library at link time (either .so/.dll or .a) and the
system will never try to link together shared and static dependencies of the
same thing (which is what libLLVM/libMLIR do today). It turns out that this is
merely a good idea on most platforms but is the core requirement on native
Windows (leaving out mingw, which uses some clever and dirty tricks to try to
blend the worlds).
> LLVM_BUILD_LLVM_DYLIB treats libLLVM.so as a "bucket" to throw
things that might benefit from shared linkage, but end binaries end up also
needing to link against the static libraries in case if what you want isn't
in libLLVM.so. When this is done just right, it can work (on Unix) but it is
very fragile and prone to multiple definition and other linkage issues that can
be extremely hard to track down.
>
> What I did:
>
> Well, first, I tried looking the other way for a few months and hoping
someone else would fix it :)
> When I started trying to generalize some of the shared library handling for
MLIR and NPCOMP, I noted that the LLVM_LINK_COMPONENTS (as in named groups of
things) are in the right direction of having a structure to the libraries, and I
found that I could actually rebase all of what the LLVM_LINK_COMPONENTS was
trying to do on the same facility, relegating the existing LLVM_LINK_COMPONENTS
to a name normalization layer on top of a more generic "LLVM
Components" facility that enforces stricter layering and more control than
the old libLLVM.so facility did.
> I rewrote it twice to progressively more modern CMake and was able to
eliminate all of the ad-hoc dependency tracking in favor of straight-forward use
of INTERFACE libraries and $<TARGET_PROPERTY> generator expressions for
selecting static or dynamic component trees based on global flags and the
presence (or absence) of per-executable LLVM_LINK_STATIC properties
>
> Note that since this is rooted only in CMake features and not LLVM macros,
out of tree, non-LLVM projects should be able to depend on LLVM components in
their own targets.
>
> I hacked up AddLLVM/LLVM-Build/LLVM-Config to (mostly) use the new facility
(leaving out a few things that can be fixed but aren't conceptual issues),
applied a bunch of fixes to the tree that were revealed by stricter checks and
got all related tests passing for LLVM and MLIR (on X86 -- some mechanical
changes need to be made to other targets) for both dynamic and static builds.
>
> What I'd like to do:
>
> Get some consensus that we'd like to improve things in this area and
that the approach I'm taking makes sense. I can do a lot of the work, but I
don't want to waste my time, and this stuff is fragile if we keep it in an
intermediate state for too long (I'm already paying this price downstream).
> Land LLVMComponents.cmake as the basis of the new facility.
> Finish implementing the "Redirection" feature that would allow us
to emulate an aggregate libLLVM as it is today.
> Start pre-staging the various stricter constraints to the build tree that
will be needed to swap AddLLVM to use the new facility.
> Rewrite component-related AddLLVM/LLVM-Build/LLVM-Config bits in a more
principled way to use the new facility (or remove features entirely that are no
longer needed) -- what I did in the above patch was just a minimal amount of
working around for a POC.
> Agree on whether we should try to have the two co-exist for a time or do a
more clean break with the old.
> Start applying the facility to downstream projects like MLIR and NPCOMP.
>
> What I would need:
>
> Help, testing and expertise. I am reasonably confident in my understanding
of how to make shared libraries work and how to use CMake, but the legacy in
LLVM here is deep -- I likely pattern matched some old features as no longer
needed when they actually are (I am not clear at all on how much of LLVM-Config
is still relevant).
> Pointers to who the stakeholders are that I should be coordinating with.
>
> Comments?
>
> Thanks!
> - Stella
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev