llvm dev - Jan 2018 - Layering Requirements in the LLVM Coding Style Guide

On Tue, Jan 16, 2018 at 10:15 AM Robinson, Paul <paul.robinson at
sony.com>
wrote:
> I have found layering to be a particularly useful and beneficial model in
> past large software projects.
>
>
>
> Is LLVM's layering actually written down anywhere?  Last time I went
> looking, there was nothing.  If there's no spec, there's no
verifiable
> conformance; you have to guess based on what other files do.
>
Fair point - Google's build system is pretty specific about this & so
we've
got it codified there, and the open source build system has to know some of
this to get the link order right - otherwise LLVM programs couldn't
successfully link (if the libraries weren't placed in the right order on
the link command)

I think the the LLVMBuild.txt files contain the library dependency lists
for the CMake build.

- Dave

> --paulr
>
>
>
> *From:* llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On Behalf Of
*David
> Blaikie via llvm-dev
> *Sent:* Tuesday, January 16, 2018 9:22 AM
> *To:* llvm-dev; Richard Smith; Chandler Carruth; Reid Kleckner
> *Subject:* [llvm-dev] Layering Requirements in the LLVM Coding Style Guide
>
>
>
> Context: I've been looking at experimenting with using Modular Code
> Generation (My talk at last year's LLVM dev meeting
> https://www.youtube.com/watch?v=lYYxDXgbUZ0 is about the best reference
> at the moment) when building the LLVM project, as a good experiment for the
> feature. This can/does enforce a stronger layering invariant than LLVM has
> historically been enforced. So I'm curious to get buy-in and maybe
document
> this if it's something people like the idea of.
>
> I'm starting this discussion here rather than in an actual code review
on
> llvm-commits since it seems like it could do with a bit of a wider
> discussion, but once/if the general direction is agreed on, I'll send a
> patch for review of specific wording for the LLVM Coding Standards.
>
>
> Currently the LLVM Coding Standards
> <https://llvm.org/docs/CodingStandards.html> doesn't say
much/anything
> about layering. 'A Public Header File *is* a Module'
>
<https://llvm.org/docs/CodingStandards.html#a-public-header-file-is-a-module>
section
> talks about modules of functionality, mostly trying to describe why a
> header file should be self contained - but uses anachronistic language
> about modules that doesn't line up with the implicit or explicit
modules
> concepts in use today, I think.
>
> I propose making this wording a bit more explicit, including:
>
> 1) Headers should be standalone (include all their dependencies - this is
> mentioned in the "is a Module" piece, by way of a technique to
help ensure
> this, but not explicit as a goal itself).
>
> 2) Files intended to be included in a particular context (that aren't
> safe/benign to include multiple times, in multiple .cpp files, etc) should
> use a '.inc' or '.def' (.def specifically for those
"define a macro,
> include the header which will reference that macro" style setups we
have in
> a few places).
>
> And the actual layering issue:
> 3) Each library should only include headers or otherwise reference
> entities from libraries it depends on. Including in headers and inline
> functions. A simple/explicit way to put this: every inline function should
> be able to be moved into a .cpp file and the build (with a unix linker -
> one that cannot handle circular library dependencies) should still succeed.
>
>
> This last point is the most interesting - and I hope one that people
> generally find desirable, so it might not be immediately obvious why it may
> be contentious or difficult:
>
> LLVM violates this constraint by using inline functions in headers to
> avoid certain layering constraints that might otherwise cause the build to
> fail. A couple of major examples I've hit are:
>
> TargetSelect.h
>
<http://lists.llvm.org/pipermail/llvm-dev/2017-December/119494.html>and
> similar: This one's especially tricky - the header is part of
libSupport,
> but each function in here depends on a different subset of targets
> (creating a circular dependency) - to call the given function the
> programmer needs to choose the right dependencies to link to or the program
> will not link.
> Clang Diagnostics <https://reviews.llvm.org/D41357> (work in
progress):
> The diagnostics for each component are in their own component directories,
> but are then all included from libClangBasic, a library none of those
> components depends on. (so this isn't so much an inlining case as
#include
> based circular dependency)
>
>
> Generally I'd like to get buy-in that stricter layering is desirable,
and
> that these few cases are at least sub-optimal, if accepted for now.
>
> Happy to go into more details about any of this, examples, etc, but I
> realize this is already a bit long.
> - Dave
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180116/9a0b3678/attachment-0001.html>

Looking at build-procedure files for link-order hints is technically
"written down" but not really human-friendly and not at all what I had
in mind. ☺
I get that writing it down on a doc page will have the usual bit-rot problems,
but if you want to tell developers (especially newer developers) "get the
layering right" you really need to point to a place that says what the
layering *is*.  Maybe you were agreeing to do that, but I'm not sure.
--paulr
I have found layering to be a particularly useful and beneficial model in past
large software projects.
Is LLVM's layering actually written down anywhere?  Last time I went
looking, there was nothing.  If there's no spec, there's no verifiable
conformance; you have to guess based on what other files do.

Fair point - Google's build system is pretty specific about this & so
we've got it codified there, and the open source build system has to know
some of this to get the link order right - otherwise LLVM programs couldn't
successfully link (if the libraries weren't placed in the right order on the
link command)

I think the the LLVMBuild.txt files contain the library dependency lists for the
CMake build.

- Dave

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180117/6340cee1/attachment-0001.html>

On Wed, Jan 17, 2018 at 11:25 AM Robinson, Paul <paul.robinson at
sony.com>
wrote:
> Looking at build-procedure files for link-order hints is technically
> "written down" but not really human-friendly and not at all what
I had in
> mind. J
>
> I get that writing it down on a doc page will have the usual bit-rot
> problems, but if you want to tell developers (especially newer developers)
> "get the layering right" you really need to point to a place that
says what
> the layering *is*.  Maybe you were agreeing to do that, but I'm not
sure.
>
I wasn't planning on writing it down anymore than it's already
necessarily
enshrined in the build system. I'm not expecting to yell at/complain to
people who violate it unknowingly (Apple/Windows developers in general
don't get layering checked today because their linkers do resolve circular
dependencies - but they get failures on Linux buildbots and fix them when
they arise) - but to point out "oh, hey, maybe you didn't notice but
this
introduces a layering violation - please fix it".

I'm not sure what makes LLVMBuild.txt not human-friendly - they're very
terse text files, contain little other than a list of dependencies. They're
probably easier to read/maintain than CMakeLists.txt which have to be
updated whenever a new source file is added. These lists have to be
updated/maintained when new libraries or dependencies are introduced &
yeah, for the most part we don't have to think about them - and most
changes won't impact layering/violate layering constraints, but then rarely
we will hit these things & check or update the layering, etc.

- Dave

> --paulr
>
> I have found layering to be a particularly useful and beneficial model in
> past large software projects.
>
> Is LLVM's layering actually written down anywhere?  Last time I went
> looking, there was nothing.  If there's no spec, there's no
verifiable
> conformance; you have to guess based on what other files do.
>
>
> Fair point - Google's build system is pretty specific about this &
so
> we've got it codified there, and the open source build system has to
know
> some of this to get the link order right - otherwise LLVM programs
couldn't
> successfully link (if the libraries weren't placed in the right order
on
> the link command)
>
> I think the the LLVMBuild.txt files contain the library dependency lists
> for the CMake build.
>
> - Dave
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180117/8b74d72c/attachment.html>