llvm dev - Aug 2020 - Organization of LLVM utilities

Could you expand on this a bit? I see from my build directory that
llvm-tblgen.exe was built before the Target/* .inc files. It sounds like
you're saying that something has to be built between the .exe and the .inc
files.

At 8/4/2020 02:44 PM, Chris Lattner wrote:

>> On Aug 4, 2020, at 11:31 AM, Paul C. Anagnostopoulos via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>> 
>> I was confused by the lack of TableGen backends in the
llvm/lib/tablegen directory until I stumbled upon the llvm/utils/tablegen
directory. Could someone explain why TableGen is divided into these two
directories?
>
>The mechanical reason is that llvm/utils builds before llvm/libs.  The basic
TableGen support is required to generate .inc files in the libs directory.
>
>-Chris

----------------------------------------------------------------
Windfall               Paul C. Anagnostopoulos 
      ----------------------------------------------------------
   Software            978 369-0839
                             www.windfall.com
----------------------------------------------------------------
My life has been filled with calamities,
some of which actually happened.
---Mark Twain 

Guga 'mzimba, sala 'nhliziyo

It may be confusing but it is not the case that “llvm/lib” contains all the
libraries, and “llvm/tools” contains all the tools.  At a course grain, the main
llvm repo builds in four phases:

1) llvm/utils.  -> this builds some executables like tablegen
2) llvm/lib.     -> This is all libraries, shouldn’t include executables.
3) llvm/tools. -> Generally executables, also some libraries that are tool
specific.
4) tests..        -> Things that depend on the above.

In theory, we could have a much finer grain build system that allows tracking
exactly which tests depend on which executables, which executables depend on
which libraries, etc, but this hasn’t been done.  In theory this could help the
build cycle if you’re running a single “llvm-mc” like test, for example.

-Chris

> On Aug 4, 2020, at 12:06 PM, Paul C. Anagnostopoulos <paul at
windfall.com> wrote:
> 
> Could you expand on this a bit? I see from my build directory that
llvm-tblgen.exe was built before the Target/* .inc files. It sounds like
you're saying that something has to be built between the .exe and the .inc
files.
> 
> At 8/4/2020 02:44 PM, Chris Lattner wrote:
> 
> 
>>> On Aug 4, 2020, at 11:31 AM, Paul C. Anagnostopoulos via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>>> 
>>> I was confused by the lack of TableGen backends in the
llvm/lib/tablegen directory until I stumbled upon the llvm/utils/tablegen
directory. Could someone explain why TableGen is divided into these two
directories?
>> 
>> The mechanical reason is that llvm/utils builds before llvm/libs.  The
basic TableGen support is required to generate .inc files in the libs directory.
>> 
>> -Chris
> 
> 
> ----------------------------------------------------------------
> Windfall               Paul C. Anagnostopoulos 
>      ----------------------------------------------------------
>   Software            978 369-0839
>                             www.windfall.com
> ----------------------------------------------------------------
> My life has been filled with calamities,
> some of which actually happened.
> ---Mark Twain 
> 
> Guga 'mzimba, sala 'nhliziyo  
>

On 2020-08-04, Chris Lattner via llvm-dev wrote:
>It may be confusing but it is not the case that “llvm/lib” contains all the
libraries, and “llvm/tools” contains all the tools.  At a course grain, the main
llvm repo builds in four phases:
>
>1) llvm/utils.  -> this builds some executables like tablegen
>2) llvm/lib.     -> This is all libraries, shouldn’t include executables.
>3) llvm/tools. -> Generally executables, also some libraries that are
tool specific.
>4) tests..        -> Things that depend on the above.
>
>In theory, we could have a much finer grain build system that allows
tracking exactly which tests depend on which executables, which executables
depend on which libraries, etc, but this hasn’t been done.  In theory this could
help the build cycle if you’re running a single “llvm-mc” like test, for
example.
>
>-Chris
llvm/test/CMakeLists.txt LLVM_TEST_DEPENDS lists many utilities. If I
understand correctly, every test under llvm/test/ requires to build all
tools listed in LLVM_TEST_DEPENDS. This is probably fine because these
utilities are all small.
>
>> On Aug 4, 2020, at 12:06 PM, Paul C. Anagnostopoulos <paul at
windfall.com> wrote:
>>
>> Could you expand on this a bit? I see from my build directory that
llvm-tblgen.exe was built before the Target/* .inc files. It sounds like
you're saying that something has to be built between the .exe and the .inc
files.
>>
>> At 8/4/2020 02:44 PM, Chris Lattner wrote:
>>
>>
>>>> On Aug 4, 2020, at 11:31 AM, Paul C. Anagnostopoulos via
llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>>
>>>> I was confused by the lack of TableGen backends in the
llvm/lib/tablegen directory until I stumbled upon the llvm/utils/tablegen
directory. Could someone explain why TableGen is divided into these two
directories?
>>>
>>> The mechanical reason is that llvm/utils builds before llvm/libs. 
The basic TableGen support is required to generate .inc files in the libs
directory.
>>>
>>> -Chris
>>
>>
>> ----------------------------------------------------------------
>> Windfall               Paul C. Anagnostopoulos
>>      ----------------------------------------------------------
>>   Software            978 369-0839
>>                             www.windfall.com
>> ----------------------------------------------------------------
>> My life has been filled with calamities,
>> some of which actually happened.
>> ---Mark Twain
>>
>> Guga 'mzimba, sala 'nhliziyo
>>
>
>_______________________________________________
>LLVM Developers mailing list
>llvm-dev at lists.llvm.org
>https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Until people pointed it out, it hadn't registered that of course TableGen is
not just one table generator, but many, depending on the backends attached. So
the common code is separate from the various backends for at least that reason.

At 8/4/2020 04:06 PM, Chris Lattner wrote:
>It may be confusing but it is not the case that “llvm/lib” contains all
the libraries, and “llvm/tools” contains all the tools.  At a course grain,
the main llvm repo builds in four phases:
>
>1) llvm/utils.  -> this builds some executables like tablegen
>2) llvm/lib.     -> This is all libraries, shouldn’t include
executables.
>3) llvm/tools. -> Generally executables, also some libraries that are
tool specific.
>4) tests..        -> Things that depend on the above.
>
>In theory, we could have a much finer grain build system that allows
tracking exactly which tests depend on which executables, which executables
depend on which libraries, etc, but this hasn’t been done.  In theory this
could help the build cycle if you’re running a single “llvm-mc” like test,
for example.
>
>-Chris