[llvm-dev] Should we split llvm Support and ADT?

Zachary Turner via llvm-dev
Sun, 28 May 2017 02:25:57 -0000

I think there are logical ways of splitting, but taken individually they
would be rather small. For example, threading support, i/o related support,
file system support.

At the same time, when you look at support there's kind of some glaringly
obvious differences in how "low level" the various things are.
Compiler.h
is pretty low level. Dwarf.h, not so much. It's hard to really quantify
this, but I could certainly imagine introducing something even more low
level than Support, perhaps with a name like Base
On Sat, May 27, 2017 at 12:55 PM David Blaikie <dblaikie at gmail.com>
wrote:

> Maybe we do and build systems aren't respecting/noticing this? I'm
not
> sure.
>
> On Sat, May 27, 2017 at 12:50 PM Craig Topper <craig.topper at
gmail.com>
> wrote:
>
>> I thought we already did write tablegen output to temporary files like
>> X86GenAsmWriter1.inc.tmp first and then diffed them with the real .inc
file
>> and conditionally copied.
>>
>> ~Craig
>>
>> On Sat, May 27, 2017 at 11:02 AM, David Blaikie via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>>
>>>
>>> On Fri, May 26, 2017 at 8:06 PM Zachary Turner via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> It would be better, because a debug tablegen is slower than an
>>>> optimized tablegen, but it's still slow and it doesn't
address the problem
>>>> that tablegen runs *at all* when it doesn't really need to.
I think if
>>>> tablegen wasn't running all the time we could incremental
builds down from
>>>> 15 minutes (and that's on my really powerful machine) to
under 5, which
>>>> seemed like a big productivity boost averaged out over all
users x time
>>>>
>>>
>>> Is that 10 minutes of running tablegen, though? I think this
touches on
>>> Hal's comment:
>>> "Maybe we should use a diff-and-update approach (I thought,
however,
>>> that we already did that)."
>>>
>>> In that the 10 minutes is presumably from timestamp-based
invalidation
>>> chaining down from running tablegen, to then rebuilding everything
that
>>> depends on tablegen's output.
>>>
>>> That could be addressed by either the build system or tablegen
writing
>>> output to a temporary file, comparing the temp to the current file,
and not
>>> modifying the current file if they're the same - thus not
invalidating
>>> everything down the chain that depends on the tablegen output (not
sure if
>>> that's actually enough to convince a build system not to
rebuild other
>>> things - but worth a shot). This is a bit of a hackaround what the
build
>>> system itself could/should be doing, but in the absence of a
content based
>>> invalidation scheme - addressing this one particularly critical
point in
>>> LLVM's build seems like it could be worth such a workaround.
(this would
>>> speed up the build for even the times when tablegen's real
dependencies
>>> change - add a new function to ArrayRef, or STLExtras, etc... and
tablegen
>>> runs, but it doesn't cascade to everything else).
>>>
>>> (plus, opt build of tablegen, so that conclusion can be reached
more
>>> quickly)
>>>
>>> As Hal also mentioned, I'd be OK with this if there's a
good logical
>>> division that can be taken more than "whatever is/isn't
used in tablegen".
>>>
>>>
>>>>
>>>> We've got a lot of random stuff in Support, not all really
related and
>>>> not all relevant to every project. It seems like we should be
able to do
>>>> better.
>>>> On Fri, May 26, 2017 at 6:48 PM Hal Finkel <hfinkel at
anl.gov> wrote:
>>>>
>>>>>
>>>>> On 05/26/2017 07:59 PM, Zachary Turner wrote:
>>>>>
>>>>> It's that TableGen depends on Support, so if you change
one file in
>>>>> support, support gets recompiled into a new static archive,
which triggers
>>>>> a rerun of tablegen on all the tablegen inputs, which is
extremely slow.
>>>>>
>>>>>
>>>>> I thought that we had a "build an optimized tablegen
even in debug
>>>>> mode" setting to work around this problem. Would that
avoid this problem?
>>>>>
>>>>>
>>>>>  -Hal
>>>>>
>>>>>
>>>>>
>>>>> On Fri, May 26, 2017 at 5:56 PM Hal Finkel <hfinkel at
anl.gov> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On 05/26/2017 07:47 PM, Zachary Turner via llvm-dev
wrote:
>>>>>>
>>>>>> Changing a header file somewhere and having to spend 10
minutes
>>>>>> waiting for a build leads to a lot of wasted developer
time.
>>>>>>
>>>>>> The real culprit here is tablegen.  Can we split
support and ADT into
>>>>>> two - the parts that tablegen depends on and the parts
that it doesn't?
>>>>>>
>>>>>>
>>>>>> What's the actual problem here? Is it that TableGen
regenerates
>>>>>> different files and so we then need to rebuild all
dependencies of those
>>>>>> files? Maybe we should use a diff-and-update approach
(I thought, however,
>>>>>> that we already did that).
>>>>>>
>>>>>>  -Hal
>>>>>>
>>>>>>
>>>>>> From what I can gather, Tablegen currently depends on
these headers
>>>>>> and all of their transitive dependencies.
>>>>>>
>>>>>> #include "llvm/Support/Casting.h"
>>>>>> #include "llvm/Support/CommandLine.h"
>>>>>> #include "llvm/Support/Compiler.h"
>>>>>> #include "llvm/Support/DataTypes.h"
>>>>>> #include "llvm/Support/Debug.h"
>>>>>> #include "llvm/Support/Error.h"
>>>>>> #include "llvm/Support/ErrorHandling.h"
>>>>>> #include "llvm/Support/Format.h"
>>>>>> #include "llvm/Support/FormattedStream.h"
>>>>>> #include "llvm/Support/LEB128.h"
>>>>>> #include "llvm/Support/LowLevelTypeImpl.h"
>>>>>> #include "llvm/Support/ManagedStatic.h"
>>>>>> #include "llvm/Support/MathExtras.h"
>>>>>> #include "llvm/Support/MemoryBuffer.h"
>>>>>> #include "llvm/Support/PrettyStackTrace.h"
>>>>>> #include "llvm/Support/Regex.h"
>>>>>> #include "llvm/Support/SMLoc.h"
>>>>>> #include "llvm/Support/ScopedPrinter.h"
>>>>>> #include "llvm/Support/Signals.h"
>>>>>> #include "llvm/Support/SourceMgr.h"
>>>>>> #include "llvm/Support/raw_ostream.h"
>>>>>>
>>>>>> #include "llvm/ADT/APInt.h"
>>>>>> #include "llvm/ADT/ArrayRef.h"
>>>>>> #include "llvm/ADT/BitVector.h"
>>>>>> #include "llvm/ADT/CachedHashString.h"
>>>>>> #include "llvm/ADT/DenseSet.h"
>>>>>> #include "llvm/ADT/IndexedMap.h"
>>>>>> #include "llvm/ADT/IntEqClasses.h"
>>>>>> #include "llvm/ADT/MapVector.h"
>>>>>> #include "llvm/ADT/Optional.h"
>>>>>> #include "llvm/ADT/PointerUnion.h"
>>>>>> #include "llvm/ADT/STLExtras.h"
>>>>>> #include "llvm/ADT/SetVector.h"
>>>>>> #include "llvm/ADT/SmallPtrSet.h"
>>>>>> #include "llvm/ADT/SmallSet.h"
>>>>>> #include "llvm/ADT/SmallVector.h"
>>>>>> #include "llvm/ADT/SparseBitVector.h"
>>>>>> #include "llvm/ADT/Statistic.h"
>>>>>> #include "llvm/ADT/StringExtras.h"
>>>>>> #include "llvm/ADT/StringMap.h"
>>>>>> #include "llvm/ADT/StringRef.h"
>>>>>> #include "llvm/ADT/StringSet.h"
>>>>>> #include "llvm/ADT/StringSwitch.h"
>>>>>> #include "llvm/ADT/TinyPtrVector.h"
>>>>>> #include "llvm/ADT/Twine.h"
>>>>>>
>>>>>>
>>>>>> Is this something worth putting effort into?  If so, I
volunteer.
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> LLVM Developers mailing listllvm-dev at
lists.llvm.orghttp://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Hal Finkel
>>>>>> Lead, Compiler Technology and Programming Languages
>>>>>> Leadership Computing Facility
>>>>>> Argonne National Laboratory
>>>>>>
>>>>>>
>>>>> --
>>>>> Hal Finkel
>>>>> Lead, Compiler Technology and Programming Languages
>>>>> Leadership Computing Facility
>>>>> Argonne National Laboratory
>>>>>
>>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170528/597c5695/attachment-0001.html>