Reid Kleckner via llvm-dev
2021-Jun-10 18:14 UTC
[llvm-dev] RFC: Revisiting LLD-as-a-library design
Hey all, Long ago, the LLD project contributors decided that they weren't going to design LLD as a library, which stands in opposition to the way that the rest of LLVM strives to be a reusable library. Part of the reasoning was that, at the time, LLD wasn't done yet, and the top priority was to finish making LLD a fast, useful, usable product. If sacrificing reusability helped LLD achieve its project goals, the contributors at the time felt that was the right tradeoff, and that carried the day. However, it is now ${YEAR} 2021, and I think we ought to reconsider this design decision. LLD was a great success: it works, it is fast, it is simple, many users have adopted it, it has many ports (COFF/ELF/mingw/wasm/new MachO). Today, we have actual users who want to run the linker as a library, and they aren't satisfied with the option of launching a child process. Some users are interested in process reuse as a performance optimization, some are including the linker in the frontend. Who knows. I try not to pre-judge any of these efforts, I think we should do what we can to enable experimentation. So, concretely, what could change? The main points of reusability are: - Fatal errors and warnings exit the process without returning control to the caller - Conflicts over global variables between threads Error recovery is the big imposition here. To avoid a giant rewrite of all error handling code in LLD, I think we should *avoid* returning failure via the llvm::Error class or std::error_code. We should instead use an approach more like clang, where diagnostics are delivered to a diagnostic consumer on the side. The success of the link is determined by whether any errors were reported. Functions may return a simple success boolean in cases where higher level functions need to exit early. This has worked reasonably well for clang. The main failure mode here is that we miss an error check, and crash or report useless follow-on errors after an error that would normally have been fatal. Another motivation for all of this is increasing the use of parallelism in LLD. Emitting errors in parallel from threads and then exiting the process is risky business. A new diagnostic context or consumer could make this more reliable. MLIR has this issue as well, and I believe they use this pattern. They use some kind of thread shard index to order the diagnostics, LLD could do the same. Finally, we'd work to eliminate globals. I think this is mainly a small matter of programming (SMOP) and doesn't need much discussion, although the `make` template presents interesting challenges. Thoughts? Tomatoes? Flowers? I apologize for the lack of context links to the original discussions. It takes more time than I have to dig those up. Reid -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210610/250f733b/attachment.html>
Petr Hosek via llvm-dev
2021-Jun-10 18:27 UTC
[llvm-dev] RFC: Revisiting LLD-as-a-library design
A big +1 from our side since we have a potential use case for LLD-as-a-library (I was going to write a similar RFC but you beat me to it). On Thu, Jun 10, 2021 at 11:15 AM Reid Kleckner via llvm-dev < llvm-dev at lists.llvm.org> wrote:> Hey all, > > Long ago, the LLD project contributors decided that they weren't going to > design LLD as a library, which stands in opposition to the way that the > rest of LLVM strives to be a reusable library. Part of the reasoning was > that, at the time, LLD wasn't done yet, and the top priority was to finish > making LLD a fast, useful, usable product. If sacrificing reusability > helped LLD achieve its project goals, the contributors at the time felt > that was the right tradeoff, and that carried the day. > > However, it is now ${YEAR} 2021, and I think we ought to reconsider this > design decision. LLD was a great success: it works, it is fast, it is > simple, many users have adopted it, it has many ports > (COFF/ELF/mingw/wasm/new MachO). Today, we have actual users who want to > run the linker as a library, and they aren't satisfied with the option of > launching a child process. Some users are interested in process reuse as a > performance optimization, some are including the linker in the frontend. > Who knows. I try not to pre-judge any of these efforts, I think we should > do what we can to enable experimentation. > > So, concretely, what could change? The main points of reusability are: > - Fatal errors and warnings exit the process without returning control to > the caller > - Conflicts over global variables between threads > > Error recovery is the big imposition here. To avoid a giant rewrite of all > error handling code in LLD, I think we should *avoid* returning failure via > the llvm::Error class or std::error_code. We should instead use an approach > more like clang, where diagnostics are delivered to a diagnostic consumer > on the side. The success of the link is determined by whether any errors > were reported. Functions may return a simple success boolean in cases where > higher level functions need to exit early. This has worked reasonably well > for clang. The main failure mode here is that we miss an error check, and > crash or report useless follow-on errors after an error that would normally > have been fatal. > > Another motivation for all of this is increasing the use of parallelism in > LLD. Emitting errors in parallel from threads and then exiting the process > is risky business. A new diagnostic context or consumer could make this > more reliable. MLIR has this issue as well, and I believe they use this > pattern. They use some kind of thread shard index to order the diagnostics, > LLD could do the same. > > Finally, we'd work to eliminate globals. I think this is mainly a small > matter of programming (SMOP) and doesn't need much discussion, although the > `make` template presents interesting challenges. > > Thoughts? Tomatoes? Flowers? I apologize for the lack of context links to > the original discussions. It takes more time than I have to dig those up. > > Reid > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210610/07c7191b/attachment-0001.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3996 bytes Desc: S/MIME Cryptographic Signature URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210610/07c7191b/attachment-0001.bin>
Andrew Kelley via llvm-dev
2021-Jun-10 20:41 UTC
[llvm-dev] RFC: Revisiting LLD-as-a-library design
Throwing flowers on behalf of the Zig project 🌻🥀 :) Currently Zig embeds LLD with a renamed main() and then exposes `zig ld.lld`, `zig ld64.lld`, `zig lld-link`, and `zig wasm-ld` which call into this renamed main() function. Then when Zig wants to invoke LLD, it executes itself as a child process, using one of these sub-commands. LLD-as-a-library would remove the need for this trick, improving performance especially on systems such as Windows which have a high cost of spawning child processes. It also has the possibility to improve error reporting, avoiding the need for parsing the stderr of LLD. Errors could be communicated in a more semantic way, with less room for bugs. With our upcoming self-hosted backend, compile times are on order of milliseconds, and so child process spawning times in order to invoke LLD end up being a substantial portion of total compilation time. That said, we are also entering the linking space to provide an alternative to LLD, but it will be some time before any possibility of removing a dependency on LLD, so this RFC would be greatly beneficial to the Zig project. Cheers, Andrew On 6/10/21 11:14 AM, Reid Kleckner via llvm-dev wrote:> Hey all, > > Long ago, the LLD project contributors decided that they weren't going > to design LLD as a library, which stands in opposition to the way that > the rest of LLVM strives to be a reusable library. Part of the reasoning > was that, at the time, LLD wasn't done yet, and the top priority was to > finish making LLD a fast, useful, usable product. If sacrificing > reusability helped LLD achieve its project goals, the contributors at > the time felt that was the right tradeoff, and that carried the day. > > However, it is now ${YEAR} 2021, and I think we ought to reconsider this > design decision. LLD was a great success: it works, it is fast, it is > simple, many users have adopted it, it has many ports > (COFF/ELF/mingw/wasm/new MachO). Today, we have actual users who want to > run the linker as a library, and they aren't satisfied with the option > of launching a child process. Some users are interested in process reuse > as a performance optimization, some are including the linker in the > frontend. Who knows. I try not to pre-judge any of these efforts, I > think we should do what we can to enable experimentation. > > So, concretely, what could change? The main points of reusability are: > - Fatal errors and warnings exit the process without returning control > to the caller > - Conflicts over global variables between threads > > Error recovery is the big imposition here. To avoid a giant rewrite of > all error handling code in LLD, I think we should *avoid* returning > failure via the llvm::Error class or std::error_code. We should instead > use an approach more like clang, where diagnostics are delivered to a > diagnostic consumer on the side. The success of the link is determined > by whether any errors were reported. Functions may return a simple > success boolean in cases where higher level functions need to exit > early. This has worked reasonably well for clang. The main failure mode > here is that we miss an error check, and crash or report useless > follow-on errors after an error that would normally have been fatal. > > Another motivation for all of this is increasing the use of parallelism > in LLD. Emitting errors in parallel from threads and then exiting the > process is risky business. A new diagnostic context or consumer could > make this more reliable. MLIR has this issue as well, and I believe they > use this pattern. They use some kind of thread shard index to order the > diagnostics, LLD could do the same. > > Finally, we'd work to eliminate globals. I think this is mainly a small > matter of programming (SMOP) and doesn't need much discussion, although > the `make` template presents interesting challenges. > > Thoughts? Tomatoes? Flowers? I apologize for the lack of context links > to the original discussions. It takes more time than I have to dig those up. > > Reid > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 840 bytes Desc: OpenPGP digital signature URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210610/cb716c98/attachment.sig>
Tom Stellard via llvm-dev
2021-Jun-10 22:16 UTC
[llvm-dev] RFC: Revisiting LLD-as-a-library design
On 6/10/21 11:14 AM, Reid Kleckner via llvm-dev wrote:> Hey all, > > Long ago, the LLD project contributors decided that they weren't going to design LLD as a library, which stands in opposition to the way that the rest of LLVM strives to be a reusable library. Part of the reasoning was that, at the time, LLD wasn't done yet, and the top priority was to finish making LLD a fast, useful, usable product. If sacrificing reusability helped LLD achieve its project goals, the contributors at the time felt that was the right tradeoff, and that carried the day. >I think it would be great to move in this directions. It's a little bit unclear at the moment whether or not the library use case is supported, because we do include headers and libraries in the install targets. As a package maintainer my wish list for an LLD library is: 1. Single shared object: https://reviews.llvm.org/D85278 2. All symbols are assumed private unless explicitly given library visibility. 3. Some subset of the API that is stable across major releases. -Tom> However, it is now ${YEAR} 2021, and I think we ought to reconsider this design decision. LLD was a great success: it works, it is fast, it is simple, many users have adopted it, it has many ports (COFF/ELF/mingw/wasm/new MachO). Today, we have actual users who want to run the linker as a library, and they aren't satisfied with the option of launching a child process. Some users are interested in process reuse as a performance optimization, some are including the linker in the frontend. Who knows. I try not to pre-judge any of these efforts, I think we should do what we can to enable experimentation. > > So, concretely, what could change? The main points of reusability are: > - Fatal errors and warnings exit the process without returning control to the caller > - Conflicts over global variables between threads > > Error recovery is the big imposition here. To avoid a giant rewrite of all error handling code in LLD, I think we should *avoid* returning failure via the llvm::Error class or std::error_code. We should instead use an approach more like clang, where diagnostics are delivered to a diagnostic consumer on the side. The success of the link is determined by whether any errors were reported. Functions may return a simple success boolean in cases where higher level functions need to exit early. This has worked reasonably well for clang. The main failure mode here is that we miss an error check, and crash or report useless follow-on errors after an error that would normally have been fatal. > > Another motivation for all of this is increasing the use of parallelism in LLD. Emitting errors in parallel from threads and then exiting the process is risky business. A new diagnostic context or consumer could make this more reliable. MLIR has this issue as well, and I believe they use this pattern. They use some kind of thread shard index to order the diagnostics, LLD could do the same. > > Finally, we'd work to eliminate globals. I think this is mainly a small matter of programming (SMOP) and doesn't need much discussion, although the `make` template presents interesting challenges. > > Thoughts? Tomatoes? Flowers? I apologize for the lack of context links to the original discussions. It takes more time than I have to dig those up. > > Reid > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >
Renato Golin via llvm-dev
2021-Jun-11 06:51 UTC
[llvm-dev] RFC: Revisiting LLD-as-a-library design
Bug +1 from me, too. The reasons of the past were clear, and they helped make the project a success. Kudos to all involved. The current reasons are also clear, especially error handling and reuse of infrastructure. Cheers, Renato On Thu, 10 Jun 2021, 19:15 Reid Kleckner via llvm-dev, < llvm-dev at lists.llvm.org> wrote:> Hey all, > > Long ago, the LLD project contributors decided that they weren't going to > design LLD as a library, which stands in opposition to the way that the > rest of LLVM strives to be a reusable library. Part of the reasoning was > that, at the time, LLD wasn't done yet, and the top priority was to finish > making LLD a fast, useful, usable product. If sacrificing reusability > helped LLD achieve its project goals, the contributors at the time felt > that was the right tradeoff, and that carried the day. > > However, it is now ${YEAR} 2021, and I think we ought to reconsider this > design decision. LLD was a great success: it works, it is fast, it is > simple, many users have adopted it, it has many ports > (COFF/ELF/mingw/wasm/new MachO). Today, we have actual users who want to > run the linker as a library, and they aren't satisfied with the option of > launching a child process. Some users are interested in process reuse as a > performance optimization, some are including the linker in the frontend. > Who knows. I try not to pre-judge any of these efforts, I think we should > do what we can to enable experimentation. > > So, concretely, what could change? The main points of reusability are: > - Fatal errors and warnings exit the process without returning control to > the caller > - Conflicts over global variables between threads > > Error recovery is the big imposition here. To avoid a giant rewrite of all > error handling code in LLD, I think we should *avoid* returning failure via > the llvm::Error class or std::error_code. We should instead use an approach > more like clang, where diagnostics are delivered to a diagnostic consumer > on the side. The success of the link is determined by whether any errors > were reported. Functions may return a simple success boolean in cases where > higher level functions need to exit early. This has worked reasonably well > for clang. The main failure mode here is that we miss an error check, and > crash or report useless follow-on errors after an error that would normally > have been fatal. > > Another motivation for all of this is increasing the use of parallelism in > LLD. Emitting errors in parallel from threads and then exiting the process > is risky business. A new diagnostic context or consumer could make this > more reliable. MLIR has this issue as well, and I believe they use this > pattern. They use some kind of thread shard index to order the diagnostics, > LLD could do the same. > > Finally, we'd work to eliminate globals. I think this is mainly a small > matter of programming (SMOP) and doesn't need much discussion, although the > `make` template presents interesting challenges. > > Thoughts? Tomatoes? Flowers? I apologize for the lack of context links to > the original discussions. It takes more time than I have to dig those up. > > Reid > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210611/edea90f2/attachment.html>
James Henderson via llvm-dev
2021-Jun-11 07:42 UTC
[llvm-dev] RFC: Revisiting LLD-as-a-library design
No objections here (although I don't have a specific use-case currently). Regarding the error handling, I support some sort of callback approach to report the errors (https://www.youtube.com/watch?v=YSEY4pg1YB0). This doesn't solve the problem of what to do after a fatal error has been reported. In the debug line parsing code which inspired that talk, we had a concept of unrecoverable and recoverable errors, whereby the parser would either stop parsing if it found something it couldn't recover from, by bailing out of the function, or it would set some assumed values and continue parsing. This may work for some cases in LLD, but the fatal cases need to stop the linking completely, so we'll need some way to bail out of the LLD call stack in those cases still somehow - personally, I think we should use llvm::Error for that up to the point of public interface with the library, to avoid the failure being unchecked. The error callbacks could then return Error to allow a client to force LLD to stop, even if the error would otherwise be non-fatal. James On Thu, 10 Jun 2021 at 19:15, Reid Kleckner via llvm-dev < llvm-dev at lists.llvm.org> wrote:> Hey all, > > Long ago, the LLD project contributors decided that they weren't going to > design LLD as a library, which stands in opposition to the way that the > rest of LLVM strives to be a reusable library. Part of the reasoning was > that, at the time, LLD wasn't done yet, and the top priority was to finish > making LLD a fast, useful, usable product. If sacrificing reusability > helped LLD achieve its project goals, the contributors at the time felt > that was the right tradeoff, and that carried the day. > > However, it is now ${YEAR} 2021, and I think we ought to reconsider this > design decision. LLD was a great success: it works, it is fast, it is > simple, many users have adopted it, it has many ports > (COFF/ELF/mingw/wasm/new MachO). Today, we have actual users who want to > run the linker as a library, and they aren't satisfied with the option of > launching a child process. Some users are interested in process reuse as a > performance optimization, some are including the linker in the frontend. > Who knows. I try not to pre-judge any of these efforts, I think we should > do what we can to enable experimentation. > > So, concretely, what could change? The main points of reusability are: > - Fatal errors and warnings exit the process without returning control to > the caller > - Conflicts over global variables between threads > > Error recovery is the big imposition here. To avoid a giant rewrite of all > error handling code in LLD, I think we should *avoid* returning failure via > the llvm::Error class or std::error_code. We should instead use an approach > more like clang, where diagnostics are delivered to a diagnostic consumer > on the side. The success of the link is determined by whether any errors > were reported. Functions may return a simple success boolean in cases where > higher level functions need to exit early. This has worked reasonably well > for clang. The main failure mode here is that we miss an error check, and > crash or report useless follow-on errors after an error that would normally > have been fatal. > > Another motivation for all of this is increasing the use of parallelism in > LLD. Emitting errors in parallel from threads and then exiting the process > is risky business. A new diagnostic context or consumer could make this > more reliable. MLIR has this issue as well, and I believe they use this > pattern. They use some kind of thread shard index to order the diagnostics, > LLD could do the same. > > Finally, we'd work to eliminate globals. I think this is mainly a small > matter of programming (SMOP) and doesn't need much discussion, although the > `make` template presents interesting challenges. > > Thoughts? Tomatoes? Flowers? I apologize for the lack of context links to > the original discussions. It takes more time than I have to dig those up. > > Reid > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210611/a1bf2246/attachment.html>
Michael Spencer via llvm-dev
2021-Jun-12 03:20 UTC
[llvm-dev] RFC: Revisiting LLD-as-a-library design
Adding Erik (not subscribed) who has previously had issues with LLD not being a library to provide some additional use cases. - Michael Spencer On Thu, Jun 10, 2021 at 12:15 PM Reid Kleckner via llvm-dev < llvm-dev at lists.llvm.org> wrote:> Hey all, > > Long ago, the LLD project contributors decided that they weren't going to > design LLD as a library, which stands in opposition to the way that the > rest of LLVM strives to be a reusable library. Part of the reasoning was > that, at the time, LLD wasn't done yet, and the top priority was to finish > making LLD a fast, useful, usable product. If sacrificing reusability > helped LLD achieve its project goals, the contributors at the time felt > that was the right tradeoff, and that carried the day. > > However, it is now ${YEAR} 2021, and I think we ought to reconsider this > design decision. LLD was a great success: it works, it is fast, it is > simple, many users have adopted it, it has many ports > (COFF/ELF/mingw/wasm/new MachO). Today, we have actual users who want to > run the linker as a library, and they aren't satisfied with the option of > launching a child process. Some users are interested in process reuse as a > performance optimization, some are including the linker in the frontend. > Who knows. I try not to pre-judge any of these efforts, I think we should > do what we can to enable experimentation. > > So, concretely, what could change? The main points of reusability are: > - Fatal errors and warnings exit the process without returning control to > the caller > - Conflicts over global variables between threads > > Error recovery is the big imposition here. To avoid a giant rewrite of all > error handling code in LLD, I think we should *avoid* returning failure via > the llvm::Error class or std::error_code. We should instead use an approach > more like clang, where diagnostics are delivered to a diagnostic consumer > on the side. The success of the link is determined by whether any errors > were reported. Functions may return a simple success boolean in cases where > higher level functions need to exit early. This has worked reasonably well > for clang. The main failure mode here is that we miss an error check, and > crash or report useless follow-on errors after an error that would normally > have been fatal. > > Another motivation for all of this is increasing the use of parallelism in > LLD. Emitting errors in parallel from threads and then exiting the process > is risky business. A new diagnostic context or consumer could make this > more reliable. MLIR has this issue as well, and I believe they use this > pattern. They use some kind of thread shard index to order the diagnostics, > LLD could do the same. > > Finally, we'd work to eliminate globals. I think this is mainly a small > matter of programming (SMOP) and doesn't need much discussion, although the > `make` template presents interesting challenges. > > Thoughts? Tomatoes? Flowers? I apologize for the lack of context links to > the original discussions. It takes more time than I have to dig those up. > > Reid > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210611/b50cdcf2/attachment.html>
Alexandre Ganea via llvm-dev
2021-Jun-12 17:28 UTC
[llvm-dev] RFC: Revisiting LLD-as-a-library design
Hello, (David Blaikie)> Reid - if you have any particular use case of your own in mind, or links to other discussions/users who are having friction with the current state of affairs, would be hand to have.The topic came up last Thursday in the Windows call, please see notes in https://docs.google.com/document/d/1A-W0Sas_oHWTEl_x_djZYoRtzAdTONMW_6l1BH9G6Bo/ Speaking for Ubisoft, there’s a short term practical usage for us in llvm-buildozer<https://reviews.llvm.org/D86351>. Neil Henning from Unity 3D raised a similar need for the Burst compiler. Since I’ve already went through these topics before, I had a list of practical things to achieve the LLD-as-a-lib goal: 1. The cl::opt data needs to live in a stack context, LLVMContext maybe. * One pain point is usages of cl::location. Folks use that kind of pattern so that the global state can be referenced from other TUs, without having to reference the cl::opt global directly. For example: bool polly::ModelReadOnlyScalars; static cl::opt<bool, true> XModelReadOnlyScalars( "polly-analyze-read-only-scalars", cl::desc("Model read-only scalar values in the scop description"), cl::location(ModelReadOnlyScalars), cl::Hidden, cl::ZeroOrMore, cl::init(true), cl::cat(PollyCategory)); * Make the CommandLineParser’s cl::opt_storage data live in a stack- or heap-based context. This is similar to point a. above, except that this applies to the implicit cl::opt state (when cl::location is omitted). There’s a PoC in https://reviews.llvm.org/D86351, lib/Support/CommandLine.cpp, L556 The rationale here is to have the ability to call LLD-as-a-lib (or Clang-as-a-lib for instance, or any other LLVM tool) in the same way as we do on the command-line. Essentially calling into LLD main() but in-process. Like mentioned in https://reviews.llvm.org/D86351 one of our objectives is to pass a CDB .json to a tool (llvm-buildozer) and build in-process. 1. The targets registry takes some time to initialize, it wouldn’t be desirable to do it every time we call into LLD-as-a-lib. With a few modifications the initialization can be made thread-safe, which is step towards making the LLD-as-a-lib entry point thread-safe. See PoC in https://reviews.llvm.org/D86351, lib/Support/TargetRegistry.cpp 2. Move LLD global variables into (LLVM? LLD?) context A first inception of this can be achieved manually, as in https://reviews.llvm.org/D86351, lld/COFF/*, see changes marked LLVM_THREAD_LOCAL, which obviously just a PoC. A better implementation would move these variables into a “LLD context”. A more advanced version would automatize this somehow, at least warn developers that a global was introduced. 3. How do we handle memory allocations? It seems the most sensible thing to do on the short term (when using LLD-as-a-lib) is to run with exitEarly = false , ie: https://github.com/llvm/llvm-project/blob/d480f968ad8b56d3ee4a6b6df5532d485b0ad01e/lld/include/lld/Common/ErrorHandler.h#L101 Later, a smarter way to get full performance would be to put all allocations into a BumpPtrAllocator-like for each LLD-as-a-lib call -- which would have the effect of exitEarly = true. 4. Externalizing ThreadPool One thing that wasn’t mentioned in Reid’s initial post, is that we won’t be able to spawn ThreadPools anymore inside LLD, at least when used as-a-lib. For example, if we call several instances of LLD-as-a-lib, from multiple client threads, a ThreadPool needs to be provided externally, by the client application. Jobs spawned internally by each LLD instance would be queued on the client ThreadPool. I don’t think this should be very hard to do, but a first iteration could (temporarily) disable multi-threading in LLD when calling it as-a-lib. 5. Returning Errors I didn’t have any issues with that in the COFF driver, but the ELF driver for example has at least one place which throws a fatal() in a constructor. This leaves a half-initialized (pooled) object on the heap, which is later destructed when destructing the SpecificAlloc<>, which in turn can corrupt the heap. See description in https://reviews.llvm.org/rG45b8a741fbbf271e0fb71294cb7cdce3ad4b9bf3 This is the primary reason why “canRunAgain” exists along with “safeLldMain”, https://github.com/llvm/llvm-project/blob/03769d9308fee79aa97149561bdbb6e3263789bd/lld/tools/lld/lld.cpp#L185 I suppose we would need to come up with a way to bubble up errors in the driver, and only return them later when the LLD-as-a-lib call completes. 6. Implicit Windows Kernel states One difficulty is that Windows stores a implicit CWD (current working directory) state for each process. When issuing Win32 API calls with relative paths, the NT Kernel would concatenate that with the internal CWD. Essentially that means we cannot pass relative paths anymore to Win32 APIs, all the paths have to be made absolute prior, and we have to store the CWD per “context”, per LLD-as-a-lib call. This isn’t terribly complicated, but requires some piping, see https://reviews.llvm.org/D86351, changes in llvm/lib/Support/Windows/Path.inc There could be additional issues like this, and on Linux as well. 7. Splitting the LLD pipeline As mentioned in this thread, we could later have a C API to provide more granularity on the LLD pipeline. But on the short term I would leave that aside, until we have a working example with just one single LLD-as-a-lib entry point (that does the same as what lldmain() does today). While we’re here, we had some other adjacent objectives with this work: 1. Being able to call Clang driver-as-a-lib in the same way as LLD Again, the objective here was to have the strictly same behavior when calling Clang-as-a-lib, as when calling clang-cl on the command-line. 2. Cache file accesses & stat We’ve seen some contention in the Windows kernel when accessing files. Being able to build in-process opens the door to sharing state between threads build different TUs, in the same way that clang-scan-deps does. There seems to be a FileSystemStatCache but it isn’t really used, anybody knows why? We could upstream a caching implementation for clang-scan-deps that could favor any other tools that do multithread in-process building. 1. LLD-as-a-DLL One point that was raised recently is, being able to compile LLVM components as DLLs on Windows. This is all adjacent to LLD-as-a-lib, perhaps it isn’t desirable to always link statically LLD into the user’s application. Does all this sound sensible? It would be nice to split the work between us, if possible. On the short term (next few weeks) I can work on 1. and 2. Best, Alex. De : Reid Kleckner <rnk at google.com> Envoyé : June 10, 2021 2:15 PM À : llvm-dev <llvm-dev at lists.llvm.org>; Fangrui Song <maskray at google.com>; Sam Clegg <sbc at chromium.org>; Shoaib Meenai <smeenai at fb.com>; gkm at fb.com; jezng at fb.com; Alexandre Ganea <alexandre.ganea at ubisoft.com>; Martin Storsjö <martin at martin.st> Objet : RFC: Revisiting LLD-as-a-library design Hey all, Long ago, the LLD project contributors decided that they weren't going to design LLD as a library, which stands in opposition to the way that the rest of LLVM strives to be a reusable library. Part of the reasoning was that, at the time, LLD wasn't done yet, and the top priority was to finish making LLD a fast, useful, usable product. If sacrificing reusability helped LLD achieve its project goals, the contributors at the time felt that was the right tradeoff, and that carried the day. However, it is now ${YEAR} 2021, and I think we ought to reconsider this design decision. LLD was a great success: it works, it is fast, it is simple, many users have adopted it, it has many ports (COFF/ELF/mingw/wasm/new MachO). Today, we have actual users who want to run the linker as a library, and they aren't satisfied with the option of launching a child process. Some users are interested in process reuse as a performance optimization, some are including the linker in the frontend. Who knows. I try not to pre-judge any of these efforts, I think we should do what we can to enable experimentation. So, concretely, what could change? The main points of reusability are: - Fatal errors and warnings exit the process without returning control to the caller - Conflicts over global variables between threads Error recovery is the big imposition here. To avoid a giant rewrite of all error handling code in LLD, I think we should *avoid* returning failure via the llvm::Error class or std::error_code. We should instead use an approach more like clang, where diagnostics are delivered to a diagnostic consumer on the side. The success of the link is determined by whether any errors were reported. Functions may return a simple success boolean in cases where higher level functions need to exit early. This has worked reasonably well for clang. The main failure mode here is that we miss an error check, and crash or report useless follow-on errors after an error that would normally have been fatal. Another motivation for all of this is increasing the use of parallelism in LLD. Emitting errors in parallel from threads and then exiting the process is risky business. A new diagnostic context or consumer could make this more reliable. MLIR has this issue as well, and I believe they use this pattern. They use some kind of thread shard index to order the diagnostics, LLD could do the same. Finally, we'd work to eliminate globals. I think this is mainly a small matter of programming (SMOP) and doesn't need much discussion, although the `make` template presents interesting challenges. Thoughts? Tomatoes? Flowers? I apologize for the lack of context links to the original discussions. It takes more time than I have to dig those up. Reid -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210612/d2760507/attachment-0001.html>
Reid Kleckner via llvm-dev
2021-Jun-15 17:35 UTC
[llvm-dev] RFC: Revisiting LLD-as-a-library design
Thanks for all the responses! I read through all the emails to try to summarize the points that were brought up. I apologize if I missed yours. I think the majority of responses were from LLD users, and they are clearly in favor of this. Maintainers seem open to change with some concerns about things like performance and merge conflicts. I think the important thing is that we've established a direction. If you review LLD code, consider reviewing and accepting patches that move in this direction. I will try to put together a patch for COFF and we can discuss the details of the code structure there. As for the shared library discussion, I think the refactoring of error handling and global state should come first. Then we can add a narrow API and a shared library around it. I think fine grained APIs and extension points should come later, if at all. I think it's important for LLD to stay focused on the performance of its primary use case, and I wouldn't want abstractions and extension points to get in the way of that. For these use cases, Orc may be more appropriate. In the long run, though, anything can happen. :) On Thu, Jun 10, 2021 at 11:14 AM Reid Kleckner <rnk at google.com> wrote:> Hey all, > > Long ago, the LLD project contributors decided that they weren't going to > design LLD as a library, which stands in opposition to the way that the > rest of LLVM strives to be a reusable library. Part of the reasoning was > that, at the time, LLD wasn't done yet, and the top priority was to finish > making LLD a fast, useful, usable product. If sacrificing reusability > helped LLD achieve its project goals, the contributors at the time felt > that was the right tradeoff, and that carried the day. > > However, it is now ${YEAR} 2021, and I think we ought to reconsider this > design decision. LLD was a great success: it works, it is fast, it is > simple, many users have adopted it, it has many ports > (COFF/ELF/mingw/wasm/new MachO). Today, we have actual users who want to > run the linker as a library, and they aren't satisfied with the option of > launching a child process. Some users are interested in process reuse as a > performance optimization, some are including the linker in the frontend. > Who knows. I try not to pre-judge any of these efforts, I think we should > do what we can to enable experimentation. > > So, concretely, what could change? The main points of reusability are: > - Fatal errors and warnings exit the process without returning control to > the caller > - Conflicts over global variables between threads > > Error recovery is the big imposition here. To avoid a giant rewrite of all > error handling code in LLD, I think we should *avoid* returning failure via > the llvm::Error class or std::error_code. We should instead use an approach > more like clang, where diagnostics are delivered to a diagnostic consumer > on the side. The success of the link is determined by whether any errors > were reported. Functions may return a simple success boolean in cases where > higher level functions need to exit early. This has worked reasonably well > for clang. The main failure mode here is that we miss an error check, and > crash or report useless follow-on errors after an error that would normally > have been fatal. > > Another motivation for all of this is increasing the use of parallelism in > LLD. Emitting errors in parallel from threads and then exiting the process > is risky business. A new diagnostic context or consumer could make this > more reliable. MLIR has this issue as well, and I believe they use this > pattern. They use some kind of thread shard index to order the diagnostics, > LLD could do the same. > > Finally, we'd work to eliminate globals. I think this is mainly a small > matter of programming (SMOP) and doesn't need much discussion, although the > `make` template presents interesting challenges. > > Thoughts? Tomatoes? Flowers? I apologize for the lack of context links to > the original discussions. It takes more time than I have to dig those up. > > Reid >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210615/4c37bc96/attachment.html>