On Tue, Nov 26, 2019 at 10:31 AM David Jones via llvm-dev < llvm-dev at lists.llvm.org> wrote:> Throw hardware at it. >While I agree that this is good advice with the current state of affairs, I don't think we should consider the current situation acceptable. LLVM has a lot of knobs that Tim alluded to, but first time users shouldn't have to search for them to get a working debug build of LLVM out of the box. As a community, we need to raise the bar here, and spend some time making first time build setup easier. I seem to recall that there were discussions at the dev meeting about simplifying our CMake build, and I think this can be part of that effort. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191126/a4d625e6/attachment.html>
I believe it is pretty clear that the stock ld linker is basically unusable for our project without a sizable amount of hardware. I’ve typically found that using gold or lld fixes this problem entirely. Perhaps we need to start having CMake detect the presence of lld and use that if possible, else check for ‘gold’ and use it if possible, and only THEN fall back on ld. Additionally, it is pretty rare for someone to need to debug TableGen (at least in my experience), and if they DO need it, they would know enough to know how to change the cmake variable. I think we should have a serious discussion about making LLVM_OPTIMIZED_TABLEGEN=true be default. From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Reid Kleckner via llvm-dev Sent: Tuesday, November 26, 2019 10:41 AM To: David Jones <david.jones at metrics.ca> Cc: llvm-dev <llvm-dev at lists.llvm.org>; Bagel <bagel99 at gmail.com> Subject: Re: [llvm-dev] debug build busts memory On Tue, Nov 26, 2019 at 10:31 AM David Jones via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote: Throw hardware at it. While I agree that this is good advice with the current state of affairs, I don't think we should consider the current situation acceptable. LLVM has a lot of knobs that Tim alluded to, but first time users shouldn't have to search for them to get a working debug build of LLVM out of the box. As a community, we need to raise the bar here, and spend some time making first time build setup easier. I seem to recall that there were discussions at the dev meeting about simplifying our CMake build, and I think this can be part of that effort. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191126/fad8f8a5/attachment.html>
On Tue, Nov 26, 2019 at 10:46 AM Keane, Erich via llvm-dev < llvm-dev at lists.llvm.org> wrote:> I believe it is pretty clear that the stock ld linker is basically > unusable for our project without a sizable amount of hardware. I’ve > typically found that using gold or lld fixes this problem entirely. > > > > Perhaps we need to start having CMake detect the presence of lld and use > that if possible, else check for ‘gold’ and use it if possible, and only > THEN fall back on ld. >+1, we should probably emit a warning if we have to fall back to ld.bfd. It's basically non-functional for large C++ apps.> Additionally, it is pretty rare for someone to need to debug TableGen (at > least in my experience), and if they DO need it, they would know enough to > know how to change the cmake variable. I think we should have a serious > discussion about making LLVM_OPTIMIZED_TABLEGEN=true be default. >If you are doing a release build already (my workflow), this represents a lot of additional overhead: second cmake config step, separate ninja subbuild. But if you are doing a debug build, it's clearly better, and there is no easy alternative. What do you think about enabling it for multi-config generators and single-config debug builds, so single-config release builds are not affected? I'd be in favor. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191126/0a28da05/attachment.html>
Alex Brachet-Mialot via llvm-dev
2019-Nov-26 19:21 UTC
[llvm-dev] debug build busts memory
The suggestion to just upgrade hardware isn't a constructive one in my opinion, not everyone can do this. As for hardware, I've built llvm with with a tiny 2 core pentium chip and 8gb of memory, (granted I needed to increase my swap space) and I left it over night. Once it's built you generally don't need to build it in full ever again and the future strain on your machine will always be less. These questions come up often enough maybe we should add a new section about building with not enough memory to llvm/docs/CMake.rst. If 16gb isn't enough then most people will have a hard time building. There has been a lot of talk recently about making llvm development more accessible to newcomers, being unable to even build has to be by far the most egregious problem. How many people had a passing interest in llvm but when their build failed didn't want to ask the mailing list what other options they had and gave up? On Tue, Nov 26, 2019 at 1:46 PM Keane, Erich via llvm-dev <llvm-dev at lists.llvm.org> wrote:> > I believe it is pretty clear that the stock ld linker is basically unusable for our project without a sizable amount of hardware. I’ve typically found that using gold or lld fixes this problem entirely. > > > > Perhaps we need to start having CMake detect the presence of lld and use that if possible, else check for ‘gold’ and use it if possible, and only THEN fall back on ld. > > > > Additionally, it is pretty rare for someone to need to debug TableGen (at least in my experience), and if they DO need it, they would know enough to know how to change the cmake variable. I think we should have a serious discussion about making LLVM_OPTIMIZED_TABLEGEN=true be default. > > > > From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Reid Kleckner via llvm-dev > Sent: Tuesday, November 26, 2019 10:41 AM > To: David Jones <david.jones at metrics.ca> > Cc: llvm-dev <llvm-dev at lists.llvm.org>; Bagel <bagel99 at gmail.com> > Subject: Re: [llvm-dev] debug build busts memory > > > > On Tue, Nov 26, 2019 at 10:31 AM David Jones via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > Throw hardware at it. > > > > While I agree that this is good advice with the current state of affairs, I don't think we should consider the current situation acceptable. > > > > LLVM has a lot of knobs that Tim alluded to, but first time users shouldn't have to search for them to get a working debug build of LLVM out of the box. As a community, we need to raise the bar here, and spend some time making first time build setup easier. I seem to recall that there were discussions at the dev meeting about simplifying our CMake build, and I think this can be part of that effort. > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
On Tue, Nov 26, 2019 at 1:41 PM Reid Kleckner via llvm-dev < llvm-dev at lists.llvm.org> wrote:> On Tue, Nov 26, 2019 at 10:31 AM David Jones via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> Throw hardware at it. >> > > While I agree that this is good advice with the current state of affairs, > I don't think we should consider the current situation acceptable. > > LLVM has a lot of knobs that Tim alluded to, but first time users > shouldn't have to search for them to get a working debug build of LLVM out > of the box. As a community, we need to raise the bar here, and spend some > time making first time build setup easier. I seem to recall that there were > discussions at the dev meeting about simplifying our CMake build, and I > think this can be part of that effort. >I don't really think of LLVM as being an especially large project (is that inaccurate, compared to other open source software) - any ideas what other projects do? But I'd at least be happy to see a warning about "hey, you're using the bfd linker - it's slow and uses lots of memory, consider installing gold or lld" - like there used to be a warning about building with asserts enabled. I'm not sure there's much else we can do if their system linker is bfd ld, right? I guess we could specifically look for non-default linkers & fallback to bfd only as a last resort (with a warning if possible)?> _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191126/8a5115b9/attachment.html>
On 11/26/2019 10:41 AM, Reid Kleckner via llvm-dev wrote:> On Tue, Nov 26, 2019 at 10:31 AM David Jones via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > > Throw hardware at it. > > > While I agree that this is good advice with the current state of affairs, I don't think we should consider the current situation acceptable. > > LLVM has a lot of knobs that Tim alluded to, but first time users shouldn't have to search for them to get a working debug build of LLVM out of the box. As a community, we need to raise the bar here, and spend some time making first time build setup easier. I seem to recall that there were discussions at the dev meeting about simplifying our CMake build, and I think this can be part of that effort. >Can we make LLVM_USE_SPLIT_DWARF=ON the default? This seems like the easiest way to solve the memory usage issues, and we are already recommending this whenever users hit this problem. -Tom> > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >
(minor cleanup required: gdb's split dwarf support needs -gdb-index, which means you need a linker that can do that (gold or lld) & also I think the LLVM_USE_SPLIT_DWARF is incomplete (because it only enables -gsplit-dwarf but doesn't add the -Wl,-gdb-index flag to the linker action) & so before recommending it as a mainstream thing should probably be tidied up a bit/fleshed out to address thoes issues) On Tue, Nov 26, 2019 at 10:17 PM Tom Stellard via llvm-dev < llvm-dev at lists.llvm.org> wrote:> On 11/26/2019 10:41 AM, Reid Kleckner via llvm-dev wrote: > > On Tue, Nov 26, 2019 at 10:31 AM David Jones via llvm-dev < > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > > > > Throw hardware at it. > > > > > > While I agree that this is good advice with the current state of > affairs, I don't think we should consider the current situation acceptable. > > > > LLVM has a lot of knobs that Tim alluded to, but first time users > shouldn't have to search for them to get a working debug build of LLVM out > of the box. As a community, we need to raise the bar here, and spend some > time making first time build setup easier. I seem to recall that there were > discussions at the dev meeting about simplifying our CMake build, and I > think this can be part of that effort. > > > > Can we make LLVM_USE_SPLIT_DWARF=ON the default? This seems like the > easiest way to solve the memory usage issues, and we are already > recommending > this whenever users hit this problem. > > -Tom > > > > > _______________________________________________ > > LLVM Developers mailing list > > llvm-dev at lists.llvm.org > > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191126/c5e56211/attachment.html>