Renato Golin via llvm-dev
2016-Sep-17 14:26 UTC
[llvm-dev] Benchmark LNT weird thread behaviour
Hi James/Chris, You guys have done this before, so I'm guessing you can help me understand what's going on. If my buildbot config is: jobs=2, nt_flags=['--cflag', '-mcpu=cortex-a15', '--use-perf', '--threads=1', '--build-threads=4'] It uses -j4 for build, -j2 for running the tests: buildmaster.tcwglab.linaro.org/builders/clang-native-arm-lnt-perf/builds/35/steps/test-suite/logs/stdio If my buildbot config is: jobs=4, nt_flags=['--cflag', '-mcpu=cortex-a15', '--use-perf', '--threads=1', '--build-threads=4'] It uses -j4 for build, -j4 for running the tests: buildmaster.tcwglab.linaro.org/builders/clang-native-arm-lnt-perf/builds/33/steps/test-suite/logs/stdio If my buildbot config is: jobs=2, nt_flags=['--cflag', '-mcpu=cortex-a15', '--use-perf', '--threads=1', '--build-threads=2'] It uses -j2 for build, -j2 for running the tests: buildmaster.tcwglab.linaro.org/builders/clang-native-arm-lnt-perf/builds/34/steps/test-suite/logs/stdio But, on the production buildbot, my config is: jobs=2, nt_flags=['--cflag', '-mcpu=cortex-a15', '--use-perf', '--threads=1', '--build-threads=2'] and it uses -j2 to build and -j1 to run the tests: lab.llvm.org:8014/builders/clang-native-arm-lnt-perf/builds/98/steps/test-suite/logs/stdio My master has up-to-date Zorg as of yesterday. The only change I did was to comment out the unused slaves. What am I doing wrong? cheers, --renato
Chris Matthews via llvm-dev
2016-Sep-20 00:17 UTC
[llvm-dev] Benchmark LNT weird thread behaviour
There seems to be three interesting flags in your LNT invocations: -j2 --threads=1 --build-threads=4 Since -j and --threads are the same flag, I think that is the problem. The first -j flag is at the start of the LNT command, perhaps that is coming from a different part of zorg? On September 19, 2016 at 3:48:55 PM, Renato Golin via llvm-dev (llvm-dev at lists.llvm.org) wrote: Hi James/Chris, You guys have done this before, so I'm guessing you can help me understand what's going on. If my buildbot config is: jobs=2, nt_flags=['--cflag', '-mcpu=cortex-a15', '--use-perf', '--threads=1', '--build-threads=4'] It uses -j4 for build, -j2 for running the tests: buildmaster.tcwglab.linaro.org/builders/clang-native-arm-lnt-perf/builds/35/steps/test-suite/logs/stdio If my buildbot config is: jobs=4, nt_flags=['--cflag', '-mcpu=cortex-a15', '--use-perf', '--threads=1', '--build-threads=4'] It uses -j4 for build, -j4 for running the tests: buildmaster.tcwglab.linaro.org/builders/clang-native-arm-lnt-perf/builds/33/steps/test-suite/logs/stdio If my buildbot config is: jobs=2, nt_flags=['--cflag', '-mcpu=cortex-a15', '--use-perf', '--threads=1', '--build-threads=2'] It uses -j2 for build, -j2 for running the tests: buildmaster.tcwglab.linaro.org/builders/clang-native-arm-lnt-perf/builds/34/steps/test-suite/logs/stdio But, on the production buildbot, my config is: jobs=2, nt_flags=['--cflag', '-mcpu=cortex-a15', '--use-perf', '--threads=1', '--build-threads=2'] and it uses -j2 to build and -j1 to run the tests: lab.llvm.org:8014/builders/clang-native-arm-lnt-perf/builds/98/steps/test-suite/logs/stdio My master has up-to-date Zorg as of yesterday. The only change I did was to comment out the unused slaves. What am I doing wrong? cheers, --renato _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <lists.llvm.org/pipermail/llvm-dev/attachments/20160919/854a8c33/attachment.html>
Renato Golin via llvm-dev
2016-Sep-20 11:29 UTC
[llvm-dev] Benchmark LNT weird thread behaviour
On 20 September 2016 at 01:17, Chris Matthews <chris.matthews at apple.com> wrote:> There seems to be three interesting flags in your LNT invocations: > > -j2 --threads=1 --build-threads=4 > > Since -j and --threads are the same flag, I think that is the problem. The > first -j flag is at the start of the LNT command, perhaps that is coming > from a different part of zorg?-j2 is from "jobs=2" in the builder's argument list, to compile Clang. This also gets passed to LNT, since nt_flags is not required, and you do want to use all cores if nothing is specified. Of course, when --threads is specified, you have the conflict, but since this argument is not required, I wouldn't want to omit the -j2 from LNT at all. I could "fix" this in our builder to not pass -jN into LNT and always use nt_flags, but I don't think that is a good fix, since other users will have odd behaviour on their side. Unless that's the kind of behaviour we *want* to encode, of course. What do you think? cheers, --renato
Chris Matthews via llvm-dev
2016-Sep-21 00:33 UTC
[llvm-dev] Benchmark LNT weird thread behaviour
I think every job should define those or use the LNT default of 1,1. The validity of compile time and exec time metrics is in question if the job is loaded incorrectly, so it makes sense to me to not allow that -j to get passed through. On September 20, 2016 at 4:30:04 AM, Renato Golin via llvm-dev (llvm-dev at lists.llvm.org) wrote: On 20 September 2016 at 01:17, Chris Matthews <chris.matthews at apple.com> wrote:> There seems to be three interesting flags in your LNT invocations: > > -j2 --threads=1 --build-threads=4 > > Since -j and --threads are the same flag, I think that is the problem. The > first -j flag is at the start of the LNT command, perhaps that is coming > from a different part of zorg?-j2 is from "jobs=2" in the builder's argument list, to compile Clang. This also gets passed to LNT, since nt_flags is not required, and you do want to use all cores if nothing is specified. Of course, when --threads is specified, you have the conflict, but since this argument is not required, I wouldn't want to omit the -j2 from LNT at all. I could "fix" this in our builder to not pass -jN into LNT and always use nt_flags, but I don't think that is a good fix, since other users will have odd behaviour on their side. Unless that's the kind of behaviour we *want* to encode, of course. What do you think? cheers, --renato _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <lists.llvm.org/pipermail/llvm-dev/attachments/20160920/2254d736/attachment.html>
Renato Golin via llvm-dev
2016-Sep-21 09:33 UTC
[llvm-dev] Benchmark LNT weird thread behaviour
On 21 September 2016 at 01:33, Chris Matthews <chris.matthews at apple.com> wrote:> I think every job should define those or use the LNT default of 1,1. The > validity of compile time and exec time metrics is in question if the job is > loaded incorrectly, so it makes sense to me to not allow that -j to get > passed through.The job's default *is* -j1. But we pass -jN to the other steps (compiling Clang, for instance). We also pass -jN to LNT, because that means both build and execute in one go. I'll change the buildbots to pass both explicitly in nt_flags, and will also change the builder to not pass -j in any case. But if users should not be passing -jN, but instead --threads and --build-threads directly, than I think we should make it into an error in LNT, no? cheers, --renato
Chris Matthews via llvm-dev
2016-Sep-21 19:57 UTC
[llvm-dev] Benchmark LNT weird thread behaviour
-j and —threads are the same flag. The problem is passing it twice with two different values. I am sort of surprised OptionParser let that past. On September 21, 2016 at 2:33:48 AM, Renato Golin via llvm-dev (llvm-dev at lists.llvm.org) wrote: On 21 September 2016 at 01:33, Chris Matthews <chris.matthews at apple.com> wrote:> I think every job should define those or use the LNT default of 1,1. The > validity of compile time and exec time metrics is in question if the job is > loaded incorrectly, so it makes sense to me to not allow that -j to get > passed through.The job's default *is* -j1. But we pass -jN to the other steps (compiling Clang, for instance). We also pass -jN to LNT, because that means both build and execute in one go. I'll change the buildbots to pass both explicitly in nt_flags, and will also change the builder to not pass -j in any case. But if users should not be passing -jN, but instead --threads and --build-threads directly, than I think we should make it into an error in LNT, no? cheers, --renato _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <lists.llvm.org/pipermail/llvm-dev/attachments/20160921/58e0dbd0/attachment.html>
Renato Golin via llvm-dev
2016-Sep-21 23:16 UTC
[llvm-dev] Benchmark LNT weird thread behaviour
Right, so the bots now pass threads directly, so the worst part is over. Now, being the same thing, I wonder if we just deprecate -j and emit an error, or a warning of which got chosen, or silently prefer - - threads over - j if both are emitted. I don't think the current behaviour of - j silently overriding - - threads is a good model, though. Cheers, Renato On 21 Sep 2016 8:57 p.m., "Chris Matthews" <chris.matthews at apple.com> wrote: -j and —threads are the same flag. The problem is passing it twice with two different values. I am sort of surprised OptionParser let that past. On September 21, 2016 at 2:33:48 AM, Renato Golin via llvm-dev ( llvm-dev at lists.llvm.org) wrote: On 21 September 2016 at 01:33, Chris Matthews <chris.matthews at apple.com> wrote:> I think every job should define those or use the LNT default of 1,1. The > validity of compile time and exec time metrics is in question if the jobis> loaded incorrectly, so it makes sense to me to not allow that -j to get > passed through.The job's default *is* -j1. But we pass -jN to the other steps (compiling Clang, for instance). We also pass -jN to LNT, because that means both build and execute in one go. I'll change the buildbots to pass both explicitly in nt_flags, and will also change the builder to not pass -j in any case. But if users should not be passing -jN, but instead --threads and --build-threads directly, than I think we should make it into an error in LNT, no? cheers, --renato _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <lists.llvm.org/pipermail/llvm-dev/attachments/20160922/54c878ae/attachment.html>
Chris Matthews via llvm-dev
2016-Sep-22 00:30 UTC
[llvm-dev] Benchmark LNT weird thread behaviour
I think it best to just drop -j totally. I don’t know if we can easily convince OptionParser to make passing both an error condition. On September 21, 2016 at 4:16:32 PM, Renato Golin via llvm-dev (llvm-dev at lists.llvm.org) wrote: Right, so the bots now pass threads directly, so the worst part is over. Now, being the same thing, I wonder if we just deprecate -j and emit an error, or a warning of which got chosen, or silently prefer - - threads over - j if both are emitted. I don't think the current behaviour of - j silently overriding - - threads is a good model, though. Cheers, Renato On 21 Sep 2016 8:57 p.m., "Chris Matthews" <chris.matthews at apple.com> wrote: -j and —threads are the same flag. The problem is passing it twice with two different values. I am sort of surprised OptionParser let that past. On September 21, 2016 at 2:33:48 AM, Renato Golin via llvm-dev (llvm-dev at lists.llvm.org) wrote: On 21 September 2016 at 01:33, Chris Matthews <chris.matthews at apple.com> wrote:> I think every job should define those or use the LNT default of 1,1. The > validity of compile time and exec time metrics is in question if the job is > loaded incorrectly, so it makes sense to me to not allow that -j to get > passed through.The job's default *is* -j1. But we pass -jN to the other steps (compiling Clang, for instance). We also pass -jN to LNT, because that means both build and execute in one go. I'll change the buildbots to pass both explicitly in nt_flags, and will also change the builder to not pass -j in any case. But if users should not be passing -jN, but instead --threads and --build-threads directly, than I think we should make it into an error in LNT, no? cheers, --renato _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <lists.llvm.org/pipermail/llvm-dev/attachments/20160921/8653217f/attachment.html>