mike tancsa
2019-Jul-24 20:56 UTC
Buildworld times (was Re: svn commit: r350256 - in stable/12: . contrib/compiler-rt/lib/sanitizer_common contrib/libunwind/src contrib/llvm/lib/DebugInfo/DWARF contrib/llvm/lib/MC contrib/llvm/lib/Object contrib/llvm/lib/Ta...)
On 7/24/2019 1:21 PM, mike tancsa wrote:> On 7/24/2019 12:02 PM, Dimitry Andric wrote: >> On 24 Jul 2019, at 17:12, mike tancsa <mike at sentex.net> wrote: >>> On 7/24/2019 9:45 AM, Dimitry Andric wrote: >>>> On 24 Jul 2019, at 14:47, mike tancsa <mike at sentex.net> wrote: >>>>> On 7/23/2019 2:40 PM, Dimitry Andric wrote: >>>>>> Author: dim >>>>>> Date: Tue Jul 23 18:40:32 2019 >>>>>> New Revision: 350256 >>>>>> URL: https://svnweb.freebsd.org/changeset/base/350256 >>>>>> >>>>> Hi, >>>>> >>>>> Not sure if this is just a difference in the versions or more things >>>>> are being compiled, but I noticed a buildworld on my Ryzen went from ~ >>>>> 31min to just over 40min. Is this expected ? >>>> Most likely this is because it will now build a bootstrap compiler, >>>> whereas previously your system may have skipped this step. Can you >>>> compare the results when setting MK_SYSTEM_COMPILER=no and >>>> MK_SYSTEM_LINKER=no ? >>> Adding those two to src.conf and make.conf still shows 40min >>> >>> time make -j14 buildworld > /var/log/build.out ; time make -j14 >>> buildkernel > /var/log/build.out. >>> [Creating objdir /usr/obj/usr/src/amd64.amd64...] >>> 25312.621u 1237.984s 40:09.72 1101.8% 45579+3473k 656666+3288880io >>> 214633pf+0w >>> 1675.467u 173.497s 2:37.90 1170.9% 37728+3167k 207118+3329460io >>> 61829pf+0w >>> >>> Should I be setting these vars somewhere else ? >> Ah, setting them in src.conf, make.conf or the environment will be >> overridden from Makefile.inc1, unfortunately. It will then show >> something like: >> >> make[1]: "/home/dim/src/stable-12/Makefile.inc1" line 343: SYSTEM_COMPILER: libclang will be built for bootstrapping a cross-compiler. >> make[1]: "/home/dim/src/stable-12/Makefile.inc1" line 348: SYSTEM_LINKER: libclang will be built for bootstrapping a cross-linker. >> >> E.g, it detects that your system compiler is of a lower version than >> the compiler in the source tree, and will thus bootstrap the latter. >> >> Can you compare the timings when setting MK_SYSTEM_COMPILER=yes and >> MK_SYSTEM_LINKER=yes? In that case, both before and after r350256, the >> results should be fairly similar, I expect. > odd, its still taking the same 40min > > ?# time make? -j14 buildworld > /var/log/build.out ; time make -j14 > buildkernel > > /var/log/build.out.???????????????????????????????????????????? > [Creating objdir /usr/obj/usr/src/amd64.amd64...] > 25281.564u 1233.340s 40:31.08 1090.6%?? 45595+3474k 633698+3288574io > 213653pf+0w > 1675.796u 172.082s 2:38.19 1168.1%????? 37746+3170k 205909+3329072io > 61654pf+0w > > > Looking at /var/log/build.out, the top line show > > # head /var/log/build.out > --- buildworld --- > make[1]: "/usr/src/Makefile.inc1" line 343: SYSTEM_COMPILER: libclang > will be built for bootstrapping a cross-compiler. > make[1]: "/usr/src/Makefile.inc1" line 348: SYSTEM_LINKER: libclang will > be built for bootstrapping a cross-linker. > --- buildworld_prologue --- > > despite > > # cat /etc/src.conf /etc/make.conf > MK_SYSTEM_COMPILER=no > MK_SYSTEM_LINKER=no > KERNCONF=server > MK_SYSTEM_COMPILER=no > MK_SYSTEM_LINKER=noHmmm, is the logic reversed somehow ?? The good news is if nothing is defined, it does the right thing. ?# head -3? /var/log/build.out.2 --- buildworld --- make[1]: "/usr/src/Makefile.inc1" line 341: SYSTEM_COMPILER: Determined that CC=cc matches the source tree.? Not bootstrapping a cross-compiler. make[1]: "/usr/src/Makefile.inc1" line 346: SYSTEM_LINKER: Determined that LD=ld matches the source tree.? Not bootstrapping a cross-linker. # time make? -j14 buildworld > /var/log/build.out.2?????????????????????????????????????????????? [Creating objdir /usr/obj/usr/src/amd64.amd64...] 19068.507u 1118.866s 31:07.44 1081.0%?? 54443+3546k 579241+2828231io 173288pf+0w ?# cat /etc/src.conf /etc/make.conf MK_SYSTEM_COMPILER=yes MK_SYSTEM_LINKER=yes KERNCONF=server MK_SYSTEM_COMPILER=yes MK_SYSTEM_LINKER=yes If I comment them out, ?head -4 /var/log/build.out.3-no-defs-in-src-or-make --- buildworld --- make[1]: "/usr/src/Makefile.inc1" line 341: SYSTEM_COMPILER: Determined that CC=cc matches the source tree.? Not bootstrapping a cross-compiler. make[1]: "/usr/src/Makefile.inc1" line 346: SYSTEM_LINKER: Determined that LD=ld matches the source tree.? Not bootstrapping a cross-linker. --- buildworld_prologue --- # cat /etc/src.conf /etc/make.conf #MK_SYSTEM_COMPILER=yes #MK_SYSTEM_LINKER=yes KERNCONF=server #MK_SYSTEM_COMPILER=yes #MK_SYSTEM_LINKER=yes vs ?head -4 /var/log/build.out.3-no-in-src-dot-conf --- buildworld --- make[1]: "/usr/src/Makefile.inc1" line 343: SYSTEM_COMPILER: libclang will be built for bootstrapping a cross-compiler. make[1]: "/usr/src/Makefile.inc1" line 348: SYSTEM_LINKER: libclang will be built for bootstrapping a cross-linker. --- buildworld_prologue --- ?# cat /etc/src.conf /etc/make.conf MK_SYSTEM_COMPILER=no MK_SYSTEM_LINKER=no KERNCONF=server MK_SYSTEM_COMPILER=no MK_SYSTEM_LINKER=no
Dimitry Andric
2019-Jul-24 21:21 UTC
Buildworld times (was Re: svn commit: r350256 - in stable/12: . contrib/compiler-rt/lib/sanitizer_common contrib/libunwind/src contrib/llvm/lib/DebugInfo/DWARF contrib/llvm/lib/MC contrib/llvm/lib/Object contrib/llvm/lib/Ta...)
On 24 Jul 2019, at 22:56, mike tancsa <mike at sentex.net> wrote:> > On 7/24/2019 1:21 PM, mike tancsa wrote: >> On 7/24/2019 12:02 PM, Dimitry Andric wrote:...>> # cat /etc/src.conf /etc/make.conf >> MK_SYSTEM_COMPILER=no >> MK_SYSTEM_LINKER=no >> KERNCONF=server >> MK_SYSTEM_COMPILER=no >> MK_SYSTEM_LINKER=no > > Hmmm, is the logic reversed somehow ? The good news is if nothing is > defined, it does the right thing.The idea is that the default is to *not* bootstrap the compiler, if the system compiler is new enough. E.g. if you build r350256 from a system built before r350256, it will normally automatically bootstrap everything. E.g., your previous builds did not have to bootstrap, and now they do, which is why they take longer. So the only good way to compare is to force MK_SYSTEM_COMPILER=yes and MK_SYSTEM_LINKER=yes, so both buildworlds will do the same thing. I did a few tests on a relatively fast machine, and buildworld with those settings on took approximately the same time at r350255 and r350256. I'm now repeating those experiments to feed the results to ministat. -Dimitry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 223 bytes Desc: Message signed with OpenPGP URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20190724/9a07fdbd/attachment.sig>