freebsd stable - Jul 2019 - Buildworld times (was Re: svn commit: r350256 - in stable/12: . contrib/compiler-rt/lib/sanitizer_common contrib/libunwind/src contrib/llvm/lib/DebugInfo/DWARF contrib/llvm/lib/MC contrib/llvm/lib/Object contrib/llvm/lib/Ta...)

On 7/24/2019 12:02 PM, Dimitry Andric wrote:
> On 24 Jul 2019, at 17:12, mike tancsa <mike at sentex.net> wrote:
>> On 7/24/2019 9:45 AM, Dimitry Andric wrote:
>>> On 24 Jul 2019, at 14:47, mike tancsa <mike at sentex.net>
wrote:
>>>> On 7/23/2019 2:40 PM, Dimitry Andric wrote:
>>>>> Author: dim
>>>>> Date: Tue Jul 23 18:40:32 2019
>>>>> New Revision: 350256
>>>>> URL: https://svnweb.freebsd.org/changeset/base/350256
>>>>>
>>>> Hi,
>>>>
>>>>    Not sure if this is just a difference in the versions or
more things
>>>> are being compiled, but I noticed a buildworld on my Ryzen went
from ~
>>>> 31min to just over 40min.   Is this expected ?
>>> Most likely this is because it will now build a bootstrap compiler,
>>> whereas previously your system may have skipped this step.  Can you
>>> compare the results when setting MK_SYSTEM_COMPILER=no and
>>> MK_SYSTEM_LINKER=no ?
>> Adding those two to src.conf and make.conf still shows 40min
>>
>>  time make -j14 buildworld > /var/log/build.out ; time make -j14
>> buildkernel > /var/log/build.out.
>> [Creating objdir /usr/obj/usr/src/amd64.amd64...]
>> 25312.621u 1237.984s 40:09.72 1101.8%   45579+3473k 656666+3288880io
>> 214633pf+0w
>> 1675.467u 173.497s 2:37.90 1170.9%      37728+3167k 207118+3329460io
>> 61829pf+0w
>>
>> Should I be setting these vars somewhere else ?
> Ah, setting them in src.conf, make.conf or the environment will be
> overridden from Makefile.inc1, unfortunately.  It will then show
> something like:
>
> make[1]: "/home/dim/src/stable-12/Makefile.inc1" line 343:
SYSTEM_COMPILER: libclang will be built for bootstrapping a cross-compiler.
> make[1]: "/home/dim/src/stable-12/Makefile.inc1" line 348:
SYSTEM_LINKER: libclang will be built for bootstrapping a cross-linker.
>
> E.g, it detects that your system compiler is of a lower version than
> the compiler in the source tree, and will thus bootstrap the latter.
>
> Can you compare the timings when setting MK_SYSTEM_COMPILER=yes and
> MK_SYSTEM_LINKER=yes?  In that case, both before and after r350256, the
> results should be fairly similar, I expect.
odd, its still taking the same 40min

?# time make? -j14 buildworld > /var/log/build.out ; time make -j14
buildkernel >
/var/log/build.out.????????????????????????????????????????????
[Creating objdir /usr/obj/usr/src/amd64.amd64...]
25281.564u 1233.340s 40:31.08 1090.6%?? 45595+3474k 633698+3288574io
213653pf+0w
1675.796u 172.082s 2:38.19 1168.1%????? 37746+3170k 205909+3329072io
61654pf+0w


Looking at /var/log/build.out, the top line show

# head /var/log/build.out
--- buildworld ---
make[1]: "/usr/src/Makefile.inc1" line 343: SYSTEM_COMPILER: libclang
will be built for bootstrapping a cross-compiler.
make[1]: "/usr/src/Makefile.inc1" line 348: SYSTEM_LINKER: libclang
will
be built for bootstrapping a cross-linker.
--- buildworld_prologue ---

despite

# cat /etc/src.conf /etc/make.conf
MK_SYSTEM_COMPILER=no
MK_SYSTEM_LINKER=no
KERNCONF=server
MK_SYSTEM_COMPILER=no
MK_SYSTEM_LINKER=no

On 7/24/2019 1:21 PM, mike tancsa wrote:
> On 7/24/2019 12:02 PM, Dimitry Andric wrote:
>> On 24 Jul 2019, at 17:12, mike tancsa <mike at sentex.net> wrote:
>>> On 7/24/2019 9:45 AM, Dimitry Andric wrote:
>>>> On 24 Jul 2019, at 14:47, mike tancsa <mike at
sentex.net> wrote:
>>>>> On 7/23/2019 2:40 PM, Dimitry Andric wrote:
>>>>>> Author: dim
>>>>>> Date: Tue Jul 23 18:40:32 2019
>>>>>> New Revision: 350256
>>>>>> URL: https://svnweb.freebsd.org/changeset/base/350256
>>>>>>
>>>>> Hi,
>>>>>
>>>>>    Not sure if this is just a difference in the versions or
more things
>>>>> are being compiled, but I noticed a buildworld on my Ryzen
went from ~
>>>>> 31min to just over 40min.   Is this expected ?
>>>> Most likely this is because it will now build a bootstrap
compiler,
>>>> whereas previously your system may have skipped this step.  Can
you
>>>> compare the results when setting MK_SYSTEM_COMPILER=no and
>>>> MK_SYSTEM_LINKER=no ?
>>> Adding those two to src.conf and make.conf still shows 40min
>>>
>>>  time make -j14 buildworld > /var/log/build.out ; time make -j14
>>> buildkernel > /var/log/build.out.
>>> [Creating objdir /usr/obj/usr/src/amd64.amd64...]
>>> 25312.621u 1237.984s 40:09.72 1101.8%   45579+3473k
656666+3288880io
>>> 214633pf+0w
>>> 1675.467u 173.497s 2:37.90 1170.9%      37728+3167k
207118+3329460io
>>> 61829pf+0w
>>>
>>> Should I be setting these vars somewhere else ?
>> Ah, setting them in src.conf, make.conf or the environment will be
>> overridden from Makefile.inc1, unfortunately.  It will then show
>> something like:
>>
>> make[1]: "/home/dim/src/stable-12/Makefile.inc1" line 343:
SYSTEM_COMPILER: libclang will be built for bootstrapping a cross-compiler.
>> make[1]: "/home/dim/src/stable-12/Makefile.inc1" line 348:
SYSTEM_LINKER: libclang will be built for bootstrapping a cross-linker.
>>
>> E.g, it detects that your system compiler is of a lower version than
>> the compiler in the source tree, and will thus bootstrap the latter.
>>
>> Can you compare the timings when setting MK_SYSTEM_COMPILER=yes and
>> MK_SYSTEM_LINKER=yes?  In that case, both before and after r350256, the
>> results should be fairly similar, I expect.
> odd, its still taking the same 40min
>
> ?# time make? -j14 buildworld > /var/log/build.out ; time make -j14
> buildkernel >
> /var/log/build.out.????????????????????????????????????????????
> [Creating objdir /usr/obj/usr/src/amd64.amd64...]
> 25281.564u 1233.340s 40:31.08 1090.6%?? 45595+3474k 633698+3288574io
> 213653pf+0w
> 1675.796u 172.082s 2:38.19 1168.1%????? 37746+3170k 205909+3329072io
> 61654pf+0w
>
>
> Looking at /var/log/build.out, the top line show
>
> # head /var/log/build.out
> --- buildworld ---
> make[1]: "/usr/src/Makefile.inc1" line 343: SYSTEM_COMPILER:
libclang
> will be built for bootstrapping a cross-compiler.
> make[1]: "/usr/src/Makefile.inc1" line 348: SYSTEM_LINKER:
libclang will
> be built for bootstrapping a cross-linker.
> --- buildworld_prologue ---
>
> despite
>
> # cat /etc/src.conf /etc/make.conf
> MK_SYSTEM_COMPILER=no
> MK_SYSTEM_LINKER=no
> KERNCONF=server
> MK_SYSTEM_COMPILER=no
> MK_SYSTEM_LINKER=no
Hmmm, is the logic reversed somehow ?? The good news is if nothing is
defined, it does the right thing.

?# head -3? /var/log/build.out.2
--- buildworld ---
make[1]: "/usr/src/Makefile.inc1" line 341: SYSTEM_COMPILER:
Determined
that CC=cc matches the source tree.? Not bootstrapping a cross-compiler.
make[1]: "/usr/src/Makefile.inc1" line 346: SYSTEM_LINKER: Determined
that LD=ld matches the source tree.? Not bootstrapping a cross-linker.


# time make? -j14 buildworld >
/var/log/build.out.2??????????????????????????????????????????????
[Creating objdir /usr/obj/usr/src/amd64.amd64...]
19068.507u 1118.866s 31:07.44 1081.0%?? 54443+3546k 579241+2828231io
173288pf+0w

?# cat /etc/src.conf /etc/make.conf
MK_SYSTEM_COMPILER=yes
MK_SYSTEM_LINKER=yes
KERNCONF=server
MK_SYSTEM_COMPILER=yes
MK_SYSTEM_LINKER=yes


If I comment them out,

?head -4 /var/log/build.out.3-no-defs-in-src-or-make
--- buildworld ---
make[1]: "/usr/src/Makefile.inc1" line 341: SYSTEM_COMPILER:
Determined
that CC=cc matches the source tree.? Not bootstrapping a cross-compiler.
make[1]: "/usr/src/Makefile.inc1" line 346: SYSTEM_LINKER: Determined
that LD=ld matches the source tree.? Not bootstrapping a cross-linker.

--- buildworld_prologue ---

# cat /etc/src.conf /etc/make.conf
#MK_SYSTEM_COMPILER=yes
#MK_SYSTEM_LINKER=yes
KERNCONF=server
#MK_SYSTEM_COMPILER=yes
#MK_SYSTEM_LINKER=yes


vs

?head -4 /var/log/build.out.3-no-in-src-dot-conf
--- buildworld ---
make[1]: "/usr/src/Makefile.inc1" line 343: SYSTEM_COMPILER: libclang
will be built for bootstrapping a cross-compiler.
make[1]: "/usr/src/Makefile.inc1" line 348: SYSTEM_LINKER: libclang
will
be built for bootstrapping a cross-linker.
--- buildworld_prologue ---

?# cat /etc/src.conf /etc/make.conf
MK_SYSTEM_COMPILER=no
MK_SYSTEM_LINKER=no
KERNCONF=server
MK_SYSTEM_COMPILER=no
MK_SYSTEM_LINKER=no