llvm dev - May 2016 - RFC: Should the default LLVM build be deterministic?

Hello LLVM-Dev,

Today if a user checks out LLVM, configures with no options specified, and
builds the result is non-deterministic. Meaning if you clean and build again the
binaries are not identical. This impacts all target platforms equally and is
caused by the default value of LLVM_ENABLE_TIMESTAMPS being On.

I believe this is incorrect behavior, and I’m curious what the wider community
thinks.

It is my belief that the default LLVM configuration (meaning no options
specified) should be deterministic, and that any option in our
build/configuration system that can impact determinism should be disabled by
default.

I would like to hold this as a central requirement for build system changes.

Thoughts?
-Chris

I don't think that "deterministic" is the right term (i.e.
"current time" is an input of the build).
Last month I spent a few hours debugging this (was trying to assert some
"NFC" changes), so I definitely agree with you that the "current
time" being part of the input of the build by default is not great, and I
support disabling this.

-- 
Mehdi
> On May 3, 2016, at 8:57 AM, Chris Bieneman via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hello LLVM-Dev,
> 
> Today if a user checks out LLVM, configures with no options specified, and
builds the result is non-deterministic. Meaning if you clean and build again the
binaries are not identical. This impacts all target platforms equally and is
caused by the default value of LLVM_ENABLE_TIMESTAMPS being On.
> 
> I believe this is incorrect behavior, and I’m curious what the wider
community thinks.
> 
> It is my belief that the default LLVM configuration (meaning no options
specified) should be deterministic, and that any option in our
build/configuration system that can impact determinism should be disabled by
default.
> 
> I would like to hold this as a central requirement for build system
changes.
> 
> Thoughts?
> -Chris
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org>
writes:
> Chris Bieneman via llvm-dev <llvm-dev at lists.llvm.org> writes:
>> It is my belief that the default LLVM configuration (meaning no
>> options specified) should be deterministic, and that any option in our
>> build/configuration system that can impact determinism should be
>> disabled by default.
>
> I don't think that "deterministic" is the right term (i.e.
"current
> time" is an input of the build).
> Last month I spent a few hours debugging this (was trying to assert
> some "NFC" changes), so I definitely agree with you that the
"current
> time" being part of the input of the build by default is not great,
> and I support disabling this.
This is especially silly because we don't even use the timestamp for
anything particularly valuable - cl::opt just likes to include it in its
version output.

For this case I'd suggest we just remove the ENABLE_TIMESTAMPS option
entirely and stop using __DATE__ in CommandLine.cpp. If someone *really*
wants a timestamp in the version output of the llvm tools for some
reason they can inject one into LLVM_VERSION_INFO.

(and yes, I agree in general that it'd be nice if building the same
source twice gave the same results by default, so +1 to that)

I agree completely. I eventually found out how to disable timestamps in the
build, but this should definitely be the default.

On Tue, May 3, 2016 at 4:57 PM, Chris Bieneman via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hello LLVM-Dev,
>
> Today if a user checks out LLVM, configures with no options specified, and
> builds the result is non-deterministic. Meaning if you clean and build
> again the binaries are not identical. This impacts all target platforms
> equally and is caused by the default value of LLVM_ENABLE_TIMESTAMPS being
> On.
>
> I believe this is incorrect behavior, and I’m curious what the wider
> community thinks.
>
> It is my belief that the default LLVM configuration (meaning no options
> specified) should be deterministic, and that any option in our
> build/configuration system that can impact determinism should be disabled
> by default.
>
> I would like to hold this as a central requirement for build system
> changes.
>
> Thoughts?
> -Chris
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160503/0fc6d885/attachment.html>

On 3 May 2016 at 17:53, Justin Bogner via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> This is especially silly because we don't even use the timestamp for
> anything particularly valuable - cl::opt just likes to include it in its
> version output.
>
> For this case I'd suggest we just remove the ENABLE_TIMESTAMPS option
> entirely and stop using __DATE__ in CommandLine.cpp. If someone *really*
> wants a timestamp in the version output of the llvm tools for some
> reason they can inject one into LLVM_VERSION_INFO.
+1

--renato

> This is especially silly because we don't even use the timestamp for
> anything particularly valuable - cl::opt just likes to include it in its
> version output.
>
> For this case I'd suggest we just remove the ENABLE_TIMESTAMPS option
> entirely and stop using __DATE__ in CommandLine.cpp. If someone *really*
> wants a timestamp in the version output of the llvm tools for some
> reason they can inject one into LLVM_VERSION_INFO.
+1 for deleting the option.

Cheers,
Rafael

On Tue, May 3, 2016 at 8:57 AM, Chris Bieneman via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hello LLVM-Dev,
>
> Today if a user checks out LLVM, configures with no options specified, and
> builds the result is non-deterministic. Meaning if you clean and build
> again the binaries are not identical. This impacts all target platforms
> equally and is caused by the default value of LLVM_ENABLE_TIMESTAMPS being
> On.
>
> I believe this is incorrect behavior, and I’m curious what the wider
> community thinks.
>
> It is my belief that the default LLVM configuration (meaning no options
> specified) should be deterministic, and that any option in our
> build/configuration system that can impact determinism should be disabled
> by default.
>
> I would like to hold this as a central requirement for build system
> changes.
>
+1 for determinism

-- Sean Silva

>
> Thoughts?
> -Chris
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160503/d29a347f/attachment.html>

The responses here are pretty one-sided.

What I’m going to do is make a commit to LLVM this afternoon to change the
default for LLVM_ENABLE_TIMESTAMPS to Off. Separately I will send out a patch
for review that removes the flag.

I will let that patch sit out on LLVM-Commits for a day or so to allow anyone
who objects to the removal of the option to speak up.

Any objections?

-Chris
> On May 3, 2016, at 1:11 PM, Sean Silva <chisophugis at gmail.com>
wrote:
> 
> 
> 
> On Tue, May 3, 2016 at 8:57 AM, Chris Bieneman via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> Hello LLVM-Dev,
> 
> Today if a user checks out LLVM, configures with no options specified, and
builds the result is non-deterministic. Meaning if you clean and build again the
binaries are not identical. This impacts all target platforms equally and is
caused by the default value of LLVM_ENABLE_TIMESTAMPS being On.
> 
> I believe this is incorrect behavior, and I’m curious what the wider
community thinks.
> 
> It is my belief that the default LLVM configuration (meaning no options
specified) should be deterministic, and that any option in our
build/configuration system that can impact determinism should be disabled by
default.
> 
> I would like to hold this as a central requirement for build system
changes.
> 
> +1 for determinism
> 
> -- Sean Silva
>  
> 
> Thoughts?
> -Chris
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160503/f01ada02/attachment-0001.html>

> On May 3, 2016, at 1:11 PM, Sean Silva via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> 
> 
> On Tue, May 3, 2016 at 8:57 AM, Chris Bieneman via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> Hello LLVM-Dev,
> 
> Today if a user checks out LLVM, configures with no options specified, and
builds the result is non-deterministic. Meaning if you clean and build again the
binaries are not identical. This impacts all target platforms equally and is
caused by the default value of LLVM_ENABLE_TIMESTAMPS being On.
> 
> I believe this is incorrect behavior, and I’m curious what the wider
community thinks.
> 
> It is my belief that the default LLVM configuration (meaning no options
specified) should be deterministic, and that any option in our
build/configuration system that can impact determinism should be disabled by
default.
> 
> I would like to hold this as a central requirement for build system
changes.
> 
> +1 for determinism
+1

-Chris

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160503/5498dd69/attachment.html>