llvm dev - May 2019 - Proposal to add preprocessor warning for unused command line macros

This is a proposal for either adding a new, or updating an existing
command line option such that a diagnostic can optionally be produced
for unused -D macros.

Long-lived large projects with thousands of files and many
contributors have a tendency to accumulate build options over time. As
time passes some build options like macros become replaced, obsolete,
or simply no longer used.

At the same time it is becoming more and more common that build
environments such as Bazel are used to try to minimize the build
effort and more importantly speed up builds. Build acceleration tools
commonly use the build options, and among them macros, to determine if
the output from a build is equivalent or not with a later build of the
same compilation unit.

The commands ...

$ clang -DFOO example.c

.. and ...

$ clang -DBAR example.c

... produce 100% identical output, since neither macro FOO nor BAR
affect the compilation unit. However, the build acceleration tools
that do not run the preprocessor will not detect that the input to the
compilation stage is identical and so will treat the output as unequal
and run the build two times, when once would have been sufficient.

There are two command line options that at present do not produce a
diagnostic for an unused command line macro, but which might be
expanded to do this task also;

-Wunused-macros

Produces diagnostic only for macros defined in the main source code
file that are not subsequenty used. No diagnostic for command line
macros, and no diagnostic for macros defined in header files included
by the source code file.

-Wunused-command-line-argument

Produces diagnostic for command line arguments that are not
applicable. But there is no diagnostic for command line macros that
are not used.

An alternative might be to add a specific option for use by the
preprocessor to produce a diagnostic for macros defined on the command
line, but not used by the source code file. This is my proposed named
for such an option;

-Wunused-command-line-macros

It is more likely than not that some of the command line macros are
intended to be consumed only by header files included by the source
code file, and not used at all in the source code itself. So the
option should be be implemented such that only macros occurring on the
command line >after< the option are subject to produce a diagnostic,
if not used. So for example;

$ clang -DFOO -Wunused-command-line-macros -DBAR example.c

... if we assume the "example.c" source code file uses neither of the
two macros FOO and BAR, a diagnostic will only be produced for BAR,
and none for FOO.

Is this a reasonable proposal? Comments?

An alternate solution might be for the preprocessor to dump out a list
of macros it has seen during processing. The build acceleration tools
can then hoover up this information in order to make more informed
decisions on which build options actually have an effect on the
produced output.

Thank you for your attention,
Mats Bengtsson

My concern would be that most build systems have their command line
arguments built up for the whole project - (eg: I want the 32 bit API
to this library, so I add -DFOO_LIB_32 to my project command line
arguments). The specific compilations that use that macro vary over
time, depending on which parts of the project use the library (maybe I
start with the usage in one source, then refactor it into anotehr
file, etc).

So I'm not sure warning on unused would suit - thinking of them sort
of like a generic header file that all the code includes - and we
can't warn about the unused macros in such a header because maybe in
one situation it's included only a fraction of its macros are used.
(same as other unused entities in headers/on the command line).

- Dave

On Mon, May 6, 2019 at 1:26 PM Mats Bengtsson via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> This is a proposal for either adding a new, or updating an existing
> command line option such that a diagnostic can optionally be produced
> for unused -D macros.
>
> Long-lived large projects with thousands of files and many
> contributors have a tendency to accumulate build options over time. As
> time passes some build options like macros become replaced, obsolete,
> or simply no longer used.
>
> At the same time it is becoming more and more common that build
> environments such as Bazel are used to try to minimize the build
> effort and more importantly speed up builds. Build acceleration tools
> commonly use the build options, and among them macros, to determine if
> the output from a build is equivalent or not with a later build of the
> same compilation unit.
>
> The commands ...
>
> $ clang -DFOO example.c
>
> .. and ...
>
> $ clang -DBAR example.c
>
> ... produce 100% identical output, since neither macro FOO nor BAR
> affect the compilation unit. However, the build acceleration tools
> that do not run the preprocessor will not detect that the input to the
> compilation stage is identical and so will treat the output as unequal
> and run the build two times, when once would have been sufficient.
>
> There are two command line options that at present do not produce a
> diagnostic for an unused command line macro, but which might be
> expanded to do this task also;
>
> -Wunused-macros
>
> Produces diagnostic only for macros defined in the main source code
> file that are not subsequenty used. No diagnostic for command line
> macros, and no diagnostic for macros defined in header files included
> by the source code file.
>
> -Wunused-command-line-argument
>
> Produces diagnostic for command line arguments that are not
> applicable. But there is no diagnostic for command line macros that
> are not used.
>
> An alternative might be to add a specific option for use by the
> preprocessor to produce a diagnostic for macros defined on the command
> line, but not used by the source code file. This is my proposed named
> for such an option;
>
> -Wunused-command-line-macros
>
> It is more likely than not that some of the command line macros are
> intended to be consumed only by header files included by the source
> code file, and not used at all in the source code itself. So the
> option should be be implemented such that only macros occurring on the
> command line >after< the option are subject to produce a diagnostic,
> if not used. So for example;
>
> $ clang -DFOO -Wunused-command-line-macros -DBAR example.c
>
> ... if we assume the "example.c" source code file uses neither of
the
> two macros FOO and BAR, a diagnostic will only be produced for BAR,
> and none for FOO.
>
> Is this a reasonable proposal? Comments?
>
> An alternate solution might be for the preprocessor to dump out a list
> of macros it has seen during processing. The build acceleration tools
> can then hoover up this information in order to make more informed
> decisions on which build options actually have an effect on the
> produced output.
>
> Thank you for your attention,
> Mats Bengtsson
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On Mon, 6 May 2019 at 21:26, Mats Bengtsson via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> Is this a reasonable proposal? Comments?
This is more of a topic for cfe-dev.

Tim.

To not unnecessarily produce a diagnostic for macros that are project
wide, my proposal suggest that the warning option be positional. By
that I mean that macros specified on the command line before the
warning option will not produce a diagnostic regardless if they are
used or not in the source code file. Only those macros appearing after
the warning option are subject to produce a diagnostic if not used at
least once.

See the last example in my original post.

It is precisely because you want to be able to determine if a macro is
actually used, and if so where it is used, that I am proposing this
warning option.

Note that I have copied the original post over to cfe-dev, so please
continue posting there if you have comments and questions.

Link to cfe-dev post:
https://groups.google.com/d/msg/cfe-dev/YjrbeSxVPI4/RfXqLY5EAQAJ

Den mån 6 maj 2019 kl 22:33 skrev David Blaikie <dblaikie at
gmail.com>:
>
> My concern would be that most build systems have their command line
> arguments built up for the whole project - (eg: I want the 32 bit API
> to this library, so I add -DFOO_LIB_32 to my project command line
> arguments). The specific compilations that use that macro vary over
> time, depending on which parts of the project use the library (maybe I
> start with the usage in one source, then refactor it into anotehr
> file, etc).
>
> So I'm not sure warning on unused would suit - thinking of them sort
> of like a generic header file that all the code includes - and we
> can't warn about the unused macros in such a header because maybe in
> one situation it's included only a fraction of its macros are used.
> (same as other unused entities in headers/on the command line).
>
> - Dave
>
> On Mon, May 6, 2019 at 1:26 PM Mats Bengtsson via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> >
> > This is a proposal for either adding a new, or updating an existing
> > command line option such that a diagnostic can optionally be produced
> > for unused -D macros.
> >
> > Long-lived large projects with thousands of files and many
> > contributors have a tendency to accumulate build options over time. As
> > time passes some build options like macros become replaced, obsolete,
> > or simply no longer used.
> >
> > At the same time it is becoming more and more common that build
> > environments such as Bazel are used to try to minimize the build
> > effort and more importantly speed up builds. Build acceleration tools
> > commonly use the build options, and among them macros, to determine if
> > the output from a build is equivalent or not with a later build of the
> > same compilation unit.
> >
> > The commands ...
> >
> > $ clang -DFOO example.c
> >
> > .. and ...
> >
> > $ clang -DBAR example.c
> >
> > ... produce 100% identical output, since neither macro FOO nor BAR
> > affect the compilation unit. However, the build acceleration tools
> > that do not run the preprocessor will not detect that the input to the
> > compilation stage is identical and so will treat the output as unequal
> > and run the build two times, when once would have been sufficient.
> >
> > There are two command line options that at present do not produce a
> > diagnostic for an unused command line macro, but which might be
> > expanded to do this task also;
> >
> > -Wunused-macros
> >
> > Produces diagnostic only for macros defined in the main source code
> > file that are not subsequenty used. No diagnostic for command line
> > macros, and no diagnostic for macros defined in header files included
> > by the source code file.
> >
> > -Wunused-command-line-argument
> >
> > Produces diagnostic for command line arguments that are not
> > applicable. But there is no diagnostic for command line macros that
> > are not used.
> >
> > An alternative might be to add a specific option for use by the
> > preprocessor to produce a diagnostic for macros defined on the command
> > line, but not used by the source code file. This is my proposed named
> > for such an option;
> >
> > -Wunused-command-line-macros
> >
> > It is more likely than not that some of the command line macros are
> > intended to be consumed only by header files included by the source
> > code file, and not used at all in the source code itself. So the
> > option should be be implemented such that only macros occurring on the
> > command line >after< the option are subject to produce a
diagnostic,
> > if not used. So for example;
> >
> > $ clang -DFOO -Wunused-command-line-macros -DBAR example.c
> >
> > ... if we assume the "example.c" source code file uses
neither of the
> > two macros FOO and BAR, a diagnostic will only be produced for BAR,
> > and none for FOO.
> >
> > Is this a reasonable proposal? Comments?
> >
> > An alternate solution might be for the preprocessor to dump out a list
> > of macros it has seen during processing. The build acceleration tools
> > can then hoover up this information in order to make more informed
> > decisions on which build options actually have an effect on the
> > produced output.
> >
> > Thank you for your attention,
> > Mats Bengtsson
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Point taken. The post has been copied to cfe-dev. Use the link below
to get to the post directly, and please do not reply to this post.

https://groups.google.com/d/msg/cfe-dev/YjrbeSxVPI4/RfXqLY5EAQAJ

Den mån 6 maj 2019 kl 22:42 skrev Tim Northover <t.p.northover at
gmail.com>:
>
> On Mon, 6 May 2019 at 21:26, Mats Bengtsson via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> > Is this a reasonable proposal? Comments?
>
> This is more of a topic for cfe-dev.
>
> Tim.