llvm dev - Jan 2016 - r250501 adds dependancy to ole32.dll on MSVC

r257499

On Sun, Jan 10, 2016 at 1:23 PM, Jakob Bornecrantz <wallbraker at
gmail.com>
wrote:
> On Thu, Dec 24, 2015 at 1:28 AM, Jakob Bornecrantz <wallbraker at
gmail.com>
> wrote:
> > On Wed, Dec 23, 2015 at 9:38 PM, Aaron Ballman <aaron at
aaronballman.com>
> wrote:
> >> On Wed, Dec 23, 2015 at 3:28 PM, Reid Kleckner <rnk at
google.com> wrote:
> >>> On Wed, Dec 23, 2015 at 11:25 AM, Aaron Ballman via llvm-dev
> >>> <llvm-dev at lists.llvm.org> wrote:
> >>>>
> >>>> On Wed, Dec 23, 2015 at 1:55 PM, Jakob Bornecrantz <
> wallbraker at gmail.com>
> >>>> wrote:
> >>>> > On Wed, Dec 23, 2015 at 5:49 PM, Aaron Ballman <
> aaron at aaronballman.com>
> >>>> > wrote:
> >>>> >> On Wed, Dec 23, 2015 at 11:29 AM, Jakob
Bornecrantz via llvm-dev
> >>>> >> <llvm-dev at lists.llvm.org> wrote:
> >>>> >>> I'm building on Windows x64 using cmake,
Ninja and VS 2013
> express on
> >>>> >>> Windows 7.
> >>>> >>>
> >>>> >>> So I have been using the LLVMSharp method on
getting a usable
> loadable
> >>>> >>> LLVM.dll[1][2].
> >>>
> >>>
> >>> One day, I would like to make it easier to make LLVM.dll...
> >
> > We (not I) have rewritten the powershell script into a python
> > script[1] and where hoping that we could commit that to LLVM and have
> > it make a DLL during regular MSVC builds. And if possible shipping
> > that DLL with the Windows package (and Windows snapshot builds[2]).
> >
> >>>
> >>>>
> >>>> > Further investigation show that for the MinGW build
OLE32 is added
> to
> >>>> > the dependencies for LLVMSupport.lib in the
CMakeLists.txt file,
> there
> >>>> > are a couple of other libraries added as well. This
lead me to find
> >>>> > that those where added for the MSVC build with a
pragma(lib, xx),
> so I
> >>>> > added such a pragma for OLE32 and that fixes my
script that is
> >>>> > generating LLVM.dll, will also fix other peoples
problems with OLE32
> >>>> > missing as well.
> >>>>
> >>>> Generally speaking, static libraries like LLVMSupport are
just an
> >>>> archive of a bunch of compiled object files, and whatever
ultimately
> >>>> consumes that library is responsible for providing
external
> >>>> definitions. So, for instance, clang.exe consumes
LLVMSupport.lib and
> >>>> so it also links in OLE32. I believe that we usually only
use
> >>>> #pragma(lib) to signal that non-standard libraries need to
be linked
> >>>> in, otherwise every source file in an archive would wind
up with an
> >>>> unwieldy number of pragmas for all its imports, or you
would have to
> >>>> manually maintain that list in some common header file. I
don't think
> >>>> that the pragma(lib) for advapi32.lib should be there
either, FWIW,
> >>>> but I don't know that we've ever had a
hard-and-fast rule for this
> >>>> sort of thing.
> >>>
> >>>
> >>> Sure, but the fact that static libraries can't encode
their
> dependencies has
> >>> always been an annoying missing feature, not something that we
want to
> >>> follow if we can avoid it.
> >>
> >> Agreed.
> >>
> >>>>
> >>>> I'm not opposed to this patch, per se, but it feels
like a slippery
> >>>> slope as to what makes the cut and what does not. I would
rather see
> >>>> *less* non-standard pragma usage instead of more.
> >>>
> >>>
> >>> For these kinds of DLLs that are available on all modern
versions of
> >>> Windows, I think it's perfectly reasonable to use the
'pragma comment
> lib'
> >>> auto linking mechanism. Realistically, no consumer of
LLVMSupport.lib
> is
> >>> going to be surprised if it needs ole32.dll.
> >>
> >> I think we're making the same point -- no consumer of
LLVMSupport.lib
> >> should expect to work if it doesn't use the stock set of
library
> >> dependencies. So use #pragma(lib) for everything that's not in
that
> >> set, but expect consumers to at least be using the stock set.
Unless
> >> you're suggesting we stick a #pragma(lib, user32) (et al)
somewhere as
> >> well?
> >
> > So ole32 and friends are added by default by cmake[3] which is why you
> > are seeing them on the executables.
> >
> > Looking through the source I find advapi32.lib, psapi.lib, shell32.lib
> > and dbghelp.lib are added via pragmas. Of these psapi.lib and
> > dghelp.lib are not added to the default libs by cmake[3]. So just
> > removing the pragmas would break the build without adding them back
> > into the linking libs in cmake. Tho it would break all non-cmake based
> > builds (that are using LLVM components) out there that don't
happen to
> > also link those libs. That is for versions < 3.8 non-cmake builds
> > worked magically thanks to the pragmas.
> >
> > I have no real say in what path you should take here. Pragmas for me
> > would be the easiest for me, since they just magically works, tho I
> > appreciate the technical ugliness they present. That said if you
> > remove the pragmas the list of libraries needed should be documented
> > somewhere and probably also be returned by llvm-config.exe.
> >
> > If we keep the pragmas I guess we could have does link.exe add the
> > libs by default as a basis for which pragmas we should add in. I
can't
> > experiment right now but user32, kernel32 and friends seems be
> > included by default by link.exe or they are pragmas in the MSVC
> > headers that the LLVM build picks.
> >
> > Thanks again for your time.
>
> Ping?
>
> Cheers, Jakob.
>
> >
> > [1] https://github.com/VoltLang/GenLLVMDLLPy/blob/master/GenLLVMDLL.py
> > [2] http://llvm.org/builds/
> > [3]
>
https://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/Platform/Windows-MSVC.cmake
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160112/163bf120/attachment.html>

I am not comfortable with this commit. I was under the mistaken
impression that we were in agreement on not linking in default
libraries like OLE32.dll, so this commit is a bit of a surprise. I
don't think it needs to be immediately reverted (though if this is
still being discussed by the time we do the 3.8 freeze, I would like
it reverted so we're not committing to supporting it until we're
ready), but I really would like to understand the design guidance. I
don't see why OLE32.dll is special, while user32.dll is not. Right now
we are very inconsistent, we use the pragma for psapi (twice, not a
default lib), dbghelp (not not a default lib), shell32 (default lib),
advapi32 (default lib), and now OLE32 (default lib).

If we don't want default libs, I would like to see shell32, ole32, and
advapi32 pragmas removed. If we do want default libs, I would like to
(1) find a common place to put all of these pragmas for the default
libs (and perhaps the nondefault libs?) so we don't needlessly
duplicate them, and (2) add pragmas for all of the standard default
libraries we rely on, and (3) set /NODEFAULTLIB to be clear that we
are managing dependencies manually. If we don't want any pragmas, we
should remove all of them and update the CMake files appropriately.

~Aaron

On Tue, Jan 12, 2016 at 1:37 PM, Reid Kleckner <rnk at google.com>
wrote:
> r257499
>
> On Sun, Jan 10, 2016 at 1:23 PM, Jakob Bornecrantz <wallbraker at
gmail.com>
> wrote:
>>
>> On Thu, Dec 24, 2015 at 1:28 AM, Jakob Bornecrantz <wallbraker at
gmail.com>
>> wrote:
>> > On Wed, Dec 23, 2015 at 9:38 PM, Aaron Ballman <aaron at
aaronballman.com>
>> > wrote:
>> >> On Wed, Dec 23, 2015 at 3:28 PM, Reid Kleckner <rnk at
google.com> wrote:
>> >>> On Wed, Dec 23, 2015 at 11:25 AM, Aaron Ballman via
llvm-dev
>> >>> <llvm-dev at lists.llvm.org> wrote:
>> >>>>
>> >>>> On Wed, Dec 23, 2015 at 1:55 PM, Jakob Bornecrantz
>> >>>> <wallbraker at gmail.com>
>> >>>> wrote:
>> >>>> > On Wed, Dec 23, 2015 at 5:49 PM, Aaron Ballman
>> >>>> > <aaron at aaronballman.com>
>> >>>> > wrote:
>> >>>> >> On Wed, Dec 23, 2015 at 11:29 AM, Jakob
Bornecrantz via llvm-dev
>> >>>> >> <llvm-dev at lists.llvm.org> wrote:
>> >>>> >>> I'm building on Windows x64 using
cmake, Ninja and VS 2013
>> >>>> >>> express on
>> >>>> >>> Windows 7.
>> >>>> >>>
>> >>>> >>> So I have been using the LLVMSharp method
on getting a usable
>> >>>> >>> loadable
>> >>>> >>> LLVM.dll[1][2].
>> >>>
>> >>>
>> >>> One day, I would like to make it easier to make
LLVM.dll...
>> >
>> > We (not I) have rewritten the powershell script into a python
>> > script[1] and where hoping that we could commit that to LLVM and
have
>> > it make a DLL during regular MSVC builds. And if possible shipping
>> > that DLL with the Windows package (and Windows snapshot
builds[2]).
>> >
>> >>>
>> >>>>
>> >>>> > Further investigation show that for the MinGW
build OLE32 is added
>> >>>> > to
>> >>>> > the dependencies for LLVMSupport.lib in the
CMakeLists.txt file,
>> >>>> > there
>> >>>> > are a couple of other libraries added as well.
This lead me to find
>> >>>> > that those where added for the MSVC build with a
pragma(lib, xx),
>> >>>> > so I
>> >>>> > added such a pragma for OLE32 and that fixes my
script that is
>> >>>> > generating LLVM.dll, will also fix other peoples
problems with
>> >>>> > OLE32
>> >>>> > missing as well.
>> >>>>
>> >>>> Generally speaking, static libraries like LLVMSupport
are just an
>> >>>> archive of a bunch of compiled object files, and
whatever ultimately
>> >>>> consumes that library is responsible for providing
external
>> >>>> definitions. So, for instance, clang.exe consumes
LLVMSupport.lib and
>> >>>> so it also links in OLE32. I believe that we usually
only use
>> >>>> #pragma(lib) to signal that non-standard libraries
need to be linked
>> >>>> in, otherwise every source file in an archive would
wind up with an
>> >>>> unwieldy number of pragmas for all its imports, or you
would have to
>> >>>> manually maintain that list in some common header
file. I don't think
>> >>>> that the pragma(lib) for advapi32.lib should be there
either, FWIW,
>> >>>> but I don't know that we've ever had a
hard-and-fast rule for this
>> >>>> sort of thing.
>> >>>
>> >>>
>> >>> Sure, but the fact that static libraries can't encode
their
>> >>> dependencies has
>> >>> always been an annoying missing feature, not something
that we want to
>> >>> follow if we can avoid it.
>> >>
>> >> Agreed.
>> >>
>> >>>>
>> >>>> I'm not opposed to this patch, per se, but it
feels like a slippery
>> >>>> slope as to what makes the cut and what does not. I
would rather see
>> >>>> *less* non-standard pragma usage instead of more.
>> >>>
>> >>>
>> >>> For these kinds of DLLs that are available on all modern
versions of
>> >>> Windows, I think it's perfectly reasonable to use the
'pragma comment
>> >>> lib'
>> >>> auto linking mechanism. Realistically, no consumer of
LLVMSupport.lib
>> >>> is
>> >>> going to be surprised if it needs ole32.dll.
>> >>
>> >> I think we're making the same point -- no consumer of
LLVMSupport.lib
>> >> should expect to work if it doesn't use the stock set of
library
>> >> dependencies. So use #pragma(lib) for everything that's
not in that
>> >> set, but expect consumers to at least be using the stock set.
Unless
>> >> you're suggesting we stick a #pragma(lib, user32) (et al)
somewhere as
>> >> well?
>> >
>> > So ole32 and friends are added by default by cmake[3] which is why
you
>> > are seeing them on the executables.
>> >
>> > Looking through the source I find advapi32.lib, psapi.lib,
shell32.lib
>> > and dbghelp.lib are added via pragmas. Of these psapi.lib and
>> > dghelp.lib are not added to the default libs by cmake[3]. So just
>> > removing the pragmas would break the build without adding them
back
>> > into the linking libs in cmake. Tho it would break all non-cmake
based
>> > builds (that are using LLVM components) out there that don't
happen to
>> > also link those libs. That is for versions < 3.8 non-cmake
builds
>> > worked magically thanks to the pragmas.
>> >
>> > I have no real say in what path you should take here. Pragmas for
me
>> > would be the easiest for me, since they just magically works, tho
I
>> > appreciate the technical ugliness they present. That said if you
>> > remove the pragmas the list of libraries needed should be
documented
>> > somewhere and probably also be returned by llvm-config.exe.
>> >
>> > If we keep the pragmas I guess we could have does link.exe add the
>> > libs by default as a basis for which pragmas we should add in. I
can't
>> > experiment right now but user32, kernel32 and friends seems be
>> > included by default by link.exe or they are pragmas in the MSVC
>> > headers that the LLVM build picks.
>> >
>> > Thanks again for your time.
>>
>> Ping?
>>
>> Cheers, Jakob.
>>
>> >
>> > [1]
https://github.com/VoltLang/GenLLVMDLLPy/blob/master/GenLLVMDLL.py
>> > [2] http://llvm.org/builds/
>> > [3]
>> >
https://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/Platform/Windows-MSVC.cmake
>
>

My feeling is that using the autolink pragma is consistent with LLVM being
distributed primarily as static libraries, and having no runtime
dependencies other than the base system libraries. All these DLLs are
"system libraries" similar to libc on Linux or libSystem.dylib on Mac.

Users should be able to take the LLVM libraries and link their own
executable, without adding extra linker flags, i.e. 'cl myapp.cpp
LLVMSupport.lib' or 'gcc myapp.cpp libLLVMSupport.a'.

I wasn't aware of the distinction between default libs and non-default libs
before you brought it up. I just think it's nice for users to not have to
screw around with flags. I don't feel strongly, and if you want to
standardize on only auto-linking non-default system libs (advapi32 and
dbghelp), that sounds good to me.

On Tue, Jan 12, 2016 at 11:25 AM, Aaron Ballman <aaron at
aaronballman.com>
wrote:
> I am not comfortable with this commit. I was under the mistaken
> impression that we were in agreement on not linking in default
> libraries like OLE32.dll, so this commit is a bit of a surprise. I
> don't think it needs to be immediately reverted (though if this is
> still being discussed by the time we do the 3.8 freeze, I would like
> it reverted so we're not committing to supporting it until we're
> ready), but I really would like to understand the design guidance. I
> don't see why OLE32.dll is special, while user32.dll is not. Right now
> we are very inconsistent, we use the pragma for psapi (twice, not a
> default lib), dbghelp (not not a default lib), shell32 (default lib),
> advapi32 (default lib), and now OLE32 (default lib).
>
> If we don't want default libs, I would like to see shell32, ole32, and
> advapi32 pragmas removed. If we do want default libs, I would like to
> (1) find a common place to put all of these pragmas for the default
> libs (and perhaps the nondefault libs?) so we don't needlessly
> duplicate them, and (2) add pragmas for all of the standard default
> libraries we rely on, and (3) set /NODEFAULTLIB to be clear that we
> are managing dependencies manually. If we don't want any pragmas, we
> should remove all of them and update the CMake files appropriately.
>
> ~Aaron
>
> On Tue, Jan 12, 2016 at 1:37 PM, Reid Kleckner <rnk at google.com>
wrote:
> > r257499
> >
> > On Sun, Jan 10, 2016 at 1:23 PM, Jakob Bornecrantz <wallbraker at
gmail.com
> >
> > wrote:
> >>
> >> On Thu, Dec 24, 2015 at 1:28 AM, Jakob Bornecrantz <
> wallbraker at gmail.com>
> >> wrote:
> >> > On Wed, Dec 23, 2015 at 9:38 PM, Aaron Ballman <
> aaron at aaronballman.com>
> >> > wrote:
> >> >> On Wed, Dec 23, 2015 at 3:28 PM, Reid Kleckner <rnk at
google.com>
> wrote:
> >> >>> On Wed, Dec 23, 2015 at 11:25 AM, Aaron Ballman via
llvm-dev
> >> >>> <llvm-dev at lists.llvm.org> wrote:
> >> >>>>
> >> >>>> On Wed, Dec 23, 2015 at 1:55 PM, Jakob
Bornecrantz
> >> >>>> <wallbraker at gmail.com>
> >> >>>> wrote:
> >> >>>> > On Wed, Dec 23, 2015 at 5:49 PM, Aaron
Ballman
> >> >>>> > <aaron at aaronballman.com>
> >> >>>> > wrote:
> >> >>>> >> On Wed, Dec 23, 2015 at 11:29 AM, Jakob
Bornecrantz via llvm-dev
> >> >>>> >> <llvm-dev at lists.llvm.org>
wrote:
> >> >>>> >>> I'm building on Windows x64
using cmake, Ninja and VS 2013
> >> >>>> >>> express on
> >> >>>> >>> Windows 7.
> >> >>>> >>>
> >> >>>> >>> So I have been using the LLVMSharp
method on getting a usable
> >> >>>> >>> loadable
> >> >>>> >>> LLVM.dll[1][2].
> >> >>>
> >> >>>
> >> >>> One day, I would like to make it easier to make
LLVM.dll...
> >> >
> >> > We (not I) have rewritten the powershell script into a python
> >> > script[1] and where hoping that we could commit that to LLVM
and have
> >> > it make a DLL during regular MSVC builds. And if possible
shipping
> >> > that DLL with the Windows package (and Windows snapshot
builds[2]).
> >> >
> >> >>>
> >> >>>>
> >> >>>> > Further investigation show that for the
MinGW build OLE32 is
> added
> >> >>>> > to
> >> >>>> > the dependencies for LLVMSupport.lib in the
CMakeLists.txt file,
> >> >>>> > there
> >> >>>> > are a couple of other libraries added as
well. This lead me to
> find
> >> >>>> > that those where added for the MSVC build
with a pragma(lib, xx),
> >> >>>> > so I
> >> >>>> > added such a pragma for OLE32 and that fixes
my script that is
> >> >>>> > generating LLVM.dll, will also fix other
peoples problems with
> >> >>>> > OLE32
> >> >>>> > missing as well.
> >> >>>>
> >> >>>> Generally speaking, static libraries like
LLVMSupport are just an
> >> >>>> archive of a bunch of compiled object files, and
whatever
> ultimately
> >> >>>> consumes that library is responsible for
providing external
> >> >>>> definitions. So, for instance, clang.exe consumes
LLVMSupport.lib
> and
> >> >>>> so it also links in OLE32. I believe that we
usually only use
> >> >>>> #pragma(lib) to signal that non-standard
libraries need to be
> linked
> >> >>>> in, otherwise every source file in an archive
would wind up with an
> >> >>>> unwieldy number of pragmas for all its imports,
or you would have
> to
> >> >>>> manually maintain that list in some common header
file. I don't
> think
> >> >>>> that the pragma(lib) for advapi32.lib should be
there either, FWIW,
> >> >>>> but I don't know that we've ever had a
hard-and-fast rule for this
> >> >>>> sort of thing.
> >> >>>
> >> >>>
> >> >>> Sure, but the fact that static libraries can't
encode their
> >> >>> dependencies has
> >> >>> always been an annoying missing feature, not
something that we want
> to
> >> >>> follow if we can avoid it.
> >> >>
> >> >> Agreed.
> >> >>
> >> >>>>
> >> >>>> I'm not opposed to this patch, per se, but it
feels like a slippery
> >> >>>> slope as to what makes the cut and what does not.
I would rather
> see
> >> >>>> *less* non-standard pragma usage instead of more.
> >> >>>
> >> >>>
> >> >>> For these kinds of DLLs that are available on all
modern versions of
> >> >>> Windows, I think it's perfectly reasonable to use
the 'pragma
> comment
> >> >>> lib'
> >> >>> auto linking mechanism. Realistically, no consumer of
> LLVMSupport.lib
> >> >>> is
> >> >>> going to be surprised if it needs ole32.dll.
> >> >>
> >> >> I think we're making the same point -- no consumer of
LLVMSupport.lib
> >> >> should expect to work if it doesn't use the stock set
of library
> >> >> dependencies. So use #pragma(lib) for everything
that's not in that
> >> >> set, but expect consumers to at least be using the stock
set. Unless
> >> >> you're suggesting we stick a #pragma(lib, user32) (et
al) somewhere
> as
> >> >> well?
> >> >
> >> > So ole32 and friends are added by default by cmake[3] which
is why you
> >> > are seeing them on the executables.
> >> >
> >> > Looking through the source I find advapi32.lib, psapi.lib,
shell32.lib
> >> > and dbghelp.lib are added via pragmas. Of these psapi.lib and
> >> > dghelp.lib are not added to the default libs by cmake[3]. So
just
> >> > removing the pragmas would break the build without adding
them back
> >> > into the linking libs in cmake. Tho it would break all
non-cmake based
> >> > builds (that are using LLVM components) out there that
don't happen to
> >> > also link those libs. That is for versions < 3.8 non-cmake
builds
> >> > worked magically thanks to the pragmas.
> >> >
> >> > I have no real say in what path you should take here. Pragmas
for me
> >> > would be the easiest for me, since they just magically works,
tho I
> >> > appreciate the technical ugliness they present. That said if
you
> >> > remove the pragmas the list of libraries needed should be
documented
> >> > somewhere and probably also be returned by llvm-config.exe.
> >> >
> >> > If we keep the pragmas I guess we could have does link.exe
add the
> >> > libs by default as a basis for which pragmas we should add
in. I can't
> >> > experiment right now but user32, kernel32 and friends seems
be
> >> > included by default by link.exe or they are pragmas in the
MSVC
> >> > headers that the LLVM build picks.
> >> >
> >> > Thanks again for your time.
> >>
> >> Ping?
> >>
> >> Cheers, Jakob.
> >>
> >> >
> >> > [1]
> https://github.com/VoltLang/GenLLVMDLLPy/blob/master/GenLLVMDLL.py
> >> > [2] http://llvm.org/builds/
> >> > [3]
> >> >
>
https://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/Platform/Windows-MSVC.cmake
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160112/57b2f074/attachment.html>