llvm dev - Mar 2016 - Problem with mingw32 DLL build

On Tue, Mar 1, 2016 at 4:21 PM Reid Kleckner via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> First, I'd like to say it would be great if we could get away from
relying
> on these globally unique pass IDs represented as addresses of globals. Long
> ago I tried to hack up a DLL build of LLVM and quickly discovered that
> these IDs are the biggest source of dllimported data, which has to be
> annotated with __declspec(dllimport/dllexport).
>
> Here's the issue as I understand it:
> - LLVM today supports using mingw64 compilers with -DBUILD_SHARED_LIBS=ON.
> This produces a DLL per LLVM library, just like Unix.
> - In lib/Analysis, each analysis inherits from a CRTP base class
> AnalysisBase.
> - AnalysisBase is very simple, it exists solely to provide the unique pass
> ID in the form of a static data member.
> - Because the ID is part of a template, it is only emitted when
> referenced. This is the same on Windows and Unix, no compiler bug here.
>   - Previously, each analysis would manually provide it's own ID, and
the
> ID would be emitted exactly once in the TU providing the analysis.
> - The Windows loader is very simple, it does not merge symbols across DSOs
> as it does on Unix. As a result, the IDs are not unique, leading to runtime
> problems.
>   - The auto-dll tools provided by mingw might not let you get this far,
> though, I haven't actually seen the logs.
>
> You introduced AnalysisBase to eliminate the boilerplate of declaring and
> defining the ID of each analysis, and discovered that it just means we have
> to put back a different kind of boilerplate in the form of explicit
> template instantiations.
>
> I don't think we can overcome the boilerplate here, if we want to
support
> BUILD_SHARED_LIBS on Windows, regardless of compiler. It's a limitation
of
> DLLs, and a useful one because it avoids doing tons of useless symbol
> lookup at startup.
>
> So, the options are:
> 1. Go back to declaring IDs in each analysis. It's tried and true and
> doesn't have vague linkage like templates.
>
I don't disagree with your analysis, but I'd like to get a build bot
that
tests DLL builds with a host toolchain other than mingw32 if this is in
fact common to those configs (as no other bot broke that i saw)... But
that's a separate issue.

The base class isn't *just* providing the static variable. It is also
providing the accessor method. I really like using a method instead of raw
access to the static data member. It is also providing the name of the
pass. So what we'd likely end up with is still having the base class but
having a friend declaration and private static data member in the derived
class.

But before going there, I'd like to ask if there is another approach that
would work: do you see any ways to effectively cause the static data member
to be emitted reliably?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160301/63b84fd3/attachment.html>

On Tue, Mar 1, 2016 at 1:32 PM, Chandler Carruth <chandlerc at gmail.com>
wrote:
>
> I don't disagree with your analysis, but I'd like to get a build
bot that
> tests DLL builds with a host toolchain other than mingw32 if this is in
> fact common to those configs (as no other bot broke that i saw)... But
> that's a separate issue.
>
We don't support BUILD_SHARED_LIBS=ON with MSVC. It doesn't come with
tools
that make it easy to support this configuration. CMake recently added some
support for building DLLs that export all symbols, but it's very new and
requires you to annotate all your exported global data, i.e. pass ids.

The base class isn't *just* providing the static variable. It is
also
> providing the accessor method. I really like using a method instead of raw
> access to the static data member. It is also providing the name of the
> pass. So what we'd likely end up with is still having the base class
but
> having a friend declaration and private static data member in the derived
> class.
>
Sure, that'd work.

> But before going there, I'd like to ask if there is another approach
that
> would work: do you see any ways to effectively cause the static data member
> to be emitted reliably?
>
I'm sure we could come up with a way to get the ID emitted reliably at the
definition of the analysis, but we also need to ensure that the ID is
unique, which means we need something like the extern template declaration
in the header.

I think your idea of using CRTP to stamp out the pass name and ID accessor
is reasonable, and then we can use a normal C++ declaration/definition pair
in the derived class for the ID.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160301/122dfcc8/attachment.html>

I'm about to switch over the the conclusion here since I've already
gotten
this wrong a few times. =[

However, I want to say for the record that I don't really think we should
keep supporting this kind of DLL builds given the limitations of DLLs when
it comes to correctly implementing these aspects of C++. I suspect we
should really move toward having an LLVM.dll (and a LTO.dll, etc) and only
using static linking within that on Windows. (I know that actually
producing an LLVM.dll is a huge challenge currently due to CMake and other
issues.) Trying to support the fully shared build with the limitations of
DLLs when it only even somewhat works on one platform (mingw32, as other
things prevent it from working w/ MSVC from my understanding of
conversations with Reid) seems like a losing battle and to have a high cost
for very little gain for the community.

On Tue, Mar 1, 2016 at 11:09 PM Reid Kleckner via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Tue, Mar 1, 2016 at 1:32 PM, Chandler Carruth <chandlerc at
gmail.com>
> wrote:
>>
>> I don't disagree with your analysis, but I'd like to get a
build bot that
>> tests DLL builds with a host toolchain other than mingw32 if this is in
>> fact common to those configs (as no other bot broke that i saw)... But
>> that's a separate issue.
>>
>
> We don't support BUILD_SHARED_LIBS=ON with MSVC. It doesn't come
with
> tools that make it easy to support this configuration. CMake recently added
> some support for building DLLs that export all symbols, but it's very
new
> and requires you to annotate all your exported global data, i.e. pass ids.
>
> The base class isn't *just* providing the static variable. It is also
>> providing the accessor method. I really like using a method instead of
raw
>> access to the static data member. It is also providing the name of the
>> pass. So what we'd likely end up with is still having the base
class but
>> having a friend declaration and private static data member in the
derived
>> class.
>>
>
> Sure, that'd work.
>
>
>> But before going there, I'd like to ask if there is another
approach that
>> would work: do you see any ways to effectively cause the static data
member
>> to be emitted reliably?
>>
>
> I'm sure we could come up with a way to get the ID emitted reliably at
the
> definition of the analysis, but we also need to ensure that the ID is
> unique, which means we need something like the extern template declaration
> in the header.
>
> I think your idea of using CRTP to stamp out the pass name and ID accessor
> is reasonable, and then we can use a normal C++ declaration/definition pair
> in the derived class for the ID.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160310/c9092414/attachment.html>