llvm dev - May 2013 - [LLVMdev] Feedback required on proper dllexport/import implementation

On 23.04.2013 19:10, Gao, Yunzhong wrote:
> I missed the discussion when I implemented dllexport/dllimport for our
local tree. I
> essentially implemented your approach#1. I was trying to avoid the various
> external_linkage + some_attribute approaches because it seems that
external_linkage
> would imply the external linkage without the dllimport/dllexport semantics,
and there
> may be existing compiler codes that rely on this semantics. If I used
external_linkage
> + some attribute, then any place in existing codes that uses
hasExternalLInkage() will
> have to be modified to check both the linkage type as well as the attribute
list, which
> seems a bit fragile to maintain; somebody somewhere will forget to check
the attributes
> when writing their new optimization or garbage collection passes.
What problems do you expect with another approach? I have a local 
prototype that removes dllimport/export as linkage and uses function 
attributes and an extension to globals. I decorate dllexported functions 
with the Used attribute and it seems to work fine. This allows such 
functions to have linkonce_odr and weak_odr linkage and not be discarded.
> I think dllimport inline functions are more closely related to the
available_externally
> linkage than linkonce_odr; no COMDAT is involved and no .linkonce discard
directive
> is needed. I would name it dllimport_inline or something similar.
Hm, I don't think so, because dllimported inline functions must never be 
instantiated, or else you get duplicated symbols. They are only allowed 
to be expanded. At the moment I solve this by checking for definitions 
during codegen (this means they survived the inliner) and dropping their 
body.
> While it makes sense to mimic MSVC behavior and MSDN doc for C++ codes, it
is
> not very clear to me what should be the correct semantics when a C99 inline
function
> carries a dllimport/dllexport declspec. [...]
> An alternative is to generate an error message whenever a C99 inline
function is used
> with a dllimport/dllexport declspec.
I agree. This should be rejected.
> In a similar manner, I wonder what should be the correct semantics of:
> inline __declspec(dllexport) int foo() { return 100; }
>
> I guess I could let dllexport imply "extern", so that an
out-of-line definition always gets
> generated, but is that the expected behavior?
Yes, such a function is always instanciated and exported.
> For the following test case, GCC ignores the dllimport attribute with a
warning (similar
> to clang's current behavior)
>
> /* test1.c */
> inline __declspec(dllimport) void foo();
> inline void foo();
> inline void foo() { }
> void foo_user() { foo(); }
> /* end of file */
Generally, MSVC allows a dllimport definition if the first declaration 
is specified as inline. This test case should import foo() (or expand it 
inline).
> But for the following two cases, GCC produces both an out-of-line
definition and also a
> reference to an dllimported symbol foo with no diagnostics. It seems wrong,
unless
> there is some problem with my build process.
>
> /* test2.c */
> inline void foo();
> __declspec(dllimport) void foo();
> inline void foo() { }
> void foo_user() { foo(); }
> /* end of file */
Having both seems wrong, as it leads to linker errors. MSVC rejects this 
code because it does not allow redeclaring a function and adding dllimport.
> /* test3.c */
> inline void foo();
> __declspec(dllimport) void foo();
> __declspec(dllimport) void foo() { } // clang complains that this is a
redeclaration without
>                                                          //
"dllimport" attribute", which seems wrong...
> void foo_user() { foo(); }
> /* end of file */
Same as above.
> What are your thoughts?
I have one issue in mind where I wonder if it's beneficial to be less 
strict than MSVC. As stated before, MSVC requires the inline keyword 
together with dllimport. This extends to member functions and prevents 
out-of-line definitions with inline:

   struct X1 {
     inline void foo();
     void bar();
   };
   void X::foo() {} // OK
   inline void X::bar() {} // Error

Same goes for class and function templates. I stumbled on this in libc++ 
where internal class templates are externed.


-Nico

Hi Nico,
Thank you for your feedback. I'll try to answer your questions below.
> What problems do you expect with another approach? I have a local
> prototype that removes dllimport/export as linkage and uses function
> attributes and an extension to globals. I decorate dllexported functions
with
> the Used attribute and it seems to work fine. This allows such functions to
> have linkonce_odr and weak_odr linkage and not be discarded.
My primary concern is what needs to be added to preserve current
compiler behavior where necessary. All the other approaches appear
to involve adding flags or attribute to an existing linkage type, so any
existing compiler codes that deal with that particular linkage type will
potentially have different behavior. For example, if you replace the
DLLExportLinkage with ExternalLinkage+DLLExport_attribute, then
codes that check hasExternalLinkage() will affect DLLExportLinkage
symbols as well, which may or may not be the right thing. For example,
LTO::addDefinedSymbol() currently does not handle dllimport/dllexport
symbols, but if you change the semantics of ExternalLinkage to include
dllimport/dllexport, then you could potentially have changed LTO
(although, that is not necessarily a bad thing; you just need to check
those places to make sure the compiler behavior still makes sense after
your change). And some contributors who have not caught up with
your change may still submit patches with the old semantics in mind,
which seems prone to errors.

If you really like your approach, I wonder if it makes sense to rename
the existing linkage type to something different, e.g., OldLinkage to
NewLinkage, so that future patches using hasOldLinkage() will have
to check their codes to make sure they work with the new semantics.
> Hm, I don't think so, because dllimported inline functions must never
be
> instantiated, or else you get duplicated symbols. They are only allowed to
be
> expanded. At the moment I solve this by checking for definitions during
> codegen (this means they survived the inliner) and dropping their body.
I agree. Hmm that is exactly why I said that dllimport inline functions bear
some similarity to the existing available_externally linkage. Maybe some
miscommunication has happened? I will make a guess as to what you
were saying.

Do you mean that you are dropping the function definition during clang
codegen instead of llvm backend? If the function definition does not
appear in the IR file, then the inliner won't be able to inline-expand the
function at call site, which seems to defeat the purpose of having an
inline definition. I solve this by having a new linkage type which works
just like available_externally (only for inline expansion but not for
out-of-line instantiation).

If I guessed wrong, please tell more details about your approach.

> > /* test1.c */
> > inline __declspec(dllimport) void foo(); inline void foo(); inline
> > void foo() { } void foo_user() { foo(); }
> > /* end of file */
> 
> Generally, MSVC allows a dllimport definition if the first declaration is
> specified as inline. This test case should import foo() (or expand it
inline).
These examples are meant to demonstrate the theoretical issues when
C99 inline is used with dllexport/dllimport. MSVC does not support C99, so I
am not sure MSVC behavior is relevant. It appears to me that MSVC will use
C++ inline semantics even when compiling C files. It appears that both you
and Reid are okay with the approach of issuing an error message whenever a
C99 inline function is used with dllimport. You also seem okay with generating
an out-of-line, dllexported, definition when a C99 inline function is used with
dllexport.
> I have one issue in mind where I wonder if it's beneficial to be less
strict than
> MSVC. As stated before, MSVC requires the inline keyword together with
> dllimport. This extends to member functions and prevents out-of-line
> definitions with inline:
> 
>    struct X1 {
>      inline void foo();
>      void bar();
>    };
>    void X::foo() {} // OK
>    inline void X::bar() {} // Error
> 
> Same goes for class and function templates. I stumbled on this in libc++
> where internal class templates are externed.
I guess you meant
__declspec(dllimport) struct X1 {
  inline void foo();
  void bar();
};
And you meant to have this declaration in a C++ file instead of a C file.

I have no objection to adding support for this usage.

- Gao.

> My primary concern is what needs to be added to preserve current
> compiler behavior where necessary. All the other approaches appear
> to involve adding flags or attribute to an existing linkage type, so any
> existing compiler codes that deal with that particular linkage type will
> potentially have different behavior.
Right. But normally LLVM API does not provide backward compatibility.
The point is to make good code / API, not to fix the broken one with
hinges and supports.
> DLLExportLinkage with ExternalLinkage+DLLExport_attribute, then
> codes that check hasExternalLinkage() will affect DLLExportLinkage
> symbols as well, which may or may not be the right thing.
In this case everything should work fine. If the behavior will be
changed, then we'll find some subtle bug hiding there for ages. normal
dllexport'ed functions are essentially external functions with some
additional semantics.

 > LTO::addDefinedSymbol() currently does not handle
dllimport/dllexport
> symbols, but if you change the semantics of ExternalLinkage to include
> dllimport/dllexport, then you could potentially have changed LTO
> (although, that is not necessarily a bad thing; you just need to check
> those places to make sure the compiler behavior still makes sense after
> your change).
Right. The change should be thoroughly tested. Some fallout from such
invasive change is a normal thing, really.
> And some contributors who have not caught up with
> your change may still submit patches with the old semantics in mind,
> which seems prone to errors.
This is irrelevant here. They need to submit patches against ToT, so,
they will need to submit already "new" patches.
> If you really like your approach, I wonder if it makes sense to rename
> the existing linkage type to something different, e.g., OldLinkage to
> NewLinkage, so that future patches using hasOldLinkage() will have
> to check their codes to make sure they work with the new semantics.
The point here is that patches should come with tests. And this is how
we will make sure they will work with "new" semantics.
> I agree. Hmm that is exactly why I said that dllimport inline functions
bear
> some similarity to the existing available_externally linkage. Maybe some
> miscommunication has happened? I will make a guess as to what you
> were saying.
This is yet another point of not introducing new linkage types :)
Almost all the stuff can be represented with current linkage types +
some modeling of additional dll semantics.

--
With best regards, Anton Korobeynikov
Faculty of Mathematics and Mechanics, Saint Petersburg State University