llvm dev - Jul 2013 - [LLVMdev] MSVC++ ABI compatibility is not a Windows requirement (was: Proposing a new 'alloca' parameter attribute to implement the Microsoft C++ ABI)

> Quite the contrary, knowing that Clang's C++ ABI is completely
> incompatible with MS is a maintenance *simplification*.
Yes, for example, as explained by Bjarne once, incompatible name
mangling schemes for ABIs that are not guaranteed to be 100% binary
compatible is a _feature_, not a bug, since it prevents anyone from
even beginning to develop a workflow that relies upon linking possibly
incompatible binaries.
>
> Saying that supporting the MS C++ ABI is an uphill battle is an
> understatement (and better say MS C++ ABI*S*, because it evolved over
> time and it is known that it will change on future releases.) As far as
> I'm concerned, I'll never base my decisions on compiler usage on
the
> advertisement of MS compatibility by Clang++, becase I *know* that for a
> very long time (maybe forever) whatever MS C++ libraries that work with
> Clang++ is by luck. That's what happens when you try to implement
> compatibility with an undocumented, propietary, complex feature.
This is my point as well; it just seems odd to spend so much effort to
implement and undocumented ABI with no guarantee of support or
stability (outside of MSVC major version releases). Furthermore, I
feel that, by reinforcing the notion that use of the MSVC++ ABI is
required for development on Windows (when no system-provided libraries
or ABIs require it), Clang developers are hindering the adoption of
Clang on Windows rather than promoting it, since it is unlikely that
any party that is not personally invested in Clang development would
be willing to depend on a backward engineered implementation of a
"required" but undocumented ABI for production use. Furthermore, it
means that Clang will always be behind the curve on Windows, since,
even if MSVC++ ABI support is fully usable one day, no one will be
willing to link it with object files from a new major version of MSVC
without some critical mass of users being guinea pigs first and
ensuring that there are no major bugaboos.

I agree that there is value in supporting implementing full MSVC++
ABI, if it can be done, but it seems like that support can never be
100% complete (or, more to the point, known to be 100% complete)
unless Microsoft itself decides to officially support the
implementation or completely stabilize and document their C++ ABIs.
However, I personally think that it is not only easier, there is more
value in implementing a C++ ABI which is compatible with officially
supported C and COM MSVC++ ABI subsets and consciously _not_
compatible in others way (in particular, which does not even attempt
to link C++ symbols with MSVC++, since that is not required for COM).
This is something that can be made to work and guaranteed (barring
some seriously misguided behavior from Microsoft) to continue to work
stably.

Furthermore, by providing and controlling its own Windows C++ ABI:
Clang can possibly offer something that MSVC *cannot* (and does not
even try to do): a development platform and ecosystem which provides a
stable-over-time C++ ABI and consistent cross-module usage of RTTI,
STL, C++ memory management, etc. (which is dicey even within MSVC
major versions unless the exact same build settings are used, last
time I checked.) This would give someone an active reason to switch to
Clang more than anything else Clang currently offers on Windows.

Stephen

This thread is odd to me. It seems that the gist of your guys' argument is
that you don't know if we will ever get full support, therefore we don't
welcome progress towards that (very useful) goal/feature.

If the specific proposal doesn't make make sense from a design standpoint,
that's one thing, but saying we shouldn't take it because of licensing
issues with MFC or because it is harmful to be partially (but not fully)
compatible with MSVC seems just weird to me.

-Chris

On Jul 31, 2013, at 9:04 AM, Stephen Lin <swlin at post.harvard.edu>
wrote:
>> Quite the contrary, knowing that Clang's C++ ABI is completely
>> incompatible with MS is a maintenance *simplification*.
> 
> Yes, for example, as explained by Bjarne once, incompatible name
> mangling schemes for ABIs that are not guaranteed to be 100% binary
> compatible is a _feature_, not a bug, since it prevents anyone from
> even beginning to develop a workflow that relies upon linking possibly
> incompatible binaries.
> 
>> 
>> Saying that supporting the MS C++ ABI is an uphill battle is an
>> understatement (and better say MS C++ ABI*S*, because it evolved over
>> time and it is known that it will change on future releases.) As far as
>> I'm concerned, I'll never base my decisions on compiler usage
on the
>> advertisement of MS compatibility by Clang++, becase I *know* that for
a
>> very long time (maybe forever) whatever MS C++ libraries that work with
>> Clang++ is by luck. That's what happens when you try to implement
>> compatibility with an undocumented, propietary, complex feature.
> 
> This is my point as well; it just seems odd to spend so much effort to
> implement and undocumented ABI with no guarantee of support or
> stability (outside of MSVC major version releases). Furthermore, I
> feel that, by reinforcing the notion that use of the MSVC++ ABI is
> required for development on Windows (when no system-provided libraries
> or ABIs require it), Clang developers are hindering the adoption of
> Clang on Windows rather than promoting it, since it is unlikely that
> any party that is not personally invested in Clang development would
> be willing to depend on a backward engineered implementation of a
> "required" but undocumented ABI for production use. Furthermore,
it
> means that Clang will always be behind the curve on Windows, since,
> even if MSVC++ ABI support is fully usable one day, no one will be
> willing to link it with object files from a new major version of MSVC
> without some critical mass of users being guinea pigs first and
> ensuring that there are no major bugaboos.
> 
> I agree that there is value in supporting implementing full MSVC++
> ABI, if it can be done, but it seems like that support can never be
> 100% complete (or, more to the point, known to be 100% complete)
> unless Microsoft itself decides to officially support the
> implementation or completely stabilize and document their C++ ABIs.
> However, I personally think that it is not only easier, there is more
> value in implementing a C++ ABI which is compatible with officially
> supported C and COM MSVC++ ABI subsets and consciously _not_
> compatible in others way (in particular, which does not even attempt
> to link C++ symbols with MSVC++, since that is not required for COM).
> This is something that can be made to work and guaranteed (barring
> some seriously misguided behavior from Microsoft) to continue to work
> stably.
> 
> Furthermore, by providing and controlling its own Windows C++ ABI:
> Clang can possibly offer something that MSVC *cannot* (and does not
> even try to do): a development platform and ecosystem which provides a
> stable-over-time C++ ABI and consistent cross-module usage of RTTI,
> STL, C++ memory management, etc. (which is dicey even within MSVC
> major versions unless the exact same build settings are used, last
> time I checked.) This would give someone an active reason to switch to
> Clang more than anything else Clang currently offers on Windows.
> 
> Stephen
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

Oh, well, I don't actually have any objection to the patch (I'm not
sure if Oscar does) or work in this direction. (So apologies for
hijacking, it's just I wanted to back up the sentiment that Oscar
expressed initially.)

I'm honestly just trying to understand why the engineering focus is
where it is, and wonders if anyone has put any thought into supporting
our own (or possibly someone else's, if it's documented) C++ ABI on
Windows, as a either a subgoal or parallel goal of full MSVC C++ ABI
implementation. As I said before, something like that wouldn't just be
a stop gap, it would provide real benefits on its own.

I'll stop hijacking this thread for that, though, sorry.

Stephen

On Wed, Jul 31, 2013 at 9:54 AM, Chris Lattner <clattner at apple.com>
wrote:
> This thread is odd to me. It seems that the gist of your guys' argument
is that you don't know if we will ever get full support, therefore we
don't welcome progress towards that (very useful) goal/feature.
>
> If the specific proposal doesn't make make sense from a design
standpoint, that's one thing, but saying we shouldn't take it because of
licensing issues with MFC or because it is harmful to be partially (but not
fully) compatible with MSVC seems just weird to me.
>
> -Chris
>
> On Jul 31, 2013, at 9:04 AM, Stephen Lin <swlin at post.harvard.edu>
wrote:
>
>>> Quite the contrary, knowing that Clang's C++ ABI is completely
>>> incompatible with MS is a maintenance *simplification*.
>>
>> Yes, for example, as explained by Bjarne once, incompatible name
>> mangling schemes for ABIs that are not guaranteed to be 100% binary
>> compatible is a _feature_, not a bug, since it prevents anyone from
>> even beginning to develop a workflow that relies upon linking possibly
>> incompatible binaries.
>>
>>>
>>> Saying that supporting the MS C++ ABI is an uphill battle is an
>>> understatement (and better say MS C++ ABI*S*, because it evolved
over
>>> time and it is known that it will change on future releases.) As
far as
>>> I'm concerned, I'll never base my decisions on compiler
usage on the
>>> advertisement of MS compatibility by Clang++, becase I *know* that
for a
>>> very long time (maybe forever) whatever MS C++ libraries that work
with
>>> Clang++ is by luck. That's what happens when you try to
implement
>>> compatibility with an undocumented, propietary, complex feature.
>>
>> This is my point as well; it just seems odd to spend so much effort to
>> implement and undocumented ABI with no guarantee of support or
>> stability (outside of MSVC major version releases). Furthermore, I
>> feel that, by reinforcing the notion that use of the MSVC++ ABI is
>> required for development on Windows (when no system-provided libraries
>> or ABIs require it), Clang developers are hindering the adoption of
>> Clang on Windows rather than promoting it, since it is unlikely that
>> any party that is not personally invested in Clang development would
>> be willing to depend on a backward engineered implementation of a
>> "required" but undocumented ABI for production use.
Furthermore, it
>> means that Clang will always be behind the curve on Windows, since,
>> even if MSVC++ ABI support is fully usable one day, no one will be
>> willing to link it with object files from a new major version of MSVC
>> without some critical mass of users being guinea pigs first and
>> ensuring that there are no major bugaboos.
>>
>> I agree that there is value in supporting implementing full MSVC++
>> ABI, if it can be done, but it seems like that support can never be
>> 100% complete (or, more to the point, known to be 100% complete)
>> unless Microsoft itself decides to officially support the
>> implementation or completely stabilize and document their C++ ABIs.
>> However, I personally think that it is not only easier, there is more
>> value in implementing a C++ ABI which is compatible with officially
>> supported C and COM MSVC++ ABI subsets and consciously _not_
>> compatible in others way (in particular, which does not even attempt
>> to link C++ symbols with MSVC++, since that is not required for COM).
>> This is something that can be made to work and guaranteed (barring
>> some seriously misguided behavior from Microsoft) to continue to work
>> stably.
>>
>> Furthermore, by providing and controlling its own Windows C++ ABI:
>> Clang can possibly offer something that MSVC *cannot* (and does not
>> even try to do): a development platform and ecosystem which provides a
>> stable-over-time C++ ABI and consistent cross-module usage of RTTI,
>> STL, C++ memory management, etc. (which is dicey even within MSVC
>> major versions unless the exact same build settings are used, last
>> time I checked.) This would give someone an active reason to switch to
>> Clang more than anything else Clang currently offers on Windows.
>>
>> Stephen
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

Chris Lattner <clattner at apple.com> writes:
> This thread is odd to me. It seems that the gist of your guys'
> argument is that you don't know if we will ever get full support,
> therefore we don't welcome progress towards that (very useful)
> goal/feature.
>
> If the specific proposal doesn't make make sense from a design
> standpoint, that's one thing, but saying we shouldn't take it
because
> of licensing issues with MFC or because it is harmful to be partially
> (but not fully) compatible with MSVC seems just weird to me.
Full disclosure: I'm the guy who filled the PR requesting the feature
the OP's is working on and, when someone tried to close it as WONTFIX, I
strongly opposed. Furthermore, MSVC++ is my Windows compiler and my
projects suffer from unwelcome complexity for working around the lack of
this feature on LLVM.

So I hope it is obvious that I'm not against Reid's proposal, quite the
contrary.

My point is that nowadays Clang++ is not a production-ready Windows
compiler. The missing features are much easier to implement than MS C++
ABI compatibility. For starters, they are well documented. So it
perplexes me to see the flux of work on MS C++ compatibility when:

 * It is an open ended, difficult, uncertain goal. You'll never know for
   sure that it is done.

 * Obviously, if you don't implement the features that every Windows C++
   compiler must have, you'll never be compatible with MS. Currently
   Clang++ cannot create or use C++ dlls (a series of patches were
   submitted but quickly rejected, I hope that Nico was not discouraged
   by this.) OTOH, implementing 32bit SEH is a must-have requirement for
   MS compatibility, but it would benefit Clang on Windows independently
   of the C++ ABI used.

 * For being the best Windows C++ compiler, Clang++ doesn't need to be
   compatible with the MS C++ ABI. However (and this is highly
   detrimental to Clang, IMO) from reading the mailing lists one could
   easily get the impression that until Clang++ gets the MS C++ ABI, it
   is not ready for Windows. It wont surprise me to learn that some
   contributors are working on the MS C++ ABI because they think that
   way.

 * Using its current C++ ABI and debug info format on Windows has the
   advantage of making Clang++ usable together with tools like existing
   debuggers and profilers. For using Clang with Visual Studio's IDE it
   must learn how to emit debug info compatible with MS, and that is
   another can of worms.

 * As the MS C++ ABI is in flux, while at the same time Clang++ is well
   ahead of MS on conformance, some weird cases might arise where
   Clang++ would be forced to extend the ABI on its own at the risk of
   being incompatible with itself after a while (MS variadic templates
   implementation required an ABI change. MS engineers admit that
   planned features might require more changes. Of course, Clang++
   already supports most of those features.)

 * Finally, legal issues remain: supposing that Clang++ gets MS C++ ABI
   compatibility, is it possible to use MS C++ runtime(s) and libraries
   with Clang++? We could end with lots of work invested on a feature
   that only benefits those who are forced to work with third-party C++
   libraries distributed on binary form. I guess that such users are a
   minority.

To recap: lots of work is being invested on a feature which is very hard
to implement, while at the same time Clang lacks basic features. I'll
dare to say that focusing the effort on those features would mean that,
in a year at most, Clang would become the best C++ compiler for Windows.

Needles to say, everyone has the right to decide on what they work. I
fully respect that. But I'll like to know what motivates those who are
working so hard on the MS C++ ABI while Clang++ remains as a compiler
not ready for production on Windows.