llvm dev - Sep 2016 - [cfe-dev] Revisiting our informal policy to support two versions of MSVC

Apart from the obvious licencing issues, each time I have moved from one
version of VC++ to another, the big problem I have had is not specifically
the ABI at the register passing, stack organisation level, but rather the
implementation details of the Standard C++ libraries, and in particular the
STL containers.

While the compiler team puts considerable effort into maintaining the ABI,
the C++ library implementation usually changes a lot.

Since this is largely in the form of very complex headers defining
templates which in turn cause other helper templates to be used, it is here
that I find things go awry.

So for C++, a function like:

std::list<int> foo();

seems simple enough, but if the caller and the callee are compiled with
different versions, it usually won't work because of some artefact of the
STL implementation tuning that occurs between versions.  In particular,
this impacts things like using C++ interfaces across DLLs and in
pre-compiled libraries.

I think that the ABI maintenance in this case tends to be for C and POD
compatability, but not for the higher level C++ compatability which is
unfortunate and restricts how we can use C++.

Is it possible that it is this aspect of the version change that is causing
your ABI difficulties?

MartinO


On 7 September 2016 at 20:18, Zachary Turner via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Can you elaborate on the abi incompatibility? I thought there were no
> breaks
> On Wed, Sep 7, 2016 at 7:59 AM James Molloy via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
>> Hi all,
>>
>> Firstly sorry I'm a bit late responding on this one. Internally to
ARM we
>> build LLVM for Windows. Our current build cluster has only VS2013
installed
>> and as a result of this thread we've been working on getting VS2015
>> installed. This involves a certain amount of IT-wrangling as the
cluster we
>> use is company-wide. There have been some hiccups regarding licensing
of
>> MSVC professional (we can't use the community edition for the same
reasons
>> mentioned by Paul previously) but we hoped to be ready in time for the
15th
>> September switchover date.
>>
>> It's recently been realised that VS2013 and VS2015 are not ABI
compatible
>> (something that really surprised me), and this means we have to
synchronize
>> moving LLVM's build to VS2015 as well as upgrading a third party
library
>> that we receive from the vendor in compiled library form. This is not
>> something we're capable of doing by September 15th.
>>
>> We try really hard at ARM to hide our internal processes because we
>> believe that they're on the whole irrelevant to the community,
however in
>> this case we'd be really stuck, unable to get production builds.
>>
>> As this is an ABI-incompatible upgrade, and it's changing the
informal
>> policy on upgrades, could we please have some more grace time? Ideally
>> another month, so the 15th October. If we haven't sorted it by
then, it's
>> our problem.
>>
>> Cheers,
>>
>> James
>>
>> On Thu, 1 Sep 2016 at 21:06 Mehdi Amini via cfe-dev <
>> cfe-dev at lists.llvm.org> wrote:
>>
>>> On Sep 1, 2016, at 1:05 PM, Reid Kleckner <rnk at google.com>
wrote:
>>>
>>> On Thu, Sep 1, 2016 at 12:53 PM, Mehdi Amini via cfe-dev <
>>> cfe-dev at lists.llvm.org> wrote:
>>>>
>>>> Isn’t a big (the most) reason for supporting “old” toolchains
to allow
>>>> downstream users to upgrade with some flexibility?
>>>> If I have a large codebase that is using LLVM (let say a few
custom
>>>> backends), and is validated with “MSVC 2013”, I can upgrade to
“2015” but I
>>>> will need some qualification/validation: this is not free and
take some
>>>> time. If you drop aggressively supports for “old” toolchain it
means that
>>>> I’m either stuck with an “old” LLVM or that I have to update
earlier than
>>>> expected.
>>>>
>>>> Isn’t this usually balanced in upstream LLVM to upgrade when
there is a
>>>> real *benefit* to it?
>>>> I’m mentioning it because it seems to conflict with the
"always upgrade
>>>> to the newest one unless there are serious issues with it” you
mentioned
>>>> above.
>>>>
>>>
>>> I agree, we should raise the minimum VS version requirement when
the
>>> benefits to the LLVM community outweigh the costs of switching for
major
>>> LLVM contributors and users. I think we'll always make that
decision in the
>>> same way: by raising it on the mailing lists and discussing the
pros and
>>> cons. That's basically what David said when he kicked this
whole discussion
>>> off, anyway:
>>>
>>> """But if we find ourselves in a situation where
asking folks to
>>> upgrade to a compiler which has been widely deployed soothes
development
>>> for the greater LLVM community, we should consider dropping support
for the
>>> older versions of that compiler."""
>>>
>>> I think everything is working as intended here.
>>>
>>>
>>> Right, to be clear there is no misunderstanding: I was absolutely
not
>>> suggesting the opposite when answering Zach..
>>>
>>> —
>>> Mehdi
>>>
>>>
>>> We raised the VS 2013 upgrade issue, discussed it, determined that
it
>>> was holding us back, and now we're doing the upgrade. If VS
"15" brings
>>> major language compatibility improvements, I imagine we'll be
having this
>>> same discussion again next year. If it doesn't, and supporting
2015 and
>>> "15" at the same time has the same cost, then we
won't bother raising the
>>> floor for a while.
>>>
>>>
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160907/d6c71f0c/attachment.html>

Note that this is intentional from the MSVC C++ library implementation 
side of things.  For major versions, no attempt is made to preserve 
library ABI compatibility.

I am aware of a language ABI break in VC++ 2013. 
https://randomascii.wordpress.com/2013/12/01/vc-2013-class-layout-change-and-wasted-space/

I'm not currently aware of any on the VC++ 2015 side of things, but that 
doesn't mean much.

On 9/7/2016 2:34 PM, Martin O'Riordan via llvm-dev
wrote:
> Apart from the obvious licencing issues, each time I have moved from 
> one version of VC++ to another, the big problem I have had is not 
> specifically the ABI at the register passing, stack organisation 
> level, but rather the implementation details of the Standard C++ 
> libraries, and in particular the STL containers.
>
> While the compiler team puts considerable effort into maintaining the 
> ABI, the C++ library implementation usually changes a lot.
>
> Since this is largely in the form of very complex headers defining 
> templates which in turn cause other helper templates to be used, it is 
> here that I find things go awry.
>
> So for C++, a function like:
>
> std::list<int> foo();
>
> seems simple enough, but if the caller and the callee are compiled 
> with different versions, it usually won't work because of some 
> artefact of the STL implementation tuning that occurs between 
> versions.  In particular, this impacts things like using C++ 
> interfaces across DLLs and in pre-compiled libraries.
>
> I think that the ABI maintenance in this case tends to be for C and 
> POD compatability, but not for the higher level C++ compatability 
> which is unfortunate and restricts how we can use C++.
>
> Is it possible that it is this aspect of the version change that is 
> causing your ABI difficulties?
>
> MartinO
>
>
> On 7 September 2016 at 20:18, Zachary Turner via llvm-dev 
> <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>
>     Can you elaborate on the abi incompatibility? I thought there were
>     no breaks
>     On Wed, Sep 7, 2016 at 7:59 AM James Molloy via cfe-dev
>     <cfe-dev at lists.llvm.org <mailto:cfe-dev at
lists.llvm.org>> wrote:
>
>         Hi all,
>
>         Firstly sorry I'm a bit late responding on this one.
>         Internally to ARM we build LLVM for Windows. Our current build
>         cluster has only VS2013 installed and as a result of this
>         thread we've been working on getting VS2015 installed. This
>         involves a certain amount of IT-wrangling as the cluster we
>         use is company-wide. There have been some hiccups regarding
>         licensing of MSVC professional (we can't use the community
>         edition for the same reasons mentioned by Paul previously) but
>         we hoped to be ready in time for the 15th September switchover
>         date.
>
>         It's recently been realised that VS2013 and VS2015 are not ABI
>         compatible (something that really surprised me), and this
>         means we have to synchronize moving LLVM's build to VS2015 as
>         well as upgrading a third party library that we receive from
>         the vendor in compiled library form. This is not something
>         we're capable of doing by September 15th.
>
>         We try really hard at ARM to hide our internal processes
>         because we believe that they're on the whole irrelevant to the
>         community, however in this case we'd be really stuck, unable
>         to get production builds.
>
>         As this is an ABI-incompatible upgrade, and it's changing the
>         informal policy on upgrades, could we please have some more
>         grace time? Ideally another month, so the 15th October. If we
>         haven't sorted it by then, it's our problem.
>
>         Cheers,
>
>         James
>
>         On Thu, 1 Sep 2016 at 21:06 Mehdi Amini via cfe-dev
>         <cfe-dev at lists.llvm.org <mailto:cfe-dev at
lists.llvm.org>> wrote:
>
>>             On Sep 1, 2016, at 1:05 PM, Reid Kleckner <rnk at
google.com
>>             <mailto:rnk at google.com>> wrote:
>>
>>             On Thu, Sep 1, 2016 at 12:53 PM, Mehdi Amini via cfe-dev
>>             <cfe-dev at lists.llvm.org <mailto:cfe-dev at
lists.llvm.org>>
>>             wrote:
>>
>>                 Isn’t a big (the most) reason for supporting “old”
>>                 toolchains to allow downstream users to upgrade with
>>                 some flexibility?
>>                 If I have a large codebase that is using LLVM (let
>>                 say a few custom backends), and is validated with
>>                 “MSVC 2013”, I can upgrade to “2015” but I will need
>>                 some qualification/validation: this is not free and
>>                 take some time. If you drop aggressively supports for
>>                 “old” toolchain it means that I’m either stuck with
>>                 an “old” LLVM or that I have to update earlier than
>>                 expected.
>>
>>                 Isn’t this usually balanced in upstream LLVM to
>>                 upgrade when there is a real *benefit* to it?
>>                 I’m mentioning it because it seems to conflict with
>>                 the "always upgrade to the newest one unless there
>>                 are serious issues with it” you mentioned above.
>>
>>
>>             I agree, we should raise the minimum VS version
>>             requirement when the benefits to the LLVM community
>>             outweigh the costs of switching for major LLVM
>>             contributors and users. I think we'll always make that
>>             decision in the same way: by raising it on the mailing
>>             lists and discussing the pros and cons. That's
basically
>>             what David said when he kicked this whole discussion off,
>>             anyway:
>>
>>             """But if we find ourselves in a situation
where asking
>>             folks to upgrade to a compiler which has been widely
>>             deployed soothes development for the greater LLVM
>>             community, we should consider dropping support for the
>>             older versions of that compiler."""
>>
>>             I think everything is working as intended here.
>
>             Right, to be clear there is no misunderstanding: I was
>             absolutely not suggesting the opposite when answering Zach..
>
>             —
>             Mehdi
>
>
>>             We raised the VS 2013 upgrade issue, discussed it,
>>             determined that it was holding us back, and now we're
>>             doing the upgrade. If VS "15" brings major
language
>>             compatibility improvements, I imagine we'll be having
>>             this same discussion again next year. If it doesn't,
and
>>             supporting 2015 and "15" at the same time has the
same
>>             cost, then we won't bother raising the floor for a
while.
>
>             _______________________________________________
>             cfe-dev mailing list
>             cfe-dev at lists.llvm.org <mailto:cfe-dev at
lists.llvm.org>
>             http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>             <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
>
>         _______________________________________________
>         cfe-dev mailing list
>         cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
>         http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>         <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
>
>
>     _______________________________________________
>     LLVM Developers mailing list
>     llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>     <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
>
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-- 
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux
Foundation Collaborative Project

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160907/66fb5f44/attachment.html>

Hi,

As I understand it the specific issue we're seeing is related to what
Martin described. But due to numerous bugs found when mixing objects
compiled with different versions of MSVC in the past, we now are shy of
doing it even if it seems to work superficially - that's no guarantee bugs
won't be found down the line. We'd much prefer to stay within the realms
of
what Microsoft support.

Cheers,

James
On Wed, 7 Sep 2016 at 20:53, Craig, Ben via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Note that this is intentional from the MSVC C++ library implementation
> side of things.  For major versions, no attempt is made to preserve library
> ABI compatibility.
> I am aware of a language ABI break in VC++ 2013.
>
https://randomascii.wordpress.com/2013/12/01/vc-2013-class-layout-change-and-wasted-space/
>
> I'm not currently aware of any on the VC++ 2015 side of things, but
that
> doesn't mean much.
>
>
> On 9/7/2016 2:34 PM, Martin O'Riordan via llvm-dev wrote:
>
> Apart from the obvious licencing issues, each time I have moved from one
> version of VC++ to another, the big problem I have had is not specifically
> the ABI at the register passing, stack organisation level, but rather the
> implementation details of the Standard C++ libraries, and in particular the
> STL containers.
>
> While the compiler team puts considerable effort into maintaining the ABI,
> the C++ library implementation usually changes a lot.
>
> Since this is largely in the form of very complex headers defining
> templates which in turn cause other helper templates to be used, it is here
> that I find things go awry.
>
> So for C++, a function like:
>
> std::list<int> foo();
>
> seems simple enough, but if the caller and the callee are compiled with
> different versions, it usually won't work because of some artefact of
the
> STL implementation tuning that occurs between versions.  In particular,
> this impacts things like using C++ interfaces across DLLs and in
> pre-compiled libraries.
>
> I think that the ABI maintenance in this case tends to be for C and POD
> compatability, but not for the higher level C++ compatability which is
> unfortunate and restricts how we can use C++.
>
> Is it possible that it is this aspect of the version change that is
> causing your ABI difficulties?
>
> MartinO
>
>
> On 7 September 2016 at 20:18, Zachary Turner via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Can you elaborate on the abi incompatibility? I thought there were no
>> breaks
>> On Wed, Sep 7, 2016 at 7:59 AM James Molloy via cfe-dev <
>> cfe-dev at lists.llvm.org> wrote:
>>
>>> Hi all,
>>>
>>> Firstly sorry I'm a bit late responding on this one. Internally
to ARM
>>> we build LLVM for Windows. Our current build cluster has only
VS2013
>>> installed and as a result of this thread we've been working on
getting
>>> VS2015 installed. This involves a certain amount of IT-wrangling as
the
>>> cluster we use is company-wide. There have been some hiccups
regarding
>>> licensing of MSVC professional (we can't use the community
edition for the
>>> same reasons mentioned by Paul previously) but we hoped to be ready
in time
>>> for the 15th September switchover date.
>>>
>>> It's recently been realised that VS2013 and VS2015 are not ABI
>>> compatible (something that really surprised me), and this means we
have to
>>> synchronize moving LLVM's build to VS2015 as well as upgrading
a third
>>> party library that we receive from the vendor in compiled library
form.
>>> This is not something we're capable of doing by September 15th.
>>>
>>> We try really hard at ARM to hide our internal processes because we
>>> believe that they're on the whole irrelevant to the community,
however in
>>> this case we'd be really stuck, unable to get production
builds.
>>>
>>> As this is an ABI-incompatible upgrade, and it's changing the
informal
>>> policy on upgrades, could we please have some more grace time?
Ideally
>>> another month, so the 15th October. If we haven't sorted it by
then, it's
>>> our problem.
>>>
>>> Cheers,
>>>
>>> James
>>>
>>> On Thu, 1 Sep 2016 at 21:06 Mehdi Amini via cfe-dev <
>>> cfe-dev at lists.llvm.org> wrote:
>>>
>>>> On Sep 1, 2016, at 1:05 PM, Reid Kleckner <rnk at
google.com> wrote:
>>>>
>>>> On Thu, Sep 1, 2016 at 12:53 PM, Mehdi Amini via cfe-dev <
>>>> cfe-dev at lists.llvm.org> wrote:
>>>>>
>>>>> Isn’t a big (the most) reason for supporting “old”
toolchains to allow
>>>>> downstream users to upgrade with some flexibility?
>>>>> If I have a large codebase that is using LLVM (let say a
few custom
>>>>> backends), and is validated with “MSVC 2013”, I can upgrade
to “2015” but I
>>>>> will need some qualification/validation: this is not free
and take some
>>>>> time. If you drop aggressively supports for “old” toolchain
it means that
>>>>> I’m either stuck with an “old” LLVM or that I have to
update earlier than
>>>>> expected.
>>>>>
>>>>> Isn’t this usually balanced in upstream LLVM to upgrade
when there is
>>>>> a real *benefit* to it?
>>>>> I’m mentioning it because it seems to conflict with the
"always
>>>>> upgrade to the newest one unless there are serious issues
with it” you
>>>>> mentioned above.
>>>>>
>>>>
>>>> I agree, we should raise the minimum VS version requirement
when the
>>>> benefits to the LLVM community outweigh the costs of switching
for major
>>>> LLVM contributors and users. I think we'll always make that
decision in the
>>>> same way: by raising it on the mailing lists and discussing the
pros and
>>>> cons. That's basically what David said when he kicked this
whole discussion
>>>> off, anyway:
>>>>
>>>> """But if we find ourselves in a situation where
asking folks to
>>>> upgrade to a compiler which has been widely deployed soothes
development
>>>> for the greater LLVM community, we should consider dropping
support for the
>>>> older versions of that compiler."""
>>>>
>>>> I think everything is working as intended here.
>>>>
>>>>
>>>> Right, to be clear there is no misunderstanding: I was
absolutely not
>>>> suggesting the opposite when answering Zach..
>>>>
>>>> —
>>>> Mehdi
>>>>
>>>>
>>>> We raised the VS 2013 upgrade issue, discussed it, determined
that it
>>>> was holding us back, and now we're doing the upgrade. If VS
"15" brings
>>>> major language compatibility improvements, I imagine we'll
be having this
>>>> same discussion again next year. If it doesn't, and
supporting 2015 and
>>>> "15" at the same time has the same cost, then we
won't bother raising the
>>>> floor for a while.
>>>>
>>>>
>>>> _______________________________________________
>>>> cfe-dev mailing list
>>>> cfe-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>
>
> _______________________________________________
> LLVM Developers mailing listllvm-dev at
lists.llvm.orghttp://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> --
> Employee of Qualcomm Innovation Center, Inc.
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux
Foundation Collaborative Project
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160907/2cae1df8/attachment.html>