llvm dev - Aug 2015 - [LLVMdev] [RFC] Developer Policy for LLVM C API

As someone who used the LLVM C API for an experiment back in 2009ish
(porting the SBCL lisp compiler to target LLVM as a backend -- never
finished), I thought it was great that LLVM provided the C API. I was sad
that it wasn't properly updated to include support for all the newly
introduced IR features, though. (E.g. atomics). I tried to send patches for
some of that stuff a while later, but didn't manage to get it in. (I
wasn't
active in llvm at that point, I was just trying to contribute a patch. I
gave up.).

It doesn't make sense to me to have the C API be less than a full wrapping
of the IR building functions. Having less than that, you almost might as
well not have it at all. (Which would be sad.)

I'd like suggest that the most important thing for the C API is NOT a 100%
promise of eternal compatibility, but to _strive_ for compatibility, as
much as is realistically possible. If it's not possible, oh well. We're
not
talking about libc here, so the costs of changing it incompatibly -- with
careful deliberation -- are not impossibly huge. I mean, even libstdc++
changes its ABI occassionally! LLVM's C API certainly shouldn't be
considered more important to keep stable than libstdc++!

That said, incompatible changes ought not be made just for trivial reasons:
they should be made when there's a true need, e.g. when an existing LLVM-C
function cannot sensibly maintain its existing behavior anymore because the
underlying functionality of LLVM was removed, or drastically changed.

I'd propose that the only 100% strict rule should be that if the ABI/API
changes, it is done in a way that *loudly* breaks old programs -- e.g. they
fail to compile, link, or run (depending on how the other-lang wrappers are
accessing the API functions) -- not that you get some random weird
misbehavior because a function's argument types or return type has been
changed.

Ideally, if you're going to remove an API, you'd deprecate it first, if
you
can see the problem coming. Otherwise, oh well -- it's not reasonable to
hold up LLVM development to adhere to deprecation policy on the C API.

That is:
1) Adding new functions for new LLVM functionality: great.
2) Removing old functions: if it's required -- after careful deliberation.
3) Modifying the signature of existing functions: no way.

A couple of concrete examples:

- Let's say you run out of space in an existing enum type (ahem,
LLVMAttribute...).

You should introduce new function names, taking a better type (perhaps some
sort of set type, instead of a fixed-width integer...), mark the old ones
deprecated so that users will update to the new functions. But, keep the
old ones around so that old code can keep working. There's no real need to
remove the old deprecated functions -- they'll keep working as well as they
ever did with the smaller-valued attributes.

- Let's say you remove types from pointers.

This is a change that can be seen coming up. So, one should fairly early on
add an LLVMBuildGEP2 function that takes the extra type argument. Leave the
old function name around for now (deprecated), because with the current
state of LLVM, it can continue to work just fine.

When LLVM is changed to *actually* remove all pointer types in its core,
then the LLVMBuildGEP function cannot reasonably continue to exist. It
simply cannot work, if pointers don't have types, and the type isn't
provided. So, remove it.

Preferably, there would be a release of LLVM in between with both functions
available to let people have some time to adjust, but if not, oh well.


On Mon, Aug 17, 2015 at 1:28 PM, deadal nix via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
>
> 2015-08-17 8:34 GMT-07:00 Reid Kleckner <rnk at google.com>:
>
>> On Sun, Aug 16, 2015 at 10:34 PM, deadal nix via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>>
>>> 2015-08-16 21:51 GMT-07:00 Eric Christopher <echristo at
gmail.com>:
>>>>
>>>> The promise of stability. We don't promise that the C++ API
will stay
>>>> stable.
>>>>
>>>
>>>
>>> Why was that promise be made in the first place ? Has it been made
in
>>> the first place ?
>>>
>>
>> It sounds like you're in favor of dropping C API stability then, if
it's
>> holding us back? That feedback is actually really helpful. :)
>>
>> There are really three goals here: flexibility to change LLVM IR,
>> completeness of the C API, and stability of the C API. Pick two.
>>
>>
> Yes but these aren't black and white IMO. These are need in tension,
sure,
> but all API have these kind of problem not specifically the C API.
>
>
>
>> The goals are mutually incompatible and we have to trade off one for
the
>> other. Most of the LLVM core developers value goal #1, the ability to
>> change the IR. Look at the pointee type changes that David Blaikie is
>> working on, and the new EH representation. If we promise both stability
and
>> completeness, these things are impossible.
>>
>>
> Yes I was actually thinking about that. This is the kind of change that
> justify a change in the C API IMO. We should try our best to keep the C API
> stable, but if somethign fundamental changes in LLVM, and that there is no
> nice way to map it to the old API, then
>
>
>
>> One way forward is to guarantee stability, but limit completeness. This
>> would mean limiting the C API to constructs that will always and
forever be
>> easily represented in LLVM.
>>
>>
> That would be a problem for me. It seems like other in this thread
> mentioned the incompleteness of the C API as a problem for them and
> mentioned the use of various hacks to work around it. On my side I tried to
> propose diff to increase completeness of the C API in the past, but the
> review process was frankly too painful and I mostly give up. I'd be up
for
> being a maintainer of the C API if that is needed.
>
>
>
>> The other choice is to forget stability and wrap the C++ API
completely,
>> potentially with something auto-generated. We could make a more limited
>> stability promise along the lines of "these APIs will be updated
to reflect
>> changes to the IR, and are otherwise stable." I think everyone
would be
>> fine with that.
>>
>
> I do not think this is a good idea. The C API is used a lot to use LLVM
> from other languages than C++ . When used directly from C, you get function
> signature, but when used from other languages, changing method randomly
> cause undefined behavior. For instance, the landing pad instruction was
> changed recently and the C API to create a landing pad was updated. As a
> result, 2 thing happened :
>  - It was not possible to set the personality function from C, which is a
> problem for many frontends.
>  - In other languages, as mangling remained the same, the thing compiles
> and fail randomly at runtime.
>
> This need to be avoided as much as possible. I'd propose the following
> policy :
> 1/ Do change the IR as per need of LLVM development. The C API must not
> prevent LLVM to move forward.
> 2/ If the change can be mapped nicely without changing the C API, then do
> that.
> 3/ If there is no way to map it nicely, then update the C API.
>
> For instance, the pointer update work would probably be a valid reason to
> go with 3/ .
>
> Right now, as far as I'm concerned, the main limitation of the C API
are
> the capability to set attributes and the capability to use atomic
> loads/stores.
>
> In any ways, I do think the patch adding tests for existing API should be
> merged. The need for test for the C API was mentioned various times in this
> thread and I do think that whatever the road taken, having test is a good
> thing.
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150817/bb688af5/attachment.html>

On Mon, Aug 17, 2015 at 12:31 PM, James Y Knight via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> I'd propose that the only 100% strict rule should be that if the
ABI/API
> changes, it is done in a way that *loudly* breaks old programs -- e.g. they
> fail to compile, link, or run (depending on how the other-lang wrappers are
> accessing the API functions) -- not that you get some random weird
> misbehavior because a function's argument types or return type has been
> changed.

This would require a level of testing that we don't have, just to make
sure that happens, no?
>
> Ideally, if you're going to remove an API, you'd deprecate it
first, if you
> can see the problem coming. Otherwise, oh well -- it's not reasonable
to
> hold up LLVM development to adhere to deprecation policy on the C API.
>
> That is:
> 1) Adding new functions for new LLVM functionality: great.
> 2) Removing old functions: if it's required -- after careful
deliberation.
> 3) Modifying the signature of existing functions: no way.
>
> A couple of concrete examples:
>
> - Let's say you run out of space in an existing enum type (ahem,
> LLVMAttribute...).
>
> You should introduce new function names, taking a better type (perhaps some
> sort of set type, instead of a fixed-width integer...), mark the old ones
> deprecated so that users will update to the new functions. But, keep the
old
> ones around so that old code can keep working. There's no real need to
> remove the old deprecated functions -- they'll keep working as well as
they
> ever did with the smaller-valued attributes.
Maybe?
Maybe not?
Maybe they were broken when built.

2015-08-17 12:46 GMT-07:00 Daniel Berlin <dberlin at dberlin.org>:
> On Mon, Aug 17, 2015 at 12:31 PM, James Y Knight via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> >
> > I'd propose that the only 100% strict rule should be that if the
ABI/API
> > changes, it is done in a way that *loudly* breaks old programs -- e.g.
> they
> > fail to compile, link, or run (depending on how the other-lang
wrappers
> are
> > accessing the API functions) -- not that you get some random weird
> > misbehavior because a function's argument types or return type has
been
> > changed.
>
>
> This would require a level of testing that we don't have, just to make
> sure that happens, no?
>
>
http://reviews.llvm.org/D10725
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150817/dd7d4239/attachment.html>

On Mon, Aug 17, 2015 at 3:46 PM, Daniel Berlin <dberlin at dberlin.org>
wrote:
> On Mon, Aug 17, 2015 at 12:31 PM, James Y Knight via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> >
> > I'd propose that the only 100% strict rule should be that if the
ABI/API
> > changes, it is done in a way that *loudly* breaks old programs -- e.g.
> they
> > fail to compile, link, or run (depending on how the other-lang
wrappers
> are
> > accessing the API functions) -- not that you get some random weird
> > misbehavior because a function's argument types or return type has
been
> > changed.
>
>
> This would require a level of testing that we don't have, just to make
> sure that happens, no?
>
Okay, kinda? I mean, more testing is always good. If you're missing a test,
and then break something by accident, well, that sucks. But, the important
first step is to agree that the *policy* is to not change the C API in that
way. If something like that does happen, treat it as a bug to be
(hopefully) fixed, rather than something that's allowed by the development
policy so "oh well".

With the way the C API is currently defined, maintaining that property
actually seems like it's pretty straightforward, because all the types it
uses are defined in the C API headers. So, basically: be careful about how
you change the C API headers. Which people already are being.

Maybe I shouldn't have said "100% strict" -- I didn't actually
mean "the
world will explode if we get this wrong and introduce a bug" -- it
certainly won't. It would just be inconvenient, so have a policy of not
doing it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150817/692e58a9/attachment.html>

On Mon, Aug 17, 2015 at 12:31 PM, James Y Knight via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> As someone who used the LLVM C API for an experiment back in 2009ish
> (porting the SBCL lisp compiler to target LLVM as a backend -- never
> finished), I thought it was great that LLVM provided the C API. I was sad
> that it wasn't properly updated to include support for all the newly
> introduced IR features, though. (E.g. atomics). I tried to send patches for
> some of that stuff a while later, but didn't manage to get it in. (I
wasn't
> active in llvm at that point, I was just trying to contribute a patch. I
> gave up.).
>
> It doesn't make sense to me to have the C API be less than a full
wrapping
> of the IR building functions. Having less than that, you almost might as
> well not have it at all. (Which would be sad.)
>
> I'd like suggest that the most important thing for the C API is NOT a
100%
> promise of eternal compatibility, but to _strive_ for compatibility, as
> much as is realistically possible. If it's not possible, oh well.
We're not
> talking about libc here, so the costs of changing it incompatibly -- with
> careful deliberation -- are not impossibly huge. I mean, even libstdc++
> changes its ABI occassionally! LLVM's C API certainly shouldn't be
> considered more important to keep stable than libstdc++!
>
> That said, incompatible changes ought not be made just for trivial
> reasons: they should be made when there's a true need, e.g. when an
> existing LLVM-C function cannot sensibly maintain its existing behavior
> anymore because the underlying functionality of LLVM was removed, or
> drastically changed.
>
FWIW we do this (drastically change LLVM) pretty often. See Debug Info
metadata, DataLayout, my typeless pointer work, Chandler's pass manager
work, etc.

>
> I'd propose that the only 100% strict rule should be that if the
ABI/API
> changes, it is done in a way that *loudly* breaks old programs -- e.g. they
> fail to compile, link, or run (depending on how the other-lang wrappers are
> accessing the API functions) -- not that you get some random weird
> misbehavior because a function's argument types or return type has been
> changed.
>
> Ideally, if you're going to remove an API, you'd deprecate it
first, if
> you can see the problem coming. Otherwise, oh well -- it's not
reasonable
> to hold up LLVM development to adhere to deprecation policy on the C API.
>
> That is:
> 1) Adding new functions for new LLVM functionality: great.
> 2) Removing old functions: if it's required -- after careful
deliberation.
> 3) Modifying the signature of existing functions: no way.
>
> A couple of concrete examples:
>
> - Let's say you run out of space in an existing enum type (ahem,
> LLVMAttribute...).
>
> You should introduce new function names, taking a better type (perhaps
> some sort of set type, instead of a fixed-width integer...), mark the old
> ones deprecated so that users will update to the new functions. But, keep
> the old ones around so that old code can keep working. There's no real
need
> to remove the old deprecated functions -- they'll keep working as well
as
> they ever did with the smaller-valued attributes.
>
But does it? What if the code is checking if two functions have the same
attributes (perhaps this is an invalid notion in the absence of information
about specific attributes) - if you compare the smaller-ranged attributes,
you might incorrectly conclude that these two functions have the same
attributes. Similarly for "no attributes specified" - you look at the
enum,
see that none are set & conclude that it's a plain, uninteresting
function
- so you create another plain, no-attributes function you think you can
make equivalent.

>
> - Let's say you remove types from pointers.
>
> This is a change that can be seen coming up. So, one should fairly early
> on add an LLVMBuildGEP2 function that takes the extra type argument. Leave
> the old function name around for now (deprecated), because with the current
> state of LLVM, it can continue to work just fine.
>
> When LLVM is changed to *actually* remove all pointer types in its core,
> then the LLVMBuildGEP function cannot reasonably continue to exist. It
> simply cannot work, if pointers don't have types, and the type
isn't
> provided. So, remove it.
>
Yep, this is one we /might/ be able to manage - because we'll probably have
to keep typed pointer information around for old bitcode deserialization.
(but it does make the API awkward - when we finally do remove LLVMBuildGEP,
do we rename LLVMBuildGEP2? Do we leave it there? & then we need
LLVMBuildGEP3, etc, as things evolve?))

Also this works for writing, but not for reading - if code was expecting to
read IR and assumed that pointers had types and bitcasts were present, etc
- it's going to have a Bad Time reading new IR... the APIs wouldn't make
any sense (there would be no getPointeeType on PointerType anymore - there
would be no Type that could be returned in response to that query)

That's part of Eric's point, I think, once the API is low level enough,
it
will churn with every change in LLVM or possible require some heroics to
avoid that churn which will slow down LLVM's API velocity (which is
somewhere around "ludicrous speed"), which is something we rather
like. The
only way to make the C API more stable is to make it higher level (see
libLTO for an example) so that we can own the migration & shelter users
from it through that abstraction. If it's low level, it's going to churn
or
slow us down (and probably both).

>
> Preferably, there would be a release of LLVM in between with both
> functions available to let people have some time to adjust, but if not, oh
> well.
>
>
> On Mon, Aug 17, 2015 at 1:28 PM, deadal nix via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>>
>>
>> 2015-08-17 8:34 GMT-07:00 Reid Kleckner <rnk at google.com>:
>>
>>> On Sun, Aug 16, 2015 at 10:34 PM, deadal nix via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>>
>>>> 2015-08-16 21:51 GMT-07:00 Eric Christopher <echristo at
gmail.com>:
>>>>>
>>>>> The promise of stability. We don't promise that the C++
API will stay
>>>>> stable.
>>>>>
>>>>
>>>>
>>>> Why was that promise be made in the first place ? Has it been
made in
>>>> the first place ?
>>>>
>>>
>>> It sounds like you're in favor of dropping C API stability
then, if it's
>>> holding us back? That feedback is actually really helpful. :)
>>>
>>> There are really three goals here: flexibility to change LLVM IR,
>>> completeness of the C API, and stability of the C API. Pick two.
>>>
>>>
>> Yes but these aren't black and white IMO. These are need in
tension,
>> sure, but all API have these kind of problem not specifically the C
API.
>>
>>
>>
>>> The goals are mutually incompatible and we have to trade off one
for the
>>> other. Most of the LLVM core developers value goal #1, the ability
to
>>> change the IR. Look at the pointee type changes that David Blaikie
is
>>> working on, and the new EH representation. If we promise both
stability and
>>> completeness, these things are impossible.
>>>
>>>
>> Yes I was actually thinking about that. This is the kind of change that
>> justify a change in the C API IMO. We should try our best to keep the C
API
>> stable, but if somethign fundamental changes in LLVM, and that there is
no
>> nice way to map it to the old API, then
>>
>>
>>
>>> One way forward is to guarantee stability, but limit completeness.
This
>>> would mean limiting the C API to constructs that will always and
forever be
>>> easily represented in LLVM.
>>>
>>>
>> That would be a problem for me. It seems like other in this thread
>> mentioned the incompleteness of the C API as a problem for them and
>> mentioned the use of various hacks to work around it. On my side I
tried to
>> propose diff to increase completeness of the C API in the past, but the
>> review process was frankly too painful and I mostly give up. I'd be
up for
>> being a maintainer of the C API if that is needed.
>>
>>
>>
>>> The other choice is to forget stability and wrap the C++ API
completely,
>>> potentially with something auto-generated. We could make a more
limited
>>> stability promise along the lines of "these APIs will be
updated to reflect
>>> changes to the IR, and are otherwise stable." I think everyone
would be
>>> fine with that.
>>>
>>
>> I do not think this is a good idea. The C API is used a lot to use LLVM
>> from other languages than C++ . When used directly from C, you get
function
>> signature, but when used from other languages, changing method randomly
>> cause undefined behavior. For instance, the landing pad instruction was
>> changed recently and the C API to create a landing pad was updated. As
a
>> result, 2 thing happened :
>>  - It was not possible to set the personality function from C, which is
a
>> problem for many frontends.
>>  - In other languages, as mangling remained the same, the thing
compiles
>> and fail randomly at runtime.
>>
>> This need to be avoided as much as possible. I'd propose the
following
>> policy :
>> 1/ Do change the IR as per need of LLVM development. The C API must not
>> prevent LLVM to move forward.
>> 2/ If the change can be mapped nicely without changing the C API, then
do
>> that.
>> 3/ If there is no way to map it nicely, then update the C API.
>>
>> For instance, the pointer update work would probably be a valid reason
to
>> go with 3/ .
>>
>> Right now, as far as I'm concerned, the main limitation of the C
API are
>> the capability to set attributes and the capability to use atomic
>> loads/stores.
>>
>> In any ways, I do think the patch adding tests for existing API should
be
>> merged. The need for test for the C API was mentioned various times in
this
>> thread and I do think that whatever the road taken, having test is a
good
>> thing.
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150817/f0a6a294/attachment-0001.html>

On Mon, Aug 17, 2015 at 1:47 PM David Blaikie via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Mon, Aug 17, 2015 at 12:31 PM, James Y Knight via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> As someone who used the LLVM C API for an experiment back in 2009ish
>> (porting the SBCL lisp compiler to target LLVM as a backend -- never
>> finished), I thought it was great that LLVM provided the C API. I was
sad
>> that it wasn't properly updated to include support for all the
newly
>> introduced IR features, though. (E.g. atomics). I tried to send patches
for
>> some of that stuff a while later, but didn't manage to get it in.
(I wasn't
>> active in llvm at that point, I was just trying to contribute a patch.
I
>> gave up.).
>>
>> It doesn't make sense to me to have the C API be less than a full
>> wrapping of the IR building functions. Having less than that, you
almost
>> might as well not have it at all. (Which would be sad.)
>>
>> I'd like suggest that the most important thing for the C API is NOT
a
>> 100% promise of eternal compatibility, but to _strive_ for
compatibility,
>> as much as is realistically possible. If it's not possible, oh
well. We're
>> not talking about libc here, so the costs of changing it incompatibly
--
>> with careful deliberation -- are not impossibly huge. I mean, even
>> libstdc++ changes its ABI occassionally! LLVM's C API certainly
shouldn't
>> be considered more important to keep stable than libstdc++!
>>
>> That said, incompatible changes ought not be made just for trivial
>> reasons: they should be made when there's a true need, e.g. when an
>> existing LLVM-C function cannot sensibly maintain its existing behavior
>> anymore because the underlying functionality of LLVM was removed, or
>> drastically changed.
>>
>
> FWIW we do this (drastically change LLVM) pretty often. See Debug Info
> metadata, DataLayout, my typeless pointer work, Chandler's pass manager
> work, etc.
>
>
>>
>> I'd propose that the only 100% strict rule should be that if the
ABI/API
>> changes, it is done in a way that *loudly* breaks old programs -- e.g.
they
>> fail to compile, link, or run (depending on how the other-lang wrappers
are
>> accessing the API functions) -- not that you get some random weird
>> misbehavior because a function's argument types or return type has
been
>> changed.
>>
>> Ideally, if you're going to remove an API, you'd deprecate it
first, if
>> you can see the problem coming. Otherwise, oh well -- it's not
reasonable
>> to hold up LLVM development to adhere to deprecation policy on the C
API.
>>
>> That is:
>> 1) Adding new functions for new LLVM functionality: great.
>> 2) Removing old functions: if it's required -- after careful
deliberation.
>> 3) Modifying the signature of existing functions: no way.
>>
>> A couple of concrete examples:
>>
>> - Let's say you run out of space in an existing enum type (ahem,
>> LLVMAttribute...).
>>
>> You should introduce new function names, taking a better type (perhaps
>> some sort of set type, instead of a fixed-width integer...), mark the
old
>> ones deprecated so that users will update to the new functions. But,
keep
>> the old ones around so that old code can keep working. There's no
real need
>> to remove the old deprecated functions -- they'll keep working as
well as
>> they ever did with the smaller-valued attributes.
>>
>
> But does it? What if the code is checking if two functions have the same
> attributes (perhaps this is an invalid notion in the absence of information
> about specific attributes) - if you compare the smaller-ranged attributes,
> you might incorrectly conclude that these two functions have the same
> attributes. Similarly for "no attributes specified" - you look at
the enum,
> see that none are set & conclude that it's a plain, uninteresting
function
> - so you create another plain, no-attributes function you think you can
> make equivalent.
>
>
>>
>> - Let's say you remove types from pointers.
>>
>> This is a change that can be seen coming up. So, one should fairly
early
>> on add an LLVMBuildGEP2 function that takes the extra type argument.
Leave
>> the old function name around for now (deprecated), because with the
current
>> state of LLVM, it can continue to work just fine.
>>
>> When LLVM is changed to *actually* remove all pointer types in its
core,
>> then the LLVMBuildGEP function cannot reasonably continue to exist. It
>> simply cannot work, if pointers don't have types, and the type
isn't
>> provided. So, remove it.
>>
>
> Yep, this is one we /might/ be able to manage - because we'll probably
> have to keep typed pointer information around for old bitcode
> deserialization. (but it does make the API awkward - when we finally do
> remove LLVMBuildGEP, do we rename LLVMBuildGEP2? Do we leave it there?
&
> then we need LLVMBuildGEP3, etc, as things evolve?))
>
> Also this works for writing, but not for reading - if code was expecting
> to read IR and assumed that pointers had types and bitcasts were present,
> etc - it's going to have a Bad Time reading new IR... the APIs
wouldn't
> make any sense (there would be no getPointeeType on PointerType anymore -
> there would be no Type that could be returned in response to that query)
>
> That's part of Eric's point, I think, once the API is low level
enough, it
> will churn with every change in LLVM or possible require some heroics to
> avoid that churn which will slow down LLVM's API velocity (which is
> somewhere around "ludicrous speed"), which is something we rather
like. The
> only way to make the C API more stable is to make it higher level (see
> libLTO for an example) so that we can own the migration & shelter users
> from it through that abstraction. If it's low level, it's going to
churn or
> slow us down (and probably both).
>
>
Precisely :)

-eric

>
>> Preferably, there would be a release of LLVM in between with both
>> functions available to let people have some time to adjust, but if not,
oh
>> well.
>>
>>
>> On Mon, Aug 17, 2015 at 1:28 PM, deadal nix via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>>
>>>
>>> 2015-08-17 8:34 GMT-07:00 Reid Kleckner <rnk at google.com>:
>>>
>>>> On Sun, Aug 16, 2015 at 10:34 PM, deadal nix via llvm-dev <
>>>> llvm-dev at lists.llvm.org> wrote:
>>>>
>>>>>
>>>>> 2015-08-16 21:51 GMT-07:00 Eric Christopher <echristo at
gmail.com>:
>>>>>>
>>>>>> The promise of stability. We don't promise that the
C++ API will stay
>>>>>> stable.
>>>>>>
>>>>>
>>>>>
>>>>> Why was that promise be made in the first place ? Has it
been made in
>>>>> the first place ?
>>>>>
>>>>
>>>> It sounds like you're in favor of dropping C API stability
then, if
>>>> it's holding us back? That feedback is actually really
helpful. :)
>>>>
>>>> There are really three goals here: flexibility to change LLVM
IR,
>>>> completeness of the C API, and stability of the C API. Pick
two.
>>>>
>>>>
>>> Yes but these aren't black and white IMO. These are need in
tension,
>>> sure, but all API have these kind of problem not specifically the C
API.
>>>
>>>
>>>
>>>> The goals are mutually incompatible and we have to trade off
one for
>>>> the other. Most of the LLVM core developers value goal #1, the
ability to
>>>> change the IR. Look at the pointee type changes that David
Blaikie is
>>>> working on, and the new EH representation. If we promise both
stability and
>>>> completeness, these things are impossible.
>>>>
>>>>
>>> Yes I was actually thinking about that. This is the kind of change
that
>>> justify a change in the C API IMO. We should try our best to keep
the C API
>>> stable, but if somethign fundamental changes in LLVM, and that
there is no
>>> nice way to map it to the old API, then
>>>
>>>
>>>
>>>> One way forward is to guarantee stability, but limit
completeness. This
>>>> would mean limiting the C API to constructs that will always
and forever be
>>>> easily represented in LLVM.
>>>>
>>>>
>>> That would be a problem for me. It seems like other in this thread
>>> mentioned the incompleteness of the C API as a problem for them and
>>> mentioned the use of various hacks to work around it. On my side I
tried to
>>> propose diff to increase completeness of the C API in the past, but
the
>>> review process was frankly too painful and I mostly give up.
I'd be up for
>>> being a maintainer of the C API if that is needed.
>>>
>>>
>>>
>>>> The other choice is to forget stability and wrap the C++ API
>>>> completely, potentially with something auto-generated. We could
make a more
>>>> limited stability promise along the lines of "these APIs
will be updated to
>>>> reflect changes to the IR, and are otherwise stable." I
think everyone
>>>> would be fine with that.
>>>>
>>>
>>> I do not think this is a good idea. The C API is used a lot to use
LLVM
>>> from other languages than C++ . When used directly from C, you get
function
>>> signature, but when used from other languages, changing method
randomly
>>> cause undefined behavior. For instance, the landing pad instruction
was
>>> changed recently and the C API to create a landing pad was updated.
As a
>>> result, 2 thing happened :
>>>  - It was not possible to set the personality function from C,
which is
>>> a problem for many frontends.
>>>  - In other languages, as mangling remained the same, the thing
compiles
>>> and fail randomly at runtime.
>>>
>>> This need to be avoided as much as possible. I'd propose the
following
>>> policy :
>>> 1/ Do change the IR as per need of LLVM development. The C API must
not
>>> prevent LLVM to move forward.
>>> 2/ If the change can be mapped nicely without changing the C API,
then
>>> do that.
>>> 3/ If there is no way to map it nicely, then update the C API.
>>>
>>> For instance, the pointer update work would probably be a valid
reason
>>> to go with 3/ .
>>>
>>> Right now, as far as I'm concerned, the main limitation of the
C API are
>>> the capability to set attributes and the capability to use atomic
>>> loads/stores.
>>>
>>> In any ways, I do think the patch adding tests for existing API
should
>>> be merged. The need for test for the C API was mentioned various
times in
>>> this thread and I do think that whatever the road taken, having
test is a
>>> good thing.
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org         http://llvm.cs.uiuc.edu
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150817/052a1c3c/attachment.html>

2015-08-17 13:47 GMT-07:00 David Blaikie <dblaikie at gmail.com>:
>
>
>
>>
>> I'd propose that the only 100% strict rule should be that if the
ABI/API
>> changes, it is done in a way that *loudly* breaks old programs -- e.g.
they
>> fail to compile, link, or run (depending on how the other-lang wrappers
are
>> accessing the API functions) -- not that you get some random weird
>> misbehavior because a function's argument types or return type has
been
>> changed.
>>
>> Ideally, if you're going to remove an API, you'd deprecate it
first, if
>> you can see the problem coming. Otherwise, oh well -- it's not
reasonable
>> to hold up LLVM development to adhere to deprecation policy on the C
API.
>>
>> That is:
>> 1) Adding new functions for new LLVM functionality: great.
>> 2) Removing old functions: if it's required -- after careful
deliberation.
>> 3) Modifying the signature of existing functions: no way.
>>
>> A couple of concrete examples:
>>
>> - Let's say you run out of space in an existing enum type (ahem,
>> LLVMAttribute...).
>>
>> You should introduce new function names, taking a better type (perhaps
>> some sort of set type, instead of a fixed-width integer...), mark the
old
>> ones deprecated so that users will update to the new functions. But,
keep
>> the old ones around so that old code can keep working. There's no
real need
>> to remove the old deprecated functions -- they'll keep working as
well as
>> they ever did with the smaller-valued attributes.
>>
>
> But does it? What if the code is checking if two functions have the same
> attributes (perhaps this is an invalid notion in the absence of information
> about specific attributes) - if you compare the smaller-ranged attributes,
> you might incorrectly conclude that these two functions have the same
> attributes. Similarly for "no attributes specified" - you look at
the enum,
> see that none are set & conclude that it's a plain, uninteresting
function
> - so you create another plain, no-attributes function you think you can
> make equivalent.
>
>
The attribute situation is fairly symptomatic of the state of the C API.
You have 2 person in this thread that tried to improve that part of the API
but eventually gave up. You have more person in that thread saying they'd
need it and have to do various workaround to get things done without it.


>
>> - Let's say you remove types from pointers.
>>
>> This is a change that can be seen coming up. So, one should fairly
early
>> on add an LLVMBuildGEP2 function that takes the extra type argument.
Leave
>> the old function name around for now (deprecated), because with the
current
>> state of LLVM, it can continue to work just fine.
>>
>> When LLVM is changed to *actually* remove all pointer types in its
core,
>> then the LLVMBuildGEP function cannot reasonably continue to exist. It
>> simply cannot work, if pointers don't have types, and the type
isn't
>> provided. So, remove it.
>>
>
> Yep, this is one we /might/ be able to manage - because we'll probably
> have to keep typed pointer information around for old bitcode
> deserialization. (but it does make the API awkward - when we finally do
> remove LLVMBuildGEP, do we rename LLVMBuildGEP2? Do we leave it there?
&
> then we need LLVMBuildGEP3, etc, as things evolve?))
>
>
LLVMBuildTypedGEP or something and we can be done with it. When The old way
is not supported anymore, LLVMBuildGEP can be removed, and eventually
recycled if the need arise.

> Also this works for writing, but not for reading - if code was expecting
> to read IR and assumed that pointers had types and bitcasts were present,
> etc - it's going to have a Bad Time reading new IR... the APIs
wouldn't
> make any sense (there would be no getPointeeType on PointerType anymore -
> there would be no Type that could be returned in response to that query)
>
> That's part of Eric's point, I think, once the API is low level
enough, it
> will churn with every change in LLVM or possible require some heroics to
> avoid that churn which will slow down LLVM's API velocity (which is
> somewhere around "ludicrous speed"), which is something we rather
like. The
> only way to make the C API more stable is to make it higher level (see
> libLTO for an example) so that we can own the migration & shelter users
> from it through that abstraction. If it's low level, it's going to
churn or
> slow us down (and probably both).
>
>
Most of it won't move from one version to another. If something fundamental
changes in LLVM from one version to the other, it is not a big deal to
update the C API to reflect it. I do not think it is going to need anything
heroic to keep it in place.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150817/9201e139/attachment.html>