On Jul 19, 2015, at 7:24 PM, Eric Christopher <echristo at gmail.com> wrote:> So, I made this proposal for what I think is a pretty good reason. There's an "unofficial" as Juergen said, policy that the C API is the stable API. There's nothing wrong with a stable C API, but that's what I'm proposing should move out of tree to where those that are most concerned with it can develop it and ensure that it remains stable for whatever guarantees they want.I don’t get it :-)> Some background here: > > Right now we definitely have this dichotomy between a "bindings" C API and a "stable" C API. The unofficial policy as I mentioned above is that there's one C API and that's the stable API. Over the last 3-5 years or so the "stable" C API has started growing in ways that encompass just about every class and API in llvm. We've occasionally denied bindings level API support because we knew the code in that area was going to change - but just imagine we'd let a few of them in, we wouldn't have been able to do the IR/Metadata split at all. As it is we technically broke the C API, just not in a way that any external user cared about.Ok, I don’t see how splitting it out to a separate project helps that.> Back to the proposal: > > What I'm proposing is that we make the C API that exists in tree a bindings API that has the same stability guarantees as the C++ API. Honestly it'll probably much more stable, but at least then we won't have to worry or revert work because the C API was "too close to the machine" or rather the C++ code. This means that someone that wants a stable C API can go off and develop one (tests and all) and we can possibly look at bringing it back into tree at some point in the future. For example, if someone comes up with a good "libjit" api then we can look at how the API design works and make sure it's general enough that it's not going to cause undue solidification of the existing APIs. > > Caveat: I'm not talking about the existing libclang or liblto libraries. Those seem to work and have a small enough API surface that they seem reasonable to support and we can move to a new API if they seem to be hindering development in the future. > > This help explain where I'm coming from here?No, this makes me even more concerned. The entire original reason for the C API is to provide a stable interface for people who didn’t want to chase the C++ API (or didn’t want to chase *as much* of it). In this sense, I consider the LLVM C API to be unlike the other “bindings” for generating IR: its purpose is to remain stable, not to be a language binding for people who prefer to write things in C. If your claim is that the C API is too restrictive and prevents C++ evolution, then we just need to be careful about what we add to the C API. Splitting it out to another project would not address this concern anyway, unless what you’re really saying is “we can break the C API even though it is trying to be stable, because its an external project”. -Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150719/3015f81a/attachment.html>
On Sun, Jul 19, 2015 at 9:58 PM Chris Lattner <clattner at apple.com> wrote:> No, this makes me even more concerned. > > The entire original reason for the C API is to provide a stable interface > for people who didn’t want to chase the C++ API (or didn’t want to chase > *as much* of it). >That may be the original reason, but it is not currently the only reason why people are using the C API today.> In this sense, I consider the LLVM C API to be unlike the other > “bindings” for generating IR: its purpose is to remain stable, not to be a > language binding for people who prefer to write things in C. >While that may have been the original intent, one of the other usages of the C APIs today are simply to provide access to the C++ APIs from languages which have C FFIs. For example, OCaml and Go both have bindings in those languages built on top of the C bindings.> If your claim is that the C API is too restrictive and prevents C++ > evolution, then we just need to be careful about what we add to the C API. >I agree. However, this desire is in tension with the desires of those that see these as bindings for the C++ APIs to mirror each and every change in the C++ APIs in order to access them from other language. Increasingly, I think that our current design of the C APIs have been overly influenced by the needs of these users. As a consequence, I think we have let a great deal enter the C API that isn't really reasonable for a stable and very long-term API. As it stands today, I think that outside of libLTO and libClang, the LLVM C API is more suitable as a set of non-stable bindings than as a stable C interface to LLVM's core functionality. I also think that this use case is a real and legitimate one. As such, I think that it would be a positive change to turn the current C API into an *explicit* set of bindings that have exactly as much API stability as the C++ APIs do. This would mean that ToT would change freely, and minor releases would remain stable. But I also completely agree with your view on how a *stable* C API should be designed, and I also think *that* is a useful thing. I just don't think we have a remotely good design for it today. I think designing such a stable API would be a big and useful project for someone to undertake. I have some ideas about how to do it in a way that would both be useful and usefully stable, but I think it would look very different from the current C API. Splitting it out to another project would not address this concern anyway,> unless what you’re really saying is “we can break the C API even though it > is trying to be stable, because its an external project”. >I also don't really see why a separate project helps. Maybe it would help for the design of a stable API for use by JIT-ing consumers or frontends would best start as separate project where it could evolve some before stabilizing? Unsure if that's useful or necessary. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150720/131249b8/attachment.html>
On Sun, Jul 19, 2015 at 9:55 PM Chris Lattner <clattner at apple.com> wrote:> On Jul 19, 2015, at 7:24 PM, Eric Christopher <echristo at gmail.com> wrote: > > So, I made this proposal for what I think is a pretty good reason. There's > an "unofficial" as Juergen said, policy that the C API is the stable API. > There's nothing wrong with a stable C API, but that's what I'm proposing > should move out of tree to where those that are most concerned with it can > develop it and ensure that it remains stable for whatever guarantees they > want. > > > I don’t get it :-) > > Some background here: > > Right now we definitely have this dichotomy between a "bindings" C API and > a "stable" C API. The unofficial policy as I mentioned above is that > there's one C API and that's the stable API. Over the last 3-5 years or so > the "stable" C API has started growing in ways that encompass just about > every class and API in llvm. We've occasionally denied bindings level API > support because we knew the code in that area was going to change - but > just imagine we'd let a few of them in, we wouldn't have been able to do > the IR/Metadata split at all. As it is we technically broke the C API, just > not in a way that any external user cared about. > > > Ok, I don’t see how splitting it out to a separate project helps that. > > Back to the proposal: > > What I'm proposing is that we make the C API that exists in tree a > bindings API that has the same stability guarantees as the C++ API. > Honestly it'll probably much more stable, but at least then we won't have > to worry or revert work because the C API was "too close to the machine" or > rather the C++ code. This means that someone that wants a stable C API can > go off and develop one (tests and all) and we can possibly look at bringing > it back into tree at some point in the future. For example, if someone > comes up with a good "libjit" api then we can look at how the API design > works and make sure it's general enough that it's not going to cause undue > solidification of the existing APIs. > > Caveat: I'm not talking about the existing libclang or liblto libraries. > Those seem to work and have a small enough API surface that they seem > reasonable to support and we can move to a new API if they seem to be > hindering development in the future. > > This help explain where I'm coming from here? > > > No, this makes me even more concerned. >Well, but it did explain ;)> > The entire original reason for the C API is to provide a stable interface > for people who didn’t want to chase the C++ API (or didn’t want to chase > *as much* of it). In this sense, I consider the LLVM C API to be unlike > the other “bindings” for generating IR: its purpose is to remain stable, > not to be a language binding for people who prefer to write things in C. > >Which part of the C API is like bindings and which part is a stable API? There's no clear delineation right now.> If your claim is that the C API is too restrictive and prevents C++ > evolution, then we just need to be careful about what we add to the C API. > Splitting it out to another project would not address this concern anyway, > unless what you’re really saying is “we can break the C API even though it > is trying to be stable, because its an external project”. >What I'm saying is that we should leave the C API in as bindings support in and if someone actually designs an API worth considering stable then we can bring that into the project. I'm assuming an external iteration on this because I can't even imagine a) the amount of bikeshedding to an a priori proposal on the lists, and b) I imagine it would take some iteration to get it to something that works that isn't too tied to the IR format. The current set of C API (modulo the caveats I mentioned) are basically bindings. You have everything from location of calling convention to where in the IR the personality function is described as current C API - I don't think this is a good design for a stable API, it's too dependent upon the ordering of things in the IR in a way that is more strict than even what we can autoupgrade. -eric -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150720/0a6e1895/attachment.html>
On 20 Jul 2015, at 05:54, Chris Lattner <clattner at apple.com> wrote:> > The entire original reason for the C API is to provide a stable interface for people who didn’t want to chase the C++ API (or didn’t want to chase *as much* of it). In this sense, I consider the LLVM C API to be unlike the other “bindings” for generating IR: its purpose is to remain stable, not to be a language binding for people who prefer to write things in C.I’m not convinced that this has worked. We’ve done some cleanup in the FreeBSD ports tree recently, but we still have LLVM 3.3-3.7 to support all of the external projects that are using it. 3.3 looks as if it is down to only one consumer (the DRI drivers for X11, which is probably the most high-profile user of LLVM for most of our users). Most external projects are using the C++ APIs because they invariably find at least one thing that isn’t covered by the C APIs that they need to do and just eat the cost of periodically moving to newer versions. The instability of the C++ APIs means that we can move fast, which is very nice, but it also means that external projects typically jump from release to release (often skipping one or more) and so we don’t get nearly as good testing as we would if we had more external users of a stable API, who could just grab head, recompile, and run their own test suites every week or so. Given libClang, the goal of a C API as bindings ought to be much simpler - we should just be able to annotate some classes that we want to export and machine-generate a C API with the same stability guarantees as the C++ API. If we want a stable API that’s actually useful, then this needs much more careful thought about what its requirements are. Splitting it into separate sets of interfaces might make sense. About the only thing that you can really do with the existing C APIs is write a front end, so making sure that we have a stable API for generating LLVM IR, a stable API for instantiating a JIT / generating object code, and (once LLD is a bit more mature) a stable API for generating linked binaries, would be of significant value. A stable API for writing out-of-tree passes would be very nice, but is almost certainly impossible as it relies on IR stability. A stable API for out-of-tree backends is likely to be completely impossible for a long time. David
----- Original Message -----> From: "David Chisnall" <David.Chisnall at cl.cam.ac.uk> > To: "Chris Lattner" <clattner at apple.com> > Cc: "Lang Hames" <lhames at apple.com>, "LLVM Dev" <llvmdev at cs.uiuc.edu> > Sent: Monday, July 20, 2015 3:02:27 AM > Subject: Re: [LLVMdev] [RFC] Developer Policy for LLVM C API > > On 20 Jul 2015, at 05:54, Chris Lattner <clattner at apple.com> wrote: > > > > The entire original reason for the C API is to provide a stable > > interface for people who didn’t want to chase the C++ API (or > > didn’t want to chase *as much* of it). In this sense, I consider > > the LLVM C API to be unlike the other “bindings” for generating > > IR: its purpose is to remain stable, not to be a language binding > > for people who prefer to write things in C. > > I’m not convinced that this has worked.I agree. To provide an explicit example, look at all the LLVM version dependencies in: https://github.com/o11c/llpy/blob/master/llpy/c/core.py https://github.com/o11c/llpy/blob/master/llpy/c/object.py (etc.)> We’ve done some cleanup in > the FreeBSD ports tree recently, but we still have LLVM 3.3-3.7 to > support all of the external projects that are using it. 3.3 looks > as if it is down to only one consumer (the DRI drivers for X11, > which is probably the most high-profile user of LLVM for most of our > users). Most external projects are using the C++ APIs because they > invariably find at least one thing that isn’t covered by the C APIs > that they need to do and just eat the cost of periodically moving to > newer versions.This has been my experience as well. -Hal> > The instability of the C++ APIs means that we can move fast, which is > very nice, but it also means that external projects typically jump > from release to release (often skipping one or more) and so we don’t > get nearly as good testing as we would if we had more external users > of a stable API, who could just grab head, recompile, and run their > own test suites every week or so. > > Given libClang, the goal of a C API as bindings ought to be much > simpler - we should just be able to annotate some classes that we > want to export and machine-generate a C API with the same stability > guarantees as the C++ API. If we want a stable API that’s actually > useful, then this needs much more careful thought about what its > requirements are. Splitting it into separate sets of interfaces > might make sense. About the only thing that you can really do with > the existing C APIs is write a front end, so making sure that we > have a stable API for generating LLVM IR, a stable API for > instantiating a JIT / generating object code, and (once LLD is a bit > more mature) a stable API for generating linked binaries, would be > of significant value. A stable API for writing out-of-tree passes > would be very nice, but is almost certainly impossible as it relies > on IR stability. A stable API for out-of-tree backends is likely to > be completely impossible for a long time. > > David > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-- Hal Finkel Assistant Computational Scientist Leadership Computing Facility Argonne National Laboratory