Evan Cheng
2014-Jul-10 18:29 UTC
[LLVMdev] Clarification on the backward compatibility promises
> On Jul 9, 2014, at 9:52 PM, Eric Christopher <echristo at gmail.com> wrote: > > On Wed, Jul 9, 2014 at 9:33 PM, Owen Anderson <resistor at mac.com> wrote: >> >> On Jul 9, 2014, at 3:51 PM, Eric Christopher <echristo at gmail.com> wrote: >> >> On Wed, Jul 9, 2014 at 3:12 PM, Evan Cheng <evan.cheng at apple.com> wrote: >> >> >> On Jun 17, 2014, at 2:10 PM, Eric Christopher <echristo at gmail.com> wrote: >> >> 2. Metadata compatibility. We already had precedence of introducing >> incompatible changes into metadata format in the past within release. >> Should we use relaxes rules for metadata compatibility? >> >> >> I think we have a special case for debug metadata (and should document >> that), but otherwise I think we should hold metadata to the same >> standard as the rest of the IR. >> >> >> The idea with metadata is that it can be removed and everything still >> works. I'm definitely not ready to lock down the debug metadata format >> and I really don't think we should for any of the other uses since >> stripping already works. (Note, I don't consider function attributes >> etc as metadata) >> >> >> We may need to rethink this. If metadata is used only as optimization / >> codegen hints, then yes I agree they can be dropped. But I suspect there is >> a need for metadata that’s *required* for correctness. As LLVM continues to >> gain clients beyond “just” compilers, we will need to be sensitive to their >> needs. I anticipate use of LLVM bitcode files as persistent object format. >> >> >> I think that metadata that's required for correctness should be baked >> into the IR and not be metadata - so if there's something we need for >> correctness we need to come up with an IR extension. See the recent >> comdat work as an example. >> >> >> That’s not really a practical suggestion for clients that aren’t essentially >> clang. The bar to changing the IR is (correctly) very high, essentially >> unreachable if the client is out-of-tree. >> > > Sure, but they likely have their own metadata format with their own > needs and can keep their own local patches for their out of tree > extensions right? As far as I know we don't have any metadata > extensions in tree that are required for any correctness. If so, > they've explicitly gone against the rules we set for metadata a long > time back: > > http://blog.llvm.org/2010/04/extensible-metadata-in-llvm-ir.html > > Unless I'm missing your point completely of course :)I don’t disagree with this. I am only cautioning against taking an absolutely hardline attitude towards metadata compatibility. As long as we take a pragmatic approach by providing ways for clients to maintain backward compatibility, I don’t think anyone will have problems with the stated policy. We will need to be very careful if / when we propose fundamental changes. Evan> > -eric
Eric Christopher
2014-Jul-10 18:40 UTC
[LLVMdev] Clarification on the backward compatibility promises
On Thu, Jul 10, 2014 at 11:29 AM, Evan Cheng <evan.cheng at apple.com> wrote:> >> On Jul 9, 2014, at 9:52 PM, Eric Christopher <echristo at gmail.com> wrote: >> >> On Wed, Jul 9, 2014 at 9:33 PM, Owen Anderson <resistor at mac.com> wrote: >>> >>> On Jul 9, 2014, at 3:51 PM, Eric Christopher <echristo at gmail.com> wrote: >>> >>> On Wed, Jul 9, 2014 at 3:12 PM, Evan Cheng <evan.cheng at apple.com> wrote: >>> >>> >>> On Jun 17, 2014, at 2:10 PM, Eric Christopher <echristo at gmail.com> wrote: >>> >>> 2. Metadata compatibility. We already had precedence of introducing >>> incompatible changes into metadata format in the past within release. >>> Should we use relaxes rules for metadata compatibility? >>> >>> >>> I think we have a special case for debug metadata (and should document >>> that), but otherwise I think we should hold metadata to the same >>> standard as the rest of the IR. >>> >>> >>> The idea with metadata is that it can be removed and everything still >>> works. I'm definitely not ready to lock down the debug metadata format >>> and I really don't think we should for any of the other uses since >>> stripping already works. (Note, I don't consider function attributes >>> etc as metadata) >>> >>> >>> We may need to rethink this. If metadata is used only as optimization / >>> codegen hints, then yes I agree they can be dropped. But I suspect there is >>> a need for metadata that’s *required* for correctness. As LLVM continues to >>> gain clients beyond “just” compilers, we will need to be sensitive to their >>> needs. I anticipate use of LLVM bitcode files as persistent object format. >>> >>> >>> I think that metadata that's required for correctness should be baked >>> into the IR and not be metadata - so if there's something we need for >>> correctness we need to come up with an IR extension. See the recent >>> comdat work as an example. >>> >>> >>> That’s not really a practical suggestion for clients that aren’t essentially >>> clang. The bar to changing the IR is (correctly) very high, essentially >>> unreachable if the client is out-of-tree. >>> >> >> Sure, but they likely have their own metadata format with their own >> needs and can keep their own local patches for their out of tree >> extensions right? As far as I know we don't have any metadata >> extensions in tree that are required for any correctness. If so, >> they've explicitly gone against the rules we set for metadata a long >> time back: >> >> http://blog.llvm.org/2010/04/extensible-metadata-in-llvm-ir.html >> >> Unless I'm missing your point completely of course :) > > I don’t disagree with this. I am only cautioning against taking an absolutely hardline attitude towards metadata compatibility. As long as we take a pragmatic approach by providing ways for clients to maintain backward compatibility, I don’t think anyone will have problems with the stated policy. We will need to be very careful if / when we propose fundamental changes. >I'm not inclined to change any metadata consumer just for kicks, and if we want to change how metadata itself works then that should be a more reasoned and careful change. I.e. I see a user of metadata as something that we don't guarantee compatibility for, but the metadata system itself I can see as a more formal IR construct. Make sense? -eric
Eric Christopher
2014-Jul-10 18:47 UTC
[LLVMdev] Clarification on the backward compatibility promises
On Thu, Jul 10, 2014 at 11:40 AM, Eric Christopher <echristo at gmail.com> wrote:> On Thu, Jul 10, 2014 at 11:29 AM, Evan Cheng <evan.cheng at apple.com> wrote: >> >>> On Jul 9, 2014, at 9:52 PM, Eric Christopher <echristo at gmail.com> wrote: >>> >>> On Wed, Jul 9, 2014 at 9:33 PM, Owen Anderson <resistor at mac.com> wrote: >>>> >>>> On Jul 9, 2014, at 3:51 PM, Eric Christopher <echristo at gmail.com> wrote: >>>> >>>> On Wed, Jul 9, 2014 at 3:12 PM, Evan Cheng <evan.cheng at apple.com> wrote: >>>> >>>> >>>> On Jun 17, 2014, at 2:10 PM, Eric Christopher <echristo at gmail.com> wrote: >>>> >>>> 2. Metadata compatibility. We already had precedence of introducing >>>> incompatible changes into metadata format in the past within release. >>>> Should we use relaxes rules for metadata compatibility? >>>> >>>> >>>> I think we have a special case for debug metadata (and should document >>>> that), but otherwise I think we should hold metadata to the same >>>> standard as the rest of the IR. >>>> >>>> >>>> The idea with metadata is that it can be removed and everything still >>>> works. I'm definitely not ready to lock down the debug metadata format >>>> and I really don't think we should for any of the other uses since >>>> stripping already works. (Note, I don't consider function attributes >>>> etc as metadata) >>>> >>>> >>>> We may need to rethink this. If metadata is used only as optimization / >>>> codegen hints, then yes I agree they can be dropped. But I suspect there is >>>> a need for metadata that’s *required* for correctness. As LLVM continues to >>>> gain clients beyond “just” compilers, we will need to be sensitive to their >>>> needs. I anticipate use of LLVM bitcode files as persistent object format. >>>> >>>> >>>> I think that metadata that's required for correctness should be baked >>>> into the IR and not be metadata - so if there's something we need for >>>> correctness we need to come up with an IR extension. See the recent >>>> comdat work as an example. >>>> >>>> >>>> That’s not really a practical suggestion for clients that aren’t essentially >>>> clang. The bar to changing the IR is (correctly) very high, essentially >>>> unreachable if the client is out-of-tree. >>>> >>> >>> Sure, but they likely have their own metadata format with their own >>> needs and can keep their own local patches for their out of tree >>> extensions right? As far as I know we don't have any metadata >>> extensions in tree that are required for any correctness. If so, >>> they've explicitly gone against the rules we set for metadata a long >>> time back: >>> >>> http://blog.llvm.org/2010/04/extensible-metadata-in-llvm-ir.html >>> >>> Unless I'm missing your point completely of course :) >> >> I don’t disagree with this. I am only cautioning against taking an absolutely hardline attitude towards metadata compatibility. As long as we take a pragmatic approach by providing ways for clients to maintain backward compatibility, I don’t think anyone will have problems with the stated policy. We will need to be very careful if / when we propose fundamental changes. >> > > I'm not inclined to change any metadata consumer just for kicks, and > if we want to change how metadata itself works then that should be a > more reasoned and careful change. > > I.e. I see a user of metadata as something that we don't guarantee > compatibility for, but the metadata system itself I can see as a more > formal IR construct. > > Make sense? >An addendum, versioning metadata once you've got a stable format would probably be a good idea if you want to worry about future compatibility. As an example, I can see the loop metadata getting a version that can be easily read by a consumer and "I can't cope with this" can just mean it can be ignored etc. However, that's a maintainability/backward compatibility thing for each metadata producer to deal with - I removed it from debug info because we were changing so fast that it was a significant overhead. That said, if someone wants to change a metadata format to be more efficient or expressive then it should have a significantly lower bar than an IR level change. -eric
Evan Cheng
2014-Jul-11 22:25 UTC
[LLVMdev] Clarification on the backward compatibility promises
> On Jul 10, 2014, at 11:40 AM, Eric Christopher <echristo at gmail.com> wrote: > > On Thu, Jul 10, 2014 at 11:29 AM, Evan Cheng <evan.cheng at apple.com> wrote: >> >>> On Jul 9, 2014, at 9:52 PM, Eric Christopher <echristo at gmail.com> wrote: >>> >>> On Wed, Jul 9, 2014 at 9:33 PM, Owen Anderson <resistor at mac.com> wrote: >>>> >>>> On Jul 9, 2014, at 3:51 PM, Eric Christopher <echristo at gmail.com> wrote: >>>> >>>> On Wed, Jul 9, 2014 at 3:12 PM, Evan Cheng <evan.cheng at apple.com> wrote: >>>> >>>> >>>> On Jun 17, 2014, at 2:10 PM, Eric Christopher <echristo at gmail.com> wrote: >>>> >>>> 2. Metadata compatibility. We already had precedence of introducing >>>> incompatible changes into metadata format in the past within release. >>>> Should we use relaxes rules for metadata compatibility? >>>> >>>> >>>> I think we have a special case for debug metadata (and should document >>>> that), but otherwise I think we should hold metadata to the same >>>> standard as the rest of the IR. >>>> >>>> >>>> The idea with metadata is that it can be removed and everything still >>>> works. I'm definitely not ready to lock down the debug metadata format >>>> and I really don't think we should for any of the other uses since >>>> stripping already works. (Note, I don't consider function attributes >>>> etc as metadata) >>>> >>>> >>>> We may need to rethink this. If metadata is used only as optimization / >>>> codegen hints, then yes I agree they can be dropped. But I suspect there is >>>> a need for metadata that’s *required* for correctness. As LLVM continues to >>>> gain clients beyond “just” compilers, we will need to be sensitive to their >>>> needs. I anticipate use of LLVM bitcode files as persistent object format. >>>> >>>> >>>> I think that metadata that's required for correctness should be baked >>>> into the IR and not be metadata - so if there's something we need for >>>> correctness we need to come up with an IR extension. See the recent >>>> comdat work as an example. >>>> >>>> >>>> That’s not really a practical suggestion for clients that aren’t essentially >>>> clang. The bar to changing the IR is (correctly) very high, essentially >>>> unreachable if the client is out-of-tree. >>>> >>> >>> Sure, but they likely have their own metadata format with their own >>> needs and can keep their own local patches for their out of tree >>> extensions right? As far as I know we don't have any metadata >>> extensions in tree that are required for any correctness. If so, >>> they've explicitly gone against the rules we set for metadata a long >>> time back: >>> >>> http://blog.llvm.org/2010/04/extensible-metadata-in-llvm-ir.html >>> >>> Unless I'm missing your point completely of course :) >> >> I don’t disagree with this. I am only cautioning against taking an absolutely hardline attitude towards metadata compatibility. As long as we take a pragmatic approach by providing ways for clients to maintain backward compatibility, I don’t think anyone will have problems with the stated policy. We will need to be very careful if / when we propose fundamental changes. >> > > I'm not inclined to change any metadata consumer just for kicks, and > if we want to change how metadata itself works then that should be a > more reasoned and careful change. > > I.e. I see a user of metadata as something that we don't guarantee > compatibility for, but the metadata system itself I can see as a more > formal IR construct. > > Make sense?Yes, this sounds generally reasonable. evan> > -eric