Greg Bedwell
2014-Sep-27 10:19 UTC
[LLVMdev] [cfe-dev] Proposal to add Bitcode version field to bitcode file wrapper
As I understand it, the bitcode compatibility promise doesn't extend as far as debug info metadata (happy to be wrong here!). I think we have a usecase where need to guarantee that debug information from any two arbitrary bitcode files going into an LTO link will result in the expected/correct debug information going into the resulting ELF file; unless we can be sure that this will always work between bitcode files generated by different versions we'd need some way of flagging up an incompatibility and providing useful information on the reason to the user. --Greg Bedwell SN Systems - Sony Computer Entertainment Group On 27 September 2014 02:24, Sean Silva <chisophugis at gmail.com> wrote:> > > On Fri, Sep 26, 2014 at 5:18 PM, Yung, Douglas < > douglas_yung at playstation.sony.com> wrote: > >> Sorry if I was unclear. There are currently no “known >> incompatibilities” that I am aware of, although I fully admit to not being >> an expert on the topic. The idea is that we add versioning information to >> the bitcode so that if an issue were discovered, it could be easily >> detected and dealt with. >> > > It sounds like time would be better invested in improving the testing of > our bitcode compatibility promise. > > -- Sean Silva > > >> >> >> Douglas Yung >> >> >> >> *From:* Bob Wilson [mailto:bob.wilson at apple.com] >> *Sent:* Friday, September 26, 2014 16:39 >> *To:* Yung, Douglas >> *Cc:* llvmdev at cs.uiuc.edu; cfe-dev at cs.uiuc.edu >> *Subject:* Re: [LLVMdev] Proposal to add Bitcode version field to >> bitcode file wrapper >> >> >> >> Bitcode backward compatibility, at least for the current major version, >> is supposed to make this unnecessary. Can you provide more information >> about what “known incompatibilities” you’re seeing? >> >> >> >> On Sep 26, 2014, at 2:40 PM, Yung, Douglas < >> douglas_yung at playstation.sony.com> wrote: >> >> >> >> Hi, >> >> >> >> We would like to add a version number to the bitcode wrapper. This >> feature would allow easier identification of what compiler produced the >> bitcode so that known incompatibilities with LTO compilation could be >> detected. Roughly speaking, this version number would consist of the major, >> minor and optionally the patch version of the compiler used to produce the >> bitcode. The version information would be encoded in 4 bytes, with the >> first byte representing the major version number, the second byte the minor >> version number, and the third and fourth bytes optionally encoding the >> patch version or other information. As to where to place this information, >> we are considering two different possibilities for updating the bitcode >> wrapper specification. >> >> >> >> The first is to simply add a single 32bit wide field at the end of the >> existing bitcode wrapper format field. This would result in the new >> structure looking like this: >> >> [Magic_{32}, Version_{32}, Offset_{32}, Size_{32}, CPUType_{32}, >> BitcodeVersion_{32}] >> >> All of the existing fields would keep their current meanings, and the new >> field BitcodeVersion is simply appended with the format described in the >> first paragraph. >> >> A second idea was to use the existing Version field in the bitcode >> wrapper format to store the bitcode version information. According to the >> documentation ( >> http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format) this >> field is currently always set to 0. This would allow us to make use of what >> is (presumably) an unused field. >> >> >> >> As this is a feature that we feel would be beneficial to the community, >> we wanted to get feedback on the design for our upcoming patches. Any >> thoughts or opinions on this would be greatly appreciated. >> >> >> >> Thanks! >> >> >> >> Douglas Yung >> >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> >> >> >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> >> > > _______________________________________________ > cfe-dev mailing list > cfe-dev at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140927/d30ffaa8/attachment.html>
David Blaikie
2014-Sep-27 15:48 UTC
[LLVMdev] [cfe-dev] Proposal to add Bitcode version field to bitcode file wrapper
In general the story is these days that we won't /crash/ on old debug info metadata formats, we'll just drop the old debug info metadata - so you won't get debug info, but you can still link/use your old IR libraries with new IR/compiler. On Sat, Sep 27, 2014 at 3:19 AM, Greg Bedwell <gregbedwell at gmail.com> wrote:> As I understand it, the bitcode compatibility promise doesn't extend as > far as debug info metadata (happy to be wrong here!). I think we have a > usecase where need to guarantee that debug information from any two > arbitrary bitcode files going into an LTO link will result in the > expected/correct debug information going into the resulting ELF file; > unless we can be sure that this will always work between bitcode files > generated by different versions we'd need some way of flagging up an > incompatibility and providing useful information on the reason to the user. > > --Greg Bedwell > SN Systems - Sony Computer Entertainment Group > > On 27 September 2014 02:24, Sean Silva <chisophugis at gmail.com> wrote: > >> >> >> On Fri, Sep 26, 2014 at 5:18 PM, Yung, Douglas < >> douglas_yung at playstation.sony.com> wrote: >> >>> Sorry if I was unclear. There are currently no “known >>> incompatibilities” that I am aware of, although I fully admit to not being >>> an expert on the topic. The idea is that we add versioning information to >>> the bitcode so that if an issue were discovered, it could be easily >>> detected and dealt with. >>> >> >> It sounds like time would be better invested in improving the testing of >> our bitcode compatibility promise. >> >> -- Sean Silva >> >> >>> >>> >>> Douglas Yung >>> >>> >>> >>> *From:* Bob Wilson [mailto:bob.wilson at apple.com] >>> *Sent:* Friday, September 26, 2014 16:39 >>> *To:* Yung, Douglas >>> *Cc:* llvmdev at cs.uiuc.edu; cfe-dev at cs.uiuc.edu >>> *Subject:* Re: [LLVMdev] Proposal to add Bitcode version field to >>> bitcode file wrapper >>> >>> >>> >>> Bitcode backward compatibility, at least for the current major version, >>> is supposed to make this unnecessary. Can you provide more information >>> about what “known incompatibilities” you’re seeing? >>> >>> >>> >>> On Sep 26, 2014, at 2:40 PM, Yung, Douglas < >>> douglas_yung at playstation.sony.com> wrote: >>> >>> >>> >>> Hi, >>> >>> >>> >>> We would like to add a version number to the bitcode wrapper. This >>> feature would allow easier identification of what compiler produced the >>> bitcode so that known incompatibilities with LTO compilation could be >>> detected. Roughly speaking, this version number would consist of the major, >>> minor and optionally the patch version of the compiler used to produce the >>> bitcode. The version information would be encoded in 4 bytes, with the >>> first byte representing the major version number, the second byte the minor >>> version number, and the third and fourth bytes optionally encoding the >>> patch version or other information. As to where to place this information, >>> we are considering two different possibilities for updating the bitcode >>> wrapper specification. >>> >>> >>> >>> The first is to simply add a single 32bit wide field at the end of the >>> existing bitcode wrapper format field. This would result in the new >>> structure looking like this: >>> >>> [Magic_{32}, Version_{32}, Offset_{32}, Size_{32}, CPUType_{32}, >>> BitcodeVersion_{32}] >>> >>> All of the existing fields would keep their current meanings, and the >>> new field BitcodeVersion is simply appended with the format described in >>> the first paragraph. >>> >>> A second idea was to use the existing Version field in the bitcode >>> wrapper format to store the bitcode version information. According to the >>> documentation ( >>> http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format) this >>> field is currently always set to 0. This would allow us to make use of what >>> is (presumably) an unused field. >>> >>> >>> >>> As this is a feature that we feel would be beneficial to the community, >>> we wanted to get feedback on the design for our upcoming patches. Any >>> thoughts or opinions on this would be greatly appreciated. >>> >>> >>> >>> Thanks! >>> >>> >>> >>> Douglas Yung >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>> >>> >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>> >>> >> >> _______________________________________________ >> cfe-dev mailing list >> cfe-dev at cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev >> >> > > _______________________________________________ > cfe-dev mailing list > cfe-dev at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140927/77227e40/attachment.html>
Alex Rosenberg
2014-Sep-28 06:35 UTC
[LLVMdev] [cfe-dev] Proposal to add Bitcode version field to bitcode file wrapper
How is this use case different from the LTO-supported toolchains shipped by other vendors such as Apple? Do they have this theoretical problem too? If the issue is solely constrained to debug info metadata, then why not use metadata to describe the format/version of the debug info? Alex> On Sep 27, 2014, at 3:19 AM, Greg Bedwell <gregbedwell at gmail.com> wrote: > > As I understand it, the bitcode compatibility promise doesn't extend as far as debug info metadata (happy to be wrong here!). I think we have a usecase where need to guarantee that debug information from any two arbitrary bitcode files going into an LTO link will result in the expected/correct debug information going into the resulting ELF file; unless we can be sure that this will always work between bitcode files generated by different versions we'd need some way of flagging up an incompatibility and providing useful information on the reason to the user. > > --Greg Bedwell > SN Systems - Sony Computer Entertainment Group > >> On 27 September 2014 02:24, Sean Silva <chisophugis at gmail.com> wrote: >> >> >>> On Fri, Sep 26, 2014 at 5:18 PM, Yung, Douglas <douglas_yung at playstation.sony.com> wrote: >>> Sorry if I was unclear. There are currently no “known incompatibilities” that I am aware of, although I fully admit to not being an expert on the topic. The idea is that we add versioning information to the bitcode so that if an issue were discovered, it could be easily detected and dealt with. >>> >> >> It sounds like time would be better invested in improving the testing of our bitcode compatibility promise. >> >> -- Sean Silva >> >>> >>> >>> Douglas Yung >>> >>> >>> >>> From: Bob Wilson [mailto:bob.wilson at apple.com] >>> Sent: Friday, September 26, 2014 16:39 >>> To: Yung, Douglas >>> Cc: llvmdev at cs.uiuc.edu; cfe-dev at cs.uiuc.edu >>> Subject: Re: [LLVMdev] Proposal to add Bitcode version field to bitcode file wrapper >>> >>> >>> >>> Bitcode backward compatibility, at least for the current major version, is supposed to make this unnecessary. Can you provide more information about what “known incompatibilities” you’re seeing? >>> >>> >>> >>> On Sep 26, 2014, at 2:40 PM, Yung, Douglas <douglas_yung at playstation.sony.com> wrote: >>> >>> >>> >>> Hi, >>> >>> >>> >>> We would like to add a version number to the bitcode wrapper. This feature would allow easier identification of what compiler produced the bitcode so that known incompatibilities with LTO compilation could be detected. Roughly speaking, this version number would consist of the major, minor and optionally the patch version of the compiler used to produce the bitcode. The version information would be encoded in 4 bytes, with the first byte representing the major version number, the second byte the minor version number, and the third and fourth bytes optionally encoding the patch version or other information. As to where to place this information, we are considering two different possibilities for updating the bitcode wrapper specification. >>> >>> >>> >>> The first is to simply add a single 32bit wide field at the end of the existing bitcode wrapper format field. This would result in the new structure looking like this: >>> >>> [Magic_{32}, Version_{32}, Offset_{32}, Size_{32}, CPUType_{32}, BitcodeVersion_{32}] >>> >>> All of the existing fields would keep their current meanings, and the new field BitcodeVersion is simply appended with the format described in the first paragraph. >>> >>> A second idea was to use the existing Version field in the bitcode wrapper format to store the bitcode version information. According to the documentation (http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format) this field is currently always set to 0. This would allow us to make use of what is (presumably) an unused field. >>> >>> >>> >>> As this is a feature that we feel would be beneficial to the community, we wanted to get feedback on the design for our upcoming patches. Any thoughts or opinions on this would be greatly appreciated. >>> >>> >>> >>> Thanks! >>> >>> >>> >>> Douglas Yung >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>> >>> >>> >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> >> >> _______________________________________________ >> cfe-dev mailing list >> cfe-dev at cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140927/18ea25fc/attachment.html>
David Blaikie
2014-Sep-28 08:01 UTC
[LLVMdev] [cfe-dev] Proposal to add Bitcode version field to bitcode file wrapper
On Sat, Sep 27, 2014 at 11:35 PM, Alex Rosenberg <alexr at leftfield.org> wrote:> How is this use case different from the LTO-supported toolchains shipped > by other vendors such as Apple? Do they have this theoretical problem too? > > If the issue is solely constrained to debug info metadata, then why not > use metadata to describe the format/version of the debug info? >FWIW (I haven't followed the rest of this thread) - that's what we/Apple have done. There's a module flag metadata that specifies the debug info metadata schema version, then the verifier (or some other pass, I forget how it's phrased) can check if the version matches the current LLVM's debug info metadata version, and if it doesn't match, it strips out all the debug info metadata.> > Alex > > On Sep 27, 2014, at 3:19 AM, Greg Bedwell <gregbedwell at gmail.com> wrote: > > As I understand it, the bitcode compatibility promise doesn't extend as > far as debug info metadata (happy to be wrong here!). I think we have a > usecase where need to guarantee that debug information from any two > arbitrary bitcode files going into an LTO link will result in the > expected/correct debug information going into the resulting ELF file; > unless we can be sure that this will always work between bitcode files > generated by different versions we'd need some way of flagging up an > incompatibility and providing useful information on the reason to the user. > > --Greg Bedwell > SN Systems - Sony Computer Entertainment Group > > On 27 September 2014 02:24, Sean Silva <chisophugis at gmail.com> wrote: > >> >> >> On Fri, Sep 26, 2014 at 5:18 PM, Yung, Douglas < >> douglas_yung at playstation.sony.com> wrote: >> >>> Sorry if I was unclear. There are currently no “known >>> incompatibilities” that I am aware of, although I fully admit to not being >>> an expert on the topic. The idea is that we add versioning information to >>> the bitcode so that if an issue were discovered, it could be easily >>> detected and dealt with. >>> >> >> It sounds like time would be better invested in improving the testing of >> our bitcode compatibility promise. >> >> -- Sean Silva >> >> >>> >>> >>> Douglas Yung >>> >>> >>> >>> *From:* Bob Wilson [mailto:bob.wilson at apple.com] >>> *Sent:* Friday, September 26, 2014 16:39 >>> *To:* Yung, Douglas >>> *Cc:* llvmdev at cs.uiuc.edu; cfe-dev at cs.uiuc.edu >>> *Subject:* Re: [LLVMdev] Proposal to add Bitcode version field to >>> bitcode file wrapper >>> >>> >>> >>> Bitcode backward compatibility, at least for the current major version, >>> is supposed to make this unnecessary. Can you provide more information >>> about what “known incompatibilities” you’re seeing? >>> >>> >>> >>> On Sep 26, 2014, at 2:40 PM, Yung, Douglas < >>> douglas_yung at playstation.sony.com> wrote: >>> >>> >>> >>> Hi, >>> >>> >>> >>> We would like to add a version number to the bitcode wrapper. This >>> feature would allow easier identification of what compiler produced the >>> bitcode so that known incompatibilities with LTO compilation could be >>> detected. Roughly speaking, this version number would consist of the major, >>> minor and optionally the patch version of the compiler used to produce the >>> bitcode. The version information would be encoded in 4 bytes, with the >>> first byte representing the major version number, the second byte the minor >>> version number, and the third and fourth bytes optionally encoding the >>> patch version or other information. As to where to place this information, >>> we are considering two different possibilities for updating the bitcode >>> wrapper specification. >>> >>> >>> >>> The first is to simply add a single 32bit wide field at the end of the >>> existing bitcode wrapper format field. This would result in the new >>> structure looking like this: >>> >>> [Magic_{32}, Version_{32}, Offset_{32}, Size_{32}, CPUType_{32}, >>> BitcodeVersion_{32}] >>> >>> All of the existing fields would keep their current meanings, and the >>> new field BitcodeVersion is simply appended with the format described in >>> the first paragraph. >>> >>> A second idea was to use the existing Version field in the bitcode >>> wrapper format to store the bitcode version information. According to the >>> documentation ( >>> http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format) this >>> field is currently always set to 0. This would allow us to make use of what >>> is (presumably) an unused field. >>> >>> >>> >>> As this is a feature that we feel would be beneficial to the community, >>> we wanted to get feedback on the design for our upcoming patches. Any >>> thoughts or opinions on this would be greatly appreciated. >>> >>> >>> >>> Thanks! >>> >>> >>> >>> Douglas Yung >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>> >>> >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>> >>> >> >> _______________________________________________ >> cfe-dev mailing list >> cfe-dev at cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev >> >> > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > _______________________________________________ > cfe-dev mailing list > cfe-dev at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140928/ae516e10/attachment.html>
Robinson, Paul
2014-Sep-28 22:10 UTC
[LLVMdev] [cfe-dev] Proposal to add Bitcode version field to bitcode file wrapper
| Bitcode backward compatibility, at least for the current major version, is
supposed to make this unnecessary.
I think the "at least for the current major version" part is one thing
that concerns us. LLVM 4.0 will promise to read LLVM 3.4 bitcode, but LLVM 4.1
will not, according to my understanding of the current promise. Smoothly
identifying that point and being able to provide an intelligent diagnostic seems
like goodness. Hard to distinguish "old" bitcode from
"broken" bitcode without recording version info of some kind, and the
sooner we start recording the version number the more completely we're able
to diagnose the situation properly when the time comes.
--paulr
From: cfe-dev-bounces at cs.uiuc.edu [mailto:cfe-dev-bounces at cs.uiuc.edu] On
Behalf Of Greg Bedwell
Sent: Saturday, September 27, 2014 3:20 AM
To: Sean Silva
Cc: Yung, Douglas; cfe-dev at cs.uiuc.edu; llvmdev at cs.uiuc.edu
Subject: Re: [cfe-dev] [LLVMdev] Proposal to add Bitcode version field to
bitcode file wrapper
As I understand it, the bitcode compatibility promise doesn't extend as far
as debug info metadata (happy to be wrong here!). I think we have a usecase
where need to guarantee that debug information from any two arbitrary bitcode
files going into an LTO link will result in the expected/correct debug
information going into the resulting ELF file; unless we can be sure that this
will always work between bitcode files generated by different versions we'd
need some way of flagging up an incompatibility and providing useful information
on the reason to the user.
--Greg Bedwell
SN Systems - Sony Computer Entertainment Group
On 27 September 2014 02:24, Sean Silva <chisophugis at
gmail.com<mailto:chisophugis at gmail.com>> wrote:
On Fri, Sep 26, 2014 at 5:18 PM, Yung, Douglas <douglas_yung at
playstation.sony.com<mailto:douglas_yung at playstation.sony.com>>
wrote:
Sorry if I was unclear. There are currently no “known incompatibilities” that I
am aware of, although I fully admit to not being an expert on the topic. The
idea is that we add versioning information to the bitcode so that if an issue
were discovered, it could be easily detected and dealt with.
It sounds like time would be better invested in improving the testing of our
bitcode compatibility promise.
-- Sean Silva
Douglas Yung
From: Bob Wilson [mailto:bob.wilson at apple.com<mailto:bob.wilson at
apple.com>]
Sent: Friday, September 26, 2014 16:39
To: Yung, Douglas
Cc: llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu>; cfe-dev at
cs.uiuc.edu<mailto:cfe-dev at cs.uiuc.edu>
Subject: Re: [LLVMdev] Proposal to add Bitcode version field to bitcode file
wrapper
Bitcode backward compatibility, at least for the current major version, is
supposed to make this unnecessary. Can you provide more information about what
“known incompatibilities” you’re seeing?
On Sep 26, 2014, at 2:40 PM, Yung, Douglas <douglas_yung at
playstation.sony.com<mailto:douglas_yung at playstation.sony.com>>
wrote:
Hi,
We would like to add a version number to the bitcode wrapper. This feature would
allow easier identification of what compiler produced the bitcode so that known
incompatibilities with LTO compilation could be detected. Roughly speaking, this
version number would consist of the major, minor and optionally the patch
version of the compiler used to produce the bitcode. The version information
would be encoded in 4 bytes, with the first byte representing the major version
number, the second byte the minor version number, and the third and fourth bytes
optionally encoding the patch version or other information. As to where to place
this information, we are considering two different possibilities for updating
the bitcode wrapper specification.
The first is to simply add a single 32bit wide field at the end of the existing
bitcode wrapper format field. This would result in the new structure looking
like this:
[Magic_{32}, Version_{32}, Offset_{32}, Size_{32}, CPUType_{32},
BitcodeVersion_{32}]
All of the existing fields would keep their current meanings, and the new field
BitcodeVersion is simply appended with the format described in the first
paragraph.
A second idea was to use the existing Version field in the bitcode wrapper
format to store the bitcode version information. According to the documentation
(http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format) this field is
currently always set to 0. This would allow us to make use of what is
(presumably) an unused field.
As this is a feature that we feel would be beneficial to the community, we
wanted to get feedback on the design for our upcoming patches. Any thoughts or
opinions on this would be greatly appreciated.
Thanks!
Douglas Yung
_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu>
http://llvm.cs.uiuc.edu<http://llvm.cs.uiuc.edu/>
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu>
http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
_______________________________________________
cfe-dev mailing list
cfe-dev at cs.uiuc.edu<mailto:cfe-dev at cs.uiuc.edu>
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140928/2e9992ff/attachment.html>
Renato Golin
2014-Sep-29 08:27 UTC
[LLVMdev] [cfe-dev] Proposal to add Bitcode version field to bitcode file wrapper
On 28 September 2014 23:10, Robinson, Paul <Paul_Robinson at playstation.sony.com> wrote:> | Bitcode backward compatibility, at least for the current major version, is > supposed to make this unnecessary.It didn't use to work that way, and I'm not sure we want it at all.> I think the "at least for the current major version" part is one thing that > concerns us. LLVM 4.0 will promise to read LLVM 3.4 bitcode, but LLVM 4.1 > will not, according to my understanding of the current promise.I've never heard such promises, but even if you're right (that there is a promise), we cannot enforce it, since we have no tests to make sure we do. Right now, the only guarantee I know exists (and it's a new one, from 3.4 onwards) is that minor releases won't break ABI or API compatibility, which includes IR logic. So 3.4.2 is guaranteed to parse 3.4 IR but not 3.3 or 3.5.x.> Smoothly identifying that point and being able to provide an intelligent diagnostic > seems like goodness. Hard to distinguish "old" bitcode from "broken" bitcode > without recording version info of some kind, and the sooner we start > recording the version number the more completely we're able to diagnose the > situation properly when the time comes.There are two problems with this: 1. Due to the nature of our development strategy, IR compatibility can be broken between two releases, which means any two commits within the same revision can fail to parse (or parse incorrectly) IR from each other. Do we care about between-release compatibility? Some people get specific commits, rather than releases, for timing reasons, for their products, and in doing so, you could get a commit that is actually IR incompatible with the next major release. If you care about compatibility, you should increment the IR version every time something radical changes, which can be multiple times between the same two releases, or spawn across multiple releases. IR versioning should be completely independent of major / minor release cycles. The hard part is to truly detect, and validate, IR compatibility changes. 2. IR incompatibility is different from metadata incompatibility. If the IR is incompatible (say we drop or add a new type, or we change how exceptions are propagated), the new parser will not understand the old and vice-versa. But if metadata changes, it can still be parsed, and as David said, if we can't understand it, we just drop it. If you want your parser to break the least, you'll have to have at least two version: IR and Debug. Other metadata versioning can be done individually (since they change at different rates). You may want to warn on stale metadata status (since it's not an error), but you should stop on stale IR. Finally, both problems end up in the same place: how do you validate this? We'd have to add a new class of tests, and for every new change in IR/metadata, we'd increase the version number and create a test that checks old parser+new syntax and old syntax+new parser and makes sure they fail/warn. You'd also need to have a table of major releases vs. IR versions, so that in the error/warning message you tell: please use LLVM M.N instead. cheers, --renato