Sean Silva
2014-Oct-06 19:27 UTC
[LLVMdev] [cfe-dev] Proposal to add Bitcode version field to bitcode file wrapper
On Sun, Oct 5, 2014 at 8:10 PM, Yung, Douglas < douglas_yung at playstation.sony.com> wrote:> Hi – > > > > I realize the thread has drifted a little, but I wanted to get back to my > original proposal. I would like to make a change to the bitcode file > wrapper to include the version of llvm that produced the bitcode. I would > like to write this version into the unused version field that currently > exists. Would there be any objections to this change? >If the version field is currently unused, I don't see the harm in filling it in. It probably would be fine to just use it to store the LLVM major version, so that we can detect incompatibilities. I still would be opposed to using a granularity finer than the "intended" breakage cycle (major versions), since that would give the impression that it is "ok" to break compatibility since it can be detected through the version field. -- Sean Silva> > > Since the original wrapper is only emitted for Darwin targets, I ran an > experiment where I took bitcode files produced by the official Apple tools > and then modified the version field to be non-zero. From my simple tests, > there seemed to be no problems with existing tools when the version field > is non-zero. > > > > Douglas Yung > > > > *From:* David Blaikie [mailto:dblaikie at gmail.com] > *Sent:* Sunday, September 28, 2014 1:02 > *To:* Alex Rosenberg > *Cc:* Greg Bedwell; Yung, Douglas; cfe-dev at cs.uiuc.edu; > llvmdev at cs.uiuc.edu > *Subject:* Re: [cfe-dev] [LLVMdev] Proposal to add Bitcode version field > to bitcode file wrapper > > > > > > > > On Sat, Sep 27, 2014 at 11:35 PM, Alex Rosenberg <alexr at leftfield.org> > wrote: > > How is this use case different from the LTO-supported toolchains shipped > by other vendors such as Apple? Do they have this theoretical problem too? > > > > If the issue is solely constrained to debug info metadata, then why not > use metadata to describe the format/version of the debug info? > > > > FWIW (I haven't followed the rest of this thread) - that's what we/Apple > have done. There's a module flag metadata that specifies the debug info > metadata schema version, then the verifier (or some other pass, I forget > how it's phrased) can check if the version matches the current LLVM's debug > info metadata version, and if it doesn't match, it strips out all the debug > info metadata. > > > > > Alex > > > On Sep 27, 2014, at 3:19 AM, Greg Bedwell <gregbedwell at gmail.com> wrote: > > As I understand it, the bitcode compatibility promise doesn't extend as > far as debug info metadata (happy to be wrong here!). I think we have a > usecase where need to guarantee that debug information from any two > arbitrary bitcode files going into an LTO link will result in the > expected/correct debug information going into the resulting ELF file; > unless we can be sure that this will always work between bitcode files > generated by different versions we'd need some way of flagging up an > incompatibility and providing useful information on the reason to the user. > > > > --Greg Bedwell > > SN Systems - Sony Computer Entertainment Group > > > > On 27 September 2014 02:24, Sean Silva <chisophugis at gmail.com> wrote: > > > > > > On Fri, Sep 26, 2014 at 5:18 PM, Yung, Douglas < > douglas_yung at playstation.sony.com> wrote: > > Sorry if I was unclear. There are currently no “known incompatibilities” > that I am aware of, although I fully admit to not being an expert on the > topic. The idea is that we add versioning information to the bitcode so > that if an issue were discovered, it could be easily detected and dealt > with. > > > > It sounds like time would be better invested in improving the testing of > our bitcode compatibility promise. > > > > -- Sean Silva > > > > > > Douglas Yung > > > > *From:* Bob Wilson [mailto:bob.wilson at apple.com] > *Sent:* Friday, September 26, 2014 16:39 > *To:* Yung, Douglas > *Cc:* llvmdev at cs.uiuc.edu; cfe-dev at cs.uiuc.edu > *Subject:* Re: [LLVMdev] Proposal to add Bitcode version field to bitcode > file wrapper > > > > Bitcode backward compatibility, at least for the current major version, is > supposed to make this unnecessary. Can you provide more information about > what “known incompatibilities” you’re seeing? > > > > On Sep 26, 2014, at 2:40 PM, Yung, Douglas < > douglas_yung at playstation.sony.com> wrote: > > > > Hi, > > > > We would like to add a version number to the bitcode wrapper. This feature > would allow easier identification of what compiler produced the bitcode so > that known incompatibilities with LTO compilation could be detected. > Roughly speaking, this version number would consist of the major, minor and > optionally the patch version of the compiler used to produce the bitcode. > The version information would be encoded in 4 bytes, with the first byte > representing the major version number, the second byte the minor version > number, and the third and fourth bytes optionally encoding the patch > version or other information. As to where to place this information, we are > considering two different possibilities for updating the bitcode wrapper > specification. > > > > The first is to simply add a single 32bit wide field at the end of the > existing bitcode wrapper format field. This would result in the new > structure looking like this: > > [Magic_{32}, Version_{32}, Offset_{32}, Size_{32}, CPUType_{32}, > BitcodeVersion_{32}] > > All of the existing fields would keep their current meanings, and the new > field BitcodeVersion is simply appended with the format described in the > first paragraph. > > A second idea was to use the existing Version field in the bitcode wrapper > format to store the bitcode version information. According to the > documentation ( > http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format) this > field is currently always set to 0. This would allow us to make use of what > is (presumably) an unused field. > > > > As this is a feature that we feel would be beneficial to the community, we > wanted to get feedback on the design for our upcoming patches. Any thoughts > or opinions on this would be greatly appreciated. > > > > Thanks! > > > > Douglas Yung > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > > _______________________________________________ > cfe-dev mailing list > cfe-dev at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > _______________________________________________ > cfe-dev mailing list > cfe-dev at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > > > > _______________________________________________ > cfe-dev mailing list > cfe-dev at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20141006/3e8b530b/attachment.html>
Robinson, Paul
2014-Oct-06 21:24 UTC
[LLVMdev] [cfe-dev] Proposal to add Bitcode version field to bitcode file wrapper
One advantage to including the minor version number is that it allows the IR to
evolve in more flexible ways (semantic difference without a syntactic
difference) but auto-upgrade stays feasible. That is, it can help *avoid*
breakages.
Another advantage would be for cases like debug-info metadata, where IIRC
there's no backward compatibility at all; we just throw away old stuff. If
we had a minor version in there, it becomes more reasonable to support
one-minor-version backward compatibility. (This is a use case that we would be
interested in.)
--paulr
From: cfe-dev-bounces at cs.uiuc.edu [mailto:cfe-dev-bounces at cs.uiuc.edu] On
Behalf Of Sean Silva
Sent: Monday, October 06, 2014 12:27 PM
To: Yung, Douglas
Cc: cfe-dev at cs.uiuc.edu; llvmdev at cs.uiuc.edu
Subject: Re: [cfe-dev] [LLVMdev] Proposal to add Bitcode version field to
bitcode file wrapper
On Sun, Oct 5, 2014 at 8:10 PM, Yung, Douglas <douglas_yung at
playstation.sony.com<mailto:douglas_yung at playstation.sony.com>>
wrote:
Hi –
I realize the thread has drifted a little, but I wanted to get back to my
original proposal. I would like to make a change to the bitcode file wrapper to
include the version of llvm that produced the bitcode. I would like to write
this version into the unused version field that currently exists. Would there be
any objections to this change?
If the version field is currently unused, I don't see the harm in filling it
in. It probably would be fine to just use it to store the LLVM major version, so
that we can detect incompatibilities. I still would be opposed to using a
granularity finer than the "intended" breakage cycle (major versions),
since that would give the impression that it is "ok" to break
compatibility since it can be detected through the version field.
-- Sean Silva
Since the original wrapper is only emitted for Darwin targets, I ran an
experiment where I took bitcode files produced by the official Apple tools and
then modified the version field to be non-zero. From my simple tests, there
seemed to be no problems with existing tools when the version field is non-zero.
Douglas Yung
From: David Blaikie [mailto:dblaikie at gmail.com<mailto:dblaikie at
gmail.com>]
Sent: Sunday, September 28, 2014 1:02
To: Alex Rosenberg
Cc: Greg Bedwell; Yung, Douglas; cfe-dev at cs.uiuc.edu<mailto:cfe-dev at
cs.uiuc.edu>; llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu>
Subject: Re: [cfe-dev] [LLVMdev] Proposal to add Bitcode version field to
bitcode file wrapper
On Sat, Sep 27, 2014 at 11:35 PM, Alex Rosenberg <alexr at
leftfield.org<mailto:alexr at leftfield.org>> wrote:
How is this use case different from the LTO-supported toolchains shipped by
other vendors such as Apple? Do they have this theoretical problem too?
If the issue is solely constrained to debug info metadata, then why not use
metadata to describe the format/version of the debug info?
FWIW (I haven't followed the rest of this thread) - that's what we/Apple
have done. There's a module flag metadata that specifies the debug info
metadata schema version, then the verifier (or some other pass, I forget how
it's phrased) can check if the version matches the current LLVM's debug
info metadata version, and if it doesn't match, it strips out all the debug
info metadata.
Alex
On Sep 27, 2014, at 3:19 AM, Greg Bedwell <gregbedwell at
gmail.com<mailto:gregbedwell at gmail.com>> wrote:
As I understand it, the bitcode compatibility promise doesn't extend as far
as debug info metadata (happy to be wrong here!). I think we have a usecase
where need to guarantee that debug information from any two arbitrary bitcode
files going into an LTO link will result in the expected/correct debug
information going into the resulting ELF file; unless we can be sure that this
will always work between bitcode files generated by different versions we'd
need some way of flagging up an incompatibility and providing useful information
on the reason to the user.
--Greg Bedwell
SN Systems - Sony Computer Entertainment Group
On 27 September 2014 02:24, Sean Silva <chisophugis at
gmail.com<mailto:chisophugis at gmail.com>> wrote:
On Fri, Sep 26, 2014 at 5:18 PM, Yung, Douglas <douglas_yung at
playstation.sony.com<mailto:douglas_yung at playstation.sony.com>>
wrote:
Sorry if I was unclear. There are currently no “known incompatibilities” that I
am aware of, although I fully admit to not being an expert on the topic. The
idea is that we add versioning information to the bitcode so that if an issue
were discovered, it could be easily detected and dealt with.
It sounds like time would be better invested in improving the testing of our
bitcode compatibility promise.
-- Sean Silva
Douglas Yung
From: Bob Wilson [mailto:bob.wilson at apple.com<mailto:bob.wilson at
apple.com>]
Sent: Friday, September 26, 2014 16:39
To: Yung, Douglas
Cc: llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu>; cfe-dev at
cs.uiuc.edu<mailto:cfe-dev at cs.uiuc.edu>
Subject: Re: [LLVMdev] Proposal to add Bitcode version field to bitcode file
wrapper
Bitcode backward compatibility, at least for the current major version, is
supposed to make this unnecessary. Can you provide more information about what
“known incompatibilities” you’re seeing?
On Sep 26, 2014, at 2:40 PM, Yung, Douglas <douglas_yung at
playstation.sony.com<mailto:douglas_yung at playstation.sony.com>>
wrote:
Hi,
We would like to add a version number to the bitcode wrapper. This feature would
allow easier identification of what compiler produced the bitcode so that known
incompatibilities with LTO compilation could be detected. Roughly speaking, this
version number would consist of the major, minor and optionally the patch
version of the compiler used to produce the bitcode. The version information
would be encoded in 4 bytes, with the first byte representing the major version
number, the second byte the minor version number, and the third and fourth bytes
optionally encoding the patch version or other information. As to where to place
this information, we are considering two different possibilities for updating
the bitcode wrapper specification.
The first is to simply add a single 32bit wide field at the end of the existing
bitcode wrapper format field. This would result in the new structure looking
like this:
[Magic_{32}, Version_{32}, Offset_{32}, Size_{32}, CPUType_{32},
BitcodeVersion_{32}]
All of the existing fields would keep their current meanings, and the new field
BitcodeVersion is simply appended with the format described in the first
paragraph.
A second idea was to use the existing Version field in the bitcode wrapper
format to store the bitcode version information. According to the documentation
(http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format) this field is
currently always set to 0. This would allow us to make use of what is
(presumably) an unused field.
As this is a feature that we feel would be beneficial to the community, we
wanted to get feedback on the design for our upcoming patches. Any thoughts or
opinions on this would be greatly appreciated.
Thanks!
Douglas Yung
_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu>
http://llvm.cs.uiuc.edu<http://llvm.cs.uiuc.edu/>
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu>
http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
_______________________________________________
cfe-dev mailing list
cfe-dev at cs.uiuc.edu<mailto:cfe-dev at cs.uiuc.edu>
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu>
http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
_______________________________________________
cfe-dev mailing list
cfe-dev at cs.uiuc.edu<mailto:cfe-dev at cs.uiuc.edu>
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
cfe-dev at cs.uiuc.edu<mailto:cfe-dev at cs.uiuc.edu>
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20141006/bb76dd3e/attachment.html>
David Blaikie
2014-Oct-06 21:39 UTC
[LLVMdev] [cfe-dev] Proposal to add Bitcode version field to bitcode file wrapper
On Mon, Oct 6, 2014 at 2:24 PM, Robinson, Paul < Paul_Robinson at playstation.sony.com> wrote:> One advantage to including the minor version number is that it allows > the IR to evolve in more flexible ways (semantic difference without a > syntactic difference) but auto-upgrade stays feasible. That is, it can help > *avoid* breakages. > > Another advantage would be for cases like debug-info metadata, where IIRC > there's no backward compatibility at all; we just throw away old stuff. If > we had a minor version in there, it becomes more reasonable to support > one-minor-version backward compatibility. (This is a use case that we > would be interested in.) >FWIW we already have a debug info metadata version flag which could be used to facilitate this functionality. Kind of a painful thing to do, depending on the particular breakage. Current strategy is that old version debug info is dropped, no reason it couldn't be upgraded instead. - David> --paulr > > > > *From:* cfe-dev-bounces at cs.uiuc.edu [mailto:cfe-dev-bounces at cs.uiuc.edu] *On > Behalf Of *Sean Silva > *Sent:* Monday, October 06, 2014 12:27 PM > *To:* Yung, Douglas > *Cc:* cfe-dev at cs.uiuc.edu; llvmdev at cs.uiuc.edu > > *Subject:* Re: [cfe-dev] [LLVMdev] Proposal to add Bitcode version field > to bitcode file wrapper > > > > > > > > On Sun, Oct 5, 2014 at 8:10 PM, Yung, Douglas < > douglas_yung at playstation.sony.com> wrote: > > Hi – > > > > I realize the thread has drifted a little, but I wanted to get back to my > original proposal. I would like to make a change to the bitcode file > wrapper to include the version of llvm that produced the bitcode. I would > like to write this version into the unused version field that currently > exists. Would there be any objections to this change? > > > > If the version field is currently unused, I don't see the harm in filling > it in. It probably would be fine to just use it to store the LLVM major > version, so that we can detect incompatibilities. I still would be opposed > to using a granularity finer than the "intended" breakage cycle (major > versions), since that would give the impression that it is "ok" to break > compatibility since it can be detected through the version field. > > > > -- Sean Silva > > > > > > Since the original wrapper is only emitted for Darwin targets, I ran an > experiment where I took bitcode files produced by the official Apple tools > and then modified the version field to be non-zero. From my simple tests, > there seemed to be no problems with existing tools when the version field > is non-zero. > > > > Douglas Yung > > > > *From:* David Blaikie [mailto:dblaikie at gmail.com] > *Sent:* Sunday, September 28, 2014 1:02 > *To:* Alex Rosenberg > *Cc:* Greg Bedwell; Yung, Douglas; cfe-dev at cs.uiuc.edu; > llvmdev at cs.uiuc.edu > *Subject:* Re: [cfe-dev] [LLVMdev] Proposal to add Bitcode version field > to bitcode file wrapper > > > > > > > > On Sat, Sep 27, 2014 at 11:35 PM, Alex Rosenberg <alexr at leftfield.org> > wrote: > > How is this use case different from the LTO-supported toolchains shipped > by other vendors such as Apple? Do they have this theoretical problem too? > > > > If the issue is solely constrained to debug info metadata, then why not > use metadata to describe the format/version of the debug info? > > > > FWIW (I haven't followed the rest of this thread) - that's what we/Apple > have done. There's a module flag metadata that specifies the debug info > metadata schema version, then the verifier (or some other pass, I forget > how it's phrased) can check if the version matches the current LLVM's debug > info metadata version, and if it doesn't match, it strips out all the debug > info metadata. > > > > > Alex > > > On Sep 27, 2014, at 3:19 AM, Greg Bedwell <gregbedwell at gmail.com> wrote: > > As I understand it, the bitcode compatibility promise doesn't extend as > far as debug info metadata (happy to be wrong here!). I think we have a > usecase where need to guarantee that debug information from any two > arbitrary bitcode files going into an LTO link will result in the > expected/correct debug information going into the resulting ELF file; > unless we can be sure that this will always work between bitcode files > generated by different versions we'd need some way of flagging up an > incompatibility and providing useful information on the reason to the user. > > > > --Greg Bedwell > > SN Systems - Sony Computer Entertainment Group > > > > On 27 September 2014 02:24, Sean Silva <chisophugis at gmail.com> wrote: > > > > > > On Fri, Sep 26, 2014 at 5:18 PM, Yung, Douglas < > douglas_yung at playstation.sony.com> wrote: > > Sorry if I was unclear. There are currently no “known incompatibilities” > that I am aware of, although I fully admit to not being an expert on the > topic. The idea is that we add versioning information to the bitcode so > that if an issue were discovered, it could be easily detected and dealt > with. > > > > It sounds like time would be better invested in improving the testing of > our bitcode compatibility promise. > > > > -- Sean Silva > > > > > > Douglas Yung > > > > *From:* Bob Wilson [mailto:bob.wilson at apple.com] > *Sent:* Friday, September 26, 2014 16:39 > *To:* Yung, Douglas > *Cc:* llvmdev at cs.uiuc.edu; cfe-dev at cs.uiuc.edu > *Subject:* Re: [LLVMdev] Proposal to add Bitcode version field to bitcode > file wrapper > > > > Bitcode backward compatibility, at least for the current major version, is > supposed to make this unnecessary. Can you provide more information about > what “known incompatibilities” you’re seeing? > > > > On Sep 26, 2014, at 2:40 PM, Yung, Douglas < > douglas_yung at playstation.sony.com> wrote: > > > > Hi, > > > > We would like to add a version number to the bitcode wrapper. This feature > would allow easier identification of what compiler produced the bitcode so > that known incompatibilities with LTO compilation could be detected. > Roughly speaking, this version number would consist of the major, minor and > optionally the patch version of the compiler used to produce the bitcode. > The version information would be encoded in 4 bytes, with the first byte > representing the major version number, the second byte the minor version > number, and the third and fourth bytes optionally encoding the patch > version or other information. As to where to place this information, we are > considering two different possibilities for updating the bitcode wrapper > specification. > > > > The first is to simply add a single 32bit wide field at the end of the > existing bitcode wrapper format field. This would result in the new > structure looking like this: > > [Magic_{32}, Version_{32}, Offset_{32}, Size_{32}, CPUType_{32}, > BitcodeVersion_{32}] > > All of the existing fields would keep their current meanings, and the new > field BitcodeVersion is simply appended with the format described in the > first paragraph. > > A second idea was to use the existing Version field in the bitcode wrapper > format to store the bitcode version information. According to the > documentation ( > http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format) this > field is currently always set to 0. This would allow us to make use of what > is (presumably) an unused field. > > > > As this is a feature that we feel would be beneficial to the community, we > wanted to get feedback on the design for our upcoming patches. Any thoughts > or opinions on this would be greatly appreciated. > > > > Thanks! > > > > Douglas Yung > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > > _______________________________________________ > cfe-dev mailing list > cfe-dev at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > _______________________________________________ > cfe-dev mailing list > cfe-dev at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > > > > > _______________________________________________ > cfe-dev mailing list > cfe-dev at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20141006/1ecc6c35/attachment.html>
Eric Christopher
2014-Oct-06 21:39 UTC
[LLVMdev] [cfe-dev] Proposal to add Bitcode version field to bitcode file wrapper
On Mon, Oct 6, 2014 at 2:24 PM, Robinson, Paul <Paul_Robinson at playstation.sony.com> wrote:> One advantage to including the minor version number is that it allows the IR > to evolve in more flexible ways (semantic difference without a syntactic > difference) but auto-upgrade stays feasible. That is, it can help *avoid* > breakages. > > Another advantage would be for cases like debug-info metadata, where IIRC > there's no backward compatibility at all; we just throw away old stuff. If > we had a minor version in there, it becomes more reasonable to support > one-minor-version backward compatibility. (This is a use case that we would > be interested in.) >This just isn't something we're willing to support now. -eric> --paulr > > > > From: cfe-dev-bounces at cs.uiuc.edu [mailto:cfe-dev-bounces at cs.uiuc.edu] On > Behalf Of Sean Silva > Sent: Monday, October 06, 2014 12:27 PM > To: Yung, Douglas > Cc: cfe-dev at cs.uiuc.edu; llvmdev at cs.uiuc.edu > > > Subject: Re: [cfe-dev] [LLVMdev] Proposal to add Bitcode version field to > bitcode file wrapper > > > > > > > > On Sun, Oct 5, 2014 at 8:10 PM, Yung, Douglas > <douglas_yung at playstation.sony.com> wrote: > > Hi – > > > > I realize the thread has drifted a little, but I wanted to get back to my > original proposal. I would like to make a change to the bitcode file wrapper > to include the version of llvm that produced the bitcode. I would like to > write this version into the unused version field that currently exists. > Would there be any objections to this change? > > > > If the version field is currently unused, I don't see the harm in filling it > in. It probably would be fine to just use it to store the LLVM major > version, so that we can detect incompatibilities. I still would be opposed > to using a granularity finer than the "intended" breakage cycle (major > versions), since that would give the impression that it is "ok" to break > compatibility since it can be detected through the version field. > > > > -- Sean Silva > > > > > > Since the original wrapper is only emitted for Darwin targets, I ran an > experiment where I took bitcode files produced by the official Apple tools > and then modified the version field to be non-zero. From my simple tests, > there seemed to be no problems with existing tools when the version field is > non-zero. > > > > Douglas Yung > > > > From: David Blaikie [mailto:dblaikie at gmail.com] > Sent: Sunday, September 28, 2014 1:02 > To: Alex Rosenberg > Cc: Greg Bedwell; Yung, Douglas; cfe-dev at cs.uiuc.edu; llvmdev at cs.uiuc.edu > Subject: Re: [cfe-dev] [LLVMdev] Proposal to add Bitcode version field to > bitcode file wrapper > > > > > > > > On Sat, Sep 27, 2014 at 11:35 PM, Alex Rosenberg <alexr at leftfield.org> > wrote: > > How is this use case different from the LTO-supported toolchains shipped by > other vendors such as Apple? Do they have this theoretical problem too? > > > > If the issue is solely constrained to debug info metadata, then why not use > metadata to describe the format/version of the debug info? > > > > FWIW (I haven't followed the rest of this thread) - that's what we/Apple > have done. There's a module flag metadata that specifies the debug info > metadata schema version, then the verifier (or some other pass, I forget how > it's phrased) can check if the version matches the current LLVM's debug info > metadata version, and if it doesn't match, it strips out all the debug info > metadata. > > > > > Alex > > > On Sep 27, 2014, at 3:19 AM, Greg Bedwell <gregbedwell at gmail.com> wrote: > > As I understand it, the bitcode compatibility promise doesn't extend as far > as debug info metadata (happy to be wrong here!). I think we have a usecase > where need to guarantee that debug information from any two arbitrary > bitcode files going into an LTO link will result in the expected/correct > debug information going into the resulting ELF file; unless we can be sure > that this will always work between bitcode files generated by different > versions we'd need some way of flagging up an incompatibility and providing > useful information on the reason to the user. > > > > --Greg Bedwell > > SN Systems - Sony Computer Entertainment Group > > > > On 27 September 2014 02:24, Sean Silva <chisophugis at gmail.com> wrote: > > > > > > On Fri, Sep 26, 2014 at 5:18 PM, Yung, Douglas > <douglas_yung at playstation.sony.com> wrote: > > Sorry if I was unclear. There are currently no “known incompatibilities” > that I am aware of, although I fully admit to not being an expert on the > topic. The idea is that we add versioning information to the bitcode so that > if an issue were discovered, it could be easily detected and dealt with. > > > > It sounds like time would be better invested in improving the testing of our > bitcode compatibility promise. > > > > -- Sean Silva > > > > > > Douglas Yung > > > > From: Bob Wilson [mailto:bob.wilson at apple.com] > Sent: Friday, September 26, 2014 16:39 > To: Yung, Douglas > Cc: llvmdev at cs.uiuc.edu; cfe-dev at cs.uiuc.edu > Subject: Re: [LLVMdev] Proposal to add Bitcode version field to bitcode file > wrapper > > > > Bitcode backward compatibility, at least for the current major version, is > supposed to make this unnecessary. Can you provide more information about > what “known incompatibilities” you’re seeing? > > > > On Sep 26, 2014, at 2:40 PM, Yung, Douglas > <douglas_yung at playstation.sony.com> wrote: > > > > Hi, > > > > We would like to add a version number to the bitcode wrapper. This feature > would allow easier identification of what compiler produced the bitcode so > that known incompatibilities with LTO compilation could be detected. Roughly > speaking, this version number would consist of the major, minor and > optionally the patch version of the compiler used to produce the bitcode. > The version information would be encoded in 4 bytes, with the first byte > representing the major version number, the second byte the minor version > number, and the third and fourth bytes optionally encoding the patch version > or other information. As to where to place this information, we are > considering two different possibilities for updating the bitcode wrapper > specification. > > > > The first is to simply add a single 32bit wide field at the end of the > existing bitcode wrapper format field. This would result in the new > structure looking like this: > > [Magic_{32}, Version_{32}, Offset_{32}, Size_{32}, CPUType_{32}, > BitcodeVersion_{32}] > > All of the existing fields would keep their current meanings, and the new > field BitcodeVersion is simply appended with the format described in the > first paragraph. > > A second idea was to use the existing Version field in the bitcode wrapper > format to store the bitcode version information. According to the > documentation > (http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format) this field > is currently always set to 0. This would allow us to make use of what is > (presumably) an unused field. > > > > As this is a feature that we feel would be beneficial to the community, we > wanted to get feedback on the design for our upcoming patches. Any thoughts > or opinions on this would be greatly appreciated. > > > > Thanks! > > > > Douglas Yung > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > > _______________________________________________ > cfe-dev mailing list > cfe-dev at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > _______________________________________________ > cfe-dev mailing list > cfe-dev at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > > > > > _______________________________________________ > cfe-dev mailing list > cfe-dev at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >