llvm dev - Sep 2014 - [LLVMdev] [cfe-dev] Proposal to add Bitcode version field to bitcode file wrapper

Sorry if I was unclear. There are currently no “known incompatibilities” that I
am aware of, although I fully admit to not being an expert on the topic. The
idea is that we add versioning information to the bitcode so that if an issue
were discovered, it could be easily detected and dealt with.

Douglas Yung

From: Bob Wilson [mailto:bob.wilson at apple.com]
Sent: Friday, September 26, 2014 16:39
To: Yung, Douglas
Cc: llvmdev at cs.uiuc.edu; cfe-dev at cs.uiuc.edu
Subject: Re: [LLVMdev] Proposal to add Bitcode version field to bitcode file
wrapper

Bitcode backward compatibility, at least for the current major version, is
supposed to make this unnecessary. Can you provide more information about what
“known incompatibilities” you’re seeing?

On Sep 26, 2014, at 2:40 PM, Yung, Douglas <douglas_yung at
playstation.sony.com<mailto:douglas_yung at playstation.sony.com>>
wrote:

Hi,

We would like to add a version number to the bitcode wrapper. This feature would
allow easier identification of what compiler produced the bitcode so that known
incompatibilities with LTO compilation could be detected. Roughly speaking, this
version number would consist of the major, minor and optionally the patch
version of the compiler used to produce the bitcode. The version information
would be encoded in 4 bytes, with the first byte representing the major version
number, the second byte the minor version number, and the third and fourth bytes
optionally encoding the patch version or other information. As to where to place
this information, we are considering two different possibilities for updating
the bitcode wrapper specification.

The first is to simply add a single 32bit wide field at the end of the existing
bitcode wrapper format field. This would result in the new structure looking
like this:

[Magic_{32}, Version_{32}, Offset_{32}, Size_{32}, CPUType_{32},
BitcodeVersion_{32}]

All of the existing fields would keep their current meanings, and the new field
BitcodeVersion is simply appended with the format described in the first
paragraph.

A second idea was to use the existing Version field in the bitcode wrapper
format to store the bitcode version information. According to the documentation
(http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format) this field is
currently always set to 0. This would allow us to make use of what is
(presumably) an unused field.

As this is a feature that we feel would be beneficial to the community, we
wanted to get feedback on the design for our upcoming patches. Any thoughts or
opinions on this would be greatly appreciated.

Thanks!

Douglas Yung
_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu>        
http://llvm.cs.uiuc.edu<http://llvm.cs.uiuc.edu/>
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140927/8f64f1d0/attachment.html>

If we discover an issue, shouldn't we just fix it?

-- dpnes
> On Sep 26, 2014, at 5:18 PM, Yung, Douglas <douglas_yung at
playstation.sony.com> wrote:
> 
> Sorry if I was unclear. There are currently no “known incompatibilities”
that I am aware of, although I fully admit to not being an expert on the topic.
The idea is that we add versioning information to the bitcode so that if an
issue were discovered, it could be easily detected and dealt with.
>  
> Douglas Yung
>  
> From: Bob Wilson [mailto:bob.wilson at apple.com] 
> Sent: Friday, September 26, 2014 16:39
> To: Yung, Douglas
> Cc: llvmdev at cs.uiuc.edu; cfe-dev at cs.uiuc.edu
> Subject: Re: [LLVMdev] Proposal to add Bitcode version field to bitcode
file wrapper
>  
> Bitcode backward compatibility, at least for the current major version, is
supposed to make this unnecessary. Can you provide more information about what
“known incompatibilities” you’re seeing?
>  
> On Sep 26, 2014, at 2:40 PM, Yung, Douglas <douglas_yung at
playstation.sony.com> wrote:
>  
> Hi,
>  
> We would like to add a version number to the bitcode wrapper. This feature
would allow easier identification of what compiler produced the bitcode so that
known incompatibilities with LTO compilation could be detected. Roughly
speaking, this version number would consist of the major, minor and optionally
the patch version of the compiler used to produce the bitcode. The version
information would be encoded in 4 bytes, with the first byte representing the
major version number, the second byte the minor version number, and the third
and fourth bytes optionally encoding the patch version or other information. As
to where to place this information, we are considering two different
possibilities for updating the bitcode wrapper specification.
>  
> The first is to simply add a single 32bit wide field at the end of the
existing bitcode wrapper format field. This would result in the new structure
looking like this:
>   
> [Magic_{32}, Version_{32}, Offset_{32}, Size_{32}, CPUType_{32},
BitcodeVersion_{32}]
>   
> All of the existing fields would keep their current meanings, and the new
field BitcodeVersion is simply appended with the format described in the first
paragraph.
>   
> A second idea was to use the existing Version field in the bitcode wrapper
format to store the bitcode version information. According to the documentation
(http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format) this field is
currently always set to 0. This would allow us to make use of what is
(presumably) an unused field.
>  
> As this is a feature that we feel would be beneficial to the community, we
wanted to get feedback on the design for our upcoming patches. Any thoughts or
opinions on this would be greatly appreciated.
>  
> Thanks!
>  
> Douglas Yung
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>  
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140926/4c9f8617/attachment.html>

On Fri, Sep 26, 2014 at 5:18 PM, Yung, Douglas <
douglas_yung at playstation.sony.com> wrote:
>  Sorry if I was unclear. There are currently no “known incompatibilities”
> that I am aware of, although I fully admit to not being an expert on the
> topic. The idea is that we add versioning information to the bitcode so
> that if an issue were discovered, it could be easily detected and dealt
> with.
>
It sounds like time would be better invested in improving the testing of
our bitcode compatibility promise.

-- Sean Silva

>
>
> Douglas Yung
>
>
>
> *From:* Bob Wilson [mailto:bob.wilson at apple.com]
> *Sent:* Friday, September 26, 2014 16:39
> *To:* Yung, Douglas
> *Cc:* llvmdev at cs.uiuc.edu; cfe-dev at cs.uiuc.edu
> *Subject:* Re: [LLVMdev] Proposal to add Bitcode version field to bitcode
> file wrapper
>
>
>
> Bitcode backward compatibility, at least for the current major version, is
> supposed to make this unnecessary. Can you provide more information about
> what “known incompatibilities” you’re seeing?
>
>
>
>  On Sep 26, 2014, at 2:40 PM, Yung, Douglas <
> douglas_yung at playstation.sony.com> wrote:
>
>
>
> Hi,
>
>
>
> We would like to add a version number to the bitcode wrapper. This feature
> would allow easier identification of what compiler produced the bitcode so
> that known incompatibilities with LTO compilation could be detected.
> Roughly speaking, this version number would consist of the major, minor and
> optionally the patch version of the compiler used to produce the bitcode.
> The version information would be encoded in 4 bytes, with the first byte
> representing the major version number, the second byte the minor version
> number, and the third and fourth bytes optionally encoding the patch
> version or other information. As to where to place this information, we are
> considering two different possibilities for updating the bitcode wrapper
> specification.
>
>
>
> The first is to simply add a single 32bit wide field at the end of the
> existing bitcode wrapper format field. This would result in the new
> structure looking like this:
>
> [Magic_{32}, Version_{32}, Offset_{32}, Size_{32}, CPUType_{32},
> BitcodeVersion_{32}]
>
> All of the existing fields would keep their current meanings, and the new
> field BitcodeVersion is simply appended with the format described in the
> first paragraph.
>
> A second idea was to use the existing Version field in the bitcode wrapper
> format to store the bitcode version information. According to the
> documentation (
> http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format) this
> field is currently always set to 0. This would allow us to make use of what
> is (presumably) an unused field.
>
>
>
> As this is a feature that we feel would be beneficial to the community, we
> wanted to get feedback on the design for our upcoming patches. Any thoughts
> or opinions on this would be greatly appreciated.
>
>
>
> Thanks!
>
>
>
> Douglas Yung
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140926/3f88835d/attachment.html>

As I understand it, the bitcode compatibility promise doesn't extend as far
as debug info metadata (happy to be wrong here!).  I think we have a
usecase where need to guarantee that debug information from any two
arbitrary bitcode files going into an LTO link will result in the
expected/correct debug information going into the resulting ELF file;
unless we can be sure that this will always work between bitcode files
generated by different versions we'd need some way of flagging up an
incompatibility and providing useful information on the reason to the user.

--Greg Bedwell
SN Systems - Sony Computer Entertainment Group

On 27 September 2014 02:24, Sean Silva <chisophugis at gmail.com> wrote:
>
>
> On Fri, Sep 26, 2014 at 5:18 PM, Yung, Douglas <
> douglas_yung at playstation.sony.com> wrote:
>
>>  Sorry if I was unclear. There are currently no “known
>> incompatibilities” that I am aware of, although I fully admit to not
being
>> an expert on the topic. The idea is that we add versioning information
to
>> the bitcode so that if an issue were discovered, it could be easily
>> detected and dealt with.
>>
>
> It sounds like time would be better invested in improving the testing of
> our bitcode compatibility promise.
>
> -- Sean Silva
>
>
>>
>>
>> Douglas Yung
>>
>>
>>
>> *From:* Bob Wilson [mailto:bob.wilson at apple.com]
>> *Sent:* Friday, September 26, 2014 16:39
>> *To:* Yung, Douglas
>> *Cc:* llvmdev at cs.uiuc.edu; cfe-dev at cs.uiuc.edu
>> *Subject:* Re: [LLVMdev] Proposal to add Bitcode version field to
>> bitcode file wrapper
>>
>>
>>
>> Bitcode backward compatibility, at least for the current major version,
>> is supposed to make this unnecessary. Can you provide more information
>> about what “known incompatibilities” you’re seeing?
>>
>>
>>
>>  On Sep 26, 2014, at 2:40 PM, Yung, Douglas <
>> douglas_yung at playstation.sony.com> wrote:
>>
>>
>>
>> Hi,
>>
>>
>>
>> We would like to add a version number to the bitcode wrapper. This
>> feature would allow easier identification of what compiler produced the
>> bitcode so that known incompatibilities with LTO compilation could be
>> detected. Roughly speaking, this version number would consist of the
major,
>> minor and optionally the patch version of the compiler used to produce
the
>> bitcode. The version information would be encoded in 4 bytes, with the
>> first byte representing the major version number, the second byte the
minor
>> version number, and the third and fourth bytes optionally encoding the
>> patch version or other information. As to where to place this
information,
>> we are considering two different possibilities for updating the bitcode
>> wrapper specification.
>>
>>
>>
>> The first is to simply add a single 32bit wide field at the end of the
>> existing bitcode wrapper format field. This would result in the new
>> structure looking like this:
>>
>> [Magic_{32}, Version_{32}, Offset_{32}, Size_{32}, CPUType_{32},
>> BitcodeVersion_{32}]
>>
>> All of the existing fields would keep their current meanings, and the
new
>> field BitcodeVersion is simply appended with the format described in
the
>> first paragraph.
>>
>> A second idea was to use the existing Version field in the bitcode
>> wrapper format to store the bitcode version information. According to
the
>> documentation (
>> http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format) this
>> field is currently always set to 0. This would allow us to make use of
what
>> is (presumably) an unused field.
>>
>>
>>
>> As this is a feature that we feel would be beneficial to the community,
>> we wanted to get feedback on the design for our upcoming patches. Any
>> thoughts or opinions on this would be greatly appreciated.
>>
>>
>>
>> Thanks!
>>
>>
>>
>> Douglas Yung
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140927/d30ffaa8/attachment.html>