Philip Reames via llvm-dev
2015-Aug-12 18:50 UTC
[llvm-dev] [LLVMdev] RFC: ThinLTO File Format
Alex already made what I consider to be the most relevant point. I would suggest removing the unwanted functionality and asking again. From my perspective, native wrapped bitcode is only interesting (and thus worth reviewing and/or talking about) once the native bitcode version is in tree and functional. Frankly, I consider the native wrapped bitcode to be an entirely orthogonal proposal that shouldn't be tied to ThinLTO at all. Fair warning, I'm not going to be particularly involved either way. This is far enough from my own immediate interests that I can't spare the cycles. I would suggest that you collaborate closely with the Sony and Apple folks who are already *using* LTO to find a proposal they're happy with. Until you do that, you are unlikely to make much progress. Philip On 08/12/2015 09:13 AM, Teresa Johnson wrote:> Ping. Explicitly adding a few more people who commented on the earlier > (high-level) ThinLTO RFC. I removed the body of the RFC here since the > original was large and had trouble getting through the mailer. I also > updated the patch mentioned below so that it was emailed to > llvm-commits properly. > > Thanks, > Teresa > > On Mon, Aug 3, 2015 at 11:59 AM, Teresa Johnson <tejohnson at google.com > <mailto:tejohnson at google.com>> wrote: > > Hi Alex, > > After outlining some of the rationale for using native-wrapped, > there were a couple of responses that indicated native-wrapped > support was reasonable, but they preferred to see bitcode-only > first (Phillip and Rafael). This is essentially what this proposal > and the patches do - I've implemented some of the basic support > for looking for and parsing the native-wrapped sections, but the > bitcode-only reading/writing support is more complete. > > In fact, as described in this RFC, I designed the native-wrapped > format to utilize the same bitcode encoding for most of the > ThinLTO information, so it uses most of the same underlying > bitcode interfaces anyway. The additional support required for > native-wrapped is not tremendous, and is similar to existing > support in the LLVM tree for reading native-wrapped bitcode. > > We believe that there will be clang/llvm users who will find > native-wrapped ThinLTO easier to use for the same reasons (e.g. > compatibility with existing native toolchains), so I don't expect > this to be Google specific. > > Thanks, > Teresa > > On Mon, Aug 3, 2015 at 12:26 PM, Alex Rosenberg > <alexr at leftfield.org <mailto:alexr at leftfield.org>> wrote: > > I think I've read all the feedback posted regarding your May > proposal. I have yet to see a single response that wants > native object wrapped bitcode. > > If the only use for native object wrapped bitcode is for your > project at Google, then it probably shouldn't go into the tree > against all of these objections. > > Alex > > On Aug 3, 2015, at 9:19 AM, Teresa Johnson > <tejohnson at google.com <mailto:tejohnson at google.com>> wrote: > >> As discussed in the high-level ThinLTO RFC >> (http://lists.cs.uiuc.edu/pipermail/llvmdev/2015-May/086211.html), >> we would like to add support for native object wrapped >> bitcode and ThinLTO information. Based on comments on the >> mailing list, I am adding support for ThinLTO in both normal >> bitcode files, as well as native-object wrapped bitcode. >> >> The following RFC describes the planned file format of >> ThinLTO information both in the bitcode-only and native >> object wrapped cases. It doesn't yet define the exact record >> format, as I would like feedback on the overall block design >> first. >> >> I've also implemented the support for reading and writing the >> bitcode blocks in the following patch: >> http://reviews.llvm.org/D11722 >> <https://urldefense.proofpoint.com/v2/url?u=http-3A__reviews.llvm.org_D11722&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Mfk2qtn1LTDThVkh6-oGglNfMADXfJdty4_bhmuhMHA&m=oUy_PB_mSfRgDO7H7bZOR04gv_DMzX5rPO_lv4PHt60&s=WVxrKkHnjKr75fCQ-UkGke8dk6KpZcFCnLWVrJ3G188&e=> >> >> The ThinLTO data structures and the file APIs are described >> in a separate RFC I will be sending simultaneously, with >> pointers to the patches implementing them. >> >> Looking forward to your feedback. Thanks! >> Teresa >> >> > > > -- > Teresa Johnson | Software Engineer | tejohnson at google.com > <mailto:tejohnson at google.com> | 408-460-2413 >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150812/d0e89b04/attachment.html>
Xinliang David Li via llvm-dev
2015-Aug-12 19:18 UTC
[llvm-dev] [LLVMdev] RFC: ThinLTO File Format
On Wed, Aug 12, 2015 at 11:50 AM, Philip Reames via llvm-dev < llvm-dev at lists.llvm.org> wrote:> Alex already made what I consider to be the most relevant point. I would > suggest removing the unwanted functionality and asking again. >I find your definition of 'unwanted' too narrow -- there are certainly users who may want this. It would be a more productive discussion if the comments can be made on the details of the RFCs themselves. David>From my perspective, native wrapped bitcode is only interesting (and thus > worth reviewing and/or talking about) once the native bitcode version is in > tree and functional. Frankly, I consider the native wrapped bitcode to be > an entirely orthogonal proposal that shouldn't be tied to ThinLTO at all. > > Fair warning, I'm not going to be particularly involved either way. This > is far enough from my own immediate interests that I can't spare the > cycles. I would suggest that you collaborate closely with the Sony and > Apple folks who are already *using* LTO to find a proposal they're happy > with. Until you do that, you are unlikely to make much progress. > > Philip > > > On 08/12/2015 09:13 AM, Teresa Johnson wrote: > > Ping. Explicitly adding a few more people who commented on the earlier > (high-level) ThinLTO RFC. I removed the body of the RFC here since the > original was large and had trouble getting through the mailer. I also > updated the patch mentioned below so that it was emailed to llvm-commits > properly. > > Thanks, > Teresa > > On Mon, Aug 3, 2015 at 11:59 AM, Teresa Johnson <tejohnson at google.com> > wrote: > >> Hi Alex, >> >> After outlining some of the rationale for using native-wrapped, there >> were a couple of responses that indicated native-wrapped support was >> reasonable, but they preferred to see bitcode-only first (Phillip and >> Rafael). This is essentially what this proposal and the patches do - I've >> implemented some of the basic support for looking for and parsing the >> native-wrapped sections, but the bitcode-only reading/writing support is >> more complete. >> >> In fact, as described in this RFC, I designed the native-wrapped format >> to utilize the same bitcode encoding for most of the ThinLTO information, >> so it uses most of the same underlying bitcode interfaces anyway. The >> additional support required for native-wrapped is not tremendous, and is >> similar to existing support in the LLVM tree for reading native-wrapped >> bitcode. >> >> We believe that there will be clang/llvm users who will find >> native-wrapped ThinLTO easier to use for the same reasons (e.g. >> compatibility with existing native toolchains), so I don't expect this to >> be Google specific. >> >> Thanks, >> Teresa >> >> On Mon, Aug 3, 2015 at 12:26 PM, Alex Rosenberg <alexr at leftfield.org> >> wrote: >> >>> I think I've read all the feedback posted regarding your May proposal. I >>> have yet to see a single response that wants native object wrapped bitcode. >>> >>> If the only use for native object wrapped bitcode is for your project at >>> Google, then it probably shouldn't go into the tree against all of these >>> objections. >>> >>> Alex >>> >>> On Aug 3, 2015, at 9:19 AM, Teresa Johnson <tejohnson at google.com> wrote: >>> >>> As discussed in the high-level ThinLTO RFC ( >>> http://lists.cs.uiuc.edu/pipermail/llvmdev/2015-May/086211.html), we >>> would like to add support for native object wrapped bitcode and ThinLTO >>> information. Based on comments on the mailing list, I am adding support for >>> ThinLTO in both normal bitcode files, as well as native-object wrapped >>> bitcode. >>> >>> The following RFC describes the planned file format of ThinLTO >>> information both in the bitcode-only and native object wrapped cases. It >>> doesn't yet define the exact record format, as I would like feedback on the >>> overall block design first. >>> >>> I've also implemented the support for reading and writing the bitcode >>> blocks in the following patch: >>> http://reviews.llvm.org/D11722 >>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__reviews.llvm.org_D11722&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Mfk2qtn1LTDThVkh6-oGglNfMADXfJdty4_bhmuhMHA&m=oUy_PB_mSfRgDO7H7bZOR04gv_DMzX5rPO_lv4PHt60&s=WVxrKkHnjKr75fCQ-UkGke8dk6KpZcFCnLWVrJ3G188&e=> >>> >>> The ThinLTO data structures and the file APIs are described in a >>> separate RFC I will be sending simultaneously, with pointers to the patches >>> implementing them. >>> >>> Looking forward to your feedback. Thanks! >>> Teresa >>> >>> >>> > > -- > Teresa Johnson | Software Engineer | tejohnson at google.com | > 408-460-2413 > > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org http://llvm.cs.uiuc.edu > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150812/2e3f4e25/attachment.html>
Teresa Johnson via llvm-dev
2015-Aug-12 20:24 UTC
[llvm-dev] [LLVMdev] RFC: ThinLTO File Format
I can remove the native wrapper portions of the associated patch and add that later. Most of the support as I mentioned is for the bitcode handling anyway, but I wanted to include a skeleton of the native wrapper part. For the RFC, I wanted to show the end goal including how the native wrapper support would look (it in fact mostly leverages the same bitcode encoding, so there isn't a lot of difference, and hence there isn't a whole lot of extra code needed to support that). The bulk of the RFC deals with the bitcode format, and I would love some feedback on that. Thanks, Teresa On Wed, Aug 12, 2015 at 11:50 AM, Philip Reames <listmail at philipreames.com> wrote:> Alex already made what I consider to be the most relevant point. I would > suggest removing the unwanted functionality and asking again. From my > perspective, native wrapped bitcode is only interesting (and thus worth > reviewing and/or talking about) once the native bitcode version is in tree > and functional. Frankly, I consider the native wrapped bitcode to be an > entirely orthogonal proposal that shouldn't be tied to ThinLTO at all. > > Fair warning, I'm not going to be particularly involved either way. This > is far enough from my own immediate interests that I can't spare the > cycles. I would suggest that you collaborate closely with the Sony and > Apple folks who are already *using* LTO to find a proposal they're happy > with. Until you do that, you are unlikely to make much progress. > > Philip > > > On 08/12/2015 09:13 AM, Teresa Johnson wrote: > > Ping. Explicitly adding a few more people who commented on the earlier > (high-level) ThinLTO RFC. I removed the body of the RFC here since the > original was large and had trouble getting through the mailer. I also > updated the patch mentioned below so that it was emailed to llvm-commits > properly. > > Thanks, > Teresa > > On Mon, Aug 3, 2015 at 11:59 AM, Teresa Johnson <tejohnson at google.com> > wrote: > >> Hi Alex, >> >> After outlining some of the rationale for using native-wrapped, there >> were a couple of responses that indicated native-wrapped support was >> reasonable, but they preferred to see bitcode-only first (Phillip and >> Rafael). This is essentially what this proposal and the patches do - I've >> implemented some of the basic support for looking for and parsing the >> native-wrapped sections, but the bitcode-only reading/writing support is >> more complete. >> >> In fact, as described in this RFC, I designed the native-wrapped format >> to utilize the same bitcode encoding for most of the ThinLTO information, >> so it uses most of the same underlying bitcode interfaces anyway. The >> additional support required for native-wrapped is not tremendous, and is >> similar to existing support in the LLVM tree for reading native-wrapped >> bitcode. >> >> We believe that there will be clang/llvm users who will find >> native-wrapped ThinLTO easier to use for the same reasons (e.g. >> compatibility with existing native toolchains), so I don't expect this to >> be Google specific. >> >> Thanks, >> Teresa >> >> On Mon, Aug 3, 2015 at 12:26 PM, Alex Rosenberg <alexr at leftfield.org> >> wrote: >> >>> I think I've read all the feedback posted regarding your May proposal. I >>> have yet to see a single response that wants native object wrapped bitcode. >>> >>> If the only use for native object wrapped bitcode is for your project at >>> Google, then it probably shouldn't go into the tree against all of these >>> objections. >>> >>> Alex >>> >>> On Aug 3, 2015, at 9:19 AM, Teresa Johnson <tejohnson at google.com> wrote: >>> >>> As discussed in the high-level ThinLTO RFC ( >>> http://lists.cs.uiuc.edu/pipermail/llvmdev/2015-May/086211.html), we >>> would like to add support for native object wrapped bitcode and ThinLTO >>> information. Based on comments on the mailing list, I am adding support for >>> ThinLTO in both normal bitcode files, as well as native-object wrapped >>> bitcode. >>> >>> The following RFC describes the planned file format of ThinLTO >>> information both in the bitcode-only and native object wrapped cases. It >>> doesn't yet define the exact record format, as I would like feedback on the >>> overall block design first. >>> >>> I've also implemented the support for reading and writing the bitcode >>> blocks in the following patch: >>> http://reviews.llvm.org/D11722 >>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__reviews.llvm.org_D11722&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Mfk2qtn1LTDThVkh6-oGglNfMADXfJdty4_bhmuhMHA&m=oUy_PB_mSfRgDO7H7bZOR04gv_DMzX5rPO_lv4PHt60&s=WVxrKkHnjKr75fCQ-UkGke8dk6KpZcFCnLWVrJ3G188&e=> >>> >>> The ThinLTO data structures and the file APIs are described in a >>> separate RFC I will be sending simultaneously, with pointers to the patches >>> implementing them. >>> >>> Looking forward to your feedback. Thanks! >>> Teresa >>> >>> >>> > > -- > Teresa Johnson | Software Engineer | tejohnson at google.com | > 408-460-2413 > > >-- Teresa Johnson | Software Engineer | tejohnson at google.com | 408-460-2413 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150812/2a6ce6c1/attachment.html>
Philip Reames via llvm-dev
2015-Aug-12 21:07 UTC
[llvm-dev] [LLVMdev] RFC: ThinLTO File Format
I went ahead and replied to two of the review threads. I only considered the parts which would be left without the native wrapped bitcode support. Philip On 08/12/2015 01:24 PM, Teresa Johnson wrote:> I can remove the native wrapper portions of the associated patch and > add that later. Most of the support as I mentioned is for the bitcode > handling anyway, but I wanted to include a skeleton of the native > wrapper part. For the RFC, I wanted to show the end goal including how > the native wrapper support would look (it in fact mostly leverages the > same bitcode encoding, so there isn't a lot of difference, and hence > there isn't a whole lot of extra code needed to support that). The > bulk of the RFC deals with the bitcode format, and I would love some > feedback on that. > > Thanks, > Teresa > > On Wed, Aug 12, 2015 at 11:50 AM, Philip Reames > <listmail at philipreames.com <mailto:listmail at philipreames.com>> wrote: > > Alex already made what I consider to be the most relevant point. > I would suggest removing the unwanted functionality and asking > again. From my perspective, native wrapped bitcode is only > interesting (and thus worth reviewing and/or talking about) once > the native bitcode version is in tree and functional. Frankly, I > consider the native wrapped bitcode to be an entirely orthogonal > proposal that shouldn't be tied to ThinLTO at all. > > Fair warning, I'm not going to be particularly involved either > way. This is far enough from my own immediate interests that I > can't spare the cycles. I would suggest that you collaborate > closely with the Sony and Apple folks who are already *using* LTO > to find a proposal they're happy with. Until you do that, you are > unlikely to make much progress. > > Philip > > > On 08/12/2015 09:13 AM, Teresa Johnson wrote: >> Ping. Explicitly adding a few more people who commented on the >> earlier (high-level) ThinLTO RFC. I removed the body of the RFC >> here since the original was large and had trouble getting through >> the mailer. I also updated the patch mentioned below so that it >> was emailed to llvm-commits properly. >> >> Thanks, >> Teresa >> >> On Mon, Aug 3, 2015 at 11:59 AM, Teresa Johnson >> <tejohnson at google.com <mailto:tejohnson at google.com>> wrote: >> >> Hi Alex, >> >> After outlining some of the rationale for using >> native-wrapped, there were a couple of responses that >> indicated native-wrapped support was reasonable, but they >> preferred to see bitcode-only first (Phillip and Rafael). >> This is essentially what this proposal and the patches do - >> I've implemented some of the basic support for looking for >> and parsing the native-wrapped sections, but the bitcode-only >> reading/writing support is more complete. >> >> In fact, as described in this RFC, I designed the >> native-wrapped format to utilize the same bitcode encoding >> for most of the ThinLTO information, so it uses most of the >> same underlying bitcode interfaces anyway. The additional >> support required for native-wrapped is not tremendous, and is >> similar to existing support in the LLVM tree for reading >> native-wrapped bitcode. >> >> We believe that there will be clang/llvm users who will find >> native-wrapped ThinLTO easier to use for the same reasons >> (e.g. compatibility with existing native toolchains), so I >> don't expect this to be Google specific. >> >> Thanks, >> Teresa >> >> On Mon, Aug 3, 2015 at 12:26 PM, Alex Rosenberg >> <alexr at leftfield.org <mailto:alexr at leftfield.org>> wrote: >> >> I think I've read all the feedback posted regarding your >> May proposal. I have yet to see a single response that >> wants native object wrapped bitcode. >> >> If the only use for native object wrapped bitcode is for >> your project at Google, then it probably shouldn't go >> into the tree against all of these objections. >> >> Alex >> >> On Aug 3, 2015, at 9:19 AM, Teresa Johnson >> <tejohnson at google.com <mailto:tejohnson at google.com>> wrote: >> >>> As discussed in the high-level ThinLTO RFC >>> (http://lists.cs.uiuc.edu/pipermail/llvmdev/2015-May/086211.html), >>> we would like to add support for native object wrapped >>> bitcode and ThinLTO information. Based on comments on >>> the mailing list, I am adding support for ThinLTO in >>> both normal bitcode files, as well as native-object >>> wrapped bitcode. >>> >>> The following RFC describes the planned file format of >>> ThinLTO information both in the bitcode-only and native >>> object wrapped cases. It doesn't yet define the exact >>> record format, as I would like feedback on the overall >>> block design first. >>> >>> I've also implemented the support for reading and >>> writing the bitcode blocks in the following patch: >>> http://reviews.llvm.org/D11722 >>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__reviews.llvm.org_D11722&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Mfk2qtn1LTDThVkh6-oGglNfMADXfJdty4_bhmuhMHA&m=oUy_PB_mSfRgDO7H7bZOR04gv_DMzX5rPO_lv4PHt60&s=WVxrKkHnjKr75fCQ-UkGke8dk6KpZcFCnLWVrJ3G188&e=> >>> >>> The ThinLTO data structures and the file APIs are >>> described in a separate RFC I will be sending >>> simultaneously, with pointers to the patches >>> implementing them. >>> >>> Looking forward to your feedback. Thanks! >>> Teresa >>> >>> >> >> >> -- >> Teresa Johnson | Software Engineer | tejohnson at google.com >> <mailto:tejohnson at google.com> | 408-460-2413 <tel:408-460-2413> >> > > > > > -- > Teresa Johnson | Software Engineer | tejohnson at google.com > <mailto:tejohnson at google.com> | 408-460-2413 <tel:408-460-2413> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150812/7c0d5315/attachment.html>