Xing GUO via llvm-dev
2020-Aug-30 17:05 UTC
[llvm-dev] GSoC'20 Add DWARF support to yaml2obj
Hi folks,
The GSoC'20 comes to an end. My project is adding DWARF support to
yaml2obj. Now, the usability of yaml2obj is improved. Some of the
outstanding improvements are listed below:
* The Length fields of DWARF sections are replaced with Format and Length.
At first, we have to hardcode the TotalLength and TotalLength64 fields
to instruct the tool to emit a proper DWARF32 or a DWARF64 section,
e.g.,
```
## To handcraft a DWARF32 debug_aranges section.
debug_aranges:
- Length:
TotalLength: 0x1234
Version: 2
...
## To handcraft a DWARF64 debug_aranges section.
debug_aranges:
- Length:
TotalLength: 0xffffffff
TotalLength64: 0x1234
Version: 2
...
```
Now, yaml2obj emits DWARF32 sections by default and the Length field
can be omitted, yaml2obj will calculate it for us (Patches that
address this issue: https://reviews.llvm.org/D82622 ,
https://reviews.llvm.org/D85880 , https://reviews.llvm.org/D81063 ,
https://reviews.llvm.org/D84008 , https://reviews.llvm.org/D86590 ,
https://reviews.llvm.org/D84911 ).
* yaml2obj supports emitting multiple abbrev tables.
yaml2obj only supported emitting a single abbrev table and multiple
compilation units had to share the same abbrev table before
https://reviews.llvm.org/D86194 and https://reviews.llvm.org/D83116 .
Now, yaml2obj is able to emit multiple abbrev tables and compilation
units can be linked to any one of them. We add an optional field ID to
abbrev tables and an optional field AbbrevTableID to compilation
units. Compilation units can use AbbrevTableID to link the abbrev
table with the same ID. Besides, the AbbrOffset field of compilation
units which corresponds to the debug_abbrev_offset field doesn't need
to be specified anymore. yaml2obj is able to calculate it for us after
https://reviews.llvm.org/D86614 .
```
debug_abbrev:
- ID: 0
Table:
...
- ID: 1
Table:
...
debug_info:
- ...
AbbrevTableID: 1 ## Reference the second abbrev table.
## AbbrOffset: ... yaml2obj will take care of it.
- ...
AbbrevTableID: 0 ## Reference the first abbrev table.
## AbbrOffset: ... yaml2obj will take care of it.
```
* The debug_str_offsets, debug_addr, debug_rnglists and debug_loclists
are supported in yaml2obj.
Check out https://reviews.llvm.org/D83624 ,
https://reviews.llvm.org/D84234 , https://reviews.llvm.org/D81541 and
https://reviews.llvm.org/D83853 for more information!
* The DWARF support is added to elf2yaml and improved in macho2yaml.
At first, the output of macho2yaml is noisy. It dumps the DWARF
sections twice, one in the Sections: entry and one in the DWARF:
entry, e.g.,
```
## The content of the debug_str section is dumped twice!
Sections:
- sectname: __debug_str
...
content: 6D61696E00 ## "main\0"
DWARF:
debug_str:
- main
```
After https://reviews.llvm.org/D85506 , if the DWARF parser fails to
parse the DWARF sections into the DWARF: entry, obj2yaml will dump
them as raw content sections, otherwise, they will be presented as
structured DWARF sections in the DWARF: entry. Besides,
https://reviews.llvm.org/D85094 adds DWARF support to elf2yaml.
Although it only supports dumping the debug_aranges section, we can
easily extend it in the future.
=========================================================
Unfinished tasks are listed below. I’m not going to leave the
community and I will improve them in the future.
* Allow users to describe DIEs at a high level.
In my original proposal, we plan to make yaml2obj support describing
DIEs at a high level. However, yaml2obj didn’t support emitting
multiple abbrev tables at that time and I spent some time on enabling
it to emit multiple abbrev tables and link compilation units with
them.
=========================================================
My username on Phabricator is @Higuoxing
(https://reviews.llvm.org/p/Higuoxing/). Please feel free to ping me
if you have trouble in or encountering bugs in crafting DWARF test
cases in YAML. I’m very happy to help.
Last but not least, I would love to express my sincere gratitude to
James Henderson for mentoring me during this project. Besides, I would
like to thank George Rimar, Fangrui Song, Pavel Labath, David Blaikie,
Adrian Prantl and Paul Robinson for reviewing my patches, patiently
answering my questions and leaving comments to my proposal!
=========================================================
My original proposal:
https://docs.google.com/document/d/13wNr4JbXtzaOly-UsFt7vxI3LKXzik_lVU58ICqslWM/edit?usp=sharing
Best Regards,
Xing
James Henderson via llvm-dev
2020-Sep-01 10:16 UTC
[llvm-dev] GSoC'20 Add DWARF support to yaml2obj
I just wanted to publicly say thanks to Xing for the hard work on this. Whilst there are still some elements that aren't quite done yet, I think the overall work he has been doing has been great, and since it has been in the form of lots of incremental improvements, I've already been able to make use of what he has done. Xing, best of luck for the future, and I look forward to continuing to work with you! James On Sun, 30 Aug 2020 at 18:05, Xing GUO <higuoxing at gmail.com> wrote:> Hi folks, > > The GSoC'20 comes to an end. My project is adding DWARF support to > yaml2obj. Now, the usability of yaml2obj is improved. Some of the > outstanding improvements are listed below: > > * The Length fields of DWARF sections are replaced with Format and Length. > > At first, we have to hardcode the TotalLength and TotalLength64 fields > to instruct the tool to emit a proper DWARF32 or a DWARF64 section, > e.g., > > ``` > ## To handcraft a DWARF32 debug_aranges section. > debug_aranges: > - Length: > TotalLength: 0x1234 > Version: 2 > ... > > ## To handcraft a DWARF64 debug_aranges section. > debug_aranges: > - Length: > TotalLength: 0xffffffff > TotalLength64: 0x1234 > Version: 2 > ... > ``` > > Now, yaml2obj emits DWARF32 sections by default and the Length field > can be omitted, yaml2obj will calculate it for us (Patches that > address this issue: https://reviews.llvm.org/D82622 , > https://reviews.llvm.org/D85880 , https://reviews.llvm.org/D81063 , > https://reviews.llvm.org/D84008 , https://reviews.llvm.org/D86590 , > https://reviews.llvm.org/D84911 ). > > * yaml2obj supports emitting multiple abbrev tables. > > yaml2obj only supported emitting a single abbrev table and multiple > compilation units had to share the same abbrev table before > https://reviews.llvm.org/D86194 and https://reviews.llvm.org/D83116 . > Now, yaml2obj is able to emit multiple abbrev tables and compilation > units can be linked to any one of them. We add an optional field ID to > abbrev tables and an optional field AbbrevTableID to compilation > units. Compilation units can use AbbrevTableID to link the abbrev > table with the same ID. Besides, the AbbrOffset field of compilation > units which corresponds to the debug_abbrev_offset field doesn't need > to be specified anymore. yaml2obj is able to calculate it for us after > https://reviews.llvm.org/D86614 . > > ``` > debug_abbrev: > - ID: 0 > Table: > ... > - ID: 1 > Table: > ... > debug_info: > - ... > AbbrevTableID: 1 ## Reference the second abbrev table. > ## AbbrOffset: ... yaml2obj will take care of it. > - ... > AbbrevTableID: 0 ## Reference the first abbrev table. > ## AbbrOffset: ... yaml2obj will take care of it. > ``` > > * The debug_str_offsets, debug_addr, debug_rnglists and debug_loclists > are supported in yaml2obj. > > Check out https://reviews.llvm.org/D83624 , > https://reviews.llvm.org/D84234 , https://reviews.llvm.org/D81541 and > https://reviews.llvm.org/D83853 for more information! > > * The DWARF support is added to elf2yaml and improved in macho2yaml. > > At first, the output of macho2yaml is noisy. It dumps the DWARF > sections twice, one in the Sections: entry and one in the DWARF: > entry, e.g., > > ``` > ## The content of the debug_str section is dumped twice! > Sections: > - sectname: __debug_str > ... > content: 6D61696E00 ## "main\0" > DWARF: > debug_str: > - main > ``` > > After https://reviews.llvm.org/D85506 , if the DWARF parser fails to > parse the DWARF sections into the DWARF: entry, obj2yaml will dump > them as raw content sections, otherwise, they will be presented as > structured DWARF sections in the DWARF: entry. Besides, > https://reviews.llvm.org/D85094 adds DWARF support to elf2yaml. > Although it only supports dumping the debug_aranges section, we can > easily extend it in the future. > > =========================================================> > Unfinished tasks are listed below. I’m not going to leave the > community and I will improve them in the future. > > * Allow users to describe DIEs at a high level. > In my original proposal, we plan to make yaml2obj support describing > DIEs at a high level. However, yaml2obj didn’t support emitting > multiple abbrev tables at that time and I spent some time on enabling > it to emit multiple abbrev tables and link compilation units with > them. > > =========================================================> > My username on Phabricator is @Higuoxing > (https://reviews.llvm.org/p/Higuoxing/). Please feel free to ping me > if you have trouble in or encountering bugs in crafting DWARF test > cases in YAML. I’m very happy to help. > > Last but not least, I would love to express my sincere gratitude to > James Henderson for mentoring me during this project. Besides, I would > like to thank George Rimar, Fangrui Song, Pavel Labath, David Blaikie, > Adrian Prantl and Paul Robinson for reviewing my patches, patiently > answering my questions and leaving comments to my proposal! > > =========================================================> > My original proposal: > > https://docs.google.com/document/d/13wNr4JbXtzaOly-UsFt7vxI3LKXzik_lVU58ICqslWM/edit?usp=sharing > > > Best Regards, > Xing >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200901/e56a3d0f/attachment-0001.html>