llvm dev - Mar 2020 - [yaml2obj] GSoC-20: Add DWARF support to yaml2obj

Hi there,

I'm proposing for the GSoC project: Add DWARF support to yaml2obj[1].
I've uploaded my proposal. If you have any suggestion or ideas, feel
free to leave a comment[2].

Thanks!
------
[1] https://llvm.org/OpenProjects.html#llvm_dwarf_yaml2obj
[2]
https://docs.google.com/document/d/1miCuMQEX8WZ9_hWXWQtOYTA4JAK-yDnPV9vyO_d5vzE/edit?usp=sharing

-- 
Best Regards,
Xing

It's great to see someone interested in improving yaml2obj!

As far as I'm concerned, the main problem with yam2obj for DWARF testcases
is that at the moment, it is both too high-level and too low-level at the same
time. For writing debug info testcases, yaml2obj at the moment does not add
anything on top of assembler. If it is only providing an alternative syntax, and
no other features, it isn't that useful. Let me explain what I mean:

1. Too high-level
 
For testing parsers we need to be able to create malformed input, so we need to
be able to manually tweak offsets and headers where necessary. IIRC currently
yaml2obj always creates section headers automatically, and can do so for only
one DWARF version. It would be valuable to support more than one version of
DWARF, and to support them on a per-section basis (it is not uncommon to mix a
DWARF 5 .debug_info section with a DWARF 4 line table). Also there needs to be a
way to optionally manually specify headers byte-for-byte, for when we do need
this.

2. Too low-level

For functionality tests, however, we don't want to hardcode things like byte
offsets, because it makes it extremely hard to write/modify tests by hand. It
would be awesome if yaml2obj could automatically generate abbreviation sections
if the user requests it, if their exact layout isn't relevant to the test.
Similarly, having to manually adjust object file metadata, such as Mach-O
section load commands or ELF headers every time we're editing a test in a
way that changes the size of, e.g., the .debug_info section prevents us from
using yaml2obj for any kind of hand-written tests.

The tasks you are describing to add explicit syntax for more DWARF constructs
are also good and necessary, but addressing one or both of these problems would
be even more important to increase the usefulness of yaml2obj for DWARF testing.

-- adrian

On 31/03/2020 18:29, Adrian Prantl via llvm-dev wrote:
> It's great to see someone interested in improving yaml2obj!
> 
> As far as I'm concerned, the main problem with yam2obj for DWARF
testcases is that at the moment, it is both too high-level and too low-level at
the same time. For writing debug info testcases, yaml2obj at the moment does not
add anything on top of assembler. If it is only providing an alternative syntax,
and no other features, it isn't that useful. Let me explain what I mean:
> 
> 1. Too high-level
>  
> For testing parsers we need to be able to create malformed input, so we
need to be able to manually tweak offsets and headers where necessary. IIRC
currently yaml2obj always creates section headers automatically, and can do so
for only one DWARF version. It would be valuable to support more than one
version of DWARF, and to support them on a per-section basis (it is not uncommon
to mix a DWARF 5 .debug_info section with a DWARF 4 line table). Also there
needs to be a way to optionally manually specify headers byte-for-byte, for when
we do need this.
> 
> 2. Too low-level
> 
> For functionality tests, however, we don't want to hardcode things like
byte offsets, because it makes it extremely hard to write/modify tests by hand.
It would be awesome if yaml2obj could automatically generate abbreviation
sections if the user requests it, if their exact layout isn't relevant to
the test. Similarly, having to manually adjust object file metadata, such as
Mach-O section load commands or ELF headers every time we're editing a test
in a way that changes the size of, e.g., the .debug_info section prevents us
from using yaml2obj for any kind of hand-written tests.
> 
> The tasks you are describing to add explicit syntax for more DWARF
constructs are also good and necessary, but addressing one or both of these
problems would be even more important to increase the usefulness of yaml2obj for
DWARF testing.
> 
> -- adrian
> 
Hello Xing,

I'd like to echo Adrian here. I currently find the assembly much more
readable and maintainable that the yaml dwarf representation, and so I
always write tests that way.

For me personally, the ability to write/edit syntactically correct dwarf
easily is much more important than being able to generate "incorrect"
dwarf -- I'm perfectly happy to continue to write the latter in
assembly, but there is a lot that could be improved about the experience
of writing "correct" dwarf. Ideally, I'd have a mode where I can
just
write the logical DIE structure (tags, attributes and their values), and
the tool would split that into the abbrev/loclist/range/etc. sections.

regards,
pavel