llvm dev - Apr 2014 - [LLVMdev] [lld] adding demangler for symbol resolution

Hi Nick, Bigcheese,

When lld is used to link C++ code, it would be required to demangle 
symbol names by default/user driven option.

The Gnu linker has the following options :-

--demangle=[style]
--no-demangle

I found that clang/llvm-symbolizer use __cxx_demangle function.

I would think that lld also need to call the same function, and I think 
the way we want to demangle is to have the function in LinkingContext as 
various flavors may choose to use different API's to demangle symbol names.

The API's that would be in LinkingContext would be :-

         * virtual bool canDemangle()  = 0; // Does the flavor provide a 
way to demangle symbol names ?
         * virtual std::string demangle(StringRef symbolName) = 0; // 
demangle the symbol name

Thoughts / Suggestions ?

Thanks

Shankar Easwaran

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by the
Linux Foundation

I agree that LinkingContext should be the right place to have a function
for demangling. Such feature is needed. It'd be useful in many situations
such as printing an error message or logging. But I'd want to keep the
number of API's each LinkingContext needs to support minimum, at least for
now, as we can add more as we need it.

On Tue, Apr 1, 2014 at 9:19 PM, Shankar Easwaran <shankare at
codeaurora.org>wrote:
> Hi Nick, Bigcheese,
>
> When lld is used to link C++ code, it would be required to demangle symbol
> names by default/user driven option.
>
> The Gnu linker has the following options :-
>
> --demangle=[style]
> --no-demangle
>
> I found that clang/llvm-symbolizer use __cxx_demangle function.
>
> I would think that lld also need to call the same function, and I think
> the way we want to demangle is to have the function in LinkingContext as
> various flavors may choose to use different API's to demangle symbol
names.
>
> The API's that would be in LinkingContext would be :-
>
>         * virtual bool canDemangle()  = 0; // Does the flavor provide a
> way to demangle symbol names ?
>         * virtual std::string demangle(StringRef symbolName) = 0; //
> demangle the symbol name
>
I wouldn't add canDemangle, but instead add "virtual std::string
demangle(StringRef symbolName)" with the default implementation that
returns a given string as is. Override the function if you want to support
demangling. Do nothing if you don't.

With this approach you can simply call demangle() unconditionally. If you
are lucky you'll get a demangled symbol name, and if not, you'll get a
fallback string.

Thoughts / Suggestions ?
>
> Thanks
>
> Shankar Easwaran
>
> --
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted
> by the Linux Foundation
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140401/7e633f6c/attachment.html>

On Apr 1, 2014, at 9:19 PM, Shankar Easwaran wrote:
> Hi Nick, Bigcheese,
> 
> When lld is used to link C++ code, it would be required to demangle symbol
names by default/user driven option.
> 
> The Gnu linker has the following options :-
> 
> --demangle=[style]
> --no-demangle
> 
> I found that clang/llvm-symbolizer use __cxx_demangle function.
> 
> I would think that lld also need to call the same function, and I think the
way we want to demangle is to have the function in LinkingContext as various
flavors may choose to use different API's to demangle symbol names.
> 
> The API's that would be in LinkingContext would be :-
> 
>        * virtual bool canDemangle()  = 0; // Does the flavor provide a way
to demangle symbol names ?
>        * virtual std::string demangle(StringRef symbolName) = 0; //
demangle the symbol name
> 
> Thoughts / Suggestions ?
Wouldn't it be simpler to have one demangle() method that does nothing
(returns input string) if demangling is not available, the string is not a
mangled symbol, or demangling was turned off (--no-demangle).   Then, you just
wrap a demangle() call around every use.

The __cxa_demangle function has an odd interface that requires a malloc
allocated block.  Having demangle() return a std::string means yet another
allocation. We might not care if this is just used in diagnostic outputs, but a
more efficient way would be to pass the stream object to demangle and have it
write directly to the stream instead of creating a std::string.

Seems like a demangling utility might be something to add at the LLVM level. 
Either directly to raw_ostream or a wrapper like format().

-Nick

On 4/2/2014 12:23 PM, Nick Kledzik wrote:
> On Apr 1, 2014, at 9:19 PM, Shankar Easwaran wrote:
>
>> Hi Nick, Bigcheese,
>>
>> When lld is used to link C++ code, it would be required to demangle
symbol names by default/user driven option.
>>
>> The Gnu linker has the following options :-
>>
>> --demangle=[style]
>> --no-demangle
>>
>> I found that clang/llvm-symbolizer use __cxx_demangle function.
>>
>> I would think that lld also need to call the same function, and I think
the way we want to demangle is to have the function in LinkingContext as various
flavors may choose to use different API's to demangle symbol names.
>>
>> The API's that would be in LinkingContext would be :-
>>
>>         * virtual bool canDemangle()  = 0; // Does the flavor provide a
way to demangle symbol names ?
>>         * virtual std::string demangle(StringRef symbolName) = 0; //
demangle the symbol name
>>
>> Thoughts / Suggestions ?
> Wouldn't it be simpler to have one demangle() method that does nothing
(returns input string) if demangling is not available, the string is not a
mangled symbol, or demangling was turned off (--no-demangle).   Then, you just
wrap a demangle() call around every use.
Are you mentioning that one demangle function in LinkingContext ?

One demangle method wouldnt work as the ItaniumABI uses one method to 
demangle, ARMCXXABI uses a different method, and MSVC uses a different 
one. I am not sure about Mach-O here ?
> The __cxa_demangle function has an odd interface that requires a malloc
allocated block.  Having demangle() return a std::string means yet another
allocation. We might not care if this is just used in diagnostic outputs, but a
more efficient way would be to pass the stream object to demangle and have it
write directly to the stream instead of creating a std::string.
I dont know if diagnostics in clang, already redirect things directly to 
a stream.

May be for now, as an initial implementation, we can have a single 
demangle function that returns a std::string.

As part of this, I was thinking to cleanup the way the errors are 
displayed to the user from the Resolver, we could have functions in 
SymbolTable with

raiseError(SymbolErrorKind, filename, symbolname)
raiseError(SymbolErrorKind, filename, symbolname, filename, symbolname)

SymbolErrorKind :
MultipleDefinition
Undefined
GroupError
Note (for tracing)
...

> Seems like a demangling utility might be something to add at the LLVM
level.  Either directly to raw_ostream or a wrapper like format().
I have browsed discussions in llvm related to this to move the demangler 
function which is housed in libcxx, and I dont think there is a plan to 
move that.

I think the format() specifier would be one thing that would be useful, 
but I am not sure on how different linking contexts in lld, could route 
calls with a central format specifier.

Can you share more info on this ?

Thanks

Shankar Easwaran

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by the
Linux Foundation

The simplest long-term solution would probably be to just add demangling in
a portable way to libSupport. Then we don't need to conditionalize or
anything. We just call llvm::demangle(). This is the only viable solution
for proper cross-linking anyway.

Given that there should be existing demanglers we can reuse, I'm not sure
that this long term solution is significantly more difficult than the short
term solution you are proposing.

-- Sean Silva

On Tue, Apr 1, 2014 at 9:19 PM, Shankar Easwaran <shankare at
codeaurora.org>
wrote:
>
> Hi Nick, Bigcheese,
>
> When lld is used to link C++ code, it would be required to demangle symbol
> names by default/user driven option.
>
> The Gnu linker has the following options :-
>
> --demangle=[style]
> --no-demangle
>
> I found that clang/llvm-symbolizer use __cxx_demangle function.
>
> I would think that lld also need to call the same function, and I think
> the way we want to demangle is to have the function in LinkingContext as
> various flavors may choose to use different API's to demangle symbol
names.
>
> The API's that would be in LinkingContext would be :-
>
>         * virtual bool canDemangle()  = 0; // Does the flavor provide a
> way to demangle symbol names ?
>         * virtual std::string demangle(StringRef symbolName) = 0; //
> demangle the symbol name
>
> Thoughts / Suggestions ?
>
> Thanks
>
> Shankar Easwaran
>
> --
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted
> by the Linux Foundation
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20141211/d33938e5/attachment.html>

One source of confusion is libstdc++ owns the API __cxa_demangle, when 
the demangle code is moved from libc++ to llvm, does libc++ link with 
libSupport ?

libc++ needs to continue to own the demangle API too, IMO.

Shankar Easwaran

On 12/11/2014 6:24 PM, Sean Silva wrote:
> The simplest long-term solution would probably be to just add demangling in
> a portable way to libSupport. Then we don't need to conditionalize or
> anything. We just call llvm::demangle(). This is the only viable solution
> for proper cross-linking anyway.
>
> Given that there should be existing demanglers we can reuse, I'm not
sure
> that this long term solution is significantly more difficult than the short
> term solution you are proposing.
>
> -- Sean Silva
>
> On Tue, Apr 1, 2014 at 9:19 PM, Shankar Easwaran <shankare at
codeaurora.org>
> wrote:
>> Hi Nick, Bigcheese,
>>
>> When lld is used to link C++ code, it would be required to demangle
symbol
>> names by default/user driven option.
>>
>> The Gnu linker has the following options :-
>>
>> --demangle=[style]
>> --no-demangle
>>
>> I found that clang/llvm-symbolizer use __cxx_demangle function.
>>
>> I would think that lld also need to call the same function, and I think
>> the way we want to demangle is to have the function in LinkingContext
as
>> various flavors may choose to use different API's to demangle
symbol names.
>>
>> The API's that would be in LinkingContext would be :-
>>
>>          * virtual bool canDemangle()  = 0; // Does the flavor provide
a
>> way to demangle symbol names ?
>>          * virtual std::string demangle(StringRef symbolName) = 0; //
>> demangle the symbol name
>>
>> Thoughts / Suggestions ?
>>
>> Thanks
>>
>> Shankar Easwaran
>>
>> --
>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted
>> by the Linux Foundation
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by the
Linux Foundation