llvm dev - Oct 2021 - RFC: A binary serialization format for MemProf

I was going to respond similarly, and add a note that it isn't clear that
gollvm (LLVM-based Go compiler) supports either PGO or the sanitizers, so
that may be more difficult than Rust which does. As Snehasish notes, we are
focused on C/C++, but this will all be done in the LLVM IR level and should
be language independent in theory.
Teresa

On Fri, Oct 1, 2021 at 10:25 AM Snehasish Kumar <snehasishk at google.com>
wrote:
> Hi Andrey,
>
> The serialization format is language independent, though our focus is
> C/C++. Note that our instrumentation is based on the LLVM sanitizer
> infrastructure and should work for Rust (supports building with sanitizers
> [1]). We have not considered using the data profile for non-C/C++ codes.
>
> Regards,
> Snehasish
>
> [1]
> https://doc.rust-lang.org/beta/unstable-book/compiler-flags/sanitizer.html
>
> On Fri, Oct 1, 2021 at 9:14 AM Andrey Bokhanko <andreybokhanko at
gmail.com>
> wrote:
>
>> Hi Snehasish, David and Theresa,
>>
>> I'm really glad to see the steady progress in this area!
>>
>> It looks like the format is pretty much language independent
>> (correct?) -- so it can be applied not only to C/C++, but other
>> languages (Rust) and even toolchains (Go) as well? If you have already
>> considered using data profile for non-C/C++, may I kindly ask you to
>> share your thoughts on this?
>>
>> Yours,
>> Andrey
>> ==>> Advanced Software Technology Lab
>> Huawei
>>
>> On Thu, Sep 30, 2021 at 1:17 AM Snehasish Kumar <snehasishk at
google.com>
>> wrote:
>> >
>>
>
-- 
Teresa Johnson |  Software Engineer |  tejohnson at google.com |
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211001/8e07c05f/attachment.html>

On Fri, Oct 1, 2021 at 10:32 AM Teresa Johnson <tejohnson at google.com>
wrote:
>
> I was going to respond similarly, and add a note that it isn't clear
that gollvm (LLVM-based Go compiler) supports either PGO or the sanitizers, so
that may be more difficult than Rust which does. As Snehasish notes, we are
focused on C/C++, but this will all be done in the LLVM IR level and should be
language independent in theory.
+Than McIntosh to comment more on PGO and sanitizer support for gollvm.

David

> Teresa
>
> On Fri, Oct 1, 2021 at 10:25 AM Snehasish Kumar <snehasishk at
google.com> wrote:
>>
>> Hi Andrey,
>>
>> The serialization format is language independent, though our focus is
C/C++. Note that our instrumentation is based on the LLVM sanitizer
infrastructure and should work for Rust (supports building with sanitizers [1]).
We have not considered using the data profile for non-C/C++ codes.
>>
>> Regards,
>> Snehasish
>>
>> [1]
https://doc.rust-lang.org/beta/unstable-book/compiler-flags/sanitizer.html
>>
>> On Fri, Oct 1, 2021 at 9:14 AM Andrey Bokhanko <andreybokhanko at
gmail.com> wrote:
>>>
>>> Hi Snehasish, David and Theresa,
>>>
>>> I'm really glad to see the steady progress in this area!
>>>
>>> It looks like the format is pretty much language independent
>>> (correct?) -- so it can be applied not only to C/C++, but other
>>> languages (Rust) and even toolchains (Go) as well? If you have
already
>>> considered using data profile for non-C/C++, may I kindly ask you
to
>>> share your thoughts on this?
>>>
>>> Yours,
>>> Andrey
>>> ==>>> Advanced Software Technology Lab
>>> Huawei
>>>
>>> On Thu, Sep 30, 2021 at 1:17 AM Snehasish Kumar <snehasishk at
google.com> wrote:
>>> >
>
>
>
> --
> Teresa Johnson | Software Engineer | tejohnson at google.com |

Thanks Teresa and others for the clarification!

On Fri, Oct 1, 2021 at 8:32 PM Teresa Johnson <tejohnson at google.com>
wrote:
> I was going to respond similarly, and add a note that it isn't clear
that
> gollvm (LLVM-based Go compiler) supports either PGO or the sanitizers, so
> that may be more difficult than Rust which does. As Snehasish notes, we are
> focused on C/C++, but this will all be done in the LLVM IR level and should
> be language independent in theory.
>
Let me note that I specifically meant gc (Google's standard Go compiler),
not gollvm. IMHO, there is an intrinsic value of data formats being unified
among different toolchains -- as very well demonstrated by DWARF.

(Yes, I'm aware that gc doesn't support even ages-long instruction
profiling. One of the reasons is the apparent lack of implemented
optimizations that can directly benefit from profiling. In case of memory
profiling, the use case is clear. Also, given that BOLT helps Go a lot (up
to +20% speed-up on our internal tests), I expect the same for memory
profiling, which will warrant extending gc capabilities to use MemProf
format.)

Yours,
Andrey

> Teresa
>
> On Fri, Oct 1, 2021 at 10:25 AM Snehasish Kumar <snehasishk at
google.com>
> wrote:
>
>> Hi Andrey,
>>
>> The serialization format is language independent, though our focus is
>> C/C++. Note that our instrumentation is based on the LLVM sanitizer
>> infrastructure and should work for Rust (supports building with
sanitizers
>> [1]). We have not considered using the data profile for non-C/C++
codes.
>>
>> Regards,
>> Snehasish
>>
>> [1]
>>
https://doc.rust-lang.org/beta/unstable-book/compiler-flags/sanitizer.html
>>
>> On Fri, Oct 1, 2021 at 9:14 AM Andrey Bokhanko <andreybokhanko at
gmail.com>
>> wrote:
>>
>>> Hi Snehasish, David and Theresa,
>>>
>>> I'm really glad to see the steady progress in this area!
>>>
>>> It looks like the format is pretty much language independent
>>> (correct?) -- so it can be applied not only to C/C++, but other
>>> languages (Rust) and even toolchains (Go) as well? If you have
already
>>> considered using data profile for non-C/C++, may I kindly ask you
to
>>> share your thoughts on this?
>>>
>>> Yours,
>>> Andrey
>>> ==>>> Advanced Software Technology Lab
>>> Huawei
>>>
>>> On Thu, Sep 30, 2021 at 1:17 AM Snehasish Kumar <snehasishk at
google.com>
>>> wrote:
>>> >
>>>
>>
>
> --
> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211004/49fc1432/attachment.html>