llvm dev - Sep 2016 - what is official way to determine if we are running lto 2nd stage?

Hi,

Imagine that your backend has valid asm instruction written like this:

"%x mnem %y, %z"

And user puts it as inline assembler:

__asm__ ("%x mnem %y, %z");

It can not be parsed with current llvm asm parser, because it starts
with % (moreover it has mnemonic in second place)

Say you written pass, that makes it "mnem %x, %y, %z".

Now this guy can be parsed, but can not be encoded by gas. You simply
havent that instruction in you assembler. For LTO it isn't a problem:
you can make arbitrary MCInst from everything that comes into
ParseInstruction. But it is problem for regular scenario where wrong
asm will be printed and then passed to gas.

So I want to apply this on 2nd lto stage where AsmParser is inevitable
and to not apply in non-LTO cases.

---
With best regards, Konstantin

On Mon, Sep 12, 2016 at 10:19 PM, Mehdi Amini <mehdi.amini at apple.com>
wrote:
>
>> On Sep 12, 2016, at 11:07 AM, Konstantin Vladimirov
<konstantin.vladimirov at gmail.com> wrote:
>>
>> Hi,
>>
>> This is really basic block level pass. It is no difference what is
>> level, problem is the same.
>
> Can you clarify what you mean? If you have a MachineFunction pass, it’ll
run in the backend only.
> I don’t understand why you would need to distinguish between LTO or no-LTO
here.
>
>>
>> After fixing for asm parser, assembler syntax is no more valid for
>> backend, without processing with asm parser.
>
> My understanding of Inline ASM is that it is supposed to be opaque to the
backend till you reach MC.
> So I don’t understand this sentence "no more valid for backend,
without processing with asm parser”.
>
> Sorry if my answers don’t make sense to you, I may still be missing a key
part of your problem.
>
> —
> Mehdi
>
>
>> May be it will be solution to process inline asm on insn printer level
>> to remove syntax fixes. But just switch it off without lto will make
>> compiler do less job
>>
>> P.S. sorry for dup, maillist CC lost on first sent.
>>
>> ---
>> WIth best regards, Konstantin
>>
>>
>> On Mon, Sep 12, 2016 at 8:52 PM, Mehdi Amini <mehdi.amini at
apple.com> wrote:
>>>
>>>> On Sep 12, 2016, at 9:26 AM, Konstantin Vladimirov
<konstantin.vladimirov at gmail.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> In LTO we have AsmParser that process inline assembler
instructions to
>>>> MCInst and I want to fix some inline assembler in order to
conform its
>>>> rules (do not start with non-identifier and so on) because asm
syntax
>>>> of our backend allows some incompatible patterns. In order to
do this
>>>> I am adding IR-level target-specific pass. But those fixes
shall not
>>>> be applied when there is no AsmParser later to process them. So
I want
>>>> to switch this pass off if we are not in 2nd lto stage.
>>>
>>> This is not clear to me: how should this be different for LTO than
for a non-LTO compile?
>>>
>>> Also, if you’re only fixing the inline ASM, why doing it as an
IR-level pass instead of MachineFunctionPass?
>>>
>>> —
>>> Mehdi
>>>
>>>
>>>>
>>>> I think, I can make target-specific option and make user to
supply it
>>>> whenever he wants to run 2nd lto stage, but this is ugly.
>>>>
>>>> Can I somehow ask, say, about whole string of options and then
parse
>>>> it to match "lto" from here?
>>>>
>>>> ---
>>>> With best regards, Konstantin
>>>>
>>>> On Mon, Sep 12, 2016 at 6:17 PM, Mehdi Amini <mehdi.amini at
apple.com> wrote:
>>>>> Hi,
>>>>>
>>>>>
>>>>>> On Sep 12, 2016, at 1:26 AM, Konstantin Vladimirov via
llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I want to enable some target-specific functionality
only if current
>>>>>> build is 2nd LTO stage (i.e. optimizer called from
plugin). What is
>>>>>> best and recommended way to do it?
>>>>>
>>>>> There is none. We can setup a different optimizer pass
pipeline for LTO, but the target specific part (i.e. the backend) isn’t supposed
to behave differently.
>>>>>
>>>>> This is an issue in general with LTO where options from the
command line (like -fno-builtins, or -fveclib=xxxx) are not correctly propagated
to LTO.
>>>>>
>>>>> What kind of behavior do you want to enable exactly?
>>>>>
>>>>> —
>>>>> Mehdi
>>>>>
>>>
>

> On Sep 12, 2016, at 10:01 PM, Konstantin Vladimirov
<konstantin.vladimirov at gmail.com> wrote:
> 
> Hi,
> 
> Imagine that your backend has valid asm instruction written like this:
> 
> "%x mnem %y, %z"
> 
> And user puts it as inline assembler:
> 
> __asm__ ("%x mnem %y, %z");
> 
> It can not be parsed with current llvm asm parser, because it starts
> with % (moreover it has mnemonic in second place)
> 
> Say you written pass, that makes it "mnem %x, %y, %z".
> 
> Now this guy can be parsed, but can not be encoded by gas. You simply
> havent that instruction in you assembler. For LTO it isn't a problem:
> you can make arbitrary MCInst from everything that comes into
> ParseInstruction. But it is problem for regular scenario where wrong
> asm will be printed and then passed to gas.
> 
> So I want to apply this on 2nd lto stage where AsmParser is inevitable
> and to not apply in non-LTO cases.
OK, from what you are describing, it does not seem a LTO vs non-LTO, but
integrated-ASM vs non-integrated-ASM, right?

— 
Mehdi


> 
> ---
> With best regards, Konstantin
> 
> On Mon, Sep 12, 2016 at 10:19 PM, Mehdi Amini <mehdi.amini at
apple.com> wrote:
>> 
>>> On Sep 12, 2016, at 11:07 AM, Konstantin Vladimirov
<konstantin.vladimirov at gmail.com> wrote:
>>> 
>>> Hi,
>>> 
>>> This is really basic block level pass. It is no difference what is
>>> level, problem is the same.
>> 
>> Can you clarify what you mean? If you have a MachineFunction pass,
it’ll run in the backend only.
>> I don’t understand why you would need to distinguish between LTO or
no-LTO here.
>> 
>>> 
>>> After fixing for asm parser, assembler syntax is no more valid for
>>> backend, without processing with asm parser.
>> 
>> My understanding of Inline ASM is that it is supposed to be opaque to
the backend till you reach MC.
>> So I don’t understand this sentence "no more valid for backend,
without processing with asm parser”.
>> 
>> Sorry if my answers don’t make sense to you, I may still be missing a
key part of your problem.
>> 
>> —
>> Mehdi
>> 
>> 
>>> May be it will be solution to process inline asm on insn printer
level
>>> to remove syntax fixes. But just switch it off without lto will
make
>>> compiler do less job
>>> 
>>> P.S. sorry for dup, maillist CC lost on first sent.
>>> 
>>> ---
>>> WIth best regards, Konstantin
>>> 
>>> 
>>> On Mon, Sep 12, 2016 at 8:52 PM, Mehdi Amini <mehdi.amini at
apple.com> wrote:
>>>> 
>>>>> On Sep 12, 2016, at 9:26 AM, Konstantin Vladimirov
<konstantin.vladimirov at gmail.com> wrote:
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> In LTO we have AsmParser that process inline assembler
instructions to
>>>>> MCInst and I want to fix some inline assembler in order to
conform its
>>>>> rules (do not start with non-identifier and so on) because
asm syntax
>>>>> of our backend allows some incompatible patterns. In order
to do this
>>>>> I am adding IR-level target-specific pass. But those fixes
shall not
>>>>> be applied when there is no AsmParser later to process
them. So I want
>>>>> to switch this pass off if we are not in 2nd lto stage.
>>>> 
>>>> This is not clear to me: how should this be different for LTO
than for a non-LTO compile?
>>>> 
>>>> Also, if you’re only fixing the inline ASM, why doing it as an
IR-level pass instead of MachineFunctionPass?
>>>> 
>>>> —
>>>> Mehdi
>>>> 
>>>> 
>>>>> 
>>>>> I think, I can make target-specific option and make user to
supply it
>>>>> whenever he wants to run 2nd lto stage, but this is ugly.
>>>>> 
>>>>> Can I somehow ask, say, about whole string of options and
then parse
>>>>> it to match "lto" from here?
>>>>> 
>>>>> ---
>>>>> With best regards, Konstantin
>>>>> 
>>>>> On Mon, Sep 12, 2016 at 6:17 PM, Mehdi Amini
<mehdi.amini at apple.com> wrote:
>>>>>> Hi,
>>>>>> 
>>>>>> 
>>>>>>> On Sep 12, 2016, at 1:26 AM, Konstantin Vladimirov
via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> I want to enable some target-specific functionality
only if current
>>>>>>> build is 2nd LTO stage (i.e. optimizer called from
plugin). What is
>>>>>>> best and recommended way to do it?
>>>>>> 
>>>>>> There is none. We can setup a different optimizer pass
pipeline for LTO, but the target specific part (i.e. the backend) isn’t supposed
to behave differently.
>>>>>> 
>>>>>> This is an issue in general with LTO where options from
the command line (like -fno-builtins, or -fveclib=xxxx) are not correctly
propagated to LTO.
>>>>>> 
>>>>>> What kind of behavior do you want to enable exactly?
>>>>>> 
>>>>>> —
>>>>>> Mehdi
>>>>>> 
>>>> 
>>

Hi,

This is possible interpretation. But really only use-case where we
need AsmParser now is lto case. So for me it is more about lto vs
non-lto, because in non-lto cases we can simply output any assembler
and call gas via custom Toolchain in clang, whereas in lto we need to
be asmparser-compatible.

---
With best regards, Konstantin


On Tue, Sep 13, 2016 at 8:06 AM, Mehdi Amini <mehdi.amini at apple.com>
wrote:
>
>> On Sep 12, 2016, at 10:01 PM, Konstantin Vladimirov
<konstantin.vladimirov at gmail.com> wrote:
>>
>> Hi,
>>
>> Imagine that your backend has valid asm instruction written like this:
>>
>> "%x mnem %y, %z"
>>
>> And user puts it as inline assembler:
>>
>> __asm__ ("%x mnem %y, %z");
>>
>> It can not be parsed with current llvm asm parser, because it starts
>> with % (moreover it has mnemonic in second place)
>>
>> Say you written pass, that makes it "mnem %x, %y, %z".
>>
>> Now this guy can be parsed, but can not be encoded by gas. You simply
>> havent that instruction in you assembler. For LTO it isn't a
problem:
>> you can make arbitrary MCInst from everything that comes into
>> ParseInstruction. But it is problem for regular scenario where wrong
>> asm will be printed and then passed to gas.
>>
>> So I want to apply this on 2nd lto stage where AsmParser is inevitable
>> and to not apply in non-LTO cases.
>
> OK, from what you are describing, it does not seem a LTO vs non-LTO, but
integrated-ASM vs non-integrated-ASM, right?
>
> —
> Mehdi
>
>
>
>>
>> ---
>> With best regards, Konstantin
>>
>> On Mon, Sep 12, 2016 at 10:19 PM, Mehdi Amini <mehdi.amini at
apple.com> wrote:
>>>
>>>> On Sep 12, 2016, at 11:07 AM, Konstantin Vladimirov
<konstantin.vladimirov at gmail.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> This is really basic block level pass. It is no difference what
is
>>>> level, problem is the same.
>>>
>>> Can you clarify what you mean? If you have a MachineFunction pass,
it’ll run in the backend only.
>>> I don’t understand why you would need to distinguish between LTO or
no-LTO here.
>>>
>>>>
>>>> After fixing for asm parser, assembler syntax is no more valid
for
>>>> backend, without processing with asm parser.
>>>
>>> My understanding of Inline ASM is that it is supposed to be opaque
to the backend till you reach MC.
>>> So I don’t understand this sentence "no more valid for
backend, without processing with asm parser”.
>>>
>>> Sorry if my answers don’t make sense to you, I may still be missing
a key part of your problem.
>>>
>>> —
>>> Mehdi
>>>
>>>
>>>> May be it will be solution to process inline asm on insn
printer level
>>>> to remove syntax fixes. But just switch it off without lto will
make
>>>> compiler do less job
>>>>
>>>> P.S. sorry for dup, maillist CC lost on first sent.
>>>>
>>>> ---
>>>> WIth best regards, Konstantin
>>>>
>>>>
>>>> On Mon, Sep 12, 2016 at 8:52 PM, Mehdi Amini <mehdi.amini at
apple.com> wrote:
>>>>>
>>>>>> On Sep 12, 2016, at 9:26 AM, Konstantin Vladimirov
<konstantin.vladimirov at gmail.com> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> In LTO we have AsmParser that process inline assembler
instructions to
>>>>>> MCInst and I want to fix some inline assembler in order
to conform its
>>>>>> rules (do not start with non-identifier and so on)
because asm syntax
>>>>>> of our backend allows some incompatible patterns. In
order to do this
>>>>>> I am adding IR-level target-specific pass. But those
fixes shall not
>>>>>> be applied when there is no AsmParser later to process
them. So I want
>>>>>> to switch this pass off if we are not in 2nd lto stage.
>>>>>
>>>>> This is not clear to me: how should this be different for
LTO than for a non-LTO compile?
>>>>>
>>>>> Also, if you’re only fixing the inline ASM, why doing it as
an IR-level pass instead of MachineFunctionPass?
>>>>>
>>>>> —
>>>>> Mehdi
>>>>>
>>>>>
>>>>>>
>>>>>> I think, I can make target-specific option and make
user to supply it
>>>>>> whenever he wants to run 2nd lto stage, but this is
ugly.
>>>>>>
>>>>>> Can I somehow ask, say, about whole string of options
and then parse
>>>>>> it to match "lto" from here?
>>>>>>
>>>>>> ---
>>>>>> With best regards, Konstantin
>>>>>>
>>>>>> On Mon, Sep 12, 2016 at 6:17 PM, Mehdi Amini
<mehdi.amini at apple.com> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>>
>>>>>>>> On Sep 12, 2016, at 1:26 AM, Konstantin
Vladimirov via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I want to enable some target-specific
functionality only if current
>>>>>>>> build is 2nd LTO stage (i.e. optimizer called
from plugin). What is
>>>>>>>> best and recommended way to do it?
>>>>>>>
>>>>>>> There is none. We can setup a different optimizer
pass pipeline for LTO, but the target specific part (i.e. the backend) isn’t
supposed to behave differently.
>>>>>>>
>>>>>>> This is an issue in general with LTO where options
from the command line (like -fno-builtins, or -fveclib=xxxx) are not correctly
propagated to LTO.
>>>>>>>
>>>>>>> What kind of behavior do you want to enable
exactly?
>>>>>>>
>>>>>>> —
>>>>>>> Mehdi
>>>>>>>
>>>>>
>>>
>