llvm dev - Dec 2010 - [LLVMdev] Function-level metadata for OpenCL (was Re: OpenCL support)

On Fri, Dec 17, 2010 at 04:21:18PM -0500, David Neto
wrote:
> However we record the fact that a function is a kernel, the mechanism
> should handle the case of a kernel calling another kernel.
> Recall that a kernel called by another kernel behaves more like a
> regular function.  For example it doesn't have workspace iteration
> automatically applied to it; rather it just adopts the work item of
> the caller.
> 
> About using a calling convention to mark a function as a kernel.  It
> seems a handy place to hang it, but is it really exclusive of the
> other calling conventions?
> In particular, does that approach nicely in the case where a CPU is
> running the kernels?  Does that lead to special casing or duplication
> in the code generator?  For example, you still have to know what
> "real" calling convention to use when a kernel is running on a
CPU.
> (Forgive my ignorance.)
As with __local variables, it may be that "kernelness" cannot be
represented in a standard form in LLVM.  For example on a CPU a
kernel function may have an additional parameter which is a pointer to
__local memory space, which would not be necessary on GPUs.  Then in
fact you would use a standard calling convention on a CPU.

But for GPUs, I think using the calling convention is appropriate.
If we standardise the calling convention number, this can be the
default behaviour.
> I would be happy to see an OpenCL-specific patch that always marked
> non-kernel functions with internal linkage.  Then you could
> distinguish the kernel/non-kernel case just by the linkage attribute.
> It might be a little unclean / unorthogonal, but I think it would be
> ok.
Some OpenCL implementations (including my own) may use runtime library
functions which live in a separate compilation unit.  These would
need to be marked external but of course would not be kernel functions.

Nick Lewycky wrote:
> Being discardable is a design point of metadata. You might add something
> else to support this, but it won't be metadata.
There's nothing intrinsic about the concept of metadata which requires
it to be discardable.  In particular, if the metadata is attached to
a function, the only case I can think of where an optimiser needs
to touch the metadata is if a function with metadata is inlined.
And as I mentioned in my previous mail I don't think this will be
any trouble for OpenCL.

The __kernel attribute isn't the only attribute we need to preserve.
There are also:

__attribute__((vec_type_hint(type)))
__attribute__((work_group_size_hint(X, Y, Z)))
__attribute__((reqd_work_group_size(X, Y, Z))) 

which provide hints to the code generator regarding the specific work
load of a particular kernel.
> Why are you trying to preserve "kernel"-ness into the LLVM IR?
What
> semantics does it have? What does __kernel actually mean to the optimizers
> and code generator?
For PTX, if __kernel is set on a function it needs to be codegen'd
with a specific directive which marks it as a kernel entry point.
What this actually means at a lower level I don't know (the low level
machine code representation is undocumented).  I believe there is
also something similar in the AMD Stream IL.

As for the other attributes mentioned above, I don't know off-hand,
but I believe there are PTX directives for at least some of them.
> Could you just make __kernel mean "externally visible" and
undecorated
> functions be "linkonce_odr"?
I think the semantics of undecorated functions is closer to "internal"
than "linkonce_odr" here (kernel programs shouldn't be able to
provide a definition for functions in another module, such as a
runtime library module).
> If that's not enough, could you swing it around
> and maintain single named metadata node with a list of functions that are
> marked __kernel?
Are you saying that named metadata nodes are non-discardable?  Even
if this were true, it would still be difficult to represent the other
attributes unless the metadata were attached to the function.

Thanks,
-- 
Peter

On Mon, Dec 20, 2010 at 3:11 PM, Peter Collingbourne <peter at pcc.me.uk>
wrote:
> On Fri, Dec 17, 2010 at 04:21:18PM -0500, David Neto wrote:
>> I would be happy to see an OpenCL-specific patch that always marked
>> non-kernel functions with internal linkage.  Then you could
>> distinguish the kernel/non-kernel case just by the linkage attribute.
>> It might be a little unclean / unorthogonal, but I think it would be
>> ok.
>
> Some OpenCL implementations (including my own) may use runtime library
> functions which live in a separate compilation unit.  These would
> need to be marked external but of course would not be kernel functions.
I stand corrected.  :-)

thanks,
david

On Mon, Dec 20, 2010 at 9:11 PM, Peter Collingbourne <peter at pcc.me.uk>
wrote:
> Nick Lewycky wrote:
>> Being discardable is a design point of metadata. You might add
something
>> else to support this, but it won't be metadata.
>
> There's nothing intrinsic about the concept of metadata which requires
> it to be discardable.  In particular, if the metadata is attached to
> a function, the only case I can think of where an optimiser needs
> to touch the metadata is if a function with metadata is inlined.
There may be cases where an optimization pass doesn't know how to
update metadata, so discarding it is the only way to prevent invalid
metadata while still performing the transformation.
If metadata would be attached to functions, that means passes like
dead argument elimination may need to remove it since they create a
new function (with a different type), and the metadata may depend on
the function type staying the same. For instance: the metadata may
contain references to parameter numbers for all it knows, and it won't
know how to change that to account for the removed parameters.

> From: Peter Collingbourne [mailto:peter at pcc.me.uk]
> Sent: 20 December 2010 20:11
> As with __local variables, it may be that "kernelness" cannot be
> represented in a standard form in LLVM.  For example on a CPU a
> kernel function may have an additional parameter which is a pointer to
> __local memory space, which would not be necessary on GPUs.  Then in
> fact you would use a standard calling convention on a CPU.
> 
> But for GPUs, I think using the calling convention is appropriate.
> If we standardise the calling convention number, this can be the
> default behaviour.
I don't think we want LLVM-IR coming from an OpenCL C frontend to be
different for GPU and CPU targets.  In my view, the frontend should be
parameterised by only two (more or less) parameters: bitness (32/64) and
endianness (little/big).  How one can even guarantee e.g. that a calling
convention for NVIDIA GPUs is appropriate for ATI GPUs?  So using calling
conventions too early on (e.g. between invoking the clBuildProgram() and
clCreateKernel() API functions) is a path to implementation divergence,
rather than standardisation.


From: Nick Lewycky [mailto:nlewycky at google.com] 
Sent: 17 December 2010 22:16
> > If that's not enough, could you swing it around and maintain
single
> > named metadata node with a list of functions that are marked __kernel?
This is exactly what we would like to do if metadata
"non-discardability"
could be guaranteed at least until the clCreateKernel() API function is
called.

Cheers,
Anton.

-----Original Message-----
From: cfe-dev-bounces at cs.uiuc.edu [mailto:cfe-dev-bounces at cs.uiuc.edu]
On Behalf Of Anton Lokhmotov
Sent: 21 December 2010 19:18
To: 'Peter Collingbourne'; David Neto; nlewycky at google.com
Cc: cfe-dev at cs.uiuc.edu; llvmdev at cs.uiuc.edu
Subject: Re: [cfe-dev] [LLVMdev] Function-level metadata for OpenCL (was
Re:OpenCL support)
> I don't think we want LLVM-IR coming from an OpenCL C frontend to be
> different for GPU and CPU targets.  In my view, the frontend should be
> parameterised by only two (more or less) parameters: bitness (32/64)
and
> endianness (little/big).  How one can even guarantee e.g. that a
calling
> convention for NVIDIA GPUs is appropriate for ATI GPUs?  So using
calling
> conventions too early on (e.g. between invoking the clBuildProgram()
and
> clCreateKernel() API functions) is a path to implementation
divergence,
> rather than standardisation.
Agreed, there should only be one IR representation.
> This is exactly what we would like to do if metadata
"non-discardability"
> could be guaranteed at least until the clCreateKernel() API function
is
> called.
LLVM doesn't know (and shouldn't know imo) anything about CL API calls
(unless you didn't mean that in which case nevermind :) ). I still don't
think that metadata is the right approach for recording certain CL
specific function attributes. The blog entry that Chris and Devang wrote
back in March detailing metadata makes it quite clear that metadata
shouldn't be used for anything that can't be discarded at whim. So
I'm
nervous about adding anything that 'breaks' this design decision,
without actually making the decision to change that design (which
probably needs input from the original authors).

Mike

On Mon, Dec 20, 2010 at 10:53:21PM +0100, Frits van Bommel
wrote:
> On Mon, Dec 20, 2010 at 9:11 PM, Peter Collingbourne <peter at
pcc.me.uk> wrote:
> > Nick Lewycky wrote:
> >> Being discardable is a design point of metadata. You might add
something
> >> else to support this, but it won't be metadata.
> >
> > There's nothing intrinsic about the concept of metadata which
requires
> > it to be discardable.  In particular, if the metadata is attached to
> > a function, the only case I can think of where an optimiser needs
> > to touch the metadata is if a function with metadata is inlined.
> 
> There may be cases where an optimization pass doesn't know how to
> update metadata, so discarding it is the only way to prevent invalid
> metadata while still performing the transformation.
> If metadata would be attached to functions, that means passes like
> dead argument elimination may need to remove it since they create a
> new function (with a different type), and the metadata may depend on
> the function type staying the same. For instance: the metadata may
> contain references to parameter numbers for all it knows, and it won't
> know how to change that to account for the removed parameters.
In general metadata should be designed in such a way as to be resilient
to changes to the IR.  For example, function metadata shouldn't
refer to parameters by index, for the same reason that instruction
metadata shouldn't refer to other instructions by relative offset.
If metadata about parameters needs to be stored then it should be
either attached to the parameter itself or by some other mechanism
(e.g. the llvm.dbg.value function used by the debug metadata).
The end result is that if an optimiser needs to copy a function it
can just copy the metadata also.

As with instruction metadata, if there is a reference to a global
or a non-constant value in function metadata it will be discarded if
those values are deleted.  For simpler use cases (including OpenCL)
the metadata will only need to use integer constants and maybe types,
so non-discardability shouldn't be an issue, as far as I'm aware.

Thanks,
-- 
Peter

On Tue, Dec 21, 2010 at 07:17:40PM -0000, Anton Lokhmotov
wrote:
> > From: Peter Collingbourne [mailto:peter at pcc.me.uk]
> > Sent: 20 December 2010 20:11
> > As with __local variables, it may be that "kernelness"
cannot be
> > represented in a standard form in LLVM.  For example on a CPU a
> > kernel function may have an additional parameter which is a pointer to
> > __local memory space, which would not be necessary on GPUs.  Then in
> > fact you would use a standard calling convention on a CPU.
> > 
> > But for GPUs, I think using the calling convention is appropriate.
> > If we standardise the calling convention number, this can be the
> > default behaviour.
> I don't think we want LLVM-IR coming from an OpenCL C frontend to be
> different for GPU and CPU targets. In my view, the frontend should be
> parameterised by only two (more or less) parameters: bitness (32/64) and
> endianness (little/big).
Not only sizes but alignment requirements will change between
platforms.  Also, what about __local on CPU?
> How one can even guarantee e.g. that a calling
> convention for NVIDIA GPUs is appropriate for ATI GPUs?
We have full control over the target code generators.  There's nothing
stopping us defining a specific constant representing the 'kernel'
calling convention and harmonising the GPU targets to use that
calling convention.

Thanks,
-- 
Peter