[llvm-dev] Opaque Pointers Help Wanted

David Blaikie via llvm-dev
Thu, 01 Jul 2021 17:43:46 -0000

On Thu, Jul 1, 2021 at 8:00 AM Nikita Popov <nikita.ppv at gmail.com>
wrote:

> On Thu, Jul 1, 2021 at 4:39 PM <paul.robinson at sony.com> wrote:
>
>> | My expectation is that after the opaque pointer migration is
complete,
>> the time the non-opaque pointer mode remains available will be measured
in
>> days, not years.
>>
>>
>>
>> That can be true for textual IR, but we need to retain backward
>> compatibility (auto-upgrade) for bitcode for much longer.  I don’t have
a
>> clear idea of what’s involved (haven’t really been tracking this
project)
>> but many users/scenarios require bitcode compatibility for long
periods.  I
>> don’t think we’ve had a true format break since about 4.0?
>>
>> --paulr
>>
>
> Sure, we definitely need to retain bitcode compatibility (i.e.
> auto-upgrade to opaque pointers).
>
> My comment was targeted at maintainers of out-of-free frontends and
> targets, who shouldn't assume there will be a long-term period of dual
> compatibility, like there was/is with the new pass manager. If you
don't
> want to be stuck on an old toolchain, you need to be ready by the time the
> switch happens.
>

Correct me if I'm wrong, but to clarify, I think: LLVM IR API likely
won't
have a long period of overlap supporting both opaque and non-opaque
pointers and especially the ability to mix/match in IR may never exist
(hard cutover) - as discussed with things like intrinsics recently, we'll
likely have to have a hard cutover to opaque pointers. Textual IR may be
able to be auto-upgraded for a time, and Bitcode certainly will be - so, in
the worst case, if a downstream project needed to pickup a newer version of
LLVM but didn't have time to invest it migrating their IR generation -
maybe they could get away with linking an older LLVM into their frontend,
then serializing roundtrip via bitcode back into a process with the newer
LLVM that would convert to opaque pointers on loading.

Is that about right?

- Dave


>
> Regards,
> Nikita
>
>
>>  *From:* llvm-dev <llvm-dev-bounces at lists.llvm.org> *On Behalf
Of *Nikita
>> Popov via llvm-dev
>> *Sent:* Wednesday, June 30, 2021 2:02 PM
>> *To:* David Blaikie <dblaikie at gmail.com>
>> *Cc:* llvm-dev at lists.llvm.org
>> *Subject:* Re: [llvm-dev] Opaque Pointers Help Wanted
>>
>>
>>
>> On Tue, Jun 22, 2021 at 7:12 PM David Blaikie via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>
>>
>>
>>
>> On Tue, Jun 22, 2021 at 9:39 AM Kaylor, Andrew via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>> > I don't think metadata can reference an LLVM type. My previous
hacky
>> suggestion was to add a new overloaded parameter with the type you want
and
>> pass undef/poison to it. In any case, we'll have to find a way to
fix these
>> sorts of intrinsics, we shouldn't block the opaque pointers project
on some
>> intrinsics.
>>
>>
>>
>> It looks like I just missed your response before getting my response to
>> David in flight. Yes, I see the metadata problem. I don’t like the
>> undef/poison approach, but I agree that it would technically work.
>>
>>
>>
>> I completely agree that we should solve this problem rather than block
>> the opaque pointer project because of it. Nevertheless, we’ll need to
solve
>> the problem.
>>
>>
>>
>>
>>
>> > LLVM already (mostly) treats memory as untyped, what is your
intrinsic
>> attempting to do?
>>
>>
>>
>> It’s kind of like a GEP but more specialized. It’s doing a pointer
>> calculation based on multidimensional composite types with a shape that
is
>> known at runtime but unknown at compile time, and we have a front end
>> that’s able to supply enough info to make this useful.
>>
>>
>>
>>
>>
>> > As for the timeline, we'll have to support mixed opaque
pointers and
>> legacy pointers for a while, we don't want out of tree users to
immediately
>> break as soon as the opaque pointers work is finished up in-tree.
>>
>>
>>
>> Is there any consensus on the scope of “for a while”? Like, how many
>> major releases after the opaque pointer work is complete? My manager
would
>> very much like to know the answer to this question. :-) I’ve been
trying to
>> prepare for the buffer being as little as one major release after the
>> in-tree work is done but hoping for more. I expect there are others
with a
>> similar interest.
>>
>>
>>
>> By default I'd vote for one major release - but likely the ongoing
burden
>> of carrying it for a bit longer if some significant contributors would
>> benefit from extra time it's probably going to be pretty harmless
to keep
>> it around a bit longer, I think?
>>
>>
>>
>> Opaque pointers are not like the pass manager switch, where we can
retain
>> support for the legacy pass manager at close to zero cost. Nearly all
tests
>> work the same on both pass managers, so there is little ongoing
maintenance
>> cost.
>>
>>
>>
>> This is not going to be the case with opaque pointers. Doing the switch
>> will require changes to nearly the whole test suite. This means that
either
>> typed pointers will end up being entirely untested (or very weakly
tested),
>> or we need to duplicate a very large fraction of our tests.
>>
>>
>>
>> My expectation is that after the opaque pointer migration is complete,
>> the time the non-opaque pointer mode remains available will be measured
in
>> days, not years.
>>
>>
>>
>> Regards,
>>
>> Nikita
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *From:* Arthur Eubanks <aeubanks at google.com>
>> *Sent:* Tuesday, June 22, 2021 12:15 PM
>> *To:* Kaylor, Andrew <andrew.kaylor at intel.com>
>> *Cc:* llvm-dev at lists.llvm.org
>> *Subject:* Re: [llvm-dev] Opaque Pointers Help Wanted
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Jun 21, 2021 at 3:27 PM Kaylor, Andrew <andrew.kaylor at
intel.com>
>> wrote:
>>
>> This has probably been discussed somewhere, but I missed it. Can you
>> elaborate a bit on this?
>>
>>
>>
>>    - Allow bitcode auto-upgrade of legacy pointer type to the new
opaque
>>    pointer type (not to be turned on until ready)
>>
>>
>>    - To support legacy bitcode, such as legacy stores/loads, we need to
>>       track pointee types for all values since legacy instructions may
infer the
>>       types from a pointer operand's pointee type
>>
>> I‘m specifically trying to understand what will happen when typed
pointer
>> support is removed. How will IR with typed pointers be auto-upgraded to
>> pure opaque pointer IR? Will the bitcode reader keep some level of
typed
>> pointer support indefinitely?
>>
>> Yes, the plan is something along the lines of associating each Value
with
>> a possible pointee type inside the bitcode reader.
>>
>>
>>
>>
>>
>> Also, do you have a plan for replacing intrinsics that currently rely
on
>> pointee types? For example, the load instruction was updated to take an
>> explicit type operand but I don’t think we can do the same thing for an
>> intrinsic like llvm.masked.load since there is Value for Type. This is
an
>> easy problem to work around for something like masked.load, but more
>> complicated if anyone has a downstream GEP-like intrinsic that needs
more
>> than the size of an element (spoiler alert: I do have such an
intrinsic).
>> Would you use a metadata argument?
>>
>> I don't think metadata can reference an LLVM type. My previous
hacky
>> suggestion was to add a new overloaded parameter with the type you want
and
>> pass undef/poison to it. In any case, we'll have to find a way to
fix these
>> sorts of intrinsics, we shouldn't block the opaque pointers project
on some
>> intrinsics.
>>
>>
>>
>> LLVM already (mostly) treats memory as untyped, what is your intrinsic
>> attempting to do?
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
<https://urldefense.com/v3/__https:/lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev__;!!JmoZiZGBv3RvKRSx!qYr6p_UsPommoeDmCaNhSX4hvxR3Uy9ym1IhWIFjgm_GXRDjrn7UxxzTONUejGpYGA$>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
<https://urldefense.com/v3/__https:/lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev__;!!JmoZiZGBv3RvKRSx!qYr6p_UsPommoeDmCaNhSX4hvxR3Uy9ym1IhWIFjgm_GXRDjrn7UxxzTONUejGpYGA$>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210701/d1dbd2bd/attachment-0001.html>