William Moses via llvm-dev
2021-Jan-26 22:51 UTC
[llvm-dev] RFC: Enzyme, Automatic Differentiation for LLVM as an LLVM Incubator Project
Hi all, Since it seems like all of the feedback here is positive, what would be the next steps (migrate Enzyme mailing list to LLVM, create discord/discourse, etc)? @Renato The split support sounds like a good solution to me. Regarding Enzyme/MLIR, the idea there isn't necessarily to use Enzyme to differentiate MLIR directly, but lowering MLIR to LLVM then running Enzyme could be an interesting (though not necessarily ideal) way to provide differentiable programming in MLIR. We're also considering extending Enzyme to work directly on MLIR as well and while indeed many parts of the analysis are specific to LLVM instructions, other differentiation specific analyses likely will have components which can be shared (e.g. Activity Analysis which determines whether there exists a path through the program that enables differentiable information to flow from input to output). Cheers, Billy On Fri, Jan 15, 2021 at 12:54 PM Renato Golin <rengolin at gmail.com> wrote:> Hi William, > > I think this is a really cool project and worthy of being in LLVM. For now > it's a plugin pass, which goes well with incubator projects, but it could > very well be a standard IR pass that is enabled by flags, etc. We have > similar examples for OpenCL, OpenMP, etc. which need integration on both > Clang and LLVM. Shouldn't be too messy. > > On Tue, 12 Jan 2021 at 22:14, William Moses via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> Obviously integration into the monorepo itself would fix this for >> subsequent versions so it's somewhat of a chicken-and-the egg issue. I'd >> love to hear any thoughts from folks on how something like this might be >> eventually handled. I'd also like to see upstream users (for example to >> differentiate MLIR -- ideally with nice integration for reductions, see >> comment below regarding parallelism). >> > > I think long term you should split support for older versions with the > trunk version. Once (and if) it gets merged into the monorepo, we can keep > the incubation repo with the previous versions only, and only to official > versions, mostly static and for historical value. People should be > encouraged to use the newer versions that have native support. > > Differentiating MLIR would probably be a separate infrastructure, not sure > using the same libraries? I haven't dug too much, but I expect some of the > analysis to be tailored to LLVM's operations and types, which have fixed > semantics and are very different from MLIR, with custom dialects, > operations and types. But it would be really nice if we had that at the > MLIR level, at least for the standard upstream dialects, as it would be a > major boost for ML compilers to start using MLIR more aggressively. > > cheers, > --renato > > -- > You received this message because you are subscribed to the Google Groups > "Enzyme AD" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to enzyme-dev+unsubscribe at googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/enzyme-dev/CAPH-gfeYP1ix4qa4jbGCSMoqY19cce_CV-q-NkjwiuEw2aMpCA%40mail.gmail.com > <https://groups.google.com/d/msgid/enzyme-dev/CAPH-gfeYP1ix4qa4jbGCSMoqY19cce_CV-q-NkjwiuEw2aMpCA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210126/c66384eb/attachment.html>
Renato Golin via llvm-dev
2021-Jan-27 12:05 UTC
[llvm-dev] RFC: Enzyme, Automatic Differentiation for LLVM as an LLVM Incubator Project
On Tue, 26 Jan 2021 at 22:51, William Moses <wmoses at mit.edu> wrote:> Since it seems like all of the feedback here is positive, what would be > the next steps (migrate Enzyme mailing list to LLVM, create > discord/discourse, etc)? >I wouldn't worry about merging the lists too soon (very high traffic). MLIR has a separate channel and that seems to be working for them, you could follow their path at least for now. I couldn't find the code's license, but since you're working with MIT, I imagine it's compatible (and convertible) to the LLVM license. Everything else checks for me, including the migration plan towards the monorepo. Mehdi, does that answer your questions? Should we look for more people to have a look and comment? Alex, perhaps mentioning on the weekly again next week to see if we get more people to look at it? Regarding Enzyme/MLIR, the idea there isn't necessarily to use Enzyme to> differentiate MLIR directly, but lowering MLIR to LLVM then running Enzyme > could be an interesting (though not necessarily ideal) way to provide > differentiable programming in MLIR. We're also considering extending Enzyme > to work directly on MLIR as well and while indeed many parts of the > analysis are specific to LLVM instructions, other differentiation specific > analyses likely will have components which can be shared (e.g. Activity > Analysis which determines whether there exists a path through the program > that enables differentiable information to flow from input to output). >MLIR doesn't always goes to LLVM IR, that's why I suggested it. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210127/cc2dfd8a/attachment.html>