Hi,
(bcc: mlir at tensorflow.org FYI)
I am following-up on the integration of MLIR in LLVM as a subproject (Re:
http://lists.llvm.org/pipermail/llvm-dev/2019-October/135687.html ).
We're aiming to integrate into the monorepo next month. Right now our
intent is for MLIR to live in a top-level directory in parallel to clang,
lldb, lld, etc.
Our top option for the integration is to perform a git subtree merge to
bring the MLIR history into the monorepo, here is a prototype:
https://github.com/joker-eph/llvm-project-with-mlir
If you're curious to try it, at the moment it needs a specific CMake
invocation:
cmake -G Ninja ../llvm/ -DLLVM_TARGETS_TO_BUILD="host"
-DLLVM_EXTERNAL_PROJECTS=mlir -DLLVM_EXTERNAL_MLIR_SOURCE_DIR={path to
repo}/mlir/
We'll hook into -DLLVM_ENABLE_PROJECTS after landing.
Let me know if you have any comment about this!
Cheers,
--
Mehdi
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191115/3cc8ead9/attachment.html>
On Fri, Nov 15, 2019 at 5:03 AM Mehdi AMINI via llvm-dev < llvm-dev at lists.llvm.org> wrote:> Hi, > > (bcc: mlir at tensorflow.org FYI) > > I am following-up on the integration of MLIR in LLVM as a subproject (Re: > http://lists.llvm.org/pipermail/llvm-dev/2019-October/135687.html ). > > We're aiming to integrate into the monorepo next month. Right now our > intent is for MLIR to live in a top-level directory in parallel to clang, > lldb, lld, etc. >Sounds right.> Our top option for the integration is to perform a git subtree merge to > bring the MLIR history into the monorepo, here is a prototype: > https://github.com/joker-eph/llvm-project-with-mlir >I'll note that this would be the very-first merge commit on master. I'm not opposed to this, but others may be. (To allow this, we'd temporarily reconfigure github to allow pushing merge-commits, for this one commit, and then disable it again.) However, another issue is that subtree merges have really weird artifacts when trying to look through history, with e.g. git log <filename>. I think I'd really prefer to avoid utilizing a subtree-merge for this. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191115/fe0be7f3/attachment-0001.html>
On Fri, Nov 15, 2019 at 8:54 AM James Y Knight <jyknight at google.com> wrote:> > > On Fri, Nov 15, 2019 at 5:03 AM Mehdi AMINI via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> Hi, >> >> (bcc: mlir at tensorflow.org FYI) >> >> I am following-up on the integration of MLIR in LLVM as a subproject (Re: >> http://lists.llvm.org/pipermail/llvm-dev/2019-October/135687.html ). >> >> We're aiming to integrate into the monorepo next month. Right now our >> intent is for MLIR to live in a top-level directory in parallel to clang, >> lldb, lld, etc. >> > > Sounds right. > > >> Our top option for the integration is to perform a git subtree merge to >> bring the MLIR history into the monorepo, here is a prototype: >> https://github.com/joker-eph/llvm-project-with-mlir >> > > I'll note that this would be the very-first merge commit on master. I'm > not opposed to this, but others may be. (To allow this, we'd temporarily > reconfigure github to allow pushing merge-commits, for this one commit, and > then disable it again.) > > However, another issue is that subtree merges have really weird artifacts > when trying to look through history, with e.g. git log <filename>. I think > I'd really prefer to avoid utilizing a subtree-merge for this. >Right: `git blame` work well though. The alternative would be to have a commit to move the files under a “mlir” directory and then do a normal merge? I am interested to hear about other advices here :) Thanks, Mehdi>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191115/d6f2c7b1/attachment-0001.html>
Since you are going to rewrite the mlir history anyway, you can probably delete accidentally checked in large files if any. * I don't know whether the file CONTRIBUTING.md is still appropriate, at least for the Code of Conduct, LLVM has its own version. * g3doc/ seems a very Google specific name. Does `docs/` work? * bindings/python/pybind.cpp - does it have to be an in-tree plugin? * The Apache 2 license headers are verbose. LLVM uses SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception. On Fri, Nov 15, 2019 at 2:03 AM Mehdi AMINI via llvm-dev <llvm-dev at lists.llvm.org> wrote:> > Hi, > > (bcc: mlir at tensorflow.org FYI) > > I am following-up on the integration of MLIR in LLVM as a subproject (Re: http://lists.llvm.org/pipermail/llvm-dev/2019-October/135687.html ). > > We're aiming to integrate into the monorepo next month. Right now our intent is for MLIR to live in a top-level directory in parallel to clang, lldb, lld, etc. > Our top option for the integration is to perform a git subtree merge to bring the MLIR history into the monorepo, here is a prototype: https://github.com/joker-eph/llvm-project-with-mlir > > If you're curious to try it, at the moment it needs a specific CMake invocation: > > cmake -G Ninja ../llvm/ -DLLVM_TARGETS_TO_BUILD="host" -DLLVM_EXTERNAL_PROJECTS=mlir -DLLVM_EXTERNAL_MLIR_SOURCE_DIR={path to repo}/mlir/ > > We'll hook into -DLLVM_ENABLE_PROJECTS after landing. > > Let me know if you have any comment about this! > > Cheers, > > -- > Mehdi > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-- 宋方睿
On Fri, Nov 15, 2019 at 10:58 AM Fāng-ruì Sòng <maskray at google.com> wrote:> Since you are going to rewrite the mlir history anyway, you can > probably delete accidentally checked in large files if any. >Good point, I checked and this is the largest file in the history of the repo as far as I can tell: https://github.com/joker-eph/llvm-project-with-mlir/blob/master/mlir/g3doc/includes/img/view-operation.svg (155kB)> > * I don't know whether the file CONTRIBUTING.md is still appropriate, > at least for the Code of Conduct, LLVM has its own version. > * g3doc/ seems a very Google specific name. Does `docs/` work? > * bindings/python/pybind.cpp - does it have to be an in-tree plugin? > * The Apache 2 license headers are verbose. LLVM uses > SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception. >Absolutely! All of these points (except the python bindings) are on my TODO list of things to do at the time of the merge. At the moment there is just a script continuously performing the merge in my test repository so this is an exact view of the current state of the public repo. I could also try to rewrite the license in the header in all the history of the repository, but I'm not sure it won't be brittle in practice, I was planning to do an update to all the files before pushing. I'll send the final version of the repo next month, I can CC you if you'd like to review this before we push it? For the python bindings, these are intended to provide some equivalent facility to the LLVM python bindings: https://github.com/joker-eph/llvm-project-with-mlir/tree/master/llvm/bindings/python ; while they need some work at the moment, I think we will want to have bindings though. -- Mehdi> > On Fri, Nov 15, 2019 at 2:03 AM Mehdi AMINI via llvm-dev > <llvm-dev at lists.llvm.org> wrote: > > > > Hi, > > > > (bcc: mlir at tensorflow.org FYI) > > > > I am following-up on the integration of MLIR in LLVM as a subproject > (Re: http://lists.llvm.org/pipermail/llvm-dev/2019-October/135687.html ). > > > > We're aiming to integrate into the monorepo next month. Right now our > intent is for MLIR to live in a top-level directory in parallel to clang, > lldb, lld, etc. > > Our top option for the integration is to perform a git subtree merge to > bring the MLIR history into the monorepo, here is a prototype: > https://github.com/joker-eph/llvm-project-with-mlir > > > > If you're curious to try it, at the moment it needs a specific CMake > invocation: > > > > cmake -G Ninja ../llvm/ -DLLVM_TARGETS_TO_BUILD="host" > -DLLVM_EXTERNAL_PROJECTS=mlir -DLLVM_EXTERNAL_MLIR_SOURCE_DIR={path to > repo}/mlir/ > > > > We'll hook into -DLLVM_ENABLE_PROJECTS after landing. > > > > Let me know if you have any comment about this! > > > > Cheers, > > > > -- > > Mehdi > > > > _______________________________________________ > > LLVM Developers mailing list > > llvm-dev at lists.llvm.org > > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > > > -- > 宋方睿 >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191115/84fa300d/attachment-0001.html>
James Y Knight via llvm-dev <llvm-dev at lists.llvm.org> writes:> However, another issue is that subtree merges have really weird artifacts > when trying to look through history, with e.g. git log <filename>. I think > I'd really prefer to avoid utilizing a subtree-merge for this.I agree. Subtree-merges are really strange when walking back through history. If someone checks out an early commit from mlir, all they will see in their working directory is the mlir sources. All other components will have disappeared. I you want to maintain history, you can look at the scripts James, I and others worked on for the LLVM git migration: https://github.com/jyknight/llvm-git-migration In particular, have a look at this pull request: https://github.com/jyknight/llvm-git-migration/pull/6 This provides a tool to graft an existing git repository into a subdirectory of the monorepo. It has the option of rewriting commits so that checkouts of those commits will maintain the other subprojects alongside. That functionality isn't well tested but I'd be happy to work on it some more if needed. I think this could work well for MLIR. -David
Reasonably Related Threads
- MLIR landing in the monorepo
- MLIR landing in the monorepo
- Google’s TensorFlow team would like to contribute MLIR to the LLVM Foundation
- Google’s TensorFlow team would like to contribute MLIR to the LLVM Foundation
- Google’s TensorFlow team would like to contribute MLIR to the LLVM Foundation