Peter Waller via llvm-dev
2021-Dec-13 16:50 UTC
[llvm-dev] Preparing BOLT for LLVM monorepo
Just as a heads up, this pulls in a few dotfiles outside the bolt tree. The
`.dockerignore` adding `.git` could conceivably break someone, so you might want
to commit that separately so that it can be discussed/reverted. The other
`.github/workflows` presumably aren’t intended for this repository either(?).
To diff the merge commit’s first parent (upstream llvm) against the merge commit
(assuming bash for brace expansion of `{^1,}`), ignoring the bolt directory:
$ git diff-tree -r 'HEAD^{/Merge BOLT into LLVM}'{^1,} :^bolt
:000000 100644 0000000000000000000000000000000000000000
6b8710a711f3b689885aa5c26c6c06bde348e82b A .dockerignore
:000000 100644 0000000000000000000000000000000000000000
477e9927279e0b6706a48f0d18f834137b92f115 A .github/workflows/Dockerfile
:000000 100644 0000000000000000000000000000000000000000
8deef62d15079ace72ac40db2335acf59d9a7e98 A
.github/workflows/Dockerfile.aarch64
:000000 100644 0000000000000000000000000000000000000000
02ee68ff868f6865825db153ee3aea7d4f76813f A
.github/workflows/docker-image.yml
Regards,
Peter
From: Rafael Auler <rafaelauler at fb.com>
Date: Friday, 10 December 2021 at 04:34
To: Peter Waller <Peter.Waller at arm.com>, Hans Wennborg <hans at
chromium.org>, Maksim Panchenko <maks at fb.com>
Cc: llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>, Chris Lattner
<clattner at llvm.org>, Andrey Bokhanko <andreybokhanko at
gmail.com>, vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at
huawei.com>
Subject: Re: Preparing BOLT for LLVM monorepo
I updated our github repo https://github.com/facebookincubator/BOLT branch
“main” with the strategy that Peter mentioned. I put a merge commit with all
BOLT commits as parent #2, and these commits have an empty root. Because of
that, the merge commit itself does not modify LLVM in any way other than
introducing a new bolt folder with all project files and their histories. Then I
have a small commit on top of that do change LLVM’s CMakeList.txt to add bolt as
a project.
At the moment, I believe the easiest way to review the code remains the one we
have been using during these past months/year: checking our “main” branch and
opening concerns in our github issues page. In case there are no further
suggestions to BOLT, we are targeting pushing the merge commit by the end of
next week.
From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Rafael
Auler via llvm-dev <llvm-dev at lists.llvm.org>
Date: Wednesday, December 8, 2021 at 4:13 PM
To: Peter Waller <Peter.Waller at arm.com>, Hans Wennborg <hans at
chromium.org>, Maksim Panchenko <maks at fb.com>
Cc: llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>, Chris Lattner
<clattner at llvm.org>, Andrey Bokhanko <andreybokhanko at
gmail.com>, vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at
huawei.com>
Subject: Re: [llvm-dev] Preparing BOLT for LLVM monorepo
Peter, thanks for your input, that’s an excellent suggestion. Let me try playing
with that, I’ll try to change our history to use an empty root as you suggested.
Best,
Rafael
From: Peter Waller <Peter.Waller at arm.com>
Date: Wednesday, December 8, 2021 at 2:19 AM
To: Rafael Auler <rafaelauler at fb.com>, Hans Wennborg <hans at
chromium.org>, Maksim Panchenko <maks at fb.com>
Cc: llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>,
vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at huawei.com>,
Andrey Bokhanko <andreybokhanko at gmail.com>, Chris Lattner <clattner
at llvm.org>
Subject: Re: Preparing BOLT for LLVM monorepo
Your approach sounds reasonable to me and looks good from a glance. One thought
to share though.
My read: You take some point in LLVM’s history, then apply a commit which
introduces a bolt directory, then have a few thousand bolt commits, then land a
merge. Correct?
If so, that means if you check out one of those historic commits, you have an
llvm-project directory with BOLT at some commit, and LLVM subprojects always
with the same ‘root’ commit.
Presumably that ‘root’ commit does not work with any commit of BOLT, and anyone
wanting to experiment with old commits (if this is a legitimate use case) would
find that those things are unlikely to work except for maybe recent commits.
Have you considered instead of rebasing onto a recent LLVM commit, taking an
empty root? That way, when a user checks out a historic commit, now they will
only get a bolt directory, and they can supply their own folders for other llvm
subprojects from an appropriate point in time without them conflicting on the
filesystem. Additionally, there is no implication for a user to think that those
directories at those historic checkouts are meaningfully related to the BOLT
commit.
I mention it because that’s what we did for the flang merge in case the same
effect is appropriate for you here.
Regards,
Peter
From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Rafael
Auler via llvm-dev <llvm-dev at lists.llvm.org>
Date: Wednesday, 8 December 2021 at 01:36
To: Hans Wennborg <hans at chromium.org>, Maksim Panchenko <maks at
fb.com>
Cc: llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>,
vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at huawei.com>,
Andrey Bokhanko <andreybokhanko at gmail.com>, Chris Lattner <clattner
at llvm.org>
Subject: Re: [llvm-dev] Preparing BOLT for LLVM monorepo
Hi folks,
https://github.com/facebookincubator/BOLT branch “main” contains a merge
proposal of BOLT into llvm-project. This is llvm from Nov 30th with 1016 commits
on top of it corresponding to the BOLT project.
These 1016 commits would ideally be committed in a merge commit, merging LLVM as
the first parent and BOLT as the second, and would be there only for the
purposes of preserving project history. In this way, they should be easily
skippable during a bisect of LLVM in the same way as the merge commit of flang.
These commits represent the linear history of BOLT on top of rebased LLVM, so
most commits are not buildable (since we can’t build a very old version of BOLT
on top of a recent LLVM base). That’s why this is for history/blame only.
We have addressed the issues in
https://github.com/facebookincubator/BOLT/issues/248 and we are happy to
continue working on any extra suggestions.
Would it be better if we put this branch as a PR into llvm-project as a way to
make it easier for people to review it? I don’t think we can put this into
phabricator. However, I guess github’s bot will probably auto-close the PR. Also
feel free to open new issues against our facebookincubator/BOLT project as a way
to review it.
Thanks
From: Hans Wennborg <hans at chromium.org>
Date: Thursday, November 4, 2021 at 5:42 AM
To: Maksim Panchenko <maks at fb.com>
Cc: Xinliang David Li <xinliangli at gmail.com>, Fāng-ruì Sòng <maskray
at google.com>, Sriraman Tallam <tmsriram at google.com>, Chris Lattner
<clattner at llvm.org>, Andrey Bokhanko <andreybokhanko at
gmail.com>, Rafael Auler <rafaelauler at fb.com>, vladislav.khmelevskyi
at huawei.com <vladislav.khmelevskyi at huawei.com>, tstellar at
redhat.com <tstellar at redhat.com>, joker.eph at gmail.com <joker.eph
at gmail.com>, echristo at gmail.com <echristo at gmail.com>, Nick
Desaulniers <ndesaulniers at google.com>, llvm-dev at lists.llvm.org
<llvm-dev at lists.llvm.org>
Subject: Re: Preparing BOLT for LLVM monorepo
Hi Maksim,
On Tue, Nov 2, 2021 at 9:52 PM Maksim Panchenko <maks at fb.com>
wrote:> We are still working on finalizing the exact logistics of the merge.
>
> However, we expect to follow the Flang project's footsteps and run the
>
> "--no-ff" merge to preserve the history of ~1K commits. We would
like to
>
> ask for help and coordination from the release managers Tom Stellard and
>
> Hans Wennborg.
I'm no longer involved in release management, and don't really have
any opinion on how to merge this.
Thanks,
Hans
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211213/dfce06ec/attachment.html>
Rafael Auler via llvm-dev
2021-Dec-13 18:06 UTC
[llvm-dev] Preparing BOLT for LLVM monorepo
Good catch! Thanks for spotting this, I’ll remove those.
Best,
Rafael
From: Peter Waller <Peter.Waller at arm.com>
Date: Monday, December 13, 2021 at 1:51 PM
To: Rafael Auler <rafaelauler at fb.com>, Hans Wennborg <hans at
chromium.org>, Maksim Panchenko <maks at fb.com>
Cc: llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>, Chris Lattner
<clattner at llvm.org>, Andrey Bokhanko <andreybokhanko at
gmail.com>, vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at
huawei.com>
Subject: Re: Preparing BOLT for LLVM monorepo
Just as a heads up, this pulls in a few dotfiles outside the bolt tree. The
`.dockerignore` adding `.git` could conceivably break someone, so you might want
to commit that separately so that it can be discussed/reverted. The other
`.github/workflows` presumably aren’t intended for this repository either(?).
To diff the merge commit’s first parent (upstream llvm) against the merge commit
(assuming bash for brace expansion of `{^1,}`), ignoring the bolt directory:
$ git diff-tree -r 'HEAD^{/Merge BOLT into LLVM}'{^1,} :^bolt
:000000 100644 0000000000000000000000000000000000000000
6b8710a711f3b689885aa5c26c6c06bde348e82b A .dockerignore
:000000 100644 0000000000000000000000000000000000000000
477e9927279e0b6706a48f0d18f834137b92f115 A .github/workflows/Dockerfile
:000000 100644 0000000000000000000000000000000000000000
8deef62d15079ace72ac40db2335acf59d9a7e98 A
.github/workflows/Dockerfile.aarch64
:000000 100644 0000000000000000000000000000000000000000
02ee68ff868f6865825db153ee3aea7d4f76813f A
.github/workflows/docker-image.yml
Regards,
Peter
From: Rafael Auler <rafaelauler at fb.com>
Date: Friday, 10 December 2021 at 04:34
To: Peter Waller <Peter.Waller at arm.com>, Hans Wennborg <hans at
chromium.org>, Maksim Panchenko <maks at fb.com>
Cc: llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>, Chris Lattner
<clattner at llvm.org>, Andrey Bokhanko <andreybokhanko at
gmail.com>, vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at
huawei.com>
Subject: Re: Preparing BOLT for LLVM monorepo
I updated our github repo https://github.com/facebookincubator/BOLT branch
“main” with the strategy that Peter mentioned. I put a merge commit with all
BOLT commits as parent #2, and these commits have an empty root. Because of
that, the merge commit itself does not modify LLVM in any way other than
introducing a new bolt folder with all project files and their histories. Then I
have a small commit on top of that do change LLVM’s CMakeList.txt to add bolt as
a project.
At the moment, I believe the easiest way to review the code remains the one we
have been using during these past months/year: checking our “main” branch and
opening concerns in our github issues page. In case there are no further
suggestions to BOLT, we are targeting pushing the merge commit by the end of
next week.
From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Rafael
Auler via llvm-dev <llvm-dev at lists.llvm.org>
Date: Wednesday, December 8, 2021 at 4:13 PM
To: Peter Waller <Peter.Waller at arm.com>, Hans Wennborg <hans at
chromium.org>, Maksim Panchenko <maks at fb.com>
Cc: llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>, Chris Lattner
<clattner at llvm.org>, Andrey Bokhanko <andreybokhanko at
gmail.com>, vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at
huawei.com>
Subject: Re: [llvm-dev] Preparing BOLT for LLVM monorepo
Peter, thanks for your input, that’s an excellent suggestion. Let me try playing
with that, I’ll try to change our history to use an empty root as you suggested.
Best,
Rafael
From: Peter Waller <Peter.Waller at arm.com>
Date: Wednesday, December 8, 2021 at 2:19 AM
To: Rafael Auler <rafaelauler at fb.com>, Hans Wennborg <hans at
chromium.org>, Maksim Panchenko <maks at fb.com>
Cc: llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>,
vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at huawei.com>,
Andrey Bokhanko <andreybokhanko at gmail.com>, Chris Lattner <clattner
at llvm.org>
Subject: Re: Preparing BOLT for LLVM monorepo
Your approach sounds reasonable to me and looks good from a glance. One thought
to share though.
My read: You take some point in LLVM’s history, then apply a commit which
introduces a bolt directory, then have a few thousand bolt commits, then land a
merge. Correct?
If so, that means if you check out one of those historic commits, you have an
llvm-project directory with BOLT at some commit, and LLVM subprojects always
with the same ‘root’ commit.
Presumably that ‘root’ commit does not work with any commit of BOLT, and anyone
wanting to experiment with old commits (if this is a legitimate use case) would
find that those things are unlikely to work except for maybe recent commits.
Have you considered instead of rebasing onto a recent LLVM commit, taking an
empty root? That way, when a user checks out a historic commit, now they will
only get a bolt directory, and they can supply their own folders for other llvm
subprojects from an appropriate point in time without them conflicting on the
filesystem. Additionally, there is no implication for a user to think that those
directories at those historic checkouts are meaningfully related to the BOLT
commit.
I mention it because that’s what we did for the flang merge in case the same
effect is appropriate for you here.
Regards,
Peter
From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Rafael
Auler via llvm-dev <llvm-dev at lists.llvm.org>
Date: Wednesday, 8 December 2021 at 01:36
To: Hans Wennborg <hans at chromium.org>, Maksim Panchenko <maks at
fb.com>
Cc: llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>,
vladislav.khmelevskyi at huawei.com <vladislav.khmelevskyi at huawei.com>,
Andrey Bokhanko <andreybokhanko at gmail.com>, Chris Lattner <clattner
at llvm.org>
Subject: Re: [llvm-dev] Preparing BOLT for LLVM monorepo
Hi folks,
https://github.com/facebookincubator/BOLT branch “main” contains a merge
proposal of BOLT into llvm-project. This is llvm from Nov 30th with 1016 commits
on top of it corresponding to the BOLT project.
These 1016 commits would ideally be committed in a merge commit, merging LLVM as
the first parent and BOLT as the second, and would be there only for the
purposes of preserving project history. In this way, they should be easily
skippable during a bisect of LLVM in the same way as the merge commit of flang.
These commits represent the linear history of BOLT on top of rebased LLVM, so
most commits are not buildable (since we can’t build a very old version of BOLT
on top of a recent LLVM base). That’s why this is for history/blame only.
We have addressed the issues in
https://github.com/facebookincubator/BOLT/issues/248 and we are happy to
continue working on any extra suggestions.
Would it be better if we put this branch as a PR into llvm-project as a way to
make it easier for people to review it? I don’t think we can put this into
phabricator. However, I guess github’s bot will probably auto-close the PR. Also
feel free to open new issues against our facebookincubator/BOLT project as a way
to review it.
Thanks
From: Hans Wennborg <hans at chromium.org>
Date: Thursday, November 4, 2021 at 5:42 AM
To: Maksim Panchenko <maks at fb.com>
Cc: Xinliang David Li <xinliangli at gmail.com>, Fāng-ruì Sòng <maskray
at google.com>, Sriraman Tallam <tmsriram at google.com>, Chris Lattner
<clattner at llvm.org>, Andrey Bokhanko <andreybokhanko at
gmail.com>, Rafael Auler <rafaelauler at fb.com>, vladislav.khmelevskyi
at huawei.com <vladislav.khmelevskyi at huawei.com>, tstellar at
redhat.com <tstellar at redhat.com>, joker.eph at gmail.com <joker.eph
at gmail.com>, echristo at gmail.com <echristo at gmail.com>, Nick
Desaulniers <ndesaulniers at google.com>, llvm-dev at lists.llvm.org
<llvm-dev at lists.llvm.org>
Subject: Re: Preparing BOLT for LLVM monorepo
Hi Maksim,
On Tue, Nov 2, 2021 at 9:52 PM Maksim Panchenko <maks at fb.com>
wrote:> We are still working on finalizing the exact logistics of the merge.
>
> However, we expect to follow the Flang project's footsteps and run the
>
> "--no-ff" merge to preserve the history of ~1K commits. We would
like to
>
> ask for help and coordination from the release managers Tom Stellard and
>
> Hans Wennborg.
I'm no longer involved in release management, and don't really have
any opinion on how to merge this.
Thanks,
Hans
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211213/7ca8b6fb/attachment.html>