llvm dev - Nov 2021 - Preparing BOLT for LLVM monorepo

Hi All,

Since the acceptance of the "BOLT framework for binary analysis,
transformation, and optimization" RFC
(https://lists.llvm.org/pipermail/llvm-dev/2020-October/145902.html),
we've been working on addressing the feedback and getting ready for
inclusion into the monorepo. We've cleaned up interfaces with core LLVM
libraries, refactored our codebase to match LLVM coding standards, and
added more open-source tests. Additionally, we've made BOLT faster and
more compact.

As of this writing, BOLT could be built as a separate project in the
LLVM monorepo without any patches other than build configuration
changes. You can view the latest sources at the monorepo fork
https://github.com/facebookincubator/BOLT, under "/bolt".

During the past year, we were happy to see an ever-growing inflow of
pull requests from external contributors. We believe that accelerating
the monorepo integration will streamline the open-source review process
and make it more transparent.

While we are finishing the final steps before the integration
(https://github.com/facebookincubator/BOLT/issues/248), we would like to
hear from the community and address any remaining concerns. If
everything goes smoothly, we anticipate the merge to happen early next
month and in time for LLVM 14 release.

We are still working on finalizing the exact logistics of the merge.
However, we expect to follow the Flang project's footsteps and run the
"--no-ff" merge to preserve the history of ~1K commits. We would like
to
ask for help and coordination from the release managers Tom Stellard and
Hans Wennborg.

We haven't made the final decision on including BOLT in
LLVM_ALL_PROJECTS, but we are ready to make BOLT the default project in
the initial merge commit. Currently, we do not support building on
Windows; thus, the project will be automatically disabled with the
warning when building on unsupported platforms. As we anticipate the
increased load on buildbots, we would like to know how we can help with
adding more build machines. Please let us know!

Thank you,
Maksim and BOLT Team
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211102/28541778/attachment.html>

On 11/2/21 1:52 PM, Maksim Panchenko wrote:
> Hi All,
> 
> Since the acceptance of the "BOLT framework for binary analysis,
> 
> transformation, and optimization" RFC
> 
> (https://lists.llvm.org/pipermail/llvm-dev/2020-October/145902.html),
> 
> we've been working on addressing the feedback and getting ready for
> 
> inclusion into the monorepo. We've cleaned up interfaces with core LLVM
> 
> libraries, refactored our codebase to match LLVM coding standards, and
> 
> added more open-source tests. Additionally, we've made BOLT faster and
> 
> more compact.
> 
> As of this writing, BOLT could be built as a separate project in the
> 
> LLVM monorepo without any patches other than build configuration
> 
> changes. You can view the latest sources at the monorepo fork
> 
> https://github.com/facebookincubator/BOLT, under "/bolt".
> 
> During the past year, we were happy to see an ever-growing inflow of
> 
> pull requests from external contributors. We believe that accelerating
> 
> the monorepo integration will streamline the open-source review process
> 
> and make it more transparent.
> 
Since you mentioned pull requests, I just wanted to make sure you are aware
that we don't use pull requests in the monorepo, so you community would need
to transition to doing reviews in Phabricator.  Do you think this will be an
issue?

-Tom
> While we are finishing the final steps before the integration
> 
> (https://github.com/facebookincubator/BOLT/issues/248), we would like to
> 
> hear from the community and address any remaining concerns. If
> 
> everything goes smoothly, we anticipate the merge to happen early next
> 
> month and in time for LLVM 14 release.
> 
> We are still working on finalizing the exact logistics of the merge.
> 
> However, we expect to follow the Flang project's footsteps and run the
> 
> "--no-ff" merge to preserve the history of ~1K commits. We would
like to
> 
> ask for help and coordination from the release managers Tom Stellard and
> 
> Hans Wennborg.
> 
> We haven't made the final decision on including BOLT in
> 
> LLVM_ALL_PROJECTS, but we are ready to make BOLT the default project in
> 
> the initial merge commit. Currently, we do not support building on
> 
> Windows; thus, the project will be automatically disabled with the
> 
> warning when building on unsupported platforms. As we anticipate the
> 
> increased load on buildbots, we would like to know how we can help with
> 
> adding more build machines. Please let us know!
> 
> Thank you,
> 
> Maksim and BOLT Team
>

On Tue, Nov 2, 2021 at 1:52 PM Maksim Panchenko <maks at fb.com> wrote:
> Hi All,
>
>
>
> Since the acceptance of the "BOLT framework for binary analysis,
>
> transformation, and optimization" RFC
>
> (https://lists.llvm.org/pipermail/llvm-dev/2020-October/145902.html),
>
> we've been working on addressing the feedback and getting ready for
>
> inclusion into the monorepo. We've cleaned up interfaces with core LLVM
>
> libraries, refactored our codebase to match LLVM coding standards, and
>
> added more open-source tests. Additionally, we've made BOLT faster and
>
> more compact.
>
I haven't reviewed everything in detail, but on a high-level, that looks
great!
Thanks for spending the effort on making this happen :)

> As of this writing, BOLT could be built as a separate project in the
>
> LLVM monorepo without any patches other than build configuration
>
> changes. You can view the latest sources at the monorepo fork
>
> https://github.com/facebookincubator/BOLT, under "/bolt".
>
>
>
> During the past year, we were happy to see an ever-growing inflow of
>
> pull requests from external contributors. We believe that accelerating
>
> the monorepo integration will streamline the open-source review process
>
> and make it more transparent.
>
>
>
> While we are finishing the final steps before the integration
>
> (https://github.com/facebookincubator/BOLT/issues/248), we would like to
>
> hear from the community and address any remaining concerns. If
>
> everything goes smoothly, we anticipate the merge to happen early next
>
> month and in time for LLVM 14 release.
>
>
>
> We are still working on finalizing the exact logistics of the merge.
>
> However, we expect to follow the Flang project's footsteps and run the
>
> "--no-ff" merge to preserve the history of ~1K commits. We would
like to
>
> ask for help and coordination from the release managers Tom Stellard and
>
> Hans Wennborg.
>
>
>
> We haven't made the final decision on including BOLT in
>
> LLVM_ALL_PROJECTS, but we are ready to make BOLT the default project in
>
> the initial merge commit.
>
I didn't quite get this sentence?

-- 
Mehdi


> Currently, we do not support building on
>
> Windows; thus, the project will be automatically disabled with the
>
> warning when building on unsupported platforms. As we anticipate the
>
> increased load on buildbots, we would like to know how we can help with
>
> adding more build machines. Please let us know!
>
>
>
> Thank you,
>
> Maksim and BOLT Team
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211102/4fd407c9/attachment.html>

Hi Maksim,

I think this sounds great in general.

Making sure things are LLVM coding style, format, etc are pretty important
to me. flang was a useful pilot here in a lot of ways and some of the
requests for fixups still don't really seem to have happened so making sure
that happens before is pretty important to me.

One inline question:

We haven't made the final decision on including BOLT
in
>
> LLVM_ALL_PROJECTS, but we are ready to make BOLT the default project in
>
> the initial merge commit. Currently, we do not support building on
>
> Windows; thus, the project will be automatically disabled with the
>
> warning when building on unsupported platforms. As we anticipate the
>
>
>
Oh? Build or work? And why :)

Thanks and good luck!

-eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211102/2fed430a/attachment-0001.html>

Hi Maksim,

there are a couple of things to do from the infrastructure perspective. So
I started a ticket with the (probably incomplete) list of things to do:
https://github.com/llvm/llvm-iwg/issues/80

Feel free to add more comments/tasks there.

Best,
Christian


On Tue, Nov 2, 2021 at 9:53 PM Maksim Panchenko via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi All,
>
>
>
> Since the acceptance of the "BOLT framework for binary analysis,
>
> transformation, and optimization" RFC
>
> (https://lists.llvm.org/pipermail/llvm-dev/2020-October/145902.html),
>
> we've been working on addressing the feedback and getting ready for
>
> inclusion into the monorepo. We've cleaned up interfaces with core LLVM
>
> libraries, refactored our codebase to match LLVM coding standards, and
>
> added more open-source tests. Additionally, we've made BOLT faster and
>
> more compact.
>
>
>
> As of this writing, BOLT could be built as a separate project in the
>
> LLVM monorepo without any patches other than build configuration
>
> changes. You can view the latest sources at the monorepo fork
>
> https://github.com/facebookincubator/BOLT, under "/bolt".
>
>
>
> During the past year, we were happy to see an ever-growing inflow of
>
> pull requests from external contributors. We believe that accelerating
>
> the monorepo integration will streamline the open-source review process
>
> and make it more transparent.
>
>
>
> While we are finishing the final steps before the integration
>
> (https://github.com/facebookincubator/BOLT/issues/248), we would like to
>
> hear from the community and address any remaining concerns. If
>
> everything goes smoothly, we anticipate the merge to happen early next
>
> month and in time for LLVM 14 release.
>
>
>
> We are still working on finalizing the exact logistics of the merge.
>
> However, we expect to follow the Flang project's footsteps and run the
>
> "--no-ff" merge to preserve the history of ~1K commits. We would
like to
>
> ask for help and coordination from the release managers Tom Stellard and
>
> Hans Wennborg.
>
>
>
> We haven't made the final decision on including BOLT in
>
> LLVM_ALL_PROJECTS, but we are ready to make BOLT the default project in
>
> the initial merge commit. Currently, we do not support building on
>
> Windows; thus, the project will be automatically disabled with the
>
> warning when building on unsupported platforms. As we anticipate the
>
> increased load on buildbots, we would like to know how we can help with
>
> adding more build machines. Please let us know!
>
>
>
> Thank you,
>
> Maksim and BOLT Team
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211103/cd4d1df9/attachment.html>

Hi Maksim,

On Tue, Nov 2, 2021 at 9:52 PM Maksim Panchenko <maks at fb.com>
wrote:
> We are still working on finalizing the exact logistics of the merge.
>
> However, we expect to follow the Flang project's footsteps and run the
>
> "--no-ff" merge to preserve the history of ~1K commits. We would
like to
>
> ask for help and coordination from the release managers Tom Stellard and
>
> Hans Wennborg.
I'm no longer involved in release management, and don't really have
any opinion on how to merge this.

Thanks,
Hans

Hi All,

As I understand, the merge is completed now
(https://github.com/llvm/llvm-project/commit/4c106cfdf7cf7eec861ad3983a3dd9a9e8f3a8ae).

My sincerest congratulations to the Meta team and all BOLT developers!

Hopefully, this will drive further adoption and development of BOLT
and BOLT-based tools!

Yours,
Andrey
==Advanced Software Technology Lab
Huawei

On Tue, Nov 2, 2021 at 11:52 PM Maksim Panchenko <maks at fb.com>
wrote:
>
> Hi All,
>
>
>
> Since the acceptance of the "BOLT framework for binary analysis,
>
> transformation, and optimization" RFC
>
> (https://lists.llvm.org/pipermail/llvm-dev/2020-October/145902.html),
>
> we've been working on addressing the feedback and getting ready for
>
> inclusion into the monorepo. We've cleaned up interfaces with core LLVM
>
> libraries, refactored our codebase to match LLVM coding standards, and
>
> added more open-source tests. Additionally, we've made BOLT faster and
>
> more compact.
>
>
>
> As of this writing, BOLT could be built as a separate project in the
>
> LLVM monorepo without any patches other than build configuration
>
> changes. You can view the latest sources at the monorepo fork
>
> https://github.com/facebookincubator/BOLT, under "/bolt".
>
>
>
> During the past year, we were happy to see an ever-growing inflow of
>
> pull requests from external contributors. We believe that accelerating
>
> the monorepo integration will streamline the open-source review process
>
> and make it more transparent.
>
>
>
> While we are finishing the final steps before the integration
>
> (https://github.com/facebookincubator/BOLT/issues/248), we would like to
>
> hear from the community and address any remaining concerns. If
>
> everything goes smoothly, we anticipate the merge to happen early next
>
> month and in time for LLVM 14 release.
>
>
>
> We are still working on finalizing the exact logistics of the merge.
>
> However, we expect to follow the Flang project's footsteps and run the
>
> "--no-ff" merge to preserve the history of ~1K commits. We would
like to
>
> ask for help and coordination from the release managers Tom Stellard and
>
> Hans Wennborg.
>
>
>
> We haven't made the final decision on including BOLT in
>
> LLVM_ALL_PROJECTS, but we are ready to make BOLT the default project in
>
> the initial merge commit. Currently, we do not support building on
>
> Windows; thus, the project will be automatically disabled with the
>
> warning when building on unsupported platforms. As we anticipate the
>
> increased load on buildbots, we would like to know how we can help with
>
> adding more build machines. Please let us know!
>
>
>
> Thank you,
>
> Maksim and BOLT Team