Serge Guelton via llvm-dev
2020-Jan-29 10:26 UTC
[llvm-dev] [RFC] Python 2 / Python 3 status
Hi folks, Python2 has reached end of support[0], and many core Python packages are dropping pthon 2 support[1]. This is a subject that raises periodically on this mlist, with a rather strong no in 2018 [-1] and a slow move in 2019 [-2, -3]. Even if Python is not a core build requirement, it's used during some configurations steps (e.g. in the cmake export_executable_symbols function), for testing, and it is definitevely part of the user experience - several clang tools depend on it, as well as many lldb features. So it's not only about utility scripts. Fedora has moved to Python3 [2], RHEL8 ships Python3 by default [3] (though RHEL7 still ships Python2, but it has reached « End of Full Support »). Latest OSX version also explicitly obsoletes Python2 [4]. Debian always ships both interpreter - even in stable [5] There has been an effort in LLVM/lldb/clang/... to make all Python scripts portable across Python2 and Python3 version [6], but if the documentation and ``GettingStarted.rst`` is to be trust, we only require >=2.7. This may be considered as a maintenance burden, see for instance this discussion [7]. Plus LLVM doesn't have a strong conservative culture, e.g. wrt CMake requirements. My personal take on this would be to start moving forward. Still supporting both version this year, but obsoleting Python 2.7 and requiring, say Python 3.6, starting January 2021 looks like a good compromise. Any thoughts? [-3] https://lists.llvm.org/pipermail/llvm-dev/2019-May/132507.html [-2] http://lists.llvm.org/pipermail/llvm-dev/2019-December/137634.html [-1] http://lists.llvm.org/pipermail/llvm-dev/2018-January/120826.html] [0] https://www.python.org/dev/peps/pep-0373/#update [1] https://python3statement.org/ [2] https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3 [3] https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/8.0_release_notes/index#web-servers-databases-dynamic-languages [4] https://developer.apple.com/documentation/macos_release_notes/macos_catalina_10_15_release_notes#3318257) [5] https://wiki.debian.org/Python [6] https://github.com/llvm/llvm-project/search?q=python+compatibility&unscoped_q=python+compatibility&type=Commits [7] https://reviews.llvm.org/D73011
Brian Gesiak via llvm-dev
2020-Jan-29 13:23 UTC
[llvm-dev] [RFC] Python 2 / Python 3 status
On Wed, Jan 29, 2020 at 5:26 AM Serge Guelton via llvm-dev <llvm-dev at lists.llvm.org> wrote:> My personal take on this would be to start moving forward. Still supporting both > version this year, but obsoleting Python 2.7 and requiring, say Python 3.6, > starting January 2021 looks like a good compromise.I'm relatively new to LLVM so my +1 doesn't count for much, but I am very supportive of this idea! I maintain a downstream fork of LLVM at Facebook, and Python 2 will no longer be made available in our developer environment at some point in 2020. So if LLVM is still maintaining Python 2-compatible programs in 2021, I'll need to install a Python 2 interpreter on a personal computer of mine to test any changes I make, which would be a big inconvenience.> Even if Python is not a core build requirement, it's used during some > configurations steps (e.g. in the cmake export_executable_symbols function), > for testing, and it is definitevely part of the user experience - several clang > tools depend on it, as well as many lldb features. So it's not only about utility > scripts.My understanding is that the build relies upon 232 LLVMBuild.txt files, which are processed by a Python program llvm-build. I think that if one uses CMake to build, then Python is in fact a core build requirement (I'm not sure if gn uses llvm-build?). I don't want to derail the discussion here, but I have been meaning to ask: I recall that it was once used more widely, but these days, why does llvm-build exist? (Again, I'm a beginner in the project, so forgive my ignorance here!) I read through some of the code and from what I understand, one of its main functions is to define the library dependencies of LLVM libraries like LLVMX86CodeGen or LLVMObject, both as global properties in CMake (build/LLVMBuild.cmake), and as a C struct that can be used by llvm-config.cpp (build/tools/llvm-config/LibraryDependencies.inc). But is there a reason these dependencies couldn't/shouldn't be hardcoded, instead of being generated by the llvm-build Python executable? I'm wondering if there's some dynamic build configuration feature of llvm-build that I'm missing. What makes it hard to remove?> Any thoughts?Pardon my tangential questions about llvm-build -- thank you for proposing this timeline! - Brian Gesiak
Reid Kleckner via llvm-dev
2020-Jan-29 18:49 UTC
[llvm-dev] [RFC] Python 2 / Python 3 status
On Wed, Jan 29, 2020 at 2:27 AM Serge Guelton via llvm-dev < llvm-dev at lists.llvm.org> wrote:> My personal take on this would be to start moving forward. Still > supporting both > version this year, but obsoleting Python 2.7 and requiring, say Python 3.6, > starting January 2021 looks like a good compromise. >Sounds good to me. Keeping the window of time during which we support both Python 2 and 3 as small as possible would be nice. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200129/a72bb7a0/attachment.html>
Robinson, Paul via llvm-dev
2020-Jan-29 20:20 UTC
[llvm-dev] [RFC] Python 2 / Python 3 status
I don’t know how Python-infested the build system is, but certainly the basic test suite (lit) is all Python. Just to add to the notes about distros, on Windows it looks like Visual Studio 2017 comes with Python 3.6. --paulr From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Reid Kleckner via llvm-dev Sent: Wednesday, January 29, 2020 1:49 PM To: Serge Guelton <sguelton at redhat.com> Cc: llvm-dev <llvm-dev at lists.llvm.org> Subject: Re: [llvm-dev] [RFC] Python 2 / Python 3 status On Wed, Jan 29, 2020 at 2:27 AM Serge Guelton via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote: My personal take on this would be to start moving forward. Still supporting both version this year, but obsoleting Python 2.7 and requiring, say Python 3.6, starting January 2021 looks like a good compromise. Sounds good to me. Keeping the window of time during which we support both Python 2 and 3 as small as possible would be nice. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200129/e3ede1c5/attachment.html>
Chris Bieneman via llvm-dev
2020-Jan-29 20:29 UTC
[llvm-dev] [RFC] Python 2 / Python 3 status
> On Jan 29, 2020, at 5:23 AM, Brian Gesiak via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > On Wed, Jan 29, 2020 at 5:26 AM Serge Guelton via llvm-dev > <llvm-dev at lists.llvm.org> wrote: >> My personal take on this would be to start moving forward. Still supporting both >> version this year, but obsoleting Python 2.7 and requiring, say Python 3.6, >> starting January 2021 looks like a good compromise. > > I'm relatively new to LLVM so my +1 doesn't count for much, but I am > very supportive of this idea! I maintain a downstream fork of LLVM at > Facebook, and Python 2 will no longer be made available in our > developer environment at some point in 2020. So if LLVM is still > maintaining Python 2-compatible programs in 2021, I'll need to install > a Python 2 interpreter on a personal computer of mine to test any > changes I make, which would be a big inconvenience. > >> Even if Python is not a core build requirement, it's used during some >> configurations steps (e.g. in the cmake export_executable_symbols function), >> for testing, and it is definitevely part of the user experience - several clang >> tools depend on it, as well as many lldb features. So it's not only about utility >> scripts. > > My understanding is that the build relies upon 232 LLVMBuild.txt > files, which are processed by a Python program llvm-build. I think > that if one uses CMake to build, then Python is in fact a core build > requirement (I'm not sure if gn uses llvm-build?). > > I don't want to derail the discussion here, but I have been meaning to > ask: I recall that it was once used more widely, but these days, why > does llvm-build exist? (Again, I'm a beginner in the project, so > forgive my ignorance here!)llvm-build exists for two reasons, one legacy, one still an issue. Back when we supported both autoconf and CMake llvm-build provided a way to express the build dependency graph in one place and have both build systems respect it. Additionally it generates the tables that drive llvm-config. Deprecating llvm-config in favor of pkg-config and CMake packages has been discussed, but there has been no decision. Removing llvm-build is technically feasible even if we continue supporting llvm-config, but a non-insignificant amount of work to get CMake to generate the headers needed in the llvm-config build. -Chris> I read through some of the code and from what I understand, one of its > main functions is to define the library dependencies of LLVM libraries > like LLVMX86CodeGen or LLVMObject, both as global properties in CMake > (build/LLVMBuild.cmake), and as a C struct that can be used by > llvm-config.cpp (build/tools/llvm-config/LibraryDependencies.inc). But > is there a reason these dependencies couldn't/shouldn't be hardcoded, > instead of being generated by the llvm-build Python executable? > > I'm wondering if there's some dynamic build configuration feature of > llvm-build that I'm missing. What makes it hard to remove? > >> Any thoughts? > > Pardon my tangential questions about llvm-build -- thank you for > proposing this timeline! > > - Brian Gesiak > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Fangrui Song via llvm-dev
2020-Jan-29 20:29 UTC
[llvm-dev] [RFC] Python 2 / Python 3 status
On 2020-01-29, Reid Kleckner via llvm-dev wrote:>On Wed, Jan 29, 2020 at 2:27 AM Serge Guelton via llvm-dev < >llvm-dev at lists.llvm.org> wrote: > >> My personal take on this would be to start moving forward. Still >> supporting both >> version this year, but obsoleting Python 2.7 and requiring, say Python 3.6, >> starting January 2021 looks like a good compromise. >> > >Sounds good to me. Keeping the window of time during which we support both >Python 2 and 3 as small as possible would be nice.One blocker may be shebang, #!/usr/bin/env python may pick either Python 2 or Python 3, if the distributions haven't migrated. Some scripts has dome `#!/usr/bin/env python` -> `#!/usr/bin/env python3` but some have not. If we can update the rest, it may be easier for our overall migration. I also recall Reid said `#!/usr/bin/env python` might make Windows developers happier but I forget the details.>_______________________________________________ >LLVM Developers mailing list >llvm-dev at lists.llvm.org >https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev