llvm dev - Jan 2018 - Migrate utils/ Python 2 scripts to Python 3

Hi LLVM-Devs,

I noticed that many Python scripts under utils/ have a shebang of
`#!/usr/bin/python` (which is a symlink to python2.7 on many platforms) and some
of them use Python 2 syntax that is not compatible with Python 3 (e.g. print
statements; str/bytes)

I created a revision to migrate utils/update_{llc_,}_test_checks.py from Python
2 to Python 3 https://reviews.llvm.org/D42450 , the arguments against making it
Python 3 is:

* There are many Mac OS X users and Mac OS X 10.8 comes with Python 2.7
pre-installed [0] but not Python 3.
* Forcing scripts to use Python 3 (by doing so we can avoid some compatibility
trouble) may not be a good idea.

Python 2.7 was published in 2010 and planned as the last of the 2.x releases. It
will not be maintained past 2020 and there is also a retirement page
https://pythonclock.org/ .

The second argument would not need to be addressed if the first one did not lay
too much burden on developers. After all, we have to install `cmake, ninja/GNU
Make / libedit(for lldb)` to build LLVM. These packages are not installed by
default on many platforms.

Thoughts on deprecating Python 2 for utils/ scripts (different from libclang or
lldb scripts) which are not user-facing?

[0]: https://docs.python.org/3/using/mac.html#getting-and-installing-macpython


Thanks,
Fangrui

Hi Fangrui, for what it's worth regarding lldb, Zachary Turner has already
done a lot of work to make the majority of its python code (most importantly,
its testsuite) work with Python 3 as well as Python 2.

J
> On Jan 29, 2018, at 12:46 PM, Fāng-ruì Sòng via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hi LLVM-Devs,
> 
> I noticed that many Python scripts under utils/ have a shebang of
> `#!/usr/bin/python` (which is a symlink to python2.7 on many platforms) and
some of them use Python 2 syntax that is not compatible with Python 3 (e.g.
print statements; str/bytes)
> 
> I created a revision to migrate utils/update_{llc_,}_test_checks.py from
Python 2 to Python 3 https://reviews.llvm.org/D42450 , the arguments against
making it Python 3 is:
> 
> * There are many Mac OS X users and Mac OS X 10.8 comes with Python 2.7
pre-installed [0] but not Python 3.
> * Forcing scripts to use Python 3 (by doing so we can avoid some
compatibility trouble) may not be a good idea.
> 
> Python 2.7 was published in 2010 and planned as the last of the 2.x
releases. It will not be maintained past 2020 and there is also a retirement
page https://pythonclock.org/ .
> 
> The second argument would not need to be addressed if the first one did not
lay too much burden on developers. After all, we have to install `cmake,
ninja/GNU Make / libedit(for lldb)` to build LLVM. These packages are not
installed by default on many platforms.
> 
> Thoughts on deprecating Python 2 for utils/ scripts (different from
libclang or lldb scripts) which are not user-facing?
> 
> [0]:
https://docs.python.org/3/using/mac.html#getting-and-installing-macpython
> 
> 
> Thanks,
> Fangrui
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

+1 from my for changing LLVMs minimum requirements (for compiling llvm) to
python3 and dropping python2 support.

In case of lit we probably need to keep the code in a way that works with python
2 and 3 though as that will affect various projects outside of core llvm that
started adopting it.

- Matthias
> On Jan 29, 2018, at 3:42 PM, Jason Molenda via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hi Fangrui, for what it's worth regarding lldb, Zachary Turner has
already done a lot of work to make the majority of its python code (most
importantly, its testsuite) work with Python 3 as well as Python 2.
> 
> J
> 
>> On Jan 29, 2018, at 12:46 PM, Fāng-ruì Sòng via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
>> 
>> Hi LLVM-Devs,
>> 
>> I noticed that many Python scripts under utils/ have a shebang of
>> `#!/usr/bin/python` (which is a symlink to python2.7 on many platforms)
and some of them use Python 2 syntax that is not compatible with Python 3 (e.g.
print statements; str/bytes)
>> 
>> I created a revision to migrate utils/update_{llc_,}_test_checks.py
from Python 2 to Python 3 https://reviews.llvm.org/D42450 , the arguments
against making it Python 3 is:
>> 
>> * There are many Mac OS X users and Mac OS X 10.8 comes with Python 2.7
pre-installed [0] but not Python 3.
>> * Forcing scripts to use Python 3 (by doing so we can avoid some
compatibility trouble) may not be a good idea.
>> 
>> Python 2.7 was published in 2010 and planned as the last of the 2.x
releases. It will not be maintained past 2020 and there is also a retirement
page https://pythonclock.org/ .
>> 
>> The second argument would not need to be addressed if the first one did
not lay too much burden on developers. After all, we have to install `cmake,
ninja/GNU Make / libedit(for lldb)` to build LLVM. These packages are not
installed by default on many platforms.
>> 
>> Thoughts on deprecating Python 2 for utils/ scripts (different from
libclang or lldb scripts) which are not user-facing?
>> 
>> [0]:
https://docs.python.org/3/using/mac.html#getting-and-installing-macpython
>> 
>> 
>> Thanks,
>> Fangrui
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Does Python 3 have feature we want to use in LLVM codebase, and no
workaround there? If so, please give some examples. I think that makes the
discussion more concrete.

2018-01-30 4:46 GMT+08:00 Fāng-ruì Sòng via llvm-dev <
llvm-dev at lists.llvm.org>:
> Hi LLVM-Devs,
>
> I noticed that many Python scripts under utils/ have a shebang of
> `#!/usr/bin/python` (which is a symlink to python2.7 on many platforms)
> and some of them use Python 2 syntax that is not compatible with Python 3
> (e.g. print statements; str/bytes)
>
> I created a revision to migrate utils/update_{llc_,}_test_checks.py from
> Python 2 to Python 3 https://reviews.llvm.org/D42450 , the arguments
> against making it Python 3 is:
>
> * There are many Mac OS X users and Mac OS X 10.8 comes with Python 2.7
> pre-installed [0] but not Python 3.
> * Forcing scripts to use Python 3 (by doing so we can avoid some
> compatibility trouble) may not be a good idea.
>
> Python 2.7 was published in 2010 and planned as the last of the 2.x
> releases. It will not be maintained past 2020 and there is also a
> retirement page https://pythonclock.org/ .
>
> The second argument would not need to be addressed if the first one did
> not lay too much burden on developers. After all, we have to install
> `cmake, ninja/GNU Make / libedit(for lldb)` to build LLVM. These packages
> are not installed by default on many platforms.
>
> Thoughts on deprecating Python 2 for utils/ scripts (different from
> libclang or lldb scripts) which are not user-facing?
>
> [0]: https://docs.python.org/3/using/mac.html#getting-and-install
> ing-macpython
>
>
> Thanks,
> Fangrui
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>


-- 
Wei-Ren Chen (陳韋任)
Homepage: https://people.cs.nctu.edu.tw/~chenwj
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180130/dd7c8853/attachment.html>

Personally, every machine I work with only has Python 2.7.

Justin is correct that there is a non-trivial amount of effort to convert the
bots.

Python 3 is wonderful. But, a Python 3 dependency seems like one burden that
could be avoided.  We have already made that trade-off in the past, for example
by only using standard python packages, so there is less/nothing to pip install
when getting started.  Dependencies likes host compiler and cmake totally make
sense given they are central to how LLVM is made.  I don’t think the same for
the python code.

I’m in favor of 2/3 compatibility until the death clock ends.
> On Jan 30, 2018, at 4:48 AM, 陳韋任 via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Does Python 3 have feature we want to use in LLVM codebase, and no
workaround there? If so, please give some examples. I think that makes the
discussion more concrete.
> 
> 2018-01-30 4:46 GMT+08:00 Fāng-ruì Sòng via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>:
> Hi LLVM-Devs,
> 
> I noticed that many Python scripts under utils/ have a shebang of
> `#!/usr/bin/python` (which is a symlink to python2.7 on many platforms) and
some of them use Python 2 syntax that is not compatible with Python 3 (e.g.
print statements; str/bytes)
> 
> I created a revision to migrate utils/update_{llc_,}_test_checks.py from
Python 2 to Python 3 https://reviews.llvm.org/D42450
<https://reviews.llvm.org/D42450> , the arguments against making it Python
3 is:
> 
> * There are many Mac OS X users and Mac OS X 10.8 comes with Python 2.7
pre-installed [0] but not Python 3.
> * Forcing scripts to use Python 3 (by doing so we can avoid some
compatibility trouble) may not be a good idea.
> 
> Python 2.7 was published in 2010 and planned as the last of the 2.x
releases. It will not be maintained past 2020 and there is also a retirement
page https://pythonclock.org/ <https://pythonclock.org/> .
> 
> The second argument would not need to be addressed if the first one did not
lay too much burden on developers. After all, we have to install `cmake,
ninja/GNU Make / libedit(for lldb)` to build LLVM. These packages are not
installed by default on many platforms.
> 
> Thoughts on deprecating Python 2 for utils/ scripts (different from
libclang or lldb scripts) which are not user-facing?
> 
> [0]:
https://docs.python.org/3/using/mac.html#getting-and-installing-macpython
<https://docs.python.org/3/using/mac.html#getting-and-installing-macpython>
> 
> 
> Thanks,
> Fangrui
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
> 
> 
> -- 
> Wei-Ren Chen (陳韋任)
> Homepage: https://people.cs.nctu.edu.tw/~chenwj
<https://people.cs.nctu.edu.tw/~chenwj>_______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180130/cc7791bf/attachment.html>