llvm dev - Feb 2018 - Migrate utils/ Python 2 scripts to Python 3

> On Jan 31, 2018, at 10:04 AM, Fāng-ruì Sòng via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Thanks for the information. Then how about standalone scripts (many
> one-file) like utils/update_check_tests.py that are unrelated to lit
> or other important infrastructure? Can they be changed from
> `#!/usr/bin/python2.7` to `#!/usr/bin/python3` shebang?
> 
No. Every machine running macOS has /usr/bin/python and /usr/bin/python2.7 and
because of system integrity protection it isn't even possible to install a
python3 binary into /usr/bin (You can install it to /usr/local/bin though).

-- adrian
> On Tue, Jan 30, 2018 at 11:20 AM, Chris Matthews
> <chris.matthews at apple.com> wrote:
>> Personally, every machine I work with only has Python 2.7.
>> 
>> Justin is correct that there is a non-trivial amount of effort to
convert
>> the bots.
>> 
>> Python 3 is wonderful. But, a Python 3 dependency seems like one burden
that
>> could be avoided.  We have already made that trade-off in the past, for
>> example by only using standard python packages, so there is
less/nothing to
>> pip install when getting started.  Dependencies likes host compiler and
>> cmake totally make sense given they are central to how LLVM is made.  I
>> don’t think the same for the python code.
>> 
>> I’m in favor of 2/3 compatibility until the death clock ends.
>> 
>> 
>> On Jan 30, 2018, at 4:48 AM, 陳韋任 via llvm-dev <llvm-dev at
lists.llvm.org>
>> wrote:
>> 
>> Does Python 3 have feature we want to use in LLVM codebase, and no
>> workaround there? If so, please give some examples. I think that makes
the
>> discussion more concrete.
>> 
>> 2018-01-30 4:46 GMT+08:00 Fāng-ruì Sòng via llvm-dev
>> <llvm-dev at lists.llvm.org>:
>>> 
>>> Hi LLVM-Devs,
>>> 
>>> I noticed that many Python scripts under utils/ have a shebang of
>>> `#!/usr/bin/python` (which is a symlink to python2.7 on many
platforms)
>>> and some of them use Python 2 syntax that is not compatible with
Python 3
>>> (e.g. print statements; str/bytes)
>>> 
>>> I created a revision to migrate utils/update_{llc_,}_test_checks.py
from
>>> Python 2 to Python 3 https://reviews.llvm.org/D42450 , the
arguments against
>>> making it Python 3 is:
>>> 
>>> * There are many Mac OS X users and Mac OS X 10.8 comes with Python
2.7
>>> pre-installed [0] but not Python 3.
>>> * Forcing scripts to use Python 3 (by doing so we can avoid some
>>> compatibility trouble) may not be a good idea.
>>> 
>>> Python 2.7 was published in 2010 and planned as the last of the 2.x
>>> releases. It will not be maintained past 2020 and there is also a
retirement
>>> page https://pythonclock.org/ .
>>> 
>>> The second argument would not need to be addressed if the first one
did
>>> not lay too much burden on developers. After all, we have to
install `cmake,
>>> ninja/GNU Make / libedit(for lldb)` to build LLVM. These packages
are not
>>> installed by default on many platforms.
>>> 
>>> Thoughts on deprecating Python 2 for utils/ scripts (different from
>>> libclang or lldb scripts) which are not user-facing?
>>> 
>>> [0]:
>>>
https://docs.python.org/3/using/mac.html#getting-and-installing-macpython
>>> 
>>> 
>>> Thanks,
>>> Fangrui
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>> 
>> 
>> 
>> --
>> Wei-Ren Chen (陳韋任)
>> Homepage: https://people.cs.nctu.edu.tw/~chenwj
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>> 
> 
> 
> 
> -- 
> 宋方睿
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

The suggested way to do this on OSX is using env:

#!/usr/bin/env python3
> On Jan 31, 2018, at 10:48 AM, Adrian Prantl <aprantl at apple.com>
wrote:
> 
> 
> 
>> On Jan 31, 2018, at 10:04 AM, Fāng-ruì Sòng via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
>> 
>> Thanks for the information. Then how about standalone scripts (many
>> one-file) like utils/update_check_tests.py that are unrelated to lit
>> or other important infrastructure? Can they be changed from
>> `#!/usr/bin/python2.7` to `#!/usr/bin/python3` shebang?
>> 
> 
> No. Every machine running macOS has /usr/bin/python and /usr/bin/python2.7
and because of system integrity protection it isn't even possible to install
a python3 binary into /usr/bin (You can install it to /usr/local/bin though).
> 
> -- adrian
> 
>> On Tue, Jan 30, 2018 at 11:20 AM, Chris Matthews
>> <chris.matthews at apple.com> wrote:
>>> Personally, every machine I work with only has Python 2.7.
>>> 
>>> Justin is correct that there is a non-trivial amount of effort to
convert
>>> the bots.
>>> 
>>> Python 3 is wonderful. But, a Python 3 dependency seems like one
burden that
>>> could be avoided.  We have already made that trade-off in the past,
for
>>> example by only using standard python packages, so there is
less/nothing to
>>> pip install when getting started.  Dependencies likes host compiler
and
>>> cmake totally make sense given they are central to how LLVM is
made.  I
>>> don’t think the same for the python code.
>>> 
>>> I’m in favor of 2/3 compatibility until the death clock ends.
>>> 
>>> 
>>> On Jan 30, 2018, at 4:48 AM, 陳韋任 via llvm-dev <llvm-dev at
lists.llvm.org>
>>> wrote:
>>> 
>>> Does Python 3 have feature we want to use in LLVM codebase, and no
>>> workaround there? If so, please give some examples. I think that
makes the
>>> discussion more concrete.
>>> 
>>> 2018-01-30 4:46 GMT+08:00 Fāng-ruì Sòng via llvm-dev
>>> <llvm-dev at lists.llvm.org>:
>>>> 
>>>> Hi LLVM-Devs,
>>>> 
>>>> I noticed that many Python scripts under utils/ have a shebang
of
>>>> `#!/usr/bin/python` (which is a symlink to python2.7 on many
platforms)
>>>> and some of them use Python 2 syntax that is not compatible
with Python 3
>>>> (e.g. print statements; str/bytes)
>>>> 
>>>> I created a revision to migrate
utils/update_{llc_,}_test_checks.py from
>>>> Python 2 to Python 3 https://reviews.llvm.org/D42450 , the
arguments against
>>>> making it Python 3 is:
>>>> 
>>>> * There are many Mac OS X users and Mac OS X 10.8 comes with
Python 2.7
>>>> pre-installed [0] but not Python 3.
>>>> * Forcing scripts to use Python 3 (by doing so we can avoid
some
>>>> compatibility trouble) may not be a good idea.
>>>> 
>>>> Python 2.7 was published in 2010 and planned as the last of the
2.x
>>>> releases. It will not be maintained past 2020 and there is also
a retirement
>>>> page https://pythonclock.org/ .
>>>> 
>>>> The second argument would not need to be addressed if the first
one did
>>>> not lay too much burden on developers. After all, we have to
install `cmake,
>>>> ninja/GNU Make / libedit(for lldb)` to build LLVM. These
packages are not
>>>> installed by default on many platforms.
>>>> 
>>>> Thoughts on deprecating Python 2 for utils/ scripts (different
from
>>>> libclang or lldb scripts) which are not user-facing?
>>>> 
>>>> [0]:
>>>>
https://docs.python.org/3/using/mac.html#getting-and-installing-macpython
>>>> 
>>>> 
>>>> Thanks,
>>>> Fangrui
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>> 
>>> 
>>> 
>>> 
>>> --
>>> Wei-Ren Chen (陳韋任)
>>> Homepage: https://people.cs.nctu.edu.tw/~chenwj
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>> 
>>> 
>> 
>> 
>> 
>> -- 
>> 宋方睿
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

On 2018-01-31, Chris Matthews wrote:
>The suggested way to do this on OSX is using env:
>
>#!/usr/bin/env python3
Sorry, I use `#!/usr/bin/env python3` in my patch but used #!/usr/bin/python3 in
the email :)
>
>> On Jan 31, 2018, at 10:48 AM, Adrian Prantl <aprantl at
apple.com> wrote:
>>
>>
>>
>>> On Jan 31, 2018, at 10:04 AM, Fāng-ruì Sòng via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>>>
>>> Thanks for the information. Then how about standalone scripts (many
>>> one-file) like utils/update_check_tests.py that are unrelated to
lit
>>> or other important infrastructure? Can they be changed from
>>> `#!/usr/bin/python2.7` to `#!/usr/bin/python3` shebang?
>>>
>>
>> No. Every machine running macOS has /usr/bin/python and
/usr/bin/python2.7 and because of system integrity protection it isn't even
possible to install a python3 binary into /usr/bin (You can install it to
/usr/local/bin though).
>>
>> -- adrian
>>
>>> On Tue, Jan 30, 2018 at 11:20 AM, Chris Matthews
>>> <chris.matthews at apple.com> wrote:
>>>> Personally, every machine I work with only has Python 2.7.
>>>>
>>>> Justin is correct that there is a non-trivial amount of effort
to convert
>>>> the bots.
>>>>
>>>> Python 3 is wonderful. But, a Python 3 dependency seems like
one burden that
>>>> could be avoided.  We have already made that trade-off in the
past, for
>>>> example by only using standard python packages, so there is
less/nothing to
>>>> pip install when getting started.  Dependencies likes host
compiler and
>>>> cmake totally make sense given they are central to how LLVM is
made.  I
>>>> don’t think the same for the python code.
>>>>
>>>> I’m in favor of 2/3 compatibility until the death clock ends.
>>>>
>>>>
>>>> On Jan 30, 2018, at 4:48 AM, 陳韋任 via llvm-dev <llvm-dev at
lists.llvm.org>
>>>> wrote:
>>>>
>>>> Does Python 3 have feature we want to use in LLVM codebase, and
no
>>>> workaround there? If so, please give some examples. I think
that makes the
>>>> discussion more concrete.
>>>>
>>>> 2018-01-30 4:46 GMT+08:00 Fāng-ruì Sòng via llvm-dev
>>>> <llvm-dev at lists.llvm.org>:
>>>>>
>>>>> Hi LLVM-Devs,
>>>>>
>>>>> I noticed that many Python scripts under utils/ have a
shebang of
>>>>> `#!/usr/bin/python` (which is a symlink to python2.7 on
many platforms)
>>>>> and some of them use Python 2 syntax that is not compatible
with Python 3
>>>>> (e.g. print statements; str/bytes)
>>>>>
>>>>> I created a revision to migrate
utils/update_{llc_,}_test_checks.py from
>>>>> Python 2 to Python 3 https://reviews.llvm.org/D42450 , the
arguments against
>>>>> making it Python 3 is:
>>>>>
>>>>> * There are many Mac OS X users and Mac OS X 10.8 comes
with Python 2.7
>>>>> pre-installed [0] but not Python 3.
>>>>> * Forcing scripts to use Python 3 (by doing so we can avoid
some
>>>>> compatibility trouble) may not be a good idea.
>>>>>
>>>>> Python 2.7 was published in 2010 and planned as the last of
the 2.x
>>>>> releases. It will not be maintained past 2020 and there is
also a retirement
>>>>> page https://pythonclock.org/ .
>>>>>
>>>>> The second argument would not need to be addressed if the
first one did
>>>>> not lay too much burden on developers. After all, we have
to install `cmake,
>>>>> ninja/GNU Make / libedit(for lldb)` to build LLVM. These
packages are not
>>>>> installed by default on many platforms.
>>>>>
>>>>> Thoughts on deprecating Python 2 for utils/ scripts
(different from
>>>>> libclang or lldb scripts) which are not user-facing?
>>>>>
>>>>> [0]:
>>>>>
https://docs.python.org/3/using/mac.html#getting-and-installing-macpython
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Fangrui
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Wei-Ren Chen (陳韋任)
>>>> Homepage: https://people.cs.nctu.edu.tw/~chenwj
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> 宋方睿
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-- 
宋方睿