llvm dev - Jul 2020 - [LNT] Build bot problems due to Python dependencies

Hello,

as of Jul 9, the LNT build bot on SystemZ shows red.  The problem occurs in
the "setup lit" test, which fails due to:
Processing dependencies for LNT==0.4.2.dev0
Searching for typing
Reading https://pypi.org/simple/typing/
Downloading
https://files.pythonhosted.org/packages/8f/00/c999df515e923dbb73281426f5032a3d12b68600e362af30ed2c495d1e79/typing-3.7.4.2.tar.gz#sha256=6f6f3553709d2234e412092e5daa93aaaaa42ea1854505442280b39f9311707f

Best match: typing 3.7.4.2
Processing typing-3.7.4.2.tar.gz
Writing /tmp/easy_install-320eoftg/typing-3.7.4.2/setup.cfg
Running typing-3.7.4.2/setup.py -q bdist_egg
--dist-dir /tmp/easy_install-320eoftg/typing-3.7.4.2/egg-dist-tmp-h7gsmgmu
ERROR: You need Python 2.7 or 3.4 to install the typing package.
error: Setup script exited with 1

I'm not really a Python expert, but from reading the docs it appears that
on Python 3.5 and newer, one should should never attempt to install the
"typing" module, as the functionality has been integrated into the
core.
(The sandbox on the build bot system uses Python 3.8.)

And in fact, if I manually remove the "typing" entry in the
install_requires section of lnt/setup.py, the installation (and subsequent
test suite run) goes through.

How is this supposed to work?  Could this be some sort of environment issue
on my machine?



Mit freundlichen Gruessen / Best Regards

Ulrich Weigand

--
  Dr. Ulrich Weigand | Phone: +49-7031/16-3727
  Distinguished Engineer, Open source compilers and toolchain
  IBM Deutschland Research & Development GmbH
  Vors. des Aufsichtsrats: Gregor Pillen | Geschäftsführung: Dirk Wittkopp
  Sitz d. Ges.: Böblingen | Registergericht: Amtsgericht Stuttgart, HRB
243294
  Data Privacy Statement der IBM: https://www.ibm.com/privacy/us/en/
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200710/e240602d/attachment.html>

It is possible we are doing that in some "old way" that is no longer
valid.  LNT does not support python 3.0-3.5, only 3.6 and newer. So it seems odd
that we would not all hit this issue.

Could you provide a link to the failing bot, as well as the python doc you
mentioned?

Did this fail because of a commit, or a was the testing environment changed?

Thanks!
> On Jul 10, 2020, at 5:07 AM, Ulrich Weigand via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hello,
> 
> as of Jul 9, the LNT build bot on SystemZ shows red. The problem occurs in
the "setup lit" test, which fails due to:
> Processing dependencies for LNT==0.4.2.dev0
> Searching for typing
> Reading https://pypi.org/simple/typing/
<https://pypi.org/simple/typing/>
> Downloading
https://files.pythonhosted.org/packages/8f/00/c999df515e923dbb73281426f5032a3d12b68600e362af30ed2c495d1e79/typing-3.7.4.2.tar.gz#sha256=6f6f3553709d2234e412092e5daa93aaaaa42ea1854505442280b39f9311707f
<https://files.pythonhosted.org/packages/8f/00/c999df515e923dbb73281426f5032a3d12b68600e362af30ed2c495d1e79/typing-3.7.4.2.tar.gz#sha256=6f6f3553709d2234e412092e5daa93aaaaa42ea1854505442280b39f9311707f>
> Best match: typing 3.7.4.2
> Processing typing-3.7.4.2.tar.gz
> Writing /tmp/easy_install-320eoftg/typing-3.7.4.2/setup.cfg
> Running typing-3.7.4.2/setup.py -q bdist_egg --dist-dir
/tmp/easy_install-320eoftg/typing-3.7.4.2/egg-dist-tmp-h7gsmgmu
> ERROR: You need Python 2.7 or 3.4 to install the typing package.
> error: Setup script exited with 1
> 
> I'm not really a Python expert, but from reading the docs it appears
that on Python 3.5 and newer, one should should never attempt to install the
"typing" module, as the functionality has been integrated into the
core. (The sandbox on the build bot system uses Python 3.8.)
> 
> And in fact, if I manually remove the "typing" entry in the
install_requires section of lnt/setup.py, the installation (and subsequent test
suite run) goes through.
> 
> How is this supposed to work? Could this be some sort of environment issue
on my machine?
> 
> 
> 
> Mit freundlichen Gruessen / Best Regards
> 
> Ulrich Weigand
> 
> -- 
> Dr. Ulrich Weigand | Phone: +49-7031/16-3727
> Distinguished Engineer, Open source compilers and toolchain
> IBM Deutschland Research & Development GmbH
> Vors. des Aufsichtsrats: Gregor Pillen | Geschäftsführung: Dirk Wittkopp
> Sitz d. Ges.: Böblingen | Registergericht: Amtsgericht Stuttgart, HRB
243294
> Data Privacy Statement der IBM: https://www.ibm.com/privacy/us/en/
<https://www.ibm.com/privacy/us/en/>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200713/aacf9494/attachment.html>

Chris Matthews <chris.matthews at apple.com> wrote on 14.07.2020 00:32:26:
> It is possible we are doing that in some "old way" that is no
longer
> valid.  LNT does not support python 3.0-3.5, only 3.6 and newer. So
> it seems odd that we would not all hit this issue.
>
> Could you provide a link to the failing bot, as well as the python
> doc you mentioned?
http://lab.llvm.org:8011/builders/clang-s390x-linux-lnt

https://pypi.org/project/typing/
> Did this fail because of a commit, or a was the testing environment
changed?

Interestingly enough, as of yesterday (Jul 13) the build bot is green
again.  I notice the following difference in the logs:

When it last used to work, we see:
Downloading
https://files.pythonhosted.org/packages/fe/2e/b480ee1b75e6d17d2993738670e75c1feeb9ff7f64452153cf018051cc92/typing-3.7.4.1-py3-none-any.whl#sha256=f38d83c5a7a7086543a0f649564d661859c5146a85775ab90c0d2f93ffaa9714

Best match: typing 3.7.4.1

While it failed, we see:
Downloading
https://files.pythonhosted.org/packages/8f/00/c999df515e923dbb73281426f5032a3d12b68600e362af30ed2c495d1e79/typing-3.7.4.2.tar.gz#sha256=6f6f3553709d2234e412092e5daa93aaaaa42ea1854505442280b39f9311707f

Best match: typing 3.7.4.2

And since it is working again, we see:
Downloading
https://files.pythonhosted.org/packages/05/d9/6eebe19d46bd05360c9a9aae822e67a80f9242aabbfc58b641b957546607/typing-3.7.4.3.tar.gz#sha256=1187fb9c82fd670d10aa07bbb6cfcfe4bdda42d6fab8d5134f04e8c4d0b71cc9

Best match: typing 3.7.4.3

So this seems a clear indication that this was actually just a
problem with the upstream 3.7.4.2 version of the typing package.

Given that seems appears to have been resolved with 3.7.4.3,
I think we're good again.

Thanks,
Ulrich
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200714/20ed14a7/attachment.html>