Hi folks,
I noticed our test-suite bot is failing randomly when getting a timeout on
the install phase:
http://lab.llvm.org:8011/builders/clang-native-arm-lnt/builds/666/steps/venv.lnt.install/logs/stdio
Downloading/unpacking Werkzeug>=0.6.1 (from Flask->LNT==0.4.1dev)
Exception:
File
"/opt/buildbot/clang-native-arm-lnt/lnt.venv/local/lib/python2.7/site-packages/pip-1.2.1-py2.7.egg/pip/download.py",
line 380, in _download_url
chunk = resp.read(4096)
(...)
timeout: timed out
The package getting time out is not always the same, sometimes it also
happens with SQLAlchemy. I'm guessing the server is loaded or the
connection is somehow bad?
cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130420/a2063ce0/attachment.html>
Ping? This is getting as often as every other run... Is that process downloading from an internal server? If not, should we? If yes, should we not? thanks, --renato On 20 April 2013 14:03, Renato Golin <renato.golin at linaro.org> wrote:> Hi folks, > > I noticed our test-suite bot is failing randomly when getting a timeout on > the install phase: > > > http://lab.llvm.org:8011/builders/clang-native-arm-lnt/builds/666/steps/venv.lnt.install/logs/stdio > > Downloading/unpacking Werkzeug>=0.6.1 (from Flask->LNT==0.4.1dev) > Exception: > File > "/opt/buildbot/clang-native-arm-lnt/lnt.venv/local/lib/python2.7/site-packages/pip-1.2.1-py2.7.egg/pip/download.py", > line 380, in _download_url > chunk = resp.read(4096) > (...) > timeout: timed out > > The package getting time out is not always the same, sometimes it also > happens with SQLAlchemy. I'm guessing the server is loaded or the > connection is somehow bad? > > cheers, > --renato >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130422/f963e2dc/attachment.html>
Hey Renato, I'll take a look this morning and now to fetching from an internal server so this doesn't happen again. - Daniel On Apr 22, 2013, at 8:00, Renato Golin <renato.golin at linaro.org> wrote: Ping? This is getting as often as every other run... Is that process downloading from an internal server? If not, should we? If yes, should we not? thanks, --renato On 20 April 2013 14:03, Renato Golin <renato.golin at linaro.org> wrote:> Hi folks, > > I noticed our test-suite bot is failing randomly when getting a timeout on > the install phase: > > > http://lab.llvm.org:8011/builders/clang-native-arm-lnt/builds/666/steps/venv.lnt.install/logs/stdio > > Downloading/unpacking Werkzeug>=0.6.1 (from Flask->LNT==0.4.1dev) > Exception: > File > "/opt/buildbot/clang-native-arm-lnt/lnt.venv/local/lib/python2.7/site-packages/pip-1.2.1-py2.7.egg/pip/download.py", > line 380, in _download_url > chunk = resp.read(4096) > (...) > timeout: timed out > > The package getting time out is not always the same, sometimes it also > happens with SQLAlchemy. I'm guessing the server is loaded or the > connection is somehow bad? > > cheers, > --renato >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130422/60301024/attachment.html>
On Mon, Apr 22, 2013 at 8:00 AM, Renato Golin <renato.golin at linaro.org>wrote:> Ping? > > This is getting as often as every other run... Is that process downloading > from an internal server? If not, should we? If yes, should we not? >Ok, so we are already downloading from a private server (lab.llvm.org), which IMHO we should be doing because we want tight control over the dependencies etc. I forgot that your machine is outside the lab, so that may be involved here. I don't fully understand the networking setup but it is very different outside vs inside. That said, I ran some timings from our network here to lab.llvm.org, and I couldn't reproduce any connectivity issues from here. Renato, can you try running some testing on your side to quantify where the connectivity issue is? Things like: $ ab -c 3 -n 100 http://lab.llvm.org/packages/SQLAlchemy-0.6.6.tar.gz work fine from here. That said, it *would* be nice to harden this code against that failure mode -- one reasonable way would be to have the buildbot slave mirror the packages directory locally (using rsync or so) and do the install from there. We can make the rsync command not cause a buildbot failure, since we rarely change the packages. Any interest in making these changes to the buildbot config? - Daniel> thanks, > --renato > > > On 20 April 2013 14:03, Renato Golin <renato.golin at linaro.org> wrote: > >> Hi folks, >> >> I noticed our test-suite bot is failing randomly when getting a timeout >> on the install phase: >> >> >> http://lab.llvm.org:8011/builders/clang-native-arm-lnt/builds/666/steps/venv.lnt.install/logs/stdio >> >> Downloading/unpacking Werkzeug>=0.6.1 (from Flask->LNT==0.4.1dev) >> Exception: >> File >> "/opt/buildbot/clang-native-arm-lnt/lnt.venv/local/lib/python2.7/site-packages/pip-1.2.1-py2.7.egg/pip/download.py", >> line 380, in _download_url >> chunk = resp.read(4096) >> (...) >> timeout: timed out >> >> The package getting time out is not always the same, sometimes it also >> happens with SQLAlchemy. I'm guessing the server is loaded or the >> connection is somehow bad? >> >> cheers, >> --renato >> > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130422/9b98a779/attachment.html>