Hello everyone,
I wanted to add some details and motivation for this.
For example, LLVM test-suite uses lit as tool for running tests and benchmarks,
great part of them can execute for quite long time. Some benchmarks with special
set of compiler flags can also have a great compile time. So it can be important
to utilize resources very effectively.
There is already opportunity to choose threads or processes to parallelize in
lit, but this works only for one machine.
There is DRMAA or Distributed Resource Management Application API, which is a
high-level Open Grid Forum API specification for the submission and control of
jobs to a Distributed Resource Management (DRM) system, such as a Cluster or
Grid computing infrastructure. For python there is package which allow to detect
if there is any grids or not, and uses threads/processes on local machine.
https://github.com/pygridtools/gridmap
But one important point, then building should be part of test in LLVM
test-suite. Now there is a mechanism that all tests firstly are built and then
they execute.
Are there any objections for such change from community?
Best regards,
Elena.
From: Elena Lepilkina
Sent: Friday, July 6, 2018 12:39 PM
To: 'llvm-dev at lists.llvm.org' <llvm-dev at lists.llvm.org>
Cc: Sergey Yakushkin <syakush at synopsys.com>
Subject: Running lit tests on grid
Hello,
We are interested in running big sets of lit tests on grid (e.g. test-suite). So
we will be grateful if someone can answer 2 questions:
1. Does anyone try using lit parallelization on any grid engine?
2. What about replacing building of benchmarks in test-suite into special
step in test script?
Best regards,
Elena Lepilkina.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180710/0667d0cf/attachment.html>