llvm dev - Apr 2020 - lit errors when relying on PYTHONPATH for modules

I apologize for the second message, but I've created a revision with my
proposed solution that works on our end.
I'd appreciate any feedback or guidance if this change is not the right way
to fix my problem.

https://reviews.llvm.org/D78229/new/

Thanks,
J.B.

From: Nagurne, James
Sent: Wednesday, April 15, 2020 1:52 PM
To: llvm-dev at lists.llvm.org
Cc: llvm_cgt at list.ti.com - LLVM Compiler Development (May contain non-TIers)
(llvm_cgt at list.ti.com)
Subject: lit errors when relying on PYTHONPATH for modules

Our team utilizes a local directory for most of our non-standard python modules.
An example of this is 'pygments'.
We put this directory into our PYTHONPATH when we provision our systems for
builds and validation.

When LLVM configuration begins, there's a section of code in
llvm/cmake/config-ix.cmake that goes and looks for 'pygments', among
other modules.
It correctly finds these modules and sets LLVM_HAVE_OPT_VIEWER_MODULES.

This variable is used to initialize a feature checked by the opt-viewer lit
tests. Since we had the modules during LLVM configuration, we will therefore run
the tests.

lit creates a test script that looks something like this:
set -o pipefail;set -x;{ : 'RUN: at line 5';   /usr/bin/python3
/<LLVM_ROOT>/llvm/tools/opt-viewer/opt-viewer.py <options>
...
}

And executes that script with:
/bin/bash <BUILD_DIR>/test/tools/opt-viewer/Output/basic.test.script

Starting a new shell via /bin/bash will not propagate the PYTHONPATH with which
we initially configured LLVM, and the test fails because it can't find
'pygments'. The function which executes this command, executeCommand,
does have an 'env' parameter, but the argument sent in is usually
'test.config.environment', which does not include PYTHONPATH.

Are we missing something fundamental in our setup?
There's a list of host variables that get propagated into the test suite
(with_system_environment) in lit.cfg.py. Should we add "PYTHONPATH" to
the list of variables propagated in llvm/test/lit.cfg.py?

Thanks,
J.B.


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200415/906519a0/attachment.html>