llvm dev - Sep 2017 - Different ways of running lit

What are all the different ways people run lit?  I'm doing some refactoring
and want to make sure I have all the based covered.  Obviously you can use
check-llvm.  And you can also run llvm-lit.py in your bin directory and
point it to your source tree.

What else?  Both of the aforementioned methods require running cmake first,
is there any use case where someone runs lit without having run cmake
first?  Or anything else I'm not thinking of?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170911/9142e0c3/attachment.html>

I often run, from my build tree: "./bin/llvm-lit -v
test/DebugInfo/whatever.ll" (where the
"test/DebugInfo/whatever.ll" only
exists in the source tree, not in the build tree where I'm running the
command from)

On Sun, Sep 10, 2017 at 10:33 PM Zachary Turner via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> What are all the different ways people run lit?  I'm doing some
> refactoring and want to make sure I have all the based covered.  Obviously
> you can use check-llvm.  And you can also run llvm-lit.py in your bin
> directory and point it to your source tree.
>
> What else?  Both of the aforementioned methods require running cmake
> first, is there any use case where someone runs lit without having run
> cmake first?  Or anything else I'm not thinking of?
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170911/179269f7/attachment.html>

LNT has some interesting uses of lit.

The lit test suite itself (check-lit) has some interesting ways to invoke
lit that might be worth looking at.

On Sun, Sep 10, 2017 at 10:32 PM, Zachary Turner via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> What are all the different ways people run lit?  I'm doing some
> refactoring and want to make sure I have all the based covered.  Obviously
> you can use check-llvm.  And you can also run llvm-lit.py in your bin
> directory and point it to your source tree.
>
> What else?  Both of the aforementioned methods require running cmake
> first, is there any use case where someone runs lit without having run
> cmake first?  Or anything else I'm not thinking of?
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170911/6efab162/attachment.html>

On 9/11/17 10:48 AM, David Blaikie via llvm-dev wrote:
> I often run, from my build tree: "./bin/llvm-lit -v 
> test/DebugInfo/whatever.ll" (where the
"test/DebugInfo/whatever.ll" only
> exists in the source tree, not in the build tree where I'm running the 
> command from)
I do this a lot too.

@zturner: this is what I was trying to explain on IRC the other day. For 
whatever reason, this setup is the easier one to get working for a new 
testsuite, and the `./bin/llvm-lit /abs/path/to/test.ll` one is harder 
(it can't find the site.cfg for the cases where I've done it).


Jon
> 
> On Sun, Sep 10, 2017 at 10:33 PM Zachary Turner via llvm-dev 
> <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
> 
>     What are all the different ways people run lit?  I'm doing some
>     refactoring and want to make sure I have all the based covered. 
>     Obviously you can use check-llvm.  And you can also run llvm-lit.py
>     in your bin directory and point it to your source tree.
> 
>     What else?  Both of the aforementioned methods require running cmake
>     first, is there any use case where someone runs lit without having
>     run cmake first?  Or anything else I'm not thinking of?
>     _______________________________________________
>     LLVM Developers mailing list
>     llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
-- 
Jon Roelofs
jonathan at codesourcery.com
CodeSourcery / Mentor Embedded / Siemens

Maybe tell us what you want to change?

Indeed lit is used in various ways outside of llvm.

libcxx and the llvm test-suite both have a bigger amount of custom python code
for their lit tests and break easily when changing lit.

- Matthias
> On Sep 10, 2017, at 10:32 PM, Zachary Turner via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> What are all the different ways people run lit?  I'm doing some
refactoring and want to make sure I have all the based covered.  Obviously you
can use check-llvm.  And you can also run llvm-lit.py in your bin directory and
point it to your source tree.
> 
> What else?  Both of the aforementioned methods require running cmake first,
is there any use case where someone runs lit without having run cmake first?  Or
anything else I'm not thinking of?
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

I'm looking at ways of simplifying the lit config files.  Currently, almost
every lit.cfg and lit.site.cfg.in has a ton of copied code.  I kind of
understand why this is necessary architecturally now, so now I'm looking at
how to break it apart.

For example, every single lit config file has a block of code that looks
like this:


# Check that the object root is known.
if config.test_exec_root is None:
    # Otherwise, we haven't loaded the site specific configuration (the
user is
    # probably trying to run on a test file directly, and either the site
    # configuration hasn't been created by the build system, or we are in an
    # out-of-tree build situation).

    # Check for 'lit_site_config' user parameter, and use that if
available.
    site_cfg = lit_config.params.get('lit_site_config', None)
    if site_cfg and os.path.exists(site_cfg):
        lit_config.load_config(config, site_cfg)
        raise SystemExit

    # Try to detect the situation where we are using an out-of-tree build by
    # looking for 'llvm-config'.
    #
    # FIXME: I debated (i.e., wrote and threw away) adding logic to
    # automagically generate the lit.site.cfg if we are in some kind of
fresh
    # build situation. This means knowing how to invoke the build system
though,
    # and I decided it was too much magic. We should solve this by just
having
    # the .cfg files generated during the configuration step.

    llvm_config = lit.util.which('llvm-config',
config.environment['PATH'])
    if not llvm_config:
        lit_config.fatal('No site specific configuration available!')

    # Get the source and object roots.
    llvm_src_root = lit.util.capture(['llvm-config',
'--src-root']).strip()
    llvm_obj_root = lit.util.capture(['llvm-config',
'--obj-root']).strip()
    clang_src_root = os.path.join(llvm_src_root, "tools",
"clang")
    clang_obj_root = os.path.join(llvm_obj_root, "tools",
"clang")

    clang_tools_extra_src_root = os.path.join(clang_src_root, "tools",
"extra")
    clang_tools_extra_obj_root = os.path.join(clang_obj_root, "tools",
"extra")
    # Validate that we got a tree which points to here, using the standard
    # tools/clang layout.
    this_src_root = os.path.dirname(config.test_source_root)
    if os.path.realpath(clang_tools_extra_src_root)
!os.path.realpath(this_src_root):
        lit_config.fatal('No site specific configuration available!')

    # Check that the site specific configuration exists.
    site_cfg = os.path.join(clang_tools_extra_obj_root, 'test',
'lit.site.cfg')
    if not os.path.exists(site_cfg):
        lit_config.fatal(
            'No site specific configuration available! You may need to '
            'run "make test" in your Clang build directory.')

    # Okay, that worked. Notify the user of the automagic, and reconfigure.
    lit_config.note('using out-of-tree build at %r' % clang_obj_root)
    lit_config.load_config(config, site_cfg)
    raise SystemExit

It's obviously copy / pasted, as even the comments are the same.

This is not the only example either, probably 50% of lit.cfg files are copy
/ paste.  I'm trying to eliminate all of this.

Again, I already kind of understand the basic idea of what needs to be
done, I just want to make sure I'm testing all use cases.

On Mon, Sep 11, 2017 at 10:32 AM Matthias Braun <mbraun at apple.com>
wrote:
> Maybe tell us what you want to change?
>
> Indeed lit is used in various ways outside of llvm.
>
> libcxx and the llvm test-suite both have a bigger amount of custom python
> code for their lit tests and break easily when changing lit.
>
> - Matthias
>
> > On Sep 10, 2017, at 10:32 PM, Zachary Turner via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > What are all the different ways people run lit?  I'm doing some
> refactoring and want to make sure I have all the based covered.  Obviously
> you can use check-llvm.  And you can also run llvm-lit.py in your bin
> directory and point it to your source tree.
> >
> > What else?  Both of the aforementioned methods require running cmake
> first, is there any use case where someone runs lit without having run
> cmake first?  Or anything else I'm not thinking of?
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170911/786981f9/attachment.html>