llvm dev - Nov 2016 - Running "different tests" on cmake based test-suite build

Hello Friends,

With configure based build deprecated on llvm, how can I run different 
tests/pass on the suite 
(http://llvm.org/docs/TestSuiteMakefileGuide.html#running-different-tests)?

Right now I am having a hacky solution, which includes running the 
"still present" configure file of test-suite to generate the 
MAKE.<MYPASS>.Makefile;
> cd external-testsuite-build-loc
>
> <llvmsrc>/project/test-suite/configure --enable-optimized 
> -disable-assertions
>
but while running 'make TEST=PASS' I am getting error for trying to 
include $(LLVM_OBJ_ROOT)/Makefile.config which is obviously not there on 
a cmake based llvm build. I have to edit the generated makefile to 
bypass this.

Can you please point me to some clean solution or let me know if am 
missing something?


-- 
*With Thanks and Regards*
Sandeep Dasgupta
Research Assistant CS @UIUC
(http://web.engr.illinois.edu/~sdasgup3/)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161123/d32d163d/attachment.html>

Hi Sandeep,

The CMake version of the test-suite has been improved (see
https://reviews.llvm.org/D21360) so that it’s easy to plug additional test
suites. A starting point would be to look at how the TEST_SUITE_SUBDIRS variable
in the top level CMakeLists.txt is used.

Kind regards,
Arnaud

From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of Sandeep
Dasgupta via llvm-dev
Sent: 24 November 2016 02:32
To: llvm-dev at lists.llvm.org
Subject: [llvm-dev] Running "different tests" on cmake based
test-suite build


Hello Friends,

With configure based build deprecated on llvm, how can I run different
tests/pass on the suite
(http://llvm.org/docs/TestSuiteMakefileGuide.html#running-different-tests)?

Right now I am having a hacky solution, which includes running the "still
present" configure file of test-suite to generate the
MAKE.<MYPASS>.Makefile;

cd external-testsuite-build-loc

<llvmsrc>/project/test-suite/configure  --enable-optimized
-disable-assertions

but while running 'make TEST=PASS' I am getting error for trying to
include $(LLVM_OBJ_ROOT)/Makefile.config which is obviously not there on a cmake
based llvm build. I have to edit the generated makefile to bypass this.

Can you please point me to some clean solution or let me know if am missing
something?

--
With Thanks and Regards
Sandeep Dasgupta
Research Assistant CS @UIUC
(http://web.engr.illinois.edu/~sdasgup3/)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161124/c532af8d/attachment.html>

On 11/24/16 3:12 AM, Arnaud De Grandmaison via llvm-dev
wrote:
>
> Hi Sandeep,
>
> The CMake version of the test-suite has been improved (see 
> https://reviews.llvm.org/D21360) so that it’s easy to plug additional 
> test suites. A starting point would be to look at how the 
> TEST_SUITE_SUBDIRS variable in the top level CMakeLists.txt is used.
>
I think this bug report is about adding new programs to the test suite.  
I don't think Sandeep is asking how to add more test programs to the 
test suite; he is asking how to add new Makefiles to the test suite to 
make it run new experiments that it did not run previously.

As context, several previous research projects used an interface within 
the old autoconf test suite to extend it to compile programs with 
different compiler options and run performance experiments on them.  For 
example, the SAFECode and Automatic Pool Allocation projects added rules 
to compile and run programs with additional transforms and to record the 
results in CSV files which could be imported into other programs for 
creating graphs for papers.  All of this could be done without modifying 
any files within the test-suite project; one simply created additional 
Makefiles, set come variables on the make command line, and fired off 
the experiments with "make."

I believe Sandeep is asking whether similar functionality exists within 
the new Cmake build system and, if so, how to use it.  I am curious, 
too.  It would make running some experiments for my research projects 
more convenient.  I just assumed this feature was deprecated as I got 
the impression that I was the only one still using it.

Regards,

John Criswell
> Kind regards,
>
> Arnaud
>
> *From:*llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On Behalf Of 
> *Sandeep Dasgupta via llvm-dev
> *Sent:* 24 November 2016 02:32
> *To:* llvm-dev at lists.llvm.org
> *Subject:* [llvm-dev] Running "different tests" on cmake based 
> test-suite build
>
> Hello Friends,
>
> With configure based build deprecated on llvm, how can I run different 
> tests/pass on the suite 
> (http://llvm.org/docs/TestSuiteMakefileGuide.html#running-different-tests)?
>
>
> Right now I am having a hacky solution, which includes running the 
> "still present" configure file of test-suite to generate the 
> MAKE.<MYPASS>.Makefile;
>
>     cd external-testsuite-build-loc
>
>     <llvmsrc>/project/test-suite/configure --enable-optimized
>     -disable-assertions
>
> but while running 'make TEST=PASS' I am getting error for trying to
> include $(LLVM_OBJ_ROOT)/Makefile.config which is obviously not there 
> on a cmake based llvm build. I have to edit the generated makefile to 
> bypass this.
>
> Can you please point me to some clean solution or let me know if am 
> missing something?
>
> -- 
> *With Thanks and Regards*
> Sandeep Dasgupta
> Research Assistant CS @UIUC
> (http://web.engr.illinois.edu/~sdasgup3/ 
> <http://web.engr.illinois.edu/%7Esdasgup3/>)
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-- 
John Criswell
Assistant Professor
Department of Computer Science, University of Rochester
http://www.cs.rochester.edu/u/criswell

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161124/4287ae88/attachment.html>