similar to: [LLVMdev] test-suite RunSafely.sh remote execution with test copy

Dan/Daniel/Eric, I'm testing a bare-metal ARM toolchain, and I've hacked up my local copy of lit to get it to run libcxx tests on qemu. I wanted to pick your collective brains to see if there was a better way of doing this. What I have implemented is here (don't take this as a formal review request, we can do that later once we figure out the best direction to go): lit part:

If you look at http://npt.cc.rsu.ru/testresults-X86-FreeBSD/index.html then you see that all program test have '*' in run time columns. I think problem in using plain 'time' command in RunSafely.sh At FreeBSD 'time <program> <args>' call generate output in format " 0,00 real 0,00 user 0,00 sys" for *.time files instead expected

Daniel Dunbar wrote: > I am going to change the LLVM test-suite RunSafely.sh to only track > 'user' time, instead of reporting 'user' + 'sys' time as it currently > does. This will probably cause a spike in nightly test numbers, > although hopefully it will be limited to the smaller tests. > > The eventual goal is to report all numbers (independently), so

I am going to change the LLVM test-suite RunSafely.sh to only track 'user' time, instead of reporting 'user' + 'sys' time as it currently does. This will probably cause a spike in nightly test numbers, although hopefully it will be limited to the smaller tests. The eventual goal is to report all numbers (independently), so that we can control for noise better. However,

On Apr 18, 2010, at 11:08 PM, John Criswell wrote: > Second, why are you only interested in user time? The reason why we had > RunSafely.sh measure user + system time is that it gives a more accurate > depiction of how well an optimization works. If a program spends most > of its time in the OS, increasing speed in user-space doesn't gain us > much. If a transform

On Thu, Apr 17, 2014 at 09:57:32AM -0400, Tom Stellard wrote: > On Fri, Apr 11, 2014 at 03:02:21PM -0700, Tom Stellard wrote: > > Hi, > > > > I have just tagged the first release candidate for the > > 3.4.1 release, so testers may begin testing. Please refer to > > http://llvm.org/docs/ReleaseProcess.html for information on how to > > validate a release.

Dear All, Is the lack of virtual memory limits for Darwin in RunSafely.sh intentional or accidential? What about Cygwin? It seems to me that some of the limits should be applied for all operating system targets. -- John T.

Hi all, The LLVM benchmarking system produces very noisy results even on quiet machines. One of the sources of inaccuracy is the timing tool we are using. Because it is a user-space tool, the OS can context switch it and we will get an outlier result. Perf stat uses SW_TASK_CLOCK counter in kernel to measure time, therefore more accurate. It also does not get context switched. I've

In running the test-suite (llvm 3.0), in the *.out.time files in the Output folder the following line appears: sh: time: command not found This occurs when the following lines are executed: #RunSafely.sh ( sh -c "$ULIMITCMD $TIMEIT -p sh -c '$COMMAND >$OUTFILE 2>&1 < $INFILE; echo exit \$?'" ) 2>&1 \ | awk -- '\ BEGIN { cpu = 0.0; } /^user/

The issues seems to be this line in the tools Makefile timeit-target: timeit.c $(LD_ENV_OVERRIDES) $(LCC) -o $@ $< $(LDFLAGS) $(CFLAGS) $(TARGET_FLAGS) -O3 It should not add target flags if we are simulating the target on the host. On 06/06/2013 06:59 PM, reed kotler wrote: > I want to get lnt to use qemu for the execution. > > In that case, RHOST= is not set. > > But I

Hello, I have code containing conditions and loops. Hence some BB execution are determined at run time depending on condition. Now I want to count only those BB that are always executed irrespective of condition result means reachable. and their execution is evident at compile time. How to do this? Please help Thank You Regards -------------- next part -------------- An HTML attachment was

Hi, I'm trying to setup the test-suite and am getting short of running it... Following the steps here: http://llvm.org/docs/lnt/quickstart.html I got as far as getting lnt installed, but when I run the "lnt runtest nt" it fails: compilers.py:81: error: unable to determine compiler version: 'clang++': 'clang version 3.2 ' Seems my string doesn't have a

I hit the same problems as you did when I tried it a few weeks ago. Haven't bothered with it since. There is a way to run tests without using lnt. You can use the Makefiles directly in your test-suite build directory. For running nightly tests, you can do: make report.html TEST=nightly That will generate a html file with the test results in. Amara On 26 October 2012 22:53, Renato Golin

Hi, I am not sure if this is the right forum. Please direct me to the appropriate place if it isn't so. Please keep in CC as i am not subscribed to this mailing list. I am trying to test riscv llvm tools in QEMU using llvm testsuite. As a trial i am trying only the Single Source C Regression folder. The steps that i took are : ]$ cmake -DCMAKE_C_COMPILER=/opt/riscv-tools/bin/clang

I want to get lnt to use qemu for the execution. In that case, RHOST= is not set. But I change the Arch because I am going to run in cross mode. Then I'm setting RUNUNDER to be a script which runs qemu. In this case it builds timeit-target as a Mips which fails because this is running on x86. ~/mysandbox/bin/lnt runtest nt --sandbox ~/mysandbox --cc /local/llvmpb_a/install/bin/clang

On 16 May 2014 18:08, Hal Finkel <hfinkel at anl.gov> wrote: > ----- Original Message ----- >> From: "Yi Kong" <kongy.dev at gmail.com> >> To: "Hal Finkel" <hfinkel at anl.gov>, "Renato Golin" <renato.golin at linaro.org>, "Tobias Grosser" <tobias at grosser.es> >> Cc: "LLVM Dev" <llvmdev at

Eric Christopher <echristo at apple.com> writes: > On Nov 8, 2011, at 7:20 PM, David A. Greene wrote: > > I couldn't figure out how to run the test-suite. Are there up-to-date > instructions somewhere? llvm-gcc doesn't exist anymore... > > Easiest is to check out the test-suite branch into projects and then after you configure and build from the top

On Sat, 11 Sep 2004 13:53:11 -0700 Reid Spencer <reid at x10sys.com> wrote: > On Sat, 2004-09-11 at 12:49, Jeff Cohen wrote: > > > > ===================== MultiSource/Applications/sgefa > > > sgefa is a known XFAIL. See the nightly test results over the last > several months. Actually, you should compare your test results with the > 1.3 release test results

I've set up a public LNT server to show the result of perf stat. There is a huge improvement compared with timeit tool. http://parkas16.inria.fr:8000/ Patch is updated to pin the process to a single core, the readings are even more accurate. It's hard coded to run everything on core 0, so don't run parallel testing with it for now. The tool now depends on Linux perf and schedtool.

----- Original Message ----- > From: "Evan Cheng" <evan.cheng at apple.com> > To: "Hal Finkel" <hfinkel at anl.gov> > Cc: "LLVM Developers Mailing List" <llvmdev at cs.uiuc.edu> > Sent: Monday, May 19, 2014 3:35:45 PM > Subject: Re: [LLVMdev] [RFC] Benchmarking subset of the test suite > > > > > On May 4, 2014, at