llvm dev - Dec 2012 - [LLVMdev] LNT compile-time performance testing

Hi,

I was looking at LNT's "compile" tests, listed here:

$ lnt-sandbox/bin/lnt showtests
Available tests:
  compile - Single file compile-time performance testing
  nt      - LLVM test-suite compile and execution tests

When trying to run it, I get Python exceptions in the code trying to
parse the output of 'ifconfig' in order to infer my MAC address (this
is on 64-bit Ubuntu 12.04). That parsing code indeed looks strange.

Is the "compile" test in a working condition at all? How should I run
it? (I don't think the LNT docs say much about it, correct me if I'm
wrong). Thanks in advance, Eli.

There is currently no one publicly using the compile tests. We use it internally
around the clock. I am not sure what Daniel's vision for the use of this
externally (i.e. I am not sure how beta/non-beta he thinks this is), but
regardless the way to use it is:

1. Create a directory.
2. Stick a bunch of tar balls you want into the directory.
3. Create a project_list.json file and write a configuration of the form:
{
	"single-file" : {
		"tests" : [
			{
				"path" : "path to single source file",
				"pch_input" : "PCH if there is one"
				"extra_flags": [
					"flag1", "flag2", ...
				]
			},
			{
				...
			}
		],
		"pch" : [
			{
				"path" : "PCH Header",
				"name" : "PCH Name",
				"output": "PCH Output name"
			}
		]
	},
	"projects" : [
		{
			"name" : "NAME",
			"archive": "ARCHIVE",   **I believe only zip/.tar.gz are
supported now,
			"patch_files" : [ "Patch 1 to apply to project",
"Patch 2 to apply to project"],
			"build_info" : {
				"style" : "make",
				"target" : "MAKE RULE" (i.e. all/etc),
				"file": "PATH TO UNZIPPED DIRECTORY" **(i.e. it assumes
you tar balled up a whole project directory so if X.tar.gz is the archive then X
is the directory [this name is not the best but I might be remembering things
incorrectly]),
				"config" : {
					"Release" : {
						"extra_env" : {
							"EXTRA_ENV_VAR" : "EXTRA_ENV_VALUE",
							...
						},
						"extra_args" : [

						],
					},
					"Debug" : {
						SAME AS RELEASE
					}
				}
			}
		},
		{
			NEXT_PROJECT ...
		}
	]
}

Note that currently it only does make and Xcode-project builds (I only showed
the interesting args for the makefiles since I am assuming you are not
interested in Xcode style builds.

On Dec 19, 2012, at 2:05 PM, Eli Bendersky <eliben at google.com> wrote:
> Hi,
> 
> I was looking at LNT's "compile" tests, listed here:
> 
> $ lnt-sandbox/bin/lnt showtests
> Available tests:
>  compile - Single file compile-time performance testing
>  nt      - LLVM test-suite compile and execution tests
> 
> When trying to run it, I get Python exceptions in the code trying to
> parse the output of 'ifconfig' in order to infer my MAC address
(this
> is on 64-bit Ubuntu 12.04). That parsing code indeed looks strange.
> 
> Is the "compile" test in a working condition at all? How should I
run
> it? (I don't think the LNT docs say much about it, correct me if
I'm
> wrong). Thanks in advance, Eli.

On Wed, Dec 19, 2012 at 2:19 PM, Michael Gottesman <mgottesman at
apple.com> wrote:
> There is currently no one publicly using the compile tests. We use it
internally around the clock. I am not sure what Daniel's vision for the use
of this externally (i.e. I am not sure how beta/non-beta he thinks this is), but
regardless the way to use it is:
>
<snip>

Thanks, Michael, I'll look into it.
Eli