llvm dev - Jul 2013 - [LLVMdev] clang searching for many linux directories that do not exist on FreeBSD host

Greetings -

I'm a user of clang (3.3), as it is the system compiler for my
installation of FreeBSD.  (In FreeBSD 10, it will be the default
compiler, but that's not my point.)  My system identifies itself
as:

FreeBSD 9.2-PRERELEASE #0: Tue Jul 16 13:00:08 EDT 2013 
lidl at nine0:/usr/obj/usr/src/sys/GENERIC

Recently, in preparation for the upcoming 9.2 release, they
imported the llvm 3.3 tree.  That works fine for me.

I did notice (while looking at an unrelated problem), that
clan looks around for a bunch of linux directories every time it
is started.  And those directories are never going to be found,
at least not on a FreeBSD system.

It was suggested that I take the issue here, rather than attempting
to fix it with a locally maintained FreeBSD patch.

I've included my trivial test program, and the ktrace output.
I trimmed output to be the legitimate accesses to shared libraries,
etc that clang must make - leaving most of the extraneous accesses.

It was easy to trace these patterns back to the file:
	tools/clang/lib/Driver/ToolChains.cpp

So my question is this:  Is there any easy modification to make that
will allow clang to skip doing all this work for no gain?  It seems
silly to me, when its being used as the system compiler, to have it
call stat() a little over two hundred times, each time
the compiler is started up.

(There's also a very Mac-looking /System/Library/... stat() at the
end too...)

Thanks for any help.

-Kurt


lidl at nine0-309: cat hello.c
#include <stdlib.h>
#include <stdio.h>

int main(int argc, char *argv[])
{
	printf("Hello world!\n");
	return 0;
}
lidl at nine0-310: ktrace -i clang -Wall hello.c
lidl at nine0-311: kdump | egrep -e NAMI -e /usr/lib | awk '{print $4}'
[...]
"/usr/lib64"
"/usr/lib"
"/usr/lib/gcc/x86_64-linux-gnu"
"/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu"
"/usr/lib/x86_64-linux-gnu"
"/usr/lib/gcc/x86_64-unknown-linux-gnu"
"/usr/lib/x86_64-unknown-linux-gnu/gcc/x86_64-unknown-linux-gnu"
"/usr/lib/x86_64-unknown-linux-gnu"
"/usr/lib/gcc/x86_64-pc-linux-gnu"
"/usr/lib/x86_64-pc-linux-gnu/gcc/x86_64-pc-linux-gnu"
"/usr/lib/x86_64-pc-linux-gnu"
"/usr/lib/gcc/x86_64-redhat-linux6E"
"/usr/lib/x86_64-redhat-linux6E/gcc/x86_64-redhat-linux6E"
"/usr/lib/x86_64-redhat-linux6E"
"/usr/lib/gcc/x86_64-redhat-linux"
"/usr/lib/x86_64-redhat-linux/gcc/x86_64-redhat-linux"
"/usr/lib/x86_64-redhat-linux"
"/usr/lib/gcc/x86_64-suse-linux"
"/usr/lib/x86_64-suse-linux/gcc/x86_64-suse-linux"
"/usr/lib/x86_64-suse-linux"
"/usr/lib/gcc/x86_64-manbo-linux-gnu"
"/usr/lib/x86_64-manbo-linux-gnu/gcc/x86_64-manbo-linux-gnu"
"/usr/lib/x86_64-manbo-linux-gnu"
"/usr/lib/gcc/x86_64-linux-gnu"
"/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu"
"/usr/lib/x86_64-linux-gnu"
"/usr/lib/gcc/x86_64-slackware-linux"
"/usr/lib/x86_64-slackware-linux/gcc/x86_64-slackware-linux"
"/usr/lib/x86_64-slackware-linux"
"/usr/lib/gcc/x86_64-unknown-freebsd9.2"
"/usr/lib/x86_64-unknown-freebsd9.2/gcc/x86_64-unknown-freebsd9.2"
"/usr/lib/x86_64-unknown-freebsd9.2"
"/usr/lib32"
"/usr/lib32/gcc/i686-linux-gnu"
"/usr/lib32/i686-linux-gnu/gcc/i686-linux-gnu"
"/usr/lib32/i686-linux-gnu"
"/usr/lib32/gcc/i686-pc-linux-gnu"
"/usr/lib32/i686-pc-linux-gnu/gcc/i686-pc-linux-gnu"
"/usr/lib32/i686-pc-linux-gnu"
"/usr/lib32/gcc/i486-linux-gnu"
"/usr/lib32/i486-linux-gnu/gcc/i486-linux-gnu"
"/usr/lib32/i486-linux-gnu"
"/usr/lib32/gcc/i386-linux-gnu"
"/usr/lib32/i386-linux-gnu/gcc/i386-linux-gnu"
"/usr/lib32/i386-linux-gnu"
"/usr/lib32/gcc/i386-redhat-linux6E"
"/usr/lib32/i386-redhat-linux6E/gcc/i386-redhat-linux6E"
"/usr/lib32/i386-redhat-linux6E"
"/usr/lib32/gcc/i686-redhat-linux"
"/usr/lib32/i686-redhat-linux/gcc/i686-redhat-linux"
"/usr/lib32/i686-redhat-linux"
"/usr/lib32/gcc/i586-redhat-linux"
"/usr/lib32/i586-redhat-linux/gcc/i586-redhat-linux"
"/usr/lib32/i586-redhat-linux"
"/usr/lib32/gcc/i386-redhat-linux"
"/usr/lib32/i386-redhat-linux/gcc/i386-redhat-linux"
"/usr/lib32/i386-redhat-linux"
"/usr/lib32/gcc/i586-suse-linux"
"/usr/lib32/i586-suse-linux/gcc/i586-suse-linux"
"/usr/lib32/i586-suse-linux"
"/usr/lib32/gcc/i486-slackware-linux"
"/usr/lib32/i486-slackware-linux/gcc/i486-slackware-linux"
"/usr/lib32/i486-slackware-linux"
"/usr/lib32/gcc/i686-montavista-linux"
"/usr/lib32/i686-montavista-linux/gcc/i686-montavista-linux"
"/usr/lib32/i686-montavista-linux"
"/usr/lib32/gcc/i386-unknown-freebsd9.2"
"/usr/lib32/i386-unknown-freebsd9.2/gcc/i386-unknown-freebsd9.2"
"/usr/lib32/i386-unknown-freebsd9.2"
"/usr/lib"
"/usr/lib/gcc/i686-linux-gnu"
"/usr/lib/i686-linux-gnu/gcc/i686-linux-gnu"
"/usr/lib/i686-linux-gnu"
"/usr/lib/gcc/i686-pc-linux-gnu"
"/usr/lib/i686-pc-linux-gnu/gcc/i686-pc-linux-gnu"
"/usr/lib/i686-pc-linux-gnu"
"/usr/lib/gcc/i486-linux-gnu"
"/usr/lib/i486-linux-gnu/gcc/i486-linux-gnu"
"/usr/lib/i486-linux-gnu"
"/usr/lib/gcc/i386-linux-gnu"
"/usr/lib/i386-linux-gnu/gcc/i386-linux-gnu"
"/usr/lib/i386-linux-gnu"
"/usr/lib/gcc/i386-redhat-linux6E"
"/usr/lib/i386-redhat-linux6E/gcc/i386-redhat-linux6E"
"/usr/lib/i386-redhat-linux6E"
"/usr/lib/gcc/i686-redhat-linux"
"/usr/lib/i686-redhat-linux/gcc/i686-redhat-linux"
"/usr/lib/i686-redhat-linux"
"/usr/lib/gcc/i586-redhat-linux"
"/usr/lib/i586-redhat-linux/gcc/i586-redhat-linux"
"/usr/lib/i586-redhat-linux"
"/usr/lib/gcc/i386-redhat-linux"
"/usr/lib/i386-redhat-linux/gcc/i386-redhat-linux"
"/usr/lib/i386-redhat-linux"
"/usr/lib/gcc/i586-suse-linux"
"/usr/lib/i586-suse-linux/gcc/i586-suse-linux"
"/usr/lib/i586-suse-linux"
"/usr/lib/gcc/i486-slackware-linux"
"/usr/lib/i486-slackware-linux/gcc/i486-slackware-linux"
"/usr/lib/i486-slackware-linux"
"/usr/lib/gcc/i686-montavista-linux"
"/usr/lib/i686-montavista-linux/gcc/i686-montavista-linux"
"/usr/lib/i686-montavista-linux"
"/usr/lib/gcc/i386-unknown-freebsd9.2"
"/usr/lib/i386-unknown-freebsd9.2/gcc/i386-unknown-freebsd9.2"
"/usr/lib/i386-unknown-freebsd9.2"
"/usr/bin/.."
"/usr/bin/../lib64"
"/usr/bin/../lib"
"/usr/bin/../lib/gcc/x86_64-linux-gnu"
"/usr/bin/../lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu"
"/usr/bin/../lib/x86_64-linux-gnu"
"/usr/bin/../lib/gcc/x86_64-unknown-linux-gnu"
"/usr/bin/../lib/x86_64-unknown-linux-gnu/gcc/x86_64-unknown-linux-gnu"
"/usr/bin/../lib/x86_64-unknown-linux-gnu"
"/usr/bin/../lib/gcc/x86_64-pc-linux-gnu"
"/usr/bin/../lib/x86_64-pc-linux-gnu/gcc/x86_64-pc-linux-gnu"
"/usr/bin/../lib/x86_64-pc-linux-gnu"
"/usr/bin/../lib/gcc/x86_64-redhat-linux6E"
"/usr/bin/../lib/x86_64-redhat-linux6E/gcc/x86_64-redhat-linux6E"
"/usr/bin/../lib/x86_64-redhat-linux6E"
"/usr/bin/../lib/gcc/x86_64-redhat-linux"
"/usr/bin/../lib/x86_64-redhat-linux/gcc/x86_64-redhat-linux"
"/usr/bin/../lib/x86_64-redhat-linux"
"/usr/bin/../lib/gcc/x86_64-suse-linux"
"/usr/bin/../lib/x86_64-suse-linux/gcc/x86_64-suse-linux"
"/usr/bin/../lib/x86_64-suse-linux"
"/usr/bin/../lib/gcc/x86_64-manbo-linux-gnu"
"/usr/bin/../lib/x86_64-manbo-linux-gnu/gcc/x86_64-manbo-linux-gnu"
"/usr/bin/../lib/x86_64-manbo-linux-gnu"
"/usr/bin/../lib/gcc/x86_64-linux-gnu"
"/usr/bin/../lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu"
"/usr/bin/../lib/x86_64-linux-gnu"
"/usr/bin/../lib/gcc/x86_64-slackware-linux"
"/usr/bin/../lib/x86_64-slackware-linux/gcc/x86_64-slackware-linux"
"/usr/bin/../lib/x86_64-slackware-linux"
"/usr/bin/../lib/gcc/x86_64-unknown-freebsd9.2"
"/usr/bin/../lib/x86_64-unknown-freebsd9.2/gcc/x86_64-unknown-freebsd9.2"
"/usr/bin/../lib/x86_64-unknown-freebsd9.2"
"/usr/bin/../lib32"
"/usr/bin/../lib32/gcc/i686-linux-gnu"
"/usr/bin/../lib32/i686-linux-gnu/gcc/i686-linux-gnu"
"/usr/bin/../lib32/i686-linux-gnu"
"/usr/bin/../lib32/gcc/i686-pc-linux-gnu"
"/usr/bin/../lib32/i686-pc-linux-gnu/gcc/i686-pc-linux-gnu"
"/usr/bin/../lib32/i686-pc-linux-gnu"
"/usr/bin/../lib32/gcc/i486-linux-gnu"
"/usr/bin/../lib32/i486-linux-gnu/gcc/i486-linux-gnu"
"/usr/bin/../lib32/i486-linux-gnu"
"/usr/bin/../lib32/gcc/i386-linux-gnu"
"/usr/bin/../lib32/i386-linux-gnu/gcc/i386-linux-gnu"
"/usr/bin/../lib32/i386-linux-gnu"
"/usr/bin/../lib32/gcc/i386-redhat-linux6E"
"/usr/bin/../lib32/i386-redhat-linux6E/gcc/i386-redhat-linux6E"
"/usr/bin/../lib32/i386-redhat-linux6E"
"/usr/bin/../lib32/gcc/i686-redhat-linux"
"/usr/bin/../lib32/i686-redhat-linux/gcc/i686-redhat-linux"
"/usr/bin/../lib32/i686-redhat-linux"
"/usr/bin/../lib32/gcc/i586-redhat-linux"
"/usr/bin/../lib32/i586-redhat-linux/gcc/i586-redhat-linux"
"/usr/bin/../lib32/i586-redhat-linux"
"/usr/bin/../lib32/gcc/i386-redhat-linux"
"/usr/bin/../lib32/i386-redhat-linux/gcc/i386-redhat-linux"
"/usr/bin/../lib32/i386-redhat-linux"
"/usr/bin/../lib32/gcc/i586-suse-linux"
"/usr/bin/../lib32/i586-suse-linux/gcc/i586-suse-linux"
"/usr/bin/../lib32/i586-suse-linux"
"/usr/bin/../lib32/gcc/i486-slackware-linux"
"/usr/bin/../lib32/i486-slackware-linux/gcc/i486-slackware-linux"
"/usr/bin/../lib32/i486-slackware-linux"
"/usr/bin/../lib32/gcc/i686-montavista-linux"
"/usr/bin/../lib32/i686-montavista-linux/gcc/i686-montavista-linux"
"/usr/bin/../lib32/i686-montavista-linux"
"/usr/bin/../lib32/gcc/i386-unknown-freebsd9.2"
"/usr/bin/../lib32/i386-unknown-freebsd9.2/gcc/i386-unknown-freebsd9.2"
"/usr/bin/../lib32/i386-unknown-freebsd9.2"
"/usr/bin/../lib"
"/usr/bin/../lib/gcc/i686-linux-gnu"
"/usr/bin/../lib/i686-linux-gnu/gcc/i686-linux-gnu"
"/usr/bin/../lib/i686-linux-gnu"
"/usr/bin/../lib/gcc/i686-pc-linux-gnu"
"/usr/bin/../lib/i686-pc-linux-gnu/gcc/i686-pc-linux-gnu"
"/usr/bin/../lib/i686-pc-linux-gnu"
"/usr/bin/../lib/gcc/i486-linux-gnu"
"/usr/bin/../lib/i486-linux-gnu/gcc/i486-linux-gnu"
"/usr/bin/../lib/i486-linux-gnu"
"/usr/bin/../lib/gcc/i386-linux-gnu"
"/usr/bin/../lib/i386-linux-gnu/gcc/i386-linux-gnu"
"/usr/bin/../lib/i386-linux-gnu"
"/usr/bin/../lib/gcc/i386-redhat-linux6E"
"/usr/bin/../lib/i386-redhat-linux6E/gcc/i386-redhat-linux6E"
"/usr/bin/../lib/i386-redhat-linux6E"
"/usr/bin/../lib/gcc/i686-redhat-linux"
"/usr/bin/../lib/i686-redhat-linux/gcc/i686-redhat-linux"
"/usr/bin/../lib/i686-redhat-linux"
"/usr/bin/../lib/gcc/i586-redhat-linux"
"/usr/bin/../lib/i586-redhat-linux/gcc/i586-redhat-linux"
"/usr/bin/../lib/i586-redhat-linux"
"/usr/bin/../lib/gcc/i386-redhat-linux"
"/usr/bin/../lib/i386-redhat-linux/gcc/i386-redhat-linux"
"/usr/bin/../lib/i386-redhat-linux"
"/usr/bin/../lib/gcc/i586-suse-linux"
"/usr/bin/../lib/i586-suse-linux/gcc/i586-suse-linux"
"/usr/bin/../lib/i586-suse-linux"
"/usr/bin/../lib/gcc/i486-slackware-linux"
"/usr/bin/../lib/i486-slackware-linux/gcc/i486-slackware-linux"
"/usr/bin/../lib/i486-slackware-linux"
"/usr/bin/../lib/gcc/i686-montavista-linux"
"/usr/bin/../lib/i686-montavista-linux/gcc/i686-montavista-linux"
"/usr/bin/../lib/i686-montavista-linux"
"/usr/bin/../lib/gcc/i386-unknown-freebsd9.2"
"/usr/bin/../lib/i386-unknown-freebsd9.2/gcc/i386-unknown-freebsd9.2"
"/usr/bin/../lib/i386-unknown-freebsd9.2"
[...]
"/System/Library/CoreServices/SystemVersion.plist"
[...]

On Thu, Jul 18, 2013 at 11:04 AM, Kurt Lidl <lidl at pix.net>
wrote:
>
> Greetings -
>
> I'm a user of clang (3.3), as it is the system compiler for my
> installation of FreeBSD.  (In FreeBSD 10, it will be the default
> compiler, but that's not my point.)  My system identifies itself
> as:
>
> FreeBSD 9.2-PRERELEASE #0: Tue Jul 16 13:00:08 EDT 2013
> lidl at nine0:/usr/obj/usr/src/sys/GENERIC
>
> Recently, in preparation for the upcoming 9.2 release, they
> imported the llvm 3.3 tree.  That works fine for me.
>
> I did notice (while looking at an unrelated problem), that
> clan looks around for a bunch of linux directories every time it
> is started.  And those directories are never going to be found,
> at least not on a FreeBSD system.
>
> It was suggested that I take the issue here, rather than attempting
> to fix it with a locally maintained FreeBSD patch.
>
> I've included my trivial test program, and the ktrace output.
> I trimmed output to be the legitimate accesses to shared libraries,
> etc that clang must make - leaving most of the extraneous accesses.
>
> It was easy to trace these patterns back to the file:
>         tools/clang/lib/Driver/ToolChains.cpp
>
> So my question is this:  Is there any easy modification to make that
> will allow clang to skip doing all this work for no gain?  It seems
> silly to me, when its being used as the system compiler, to have it
> call stat() a little over two hundred times, each time
> the compiler is started up.
It's straightforward: you just need to make toolchains::FreeBSD
inherit directly from ToolChain and implement all the methods it would
otherwise inherit from Generic_ELF (which in turn inherits from
Generic_GCC).

-Eli

On 19 Jul 2013, at 02:14, Eli Friedman <eli.friedman at gmail.com> wrote:
> It's straightforward: you just need to make toolchains::FreeBSD
> inherit directly from ToolChain and implement all the methods it would
> otherwise inherit from Generic_ELF (which in turn inherits from
> Generic_GCC).
Wouldn't it make more sense to move the Linux-specific code out of
Generic_GCC and into the Linux toolchain, rather than making all of the other
subclasses of Generic_GCC reimplement the common code?

David