Harry Schmalzbauer
2014-Dec-19 16:06 UTC
CPUTYPEs includig avx are suspicious with clang in stable/10
I'm seeing sporadic problems (core dumps) with binaries compiled with CPUTYPE=core-i-avx. They run fine on IvyBridge, but fail on Haswells. Stripping avx from MACHINE_CPU (by defining CPUTYPE=corei7 instead of core-i-avx) solved all crashes on the haswell system. I have no idea if applications like pkg(8) make use of AVX, I'd bet they don't do. So I guess it's something wrong with clang. Unfortuantely I currently don't have a development installation on any haswell system to provide some useful backtraces, but perhaps somebody with compiler knowledge can have a look at this problem? Thanks, -Harry
John Baldwin
2014-Dec-22 14:58 UTC
CPUTYPEs includig avx are suspicious with clang in stable/10
On Friday, December 19, 2014 11:06:32 am Harry Schmalzbauer wrote:> I'm seeing sporadic problems (core dumps) with binaries compiled with > CPUTYPE=core-i-avx. > They run fine on IvyBridge, but fail on Haswells. > Stripping avx from MACHINE_CPU (by defining CPUTYPE=corei7 instead of > core-i-avx) solved all crashes on the haswell system. > > I have no idea if applications like pkg(8) make use of AVX, I'd bet they > don't do. So I guess it's something wrong with clang. > Unfortuantely I currently don't have a development installation on any > haswell system to provide some useful backtraces, but perhaps somebody > with compiler knowledge can have a look at this problem?If you are getting SIGILL (signal 4), all you really need is to get a core (even from a non-debug binary), then use gdb on the core and run 'x/i $rip' at the gdb prompt to see the instruction that triggered the fault. -- John Baldwin
Dimitry Andric
2014-Dec-22 20:49 UTC
CPUTYPEs includig avx are suspicious with clang in stable/10
On 19 Dec 2014, at 17:06, Harry Schmalzbauer <freebsd at omnilan.de> wrote:> > I'm seeing sporadic problems (core dumps) with binaries compiled with > CPUTYPE=core-i-avx. > They run fine on IvyBridge, but fail on Haswells. > Stripping avx from MACHINE_CPU (by defining CPUTYPE=corei7 instead of > core-i-avx) solved all crashes on the haswell system.Can you provide a concrete test case? E.g. binaries with both optimizations, of which one exhibits the crash?> I have no idea if applications like pkg(8) make use of AVX, I'd bet they > don't do.If you put CPUTYPE?=core-i-avx in /etc/make.conf, and recompile ports with that configuration, they can certainly use AVX, if there are parts in the code that can be vectorized (e.g. certain loops).> So I guess it's something wrong with clang. > Unfortuantely I currently don't have a development installation on any > haswell system to provide some useful backtraces, but perhaps somebody > with compiler knowledge can have a look at this problem?It is hard to say anything without more concrete information. :) -Dimitry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 194 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20141222/c939d9e6/attachment.sig>
Jamie Landeg-Jones
2014-Dec-23 04:19 UTC
CPUTYPEs includig avx are suspicious with clang in stable/10
Harry Schmalzbauer <freebsd at omnilan.de> wrote:> I'm seeing sporadic problems (core dumps) with binaries compiled with > CPUTYPE=core-i-avx. > They run fine on IvyBridge, but fail on Haswells. > Stripping avx from MACHINE_CPU (by defining CPUTYPE=corei7 instead of > core-i-avx) solved all crashes on the haswell system.I wonder if it's the same problem I've encountered with a KVM virtual host? In this case, the fault was BMI (BMI2 is ok) I never isolated the cause - my BMI tests worked ok with clang (though they were direct calls to the BMI family, rather than 'stuff clang could use BMI Op-codes stuff for") Maybe you could try with -mno-bmi ? My details: FreeBSD 10.1-PRERELEASE #0: Sun Nov 9 10:07:32 GMT 2014 root at catseye.dyslexicfish.net:/usr/obj/usr/src/sys/CATSEYE amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 CPU: Vultr Virtual CPU 2 (3392.29-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x306c1 Family = 0x6 Model = 0x3c Stepping = 1 Features=0x178bfbfd<FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT> Features2=0x9ffa3203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,HV> AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM> AMD Features2=0x1<LAHF> Structured Extended Features=0xf38<BMI1,HLE,AVX2,BMI2,ERMS,INVPCID,RTM TSC: P-state invariant 3:21 [2] (8) "/tmp" jamie at catseye% printf '#include <stdio.h>\nint main()\n{\nprintf ("boo");\n}\n' | /usr/bin/cc -v -x c -march=native -fsyntax-only -o /dev/null - 2>&1 | awk '($1 == "\"/usr/bin/cc\"") {printf "\n%s\n\n", $0; sub ("^.* -target-cpu ",""); sub (" .*$", ""); print "Clang \"native\" target for this machine: " $0} ($1 == "error:")' "/usr/bin/cc" -cc1 -triple x86_64-unknown-freebsd10.1 -fsyntax-only -disable-free -disable-llvm-verifier -main-file-name - -mrelocation-model static -mdisable-fp-elim -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu core-avx2 -v -resource-dir /usr/bin/../lib/clang/3.4.1 -fdebug-compilation-dir /tmp -ferror-limit 19 -fmessage-length 0 -mstackrealign -fobjc-runtime=gnustep -fdiagnostics-show-option -vectorize-slp -x c - Clang "native" target for this machine: core-avx2 3:21 [2] (9) "/tmp" jamie at catseye% A full test for me shows this works OK: -mtune=core-avx2 -march=core-avx2 -mmmx -msse -msse2 -msse3 -mpclmul -mssse3 -mfma -mcx16 -msse4.1 -msse4.2 -mpopcnt -maes -mavx -mno-bmi -mavx2 -mbmi2 -mrtm (note -mno-nbmi instead of -mbmi) Cheers, Jamie