Kuba Brecka via llvm-dev
2016-Jan-22 15:11 UTC
[llvm-dev] greendragon build noisy due to mmap_stress.cc
Hm, I tried to reproduce this as well, but unsuccessfully. From the crash report: EXC_I386_GPFLT means we’re dereferencing a non-canonical pointer, in this case “0x00486000000025df”. This happens at wrap_OSSpinLockLock+17, which is just after the prologue and just after calling cur_thread(). So I’d say it happens when we’re dereferencing the pointer returned by cur_thread(). On OS X, we’re doing this trick where we store the ThreadState pointer in the shadow memory. Could it be that something actually read/wrote the memory that is backed by the same place as the shadow memory? Does “0x00486000000025df” like a reasonable content of a shadow cell? Kuba> On 21 Jan 2016, at 22:32, Kostya Serebryany <kcc at google.com> wrote: > > +kuba, who ported tsan to OSX. > > On Thu, Jan 21, 2016 at 1:26 PM, Chris Matthews <chris.matthews at apple.com <mailto:chris.matthews at apple.com>> wrote: > Ah ha! I found crash reports: > > > green-dragon-03:DiagnosticReports buildslave$ cat mmap_stress.cc.tmp_2016-01-19-231335_green-dragon-03.crash > Process: mmap_stress.cc.tmp [95010] > Path: /Users/USER/*/mmap_stress.cc.tmp > Identifier: mmap_stress.cc.tmp > Version: 0 > Code Type: X86-64 (Native) > Parent Process: bash [95004] > User ID: 501 > > Date/Time: 2016-01-19 23:13:22.169 -0800 > OS Version: Mac OS X 10.10.5 (14F27) > Report Version: 11 > Anonymous UUID: DFD97AE9-5E0B-44AD-8848-A057000D1FEC > > > Time Awake Since Boot: 9700000 seconds > > Crashed Thread: 21 > > ^^^^^^^^^^^^^^^ > > > Exception Type: EXC_BAD_ACCESS (SIGSEGV) > Exception Codes: EXC_I386_GPFLT > > Thread 0:: Dispatch queue: com.apple.main-thread > 0 libsystem_kernel.dylib 0x00007fff98f915da syscall_thread_switch + 10 > 1 libsystem_platform.dylib 0x00007fff965f982d _OSSpinLockLockSlow + 63 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063d5e4d __sanitizer::BlockingMutex::Lock() + 29 > 3 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063db7bb __sanitizer::ThreadRegistry::CreateThread(unsigned long, bool, unsigned int, void*) + 43 > > Thread 1: > 0 libsystem_kernel.dylib 0x00007fff98f9648a __semwait_signal + 10 > 1 libclang_rt.tsan_osx_dynamic.dylib 0x0000000106393a0e wrap_nanosleep + 94 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063c2bea __tsan::BackgroundThread(void*) + 346 > > Thread 2: > 0 libsystem_kernel.dylib 0x00007fff98f9648a __semwait_signal + 10 > 1 libclang_rt.tsan_osx_dynamic.dylib 0x0000000106395650 wrap_pthread_join + 224 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x000000010639531a __tsan_thread_start_func + 122 > 3 libsystem_pthread.dylib 0x00007fff93b5bfd7 _pthread_start + 176 > 4 libsystem_pthread.dylib 0x00007fff93b593ed thread_start + 13 > > Thread 3: > 0 libsystem_kernel.dylib 0x00007fff98f9648a __semwait_signal + 10 > 1 libclang_rt.tsan_osx_dynamic.dylib 0x0000000106395650 wrap_pthread_join + 224 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x000000010639531a __tsan_thread_start_func + 122 > 3 libsystem_pthread.dylib 0x00007fff93b5bfd7 _pthread_start + 176 > 4 libsystem_pthread.dylib 0x00007fff93b593ed thread_start + 13 > > Thread 4: > 0 libsystem_kernel.dylib 0x00007fff98f915da syscall_thread_switch + 10 > 1 libsystem_platform.dylib 0x00007fff965f982d _OSSpinLockLockSlow + 63 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063d5e4d __sanitizer::BlockingMutex::Lock() + 29 > 3 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063dc076 __sanitizer::ThreadRegistry::FinishThread(unsigned int) + 22 > > Thread 5: > 0 libsystem_kernel.dylib 0x00007fff98f915c2 swtch_pri + 10 > 1 libsystem_pthread.dylib 0x00007fff93b5d381 sched_yield + 11 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063d5946 __sanitizer::internal_sched_yield() + 6 > 3 libclang_rt.tsan_osx_dynamic.dylib 0x0000000106395495 wrap_pthread_create + 341 > 4 ??? 0x0000000000002009 0 + 8201 > > Thread 6: > 0 libsystem_kernel.dylib 0x00007fff98f915c2 swtch_pri + 10 > 1 libsystem_pthread.dylib 0x00007fff93b5d381 sched_yield + 11 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063d5946 __sanitizer::internal_sched_yield() + 6 > 3 libclang_rt.tsan_osx_dynamic.dylib 0x0000000106395495 wrap_pthread_create + 341 > 4 ??? 0x000000000000200c 0 + 8204 > > Thread 7: > 0 libsystem_kernel.dylib 0x00007fff98f915c2 swtch_pri + 10 > 1 libsystem_pthread.dylib 0x00007fff93b5d381 sched_yield + 11 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063d5946 __sanitizer::internal_sched_yield() + 6 > 3 libclang_rt.tsan_osx_dynamic.dylib 0x0000000106395495 wrap_pthread_create + 341 > 4 ??? 0x0000000000002006 0 + 8198 > > Thread 8: > 0 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063cb015 __tsan::MetaMap::FreeRange(__tsan::ThreadState*, unsigned long, unsigned long, unsigned long) + 149 > > Thread 9: > 0 libsystem_kernel.dylib 0x00007fff98f915c2 swtch_pri + 10 > 1 libsystem_pthread.dylib 0x00007fff93b5d381 sched_yield + 11 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063d5946 __sanitizer::internal_sched_yield() + 6 > 3 libclang_rt.tsan_osx_dynamic.dylib 0x0000000106395495 wrap_pthread_create + 341 > 4 ??? 0x0000000000004006 0 + 16390 > > Thread 10: > 0 libsystem_kernel.dylib 0x00007fff98f915c2 swtch_pri + 10 > 1 libsystem_pthread.dylib 0x00007fff93b5d381 sched_yield + 11 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063d5946 __sanitizer::internal_sched_yield() + 6 > 3 libclang_rt.tsan_osx_dynamic.dylib 0x0000000106395495 wrap_pthread_create + 341 > 4 ??? 0x0000000000002006 0 + 8198 > > Thread 11: > 0 libsystem_kernel.dylib 0x00007fff98f95eda __mmap + 10 > 1 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063e0404 __sanitizer::MmapFixedNoReserve(unsigned long, unsigned long, char const*) + 100 > > Thread 12: > 0 libsystem_kernel.dylib 0x00007fff98f95f9a __munmap + 10 > 1 libclang_rt.tsan_osx_dynamic.dylib 0x0000000106394fb6 wrap_munmap + 134 > 2 ??? 0x0000000000002000 0 + 8192 > > Thread 13: > 0 libsystem_kernel.dylib 0x00007fff98f95eda __mmap + 10 > 1 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063e0404 __sanitizer::MmapFixedNoReserve(unsigned long, unsigned long, char const*) + 100 > > Thread 14: > 0 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063b8759 __tsan::CreateThreadContext(unsigned int) + 233 > > Thread 15: > 0 libsystem_kernel.dylib 0x00007fff98f95f9a __munmap + 10 > 1 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063d5846 __sanitizer::internal_munmap(void*, unsigned long) + 6 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063d6bcf __sanitizer::UnmapOrDie(void*, unsigned long) + 31 > 3 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063ba734 __tsan::MemoryRangeSet(__tsan::ThreadState*, unsigned long, unsigned long, unsigned long, unsigned long long) + 612 > 4 libclang_rt.tsan_osx_dynamic.dylib 0x0000000106394f20 wrap_mmap + 496 > 5 ??? 0x0000000000002000 0 + 8192 > > Thread 16: > 0 libsystem_kernel.dylib 0x00007fff98f915da syscall_thread_switch + 10 > 1 libsystem_platform.dylib 0x00007fff965f982d _OSSpinLockLockSlow + 63 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063d5e4d __sanitizer::BlockingMutex::Lock() + 29 > 3 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063dc1ef __sanitizer::ThreadRegistry::StartThread(unsigned int, unsigned long, void*) + 31 > > Thread 17: > 0 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063cb018 __tsan::MetaMap::FreeRange(__tsan::ThreadState*, unsigned long, unsigned long, unsigned long) + 152 > > Thread 18: > 0 libsystem_kernel.dylib 0x00007fff98f915c2 swtch_pri + 10 > 1 libsystem_pthread.dylib 0x00007fff93b5d381 sched_yield + 11 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063d5946 __sanitizer::internal_sched_yield() + 6 > 3 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063952e5 __tsan_thread_start_func + 69 > 4 libsystem_pthread.dylib 0x00007fff93b5bfd7 _pthread_start + 176 > 5 libsystem_pthread.dylib 0x00007fff93b593ed thread_start + 13 > > Thread 19: > 0 libsystem_kernel.dylib 0x00007fff98f915c2 swtch_pri + 10 > 1 libsystem_pthread.dylib 0x00007fff93b5d381 sched_yield + 11 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063d5946 __sanitizer::internal_sched_yield() + 6 > 3 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063952e5 __tsan_thread_start_func + 69 > 4 libsystem_pthread.dylib 0x00007fff93b5bfd7 _pthread_start + 176 > 5 libsystem_pthread.dylib 0x00007fff93b593ed thread_start + 13 > > Thread 20: > 0 libsystem_kernel.dylib 0x00007fff98f915c2 swtch_pri + 10 > 1 libsystem_pthread.dylib 0x00007fff93b5d381 sched_yield + 11 > 2 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063d5946 __sanitizer::internal_sched_yield() + 6 > 3 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063952e5 __tsan_thread_start_func + 69 > 4 libsystem_pthread.dylib 0x00007fff93b5bfd7 _pthread_start + 176 > 5 libsystem_pthread.dylib 0x00007fff93b593ed thread_start + 13 > > Thread 21 Crashed: > > ^^^^^^^^^^^^^ > > 0 libclang_rt.tsan_osx_dynamic.dylib 0x00000001063cbe01 wrap_OSSpinLockLock + 17 > 1 libsystem_pthread.dylib 0x00007fff93b5bfd7 _pthread_start + 176 > 2 libsystem_pthread.dylib 0x00007fff93b593ed thread_start + 13 > > Thread 21 crashed with X86 Thread State (64-bit): > rax: 0x00486000000025df rbx: 0x00000001095ea000 rcx: 0x00486000000025df rdx: 0x0000000000000000 > rdi: 0x00007fff7d4472d8 rsi: 0x0000000000000000 rbp: 0x00000001095e9f10 rsp: 0x00000001095e9ec0 > r8: 0x00000000000028df r9: 0x0000000000083000 r10: 0x0000000000000000 r11: 0x0000000000000206 > r12: 0x000000000000200f r13: 0x00000000000008ff r14: 0x00007fff7d4472d8 r15: 0x00000001063952a0 > rip: 0x00000001063cbe01 rfl: 0x0000000000010202 cr2: 0x00003100849e0000 > > Logical CPU: 0 > Error Code: 0x00000000 > Trap Number: 13 > > > Binary Images: > 0x106387000 - 0x106387ff7 +mmap_stress.cc.tmp (0) <A98F81CD-CE7F-3DD1-9C04-79A188AA7340> /Users/USER/*/mmap_stress.cc.tmp > 0x10638e000 - 0x1063fdff7 +libclang_rt.tsan_osx_dynamic.dylib (0) <D1A58E40-8472-3DFE-85F3-3E1430258010> /Users/USER/*/libclang_rt.tsan_osx_dynamic.dylib > 0x7fff69ba0000 - 0x7fff69bd6887 dyld (353.2.3) <B1B370A5-479F-3533-8AD7-97B687D4F989> /usr/lib/dyld > 0x7fff8ca26000 - 0x7fff8ca27ff7 libsystem_blocks.dylib (65) <9615D10A-FCA7-3BE4-AA1A-1B195DACE1A1> /usr/lib/system/libsystem_blocks.dylib > 0x7fff8d4d6000 - 0x7fff8d4dbfff libsystem_stats.dylib (163.30.2) <D0E96837-3CF6-323D-B711-6DF6F660E530> /usr/lib/system/libsystem_stats.dylib > 0x7fff8d4dc000 - 0x7fff8d4dcff7 libkeymgr.dylib (28) <77845842-DE70-3CC5-BD01-C3D14227CED5> /usr/lib/system/libkeymgr.dylib > 0x7fff8d55d000 - 0x7fff8d75746f libobjc.A.dylib (647) <759E155D-BC42-3D4E-869B-6F57D477177C> /usr/lib/libobjc.A.dylib > 0x7fff8dab1000 - 0x7fff8dabcfff libcommonCrypto.dylib (60061.30.1) <E789748D-F9A7-3CFF-B317-90DF348B1E95> /usr/lib/system/libcommonCrypto.dylib > 0x7fff8dd4b000 - 0x7fff8dd53ffb libcopyfile.dylib (118.1.2) <0C68D3A6-ACDD-3EF3-991A-CC82C32AB836> /usr/lib/system/libcopyfile.dylib > 0x7fff8de1e000 - 0x7fff8de23ff7 libmacho.dylib (862) <126CA2ED-DE91-308F-8881-B9DAEC3C63B6> /usr/lib/system/libmacho.dylib > 0x7fff8e1e8000 - 0x7fff8e274fe7 libsystem_c.dylib (1044.40.1) <F0635E0F-FE4B-34DB-ACF9-A58C1E9070E9> /usr/lib/system/libsystem_c.dylib > 0x7fff8e9df000 - 0x7fff8e9e7fff libsystem_dnssd.dylib (576.30.4) <0CEB5910-843F-315C-A1DE-5D955A48A045> /usr/lib/system/libsystem_dnssd.dylib > 0x7fff8eb0d000 - 0x7fff8eb13ff7 libsystem_networkextension.dylib (167.40.3) <BA58B30B-8377-3B0A-8AE3-4F84021D9D4E> /usr/lib/system/libsystem_networkextension.dylib > 0x7fff8eb18000 - 0x7fff8eb1efff libsystem_trace.dylib (72.20.1) <840F5301-B55A-3078-90B9-FEFFD6CD741A> /usr/lib/system/libsystem_trace.dylib > 0x7fff8f494000 - 0x7fff8f494ff7 libunc.dylib (29) <5676F7EA-C1DF-329F-B006-D2C3022B7D70> /usr/lib/system/libunc.dylib > 0x7fff8f513000 - 0x7fff8f515fff libsystem_sandbox.dylib (358.20.5) <3F5E973F-C702-31AC-97BC-05F5C195683C> /usr/lib/system/libsystem_sandbox.dylib > 0x7fff8f54d000 - 0x7fff8f55eff3 libsystem_coretls.dylib (35.40.1) <155DA0A9-2046-332E-BFA3-D7974A51F731> /usr/lib/system/libsystem_coretls.dylib > 0x7fff8f55f000 - 0x7fff8f566ff7 libcompiler_rt.dylib (35) <BF8FC133-EE10-3DA6-9B90-92039E28678F> /usr/lib/system/libcompiler_rt.dylib > 0x7fff8fb97000 - 0x7fff8fbb3ff7 libsystem_malloc.dylib (53.30.1) <DDA8928B-CC0D-3255-BD8A-3FEA0982B890> /usr/lib/system/libsystem_malloc.dylib > 0x7fff90410000 - 0x7fff90411ff3 libSystem.B.dylib (1213) <1866C519-C5F3-3D09-8C17-A8F703664521> /usr/lib/libSystem.B.dylib > 0x7fff9195d000 - 0x7fff91985fff libxpc.dylib (559.40.1) <5C829202-962E-3744-8B50-00D38CC88E84> /usr/lib/system/libxpc.dylib > 0x7fff928f9000 - 0x7fff9290fff7 libsystem_asl.dylib (267) <F153AC5B-0542-356E-88C8-20A62CA704E2> /usr/lib/system/libsystem_asl.dylib > 0x7fff92c13000 - 0x7fff92c16ff7 libdyld.dylib (353.2.3) <CFBBE540-D503-3AFC-B5D6-644F1E69949B> /usr/lib/system/libdyld.dylib > 0x7fff93b58000 - 0x7fff93b61fff libsystem_pthread.dylib (105.40.1) <ACE90967-ECD0-3251-AEEB-461E3C6414F7> /usr/lib/system/libsystem_pthread.dylib > 0x7fff93b62000 - 0x7fff93b8cff7 libdispatch.dylib (442.1.4) <502CF32B-669B-3709-8862-08188225E4F0> /usr/lib/system/libdispatch.dylib > 0x7fff94527000 - 0x7fff94530ff7 libsystem_notify.dylib (133.1.1) <61147800-F320-3DAA-850C-BADF33855F29> /usr/lib/system/libsystem_notify.dylib > 0x7fff94531000 - 0x7fff94531ff7 liblaunch.dylib (559.40.1) <4F81CA3A-D2CE-3030-A89D-42F3DAD7BA8F> /usr/lib/system/liblaunch.dylib > 0x7fff94535000 - 0x7fff94539fff libcache.dylib (69) <45E9A2E7-99C4-36B2-BEE3-0C4E11614AD1> /usr/lib/system/libcache.dylib > 0x7fff94bd6000 - 0x7fff94bd8ff7 libsystem_coreservices.dylib (9) <41B7C578-5A53-31C8-A96F-C73E030B0938> /usr/lib/system/libsystem_coreservices.dylib > 0x7fff965f6000 - 0x7fff965fefff libsystem_platform.dylib (63) <64E34079-D712-3D66-9CE2-418624A5C040> /usr/lib/system/libsystem_platform.dylib > 0x7fff984d8000 - 0x7fff98551fe7 libcorecrypto.dylib (233.30.1) <5779FFA0-4D9A-3AD4-B7F2-618227621DC8> /usr/lib/system/libcorecrypto.dylib > 0x7fff98935000 - 0x7fff98936fff libDiagnosticMessagesClient.dylib (100) <2EE8E436-5CDC-34C5-9959-5BA218D507FB> /usr/lib/libDiagnosticMessagesClient.dylib > 0x7fff98f80000 - 0x7fff98f9dfff libsystem_kernel.dylib (2782.40.9) <16AD15EF-3DAE-3F63-9D26-26CCE1920762> /usr/lib/system/libsystem_kernel.dylib > 0x7fff98fb1000 - 0x7fff98fb3fff libquarantine.dylib (76.20.1) <7AF90041-2768-378A-925A-D83161863642> /usr/lib/system/libquarantine.dylib > 0x7fff991f1000 - 0x7fff99219fff libsystem_info.dylib (459.40.1) <2E16C4B3-A327-3957-9C41-143911979A1E> /usr/lib/system/libsystem_info.dylib > 0x7fff9a3c0000 - 0x7fff9a3f8fff libsystem_network.dylib (412.20.3) <6105C134-6722-3C0A-A4CE-5E1261E2E1CC> /usr/lib/system/libsystem_network.dylib > 0x7fff9a401000 - 0x7fff9a402ffb libremovefile.dylib (35) <3485B5F4-6CE8-3C62-8DFD-8736ED6E8531> /usr/lib/system/libremovefile.dylib > 0x7fff9ab09000 - 0x7fff9ab39fff libsystem_m.dylib (3086.1) <1E12AB45-6D96-36D0-A226-F24D9FB0D9D6> /usr/lib/system/libsystem_m.dylib > 0x7fff9ad92000 - 0x7fff9ad94fff libsystem_configuration.dylib (699.40.2) <56F94DCE-DBDE-3615-8F07-DE6270D9F8BE> /usr/lib/system/libsystem_configuration.dylib > 0x7fff9af2a000 - 0x7fff9af55fff libc++abi.dylib (125) <88A22A0F-87C6-3002-BFBA-AC0F2808B8B9> /usr/lib/libc++abi.dylib > 0x7fff9b2b4000 - 0x7fff9b2faff7 libauto.dylib (186) <A260789B-D4D8-316A-9490-254767B8A5F1> /usr/lib/libauto.dylib > 0x7fff9b315000 - 0x7fff9b31aff7 libunwind.dylib (35.3) <BE7E51A0-B6EA-3A54-9CCA-9D88F683A6D6> /usr/lib/system/libunwind.dylib > 0x7fff9b69d000 - 0x7fff9b6f1fff libc++.1.dylib (120) <1B9530FD-989B-3174-BB1C-BDC159501710> /usr/lib/libc++.1.dylib > 0x7fff9c06c000 - 0x7fff9c06dfff libsystem_secinit.dylib (18) <581DAD0F-6B63-3A48-B63B-917AF799ABAA> /usr/lib/system/libsystem_secinit.dylib > > External Modification Summary: > Calls made by other processes targeting this process: > task_for_pid: 0 > thread_create: 0 > thread_set_state: 0 > Calls made by this process: > task_for_pid: 0 > thread_create: 0 > thread_set_state: 0 > Calls made by all processes on this machine: > task_for_pid: 3447975 > thread_create: 0 > thread_set_state: 143262 > > > System Profile: > AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x10E), Broadcom BCM43xx 1.0 (7.15.166.24.3) > Bluetooth: Version 4.3.6f3 16238, 3 services, 27 devices, 1 incoming serial ports > Thunderbolt Bus: Mac mini, Apple Inc., 23.4 > Memory Module: BANK 0/DIMM0, 8 GB, DDR3, 1600 MHz, 0x802C, 0x31364B544631473634485A2D314736453220 > Memory Module: BANK 1/DIMM0, 8 GB, DDR3, 1600 MHz, 0x802C, 0x31364B544631473634485A2D314736453220 > USB Device: Hub > USB Device: Hub > USB Device: Hub > USB Device: BRCM20702 Hub > USB Device: Bluetooth USB Host Controller > USB Device: IR Receiver > Serial ATA Device: APPLE SSD SM256E, 251 GB > Model: Macmini6,2, BootROM MM61.0106.B09, 4 processors, Intel Core i7, 2.6 GHz, 16 GB, SMC 2.8f1 > Network Service: Ethernet, Ethernet, en0 > Graphics: Intel HD Graphics 4000, Intel HD Graphics 4000, Built-In > > > > > > > >> On Jan 21, 2016, at 1:20 PM, Chris Matthews via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: >> >> The lit test directly pipes the output into the check, and does not store it in a temporary file. >> >> Last week, I ran this test in a loop for 8 hours on a machine right after it failed in a build. It did not reproduce. Do you have any suggestions for reproducing? Maybe we need to have the system under more stress before running the test? >> >>> On Jan 20, 2016, at 1:41 PM, Kostya Serebryany <kcc at google.com <mailto:kcc at google.com>> wrote: >>> >>> >>> >>> On Wed, Jan 20, 2016 at 1:31 PM, Chris Matthews <chris.matthews at apple.com <mailto:chris.matthews at apple.com>> wrote: >>> I worded that poorly, the Jenkins check I added will explain to the user that we know this fails sometimes. >>> >>>> On Jan 20, 2016, at 1:30 PM, Chris Matthews <chris.matthews at apple.com <mailto:chris.matthews at apple.com>> wrote: >>>> >>>> I have added a Jenkins check for this test, which explains why it fails on some builds. >>>> >>>> Can we change the test to keep its output? Will it just be blank anyways? >>> >>> >>> The test simply prints everything to stdout. >>> It's the lit driver that hides the output. >>> >>> Maybe someone with a Mac can run this test manually (with exact same build command line) >>> a few thousand times? I continue to suspect a kernel bug. >>> >>>> >>>>> On Jan 20, 2016, at 1:23 PM, Kostya Serebryany <kcc at google.com <mailto:kcc at google.com>> wrote: >>>>> >>>>> The test fails again: http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA_check/9739/testReport/junit/ThreadSanitizer/ThreadSanitizer/mmap_stress_cc/ <http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA_check/9739/testReport/junit/ThreadSanitizer/ThreadSanitizer/mmap_stress_cc/> >>>>> But unfortunately lit does not provide the original log, so we still don't know why... >>>>> >>>>> On Wed, Jan 13, 2016 at 1:11 PM, Chris Matthews <chris.matthews at apple.com <mailto:chris.matthews at apple.com>> wrote: >>>>> These ran to completion without error. :( >>>>> >>>>>> On Jan 12, 2016, at 3:23 PM, Chris Matthews <chris.matthews at apple.com <mailto:chris.matthews at apple.com>> wrote: >>>>>> >>>>>> I’m running this on green-dragon-03 which is running OSX 10.10.5. >>>>>> >>>>>> /Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace\@2/clang-build/bin/clang++ ../llvm/projects/compiler-rt/test/tsan/mmap_stress.cc -lpthread >>>>>> while true; do for((i=0;i<10;i++)); do ./a.out || echo ===============FAILED================ & done ; wait; done >>>>>> >>>>>> Seems like it is getting about 225 runs a minute, so it will take about 8 hours to complete 100k runs. >>>>>> >>>>>>> On Jan 12, 2016, at 1:28 PM, Tobias Grosser via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: >>>>>>> >>>>>>> On 01/12/2016 10:26 PM, Kostya Serebryany wrote: >>>>>>>> Hi Tobias, >>>>>>>> >>>>>>>> What machine is that? >>>>>>>> We have seen this and similar tests be flaky on older Linux kernels due >>>>>>>> to kernel bug(s). >>>>>>>> May I ask you to run the same test (just built with clang) on the same >>>>>>>> machine for ~100000 times and see if it ever crashes? >>>>>>>> >>>>>>>> clang++ ~/llvm/projects/compiler-rt/test/tsan/mmap_stress.cc -lpthread >>>>>>>> while true; do for((i=0;i<10;i++)); do ./a.out || echo >>>>>>>> ===============FAILED================ & done ; wait; done >>>>>>> >>>>>>> Add Michael again (who might have access to this Machine). >>>>>>> >>>>>>> Tobias >>>>>>> _______________________________________________ >>>>>>> LLVM Developers mailing list >>>>>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> >>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev> >>>>>> >>>>> >>>>> >>>> >>> >>> >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160122/145dd506/attachment-0001.html>
Dmitry Vyukov via llvm-dev
2016-Jan-22 15:17 UTC
[llvm-dev] greendragon build noisy due to mmap_stress.cc
On Fri, Jan 22, 2016 at 4:11 PM, Kuba Brecka <jbrecka at apple.com> wrote:> Hm, I tried to reproduce this as well, but unsuccessfully. From the crash > report: EXC_I386_GPFLT means we’re dereferencing a non-canonical pointer, > in this case “0x00486000000025df”. This happens at wrap_OSSpinLockLock+17, > which is just after the prologue and just after calling cur_thread(). So > I’d say it happens when we’re dereferencing the pointer returned by > cur_thread(). On OS X, we’re doing this trick where we store the > ThreadState pointer in the shadow memory. Could it be that something > actually read/wrote the memory that is backed by the same place as the > shadow memory? > > Does “0x00486000000025df” like a reasonable content of a shadow cell?Yes, it looks like a reasonable shadow: // Shadow (from most significant bit): // freed : 1 // tid : kTidBits // is_atomic : 1 // is_read : 1 // size_log : 2 // addr0 : 3 // epoch : kClkBits
Dmitry Vyukov via llvm-dev
2016-Jan-22 15:53 UTC
[llvm-dev] greendragon build noisy due to mmap_stress.cc
On Fri, Jan 22, 2016 at 4:17 PM, Dmitry Vyukov <dvyukov at google.com> wrote:> On Fri, Jan 22, 2016 at 4:11 PM, Kuba Brecka <jbrecka at apple.com> wrote: >> Hm, I tried to reproduce this as well, but unsuccessfully. From the crash >> report: EXC_I386_GPFLT means we’re dereferencing a non-canonical pointer, >> in this case “0x00486000000025df”. This happens at wrap_OSSpinLockLock+17, >> which is just after the prologue and just after calling cur_thread(). So >> I’d say it happens when we’re dereferencing the pointer returned by >> cur_thread(). On OS X, we’re doing this trick where we store the >> ThreadState pointer in the shadow memory. Could it be that something >> actually read/wrote the memory that is backed by the same place as the >> shadow memory? >> >> Does “0x00486000000025df” like a reasonable content of a shadow cell? > > > > Yes, it looks like a reasonable shadow: > > // Shadow (from most significant bit): > // freed : 1 > // tid : kTidBits > // is_atomic : 1 > // is_read : 1 > // size_log : 2 > // addr0 : 3 > // epoch : kClkBits+Nico pointed to another failure: http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA_check/9782/testReport/ThreadSanitizer/ThreadSanitizer/mmap_stress_cc/