Tobias Hieta via llvm-dev
2021-Aug-16 09:12 UTC
[llvm-dev] Many LLDB test failures in 13.0.0-rc1 on macOS
Hello, While testing LLVM 13.0.0-rc1 on macOS I noticed many lldb test failures that are regressions from 12.0.1. I was going to post this to lldb-dev - but I was not subscribed and the process seems manual still. So forgive me for posting it here. Let me know if I can share anymore information about these test failures with the team. List of failing tests: FAIL: lldb-shell :: Commands/command-thread-select.test (94817 of 96071) FAIL: lldb-shell :: Breakpoint/jit-loader_jitlink_elf.test (94818 of 96071) FAIL: lldb-shell :: Breakpoint/jit-loader_rtdyld_elf.test (94819 of 96071) FAIL: lldb-shell :: ExecControl/StopHook/stop-hook.test (94823 of 96071) FAIL: lldb-shell :: Host/TestCustomShell.test (94842 of 96071) FAIL: lldb-shell :: Expr/TestIRMemoryMap.test (94849 of 96071) FAIL: lldb-shell :: Process/Optimization.test (94852 of 96071) FAIL: lldb-shell :: Process/UnsupportedLanguage.test (94853 of 96071) FAIL: lldb-shell :: Process/TestEnvironment.test (94854 of 96071) FAIL: lldb-shell :: Register/x86-64-gp-read.test (94855 of 96071) FAIL: lldb-shell :: Register/x86-64-read.test (94862 of 96071) FAIL: lldb-shell :: Register/x86-64-ymm-read.test (94866 of 96071) FAIL: lldb-shell :: Register/x86-multithread-read.test (94873 of 96071) FAIL: lldb-shell :: Register/x86-multithread-write.test (94876 of 96071) FAIL: lldb-shell :: Reproducer/Functionalities/TestImageList.test (94879 of 96071) FAIL: lldb-shell :: Reproducer/Functionalities/TestDataFormatter.test (94880 of 96071) FAIL: lldb-shell :: Reproducer/Functionalities/TestExpressionEvaluation.test (94882 of 96071) FAIL: lldb-shell :: Reproducer/Functionalities/TestStepping.test (94883 of 96071) FAIL: lldb-shell :: Reproducer/Modules/TestModuleCXX.test (94884 of 96071) FAIL: lldb-shell :: Reproducer/TestDSYM.test (94902 of 96071) FAIL: lldb-shell :: Reproducer/TestDump.test (94922 of 96071) FAIL: lldb-shell :: Reproducer/TestFileRepro.test (94924 of 96071) FAIL: lldb-shell :: Reproducer/TestGDBRemoteRepro.test (94931 of 96071) FAIL: lldb-shell :: Reproducer/TestMultipleTargets.test (94932 of 96071) FAIL: lldb-shell :: Reproducer/TestRelativePath.test (94934 of 96071) FAIL: lldb-shell :: Reproducer/TestReuseDirectory.test (94946 of 96071) FAIL: lldb-shell :: Reproducer/TestVersionCheck.test (94947 of 96071) FAIL: lldb-shell :: Reproducer/TestVerify.test (94948 of 96071) FAIL: lldb-shell :: Reproducer/TestWorkingDir.test (94950 of 96071) FAIL: lldb-shell :: Settings/TestFrameFormatColor.test (94957 of 96071) FAIL: lldb-shell :: Settings/TestFrameFormatMangling.test (94961 of 96071) FAIL: lldb-shell :: Settings/TestFrameFormatNoColor.test (94969 of 96071) FAIL: lldb-shell :: Settings/TestLineMarkerColor.test (94975 of 96071) FAIL: lldb-shell :: Subprocess/fork-follow-parent-wp.test (95010 of 96071) FAIL: lldb-shell :: Subprocess/fork-follow-parent.test (95032 of 96071) FAIL: lldb-shell :: Subprocess/vfork-follow-parent.test (95045 of 96071) FAIL: lldb-shell :: SymbolFile/DWARF/deterministic-build.cpp (95046 of 96071) FAIL: lldb-shell :: Unwind/eh-frame-dwarf-unwind.test (95219 of 96071) FAIL: lldb-shell :: Unwind/thread-step-out-ret-addr-check.test (96063 of 96071) FAIL: lldb-shell :: Unwind/trap_frame_sym_ctx.test (96066 of 96071) FAIL: lldb-shell :: Watchpoint/SetErrorCases.test (96067 of 96071) FAIL: lldb-unit :: tools/lldb-server/tests/./LLDBServerTests/StandardStartupTest.TestStopReplyContainsThreadPcs (96068 of 96071) -- Tobias
Stefan Gränitz via llvm-dev
2021-Aug-22 14:11 UTC
[llvm-dev] Many LLDB test failures in 13.0.0-rc1 on macOS
Hi Tobias What's the latest status here? I just made a local build from the tagged version on macOS 11.5.2 and CommandLineTools 12.5.1. Running the entire test suite, I do see the typical flakiness but nothing surprising: Failed Tests (8): lldb-api :: commands/expression/diagnostics/TestExprDiagnostics.py lldb-api :: commands/platform/sdk/TestPlatformSDK.py lldb-api :: lang/c/modules/TestCModules.py lldb-api :: lang/objc/modules-auto-import/TestModulesAutoImport.py lldb-api :: lang/objc/modules-non-objc-target/TestObjCModulesNonObjCTarget.py lldb-api :: lang/objc/modules-objc-property/TestModulesObjCProperty.py lldb-api :: lang/objc/modules/TestObjCModules.py lldb-api :: macosx/posix_spawn/TestLaunchProcessPosixSpawn.py Unexpectedly Passed Tests (1): lldb-api :: lang/objc/foundation/TestRuntimeTypes.py Am I missing anything here? My major concern is that the Shell/Breakpoint/jit-loader tests keep working in upcoming releases. Thanks Stefan On 16/08/2021 11:12, Tobias Hieta via llvm-dev wrote:> Hello, > > While testing LLVM 13.0.0-rc1 on macOS I noticed many lldb test > failures that are regressions from 12.0.1. > > I was going to post this to lldb-dev - but I was not subscribed and > the process seems manual still. So forgive me for posting it here. Let > me know if I can share anymore information about these test failures > with the team. > > List of failing tests: > FAIL: lldb-shell :: Commands/command-thread-select.test (94817 of 96071) > FAIL: lldb-shell :: Breakpoint/jit-loader_jitlink_elf.test (94818 of 96071) > FAIL: lldb-shell :: Breakpoint/jit-loader_rtdyld_elf.test (94819 of 96071) > FAIL: lldb-shell :: ExecControl/StopHook/stop-hook.test (94823 of 96071) > FAIL: lldb-shell :: Host/TestCustomShell.test (94842 of 96071) > FAIL: lldb-shell :: Expr/TestIRMemoryMap.test (94849 of 96071) > FAIL: lldb-shell :: Process/Optimization.test (94852 of 96071) > FAIL: lldb-shell :: Process/UnsupportedLanguage.test (94853 of 96071) > FAIL: lldb-shell :: Process/TestEnvironment.test (94854 of 96071) > FAIL: lldb-shell :: Register/x86-64-gp-read.test (94855 of 96071) > FAIL: lldb-shell :: Register/x86-64-read.test (94862 of 96071) > FAIL: lldb-shell :: Register/x86-64-ymm-read.test (94866 of 96071) > FAIL: lldb-shell :: Register/x86-multithread-read.test (94873 of 96071) > FAIL: lldb-shell :: Register/x86-multithread-write.test (94876 of 96071) > FAIL: lldb-shell :: Reproducer/Functionalities/TestImageList.test > (94879 of 96071) > FAIL: lldb-shell :: Reproducer/Functionalities/TestDataFormatter.test > (94880 of 96071) > FAIL: lldb-shell :: > Reproducer/Functionalities/TestExpressionEvaluation.test (94882 of > 96071) > FAIL: lldb-shell :: Reproducer/Functionalities/TestStepping.test > (94883 of 96071) > FAIL: lldb-shell :: Reproducer/Modules/TestModuleCXX.test (94884 of 96071) > FAIL: lldb-shell :: Reproducer/TestDSYM.test (94902 of 96071) > FAIL: lldb-shell :: Reproducer/TestDump.test (94922 of 96071) > FAIL: lldb-shell :: Reproducer/TestFileRepro.test (94924 of 96071) > FAIL: lldb-shell :: Reproducer/TestGDBRemoteRepro.test (94931 of 96071) > FAIL: lldb-shell :: Reproducer/TestMultipleTargets.test (94932 of 96071) > FAIL: lldb-shell :: Reproducer/TestRelativePath.test (94934 of 96071) > FAIL: lldb-shell :: Reproducer/TestReuseDirectory.test (94946 of 96071) > FAIL: lldb-shell :: Reproducer/TestVersionCheck.test (94947 of 96071) > FAIL: lldb-shell :: Reproducer/TestVerify.test (94948 of 96071) > FAIL: lldb-shell :: Reproducer/TestWorkingDir.test (94950 of 96071) > FAIL: lldb-shell :: Settings/TestFrameFormatColor.test (94957 of 96071) > FAIL: lldb-shell :: Settings/TestFrameFormatMangling.test (94961 of 96071) > FAIL: lldb-shell :: Settings/TestFrameFormatNoColor.test (94969 of 96071) > FAIL: lldb-shell :: Settings/TestLineMarkerColor.test (94975 of 96071) > FAIL: lldb-shell :: Subprocess/fork-follow-parent-wp.test (95010 of 96071) > FAIL: lldb-shell :: Subprocess/fork-follow-parent.test (95032 of 96071) > FAIL: lldb-shell :: Subprocess/vfork-follow-parent.test (95045 of 96071) > FAIL: lldb-shell :: SymbolFile/DWARF/deterministic-build.cpp (95046 of 96071) > FAIL: lldb-shell :: Unwind/eh-frame-dwarf-unwind.test (95219 of 96071) > FAIL: lldb-shell :: Unwind/thread-step-out-ret-addr-check.test (96063 of 96071) > FAIL: lldb-shell :: Unwind/trap_frame_sym_ctx.test (96066 of 96071) > FAIL: lldb-shell :: Watchpoint/SetErrorCases.test (96067 of 96071) > FAIL: lldb-unit :: > tools/lldb-server/tests/./LLDBServerTests/StandardStartupTest.TestStopReplyContainsThreadPcs > (96068 of 96071) > > -- Tobias > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-- https://weliveindetail.github.io/blog/about/