Mingming Liu via llvm-dev
2021-Dec-13 20:49 UTC
[llvm-dev] Suggestions on debugging pre-merge test failure that looks irrelevant.
Hi,
I'm looking for some suggestions on debugging pre-merge test failures
that look irrelevant.
Two examples with links (while `ninja check-all` passed for the patch,
although the DLLVM_ENABLE_PROJECTS doesn't necessarily cover the
subprojects that fail)
1) This is caused by stale cmake configurations, as indicated by the
error.
Link of build artifact
<https://buildkite.com/llvm-project/premerge-checks/builds/68728#6abfb9c3-e82d-4bbf-835b-551344e469c5/31-46>
.
Sync up to pick up new cmake configurations resolved the error.
2) Link of build artifact
<https://buildkite.com/llvm-project/premerge-checks/builds/69795#22733c7d-1ef1-488d-ad08-d77542b0d7e0>
Test failed with error
1. Failed Tests (1):
2. LLVM :: Bindings/Go/go.test
I didn't find other information in the log that might help.
There is a playbook
<https://github.com/google/llvm-premerge-checks/blob/main/docs/playbooks.md>
yet
I'm wondering if there are more lightweight procedures besides restarting
build (e.g., looking at a continuous build dashboard for tests that might
already fail or might be flaky before the patch) to debug tests that look
irrelevant.
--
Thanks,
Mingming
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211213/dce864d5/attachment.html>
Geoffrey Martin-Noble via llvm-dev
2021-Dec-13 21:13 UTC
[llvm-dev] Suggestions on debugging pre-merge test failure that looks irrelevant.
The first error you linked is for Bazel, which is not officially supported, but you have opted in to the pre-merge checks by joining the Bazel phab project. That particular message comes from the tests that detect when config files have changed. As you noted, this is due to an old baseline for your patch (that was fixed 11 days ago in https://github.com/llvm/llvm-project/commit/c6cfd385b1). I'd generally recommend rebasing to more recent commits before sending something for review/testing anyway. For the second failure, you can see more details about the error in the error logs. The raw logs <https://buildkite-cloud.s3.amazonaws.com/logs-by-pipeline/f8ab115f-a384-49e8-a048-0f71ab03c5d0/369628b0-3b82-4b9f-bd36-8859c7273baa/22733c7d-1ef1-488d-ad08-d77542b0d7e0.log?response-content-disposition=inline&response-content-type=text%2Fplain&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAQPCP3C7L6NRLG4VQ%2F20211213%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211213T210303Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEGEaCXVzLWVhc3QtMSJIMEYCIQC7Zk8lLsq%2BNiijVeHzF0ee9GJzj7tZ8JyqmkoyMv7cbgIhANEpzUfSc4aQrVPxpsME4OO7SFHDSaUupbZ%2BvQaFt00GKvoDCEoQABoMMDMyMzc5NzA1MzAzIgw7Nbvd4DnPsSzrViYq1wOWra8AZ5khDyEu2pmLazmHvkJyhCD3pWkecJqzfgbDHr2%2BAP4yFlJEqIC5z0l56Hc7IWKOIl9V4zZ0FvK%2FZ7%2BV3i9OHgAZSQB072E2RwuBPdzXJG7udvCFZLqfv2M8Z%2FXLFj54KZ5VUD7%2FifIFaFNFjyCJlkfNDzNCHph7c0dK1HW4V4TkhSh659lTxSZmFeQf4WArqbwJz70h0LaP9Evl6dvdrjg%2B%2FaE5OULzPoLzIBdj0jtV04zHhO8XLL4HBpzqhsQynXTCeMMupNxCxxZAYBGt3Hio45xGeE0%2Fyo0FuIYB0z8uqqK3vESVGGZ2jp6D8KzFqZmomeikGxc9grKZRrWLCEwCdjxPE3lQcS9ScIQkjww2G9q%2F5FbM%2FTdvW9Du5hv%2F5by5MaNAUMDc88xThN5lXVc5rTxd6BYA4OCHW4aZF7I7%2Bq2CCYjczGhSvxw3uvcJVOX0%2BgdRcxg%2Fm9HewQPW72IZmxv4eJab8gTMjqu3q%2BvMJhL1%2BcA3f8UMSPo7nyFrz%2FgW9hkI6fSxmqAYXoxDPBe6QtE7JSRAHdfZnzrG1IFt9Z8lYa6fXb69lNkjlVGiIexSXZsh0BoJ8AKIoPTJAQqyiye%2FUG0nTwqdi3ETSAlbjm0wm%2BrdjQY6pAGz38I6pe29x7UfHM8Ci8xd5lV4jvzWvYhRQC2URs91Anx%2BCBn8JnGrCbfdZkSJLkfPGArbUX%2FO9qpfRwq9BysO98fbIAvHWU9%2FCNv0uGp8jV2f2FXMkXF7%2F2WWhi7X6n7iLzf%2F1r9XjCYDCQQJhVI7J6vSuh%2BdxENmakzJMJ1baT060GhtfjFHm%2BuLXw35%2FzK96mxEe6rsFdZ4gWDAmRWs5Sac3Q%3D%3D&X-Amz-Signature=9223509378e29ef5c1ad12c76763dc3e19602a4f23be33107428869b1fd64e9a> have everything (whereas the buildkite UI only shows the last 1MB, as noted at the top). There you can see the output for that particular test failure <https://buildkite-cloud.s3.amazonaws.com/logs-by-pipeline/f8ab115f-a384-49e8-a048-0f71ab03c5d0/369628b0-3b82-4b9f-bd36-8859c7273baa/22733c7d-1ef1-488d-ad08-d77542b0d7e0.log?response-content-disposition=inline&response-content-type=text%2Fplain&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAQPCP3C7L6NRLG4VQ%2F20211213%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211213T210303Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEGEaCXVzLWVhc3QtMSJIMEYCIQC7Zk8lLsq%2BNiijVeHzF0ee9GJzj7tZ8JyqmkoyMv7cbgIhANEpzUfSc4aQrVPxpsME4OO7SFHDSaUupbZ%2BvQaFt00GKvoDCEoQABoMMDMyMzc5NzA1MzAzIgw7Nbvd4DnPsSzrViYq1wOWra8AZ5khDyEu2pmLazmHvkJyhCD3pWkecJqzfgbDHr2%2BAP4yFlJEqIC5z0l56Hc7IWKOIl9V4zZ0FvK%2FZ7%2BV3i9OHgAZSQB072E2RwuBPdzXJG7udvCFZLqfv2M8Z%2FXLFj54KZ5VUD7%2FifIFaFNFjyCJlkfNDzNCHph7c0dK1HW4V4TkhSh659lTxSZmFeQf4WArqbwJz70h0LaP9Evl6dvdrjg%2B%2FaE5OULzPoLzIBdj0jtV04zHhO8XLL4HBpzqhsQynXTCeMMupNxCxxZAYBGt3Hio45xGeE0%2Fyo0FuIYB0z8uqqK3vESVGGZ2jp6D8KzFqZmomeikGxc9grKZRrWLCEwCdjxPE3lQcS9ScIQkjww2G9q%2F5FbM%2FTdvW9Du5hv%2F5by5MaNAUMDc88xThN5lXVc5rTxd6BYA4OCHW4aZF7I7%2Bq2CCYjczGhSvxw3uvcJVOX0%2BgdRcxg%2Fm9HewQPW72IZmxv4eJab8gTMjqu3q%2BvMJhL1%2BcA3f8UMSPo7nyFrz%2FgW9hkI6fSxmqAYXoxDPBe6QtE7JSRAHdfZnzrG1IFt9Z8lYa6fXb69lNkjlVGiIexSXZsh0BoJ8AKIoPTJAQqyiye%2FUG0nTwqdi3ETSAlbjm0wm%2BrdjQY6pAGz38I6pe29x7UfHM8Ci8xd5lV4jvzWvYhRQC2URs91Anx%2BCBn8JnGrCbfdZkSJLkfPGArbUX%2FO9qpfRwq9BysO98fbIAvHWU9%2FCNv0uGp8jV2f2FXMkXF7%2F2WWhi7X6n7iLzf%2F1r9XjCYDCQQJhVI7J6vSuh%2BdxENmakzJMJ1baT060GhtfjFHm%2BuLXw35%2FzK96mxEe6rsFdZ4gWDAmRWs5Sac3Q%3D%3D&X-Amz-Signature=9223509378e29ef5c1ad12c76763dc3e19602a4f23be33107428869b1fd64e9a#:~:text=********************%20test%20'llvm%20%3A%3A%20bindings%2Fgo%2Fgo.test'%20failed%20********************> (it takes a minute to load). The relevant bit is: no required module provides package llvm.org/llvm/bindings/go/llvm: go.mod> file not found in current directory or any parent directory; see 'go help > modules'So it's something about a missing go dependency. I think I've encountered that failure before. It seems flaky. Maybe people working on go bindings can help root cause? Sorry that doesn't answer your meta-question. Perhaps +Mikhail Goncharov <goncharov at google.com> has some ideas for better workflows here On Mon, Dec 13, 2021 at 12:49 PM Mingming Liu via llvm-dev < llvm-dev at lists.llvm.org> wrote:> Hi, > I'm looking for some suggestions on debugging pre-merge test failures > that look irrelevant. > > Two examples with links (while `ninja check-all` passed for the patch, > although the DLLVM_ENABLE_PROJECTS doesn't necessarily cover the > subprojects that fail) > 1) This is caused by stale cmake configurations, as indicated by the > error. > Link of build artifact > <https://buildkite.com/llvm-project/premerge-checks/builds/68728#6abfb9c3-e82d-4bbf-835b-551344e469c5/31-46> > . > Sync up to pick up new cmake configurations resolved the error. > > 2) Link of build artifact > <https://buildkite.com/llvm-project/premerge-checks/builds/69795#22733c7d-1ef1-488d-ad08-d77542b0d7e0> > > Test failed with error > > > > 1. Failed Tests (1): > 2. LLVM :: Bindings/Go/go.test > > I didn't find other information in the log that might help. > > There is a playbook > <https://github.com/google/llvm-premerge-checks/blob/main/docs/playbooks.md> yet > I'm wondering if there are more lightweight procedures besides restarting > build (e.g., looking at a continuous build dashboard for tests that might > already fail or might be flaky before the patch) to debug tests that look > irrelevant. > > -- > Thanks, > Mingming > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20211213/329b5afd/attachment.html>