Yes. I'm seeing this as well and it predates Kostya's change. Shows up as memory corruption or double free on Linux. On Nov 14, 2013 10:46 PM, "Alexey Samsonov" <samsonov at google.com> wrote:> FYI I've seen what looked like a memory corruption in (private) Clang > bootstrap process long before Kostya's changes were submitted. I hope I'll > have the chance to investigate it soon. > > > On Fri, Nov 15, 2013 at 10:20 AM, Bob Wilson <bob.wilson at apple.com> wrote: > >> I don’t know yet, but I will let you know as soon as I can. >> >> On Nov 14, 2013, at 10:18 PM, Kostya Serebryany <kcc at google.com> wrote: >> >> > On Fri, Nov 15, 2013 at 10:16 AM, Bob Wilson <bob.wilson at apple.com> >> wrote: >> >> No, not that I am aware of. >> > >> > So, if my commits did indeed trigger the failures it could be >> > something like binary size change that caused different code alignment >> > or some such >> > and which triggered a latent memory bug somewhere else. >> > >> > It's already late evening for you now. Will you have a chance to >> > reapply changes today? >> > >> > --kcc >> > >> >> >> >> On Nov 14, 2013, at 10:15 PM, Kostya Serebryany <kcc at google.com> >> wrote: >> >> >> >>> On Fri, Nov 15, 2013 at 10:14 AM, Bob Wilson <bob.wilson at apple.com> >> wrote: >> >>>> The bit code file produced by the stage 1 compiler for one of the >> files in the clang driver is corrupt and causes the linker for stage 2 to >> crash. >> >>> >> >>> Is AddressSanitizer involved in any of the stages of the LTO build? >> >>> >> >>>> >> >>>> On Nov 14, 2013, at 10:13 PM, Kostya Serebryany <kcc at google.com> >> wrote: >> >>>> >> >>>>> What are the symptoms? >> >>>>> >> >>>>> On Fri, Nov 15, 2013 at 9:37 AM, Bob Wilson <bob.wilson at apple.com> >> wrote: >> >>>>>> I’m waiting to see if this fixes the buildbots. Unfortunately, >> because they were failing all day, there are a bunch of other regressions >> that have come up, and I’m still working through them. It takes quite a >> while to run a bootstrapped LTO clang build, so it will take a while longer. >> >>>>>> >> >>>>>> I don’t have any other useful information at this point, and I >> share your puzzlement about how your changes could possibly break the >> compiler. My only hypothesis is some sort of memory corruption. >> >>>>>> >> >>>>>> I will keep you posted. >> >>>>>> >> >>>>>> On Nov 14, 2013, at 9:22 PM, Kostya Serebryany <kcc at google.com> >> wrote: >> >>>>>> >> >>>>>>> Also, when are you planing to "reapply the changes or help debug"? >> >>>>>>> >> >>>>>>> On Fri, Nov 15, 2013 at 9:15 AM, Kostya Serebryany < >> kcc at google.com> wrote: >> >>>>>>>> On Fri, Nov 15, 2013 at 7:26 AM, Bob Wilson < >> bob.wilson at apple.com> wrote: >> >>>>>>>>> Hi Kostya, >> >>>>>>>>> >> >>>>>>>>> Thanks for the heads-up on this. I haven’t had a chance to >> look into the >> >>>>>>>>> details yet, but it looks like these patches may be breaking our >> >>>>>>>>> bootstrapped LTO build. Our buildbots have been failing all >> day, and we’re >> >>>>>>>>> still trying to figure out the problem. I’m going to >> speculatively revert >> >>>>>>>>> those changes, since they were the only patches on the buildbot >> blame list. >> >>>>>>>>> I will either reapply the changes or help debug the problem. >> >>>>>>>> >> >>>>>>>> How could this possibly affect your LTO build? >> >>>>>>>> The option is off by default. >> >>>>>>>> Do you have any details, logs, etc? >> >>>>>>>> >> >>>>>>>>> >> >>>>>>>>> —Bob >> >>>>>>>>> >> >>>>>>>>> On Nov 14, 2013, at 5:42 AM, Kostya Serebryany <kcc at google.com> >> wrote: >> >>>>>>>>> >> >>>>>>>>> Bob, Justin, >> >>>>>>>>> >> >>>>>>>>> I've just committed a poor man's coverage implementation that >> works with >> >>>>>>>>> asan. >> >>>>>>>>> http://llvm.org/viewvc/llvm-project?rev=194701&view=rev >> >>>>>>>>> http://llvm.org/viewvc/llvm-project?rev=194702&view=rev >> >>>>>>>>> It provides only function-level boolean coverage (i.e. no >> counters, just >> >>>>>>>>> "visited or not"), >> >>>>>>>>> but is very fast and very simple (no extra sections to the >> binary file, etc) >> >>>>>>>>> I've tried it for Chrome's content_shell (huge and heavy >> binary) and the >> >>>>>>>>> overhead >> >>>>>>>>> is negligible at both run-time and shutdown-time. >> >>>>>>>>> >> >>>>>>>>> We'll be evaluating this implementation and collecting usage >> stats. >> >>>>>>>>> Maybe we want to implement something simple like this in the >> Clang coverage. >> >>>>>>>>> >> >>>>>>>>> --kcc >> >>>>>>>>> >> >>>>>>>>> >> >>>>>> >> >>>> >> >> >> >> >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> > > > > -- > Alexey Samsonov, MSK > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131114/e8fa2370/attachment.html>
I was able to successfully build with r194701, so I have reapplied that along with the compiler-rt changes. Somehow we need to get to the bottom of the problem. If you can reproduce it, please help!! On Nov 14, 2013, at 11:01 PM, Eric Christopher <echristo at gmail.com> wrote:> Yes. I'm seeing this as well and it predates Kostya's change. Shows up as memory corruption or double free on Linux. > > On Nov 14, 2013 10:46 PM, "Alexey Samsonov" <samsonov at google.com> wrote: > FYI I've seen what looked like a memory corruption in (private) Clang bootstrap process long before Kostya's changes were submitted. I hope I'll have the chance to investigate it soon. > > > On Fri, Nov 15, 2013 at 10:20 AM, Bob Wilson <bob.wilson at apple.com> wrote: > I don’t know yet, but I will let you know as soon as I can. > > On Nov 14, 2013, at 10:18 PM, Kostya Serebryany <kcc at google.com> wrote: > > > On Fri, Nov 15, 2013 at 10:16 AM, Bob Wilson <bob.wilson at apple.com> wrote: > >> No, not that I am aware of. > > > > So, if my commits did indeed trigger the failures it could be > > something like binary size change that caused different code alignment > > or some such > > and which triggered a latent memory bug somewhere else. > > > > It's already late evening for you now. Will you have a chance to > > reapply changes today? > > > > --kcc > > > >> > >> On Nov 14, 2013, at 10:15 PM, Kostya Serebryany <kcc at google.com> wrote: > >> > >>> On Fri, Nov 15, 2013 at 10:14 AM, Bob Wilson <bob.wilson at apple.com> wrote: > >>>> The bit code file produced by the stage 1 compiler for one of the files in the clang driver is corrupt and causes the linker for stage 2 to crash. > >>> > >>> Is AddressSanitizer involved in any of the stages of the LTO build? > >>> > >>>> > >>>> On Nov 14, 2013, at 10:13 PM, Kostya Serebryany <kcc at google.com> wrote: > >>>> > >>>>> What are the symptoms? > >>>>> > >>>>> On Fri, Nov 15, 2013 at 9:37 AM, Bob Wilson <bob.wilson at apple.com> wrote: > >>>>>> I’m waiting to see if this fixes the buildbots. Unfortunately, because they were failing all day, there are a bunch of other regressions that have come up, and I’m still working through them. It takes quite a while to run a bootstrapped LTO clang build, so it will take a while longer. > >>>>>> > >>>>>> I don’t have any other useful information at this point, and I share your puzzlement about how your changes could possibly break the compiler. My only hypothesis is some sort of memory corruption. > >>>>>> > >>>>>> I will keep you posted. > >>>>>> > >>>>>> On Nov 14, 2013, at 9:22 PM, Kostya Serebryany <kcc at google.com> wrote: > >>>>>> > >>>>>>> Also, when are you planing to "reapply the changes or help debug"? > >>>>>>> > >>>>>>> On Fri, Nov 15, 2013 at 9:15 AM, Kostya Serebryany <kcc at google.com> wrote: > >>>>>>>> On Fri, Nov 15, 2013 at 7:26 AM, Bob Wilson <bob.wilson at apple.com> wrote: > >>>>>>>>> Hi Kostya, > >>>>>>>>> > >>>>>>>>> Thanks for the heads-up on this. I haven’t had a chance to look into the > >>>>>>>>> details yet, but it looks like these patches may be breaking our > >>>>>>>>> bootstrapped LTO build. Our buildbots have been failing all day, and we’re > >>>>>>>>> still trying to figure out the problem. I’m going to speculatively revert > >>>>>>>>> those changes, since they were the only patches on the buildbot blame list. > >>>>>>>>> I will either reapply the changes or help debug the problem. > >>>>>>>> > >>>>>>>> How could this possibly affect your LTO build? > >>>>>>>> The option is off by default. > >>>>>>>> Do you have any details, logs, etc? > >>>>>>>> > >>>>>>>>> > >>>>>>>>> —Bob > >>>>>>>>> > >>>>>>>>> On Nov 14, 2013, at 5:42 AM, Kostya Serebryany <kcc at google.com> wrote: > >>>>>>>>> > >>>>>>>>> Bob, Justin, > >>>>>>>>> > >>>>>>>>> I've just committed a poor man's coverage implementation that works with > >>>>>>>>> asan. > >>>>>>>>> http://llvm.org/viewvc/llvm-project?rev=194701&view=rev > >>>>>>>>> http://llvm.org/viewvc/llvm-project?rev=194702&view=rev > >>>>>>>>> It provides only function-level boolean coverage (i.e. no counters, just > >>>>>>>>> "visited or not"), > >>>>>>>>> but is very fast and very simple (no extra sections to the binary file, etc) > >>>>>>>>> I've tried it for Chrome's content_shell (huge and heavy binary) and the > >>>>>>>>> overhead > >>>>>>>>> is negligible at both run-time and shutdown-time. > >>>>>>>>> > >>>>>>>>> We'll be evaluating this implementation and collecting usage stats. > >>>>>>>>> Maybe we want to implement something simple like this in the Clang coverage. > >>>>>>>>> > >>>>>>>>> --kcc > >>>>>>>>> > >>>>>>>>> > >>>>>> > >>>> > >> > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > -- > Alexey Samsonov, MSK > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131114/9854a243/attachment.html>
On Fri, Nov 15, 2013 at 11:23 AM, Bob Wilson <bob.wilson at apple.com> wrote:> I was able to successfully build with r194701, so I have reapplied that > along with the compiler-rt changes.Good! When you have a chance, please take a look at the change. I'd like to see if we can do something like this for clang coverage.> Somehow we need to get to the bottom of > the problem. If you can reproduce it, please help!!Alexey sees some memory corruption which is blocking him too --kcc> > On Nov 14, 2013, at 11:01 PM, Eric Christopher <echristo at gmail.com> wrote: > > Yes. I'm seeing this as well and it predates Kostya's change. Shows up as > memory corruption or double free on Linux. > > On Nov 14, 2013 10:46 PM, "Alexey Samsonov" <samsonov at google.com> wrote: >> >> FYI I've seen what looked like a memory corruption in (private) Clang >> bootstrap process long before Kostya's changes were submitted. I hope I'll >> have the chance to investigate it soon. >> >> >> On Fri, Nov 15, 2013 at 10:20 AM, Bob Wilson <bob.wilson at apple.com> wrote: >>> >>> I don’t know yet, but I will let you know as soon as I can. >>> >>> On Nov 14, 2013, at 10:18 PM, Kostya Serebryany <kcc at google.com> wrote: >>> >>> > On Fri, Nov 15, 2013 at 10:16 AM, Bob Wilson <bob.wilson at apple.com> >>> > wrote: >>> >> No, not that I am aware of. >>> > >>> > So, if my commits did indeed trigger the failures it could be >>> > something like binary size change that caused different code alignment >>> > or some such >>> > and which triggered a latent memory bug somewhere else. >>> > >>> > It's already late evening for you now. Will you have a chance to >>> > reapply changes today? >>> > >>> > --kcc >>> > >>> >> >>> >> On Nov 14, 2013, at 10:15 PM, Kostya Serebryany <kcc at google.com> >>> >> wrote: >>> >> >>> >>> On Fri, Nov 15, 2013 at 10:14 AM, Bob Wilson <bob.wilson at apple.com> >>> >>> wrote: >>> >>>> The bit code file produced by the stage 1 compiler for one of the >>> >>>> files in the clang driver is corrupt and causes the linker for stage 2 to >>> >>>> crash. >>> >>> >>> >>> Is AddressSanitizer involved in any of the stages of the LTO build? >>> >>> >>> >>>> >>> >>>> On Nov 14, 2013, at 10:13 PM, Kostya Serebryany <kcc at google.com> >>> >>>> wrote: >>> >>>> >>> >>>>> What are the symptoms? >>> >>>>> >>> >>>>> On Fri, Nov 15, 2013 at 9:37 AM, Bob Wilson <bob.wilson at apple.com> >>> >>>>> wrote: >>> >>>>>> I’m waiting to see if this fixes the buildbots. Unfortunately, >>> >>>>>> because they were failing all day, there are a bunch of other regressions >>> >>>>>> that have come up, and I’m still working through them. It takes quite a >>> >>>>>> while to run a bootstrapped LTO clang build, so it will take a while longer. >>> >>>>>> >>> >>>>>> I don’t have any other useful information at this point, and I >>> >>>>>> share your puzzlement about how your changes could possibly break the >>> >>>>>> compiler. My only hypothesis is some sort of memory corruption. >>> >>>>>> >>> >>>>>> I will keep you posted. >>> >>>>>> >>> >>>>>> On Nov 14, 2013, at 9:22 PM, Kostya Serebryany <kcc at google.com> >>> >>>>>> wrote: >>> >>>>>> >>> >>>>>>> Also, when are you planing to "reapply the changes or help >>> >>>>>>> debug"? >>> >>>>>>> >>> >>>>>>> On Fri, Nov 15, 2013 at 9:15 AM, Kostya Serebryany >>> >>>>>>> <kcc at google.com> wrote: >>> >>>>>>>> On Fri, Nov 15, 2013 at 7:26 AM, Bob Wilson >>> >>>>>>>> <bob.wilson at apple.com> wrote: >>> >>>>>>>>> Hi Kostya, >>> >>>>>>>>> >>> >>>>>>>>> Thanks for the heads-up on this. I haven’t had a chance to >>> >>>>>>>>> look into the >>> >>>>>>>>> details yet, but it looks like these patches may be breaking >>> >>>>>>>>> our >>> >>>>>>>>> bootstrapped LTO build. Our buildbots have been failing all >>> >>>>>>>>> day, and we’re >>> >>>>>>>>> still trying to figure out the problem. I’m going to >>> >>>>>>>>> speculatively revert >>> >>>>>>>>> those changes, since they were the only patches on the buildbot >>> >>>>>>>>> blame list. >>> >>>>>>>>> I will either reapply the changes or help debug the problem. >>> >>>>>>>> >>> >>>>>>>> How could this possibly affect your LTO build? >>> >>>>>>>> The option is off by default. >>> >>>>>>>> Do you have any details, logs, etc? >>> >>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> —Bob >>> >>>>>>>>> >>> >>>>>>>>> On Nov 14, 2013, at 5:42 AM, Kostya Serebryany <kcc at google.com> >>> >>>>>>>>> wrote: >>> >>>>>>>>> >>> >>>>>>>>> Bob, Justin, >>> >>>>>>>>> >>> >>>>>>>>> I've just committed a poor man's coverage implementation that >>> >>>>>>>>> works with >>> >>>>>>>>> asan. >>> >>>>>>>>> http://llvm.org/viewvc/llvm-project?rev=194701&view=rev >>> >>>>>>>>> http://llvm.org/viewvc/llvm-project?rev=194702&view=rev >>> >>>>>>>>> It provides only function-level boolean coverage (i.e. no >>> >>>>>>>>> counters, just >>> >>>>>>>>> "visited or not"), >>> >>>>>>>>> but is very fast and very simple (no extra sections to the >>> >>>>>>>>> binary file, etc) >>> >>>>>>>>> I've tried it for Chrome's content_shell (huge and heavy >>> >>>>>>>>> binary) and the >>> >>>>>>>>> overhead >>> >>>>>>>>> is negligible at both run-time and shutdown-time. >>> >>>>>>>>> >>> >>>>>>>>> We'll be evaluating this implementation and collecting usage >>> >>>>>>>>> stats. >>> >>>>>>>>> Maybe we want to implement something simple like this in the >>> >>>>>>>>> Clang coverage. >>> >>>>>>>>> >>> >>>>>>>>> --kcc >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>> >>> >>>> >>> >> >>> >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> >> >> >> >> -- >> Alexey Samsonov, MSK >> >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >