Reid Kleckner
2015-May-28 21:24 UTC
[LLVMdev] WinEH work to be done (in progress and otherwise)
I think we should make a new svn repository for these. test-suite is the logical place, but it's way too bloated and has too much Unix specific baggage. I'd really like to have something that just uses lit, can be checked out into projects, and can be run as part of check-all, like the compiler-rt execution test suite. My suggestion is to call it 'execution-tests', and we can arrange to auto-detect it if it gets checked out under llvm/projects/ and set up a test suite. On Thu, May 28, 2015 at 1:35 PM, Joseph Tremoulet <jotrem at microsoft.com> wrote:> Hi, > > > > Have any decisions been made on where to put executable WinEH tests? > Wherever they go, would it be helpful if I were to add some C++EH and/or > SEH tests pulled from MSVC's test suite, since compatibility is the goal > here? > > > > Thanks > > -Joseph > > > > *From:* llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] *On > Behalf Of *Reid Kleckner > *Sent:* Tuesday, May 19, 2015 5:56 PM > *To:* Kaylor, Andrew > *Cc:* llvmdev at cs.uiuc.edu > *Subject:* Re: [LLVMdev] WinEH work to be done (in progress and otherwise) > > > > On Tue, May 19, 2015 at 2:16 PM, Kaylor, Andrew <andrew.kaylor at intel.com> > wrote: > > Apart from this, the other WinEH-related task that I have on my radar > (but as of yet haven’t done anything about) is getting “nightly” tests > running on Windows. I did look into the instructions for running the > “test-suite” tests, and it looks like that test suite is assuming a > Unix-type platform to some extent. I tried to re-interpret the instruction > and run this suite on Windows, but it didn’t work and I haven’t looked into > trying to fix whatever the problems were. I saw that someone submitted a > patch last November that claimed to improve the running of these tests on > Windows, but the patch was never reviewed or committed. > > > > So I was wondering, is anyone running compile-and-execute type tests > with clang on a native (i.e. non-cygwin/mingw) Windows platform? It seems > like this will be an important thing to do in general, but particularly so > for Windows EH, dependent as it is on interaction with the runtime > library. That is, having tests which verify that we’re producing the xdata > entries we intend to produce doesn’t seem like a very thorough method of > testing and is no guarantee at all that we are producing working > executables. In addition to the existing C++ EH tests in the “test-suite” > tests, I have a suite of tests (not currently in any public repository) > that were specifically developed to exercise Windows-specific problem > cases. I’d eventually like to see these tests run on some regular basis > against clang on Windows. Is there any work currently under way to make > something like this happen? > > > > I'd like to have a place where we can put small, general purpose execution > tests for clang, but we don't have a good place. test-suite is just too big > and non-portable. The results it generates are also not suitable for > consumption by developers. If someone upstream breaks a C++ EH execution > test, what should they do, how should they debug? A portable > assembly-emission test works because they can see the before and after and > evaluate whether their change was correct or not. > > > > For the rest of the MS C++ ABI, we've gotten by self-hosting Clang and > then building Chromium, and that's enough coverage. Neither project uses > C++ exceptions, so that's not going to cut it for this. Long ago Timur used > to maintain a small repo in Google Code for this: > > https://code.google.com/p/smoke-cpp-tests/ > > Nobody runs it though and we haven't updated as we encountered and solved > problems. > > > > I think the best we can do is add a subdirectory of test-suite that's > outside it's makefile system and go from there. Or we could add a new repo > similar to the ABI test suite that Sony contributed. However, you may see > that as a cautionary tale. Sony added the test suite, and nobody else runs > it and the bot has been broken for a month. > > > > I guess my conclusion is that this will probably be a lot of work, and I > think it'll provide less value than you expect from it. But, if you think > it's worth it and want to push it through, then it costs LLVM very little > check a few test cases in outside the main clang/llvm repos. >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150528/2ebf9d08/attachment.html>
Jim Radigan
2015-Jun-19 15:24 UTC
[LLVMdev] WinEH work to be done (in progress and otherwise)
Hi, I have all the C/C++ legacy tests for exception handling on all the Windows platforms. We use these daily for all our verification. I’d like to donate and participate somehow. Can I just I simply add these to your existing testing infrastructure. Thanks, -Jim Radigan From: llvmdev-bounces at cs.uiuc.edu<mailto:llvmdev-bounces at cs.uiuc.edu> [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of Chandler Carruth Sent: Thursday, May 28, 2015 3:29 PM To: Reid Kleckner; Joseph Tremoulet CcSubject: Re: [LLVMdev] WinEH work to be done (in progress and otherwise) We already have a "unittests" tree in the test-suite. If you want to create a new repository, you need to widely discuss it and have a good plan for integrating it into build bots etc. On Thu, May 28, 2015 at 2:36 PM Reid Kleckner <rnk at google.com<mailto:rnk at google.com>> wrote: I think we should make a new svn repository for these. test-suite is the logical place, but it's way too bloated and has too much Unix specific baggage. I'd really like to have something that just uses lit, can be checked out into projects, and can be run as part of check-all, like the compiler-rt execution test suite. My suggestion is to call it 'execution-tests', and we can arrange to auto-detect it if it gets checked out under llvm/projects/ and set up a test suite. On Thu, May 28, 2015 at 1:35 PM, Joseph Tremoulet <jotrem at microsoft.com<mailto:jotrem at microsoft.com>> wrote: Hi, Have any decisions been made on where to put executable WinEH tests? Wherever they go, would it be helpful if I were to add some C++EH and/or SEH tests pulled from MSVC's test suite, since compatibility is the goal here? Thanks -Joseph From: llvmdev-bounces at cs.uiuc.edu<mailto:llvmdev-bounces at cs.uiuc.edu> [mailto:llvmdev-bounces at cs.uiuc.edu<mailto:llvmdev-bounces at cs.uiuc.edu>] On Behalf Of Reid Kleckner Sent: Tuesday, May 19, 2015 5:56 PM To: Kaylor, Andrew Cc: llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu> Subject: Re: [LLVMdev] WinEH work to be done (in progress and otherwise) On Tue, May 19, 2015 at 2:16 PM, Kaylor, Andrew <andrew.kaylor at intel.com<mailto:andrew.kaylor at intel.com>> wrote: Apart from this, the other WinEH-related task that I have on my radar (but as of yet haven’t done anything about) is getting “nightly” tests running on Windows. I did look into the instructions for running the “test-suite” tests, and it looks like that test suite is assuming a Unix-type platform to some extent. I tried to re-interpret the instruction and run this suite on Windows, but it didn’t work and I haven’t looked into trying to fix whatever the problems were. I saw that someone submitted a patch last November that claimed to improve the running of these tests on Windows, but the patch was never reviewed or committed. So I was wondering, is anyone running compile-and-execute type tests with clang on a native (i.e. non-cygwin/mingw) Windows platform? It seems like this will be an important thing to do in general, but particularly so for Windows EH, dependent as it is on interaction with the runtime library. That is, having tests which verify that we’re producing the xdata entries we intend to produce doesn’t seem like a very thorough method of testing and is no guarantee at all that we are producing working executables. In addition to the existing C++ EH tests in the “test-suite” tests, I have a suite of tests (not currently in any public repository) that were specifically developed to exercise Windows-specific problem cases. I’d eventually like to see these tests run on some regular basis against clang on Windows. Is there any work currently under way to make something like this happen? I'd like to have a place where we can put small, general purpose execution tests for clang, but we don't have a good place. test-suite is just too big and non-portable. The results it generates are also not suitable for consumption by developers. If someone upstream breaks a C++ EH execution test, what should they do, how should they debug? A portable assembly-emission test works because they can see the before and after and evaluate whether their change was correct or not. For the rest of the MS C++ ABI, we've gotten by self-hosting Clang and then building Chromium, and that's enough coverage. Neither project uses C++ exceptions, so that's not going to cut it for this. Long ago Timur used to maintain a small repo in Google Code for this: https://code.google.com/p/smoke-cpp-tests/<https://urldefense.proofpoint.com/v2/url?u=https-3A__code.google.com_p_smoke-2Dcpp-2Dtests_&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Mfk2qtn1LTDThVkh6-oGglNfMADXfJdty4_bhmuhMHA&m=nSJRS9-5NwSLdiLng62PIvrk3rcZRDmpF2oboUMhpgE&s=4QsKoKOgmJ6EUy3biMinr91bQc7pSbgjbvRibkg6lIA&e=> Nobody runs it though and we haven't updated as we encountered and solved problems. I think the best we can do is add a subdirectory of test-suite that's outside it's makefile system and go from there. Or we could add a new repo similar to the ABI test suite that Sony contributed. However, you may see that as a cautionary tale. Sony added the test suite, and nobody else runs it and the bot has been broken for a month. I guess my conclusion is that this will probably be a lot of work, and I think it'll provide less value than you expect from it. But, if you think it's worth it and want to push it through, then it costs LLVM very little check a few test cases in outside the main clang/llvm repos. _______________________________________________ LLVM Developers mailing list LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu> http://llvm.cs.uiuc.edu http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150619/40652105/attachment.html>