On Oct 11, 2008, at 8:41 PM, OvermindDL1 wrote:>> If you were interested in working on this project, it seems that the >> place to start is not by adding new crazy sorts of API tests for >> LLVM. It would be a better use of your time to take what we already >> have and make it work (with one of your proposals) on windows. When >> the infrastructure is set up, others could add tests in areas that >> they are knowledgeable about. > > I am not sure how well the current tests could be ported though, they > are very gcc-centric, as in compile with llvm-gcc and execute what > llvm-gcc compiled, not very useful as an api test. I have not really > looked much into the current test setup once I noticed how useless it > was for me, but is there a certain part of it that does not touch gcc > type things, that literally just tests llvm directly without any > middle stuff like llvm-gcc? If so, I could probably work on getting > that into a more unified framework...Are you talking about the test-suite SVN module? If so, I agree with you. It seems much more useful to start with the llvm/test subdirectory of unit tests. Those mostly don't depend on llvm-gcc and the few ones that do should be moved elsewhere anyway. -Chris
Chris Lattner <clattner at apple.com> writes:> On Oct 11, 2008, at 8:41 PM, OvermindDL1 wrote: > >> I am not sure how well the current tests could be ported though, they >> are very gcc-centric, as in compile with llvm-gcc and execute what >> llvm-gcc compiled, not very useful as an api test. I have not really >> looked much into the current test setup once I noticed how useless it >> was for me, but is there a certain part of it that does not touch gcc >> type things, that literally just tests llvm directly without any >> middle stuff like llvm-gcc? If so, I could probably work on getting >> that into a more unified framework... > > Are you talking about the test-suite SVN module? If so, I agree with > you. It seems much more useful to start with the llvm/test > subdirectory of unit tests. Those mostly don't depend on llvm-gcc and > the few ones that do should be moved elsewhere anyway.How much value is on adding cross-platform support for the tests in llvm/test? I've looked at them and the task seems achievable with not too much work. Am I right suppossing that a non-zero return value for the "RUN:" command(s) means failure, and zero means success, and there is no more stuff than that to worry about for knowing the result? -- Oscar
Óscar Fuentes wrote:> Chris Lattner <clattner at apple.com> writes: > > >> On Oct 11, 2008, at 8:41 PM, OvermindDL1 wrote: >> >> >>> I am not sure how well the current tests could be ported though, they >>> are very gcc-centric, as in compile with llvm-gcc and execute what >>> llvm-gcc compiled, not very useful as an api test. I have not really >>> looked much into the current test setup once I noticed how useless it >>> was for me, but is there a certain part of it that does not touch gcc >>> type things, that literally just tests llvm directly without any >>> middle stuff like llvm-gcc? If so, I could probably work on getting >>> that into a more unified framework... >>> >> Are you talking about the test-suite SVN module? If so, I agree with >> you. It seems much more useful to start with the llvm/test >> subdirectory of unit tests. Those mostly don't depend on llvm-gcc and >> the few ones that do should be moved elsewhere anyway. >> > > How much value is on adding cross-platform support for the tests in > llvm/test? I've looked at them and the task seems achievable with not > too much work. >For the *.ll tests, quite a bit. It's not too much work up-front if you hardcode the test sequence in the alternate test driver framework.> Am I right suppossing that a non-zero return value for the "RUN:" > command(s) means failure, and zero means success, and there is no more > stuff than that to worry about for knowing the result? >In general there are two return codes of interest: the one from the actual tool run (usually llvm-as), and the one from the final grep. I'm a bit shaky on how pipes behave, but I think that in a pipe sequence only the last command's code is available. For grep (MingW32), the return codes are: "Exit status is 0 if match, 1 if no match, and 2 if trouble." We can get away with eliding no-match vs trouble for grep, as presumably a release version would never report trouble when properly invoked. Kenneth
On Oct 11, 2008, at 11:35 PM, Óscar Fuentes wrote:> Chris Lattner <clattner at apple.com> writes: > >> On Oct 11, 2008, at 8:41 PM, OvermindDL1 wrote: >> >>> I am not sure how well the current tests could be ported though, >>> they >>> are very gcc-centric, as in compile with llvm-gcc and execute what >>> llvm-gcc compiled, not very useful as an api test. I have not >>> really >>> looked much into the current test setup once I noticed how useless >>> it >>> was for me, but is there a certain part of it that does not touch >>> gcc >>> type things, that literally just tests llvm directly without any >>> middle stuff like llvm-gcc? If so, I could probably work on getting >>> that into a more unified framework... >> >> Are you talking about the test-suite SVN module? If so, I agree with >> you. It seems much more useful to start with the llvm/test >> subdirectory of unit tests. Those mostly don't depend on llvm-gcc >> and >> the few ones that do should be moved elsewhere anyway. > > How much value is on adding cross-platform support for the tests in > llvm/test? I've looked at them and the task seems achievable with not > too much work.It would be a huge benefit. Those tests cover a significant portion of the sanity checks in the compiler as well as indirect api tests.> Am I right suppossing that a non-zero return value for the "RUN:" > command(s) means failure, and zero means success, and there is no more > stuff than that to worry about for knowing the result?Yep exactly. -Chris