I'm working on an update to the patch. The only thing holding me up is trying to come to a final decision as to where all the various pieces should live. Specifically, the Google Test library, and the actual unit tests themselves. On Mon, Dec 29, 2008 at 9:17 AM, Chris Lattner <clattner at apple.com> wrote:> On Dec 27, 2008, at 8:54 PM, Talin wrote: > > Although I haven't actually tried Boost.Test, I kind of figured that > > this would be the case - that you pretty much have to drink the "Boost > > Kool-Aid" in order to use it. > > I agree, boost.test seems like a non-starter from many reasons. > > >> So are you planning on maintaining whatever test system, or just have > >> them as a pre-requisite. For example are you going to have the gtest > >> incorporated, or have them install it separately first? I was under > >> the impression that the user would have to install gtest first. > > So the plan is to take a snapshot of gtest and check it in to the LLVM > > tree, rather than have it installed separately. > > Yes, we'd want to either do this, or have the configure machinery > detect an installed copy and only enable unit testing if present. If > the goal is to make everyone run unit tests, embedding a copy would be > the best way to go. > > > This means that keeping the gtest snapshot up to date will be trivial, > > since it will only require copying in the latest gtest snapshot and > > checking it in to LLVM - presuming that gtest remains backwards > > compatible, which I assume it will. > > This is only an issue if the latest and greatest versions of gtest add > something we need. My understanding of unit testing is fairly > limited, but I don't forsee major new innovations regularly happening > that we'll need to bring in on a frequent basis. Doing a periodic > synch (once every year or two) will probably be fine. > > > > > Licensing-wise, both LLVM and gtest are distributed under a fairly > > permissive BSD-style license. I don't know who would make the > > judgement > > call as to whether or not the licenses can co-exist. However, since > > neither license is "viral" in the sense of wanting to apply any sort > > of > > restriction on derived works or the "work as a whole", I see no > > barrier > > to shipping a combined product with different portions falling under > > different licenses. Thus, the unit tests themselves would still fall > > under the LLVM license, and linking the unit tests with gtest would > > not > > violate either license. Of course, IANAL. > > The licenses coexist. Please just add an entry to the llvm/ > LICENSES.txt file. > > > My personal goal is that one should be able to check out the head of > > LLVM on a generic Linux/OS-X system, with only the standard > > development > > environment (i.e. make/gcc/etc) and type "make unittest" and have the > > tests run. > > Yep. BTW, do these tests run in parallel with make -jN? If not, > please make them :) > > > In the longer term, I'd like to see LLVM have an automated build that > > runs the unit tests as part of the build. I noticed that gtest has an > > option to output test results in XML form, although I have not played > > with this personally, it might be useful in this regard. > > The only thing I'm concerned about here is extending build time. For > example, if we have a lot of unit tests for libsupport (e.g. > densemap), I wouldn't want those tests to be run unless libsupport > changes. It should just be a "small matter of makefile hackery" to > get this going I suppose. > > Is there a final proposed version of the patch with the changes Misha > asked for? If so, I'll review and commit it when I have time. > > -Chris > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-- -- Talin -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20081229/e8dafd6a/attachment.html>
I'd put gtest into llvm/utils and the tests themselves into a new llvm/ unittests directory. Also please update TestingGuide.html, thanks! -Chris On Dec 29, 2008, at 1:30 PM, Talin <viridia at gmail.com> wrote:> I'm working on an update to the patch. The only thing holding me up > is trying to come to a final decision as to where all the various > pieces should live. Specifically, the Google Test library, and the > actual unit tests themselves. > > On Mon, Dec 29, 2008 at 9:17 AM, Chris Lattner <clattner at apple.com> > wrote: > On Dec 27, 2008, at 8:54 PM, Talin wrote: > > Although I haven't actually tried Boost.Test, I kind of figured that > > this would be the case - that you pretty much have to drink the > "Boost > > Kool-Aid" in order to use it. > > I agree, boost.test seems like a non-starter from many reasons. > > >> So are you planning on maintaining whatever test system, or just > have > >> them as a pre-requisite. For example are you going to have the > gtest > >> incorporated, or have them install it separately first? I was under > >> the impression that the user would have to install gtest first. > > So the plan is to take a snapshot of gtest and check it in to the > LLVM > > tree, rather than have it installed separately. > > Yes, we'd want to either do this, or have the configure machinery > detect an installed copy and only enable unit testing if present. If > the goal is to make everyone run unit tests, embedding a copy would be > the best way to go. > > > This means that keeping the gtest snapshot up to date will be > trivial, > > since it will only require copying in the latest gtest snapshot and > > checking it in to LLVM - presuming that gtest remains backwards > > compatible, which I assume it will. > > This is only an issue if the latest and greatest versions of gtest add > something we need. My understanding of unit testing is fairly > limited, but I don't forsee major new innovations regularly happening > that we'll need to bring in on a frequent basis. Doing a periodic > synch (once every year or two) will probably be fine. > > > > > Licensing-wise, both LLVM and gtest are distributed under a fairly > > permissive BSD-style license. I don't know who would make the > > judgement > > call as to whether or not the licenses can co-exist. However, since > > neither license is "viral" in the sense of wanting to apply any sort > > of > > restriction on derived works or the "work as a whole", I see no > > barrier > > to shipping a combined product with different portions falling under > > different licenses. Thus, the unit tests themselves would still fall > > under the LLVM license, and linking the unit tests with gtest would > > not > > violate either license. Of course, IANAL. > > The licenses coexist. Please just add an entry to the llvm/ > LICENSES.txt file. > > > My personal goal is that one should be able to check out the head of > > LLVM on a generic Linux/OS-X system, with only the standard > > development > > environment (i.e. make/gcc/etc) and type "make unittest" and have > the > > tests run. > > Yep. BTW, do these tests run in parallel with make -jN? If not, > please make them :) > > > In the longer term, I'd like to see LLVM have an automated build > that > > runs the unit tests as part of the build. I noticed that gtest has > an > > option to output test results in XML form, although I have not > played > > with this personally, it might be useful in this regard. > > The only thing I'm concerned about here is extending build time. For > example, if we have a lot of unit tests for libsupport (e.g. > densemap), I wouldn't want those tests to be run unless libsupport > changes. It should just be a "small matter of makefile hackery" to > get this going I suppose. > > Is there a final proposed version of the patch with the changes Misha > asked for? If so, I'll review and commit it when I have time. > > -Chris > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > -- > -- Talin > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20081229/97cfe32f/attachment.html>
2008/12/29 Talin <viridia at gmail.com>> I'm working on an update to the patch. The only thing holding me up is > trying to come to a final decision as to where all the various pieces should > live. Specifically, the Google Test library, and the actual unit tests > themselves.I would recommend putting Google Test in llvm/test/googletest . The unittests should probably go in llvm/test with other tests, in the same directories, e.g., tests of the various analysis algorithms would live in llvm/test/Analysis, 1 file per analysis, unless folks think they should go further into the tree: llvm/test/Analysis/Andersens/, etc., but I think it would be cleaner and easier if the files for a single library were in the same directory, as we would be able to create a single unittest from all of them, to speed up the build/link/test cycle. Thoughts? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20081229/b4be9b64/attachment.html>
Do the existing tests use a wildcard rule to gather all test sources? We would need to insure that the wildcard rule for the unit tests and the large tests are mutually exclusive. Also, will the unit tests be one executable or many? They will probably compile/run faster if there is a smaller number of executables. On Mon, Dec 29, 2008 at 1:43 PM, Misha Brukman <brukman at gmail.com> wrote:> 2008/12/29 Talin <viridia at gmail.com> > >> I'm working on an update to the patch. The only thing holding me up is >> trying to come to a final decision as to where all the various pieces should >> live. Specifically, the Google Test library, and the actual unit tests >> themselves. > > > I would recommend putting Google Test in llvm/test/googletest . The > unittests should probably go in llvm/test with other tests, in the same > directories, e.g., tests of the various analysis algorithms would live in > llvm/test/Analysis, 1 file per analysis, unless folks think they should go > further into the tree: llvm/test/Analysis/Andersens/, etc., but I think it > would be cleaner and easier if the files for a single library were in the > same directory, as we would be able to create a single unittest from all of > them, to speed up the build/link/test cycle. > > Thoughts? > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > >-- -- Talin -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20081229/f9a6175e/attachment.html>