David Greene <dag at cray.com> writes:>> For my test suite I use Tcl (with TclX, no Expect). It watches stdout >> and stderr, gets exit codes and has a timer for killing hanged >> processes. Process control works the same on Windows and Unix and takes >> a less than 30 lines of code. >> >> What else do you need? > > A way to examine asm output and compate to expected patterns.This is a text manipulation task, isn't it? No problem.> A way to run performance regression tests (i.e. looking at CPU time > and allowing for fuzz between test runs).Do you mean that you want a language with that feature built-in?> In my experience Tcl is very hard to work with due to the > non-existence of any reasonable debugging tools on all platforms.For one reason or another, my Tcl code never grows so large that I miss a debugger. A `puts' here and there plus the repl is enough :-) -- Oscar
On Monday 04 August 2008 16:59, Óscar Fuentes wrote:> David Greene <dag at cray.com> writes: > >> For my test suite I use Tcl (with TclX, no Expect). It watches stdout > >> and stderr, gets exit codes and has a timer for killing hanged > >> processes. Process control works the same on Windows and Unix and takes > >> a less than 30 lines of code. > >> > >> What else do you need? > > > > A way to examine asm output and compate to expected patterns. > > This is a text manipulation task, isn't it? No problem.No problem with Perl either, or Python. Tcl is much less well-known. Note that I don't particularly like any of these languages but I'm trying not to let personal preference get in the way. :)> > A way to run performance regression tests (i.e. looking at CPU time > > and allowing for fuzz between test runs). > > Do you mean that you want a language with that feature built-in?No, I mean in the future we should have tests that actually pass/fail based on their runtime performance. To do that you need a way to time the test and a way to account for normal system variations (the fuzz bit). We don't have any of these kinds of tests yet. But I hope we do in the future.> > In my experience Tcl is very hard to work with due to the > > non-existence of any reasonable debugging tools on all platforms. > > For one reason or another, my Tcl code never grows so large that I miss > a debugger. A `puts' here and there plus the repl is enough :-)How many people know Tcl? That has a direct impact on maintanability. -Dave
David Greene <dag at cray.com> writes:> No problem with Perl either, or Python. Tcl is much less well-known. > > Note that I don't particularly like any of these languages but I'm trying > not to let personal preference get in the way. :)Neither I do. This sub-thread started with a discussion about the feasibility of leaving behing DejaGNU and using `bash' instead or:> > If a language conceals the differences between POSIX and Microsoftian C > > for interprocess control, hides badly non-POSIX filepaths from me, and > > has a reasonable track record of forward compatibility, and has a > > reasonably universal build process, I'll consider crash-learning the > > language if I'm not already working in it.Then I proposed Tcl as an example of such language. Maybe Perl, Python, Ruby, Lua and others are okay too, but: [snip]> How many people know Tcl? That has a direct impact on maintanability.DejaGNU is built around Expect, wich is a Tcl extension, so LLVM is already using Tcl. Tcl is pretty simple. You can learn Tcl in minutes, although there are some very common pitfalls (related to quoting issues, etc). OTOH, right now I'm reading llvm-config and GenLibDeps.pl, wich are Perl. I don't know Perl at all, but more or less I understand what I'm reading (1). This is because whoever wrote those scripts did it well. Had similar experiences with Python: from knowing nothing at all to extending functionality of an existing large application in minutes. So the real issue, IMHO, is not the language itself, but *who* does the job. To a large extent, he has the right to choose the tool he prefers. If he does a good job, rejecting it on the basis of the language used would be foolish, as anyone could maintain the code with minimal effort. (1) Fortunately, I have no need to understand the code that has this comment on top of it: # To understand this code, you'll need a working knowledge of Perl 5, # and possibly some quality time with 'man perlref'. -- Oscar
David Greene wrote:> On Monday 04 August 2008 16:59, Óscar Fuentes wrote: > >> David Greene <dag at cray.com> writes: >> >>>> For my test suite I use Tcl (with TclX, no Expect). It watches stdout >>>> and stderr, gets exit codes and has a timer for killing hanged >>>> processes. Process control works the same on Windows and Unix and takes >>>> a less than 30 lines of code. >>>> >>>> What else do you need? >>>> >>> A way to examine asm output and compate to expected patterns. >>> >> This is a text manipulation task, isn't it? No problem. >> > > No problem with Perl either, or Python. Tcl is much less well-known. > > Note that I don't particularly like any of these languages but I'm trying > not to let personal preference get in the way. :) >I already use both Python and Perl. I ruled out Python, even though it is a very good match feature-wise (subprocess module For The Win), because my in-house applications (about 600 lines of code, maybe 800 lines of code and comments) are not nearly as forward-compatible when avoiding deprecated constructions as the public relations statements imply; something has broken on *every* major release since I started using it (2.2 |-> 2.3. |-> 2.4 |-> 2.5). My coding style just finds those breakages that aren't caught in Python's regression testing :) I'm fine with this for small applications, but reimplementing the testcase runner looks like it's going to be large enough to make the 2.5 |-> 2.6 change a large breakage.>>> A way to run performance regression tests (i.e. looking at CPU time >>> and allowing for fuzz between test runs). >>> >> Do you mean that you want a language with that feature built-in? >> > > No, I mean in the future we should have tests that actually pass/fail based > on their runtime performance. To do that you need a way to time the test > and a way to account for normal system variations (the fuzz bit). > > We don't have any of these kinds of tests yet. But I hope we do in the > future. >Agreed. This (and the lack of a reliable cross-platform way to grab both stdout and stderr) is what has me nervous about using Perl. Kenneth