Quoting from Thomas's message -
> "On the question of which system really is easier to learn I can
> only comment that this isn't the only question where education,
> as a field, would benefit from some good randomized controlled
> trials."
A Randomized Controlled Trial?:
Doing such trials would be a 30-year project. The entry criterion
might be at least a pass score on a test that was designed to
identify students with the potential to be reasonable statistical
practitioners. (To make this work, coaching at a summer camp might
be a necessary preliminary.) Students would be introduced to
whatever system at various times in their educational development --
ages 11, 14, 18 or 24. For each age/system combination, there'd be a
variety of dose levels(!). Half would be introduced via the GUI and
half via the command line. Outcome measures would be (1) liking for
the system; (2) quality of analysis, on several analysis tasks of a
type that are likely to arise in several different analysis areas.
Assessments would be made in early career and in mid-career.
Analyses would of course be done using both SAS Proc Mixed and lmer(0
in lme4. There'd be bound to be enough missing data to make the
design unbalanced, hence allowing plenty of room for argument about
the informativeness of the missingness, and about the adequacy of the
degrees of freedom approximation, or whether an approach that uses a
df approximation was even worth considering.
What happens with those who decide, of their own accord or from
necessity, to learn a system additional to the one to which they were
assigned? (This may itself be an outcome.) Should there be control
for exposure to another language?
The more one thinks about it, the worse the design problem gets. The
situation is a bit different from the teaching of reading, where high
quality randomized trials can and should be done, notwithstanding the
complications of controlling for teacher effects. As always, it is
however insightful to think about the randomized trial that would be
required.
I can envisage a simple randomized trial, still extending over some
years, where the outcome measure is the quality of statistical
analysis, on problems that meet the criteria given above.
The height of the bar:
For proper comparison of ease of doing analyses, a staged set of
analysis problems is required, from cases where most would agree that
a t-test or chi-square test ot CI or ... "answers" the question of
interest, through to a variety of realistic regression problems.
Agreement on some minimal set of steps needed to do an adequate
analysis would be a necessary part of the process. This insists that
the goalposts are always at the same height. Such an exercise could
be highly insightful, and a useful contribution to the public
scientific good.
Research questions:
To a smaller or larger extent, R is a component of a research
exercise in the development of statistical computational abilities.
Perhaps to the majority of users on this list, it is primarily an
effective tool for the handling of statistical and other scientific
computing tasks. Some see these two goals as somewhat distinct (at
the boundaries, they obviously are); others see a large overlap.
In any case, this latter role has enormous importance, actual and
potential, for the scientific community, and indeed for any area
(especially business) where there is a continual and insistent demand
to make sense of data. A variety of research questions that warrant
attention:
(1) Who should learn R?
[In my view R is such a versatile tool for scientific computing that
anyone contemplating a career in science, and who expects to to their
own computations that have a substantial data analysis component,
should learn R. The only serious competitors, in my view and
depending on the area of application, are Genstat, Stata, and Matlab
-- Genstat for the analysis of designed experiments and for the
quality of its GUI, Stata for the reasons given by others, and Matlab
for signal processsing. SAS may be important for its efficiency in
certain types of batch processing with large data sets, and because
of the extent of existing large SAS repositories, SPSS may be
important because of the extent of existing large SPSS data
repositories. Some comment is also needed on S-PLUS? I am of course
ignoring the skill investment that many researchers have made in
these other packages. While this has somehow to be factored in, it
surely has limited relevance to assessing priorities for those who
are currently starting out.]
(2) R has clearly reduced the time lag between the development of new
theory, and availability of the associated methodology to statistical
practitioners. It has also, incidentally, raised the bar for
commercial statistical software systems. What are the implications
for statistical research, and for professional practice and training?
(3) Should learners use a GUI, or the command line, in getting started?
[A major issue for GUIs is documentation of steps in an analysis.
This will become increasingly important as more journals demand, as I
hope will happen, publication of Sweave or other reproducible
versions of analyses. Some ultimate familiarity with the command
line may in the medium term be essential.]
(4) When should students start learning R?
[Students should get their first exposure to a high-level programming
language, in the style of R then Python or Octave, at age 11-14.
There are now good alternatives to the former use of Fortran or
Pascal, languages which have for good reason dropped out of favour
for learning experience. They should start on R while their minds are
still malleable, and long before they need it for serious research use.]
(5) What are the traps, in using R, for relative novices?
[Mechanisms are needed for identifying traps that routinely catch
novices (even novices who may be quite sophisticated statistically),
with a program to tackle these, in the medium to long term.]
(6) Default output requires (continuing) careful scrutiny from a
"what will encourage good statistical practice" perspective.
(7) What, more widely, should go on the wish list?
John Maindonald email: john.maindonald at anu.edu.au
phone : +61 2 (6125)3473 fax : +61 2(6125)5549
Centre for Mathematics & Its Applications, Room 1194,
John Dedman Mathematical Sciences Building (Building 27)
Australian National University, Canberra ACT 0200.
On 4 Jan 2006, at 10:00 PM, r-help-request at stat.math.ethz.ch wrote:
> From: Thomas Lumley <tlumley at u.washington.edu>
> Date: 4 January 2006 6:23:18 AM
> To: Peter Dalgaard <p.dalgaard at biostat.ku.dk>
> Cc: R-help at stat.math.ethz.ch, Patrick Burns <pburns at
pburns.seanet.com>
> Subject: Re: [R] A comment about R:
>
>
> On Tue, 3 Jan 2006, Peter Dalgaard wrote:
>> One thing that is often overlooked, and hasn't yet been mentioned
in
>> the thread, is how much *simpler* R can be for certain completely
>> basic tasks of practical or pedagogical relevance: Calculate a simple
>> derived statistic, confidence intervals from estimate and SE,
>> percentage points of the binomial distribution - using dbinom or from
>> the formula, take the sum of each of 10 random samples from a set of
>> numbers, etc. This is where other packages get stuck in the
>> procedure+dataset mindset.
>
> Some of these things are actually fairly straightforward in Stata.
> For example, Stata will give confidence intervals and tests for
> linear combinations of coefficients and even (using symbolic
> differentiation and the delta method) for nonlinear combinations.
> The first is available in packages in R, the second is in "S
> Programming" but doesn't seem to be packaged.
>
> <snip>
>
> Now, I still prefer R both for data analysis and (even more so) for
> programming. There are some things that it is genuinely difficult
> to program in Stata -- and as evidence that this isn't just my
> ignorance of the best approaches, the language was substantially
> reworked in both versions 8 and 9 to allow the vendor to implement
> better graphics and
> linear mixed models.
>
> On the question of which system really is easier to learn I can
> only comment that this isn't the only question where education, as
> a field, would benefit from some good randomized controlled trials.
>
> -thomas
>
> Thomas Lumley Assoc. Professor, Biostatistics
> tlumley at u.washington.edu University of Washington, Seattle