I'm making the move of the survival package from my own environment to,
and have stumbled into a vacuum. The R Extensions manual has really nice
instructions about how to lay out the directories, order the files, and
run tests for DISTRIBUTION of a product, but I can't find anything on how
to set up a reasonable DEVELOPMENT environment.
In my local world, I had the .c and .s files in a common directory, with
a Makefile that I had created, and the test suite in a subdirectory. Debugging
and development was quite nice.
make
cd test
R
attach("..")
try something and perhaps it fails
q()
cd ..
Fix and repeat. The Makefile took some time to create, but paid for itself a
hundred times over.
So, I've now rearranged everything into standard R order. Then I did the
only thing I could find
R CMD INSTALL ~/myRlib survival
where "survival" is said directory. This turns out to be not useful
at all.
The survival package is large, and I rather suspected that I would goof
something up, and I did, resulting in the following message
Error in parse(n = -1, file = file) : unexpected end of input at
14450: }
14451:
It is not exactly obvious which of the 132 files in my R/ directory is the
culprit here.
In general:
1. The library is large, and recompiling/reparsing everything is very far from
instantaneous. It is not the edit/load cycle I desire.
2. I take testing seriously: the test suite takes on the order of 15 minutes to
run on a fast machine. I most certainly don't want to run it in the mid
cycle.
Someone must have tackled this. I'm hoping that there is some
documentation
that I have managed to overlook which discussess a good setup for this middle
ground between concieving of a library and packaging it for delivery; the
"build, test, make sure it acually works" part of development.
Terry Therneau
Dear Terry, One way to locate which file is wrong - surely not the most brillant way! You could do an R script that sources each of your .R files within a "for (file in file.names)" loop. When R will stop, it will indicate which file has a wrong syntax and more info. Cheers, Mathieu Terry Therneau a ?crit :> I'm making the move of the survival package from my own environment to, > and have stumbled into a vacuum. The R Extensions manual has really nice > instructions about how to lay out the directories, order the files, and > run tests for DISTRIBUTION of a product, but I can't find anything on how > to set up a reasonable DEVELOPMENT environment. > In my local world, I had the .c and .s files in a common directory, with > a Makefile that I had created, and the test suite in a subdirectory. Debugging > and development was quite nice. > make > cd test > R > attach("..") > try something and perhaps it fails > q() > cd .. > Fix and repeat. The Makefile took some time to create, but paid for itself a > hundred times over. > > So, I've now rearranged everything into standard R order. Then I did the > only thing I could find > R CMD INSTALL ~/myRlib survival > where "survival" is said directory. This turns out to be not useful at all. > The survival package is large, and I rather suspected that I would goof > something up, and I did, resulting in the following message > > Error in parse(n = -1, file = file) : unexpected end of input at > 14450: } > 14451: > > It is not exactly obvious which of the 132 files in my R/ directory is the > culprit here. > > In general: > 1. The library is large, and recompiling/reparsing everything is very far from > instantaneous. It is not the edit/load cycle I desire. > > 2. I take testing seriously: the test suite takes on the order of 15 minutes to > run on a fast machine. I most certainly don't want to run it in the mid cycle. > > Someone must have tackled this. I'm hoping that there is some documentation > that I have managed to overlook which discussess a good setup for this middle > ground between concieving of a library and packaging it for delivery; the > "build, test, make sure it acually works" part of development. > > Terry Therneau > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >-- Institute of Mathematics Ecole Polytechnique F?d?rale de Lausanne STAT-IMA-FSB-EPFL, Station 8 CH-1015 Lausanne Switzerland http://stat.epfl.ch/ Tel: + 41 (0)21 693 7907
Hi Terry I suspect many people struggle with similar issues. The new version of mvbutils contains a number of routines that facilitate creation & maintenance of a package, hopefully through its entire life-cycle: from "documenting your own stuff for personal use", through "giving out semi-documented code that your mates can use", through "getting it through all the CRAN hoops", through "maintaining & changing an already-installed-and-loaded package". There is support for creating & editing functions (etc) in your own "development" package while it's already loaded, plus speedy on-the-fly updating of the installed version of the package. These in particular are things that R does not make easy by default (probably for good reasons). Plus: you don't have to learn Rd format if you don't want to-- the package-creator in mvbutils will generate Rd documentation for you based on plain-text documentation. Use of these package-creation-maintenance routines does require some level of buy-in to 'mvbutils's view of how your working life in R should be organized... basically 'cd' and 'fixr'. I don't reckon this should be much of a drama for most people, but there are so many different ways of working in R, so who knows? I use these routines perpetually (maintaining about 6 packages), but as far as the rest of the universe goes, they are still alpha-releases. They are geared to the way I work, and so as yet might not do what others want; however, if there's demand and feasibility, I'm happy to try to incorporate other features. In particular, the documentation is incomplete at present, and I therefore haven't over-advertised the package-maintenance stuff until I get time to write a "how-to" article. [I've had to release the package-maintenance stuff in a slightly undercooked state, because it is intimately bound to the guts of mvbutils, and it became necessary to roll out a maintenance release of mvbutils for other reasons.] Let me know if this might be of further interest to you Mark -- Mark Bravington CSIRO Mathematical & Information Sciences Marine Laboratory Castray Esplanade Hobart 7001 TAS ph (+61) 3 6232 5118 fax (+61) 3 6232 5012 mob (+61) 438 315 623 Terry Therneau wrote:> I'm making the move of the survival package from my own environment > to, > and have stumbled into a vacuum. The R Extensions manual has really > nice instructions about how to lay out the directories, order the > files, and > run tests for DISTRIBUTION of a product, but I can't find anything on > how > to set up a reasonable DEVELOPMENT environment. > In my local world, I had the .c and .s files in a common > directory, with > a Makefile that I had created, and the test suite in a subdirectory. > Debugging and development was quite nice. > make > cd test > R > attach("..") > try something and perhaps it fails > q() > cd .. > Fix and repeat. The Makefile took some time to create, but paid for > itself a hundred times over. > > So, I've now rearranged everything into standard R order. Then I > did the only thing I could find > R CMD INSTALL ~/myRlib survival > where "survival" is said directory. This turns out to be not useful > at all. The survival package is large, and I rather suspected that I > would goof something up, and I did, resulting in the following message > > Error in parse(n = -1, file = file) : unexpected end of input at > 14450: } > 14451: > > It is not exactly obvious which of the 132 files in my R/ directory > is the culprit here. > > In general: > 1. The library is large, and recompiling/reparsing everything is very > far from instantaneous. It is not the edit/load cycle I desire. > > 2. I take testing seriously: the test suite takes on the order of 15 > minutes to run on a fast machine. I most certainly don't want to run > it in the mid cycle. > > Someone must have tackled this. I'm hoping that there is some > documentation that I have managed to overlook which discussess a good > setup for this middle ground between concieving of a library and > packaging it for delivery; the "build, test, make sure it acually > works" part of development. > > Terry Therneau > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel
On 11/12/2008 6:04 PM, Terry Therneau wrote:> I'm making the move of the survival package from my own environment to, > and have stumbled into a vacuum. The R Extensions manual has really nice > instructions about how to lay out the directories, order the files, and > run tests for DISTRIBUTION of a product, but I can't find anything on how > to set up a reasonable DEVELOPMENT environment. > In my local world, I had the .c and .s files in a common directory, with > a Makefile that I had created, and the test suite in a subdirectory. Debugging > and development was quite nice. > make > cd test > R > attach("..") > try something and perhaps it fails > q() > cd .. > Fix and repeat. The Makefile took some time to create, but paid for itself a > hundred times over. > > So, I've now rearranged everything into standard R order. Then I did the > only thing I could find > R CMD INSTALL ~/myRlib survival > where "survival" is said directory. This turns out to be not useful at all. > The survival package is large, and I rather suspected that I would goof > something up, and I did, resulting in the following message > > Error in parse(n = -1, file = file) : unexpected end of input at > 14450: } > 14451: > > It is not exactly obvious which of the 132 files in my R/ directory is the > culprit here.That's something I would like to fix too. There are (at least) two possible ways: stop concatenating the files (which is not really needed nowadays, most packages install saved images), or add some markup to the concatenated file so that the parser can report on the original filename and line number (like the #LINE directives output by the C preprocessor).> > In general: > 1. The library is large, and recompiling/reparsing everything is very far from > instantaneous. It is not the edit/load cycle I desire.If you install from the directory, the compiling should only be done once (unless you change a file, of course). (The alternative is installing from the tarball, which is recommended for later stages of testing before distribution, because it's possible something could go wrong in building the tarball. But it won't include any object files, so you'll recompile every time.) You can also use option "--docs=none" to skip building the help system; this will save a bit of time.> > 2. I take testing seriously: the test suite takes on the order of 15 minutes to > run on a fast machine. I most certainly don't want to run it in the mid cycle.I don't quite follow this. If you want to run all your tests, you would use R CMD check. If you only want to run some of them, then you can source things out of the tests directory while running interactively.> Someone must have tackled this. I'm hoping that there is some documentation > that I have managed to overlook which discussess a good setup for this middle > ground between concieving of a library and packaging it for delivery; the > "build, test, make sure it acually works" part of development.I find the process I follow is to organize the files in the distribution structure from the beginning. When adding new functions, I'll generally use source() a few times to get the syntax right, and perhaps run simple tests. (But remember, if you use a NAMESPACE, the functions may not behave the same when they're sourced into the global environment.) In the early stages, I'll do a lot of installs of the packages. If I was porting a big package and wanted to find syntax errors, to work around the not-very-helpful error message you saw I'd do something like for (f in list.files("pkg/R", full=TRUE)) source(f) which will report the error more informatively. Duncan Murdoch
Thanks to all for the replys. I've learned several things First -- I didn't miss anything obvious in the documentation. This is good in one sense at least: I wasn't being blind. I had one misunderstanding pointed out, which is that I need not worry about my test suite being run with each INSTALL; that happens with CHECK. Second, I heard several good ideas for attacking the problem. Terry T.
Mathieu Ribatet <mathieu.ribatet <at> epfl.ch> writes:> > Dear Terry, > > One way to locate which file is wrong - surely not the most brillant > way! You could do an R script that sources each of your .R files within > a "for (file in file.names)" loop. > When R will stop, it will indicate which file has a wrong syntax and > more info.Yes, I also use this approach, however making use of the sourceDir() utility function that you get by example(source). I had contemplated on and off, it was worth of making sourceDir() an "official" function and had found that I could not reasonably see the case for it.. Martin Maechler, ETH Zurich