similar to: Mapping parse tree elements to tokens

Probably need a generic tree based on "ParseNode" objects that associate the line information with the symbol (for leaf nodes). As Duncan notes, it should be possible to gather that from the table. But it would be nice if there was an "expr" column in the parse data column in addition to "text". It would contain the parsed object. Otherwise, to use the table, one is

I have two use cases in mind: 1) Code indexing/searching, where the table gets me almost all of the way there, except I ask for all of the text (including the calls) and then parse that, because it's nice to get back an actual code object when you are searching code (in addition to where the code lives). The extra parsing step is just a minor inconvenience. 2) Code analysis, which I'm

As Michael guessed my main use cases was code analysis. A concrete example where this would help is with my test code coverage tool covr. There is currently a bug when tracking coverage for if / else statements when the clauses do not contain brackets (https://github.com/jimhester/covr/issues/39). Because only one source reference is generated in this case (because it is parsed as a single

Hi, With R-3.1.0 I get: > getParseData(parse(text = "{1}", keep.source = TRUE)) line1 col1 line2 col2 id parent token terminal text 7 1 1 1 3 7 9 expr FALSE 1 1 1 1 1 1 7 '{' TRUE { 2 1 2 1 2 2 3 NUM_CONST TRUE 1 3 1 2 1 2 3 5 expr FALSE 4 1 3 1

On 29/07/2015 2:30 PM, Michael Lawrence wrote: > Probably need a generic tree based on "ParseNode" objects that > associate the line information with the symbol (for leaf nodes). As > Duncan notes, it should be possible to gather that from the table. > > But it would be nice if there was an "expr" column in the parse data > column in addition to

Hi all, Is the next behaviour suitable? identical(F,FALSE) ## [1] TRUE utils::getParseData(parse(text = "c(F,FALSE)", keep.so=rce = TRUE)) ## line1 col1 line2 col2 id parent token terminal text ## 14 1 1 1 10 14 0 expr FALSE ## 1 1 1 1 1 1 3 SYMBOL_FUNCTION_CALL TRUE c ## 3 1 1 1 1 3

Both codetools and compiler get by without this. codetools uses source refs to generate messages; I don't recall if compiler does but it could easily do so. I would be wary about committing to this sort of implementation specific stuff -- we might want to go to completely different parser technology at tome point, which would be harder if we committed to these sort of details. Best, luke On

On 29/07/2015 12:13 PM, Jim Hester wrote: > I would like to map the parsed tokens obtained from utils::getParseData() > to the parse tree and elements obtained by base::parse(). > > It looks like back when this code was in the parser package the parse() > function annotated the elements in the tree with their id, which would > allow you to perform this mapping. However when the

Hi, The imaginary unit is gone in the 'text' column in the returned data frame from getParseData(), e.g. in the example below, perhaps the text should be 1i instead of 1: > p=parse(text='1i') > getParseData(p) line1 col1 line2 col2 id parent token terminal text 1 1 1 1 2 1 2 NUM_CONST TRUE 1 2 1 1 1 2 2 0 expr

Hi, The text column for '->' becomes '<-' in the data frame returned by getParseData(): > getParseData(parse(text='1->x')) line1 col1 line2 col2 id parent token terminal text 7 1 1 1 4 7 0 expr FALSE 1 1 1 1 1 1 2 NUM_CONST TRUE 1 2 1 1 1 1 2 7 expr FALSE 3

This seems like a bug to me: code <- 'x <- r"(hello, "world")"' getParseData(parse(text = code)) #> line1 col1 line2 col2 id parent token terminal text #> 7 1 1 1 24 7 0 expr FALSE #> 1 1 1 1 1 1 3 SYMBOL TRUE x #> 3 1 1 1 1 3 7 expr

I don't know, maybe it would make sense to keep the whole expression, that's the text of the tag after all. Also, if we don't keep the whole expression, then it is not a valid string literal any more, because it does not have quoting. I can try to look into a patch. This is for 4.1 I believe, so in some sense it is not urgent? Gabor On Wed, Apr 22, 2020 at 3:31 PM <luke-tierney

I have just committed (in r59883) some changes to the R parser based on Romain Francois' parser package. Packages that made use of parser will hopefully find that the information in base R gives them what they need to work with, but the data is not identical to what parser recorded (since it was not consistent with some things already in R). One reason for the change was that the parser

Hi, I have run into a problem with parseData from the utils package.? When an assignment is done with = instead of <-, the information provided by parseData does not include an entry for the assignment. For this input, stored in file "BadPosition.R": y <- 5 foo = 7 And running this code: parsed <- parse("BadPosition.R", keep.source=TRUE) parsedData <-

The fix is now in R-devel, 75386. I have not ported to R-patched, because the fix breaks two packages which are working around this bug (and to my knowledge without having reported it before). So thanks again for the report! Best Tomas On 08/16/2018 10:06 AM, Tomas Kalibera wrote: > Dear Barbara, > > thank you for the report. This is something to be fixed in R - I am > now

Hello, I am migrating my package lambda.r to R3.0.0 and am experiencing some issues with the getParserData function (which replaces the parser package). Basically the function works in the R shell but fails when either called from RUnit or from R CMD check. I've narrowed it down to the function getSrcfile, which is returning different values depending on the code path. From the command line

Maybe someone has already suggested this, but if your functions accepted strings you could use sub or gsub to replace the -> with a symbol that parsed at the same precedence as <-, say <<-. Then parse it and deal with it. When it is time to display the parsed and perhaps manipulated formulae to the user, deparse it and do the reverse replacement. > encode <-

Hi, I've noticed that partial parse data from parse() is irretrivable when it errors due to encountering '``' - two backticks in sequence. The print output also looks more limited than a regular parser error which leads me to suspect this is a bug. If this is the case could someone with the authority please add it to R's bug tracker? My reprex follows. Thanks, Miles ``` r

Hi, I'm trying to make sense of the output of getParseData(). The "parent" column is supposed to refer to the "id" of the parent of the given item, but there are numbers in the parent column that do not exist in the id column. Example: > p <- parse(text="f<-function(){if(TRUE)1 else 2}") > df <- getParseData(p) > df line1 col1 line2 col2 id

Looks like a bug. Will have a look when I get a chance. Simpler version: getParseData(parse(text = 'r"-|hello|-"')) > getParseData(parse(text = 'r"(hello)"')) line1 col1 line2 col2 id parent token terminal text 1 1 1 1 10 1 3 STR_CONST TRUE "hello) 3 1 1 1 10 3 0 expr FALSE The opening