R help - Mar 2018 - Using R and the Tidyverse for an economic model

I've been translating an economic model from Python into R, and I thought
members of the list would like to see a presentation I've written about it.
I've blogged this at
http://www.j-paine.org/blog/2018/03/r-taxben-a-microsimulation-economic-model-in-r.html
, and the presentation itself is a slideshow at
http://www.j-paine.org/rtaxben/R/reveal/rtaxben.html . The slideshow is
written as one side of a conversation which reveals R and the Tidyverse a
feature at a time to a colleague not familiar with R. Those who _are_
familar with R might prefer the version at
http://www.j-paine.org/rtaxben/R/reveal/rtaxben_anim.html . Exactly the
same material, but, as explained in my introduction, quicker to read. Read
the blog post first.

Our model, R-Taxben, is a microeconomic model, which simulates at the level
of individual people rather than bulk variables such as unemployment and
inflation. It works, roughly speaking, by reading survey data about actual
households, then applying taxes and benefits to calculate net income and
expenditure from gross. It has four main parts: (1) read and process
parameters which describe the taxes and benefits; (2) read the household
data from CSV files and transform into data frames usable by the model; (3)
apply the taxes and benefits, calculating such things as council tax, VAT,
child benefit, and pensions; (4) display the results.

My slides are mainly about (2) and (4), but do touch on the others. I
suggest, for example, that legible R code for (3) could be used as a
"reference standard" against which to describe the notoriously complex
UK
benefits system. Organisations such as the Child Poverty Action Group have
written handbooks for benefits advisers which try to specify the system
precisely. We'd like to use R for an electronic version of these.

I've said quite a bit about R for probing and plotting data. Not only for
economists, but for students learning about economics, fiscal policy, and
statistics. And after a brief intro to base R, I've concentrated on the
Tidyverse, because of what I see as its advantages. There are lots of small
demos of the Tidyverse scattered around the web, but fewer of big projects
which use lots of different features from it. So my examples here might be
useful.

Reliability and accuracy are vital, which is why I have more slides about
testing than about anything else, with examples of "testthat".

Near the end, I show a web interface, built using Vis.js , which displays
dataflow in the model. The aim is to make it completely scrutable, so that
none of its economic assumptions are a mystery.

We're looking for funding to go beyond this prototype. There are places
where we'll probably need help with such things as efficiency (see the
section on representation-independent selectors), efficiency again
(multiple JOINs), and the best way to overcome lack of static typing. It
would be great to have R experts, even R implementors, who were willing to
advise on this, and even to collaborate on our grant applications.

	[[alternative HTML version deleted]]

Looks like you have made an impressive start and some attractive 
introductions. I have no significant interest in your topic (sorry), but 
it seems that you are re-inventing the wheel a bit in regards to much of 
your documentation and modularization... R packages can help you solve 
these problems in a cross-platform way. You might try starting with [1] 
and referring to [2] as needed.

Regarding your representation-independent selectors... this looks to me 
like yet another representation (I think the term is "network
database"),
and subject to specific advantages and limitations that this 
representation imposes. For most work I do, tidy data frames have an 
excellent balance of speed and adaptability. For other types of analyses, 
multi-dimensional arrays would be better. Nested lists are extremely 
flexible, but not particularly fast (some would say quite slow, but that 
depends on your use case). Sometimes a relational database or the 
data.table package [3] can be used for increased performance, but your 
functional interface would not be compatible with _merging_ the 
information efficiently, while dplyr can theoretically support any data 
store that presents a tabular data interface with data merge capability.

R seems to work best when used in the functional paradigm operating on 
general-purpose objects... functions that transform, analyze, and present 
data. Having more general classes of objects means more re-use and ad-hoc 
analysis can occur. If I make an object of class "myspecial", only 
functions I write will be useful. Making it a subclass of a more general 
class is one way to make it more widely useful, but avoiding making it a 
subclass of the general class at all can be the most flexible design 
principle... which is what "tidy data" aspires to do with data frames.

That is, I think you should not be avoiding $ (or more generally the
"[["
operator)... you should be embracing it and enabling users to use it as 
well. Just don't go all multi-level with it... prefer multi-column indexes 
in data frames in most cases (e.g [4]).

[1] http://r-pkgs.had.co.nz/
[2] https://cran.r-project.org/doc/manuals/R-exts.html
[3]
https://cran.r-project.org/web/packages/data.table/vignettes/datatable-intro.html
[4] https://www.jstatsoft.org/article/view/v040i01/v40i01.pdf

On Mon, 26 Mar 2018, Jocelyn Ireson-Paine via R-help wrote:
> I've been translating an economic model from Python into R, and I
thought
> members of the list would like to see a presentation I've written about
it.
> I've blogged this at
>
http://www.j-paine.org/blog/2018/03/r-taxben-a-microsimulation-economic-model-in-r.html
> , and the presentation itself is a slideshow at
> http://www.j-paine.org/rtaxben/R/reveal/rtaxben.html . The slideshow is
> written as one side of a conversation which reveals R and the Tidyverse a
> feature at a time to a colleague not familiar with R. Those who _are_
> familar with R might prefer the version at
> http://www.j-paine.org/rtaxben/R/reveal/rtaxben_anim.html . Exactly the
> same material, but, as explained in my introduction, quicker to read. Read
> the blog post first.
>
> Our model, R-Taxben, is a microeconomic model, which simulates at the level
> of individual people rather than bulk variables such as unemployment and
> inflation. It works, roughly speaking, by reading survey data about actual
> households, then applying taxes and benefits to calculate net income and
> expenditure from gross. It has four main parts: (1) read and process
> parameters which describe the taxes and benefits; (2) read the household
> data from CSV files and transform into data frames usable by the model; (3)
> apply the taxes and benefits, calculating such things as council tax, VAT,
> child benefit, and pensions; (4) display the results.
>
> My slides are mainly about (2) and (4), but do touch on the others. I
> suggest, for example, that legible R code for (3) could be used as a
> "reference standard" against which to describe the notoriously
complex UK
> benefits system. Organisations such as the Child Poverty Action Group have
> written handbooks for benefits advisers which try to specify the system
> precisely. We'd like to use R for an electronic version of these.
>
> I've said quite a bit about R for probing and plotting data. Not only
for
> economists, but for students learning about economics, fiscal policy, and
> statistics. And after a brief intro to base R, I've concentrated on the
> Tidyverse, because of what I see as its advantages. There are lots of small
> demos of the Tidyverse scattered around the web, but fewer of big projects
> which use lots of different features from it. So my examples here might be
> useful.
>
> Reliability and accuracy are vital, which is why I have more slides about
> testing than about anything else, with examples of "testthat".
>
> Near the end, I show a web interface, built using Vis.js , which displays
> dataflow in the model. The aim is to make it completely scrutable, so that
> none of its economic assumptions are a mystery.
>
> We're looking for funding to go beyond this prototype. There are places
> where we'll probably need help with such things as efficiency (see the
> section on representation-independent selectors), efficiency again
> (multiple JOINs), and the best way to overcome lack of static typing. It
> would be great to have R experts, even R implementors, who were willing to
> advise on this, and even to collaborate on our grant applications.
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
---------------------------------------------------------------------------
Jeff Newmiller                        The     .....       .....  Go Live...
DCN:<jdnewmil at dcn.davis.ca.us>        Basics: ##.#.       ##.#.  Live
Go...
                                       Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/Batteries            O.O#.       #.O#.  with
/Software/Embedded Controllers)               .OO#.       .OO#.  rocks...1k