Given that we occasionally run into problems with comparing Excel results to R results, and other spreadsheet-induced errors, I thought this might be of interest. http://www.nextnewdeal.net/rortybomb/researchers-finally-replicated-reinhart-rogoff-and-there-are-serious-problems The punchline: "If this error turns out to be an actual mistake Reinhart-Rogoff made, well, all I can hope is that future historians note that one of the core empirical points providing the intellectual foundation for the global move to austerity in the early 2010s was based on someone accidentally not updating a row formula in Excel." Ouch. (Note: I know nothing about the site, the author of the article, or the study in question. I was pointed to it by someone else. But if true: highly problematic.) Sarah -- Sarah Goslee http://www.functionaldiversity.org
When in doubt, assume the spreadsheet is wrong. I suggested this to someone have a problem with R vs Excel results a while ago. When I checked back with him -- there was a spreadsheet error. I think a t-shirt with the motto "Friends don't let friends use spreadsheets"[1] sounds like a good idea. Unfortunately I am not artistic enough to do a design. 1. Slight paraphrase of J. D Cryer's statement http://homepage.cs.uiowa.edu/~jcryer/JSMTalk2001.pdf John Kane Kingston ON Canada> -----Original Message----- > From: sarah.goslee at gmail.com > Sent: Tue, 16 Apr 2013 13:25:57 -0400 > To: r-help at r-project.org > Subject: [R] the joy of spreadsheets (off-topic) > > Given that we occasionally run into problems with comparing Excel > results to R results, and other spreadsheet-induced errors, I thought > this might be of interest. > > http://www.nextnewdeal.net/rortybomb/researchers-finally-replicated-reinhart-rogoff-and-there-are-serious-problems > > The punchline: > > "If this error turns out to be an actual mistake Reinhart-Rogoff made, > well, all I can hope is that future historians note that one of the > core empirical points providing the intellectual foundation for the > global move to austerity in the early 2010s was based on someone > accidentally not updating a row formula in Excel." > > Ouch. > > (Note: I know nothing about the site, the author of the article, or > the study in question. I was pointed to it by someone else. But if > true: highly problematic.) > > Sarah > > -- > Sarah Goslee > http://www.functionaldiversity.org > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.____________________________________________________________ FREE ONLINE PHOTOSHARING - Share your photos online with your friends and family! Visit http://www.inbox.com/photosharing to find out more!
What a terrific article. Thanks for sharing! The more we critically
examine how research is actually done the more frightened we become.
Frank
--
Frank E Harrell Jr Professor and Chairman School of Medicine
Department of Biostatistics Vanderbilt University
I tend to live in fear that some spreadsheet calculating a drug dose for me will use my telephone number rather than my weight. John Kane Kingston ON Canada> -----Original Message----- > From: f.harrell at vanderbilt.edu > Sent: Tue, 16 Apr 2013 13:20:46 -0500 > To: r-help at stat.math.ethz.ch > Subject: Re: [R] the joy of spreadsheets (off-topic) > > What a terrific article. Thanks for sharing! The more we critically > examine how research is actually done the more frightened we become. > > Frank > > -- > Frank E Harrell Jr Professor and Chairman School of Medicine > Department of Biostatistics Vanderbilt University > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.____________________________________________________________ FREE 3D MARINE AQUARIUM SCREENSAVER - Watch dolphins, sharks & orcas on your desktop!
On 04/17/2013 03:25 AM, Sarah Goslee wrote:> ... > Ouch. > > (Note: I know nothing about the site, the author of the article, or > the study in question. I was pointed to it by someone else. But if > true: highly problematic.) > > Sarah >There seem to be three major problems described here, and only one is marginally related to Excel (and similar spreadsheets). Cherry picking data is all too common. Almost anyone who reviews papers for publication will have encountered it, and there are excellent books describing examples that have had great influence on public policy. Similarly, applying obscure and sometimes inappropriate statistical methods that produce the desired results when nothing else will appears with depressing frequency. The final point does relate to Excel and any application that hides what is going on to the casual observer. I will treasure this URL to give to anyone who chastises my moaning when I have to perform some task in Excel. It is not an error in the application (although these certainly exist) but a salutory caution to those who think that if a reasonable looking number appears in a cell, it must be the correct answer. I have found not one, but two such errors in the simple calculation of a "birthday age" from the date of birth and date of death. Jim
On Tue, Apr 16, 2013 at 1:25 PM, Sarah Goslee <sarah.goslee at gmail.com> wrote:> Given that we occasionally run into problems with comparing Excel > results to R results, and other spreadsheet-induced errors, I thought > this might be of interest. > > http://www.nextnewdeal.net/rortybomb/researchers-finally-replicated-reinhart-rogoff-and-there-are-serious-problems > > The punchline: > > "If this error turns out to be an actual mistake Reinhart-Rogoff made, > well, all I can hope is that future historians note that one of the > core empirical points providing the intellectual foundation for the > global move to austerity in the early 2010s was based on someone > accidentally not updating a row formula in Excel." > > Ouch. > > (Note: I know nothing about the site, the author of the article, or > the study in question. I was pointed to it by someone else. But if > true: highly problematic.) >Herndon, Ash and Pollin (HAP), the authors of the critique, found that in the highest debt category the Excel error in Rienhart and Rogoff (RR) was -0.3 percent points compared to a total error (from that plus RR's other 2 mistakes) of -2.3 percentage points. See Figure 1 of HAP. Thus aside from the dubiousness of attributing the coding error in Excel to Excel itself it was not the main source of the discrepancy. Also even if one backs out all three errors that they found, the key conclusion that GDP growth is declining with debt still occurs (but to a lesser extent) as pointed out by RR in an initial responding email reported by Bloomberg News. The key takeaway here is really unrelated to Excel but rather is that until data and analyses are shared or made public so that the analysis can be reproduced one cannot have any real confidence in research results. RR http://www.nber.org/papers/w15639.pdf HAP http://www.peri.umass.edu/fileadmin/pdf/working_papers/working_papers_301-350/WP322.pdf Bloomberg News http://www.bloomberg.com/news/2013-04-16/reinhart-rogoff-paper-cited-by-ryan-faulted-for-serious-errors-.html -- Statistics & Software Consulting GKX Group, GKX Associates Inc. tel: 1-877-GKX-GROUP email: ggrothendieck at gmail.com
In case you haven't noticed, this is making the rounds in the media, including a handful of references to R. See e.g. http://news.slashdot.org/story/13/04/17/0215211/excel-error-contributes-to-problems-with-austerity-study I suppose we can't fortune()'ify anonymous quotes, but I kind of like this exchange: "Bacon Bits": "SPSS and R are very good at statistical analysis. Quantrix, MapleSoft, IBM Algorithmics, and other software is for financial data modeling. None of those is particularly appropriate for sharing data in a useful format with peers. Excel is." "Hatta": "R is extremely appropriate for sharing data in a useful format with peers. It's completely free for one. But more importantly, it saves every single step of your analysis. Send someone an Excel file, and who knows what they've done to the data. Send someone your R project directory and they can see exactly what you did. The problem with sending R files to your peers isn't that the R files aren't useful. It's that your peers aren't." On Apr 16, 2013, at 19:25 , Sarah Goslee wrote:> Given that we occasionally run into problems with comparing Excel > results to R results, and other spreadsheet-induced errors, I thought > this might be of interest. > > http://www.nextnewdeal.net/rortybomb/researchers-finally-replicated-reinhart-rogoff-and-there-are-serious-problems > > The punchline: > > "If this error turns out to be an actual mistake Reinhart-Rogoff made, > well, all I can hope is that future historians note that one of the > core empirical points providing the intellectual foundation for the > global move to austerity in the early 2010s was based on someone > accidentally not updating a row formula in Excel." > > Ouch. > > (Note: I know nothing about the site, the author of the article, or > the study in question. I was pointed to it by someone else. But if > true: highly problematic.) > > Sarah > > -- > Sarah Goslee > http://www.functionaldiversity.org > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.-- Peter Dalgaard, Professor Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
From a quick read, the Excel error prior for incompetence looks high but some of the other issues hint that the prior for the overall findings was remarkably in favor of malice. John Kane Kingston ON Canada> -----Original Message----- > From: s.ellison at lgcgroup.com > Sent: Fri, 26 Apr 2013 17:07:55 +0100 > To: tea3rd at gmail.com, pdalgd at gmail.com > Subject: Re: [R] the joy of spreadsheets (off-topic) > > > >> One might wonder if the "Excel error" was indeed THAT or >> perhaps a way to get the desired results, give the other >> issues in their analysis? > > The prior for the incompetence/malice question is usually best set pretty > heavily in favour of incompetence ... > > S > > > ******************************************************************* > This email and any attachments are confidential. Any =...{{dropped:15}}
Apparently Analagous Threads
- Excel shared spreadsheets behavior
- RGoogleDocs ability to write to spreadsheets broken as of yesterday - CAN PAY FOR FIX
- RGoogleDocs ability to write to spreadsheets broken as of yesterday
- Is there in R a function equivalent to the mround, as found in most spreadsheets?
- Excel shared spreadsheets on Samba share