search for: overfitted

Dear listers, I'm using gam(from mgcv) for semi-parametric regression on small and noisy datasets(10 to 200 observations), and facing a problem of overfitting. According to the book(Simon N. Wood / Generalized Additive Models: An Introduction with R), it is suggested to avoid overfitting by inflating the effective degrees of freedom in GCV evaluation with increased "gamma"

I am analyzing particulate matter data (PM10) on a small data set (147 observations). I fitted a semi-parametric model and am worried about overfitting. How can one check for model fit in GAM? Jean G. Orelien

...r, when I tried to generate the standard errors, things went awry. (Please see http://tinyurl.com/38ej2t ) There are three curves, seemingly the fitted curve and the curves for plus and minus two standard errors. The shapes seem okay, but there are large errors in the y values. Question: Have I overfitted the data? Feedback? Tom Thomas L. Jones, PhD, Computer Science

Hello, I'm trying to understand how to use the pbo package by looking at a vignette. I'm curious about a part of the vignette that creates simulated returns data. The package author transforms his simulated returns in a way that I'm unfamiliar with, and that I haven't been able to find an explanation for after searching around. I'm curious if I need to replicate the

I am accustomed to the LIBSVM package, which provides cross-validation on training with the -v option % svm-train -v 5 ... This does 5 fold cross validation while building the model and avoids over-fitting. But I don't see how to accomplish that in the e1071 package. (I learned that svm(... cross=5 ...) only _tests_ using cross-validation -- it doesn't affect the training.) Can

This isn't a question about R, but I'm hoping someone will be willing to help. I've been looking at calibration plots in multiple regression (plotting observed response Y on the vertical axis versus predicted response [Y hat] on the horizontal axis). According to Frank Harrell's "Regression Modeling Strategies" book (pp. 61-63), when making such a plot on new data

Hi all, I am using the "svm" command in the e1071 package. Does it have an automatic way of setting the "cost" parameter? I changed a few values for the "cost" parameter but I hope there is a systematic way of obtaining the best "cost" value. I noticed that there is a "cross" (Cross validation) parameter in the "svm" function. But I

I?ve been using Frank Harrell?s rms package to do bootstrap model validation. Is it the case that the optimum penalization may still give a model which is substantially overfitted? I calculated corrected R^2, optimism in R^2, and corrected slope for various penalties for a simple example: x1 <- rnorm(45) x2 <- rnorm(45) x3 <- rnorm(45) y <- x1 + 2*x2 + rnorm(45,0,3) ols0 <- ols(y ~ x1 + x2 + x3, x=TRUE, y=TRUE) corrected.Rsq <- rep(0,60) optimism.Rsq &l...

Hi Joe, The centering and re-scaling is done for the purposes of his example, and also to be consistent with his definition of the sharpe function. In particular, note that the sharpe function has the rf (riskfree) parameter with a default value of .03/252 i.e. an ANNUAL 3% rate converted to a DAILY rate, expressed in decimal. That means that the other argument to this function, x, should be DAILY

On Fri, May 4, 2018 at 10:16 AM Sanjay Patel <spatel at rotateright.com> wrote: > I understand the overfit argument (but in most cases it just shows that a > unit test isn't minimized)... > Even minimized tests sometimes need a few other things to setup the circumstance (many DWARF tests, for example - produce the full DWARF output, but maybe you only care about one part of it

Hi, I have a question about the parameter C (cost) in svm function in e1071. I thought larger C is prone to overfitting than smaller C, and hence leads to more support vectors. However, using the Wisconsin breast cancer example on the link: http://planatscher.net/svmtut/svmtut.html I found that the largest cost have fewest support vectors, which is contrary to what I think. please see the scripts

Wrong list. Post on r-sig-finance instead. Cheers, Bert On Nov 20, 2017 11:25 PM, "Joe O" <joerodonnell at gmail.com> wrote: Hello, I'm trying to understand how to use the pbo package by looking at a vignette. I'm curious about a part of the vignette that creates simulated returns data. The package author transforms his simulated returns in a way that I'm

Hi, I am trying to understand the alternative methods that are available for selecting variables in a regression without simply imposing my own bias (having "good judgement"). The methods implimented in leaps and step and stepAIC seem to fall into the general class of stepwise procedures. But these are commonly condemmed for inducing overfitting. In Hastie, Tibshirani and Friedman

Hi there, Does anyone know how I solve for x from a given y in a polynomial function? Here's some example code: ##example file a<-1:10 b<-c(1,2,2.5,3,3.5,4,6,7,7.5,8) po.lm<-lm(a~b+I(b^2)+I(b^3)+I(b^4)); summary(po.lm) (please ignore that the model is severely overfit- that's not the point). Let's say I want to solve for the value b where a = 5.5. Any thoughts? I did

I understand the overfit argument (but in most cases it just shows that a unit test isn't minimized)...but I don't see how the complete auto-generated assertions could be worse at detecting a miscompile than incomplete manually-generated assertions? The whole point of auto-generating complete checks is to catch miscompiles/regressions sooner. Ie, before they get committed and result in

Yep - all about balance. The main risk are tests that overfit (golden files being the worst case - checking that the entire output matches /exactly/ - this is what FileCheck is intended to help avoid) and maintainability. In the case of the autogenerated FileCheck lines I've seen so far - they seem like they still walk a fairly good line of checking exactly what's intended. Though I

Hi Eric, Thank you, that helps a lot. If I'm understanding correctly, if I?m wanting to use actual returns from backtests rather than simulated returns, I would need to make sure my risk-adjusted return measure, sharpe ratio in this case, matches up in scale with my returns (i.e. daily returns with daily sharpe, monthly with monthly, etc). And I wouldn?t need to transform returns like the

Correct Sent from my iPhone > On 21 Nov 2017, at 22:42, Joe O <joerodonnell at gmail.com> wrote: > > Hi Eric, > > Thank you, that helps a lot. If I'm understanding correctly, if I?m wanting to use actual returns from backtests rather than simulated returns, I would need to make sure my risk-adjusted return measure, sharpe ratio in this case, matches up in scale with

On Fri, May 4, 2018 at 11:30 AM, David Blaikie <dblaikie at gmail.com> wrote: > > > On Fri, May 4, 2018 at 10:16 AM Sanjay Patel <spatel at rotateright.com> > wrote: > >> I understand the overfit argument (but in most cases it just shows that a >> unit test isn't minimized)... >> > > Even minimized tests sometimes need a few other things to

[re-sending - previous email went out by accident before complete] Hi Joe, The centering and re-scaling is done for the purposes of his example, and also to be consistent with his definition of the sharpe function. In particular, note that the sharpe function has the rf (riskfree) parameter with a default value of .03/252 i.e. an ANNUAL 3% rate converted to a DAILY rate, expressed in decimal. That