similar to: How to avoid overfitting in gam(mgcv)

Dear all, I'm trying to fit a pure additive model of the following formula : fit <- gam(y~x1+te(x2, x3, bs="cr")) ,with the smoothing parameter estimation method "magic"(default). Regarding this, I have two questions : Question 1 : In some cases the value of "mgcv.conv$fully.converged" becomes "FALSE", which tells me that the method stopped with a

The subject is a Generalized Additive Model. Experts caution us against overfitting the data, which can cause inaccurate results. I am not a statistician (my background is in Computer Science). Perhaps some kind soul would take a look and vet the model for overfitting the data. The study estimated the ebb and flow of traffic through a voting place. Just one voting place was studied; the

I am analyzing particulate matter data (PM10) on a small data set (147 observations). I fitted a semi-parametric model and am worried about overfitting. How can one check for model fit in GAM? Jean G. Orelien

This isn't a question about R, but I'm hoping someone will be willing to help. I've been looking at calibration plots in multiple regression (plotting observed response Y on the vertical axis versus predicted response [Y hat] on the horizontal axis). According to Frank Harrell's "Regression Modeling Strategies" book (pp. 61-63), when making such a plot on new data

Hi, When I tested xm vcpu-pin command, I encountered the following strange problem. I encountered it on x86, but I didn''t encounter it on ia64. On x86: # xm vcpu-list Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 0 -b- 31.3 any cpu Domain-0 0 1 1 r--

Hi, I tested unlikely operations about the xm reboot/shutdown command. As a result, I found the following problems. Problem 1: Can reboot/shutdown Domain-0 by mistake. # xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 1024 2 r----- 31.0 # xm reboot 0 Error: Can''t specify Domain-0 # xm reboot 00 #

Hi, When I tested xm create command, I saw the following error message. I expected an error message "Error: (12, ''Cannot allocate memory'')" because I intentionally caused a memory shortage on the test. But the error message was different from my expectation. # xm create /xen/HVMdomain.1 Using config file "/xen/HVMdomain.1". Error: an integer is required

Hi,everyone, I am studying the generalized additive model and employ the package 'mgcv' developed by professor Wood. However,I can not understand the example listed in check.in function. For example, library(mgcv) set.seed(1) dat <- gamSim(1,n=400,scale=2) ## fit a GAM with quite low `k' b<-gam(y~s(x0,k=6)+s(x1,k=6)+s(x2,k=6)+s(x3,k=6),data=dat) plot(b,pages=1,residuals=TRUE)

Hi, This patch adds more user registers to show them to xenctx for ia64. Tested domU/domVTi on ia64. Sample is the below. # ./xenctx 1 0 iip: e000000000000810 ipsr: 00001012087a6010 b0: a000000100068a70 b6: a00000010014ff60 b7: e000000000000800 cr_ifs: 800000000000050a ar_unat:

Hello, I'm trying to understand how to use the pbo package by looking at a vignette. I'm curious about a part of the vignette that creates simulated returns data. The package author transforms his simulated returns in a way that I'm unfamiliar with, and that I haven't been able to find an explanation for after searching around. I'm curious if I need to replicate the

Hi, I have a question about the parameter C (cost) in svm function in e1071. I thought larger C is prone to overfitting than smaller C, and hence leads to more support vectors. However, using the Wisconsin breast cancer example on the link: http://planatscher.net/svmtut/svmtut.html I found that the largest cost have fewest support vectors, which is contrary to what I think. please see the scripts

Hi, I am trying to understand the alternative methods that are available for selecting variables in a regression without simply imposing my own bias (having "good judgement"). The methods implimented in leaps and step and stepAIC seem to fall into the general class of stepwise procedures. But these are commonly condemmed for inducing overfitting. In Hastie, Tibshirani and Friedman

I?ve been using Frank Harrell?s rms package to do bootstrap model validation. Is it the case that the optimum penalization may still give a model which is substantially overfitted? I calculated corrected R^2, optimism in R^2, and corrected slope for various penalties for a simple example: x1 <- rnorm(45) x2 <- rnorm(45) x3 <- rnorm(45) y <- x1 + 2*x2 + rnorm(45,0,3) ols0 <- ols(y

Hi Joe, The centering and re-scaling is done for the purposes of his example, and also to be consistent with his definition of the sharpe function. In particular, note that the sharpe function has the rf (riskfree) parameter with a default value of .03/252 i.e. an ANNUAL 3% rate converted to a DAILY rate, expressed in decimal. That means that the other argument to this function, x, should be DAILY

Wrong list. Post on r-sig-finance instead. Cheers, Bert On Nov 20, 2017 11:25 PM, "Joe O" <joerodonnell at gmail.com> wrote: Hello, I'm trying to understand how to use the pbo package by looking at a vignette. I'm curious about a part of the vignette that creates simulated returns data. The package author transforms his simulated returns in a way that I'm

Hi Eric, Thank you, that helps a lot. If I'm understanding correctly, if I?m wanting to use actual returns from backtests rather than simulated returns, I would need to make sure my risk-adjusted return measure, sharpe ratio in this case, matches up in scale with my returns (i.e. daily returns with daily sharpe, monthly with monthly, etc). And I wouldn?t need to transform returns like the

Correct Sent from my iPhone > On 21 Nov 2017, at 22:42, Joe O <joerodonnell at gmail.com> wrote: > > Hi Eric, > > Thank you, that helps a lot. If I'm understanding correctly, if I?m wanting to use actual returns from backtests rather than simulated returns, I would need to make sure my risk-adjusted return measure, sharpe ratio in this case, matches up in scale with

I did this: nb <- naiveBayes(users, platform) pl <- predict(nb,users) nrow(users) ==> 314781 ncol(users) ==> 109 1. naiveBayes() was quite fast (~20 seconds), while predict() was slow (tens of minutes). why? 2. the predict results were completely off the mark (quite the opposite of the expected overfitting). suffice it to show the tables: pl: android blackberry ipad

I am accustomed to the LIBSVM package, which provides cross-validation on training with the -v option % svm-train -v 5 ... This does 5 fold cross validation while building the model and avoids over-fitting. But I don't see how to accomplish that in the e1071 package. (I learned that svm(... cross=5 ...) only _tests_ using cross-validation -- it doesn't affect the training.) Can

[re-sending - previous email went out by accident before complete] Hi Joe, The centering and re-scaling is done for the purposes of his example, and also to be consistent with his definition of the sharpe function. In particular, note that the sharpe function has the rf (riskfree) parameter with a default value of .03/252 i.e. an ANNUAL 3% rate converted to a DAILY rate, expressed in decimal. That