similar to: Possible overfitting of a GAM

Dear listers, I'm using gam(from mgcv) for semi-parametric regression on small and noisy datasets(10 to 200 observations), and facing a problem of overfitting. According to the book(Simon N. Wood / Generalized Additive Models: An Introduction with R), it is suggested to avoid overfitting by inflating the effective degrees of freedom in GCV evaluation with increased "gamma"

I am analyzing particulate matter data (PM10) on a small data set (147 observations). I fitted a semi-parametric model and am worried about overfitting. How can one check for model fit in GAM? Jean G. Orelien

Hello, I have got a cluster sample using an election dataset where I already had the final results of a county-specific election. I am trying to figure out what would be the best sampling design for my data. The structure of the dataset is: 1) polling station (in general schools where people vote, for a county, for example, there are 15 polling stations) 2) inside each polling station, there

This isn't a question about R, but I'm hoping someone will be willing to help. I've been looking at calibration plots in multiple regression (plotting observed response Y on the vertical axis versus predicted response [Y hat] on the horizontal axis). According to Frank Harrell's "Regression Modeling Strategies" book (pp. 61-63), when making such a plot on new data

Hello! Can anyone give a tip how to plot parametric effects in an Generalized Additive Model, from mgcv package? Thanks, PM

I?ve been using Frank Harrell?s rms package to do bootstrap model validation. Is it the case that the optimum penalization may still give a model which is substantially overfitted? I calculated corrected R^2, optimism in R^2, and corrected slope for various penalties for a simple example: x1 <- rnorm(45) x2 <- rnorm(45) x3 <- rnorm(45) y <- x1 + 2*x2 + rnorm(45,0,3) ols0 <- ols(y

Hello, I have a question regarding model selection and dropping of terms for GAMs fitted with package mgcv. I am following the approach suggested in Wood (2001), Wood and Augustin (2002). I fitted a saturated model, and I find from the plots that for two of the covariates, 1. The confidence interval includes 0 almost everywhere 2. The degrees of freedom are NOT close to 1 3. The partial

Hi Steven- Nice, but will this not break Windows? From an initial glance over your patch, it seems to assume the existence of mmap() in some form or other. Alistair On 8 Aug 2010, at 03:05, Steven Noonan wrote: > Hi folks, > > I've been doing work on memory reduction in Unladen Swallow, and > during testing, LiveRanges seemed to be consuming one of the largest > chunks of

Hi folks, I've been doing work on memory reduction in Unladen Swallow, and during testing, LiveRanges seemed to be consuming one of the largest chunks of memory. I wrote a replacement allocator for use by BumpPtrAllocator which uses mmap()/munmap() in place of malloc()/free(). It has worked flawlessly in testing, and reduces memory usage quite nicely in Unladen Swallow. The code is available

I write about R every weekday at?http://blog.revolution-computing.com . In case you missed them, here are some articles from the month of April of particular interest to r-help subscribers. Thanks to everyone who has been following the blog and sending me messages and/or leaving comments -- it always brightens my day to hear from readers! http://tinyurl.com/cy7x9a?(from April 1) announced the new

We are seeking nominations for candidates for election to the X.Org Foundation Board of Directors. All X.Org Foundation members are eligible for election to the board. Nominations for the 2024 election are now open and will remain open until 23:59 UTC on 26 February 2024. The Board consists of directors elected from the membership. Each year, an election is held to bring the total number of

Thanks, Vedant. Yes, we have done a lot of testing of Clang/LLVM (and GCC) in the past several years (more details at https://people.inf.ethz.ch/suz/emi/index.html): [GCC/LLVM bugs: *1,602* (total) / *1,007* (fixed)] [Reports: GCC (link1 <http://tinyurl.com/ojzdt78>, link2 <http://tinyurl.com/oxlkmjc>, link3 <http://tinyurl.com/jd3jnl3>, link4

You may also be interested in the following resources on compilers correctness (articles, software, and talks -- from the general topics to the ones specifically focused on testing, validation, and verification): https://github.com/MattPD/cpplinks/blob/master/compilers.correctness.md Best regards, Matt P. Dziubinski On 12/7/2018 20:19, Philip Reames via llvm-dev wrote: > There's also

I''m a graduating student of IT right now and in a couple of months, we''ll be starting on a new project in the University. My group and I decided to create a voting system which requires Fingerprint scanner to identify the voters and make the voting process more secured. I still don''t know how biometrics fingerprint scanner works exactly and I''ve just started

Hi, I need help for cleaning this: "[2440810] / www.tinyurl.com/hgaco4fha3" My desired output is: "[2440810] / tinyurl". My attemps: stringa <- "[2440810] / www.tinyurl.com/hgaco4fha3" b <- sub('^www.', '', stringa) #wanted to get rid of "www." part. Until first dot. b <- sub('[.].*', '', b) #clean from

I just finished going through the time sync problem, and here's the info I used: To configure ESX 2.5.x Server: http://tinyurl.com/ycpcfv <http://tinyurl.com/ycpcfv> To configure Windows Guest OS: http://tinyurl.com/yctxrr <http://tinyurl.com/yctxrr> To configure CentOS w/o X installed, you still need to install the vmware tools. Edit the vmx file for your guest,

Hi, I'm running gam (mgcv version 1.3-29) and glm (logit) (stats R 2.61) on the same models/data, and I got error messages for the gam() model and warnings for the glm() model. R-help suggested that the glm() warning messages are due to the model perfectly predicting binary output. Perhaps the model overfits the data? I inspected my data and it was not immediately obvious to me (though I

Hello, I'm trying to understand how to use the pbo package by looking at a vignette. I'm curious about a part of the vignette that creates simulated returns data. The package author transforms his simulated returns in a way that I'm unfamiliar with, and that I haven't been able to find an explanation for after searching around. I'm curious if I need to replicate the

Hello, I am having a issue with my Xen setup, that other people have reported as well. No one seems to have a solution to this problem or an idea what might cause it. My only hope is that I might get some more ideas here in the devel list. The basic problem is that if there is sustained high data rate traffic through a virtual interface, it seems that the queues fill up and the link

Hi all, I put up an exhaustive model to use R's "step" function: ------------------------ mygam=gam(col1 ~ 1 + col2 + col3 + col4 + col2 ^ 2 + col3 ^ 2 + col4 ^ 2 + col2 ^ 3 + col3 ^ 3 + col4 ^ 3 + s(col2, 1) + s(col3, 1) + s(col4, 1) + s(col2, 2) + s(col3, 2) + s(col4, 2) + s(col2, 3) + s(col3, 3) + s(col4, 3) + s(col2, 4) + s(col3, 4) + s(col4, 4) + s(col2, 5) + s(col3,