similar to: glm binomial logit - removing extra computations

Hi, With reference to the mail below, I have large datasets, coming from various different sources, which I can read into filebacked big.matrix using library bigmemory. I want to merge them all into one 'big.matrix' object. (Later, I want to run regression using library 'biglm'). I am unsuccessfully trying to do this from quite some time now. Can you please

I am using the library(bigmemory) to handle large datasets, say 1 GB, and facing following problems. Any hints from anybody can be helpful. _Problem-1: _ I am using "read.big.matrix" function to create a filebacked big matrix of my data and get the following warning: > x = read.big.matrix("/home/utkarsh.s/data.csv",header=T,type="double",shared=T,backingfile

Hi all, I am performing a stepwise regression by running the "step" function on an "lm" object. Now I want to save the intermediate iterations. I know the argument trace=T will print it on the console, but I rather want to assign it to some R object or may be output it in a CSV or text file. Any help will be appreciated. Regards Utkarsh

Hi All, I have vector of length 52, say, x=sample(30,52,replace=T). I want to sort x and split into five *nearly equal groups*. Note that the observations are repeated in x so in case of a tie I want both the observations to fall in same group. This seems a very common task to do, but still I couldn't find an R function to do this. Any help would be highly appreciated. Regards Utkarsh

Hi Sir, Thanks for making package available to us. I am facing few problems if you can give some hints: Problem-1: The model summary and residual deviance matched (in the mail below) but I didn't understand why AIC is still different. > AIC(m1) [1] 532965 > AIC(m1big_longer) [1] 101442.9 Problem-2: chunksize argument is there in bigglm but not in biglm, consequently,

Hi All, I am currently using R version 2.8.1 on linux cent os 4.4 (i386) and want to upgrade to version 2.9.1. It seems to me that version-2.9.1 is it not for my OS. Am I right? Regards Utkarsh [[alternative HTML version deleted]]

Hi All, When modeling with glm and family = binomial (link = logit) and response values of 0 and 1, I get the predicted probabilities of assigning to my class one, then I would like to compare it with my vector y which does have the original labels. How should I change the probabilities into values of zero and 1 and then compare it with my vector y to find out about the accuracy of my

Hi All, I am currently doing the following to compute summary statistics of aggregated data: a = aggregate(warpbreaks$breaks, warpbreaks[,-1], mean) b = aggregate(warpbreaks$breaks, warpbreaks[,-1], sum) c = aggregate(warpbreaks$breaks, warpbreaks[,-1], length) ans = cbind(a, b[,3], c[,3]) This seems unnecessarily complex to me so I tried > aggregate(warpbreaks$breaks, warpbreaks[,-1],

Hi I am trying to make glm() work to analyze a toy logit system. I have a dataframe with x and y independent variables. I have L=1+x-y (ie coefficients 1,1,-1) then if I have a logit relation with L=log(p/(1-p)), p=1/(1+exp(L)). If I interpret "p" as the probability of success in a Bernouilli trial, and I can observe the result (0 for "no", 1 for

Hi All, I am using "glm" function to build logistic regression. I noticed that glm function glm function is computing many other statistics which are not required for our analysis. As our dataset is very big and we have to run logistic regression on several samples the run time drastically increases if all those statistics are computed. Is these any way to skip computation in glm

I never understood that why is the value returned by as.date function in the library(survival) never matches with the description given in the help file: Following is the extract from ?as.date Description: Converts any of the following character forms to a Julian date: 8/31/56, 8-31-1956, 31 8 56, 083156, 31Aug56, or August 31 1956. Usage: as.date(x, order = "mdy", ...)

Hi R, I am getting this error while trying to use 'lrm' function with nine independent variables: > res = lrm(y1994~WC08301+WC08376+WC08316+WC08311+WC01001+WC08221+WC08106+WC0810 1+WC08231,data=y) singular information matrix in lrm.fit (rank= 8 ). Offending variable(s): WC08101 WC08221 Error in j:(j + params[i] - 1) : NA/NaN argument Now, if I take choose only four

I had a look at this a while back and it didn't seem to be easy. The path of least resistance would seem to be to use SAS itself to create a data set with fewer columns, but of course that requires you to get access to SAS. Otherwise, I think you'd have to modify sas7bdat::read.sas7bdat to drop unselected columns. That function is pure R code, so it might not be quite as hard as it

hi, the sas universal viewer might be a free, non-R way to convert a sas7bdat file to non-proprietary formats, not sure if it's windows-only. those other formats should be easier to import only a subset of columns into R.. https://support.sas.com/downloads/browse.htm?fil=&cat=74 On Thu, Aug 10, 2017 at 7:42 AM, peter dalgaard <pdalgd at gmail.com> wrote: > I had a look at this

I define the following function: (Please don't wonder about the use of this function, this is just a simplified version of my actual function. And please don't spend your time in finding an alternate way of doing the same as the following does not exactly represent my function. I am only interested in a good explanation) > f1 =

Hi all, Could anybody tell me what happened to my logistic regression in R? mylog=glm(mytraindata$V1 ~ ., data=mytraindata, family=binomial("logit")) It generated the following error message: Error in model.frame.default(Terms, newdata, na.action = na.action, xlev = object$xlevels) : factor 'state1' has new level(s) AP Thank you!

Hello, I am using the glm model with a poisson distribution. The model runs just fine but when I try to get the null deviance for the model of the null degrees of freedom I get the following errors: > null.deviance(pAmeir_1) Error: couldn't find function "null.deviance" > df.null(pAmeir_1) Error: couldn't find function "df.null" When I do: >

Hi, I am using rpart package to fit classification trees. library(rpart) fit <- rpart(Kyphosis ~ Age + Number + Start, data=kyphosis) plot(fit,uniform=T) text(fit, use.n=TRUE) But I am unable to label the branches (not the nodes) of the tree. Can somebody help me out in this? Thank you, Regards Utkarsh Singhal | Amba Research Ph +91 80 3980 8017 | Mob +91 99 0295 8815

Hello everyone, I want to import data from huge sas files with 100s of columns. The good thing is that I am only interested in a few selected columns. Is there any way to do that without loading the full dataset. I have tried two functions: (1) read.sas7bdat *[from library 'sas7bdat']*, and (2) read_sas *[from library 'haven']. *But couldn't find what I am looking for. Best

Hi All, There are several replies to the question below, but I think there must exist a better way of doing so. I just want to check whether all the elements of a vector are same. My vector has one million elements and it is highly likely that there are distinct elements in the first few itself. For example: > x = c(1,2,rep(1,100000)) I want the answer as FALSE, which is clear from the