search for: noahsilverman

Hello, I was just assigned to perform a cost effectiveness study in healthcare. We are studying the cost effectiveness of a proposed diagnostic vs. current screening procedures. One of the team members suggest a commercial software package called "TreeAge Pro". Looking at the description, it appears to be a nice GUI to some very simple models that could be easily constructed in R.

Hi, I have a very large data set (aprox. 100,000 rows.) The data comes from around 10,000 "groups" with about 10 entered per group. The values are in one column, the group ID is an integer in the second column. I want to normalize the values by group: for(g in unique(groups){ x[group==g] / sum(x[group==g]) } This works find in a loop, but is slow. Is there a faster way to do

Hello, I noticed something odd when working with data frames and xts objects. If I read in a CSV file, R creates a nice data.frame. This works well. If I then convert to an XTS object, I see that all the values in the data are now quoted. My data is a mix of numeric and character. This is usually seen when converting a data.frame to a matrix, as R will treat all the data as the same class.

Hello, I'm attempting to build a package using R 2.15.0 on OS X I am getting a generic failure when performing a cran type check on the package. Even with a very simple test package, it still fails int he same place. Example: In R: rm(list=ls()) foo <- function(x){print(x)} package.skeleton(name="foo") Then, at the command line: R CMD build foo R CMD check --as-cran

Hi, I have a CSV file that is formatted well, except that the last line is a "summary" not is CSV format. Toy example: label_1, label_2, label_3 1,2,3 3,2,4 2,3,4 Total Rows: 3 When I try to import this into R with: d <- read.table("foo.csv", header=T, sep=",") It fails to import properly because of the last line. Currently, I have a shell script that strips

I have a bunch of data sets that were created for the libsvm tool. They are in "colon separated sparse format". i.e. 1 5:1 27:3 345:10 Is a row with the label of "1" and only has values in columns 5, 27, and 345. I want to read these into a data.frame in R. Is there a simple way to do this? -- Noah Silverman, M.S. UCLA Department of Statistics 8117 Math Sciences

Unfortunately, some data I need to work with was delivered in a rather messy Excel file. I want to import into R and clean up some things so that I can do my analysis. Pulling in a CSV from Excel is the easy part. My current challenge is dealing with some text mixed in the values. i.e. 118 5.7 <2.0 3.7 Since this column in Excel has a "<2.0" value, then R reads the

Hello, I'm trying to read in a fairly large file into R, and am getting an odd error (65000 rows, 37 columns) Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings, : line 25628 did not have 37 elements That line DOES have 37 elements. As A test, I tried deleting it, and a few surrounding lines. Same error occurs with a different line number. Is there some hard

Hi, I want to store the output of my program to a file. However, With subsequent runs of my code, I'd like to append to the same log file. Currently, I'm using: outfile <- file("log.txt", open="w") cat(results, file=outfile) This works, but will overwrite the log file each time. Is there a way to open a file and have R append to the end? Thanks! -- Noah

Hi, I have several data structures (xts structures). I then have a list of the names of those objects. I'd like to access the object by name. For example: foo1 <- as.xts(....) foo2 <- as.xts(...) foo3 <- as.xts(...) structs <- c("foo1", "foo2", "foo3") for (thisOne in structs){ print(thisOne$colA) } The above fails. Clearly I'm missing a

Hi, When using the glmnet() function of the package glmnet, A series of coefficients is returned for a list of descending lambda values. I am unable to locate anything in the documentation that explains HOW this choice of lambda series is made. (There is documentation about how to choose my own, but I want to understand how the authors are doing it) Any ideas? -- Noah Silverman UCLA

Hi, I'm interested in doing some sound analysis with R. Does anyone have any experience/methods for reading in a wav file? -- Noah Silverman UCLA Department of Statistics 8208 Math Sciences Building Los Angeles, CA 90095 [[alternative HTML version deleted]]

Hi, i have some data in a matrix. It has zero values scattered throughout, at random. I'd like to create a line plot, with a line for each row, that *excludes* the zero or NA values. The data looks like this (toy example) 10 12 21 0 23 0 43 0 NA 41 0 0 0 34 35 0 35 0 44 0 NA NA NA 3 2 5 0 3 2 etc... Suggestions on an easy way to do this? Thanks! -- Noah Silverman UCLA Department of

Hi, I have a rather large data frame (500 x 5000) that I want to convert to a proper xts object. I am able to properly generate an xts object with the correct time index. However, all of my numerical values are now strings. b <- as.xts(a[,2:dim(a)[2]], order.by=as.POSIXct(strptime(paste(a$Date), '%m/%d/%Y'))) My guess is that somewhere in the large data frame there are a few

Hello, I am attempting to use elasticnet to classify a number of documents. The features are words. The data is coded into a matrix with each document as a row and each word as a column. The data is binary, with {0,1} indicating the presence of a word. I want to use the cross validation function of elasticnet (cv.enet). However, when the code selects a random subset of the data for a given

Hello, I coming across a strange problem doing math on an xts object. If I have an xts object of stock prices (perhaps 5 minute bars of open, high, low,close) and want to do some math, the results fail. For example: d$close[10] - d$open[10] works perfectly d$close[10] - d$open[9] fails. I just get an answer of "numeric(0) Index: numeric(0)". My guess is that xts is breaking

Hi, I want to keep a running file of some statistics generated by a running process. Occasionally the process will be restarted. On restart, I'd like to read the last line of the log file to set some variables to their last state, and then continue to append values to that same file. I can't seem to get the appending part to work. I can read values form the file, but then upon the

Hi, I have a very large data set stored as an xts object. xts is very nice about showing row labels as "human readable" dates and times. I want the actual epoch values that are stored internally. The only way I can find to access them is one-at-a-time using the internal function: xcoredata() Calling this in an entire column, the "R" way doesn't work. It will only

Hi, Looking at a large data set with many factors. I would like to expand each factor variable into multiple new variables for each level. (0,1) coding. My first though was just to code a big nasty loop, to take each level and cbind a column onto my data set. But, that seems painful. There must be a better way. Is there an "easy" way to do this in R? (Note, I don't want to

Hello, I need to calculate a moving average and an exponentially weighted moving average over a fairly large data set (500K rows). Doing this in a for loop works nicely, but is slow. ewma <- data$col[1] N <- dim(data)[1] for(i in 2:N){ data$ewma <- alpha * data$ewma[i-1] + (1-alpha) * data$value[i] } Since the moving average "accumulates" as we move through the data,