similar to: Cleaning up messy Excel data

Displaying 20 results from an estimated 20000 matches similar to: "Cleaning up messy Excel data"

2012 Jun 11
3
Decision Trees or Markov Models for Cost Effectiveness
Hello, I was just assigned to perform a cost effectiveness study in healthcare. We are studying the cost effectiveness of a proposed diagnostic vs. current screening procedures. One of the team members suggest a commercial software package called "TreeAge Pro". Looking at the description, it appears to be a nice GUI to some very simple models that could be easily constructed in R.
2012 Oct 09
4
Convert COLON separated format
I have a bunch of data sets that were created for the libsvm tool. They are in "colon separated sparse format". i.e. 1 5:1 27:3 345:10 Is a row with the label of "1" and only has values in columns 5, 27, and 345. I want to read these into a data.frame in R. Is there a simple way to do this? -- Noah Silverman, M.S. UCLA Department of Statistics 8117 Math Sciences
2012 Oct 14
4
Date Math
Hello, I have a time series object (xts) that I iterate over in a loop. Works fine. My challenge is that I want to be able to reference other entries in the series by math. i.e. For today's observation, what were the last 5 observations? If indexed numerically, it is trivial, but I can figure out how to do this with dates. This is slightly more difficult as there may not be an
2012 May 18
4
Menus - best practices?
Hello, I need to design a fairly simple front-end for someone to use an R script system that I've built. My thought was to just use the text based menus available in the base R package, perhaps in some kind of loop. How have other people done this? Any "best practices" that you can recommend? Thanks! -- Noah Silverman UCLA Department of Statistics 8117 Math Sciences Building
2011 Sep 02
2
Avoiding for Loop for moving average
Hello, I need to calculate a moving average and an exponentially weighted moving average over a fairly large data set (500K rows). Doing this in a for loop works nicely, but is slow. ewma <- data$col[1] N <- dim(data)[1] for(i in 2:N){ data$ewma <- alpha * data$ewma[i-1] + (1-alpha) * data$value[i] } Since the moving average "accumulates" as we move through the data,
2012 May 18
2
Failure building any package
Hello, I'm attempting to build a package using R 2.15.0 on OS X I am getting a generic failure when performing a cran type check on the package. Even with a very simple test package, it still fails int he same place. Example: In R: rm(list=ls()) foo <- function(x){print(x)} package.skeleton(name="foo") Then, at the command line: R CMD build foo R CMD check --as-cran
2011 Aug 23
2
dummy variables from factors
Hi, Looking at a large data set with many factors. I would like to expand each factor variable into multiple new variables for each level. (0,1) coding. My first though was just to code a big nasty loop, to take each level and cbind a column onto my data set. But, that seems painful. There must be a better way. Is there an "easy" way to do this in R? (Note, I don't want to
2011 Aug 20
1
Raw epoch time from XTS
Hi, I have a very large data set stored as an xts object. xts is very nice about showing row labels as "human readable" dates and times. I want the actual epoch values that are stored internally. The only way I can find to access them is one-at-a-time using the internal function: xcoredata() Calling this in an entire column, the "R" way doesn't work. It will only
2013 Apr 16
2
Strange error with log-normal models
Hi, I have some data, that when plotted looks very close to a log-normal distribution. My goal is to build a regression model to test how this variable responds to several independent variables. To do this, I want to use the fitdistr tool from the MASS package to see how well my data fits the actual distribution, and also build a generalized linear model using the glm command. The summary
2011 Jun 20
1
access objects by name
Hi, I have several data structures (xts structures). I then have a list of the names of those objects. I'd like to access the object by name. For example: foo1 <- as.xts(....) foo2 <- as.xts(...) foo3 <- as.xts(...) structs <- c("foo1", "foo2", "foo3") for (thisOne in structs){ print(thisOne$colA) } The above fails. Clearly I'm missing a
2011 Aug 23
1
Glmnet lambda value choice
Hi, When using the glmnet() function of the package glmnet, A series of coefficients is returned for a list of descending lambda values. I am unable to locate anything in the documentation that explains HOW this choice of lambda series is made. (There is documentation about how to choose my own, but I want to understand how the authors are doing it) Any ideas? -- Noah Silverman UCLA
2011 Sep 14
1
Open file for reading and writing with APPEND
Hi, I want to keep a running file of some statistics generated by a running process. Occasionally the process will be restarted. On restart, I'd like to read the last line of the log file to set some variables to their last state, and then continue to append values to that same file. I can't seem to get the appending part to work. I can read values form the file, but then upon the
2011 Jul 24
3
Variable scope in functions - best practices
Hi, I'm working on coding some more complex things in R and have need to break much of the logic into functions. I have several "global" variables that I want to change with a given function. (The variable has a different value after the function is called.) In other languages like C, this is simple. However, in R, if a function changes a variable, that change only occurs in the
2012 Nov 25
1
rjags and parallel chains
Hello, I have a fairly complex hierarchical model that I using rjags to fit. Short test runs verify that it works and everything appears to be setup correctly. Now that I want to collect a larger sample from the posterior (5,000 or more). This looks like it will take several days to run on my hardware (Intel core i7, 16GB RAM) I read in the rjags documentation that there is an option for
2011 Oct 17
1
Independent component analysis with only one "source" of data
Hi, Looking at the fastICA library. I want to test separating out different sounds from a recorded wav file. But, I only have a SINGLE stream of data (one channel wav.) It appears as if the fastICA won't let me separate more sounds than I have columns in my data matrix. is this correct? If so, is there a work around that is commonly used? Thanks. -- Noah Silverman UCLA Department of
2011 Aug 25
1
Optim function with multivariate inputs
Hi, I have function that I want to optimize. Am playing with the optim() function in R Two issues: 1) I can't seem to get it to work with a function that takes multiple inputs. Dummy Example: myFunc <- function(A,B,D,D){ # Do stuff return E } > myFunc(1,2,3,4) [1] 12 # works fine from command line > optim( par=c(1,2,3,4), fn=myFunc) Error in A+B : 'B' is missing
2011 Oct 16
2
Use of ICA for sound
Hi, I'm looking at the "cocktail party" classic problem. I can see how to use ICA to separate the components. But, How do I then create new wav files of the separated sounds so that they can be played? Thanks -- Noah Silverman UCLA Department of Statistics 8208 Math Sciences Building Los Angeles, CA 90095 [[alternative HTML version deleted]]
2012 Feb 13
1
Warnings from script - where?
Hi, Next challenge today. I have a script that I call within R: source("foo.R") When it finishes, I get the dreaded output: "There were 50 or more warnings (use warnings() to see the first 50)" So, I type warnings() and get a nice list. (Same error repeatedly, so probably something I'm using in a loop.) The difficult part is I have no idea *where* the warning is
2013 Mar 19
1
RGL 3D plots are flat. Please Help
Hello, I have a matrix of simulated data that I want to plot as a 3D surface using RGL. Have followed the documentation carefully, but my plot only contains a weird "single slice" of the data. The actual RGL library and dependance seem fine as all of the demo code plots beautifully. The actual command I'm using is: rgl.surface(x,z,d,
2012 May 29
2
Converting to XTS loses data.frame structure
Hello, I noticed something odd when working with data frames and xts objects. If I read in a CSV file, R creates a nice data.frame. This works well. If I then convert to an XTS object, I see that all the values in the data are now quoted. My data is a mix of numeric and character. This is usually seen when converting a data.frame to a matrix, as R will treat all the data as the same class.