similar to: gauss fit with outlier removal

Hello, I need to analyse a data matrix with dimensions of 30x100. Before analysing the data there is, however, a need to remove outliers from the data. I read quite a lot about outlier removal already and I think the most common technique for that seems to be Principal Component Analysis (PCA). However, I think that these technqiue is quite subjective. When is an outlier an outlier? I uploaded

Hi, I am currently analysis Raman spectroscopic data with the hyperSpec package. I consulted the documentation on this package and I found an example work-flow dedicated to Raman spectroscopy (see the address : http://hyperspec.r-forge.r-project.org/chondro.pdf) I am currently trying to remove outliers thanks to PCA just as they did in the documentation, but I get a message error I can't

I have a statistical question. The data sets I am working with are right-skewed so I have been plotting the log transformations of my data. I am using a Grubbs Test to detect outliers in the data, but I get different outcomes depending on whether I run the test on the original data or the log(data). Here is one of the problematic sets: fgf2p50=c(1.563,2.161,2.529,2.726,2.442,5.047)

For the analysis of financial data wih a large variance, what is the best way to select an outlier threshold? Listed below, is there a best method to select an outlier threshold and how does R calculate it? In R, how do you find the outlier threshold through an interquartile range? In R, how do you find the outlier threshold using the hist command? In R, how do you find the outlier threshold

Hi! I'm trying to implement an outlier test once/row in a large dataframe. Ideally, I'd do this then add the Pvalue results and the number flagged as an outlier as two new separate columns to the dataframe. Grubbs outlier test requires a vector and I'm confused how to make each row of my dataframe a vector, followed by doing a Grubbs test for each row containing the vector of numbers

I have been learning about some outlier tests -- Dixon and Grubb, specifically -- for small data sets. When I try help.start() and search for outlier tests, the only response I manage to find is the Bonferroni test avaiable from the CAR package... are there any other packages the offer outlier tests? Are the Dixon and Grubb tests "good" for small samples or are others more

hi - if I sample from a normal distribution with something like n100<-rnorm(100,0,1) and add an outlier with n100[10]<-4 then qqnorm(n100) visually shows the point 4 as an outlier and calculating the probablity of a value of 4 or bigger in 100 samples of norm(0,1) gives > 1-exp(log(pnorm(4,0,1))*100) [1] 0.003162164 If I have more than 1 sample above outlier threshold the math is a

I try to use ?randomForest to find variables that are the most important to divide my dataset (continuous, categorical variables) in two given groups. But when I plot the outliers: plot(outlier(FemMalSex_NAavoid88.rf33, cls=FemMalSex_NAavoid88$Sex), type="h",col=c("red","green")[as.numeric(FemMalSex_NAavoid88$Sex)]) it seems to me that all my values appear as

I am a new user of R so please bear with me. I have reviewed some R books, FAQs and such but the volume of material is great. I am in the process of porting my current SAS and SVS Script code to Lotus Approach, R and WordPerfect. My question is, can you help me determine the best R method to implement the Hoaglin Outlier Method? It is used in the Appendix A and B of the fo llowing link.

Hi, I was just trying to figure out how to beautify the output of my bwplot-output. Altogether I figured most of the things out on my own. The one thing which puzzles me though are the symbols for the outliers. I can easily change the form of the median symbol by using "pch" but I don't know how to do this for outliers. Obviously the "outpch" of the

Hi there, I am using SAS Proc NLMIXED to maximize a likelihood with multivariate normal random effects. An example is the two part random effects model for repeated measures semi-continous data with a cluster at 0. I use the "model y ~ general(loglike)" statement in Proc NLMIXED, so I can specify a general log likelihood function constructed by SAS programming statements. Then the

I need to detect outliers in a large data set which is highly right-skewed. I plan to use medcouple-based outlier detection. Is there any support for medcouple-based outlier detection in R? Are there any other routines in R to perform outlier detection in highly right-skewed data? Manuj Sharma See the Web&#39;s breaking stories, chosen by people like you. Check out Yahoo! Buzz.

I recommend that you consult with a local statistical expert. Much of what you say (outliers?!?) seems to make little sense, and your statistical knowledge seems minimal. Perhaps more to the point, none of your questions can be properly answered without subject matter context, which this list is not designed to provide. That's why I believe you need local expertise. Bert Gunter "The

Dear R-experts, Searching the help archives I found a recommendation to do multivariate outlier identification by mahalanobis distances based on a robustly estimated covariance matrix and compare the resulting distances to a chi^2-distribution with p (number of your variables) degrees of freedom. I understand that compared to euclidean distances this has the advantage of being scale-invariant.

Hi, I was looking for a package that would help with outlier detection for bimodal distributions. I have tried 'outliers' and 'extremevalues' packages, but am not sure if they are ok for bimodal distribution. Any help would be highly appreciated! thanks, [[alternative HTML version deleted]]

Hi, I am trying to use rm.outlier method but encountering following error: > y <- rnorm(100) > rm.outlier(y) Error: Error in if (nrow(x) != ncol(x)) stop("x must be a square matrix") : argument is of length zero Whats wrong here? TIA Sachin __________________________________________________ [[alternative HTML version

You could assign a density value to each point. Maybe you've done that already...? Then trim the lowest n (number of) data points Or trim the lowest p (proportion of) data points. e.g. Remove the data points with the 20 lowest density values. Or remove the data points with the lowest 5% of density values. I'll let you decide whether that is a good idea or a bad idea. And if it's a

I have a large set of data from an experiment in which each individual has trials 1-40 with approx. 150 points sampled every 0.1 sec within each trial. Each trial is a column and the rows are organized by time.  I know there is a code to take out single outliers within a certain trial by using the chi square test code. But, I want to be able to determine if trials are outliers. I am having a tough

Hi Bert, Another confrontational response from you... You might have noticed that I use the word "outlier" carefully in this post and only in relation to the plotted ellipses. I do not know the underlying algorithm of geom_density_2d() and therefore I am having an issue of how to interpret the plot. I was hoping someone here knows that and can help me. Ana On Fri, Oct 9, 2020 at

Hello R users, Can someone tell if there is a package in R that can do outlier detection that give outputs simiilar to what I got from SAS below. Many thanks in advance for any help! Outlier Details Approx Chi-