similar to: Memory usage in prcomp

Following from the R-help thread of March 22 on "Memory usage in prcomp", I've started looking into adding an optional 'rank.' argument to prcomp allowing to more efficiently get only a few PCs instead of the full p PCs, say when p = 1000 and you know you only want 5 PCs. (https://stat.ethz.ch/pipermail/r-help/2016-March/437228.html As it was mentioned, we already

I agree with Kasper, this is a 'big' issue. Does your method of taking only n PCs reduce the load on memory? The new addition to the summary looks like a good idea, but Proportion of Variance as you describe it may be confusing to new users. Am I correct in saying Proportion of variance describes the amount of variance with respect to the number of components the user chooses to show? So

> On 25 Mar 2016, at 10:41 am, peter dalgaard <pdalgd at gmail.com> wrote: > > As I see it, the display showing the first p << n PCs adding up to 100% of the variance is plainly wrong. > > I suspect it comes about via a mental short-circuit: If we try to control p using a tolerance, then that amounts to saying that the remaining PCs are effectively zero-variance, but

Hello, not understanding the output of prcomp, I reduce the number of components and the output continues to show cumulative 100% of the variance explained, which can't be the case dropping from 8 components to 3. How do i get the output in terms of the cumulative % of the total variance, so when i go from total solution of 8 (8 variables in the data set), to a reduced number of

The following seems to be an bug in prcomp(): > test <- ts( matrix( c(NA, 2:5, NA, 7:10), 5, 2)) > test Time Series: Start = 1 End = 5 Frequency = 1 Series 1 Series 2 1 NA NA 2 2 7 3 3 8 4 4 9 5 5 10 > prcomp(test, scale.=TRUE, na.action=na.omit) Erro en svd(x, nu = 0) : infinite or missing values in 'x'

Martin, I fully agree. This becomes an issue when you have big matrices. (Note that there are awesome methods for actually only computing a small number of PCs (unlike your code which uses svn which gets all of them); these are available in various CRAN packages). Best, Kasper On Thu, Mar 24, 2016 at 1:09 PM, Martin Maechler <maechler at stat.math.ethz.ch > wrote: > Following from

Full_Name: Ryszard Czerminski Version: 1.8.1 OS: GNU/Linux Submission from: (NULL) (205.181.102.120) prcomp(..., scale = TRUE) does not work correctly: $ uname -a Linux 2.4.20-28.9bigmem #1 SMP Thu Dec 18 13:27:33 EST 2003 i686 i686 i386 GNU/Linux $ gcc --version gcc (GCC) 3.2.2 20030222 (Red Hat Linux 3.2.2-5) > a <- matrix(rnorm(6), nrow = 3) > sum((scale(a %*% svd(cov(a))$u, scale

As I see it, the display showing the first p << n PCs adding up to 100% of the variance is plainly wrong. I suspect it comes about via a mental short-circuit: If we try to control p using a tolerance, then that amounts to saying that the remaining PCs are effectively zero-variance, but that is (usually) not the intention at all. The common case is that the remainder terms have a roughly

According to R help: princomp() uses eigenvalues of covariance data. prcomp() uses the SVD method. yet when I run the (eg., USArrests) data example and compare with my own "hand-written" versions of PCA I get what looks like the opposite. Example: comparing the variances I see: Using prcomp(USArrests) ------------------------------------- Standard deviations: [1] 83.732400 14.212402

Full_Name: Juha Heljoranta Version: R 2.1.1 (2005-06-20) OS: Gentoo Linux Submission from: (NULL) (88.112.29.250) prcomp has a bug which causes following error Error in svd(x, nu = 0) : infinite or missing values in 'x' on a valid data set (no Infs, no missing values). The error is most likely caused by the zeros in data. My code and temporary workaround: m = matrix(... ...

Hi there, I was wondering if anyone could explain how you should set tol in the prcomp function. Using help(prcomp) explains it as "a value indicating the magnitude below which components should be omitted. (Components are omitted if their standard deviations are less than or equal to tol times the standard deviation of the first component.) With the default null setting, no components

> On 25 Mar 2016, at 10:08 , Jari Oksanen <jari.oksanen at oulu.fi> wrote: > >> >> On 25 Mar 2016, at 10:41 am, peter dalgaard <pdalgd at gmail.com> wrote: >> >> As I see it, the display showing the first p << n PCs adding up to 100% of the variance is plainly wrong. >> >> I suspect it comes about via a mental short-circuit: If we

Hello, Can you get eigenvalues in addition to eigevectors using prcomp? If so how? I am unable to use princomp due to small sample sizes. Thank you in advance for your help! Rebecca Young -- Rebecca Young Graduate Student Ecology & Evolutionary Biology, Badyaev Lab University of Arizona 1041 E Lowell Tucson, AZ 85721-0088 Office: 425BSW rlyoung at email.arizona.edu (520) 621-4005

Dear all I have a big matrix of standardized values (dimensions 285x5829) and R fails to calculate the principal components using prcomp() with the following error message: pc <- prcomp(my.matrix) Error in La.svd(x, nu, nv, method) : error code 1 from Lapack routine dgesdd Is the matrix too big? I'm using R-1.8.1 under Unix (Solaris8) and Linux(Suse 8.2). I tried to perform a principal

Hello, I have a short question about the prcomp function. First I cite the associated help page (help(prcomp)): "Value: ... SDEV the standard deviations of the principal components (i.e., the square roots of the eigenvalues of the covariance/correlation matrix, though the calculation is actually done with the singular values of the data matrix). ROTATION the matrix of variable loadings

Hi, I am trying to do a PCA on my data but I keep getting the error message svd(x, nu=0) infinite or missing values >From the messages posted on the subject, I understand that the NAs in my data might be the problem, but I thought na.omit would take care of that. Less than 5% of my cells are missing data. However, the NAs are not regularly distributed across my matrix: certain cases and

In the manual page for prcomp(), it says that sdev is "the standard deviations of the principal components (i.e., the square roots of the eigenvalues of the covariance/correlation matrix, though the calculation is actually done with the singular values of the data matrix)." ?However, this is not what I'm finding. ?The values appear to be the standard deviations of a reprojection of

The help for prcomp on R 1.0.0 states that the component sdev of the return value is the eigenvalues of the cov matrix. Am I completely mistaken, or should this be the _square root_ of the eigenvalues? Also, the documentation is not very clear about how tol is used to omit components. (The _code_ is clear, though. :-) -- B/H

Per the discussion about the problems with prcomp() when n << p, which boils down to a problem with svd() when n << p, here is a patch to prcomp() which substitutes La.svd() instead of svd(). -Greg (This is really a feature enhancement, but submitted to R-bugs to make sure it doesn't get lost. ) *** R-1.6.0/src/library/mva/R/prcomp.R Mon Aug 13 17:41:50 2001 ---

In comparing the results of princomp and prcomp I find: 1. The reported standard deviations are similar but about 1% from each other, which seems well above round-off error. 2. princomp returns what I understand are variances and cumulative variances accounted for by each principal component which are all equal. "SS loadings" is always 1. 3. Same happens