Hyun Soo Park
2021-May-08 10:05 UTC
[R] factor analysis of dynamic structure (FADS) for a huge time-series data
Dear R users, I want to find the latent factors from a kind of time-series data describing temporal changes of concentration using a factor analysis technique called 'factor analysis of dynamic structure (FADS).' I learned how to form the data for the analysis using a proper package embedding FADS, such as 'fad' package. The analysis with 'fad' worked and gave me results, but the problem was raised when the time-series data is vast. The time-series data extracted from the 3-dimensional matrix (i.e., 3D image volume of 50 x 50 x 163) repeatedly acquired at 54-time points is consisted of 50 x 50 x 163 x 54 = 22,005,000 observations. The desired number of the latent factor (k) is 4. What I got from fad(MATRIX, k) is following: Error in fun(A, k, nu, nv, opts, mattype = "matrix") : TridiagEigen: eigen decomposition failed When I resize the matrix smaller into 5 x 5 x 15, it gives me what I wanted properly. I found that some resampling methods such as random sampling, data stratification, etc., could resolve this kind of problem, but I have no ideas which one could be appropriate. Please teach me with any ideas and comments. Thanks in advance, Park -- *??????, ?????????* *????:* (???) +82-31-787-2936 (????) +82-10-8833-2806 *??:* +82-31-787-4018 *???:* hyuns at snu.ac.kr *Hyun Soo Park, PhD* *--* *Research professor* Department of Nuclear Medicine Seoul National University Bundang Hospital, Seongnam, Korea *Telephone:* (Office) +82-31-787-2936 (Mobile) +82-10-8833-2806 *Fax:* +82-31-787-4018 *email:* hyuns at snu.ac.kr [[alternative HTML version deleted]]
Jeff Newmiller
2021-May-08 21:00 UTC
[R] factor analysis of dynamic structure (FADS) for a huge time-series data
This not being a question about R, but rather about statistics, or possibly about a contributed package, means (per the Posting Guide) that you should be asking in a statistics forum like stats.stackexchange.com or corresponding with the author of the package in question. If you are lucky someone here will have something to offer, but it is not very likely. On May 8, 2021 3:05:12 AM PDT, Hyun Soo Park <hyuns at snu.ac.kr> wrote:>Dear R users, > >I want to find the latent factors from a kind of time-series data >describing temporal changes of concentration using a factor analysis >technique called 'factor analysis of dynamic structure (FADS).' I >learned >how to form the data for the analysis using a proper package embedding >FADS, such as 'fad' package. > >The analysis with 'fad' worked and gave me results, but the problem was >raised when the time-series data is vast. > >The time-series data extracted from the 3-dimensional matrix (i.e., 3D >image volume of 50 x 50 x 163) repeatedly acquired at 54-time points is >consisted of 50 x 50 x 163 x 54 = 22,005,000 observations. The desired >number of the latent factor (k) is 4. What I got from fad(MATRIX, k) is >following: > >Error in fun(A, k, nu, nv, opts, mattype = "matrix") : > TridiagEigen: eigen decomposition failed > >When I resize the matrix smaller into 5 x 5 x 15, it gives me what I >wanted >properly. > >I found that some resampling methods such as random sampling, data >stratification, etc., could resolve this kind of problem, but I have no >ideas which one could be appropriate. > >Please teach me with any ideas and comments. > >Thanks in advance, > >Park-- Sent from my phone. Please excuse my brevity.
Ranjan Maitra
2021-May-09 02:28 UTC
[R] factor analysis of dynamic structure (FADS) for a huge time-series data
I am an author of the paper behind the fad package. I suspect that the call is not correct. Actually, fad does not quite account for time series or other structured data and you have to enter it, as in all general EFA packages as a n x p matrix, with n the number of observations and p the number of coordinates. So, if you can provide a reproducible example, I can look into it, or you can also file an issue on the github site. One thing to note that EFA requires all variances in the dispersion matrix to be positive, and it is possible that your images have some background where there is no activity and hence the sd for those pixel/voxels are zero. Of course, ideally, your EFA should account for the image structure, but that is a different topic and not part of fad or any similar package. Ranjan PS: I monitor this e-mail address only through this list. ? ? ? Sent:?Saturday, May 08, 2021 at 5:05 AM From:?"Hyun Soo Park" <hyuns at snu.ac.kr> To:?"r-help at r-project.org" <r-help at r-project.org> Subject:?[R] factor analysis of dynamic structure (FADS) for a huge time-series data Dear R users, I want to find the latent factors from a kind of time-series data describing temporal changes of concentration using a factor analysis technique called 'factor analysis of dynamic structure (FADS).' I learned how to form the data for the analysis using a proper package embedding FADS, such as 'fad' package. The analysis with 'fad' worked and gave me results, but the problem was raised when the time-series data is vast. The time-series data extracted from the 3-dimensional matrix (i.e., 3D image volume of 50 x 50 x 163) repeatedly acquired at 54-time points is consisted of 50 x 50 x 163 x 54 = 22,005,000 observations. The desired number of the latent factor (k) is 4. What I got from fad(MATRIX, k) is following: Error in fun(A, k, nu, nv, opts, mattype = "matrix") : TridiagEigen: eigen decomposition failed When I resize the matrix smaller into 5 x 5 x 15, it gives me what I wanted properly. I found that some resampling methods such as random sampling, data stratification, etc., could resolve this kind of problem, but I have no ideas which one could be appropriate. Please teach me with any ideas and comments. Thanks in advance, Park -- *??????, ?????????* *????:* (???) +82-31-787-2936 (????) +82-10-8833-2806 *??:* +82-31-787-4018 *???:* hyuns at snu.ac.kr *Hyun Soo Park, PhD* *--* *Research professor* Department of Nuclear Medicine Seoul National University Bundang Hospital, Seongnam, Korea *Telephone:* (Office) +82-31-787-2936 (Mobile) +82-10-8833-2806 *Fax:* +82-31-787-4018 *email:* hyuns at snu.ac.kr [[alternative HTML version deleted]] ______________________________________________ R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html[http://www.R-project.org/posting-guide.html] and provide commented, minimal, self-contained, reproducible code.