Hello-
I have a script which steps through a series of subjects, and for the
subjects I remove outlying values. After removing these outliers, I
specify a cutoff, keeping only values over a certain value (e.g., 1.96).
I want to populate a matrix with a statistic of the values that make the
cutoff (for example, the mean). However, in some subjects, after
outliers and the cutoff are specified, there are no data that meet the
criteria (I get <0 rows> (or 0-length row.names)). Here the script dies.
The solution I think is to specify a break so that the matrix will be
populated with a value (such as NA) and it will move on to the next
subject in the loop. However, I haven't been able to figure this out. If
anyone has any suggestions I would very much appreciate them. I have
paster part of the script below up to the point where it dies.
Thanks.
for (ss in levels(ss.list)) {
print(ss)
ss.count = ss.count + 1
query.string <- paste("SELECT * FROM
",ss,";",sep="")
#print(query.string)
data_gcs <- dbGetQuery(con, query.string)
attach(data_gcs)
names(data_gcs)
mat_row = 0
for(i in levels(roi.list)){
print (i)
current.roi <- data_gcs[data_gcs$ROI==i,]
current.roi.plus.z <- data.frame(current.roi,
scale(current.roi[,15]), scale(current.roi[,18]),
scale(current.roi[,21]), scale(current.roi[,24]))
testextrem <- function(x) {if ((abs(x[1]) < 2.5) &
(abs(x[2]) < 2.5) & (abs(x[3]) < 2.5) & (abs(x[4]) < 2.5))
return(1)
else return(0)}
filtervector <- apply(as.vector(current.roi.plus.z[,c(27,
28, 29, 30)]), 1, FUN=testextrem)
current.roi.plus.z.filter <- data.frame(current.roi.plus.z,
filtervector)
final.roi.df <-
current.roi.plus.z.filter[which(current.roi.plus.z.filter$filtervector==1),]
#print (final.roi.df)
kicked.out<-(length(filtervector) -
sum(filtervector))/length(filtervector)
print(kicked.out)
matrix.col = matrix.col + 1
attach(final.roi.df)
names(final.roi.df)
print(matrix.col)
#set cutoff for FDR. per voxel p values (of Z dist) .05 =
1.96, .01 = 2.575, .001 = 3.277, .005 = 3.479 BE SURE TO CHANGE THE
VARIABLE EACH TIME YOU CHANGE CONDITION
pre.outliers<-subset(final.roi.df, gFDR4FWHM >= 1.96)
detach(final.roi.df)
attach(pre.outliers)
outliers<-subset(pre.outliers, gtvalue4FWHM >= 0.00)
...and the script dies here because there are no data in "outliers".
--
Anthony Steven Dick, Ph.D.
Post-Doctoral Scholar
Human Neuroscience Laboratory
Biological Sciences Division
University of Chicago
5841 S. Maryland Ave. MC-2030
Chicago, IL 60637
Phone: (773)-834-7770
Email: adick at uchicago.edu
Alternate email: anthony at anthonymail.com
Try changing
outliers <- subset(pre.outliers, gtvalue4FWHM >= 0.00)
to
w <- which( gtvalue4FWHM >= 0.00 )
outliers( length(w) > 0, pre.outliers[ w, ], NA )
Other comments:
1. Make sure you detach(data_gcs) at the end of the loops
2. scale() works on columns by default, so try
current.roi.plus.z <- data.frame(current.roi,
scale(current.roi[, c(15,18,21,24)]) )
3. For simple statements, you can use ifelse() syntax. Also see all().
Using these two functions, you can try
testextrem <- function(x) ifelse( all(abs(x[ 1:4 ]) < c(2.5)), 1, 0)
4. Try to minimise the use of attach() in loops.
5. You might be interested in learning more about apply(), tapply(),
split() etc.
If this does not help, then please resend a simplified version of the
codes, preferably with a simple toy example to illustrate.
Regards, Adai
Anthony Steven Dick wrote:> Hello-
>
> I have a script which steps through a series of subjects, and for the
> subjects I remove outlying values. After removing these outliers, I
> specify a cutoff, keeping only values over a certain value (e.g., 1.96).
> I want to populate a matrix with a statistic of the values that make the
> cutoff (for example, the mean). However, in some subjects, after
> outliers and the cutoff are specified, there are no data that meet the
> criteria (I get <0 rows> (or 0-length row.names)). Here the script
dies.
>
> The solution I think is to specify a break so that the matrix will be
> populated with a value (such as NA) and it will move on to the next
> subject in the loop. However, I haven't been able to figure this out.
If
> anyone has any suggestions I would very much appreciate them. I have
> paster part of the script below up to the point where it dies.
>
> Thanks.
>
> for (ss in levels(ss.list)) {
> print(ss)
> ss.count = ss.count + 1
> query.string <- paste("SELECT * FROM
",ss,";",sep="")
> #print(query.string)
> data_gcs <- dbGetQuery(con, query.string)
> attach(data_gcs)
> names(data_gcs)
> mat_row = 0
> for(i in levels(roi.list)){
> print (i)
> current.roi <- data_gcs[data_gcs$ROI==i,]
> current.roi.plus.z <- data.frame(current.roi,
> scale(current.roi[,15]), scale(current.roi[,18]),
> scale(current.roi[,21]), scale(current.roi[,24]))
> testextrem <- function(x) {if ((abs(x[1]) < 2.5) &
> (abs(x[2]) < 2.5) & (abs(x[3]) < 2.5) & (abs(x[4]) < 2.5))
return(1)
> else return(0)}
> filtervector <- apply(as.vector(current.roi.plus.z[,c(27,
> 28, 29, 30)]), 1, FUN=testextrem)
> current.roi.plus.z.filter <- data.frame(current.roi.plus.z,
> filtervector)
> final.roi.df <-
>
current.roi.plus.z.filter[which(current.roi.plus.z.filter$filtervector==1),]
> #print (final.roi.df)
> kicked.out<-(length(filtervector) -
> sum(filtervector))/length(filtervector)
> print(kicked.out)
> matrix.col = matrix.col + 1
> attach(final.roi.df)
> names(final.roi.df)
> print(matrix.col)
> #set cutoff for FDR. per voxel p values (of Z dist) .05 =
> 1.96, .01 = 2.575, .001 = 3.277, .005 = 3.479 BE SURE TO CHANGE THE
> VARIABLE EACH TIME YOU CHANGE CONDITION
> pre.outliers<-subset(final.roi.df, gFDR4FWHM >= 1.96)
> detach(final.roi.df)
> attach(pre.outliers)
> outliers<-subset(pre.outliers, gtvalue4FWHM >= 0.00)
> ...and the script dies here because there are no data in
"outliers".
>