[R] NA command in a 'for' loop

Jim Lemon
Tue, 21 Apr 2020 04:24:59 -0000

Hi Helen,
I can follow you this far:

# just read in the data from your example
d<-read.csv(text="2.90546E+11,threat,1,2,1,2,1,death,stove,NA,NA,205,0,394
2.90546E+11,threat,2,2,2,1,1,emaciated,shortened,NA,NA,205,0,502
2.90546E+11,threat,3,1,1,1,2,mutilate,consider,NA,NA,205,1,468
2.90546E+11,threat,6,1,2,2,1,weep,shop,NA,NA,203,1,345
2.90546E+11,threat,9,2,1,2,2,tormented,easygoing,NA,NA,205,1,373
2.90546E+11,threat,10,1,2,2,2,snake,table,NA,NA,205,1,343
2.90546E+11,threat,11,2,2,1,1,crisis,faucet,NA,NA,203,1,437
2.90546E+11,threat,12,1,1,1,1,victim,utensil,NA,NA,203,1,343
2.90546E+11,threat,14,1,2,2,1,depressed,repentant,NA,NA,203,1,441
2.90546E+11,threat,15,2,2,1,2,scum,shoe,NA,NA,205,1,475",
header=FALSE,stringsAsFactors=FALSE)
# get rows of d you want
d2<-d[d$V13==1,]
d2
congruent<-(d2$V4 == 1)
congruent

and things are as I expect even though I have shortened the code somewhat.
I'm not familiar with the "get_tlbs" function which I can find on
GitHub
but not on CRAN, so if this:

x<-get_tlbs(d2$V14,congruent,prior_weights=NULL,method="weighted",
 fill_gaps = FALSE)

gives you the "x" that you want, I think we're close.

Jim

On Tue, Apr 21, 2020 at 1:53 PM Helen Sawaya <helensawaya at hotmail.com>
wrote:

> Thank you all for your input.
>
> This is an example of one data file (I have 74 data files):
>
> 2.90546E+11, threat,    1, 2, 1, 2, 1,        death,        stove,
>   NA,           NA,  205,    0,  394
> 2.90546E+11, threat,    2, 2, 2, 1, 1,    emaciated,    shortened,
>   NA,           NA,  205,    0,  502
> 2.90546E+11, threat,    3, 1, 1, 1, 2,     mutilate,     consider,
>   NA,           NA,  205,    1,  468
> 2.90546E+11, threat,    6, 1, 2, 2, 1,         weep,         shop,
>   NA,           NA,  203,    1,  345
> 2.90546E+11, threat,    9, 2, 1, 2, 2,    tormented,    easygoing,
>   NA,           NA,  205,    1,  373
> 2.90546E+11, threat,   10, 1, 2, 2, 2,        snake,        table,
>   NA,           NA,  205,    1,  343
> 2.90546E+11, threat,   11, 2, 2, 1, 1,       crisis,       faucet,
>   NA,           NA,  203,    1,  437
> 2.90546E+11, threat,   12, 1, 1, 1, 1,       victim,      utensil,
>   NA,           NA,  203,    1,  343
> 2.90546E+11, threat,   14, 1, 2, 2, 1,    depressed,    repentant,
>   NA,           NA,  203,    1,  441
> 2.90546E+11, threat,   15, 2, 2, 1, 2,         scum,         shoe,
>   NA,           NA,  205,    1,  475
>
> ?Column 13 has values of 0s and 1s which my cognitive task outputted.
> Column 14 is the reaction time (ms) data. I want to get rid of the rows
> that contain zeros so I thought I'd first replace zeros with NAs then
use
> complete.cases function to get rid of the NAs. I also wanted to apply other
> functions so I included them all in a loop. All work fine except for the
> one where I try to turn the zeros to NAs.
>
> Jim when I tried your mockdata example, it worked fine. But when I
> translated it to my data, I still get zeros in the output. Can you identify
> any mistranslations I'm doing?
>
> txt.files<-list.files(".",pattern="dotprobe") #all
my data files are text
> files in one folder
> for(tf in txt.files) {
>   d<-read.table(tf)
>   d[,13][d[,13]==0]<-NA #column 13 contains zeros
>   d<-d[ ,-c(10,11)] #get rid of columns 10 and 11
>   write.table(d,sub("[.]",".tlbs.",tf),quote=FALSE,
row.names=FALSE)
> }
>
> That's an example of one of the output I get:
>
> V1 V2 V3 V4 V5 V6 V7 V8 V9 V12 V13 V14
> 2.90546E+11, threat, 1, 2, 1, 2, 1, death, stove, 205, 0, 394
> 2.90546E+11, threat, 2, 2, 2, 1, 1, emaciated, shortened, 205, 0, 502
> 2.90546E+11, threat, 3, 1, 1, 1, 2, mutilate, consider, 205, 1, 468
> 2.90546E+11, threat, 6, 1, 2, 2, 1, weep, shop, 203, 1, 345
> 2.90546E+11, threat, 9, 2, 1, 2, 2, tormented, easygoing, 205, 1, 373
> 2.90546E+11, threat, 10, 1, 2, 2, 2, snake, table, 205, 1, 343
>
> Columns 10 and 11 were deleted. But zeros were not replaced by NAs.
> After all the data cleaning, the functions I'm interested in including
in
> the loop are: get_tlbs and summarize_bias (and these also work fine in my
> loop).
>
> Thanks again ?
> Sincerely
> Helen
> ------------------------------
> *From:* Jim Lemon <drjimlemon at gmail.com>
> *Sent:* Tuesday, April 21, 2020 2:52 AM
> *To:* Rui Barradas <ruipbarradas at sapo.pt>
> *Cc:* Helen Sawaya <helensawaya at hotmail.com>; Michael Dewey <
> lists at dewey.myzen.co.uk>; r-help at R-project.org <r-help at
r-project.org>
> *Subject:* Re: [R] NA command in a 'for' loop
>
> Hi Helen,
> Your problem may lie in using row.names=TRUE. I was puzzled when an
> extra column kept popping up in the output files. For reading in and
> replacing zeros with NAs, this seems to work:
>
> for(mockdata in 1:3) {
>  mdf<-data.frame(sample(2:20,10),sample(2:20,10),sample(0:1,10,TRUE))
>
write.table(mdf,file=paste0("threat",mockdata,".txt"),quote=FALSE,
>   row.names=FALSE,col.names=FALSE)
> }
> txt.files<-list.files(".",pattern="threat[1-3]")
> for(tf in txt.files) {
>  d<-read.table(tf)
>  d[,3][d[,3]==0]<-NA
>
write.table(d,sub("[.]",".tbls.",tf),quote=FALSE,row.names=FALSE)
> }
>
> Jim
>
> On Tue, Apr 21, 2020 at 7:57 AM Rui Barradas <ruipbarradas at
sapo.pt> wrote:
> >
> > Hello,
> >
> > I believe the only way we have to see what is happening is for you to
> > post the output of
> >
> >
> > dput(head(d, 20))  # or 30
> >
> >
> > or, with d2 a subset of d that includes zeros,
> >
> >
> > dput(head(d2, 20))
> >
> >
> > Hope this helps,
> >
> > Rui Barradas
> >
> > ?s 17:48 de 20/04/20, Helen Sawaya escreveu:
> > > I have one column that represents correct response versus error
> (correct
> > > is coded as 1 and error is coded as 0). Nowhere else in the
dataset are
> > > there values of 0. The vector is treated as an integer.
> > >
> ------------------------------------------------------------------------
> > > *From:* Michael Dewey <lists at dewey.myzen.co.uk>
> > > *Sent:* Monday, April 20, 2020 7:35 PM
> > > *To:* Helen Sawaya <helensawaya at hotmail.com>; Rui
Barradas
> > > <ruipbarradas at sapo.pt>; r-help at R-project.org
<r-help at R-project.org>
> > > *Subject:* Re: [R] NA command in a 'for' loop
> > > Just a thought Helen but is x being treated as a real and what
you
> think
> > > are zero and are printed as zero are in fact some very small
number? If
> > > so you need to alter your test appropriately.
> > >
> > > Michael
> > >
> > > On 20/04/2020 17:25, Helen Sawaya wrote:
> > >> Thank you for your reply.
> > >>
> > >> I tried d[] <- lapply(d, function(x) {is.na(x) <- x ==
0; x})
> > >> but I am still getting zeros instead of NAs in my output..
> > >>
> > >> I wonder if the problem is that some of my data files
don't have any
> zeros (participants made no errors)..
> > >> ________________________________
> > >> From: Rui Barradas <ruipbarradas at sapo.pt>
> > >> Sent: Monday, April 20, 2020 9:05 AM
> > >> To: Helen Sawaya <helensawaya at hotmail.com>; r-help
at R-project.org
> <r-help at R-project.org>
> > >> Subject: Re: [R] NA command in a 'for' loop
> > >>
> > >> Hello,
> > >>
> > >> Instead of
> > >>
> > >> d[d == 0] <- NA
> > >>
> > >> try
> > >>
> > >> d[] <- lapply(d, function(x) {is.na(x) <- x == 0; x})
> > >>
> > >>
> > >> Also, in the first for loop
> > >>
> > >> paste(i, sep = "")
> > >>
> > >> does nothing, it's the same as i.
> > >> And the same for
> > >>
> > >> (d2$V4 == 1) == TRUE
> > >>
> > >> Since (d2$V4 == 1)  already is FALSE/TRUE there is no need
for
> > >>
> > >> (.) == TRUE
> > >>
> > >>
> > >> Hope this helps,
> > >>
> > >> Rui Barradas
> > >>
> > >>
> > >>
> > >> ?s 20:52 de 19/04/20, Helen Sawaya escreveu:
> > >>> Dear R experts,
> > >>>
> > >>> I am using a 'for' loop to apply commands to
multiple datasets (each
> file is one participant). The only one not working is the command that
> identifies zeros in my datasets and changes them to NAs. But when I look at
> the output, zeros ("0") are still present.  Surprisingly, the
functions
> work fine when I apply them to a single
> > > dataset (outside the loop). I've tried:
> > >>>
> > >>> all.files <- list.files(".")
> > >>> txt.files <-
grep("threat.txt",all.files,value=T)
> > >>>
> > >>> for(i in txt.files){
> > >>>     d <-
read.table(paste(i,sep=""),header=F)
> > >>>     d[d==0] <- NA #replace zeros with NA
> > >>>     write.table(d, paste0(i,".tlbs.txt"),
quote=FALSE,
> row.names=TRUE)}
> > >>>     d<-d[ ,-c(10,11)]
> > >>>     d2<-d[complete.cases(d), ]
> > >>>     d2$V4<-as.numeric(d2$V4)
> > >>>     congruent <- (d2$V4 == 1) == TRUE
> > >>>     x <- get_tlbs(d2$V14, congruent, prior_weights =
NULL, method > "weighted", fill_gaps = FALSE)
> > >>>     write.table(x, paste0(i,".tlbs.txt"),
quote=FALSE,
> row.names=TRUE)}
> > >>>
> > >>> I've also tried:
> > >>>
> > >>> for(i in txt.files){
> > >>>     d <-
read.table(paste(i,sep=""),header=F)
> > >>>     if (0 %in% d)
> > >>>     {replace_with_na(d,replace = list(x = 0))} # replace
zeros with
> NA
> > >>>     d<-d[ ,-c(10,11)]
> > >>>     d2<-d[complete.cases(d), ]
> > >>>     d2$V4<-as.numeric(d2$V4)
> > >>>     congruent <- (d2$V4 == 1) == TRUE
> > >>>     x <- get_tlbs(d2$V14, congruent, prior_weights =
NULL, method > "weighted", fill_gaps = FALSE)
> > >>>     write.table(x,
paste0(i,".summaryoutput.txt"), quote=FALSE,
> row.names=TRUE)}
> > >>>
> > >>> Thank you for your help.
> > >>> Sincerely
> > >>> Helen
> > >>>
> > >>>         [[alternative HTML version deleted]]
> > >>>
> > >>> ______________________________________________
> > >>> R-help at r-project.org mailing list -- To UNSUBSCRIBE
and more, see
> > >>> https://stat.ethz.ch/mailman/listinfo/r-help
> > >>> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > >>> and provide commented, minimal, self-contained,
reproducible code.
> > >>>
> > >>
> > >>        [[alternative HTML version deleted]]
> > >>
> > >> ______________________________________________
> > >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and
more, see
> > >> https://stat.ethz.ch/mailman/listinfo/r-help
> > >> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > >> and provide commented, minimal, self-contained, reproducible
code.
> > >>
> > >>
> > >
> > > --
> > > Michael
> > > http://www.dewey.myzen.co.uk/home.html
> >
> > ______________________________________________
> > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html