R help - Jan 2013 - a function more appropriate than 'sapply'?

I'm wondering if I need to use a function other than sapply as the following
line of code runs indefinitely (or > 30 min so far) and uses up all 16Gb of
memory on my machine for what seems like a very small dataset (data attached
in a txt file  wells.txt
<http://r.789695.n4.nabble.com/file/n4656723/wells.txt>  ).  The R code
is:

wells<-read.table("c:/temp/wells.txt",col.names=c("name","plc_hldr"))
wells2<-wells[sapply(wells[,1],function(x)length(strsplit(as.character(x),
"_")[[1]])==2),]

The 2nd line of R code above gets bogged down and takes all my RAM with it:
<http://r.789695.n4.nabble.com/file/n4656723/memory_loss.png> 

I'm simply trying to extract all of the lines of data that have a single
"_"
in the first column and place them into a dataset called "wells2".  If
that
were to work, I then want to extract the lines of data that have two
"_" and
put them into a separate dataset, say "wells3".  Is there a better way
to do
this than the one-liner above?

-Eric



--
View this message in context:
http://r.789695.n4.nabble.com/a-function-more-appropriate-than-sapply-tp4656723.html
Sent from the R help mailing list archive at Nabble.com.

Hi,
May be this helps:
?wells<-read.table("wells.txt",header=FALSE,stringsAsFactors=F)


?wells2<-wells[-grep(".*\\_.*\\_",wells[,1]),]
? head(wells2)
? # ? V1 V2
#1? w7_1? 0
#2 w11_1? 0
#3 w12_1? 0
#4 w13_1? 0
#5 w14_1? 0
#6 w15_1? 0



wellsNew<-wells[grep(".*\\_.*\\_",wells[,1]),]
?head(wellsNew)
#??????????? V1 V2
#851 99_10_4395? 0
#852 99_10_4396? 0
#853 99_10_4400? 0
#854 99_10_4403? 0
#855 99_10_4404? 0
#856 99_10_4606? 0
?nrow(wells)
#[1] 46366
nrow(wells2)
#[1] 38080
?nrow(wellsNew)
#[1] 8286
?38080+8286
#[1] 46366
A.K.



----- Original Message -----
From: emorway <emorway at usgs.gov>
To: r-help at r-project.org
Cc: 
Sent: Saturday, January 26, 2013 1:43 PM
Subject: [R] a function more appropriate than 'sapply'?

I'm wondering if I need to use a function other than sapply as the following
line of code runs indefinitely (or > 30 min so far) and uses up all 16Gb of
memory on my machine for what seems like a very small dataset (data attached
in a txt file? wells.txt
<http://r.789695.n4.nabble.com/file/n4656723/wells.txt>? ).? The R code
is:

wells<-read.table("c:/temp/wells.txt",col.names=c("name","plc_hldr"))
wells2<-wells[sapply(wells[,1],function(x)length(strsplit(as.character(x),
"_")[[1]])==2),]

The 2nd line of R code above gets bogged down and takes all my RAM with it:
<http://r.789695.n4.nabble.com/file/n4656723/memory_loss.png> 

I'm simply trying to extract all of the lines of data that have a single
"_"
in the first column and place them into a dataset called "wells2".? If
that
were to work, I then want to extract the lines of data that have two
"_" and
put them into a separate dataset, say "wells3".? Is there a better way
to do
this than the one-liner above?

-Eric



--
View this message in context:
http://r.789695.n4.nabble.com/a-function-more-appropriate-than-sapply-tp4656723.html
Sent from the R help mailing list archive at Nabble.com.

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

On 26-01-2013, at 19:43, emorway <emorway at usgs.gov> wrote:
> I'm wondering if I need to use a function other than sapply as the
following
> line of code runs indefinitely (or > 30 min so far) and uses up all 16Gb
of
> memory on my machine for what seems like a very small dataset (data
attached
> in a txt file  wells.txt
> <http://r.789695.n4.nabble.com/file/n4656723/wells.txt>  ).  The R
code is:
> 
>
wells<-read.table("c:/temp/wells.txt",col.names=c("name","plc_hldr"))
>
wells2<-wells[sapply(wells[,1],function(x)length(strsplit(as.character(x),
> "_")[[1]])==2),]
> 
> The 2nd line of R code above gets bogged down and takes all my RAM with it:
> <http://r.789695.n4.nabble.com/file/n4656723/memory_loss.png> 
> 
> I'm simply trying to extract all of the lines of data that have a
single "_"
> in the first column and place them into a dataset called
"wells2".  If that
> were to work, I then want to extract the lines of data that have two
"_" and
> put them into a separate dataset, say "wells3".  Is there a
better way to do
> this than the one-liner above?

Read your file with

	wells<-read.table("wells.txt",col.names=c("name","plc_hldr"),
stringsAsFactors=FALSE)

Remove all non underscores with

	w.sub <- gsub("[^_]+","",wells[,1])

then select elements of w.sub with 2 underscores and a single underscore with

	u.2 <- which(w.sub=="__")
	u.1 <- which(w.sub=="_")

and use u.1 and u.2 to select the appropriate rows of wells.

I tried to select rows containing 1 or 2 underscores with grep regular
expressions but that appeared to be more difficult than I had expected.
The method above is quick.

Berend