Displaying 20 results from an estimated 1000 matches similar to: "my function does not work for large data set"
2010 May 26
1
how to Store loop output from a function
HI, Dear R community,
I am writing the following function to create one data set(*tree.pred*) and
one vector(*valid.out*) from loops. Later, I want to use the data set from
this loop to plot curves. I have tried return, list, but I can not use the
*tree.pred* data and *valid.out* vector.
auc.tree<- function(msplit,mbucket) {
* tree.pred<-data.frame()
2010 Sep 07
1
change the for loops with lapply
cv.fold<-function(i, size=3, rang=0.3){
cat('Fold ', i, '\n')
out.fold.c <-((i-1)*c.each.part +1):(i*c.each.part)
out.fold.n <-((i-1)*n.each.part +1):(i*n.each.part)
train.cv <- n.cc[-out.fold.c, c(2:2401, 2417)]
train.nv <- n.nn[-out.fold.n, c(2:2401, 2417)]
train.v<-rbind(train.cv, train.nv) #training data for feature
2011 Jun 22
1
question about read.columns
HI, Dear R community,
I have a large data set names dd.txt, the columns are: there are 2402
variables.
a1, b1, ..z1, a11, b11, ...z11, a111, b111, ..z111..
IF I dont know the relative position of the columns, but I know I need the
following variables:
var<-c(a1, c1,a11,b11,f111)
Can I use read.columns to read the data into R?
I have tried the following codes, but it does not work
2010 May 12
1
exact the variables used in tree construction
> fit.dimer <- rpart(as.factor(out) ~ ., method="class", data=p_df)
>
> fit.dimer$frame[, "var"]
[1] NE WC <leaf> TA <leaf> <leaf> WG WD WW WC
[11] <leaf> <leaf> <leaf> CT <leaf> FC <leaf> YG QT <leaf>
[21] <leaf> <leaf> NW DP DY <leaf> SK
2010 Apr 06
2
help output figures in R
somfunc<- function (file) {
aa_som<-scale(file)
final.som<-som(data=aa_som, rlen=10000, grid=somgrid(5,4, "hexagonal"))
pdf(file="/home/cdu/changbin/file.pdf") #output graphic file.
plot(final.som, main="Unsupervised SOM")
dev.off()
}
I have many different files, if I want output pdf file with the same name
as for each dataset I feed to the function
2010 May 18
2
get the row sums
> head(en.id.pr)
valid.gene_id b.pred rf.pred svm.pred
1521 2500151211 0 0 0
366 639679745 0 0 0
1965 2502081603 1 1 1
1420 644148030 1 1 1
1565 2500626489 1 1 1
1816 2501711016 1 1 1
> p.pred <- data.frame(en.id.pr, sum=apply(en.id.pr[,2:4], 1, sum)) #
2010 Apr 21
2
?rpart
HI, Dear R community,
Last friday, I used the codes, it works, but today, it does not run?
> fit.dimer <- rpart(outcome ~., method="class", data=p.df)
Error in `[.data.frame`(frame, predictors) : undefined columns selected
DOEs anyone have comments or suggestions? Thanks in advance!
--
Sincerely,
Changbin
--
[[alternative HTML version deleted]]
2010 Jun 15
1
output from the gbm package
HI, Dear Greg and R community,
I have one question about the output of gbm package. the output of Boosting
should be f(x), from it , how to calculate the probability for each
observations in data set?
SInce it is stochastic, how can guarantee that each observation in training
data are selected at least once? IF SOME obs are not selected, how to
calculate the training error?
Thanks?
--
2010 May 05
2
probabilities in svm output in e1071 package
svm.fit<-svm(as.factor(out) ~ ., data=all_h, method="C-classification",
kernel="radial", cost=bestc, gamma=bestg, cross=10) # model fitting
svm.pred<-predict(svm.fit, hh, decision.values = TRUE, probability = TRUE) #
find the probability, but can not find.
attr(svm.pred, "probabilities")
> attr(svm.pred, "probabilities")
1 0
1 0 0
2 0
2010 May 11
1
how to extract the variables used in decision tree
HI, Dear R community,
How to extract the variables actually used in tree construction? I want to
extract these variables and combine other variable as my features in next
step model building.
> printcp(fit.dimer)
Classification tree:
rpart(formula = outcome ~ ., data = p_df, method = "class")
Variables actually used in tree construction:
[1] CT DP DY FC NE NW QT SK TA WC WD WG WW
2010 May 25
4
R eat my data
HI, Dear R community,
My original file has 1932 lines, but when I read into R, it changed to 1068
lines, how comes?
cdu@nuuk:~/operon$ wc -l id_name_gh5.txt
1932 id_name_gh5.txt
> gene_name<-read.table("/home/cdu/operon/id_name_gh5.txt", sep="\t",
skip=0, header=F, fill=T)
> dim(gene_name)
[1] 1068 3
--
Sincerely,
Changbin
--
Changbin Du
DOE Joint Genome
2010 Apr 07
2
help in attach function
Hi, r-community,
This morning, I MET the following problem several times when I try to attach
the data set.
When I closed the current console and reopen the R console, the problem
disappear. BUt with the time passed on, the problem occurs again.
Can anyone help me with this?
> attach(total)
The following object(s) are masked from total ( position 3 ) :
acid base cell_evalue
2010 Apr 19
0
help in output file
HI, Dear R-community,
I AM using the following codes to grow tree and plot tree:
# Classification Tree with rpart
library(rpart)
pdf(file="/home/cdu/changbin/dimer_tree.pdf")
# grow tree
fit.dimer <- rpart(outcome ~ ., method="class", data=p.dimer[,2:402])
plotcp(fit.dimer) # visualize cross-validation results
# plot tree
plot(fit.dimer, uniform=TRUE,
2010 Apr 30
0
ROC curve in randomForest
require(randomForest)
rf.pred<-predict(fit, valid, type="prob")
> rf.pred[1:20, ]
0 1
16 0.0000 1.0000
23 0.3158 0.6842
43 0.3030 0.6970
52 0.0886 0.9114
55 0.1216 0.8784
75 0.0920 0.9080
82 0.4332 0.5668
120 0.2302 0.7698
128 0.1336 0.8664
147 0.4272 0.5728
148 0.0490 0.9510
153 0.0556 0.9444
161 0.0760 0.9240
162 0.4564 0.5436
172 0.5148 0.4852
176 0.1730
2010 Apr 15
2
r-loop
HI, Dear community,
I am building the following loop,
ww<-function(file) {
lossw<-vector()
for (x in seq(0.1, 0.9, by=0.1)) {
cat('xweight ', x, '\n')
lossw[i] <- cross.validation(file, x)$avg
}
return(lossw) }
MY question is how to index the lossw[i]?
for (i in 1:9)
for (x in seq(0.1, 0.9, by=0.1))
Thanks so much!
2010 Oct 12
1
need help with nnet
HI, Dear R community,
My data set has 2409 variables, the last one is response variable. I have
used the nnet after feature selection and works. But this time, I am using
nnet to fit a model without feature selection. I got the following error
information:
> dim(train)
[1] 1827 2409
nnet.fit<-nnet(as.factor(out) ~ ., data=train, size=3, rang=0.3,
decay=5e-4, maxit=500) # model
2010 May 05
3
sort the data set by one variable
> #sort the data by predicted probability
> b.order<-bo.id.pred[(order(-predict)),]
> b.order[1:20,]
gene_id predict
43 637882902 0.07823997
53 638101634 0.66256490
61 639084581 0.08587504
41 637832824 0.02461066
25 637261662 0.11613879
22 637240022 0.06350477
62 639084582 0.02238538
63 639097718 0.06792841
44 637943079 0.04532625
80 640158389 0.06582658
3 637006517 0.57648451
2010 Apr 26
3
R.GBM package
HI, Dear Greg,
I AM A NEW to GBM package. Can boosting decision tree be implemented in
'gbm' package? Or 'gbm' can only be used for regression?
IF can, DO I need to combine the rpart and gbm command?
Thanks so much!
--
Sincerely,
Changbin
--
[[alternative HTML version deleted]]
2010 Nov 04
4
how to work with long vectors
HI, Dear R community,
I have one data set like this, What I want to do is to calculate the
cumulative coverage. The following codes works for small data set (#rows =
100), but when feed the whole data set, it still running after 24 hours.
Can someone give some suggestions for long vector?
id reads
Contig79:1 4
Contig79:2 8
Contig79:3 13
Contig79:4 14
Contig79:5 17
2011 Jan 20
2
auc function
Hi, there.
Suppose I already have sensitivities and specificities. What is the quick R-function to calculate AUC for the ROC plot? There seem to be many R functions to calculate AUC.
Thanks.
Yulei
[[alternative HTML version deleted]]