Please check this code :
library(parallel)
workerFunc <- function(n) { return(n^2) }
a <- function(){
CurrentTime <- Sys.time()
res <- lapply(values, workerFunc)
TimeTaken <- Sys.time() - CurrentTime
print(TimeTaken)
}
b <- function(){
CurrentTime <- Sys.time()
numWorkers <- detectCores()
res <- mclapply(values, workerFunc, mc.cores = numWorkers-2)
TimeTaken <- Sys.time() - CurrentTime
print(TimeTaken)
}
c <- function(n){
values <<- 1:n
print("Evaluating a : ")
a()
print("Evaluating b : ")
b()
}
For large values of n, the code calculates a() but the hangs indefinitely
on reaching b(). How do I correct it ?
I am using Ubuntu 14.04 and core i7 Processor
--
*AROONALOK PYNE*
BE Graduate
Department Of Computer Science And Engineering
Jadavpur University, Kolkata-32
India
[[alternative HTML version deleted]]
AROONALOK PYNE <aroonalok.pyne at gmail.com> writes:> Please check this code : > > library(parallel) > workerFunc <- function(n) { return(n^2) } > a <- function(){ > CurrentTime <- Sys.time() > res <- lapply(values, workerFunc) > TimeTaken <- Sys.time() - CurrentTime > print(TimeTaken) > } > b <- function(){ > CurrentTime <- Sys.time() > numWorkers <- detectCores() > res <- mclapply(values, workerFunc, mc.cores = numWorkers-2) > TimeTaken <- Sys.time() - CurrentTime > print(TimeTaken) > } > c <- function(n){ > values <<- 1:n > print("Evaluating a : ") > a() > print("Evaluating b : ") > b() > } > > For large values of n, the code calculates a() but the hangs indefinitely > on reaching b(). How do I correct it ? > I am using Ubuntu 14.04 and core i7 ProcessorWhich version of R and what are "large values"? I just did the following successfully: ,---- | > c(100) | [1] "Evaluating a : " | Time difference of 0.0002059937 secs | [1] "Evaluating b : " | Time difference of 0.02037406 secs | > c(1000) | [1] "Evaluating a : " | Time difference of 0.001929998 secs | [1] "Evaluating b : " | Time difference of 0.01943302 secs | > c(1000000) | [1] "Evaluating a : " | Time difference of 1.822959 secs | [1] "Evaluating b : " | Time difference of 1.46111 secs | > c(10000000) | [1] "Evaluating a : " | Time difference of 23.55863 secs | [1] "Evaluating b : " | Time difference of 15.90938 secs | > `---- Cheers, Rainer -- Rainer M. Krug email: Rainer<at>krugs<dot>de PGP: 0x0F52F982 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 480 bytes Desc: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20150611/6a686214/attachment.bin>
Please keep this on the r-help mailing list. AROONALOK PYNE <aroonalok.pyne at gmail.com> writes:> R version 3.0.2 (2013-09-25) > > > Large value : 9999999 > > > I rerun the code as c(10000000) for which your machine works fine but > my code still hangs on reaching b(). I am running it from Linux > Terminal.Might be memory allocation issue (hardware)? I have 1GB of memory. If I am not mistaken, all the data is copied into different R threads when using mclapply - so the memory requirement is much bigger. Cheers, Rainer> > > On Thu, Jun 11, 2015 at 2:15 PM, Rainer M Krug <Rainer at krugs.de> wrote: > >> AROONALOK PYNE <aroonalok.pyne at gmail.com> writes: >> >> > Please check this code : >> > >> > library(parallel) >> > workerFunc <- function(n) { return(n^2) } >> > a <- function(){ >> > CurrentTime <- Sys.time() >> > res <- lapply(values, workerFunc) >> > TimeTaken <- Sys.time() - CurrentTime >> > print(TimeTaken) >> > } >> > b <- function(){ >> > CurrentTime <- Sys.time() >> > numWorkers <- detectCores() >> > res <- mclapply(values, workerFunc, mc.cores = numWorkers-2) >> > TimeTaken <- Sys.time() - CurrentTime >> > print(TimeTaken) >> > } >> > c <- function(n){ >> > values <<- 1:n >> > print("Evaluating a : ") >> > a() >> > print("Evaluating b : ") >> > b() >> > } >> > >> > For large values of n, the code calculates a() but the hangs indefinitely >> > on reaching b(). How do I correct it ? >> > I am using Ubuntu 14.04 and core i7 Processor >> >> Which version of R and what are "large values"? >> >> I just did the following successfully: >> >> ,---- >> | > c(100) >> | [1] "Evaluating a : " >> | Time difference of 0.0002059937 secs >> | [1] "Evaluating b : " >> | Time difference of 0.02037406 secs >> | > c(1000) >> | [1] "Evaluating a : " >> | Time difference of 0.001929998 secs >> | [1] "Evaluating b : " >> | Time difference of 0.01943302 secs >> | > c(1000000) >> | [1] "Evaluating a : " >> | Time difference of 1.822959 secs >> | [1] "Evaluating b : " >> | Time difference of 1.46111 secs >> | > c(10000000) >> | [1] "Evaluating a : " >> | Time difference of 23.55863 secs >> | [1] "Evaluating b : " >> | Time difference of 15.90938 secs >> | > >> `---- >> >> Cheers, >> >> Rainer >> -- >> Rainer M. Krug >> email: Rainer<at>krugs<dot>de >> PGP: 0x0F52F982 >>-- Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) Centre of Excellence for Invasion Biology Stellenbosch University South Africa Tel : +33 - (0)9 53 10 27 44 Cell: +33 - (0)6 85 62 59 98 Fax : +33 - (0)9 58 10 27 44 Fax (D): +49 - (0)3 21 21 25 22 44 email: Rainer at krugs.de Skype: RMkrug PGP: 0x0F52F982 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 480 bytes Desc: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20150611/1fbd7586/attachment.bin>
---------- Forwarded message ---------- From: AROONALOK PYNE <aroonalok.pyne at gmail.com> Date: Thu, Jun 11, 2015 at 2:25 PM Subject: Re: Issue with mcapply To: Rainer M Krug <Rainer at krugs.de> R version 3.0.2 (2013-09-25) Large value : 9999999 I rerun the code as c(10000000) for which your machine works fine but my code still hangs on reaching b(). I am running it from Linux Terminal. On Thu, Jun 11, 2015 at 2:15 PM, Rainer M Krug <Rainer at krugs.de> wrote:> AROONALOK PYNE <aroonalok.pyne at gmail.com> writes: > > > Please check this code : > > > > library(parallel) > > workerFunc <- function(n) { return(n^2) } > > a <- function(){ > > CurrentTime <- Sys.time() > > res <- lapply(values, workerFunc) > > TimeTaken <- Sys.time() - CurrentTime > > print(TimeTaken) > > } > > b <- function(){ > > CurrentTime <- Sys.time() > > numWorkers <- detectCores() > > res <- mclapply(values, workerFunc, mc.cores = numWorkers-2) > > TimeTaken <- Sys.time() - CurrentTime > > print(TimeTaken) > > } > > c <- function(n){ > > values <<- 1:n > > print("Evaluating a : ") > > a() > > print("Evaluating b : ") > > b() > > } > > > > For large values of n, the code calculates a() but the hangs indefinitely > > on reaching b(). How do I correct it ? > > I am using Ubuntu 14.04 and core i7 Processor > > Which version of R and what are "large values"? > > I just did the following successfully: > > ,---- > | > c(100) > | [1] "Evaluating a : " > | Time difference of 0.0002059937 secs > | [1] "Evaluating b : " > | Time difference of 0.02037406 secs > | > c(1000) > | [1] "Evaluating a : " > | Time difference of 0.001929998 secs > | [1] "Evaluating b : " > | Time difference of 0.01943302 secs > | > c(1000000) > | [1] "Evaluating a : " > | Time difference of 1.822959 secs > | [1] "Evaluating b : " > | Time difference of 1.46111 secs > | > c(10000000) > | [1] "Evaluating a : " > | Time difference of 23.55863 secs > | [1] "Evaluating b : " > | Time difference of 15.90938 secs > | > > `---- > > Cheers, > > Rainer > -- > Rainer M. Krug > email: Rainer<at>krugs<dot>de > PGP: 0x0F52F982 >-- *AROONALOK PYNE* BE Graduate Department Of Computer Science And Engineering Jadavpur University, Kolkata-32 India -- *AROONALOK PYNE* BE Graduate Department Of Computer Science And Engineering Jadavpur University, Kolkata-32 India [[alternative HTML version deleted]]
---------- Forwarded message ---------- From: AROONALOK PYNE <aroonalok.pyne at gmail.com> Date: Thu, Jun 11, 2015 at 3:49 PM Subject: Re: Issue with mcapply To: Rainer M Krug <Rainer at krugs.de> Hi, My system has 4GB memory. On running htop command on linux terminal, for c(9999999), a() runnnng on single cpu core showed stats for 100% cpu utilization. On reaching b(), the entire system hanged (no response and the cpu utilization froze at 100%, RAM utilization never crossed 1.5 GB) -- *AROONALOK PYNE* BE Graduate Department Of Computer Science And Engineering Jadavpur University, Kolkata-32 India [[alternative HTML version deleted]]
Again: please keep this on r-help! AROONALOK PYNE <aroonalok.pyne at gmail.com> writes:> Hi, > > My system has 4GB memory. On running htop command on linux terminal, for > c(9999999), a() runnnng on single cpu core showed stats for 100% cpu > utilization. On reaching b(), the entire system hanged (no response and the > cpu utilization froze at 100%, RAM utilization never crossed 1.5 GB)Please read up on memory allocation in R. top (and htop) are not appropriate to see if R has memory or not as vectors need to be allocated in consecutive blocks. There have been several questions about this on this list. Cheers, Rainer -- Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) Centre of Excellence for Invasion Biology Stellenbosch University South Africa Tel : +33 - (0)9 53 10 27 44 Cell: +33 - (0)6 85 62 59 98 Fax : +33 - (0)9 58 10 27 44 Fax (D): +49 - (0)3 21 21 25 22 44 email: Rainer at krugs.de Skype: RMkrug PGP: 0x0F52F982 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 480 bytes Desc: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20150611/11b243b9/attachment.bin>
Thanks a lot Rainer On Thu, Jun 11, 2015 at 4:24 PM, Rainer M Krug <Rainer at krugs.de> wrote:> > Again: please keep this on r-help! > > AROONALOK PYNE <aroonalok.pyne at gmail.com> writes: > > > Hi, > > > > My system has 4GB memory. On running htop command on linux terminal, for > > c(9999999), a() runnnng on single cpu core showed stats for 100% cpu > > utilization. On reaching b(), the entire system hanged (no response and > the > > cpu utilization froze at 100%, RAM utilization never crossed 1.5 GB) > > Please read up on memory allocation in R. top (and htop) are not > appropriate to see if R has memory or not as vectors need to be > allocated in consecutive blocks. > > There have been several questions about this on this list. > > Cheers, > > Rainer > > -- > Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation > Biology, UCT), Dipl. Phys. (Germany) > > Centre of Excellence for Invasion Biology > Stellenbosch University > South Africa > > Tel : +33 - (0)9 53 10 27 44 > Cell: +33 - (0)6 85 62 59 98 > Fax : +33 - (0)9 58 10 27 44 > > Fax (D): +49 - (0)3 21 21 25 22 44 > > email: Rainer at krugs.de > > Skype: RMkrug > > PGP: 0x0F52F982 >-- *AROONALOK PYNE* BE Graduate Department Of Computer Science And Engineering Jadavpur University, Kolkata-32 India [[alternative HTML version deleted]]