search for: shian

Yes I am running on Rstudio 1.2.5033. I was also running this code without error on Ubuntu in Rstudio. Checking again on the terminal and it does indeed work fine even with large data.frames. Any idea as to what interaction between Rstudio and mclapply causes this? Thanks, Shian On 28 Apr 2020, at 7:29 pm, Simon Urbanek <simon.urbanek at R-project.org<mailto:simon.urbanek at R-project.org>> wrote: Sorry, the code works perfectly fine for me in R even for 1e6 observations (but I was testing with R 4.0.0). Are you using some kind of GUI? Cheers, Simon On 28/...

...ms like that it?s quite intrinsic to how forks interact with RStudio. Given this code is eventually going to be a part of a package, should I expect it to fail mysteriously in RStudio for my users? Is the best solution here to migrate all my parallelism to PSOCK for the foreseeable future? Thanks, Shian > On 29 Apr 2020, at 2:08 am, Henrik Bengtsson <henrik.bengtsson at gmail.com> wrote: > > Hi, a few comments below. > > First, from my experience and troubleshooting similar reports from > others, a returned NULL from parallel::mclapply() is often because the > correspon...

...= df, + mc.cores = 2L + ) [[1]] NULL [[2]] NULL There is no error message returned, and the code runs perfectly fine in lapply. I am on a MacBook 15 (2016) running MacOS 10.14.6 (Mojave) and R version 3.6.2. This bug could not be reproduced on my Ubuntu 19.10 running R 3.6.1. Kind regards, Shian Su ---- Shian Su PhD Student, Ritchie Lab 6W, Epigenetics and Development Walter & Eliza Hall Institute of Medical Research 1G Royal Parade, Parkville VIC 3052, Australia _______________________________________________ The information in this email is confidential and intend...{{dropped:15}}

...lelism. I?m already running into trouble with the fact that PSOCK doesn?t seem to retain loaded packages in spawned processes. I would love to know if there reliable options in R that allow a similar interface to mclapply but use a different and more RStudio-stable mode of parallelisation? Thanks, Shian > On 29 Apr 2020, at 1:33 pm, Simon Urbanek <simon.urbanek at R-project.org> wrote: > > Do NOT use mcparallel() in packages except as a non-default option that user can set for the reasons Henrik explained. Multicore is intended for HPC applications that need to use many cores for c...

...ou need to share a lot of data for small results. If the data is small, or you need to read it first, then other methods like PSOCK may be preferable. In any case, parallelization only makes sense for code that you know will take a long time to run. Cheers, Simon > On 29/04/2020, at 11:54 AM, Shian Su <su.s at wehi.edu.au> wrote: > > Thanks Henrik, > > That clears things up significantly. I did see the warning but failed to include it my initial email. It sounds like an RStudio issue, and it seems like that it?s quite intrinsic to how forks interact with RStudio. Given thi...

On Tue, Apr 28, 2020 at 9:00 PM Shian Su <su.s at wehi.edu.au> wrote: > > Thanks Simon, > > I will take note of the sensible default for core usage. I?m trying to achieve small scale parallelism, where tasks take 1-5 seconds and make fuller use of consumer hardware. Its not a HPC-worthy computation but even laptops th...

...me package code that you use directly or indirectly. You could even use stop() here if you wanna be conservative. [0] https://github.com/rstudio/rstudio/issues/2597#issuecomment-482187011 [1] https://stat.ethz.ch/pipermail/r-devel/2020-January/078896.html /Henrik On Tue, Apr 28, 2020 at 2:39 AM Shian Su <su.s at wehi.edu.au> wrote: > > Yes I am running on Rstudio 1.2.5033. I was also running this code without error on Ubuntu in Rstudio. Checking again on the terminal and it does indeed work fine even with large data.frames. > > Any idea as to what interaction between Rstudio a...

Sorry, the code works perfectly fine for me in R even for 1e6 observations (but I was testing with R 4.0.0). Are you using some kind of GUI? Cheers, Simon > On 28/04/2020, at 8:11 PM, Shian Su <su.s at wehi.edu.au> wrote: > > Dear R-devel, > > I am experiencing issues with running GAM models using mclapply, it fails to return any values if the data input becomes large. For example here the code runs fine with a df of 100 rows, but fails at 1000. > > library(...