similar to: PSOCK cluster and renice

Hi Henrik, Thanks for the detailed in fast reply! My guess would be that the confusion comes from the different use of nice and renice. The workraund you provided work fine! Thanks a lot. Best, Andreas Henrik Bengtsson <henrik.bengtsson at gmail.com> writes: > Looks like a bug to me due to wrong assumptions about 'nice' > arguments, but could be because a

Looks like a bug to me due to wrong assumptions about 'nice' arguments, but could be because a "non-standard" 'nice' is used. If we do: > trace(system, tracer = quote(print(command))) Tracing function "system" in package "base" we see that the system call used is: > cl <- parallel::makePSOCKcluster(2L, renice = 19) Tracing system(cmd, wait

(Bcc: CRAN) This is a proposal helping CRAN and alike as well as individual developers to avoid stray R processes being left behind that might be produced when an example or a package test fails to set up a parallel::makeCluster(). ISSUE If a package test sets up a PSOCK cluster and then the master process dies for one reason or the other, the PSOCK worker processes will remain running for 30

BACKGROUND: While troubleshooting random, occasionally occurring, errors from parallel::makePSOCKcluster("localhost", port = 11000); Error in socketConnection("localhost", port = port, server = TRUE, blocking = TRUE, : cannot open the connection I had another look at parallel:::newPSOCKnode(), which is used internally to set up each background worker. It is designed to,

A solution is to have parallel:::.slaveRSOCK() attempt to connect multiple times before failing, e.g. makeSOCKmaster <- function(master, port, timeout, useXDR, maxTries = 10L, interval = 1.0) { port <- as.integer(port) for (i in seq_len(maxTries)) { con <- tryCatch({ socketConnection(master, port = port, blocking = TRUE,

Hi all, The call parallel::makeCluster(1L) hangs infinitely on my MacOS machine which seems to be already reported by some people (e.g., https://stat.ethz.ch/pipermail/r-devel/2018-February/075565.html). However, the solutions posted on SO, GH or R-devel do not work in my case. So far, I unsuccessfully tested ? 1. Couple of reboots 2. Adding 192.0.0.1 to /etc/hosts 3. Using R.app

On Mon, 31 Oct 2016, Henrik Bengtsson wrote: > Thank you for looking into this Luke. > > On Thu, Oct 27, 2016 at 9:26 AM, <luke-tierney at uiowa.edu> wrote: >> On unix, unless event polling is enabled Sys.sleep just waits in a >> select() call (with a SIGINT handler in place) so the elapsed time >> isn't checked until after the select call is complete.

Great. For the record of this thread, I've submitted patch PR17391 (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17391). I've patched it against the latest R-devel on the SVN, passes 'make check-all', and I've verified it works with the above tests. /Henrik On Fri, Mar 9, 2018 at 4:37 AM, <luke-tierney at uiowa.edu> wrote: > I'm happy to look at a

Hi Thomas, thanks for your reply (and thanks for your patience...). I am now using the following minimal reprex: > library(parallel) > cl <- makeCluster(2L) I freshly started the machine and did not open any other app. Just R.app (3.6.1). After executing the second line of code, R seems to hang infinitely and does not respond. The R process itself uses almost no CPU. Unfortunately,

The problem causing the stray worker processes when the master fails to open a server socket to listen to connections from workers is not related to timeout in socketConnection(), because socketConnection() will fail right away. It is caused by a bug in checking the setup timeout (PR 17391). Fixed in 76275. Best Tomas On 3/18/19 2:23 AM, Henrik Bengtsson wrote: > (Bcc: CRAN) > >

Dear all, I?m working as an administrator of a High-Performance Computing (HPC) Cluster which runs on Linux. A lot of people are using R on this Linux cluster and, of course, the *parallel* package to speed up their computations. It has been our collective experience, that using |makeForkCluster| yields an overall better experience /on Linux/ than the |makePSOCKcluster|, for whatever definition

Hi, for a job of lesser priority I would like to connect to a Samba server. When I'm doing a 'smbmount', enter the server by SSH, pgrep for the new smbd process and then renice it I have exactly the behaviour I desire. Now I would like to automate this and yet start the server process with lower priority. Connections to other clients shall not be touched at all. Are there any

Hi all, I can't get the functionality of the package parallel to work. Specifically, makeCluster() hangs when I run it. I first noticed the problem when trying to run Rstan with multiple cores and the traced it back to the core package parallel. The following results in R hanging after the call to makeCluster. library(parallel) # Calculate the number of cores no_cores <-

I'm exploring latency overhead of parallel PSOCK workers and noticed that serializing/unserializing data back to the main R session is significantly slower on Linux than it is on Windows/MacOS with similar hardware. Is there a reason for this difference and is there a way to avoid the apparent additional Linux overhead? I attempted to isolate the behavior with a test that simply returns

I have 4 gig in my * box. I'm tuning for performance and I'd like to ask opinions: 1. asterisk -p == renice -20 ?? 2. I've turned off swap with no apparent ill effects. Can anyone commment on long term effects with moderate load (say, 30 SIP phones / 2-3K calls /day) 3. Can anyone comment on using ramdisk as swap and whether this is a good idea or bad idea? I'm using 2.6

Dear Henrik, thank you, for the quick reply. Bizarrely enough, the problem vanished when I woke the computer from sleep (I had previously replicated the problem after several restarts of both R and the MacOS). I will follow-up if I can again replicate the problem. Florian On 2/10/18 4:39 PM, Henrik Bengtsson wrote: > A few quick comments: > > * You mention R --vanilla, but make sure

On Mon, 2 Nov 2020 at 02:22, Simon Urbanek <simon.urbanek at r-project.org> wrote: > > It looks like R sockets on Linux could do with TCP_NODELAY -- without (status quo): How many network packets are generated with and without it? If there are many small writes and thus setting TCP_NODELAY causes many small packets to be sent, it might make more sense to set TCP_QUICKACK instead.

I'm not sure the user would know ;). This is very system-specific issue just because the Linux network stack behaves so differently from other OSes (for purely historical reasons). That makes it hard to abstract as a "feature" for the R sockets that are supposed to be platform-independent. At least TCP_NODELAY is actually part of POSIX so it is on better footing, and disabling

FWIW, there are indeed a few low hanging bug fixes in 'parallelly' that should be easy to incorporate into 'parallel' without adding extra maintenance. For example, in parallel::makePSOCKcluster(), it is not possible to disable SSH option '-l USER' so that it can be set in ~/.ssh/config. The remote user name will be the user name of your local machine and if you try to

This is the specific error messsage from R CMD check --as-cran Error in .check_ncores(length(names)) : 16 simultaneous processes spawned Calls: prepost -> makeCluster -> makePSOCKcluster -> .check_ncores Execution halted Thanks, Ravi ________________________________ From: Ravi Varadhan Sent: Saturday, July 1, 2023 1:15 PM To: R-Help <r-help at r-project.org> Subject: Number