Jan Gorecki
2020-Nov-21 17:51 UTC
[Rd] .Internal(quit(...)): system call failed: Cannot allocate memory
Dear R-developers, Some of the more fat scripts (50+ GB mem used by R) that I am running, when they finish they do quit with q("no", status=0) Quite often it happens that there is an extra stderr output produced at the very end which looks like this: Warning message: In .Internal(quit(save, status, runLast)) : system call failed: Cannot allocate memory Is there any way to avoid this kind of warnings? I am using stderr output for detecting failures in scripts and this warning is a false positive of a failure. Maybe quit function could wait little bit longer trying to allocate before it raises this warning? Best regards, Jan Gorecki
Duncan Murdoch
2020-Nov-21 19:40 UTC
[Rd] .Internal(quit(...)): system call failed: Cannot allocate memory
On 21/11/2020 12:51 p.m., Jan Gorecki wrote:> Dear R-developers, > > Some of the more fat scripts (50+ GB mem used by R) that I am running, > when they finish they do quit with q("no", status=0) > Quite often it happens that there is an extra stderr output produced > at the very end which looks like this: > > Warning message: > In .Internal(quit(save, status, runLast)) : > system call failed: Cannot allocate memory > > Is there any way to avoid this kind of warnings? I am using stderr > output for detecting failures in scripts and this warning is a false > positive of a failure. > > Maybe quit function could wait little bit longer trying to allocate > before it raises this warning?I don't know what waiting would accomplish. Generally speaking the allocation functions in R will try garbage collection before failing, so it looks like you are in a situation where there really is no memory available. (I think code can prevent gc; maybe your code is doing that and not re-enabling it?) Having a reproducible example would help, but I imagine it's not easy to put one together. Duncan Murdoch
Tomas Kalibera
2020-Nov-23 11:14 UTC
[Rd] .Internal(quit(...)): system call failed: Cannot allocate memory
On 11/21/20 6:51 PM, Jan Gorecki wrote:> Dear R-developers, > > Some of the more fat scripts (50+ GB mem used by R) that I am running, > when they finish they do quit with q("no", status=0) > Quite often it happens that there is an extra stderr output produced > at the very end which looks like this: > > Warning message: > In .Internal(quit(save, status, runLast)) : > system call failed: Cannot allocate memory > > Is there any way to avoid this kind of warnings? I am using stderr > output for detecting failures in scripts and this warning is a false > positive of a failure. > > Maybe quit function could wait little bit longer trying to allocate > before it raises this warning?If you see this warning, some call to system() or system2() or similar, which executes an external program, failed to even run a shell to run that external program, because there was not enough memory. You should be able to find out where it happens by checking the exit status of system(). Tomas> > Best regards, > Jan Gorecki > > ______________________________________________ > R-devel at r-project.org mailing list > stat.ethz.ch/mailman/listinfo/r-devel
Gregory Warnes
2020-Nov-23 15:04 UTC
[Rd] .Internal(quit(...)): system call failed: Cannot allocate memory
Try explicitly deleting large data objects by calling `rm`, then `gc`. On Mon, Nov 23, 2020 at 6:15 AM Tomas Kalibera <tomas.kalibera at gmail.com> wrote:> On 11/21/20 6:51 PM, Jan Gorecki wrote: > > Dear R-developers, > > > > Some of the more fat scripts (50+ GB mem used by R) that I am running, > > when they finish they do quit with q("no", status=0) > > Quite often it happens that there is an extra stderr output produced > > at the very end which looks like this: > > > > Warning message: > > In .Internal(quit(save, status, runLast)) : > > system call failed: Cannot allocate memory > > > > Is there any way to avoid this kind of warnings? I am using stderr > > output for detecting failures in scripts and this warning is a false > > positive of a failure. > > > > Maybe quit function could wait little bit longer trying to allocate > > before it raises this warning? > > If you see this warning, some call to system() or system2() or similar, > which executes an external program, failed to even run a shell to run > that external program, because there was not enough memory. You should > be able to find out where it happens by checking the exit status of > system(). > > Tomas > > > > > > Best regards, > > Jan Gorecki > > > > ______________________________________________ > > R-devel at r-project.org mailing list > > stat.ethz.ch/mailman/listinfo/r-devel > > ______________________________________________ > R-devel at r-project.org mailing list > stat.ethz.ch/mailman/listinfo/r-devel >-- "Whereas true religion and good morals are the only solid foundations of public liberty and happiness . . . it is hereby earnestly recommended to the several States to take the most effectual measures for the encouragement thereof." Continental Congress, 1778 [[alternative HTML version deleted]]
Bill Dunlap
2020-Nov-23 17:10 UTC
[Rd] .Internal(quit(...)): system call failed: Cannot allocate memory
The call to system() probably is an internal call used to delete the session's tempdir(). This sort of failure means that a potentially large amount of disk space is not being recovered when R is done. Perhaps R_CleanTempDir() could call R_unlink() instead of having a subprocess call 'rm -rf ...'. Then it could also issue a specific warning if it was impossible to delete all of tempdir(). (That should be very rare.)> q("no")Breakpoint 1, R_system (command=command at entry=0x7fffffffa1e0 "rm -Rf /tmp/RtmppoKPXb") at sysutils.c:311 311 { (gdb) where #0 R_system (command=command at entry=0x7fffffffa1e0 "rm -Rf /tmp/RtmppoKPXb") at sysutils.c:311 #1 0x00005555557c30ec in R_CleanTempDir () at sys-std.c:1178 #2 0x00005555557c31d7 in Rstd_CleanUp (saveact=<optimized out>, status=0, runLast=<optimized out>) at sys-std.c:1243 #3 0x00005555557c593d in R_CleanUp (saveact=saveact at entry=SA_NOSAVE, status=status at entry=0, runLast=<optimized out>) at system.c:87 #4 0x00005555556cc85e in do_quit (call=<optimized out>, op=<optimized out>, args=0x555557813f90, rho=<optimized out>) at main.c:1393 -Bill On Mon, Nov 23, 2020 at 3:15 AM Tomas Kalibera <tomas.kalibera at gmail.com> wrote:> On 11/21/20 6:51 PM, Jan Gorecki wrote: > > Dear R-developers, > > > > Some of the more fat scripts (50+ GB mem used by R) that I am running, > > when they finish they do quit with q("no", status=0) > > Quite often it happens that there is an extra stderr output produced > > at the very end which looks like this: > > > > Warning message: > > In .Internal(quit(save, status, runLast)) : > > system call failed: Cannot allocate memory > > > > Is there any way to avoid this kind of warnings? I am using stderr > > output for detecting failures in scripts and this warning is a false > > positive of a failure. > > > > Maybe quit function could wait little bit longer trying to allocate > > before it raises this warning? > > If you see this warning, some call to system() or system2() or similar, > which executes an external program, failed to even run a shell to run > that external program, because there was not enough memory. You should > be able to find out where it happens by checking the exit status of > system(). > > Tomas > > > > > > Best regards, > > Jan Gorecki > > > > ______________________________________________ > > R-devel at r-project.org mailing list > > stat.ethz.ch/mailman/listinfo/r-devel > > ______________________________________________ > R-devel at r-project.org mailing list > stat.ethz.ch/mailman/listinfo/r-devel >[[alternative HTML version deleted]]
Apparently Analagous Threads
- .Internal(quit(...)): system call failed: Cannot allocate memory
- .Internal(quit(...)): system call failed: Cannot allocate memory
- [External] Re: .Internal(quit(...)): system call failed: Cannot allocate memory
- .Internal(quit(...)): system call failed: Cannot allocate memory
- .Internal(quit(...)): system call failed: Cannot allocate memory