R devel - Apr 2020 - The finalizer of the externalPtr does not work when closing R?

Hi all,

I found that the finalizer of the externalPtr is not called when R is
quitting. However, manually calling GC() works fine. This behavior is
observed on devel R 2020-04-02 r78142 on Win and R 3.6.3 on Ubuntu.  I make
a reproducible package here: https://github.com/Jiefei-Wang/example

Here is the detail of how to reproduce the problem, I create a temporary
file in the package root path and make an external pointer. The finalizer
of the external pointer will delete the  temporary file when it is called.
In the first round, I manually remove the external pointer from the global
environment and call GC() to verify if the finalizer is programmed
properly. The temporary file is deleted successfully. Then I create the
file and the pointer again and close the R session without saving the
global environment. Since the external pointer is removed when closing R,
so the finalizer should be called in this procedure. However, the temp file
still exists after closing the R session.

Here is the test code(which can be found in inst/example/example.R)

## Create a temporary file
tmpFile <- paste0(system.file(package = "testPackage"),
"/tmp")
tmpFile
file.create(tmpFile)
file.exists(tmpFile)
## Create an external pointer whose finalizer will delete
## the file when the variable is not in used
x <- testPackage:::makeExtPtr(file.remove,tmpFile)
## GC is working fine
rm(list="x")
gc()
file.exists(tmpFile)

## Create the temporary file again
file.create(tmpFile)
file.exists(tmpFile)
x <- testPackage:::makeExtPtr(file.remove,tmpFile)
## Quit R session without explicitly cleaning the working space
quit(save = "no")


##=====Open a new R session======## The temporary file still exist
tmpFile <- paste0(system.file(package = "testPackage"),
"/tmp")
file.exists(tmpFile)

Not sure if this behavior is designed on purpose, but it sounds wired to me
and can cause memory leaking if not properly handled.

Best,
Jiefei

	[[alternative HTML version deleted]]

See R_RegisterCFinalizerEx() and set onexit to nonzero. Here:
https://github.com/wch/r-source/blob/9353ddfa8d30069ad8975e0364307d710f2488d5/src/include/Rinternals.h#L1279-L1280

Gabor

On Fri, Apr 3, 2020 at 1:56 PM Wang Jiefei <szwjf08 at gmail.com>
wrote:
>
> Hi all,
>
> I found that the finalizer of the externalPtr is not called when R is
> quitting. However, manually calling GC() works fine. This behavior is
> observed on devel R 2020-04-02 r78142 on Win and R 3.6.3 on Ubuntu.  I make
> a reproducible package here: https://github.com/Jiefei-Wang/example
>
> Here is the detail of how to reproduce the problem, I create a temporary
> file in the package root path and make an external pointer. The finalizer
> of the external pointer will delete the  temporary file when it is called.
> In the first round, I manually remove the external pointer from the global
> environment and call GC() to verify if the finalizer is programmed
> properly. The temporary file is deleted successfully. Then I create the
> file and the pointer again and close the R session without saving the
> global environment. Since the external pointer is removed when closing R,
> so the finalizer should be called in this procedure. However, the temp file
> still exists after closing the R session.
>
> Here is the test code(which can be found in inst/example/example.R)
>
> ## Create a temporary file
> tmpFile <- paste0(system.file(package = "testPackage"),
"/tmp")
> tmpFile
> file.create(tmpFile)
> file.exists(tmpFile)
> ## Create an external pointer whose finalizer will delete
> ## the file when the variable is not in used
> x <- testPackage:::makeExtPtr(file.remove,tmpFile)
> ## GC is working fine
> rm(list="x")
> gc()
> file.exists(tmpFile)
>
> ## Create the temporary file again
> file.create(tmpFile)
> file.exists(tmpFile)
> x <- testPackage:::makeExtPtr(file.remove,tmpFile)
> ## Quit R session without explicitly cleaning the working space
> quit(save = "no")
>
>
> ##=====Open a new R session======> ## The temporary file still exist
> tmpFile <- paste0(system.file(package = "testPackage"),
"/tmp")
> file.exists(tmpFile)
>
> Not sure if this behavior is designed on purpose, but it sounds wired to me
> and can cause memory leaking if not properly handled.
>
> Best,
> Jiefei
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

+ 5.13 of Writing R Extensions

Tomas
On 4/3/20 3:04 PM, G?bor Cs?rdi wrote:
> See R_RegisterCFinalizerEx() and set onexit to nonzero. Here:
>
https://github.com/wch/r-source/blob/9353ddfa8d30069ad8975e0364307d710f2488d5/src/include/Rinternals.h#L1279-L1280
>
> Gabor
>
> On Fri, Apr 3, 2020 at 1:56 PM Wang Jiefei <szwjf08 at gmail.com>
wrote:
>> Hi all,
>>
>> I found that the finalizer of the externalPtr is not called when R is
>> quitting. However, manually calling GC() works fine. This behavior is
>> observed on devel R 2020-04-02 r78142 on Win and R 3.6.3 on Ubuntu.  I
make
>> a reproducible package here: https://github.com/Jiefei-Wang/example
>>
>> Here is the detail of how to reproduce the problem, I create a
temporary
>> file in the package root path and make an external pointer. The
finalizer
>> of the external pointer will delete the  temporary file when it is
called.
>> In the first round, I manually remove the external pointer from the
global
>> environment and call GC() to verify if the finalizer is programmed
>> properly. The temporary file is deleted successfully. Then I create the
>> file and the pointer again and close the R session without saving the
>> global environment. Since the external pointer is removed when closing
R,
>> so the finalizer should be called in this procedure. However, the temp
file
>> still exists after closing the R session.
>>
>> Here is the test code(which can be found in inst/example/example.R)
>>
>> ## Create a temporary file
>> tmpFile <- paste0(system.file(package = "testPackage"),
"/tmp")
>> tmpFile
>> file.create(tmpFile)
>> file.exists(tmpFile)
>> ## Create an external pointer whose finalizer will delete
>> ## the file when the variable is not in used
>> x <- testPackage:::makeExtPtr(file.remove,tmpFile)
>> ## GC is working fine
>> rm(list="x")
>> gc()
>> file.exists(tmpFile)
>>
>> ## Create the temporary file again
>> file.create(tmpFile)
>> file.exists(tmpFile)
>> x <- testPackage:::makeExtPtr(file.remove,tmpFile)
>> ## Quit R session without explicitly cleaning the working space
>> quit(save = "no")
>>
>>
>> ##=====Open a new R session======>> ## The temporary file still
exist
>> tmpFile <- paste0(system.file(package = "testPackage"),
"/tmp")
>> file.exists(tmpFile)
>>
>> Not sure if this behavior is designed on purpose, but it sounds wired
to me
>> and can cause memory leaking if not properly handled.
>>
>> Best,
>> Jiefei
>>
>>          [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

Use R_RegisterFinalizerEx in your C code. See

https://cran.r-project.org/doc/manuals/r-release/R-exts.html#External-pointers-and-weak-references

This still gives you only "best effort"; for anything stronger you
would need a different approach.

In general, finalizers should only be used as a backstop, not as a
primary resource management tool (in R or any other garbage-collected
language).

Memory leaks are not an issue -- unless you are doing very unusual
things your OS will reclaim memory resources used by your process when
it exits, cleanly or otherwise.

Best,

luke

On Fri, 3 Apr 2020, Wang Jiefei wrote:
> Hi all,
>
> I found that the finalizer of the externalPtr is not called when R is
> quitting. However, manually calling GC() works fine. This behavior is
> observed on devel R 2020-04-02 r78142 on Win and R 3.6.3 on Ubuntu.  I make
> a reproducible package here: https://github.com/Jiefei-Wang/example
>
> Here is the detail of how to reproduce the problem, I create a temporary
> file in the package root path and make an external pointer. The finalizer
> of the external pointer will delete the  temporary file when it is called.
> In the first round, I manually remove the external pointer from the global
> environment and call GC() to verify if the finalizer is programmed
> properly. The temporary file is deleted successfully. Then I create the
> file and the pointer again and close the R session without saving the
> global environment. Since the external pointer is removed when closing R,
> so the finalizer should be called in this procedure. However, the temp file
> still exists after closing the R session.
>
> Here is the test code(which can be found in inst/example/example.R)
>
> ## Create a temporary file
> tmpFile <- paste0(system.file(package = "testPackage"),
"/tmp")
> tmpFile
> file.create(tmpFile)
> file.exists(tmpFile)
> ## Create an external pointer whose finalizer will delete
> ## the file when the variable is not in used
> x <- testPackage:::makeExtPtr(file.remove,tmpFile)
> ## GC is working fine
> rm(list="x")
> gc()
> file.exists(tmpFile)
>
> ## Create the temporary file again
> file.create(tmpFile)
> file.exists(tmpFile)
> x <- testPackage:::makeExtPtr(file.remove,tmpFile)
> ## Quit R session without explicitly cleaning the working space
> quit(save = "no")
>
>
> ##=====Open a new R session======> ## The temporary file still exist
> tmpFile <- paste0(system.file(package = "testPackage"),
"/tmp")
> file.exists(tmpFile)
>
> Not sure if this behavior is designed on purpose, but it sounds wired to me
> and can cause memory leaking if not properly handled.
>
> Best,
> Jiefei
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu

Thanks for all your guys responses! It is amazing to get so many answers in
a few minutes! My package needs to handle the shared memory so the
finalizer does cause memory leaking. The Ex version should be my solution
for that. Thanks for all your help.

Best
Jiefei


On Fri, Apr 3, 2020, 9:14 PM <luke-tierney at uiowa.edu> wrote:
> Use R_RegisterFinalizerEx in your C code. See
>
>
>
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#External-pointers-and-weak-references
>
> This still gives you only "best effort"; for anything stronger
you
> would need a different approach.
>
> In general, finalizers should only be used as a backstop, not as a
> primary resource management tool (in R or any other garbage-collected
> language).
>
> Memory leaks are not an issue -- unless you are doing very unusual
> things your OS will reclaim memory resources used by your process when
> it exits, cleanly or otherwise.
>
> Best,
>
> luke
>
> On Fri, 3 Apr 2020, Wang Jiefei wrote:
>
> > Hi all,
> >
> > I found that the finalizer of the externalPtr is not called when R is
> > quitting. However, manually calling GC() works fine. This behavior is
> > observed on devel R 2020-04-02 r78142 on Win and R 3.6.3 on Ubuntu.  I
> make
> > a reproducible package here: https://github.com/Jiefei-Wang/example
> >
> > Here is the detail of how to reproduce the problem, I create a
temporary
> > file in the package root path and make an external pointer. The
finalizer
> > of the external pointer will delete the  temporary file when it is
> called.
> > In the first round, I manually remove the external pointer from the
> global
> > environment and call GC() to verify if the finalizer is programmed
> > properly. The temporary file is deleted successfully. Then I create
the
> > file and the pointer again and close the R session without saving the
> > global environment. Since the external pointer is removed when closing
R,
> > so the finalizer should be called in this procedure. However, the temp
> file
> > still exists after closing the R session.
> >
> > Here is the test code(which can be found in inst/example/example.R)
> >
> > ## Create a temporary file
> > tmpFile <- paste0(system.file(package = "testPackage"),
"/tmp")
> > tmpFile
> > file.create(tmpFile)
> > file.exists(tmpFile)
> > ## Create an external pointer whose finalizer will delete
> > ## the file when the variable is not in used
> > x <- testPackage:::makeExtPtr(file.remove,tmpFile)
> > ## GC is working fine
> > rm(list="x")
> > gc()
> > file.exists(tmpFile)
> >
> > ## Create the temporary file again
> > file.create(tmpFile)
> > file.exists(tmpFile)
> > x <- testPackage:::makeExtPtr(file.remove,tmpFile)
> > ## Quit R session without explicitly cleaning the working space
> > quit(save = "no")
> >
> >
> > ##=====Open a new R session======> > ## The temporary file still
exist
> > tmpFile <- paste0(system.file(package = "testPackage"),
"/tmp")
> > file.exists(tmpFile)
> >
> > Not sure if this behavior is designed on purpose, but it sounds wired
to
> me
> > and can cause memory leaking if not properly handled.
> >
> > Best,
> > Jiefei
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
> --
> Luke Tierney
> Ralph E. Wareham Professor of Mathematical Sciences
> University of Iowa                  Phone:             319-335-3386
> Department of Statistics and        Fax:               319-335-3017
>     Actuarial Science
> 241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
> Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu
>
	[[alternative HTML version deleted]]