Ramiro Barrantes
2014-Dec-02 20:57 UTC
[R] How to have NLME return when convergence not reached
Hello, I am trying to fit many hundreds of simulated datasets using NLME (it's all in a big loop in R). Some don't seem to converge. I am working on addressing the issues by perhaps adjusting my simulation, or tweaking iteration steps in nlme, etc. However, when it doesn't converge, NLME just hangs, and my program either stalls for hours/days or takes over the computer memory and everything crashes eventually. Is there a way to tell nlme to stop when it doesn't seem to be converging somehow? I have been looking at the parameters in nlmeControl() but see nothing obvious. Thanks in advance, Ramiro [[alternative HTML version deleted]]
Bert Gunter
2014-Dec-02 21:30 UTC
[R] How to have NLME return when convergence not reached
?try Or ?tryCatch Bert Sent from my iPhone -- please excuse typos.> On Dec 2, 2014, at 12:57 PM, Ramiro Barrantes <ramiro at precisionbioassay.com> wrote: > > Hello, > > I am trying to fit many hundreds of simulated datasets using NLME (it's all in a big loop in R). Some don't seem to converge. I am working on addressing the issues by perhaps adjusting my simulation, or tweaking iteration steps in nlme, etc. However, when it doesn't converge, NLME just hangs, and my program either stalls for hours/days or takes over the computer memory and everything crashes eventually. Is there a way to tell nlme to stop when it doesn't seem to be converging somehow? I have been looking at the parameters in nlmeControl() but see nothing obvious. > > Thanks in advance, > > Ramiro > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
William Dunlap
2014-Dec-02 21:49 UTC
[R] How to have NLME return when convergence not reached
You could try using R.utils::withTimeout(expr, timeout=10) to cause an error when evaluating the expression expr takes longer than 10 seconds. Wrap that in tryCatch or try to catch the error and examine the outputs of lapply for ones of class "TimeoutException" to find the ones that took too long. E.g., > > system.time(z <- lapply(1:10, function(p)tryCatch(withTimeout(timeout=5, + { for(i in seq_len(10^p))log(sqrt(i)) ; p }), + error=function(e)e)) + ) user system elapsed 13.521 1.516 15.065 > sapply(z, function(zi)class(zi)[1]) [1] "integer" "integer" "integer" "integer" [5] "integer" "integer" "integer" "TimeoutException" [9] "TimeoutException" "simpleError" The first 7 iterations finished within 5 seconds, the next 2 timed out, and the last had a different error (cannot allocate 74.5 Gb vector). (R.utils's TimeoutException has an exceptionally long printout so I didn't show the entire result.) Bill Dunlap TIBCO Software wdunlap tibco.com On Tue, Dec 2, 2014 at 12:57 PM, Ramiro Barrantes < ramiro at precisionbioassay.com> wrote:> Hello, > > I am trying to fit many hundreds of simulated datasets using NLME (it's > all in a big loop in R). Some don't seem to converge. I am working on > addressing the issues by perhaps adjusting my simulation, or tweaking > iteration steps in nlme, etc. However, when it doesn't converge, NLME just > hangs, and my program either stalls for hours/days or takes over the > computer memory and everything crashes eventually. Is there a way to tell > nlme to stop when it doesn't seem to be converging somehow? I have been > looking at the parameters in nlmeControl() but see nothing obvious. > > Thanks in advance, > > Ramiro > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >[[alternative HTML version deleted]]
Ramiro Barrantes
2014-Dec-02 21:59 UTC
[R] How to have NLME return when convergence not reached
Thanks so much for your reply. I am using try but nlme never returns!! and I think the process is getting killed by the system as it is taking over all the memory. However, I do like William Dunlap's idea of using R.utils::withTimeout to limit the time. Thanks again for your help! ________________________________________ From: Bert Gunter [gunter.berton at gene.com] Sent: Tuesday, December 02, 2014 4:30 PM To: Ramiro Barrantes Cc: r-help at r-project.org Subject: Re: [R] How to have NLME return when convergence not reached ?try Or ?tryCatch Bert Sent from my iPhone -- please excuse typos.> On Dec 2, 2014, at 12:57 PM, Ramiro Barrantes <ramiro at precisionbioassay.com> wrote: > > Hello, > > I am trying to fit many hundreds of simulated datasets using NLME (it's all in a big loop in R). Some don't seem to converge. I am working on addressing the issues by perhaps adjusting my simulation, or tweaking iteration steps in nlme, etc. However, when it doesn't converge, NLME just hangs, and my program either stalls for hours/days or takes over the computer memory and everything crashes eventually. Is there a way to tell nlme to stop when it doesn't seem to be converging somehow? I have been looking at the parameters in nlmeControl() but see nothing obvious. > > Thanks in advance, > > Ramiro > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
Ramiro Barrantes
2014-Dec-02 21:59 UTC
[R] How to have NLME return when convergence not reached
Great suggestion!! Will give that a try. ________________________________ From: William Dunlap [wdunlap at tibco.com] Sent: Tuesday, December 02, 2014 4:49 PM To: Ramiro Barrantes Cc: r-help at r-project.org Subject: Re: [R] How to have NLME return when convergence not reached You could try using R.utils::withTimeout(expr, timeout=10) to cause an error when evaluating the expression expr takes longer than 10 seconds. Wrap that in tryCatch or try to catch the error and examine the outputs of lapply for ones of class "TimeoutException" to find the ones that took too long. E.g., > > system.time(z <- lapply(1:10, function(p)tryCatch(withTimeout(timeout=5, + { for(i in seq_len(10^p))log(sqrt(i)) ; p }), + error=function(e)e)) + ) user system elapsed 13.521 1.516 15.065 > sapply(z, function(zi)class(zi)[1]) [1] "integer" "integer" "integer" "integer" [5] "integer" "integer" "integer" "TimeoutException" [9] "TimeoutException" "simpleError" The first 7 iterations finished within 5 seconds, the next 2 timed out, and the last had a different error (cannot allocate 74.5 Gb vector). (R.utils's TimeoutException has an exceptionally long printout so I didn't show the entire result.) Bill Dunlap TIBCO Software wdunlap tibco.com<http://tibco.com> On Tue, Dec 2, 2014 at 12:57 PM, Ramiro Barrantes <ramiro at precisionbioassay.com<mailto:ramiro at precisionbioassay.com>> wrote: Hello, I am trying to fit many hundreds of simulated datasets using NLME (it's all in a big loop in R). Some don't seem to converge. I am working on addressing the issues by perhaps adjusting my simulation, or tweaking iteration steps in nlme, etc. However, when it doesn't converge, NLME just hangs, and my program either stalls for hours/days or takes over the computer memory and everything crashes eventually. Is there a way to tell nlme to stop when it doesn't seem to be converging somehow? I have been looking at the parameters in nlmeControl() but see nothing obvious. Thanks in advance, Ramiro [[alternative HTML version deleted]] ______________________________________________ R-help at r-project.org<mailto:R-help at r-project.org> mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]]