Dear fellow R coders,
I am observing differences in results obtained using glmer when using a Mac or
Linux computer versus a PC. Specifically, I am talking about a relatively
complex glmer model with a nested random effects structure. The model is set up
in the following way:
gcctrl = glmerControl(optimizer=c('nloptwrap'), optCtrl = list(maxfun =
500000), calc.derivs = FALSE)
glmer_pre_instr1 = glmer(
formula = cbind(FREQ, NSAMP-FREQ) ~ FDIST_minz + poly(RFREQ,2) + ROI + (1
+ FDIST_minz + RFREQ + ROI|ID/COL),
data = cdf_pre_instr,
family = binomial,
control = gcctrl)
Code and data of an example for which I find reproducible, non-negligible
differences between Mac/Win can be found here:
https://gitlab.com/nschuck/glmer_sandbox/tree/master
<https://gitlab.com/nschuck/glmer_sandbox/tree/master>
The differences between the fitted models seem to be most pronounced regarding
the estimated correlation structure of the random effects terms. Mac and Linux
yield very similar results, but Windows deviates quite a bit in some cases. This
has a large impact on p values obtained when performing model comparisons. I
have tried this on Mac OS 10.14, Windows 10 and Ubuntu and Debian. All systems I
have tried are using lme 1.1.21 and R 3.5+.
Does anyone have an idea what the underlying cause might be?
Thanks,
Nico
[[alternative HTML version deleted]]
The BLAS in use on each? Bert Gunter "The trouble with having an open mind is that people keep coming along and sticking things into it." -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip ) On Thu, May 30, 2019 at 5:27 AM Nicolas Schuck <nico.schuck at gmail.com> wrote:> Dear fellow R coders, > > I am observing differences in results obtained using glmer when using a > Mac or Linux computer versus a PC. Specifically, I am talking about a > relatively complex glmer model with a nested random effects structure. The > model is set up in the following way: > gcctrl = glmerControl(optimizer=c('nloptwrap'), optCtrl = list(maxfun > 500000), calc.derivs = FALSE) > > glmer_pre_instr1 = glmer( > formula = cbind(FREQ, NSAMP-FREQ) ~ FDIST_minz + poly(RFREQ,2) + ROI > + (1 + FDIST_minz + RFREQ + ROI|ID/COL), > data = cdf_pre_instr, > family = binomial, > control = gcctrl) > > Code and data of an example for which I find reproducible, non-negligible > differences between Mac/Win can be found here: > https://gitlab.com/nschuck/glmer_sandbox/tree/master < > https://gitlab.com/nschuck/glmer_sandbox/tree/master> > The differences between the fitted models seem to be most pronounced > regarding the estimated correlation structure of the random effects terms. > Mac and Linux yield very similar results, but Windows deviates quite a bit > in some cases. This has a large impact on p values obtained when performing > model comparisons. I have tried this on Mac OS 10.14, Windows 10 and Ubuntu > and Debian. All systems I have tried are using lme 1.1.21 and R 3.5+. > > Does anyone have an idea what the underlying cause might be? > > Thanks, > Nico > > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >[[alternative HTML version deleted]]
Unless there us good reason not to, always cc the list. I have done so here. The R Installation manual has some info on how to use different BLASes I believe, but someone with expertise (I have none) needs to respond to your queries. On Thu, May 30, 2019 at 7:50 AM Nicolas Schuck <nico.schuck at gmail.com> wrote:> I know that it is in use on the Mac, see sessionInfo below. I have to > check on the Win system. Why would that make such a difference and how > could I make the Win get the same results as the Unix Systems? > > R version 3.6.0 (2019-04-26) > > Platform: x86_64-apple-darwin15.6.0 (64-bit) > > Running under: macOS Mojave 10.14.5 > > Matrix products: default > > BLAS: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRblas.0.dylib > > LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib > > Random number generation: > > RNG: Mersenne-Twister > > Normal: Inversion Sample: Rounding > > locale:[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 > > attached base packages:[1] stats graphics grDevices utils datasets methods base > > Thanks, Nico > > On 30. May 2019, at 16:34, Bert Gunter <bgunter.4567 at gmail.com> wrote: > > The BLAS in use on each? > > Bert Gunter > > "The trouble with having an open mind is that people keep coming along and > sticking things into it." > -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip ) > > > On Thu, May 30, 2019 at 5:27 AM Nicolas Schuck <nico.schuck at gmail.com> > wrote: > >> Dear fellow R coders, >> >> I am observing differences in results obtained using glmer when using a >> Mac or Linux computer versus a PC. Specifically, I am talking about a >> relatively complex glmer model with a nested random effects structure. The >> model is set up in the following way: >> gcctrl = glmerControl(optimizer=c('nloptwrap'), optCtrl = list(maxfun >> 500000), calc.derivs = FALSE) >> >> glmer_pre_instr1 = glmer( >> formula = cbind(FREQ, NSAMP-FREQ) ~ FDIST_minz + poly(RFREQ,2) + >> ROI + (1 + FDIST_minz + RFREQ + ROI|ID/COL), >> data = cdf_pre_instr, >> family = binomial, >> control = gcctrl) >> >> Code and data of an example for which I find reproducible, non-negligible >> differences between Mac/Win can be found here: >> https://gitlab.com/nschuck/glmer_sandbox/tree/master < >> https://gitlab.com/nschuck/glmer_sandbox/tree/master> >> The differences between the fitted models seem to be most pronounced >> regarding the estimated correlation structure of the random effects terms. >> Mac and Linux yield very similar results, but Windows deviates quite a bit >> in some cases. This has a large impact on p values obtained when performing >> model comparisons. I have tried this on Mac OS 10.14, Windows 10 and Ubuntu >> and Debian. All systems I have tried are using lme 1.1.21 and R 3.5+. >> >> Does anyone have an idea what the underlying cause might be? >> >> Thanks, >> Nico >> >> >> >> >> [[alternative HTML version deleted]] >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> >[[alternative HTML version deleted]]
bert: you are right, sorry for not cc-ing the list. thanks also for the hint. I wanted to bring this up here again, emphasising that we do find in at least one case *a very large difference* in the p value, using the same scripts and data on a windows versus mac machine (see reproducible example in the gitlab link posted below). I have now come across several instances in which results of (g)lmer models don?t agree on windows vs unix-based machines, which I find a bit disturbing. any ideas where non-negligible differences could come from? thanks, nico> On 30. May 2019, at 16:58, Bert Gunter <bgunter.4567 at gmail.com> wrote: > > > Unless there us good reason not to, always cc the list. I have done so here. > > The R Installation manual has some info on how to use different BLASes I believe, but someone with expertise (I have none) needs to respond to your queries. > > On Thu, May 30, 2019 at 7:50 AM Nicolas Schuck <nico.schuck at gmail.com <mailto:nico.schuck at gmail.com>> wrote: > I know that it is in use on the Mac, see sessionInfo below. I have to check on the Win system. Why would that make such a difference and how could I make the Win get the same results as the Unix Systems? > > R version 3.6.0 (2019-04-26) > Platform: x86_64-apple-darwin15.6.0 (64-bit) > Running under: macOS Mojave 10.14.5 > Matrix products: default > BLAS: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRblas.0.dylib > LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib > Random number generation: > RNG: Mersenne-Twister > Normal: Inversion Sample: Rounding > locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 > attached base packages: [1] stats graphics grDevices utils datasets methods base > Thanks, Nico > On 30. May 2019, at 16:34, Bert Gunter <bgunter.4567 at gmail.com <mailto:bgunter.4567 at gmail.com>> wrote: > >> The BLAS in use on each? >> >> Bert Gunter >> >> "The trouble with having an open mind is that people keep coming along and sticking things into it." >> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip ) >> >> >> On Thu, May 30, 2019 at 5:27 AM Nicolas Schuck <nico.schuck at gmail.com <mailto:nico.schuck at gmail.com>> wrote: >> Dear fellow R coders, >> >> I am observing differences in results obtained using glmer when using a Mac or Linux computer versus a PC. Specifically, I am talking about a relatively complex glmer model with a nested random effects structure. The model is set up in the following way: >> gcctrl = glmerControl(optimizer=c('nloptwrap'), optCtrl = list(maxfun = 500000), calc.derivs = FALSE) >> >> glmer_pre_instr1 = glmer( >> formula = cbind(FREQ, NSAMP-FREQ) ~ FDIST_minz + poly(RFREQ,2) + ROI + (1 + FDIST_minz + RFREQ + ROI|ID/COL), >> data = cdf_pre_instr, >> family = binomial, >> control = gcctrl) >> >> Code and data of an example for which I find reproducible, non-negligible differences between Mac/Win can be found here: https://gitlab.com/nschuck/glmer_sandbox/tree/master <https://gitlab.com/nschuck/glmer_sandbox/tree/master> <https://gitlab.com/nschuck/glmer_sandbox/tree/master <https://gitlab.com/nschuck/glmer_sandbox/tree/master>> >> The differences between the fitted models seem to be most pronounced regarding the estimated correlation structure of the random effects terms. Mac and Linux yield very similar results, but Windows deviates quite a bit in some cases. This has a large impact on p values obtained when performing model comparisons. I have tried this on Mac OS 10.14, Windows 10 and Ubuntu and Debian. All systems I have tried are using lme 1.1.21 and R 3.5+. >> >> Does anyone have an idea what the underlying cause might be? >> >> Thanks, >> Nico >> >> >> >> >> [[alternative HTML version deleted]] >> >> ______________________________________________ >> R-help at r-project.org <mailto:R-help at r-project.org> mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help <https://stat.ethz.ch/mailman/listinfo/r-help> >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html <http://www.r-project.org/posting-guide.html> >> and provide commented, minimal, self-contained, reproducible code.[[alternative HTML version deleted]]
On Wed, 29 May 2019 18:44:26 +0200 Nicolas Schuck <nico.schuck at gmail.com> wrote:> Dear fellow R coders, > > I am observing differences in results obtained using glmer when using > a Mac or Linux computer versus a PC....Just as an aside and point of information, both the Mac and the Linux systems are PCs, just as much so as one running Windows. The designation "PC" refers to hardware. Your question demands the extra steps of thinking about what other operating system might be in use. There are others though I doubt R is implemented on any of them. JWDougherty