Hi all I am using bam to analyse the data from my experiment. It's a learning experiment, "acc" denotes accuracy and "cnd" denotes a within-subjects variable (with two levels, "label" and "ideo")."Ctrial" is centered trial (ranging from 1 to 288). The model is: bam(acc~ 1 + cnd + s(ctrial) + s(ctrial, sbj, bs = "fs", m = 1), data=data, family=binomial) The model doesn't include two different smooths (one for each condition) since including two smooths does not result to a more parsimonious model, according to following model comparison:> compareML(m0.2, m1.2)m0.2: acc ~ 1 + cnd + s(ctrial) + s(ctrial, sbj, bs = "fs", m = 1) m1.2: acc ~ 1 + cnd + s(ctrial, by = cnd) + s(ctrial, sbj, bs = "fs", m = 1) Chi-square test of fREML scores ----- Model Score Edf Chisq Df p.value Sig. 1 m0.2 10183.31 6 2 m1.2 10173.33 8 9.975 2.000 4.654e-05 *** AIC difference: -2.16, model m0.2 has lower AIC. So, I'm trying to assess if there's a difference in accuracy between the two conditions. When using the plot_smooth function, the model predictions are the ones shown in Fig.1. The code used is: plot_smooth(fm, view="ctrial", cond=list(cnd="pseudo"),main="Model",xaxt="n", xlab="Trial",ylab="Proportion Correct", lwd=2, las=2, rm.ranef=TRUE, rug=FALSE, shade=T, col="red" ) plot_smooth(fm, view="ctrial", cond=list(cnd="ideo"), xaxt="n", rm.ranef=TRUE, rug=FALSE, shade=T, col="blue", add=T , lty=2, lwd=2) legend(x=0.8, y=1.5,legend=c('Label', 'Ideogram'),col=c('red', 'blue'), lty=c(1,2), bty="n", lwd=2) Since the 95% confidence intervals overlap, I would assume that there is no difference in accuracy between the two conditions. I am also using plot_diff to directly plot the difference: plot_diff(fm, view="ctrial",comp=list(cnd=c("pseudo", "ideo")), transform.view=dnrmlz,rm.ranef=T) (dnrmlz is a simple function to de-normalize trial) The output of the function is: Summary: * ctrial : numeric predictor; with 100 values ranging from -1.725936 to 1.725936. * sbj : factor; set to the value(s): aggmpo96. (Might be canceled as random effect, check below.) * NOTE : The following random effects columns are canceled: s(ctrial,sbj) * Note: x-values are transformed. Significant 1 0.759461 - 288.240539 So, it seems that accuracy in the label condition is higher compared to the ideo condition throughout the experiment. This result seems to contradict the previous one. I am obviously misinterpreting something. Any ideas on what am I doing wrong? Thank you in advance for your time, Fotis -- PhD Candidate Department of Philosophy and History of Science University of Athens, Greece. http://users.uoa.gr/~aprotopapas/LLL/en/members.html#fotisfotiadis Notice: Please do not use this account for social networks invitations, for sending chain-mails to me, or as it were a facebook account. Thank you for respecting my privacy. <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virus-free. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2> -------------- next part -------------- A non-text attachment was scrubbed... Name: Fig1.png Type: image/png Size: 6837 bytes Desc: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20160612/d630b0d8/attachment.png> -------------- next part -------------- A non-text attachment was scrubbed... Name: Fig2.png Type: image/png Size: 6915 bytes Desc: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20160612/d630b0d8/attachment-0001.png>
To be clear, I know nothing about bam; I just wanted to correct a statistical error: "Since the 95% confidence intervals overlap, I would assume that there is no difference in accuracy between the two conditions." That is false. You need to look at a CI for the difference. As you appear to be confused about the statistical issues, I suggest you post on a statistical site like stats.stackexchange.com or consult a local statistician. Cheers, Bert Bert Gunter "The trouble with having an open mind is that people keep coming along and sticking things into it." -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip ) On Sun, Jun 12, 2016 at 7:03 AM, Fotis Fotiadis <fotisfotiadis at gmail.com> wrote:> Hi all > > I am using bam to analyse the data from my experiment. > It's a learning experiment, "acc" denotes accuracy and "cnd" denotes a > within-subjects variable (with two levels, "label" and "ideo")."Ctrial" is > centered trial (ranging from 1 to 288). > > The model is: > bam(acc~ 1 + cnd + s(ctrial) + s(ctrial, sbj, bs = "fs", m = 1), data=data, > family=binomial) > > The model doesn't include two different smooths (one for each condition) > since including two smooths does not result to a more parsimonious model, > according to following model comparison: >> compareML(m0.2, m1.2) > m0.2: acc ~ 1 + cnd + s(ctrial) + s(ctrial, sbj, bs = "fs", m = 1) > > m1.2: acc ~ 1 + cnd + s(ctrial, by = cnd) + s(ctrial, sbj, bs = "fs", > m = 1) > > Chi-square test of fREML scores > ----- > Model Score Edf Chisq Df p.value Sig. > 1 m0.2 10183.31 6 > 2 m1.2 10173.33 8 9.975 2.000 4.654e-05 *** > > AIC difference: -2.16, model m0.2 has lower AIC. > > > So, I'm trying to assess if there's a difference in accuracy between the > two conditions. > > When using the plot_smooth function, the model predictions are the ones > shown in Fig.1. > The code used is: > plot_smooth(fm, view="ctrial", > cond=list(cnd="pseudo"),main="Model",xaxt="n", > xlab="Trial",ylab="Proportion Correct", lwd=2, las=2, rm.ranef=TRUE, > rug=FALSE, shade=T, col="red" ) > plot_smooth(fm, view="ctrial", cond=list(cnd="ideo"), xaxt="n", > rm.ranef=TRUE, rug=FALSE, shade=T, col="blue", add=T , lty=2, lwd=2) > legend(x=0.8, y=1.5,legend=c('Label', 'Ideogram'),col=c('red', 'blue'), > lty=c(1,2), bty="n", lwd=2) > > Since the 95% confidence intervals overlap, I would assume that there is no > difference in accuracy between the two conditions. > > I am also using plot_diff to directly plot the difference: > plot_diff(fm, view="ctrial",comp=list(cnd=c("pseudo", "ideo")), > transform.view=dnrmlz,rm.ranef=T) > (dnrmlz is a simple function to de-normalize trial) > > The output of the function is: > Summary: > * ctrial : numeric predictor; with 100 values ranging from -1.725936 to > 1.725936. > * sbj : factor; set to the value(s): aggmpo96. (Might be canceled as random > effect, check below.) > * NOTE : The following random effects columns are canceled: s(ctrial,sbj) > > * Note: x-values are transformed. > Significant > 1 0.759461 - 288.240539 > > So, it seems that accuracy in the label condition is higher compared to the > ideo condition throughout the experiment. > This result seems to contradict the previous one. > > I am obviously misinterpreting something. > Any ideas on what am I doing wrong? > > Thank you in advance for your time, > Fotis > > > > > > > > -- > PhD Candidate > Department of Philosophy and History of Science > University of Athens, Greece. > http://users.uoa.gr/~aprotopapas/LLL/en/members.html#fotisfotiadis > > Notice: Please do not use this account for social networks invitations, for > sending chain-mails to me, or as it were a facebook account. Thank you for > respecting my privacy. > > <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> > Virus-free. > www.avast.com > <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> > <#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2> > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
Dear Bert, Thank you for your response Best, Fotis On Sun, Jun 12, 2016 at 5:50 PM, Bert Gunter <bgunter.4567 at gmail.com> wrote:> To be clear, I know nothing about bam; I just wanted to correct a > statistical error: > > "Since the 95% confidence intervals overlap, I would assume that there is > no > difference in accuracy between the two conditions." > > That is false. You need to look at a CI for the difference. > > As you appear to be confused about the statistical issues, I suggest > you post on a statistical site like stats.stackexchange.com or consult > a local statistician. > > Cheers, > Bert > > > Bert Gunter > > "The trouble with having an open mind is that people keep coming along > and sticking things into it." > -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip ) > > > On Sun, Jun 12, 2016 at 7:03 AM, Fotis Fotiadis <fotisfotiadis at gmail.com> > wrote: > > Hi all > > > > I am using bam to analyse the data from my experiment. > > It's a learning experiment, "acc" denotes accuracy and "cnd" denotes a > > within-subjects variable (with two levels, "label" and "ideo")."Ctrial" > is > > centered trial (ranging from 1 to 288). > > > > The model is: > > bam(acc~ 1 + cnd + s(ctrial) + s(ctrial, sbj, bs = "fs", m = 1), > data=data, > > family=binomial) > > > > The model doesn't include two different smooths (one for each condition) > > since including two smooths does not result to a more parsimonious model, > > according to following model comparison: > >> compareML(m0.2, m1.2) > > m0.2: acc ~ 1 + cnd + s(ctrial) + s(ctrial, sbj, bs = "fs", m = 1) > > > > m1.2: acc ~ 1 + cnd + s(ctrial, by = cnd) + s(ctrial, sbj, bs = "fs", > > m = 1) > > > > Chi-square test of fREML scores > > ----- > > Model Score Edf Chisq Df p.value Sig. > > 1 m0.2 10183.31 6 > > 2 m1.2 10173.33 8 9.975 2.000 4.654e-05 *** > > > > AIC difference: -2.16, model m0.2 has lower AIC. > > > > > > So, I'm trying to assess if there's a difference in accuracy between the > > two conditions. > > > > When using the plot_smooth function, the model predictions are the ones > > shown in Fig.1. > > The code used is: > > plot_smooth(fm, view="ctrial", > > cond=list(cnd="pseudo"),main="Model",xaxt="n", > > xlab="Trial",ylab="Proportion Correct", lwd=2, las=2, rm.ranef=TRUE, > > rug=FALSE, shade=T, col="red" ) > > plot_smooth(fm, view="ctrial", cond=list(cnd="ideo"), xaxt="n", > > rm.ranef=TRUE, rug=FALSE, shade=T, col="blue", add=T , lty=2, lwd=2) > > legend(x=0.8, y=1.5,legend=c('Label', 'Ideogram'),col=c('red', 'blue'), > > lty=c(1,2), bty="n", lwd=2) > > > > Since the 95% confidence intervals overlap, I would assume that there is > no > > difference in accuracy between the two conditions. > > > > I am also using plot_diff to directly plot the difference: > > plot_diff(fm, view="ctrial",comp=list(cnd=c("pseudo", "ideo")), > > transform.view=dnrmlz,rm.ranef=T) > > (dnrmlz is a simple function to de-normalize trial) > > > > The output of the function is: > > Summary: > > * ctrial : numeric predictor; with 100 values ranging from -1.725936 to > > 1.725936. > > * sbj : factor; set to the value(s): aggmpo96. (Might be canceled as > random > > effect, check below.) > > * NOTE : The following random effects columns are canceled: s(ctrial,sbj) > > > > * Note: x-values are transformed. > > Significant > > 1 0.759461 - 288.240539 > > > > So, it seems that accuracy in the label condition is higher compared to > the > > ideo condition throughout the experiment. > > This result seems to contradict the previous one. > > > > I am obviously misinterpreting something. > > Any ideas on what am I doing wrong? > > > > Thank you in advance for your time, > > Fotis > > > > > > > > > > > > > > > > -- > > PhD Candidate > > Department of Philosophy and History of Science > > University of Athens, Greece. > > http://users.uoa.gr/~aprotopapas/LLL/en/members.html#fotisfotiadis > > > > Notice: Please do not use this account for social networks invitations, > for > > sending chain-mails to me, or as it were a facebook account. Thank you > for > > respecting my privacy. > > > > < > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > > Virus-free. > > www.avast.com > > < > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > > <#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2> > > > > ______________________________________________ > > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. >-- PhD Candidate Department of Philosophy and History of Science University of Athens, Greece. http://users.uoa.gr/~aprotopapas/LLL/en/members.html#fotisfotiadis Notice: Please do not use this account for social networks invitations, for sending chain-mails to me, or as it were a facebook account. Thank you for respecting my privacy. [[alternative HTML version deleted]]