Sean Porter
2015-Oct-27 08:56 UTC
[R] monte carlo simulations in permanova in vegan package
Dear colleagues, I am trying to run a PERMANOVA in the vegan package with an appropriate number of permutations (see example below), ideally 9999. Obviously that number of permutations does not exists so I would like to use Monte Carlo permutation tests to derive the probability value, as is done in the commercial package PERMANOVA+ for PRIMER. How can I adapt my code so that adonis will do so ? Many thanks, Sean> permanova <- adonis(species ~ time, data = time, permutations=99,method="bray")> permanovaCall: adonis(formula = species ~ time, data = time, permutations = 99, method = "bray") Permutation: free Number of permutations: 99 Terms added sequentially (first to last) Df SumsOfSqs MeanSqs F.Model R2 Pr(>F) time 1 0.070504 0.070504 123.65 0.96866 0.01 ** Residuals 4 0.002281 0.000570 0.03134 Total 5 0.072785 1.00000 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1> permanova <- adonis(species ~ time, data = time, permutations=999,method="bray") 'nperm' > set of all permutations; Resetting 'nperm'. [[alternative HTML version deleted]]
stephen sefick
2015-Oct-27 13:10 UTC
[R] monte carlo simulations in permanova in vegan package
The example code works, and reports 9999 permutations. Can you provide more information? data(dune) data(dune.env) adonis(dune ~ Management*A1, data=dune.env, permutations=9999) On Tue, Oct 27, 2015 at 3:56 AM, Sean Porter <sporter at ori.org.za> wrote:> Dear colleagues, > > > > I am trying to run a PERMANOVA in the vegan package with an appropriate > number of permutations (see example below), ideally 9999. Obviously that > number of permutations does not exists so I would like to use Monte Carlo > permutation tests to derive the probability value, as is done in the > commercial package PERMANOVA+ for PRIMER. How can I adapt my code so that > adonis will do so ? Many thanks, Sean > > > > > permanova <- adonis(species ~ time, data = time, permutations=99, > method="bray") > > > permanova > > > > Call: > > adonis(formula = species ~ time, data = time, permutations = 99, > method > = "bray") > > > > Permutation: free > > Number of permutations: 99 > > > > Terms added sequentially (first to last) > > > > Df SumsOfSqs MeanSqs F.Model R2 Pr(>F) > > time 1 0.070504 0.070504 123.65 0.96866 0.01 ** > > Residuals 4 0.002281 0.000570 0.03134 > > Total 5 0.072785 1.00000 > > --- > > Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 > > > > > > > permanova <- adonis(species ~ time, data = time, permutations=999, > method="bray") > > 'nperm' > set of all permutations; Resetting 'nperm'. > > > > > > > > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Stephen Sefick ************************************************** Auburn University Biological Sciences 331 Funchess Hall Auburn, Alabama 36849 ************************************************** sas0025 at auburn.edu http://www.auburn.edu/~sas0025 ************************************************** Let's not spend our time and resources thinking about things that are so little or so large that all they really do for us is puff us up and make us feel like gods. We are mammals, and have not exhausted the annoying little problems of being mammals. -K. Mullis "A big computer, a complex algorithm and a long time does not equal science." -Robert Gentleman [[alternative HTML version deleted]]
Sean Porter
2015-Oct-27 13:42 UTC
[R] monte carlo simulations in permanova in vegan package
Hi Stephen and others, I am trying to run a one-way permanova where I have only 2 levels in the factor ?time?, and each level contains only 3 replicates. So because I have such few observations (6 in total) and levels (2) there are not enough possible permutations to get a reasonable test (i.e. (2*3)!/ [2!(3!)^2]. That is why for example if I run the analysis with only 99 permutations it completes the task. However, if I set the number of permutations to anything larger it returns the message ?'nperm' > set of all permutations; Resetting 'nperm'.? as the number of possible permutations exceeds the number set by the argument ?permutations=?. In PERMANOVA + for PRIMER there is a way of dealing with this issue ? by using Monte Carlo simulations to generate the p value with a reasonable number of permutations. Hopefully this clarifies my situation and aim? I was therefore hoping there was a way of coding for the Monte-Carlo permutation procedure into adonis? Thanks for your help! From: stephen sefick [mailto:ssefick at gmail.com] Sent: 27 October 2015 03:11 PM To: Sean Porter Cc: r-help at r-project.org Subject: Re: [R] monte carlo simulations in permanova in vegan package The example code works, and reports 9999 permutations. Can you provide more information? data(dune) data(dune.env) adonis(dune ~ Management*A1, data=dune.env, permutations=9999) On Tue, Oct 27, 2015 at 3:56 AM, Sean Porter <sporter at ori.org.za> wrote: Dear colleagues, I am trying to run a PERMANOVA in the vegan package with an appropriate number of permutations (see example below), ideally 9999. Obviously that number of permutations does not exists so I would like to use Monte Carlo permutation tests to derive the probability value, as is done in the commercial package PERMANOVA+ for PRIMER. How can I adapt my code so that adonis will do so ? Many thanks, Sean> permanova <- adonis(species ~ time, data = time, permutations=99,method="bray")> permanovaCall: adonis(formula = species ~ time, data = time, permutations = 99, method = "bray") Permutation: free Number of permutations: 99 Terms added sequentially (first to last) Df SumsOfSqs MeanSqs F.Model R2 Pr(>F) time 1 0.070504 0.070504 123.65 0.96866 0.01 ** Residuals 4 0.002281 0.000570 0.03134 Total 5 0.072785 1.00000 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1> permanova <- adonis(species ~ time, data = time, permutations=999,method="bray") 'nperm' > set of all permutations; Resetting 'nperm'. [[alternative HTML version deleted]] ______________________________________________ R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Stephen Sefick ************************************************** Auburn University Biological Sciences 331 Funchess Hall Auburn, Alabama 36849 ************************************************** sas0025 at auburn.edu http://www.auburn.edu/~sas0025 ************************************************** Let's not spend our time and resources thinking about things that are so little or so large that all they really do for us is puff us up and make us feel like gods. We are mammals, and have not exhausted the annoying little problems of being mammals. -K. Mullis "A big computer, a complex algorithm and a long time does not equal science." -Robert Gentleman [[alternative HTML version deleted]]
Jari Oksanen
2015-Oct-29 13:23 UTC
[R] monte carlo simulations in permanova in vegan package
Sean Porter <sporter <at> ori.org.za> writes:> I am trying to run a PERMANOVA in the vegan package with an appropriate > number of permutations (see example below), ideally 9999. Obviously that > number of permutations does not exists so I would like to use Monte Carlo > permutation tests to derive the probability value, as is done in the > commercial package PERMANOVA+ for PRIMER. How can I adapt my code so that > adonis will do so ? Many thanks, Sean[...clip...]> > > permanova <- adonis(species ~ time, data = time, permutations=999, > method="bray") > > 'nperm' > set of all permutations; Resetting 'nperm'. >I assume we are talking about the latest version of vegan and permute packages. In that case you really should switch to complete enumeration if you request exceeds the number of distinct permutations. As people have told you, you should be satisfied with that because there are no more distinct permutations. Alternatively, you need more data. If you mean by Monte Carlo that the same that you have a sampling with return instead of permutation, or that the same observation can appear several times and therefore some other unit is missing, then there are two pieces of advice: 1. You should not do so. 2. If you want to do so, you can generate your resampling matrices by hand and use that matrix as the argument of permutations=. See the documentations (?adonis) which tells how to do so. Cheers, Jari Oksanen
Sean Porter
2015-Oct-30 10:13 UTC
[R] monte carlo simulations in permanova in vegan package
Thank you Jari, It seems now that my question is morphing more into a statistical one, and perhaps not appropriate for R-help list, so apologies. Yes we are talking about the latest versions of the vegan and permute packages. When there are an insufficient number of permutations available due to low sample sizes apparently an alternative is to use the result given in Anderson & Robinson (2003) regarding the asymptotic permutation of the numerator (or denominator) of the test statistic under permutation. And I quote from Anderson et al. 2008 "It is demonstrated that each of the sums of squares has, under permutation, an asymptotic distribution that is a linear form in chi-square variables, where the coefficients are actually the eigenvalues from a PCO of the resemblance matrix. Thus, chi-square variables can be drawn randomly and independently, using Monte Carlo sampling, and these can be combined with the eigenvalues to construct the asymptotic permutation distribution for each of the numerator and denominator and, thus, for the entire pseudo-F statistic, in the event that too few actual unique permutations exist." Anderson, Gorley & Clarke. 2008. PERMANOVA+ for PRIMER: Guide to software and statistical models. Anderson & Robinson 2003. Generalised discriminant analysis based on distances. Australian and New Zealand Journal of Statistics. 45: 301-318 I am sure you already know this! The above is what I am trying to do in the vegan package though.. Apologies if I am missing something and if what you have said still applies (that is not appropriate to exceed the possible number of permutations), I am not a statistician..so any help/clarity would be welcome.. Regards, sean ????????? -----Original Message----- From: R-help [mailto:r-help-bounces at r-project.org] On Behalf Of Jari Oksanen Sent: 29 October 2015 03:23 PM To: r-help at stat.math.ethz.ch Subject: Re: [R] monte carlo simulations in permanova in vegan package Sean Porter <sporter <at> ori.org.za> writes:> I am trying to run a PERMANOVA in the vegan package with an > appropriate number of permutations (see example below), ideally 9999. > Obviously that number of permutations does not exists so I would like > to use Monte Carlo permutation tests to derive the probability value, > as is done in the commercial package PERMANOVA+ for PRIMER. How can I > adapt my code so that adonis will do so ? Many thanks, Sean[...clip...]> > > permanova <- adonis(species ~ time, data = time, permutations=999, > method="bray") > > 'nperm' > set of all permutations; Resetting 'nperm'. >I assume we are talking about the latest version of vegan and permute packages. In that case you really should switch to complete enumeration if you request exceeds the number of distinct permutations. As people have told you, you should be satisfied with that because there are no more distinct permutations. Alternatively, you need more data. If you mean by Monte Carlo that the same that you have a sampling with return instead of permutation, or that the same observation can appear several times and therefore some other unit is missing, then there are two pieces of advice: 1. You should not do so. 2. If you want to do so, you can generate your resampling matrices by hand and use that matrix as the argument of permutations=. See the documentations (?adonis) which tells how to do so. Cheers, Jari Oksanen ______________________________________________ R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Sean Porter
2015-Oct-30 11:48 UTC
[R] monte carlo simulations in permanova in vegan package
Thank you Jari, It seems now that my question is morphing more into a statistical one, and perhaps not appropriate for R-help list, so apologies. Yes we are talking about the latest versions of the vegan and permute packages. When there are an insufficient number of permutations available due to low sample sizes apparently an alternative is to use the result given in Anderson & Robinson (2003) regarding the asymptotic permutation of the numerator (or denominator) of the test statistic under permutation. And I quote from Anderson et al. 2008 "It is demonstrated that each of the sums of squares has, under permutation, an asymptotic distribution that is a linear form in chi-square variables, where the coefficients are actually the eigenvalues from a PCO of the resemblance matrix. Thus, chi-square variables can be drawn randomly and independently, using Monte Carlo sampling, and these can be combined with the eigenvalues to construct the asymptotic permutation distribution for each of the numerator and denominator and, thus, for the entire pseudo-F statistic, in the event that too few actual unique permutations exist." Anderson, Gorley & Clarke. 2008. PERMANOVA+ for PRIMER: Guide to software and statistical models. Anderson & Robinson 2003. Generalised discriminant analysis based on distances. Australian and New Zealand Journal of Statistics. 45: 301-318 I am sure you already know this! The above is what I am trying to do in the vegan package though.. Apologies if I am missing something and if what you have said still applies (that is not appropriate to exceed the possible number of permutations), I am not a statistician..so any help/clarity would be welcome.. Regards, sean ????????? -----Original Message----- From: R-help [mailto:r-help-bounces at r-project.org] On Behalf Of Jari Oksanen Sent: 29 October 2015 03:23 PM To: r-help at stat.math.ethz.ch Subject: Re: [R] monte carlo simulations in permanova in vegan package Sean Porter <sporter <at> ori.org.za> writes:> I am trying to run a PERMANOVA in the vegan package with an > appropriate number of permutations (see example below), ideally 9999. > Obviously that number of permutations does not exists so I would like > to use Monte Carlo permutation tests to derive the probability value, > as is done in the commercial package PERMANOVA+ for PRIMER. How can I > adapt my code so that adonis will do so ? Many thanks, Sean[...clip...]> > > permanova <- adonis(species ~ time, data = time, permutations=999, > method="bray") > > 'nperm' > set of all permutations; Resetting 'nperm'. >I assume we are talking about the latest version of vegan and permute packages. In that case you really should switch to complete enumeration if you request exceeds the number of distinct permutations. As people have told you, you should be satisfied with that because there are no more distinct permutations. Alternatively, you need more data. If you mean by Monte Carlo that the same that you have a sampling with return instead of permutation, or that the same observation can appear several times and therefore some other unit is missing, then there are two pieces of advice: 1. You should not do so. 2. If you want to do so, you can generate your resampling matrices by hand and use that matrix as the argument of permutations=. See the documentations (?adonis) which tells how to do so. Cheers, Jari Oksanen ______________________________________________ R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.