4ap1 at queensu.ca
2009-Apr-13 01:40 UTC
[Rd] dnbinom with a large size parameter (PR#13650)
Full_Name: Andrey Pavlov Version: 2.7.1 (2008-06-23) OS: Windows Vista Submission from: (NULL) (67.193.233.43) Dear developers, I discovered an issue with the dnbinom function while fitting a negative binomial model to my data. I was using the size and mu parameterization. When the size gets large enough, the function begins to return 1, while it should instead return the respective Poisson probability. This can be seen in the following simple example:> dpois(1,lambda=1)[1] 0.3678794> dnbinom(1,size=1e+15,mu=1)[1] 0.3678793> dnbinom(1,size=3e+15,mu=1)[1] 0.3678793 - very close to Poisson. But then I increase the size further, and it goes off somewhat:> dnbinom(1,size=5e+15,mu=1)[1] 0.3658024> dnbinom(1,size=7e+15,mu=1)[1] 0.3572676 ...until suddenly it returns 1:> dnbinom(1,size=10e+15,mu=1)[1] 1 This turned out to be a big and hard to track down issue for me. The bug confused the optimizer of the likelihood function, which happened to move too far on the size dimension and began to discover "very good" parameters. I fixed the problem by adding a logical check, which replaced the negative binomial probability with the Poisson one in case the size was large, but this is a very crude solution. Perhaps it is worth fixing the internal routine. Thanks, Andrey
4ap1 at queensu.ca wrote:> Full_Name: Andrey Pavlov > Version: 2.7.1 (2008-06-23)!!!> OS: Windows Vista > Submission from: (NULL) (67.193.233.43) > > > Dear developers, > > I discovered an issue with the dnbinom function while fitting a negative > binomial model to my data. I was using the size and mu parameterization. When > the size gets large enough, the function begins to return 1, while it should > instead return the respective Poisson probability. This can be seen in the > following simple example: > >> dpois(1,lambda=1) > [1] 0.3678794 >> dnbinom(1,size=1e+15,mu=1) > [1] 0.3678793 >> dnbinom(1,size=3e+15,mu=1) > [1] 0.3678793 > > - very close to Poisson. But then I increase the size further, and it goes off > somewhat: > >> dnbinom(1,size=5e+15,mu=1) > [1] 0.3658024 >> dnbinom(1,size=7e+15,mu=1) > [1] 0.3572676 > > ...until suddenly it returns 1: >> dnbinom(1,size=10e+15,mu=1) > [1] 1 > > This turned out to be a big and hard to track down issue for me. The bug > confused the optimizer of the likelihood function, which happened to move too > far on the size dimension and began to discover "very good" parameters. I fixed > the problem by adding a logical check, which replaced the negative binomial > probability with the Poisson one in case the size was large, but this is a very > crude solution. Perhaps it is worth fixing the internal routine.Doesn't happen in 2.8.1: > dnbinom(1,size=10e+15,mu=1) [1] 0.3678794 This was fixed (if you define it as a "bug" rather than an "FP accuracy issue") in 2.7.2/2.7.2patched. You might at least have checked the NEWS file for references to dnbinom() before submitting a bug report on an older version! -- O__ ---- Peter Dalgaard ?ster Farimagsgade 5, Entr.B c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk) FAX: (+45) 35327907