R help - Dec 2011 - Non-negativity constraints for logistic regression

Dear R users,

I am currently attempting to fit logistic regression models in R, where
the slopes should be restricted to positive values. Although I am aware
of the package nnls (which does the trick for linear regression models),
I did not find any solution for logistic regression. If there is any
package available for this purpose, I would be interested to know them.

Alternatively, I realize it is possible to optimize a specialized
likelihood function that does the trick. Although I know how to optimize
the log-likelihood of logistic regression models, I am not sure how to
implement non-negativity constraints for slope parameters without
messing up the Newton optimization. Therefore, I am also interested in
solutions for this problem.

Best regards,

Thomas Debray

On 21/12/2011 18:26, tw at netstorm.be wrote:
> Dear R users,
>
> I am currently attempting to fit logistic regression models in R, where
> the slopes should be restricted to positive values. Although I am aware
I guess non-negative, as in the subject line, so there actually is a 
solution.
> of the package nnls (which does the trick for linear regression models),
> I did not find any solution for logistic regression. If there is any
> package available for this purpose, I would be interested to know them.
>
> Alternatively, I realize it is possible to optimize a specialized
> likelihood function that does the trick. Although I know how to optimize
> the log-likelihood of logistic regression models, I am not sure how to
> implement non-negativity constraints for slope parameters without
> messing up the Newton optimization. Therefore, I am also interested in
> solutions for this problem.
There is an example of this in the 'Optimization' chapter of MASS (the 
book, page 445 to be precise).  You simply use an optimizer with box 
constraints: see ?optim and ?nlminb, for example.
> Best regards,
>
> Thomas Debray

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

> Dear R users,
> I am currently attempting to fit logistic regression models in R, where
> the slopes should be restricted to positive values. Although I am aware
>> I guess non-negative, as in the subject line, so there actually is a
solution.

Indeed, I meant non-negative, zero slopes are also possible parameter
values for my case.

> of the package nnls (which does the trick for linear regression models),
> I did not find any solution for logistic regression. If there is any
> package available for this purpose, I would be interested to know them.
> Alternatively, I realize it is possible to optimize a specialized
> likelihood function that does the trick. Although I know how to optimize
> the log-likelihood of logistic regression models, I am not sure how to
> implement non-negativity constraints for slope parameters without
> messing up the Newton optimization. Therefore, I am also interested in
> solutions for this problem.
>> There is an example of this in the 'Optimization' chapter of
MASS (the
book, page 445 to be precise).  You simply use an optimizer with box
constraints: see ?optim and ?nlminb, for example.

Thanks a lot, I managed to get it fully working by passing the constraints
to L-BFGS-B.

Hi Thomas,

Using box-constrained optimizer in glm.fit is a good suggestion for finding the
point estimates.  However, there is still the issue of making inference, i.e.,
computing the variances and p-values for the estimates.  You have to deal with
the issue of MLE possibly being on the boundary.  Asymptotic distribution of MLE
estimators will not be normal in the case of convergence at the boundary.  This
is a difficult problem.

Best,
Ravi

-------------------------------------------------------
Ravi Varadhan, Ph.D.
Assistant Professor,
Division of Geriatric Medicine and Gerontology School of Medicine Johns Hopkins
University

Ph. (410) 502-2619
email: rvaradhan@jhmi.edu<mailto:rvaradhan@jhmi.edu>


	[[alternative HTML version deleted]]