R help - Apr 2025 - Estimating regression with constraints in model coefficients

Hi Gregg,

I thank you for for information about the function vglm()

However it appears that my constraints are a little different.

My parameters have lower and upper bounds and also sum of the
estimated coefficients should be equal to some predefined value.

Other than that, there is no Intercept in my model.

I am more concerned on how can I put the above sum constraint and no
intercept in the model design.

One way to impose that perhaps is to use some Penalty function to the
log-likelihood function, but probably that would become too complex
for this type of constraint. Also, I tried to look into the internal
function vglm.fitter() to explore how log-likelihood has been
constructed, however I could not see the code inside this function.

Any further help would be appreciated.

On Wed, Apr 9, 2025 at 12:33?AM Gregg Powell <g.a.powell at
protonmail.com> wrote:
>
> there are ways to implement constraints on parameter estimates in ordinal
logistic regression in R. Here are a few approaches:
>
> The rms package (Regression Modeling Strategies) by Frank Harrell offers
the lrm function which can handle constraints through its penalty parameter,
though it's primarily designed for regularization.
> For more flexible constraints, you can use the constrOptim or optim
functions from base R along with a custom likelihood function for ordinal
logistic regression.
> The VGAM package provides the vglm function with family cumulative that can
handle certain types of constraints.
> For Bayesian approaches, you can use brms or rstan to impose informative
priors that effectively constrain parameters.
>
> Here's a simple example using the VGAM package:
>
> > library(VGAM)
> > library(foreign)
> >
> > # Load data
> > dat <-
foreign::read.dta("https://stats.idre.ucla.edu/stat/data/ologit.dta")
> >
>
> > # Unconstrained model (for comparison)
> > model_unconstrained <- vglm(apply ~ pared + public + gpa,
>
> >                           family = cumulative(parallel = TRUE),
> >                           data = dat)
> > summary(model_unconstrained)
> >
> > # Constrained model (example: constraining pared coefficient to be
positive)
> > # This uses the "constraint" matrix approach
> > constraint <- rbind(
> >  c(0, 1, 0, 0),  # This row corresponds to pared coefficient
> >  c(0, 0, 0, 0),  # These rows do nothing (identity constraints for
other parameters)
> >  c(0, 0, 0, 0)
> > )
>
> > model_constrained <- vglm(apply ~ pared + public + gpa,
>
> >                         family = cumulative(parallel = TRUE),
> >                         constraints = constraint,
> >                         data = dat)
> > summary(model_constrained)
>
>
> For more complex constraints, you might need to work with optimization
functions directly.
>
>
> r/
> Gregg
>
>
>
> On Tuesday, April 8th, 2025 at 11:20 AM, Christofer Bogaso
<bogaso.christofer at gmail.com> wrote:
>
> >
>
> >
>
> > Hi,
> >
>
> > I have below fit with ordinal logistic regression
> >
>
> > dat =
foreign::read.dta("https://stats.idre.ucla.edu/stat/data/ologit.dta")
> >
>
> > summary(MASS::polr(formula = apply ~ pared + public + gpa, data =
dat))
> >
>
> > However, instead of obtaining unconstrained estimates of model
> > parameters, I would like to impose certain constraints on each of the
> > model parameters, based on some non-sample information.
> >
>
> > Is there any R function to estimate model coefficients with imposing
> > some unser-defined constraints on the model parameters?
> >
>
> > Any pointer will be very helpful.
> >
>
> > ______________________________________________
> > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
https://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.

It may be overkill, but package nlsr has function nlxb() that can handle
various models and bound the parameters. Note that bounds can sometimes give
weird results if the bounds and initial parameter guesses are such that the
minimization of the sum of squares gets "stuck".

JN


On 2025-04-21 09:00, Christofer Bogaso wrote:
> Hi Gregg,
> 
> I thank you for for information about the function vglm()
> 
> However it appears that my constraints are a little different.
> 
> My parameters have lower and upper bounds and also sum of the
> estimated coefficients should be equal to some predefined value.
> 
> Other than that, there is no Intercept in my model.
> 
> I am more concerned on how can I put the above sum constraint and no
> intercept in the model design.
> 
> One way to impose that perhaps is to use some Penalty function to the
> log-likelihood function, but probably that would become too complex
> for this type of constraint. Also, I tried to look into the internal
> function vglm.fitter() to explore how log-likelihood has been
> constructed, however I could not see the code inside this function.
> 
> Any further help would be appreciated.
> 
> On Wed, Apr 9, 2025 at 12:33?AM Gregg Powell <g.a.powell at
protonmail.com> wrote:
>>
>> there are ways to implement constraints on parameter estimates in
ordinal logistic regression in R. Here are a few approaches:
>>
>> The rms package (Regression Modeling Strategies) by Frank Harrell
offers the lrm function which can handle constraints through its penalty
parameter, though it's primarily designed for regularization.
>> For more flexible constraints, you can use the constrOptim or optim
functions from base R along with a custom likelihood function for ordinal
logistic regression.
>> The VGAM package provides the vglm function with family cumulative that
can handle certain types of constraints.
>> For Bayesian approaches, you can use brms or rstan to impose
informative priors that effectively constrain parameters.
>>
>> Here's a simple example using the VGAM package:
>>
>>> library(VGAM)
>>> library(foreign)
>>>
>>> # Load data
>>> dat <-
foreign::read.dta("https://stats.idre.ucla.edu/stat/data/ologit.dta")
>>>
>>
>>> # Unconstrained model (for comparison)
>>> model_unconstrained <- vglm(apply ~ pared + public + gpa,
>>
>>>                            family = cumulative(parallel = TRUE),
>>>                            data = dat)
>>> summary(model_unconstrained)
>>>
>>> # Constrained model (example: constraining pared coefficient to be
positive)
>>> # This uses the "constraint" matrix approach
>>> constraint <- rbind(
>>>   c(0, 1, 0, 0),  # This row corresponds to pared coefficient
>>>   c(0, 0, 0, 0),  # These rows do nothing (identity constraints for
other parameters)
>>>   c(0, 0, 0, 0)
>>> )
>>
>>> model_constrained <- vglm(apply ~ pared + public + gpa,
>>
>>>                          family = cumulative(parallel = TRUE),
>>>                          constraints = constraint,
>>>                          data = dat)
>>> summary(model_constrained)
>>
>>
>> For more complex constraints, you might need to work with optimization
functions directly.
>>
>>
>> r/
>> Gregg
>>
>>
>>
>> On Tuesday, April 8th, 2025 at 11:20 AM, Christofer Bogaso
<bogaso.christofer at gmail.com> wrote:
>>
>>>
>>
>>>
>>
>>> Hi,
>>>
>>
>>> I have below fit with ordinal logistic regression
>>>
>>
>>> dat =
foreign::read.dta("https://stats.idre.ucla.edu/stat/data/ologit.dta")
>>>
>>
>>> summary(MASS::polr(formula = apply ~ pared + public + gpa, data =
dat))
>>>
>>
>>> However, instead of obtaining unconstrained estimates of model
>>> parameters, I would like to impose certain constraints on each of
the
>>> model parameters, based on some non-sample information.
>>>
>>
>>> Is there any R function to estimate model coefficients with
imposing
>>> some unser-defined constraints on the model parameters?
>>>
>>
>>> Any pointer will be very helpful.
>>>
>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more,
see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
https://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
> 
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
https://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

Christofer,

Given the constraints you mentioned?bounded parameters, no intercept, and a sum
constraint?you're outside the capabilities of most off-the-shelf ordinal
logistic regression functions in R like vglm or polr.

The most flexible recommendation at this point is to implement custom likelihood
optimization using constrOptim() or nloptr::nloptr() with a manually coded
log-likelihood function for the cumulative logit model.

Since you need:

Coefficient bounds (e.g., lb ? ? ? ub),

No intercept,

And a constraint like sum(?) = C,

?you'll need to code your own objective function. Here's something of a
high-level outline of the approach:

A. Model Setup
Let your design matrix X be n x p, and let Y take ordered values 1, 2, ..., K.

B. Parameters
Assume the thresholds (?_k) are fixed (or removed entirely), and you?re
estimating only the coefficient vector ?. Your constraints are:

Box constraints: lb ? ? ? ub

Equality constraint: sum(?) = C

C. Likelihood
The probability of category k is given by:

P(Y = k) = logistic(?_k - X?) - logistic(?_{k-1} - X?)

Without intercepts, this becomes more like:

P(Y ? k) = 1 / (1 + exp(-(c_k - X?)))

?where c_k are fixed thresholds.

Implementation using nloptr
This example shows a rough sketch using nloptr, which allows both equality and
bound constraints:
>library(nloptr)
>
># Custom negative log-likelihood function
>negLL <- function(beta, X, y, K, cutpoints) {
>  eta <- X %*% beta
>  loglik <- 0
>  for (k in 1:K) {
>    pk <- plogis(cutpoints[k] - eta) - plogis(cutpoints[k - 1] - eta)
>    loglik <- loglik + sum(log(pk[y == k]))
>  }
>  return(-loglik)
>}
>
># Constraint: sum(beta) = C
>sum_constraint <- function(beta, C) {
>  return(sum(beta) - C)
>}
>
># Define objective and constraint wrapper
>objective <- function(beta) negLL(beta, X, y, K, cutpoints)
>eq_constraint <- function(beta) sum_constraint(beta, C = 2)  # example
>C
>
># Run optimization
>res <- nloptr(
>  x0 = rep(0, ncol(X)),
>  eval_f = objective,
>  lb = lower_bounds,
>  ub = upper_bounds,
>  eval_g_eq = eq_constraint,
>  opts = list(algorithm = "NLOPT_LD_SLSQP", xtol_rel = 1e-8)
>)


The next step would be writing the actual log-likelihood for your specific
problem or verifying constraint implementation.

Manually coding the log-likelihood for an ordinal model is nontrivial... so a
bit of a challenge :)



All the best,
gregg powell
Sierra Vista, AZ
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 603 bytes
Desc: OpenPGP digital signature
URL:
<https://stat.ethz.ch/pipermail/r-help/attachments/20250421/b7f8cea4/attachment.sig>