Sorkin, John
2023-Aug-19 22:39 UTC
[R] Determining Starting Values for Model Parameters in Nonlinear Regression
Colleagues,
At the risk of starting a forest fire, or perhaps a brush fire, while it is good
to see that nlxb can find a solution from arbitrary starting values, I think
Paul?s question has merit despite Professor Nash?s excellent and helpful
observation.
Although non-linear algorithms can converge, they can converge to a false
solution if starting values are sub-optimally specified. When possible, I try to
specify thought-out starting values. Would it make sense to plot y as a function
of (x1, x2) at different values of x3 to get a sense of possible starting
values? Or, perhaps using median values of x1, x2, and x3 as starting values.
Comparing results from different starting values can give some confidence that
the solution obtained using arbitrary starting values are likely ?correct?.
I freely admit that my experience (and thus expertise) using non-linear
solutions is limited. Please do not flame me, I am simply urging caution.
John
John David Sorkin M.D., Ph.D.
Professor of Medicine
Chief, Biostatistics and Informatics
University of Maryland School of Medicine Division of Gerontology and Geriatric
Medicine
Baltimore VA Medical Center
10 North Greene Street<x-apple-data-detectors://12>
GRECC<x-apple-data-detectors://12> (BT/18/GR)
Baltimore, MD 21201-1524<x-apple-data-detectors://13/0>
(Phone) 410-605-711<tel:410-605-7119>9
(Fax) 410-605-7913<tel:410-605-7913> (Please call phone number above prior
to faxing)
On Aug 19, 2023, at 4:35 PM, J C Nash <profjcnash at
gmail.com<mailto:profjcnash at gmail.com>> wrote:
Why bother. nlsr can find a solution from very crude start.
Mixture <- c(17, 14, 5, 1, 11, 2, 16, 7, 19, 23, 20, 6, 13, 21, 3, 18, 15,
26, 8, 22)
x1 <- c(69.98, 72.5, 77.6, 79.98, 74.98, 80.06, 69.98, 77.34, 69.99, 67.49,
67.51, 77.63,
72.5, 67.5, 80.1, 69.99, 72.49, 64.99, 75.02, 67.48)
x2 <- c(29, 25.48, 21.38, 19.85, 22, 18.91, 29.99, 19.65, 26.99, 29.49,
32.47,
20.35, 26.48, 31.47, 16.87, 27.99, 24.49, 31.99, 24.96, 30.5)
x3 <- c(1, 2, 1, 0, 3, 1, 0, 2.99, 3, 3, 0, 2, 1, 1, 3, 2,
3, 3, 0, 2)
y <- c(1.4287, 1.4426, 1.4677, 1.4774, 1.4565,
1.4807, 1.4279, 1.4684, 1.4301, 1.4188, 1.4157, 1.4686, 1.4414,
1.4172, 1.4829, 1.4291, 1.4438, 1.4068, 1.4524, 1.4183)
mydata<-data.frame(Mixture, x1, x2, x3, y)
mydata
mymod <- y ~ 1/(Beta1*x1 + Beta2*x2 + Beta3*x3)
library(nlsr)
strt<-c(Beta1=1, Beta2=2, Beta3=3)
trysol<-nlxb(formula=mymod, data=mydata, start=strt, trace=TRUE)
trysol
# or pshort(trysol)
Output is
residual sumsquares = 1.5412e-05 on 20 observations
after 29 Jacobian and 43 function evaluations
name coeff SE tstat pval gradient
JSingval
Beta1 0.00629212 5.997e-06 1049 2.425e-42 4.049e-08
721.8
Beta2 0.00867741 1.608e-05 539.7 1.963e-37 -2.715e-08
56.05
Beta3 0.00801948 8.809e-05 91.03 2.664e-24 1.497e-08
10.81
J Nash
On 2023-08-19 16:19, Paul Bernal wrote:
Dear friends,
Hope you are all doing well and having a great weekend. I have data that
was collected on specific gravity and spectrophotometer analysis for 26
mixtures of NG (nitroglycerine), TA (triacetin), and 2 NDPA (2 -
nitrodiphenylamine).
In the dataset, x1 = %NG, x2 = %TA, and x3 = %2 NDPA.
The response variable is the specific gravity, and the rest of the
variables are the predictors.
This is the dataset:
dput(mod14data_random)
structure(list(Mixture = c(17, 14, 5, 1, 11, 2, 16, 7, 19, 23,
20, 6, 13, 21, 3, 18, 15, 26, 8, 22), x1 = c(69.98, 72.5, 77.6,
79.98, 74.98, 80.06, 69.98, 77.34, 69.99, 67.49, 67.51, 77.63,
72.5, 67.5, 80.1, 69.99, 72.49, 64.99, 75.02, 67.48), x2 = c(29,
25.48, 21.38, 19.85, 22, 18.91, 29.99, 19.65, 26.99, 29.49, 32.47,
20.35, 26.48, 31.47, 16.87, 27.99, 24.49, 31.99, 24.96, 30.5),
x3 = c(1, 2, 1, 0, 3, 1, 0, 2.99, 3, 3, 0, 2, 1, 1, 3, 2,
3, 3, 0, 2), y = c(1.4287, 1.4426, 1.4677, 1.4774, 1.4565,
1.4807, 1.4279, 1.4684, 1.4301, 1.4188, 1.4157, 1.4686, 1.4414,
1.4172, 1.4829, 1.4291, 1.4438, 1.4068, 1.4524, 1.4183)), row.names c(NA,
-20L), class = "data.frame")
The model is the following:
y = 1/(Beta1x1 + Beta2x2 + Beta3x3)
I need to determine starting (initial) values for the model parameters for
this nonlinear regression model, any ideas on how to accomplish this using
R?
Cheers,
Paul
[[alternative HTML version deleted]]
______________________________________________
R-help at r-project.org<mailto:R-help at r-project.org> mailing list -- To
UNSUBSCRIBE and more, see
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-help&data=05%7C01%7CJSorkin%40som.umaryland.edu%7C34eca026294a401cee6e08dba0f3e0d0%7C717009a620de461a88940312a395cac9%7C0%7C0%7C638280741555924966%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aQ9ApqQ%2BTJfvErHnTy4196dqj%2FZ2ed4vjXp50%2F%2B8uRs%3D&reserved=0<https://stat.ethz.ch/mailman/listinfo/r-help>
PLEASE do read the posting guide
https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.r-project.org%2Fposting-guide.html&data=05%7C01%7CJSorkin%40som.umaryland.edu%7C34eca026294a401cee6e08dba0f3e0d0%7C717009a620de461a88940312a395cac9%7C0%7C0%7C638280741555924966%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=v4P23R9qpXODcANDWsY99dnMHDI7rpvi2SNFu0e%2B85I%3D&reserved=0<http://www.r-project.org/posting-guide.html>
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help at r-project.org<mailto:R-help at r-project.org> mailing list -- To
UNSUBSCRIBE and more, see
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-help&data=05%7C01%7CJSorkin%40som.umaryland.edu%7C34eca026294a401cee6e08dba0f3e0d0%7C717009a620de461a88940312a395cac9%7C0%7C0%7C638280741555924966%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aQ9ApqQ%2BTJfvErHnTy4196dqj%2FZ2ed4vjXp50%2F%2B8uRs%3D&reserved=0<https://stat.ethz.ch/mailman/listinfo/r-help>
PLEASE do read the posting guide
https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.r-project.org%2Fposting-guide.html&data=05%7C01%7CJSorkin%40som.umaryland.edu%7C34eca026294a401cee6e08dba0f3e0d0%7C717009a620de461a88940312a395cac9%7C0%7C0%7C638280741555924966%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=v4P23R9qpXODcANDWsY99dnMHDI7rpvi2SNFu0e%2B85I%3D&reserved=0<http://www.r-project.org/posting-guide.html>
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
John Fox
2023-Aug-19 22:48 UTC
[R] Determining Starting Values for Model Parameters in Nonlinear Regression
Dear John, John, and Paul,
In this case, one can start values by just fitting
> lm(1/y ~ x1 + x2 + x3 - 1, data=mydata)
Call:
lm(formula = 1/y ~ x1 + x2 + x3 - 1, data = mydata)
Coefficients:
x1 x2 x3
0.00629 0.00868 0.00803
Of course, the errors enter this model differently, so this isn't the
same as the nonlinear model, but the regression coefficients are very
close to the estimates for the nonlinear model.
Best,
John
--
John Fox, Professor Emeritus
McMaster University
Hamilton, Ontario, Canada
web: https://www.john-fox.ca/
On 2023-08-19 6:39 p.m., Sorkin, John wrote:> Caution: External email.
>
>
> Colleagues,
>
> At the risk of starting a forest fire, or perhaps a brush fire, while it is
good to see that nlxb can find a solution from arbitrary starting values, I
think Paul?s question has merit despite Professor Nash?s excellent and helpful
observation.
>
> Although non-linear algorithms can converge, they can converge to a false
solution if starting values are sub-optimally specified. When possible, I try to
specify thought-out starting values. Would it make sense to plot y as a function
of (x1, x2) at different values of x3 to get a sense of possible starting
values? Or, perhaps using median values of x1, x2, and x3 as starting values.
Comparing results from different starting values can give some confidence that
the solution obtained using arbitrary starting values are likely ?correct?.
>
> I freely admit that my experience (and thus expertise) using non-linear
solutions is limited. Please do not flame me, I am simply urging caution.
>
> John
>
> John David Sorkin M.D., Ph.D.
> Professor of Medicine
> Chief, Biostatistics and Informatics
> University of Maryland School of Medicine Division of Gerontology and
Geriatric Medicine
> Baltimore VA Medical Center
> 10 North Greene Street<x-apple-data-detectors://12>
> GRECC<x-apple-data-detectors://12> (BT/18/GR)
> Baltimore, MD 21201-1524<x-apple-data-detectors://13/0>
> (Phone) 410-605-711<tel:410-605-7119>9
> (Fax) 410-605-7913<tel:410-605-7913> (Please call phone number above
prior to faxing)
>
> On Aug 19, 2023, at 4:35 PM, J C Nash <profjcnash at
gmail.com<mailto:profjcnash at gmail.com>> wrote:
>
> Why bother. nlsr can find a solution from very crude start.
>
> Mixture <- c(17, 14, 5, 1, 11, 2, 16, 7, 19, 23, 20, 6, 13, 21, 3, 18,
15, 26, 8, 22)
> x1 <- c(69.98, 72.5, 77.6, 79.98, 74.98, 80.06, 69.98, 77.34, 69.99,
67.49, 67.51, 77.63,
> 72.5, 67.5, 80.1, 69.99, 72.49, 64.99, 75.02, 67.48)
> x2 <- c(29, 25.48, 21.38, 19.85, 22, 18.91, 29.99, 19.65, 26.99, 29.49,
32.47,
> 20.35, 26.48, 31.47, 16.87, 27.99, 24.49, 31.99, 24.96, 30.5)
> x3 <- c(1, 2, 1, 0, 3, 1, 0, 2.99, 3, 3, 0, 2, 1, 1, 3, 2,
> 3, 3, 0, 2)
> y <- c(1.4287, 1.4426, 1.4677, 1.4774, 1.4565,
> 1.4807, 1.4279, 1.4684, 1.4301, 1.4188, 1.4157, 1.4686, 1.4414,
> 1.4172, 1.4829, 1.4291, 1.4438, 1.4068, 1.4524, 1.4183)
> mydata<-data.frame(Mixture, x1, x2, x3, y)
> mydata
> mymod <- y ~ 1/(Beta1*x1 + Beta2*x2 + Beta3*x3)
> library(nlsr)
> strt<-c(Beta1=1, Beta2=2, Beta3=3)
> trysol<-nlxb(formula=mymod, data=mydata, start=strt, trace=TRUE)
> trysol
> # or pshort(trysol)
>
>
> Output is
>
> residual sumsquares = 1.5412e-05 on 20 observations
> after 29 Jacobian and 43 function evaluations
> name coeff SE tstat pval gradient
JSingval
> Beta1 0.00629212 5.997e-06 1049 2.425e-42 4.049e-08
721.8
> Beta2 0.00867741 1.608e-05 539.7 1.963e-37 -2.715e-08
56.05
> Beta3 0.00801948 8.809e-05 91.03 2.664e-24 1.497e-08
10.81
>
> J Nash
>
>
> On 2023-08-19 16:19, Paul Bernal wrote:
> Dear friends,
> Hope you are all doing well and having a great weekend. I have data that
> was collected on specific gravity and spectrophotometer analysis for 26
> mixtures of NG (nitroglycerine), TA (triacetin), and 2 NDPA (2 -
> nitrodiphenylamine).
> In the dataset, x1 = %NG, x2 = %TA, and x3 = %2 NDPA.
> The response variable is the specific gravity, and the rest of the
> variables are the predictors.
> This is the dataset:
> dput(mod14data_random)
> structure(list(Mixture = c(17, 14, 5, 1, 11, 2, 16, 7, 19, 23,
> 20, 6, 13, 21, 3, 18, 15, 26, 8, 22), x1 = c(69.98, 72.5, 77.6,
> 79.98, 74.98, 80.06, 69.98, 77.34, 69.99, 67.49, 67.51, 77.63,
> 72.5, 67.5, 80.1, 69.99, 72.49, 64.99, 75.02, 67.48), x2 = c(29,
> 25.48, 21.38, 19.85, 22, 18.91, 29.99, 19.65, 26.99, 29.49, 32.47,
> 20.35, 26.48, 31.47, 16.87, 27.99, 24.49, 31.99, 24.96, 30.5),
> x3 = c(1, 2, 1, 0, 3, 1, 0, 2.99, 3, 3, 0, 2, 1, 1, 3, 2,
> 3, 3, 0, 2), y = c(1.4287, 1.4426, 1.4677, 1.4774, 1.4565,
> 1.4807, 1.4279, 1.4684, 1.4301, 1.4188, 1.4157, 1.4686, 1.4414,
> 1.4172, 1.4829, 1.4291, 1.4438, 1.4068, 1.4524, 1.4183)), row.names
> c(NA,
> -20L), class = "data.frame")
> The model is the following:
> y = 1/(Beta1x1 + Beta2x2 + Beta3x3)
> I need to determine starting (initial) values for the model parameters for
> this nonlinear regression model, any ideas on how to accomplish this using
> R?
> Cheers,
> Paul
> [[alternative HTML version deleted]]
> ______________________________________________
> R-help at r-project.org<mailto:R-help at r-project.org> mailing list
-- To UNSUBSCRIBE and more, see
>
https://stat.ethz.ch/mailman/listinfo/r-help<https://stat.ethz.ch/mailman/listinfo/r-help>
> PLEASE do read the posting guide
http://www.r-project.org/posting-guide.html<http://www.r-project.org/posting-guide.html>
> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> R-help at r-project.org<mailto:R-help at r-project.org> mailing list
-- To UNSUBSCRIBE and more, see
>
https://stat.ethz.ch/mailman/listinfo/r-help<https://stat.ethz.ch/mailman/listinfo/r-help>
> PLEASE do read the posting guide
http://www.r-project.org/posting-guide.html<http://www.r-project.org/posting-guide.html>
> and provide commented, minimal, self-contained, reproducible code.
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.r-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.