[R] Orthogonal polynomials used by R

Fox, John
Wed, 27 Nov 2019 17:11:04 -0000

Dear Ashim,

Orthogonal polynomials are used because they tend to produce more accurate
numerical computations, not because their coefficients are interpretable, so I
wonder why you're interested in the coefficients.

The regressors produced are orthogonal to the constant regressor and are
orthogonal to each other (and in fact are orthonormal), as it's simple to
demonstrate:

------- snip -------

> x <- 1:93
> y <- 1 + x + x^2 + x^3 + x^4 + rnorm(93)
> (m <- lm(y ~ poly(x, 4)))

Call:
lm(formula = y ~ poly(x, 4))

Coefficients:
(Intercept)  poly(x, 4)1  poly(x, 4)2  poly(x, 4)3  poly(x, 4)4
   15574516    172715069     94769949     27683528      3429259

> X <- model.matrix(m)
> head(X)
  (Intercept) poly(x, 4)1 poly(x, 4)2 poly(x, 4)3 poly(x, 4)4
1           1  -0.1776843   0.2245083  -0.2572066  0.27935949
2           1  -0.1738216   0.2098665  -0.2236579  0.21862917
3           1  -0.1699589   0.1955464  -0.1919525  0.16390514
4           1  -0.1660962   0.1815482  -0.1620496  0.11487597
5           1  -0.1622335   0.1678717  -0.1339080  0.07123722
6           1  -0.1583708   0.1545171  -0.1074869  0.03269145

> zapsmall(crossprod(X))# X'X
            (Intercept) poly(x, 4)1 poly(x, 4)2 poly(x, 4)3 poly(x, 4)4
(Intercept)          93           0           0           0           0
poly(x, 4)1           0           1           0           0           0
poly(x, 4)2           0           0           1           0           0
poly(x, 4)3           0           0           0           1           0
poly(x, 4)4           0           0           0           0           1

------- snip -------

If for some not immediately obvious reason you're interested in the
regression coefficients, why not just use a "raw" polynomial:

------- snip -------

> (m1 <- lm(y ~ poly(x, 4, raw=TRUE)))

Call:
lm(formula = y ~ poly(x, 4, raw = TRUE))

Coefficients:
            (Intercept)  poly(x, 4, raw = TRUE)1  poly(x, 4, raw = TRUE)2
poly(x, 4, raw = TRUE)3
                 1.5640                   0.8985                   1.0037
1.0000
poly(x, 4, raw = TRUE)4
                 1.0000

------- snip -------

These coefficients are simply interpretable but the model matrix is more poorly
conditioned:

------- snip -------

> head(X1)
  (Intercept) poly(x, 4, raw = TRUE)1 poly(x, 4, raw = TRUE)2 poly(x, 4, raw =
TRUE)3
1           1                       1                       1
1
2           1                       2                       4
8
3           1                       3                       9
27
4           1                       4                      16
64
5           1                       5                      25
125
6           1                       6                      36
216
  poly(x, 4, raw = TRUE)4
1                       1
2                      16
3                      81
4                     256
5                     625
6                    1296
> round(cor(X1[, -1]), 2)
                        poly(x, 4, raw = TRUE)1 poly(x, 4, raw = TRUE)2 poly(x,
4, raw = TRUE)3
poly(x, 4, raw = TRUE)1                    1.00                    0.97
0.92
poly(x, 4, raw = TRUE)2                    0.97                    1.00
0.99
poly(x, 4, raw = TRUE)3                    0.92                    0.99
1.00
poly(x, 4, raw = TRUE)4                    0.87                    0.96
0.99
                        poly(x, 4, raw = TRUE)4
poly(x, 4, raw = TRUE)1                    0.87
poly(x, 4, raw = TRUE)2                    0.96
poly(x, 4, raw = TRUE)3                    0.99
poly(x, 4, raw = TRUE)4                    1.00

------- snip -------

The two parametrizations are equivalent, however, in that they represent the
same regression surface, and so, e.g., produce the same fitted values:

------- snip -------

> all.equal(fitted(m), fitted(m1))
[1] TRUE

------- snip -------

Because one is usually not interested in the individual coefficients of a
polynomial there usually isn't a reason to prefer one parametrization to the
other on the grounds of interpretability, so why do you need to interpret the
regression equation?

I hope this helps,
 John

  -----------------------------
  John Fox, Professor Emeritus
  McMaster University
  Hamilton, Ontario, Canada
  Web: http::/socserv.mcmaster.ca/jfox

> On Nov 27, 2019, at 10:17 AM, Ashim Kapoor <ashimkapoor at gmail.com>
wrote:
>
> Dear Petr,
>
> Many thanks for the quick response.
>
> I also read this:-
> https://en.wikipedia.org/wiki/Discrete_orthogonal_polynomials
>
> Also I read  in ?poly:-
>     The orthogonal polynomial is summarized by the coefficients, which
>     can be used to evaluate it via the three-term recursion given in
>     Kennedy & Gentle (1980, pp. 343-4), and used in the ?predict? part
>     of the code.
>
> I don't have access to the mentioned book.
>
> Out of curiosity, what is the name of the discrete orthogonal polynomial
> used by R ?
> What discrete measure is it orthogonal with respect to ?
>
> Many thanks,
> Ashim
>
>
>
>
> On Wed, Nov 27, 2019 at 6:11 PM PIKAL Petr <petr.pikal at
precheza.cz> wrote:
>
>> You could get answer quickly by searching net.
>>
>>
>>
https://stackoverflow.com/questions/39031172/how-poly-generates-orthogonal-p
>> olynomials-how-to-understand-the-coefs-ret/39051154#39051154
>>
<https://stackoverflow.com/questions/39031172/how-poly-generates-orthogonal-polynomials-how-to-understand-the-coefs-ret/39051154#39051154>
>>
>> Cheers
>> Petr
>>
>>> -----Original Message-----
>>> From: R-help <r-help-bounces at r-project.org> On Behalf Of
Ashim Kapoor
>>> Sent: Wednesday, November 27, 2019 12:55 PM
>>> To: R Help <r-help at r-project.org>
>>> Subject: [R] Orthogonal polynomials used by R
>>>
>>> Dear All,
>>>
>>> I have created a time trend by doing x<-1:93 because I have a
time series
>>> with 93 data points. Next I did :-
>>>
>>> y = lm(series ~ poly(x,4))$residuals
>>>
>>> to detrend series.
>>>
>>> I choose this 4 as the order of my polynomial using cross
validation/
>>> checking the absence of trend in the residuals so I think I have
not
>> overfit
>>> this series.
>>>
>>> I wish to document the formula of poly(x,4). I am not able to find
it in
>> ?poly
>>>
>>> Can someone please tell me what the formula for the orthogonal
>>> polynomial used by R is ?
>>>
>>> Thank you,
>>> Ashim
>>>
>>>      [[alternative HTML version deleted]]
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more,
see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-
>>> guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.