Hi - In an effort to learn some basic arima modeling in R i went through the tutorial found at http://www.stat.pitt.edu/stoffer/tsa2/R_time_series_quick_fix.htm One of the examples gave me a log likelihood of 77. Now I am simply wondering if this is the expected behavior? Looking in my text book this should not be possible. I have actually spent some time on this but neither the documentation ?arima or google gave me a satisfying answer. Data and code: gTemp.raw = c(-0.11, -0.13, -0.01, -0.04, -0.42, -0.23, -0.25, -0.45, -0.23, 0.04, -0.22, -0.55 , -0.40, -0.39, -0.32, -0.32, -0.27, -0.15, -0.21, -0.25, -0.05, -0.05, -0.30, -0.35 , -0.42, -0.25, -0.15, -0.41, -0.30, -0.31, -0.21, -0.25, -0.33, -0.28, -0.02, 0.06 , -0.20, -0.46, -0.33, -0.09, -0.15, -0.04, -0.09, -0.16, -0.11, -0.15, 0.04, -0.05 , 0.01, -0.22, -0.03, 0.03, 0.04, -0.11, 0.05, -0.08, 0.01, 0.12, 0.15, -0.02 , 0.14, 0.11, 0.10, 0.06, 0.10, -0.01, 0.01, 0.12, -0.03, -0.09, -0.17, -0.02 , 0.03, 0.12, -0.09, -0.09, -0.18, 0.08, 0.10, 0.05, -0.02, 0.10, 0.05, 0.03 , -0.25, -0.15, -0.07, -0.02, -0.09, 0.00, 0.04, -0.10, -0.05, 0.18, -0.06, -0.02 , -0.21, 0.16, 0.07, 0.13, 0.27, 0.40, 0.10, 0.34, 0.16, 0.13, 0.19, 0.35 , 0.42, 0.28, 0.49, 0.44, 0.16, 0.18, 0.31, 0.47, 0.36, 0.40, 0.71, 0.43 , 0.41, 0.56, 0.70, 0.66, 0.60) gTemp.ts = ts(gTemp.raw, start=1880, freq=1) gTemp.model = arima(diff(gTemp.ts), order=c(1,0,1)) Results:> gTemp.modelCall: arima(x = diff(gTemp.ts), order = c(1, 0, 1)) Coefficients: ar1 ma1 intercept 0.2695 -0.8180 0.0061 s.e. 0.1122 0.0624 0.0030 sigma^2 estimated as 0.01680: log likelihood = 77.05, aic = -146.11
Likelihood is a function of the parameters, conditioned upon the data. It is not the same as a probability density function. Terms or factors which do not involve parameters can be omitted from the likelihood function. For continuous random variables, the density function can be in (0, Inf). Therefore, the likelihood function can assume any value between 0 and Inf. Hence the log-likelihood can be in (-Inf, Inf). When the random variable is discrete, the density or probability mass function cannot be greater than 1. Hence the likelihood cannot be greater than 1, in which case, the log-likelihood cannot be positive. Ravi. ____________________________________________________________________ Ravi Varadhan, Ph.D. Assistant Professor, Division of Geriatric Medicine and Gerontology School of Medicine Johns Hopkins University Ph. (410) 502-2619 email: rvaradhan at jhmi.edu ----- Original Message ----- From: Daniel Haugstvedt <daniel.haugstvedt at gmail.com> Date: Tuesday, October 5, 2010 9:16 am Subject: [R] subject: Log likelihood above 0 To: r-help at r-project.org> Hi - > > In an effort to learn some basic arima modeling in R i went through > the tutorial found at > > > One of the examples gave me a log likelihood of 77. Now I am simply > wondering if this is the expected behavior? Looking in my text book > this should not be possible. I have actually spent some time on this > but neither the documentation ?arima or google gave me a satisfying > answer. > > > > Data and code: > > gTemp.raw = c(-0.11, -0.13, -0.01, -0.04, -0.42, -0.23, -0.25, -0.45, > -0.23, 0.04, -0.22, -0.55 > , -0.40, -0.39, -0.32, -0.32, -0.27, -0.15, -0.21, -0.25, -0.05, > -0.05, -0.30, -0.35 > , -0.42, -0.25, -0.15, -0.41, -0.30, -0.31, -0.21, -0.25, -0.33, > -0.28, -0.02, 0.06 > , -0.20, -0.46, -0.33, -0.09, -0.15, -0.04, -0.09, -0.16, -0.11, > -0.15, 0.04, -0.05 > , 0.01, -0.22, -0.03, 0.03, 0.04, -0.11, 0.05, -0.08, 0.01, > 0.12, 0.15, -0.02 > , 0.14, 0.11, 0.10, 0.06, 0.10, -0.01, 0.01, 0.12, -0.03, > -0.09, -0.17, -0.02 > , 0.03, 0.12, -0.09, -0.09, -0.18, 0.08, 0.10, 0.05, -0.02, > 0.10, 0.05, 0.03 > , -0.25, -0.15, -0.07, -0.02, -0.09, 0.00, 0.04, -0.10, -0.05, > 0.18, -0.06, -0.02 > , -0.21, 0.16, 0.07, 0.13, 0.27, 0.40, 0.10, 0.34, 0.16, > 0.13, 0.19, 0.35 > , 0.42, 0.28, 0.49, 0.44, 0.16, 0.18, 0.31, 0.47, 0.36, > 0.40, 0.71, 0.43 > , 0.41, 0.56, 0.70, 0.66, 0.60) > > gTemp.ts = ts(gTemp.raw, start=1880, freq=1) > > gTemp.model = arima(diff(gTemp.ts), order=c(1,0,1)) > > > > Results: > > > gTemp.model > > Call: > arima(x = diff(gTemp.ts), order = c(1, 0, 1)) > > Coefficients: > ar1 ma1 intercept > 0.2695 -0.8180 0.0061 > s.e. 0.1122 0.0624 0.0030 > > sigma^2 estimated as 0.01680: log likelihood = 77.05, aic = -146.11 > > ______________________________________________ > R-help at r-project.org mailing list > > PLEASE do read the posting guide > and provide commented, minimal, self-contained, reproducible code.
On Oct 5, 2010, at 15:36 , Ravi Varadhan wrote:> Likelihood is a function of the parameters, conditioned upon the data. It is not the same as a probability density function. Terms or factors which do not involve parameters can be omitted from the likelihood function. For continuous random variables, the density function can be in (0, Inf). Therefore, the likelihood function can assume any value between 0 and Inf. Hence the log-likelihood can be in (-Inf, Inf). > > When the random variable is discrete, the density or probability mass function cannot be greater than 1. Hence the likelihood cannot be greater than 1, in which case, the log-likelihood cannot be positive....unless one of the above mentioned terms that do not involve parameters is omitted. E.g. the Poisson likelihood is x log lambda - lambda - log(x!) and the sum of the first two terms can easily be positive.> > Ravi. > ____________________________________________________________________ > > Ravi Varadhan, Ph.D. > Assistant Professor, > Division of Geriatric Medicine and Gerontology > School of Medicine > Johns Hopkins University > > Ph. (410) 502-2619 > email: rvaradhan at jhmi.edu > > > ----- Original Message ----- > From: Daniel Haugstvedt <daniel.haugstvedt at gmail.com> > Date: Tuesday, October 5, 2010 9:16 am > Subject: [R] subject: Log likelihood above 0 > To: r-help at r-project.org > > >> Hi - >> >> In an effort to learn some basic arima modeling in R i went through >> the tutorial found at >> >> >> One of the examples gave me a log likelihood of 77. Now I am simply >> wondering if this is the expected behavior? Looking in my text book >> this should not be possible. I have actually spent some time on this >> but neither the documentation ?arima or google gave me a satisfying >> answer. >> >> >> >> Data and code: >> >> gTemp.raw = c(-0.11, -0.13, -0.01, -0.04, -0.42, -0.23, -0.25, -0.45, >> -0.23, 0.04, -0.22, -0.55 >> , -0.40, -0.39, -0.32, -0.32, -0.27, -0.15, -0.21, -0.25, -0.05, >> -0.05, -0.30, -0.35 >> , -0.42, -0.25, -0.15, -0.41, -0.30, -0.31, -0.21, -0.25, -0.33, >> -0.28, -0.02, 0.06 >> , -0.20, -0.46, -0.33, -0.09, -0.15, -0.04, -0.09, -0.16, -0.11, >> -0.15, 0.04, -0.05 >> , 0.01, -0.22, -0.03, 0.03, 0.04, -0.11, 0.05, -0.08, 0.01, >> 0.12, 0.15, -0.02 >> , 0.14, 0.11, 0.10, 0.06, 0.10, -0.01, 0.01, 0.12, -0.03, >> -0.09, -0.17, -0.02 >> , 0.03, 0.12, -0.09, -0.09, -0.18, 0.08, 0.10, 0.05, -0.02, >> 0.10, 0.05, 0.03 >> , -0.25, -0.15, -0.07, -0.02, -0.09, 0.00, 0.04, -0.10, -0.05, >> 0.18, -0.06, -0.02 >> , -0.21, 0.16, 0.07, 0.13, 0.27, 0.40, 0.10, 0.34, 0.16, >> 0.13, 0.19, 0.35 >> , 0.42, 0.28, 0.49, 0.44, 0.16, 0.18, 0.31, 0.47, 0.36, >> 0.40, 0.71, 0.43 >> , 0.41, 0.56, 0.70, 0.66, 0.60) >> >> gTemp.ts = ts(gTemp.raw, start=1880, freq=1) >> >> gTemp.model = arima(diff(gTemp.ts), order=c(1,0,1)) >> >> >> >> Results: >> >>> gTemp.model >> >> Call: >> arima(x = diff(gTemp.ts), order = c(1, 0, 1)) >> >> Coefficients: >> ar1 ma1 intercept >> 0.2695 -0.8180 0.0061 >> s.e. 0.1122 0.0624 0.0030 >> >> sigma^2 estimated as 0.01680: log likelihood = 77.05, aic = -146.11 >> >> ______________________________________________ >> R-help at r-project.org mailing list >> >> PLEASE do read the posting guide >> and provide commented, minimal, self-contained, reproducible code. > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.-- Peter Dalgaard Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
Yes, of course! So, the complete answer is: the log-likelihood can be in (-Inf, Inf), regardless of whether the random variable is continuous or discrete or mixed. Ravi. ____________________________________________________________________ Ravi Varadhan, Ph.D. Assistant Professor, Division of Geriatric Medicine and Gerontology School of Medicine Johns Hopkins University Ph. (410) 502-2619 email: rvaradhan at jhmi.edu ----- Original Message ----- From: peter dalgaard <pdalgd at gmail.com> Date: Tuesday, October 5, 2010 9:49 am Subject: Re: [R] subject: Log likelihood above 0 To: Ravi Varadhan <rvaradhan at jhmi.edu> Cc: Daniel Haugstvedt <daniel.haugstvedt at gmail.com>, r-help at r-project.org> On Oct 5, 2010, at 15:36 , Ravi Varadhan wrote: > > > Likelihood is a function of the parameters, conditioned upon the > data. It is not the same as a probability density function. Terms or > factors which do not involve parameters can be omitted from the > likelihood function. For continuous random variables, the density > function can be in (0, Inf). Therefore, the likelihood function can > assume any value between 0 and Inf. Hence the log-likelihood can be > in (-Inf, Inf). > > > > When the random variable is discrete, the density or probability > mass function cannot be greater than 1. Hence the likelihood cannot > be greater than 1, in which case, the log-likelihood cannot be positive. > > ...unless one of the above mentioned terms that do not involve > parameters is omitted. E.g. the Poisson likelihood is > > x log lambda - lambda - log(x!) > > and the sum of the first two terms can easily be positive. > > > > > > Ravi. > > ____________________________________________________________________ > > > > Ravi Varadhan, Ph.D. > > Assistant Professor, > > Division of Geriatric Medicine and Gerontology > > School of Medicine > > Johns Hopkins University > > > > Ph. (410) 502-2619 > > email: rvaradhan at jhmi.edu > > > > > > ----- Original Message ----- > > From: Daniel Haugstvedt <daniel.haugstvedt at gmail.com> > > Date: Tuesday, October 5, 2010 9:16 am > > Subject: [R] subject: Log likelihood above 0 > > To: r-help at r-project.org > > > > > >> Hi - > >> > >> In an effort to learn some basic arima modeling in R i went through > >> the tutorial found at > >> > >> > >> One of the examples gave me a log likelihood of 77. Now I am simply > >> wondering if this is the expected behavior? Looking in my text book > >> this should not be possible. I have actually spent some time on this > >> but neither the documentation ?arima or google gave me a satisfying > >> answer. > >> > >> > >> > >> Data and code: > >> > >> gTemp.raw = c(-0.11, -0.13, -0.01, -0.04, -0.42, -0.23, -0.25, -0.45, > >> -0.23, 0.04, -0.22, -0.55 > >> , -0.40, -0.39, -0.32, -0.32, -0.27, -0.15, -0.21, -0.25, -0.05, > >> -0.05, -0.30, -0.35 > >> , -0.42, -0.25, -0.15, -0.41, -0.30, -0.31, -0.21, -0.25, -0.33, > >> -0.28, -0.02, 0.06 > >> , -0.20, -0.46, -0.33, -0.09, -0.15, -0.04, -0.09, -0.16, -0.11, > >> -0.15, 0.04, -0.05 > >> , 0.01, -0.22, -0.03, 0.03, 0.04, -0.11, 0.05, -0.08, 0.01, > >> 0.12, 0.15, -0.02 > >> , 0.14, 0.11, 0.10, 0.06, 0.10, -0.01, 0.01, 0.12, -0.03, > >> -0.09, -0.17, -0.02 > >> , 0.03, 0.12, -0.09, -0.09, -0.18, 0.08, 0.10, 0.05, -0.02, > >> 0.10, 0.05, 0.03 > >> , -0.25, -0.15, -0.07, -0.02, -0.09, 0.00, 0.04, -0.10, -0.05, > >> 0.18, -0.06, -0.02 > >> , -0.21, 0.16, 0.07, 0.13, 0.27, 0.40, 0.10, 0.34, 0.16, > >> 0.13, 0.19, 0.35 > >> , 0.42, 0.28, 0.49, 0.44, 0.16, 0.18, 0.31, 0.47, 0.36, > >> 0.40, 0.71, 0.43 > >> , 0.41, 0.56, 0.70, 0.66, 0.60) > >> > >> gTemp.ts = ts(gTemp.raw, start=1880, freq=1) > >> > >> gTemp.model = arima(diff(gTemp.ts), order=c(1,0,1)) > >> > >> > >> > >> Results: > >> > >>> gTemp.model > >> > >> Call: > >> arima(x = diff(gTemp.ts), order = c(1, 0, 1)) > >> > >> Coefficients: > >> ar1 ma1 intercept > >> 0.2695 -0.8180 0.0061 > >> s.e. 0.1122 0.0624 0.0030 > >> > >> sigma^2 estimated as 0.01680: log likelihood = 77.05, aic = -146.11 > >> > >> ______________________________________________ > >> R-help at r-project.org mailing list > >> > >> PLEASE do read the posting guide > >> and provide commented, minimal, self-contained, reproducible code. > > > > ______________________________________________ > > R-help at r-project.org mailing list > > > > PLEASE do read the posting guide > > and provide commented, minimal, self-contained, reproducible code. > > -- > Peter Dalgaard > Center for Statistics, Copenhagen Business School > Solbjerg Plads 3, 2000 Frederiksberg, Denmark > Phone: (+45)38153501 > Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com >