Econometrics 2 - Lecture 1
ML Estimation, Diagnostic Tests


Contents
nLinear Regression: A Review
nEstimation of Regression Parameters
nEstimation Concepts
nML Estimator: Idea and Illustrations
nML Estimator: Notation and Properties
nML Estimator: Two Examples
nAsymptotic Tests
nSome Diagnostic Tests
nQuasi-maximum Likelihood Estimator
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
2

The Linear Model
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
3
Y: explained variable
X: explanatory or regressor variable
The model describes the data-generating process of Y
      under the condition X
A simple linear regression model
     Y = a + bX
b: coefficient of X
a: intercept
A multiple linear regression model
     Y = b1 + b2X2 + … + bKXK

Fitting a Model to Data
nChoice of values b1, b2 for model parameters b1, b2 of Y = b1 + b2 X,
n    given the observations (yi, xi), i = 1,…,N
n
nFitted values: ŷi = b1 + b2 xi, i = 1,…,N
n
nPrinciple of (Ordinary) Least Squares gives the OLS estimators
n bi = arg minb1,b2 S(b1, b2), i=1,2
n
nObjective function: sum of the squared deviations
n S(b1, b2) = Si [yi - ŷi]2 = Si [yi - (b1 + b2xi)]2 = Si ei2
n
nDeviations between observation and fitted values, residuals:
n ei = yi - ŷi = yi - (b1 + b2xi)
n
n
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
4

Observations and Fitted Regression Line
n
nSimple linear regression: Fitted line and observation points (Verbeek, Figure 2.1)
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
5

Contents
nLinear Regression: A Review
nEstimation of Regression Parameters
nEstimation Concepts
nML Estimator: Idea and Illustrations
nML Estimator: Notation and Properties
nML Estimator: Two Examples
nAsymptotic Tests
nSome Diagnostic Tests
nQuasi-maximum Likelihood Estimator
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
6

OLS Estimators
nOLS estimators b1 und b2 result in
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
7
with mean values     and
and second moments
Equating the partial derivatives of S(b1, b2) to zero: normal equations

OLS Estimators: The General Case
nModel for Y contains K-1 explanatory variables
n Y = b1 + b2X2 + … + bKXK = x’b
n with x = (1, X2, …, XK)’ and b = (b1, b2, …, bK)’
nObservations: [yi, xi] = [yi, (1, xi2, …, xiK)’], i = 1, …, N
nOLS-estimates b = (b1, b2, …, bK)’ are obtained by minimizing
n
n
n this results in the OLS estimators
n
n
n
n
n
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
8

Matrix Notation
nN observations
n (y1,x1), … , (yN,xN)
nModel: yi = b1 + b2xi + εi, i  = 1, …,N, or
n y = Xb + ε
n with
n
n
n
n
nOLS estimators
n b = (X’X)-1X’y
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
9

Gauss-Markov Assumptions
A1
E{εi} = 0 for all i
A2
all εi are independent of all xi (exogenous xi)
A3
V{ei} = s2 for all i (homoskedasticity)
A4
Cov{εi, εj} = 0 for all i and j with i ≠ j (no autocorrelation)
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
10
Observation yi (i = 1, …, N) is a linear function
yi = xi'b + εi
of observations xik, k =1, …, K, of the regressor variables and the error term εi
xi = (xi1, …, xiK)'; X = (xik)
n

Normality of Error Terms
n
n
nTogether with assumptions (A1), (A3), and (A4), (A5) implies
n εi ~ NID(0,σ2) for all i
n i.e., all εi are
qindependent drawings
qfrom the normal distribution N(0,σ2)
qwith mean 0
qand variance σ2
nError terms are “normally and independently distributed” (NID, n.i.d.)
n
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
11
A5
εi  normally distributed for all i

Properties of OLS Estimators
nOLS estimator b = (X’X)-1X’y
n1. The OLS estimator b is unbiased: E{b} = β
n2. The variance of the OLS estimator is given by
n V{b} = σ2(Σi xi xi’ )-1
n3. The OLS estimator b is a BLUE (best linear unbiased estimator) for β
n4. The OLS estimator b is normally distributed with mean β and covariance matrix V{b} =
σ2(Σi xi xi’ )-1
nProperties
n1., 2., and 3. follow from Gauss-Markov assumptions
n4. needs in addition the normality assumption (A5)
n
n
n
n
n
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
12

Distribution of t-statistic
nt-statistic
n
n
nfollows
1.the t-distribution with N-K d.f. if the Gauss-Markov assumptions (A1) - (A4) and the normality
assumption (A5) hold
2.approximately the t-distribution with N-K d.f. if the Gauss-Markov assumptions (A1) - (A4) hold
but not the normality assumption (A5)
3.asymptotically (N → ∞) the standard normal distribution N(0,1)
4.approximately the standard normal distribution N(0,1)
nThe approximation errors decrease with increasing sample size N
n
n
n
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
13

OLS Estimators: Consistency
nThe OLS estimators b are consistent,
n plimN → ∞ b = β,
n if one of the two set of conditions are fulfilled:
n(A2) from the Gauss-Markov assumptions and the assumption (A6), or
nthe assumption (A7), weaker than (A2), and the assumption (A6)
nAssumptions (A6) and (A7):
n
n
n
n
n
nAssumption (A7) is weaker than assumption (A2)!
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
14
A6
1/N ΣNi=1 xi xi’ converges with growing N to a finite, nonsingular matrix Σxx
A7
The error terms have zero mean and are uncorrelated with each of the regressors: E{xi εi} = 0

Contents
nLinear Regression: A Review
nEstimation of Regression Parameters
nEstimation Concepts
nML Estimator: Idea and Illustrations
nML Estimator: Notation and Properties
nML Estimator: Two Examples
nAsymptotic Tests
nSome Diagnostic Tests
nQuasi-maximum Likelihood Estimator
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
15

Estimation Concepts
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
16
OLS estimator: minimization of objective function S(b) gives
nK first-order conditions Si (yi – xi’b) xi = Si ei xi = 0, the normal equations
nMoment conditions
 E{(yi – xi’ b) xi} = E{ei xi} = 0
nOLS estimators are solution of the normal equations
IV estimator: Model allows derivation of moment conditions
E{(yi – xi’ b) zi} = E{ei zi} = 0
which are functions of
nobservable variables yi, xi, instrument variables zi, and unknown parameters b
nMoment conditions are used for deriving IV estimators
nOLS estimators are special case of IV estimators
n
•

Estimation Concepts, cont’d
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
17
GMM estimator: generalization of the moment conditions
E{f(wi, zi, b)} = 0
nwith observable variables wi, instrument variables zi, and unknown parameters b; f:
multidimensional function with as many components as conditions
nAllows for non-linear models
nUnder weak regularity conditions, the GMM estimators are
qconsistent
qasymptotically normal
Maximum likelihood estimation
nBasis is the distribution of yi  conditional on regressors xi
nDepends on unknown parameters b
nThe estimates of the parameters b are chosen so that the distribution corresponds as well as
possible to the observations yi and xi
•

Contents
nLinear Regression: A Review
nEstimation of Regression Parameters
nEstimation Concepts
nML Estimator: Idea and Illustrations
nML Estimator: Notation and Properties
nML Estimator: Two Examples
nAsymptotic Tests
nSome Diagnostic Tests
nQuasi-maximum Likelihood Estimator
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
18

Example: Urn Experiment
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
19
Urn experiment:
nThe urn contains red and black balls
nProportion of red balls: p (unknown)
nN  random draws
nRandom draw i: yi = 1 if ball i is red, 0 otherwise; P{yi = 1} = p
nSample: N1 red balls, N-N1 black balls
nProbability for this result:
   P{N1 red balls, N-N1 black balls} = pN1 (1 – p)N-N1
Likelihood function: the probability of the sample result, interpreted as a function of the unknown
parameter p

Hackl, Econometrics 2, Lecture 1
20
Urn Experiment: Likelihood Function
nLikelihood function: the probability of the sample result, interpreted as a function of the
unknown parameter p
n    L(p) = pN1 (1 – p)N-N1
nMaximum likelihood estimator: that value     of p which maximizes L(p)
n
nCalculation of    : maximization algorithms
nAs the log-function is monotonous, extremes of L(p) and log L(p) coincide
nUse of log-likelihood function is often more convenient
n    log L(p) = N1 log p + (N - N1) log (1 – p)
n
n
Feb 22, 2013

Urn Experiment: Likelihood Function, cont’d
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
21
Verbeek, Fig.6.1

Hackl, Econometrics 2, Lecture 1
22
Urn Experiment: ML Estimator
nMaximizing log L(p) with respect to p gives the first-order condition
n
n
nSolving this equation for p gives the maximum likelihood estimator (ML estimator)
n
n
nFor N = 100, N1 = 44, the ML estimator for the proportion of red balls is     = 0.44
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
23
Maximum Likelihood Estimator: The Idea
nSpecify the distribution of the data (of y or y given x)
nDetermine the likelihood of observing the available sample as a function of the unknown parameters
nChoose as ML estimates those values for the unknown parameters that give the highest likelihood
nIn general, this leads to
qconsistent
qasymptotically normal
qefficient estimators
n provided the likelihood function is correctly specified, i.e., distributional assumptions are
correct
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
24
Example: Normal Linear Regression
nModel
n yi = β1 + β2xi + εi
n with assumptions (A1) – (A5)
nFrom the normal distribution of εi follows: contribution of  observation i  to the likelihood
function:
n
n
ndue to independent observations, the log-likelihood function is given by
n
n
n
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
25
Normal Linear Regression, cont’d
nMaximizing log L with respect to β and σ2 gives the ML estimators
n
n
n
n which coincide with the OLS estimators, and
n
n
n
n which is biased and underestimates σ²!
nRemarks:
nThe results are obtained assuming normally and independently distributed (NID) error terms
nML estimators are consistent but not necessarily unbiased; see the properties of ML estimators
below
n
Feb 22, 2013

Contents
nLinear Regression: A Review
nEstimation of Regression Parameters
nEstimation Concepts
nML Estimator: Idea and Illustrations
nML Estimator: Notation and Properties
nML Estimator: Two Examples
nAsymptotic Tests
nSome Diagnostic Tests
nQuasi-maximum Likelihood Estimator
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
26

ML Estimator: Notation
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
27
Let the density (or probability mass function) of yi, given xi, be given by f(yi|xi,θ) with
K-dimensional vector θ of unknown parameters
Given independent observations, the likelihood function for the sample of size N is
The ML estimators are the solutions of
maxθ log L(θ) = maxθ Σi log Li(θ)
or the solutions of the first-order conditions
s(θ) = Σi si(θ), the vector of gradients, also denoted as score vector
Solution of s(θ) = 0
§analytically (see examples above) or
§by use of numerical optimization algorithms

Matrix Derivatives
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
28
The scalar-valued function
or – shortly written as log L(θ) – has the K arguments θ1, …, θK
§K-vector of partial derivatives or gradient vector or gradient
§
§
§KxK matrix of second derivatives or Hessian matrix

ML Estimator: Properties
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
29
The ML estimator
1.is consistent
2.is asymptotically efficient
3.is asymptotically normally distributed:
4.
  V: asymptotic covariance matrix of

The Information Matrix
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
30
Information matrix I(θ)
§I(θ) is the limit (for N → ∞) of
§
§
§For the asymptotic covariance matrix V can be shown: V = I(θ)-1
§I(θ)-1 is the lower bound of the asymptotic covariance matrix for any consistent, asymptotically
normal estimator for θ: Cramèr-Rao lower bound
Calculation of Ii(θ) can also be based on the outer product of the score vector
for a miss-specified likelihood function, Ji(θ) can deviate from Ii(θ)

Hackl, Econometrics 2, Lecture 1
31
Covariance Matrix V: Calculation
nTwo ways to calculate V:
nA consistent estimate is based on the information matrix I(θ):
n
n
n
n index “H”: the estimate of V is based on the Hessian matrix
nThe BHHH (Berndt, Hall, Hall, Hausman) estimator
n
n
n
n with score vector s(θ); index “G”: the estimate of V is based on gradients
qalso called: OPG (outer product of gradient) estimator
qE{si(θ) si(θ)’} coincides with Ii(θ) if f(yi| xi,θ) is correctly specified
n
n
n
n
Feb 22, 2013

Contents
nLinear Regression: A Review
nEstimation of Regression Parameters
nEstimation Concepts
nML Estimator: Idea and Illustrations
nML Estimator: Notation and Properties
nML Estimator: Two Examples
nAsymptotic Tests
nSome Diagnostic Tests
nQuasi-maximum Likelihood Estimator
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
32

Hackl, Econometrics 2, Lecture 1
33
Urn Experiment: Once more
nLikelihood contribution of the i-th observation
n log Li(p) = yi log p + (1 - yi) log (1 – p)
nThis gives scores
n
n
n and
n
n
n
nWith E{yi} = p, the expected value turns out to be
n
n
n
nThe asymptotic variance of the ML estimator V = I-1 = p(1-p)
Feb 22, 2013

Urn Experiment and Binomial Distribution
nThe asymptotic distribution is
n
nSmall sample distribution:
n             N     ~ B(N, p)
nUse of the approximate normal distribution for portions
n rule of thumb:
n N p (1-p) > 9
nTest of H0: p = p0 can be based on test statistic
n
n
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
34

Hackl, Econometrics 2, Lecture 1
35
Example: Normal Linear Regression
nModel
n yi = xi’β + εi
n with assumptions (A1) – (A5)
nLog-likelihood function
n
n
nScore contributions:
n
n
n
n
nThe first-order conditions – setting both components of Σisi(β,σ²) to zero – give as ML
estimators: the OLS estimator for β, the average squared residuals for σ²:
n
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
36
Normal Linear Regression, cont’d
n
n
nAsymptotic covariance matrix: Contribution of the i-th observation (E{εi} = E{εi3} = 0, E{εi2} =
σ², E{εi4} = 3σ4)
n
n gives
n V = I(β,σ²)-1 = diag (σ²Σxx-1, 2σ4)
n with Σxx = lim (Σixixi‘)/N
nThe ML estimate for β and σ² follow asymptotically
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
37
Normal Linear Regression, cont’d
nFor finite samples: covariance matrix of ML estimators for β
n
n similar to OLS results
Feb 22, 2013

Contents
nLinear Regression: A Review
nEstimation of Regression Parameters
nEstimation Concepts
nML Estimator: Idea and Illustrations
nML Estimator: Notation and Properties
nML Estimator: Two Examples
nAsymptotic Tests
nSome Diagnostic Tests
nQuasi-maximum Likelihood Estimator
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
38

Hackl, Econometrics 2, Lecture 1
39
Diagnostic Tests
nDiagnostic (or specification) tests based on ML estimators
nTest situation:
nK-dimensional parameter vector θ = (θ1, …, θK)’
nJ  ≥ 1 linear restrictions (K ≥ J)
nH0: R θ = q with JxK matrix R, full rank; J-vector q
nTest principles based on the likelihood function:
1.Wald test: Checks whether the restrictions are fulfilled for the unrestricted ML estimator for θ;
test statistic ξW
2.Likelihood ratio test: Checks whether the difference between the log-likelihood values with and
without the restriction is close to zero; test statistic ξLR
3.Lagrange multiplier test (or score test): Checks whether the first-order conditions (of the
unrestricted model) are violated by the restricted ML estimators; test statistic ξLM
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
40
Likelihood and Test Statistics
teststat
g(b) = 0: restriction
log L: log-likelihood
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
41
The Asymptotic Tests
nUnder H0, the test statistics of all three tests
nfollow asymptotically, for finite sample size approximately, the Chi-square distribution with J df
nThe tests are asymptotically (large N) equivalent
nFinite sample size: the values of the test statistics obey the relation
n ξW ≥ ξLR ≥ ξLM
nChoice of the test: criterion is computational effort
1.Wald test: Requires estimation only of the unrestricted model; e.g., testing for omitted
regressors: estimate the full model, test whether the coefficients of potentially omitted
regressors are different from zero
2.Lagrange multiplier test: Requires estimation only of the restricted model; preferable if
restrictions complicate estimation
3.Likelihood ratio test: Requires estimation of both the restricted  and the unrestricted model
4.
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
42
Wald Test
nChecks whether the restrictions are fulfilled for the unrestricted ML estimator for θ
nAsymptotic distribution of the unrestricted ML estimator:
n
nHence, under H0: R θ = q,
n
n
nThe test statistic
n
n
qunder H0, ξW is expected to be close to zero
qp-value to be read from the Chi-square distribution with J df
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
43
Wald Test, cont’d
nTypical application: tests of linear restrictions for regression coefficients
nTest of H0: βi = 0
n ξW = bi2/[se(bi)2]
qξW follows the Chi-square distribution with 1 df
qξW is the square of the t-test statistic
nTest of the null-hypothesis that a subset of J of the coefficients β are zeros
n ξW = (eR’eR – e’e)/[e’e/(N-K)]
qe: residuals from unrestricted model
qeR: residuals from restricted model
qξW follows the Chi-square distribution with J df
qξW is related to the F-test statistic by ξW = FJ
q
q
n
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
44
Likelihood Ratio Test
nChecks whether the difference between the ML estimates obtained with and without the restriction
is close to zero
n for nested models
nUnrestricted ML estimator:
nRestricted ML estimator:    ; obtained by minimizing the log-likelihood subject to R θ = q
nUnder H0: R θ = q, the test statistic
n
n
qis expected to be close to zero
qp-value to be read from the Chi-square distribution with J df
n
n
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
45
Likelihood Ratio Test, cont’d
nTest of linear restrictions for regression coefficients
nTest of the null-hypothesis that J linear restrictions of the coefficients β are valid
n ξLR = N log(eR’eR/e’e)
qe: residuals from unrestricted model
qeR: residuals from restricted model
qξLR follows the Chi-square distribution with J df
nRequires that the restricted model is nested within the unrestricted model
n
n
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
46
Lagrange Multiplier Test
nChecks whether the derivative of the likelihood for the constrained ML estimator is close to zero
nBased on the Lagrange constrained maximization method
nLagrangian, given θ = (θ1’, θ2’)’ with restriction θ2 = q, J-vectors θ2, q
n H(θ, λ) = Σi log L i(θ) – λ‘(θ-q)
nFirst-order conditions give the constrained ML estimators
n and
n
n
n
n
nλ measures the extent of violation of the restriction, basis for ξLM
n si are the scores; LM test is also called score test
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
47
Lagrange Multiplier Test, cont’d
nFor     can be shown that           follows asymptotically the normal distribution N(0,Vλ)  with
n
n i.e., the lower block diagonal of the inverted information matrix
n
n
n
nThe Lagrange multiplier test statistic
n
n has under H0 an asymptotic Chi-square distribution with J df
n    is the block diagonal of the estimated inverted information
n     matrix, based on the constrained estimators for θ
n
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
48
Calculation of the LM Test Statistic
nOuter product gradient (OPG) of ξLM
nNxK  matrix of first derivatives S’ = [s1(  ), …, sN(  )]
n    can be calculated as
nInformation matrix
n
nTherefore
n
nAuxiliary regression of a N-vector i = (1, …, 1)’ on the scores si(  ),  i.e., on the columns of
S; no intercept
nPredicted values from auxiliary regression: S(S’S)-1S’i
nExplained  sum of squares: i’S(S’S)-1S’S(S’S)-1S’i = i’S(S’S)-1S’i
nLM test statistic ξLM = N R² with the uncentered R² of the auxiliary regression; cf. total sum of
squares i’i
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
49
An Illustration
nThe urn experiment: test of H0: p = p0 (J = 1, R = I)
nThe likelihood contribution of the i-th observation is
n log Li(p) = yi log p + (1 - yi) log (1 – p)
nThis gives
n si(p) = yi/p – (1-yi)/(1-p) and Ii(p) = [p(1-p)]-1
nWald test:
n
nLikelihood ratio test:
n
n with
n
n
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
50
An Illustration, cont’d
nLagrange multiplier test:
n with
n
n
n and the inverted information matrix [I(p)]-1 = p(1-p), calculated for the restricted case, the LM
test statistic is
n
n
n
nExample
nIn a sample of N = 100 balls, 44 are red
nH0: p0 = 0.5
nξW = 1.46, ξLR = 1.44, ξLM = 1.44
nCorresponding p-values are 0.227, 0.230, and 0.230
n
Feb 22, 2013

Contents
nLinear Regression: A Review
nEstimation of Regression Parameters
nEstimation Concepts
nML Estimator: Idea and Illustrations
nML Estimator: Notation and Properties
nML Estimator: Two Examples
nAsymptotic Tests
nSome Diagnostic Tests
nQuasi-maximum Likelihood Estimator
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
51

Hackl, Econometrics 2, Lecture 1
52
Normal Linear Regression: Scores
nLog-likelihood function
n
n
nScores:
n
n
n
n
n
nCovariance matrix
n V = I(β,σ²)-1 = diag(σ²Σxx-1, 2σ4)
n
n
n
n
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
53
Testing for Omitted Regressors
nModel: yi = xi’β + zi’γ + εi, εi ~ NID(0,σ²)
nTest whether the J regressors zi are erroneously omitted:
nFit the restricted model
nApply the LM test to check H0: γ = 0
nFirst-order conditions give the scores
n
n
n with constrained ML estimators for β and σ²; ML-residuals
nAuxiliary regression of N-vector i = (1, …, 1)’ on the scores gives the uncentered R²
nThe LM test statistic is ξLM = N R²
nAn asymptotically equivalent LM test statistic is NRe² with Re² from the regression of the ML
residuals on xi and zi
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
54
Testing for Heteroskedasticity
nModel: yi = xi’β + εi, εi ~ NID, V{εi} = σ² h(zi’α), h(.) > 0 but unknown, h(0) = 1, ∂/∂α{h(.)} �
0, J-vector zi
nTest for homoskedasticity: Apply the LM test to check H0: α = 0
nFirst-order conditions with respect to σ² and α give the scores
n
n with constrained ML estimators for β and σ²; ML-residuals
nAuxiliary regression of N-vector i = (1, …, 1)’ on the scores gives the uncentered R²
nLM test statistic ξLM = NR²; a version of Breusch-Pagan test
nAn asymptotically equivalent version of the Breusch-Pagan test is based on NRe² with Re² from the
regression of the squared ML residuals on zi and an intercept
nAttention: no effect of the functional form of h(.)
n
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
55
Testing for Autocorrelation
nModel: yt = xt’β + εt, εt = ρεt-1 + vt, vt ~ NID(0,σ²)
nLM test of H0: ρ = 0
nFirst-order conditions give the scores
n
n with constrained ML estimators for β and σ²
nThe LM test statistic is ξLM = (T-1) R² with R² from the auxiliary regression of the N-vector i =
(1,…,1)’ on the scores
nIf xt contains no lagged dependent variables: products with xt can be dropped from the regressors
nAn asymptotically equivalent test is the Breusch-Godfrey test based on NRe² with Re² from the
regression of the ML residuals on xt and the lagged residuals
n
Feb 22, 2013

Contents
nLinear Regression: A Review
nEstimation of Regression Parameters
nEstimation Concepts
nML Estimator: Idea and Illustrations
nML Estimator: Notation and Properties
nML Estimator: Two Examples
nAsymptotic Tests
nSome Diagnostic Tests
nQuasi-maximum Likelihood Estimator
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
56

Hackl, Econometrics 2, Lecture 1
57
Quasi ML Estimator
nThe quasi-maximum likelihood estimator
nrefers to moment conditions
ndoes not refer to the entire distribution
nuses the GMM concept
nDerivation of the ML estimator as a GMM estimator
nweaker conditions
nconsistency applies
n
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
58
Generalized Method of Moments (GMM)
nThe model is characterized by R moment conditions
n E{f(wi, zi, θ)} = 0
qf(.): R-vector function
qwi: vector of observable variables, zi: vector of instrument variables
qθ: K-vector of unknown parameters
nSubstitution of the moment conditions by sample equivalents:
n gN(θ) = (1/N) Σi f(wi, zi, θ) = 0
nMinimization wrt θ of the quadratic form
n QN(θ) = gN(θ)‘ WN gN(θ)
n with the symmetric, positive definite weighting matrix WN gives the GMM estimator
n
n
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
59
Quasi-ML Estimator
nThe quasi-maximum likelihood estimator
nrefers to moment conditions
ndoes not refer to the entire distribution
nuses the GMM concept
nML estimator can be interpreted as GMM estimator: first-order conditions
n
n correspond to sample averages based on theoretical moment conditions
nStarting point is
n E{si(θ)} = 0
n valid for the K-vector θ if the likelihood is correctly specified
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
60
E{si(θ)} = 0
nFrom ∫f(yi|xi;θ) dyi = 1 follows
n
n
nTransformation
n
n
n gives
n
nThis  theoretical moment for the scores is valid for any density f(.)
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
61
Quasi-ML Estimator, cont’d
nUse of the GMM idea – substitution of moment conditions by sample equivalents – suggests to
transform E{si(θ)} = 0 into its sample equivalent and solve the first-order conditions
n
n
nThis reproduces the ML estimator
nExample: For the linear regression yi = xi’β + εi, application of the Quasi-ML concept starts from
the sample equivalents of
n E{(yi - xi’β) xi} = 0
n this corresponds to the moment conditions of the OLS and the first-order condition of the ML
estimators
qdoes not depend of the normality assumption of εi!
Feb 22, 2013

Hackl, Econometrics 2, Lecture 1
62
Quasi-ML Estimator, cont’d
nCan be based on a wrong likelihood assumption
nConsistency is due to starting out from E{si(θ)} = 0
nHence, “quasi-ML” (or “pseudo ML”) estimator
nAsymptotic distribution:
nMay differ from that of the ML estimator:
n
nUsing the asymptotic distribution of the GMM estimator gives
n
n
n with J(θ) = lim (1/N)ΣiE{si(θ) si(θ)’}
n and I(θ) = lim (1/N)ΣiE{-∂si(θ)/∂θ’}
nFor linear regression: heteroskedasticity-consistent covariance matrix
Feb 22, 2013

Your Homework
1.Open the Greene sample file “greene7_8, Gasoline price and consumption”, offered within the Gretl
system. The variables to be used in the following are: G = total U.S. gasoline consumption,
computed as total expenditure divided by price index; Pg = price index for gasoline; Y = per capita
disposable income; Pnc = price index for new cars; Puc = price index for used cars; Pop = U.S.
total population in millions. Perform the following analyses and interpret the results:
a.Produce and interpret the scatter plot of the per capita (p.c.) gasoline consumption (Gpc) over
the p.c. disposable income.
b.Fit the linear regression for log(Gpc) with regressors log(Y), Pg, Pnc and Puc to the data and
give an interpretation of the outcome.
q
1.
1.
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
63

Your Homework, cont’d
c.Test for autocorrelation of the error terms using the LM test statistic ξLM = (T-1) R² with R²
from the auxiliary regression of the ML residuals on the lagged residuals with appropriately chosen
lags.
d.Test for autocorrelation using NRe² with Re² from the regression of the ML residuals on xt and
the lagged residuals.
2.Assume that the errors εt of the linear regression yt = β1 + β2xt + εt are NID(0, σ2)
distributed. (a) Determine the log-likelihood function of the sample for t = 1, …,T; (b) show that
the first-order conditions for the ML estimators have expectations zero for the true parameter
values; (c) derive the asymptotic covariance matrix on the basis (i) of the information matrix and
(ii) of the score vector; (d) derive the matrix S of scores for the omitted variable LM test [cf.
eq. (6.38) in Veebeek].
Feb 22, 2013
Hackl, Econometrics 2, Lecture 1
64