Nls

Non-linear Regression

Walter Sosa-Escudero

Econ 507. Econometric Analysis. Spring 2009

April 23, 2009

Walter Sosa-Escudero Non-linear Regression

The Model

E(y|x) = x(β)

where x(β) is a possibly non-linear function <k → < indexed by Kpparametros β.

Example:x(β) = β1x

β2


Example: AR(1) autocorrelation.

Yt = β1 + β2Xt + ut, t = 1, . . . , Tut = φut−1 + εt, |φ| < 1

Since the model is valid for every period, the two followingstatements hold:

Yt = β1 + β2Xt + ut

Yt−1 = β1 + β2Xt−1 + ut−1


Yt = β1 + β2Xt + ut

Yt−1 = β1 + β2Xt−1 + ut−1

Now multiply both sides by φ, and substract, and given thatut = φut−1 + εt we get:

Yt = β1 − φβ1 + φYt−1 + β2Xt − β2φXt−1 + ut − φut−1

= β1 − φβ1 + φYt−1 + β2Xt − β2φXt−1 + ε

This is a non-linear (in parameters) regression model with noautocorrelation: we have been able to get rid of serial correlation,but now we need to estimate a non-linear model.


For estimation purposes, we will assume there is a set of ofvariables Ω that satisfy

E(y|Ω) = x(β)

By definition, x ∈ Ω, but there may be other variables with thisproperty.

Using properties of the conditional expectation, we can express themodel as

y = x(β) + u

where u satisfies E(u|x) = 0.


Method-of-moments estimation

If w is a vector of K rv’s in Ω, then the following momentconditions hold:

E(wu) = E(w (y − x(β0)) = 0

Suppose we have an iid sample (yi, wi, xi), i = 1, . . . , n of themodel

yi = xi(β) + ui, ui ∼ IID(0, σ2)

with E(ui|Ωi) = 0, wi ∈ Ωt, so E(wiui) = 0. β0 is the true value.


The MM estimator β is defined implicitely as:

1n

n∑i=1

wi(y − x(β)) = 0,

a possibly non-linear system of K equations and K unknowns.

Asymptotic properties

Consistency: we will establish a more general result when westudy extremum estimators.

Asymptotic normality and asymptotic variance.

Asymptotic efficiency.


Asymptotic normality

We will assume (and prove later) that β is consistent.

Let

αn(β) ≡ 1n

n∑i=1

wi(yi − xi(β))

and take a mean value expansion around β0:

αn(β) = αn(β0) + α′n(β)(β − β0)

=1n

∑wiui +

[1n

∑wiXi(β)′

](β − β0)

Xi(β) is a column vector of the K derivatives of x(β) with respectof its arguments. Then the FOC is

1n

∑wiui +

[1n

∑wiXi(β)′

](β − β0) = 0


Premultiply by√n and solving:

√n (β − β0) =

[1n

∑wiXi(β)′

]−1 (√n

1n

∑wiui

)Now we can use standard tools. We need to show

1n

∑wiXi(β)′

p→ E(wX(β0)′) ≡ SwX′ .√n 1n

∑wiui

d→ N(, Sww) with Sww ≡ E(ww′)

HW: write down regularity conditions that guarantee that these results hold


Then √n (β − β0) d→ N

(0, σ2S−1

wX′Sww′S−1wX′)


Asymptotic efficiency

Our consistency-AN argument works for any w ∈ Ω.

We will prove that a MM estimator based on w = X ′(β0) isasymptotically efficient. The asymptotic variance of the MM

estimator is

V = σ2S−1wX′Sww′S

−1wX′

= plim σ2

(1nW ′X0

)−1( 1nW ′W

)(1nW ′X0

)′−1

When W = X0 it reduces to:

V ∗ = plim σ2

(1nX ′0X0

)−1


Then

V − V ∗ = σ2plim

[(1

nW ′X0

)−1(1

nW ′W

)(1

nW ′X0

)−1

−(

1

nX ′0X0

)−1]

= σ2plim

(1

n

)−1 [(W ′X0

)−1 (W ′W

) (W ′X0

)−1 −(X ′0X0

)−1]

which is psd iff:X ′0X0 −X ′0W (W ′W )−1W ′X0

Now

X ′0(I −W (W ′W )−1W ′

)X ′0 = X ′0MWX ′0 = X ′0M

′WMWX0 = m′m ≥ 0

where m ≡MWX0. The desired result follows since MW is symmetric

and idempotent.


MM and Non-Linear Least Squares

A problem with the optimal MM estimator is that it is not feasible.Why?

Consider the following set of sample moment conditions

X ′(β)(y − x(β)) = 0

If we can show that the estimator implied by these momentconditions is consistent, then it will be asymptotically equivalent tothe unfeasible MM estimator. (we will do it, later on).


It is easy to see that

X ′(β)(y − x(β)) = 0

are the FOC’s of the following optimization problem

min(y − x(β))′(y − x(β))

that is, minimize the SSR of the non-linear regression model. Then,the resulting estimator is the non-linear least squares estimator.

We will prove its consistency later on, and asymptotic normality inthe homework.


Empirical example: the black economy

Based on Lyssiotou and Stengos (2004), ‘Estimates of the Black EconomyBased on Consumer Demand Approaches’, The Economic Journal, 114 (July),2004, pp. 622-640.

Suppose log food consumption is related log total income asfollows:

ln ci = β0 + β1 ln yi + ui (1)

Total income comes from two sources, ywi and yni , say wages andnon-wage income:

yi = ywi + yni

We will assume wage income is correctly reported, henceunderreport occurs only related to non-wage income as follows:

yrni = θyni

so individuals report to household surveys a fraction θ of theirnon-wage income.


Replacing yni = 1/θyrni above and then in (1), and calling δ ≡ 1/θwe get:

ln ci = β0 + β1 ln(ywi + δ yrni ) + ui (2)

Under standard assumptions about ui, we can use non-linearleast squares, the explained variable is ln ci and the dependentvariables are wage income and reported non-wage income.

Underreport can be estimated as 1− θ = 1− 1/δ, How wouldyou estimate its standard error?

The null of no underreport corresponds to H0 : δ = 1, inwhich case the model becomes linear in the parameters, andcan be estimated using standard methods.

Regarding identification, note that the model is not identifiedif β1 = 0, for which we will restrict the analysis to β1 > 0.Also, δ must be such that ywi + δ yrni > 0, which we willassume.


The model was estimated in R for Windows with starting valuesequal to (0,1,0.5). Regression results are as follows:

Formula: logc ~ c0 + c1 * log(ya + c2 * yn)

Parameters:

Estimate Std. Error t value Pr(>|t|)

c0 0.91727 0.06731 13.628 < 2e-16 ***

c1 0.38787 0.04961 7.819 4.84e-14 ***

c2 1.63359 0.46596 3.506 0.000507 ***

---

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4932 on 397 degrees of freedom

Correlation of Parameter Estimates:

c0 c1

c1 0.1991

c2 -0.9234 -0.3328

Then estimated underreport is 1- 1/1.63359= 0.612 = 0.388


Nls

Documents

Transcript of Nls