Regression Analysis 04

download Regression Analysis 04

of 90

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Regression Analysis 04

1 QM IIQM IILecture 9: Exploration of Multiple Regression Assumptions. 2Organization of Lecture 9Review of Gauss-Markov AssumptionsAssumptions to calculate hat - MulticollinearityAssumptions to show hat is unbiased - Omitted Variable Bias - Irrelevant VariablesAssumptions to calculate variance of hat- Limited Degrees of Freedom- Non-Constant Variance 31. 6 Classical Linear Model Assumptions(Wooldridge, p. 167) Linear in Parameters Random Sampling of n observations No perfect collinearity Zero conditional mean. The error u has an expected value of 0, given any values of the independent variables Homoskedasticity. The error has the same variance given any value of the explanatory variables.6. Normality. The population error u is independent of the explanatory variables and is normally distributed0 1 1 2 2... k ky x x x u + + + +1, 2, 2, ,( ..., , ) : 1, 2,...i i i ik ix x x x y i n None of the independent variables are constant and there are no exact linear relationship among independent variables21( , ,... ) 0kE u x x x 221( , ,... )kVar u x x x o 20 1 1 2 2| ( ... , )k ky x N x x x o + + + : 4Categories of AssumptionsTotal number of assumptions necessary depends upon how you countMatrix vs. ScalarFour categories of assumptions:Assumptions to calculate hat Assumptions to show hat is unbiasedAssumptions to calculate variance of hatAssumption for Hypotheses Testing (Normality of the Errors) 5Categories of AssumptionsNote that these sets of assumptions follow in sequenceEach step builds on the previous resultsThus if an assumption is necessary to calculate hat , it is also necessary to show that hat is unbiased, etc.If an assumption is only necessary for a later step, earlier results are unaffected 62. Assumptions to calculate hat 7Assumptions to Calculate hat :GM3: xs varyEvery x takes on at least two distinct valuesRecall our bivariate estimatorIf x does not vary, then the denominator is 012( )( )( )i iix x y yx x

8If x does not vary then our data points become a vertical line.The slope of a vertical line is undefinedConceptually, if we do not observe variation in x, we cannot draw conclusions about how y varies with xAssumptions to Calculate hat :GM3: xs vary 9Assumptions to Calculate hat :GM3 New: xs are Not Perfectly CollinearMatrix (XX)-1 existsRecall our multivariate estimator This means that (XX) must be of full rankNo columns can be linearly related1' (X'X) X y 10If one x is a perfect linear function of another then OLS cannot distinguish among their effectsIf an x has no variation independent of other xs, OLS has no information to estimate its effect.This is more general statement of the previous assumption x varies. Assumptions to Calculate hat :GM3 New: xs are Not Perfectly Collinear 11Multicollinearity 12MulticollinearityWhat happens if two of variables are not perfectly related, but are still highly correlated?Excerpts from readings: Wooldridge, p. 866: A term that refers to correlation among the independent variables in a multiple regression model; it is usually invoked when some correlations are large, but an actual magnitude is not well-defined. KKV, p. 122. Any situation where we can perfectly predict one explanatory variable from one or more of the remaining explanatory variables. UCLA On-line Regression Course: The primary concern is that as the degree of multicollinearity increases, the regression model estimates of the coefficients become unstable and the standard errors for the coefficients can get wildly inflated. Wooldrige, p. 144. Large standard errors can result from multicollinearity. 13Multicollinearity: DefinitionIf multicollinearity is not perfect, then (XX)-1 exists and analysis can proceed.But multicollinearity can cause problems for hypothesis testing even if correlations are not perfect.As the level of multicollinearity increases, the amount of independent information about the xs decreasesThe problem is insufficient information in the sample 14Multicollinearity: ImplicationsOur only assumption in deriving hat and showing it is BLUE was that (XX)1 exists.Thus if multicollinearity is not perfect, then OLS is unbiased and is still BLUE.But while hat will have the least variance among unbiased linear estimators, its variance will increase. 15Multicollinearity: ImplicationsRecall our MR equation for hat in scalar notation:As multicollinearity increases, the correlation between xik and xikhat increases2( )( )ik ik ikik ikx x yx x

16Multicollinearity: ImplicationsReducing xs variation from xhat is the same as reducing xs variation from the mean of x in the bivariate model. Recall the equation for the variance of hat in the bivariate regression model: Now, we have the analogous formula for Multiple Regression122 12( )( )uiVarx xo o

12 22 12 21 1 1 121 1 11( )( ) (1 ), where is the total variation in x and is the R-squared fromthe regression of x on all the other x's. u ui iVarx x SST RSST Ro o o = The closer x is to its predicted value, the smaller the quantity and therefore the larger the variance around Beta1_hat. (a.k.a a larger standard error). 17Multicollinearity: ImplicationsThus as the correlation between x and xhat increases, the denominator for the variance of hat decreases increasing the variance of hat Notice if x1 is uncorrelated with x2xn, then the formula is the same as in the bivariate case. 18Multicollinearity: ImplicationsIn practice, xs that are causing the same y are often correlated to some extentIf the correlation is high, it becomes difficult to distinguish the impact of x1 on y from the impact of x2xnOLS estimates tend to be sensitive to small changes in the data. 19Illustrating MulticollinearityWhen correlation among xs is low, OLS has lots of information to estimate hat This gives us confidence in our estimates of hat YX1X2 20Illustrating MulticollinearityWhen correlation among xs is high, OLS has very little information to estimate hat This makes us relatively uncertain about our estimate of hat yx1x2 21Multicollinearity: Causesxs are causally related to one another and you put them both in your model. Proximate versus ultimate causality. (i.e. legacy v. political institutions and economic reform)Poorly constructed sampling design causes correlation among xs.(i.e. You survey 50 districts in Indonesia, where the richest districts are all ethnically homogenous, so you cannot distinguish between ethnic tension and wealth on propensity to violence.).Poorly constructed measures over aggregate information can make cases correlate.(i.e. Freedom House Political Liberties, Global Competitiveness Index) 22Multicollinearity: CausesStatistical model specification: adding polynomial terms or trend indicators.(i.e. Time since the last independent election correlates with Eastern Europe or Former Soviet Union).Too many variables in the model xs measure the same conceptual variable.(i.e. Two causal variables essentially pick-up up on the same underlying variable). 23Multicollinearity: Warning SignsF-test of joint-significance of several variables is significant but coefficients are not.Coefficients are substantively large but statistically insignificant.Standard errors of hat s change when other variables included or removed, but estimated value of hat does not. 24Multicollinearity: Warning SignsMulticollinearity could be a problem any time that a coefficient is not statistically significantShould always check analyses for multicollinearity levelsIf coefficients are significant, then multicollinearity is not a problem for hypothesis testing. Its only effect is to increase the hat2 25Multicollinearity: The DiagnosticDiagnostic of multicollinearity is the auxiliary R-squaredRegress each x on all other xs in the modelR-squared will show you linear correlation between each x and all other xs in the model 26Multicollinearity: The DiagnosticThere is no definitive threshold when the auxiliary R-squared is too high. Depends on whether is significantTolerance for multicollinearity depends on n Larger n means more informationIf auxiliary R-squared is close to 1 and hat is insignificant, you should be concerned If n is small then .7 or .8 may be too high 27Multicollinearity: Remedies Increase sample size to get more informationChange sampling mechanism to allow greater variation in xsChange unit of analysis to allow more cases and more variation in xs (districts instead of states)Look at bivariate correlations prior to modeling 28Multicollinearity: RemediesDisaggregate measures to capture independent variationCreate a composite scale or index if variables measure the same conceptConstruct measures to avoid correlations 29Multicollinearity: An Example reg approval cpi unemrate realgnp Source | SS df MS Number of obs = 148---------+------------------------------ F( 3, 144) = 5.38 Model | 2764.51489 3 921.504964 Prob > F = 0.0015Residual | 24686.582 144 171.434597 R-squared = 0.1007---------+------------------------------ Adj R-squared = 0.0820 Total | 27451.0969 147 186.742156 Root MSE = 13.093------------------------------------------------------------------------------approval | Coef. Std. Err. t P>|t| [95% Conf. Interval]---------+-------------------------------------------------------------------- cpi | -.0519523 .1189559 -0.437 0.663 -.2870775 .1831729unemrate | 1.38775 .9029763 1.537 0.127 -.3970503 3.172551 realgnp | -.0120317 .007462 -1.612 0.109 -.0267809 .0027176 _cons | 61.52899 5.326701 11.551 0.000 51.00037 72.05762---------------------------------------------------------------------