A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

14
Multivariate Analysis and Its Applications IMS Lecture Notes Monograph Series (1994) Volume 24 A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS BY P. M. BENTLER University of California, Los Angeles Covariance structure analysis is a well known method for testing theories on nonexperimental data. Under the null hypothesis, the population covariance matrix Σ is hypothesized to be a function of a vector of more basic parameters 0, i.e., Σ = Σ(0). An illustration is Σ = ΛΦΛ' + Φ, the confirmatory factor analysis model. The null hypothesis is typically evaluated with test statistics that are presumed to have X distributions in large samples. Previous work by Satorra and Bentler (1986, 1988a, 1988b) has shown that the general null distribution of these statistics is not \ (df), but rather a weighted sum of 1 df X statistics. In this paper, this mixture distribution is suggested to be approximated using a method proposed by Gabler and Wolff (1987). A sampling experiment evaluates the performance of this approximation. When applied to correcting the estimated probability of the maximum likelihood test statistic, it is found to work well under conditions of independence of latent variates underlying the model, except at the smallest sample sizes, but to perform poorly under conditions of dependence. When applied to correcting the Satorra Bentler scaled test statistic, it is found to work well under independence, but to overcorrect under dependence. A theoretical basis for these divergent results remains to be found. 1. Introduction. Hu, Bentler, and Kano (1992) recently studied the performance of six goodness of fit test statistics in covariance structure analy sis using Monte Carlo sampling under the null hypothesis. For an introduction to covariance structure analysis, see, e.g., Bollen (1989). Under an assumed distribution of variables and a hypothesized model Σ(0) for the population covariance matrix Σ, these statistics have an asymptotic central χ 2 * Supported in part by USPHS grants DA01070 and DA00017. AMS 1980 Subject Classifications: Primary 62E05, 62H10, 62N05. Key words and phrases: Chi square, dependent observations, factor analysis

Transcript of A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

Page 1: A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

Multivariate Analysis and Its Applications

IMS Lecture Notes - Monograph Series (1994) Volume 24

A TESTING METHOD FORCOVARIANCE STRUCTURE ANALYSIS

BY P. M. BENTLER

University of California, Los Angeles

Covariance structure analysis is a well-known method for testing theories

on nonexperimental data. Under the null hypothesis, the population covariance

matrix Σ is hypothesized to be a function of a vector of more basic parameters

0, i.e., Σ = Σ(0). An illustration is Σ = ΛΦΛ' + Φ, the confirmatory factor

analysis model. The null hypothesis is typically evaluated with test statistics

that are presumed to have X distributions in large samples. Previous work

by Satorra and Bentler (1986, 1988a, 1988b) has shown that the general null

distribution of these statistics is not \ (df), but rather a weighted sum of 1-

df X statistics. In this paper, this mixture distribution is suggested to be

approximated using a method proposed by Gabler and Wolff (1987). A sampling

experiment evaluates the performance of this approximation. When applied to

correcting the estimated probability of the maximum likelihood test statistic, it is

found to work well under conditions of independence of latent variates underlying

the model, except at the smallest sample sizes, but to perform poorly under

conditions of dependence. When applied to correcting the Satorra-Bentler scaled

test statistic, it is found to work well under independence, but to overcorrect

under dependence. A theoretical basis for these divergent results remains to be

found.

1. Introduction. Hu, Bentler, and Kano (1992) recently studied theperformance of six goodness-of-fit test statistics in covariance structure analy-sis using Monte Carlo sampling under the null hypothesis. For an introductionto covariance structure analysis, see, e.g., Bollen (1989). Under an assumeddistribution of variables and a hypothesized model Σ(0) for the populationcovariance matrix Σ, these statistics have an asymptotic central χ2

* Supported in part by USPHS grants DA01070 and DA00017.

AMS 1980 Subject Classifications: Primary 62E05, 62H10, 62N05.

Key words and phrases: Chi-square, dependent observations, factor analysis

Page 2: A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

124 A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

distribution that describes the mean, variance, and tail performance of thestatistics. Hu et al. investigated three ways of violating theoretical conditionsrelevant to the choice of χ2 as the reference distribution: they violated distribu-tional assumptions, assumed independence conditions, and asymptotic samplesize requirements. The effects of these violations on normal theory maximumlikelihood (ML) and generalized least squares (GLS), elliptical theory (ERLS),heterogeneous kurtosis (HK), asymptotic distribution-free (ADF), and scaling-corrected (SCALED) test statistics (TML,7GLS 97ERLS,THK,ΪADF,7SCALED)

were studied. They found that: the normal theory tests worked well undersome conditions but completely broke down under other conditions; the ellipti-cal test performed variably; the heterogeneous kurtosis test performed better;the asymptotic distribution free test performed very badly at all but the largestsample sizes; and the scaled test statistic performed best overall. In relatedwork, Chou, Bentler, and Satorra (1991) and Muthen and Kaplan (1992) alsofound that standard test statistics in covariance structure analysis could per-form badly under conditions of violation of assumptions. Since the statisticaltheory is asymptotic, an especially important practical problem continues tobe how to improve the performance in small samples of the existing statistics.

The purpose of this study is to propose a method for more accuratelyapproximating the distribution of the TML and TSCALED test statistics andto evaluate the performance of this approximation. Satorra and Bentler (1986,1988a, 1988b) had shown that test statistics used in covariance structure analy-sis are not in general χ2 distributed, though under precise modeling conditionsthis reference distribution would be appropriate. These modeling conditionsreduce a mixture distribution to that of the standard χ2 variate. Satorraand Bentler provided no procedures for actually implementing their theory inpractice. The contribution of this paper is to propose an implementation ofthe Satorra-Bentler theory using an approximation developed by Gabler andWolff (1987) for a similar problem. This implementation is evaluated with asmall sampling study based on the Monte Carlo conditions previously studiedby Hu et al.

2. Test Statistics. In this section, the notation is introduced,some currently available test statistics in covariance structure analysis arereviewed, and the technical problem is defined. This review must necessarilybe short; summaries of various aspects of the theory are provided, for example,by Bentler and Dijkstra (1985), Browne (1984), Satorra (1989), and Wakaki,Eguchi, and Fujikoshi (1990). The review is provided to provide a context forthe current work, as well as to provide the definitions needed to compare theproposed method with previously existing methods.

Following Hu et al. (1992, Appendix), let S represent the usual unbiasedestimator based on a sample of size n of a p X p population covariance matrixΣ, whose elements are functions of a q x 1 parameter vector θ : Σ = Σ(0).

Page 3: A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

P. M. BENTLER 125

A discrepancy function F = F(S,Σ(Θ)) can be considered to be a measure ofthe discrepancy between S and Σ(θ) evaluated at an estimator θ. The normaltheory maximum-likelihood (ML) discrepancy function (Jδreskog, 1969) is :

F M L = log |Σ | - log |5 | + t r ^ E " 1 ) - p.

At the minimum, Σ = Σ(θ) and FML takes on the value FML, where T M L ={Π-1)FML is distributed, under the null hypothesis, as an asymptotic goodness-of-ίit χ2 variate with (p* - q) degrees of freedom, where p* = p(p + l)/2. TMLcan be used as a test statistic to evaluate the null hypothesis Σ = Σ(0). Thenull hypothesis is rejected if TML exceeds a critical value in the χ2 distributionat an α-level of significance.

A quadratic form discrepancy function is:

where s and σ{θ) are p * x l column vectors formed from the nonduplicatedelements of S and Σ(0), respectively, and W is a p* X p* positive-definiteweight matrix. The asymptotically distribution-free (ADF) covariance struc-ture method used by Hu et al. minimizes FQV under the choice of optimalweight matrix W with typical elements

where σijki = E(xu — μi)(xtj — βj)(χtk — μ>k)(%tι — μι) is the fourth-order mul-

tivariate moment of variables X{ about their means /ij, and σij is an element ofΣ. In practice, sample moment estimators Sijki = Σι(xti-Xi)(%tj — Xj)(%tk-Xk)(xtι-%ι)/n and S{j = Σi(χti-Xi)(χtj-Xj)/(n-l) are used to consistentlyestimate σ^x and σ2j. The ADF estimator provides an asymptotically effi-cient estimator θ without the need for distributional assumptions on variables.Under the null hypothesis, the associated test statistic TADF = (n — 1)FQD

has an asymptotic χ2 distribution based on (p* - q) degrees of freedom. SeeBrowne (1984) or Chamberlain (1982).

The fitting function FQD for normal theory GLS can be simplified to

if W = 2K'p(V ® V)KP, where V is a positive definite matrix that convergesto Σ probability, and Kp is a known transition matrix. At the minima ofthe respective functions, both Γ M L and TGLS = (n - 1)FGLS have asymptoticχ2 distributions with (p* - q) degrees of freedom; they are asymptoticallyequivalent when the model is correct. Browne (1974) has shown that if Vconverges in probability to Σ (V = S is typically used in practice) then GLS

Page 4: A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

126 A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

estimators are asymptotically equivalent to maximum likelihood estimators.See also Lee and Bentler (1980).

Under the assumption that all marginal distributions of a multivariatedistribution are symmetric and have the same relative kurtosis, elliptical the-ory parameter estimators and test statistics can be obtained by readjustingthe statistics derived from normal theory methods. Let n = σuu/3σ^i — 1be the common kurtosis parameter of a distribution from the elliptical class.Multivariate normal distributions are members of this class with K = 0. Thefourth-order multivartiate moments σijki are related to K by

= (« + l){σijGkι + σikσji +

where σ^ is an element of Σ. As a result of this simplification, the discrepancyfunction for an elliptical distribution may be written as

FE = \{κ + l)"1tr((5 - ΣiθW-1)2 - δ(tτ(S -

where as before V is any consistent estimator of Σ, and δ = «/(4(« + I) 2 +2pκ(κ+l)) (Bentler (1983); Browne (1984)). The selection of V as a consistentestimator of Σ leads, under the model and assumptions, to an asymptoticallyefficient estimator of θ with Tβ = {n - l)Fβ at θ asymptotically distributedas a χL*_ ) variate. A standard implementation is to choose V = Σ at theminimum and (K + 1) = Σ i ((& ~ x)S~1(x - x))2/np(p + 2). Since the modelsto be investigated are invariant with respect to a constant scaling factor, atthe minimum of Fβ the second term drops out yielding TE — TERLS

a s usedin the Hu et al. study. See Shapiro and Browne (1987).

Heterogeneous kurtosis theory (Kano, Berkane, & Bentler, 1990) definesa more general class of multivariate distributions that allows marginal distri-butions to have heterogeneous kurtosis parameters. The elliptical distributionis a special case of this class of distributions. Let κ\ = σmt /3σ?t represent ameasure of excess kurtosis of the i-th variable, and the fourth-order momentshave the structure

where oy = (κi + Kj)/2. If the covariance structure Σ(0) is fully scale invariantand the modeling and distributional assumptions are met, the JFQD discrepancyfunction can be expressed as

where C = A * Σ, and * denotes the elementwise (Hadamard) product of thetwo matrices of the same order. Hu et al. used A = (αy) = (ίq + Kj)/2

Page 5: A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

P. M. BENTLER 127

based on the usual moment estimators /?2 = saa/Ssf^ with C = A * 5.Kano et al. (1990) demonstrated that the simple adjustment of the weightmatrix C of the normal theory generalized least squares procedure (see FGLS

above) produces asymptotically efficient estimators. The associated test statis-tic THK = (n- l)ίHK at the minimum has an asymptotic χlp*_q) distributionunder the assumed model.

Satorra and Bentler (1988a, b) developed two modifications of any stan-dard goodness of fit statistic test T = (ΓML?ΪHK etc.) so that its distributionalbehavior should more closely approximate χ 2 . One of these, the scaled teststatistic, is available in the computer program EQS (Bentler, 1989, p.218) andis studied here. Satorra and Bentler (1986) noted that the general distributionof T is in fact not χ 2, but rather a mixture

df

where α t is one of the df (=degrees of freedom ) nonnull eigenvalues of thematrix UVSS, Vss is the asymptotic covariance matrix of y/n(s - σ(0)), % is oneof the df independent χ2 variates, and, when there are no constraints on freeparameters (as in this study)

U = W'1 - W^σi&'W^σ^σ'W'1

is the residual weight matrix under the model and the weight matrix W used inthe estimation. The scaled statistic used by Hu et al. was based on ΓML ? withW = 2Kr

p(Σ ® Σ)/ίp, the normal theory ML weight matrix at the minimumof FML and σ = dFMh/dθf evaluated at θ. The mean of the asymptoticdistribution of ΓML is given by tτ(UVss). Then, defining the scaling estimatek =tτ(UV88)/df , where U is a consistent estimator of U based on 0, and Vss

is the distribution-free estimator with elements Sijki — SijSki (see above), thescaled ML statistic

ΪSCALED =

defines Satorra and Bentler's SCALED test statistic as applied by Hu et al.This statistic is easier to implement than the general form.

3. The Hu, Bentler, and Kano (1992) Study. The Hu et al.sampling study is now described further to provide a more detailed summaryof prior findings, and also because their conditions are replicated in the currentinvestigation to study the tests proposed below. They used the confirmatoryfactor model x = Λ£ + ε to generate measured variables x under variousconditions on the common factors ξ and unique variates ("errors") ε. Theyused 15 measured variables in z, with Λ being a 15 X 3 matrix having a simplecluster structure. See Hu et al. for details. In the standard approach to factor

Page 6: A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

128 A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

analysis, factors and errors are assumed to be normally distributed, factors areallowed to correlate with covariance matrix ε(f £') = Φ, errors are uncorrelatedwith factors, i.e., ε(ξε/) = 0 and various error variates are uncorrelated andhave a diagonal covariance matrix ε(εε') = Φ. As a result, Σ = Σ(0) =ΛΦΛ' + Φ, and the elements of θ are the unknown parameters in Λ, Φ, andΦ. Hu et al. had one condition in which factors and errors were multivariatenormally distributed, so that the latent variates that are uncorrelated in thefactor model are also independent of each other. They also used conditions inwhich the factors and/or errors were not normally distributed. In some of theseconditions, factor/error variates that are uncorrelated under the model alsowere independent, while in other conditions these variables were uncorrelatedbut not independent. Independence of latent variates is a key condition inso-called asymptotic robustness theory that describes conditions under whichnormal theory ML and GLS test statistics are robust to violations of normality(e.g., Amemiya k Anderson (1990); Anderson k Amemiya (1988); Browne kShapiro (1988); Mooijaart k Bentler (1991); Satorra k Bentler (1990)).

After generation of the population covariance matrix Σ under the as-sumed conditions, random samples of a given size from the population weretaken, the null model was estimated, and the statistics T = (n - 1)F (forT = TML,TGLSJTERLS,THK,TADF and TSCALED) were computed. The per-formance of these statistics across the sampling replications at a given samplesize were the main data of their study. They used sample sizes of 150, 250, 500,1000, 2500, and 5000 to evaluate the effects of sample size. In each conditionat each sample size, 200 replications (samples) were drawn from the popula-tion, and the various estimators and goodness-of-fit tests were computed. Themean values and standard deviations of T across the 200 replications, andthe empirical rejection rates at the α = .05 level based on the assumed χ2

distribution, were used to compare their methods.

Hu et al. found that when the latent common and unique factors wereindependently distributed, the anticipated asymptotic robustness properties ofthe χ2 test were retained for normal theory methods when the sample size wasrelatively large. That is, the test statistics behaved as χ2 variates even thoughassumed distributional assumptions were violated. Asymptotic robustness,however, could not be guaranteed at smaller sample sizes with ML. ERLSslightly, and HK somewhat more, overcorrected the test statistics when someor all the latent variates were nonnormal. The ADF method was very sensitiveto sample size except under normality, it did not even perform acceptablywith a sample size as large as 2500. The SCALED statistic outperformed ADFat all but the largest sample sizes. They also found that under conditions ofdependency among latent factors and unique variates, normal theory methodscould not be trusted, HK worked substantially better, ADF performed wellonly at very large sample sizes, and, across all sample sizes, the Satorra-BentlerSCALED statistic performed at closest to nominal levels of all the methods

Page 7: A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

P. M. BENTLER 129

considered.The question is whether a new approximation to the distribution of statis-

tics T = Γ M L,^GLS,^ERLS,ΪHK,2ADF and ΓS CALED could perform betterunder the same conditions. For simplicity, this study is limited to the per-formance of ΓML and ΓSCALED when evaluated not by χ2 but rather by theproposed approximation to the weighted sum of 1-df χ2 variates. In addition,the performance of these statistics at smaller sample sizes than considered byHu et al. is evaluated.

4. Approximating the General Distribution. As noted above,under general conditions the distribution of T is the distribution of a weightedsum of chi-square variables

dfJwhere OL{ is one of the nonnull eigenvalues of UV8SJVS8 is the asymptoticcovariance matrix of yfn(s — σ{θ)), U is the residual matrix U = W~λ —W~1σ(σfW~1σ)~1σfW~1, and % is one of the df independent χ2 variates.Specific implementation depends on the estimator, here taken to be ML.Thus, W = 2Kf

p(Σ ® Σ)KP and σ = dFuh/dθ'. In practice these matri-ces are evaluated at the ML estimator θ, yielding c?;.

An explicit expression for the distribution of T is given in Johnson andKotz (1970), but it is difficult to evaluate in practice. Gabler and Wolff (1987)proposed to do this by constructing a random variable Y that has the samefirst three moments as those of Γ, and has only minor differences in the highermoments. They standardized the problem so that Σα; = 1, and took Y to bethe positive random variable with density function

For m = 1,2, , they verified that E(Tm) < E(Ym), derived the distributionfunction of Y, obtained the laplace transforms of T and Y, and showed thatfor small t > 0

is a good approximation for the distribution function F(t) of Γ where 7 denotesthe incomplete gamma function. Based on this theory they proposed an algo-rithm for approximating the probability Pr(X)f afc < t) by the minimum of

Page 8: A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

130 A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

the functions G(t) and H(t), where

A

and

They indicate that comparisons made for those cases in which exact resultsare available showed extremely good accuracy to their approximation.

In the application of this approach to covariance structure analysis, spe-cific test statistics T must be chosen. Here, only TML and ^SCALED are stud-ied. Further, the population eigenvalues ai have to be replaced by their sam-ple counterparts 2;. The accuracy of the procedure thus depends on severalfeatures. First, is the distribution of TML and TSCALED well described as amixture of 1-df χ2 statistics under the conditions of the study (small samplesize, dependence, nonnormality). Second, is the Gabler-Wolff approximationa good one to the theoretical distribution. Third, what is the quality of theestimators of the population eigenvalues. The proposed procedure can breakdown due to problems at any of these points. Regarding the eigenvalues,only the simplest estimator is considered based on the eigenvalues of the sam-ple moment matrix product UVSS. These eigenvalues can be computed usingstandard approaches for obtaining the eigenvalues of nonsymmetric matrices.

5. Simulation Results. Seven conditions were studied using the Hu,Bentler, Kano (1992) procedures. These conditions depended on the distribu-tion of the common factors ξ and the unique factors ε, as well as the mutualdependence/independence of these sets of factors. The conditions were asfollows:

1. Normal ξ Normal ε Mutual independence of f and ε2. Nonnormal ξ Nonnormal ε Mutual independence of ξ and ε3. Nonnormal ξ Nonnormal ε Mutual independence of ζ and ε4. Normal ξ Nonnormal ε Mutual independence of ξ and ε5. Nonnormal ξ Nonnormal ε Dependent ξ and ε6. Nonnormal £ Nonnormal ε Dependent £ and ε7. Nonnormal ξ Nonnormal ε Dependent ξ and ε

The conditions varied in several ways beyond that stated above, see Hu etal., but for the current purpose the main points are that: condition 1 is thestandard case, leading to multivariate normal measured variables; conditions3-4 with mutually independent latent variables are consistent with the theory

Page 9: A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

P. M. BENTLER 131

of asymptotic robustness for normal theory test statistics; and conditions 5-7violate asymptotic robustness conditions. In all conditions except condition1, the measured variables were symmetrically but nonnormally distributed.Marginal kurtoses for factors and errors were in the range 0-28. In conditions2 and 3, the true kurtoses for the nonnormal factors were -1.0, 2.0, and 5.0.In conditions 2-4, the kurtoses of the unique variates ranged from -1.0 to 7.5.In conditions 5-7, similarly defined nonnormal factors and errors were furtherdivided by a rescaled chi random variable that was independent of the factorsand errors. In addition to the sample sizes 150-5000 as studied by Hu et al.,samples of size 50 and 100 also were drawn in the current study.

The behavior of the statistics TML and TSCALED were evaluated in thesimulation using proposed procedure to obtain estimated probability values.With 200 replications, at a nominal alpha level of .05, the true model shouldbe rejected about 10 times. The following table summarizes the rejection rateunder the seven conditions for the statistic

Table 1. Summary of Simulation Results forNumber of Model Rejections

Condi-tion

1

2

3

4

5

6

7

50

195

198/199

195

191

177/177

100

7

9

4

5

148

140

134

Sample Size

150

7

7

8

8

158

147

141

250

12

8

6

8

175

172

169

500

6

9

9

7

195

190

188

1000

5

17

15

15

197

196

195

2500

4

14

10

9

200

199

199

5000

8

10

10

9

199

200

199

Several major results can be seen in Table 1. When the data are normal(condition 1), the proposed procedure works well at all sample sizes except thesmallest. At n = 50, the true model is rejected in 195 out of 200 replications.Essentially the same results are obtained even if the factors and errors arenot normally distributed but they are mutually independent (conditions 2-4).At the smallest sample size, fewer than 200 replications sometimes yieldedconverged solutions. For example in condition 2, 199 out of 200 replications

Page 10: A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

132 A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

yielded converged solutions; all but one of these yielded a probability valuesuggesting model rejection. Most dramatically, the true model was rejectedin an extremely high proportion of model tests under factor and error depen-dence (conditions 5-7). At the smallest sample sizes in conditions 6 and 7,no converged solutions were obtained, while at larger sample sizes about 3/4to almost all true models were rejected. Results for the ΓSCALED statistic aregiven in Table 2.

Table 2. Summary of Simulation Results for Γ'SCALED-

Number of Model Rejections

Condi-

tion

1

2

3

4

5

6

7

50

11

13/199

9

10

2/177

100

10

11

4

8

14

12

24

Sample Size

150

11

11

9

8

0

0

0

250

16

9

8

7

0

0

0

500

4

9

7

8

0

0

0

1000

5

15

15

15

0

1

0

2500

5

11

9

11

1

1

1

5000

8

7

10

9

2

1

1

When evaluating the results using the Gabler-Wolff procedure as imple-mented here, it appears that as compared to the ΓML statistic, the ΓSCALED

statistic performs better under conditions 1-4 at sample size 50. At n — 100or beyond, in conditions 1-4 the results for Γ M L and ΓSCALED are approxi-mately the same. On the other hand, under conditions of variate dependence(conditions 5-7), ΓSCALED generally yields probability estimates that are toohigh. At sample sizes 150 and larger, the true model is almost always acceptedrather than rejected at the nominal alpha level. An exception seems to be oc-cur at n = 100, where performance closer to nominal is found. But this trenddoes not hold for n = 50, where nonconvergence is a major problem.

6. Conclusions. The proposed method for evaluating the fit of covari-ance structure methods seems to be only marginally better than previouslyreported methods. Hu et al. (1992) had reported that the ML method un-der conditions of variate independence performed well when sample size was

Page 11: A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

P. M. BENTLER 133

500 or greater, but that at samples size 150 and 250 the statistic TML? whenevaluated according to the χ2 reference distribution, rejected models abouttwice as often as would be expected nominally. When evaluating TML usingthe weighted sum of χ 2 ( l ) variates as proposed here, the proportions of modelrejections are much closer to nominal at samples sized of 100 or above. This isa clear impiovement. On the other hand, at sample size 50, model rejectionsare so frequent as to make the proposed procedure useless. At this time, thereis no explanation for this surprising decrement in performance at the smallestsample size. It is likely that it has to do with the inadequate estimation ofthe eigenvalues of UVSS, especially the smallest nonzero population eigenvalueswhich may be null at the smallest sample size . Further research will have toevaluate this hypothesis.

On the other hand, the proposed method when applied to evaluating TMLunder conditions of dependence of factors and errors was no better than whenTML was evaluated with reference to the χ2 distribution. Since this method wasinadequate at even the largest sample size, and since the method does workwell under conditions of variate independence, the poor performance muststem from either the fact that the distribution of TML is n°t well describedas a mixture of χ 2 ( l ) variates, or the fact that the Gabler-Wolff approxima-tion breaks down under these conditions. Clearly, the derivations of Satorraand Bentler (1986, 1988a, 1988b) and Gabler and Wolff (1987) will have tobe examined to determine whether there is a failure in some theoretical as-sumption that may invalidate either or both of their results under conditionsof dependence.

Hu et al. had already shown that the TSCALED statistic performed betteron average than any other statistic they considered, when evaluated against aχ2 distribution. In the present study, this statistic was referred to the Gabler-Wolff approximating distribution. As with the TML statistic, the TSCALED

statistic performed better at smaller sample sized under conditions of variateindependence. However, when evaluated under conditions of variate depen-dence, TSCALED under Gabler-Wolff performed substantially worse than whenevaluated by the χ2 distribution. Thus, there does not seem to be much virtueto the use of TSCALED using the probability calculations based on Gabler andWolff (1987) as applied in this study.

As noted by a reviewer, there is some non-smoothness in results acrosssample sizes for both test statistics. In Table 1, an example occurs at n = 1000for condition 3-4. In Table 2, an example occurs at n = 100 for conditions 5-7.An obvious problem with the simulation is that the number of replications isundoubtedly too small to accurately describe the tail behavior of these teststatistics. However, it does not seem likely that this design feature explainsthe apparent anomalies in trends of results, so there may be other technicaldifficulties with the simulation that we have not been able to locate.

Page 12: A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

134 A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

Acknowledgment. The generous assistance of Eric J.C. Wu and ShinnWu in programming, Li-tze Hu in performing the simulations, and severalreviewers, is gratefully acknowledged.

REFERENCES

AMEMIYA, Y. and ANDWESON, T. W. (1990). Asymptotic chi-square test fora large class of factor analysis models, Ann. Statist. 18, 1453-1463.

ANDERSON, T. W. and AMEMIYA, Y.(1988). The asymptotic normal distribu-tion of estimators in factor analysis under general conditions, Ann. Statist.16, 759-771.

BENTLER, P. M. (1983). Some contributions to efficient statistics for structuralmodels: Specification and estimation of moment structures, Psychometrika48, 493-517.

BENTLER, P. M. (1989). EQS Structural Equations Program Manual. BMDPStatistical Software, Los Angeles.

BENTLER, P. M. and DIJKSTRA, T. (1985). Efficient estimation via lineariza-tion in structural models. In Multivariate Analysis VI (P.R. Krishnaiah,ed.) 9-42. North-Holland, Amsterdam.

BOLLEN, K. A. (1989). Structural Equations with Latent Variables. Wiley,New York.

BROWNE, M. W. (1974), Generalized least squares estimators in the analysisof covariance structure, S. African Statist. J. 8, 1-24.

BROWNE, M. W. (1984). Asymptotically distribution-free methods for theanalysis of covariance structures, Brit. J. Math. Statist. Psychol. 37,62-83.

BROWNE, M. W. and SHAPIRO, A. (1988). Robustness of normal theory meth-ods in the analysis of linear latent variate models, Brit. J. Math. Statist.Psychol. 41, 193-208.

CHAMBERLAIN, G. (1982). Multivariate regression models for panel data, J.Econometrics 18, 5-46

CHOU, C.-P., BENTLER, P. M. and SATORRA, A. (1991). Scaled test statisticsand robust standard errors for nonnormal data in covariance structureanalysis: A Monte Carlo study, Brit. J. Math. Statist. Psychol. 44,347-357.

GABLER, S. and WOLFF, C. (1987). A quick and easy approximation to thedistribution of a sum of weighted chi-square variables, Statist. Hefte 28,317-325.

Page 13: A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

P. M. BENTLER 135

Hu, L., BENTLER, P. M. and KANO, Y. (1992). Can test statistics in covari-ance structure analysis be trusted? PsychoL Bull 112, 351-362.

JOHNSON, N. L. and KOTZ, S. (1970). Continuous Univariate Distributions-2. Houghton Mifflin, Boston.

JORESKOG, K. G., (1969). A general approach to confirmatory maximumlikelihood factor analysis, Psychometrika 34, 183-202.

KANO, Y., BERKANE, M and BENTLER, P. M. (1990). Covariance structureanalysis with heterogeneous kurtosis parameters, Biometrika 77, 575-585.

LEE, S.-Y and BENTLER, P. M. (1980). Some asymptotic properties of con-strained generalized least squares estimation in covariance structure mod-els, S. African Statist. J. 14, 121-136.

MOOIJAART, A. and BENTLER, P. M. (1991) Robustness of normal theorystatistics in structural equation models, Statist. Neerlandica 45, 159-171.

MUTHEN, B. and KAPLAN, D. (1992). A comparison of some methodologiesfor the factor analysis of non-normal Likert variables: A note on the sizeof the model, Brit. J. Math. Statist. PsychoL 45, 19-30.

SATORRA, A. (1989). Alternative test criteria in covariance structure analysis:A unified approach, Psychometrika 54, 131-151.

SATORRA, A. and fred Bentler, P.M. (1986). Some robustness properties ofgoodness of fit statistics in covariance structure analysis, Proc. Bus. Econ.Stat. Amer. statist. Assoc. 549-554.

SATORRA, A. and BENTLER, P. M. (1988a). Scaling corrections for chi-squarestatistics in covariance structure analysis, Proc. Amer. Statist. Assoc.308-313.

SATORRA, A. and BENTLER, P. M. (1988b). Scaling corrections for statisticsin covariance structure analysis. Technical Report, UCLA Statistics Series#2.

SATORRA, A. and BENTLER, P. M. (1990), Model conditions for asymptoticrobustness in the analysis of linear relations, Comp. Statist. Data Anal10, 235-249.

SHAPIRO, A. and BROWNE, M. (1987) Analysis of covariance structures underelliptical distributions, J. Amer. Statist. Assoc. 82, 1092-1097.

WAKAKI, H., EGUCHI, S. and FUJIKOSHI, Y. (1990). A class of tests for ageneral covariance structure, J. Multiv. Anal 32, 313-325.

Page 14: A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

136 A TESTING METHOD FOR COVARIANCE STRUCTURE ANALYSIS

DEPARTMENT OF PSYCHOLOGY

UNIVERSITY OF CALIFORNIA, LOS ANGELES

Los ANGELES, CA 90024-1563USA