August 04, 2020
Chi-Square Difference Testing Using the Satorra-Bentler Scaled Chi-Square
Chi-square testing for continuous non-normal outcomes has been discussed in a series of papers by Satorra and Bentler. A popular test statistic is the Satorra-Bentler scaled (mean-adjusted) chi-square, where the usual normal-theory chi-square statistic is divided by a scaling correction to better approximate chi-square under non-normality.
A little-known fact, however, is that such a scaled chi-square cannot be used for chi-square difference testing of nested models because a difference between two scaled chi-squares for nested models is not distributed as chi-square. Mplus issues a warning about this.
In discussions with Albert Satorra, Bengt suggested that Albert might want to figure out how to get a chi-square difference test for the Satorra-Bentler scaled chi-square and he did, producing the following book chapter which can be downloaded as a working paper (in postscript format).
Satorra, A. (2000). Scaled and adjusted restricted tests in multi-sample analysis of moment structures. In Heijmans, R.D.H., Pollock, D.S.G. & Satorra, A. (eds.), Innovations in multivariate statistical analysis. A Festschrift for Heinz Neudecker (pp.233-247). London: Kluwer Academic Publishers.
The formulas in the paper are, however, complex and subsequently Albert and Peter Bentler wrote a paper showing that simple hand calculations using output from nested runs can give the desired chi-square difference test of nested models using the scaled chi-square. This paper is available here: Satorra, A. & Bentler, P.M. (2010). Ensuring positiveness of the scaled difference chi-square test statistic. Psychometrika 75: 243. doi:10.1007/s11336-009-9135-y
Difference Testing Using Chi-Square
Following are the steps needed to compute a chi-square difference test in Mplus using the MLM (Satorra-Bentler), MLR, and WLSM chi-square. DIFFTEST should be used for MLMV and WLSMV. The nested model is the more restrictive model with more degrees of freedom than the comparison model.
Difference Testing Using the Loglikelihood
Following are the steps needed to compute a chi-square difference test based on loglikelihood values and scaling correction factors obtained with the MLR estimator.
Computing the Strictly Positive Satorra-Bentler Chi-Square Difference Test
The robust chi-square difference test can sometimes produce a negative value. An alternative approach that avoids this is given in
Satorra, A., & Bentler, P.M. (2010). Ensuring positiveness of the scaled difference chi-square test statistic. Psychometrika, 75, 243-248.
Mplus Web Note No. 12 shows how to compute this new alternative test.
For an application article, see Bryant and Satorra (2011).