The power of multiple battery factor analysis in coping with the effects of differential skewness of variables

The principal objective of the study was to determine the power of Multiple Battery Factor Analysis (MBFA) in coping with the effects of differential skewness of the variables used. Generally speaking, joint analyses result in factors of skewness. To examine the problem the General Scholastic Aptitude Test (GSAT) and Senior Ability Tests (SAT) were jointly applied to a sample of 1 598 first-year university students, and subjected to both a Principal Factor Analysis (PFA) and a MBFA. Three factors were obtained in both instances. The PFA yielded factors of skewness and the MBFA factors of content. The implications of the findings are discussed. Opsomming Die hoofdoelwit van die studie was om die krag van Veelvuldigebattery-faktorontleding (VBFO) te bepaal ten einde die gevolge van differensiele skeefheid van veranderlikes te bowe te kom. In die algemeen lei gesamentlike faktorontledings van batterye toetse tot faktore van skeefheid. Om die probleem te ondersoek, is die Algemene Skolastiese Aanlegtoets (ASAT) en die Senior Aanlegtoetse (SAT) gesamentlik op ’n steekproef van 1 598 eerstejaaruniversiteitstudente toegepas en aan sowel ’n Hooffaktorontleding (HFO) as ’n VBFO onderwerp. Drie faktore is in albei gevalle verkry. Die HFO het faktore van skeefheid opgelewer en die VBFO faktore van inhoud. Die implikasies van die bevindinge word bespreek.

It was shown by Schepers (2004) that the degree of skewness of binary test items places an upper limit on the correlations between the items, regardless of the contents of the items. Factoring intercorrelation matrices based on such items usually result in a multiplicity of factors most of which are artefacts (Guttman, 1955(Guttman, , 1957. A similar tendency was also indicated in respect of continuous variables (Schepers, 2004). Ferguson (1941) was one of the first to highlight the problem and referred to the phenomenon as "difficulty factors".
Various solutions have already been proposed to overcome the problem of artefactual factors: Gorsuch (1974, p. 262), for instance, maintained that the best solution is to avoid working with variables that are too skew. His suggestion is highly acceptable, provided the variables are well constructed measuring instruments with acceptable metrical properties. Horst (1965, p. 516) maintained that one way of getting rid of the artefactual factors is to fit an appropriate simplex to the data matrix and to separate it from the true structure (Jöreskog, 1970). This, however, is equivalent to throwing the baby away with the bath water. Thurstone (1947), the pioneer of factor analysis, stressed the fact that the factor analytical model made no assumption of normality regarding the underlying distributions (p. 325). He recommended, however, that the raw scores be normalised before factoring as this would result in the clearest structure being obtained (p. 369).
According to Finch and West (1997, p. 454) "the maximum likelihood (ML) estimation procedure typically utilized in confirmatory factor analysis assumes that the measured variables have a multivariate normal distribution". However, clear guidelines for judging deviations from multivariate normality have as yet not been formulated. West, Finch and Curran (1995) recommend avoiding variables with univariate skewness and kurtosis indices above 2 and 7 respectively (p. 454). Their recommendation regarding kurtosis, however, is not well founded. As far as measuring instruments are concerned, leptokurtosis is normally associated with low reliabilities and should be avoided at all costs. Indices as high as 7 are rather extreme and signify very low reliabilities. As far as skewness is concerned, differential skewness of variables (skewness in opposite directions) is more critical than high values of skewness as it places an upper limit on the correlations between variables (Schepers, 2004).
An issue which often arises is whether two or more test batteries, given to the same sample of subjects, have a common factor structure. Traditionally, researchers have simply conducted a joint factor analysis by using the intercorrelation matrix of all the variables in all the batteries combined. However, the outcome of such analyses have not always been acceptable due to the effects of differential skewness on the underlying structures. Finch and West (1997, p. 470) point out in this regard that joint factor analyses confound two sources of covariation, namely covariation within batteries and covariation between batteries. To overcome this problem Tucker (1958) developed his Inter-battery Factor Analysis technique (IBFA). IBFA uses the cross-correlations between two batteries of tests and determines only the factors common to the two batteries. Tucker's technique has subsequently been extended to more than two batteries by Browne (1980) and programmed by Cudeck (1980Cudeck ( , 1991. It is known as Multiple Battery Factor Analysis (MBFA). For a practical illustration of the technique see de Bruin (2000). He applied IBFA to Comrey's Personality Scales and the 16 Personality Factor Questionnaire.
The principal objective of the present study was to demonstrate the power of Multiple Battery Factor Analysis (MBFA) in coping with the effects of differential skewness of variables in factor analysis.

METHOD Sample
The full complement of first-year university students at the Rand Afrikaans University, during 1995, was subjected to an extensive psychometric test programme. The programme stretched over four days, and students who did not arrive at a particular session were not tested subsequently. If the assumption is made that absence from a test session was random, then the sample could be considered representative of the population of first-year university students at the Rand Afrikaans University, during 1995. All four cultural groups were represented in the sample. Complete records for 1 598 students were obtained in respect of the General Scholastic Aptitude Test (GSAT) and Senior Aptitude Tests (SAT), amongst others.

Measuring instruments
The present study is based on the test scores of students in respect of the GSAT and SAT.

General Scholastic Aptitude Test (GSAT)
The GSAT yields a measure of academic intelligence or scholastic aptitude. It consists of six subtests -three verbal and three nonverbal, and measures both verbal and non-verbal intelligence. A measure of Total IQ is given by the weighted sum of all six subtests. The reliability of the total score is 0,94 according to Kuder-Richardson Formula 8, in respect of a sample of 18 yearolds (Claassen, De Beer, Hugo & Meyer, 1998).

Senior Aptitude Tests (SAT)
The SAT was constructed for the measurement of a number of aptitudes of pupils in Standards 8, 9 and 10, and of adults. It consists of verbal, numerical, non-verbal reasoning, spatial and memory tests. Tests 11 and 12 (Coordination and Writing Speed) were excluded for the purposes of this study. The reliabilities of the various tests ranged from 0,71 to 0,93 for standard 10 pupils, according to Kuder-Richardson Formula 8 (Fouché & Verwey, 1991).

Procedure
For the purposes of this study only the records of students who had completed both the GSAT and the SAT were used. A total of 1 598 complete records were obtained.

Statistical analysis
To demonstrate the effects of differential skewness of variables in factor analysis a joint analysis of the subtests of the GSAT and SAT was done. Principal Factor Analysis was used to extract the factors and rotation to simple structure was done by means of a Direct Oblimin rotation. Coefficients of skewness and kurtosis were computed for all the variables. Following that a Multiple Battery Factor Analysis (MBFA) was done using the extension of Browne (1980) and the programme of Cudeck (1991).

RESULTS
Principal objective: The power of Multiple Battery Factor Analysis in coping with the effects of differential skewness of variables in factor analysis.
As a first step in the analysis, the matrix of intercorrelations of the GSAT and SAT was computed. It is given in Table 1.
From Table 1 it can be seen that the cross-correlations (shaded area) between the two batteries vary from low to moderate in magnitude.
Next, a joint factor analysis of the GSAT and SAT was done: The eigenvalues of the unreduced intercorrelation matrix are given in Table 2. As can be seen three of the eigenvalues were greater than unity. Accordingly three factors were extracted (Kaiser, 1961).  The obtained factor matrix was rotated to simple structure by means of a Direct Oblimin rotation and is given in Table 3.
Factor 2 has moderate to high loadings on SAT 1 (Verbal Comprehension), SAT 2 (Calculations), SAT 3 (Disguised Words), SAT 4 (Comparison), SAT 9 (Memory-paragraph), and SAT 10 (Memory-symbols). The meaning of Factor 2 is not very clear. It has loadings on the verbal subtests of the SAT and also on the numerical subtests.
Factor 3 has moderate to high loadings on GSAT 1 (Word Analogies), GSAT 3 (Verbal Reasoning), and GSAT 5 (Word Pairs). Factor 3 is clearly a verbal factor, but loads only on the GSAT and not the SAT.
The coefficients of skewness of the various measures of the GSAT and the SAT are also given in Table 3. From these coefficients it would appear that the distributions of the GSAT are quite skew. The indices range from 1,818 to -2,111. By contrast the distributions of the SAT are only moderately skew. The indices range from 0,450 to -1,248.
From the foregoing it would appear that differential skewness of the variables have clouded the issue and have yielded factors of skewness.
Following this, the GSAT and SAT were subjected to a Multiple Battery Factor Analysis (Browne, 1980;Cudeck, 1980Cudeck, , 1991. As a first step in the analysis goodness of fit statistics were computed successively for one, two, three and four factors, and are given in Table 4. From the upper-tail probabilities, given in Table 4, it is clear that a three-factor-solution is optimal. The Tucker-Lewis reliability coefficient of 0,967 is also highly acceptable. The average absolute off-diagonal residual is 0,046, and indicates a very good fit. The rotated factor matrix (Direct Quartimin) is given in Table 5. Table 5 shows that Factor 1 is well determined with high loadings on GSAT 4 (Pattern Completion), GSAT 6 (Figure Analogies), SAT 5 (Pattern Completion), SAT 6 (Figure Series), SAT 7 (Spatial 2D), and SAT 8 (Spatial 3D). Factor 1 is clearly a factor of non-verbal reasoning.
Factor 2 has moderate to high loadings on GSAT 1 (Word Analogies), GSAT 3 (Verbal Reasoning), GSAT 5 (Word Pairs), SAT1 (Verbal Comprehension), SAT 3 (Disguised Words), SAT 9 (Memory-paragraph) and SAT 10 (Memory-symbols). Factor 2 is a well determined verbal factor and has loadings on both the GSAT and SAT. Factor 3 has moderate to high loadings on GSAT 2 (Number Series), SAT 2 (Calculations), and SAT 4 (Comparison). SAT 4 is a measure of speed and accuracy. However, several of the items are numerical in form. This probably accounts for the loading of 0,303 on Factor 3. Factor 3 is thus essentially a number factor.
The three factors are strongly positively correlated, suggesting an underlying factor of general intelligence.
From the foregoing it is clear that MBFA successfully identified the underlying structure of the GSAT and SAT, despite differential skewness of the variables.

DISCUSSION
As far as the principal objective of the study is concerned, it was found that a joint factor analysis of the GSAT and SAT resulted in a distorted structure due to the effects of differential skewness of the variables. However, in applying MBFA to the same data a clear and well defined structure was obtained. MBFA is therefore not disturbed by moderate degrees of skewness of the variables used.
The value of MBFA in coping with moderate degrees of skewness of the variables used, has been shown in the present study, but further research in this regard is essential. Cut-off points in respect of the indices of skewness and kurtosis of the variables need to be established. The following questions need to be answered: How extreme must the indices of skewness and kurtosis be before the structure underlying the multiple batteries of tests will be distorted?
To what extent can normalisation of variables be used to overcome the effects of skewness and kurtosis?
Why is MBFA not disturbed by the effects of differential skewness of variables?
An interesting challenge to creative statisticians would be to partial out the effects of skewness from product-moment correlations.
From the foregoing it is clear that MBFA is a very useful technique and can be used to identify the common factors (constructs) underlying two or more batteries of test.