The differential item functioning and structural equivalence of a nonverbal cognitive ability test for five language groups

Abstract

Orientation: For a number of years, eliminating a language component in testing by using nonverbal cognitive tests has been proposed as a possible solution to the effect of groups’ languages (mother tongues or first languages) on test performance. This is particularly relevant in South Africa with its 11 official languages.

Research purpose: The aim of the study was to determine the differential item functioning (DIF) and structural equivalence of a nonverbal cognitive ability test (the PiB/SpEEx Observance test [401]) for five South African language groups.

Motivation for study: Cultural and language group sensitive tests can lead to unfair discrimination and is a contentious workplace issue in South Africa today. Misconceptions about psychometric testing in industry can cause tests to lose credibility if industries do not use a scientifically sound test-by-test evaluation approach.

Research design, approach and method: The researcher used a quasi-experimental design and factor analytic and logistic regression techniques to meet the research aims. The study used a convenience sample drawn from industry and an educational institution.

Main findings: The main findings of the study show structural equivalence of the test at a holistic level and nonsignificant DIF effect sizes for most of the comparisons that the researcher made.

Practical/managerial implications: This research shows that the PIB/SpEEx Observance Test (401) is not completely language insensitive. One should see it rather as a language-reduced test when people from different language groups need testing.

Contribution/value-add: The findings provide supporting evidence that nonverbal cognitive tests are plausible alternatives to verbal tests when one compares people from different language groups.