An investigation of gender-based differences in assessment instruments: A test of measurement invariance

Abstract

Orientation: Practitioners and researchers often assume that the psychometric instruments they use are invariant and that they therefore measure similar constructs in a comparable manner across men and women respondents. This assumption is, however, rarely tested, leading to an undetected bias in research findings or an adverse impact because of the presence of non-invariance.

Research purpose: After presenting essential information about measurement invariance (MI) and arguing for the testing thereof, this research aims to reveal the prevalence of MI across several frequently used psychometric instruments credulously used based on the assumption the revenant constructs are measured equivalently across gender exists.

Motivation for the study: Firstly, this study aims to increase awareness regarding MI, a property that can be tested statistically. Secondly, the research aims to make practitioners aware of the presence of bias in psychometric instruments, specifically to identify instruments that could be included in investigations which attempt to understand gender matters in the workplace.

Research approach/design and method: Cross-sectional survey data, pertaining to seven standard instruments, related to innovative work behaviour, were analysed. Pairwise, multigroup confirmatory factor analyses with robust maximum likelihood estimation were used to examine configural, metric, intercept and strict invariance, as well as the equivalence of the latent means.

Main findings: The findings were binary, with four of the instruments showing MI at an equal latent means level, whilst three instruments were non-invariant at the configural level. Measurement invariance was either accepted completely or rejected completely.

Practical/managerial implications: The serratedness of findings, even when using well-recognised and frequently used psychometric instruments, exposes the prevalence of non-invariance in some instruments, thereby necessitating the standard testing for MI. These findings also specify the instruments that are MI (in terms of gender), which allow other researchers and practitioners to use these instruments with more confidence when measuring and comparing men and women respondents in their studies.

Contribution/value-add: This research demonstrates the ease with which MI testing can be performed and alerts researchers to do MI testing when conducting cross-group studies, as the prevalence of measurement non-invariance is high.