Derivation of individualised item interrelatedness indices of careless responding

Abstract

Orientation: Presently, carelessly responding (CR) individuals are omitted in terms of several individual indices, including consistency-type indices (that compare performance on only a limited number of matched item pairs), and subsequently, the effectiveness of such screening is evaluated in terms of, among others, the group mean item interrelatedness (IIR) (based on all J[J – 1] item pairs).

Research purpose: This research aims to develop individualised versions of the group IIR measures to render them applicable during the screening phase as substitutes for the presently used consistency-type indices.

Motivation for the study: Such individual consistency indices may be used together with other CR indices to jointly determine the eventual evaluation results.

Research approach/design and method: To develop the intended CR indices mathematical-statistical principles were applied on the product moment correlation and coefficient alpha formulae.

Main findings: Three individual IIR indices have been developed which show individual respondents’ respective contributions to the mean item inter-correlation and to coefficient alpha, as measures of group mean IIR.

Practical/managerial implications: These indices may be used during screening in lieu of the existing restrictive consistency indices.

Contribution/value-add: Carelessly responding respondents who previously may have survived screening because of less-inclusive consistency-type IIR indices and consequently may have negatively affected the eventual evaluation results, are now screened out.