Situational effects may account for gain scores in cognitive ability testing: A longitudinal SEM approach

Mean gain scores for cognitive ability tests between two sessions in a selection setting are now a robust finding, yet not fully understood. Many authors do not attribute such gain scores to an increase in the target abilities. Our approach consists of testing a longitudinal SEM model suitable to this view. We propose to model the scores' changes of a battery of tests between two sessions with a single factor, namely the change in the situational component of the scores. The situational component encompasses all effects due to the specificity of the state of the person in the current situation (e.g., anxiety level, tiredness, test-taking practice) and is allowed to vary from one session to another. By definition, this single component is supposed to influence all tests at a given session. In particular cases such as high-stake selection settings, where applicants are likely to train themselves before retaking the tests, situational factors might even suffice to explain mean score increases. Empirically, our latent change model closely fitted the scores of 752 applicants for entry into the French Aircraft Pilot Training, gathered on a set of three tests (visual perception, mechanical comprehension, and selective attention). Gain scores of moderate to strong effect sizes could be explained by common situational effects, with no need for admitting change on ability components. Therefore, gain scores may be understood as construct-irrelevant changes.