Modern psychometrics for assessing achievement goal orientation: A Rasch analysis

Background A program of research is needed that assesses the psychometric properties of instruments designed to quantify students' achievement goal orientations to clarify inconsistencies across previous studies and to provide a stronger basis for future research. Aim We conducted traditional psychometric and modern Rasch‐model analyses of the Achievement Goals Questionnaire (AGQ, Elliot & McGregor, 2001 ) and the Patterns of Adaptive Learning Scale (PALS, Midgley et al., 2000 ) to provide an in‐depth analysis of the two most popular instruments in educational psychology. Samples and methods For Study 1, 217 undergraduate students enrolled in educational psychology courses participated. Thirty‐four were male and 181 were female (two did not respond). Participants completed the AGQ in the context of their educational psychology class. For Study 2, 126 undergraduate students enrolled in educational psychology courses participated. Thirty were male and 95 were female (one did not respond). Participants completed the PALS in the context of their educational psychology class. Results Traditional psychometric assessments of the AGQ and PALS replicated previous studies. For both, reliability estimates ranged from good to very good for raw subscale scores and fit for the models of goal orientations were good. Based on traditional psychometrics, the AGQ and PALS are valid and reliable indicators of achievement goals. Rasch analyses revealed that estimates of reliability for items were very good but respondent ability estimates varied from poor to good for both the AGQ and PALS. These findings indicate that items validly and reliably reflect a group's aggregate goal orientation, but using either instrument to characterize an individual's goal orientation is hazardous.