Identifying and Understanding the Effects of Unmotivated Examinees on Test Dimensionality Using Optimal Appropriateness Measurement

Recently, a question was raised as to whether the multidimensionality of some professional licensing exams is due to the administration of subtests measuring conceptually distinct skills or, alternatively, strategic preparation on the part of groups of examinees attempting to cope with the demands of multiple hurdle certification systems. This article illustrates a way to investigate this issue with optimal appropriateness measurement (OAM) methods and confirmatory factor analysis (CFA). Specifically, using the former paper-and-pencil American Institute of Certified Public Accountants (AICPA) Uniform Examination as an example, OAM methods were used to identify examinees that appeared unmotivated on 2 of the 4 AICPA exam subtests. Dimensionality was studied by using CFA to compare the fit of single- and 4-factor models before and after removing flagged examinees. The results indicated that the 4-factor model provided better fit than a unidimensional model even after removing nearly 30% of respondents, thus weakening the claim that multidimensionality could be attributed solely to strategic preparation.