MULTIPLE-CHOICE AND CONSTRUCTED RESPONSE TESTS OF ABILITY: RACE-BASED SUBGROUP PERFORMANCE DIFFERENCES ON ALTERNATIVE PAPER-AND-PENCIL TEST FORMATS

We present an example of an innovative constructed response test format–a write-in/mark-in paper-and-pencil test–as an alternative to the traditional multiple-choice paper-and-pencil test, with the potential for reducing subgroup differences. We present subgroup differences data on these 2 paper-and-pencil test formats on an operational promotional exam in a sample of African American and White firefighters. The tests were designed to measure the same content domain. Using within-subjects data that compared the performance of 13 African American and 14 White fire captains, and between-subjects data that compared the performance of 21 African American and 49 White fire captains, several results were in the predicted direction such that subgroup differences were reduced on the constructed response test. However, these results did not reach statistical significance. Therefore, the study points to the need for additional research to further evaluate the promise of the constructed response test format.