DEVELOPMENT AND VALIDATION OF A COMPUTERIZED INTERPRETATION SYSTEM FOR PERSONNEL TESTS

A computerized system was developed for generating narrative interpretations of scores from a battery of personnel screening tests. The report structure and interpretive statement library were designed to capture the test expertise and interpretive strategies of a panel of testing experts. This was accomplished by enumerating the questions that the experts believed the battery could answer, developing answers to these questions, and devising rules for selecting the appropriate answers based on test-battery scores. The accuracy, thoroughness, readability, and coherence of the computer-generated reports were evaluated in comparison to reports generated by human experts for the same examinees. Results of the evaluation showed the computerized reports to be more accurate and thorough, as readable, and somewhat less coherent than interpretations generated by the typical human expert. The computerized system development and validation strategies described are useful for other applications in which numbers are interpreted in a narrative report format.