Towards a theory of algorithm-determined cognitive test construction

A rationale for, and data from, a trial of a theory of item generation by algorithms whose origins are cognitive models of task performance are presented. Since Spearman (1904), intelligence has been operationally defined and assessed in human subjects by administering identical test items whose content and order have been fixed only after empirical iterations. In our approach, intelligence is ostensively defined by theoretically determined algorithms used for item construction and presentation. Knowledge of what cognitive factors limit human performance makes it possible to vary within tightly specified parameters those features of the tasks that contribute to difficulty, which we call radicals, to let those components of the tasks that do not contribute to difficulty vary randomly, and to counterbalance aspects of answer production that might induce biases of response. Empirical data are based on the generation of five different short tests demanding only functional literacy as a prerequisite for their execution. Four parallel forms of each test were administered to young male Army recruits whose scores were collated with their Army Entrance Test results, which were not previously known to us. Results show that the parallel, algorithm-generated item sets are statistically invariant, which item generation theory demands; and that the individual tests differentially predict Army Entrance Test scores. We conclude that IQ test performances are parsimoniously explained by individual differences in encoding, comparison and reconstructive memory processes.