Reliability coefficients as a function of individual differences induced by a learning process assuming identical organisms

In this article we show that a particular mathematical learning model, the Bower-Trabasso (1964) concept identification model taken together with an assumption of independence of replicate measurements, implies the existence of substantial and statistically significant performance differences across individuals. The individual differences in turn imply a sizeable reliability coefficient. These results contradict naive intuition, for this model (like many other mathematical models of learning) assumes that all individuals begin the experiment with identical parameter values for the process under study. Thus at least one such model has the characteristic of implying the generation of individual performance differences among originally identical organisms.Examination of data from an experiment by Cotton shows that the Hoyt reliability coefficient under classical test theory, a lower bound for the (composite) reliability of total scores for a series of trials, increases with the number of trials analyzed and exceeds the corresponding theoretical values implied by the Bower-Trabasso model. An experiment by Levine was also analyzed because its use of blank trials between feedback trials permitted direct calculation of composite reliability (or more properly stated composite consistency). For this experiment, the theoretical development just discussed (Case I) was used together with Restle's hypothesis selection model specialized to include a local consistency assumption, the so-called P₂ model of Gregg and Simon (Case II). Moderate conformity of empirical and theoretical reliabilities was found, with discrepancies between observed and predicted values usually being smaller with Case II. However, the Hoyt reliability coefficient is not a lower bound for composite reliability in Case II, because composite reliability is underestimated when identical stimuli are not used for comparable trials.Despite the Bower-Trabasso assumption of no initial differences, it seems reasonable to attribute the difference between predicted and obtained reliabilities to preexisting individual differences. Implications of the tentative conclusion that individual differences in concept identification performance are attributable to a combination of preexisting differences and differences induced in a current task are discussed briefly.