Sampling Methods for Identifying Differences in Source Reliability

This study explored observer performance in using across-trial and within-trial variability information to weight sources based on reliability. Three trained observers performed a multi-element, visual signal-detection task under 3 block-type conditions: a fixed block condition and 2 random conditions. In the fixed block condition, the observers had both within- and across-trial variability information to identify differences in source reliability. The random conditions eliminated the across-trial information, leaving only within-trial variability information in 1 case and neither within- nor across-trial variability information in another case. There was a significant block-type effect. Observers could use differences in across-trial variability of individual sources to assign weights. Although there was a difference in the weights assigned to reliable and unreliable sources in the partial-random condition (in which within-trial variability information was available), only 1 participant showed a significant difference in the weight assignment relative to the full-random condition. Thus, altogether, observers were not very efficient at using within-trial variability to weight sources accordingly.