| Abstract: | This paper examines the accuracy with which decision makers can predict the effects of the characteristics of a research study on their evaluations of its usefulness to their work. It draws on survey data from a sample of 155 decision makers in mental health: each of them predicted the effects of 26 research characteristics on usefulness and rated each of two research studies on usefulness and the extent to which each characteristic described the study. The principal finding is that decision makers' predictions of the effects of characteristics on usefulness correspond much more closely to the effects actually observed when these effects are calculated across individuals than when they are taken for a single individual rating one (or two) studies. After examining several alternative explanations, it is suggested that an individual decision maker uses multiple evaluation functions to judge research, weighting the characteristics differently in each, and that investigators should consider the possibility of such variation whenever they attempt to model individuals' evaluation functions. |
