On the form of ROCs constructed from confidence ratings

A classical question for memory researchers is whether memories vary in an all-or-nothing, discrete manner (e.g., stored vs. not stored, recalled vs. not recalled), or whether they vary along a continuous dimension (e.g., strength, similarity, or familiarity). For yes-no classification tasks, continuous- and discrete-state models predict nonlinear and linear receiver operating characteristics (ROCs), respectively (D. M. Green & J. A. Swets, 1966; N. A. Macmillan & C. D. Creelman, 1991). Recently, several authors have assumed that these predictions are generalizable to confidence ratings tasks (J. Qin, C. L. Raye, M. K. Johnson, & K. J. Mitchell, 2001; S. D. Slotnick, S. A. Klein, C. S. Dodson, & A. P. Shimamura, 2000, and A. P. Yonelinas, 1999). This assumption is shown to be unwarranted by showing that discrete-state ratings models predict both linear and nonlinear ROCs. The critical factor determining the form of the discrete-state ROC is the response strategy adopted by the classifier.     

Finite-state recall revealed by backward learning curves constructed from confidence ratings

Two experiments were carried out to show that the constancy of Type II d’ in recall is not an artifact, but is due to finite-state processes as suggested by Bernbach (1967). The following conclusions were reached: (1) The constancy of Type II d’ is not an artifact due to the inclusion of intralist intrusions in the class of incorrect responses; (2) backward learning curves constructed from confidence ratings gave clear evidence of all-or-none recall; (3) recall appeared to be all or none whether the subjects learned by rote or with a mnemonic; (4) the discriminability index derived from Luce’s (1959) choice theory did not exhibit constancy over trials, unlike Type II d’ and despite the fact that backward rating curves indicated all-or-none processes.     

A comparison of type II operating characteristics derived from confidence ratings and from latencies

On each of a series of trials, S judged two stimuli to be the same or different. S then made a confidence rating of his judgment. The difficulty of discrimination and feedback of response information (latency and correctness) were manipulated. Feedback had no effect. Latencies conditional on confidence rating decreased with increasing confidence and remained relatively constant with changes in discrimination difficulty. Type II operating characteristics were derived from latencies and from confidence ratings; both appropriately reflected differences in task difficulty.     

The ROC Toolbox: A toolbox for analyzing receiver-operating characteristics derived from confidence ratings

Signal-detection theory, and the analysis of receiver-operating characteristics (ROCs), has played a critical role in the development of theories of episodic memory and perception. The purpose of the current paper is to present the ROC Toolbox. This toolbox is a set of functions written in the Matlab programming language that can be used to fit various common signal detection models to ROC data obtained from confidence rating experiments. The goals for developing the ROC Toolbox were to create a tool (1) that is easy to use and easy for researchers to implement with their own data, (2) that can flexibly define models based on varying study parameters, such as the number of response options (e.g., confidence ratings) and experimental conditions, and (3) that provides optimal routines (e.g., Maximum Likelihood estimation) to obtain parameter estimates and numerous goodness-of-fit measures.The ROC toolbox allows for various different confidence scales and currently includes the models commonly used in recognition memory and perception: (1) the unequal variance signal detection (UVSD) model, (2) the dual process signal detection (DPSD) model, and (3) the mixture signal detection (MSD) model. For each model fit to a given data set the ROC toolbox plots summary information about the best fitting model parameters and various goodness-of-fit measures. Here, we present an overview of the ROC Toolbox, illustrate how it can be used to input and analyse real data, and finish with a brief discussion on features that can be added to the toolbox.     

The cognitive interview technique: Reviving the issues

The cognitive interview technique (CI) has received considerable attention as one of the most succssful interview techniques applied to real-life investigations, particularly police investigations. Although the general effects of CI are well documented, there are grounds for critical assessment. In this paper the theoretical principles behind the cognitive components of CI are provided. Variables that have been manipulated will be reviewed, as will conditions under which CI has been studied. The intention is to generalize across the major studies of CI and explore what patterns are present in the data. Issues are introduced that attempt to refine the theoretical underpinnings of CI, and provide direction for research.     

An independent replication of the effectiveness of the cognitive interview

The cognitive interview developed by Geiselman and co-workers is a set of memory retrieval aids to enhance recall in memory tasks such as eyewitness reports. This research is a replication of the study of Geiselman, Fischer, Firstenberg, Hutton, Sullivan, Avetissian and Prosk (1984) with a German version of the cognitive interview. Twenty-nine subjects participated and saw a short film in an incidental learning situation. Approximately 6 days later the participants were questioned about one scene of the film. Fourteen subjects received the cognitive interview instructions and 15 were questioned without special instructions. First, subjects gave a written free report of the scene and subsequently answered more specific questions concerning details. The results clearly indicate that the cognitive interview enhances memory retrieval in free recall conditions but this effect is reduced to specific questions. The effects are discussed in terms of a possible underlying memory organization and the use of retrieval strategies.     

Using ecphoric confidence ratings to discriminate seen from unseen faces: The effects of retention interval and distinctiveness

Theories of confidence processing for recognition judgments suggest that confidence indexes the degree of match between a presented stimulus and an image in memory (ecphoric similarity). Recent research has demonstrated that having participants rate their confidence that a face had been previously seen provides an equivalent or a better index of the stimulus's status than does eliciting a simple binary response (Sauer, Brewer, & Weber, Journal of Experimental Psychology: General 137: 528-547, 2008). Using a face recognition paradigm, we manipulated retention interval and stimulus distinctiveness to directly test the suggestion that confidence indexes ecphoric similarity and to probe the boundary conditions for using confidence ratings to discriminate seen from unseen faces. Consistent with the proposed ecphoric basis for confidence ratings, mean confidence was higher for previously seen than for unseen faces, and conditions conducive to the formation of strong memories improved confidence-based discrimination. In all conditions, after the application of a classification algorithm, confidence ratings provided a more sensitive index of face status (i.e., seen or unseen) than did binary responses.     

Towards understanding the effects of interviewer training in evaluating the cognitive interview

The effectiveness of the cognitive interview (CI) technique has been demonstrated in experimental studies of eyewitness memory conducted in the USA, Germany, and the UK. Much of the research, however, has used college students as interviewers rather than professionals who regularly interview witnesses (e.g. police officers). The aim of this research was to examine the effects of such CI training using experienced police officers with established interviewing styles. An armed robbery was staged and information was elicited in witness interviews. The CI did not significantly increase correct recall but tended to produce more suppositions and errors. A qualitative analysis of interviewer behaviour showed that many officers had difficulty in applying the new cognitive techniques and in avoiding the use of closed and leading questions. The paper addresses two issues: first, that of training experienced police officers who have received relatively little prior training in investigative interviewing, and second, the extent to which we can generalize from the laboratory data elicited in previous studies of CI.     

On the trait implications of social behaviors: Kindness,intelligence, goodness,and normality ratings for 400 behavior statements

Four hundred stimulus behaviors and their mean normative ratings for kindness, intelligence, goodness, and normality were presented for use in person perception and memory studies. Each of the four normative ratings was based on a separate sample of 35 to 39 undergraduate students from the University of Illinois. Rankings of the mean ratings were provided to facilitate a quick comparison of the behavior ratings along each of the four trait dimensions. In addition, a cluster analysis of the behaviors was reported, using the mean kindness, intelligence, and normality ratings as defining variables. Six clusters were distinguished: (1) behaviors that primarily conveyed kindness; (2) behaviors that primarily conveyed unkindness; (3) behaviors that conveyed an unusual amount of intelligence; (4) behaviors that conveyed general intelligence; (5) behaviors that conveyed a general lack of intelligence; and (6) behaviors that conveyed very little information about kindness or intelligence.     

The relative efficacy of predicting IQ from mother-child interactions using ratings versus behavioral count measures

The relative efficacy of using ratings versus behavioral count measures to predict intelligence from mother-child interactions was investigated for 45 mother-child dyads who constituted a heterogeneous sample with respect to socioeconomic status. These dyads were observed when the children were 36 months of age; children were tested with standardized IQ tests at 36 and 60 months. Social interactions between mothers and children were coded from video tapes with two different systems by independent observers. The behavioral count system was used to code second-by-second the duration and frequency of behaviors during the session. The rating system was used to judge maternal behaviors on three scales following the session. A series of forced stepwise multiple regression analyses compared the predictive utility of the two systems both concurrently and over time. The ratings of maternal behavior yielded high correlations with child IQ both concurrently and over time and were not contributed to significantly by the behavioral count measures. The authors speculate that ratings proved more efficacious because the raters could make more subjective and intuitive judgments concerning maternal behavior.     

Adapting the cognitive interview to enhance long-term (35 years) recall of physical activities

The cognitive interview (CI) was modified for use in an epidemiological study in which respondents were asked to recall their daily physical activities from the distant past (35 years ago). In comparison to a traditional epidemiological interview, the CI elicited many more responses and also more precise responses. Several practical costs, however, were incurred by the CI: additional time to train interviewers and to conduct interviews and difficulties in coding the responses. The costs and benefits of conducting the CI are addressed, along with conceptual and methodological challenges. The article ends with an existential question: Is the CI a singular technique if it can be modified so radically for different settings?     

Gambling on the unconscious: A comparison of wagering and confidence ratings as measures of awareness in an artificial grammar task

We explore three methods for measuring the conscious status of knowledge using the artificial grammar learning paradigm. We show wagering is no more sensitive to conscious knowledge than simple verbal confidence reports but is affected by risk aversion. When people wager rather than give verbal confidence they are less ready to indicate high confidence. We introduce a “no-loss gambling” method which is insensitive to risk aversion. We show that when people are just as ready to bet on a genuine random process as their own classification decisions, their classifications are still above baseline, indicating knowledge participants are not aware of having. Our results have methodological implications for any study investigating whether people are aware of knowing.     

Evaluation of the BSRI masculine and feminine items using desirability and stereotype ratings

This study empirically tested Bem's (1974) assumption that the BSRI Masculinity and Femininity scales measure sex-typed standards of desirable behavior for men and women in American society. The adequacy of items in the two scales was evaluated by Bem's (1974) criteria using two types of desirability and stereotype ratings. Results obtained in all the experimental conditions except one involving Bem's desirability instructions and Bem's rating scale did not support the tested assumption. Implications of these results for revising the BSRI scales were discussed.     

Adult age differences in the realism of confidence judgments: overconfidence, format dependence, and cognitive predictors

Realistic confidence judgments are essential to everyday functioning, but few studies have addressed the issue of age differences in overconfidence. Therefore, the authors examined this issue with probability judgment and intuitive confidence intervals in a sample of 122 healthy adults (ages: 35-40, 55-60, 70-75 years). In line with predictions based on the na?ve sampling model (P. Juslin, A. Winman, & P. Hansson, 2007), substantial format dependence was observed, with extreme overconfidence when confidence was expressed as an intuitive confidence interval but not when confidence was expressed as a probability judgment. Moreover, an age-related increase in overconfidence was selectively observed when confidence was expressed as intuitive confidence intervals. Structural equation modeling indicated that the age-related increases in overconfidence were mediated by a general cognitive ability factor that may reflect executive processes. Finally, the results indicated that part of the negative influence of increased age on general ability may be compensated for by an age-related increase in domain-relevant knowledge.