Useful scientific theories are useful: A reply to Rouder, Pratte, and Morey (2010)

In a recognition memory experiment, Mickes, Wixted, and Wais (2007) reported that distributional statistics computed from ratings made using a 20-point confidence scale (which showed that the standard deviation of the ratings made to lures was approximately 0.80 times that of the targets) essentially matched the distributional statistics estimated indirectly by fitting a Gaussian signal-detection model to the receiver-operating characteristic (ROC). We argued that the parallel results serve to increase confidence in the Gaussian unequal-variance model of recognition memory. Rouder, Pratte, and Morey (2010) argue that the results are instead uninformative. In their view, parametric models of latent memory strength are not empirically distinguishable. As such, they argue, our conclusions are arbitrary, and parametric ROC analysis should be abandoned. In an attempt to demonstrate the inherent untestability of parametric models, they describe a non-Gaussian equal-variance model that purportedly accounts for our findings just as well as the Gaussian unequal-variance model does. However, we show that their new model—despite being contrived after the fact and in full view of the to-be-explained data—does not account for the results as well as the unequal-variance Gaussian model does. This outcome manifestly demonstrates that parametric models are, in fact, testable. Moreover, the results differentially favor the Gaussian account over the probit model and over several other reasonable distributional forms (such as the Weibull and the lognormal).     

Positive and negative remember judgments and ROCs in the plurals paradigm: Evidence for alternative decision strategies

Using old-new ratings and remember-know judgments we explored the plurals paradigm, in which studied words must be distinguished from plurality-changed lures. The paradigm allowed us to investigate negative remembering—that is, the remembering of a plural-altered study item; capacity for this judgment was found to be poorer than or equivalent to the conventional positive remembering. A response-bias manipulation affected positive but not negative remembering. The ratings were used to construct ROC curves and test the prediction of the most common dual-process theory of recognition memory (Yonelinas, 2001) that the amount of recollection can be independently estimated from ROC curves and from remember judgments. By fitting the individual data with pure signal detection theory (SDT) models and dual-process models that combined SDT and high-threshold components (HTSDT), we identified two types of subjects. For those who were better described by HTSDT, the predicted convergence of remember-know and ROC measures was observed. For those who were better described by SDT, the ROC intercept could not predict the remember rate. The data are consistent with the idea that all subjects rely on the same representation but base their decisions on different partitions of a decision space.     

The strength-based mirror effect in subjective strength ratings: The evidence for differentiation can be produced without differentiation

Criss (Cognitive Psychology 59:297?C319, 2009) reported that subjective ratings of memory strength showed a mirror effect pattern in which strengthening the studied words increased ratings for targets and decreased ratings for lures. She interpreted the effect on lure items as evidence for differentiation, a process whereby lures produce a poorer match to strong than to weak memory traces. However, she also noted that participants might use different mappings between memory evidence and levels of the rating scale when they expected strong versus weak targets; that is, the effect might be produced by decision processes rather than differentiation. We report two experiments designed to distinguish these accounts. Some participants studied pure lists of weak or strong items (presented once or five times, respectively), while others studied mixed lists of half weak and half strong items. The participants from both groups had pure-strength tests: Only strong or only weak items were tested, and the participants were informed of which it would be before the test. The results showed that strength ratings for lures were lower when strong versus weak targets were tested, regardless of whether the study list was pure or mixed. In the mixed-study condition, the effect was produced even after identical study lists, and thus the same degree of differentiation in the studied traces. Therefore, our results suggest that the strength-rating mirror effect is produced by changes in decision processes.     

Subjective frequency and imageability ratings for 3,600 French nouns

Subjective frequency and imageability estimates for a sample of 3,600 French nouns were collected from two independent groups of 72 young adults each. Both groups received standard instructions and provided their ratings on a 7-point scale. The timing, sequencing, presentation of lexical stimuli, and recording of responses were controlled by a computer. All estimates of internal consistency and test—retest reliability (≥.98) confirm the high level of precision and reliability of the ratings. Correlations with ratings drawn from similar studies were found to be positive and significant for subjective frequency (r ≥ .85) and for imageability (r ≥ .69). Subjective frequency was positively and significantly correlated with objective frequency estimates drawn from 10 different sources (r ≥ .42). Subjective frequency and imageability were significantly correlated (r=.26), a relationship that was driven primarily by a sudden drop in imageability ratings for words with a subjective frequency rating below 2.5. The methodological implications of these findings are discussed. The ratings can be downloaded as supplemental materials from brm.psychonomic-journals.org/content/supplemental.     

Comparing Four Methods of Integrating Parent and Teacher Symptom Ratings of Attention-deficit/hyperactivity Disorder (ADHD)

Although parents and teachers are valid informants in the assessment of childhood attention-deficit/hyperactivity disorder (ADHD), there is relatively little systematic research on how these ratings should be optimally combined. We compared four methods of ADHD assessment to determine how well they identified impaired children: (1) parent only, (2) teacher only, (3) parent or teacher (‘or rule’), and (4) parent and teacher (‘and rule’). We obtained parent and teacher ratings of ADHD from the Disruptive Behavior Disorder Rating Scale on 232 5- to 10-year-old children (69% male; 47% Caucasian) with (n = 121) and without (n = 111) ADHD. We used receiver operating characteristic curves (ROC) and seemingly unrelated regression analyses (SUR) to evaluate how accurately each method identified categorically- and dimensionally-defined measures of functional impairment. Parent ratings of ADHD optimally identified globally impaired children based on categorical and dimensional measures. However, teacher ratings of ADHD most accurately identified children who were negatively regarded by peers using categorical, but not dimensional, measures. No ADHD assessment method effectively identified children with academic difficulties. Although multiple informants are valuable in the assessment of ADHD, no single method was consistently superior in identifying impaired children across domains. We consider alternative assessment strategies in ADHD as well as other potential factors that may contribute to modest agreement among informants.     

A note on the estimation of test reliability by the Kuder-Richardson formula (20)

The Kuder-Richardson formula (20) is rewritten to be identical with the simplest formula, (21), except for the addition of a term involving the standard deviation, _p , of the itemp's. If _p can be estimated, a rapid and superior estimate of test reliability is possible in contrast to the simpler formula (21) used when the number of items and mean and standard deviation of test scores are known.Kuder, G. F. and Richardson, M. W. The theory of the estimation of test reliability.Psychometrika. 1937, 2, 151–160.     

The influence of averaging and noisy decision strategies on the recognition memory ROC

Many single- and dual-process models of recognition memory predict that the ratings and remember-know receiver operating characteristics (ROCs) are the same, but Rotello, Macmillan, and Reeder (2004) reported that the slopes of the remember-know and ratingsz-transformed ROCs (zROCs) are different. The authors show that averaging introduces nonlinearities to the form of thezROC and that ratings and remember-knowzROCs are indistinguishable when constructed in a conventional manner. The authors show, further, that some nonoptimal decision strategies have a distinctive, nonlinear effect on the form of the single-process continuous-statezROC. The conclusion is that many factors having nothing to do with the nature of recognition memory can affect the shape ofzROCs, and that therefore, the shape of thezROC does not, alone, characterize different memory models.     

Conscious and unconscious discriminations between true and false memories

When subjects give higher confidence or memory ratings to a test word in a recognition test, do they simply raise their criterion without making better discrimination, or do they raise both criterion and true discrimination between the studied words (SW) and the lures? Given that previous studies found subjects’ false alarm responses to lures slower than to SW, and recognition latency inversely correlated with the confidence rating, can the latency difference between the lures and SW be accounted for by confidence or memory ratings? The present results showed that when subjects gave higher confidence or memory ratings, both their bias and sensitivity were raised, indicating that they could consciously distinguish the lures from the SW. However, a latency difference between true and false recognitions persisted after confidence and memory ratings were held constant, suggesting an unconscious source of discrimination between the two types of memory.     

Reliability for the law of comparative judgment

A variance-components analysis is presented for paired comparisons in terms of three components:s, the scale value of the stimuli;d, a deviation from the linear model specified by the law of comparative judgment; andb, a binomial error component. Estimates are given for each of the three variances, _s ² , _d ² , and _b ² . Several coefficients, analogous to reliability coefficients, based on these three variances are indicated. The techniques are illustrated in a replicated comparison of handwriting specimens.This research was jointly supported in part by Princeton University, the Office of Naval Research under contract Nonr-1858(15), and the National Science Foundation under grant NSF G-642, and in part by Educational Testing Service. Reproduction in whole or in part is permitted for any purpose of the United States Government.Thanks are due to Ledyard Tucker and Frederic Lord for valuable suggestions on the development presented here.     

The revelation effect for autobiographical memory: A mixture-model analysis

Participants provided information about their childhood by rating their confidence about whether they had experienced various events (e.g., “broke a window playing ball”). On some trials, participants unscrambled a key word from the event phrase (e.g., wdinwo—window) or an unrelated word (e.g., gnutge—nugget) before seeing the event and giving their confidence ratings. The act of unscrambling led participants to increase their confidence that the event occurred in their childhood, but only when the confidence rating immediately followed the act of unscrambling. This increase in confidence mirrors the “revelation effect” observed in word recognition experiments. In the present article, we analyzed our data using a new signal detection mixture distribution model that does not require the researcher to know the veracity of memory judgments a priori. Our analysis reveals that unscrambling a key word or an unrelated word affects response bias and discriminability in autobiographical memory tests in ways that are very similar to those that have been previously found for word recognition tasks.     

Matching of numerical symbols with number of responses by pigeons

Pigeons were trained to peck a certain number of times on a key that displayed one of several possible numerical symbols. The particular symbol displayed indicated the number of times that the key had to be pecked. The pigeons signalled the completion of the requirement by operating a separate key. They received a food reward for correct response sequences and time-out penalties for incorrect response sequences. In the first experiment nine pigeons learned to allocate 1, 2, 3 or 4 pecks to the corresponding numerosity symbols s ₁, s ₂, s ₃ and s ₄ with levels of accuracy well above chance. The second experiment explored the maximum set of numerosities that the pigeons were capable of handling concurrently. Six of the pigeons coped with an s ₁–s ₅ task and four pigeons even managed an s ₁–s ₆ task with performances that were significantly above chance. Analysis of response times suggested that the pigeons were mainly relying on a number-based rather than on a time-based strategy. Received: 11 October 1999 / Accepted after revision: 27 January 2000     

Using response time modeling to distinguish memory and decision processes in recognition and source tasks

Receiver operating characteristic (ROC) functions are often used to make inferences about memory processes, such as claiming that memory strength is more variable for studied versus nonstudied items. However, decision processes can produce the ROC patterns that are usually attributed to memory, so independent forms of data are needed to support strong conclusions. The present experiments tested ROC-based claims about the variability of memory evidence by modeling response time (RT) data with the diffusion model. To ensure that the model can correctly discriminate equal- and unequal-variance distributions, Experiment 1 used a numerousity discrimination task that had a direct manipulation of evidence variability. Fits of the model produced correct conclusions about evidence variability in all cases. Experiments 2 and 3 explored the effect of repeated learning trials on evidence variability in recognition and source memory tasks, respectively. Fits of the diffusion model supported the same conclusions about variability as the ROC literature. For recognition, evidence variability was higher for targets than for lures, but it did not differ on the basis of the number of learning trials for target items. For source memory, evidence variability was roughly equal for source 1 and source 2 items, and variability increased for items with additional learning attempts. These results demonstrate that RT modeling can help resolve ambiguities regarding the processes that produce different patterns in ROC data. The results strengthen the evidence that memory strength distributions have unequal variability across item types in recognition and source memory tasks.     

Evaluating characteristics of false memories: Remember/know judgments and memory characteristics questionnaire compared

Subjects hearing a list of associates to a nonpresented lure word later often claim to have heard the lure (Deese, 1959; Roediger & McDermott, 1995). To examine the characteristics of such false memories, subjects completed a memory characteristics questionnaire (MCQ; Johnson, Foley, Suengas, & Raye, 1988) or made remember/know (RK; Gardiner & Java, 1993) judgments for previously heard theme associates and nonpresented lures. MCQ ratings indicated that false memories for lures had less auditory detail and less remembered feelings and reactions than memories for presented words. In addition, rates of false recognition for lures were significantly lower than rates of correct recognition when items from various themes were intermixed instead of blocked at acquisition and subjects made MCQ ratings instead of RK judgments. This demonstrates that false memories can be affected both by how they are acquired and by how extensively they are examined at retrieval.     

Working memory predicts the rejection of false memories

The relationship between working memory capacity (WMC) and false memories in the memory conjunction paradigm was explored. Previous research using other paradigms has shown that individuals high in WMC are not as likely to experience false memories as low-WMC individuals, the explanation being that high-WMC individuals are better able to engage in source monitoring. In the memory conjunction paradigm participants are presented at study with parent words (e.g., eyeglasses, whiplash). At test, in addition to being presented with targets and foils, participants are presented with lures that are composed of previously studied features (e.g., eyelash). It was found that high-WMC individuals had lower levels of false recognition than low-WMC individuals. Furthermore, recall-to-reject responses were analysed (e.g., “I know I didn't see eyelash because I remember seeing eyeglasses”) and it was found that high-WMC individuals were more likely to utilise this memory editing strategy, providing direct evidence that one reason that high-WMC individuals are not as prone to false memories is because they are better able to engage in source monitoring.     

Strategic influences on recollection in the exclusion task: Electrophysiological evidence

One assumption underlying the use of the exclusion task as part of the process dissociation procedure is that studied items are successfully excluded only when they are recollected. The present study employed event-related potentials (ERPs) to demonstrate that successful exclusion does not necessarily require recollection. In two experiments, the study tasks for to-be-excluded items were identical, but the tasks employed withtarget items differed, giving better memory for these items in Experiment 1 than in Experiment 2. Successfully excluded items elicited the ERP signature for recollection—theleft parietal old/new effect—in Experiment 2 only. These findings indicate that the subjects adopted different retrieval strategies in the two experiments. It is suggested that they made more use of source information about to-be-excluded items in the second experiment than in the first.     

Early activation of object names in visual search

In a visual search experiment, participants had to decide whether or not a target object was present in a fourobject search array. One of these objects could be a semantically related competitor (e.g.,shirt for the targettrousers) or a conceptually unrelated object with the same name as the target—for example,bat (baseball) for the targetbat (animal). In the control condition, the related competitor was replaced by an unrelated object. The participants’ response latencies and eye movements demonstrated that the two types of related competitors had similar effects: Competitors attracted the participants’ visual attention and thereby delayed positive and negative decisions. The results imply that semantic and name information associated with the objects becomes rapidly available and affects the allocation of visual attention.     

Memory conjunction clusters: Influence of familiarity and recollection

In the memory conjunction paradigm, the number of times that constituents of conjunction lures were studied and the method of presentation were varied. In two experiments, participants were presented with eight parent items that could be recombined at test to form a conjunction lure. The constituents that were shared between the parent items and the conjunction lures were either presented in the same words (e.g., blackmail and jailbird presented four times each for the conjunction lure blackbird) or in different words (e.g., the targets footstool, footlocker, foothill, footbridge, baseball, softball, basketball, and golfball for the conjunction lure football). In both experiments, rates of false recognition were higher in the Different condition as opposed to the Same condition. These results provide evidence that participants in the Same condition were able to utilise a recall-to-reject strategy by remembering the repeatedly presented parent word. In the Different condition, participants were not able to utilise that strategy and instead relied on the familiarity of the repeatedly presented constituents which led to higher rates of false recognition.     

Using logistic regression to estimate delay-discounting functions

The monetary choice questionnaire (MCQ) and similar computer tasks ask preference questions in order to ascertain indifference, the perceived equivalence of immediate versus larger delayed rewards. Indifference data are then fitted with a hyperbolic function, summarizing the decline in perceived value with delay time. We present a fitting method that estimates the hyperbolic parameterk directly from survey responses. Binary preferences are modeled as a function of time (X₂) and a transformed reward ratio (X ₁), yielding logistic regression coefficients β₂ and β₁. The hyperbolic parameter emerges ask = β₂/β₁, where the logistic predictedp = .5 (the definition of indifference). The MCQ was administered to 1,073 adolescents and was scored using both standard and logistic methods. The means for ln(k) were similar (standard, −4.53; logistic, −4.51), and the results were highly correlated (ρ = .973). Simulated MCQ data showed thatk was unbiased, except where β₁ ≥ −1, indicating a vague survey response. Jackknife standard errors provided excellent coverage.     

Forced confabulation affects memory sensitivity as well as response bias

A signal detection analysis assessed the extent to which forced confabulation results from a change in memory sensitivity (d _a ), as well as response criterion (β). After viewing a crime video, participants answered 14 answerable and 6 unanswerable questions. Those in the voluntary guess condition had a “don’t know” response option; those in the forced guess condition did not. One week later, the same questions were answered using a recognition memory test that included each participant’s initial responses. As was predicted, on both answerable and unanswerable questions, participants in the forced guess condition had significantly lower response criteria than did those who voluntarily guessed. Furthermore, on both answerable and unanswerable questions, d _a scores were also significantly lower in the forced than in the voluntary guess condition. Thus, the forced confabulation effect is a real memory effect above and beyond the effects of response bias; forcing eyewitnesses to guess or speculate can actually change their memory.     

Examining the basis for illusory recollection: The role of remember/know instructions

Curiously, studies using the remember/know paradigm to measure recollective experience show that people often vividly remember events that never occurred, a phenomenon referred to asillusory recollection. Two experiments tested the hypothesis that false remember responses in the converging associates, or Deese/Roediger-McDermott (DRM) paradigm, reflect accurate memory for the study episode, rather than false recollection of critical lures. To test this hypothesis, we used standard remember instructions that emphasized recollection of the study context by allowing participants to use memory of surrounding list items as evidence for recollection, or we used modified instructions that did not include memory for surrounding list items as a basis for recollection. Results showed that, as compared with the standard instruction condition, the modified instructions selectively reduced reports of false remember responses to critical lures, but did not affect remember responses to studied items. By contrast, remember responses to critical lures were unaffected by an instruction condition that excluded the use of voice information as evidence for remembering. These results suggest that remember responses to falsely recognized items are driven partly by retrieval of studied items, rather than illusory recollection of the critical lures themselves. They further point to the importance of instructions in influencing subjective reports.