A two-stage logistic regression model for analyzing inter-rater agreement

Studies of agreement commonly occur in psychiatric research. For example, researchers are often interested in the agreement among radiologists in their review of brain scans of elderly patients with dementia or in the agreement among multiple informant reports of psychopathology in children. In this paper, we consider the agreement between two raters when rating a dichotomous outcome (e.g., presence or absence of psychopathology). In particular, we consider logistic regression models that allow agreement to depend on both rater- and subject-level covariates. Logistic regression has been proposed as a simple method for identifying covariates that are predictive of agreement (Coughlin et al., 1992). However, this approach is problematic since it does not take account of agreement due to chance alone. As a result, a spurious association between the probability (or odds) of agreement and a covariate could arise due entirely to chance agreement. That is, if the prevalence of the dichotomous outcome varies among subgroups of the population, then covariates that identify the subgroups may appear to be predictive of agreement. In this paper we propose a modification to the standard logistic regression model in order to take proper account of chance agreement. An attractive feature of the proposed method is that it can be easily implemented using existing statistical software for logistic regression. The proposed method is motivated by data from the Connecticut Child Study (Zahner et al., 1992) on the agreement among parent and teacher reports of psychopathology in children. In this study, parents and teachers provide dichotomous assessments of a child's psychopathology and it is of interest to examine whether agreement among the parent and teacher reports is related to the age and gender of the child and to the time elapsed between parent and teacher assessments of the child.The authors thank the Associate Editor and the referees for their helpful comments and suggestions. We also thank Gwen Zahner for use of data from the Connecticut Child Study, which was conducted under contract to the Connecticut Department of Children and Youth Services. This research was supported by grants HL 69800, AHRQ 10871, HL52329, HL61769, GM 29745, MH 54693 and MH 17119 from the National Institutes of Health.     

The confidence-accuracy relationship in eyewitness identification: effects of lineup instructions, foil similarity, and target-absent base rates

Discriminating accurate from mistaken eyewitness identifications is a major issue facing criminal justice systems. This study examined whether eyewitness confidence assists such decisions under a variety of conditions using a confidence-accuracy (CA) calibration approach. Participants (N = 1,200) viewed a simulated crime and attempted 2 separate identifications from 8-person target-present or target-absent lineups. Confidence and accuracy were calibrated for choosers (but not nonchoosers) for both targets under all conditions. Lower overconfidence was associated with higher diagnosticity, lower target-absent base rates, and shorter identification latencies. Although researchers agree that courtroom expressions of confidence are uninformative, our findings indicate that confidence assessments obtained immediately after a positive identification can provide a useful guide for investigators about the likely accuracy of an identification.     

DIMENSIONS OF FRUSTRATION INTOLERANCE AND THEIR RELATIONSHIP TO SELF-CONTROL PROBLEMS

Frustration intolerance beliefs are central to the theory and practice of Rational Emotive Behavior Therapy. However, there has been little investigation of the content of these beliefs, and empirical evidence linking specific beliefs to distinct psychological problems is sparse. To redress this, the Frustration–Discomfort Scale has been developed as a multidimensional measure. This was used to explore the relationship between the four dimensions of frustration intolerance (emotional intolerance, demands for entitlement, comfort, and achievement) and problems of self-control. Results indicated that the Frustration–Discomfort sub-scales showed differential relationships with self-control problems, independent of self-worth beliefs. This supported the validity and usefulness of the scale and the importance of distinguishing between dimensions of frustration intolerance. This paper is based on research submitted to the University of Edinburgh in part fulfillment of a Doctorate of Philosophy degree. Address correspondence to Neil Harrington, Psychology Department, Stratheden Hospital, Cupar, Fife, KY15 5RR, UK; e-mail: neil@nharr.freeserve.co.uk.     

Sex differences in righting from supine to prone in rats (Rattus norvegicus): a masculinized skeletomusculature is not required

Previous research has shown that sex differences exist in the composition of lateral movements (E. F. Field, I. Q. Whishaw, & S. M. Pellis, 1996, 1997a, 1997b; see also records 1996-06132-009, 1997-05322-015, and 1997-04722-005). An unresolved question is whether sex differences are present in other movements, such as rotation around the longitudinal axis, and whether this difference is dependent on a feminine or masculine skeletomusculature. Female rats (Rattus norvegicus) first rotate their forequarters and then their hindquarters in the same direction. Male rats exhibit rotation of the hindquarters counter to the direction of forequarter rotation. Males with the testicular feminized mutation, who have a feminized skeletomusculature and masculinized central nervous system, are similar to male controls. This study provides evidence that sex differences in movement integration are not restricted to the lateral plane, are not solely due to sex differences in skeletomusculature, and thus are likely mediated by the central nervous system.     

How informed is online informed consent?

We examined participants' reading and recall of informed consent documents presented via paper or computer. Within each presentation medium, we presented the document as a continuous or paginated document to simulate common computer and paper presentation formats. Participants took slightly longer to read paginated and computer informed consent documents and recalled slightly more information from the paginated documents. We concluded that obtaining informed consent online is not substantially different than obtaining it via paper presentation. We also provide suggestions for improving informed consent--in both face-to-face and online experiments.     

Amputees by choice: body integrity identity disorder and the ethics of amputation

ABSTRACT Should surgeons be permitted to amputate healthy limbs if patients request such operations? We argue that if such patients are experiencing significant distress as a consequence of the rare psychological disorder named Body Integrity Identity Disorder (BIID), such operations might be permissible. We examine rival accounts of the origins of the desire for healthy limb amputations and argue that none are as plausible as the BIID hypothesis. We then turn to the moral arguments against such operations, and argue that on the evidence available, none is compelling. BIID sufferers meet reasonable standards for rationality and autonomy: so as long as no other effective treatment for their disorder is available, surgeons ought to be allowed to accede to their requests.     

The dynamics of threat and challenge appraisals prior to stressful achievement events

Research on cognitive appraisal of stressful achievement events has emphasized threat appraisals and anxiety. The present research also focused on challenge and positive emotion. Study 1 used hypothetical scenarios of stressful events. Study 2 explored temporal pattems of appraisal and emotion prior to an exam. Compared with threat appraisals, trait and state challenge appraisals were associated with more confident coping expectancies, lower perceptions of threat, higher positive emotion, and more beneficial perceptions of the effects of appraisal and emotion on performance. Beneficial perceptions of state appraisals were associated with higher exam performance. These findings were interpreted in the context of theoretical perspectives on the cognitive appraisal of stressful events and the adaptive functions of challenge and positive emotion.     

The emergent generation effect and hypermnesia: influences of semantic and nonsemantic generation tasks

The generation effect is moderated by experimental design, affecting recall in within-subjects designs but typically not in between-subjects designs. However, N. W. Mulligan (2001) found that the generation effect emerged over repeated recall tests in a between-subjects design, calling into question the generality of this limiting condition. In addition, the generate condition but not the read condition produced hypermnesia (increased recall over tests). The present experiments demonstrate that semantic-based (semantic-associate and category-associate) generation tasks produce this pattern of results whereas nonsemantic (letter transposition, rhyme, word fragment) generation tasks do not. Thus, the emergent generation effect appears to be a byproduct of semantic elaboration rather than a direct product of generation. In addition, high- and low-imagery words produced equivalent hypermnesia and emergent generation effects, arguing against a mediating role for imagistic encoding. Finally, there is no evidence of an emergent generation effect for nonwords, another traditional limiting condition of the generation effect.     

Visual similarity is greater for line drawings of nonliving than living things: the importance of musical instruments and body parts

It has been argued that greater intra-category structural similarity for living things, and the subsequent "visual crowding," makes them more difficult to recognize and name for neurologically damaged individuals and normal subjects (Humphreys et al., 1988). Nevertheless, the precise meaning and quantification of structural similarity remains unclear, as does the rationale for why it necessarily should be greater for living things. We derived a new measure of visual overlap from the Snodgrass and Vanderwart corpus of line drawings: the degree of pixel overlap within subcategories (Euclidean distance: ED). Contrary to existing notions of visual crowding and extant measures of contour overlap, within-category ED indicated less within-category visual overlap for living things. Furthermore, musical instruments clustered with living things (having low overlap), while body parts clustered with nonliving things (having high overlap). These counter-intuitive findings accord with patient data and thus, provide evidence for the psychological reality and utility of ED.     

Effects of death within 11 years on cognitive performance in old age

Six different cognitive tests and the Heck Depression Inventory (BDI) were given to 3,572 active community residents aged 49 to 93 years. Causes of death were ascertained for 443 who died between 36 and 3,903 days later. Subsequent survival predicted test scores during the 3,903 days and independently during Days 36 to 1,826 and Days 1,827 to 3,903. Scores on the BDI and cumulative verbal learning and vocabulary tests predicted mortality after demographics and performance on other cognitive tests had been considered. Predictors were similar for deaths from heart disease, malignancies, and other causes. A new finding that cognitive tests did not predict survival duration within the sample of deceased explains previous findings of greater terminal decline in performance for young than for elderly adults.