Interpretation of a full-information item-level factor analysis of the MMPI-2: normative sampling and nonpathognomonic descriptors

An exploratory item-level full-information factor analysis was performed on the normative sample for the MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989). This method of factor analysis, developed by Schilling and Bock (Bock & Schilling, 1997) and based on item response theory, works directly with the response patterns and avoids the artifacts associated with phi coefficients and tetrachoric coefficients. Promax rotation of the factor solution organizes the clinical scale items into 10 factors that we labeled Distrust, Self-Doubt, Fitness, Serenity, Rebelliousness, Instrumentality, Irritability, Artistry, Sociability, and Self-Reliance. A comparison was made to the results of Johnson, Butcher, Null, and Johnson (1984), who performed a principal-component analysis on an item set of 550 items from the previous version of the MMPI (Hathaway & McKinley, 1943). Along with version changes and sampling differences, the essential differences between Johnson et al.'s results and ours may be attributed to differences between the Schilling and Bock method, which uses all information in the item responses, and the principal-component analysis, which uses the partial information contained in pairwise correlation coefficients. This study included 518 of the complete 567 items of the MMPI-2, versus Johnson et al.'s retention of 309 of the initially included 550 items of the previous MMPI. The full-information analysis retained all 518 initially included items and more evenly distributed the items over the 10 resulting factors, all sharply defined by their highest loading items and easy to interpret. Sampling effects and factor label considerations are discussed, along with recommendations for research that would validate the clinical utility of the implied scales for describing normal personality profiles. The full-information procedure provides for Bayes estimation of scores on these scales.     

Assessing the belief bias effect with ROCs: reply to Dube, Rotello, and Heit (2010)

Dube, Rotello, and Heit (2010) argued (a) that the so-called receiver operating characteristic is nonlinear for data on belief bias in syllogistic reasoning; (b) that their data are inconsistent with Klauer, Musch, and Naumer's (see record 2000-02818-008) model of belief bias; (c) that their data are inconsistent with any of the existing accounts of belief bias and only consistent with a theory provided by signal detection theory; and (d) that in fact, belief bias is a response bias effect. In this reply, we present reanalyses of Dube et al.'s data and of old data suggesting (a) that the receiver operating characteristic is linear for binary "valid" versus "invalid" responses, as employed by the bulk of research in this field; (b) that Klauer et al.'s model describes the old data significantly better than does Dube et al.'s model and that it describes Dube et al.'s data somewhat better than does Dube et al.'s model; (c) that Dube et al.'s data are consistent with the account of belief bias by misinterpreted necessity, whereas Dube et al.'s signal detection model does not fit their data; and (d) that belief bias is more than a response bias effect.     

Explicit and motoric dependent measures of geographical slant are dissociable: A reassessment of the findings of Durgin, Hajnal, Li, Tonge, and Stigliani (2010)

Durgin et al. (2010) argued that the apparent accuracy of the palmboard measure of geographical slant is accidental and reflects limitations in wrist flexion that reduce palmboard adjustments by just the right amount given the perceptual overestimations upon which they are based. This account is inconsistent with findings that verbal reports and palmboard adjustments are dissociable. In addition to previous evidence found for such dissociation, Durgin et al. also found verbal/palmboard dissociations in Experiment 2. Experiments 1 and 3 of Durgin et al. lacked verbal reports and instead compared palmboard adjustments to free-hand estimates in the context of small wooden surfaces. These experiments are not relevant to the issue of verbal/palmboard dissociability. Across studies, the accuracy of Durgin et al.'s palmboard implementation is far less than that found by others (Feresin & Agostini, 2007). The design of Durgin et al.'s Experiment 5 misrepresented the experimental conditions of Creem and Proffitt (1998), and consequently, the findings of this study have no bearing on the issue at hand.     

Partial support from a non-replication: comment on Roma, Silberberg, Ruggiero, and Suomi (2006)

A purported replication by P. G. Roma, A. Silberberg, A. M. Ruggiero, and S. J. Suomi of the authors' previous study claims to contradict their finding that capuchin monkeys (Cebus apella) refuse to exchange with an experimenter if their partner receives a superior reward. Roma et al. used no exchange task, however, or any other task. Roma et al. offered frustration as explanation of their findings, yet failed to statistically prove that the effect of frustration is stronger than that of inequity. They also misrepresented the dependent measure of the authors' study. Reanalysis of the authors' own data indicated no role of frustration, that is, no effect of previous experience with a superior reward. The authors conclude that Roma et al.'s study is not a replication and does not disprove the authors' findings.     

A reminder about procedures needed to reliably produce perfect timesharing: comment on Lien, McCann, Ruthruff, and Proctor (2005)

M.-C. Lien, R. S. McCann, E. Ruthruff, and R. W. Proctor (2005) argued that simultaneous ideomotor-compatible choice tasks cannot be perfectly timeshared. Their conclusion is limited in generalizability for 2 reasons: (a) Their experiments did not include procedures that previous research has shown to be necessary for obtaining perfect timesharing (motivating subjects to perform the 2 tasks rapidly and simultaneously; homogeneous blocks of simultaneous stimuli for the 2 tasks), and (b) their experiments included a procedure that previous research has shown to interfere with perfect timesharing of simultaneous tasks (within-block variation of task interstimulus intervals). Also discussed here are problems in M.-C. Lien et al.'s (2005) analysis of slopes relating Task 2 latency to Task 1 latency and their advocacy of a central bottleneck theory that may not be disconfirmable.     

PERSONALITY TESTS AT THE CROSSROADS: A RESPONSE TO MORGESON, CAMPION, DIPBOYE, HOLLENBECK, MURPHY, AND SCHMITT (2007)

Despite widespread and growing acceptance that published personality tests are valid predictors of job performance, Morgeson et al. (2007) propose they be abandoned in personnel selection because average validity estimates are low. Our review of the literature shows that Morgeson et al.'s skepticism is unfounded. Meta-analyses have demonstrated that published personality tests, in fact, yield useful validity estimates when validation is based on confirmatory research using job analysis and taking into account the bidirectionality of trait–performance linkages. Further gains are likely by use of narrow over broad measures, multivariate prediction, and theory attuned to the complexities of trait expression and evaluation at work. Morgeson et al. also suggest that faking has little, if any, impact on personality test validity and that it may even contribute positively to job performance. Job applicant research suggests that faking under true hiring conditions attenuates personality test validity but that validity is still sufficiently strong to warrant personality test use in hiring. Contrary to Morgeson et al., we argue that the full value of published personality tests in organizations has yet to be realized, calling for programmatic theory-driven research.     

Jousting with straw men: comment on Westen, Novotny, and Thompson-Brenner (2004)

Empirically supported treatments (ESTs) do not cure every patient, and the randomized trial is not a flawless methodology. Upon these often-noted and widely accepted points, D. Westen, C. M. Novotny, and H. Thompson-Brenner (2004a; see record 2004-15935-005) built a critique of ESTs and EST research. However, important work developing effective, clinically relevant treatments for serious problems was omitted from the Westen et al. (2004a) review. Little documentation was offered for the purported "assumptions" of EST methodology that Westen et al. (2004a) criticized; and different review standards were applied to studies supporting versus those disagreeing with Westen et al.'s (2004a) views. Finally, the correlational research designs proposed as a remedy by Westen et al. (2004a) have far more serious weaknesses than randomized trials, thoughtfully applied to real-world clinical care.     

The validity and appropriateness of methods, analyses, and conclusions in Rind et al. (1998): A rebuttal of victimological critique from Ondersma et al. (2001) and Dallam et al. (2001).

The authors respond to 2 victimological critiques of their 1998 meta-analysis on child sexual abuse (CSA). S. J. Dallam et al. (2001) claimed that B. Rind, P. Tromovitch, and R. Bauserman (1998) committed numerous methodological and statistical errors, and often miscoded and misinterpreted data. The authors show all these claims to be invalid. To the contrary, they demonstrate frequent bias in Dallam et al.'s criticisms. S. J. Ondersma et al. (2001) claimed that Rind et al.'s study is part of a backlash against psychotherapists, that its suggestions regarding CSA definitions were extrascientific, and that the moral standard is needed to understand CSA scientifically. The authors show their suggestions to have been scientific and argue that it is Ondersma et al.'s issue-framing and moral standard that are extrascientific. This reply supports the original methods, analyses, recommendations, and conclusions of Rind et al.     

Toward a complete decision model of item and source recognition: A discrete-state approach

In source-monitoring experiments, participants study items from two sources (A and B). At test, they are presented Source A items, Source B items, and new items. They are asked to decide whether a test item is old or new (item memory) and whether it is a Source A or a Source B item (source memory). Hautus, Macmillan, and Rotello (2008) developed models, couched in a bivariate signal detection framework, that account for item and source memory across several data sets collected in a confidence-rating response format. The present article enlarges the set of candidate models with a discrete-state model. The model is a straightforward extension of Bayen, Murnane, and Erdfelder's (1996) multinomial model of source discrimination to confidence ratings. On the basis of the evaluation criteria adopted by Hautus et al., it provides a better account of the data than do Hautus et al.'s models.     

Not evidence for separable controlled and automatic influences in artificial grammar learning: comment on Higham, Vokey, and Pritchard (2000)

P. A. Higham, J. R. Vokey, and J. L. Pritchard (2000) claimed to provide evidence for separable controlled and automatic processes in artificial grammar learning. It is argued that their results are compatible with a single controlled influence: Participants might mistakenly identify more grammatical items than nongrammatical items as belonging to the other grammar, because the grammars are very similar to each other, and the nongrammatical items are relatively highly dissimilar. Participants' knowledge may be ambiguous, rather than automatic. It is further argued that even if Higham et al.'s data do support automatic effects, opposition logic, in this case, cannot be said to have succeeded where dissociation logic has failed, because it is used to address the issue of whether participants have conscious control over the knowledge they acquire, rather than whether they possess conscious awareness of that knowledge.     

Is the focus of attention in working memory expanded through practice?

This article reinvestigates the claim by P. Verhaeghen, J. Cerella, and C. Basak (2004) that the focus of attention in working memory can be expanded from 1 to 4 items through practice. Using a modified version of Verhaeghen et al.'s n-back paradigm, Experiments 1 and 3 show that a signature of a one-item focus, the time cost for switching between items in working memory, persists over practice. Verhaeghen et al. reported a shift over practice from a step function to a linear slope of reaction times over set size and argued that it reflects the expansion of the focus. With an improved counterbalancing scheme, a continuously increasing slope was found even without practice in Experiment 2. The results question the hypothesis that the focus is expanded through practice. They are in line with predictions from a model that distinguishes a one-item focus from a direct-access region holding about 4 items.     

Occlusion, symmetry, and object-based attention: comment on Behrmann, Zemel, and Mozer (1998)

The validity of M. Behrmann, R. Zemel, and M. Mozer's (1998) finding that object-based attention can be directed toward occluded objects is examined in 3 experiments. In M. Behrmann et al.'s (1998) original study, participants made speeded judgments of whether the numbers of bumps attached to 2 arms of an X shape were the same or different. The 2 sets of bumps belonged either to a single object, 2 different objects, or 2 separated parts of an occluded object. Unfortunately, this objecthood manipulation was confounded by the symmetry of the stimuli. Experiment 1 replicated M. Behrmann et al.'s main results using identical stimuli. Experiments 2a and 2b dissociated objecthood from symmetry. The results suggest that the effects of object-based attention found by M. Behrmann et al. are largely due to symmetry. The stimuli used in M. Behrmann et al. are not appropriate for examining the relation between object-based attention and occlusion.     

Factorial structure of short scales measuring manic-depression in Kuwaiti undergraduates

A sample of 202 Kuwaiti college students (63 men and 139 women; Mage=21.6 yr., SD=2.4) responded to the Thalbourne, et al.'s Manic-Depressiveness Scale. In a factor analysis with a varimax rotation, 14 of 18 items had significant loadings (>.3) on the first factor, including 8 items of the original depression scale. 6 items significantly loaded on the second factor, including 3 items of the original mania scale. This did not match the hypothetical structure of the scale.     

Oddities in Kirkpatrick, et al.'s study of children of lesbian mothers

Kirkpatrick, et al.'s 1976 study of what happened to 20 lesbians' children has received considerable attention, apparently later being matched with 20 heterosexuals' children. In 2004, Kirkpatrick generally acknowledged Schumm's caution that her findings are less impressive than are needed, yet, Kirkpatrick stated her "early findings have been reinforced" and that "no evidence of differences in the children grouped by the mother's sexual orientation" have been documented by subsequent research. Close examination of the data of these studies indicates that children from 13 lesbian mothers were compared with children from 13 divorced heterosexuals. Further, there are contradictions between the published reports regarding the nature of samples and various findings. Analysis of Kirkpatrick, et al.'s study suggests that children do less well when raised by homosexual parents.     

Further evidence on the validity of the MMPI-2 Restructured Clinical (RC) Scales: addressing questions raised by Rogers, Sewell, Harrison, and Jordan and Nichols

The reviews by Rogers, Sewell, Harrison, and Jordan (2006/this issue), and by Nichols (2006/this issue) offer markedly contrasting appraisals of the MMPI-2 Restructured Clinical (RC) Scales introduced by Tellegen et al. (2003). The one common feature is that both reviews draw on the same atypical MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) data set for their empirical analyses, with results warranting critical scrutiny. Rogers et al.'s critique provides an evaluation of the RC Scales from the perspective of Jackson's (1970) method of test development. One significant issue in Rogers et al.'s review concerns social desirability, prompting us to clarify our own views on this topic. We also highlight and discuss problems associated with Rogers et al.'s use of the unrepresentative data set. Nichols's polemical critique neglects empirical and theoretical support for demoralization as a central construct and misconstrues as "construct drift" the purposeful process of developing the RC scales. Nichols's criticisms and proposals overlook requirements for assessing syndromes and for construct validation and even rudiments of scale development. Our reply incorporates evidence, including new findings, refuting his criticisms and confirming that demoralization is a pervasive MMPI dimension, that the RC Scales capture the major distinctive features of the original Clinical Scales, and that they generate correspondingly meaningful validity patterns.     

Learning from implicit learning literature: Comment on Shea, Wulf, Whitacre, and Park (2001)

In their analysis of complex motor skill learning, Shea, Wulf, Whitacre, and Park (2001) have overlooked one of the most robust conclusions of the experimental studies on implicit learning conducted during the last decade--namely that participants usually learn things that are different from those that the experimenter expected them to learn. Weshow that the available literature on implicit learning strongly suggests that the improved performance in Shea et al.'s Experiments 1 and 2 (and similar earlier experiments, e.g., Wulf & Schmidt, 1997) was due to the exploitation of regularities in the target pattern different from those on which the postexperimental interview focused. This rules out the conclusions drawn from the failure of this interview to reveal any explicit knowledge about the task structure on the part of the participants. Similarly, because the information about the task structure provided to an instructed group of participants in Shea et al.'s Experiment 2 did not concern the regularities presumably exploited by the standard, so-called implicit, group, Shea et al.'s claim that explicit knowledge may be less effective than implicit knowledge is misleading.     

Are effects of spelling-to-sound irregularity on pronunciation latency confounded by stimulus familiarity?

Summary This article reports subjective experiential familiarity ratings for the experimental items used by Parkin, McMullen, and Graystone (1986). These data were used to examine the possibility that the regularity effects obtained when measuring pronunciation latencies to these materials arose from a confounding of familiarity. It was shown that Parkin et al.'s materials were controlled along the dimension of subjective familiarity. Correlational analyses showed that there was a significant relationship between familiarity and pronunciation latency — the more familiar a word the shorter the latency. Analysis of co-variance revealed that the regularity effects obtained by Parkin et al. remained when the effects of familiarity on latency were partialled out. It is concluded that effects of spelling-to-sound regularity are not due to unintentional confounding of stimulus familiarity.     

A misleading review of response bias: comment on McGrath, Mitchell, Kim, and Hough (2010)

In the May 2010 issue of Psychological Bulletin, R. E. McGrath, M. Mitchell, B. H. Kim, and L. Hough published an article entitled "Evidence for Response Bias as a Source of Error Variance in Applied Assessment" (pp. 450-470). They argued that response bias indicators used in a variety of settings typically have insufficient data to support such use in everyday clinical practice. Furthermore, they claimed that despite 100 years of research into the use of response bias indicators, "a sufficient justification for [their] use… in applied settings remains elusive" (p. 450). We disagree with McGrath et al.'s conclusions. In fact, we assert that the relevant and voluminous literature that has addressed the issues of response bias substantiates validity of these indicators. In addition, we believe that response bias measures should be used in clinical and research settings on a regular basis. Finally, the empirical evidence for the use of response bias measures is strongest in clinical neuropsychology. We argue that McGrath et al.'s erroneous perspective on response bias measures is a result of 3 errors in their research methodology: (a) inclusion criteria for relevant studies that are too narrow; (b) errors in interpreting results of the empirical research they did include; (c) evidence of a confirmatory bias in selectively citing the literature, as evidence of moderation appears to have been overlooked. Finally, their acknowledging experts in the field who might have highlighted these errors prior to publication may have prevented critiques during the review process.     

Deconstructing multicultural counseling competencies research: comment on Owen, Leach, Wampold, and Rodolfa (2011)

The present article offers a commentary and critique of the research presented by J. Owen, M. M. Leach, B. Wampold and E. Rodolfa (2011). Given the complexity of the research methodology, findings, and conclusions, the authors provide a concise summary of findings, study limitations, and conclusions followed by a detailed critique of the study. The authors respect and appreciate the ambitious efforts made by Owen et al. to address gaps in the literature regarding outcome research using client ratings of counselors' multicultural counseling competencies (MCCs). The authors provide a critical analysis of some of J. Owen et al.'s specific conclusions and offer alternative conclusions based on conceptual and methodological bases. The authors use the opportunity to comment on this study as a means of advancing recommendations regarding future research on MCCs that might contribute to substantive revisions to the long-standing theoretical foundation in this area.     

Reexamining the factor structure of the 20-item Toronto alexithymia scale: commentary on Gignac, Palmer, and Stough

Gignac, Palmer, and Stough (2007/this issue) test a number of different latent factor models for the TAS-20 using a community sample of 355 participants and conclude that this scale is best represented by a "nested factors model," with five substantive factors and a method factor. Gignac et al. also report that the correlated three-factor model and a comparable higher order model supported by most studies produced poor levels of incremental close fit. In this article, we challenge Gignac et al.'s unheralded and largely unsupported use of nested model fitting and the uncritical acceptance of exceptionally high cutoff levels to assess goodness of fit (GOF). Using more traditional and empirically supported model testing procedures and a more flexible approach to the interpretation of multiple tests of GOF, we interpret Gignac et al.'s results as actually supportive of the traditional three-factor model and one that has been recovered in 17 of the 24 factor analytic studies of the TAS-20.