国家公务员结构化面试中评委偏差的IRT分析

使用项目反应理论(IRT)中的多面Rasch模型,对两组共12名评委在国家公务员结构化面试中的评委偏差进行了分析。提出并验证了两种评委偏差：评委之间在宽严程度上的差异和评委自身的一致性问题。结果发现：不同评委之间在宽严程度上差异显著,且不同评委评定行为的跨考生、跨维度、跨性别、跨时间的自身一致性也存在差异。研究表明,这种进入到评委个体层次的分析突破了经典测量理论(CTT)定位于评委群体进行分析的局限,针对每位评委的偏差行为提供了详细具体的诊断信息,从而为评委的针对性培训和评委库的建立提供了现代测量学的新方法     

多面Rasch模型在结构化面试中的应用

使用项目反应理论中的多面Rasch模型,对66名考生在结构化面试中的成绩进行分析,剔除了由于评委等具体测量情境因素引入的误差对原始分数的影响,得到考生的能力估计值以及个体水平的评分者一致性信息。对基于考生能力估计值和考生面试分得到的决策结果进行比较,发现测量误差的确对决策造成影响,对个别考生的影响甚至相当巨大。进一步使用Facets偏差分析以及评委宽严程度的Facets分析追踪误差源。结果表明,将来自不同面试组的被试进行面试原始成绩的直接比较,评委的自身一致性和评委彼此之间在宽严程度上的差异均将导致误差。研究表明,采用Facets的考生能力估计值作为决策的依据将提高选拔的有效性。同时,Facets分析得到的考生个体层次的评分者一致性指标,以及评委与考生的偏差分析等研究结果还可以为面试误差来源的定位提供详细的诊断信息     

Rater bias in the EASI temperament scales: a twin study

Under trait theory, ratings may be modeled as a function of the temperament of the child and the bias of the rater. Two linear structural equation models are described, one for mutual self- and partner ratings, and one for multiple ratings of related individuals. Application of the first model to EASI temperament data collected from spouses rating each other shows moderate agreement between raters and little rating bias. Spouse pairs agree moderately when rating their twin children, but there is significantly rater bias, with greater bias for monozygotic than for dizygotic twins. MLE's of heritability are approximately .5 for all temperament scales with no common environmental variance. Results are discussed with reference to trait validity, the person-situation debate, halo effects, and stereotyping. Questionnaire development using ratings on family members permits increased rater agreement and reduced rater bias.     

The parser doesn't ignore intransitivity, after all

Several previous studies (B. C. Adams, C. Clifton, & D. C. Mitchell, 1998; D. C. Mitchell, 1987; R. P. G. van Gompel & M. J. Pickering, 2001) have explored the question of whether the parser initially analyzes a noun phrase that follows an intransitive verb as the verb's direct object. Three eye-tracking experiments examined this issue in more detail. Experiment 1 replicated the finding that readers experience difficulty on this noun phrase in normal reading and found that this difficulty occurs even with intransitive verbs for which a direct object is categorically prohibited. Experiment 2, however, demonstrated that this effect is not due to syntactic misanalysis but to disruption that occurs when a comma is absent at a subordinate clause/main clause boundary. Experiment 3 replicated the finding (M. J. Pickering & M. J. Traxler, 2003; M. J. Traxler & M. J. Pickering, 1996) that when a noun phrase "filler" is an implausible direct object for an optionally transitive relative clause verb, processing difficulty results; however, there was no evidence for such difficulty when the relative clause verb was strictly intransitive. Taken together, the 3 experiments undermine the support for the claim that the parser initially ignores a verb's subcategorization restrictions.     

Rater Model Using Signal Detection Theory for Latent Differential Rater Functioning

Differential rater functioning (DRF) occurs when raters show evidence of exercising differential severity or leniency when scoring examinees within different subgroups. Previous studies of DRF have examined rater bias using manifest variables (e.g., use of covariates) to determine the subgroups. These manifest variables include gender and the ethnicity of the examinee. For example, a rater may score males more severely. Ideally, each rater’s severity should be invariant across subgroups. This study examines DRF in the context of latent subgroups that classify possible sources of DRF based on raters’ scoring behavior rather than manifest factors. An extension of the latent class signal detection theory (LC-SDT) model for identifying DRF is proposed and examined using real-world data and simulations. Results from real-world data show that the signal detection approach leads to an effective method to identify latent DRF. Simulations with varying sample sizes and conditions of rater precision were shown to recover parameters at an adequate level, supporting its use to identify latent DRF in large-scale data. These findings suggest that the DRF extension of the LC-SDT can be a useful model to examine characteristics of raters and add information that can aid rater training.     

Body dissatisfaction, bulimic symptoms, and clothing practices among college women

Clothing use may be a behavioral avoidance strategy for individuals with body dissatisfaction and eating pathology. The authors administered the Body Image Avoidance Questionnaire (J. C. Rosen, D. Srebnik, E. Saltzberg, & S. Wendt, 1991), the Bulimia Test-Revised (M. Thelen, J. Farmer, S. Wonderlich, & M. Smith, 1991), and the Body dissatisfaction subscale of the Eating Disorder Inventory (D. M. Garner, M. P. Olmstead, & J. Polivy, 1983) to undergraduate college women from two universities (N = 540). Results indicated that women who were more dissatisfied with their bodies (beta = .396) and had greater disordered eating behaviors (beta = .378) were more likely to engage in clothing-related appearance-management behaviors (p < .001), including wearing apparel to camouflage their bodies; avoiding revealing, brightly colored, or tightly fitting clothing; and avoiding shopping for clothing. These findings suggest that the presence of certain clothing-related appearance-management behaviors may be a warning sign that an individual is at risk for developing an eating disorder or may currently have an eating disorder.     

Child Care in The Netherlands: Trends in Quality Over the Years 1995-2005

The authors assessed the quality of child care in a representative national sample of 42 child-care centers in the Netherlands and compared it with the quality of care that researchers have found using similar samples in 1995 (M. H. van IJzendoorn, L. W. C. Tavecchio, G. J. J. M. Stams, M. J. E. Verhoeven, & E. J. Reiling, 1998) and 2001 (M. J. J. M. Gevers Deynoot-Schaub & J. M. A. Riksen-Walraven, 2005). In the present study, results showed a low level of overall process quality for the 2005 sample, measured by the Infant/Toddler Environment Rating Scale-Revised (T. Harms, D. Cryer, & R. M. Clifford, 2003) and the Early Childhood Environment Rating Scale-Revised (T. Harms, R. M. Clifford, & D. Cryer, 1998). The present authors found a significant decline in process quality in comparison with the 1995 and 2001 findings. They concluded that, from an international perspective, the Netherlands has lost its leading position in child-care quality compared with that from 10 years ago.     

Evidence for genetic influences on personality from self-reports and informant ratings.

Self-report data on Extraversion (E) and Neuroticism (N), together with ratings by the co-twin, were obtained from a sample of 826 adult female twin pairs ascertained through a population-based twin register. Data were analyzed using a model that allowed for the contributions to personality ratings of the rater's personality (rater bias) as well as of the personality of the person being rated. For E, but not for N, significant rater bias was found, with extraverted respondents tending to underestimate, and introverted respondents tending to overestimate, the Extraversion of their co-twins. Good agreement between self-reports and ratings by the respondent's co-twin was found for both E and N. Substantial genetic influences were found for both personality traits, confirming findings from genetic studies of personality that have relief only on self-reports of respondents.     

Child care under pressure: the quality of Dutch centers in 1995 and in 2001

In 2001, the authors assessed the quality of care provided to children in 51 care groups from 39 child-care centers in The Netherlands using the Infant/Toddler Environment Rating Scale (T. Harms, D. Cryer, & R. M. Clifford, 1990) and compared the results with the quality of child care assessed in 1995 (M. H. van IJzendoorn, L. W. C. Tavecchio, G. J. Stams, M. J. E. Verhoeven, & E. J. Reiling, 1998). The overall quality and scale scores for language and learning activities were significantly lower in 2001 than in 1995. Child-care centers founded within the past 6 years (all nonsubsidized centers) scored considerably lower than did older, mostly subsidized, centers. The results are discussed from the perspective of changes in the socioeconomic and political context of child care.     

Factor structure of the Psychopathic Personality Inventory (PPI): findings from a large incarcerated sample

Exploratory factor analysis (EFA) of the Psychopathic Personality Inventory (PPI; S. O. Lilienfeld, 1990; S. O. Lilienfeld & B. P. Andrews, 1996) with a community sample has suggested that the PPI subscales may comprise 2 higher order factors (S. D. Benning, C. J. Patrick, B. M. Hicks, D. M. Blonigen, & R. F. Krueger, 2003). However, substantive and structural evidence raises concerns about the viability of this 2-factor model, particularly in offender populations. The authors attempted to replicate the S. D. Benning et al. 2-factor solution using a large (N = 1,224) incarcerated male sample. Confirmatory factor analysis of this model resulted in poor model fit. Similarly, using the same EFA procedures as did S. D. Benning et al., the authors found little evidence for a 2-factor model. When they followed the recommendations of J.-W. van Prooijen and W. A. van der Kloot (2001) for recovering EFA solutions, model fit results provided some evidence that a 3-factor EFA solution could be recovered via confirmatory factor analysis.     

A misleading review of response bias: comment on McGrath, Mitchell, Kim, and Hough (2010)

In the May 2010 issue of Psychological Bulletin, R. E. McGrath, M. Mitchell, B. H. Kim, and L. Hough published an article entitled "Evidence for Response Bias as a Source of Error Variance in Applied Assessment" (pp. 450-470). They argued that response bias indicators used in a variety of settings typically have insufficient data to support such use in everyday clinical practice. Furthermore, they claimed that despite 100 years of research into the use of response bias indicators, "a sufficient justification for [their] use… in applied settings remains elusive" (p. 450). We disagree with McGrath et al.'s conclusions. In fact, we assert that the relevant and voluminous literature that has addressed the issues of response bias substantiates validity of these indicators. In addition, we believe that response bias measures should be used in clinical and research settings on a regular basis. Finally, the empirical evidence for the use of response bias measures is strongest in clinical neuropsychology. We argue that McGrath et al.'s erroneous perspective on response bias measures is a result of 3 errors in their research methodology: (a) inclusion criteria for relevant studies that are too narrow; (b) errors in interpreting results of the empirical research they did include; (c) evidence of a confirmatory bias in selectively citing the literature, as evidence of moderation appears to have been overlooked. Finally, their acknowledging experts in the field who might have highlighted these errors prior to publication may have prevented critiques during the review process.     

Counterfactuals for free

I would like to thank D. Føllesdal, R. Jeffrey, J. Maienschein, B. van Fraassen, and M. White for comments on earlier versions of this paper and for discussion of the ideas involved. Funds to support research leaves during which this was written came from several sources; the NEH (#FB-24865-87), the NSF (#SES 19308), and Arizona State University. All these are gratefully acknowledged.     

BOOK REVIEWS

Books reviewed in this articles:
The Midwife of Platonism: Text and Subtext in Plato's Theaetetus . D avid S edley .
Gilles Deleuze's Difference and Repetition: a Critical Introduction and Guide . J ames W illiams .
John McDowell . M aximilian D e G aynesford .
Real Natures and Familiar Objects . C rawford E lder .
Natural Kinds and Conceptual Change . J oseph L a P orte .
Powers: a Study in Metaphysics . G eorge M olnar E dited with an I ntroduction by S tephen M umford and A F oreword by D.M. A rmstrong .
Anti-Individualism and Knowledge . J essica B rown .
Consciousness and its Objects . C olin M c G inn .
Reasonably Vicious . C andace V ogler .
Ethics Without Principles . J onathan D ancy .
Getting Even: Forgiveness and its Limits . B y J effrie E. M urphy .
Justice . B y H arry B righouse .
Faithful Reason: Essays Catholic and Philosophical . B y J ohn H aldane .
The Non-Existence of God . B y N icholas E veritt .
Art as Performance . B y D avid D avies .     

Reviews

Book Reviews in this article:
The Great Devonian Controversy: The Shaping of Scientific Knowledge among Gentlemanly Specialists . By M artin J. S. R udwick
Evolution: A Theory in Crisis . By M ichael D enton .
Images of Science: Essays on Realism and Empiricism, with a Reply from Bos C. van Fraassen . Edited by Paul M. C hurchland and C lifford A. H ooker .
In Search of the Person–Philosophical Explorations in Cognitive Science . By M ichael A. A rbib .
What They Are Saying About Genetic Engineering . By T homas A. S hannon .
Cosmogony and Ethical Order: New Studies in Comparative Ethics . Edited by R obin W. L ovin and F rank E. R eynolds .     

Holography does not account for goodness: a critique of van der Helm and Leeuwenberg (1996)

P. A. van der Helm and E. L. J. Leeuwenberg (1996) outlined a holographic account of figural goodness of a perceptual stimulus. The theory is mathematically precise and can be applied to a broad spectrum of empirical data. The authors argue, however, that the account is inadequate on both theoretical and empirical grounds. The theoretical difficulties concern the internal consistency of the account and its reliance on unspecified auxiliary assumptions. The account also makes counterintuitive empirical predictions, which do not fit past data or the results of a series of new experimental studies.     

Book Reviews

Book review in this article
G hiselli , E dwin E., and B rown , C larence W. Personnel and Industrial Psychology.
L ombard , G eorge F. F. Behavior in a Selling Group.
H ersey , R exford . Zest for Work.
H enriksson , E rik . Frånvaro från Arbetet.
B ursk , E dward C. (Ed.) The Management Team.
L indgren , H enry C. Effective Leadership in Human Relations.
P lanty , E arl G., and F reeston , J. T homas . Developing Management Ability.
W olfle , D ael . America's Resources of Specialized Talent.
N ational M anpower C ouncil . A Policy for Skilled Manpower.
W hitehill , A rthur M., J r . Personnel Relations: The Human Aspects of Administration.
B urlington , R ichard S tevens , and M ay , D onald C urtis . Handbook of Probability and Statistics with Tables.
W ert , J ames E., N eidt , C harles C., and A hmann , J. S tanley . Statistical Methods in Educational and psychological Research     

Cultural difference and the therapeutic alliance: an evidence-based analysis

The research on positive psychotherapy outcome consistently indicates that the quality of the alliance is important across different models of psychotherapy (D. E. Orlinsky, M. H. Ronnestad, & U. Willutzki, 2004; B. E. Wampold, 2000). Social psychological research has documented how "unintentional bias" can produce barriers to university admissions, employment, and advancement of well-qualified members of ethnic minority groups (J. F. Dovidio, S. L. Gaertner, K. Kawakami, & G. Hodson, 2002). Neuroscience is further confirming social psychological responses associated with race (J. L. Eberhard, 2005). Unintentional bias identified in social psychological research may be part of the psychotherapist/client interaction, interfere with the therapeutic alliance, and partly account for the high dropout rates and underutilization of psychotherapeutic services by people of color. The purpose of this article is to provide an evidence-based analysis of how psychologists in practice may unintentionally interfere with development of quality alliances with culturally different clients or patients and thus contribute to the barriers to effective multicultural counseling and psychotherapy. Principles from the American Psychological Association's (2003) multicultural guidelines and a review of relevant research are applied in suggesting strategies to reduce bias and to develop culturally appropriate skills in psychological practices.     

Language switching in the bilingual brain: What’s next?

Recent work using functional neuroimaging with early bilinguals has found little evidence for separate neural systems for each language during picture naming (Hernandez, A. E., Dapretto, M., Mazziotta, J., & Bookheimer, S. (2001). Language switching and language representation in Spanish–English bilinguals: An fMRI study. Neuroimage, 14, 510–520). However, switching between languages in early bilinguals during picture naming shows increased activity in the Dorsolateral Prefrontal Cortex (DLPFC) suggesting the importance of maintaining goal related information in order to bias subsequent response selection (Braver, T. S., Barch, D. M., Kelley, W. M., Buckner, R. L., Cohen, N. J., Miezin, F. M., et al. (2001). Direct comparison of prefrontal cortex regions engaged by working and long-term memory tasks. Neuroimage, 14, 48–59; Cohen, J. D., Braver, T. S., & O’Reilly, R. C. (1996). A computational approach to prefrontal cortex, cognitive control and schizophrenia: Recent developments and current challenges. Philosophical Transactions of the Royal Society of London Series B-Biological Sciences, 351, 1515–1527; O’Reilly, R. C., Braver, T. S., & Cohen, J. D. (1999). A biologically based computational model of working memory. In E. Akira Miyake, E. Priti Shah & et al. (Eds.), Models of working memory: Mechanisms of active maintenance and executive control. (pp. 375–411): New York, NY, USA). The current study set out to test early bilinguals using a picture naming paradigm. Results revealed increased activity in the DLPFC and the superior parietal lobule during language switching compared to naming of pictures in a single language. Increased activity was also observed between early learned first and second languages. The results from single language conditions revealed differences in areas devoted to language processing such as the Superior Temporal Gyrus. However, increased activity in brain areas devoted to memory, somatosensory processing and emotion were also observed. Taken together these results replicate previous studies on language switching. They also extend studies on the neural bases of bilingualism by suggesting that early bilinguals’ representation of the two languages may be mediated by neural systems not typically associated with language. The article ends by considering future directions in understanding the brain bases of language switching and single language processing in bilinguals.     

Publication bias in psychological science: prevalence, methods for identifying and controlling, and implications for the use of meta-analyses

The issue of publication bias in psychological science is one that has remained difficult to address despite decades of discussion and debate. The current article examines a sample of 91 recent meta-analyses published in American Psychological Association and Association for Psychological Science journals and the methods used in these analyses to identify and control for publication bias. Of the 91 studies analyzed, 64 (70%) made some effort to analyze publication bias, and 26 (41%) reported finding evidence of bias. Approaches to controlling publication bias were heterogeneous among studies. Of these studies, 57 (63%) attempted to find unpublished studies to control for publication bias. Nonetheless, those studies that included unpublished studies were just as likely to find evidence for publication bias as those that did not. Furthermore, authors of meta-analyses themselves were overrepresented in unpublished studies acquired, as compared with published studies, suggesting that searches for unpublished studies may increase rather than decrease some sources of bias. A subset of 48 meta-analyses for which study sample sizes and effect sizes were available was further analyzed with a conservative and newly developed tandem procedure of assessing publication bias. Results indicated that publication bias was worrisome in about 25% of meta-analyses. Meta-analyses that included unpublished studies were more likely to show bias than those that did not, likely due to selection bias in unpublished literature searches. Sources of publication bias and implications for the use of meta-analysis are discussed.     

Using dynamic risk and protective factors to predict inpatient aggression: Reliability and validity of START assessments

The Short-Term Assessment of Risk and Treatability (START; C. D. Webster, M. L. Martin, J. Brink, T. L. Nicholls, & S. L. Desmarais, 2009; C. D. Webster, M. L. Martin, J. Brink, T. L. Nicholls, & C. Middleton, 2004) is a relatively new structured professional judgment guide for the assessment and management of short-term risks associated with mental, substance use, and personality disorders. The scheme may be distinguished from other violence risk assessment instruments because of its inclusion of 20 dynamic factors that are rated in terms of both vulnerability and strength. This study examined the reliability and validity of START assessments in predicting inpatient aggression. Research assistants completed START assessments for 120 male forensic psychiatric patients through review of hospital files. They also completed Historical-Clinical-Risk Management-20 (HCR-20; C. D. Webster, K. S. Douglas, D. Eaves, & S. D. Hart, 1997) and Hare Psychopathy Checklist: Screening Version (PCL:SV; S. D. Hart, D. N. Cox, & R. D. Hare, 1995) assessments. Outcome data were coded from hospital files for a 12-month follow-up period using the Overt Aggression Scale (OAS; S. C. Yudofsky, J. M. Silver, W. Jackson, J. Endicott, & D. W. Williams, 1986). START assessments evidenced excellent interrater reliability and demonstrated both predictive and incremental validity over the HCR-20 Historical subscale scores and PCL:SV total scores. Overall, results support the reliability and validity of START assessments and use of the structured professional judgment approach more broadly, as well as the value of using dynamic risk and protective factors to assess violence risk. (PsycINFO Database Record (c) 2012 APA, all rights reserved).