A comparison of the efficiency and accuracy of BILOG and LOGIST

Comparisons are made between BILOG version 2.2 and LOGIST 5.0 Version 2.5 in estimating the item parameters, traits, item characteristic functions (ICFs), and test characteristic functions (TCFs) for the three-parameter logistic model. Data analyzed are simulated item responses for 1000 simulees and one 10-item test, four 20-item tests, and four 40-item tests. LOGIST usually was faster than BILOG in producing maximum likelihood estimates. BILOG almost always produced more accurate estimates of individual item parameters. In estimating ICFs and TCFs BILOG was more accurate for the 10-item test, and the two programs were about equally accurate for the 20- and 40-item tests.I am grateful to Robert J. Mislevy, Martha L. Stocking, and Marilyn S. Wingersky for many helpful comments on an earlier version of this paper. I would also like to thank Hamid Kamrani and Bongmyoung Park for getting LOGIST and BILOG running and keeping them running under changing computer systems at CTB/McGraw-Hill.     

Evaluation of self-contained training procedures for the Time-Sample Behavioral Checklist

The comparative effectiveness of two time-limited modes of training observers to code the behavior of clients in residential treatment programs on the Time-Sample Behavioral Checklist (TSBC) was evaluated. The susceptibility of training procedures to consensual observer drift and the predictability of TSBC mastery from trainee characteristics were also examined. Two equated groups of undergraduate students (N=15 each) participated in full-time training on the TSBC and another instrument for 27 days, followed by criterion testingin vivo and on videotapes. One group was trained by experienced personnel using procedures known to be effective but potentially subject to consensual observer drift. The other group was trained using a previously untested set of written and videotape procedures that do not rely on experienced personnel. Comparative effectiveness and observer drift were evaluated by multivariate and univariate ANOVAs on mastery scores reflecting both pattern agreement and level differences between each trainee and criterion codings. The new, more efficient training procedures were found to be as effective as the original training procedures in the degree of mastery achieved by trainees. Original training procedures were found to be resistant to consensual observer drift, with such a phenomenon appearing in only 1 instance of 156 opportunities. The high degree of mastery achieved by trainees during the time-limited training period was comparable to that previously achieved with the original training procedures. No meaningful predictions of coding mastery were found, with only one trainee characteristic being significantly related to one of six mastery criteria. The results document procedures that are both efficient and resistant to invalidity for training observers in the use of multidimensional observational systems, as well as providing guidelines for the development of standardized procedures.This article is based on a thesis submitted to the Graduate College of the University of Illinois at Urbana-Champaign in partial fulfillment of the requirements for the Ph.D. degree in psychology by the first author and under direction of the second author. The third and fourth authors also participated as supervisors. Appreciation is extended to other members of the thesis committee, Fred Kanfer, Ross Parke, and Julian Rappaport, and to John Gottman and W. Robert Nay for their comments and recommendations. This study was partially supported by Public Health Service Grants MH-25464 and MH-14257 from the National Institute of Mental Health and by grants from the Joyce Foundation, the MacArthur Foundation, and the Illinois Department of Mental Health and Developmental Disabilities.     

The comparative effectiveness of two modes of observer training on the Staff-Resident Interaction Chronograph

The comparative effectiveness of two time-limited modes of training observers to code activity on the Staff-Resident Interaction Chronograph (SRIC) in residential treatment programs for mentally disabled adults was evaluated. The susceptibility of training procedures for consensual observer drift was also examined, as was the predictability of SRIC mastery from trainee characteristics. Two equated groups of undergraduate student trainees (N=15 each) participated in full-time training for 27 days, followed by two weeks of criterion testing in vivo and on videotapes. One group received training by experience personnel using procedures known to be effective (original method). The other group received training via a previously untested set of written and videotaped procedures that do not rely on experienced personnel (package method). Multivariate and univariate analyses of variance found both methods to be equally effective in the degree of mastery achieved by trainees, without evidence of observer drift. No meaningful predictions of coding mastery were found, but conceptual mastery was predictable from individual characteristics. Differences were obtained for both groups between in vivo versus videotaped criterion tests. The results document procedures that are both efficient and resistant to invalidity for complex observational methodology as well as feasible for standardizing assessment of staff functioning across residential settings.This article is based on a thesis submitted to the Graduate College of the University of Illinois at Urbana-Champaign in partial fulfillment of the requirements for the Ph.D. degree in psychology by the first author under direction of the second author. The third and fourth authors also participated as supervisors. Appreciation is extended to other members of the thesis committee, Fred Kanfer, W. Robert Nay, Julian Rappaport, and James Wardrop, for their comments and recommendations. This study was partially supported by Public Health Service Grants MH-25464 and MH-14257 from the National Institute of Mental Health, and by grants from The Joyce Foundation and the Illinois Department of Mental Health and Developmental Disabilities.     

Power of the likelihood ratio test in covariance structure analysis

A procedure for computing the power of the likelihood ratio test used in the context of covariance structure analysis is derived. The procedure uses statistics associated with the standard output of the computer programs commonly used and assumes that a specific alternative value of the parameter vector is specified. Using the noncentral Chi-square distribution, the power of the test is approximated by the asymptotic one for a sequence of local alternatives. The procedure is illustrated by an example. A Monte Carlo experiment also shows how good the approximation is for a specific case.This research was made possible by a grant from the Dutch Organization for Advancement of Pure Research (ZWO). The authors also like to acknowledge the helpful comments and suggestions from the editor and anonymous reviewers.     

Binary programming and test design

An algorithmic approach to test design, using information functions, is presented. The approach uses a special branch of linear programming, i.e. binary programming. In addition, results of some benchmark problems are presented. Within the same framework, it is also possible to formulate the problem of individualized testing.I would like to thank my colleagues N. Veldhuijzen, H. Verstralen and M. Zwarts for their suggestions and comments. Furthermore, I would like to thank Professor W. van der Linden, Department of Educational Measurement and Data Analysis, Technological University Twente, for offering facilities at his department; Ellen Timminga of the same department and S. Baas, department of Operational Research at the same University for their efforts in linear programming.     

First‐line therapy for post‐traumatic stress disorder: A systematic review of cognitive behavioural therapy and psychodynamic approaches

Background

Despite evidence supporting cognitive behavioural therapy (CBT)‐based interventions as the most effective approach for treating post‐traumatic stress disorder (PTSD) in randomised control trials, alternative treatment interventions are often used in clinical practice. Psychodynamic (PDT)‐based interventions are one example of such preferred approaches, this is despite comparatively limited available evidence supporting their effectiveness for treating PTSD.

Aims

Existing research exploring effective therapeutic interventions for PTSD includes trauma‐focused CBT involving exposure techniques. The present review sought to establish the treatment efficacy of CBT and PDT approaches and considers the potential impact of selecting PDT‐based techniques over CBT‐based techniques for the treatment of PTSD.

Results

The evidence reviewed provided examples supporting PDT‐based therapy as an effective treatment for PTSD, but confirmed CBT as more effective in the treatment of this particular disorder. Comparable dropout rates were reported for both treatment approaches, suggesting that relative dropout rate should not be a pivotal factor in the selection of a PDT approach over CBT for treatment of PTSD.

Conclusion/Implications

The need to routinely observe evidence‐based recommendations for effective treatment of PTSD is highlighted and factors undermining practitioner engagement with CBT‐based interventions for the treatment of PTSD are identified.     

Exploring a Source of Uneven Score Equity across the Test Score Range

Score equity assessment (SEA) refers to an examination of population invariance of equating across two or more subpopulations of test examinees. Previous SEA studies have shown that score equity may be present for examinees scoring at particular test score ranges but absent for examinees scoring at other score ranges. No studies to date have performed research for the purpose of understanding why score equity can be inconsistent across the score range of some tests. The purpose of this study is to explore a source of uneven subpopulation score equity across the score range of a test. It is hypothesized that the difficulty of anchor items displaying differential item functioning (DIF) is directly related to the score location at which issues of score inequity are observed. The simulation study supports the hypothesis that the difficulty of DIF items has a systematic impact on the uneven nature of conditional score equity.     

眼动追踪在价值导向元记忆研究中的应用

价值导向元记忆关注人们在面对不同重要性信息时,通过元记忆监测和调节,有选择地优先加工高价值信息,以实现记忆效率最大化的目的。价值导向元记忆包括价值导向元记忆监测和控制,眼动追踪技术以其无干扰性、生态效度高等优势可以时时追踪这一监控过程。当前该领域研究中已采用的眼动指标集中在项目选择、学习时间分配、学习进程等方面。未来在项目选择、学习效率和策略比较等研究中可以探索眼动追踪技术的进一步应用。     

Two hands are better than one: a new assessment method and a new interpretation of the non-visual illusion of self-touch

A simple experimental paradigm creates the powerful illusion that one is touching one’s own hand even when the two hands are separated by 15 cm. The participant uses her right hand to administer stimulation to a prosthetic hand while the Examiner provides identical stimulation to the participant’s receptive left hand. Change in felt position of the receptive hand toward the prosthetic hand has previously led to the interpretation that the participant experiences self-touch at the location of the prosthetic hand, and experiences a sense of ownership of the prosthetic hand. Our results argue against this interpretation. We assessed change in felt position of the participant’s receptive hand but we also assessed change in felt position of the participant’s administering hand. Change in felt position of the administering hand was significantly greater than change in felt position of the receptive hand. Implications for theories of ownership are discussed.     

Two interpretations of the discrimination parameter

In this paper we propose two interpretations for the discrimination parameter in the two-parameter logistic model (2PLM). The interpretations are based on the relation between the 2PLM and two stochastic models. In the first interpretation, the 2PLM is linked to a diffusion model so that the probability of absorption equals the 2PLM. The discrimination parameter is the distance between the two absorbing boundaries and therefore the amount of information that has to be collected before a response to an item can be given. For the second interpretation, the 2PLM is connected to a specific type of race model. In the race model, the discrimination parameter is inversely related to the dependency of the information used in the decision process. Extended versions of both models with person-to-person variability in the difficulty parameter are considered. When fitted to a data set, it is shown that a generalization of the race model that allows for dependency between choices and response times (RTs) is the best-fitting model.