Multiple-answer multiple-choice test items: Responding and scoring through bayes and minimax strategies

A multiple-answer multiple-choice test item has a certain number of alternatives,any number of which might be keyed. The examinee is also allowed to mark any number of alternatives. This increased flexibility over the one keyed alternative case is useful in practice but raises questions about appropriate scoring rules. In this article a certain class of item scoring rules called thebinary class is considered. The concepts ofstandard scoring rules and equivalence among these scoring rules are introduced in the misinformation model for which the traditional knowledge model is a special case. The examinee's strategy with respect to a scoring rule is examined. The critical role of a quantity called the scoring ratio is emphasized. In the case of examinee uncertainty about the number of correct alternatives on an item, a Bayes and a minimax strategy for the examinee are developed. Also an appropriate response for the examiner to the minimax strategy is outlined.Research partially supported under Grants N00014-67-A-0314-0022 from the Office of Naval Research and GS-32514 and MPS 75-07539 from the National Science Foundation.     

Reliability of multiple classifications

Cohen's kappa index is reformulated for multiple classifications based on exchangeable random variables. It is found that kappa is between 0 and 1 inclusive. Two characterizations for kappa are stated in terms of the relationship between such random variables. Within the normal test score model, kappa increases with test reliability and test length. Furthermore, when based on binary classifications, kappa is an inverse U-shaped function of the cutoff score. These trends also hold for the beta-binomial test score model.Paper read at the Spring meeting of the Psychometric Society, Bell Laboratories (Murray Hill, New Jersey), March 1976.The editorial assistance and helpful comments of Leonard S. Feldt, Sarah P. Seaman-Huynh, and the three referees are gratefully acknowledged.     

The Del Greco Assertive Behavior Inventory

Previous research in the area of assertiveness training has been examined with respect to the definitions of assertive, aggressive, and nonassertive behavior and the prominent assertion model. Logical inconsistencies have led to a refinement of the definitions of these behaviors, the inclusion of passive aggressive behavior, and the testing of a new two-dimensional model of assertion. The present study has developed a psychometric test to measure assertive, aggressive, nonassertive, and passive aggressive behavior in the college dormitory population. In addition, support was demonstrated for the validity of the two dimensional model of assertion. The instrument development was divided into four phases: item generation, item evaluation and revision, reliability testing and item analysis, and validity testing. Data analysis supported the two-dimensional model of assertion. In addition, an 86-item Del Greco Assertive Behavior Inventory designed for use with college dormitory students has weathered preliminary validity and reliability testing.     

Maximum likelihood estimation of the joint covariance matrix for sections of tests given to distinct samples with application to test equating

Consider an old testX consisting ofs sections and two new testsY andZ similar toX consisting ofp andq sections respectively. All subjects are given testX plus two variable sections from either testY orZ. Different pairings of variable sections are given to each subsample of subjects. We present a method of estimating the covariance matrix of the combined test (X ₁, ...,X _s ,Y ₁, ...,Y _p ,Z ₁, ...,Z _q ) and describe an application of these estimation techniques to linear, observed-score, test equating.The author is indebted to Paul W. Holland and Donald B. Rubin for their encouragement and many helpful comments and suggestions that contributed significantly to the development of this paper.This research was supported by the Program Statistics Research Project of the ETS Research Statistics Group.     

Parental divorce as a factor in children''s perception of human relationships: An analysis of human movement responses in the Rorschach test

The subjects were 108 Swedish children between 10 and 12 years old, who constituted a divorce and a non-divorce group of 27 girls and 27 boys each. Rorschach Tests were administered individually and human movement responses were analysed. Non-divorce children perceived more cooperative and fewer hostile interactions than the divorce children. Responses containing figures engaged in hostile interaction were most common among divorce girls, whereas the divorce boys had difficulties in perceiving any interaction whatsoever. The different effect of parental divorce on the ability of girls and boys to develop satisfactory social relationships is discussed.     

Covariance structure analysis in several populations

This paper is concerned with the study of covariance structural models in several populations. Estimation theory of the parameters that are subject to general functional restraints is developed based on the generalized least squares approach. Asymptotic properties of the constrained estimator are studied; and asymptotic chi-square tests are presented to evaluate appropriate model comparisons. The method of multipliers and the standard reparametrization technique are discussed in obtaining the estimates. The methodology is demonstrated by a set of real data.Computer facilities were provided by the Computer Services Center, The Chinese University of Hong Kong. The authors are indebted to several anonymous reviewers for suggestions for improvement of this paper.     

Two simple classes of mastery scores based on the beta-binomial model

Two simple classes of mastery scores which are suitable for hand calculations are presented for beta-binomial test score distributions combined with linear and cubic referral success. The models provide a simple way to explore the consequences of selecting an arbitrary mastery score. Such assessment would be useful whenever the test user is not willing to posta priori a loss ratio, but wishes to look at the various consequences before aiming at a particular score.     

Stopping rules for multi-category computerized classification testing

Computerized classification testing (CCT) aims to classify persons into one of two or more possible categories to make decisions such as mastery/non-mastery or meet most/meet all/exceed. A defining feature of CCT is its stopping criterion: the test terminates when there is enough confidence to make a decision. There is abundant research on CCT with a single cut-off, and two common stopping criteria are the sequential probability ratio test (SPRT) statistic and the generalized likelihood ratio statistic (GLR). However, there is a relative scarcity of research extending the SPRT to the multi-hypothesis case for when there is more than one cut-off. In this paper, we propose a new multi-category GLR (mGLR) statistic as well as a stochastically curtailed version of the CCT with three or more categories. A simulation study was conducted to show that the mGLR statistic outperformed the existing stopping rules by generating shorter average test length without sacrificing classification accuracy. Results also revealed that the stochastically curtailed mGLR successfully increased test efficiency in certain testing conditions.     

权力感对道德判断的影响及其理论解释:基于CNI模型的视角

权力感指个体对控制他人和自身的能力的感知。现有权力对道德判断的影响研究主要使用传统两难困境范式,围绕权力感影响主体道德判断的功利论倾向与道义论倾向展开,且研究观点尚未达成共识。由于传统两难困境范式的局限性,研究者开发了道德判断CNI模型探究权力感对道德判断的影响,该模型为两者之间的关系提供了新的解释路径。未来研究可关注CNI模型视角下不同类型权力感对道德判断的影响以及探讨特质性权力感与情境性权力感对个体道德判断差异的联合作用。     

Recommandations de l’ITC pour la traduction et l’adaptation de tests (seconde édition)

The second edition of the ITC guidelines for translating and adapting tests was prepared between 2005 and 2015 to improve upon the first edition, and to respond to advances in testing technology and practices. The 18 guidelines are organized into six categories to facilitate their use: pre-condition (3), test development (5), confirmation (4), administration (2), scoring and interpretation (2), and documentation (2). For each guideline, an explanation is provided along with suggestions for practice. A checklist is provided to improve the implementation of the guidelines.