A brief inventory of Interpersonal Problems-Circumplex using nonparametric item response theory: introducing the IIP-C-IRT

Nonparametric item response theory methods were applied to the responses of 1,000 college students on the 64 items of the Inventory of Interpersonal Problems-Circumplex (IIP-C; Alden, Wiggins, & Pincus, 1990) to develop an abbreviated 32-item version of the instrument. In a separate validation sample of 981 students, the newly selected scale items did not show evidence of differential item functioning across males and females. There was high convergence found between the new scales and IIP-C parent scales, along with commensurate or improved fits to the circular structural model relative to the full scale and its existing brief derivatives-the IIP-32 and the IIP-SC. Results provide evidence that the new brief scales can improve the level of precision and information yielded in brief assessments of interpersonal problems without gender bias.     

Does the rose still smell as sweet? Item variability across test forms and revisions

This article examines item stability when the same item appears in different contexts. The 1st section considers the assumptions in classical test theory and item response theory concerning the relationship between the item and the trait it is presumed to measure. The 2nd section presents contextualist challenges to the measurement theory assumptions about item properties and shows the instability of item characteristics across different testing contexts. The 3rd section describes methods for checking the relationship between items and traits. Classical test methods, item response methods, and structural equation methods for assessing item stability are reviewed. The instability of item characteristics across contexts should caution researchers to assess, and not assume, that items operate the same way on different test versions. Item instability also indicates the need for a more detailed understanding of the psychological processes that occur between item and answer.     

非词复述测验的编制和初步分析

本研究编制了一种用于测量儿童语音工作记忆的测验———非词复述测验,并在48名四年级小学生中初步进行信度、效度检验和项目分析。结果表明:(1)该测验有较高的重测信度;(2)该测验具有较高的结构效度和效标效度;(3)分测验二的项目难度分布比较合理,多数项目鉴别力较高,而分测验一的项目难度分布和项目鉴别力则有待于在今后的研究中进一步提高。     

Pictures in Test Items: Effects on Response Time and Response Correctness

Although pictures are often added to text in items of educational tests, little is known about their influence on item solving. Therefore, we conducted an experiment in which we examined how pictures affected item solving. A total of N = 158 fourth‐grade students completed a physics knowledge test under one of six experimental conditions. The experimental conditions varied according to whether or not pictures were presented in the stem and in the answer options of the test items. The results showed that pictures in the stem and in the answer options increased the correctness with which students responded to the test items. This was particularly true for test items that required the application of relationships. In addition, response time was reduced when pictures were added to the answer options of the test items. Hence, pictures are an important feature of test items that produce changes in item processing. Copyright © 2011 John Wiley & Sons, Ltd.     

Psychopathy in adolescent female offenders: an item response theory analysis of the psychopathy checklist: youth version

The present study examined the applicability of the PCL:YV items to a sample of detained adolescent girls. Item response theory (IRT) was used to analyze test and item functioning of the PCL:YV. Examination of IRT trace lines indicated that the items most discriminating of the underlying construct of psychopathy included "callousness and a lack of empathy", "conning and manipulation", and "a grandiose sense of self-worth". Results from the analyses also demonstrated that the items least discriminating in this sample, or least useful for identifying psychopathy, included "poor anger control", "shallow affect", or engaging in a "serious violation of conditional release". Consistent with previous research (Cooke & Michie, 1997; Hare, 2003), interpersonal and affective components of psychopathy provided more information than behavioral features. Moreover, although previous research has also found affective features to provide the most information in past studies, it was interpersonal features of psychopathy in this case, followed by affective features, that provided greater levels of information. Implications of these results are discussed.     

Construction of Circumplex Scales for the Inventory of Interpersonal Problems

We constructed a set of circumplex scales for the Inventory of Interpersonal Problems (IIP; Horowitz, Rosenberg, Baer, Ureno, & Villasenor, 1988). Initial scale construction used all 127 items from this instrument in two samples of university undergraduates (n = 197; n = 273). Cross-sample stability of item locations plotted against the first two principal components was high. A final set of eight 8-item circumplex scales was derived from the combined sample (n = 470) and cross-validated in a third university sample (n = 974). Finally, we examined the structural convergence of the IIP circumplex scales with an established measure of interpersonal dispositions, the Revised Interpersonal Adjective Scales (IAS-R; Wiggins, Trapnell, & Phillips, 1988). Although both circumplex instruments were derived independently, they shared a common Circular space. Implications of these results are discussed with reference to current research methods for the study of interpersonal behavior.     

Item difficulty parameter estimation using the idea of the graded response model and computerized adaptive testing

Abstract:  In test operations using IRT (item response theory), items are included in a test before being used to rate subjects and the response data is used to estimate their item parameters. However, this method of test operation may lead to item content leakage and an adequate test operation can become difficult. To address this problem, Ozaki and Toyoda (2005, 2006 ) developed item difficulty parameter estimation methods that use paired comparison data from the perspective of the difficulty of items as judged by raters familiar with the field. In the present paper, an improved method of item difficulty parameter estimation is developed. In this new method, an item for which the difficulty parameter is to be estimated is compared with multiple items simultaneously, from the perspective of their difficulty. This is not a one-to-one comparison but a one-to-many comparison. In the comparisons, raters are informed that items selected from an item pool are ordered according to difficulty. The order will provide insight to improve the accuracy of judgment.     

计算机化自适应测验中原始题项目参数的估计

计算机化自适应测验(Computerized Adaptive Testing, 简称CAT)其安全性面临着新的挑战, 小题库的安全更受威胁。如何建设一个大型、优质的题库成为CAT研究中一个非常重要的课题。目前CAT题库的建设存在一些问题, 如成本高且保密性较差。尤其是等值技术较复杂且锚题重复使用容易造成泄露。如能在实施CAT过程中插入未经过参数估计的项目(原始题), 同时对原始题项目参数进行估计, 这对建设大型、优质的CAT题库来说其意义是不言而喻的。本文基于1PLM和2PLM对此进行研究, 提出了原始题在线估计的新方法以及推导出了求区分度参数a迭代初值的计算公式。研究结果表明：无论是模拟研究还是实证研究, 原始题被作答的次数对项目参数估计结果都会产生不同的影响, 并且原始题作答人数越多项目参数估计精度也越高。     

Use of item ratings to examine personality test item cognitive response processes

This research examines the processes respondents use to answer personality test items. A total of 158 true/false items from four scales of the Personality Research Form and the California Psychological Inventory were used as stimuli. University students (N = 120) responded to each item and indicated one of nine strategies used in deciding on a response. Obtained response strategy ratings for items were reliable and their frequencies corresponded closely to previous findings with other items. Subsequently, the associations between item response strategy frequencies and item-total correlations were computed. Congruent with previous research, better items avoided behaviours or experiences and evoked responding based on traits and on referring to the statements of others. The associations between item response strategies and other indices of item quality are discussed and implications regarding scale development are offered.     

Examining the Process of Responding to Circumplex Scales of Interpersonal Values Items: Should Ideal Point Scoring Methods Be Considered?

The Circumplex Scales of Interpersonal Values (CSIV) is a 64-item self-report measure of goals from each octant of the interpersonal circumplex. We used item response theory methods to compare whether dominance models or ideal point models best described how people respond to CSIV items. Specifically, we fit a polytomous dominance model called the generalized partial credit model and an ideal point model of similar complexity called the generalized graded unfolding model to the responses of 1,893 college students. The results of both graphical comparisons of item characteristic curves and statistical comparisons of model fit suggested that an ideal point model best describes the process of responding to CSIV items. The different models produced different rank orderings of high-scoring respondents, but overall the models did not differ in their prediction of criterion variables (agentic and communal interpersonal traits and implicit motives).     

Item placement on a personality measure: effects on faking behavior and test measurement properties

Although personality tests are widely used to select applicants for a variety of jobs, there is concern that such measures are fakable. One procedure used to minimize faking has been to disguise the true intent of personality tests by randomizing items such that items measuring similar constructs are dispersed throughout the test. In this study, we examined if item placement does influence the fakability and psychometric properties of a personality measure. Study participants responded to 1 of 2 formats (random vs. grouped items) of a personality test honestly and also under instructions to fake or to behave like an applicant. Results indicate that the grouped item placement format was more fakable for the Neuroticism and Conscientiousness scales. The test with items randomly placed fit the data better within the honest and applicant conditions. These findings demonstrate that the issue of item placement should be seriously considered before administering personality measures because different item presentations may affect the incidence of faking and the psychometric properties of the measure.     

Predicting consistent psychological test item responses: A comparison of models

The efficacy of four models for predicting the stability of a given individual's test item responses on a structured inventory was examined. Two models were based on item characteristics alone and predicted that an individual would be most likely to change responses to items with moderate endorsement probabilities, or with moderate social desirability scale values. Two other prediction models incorporated individual differences in the perception of item characteristics by predicting that unstable items would have relatively long response latencies for an individual, or would be near an individual's threshold for responding desirably to items. Results from two studies yielded support for the following conclusions: (a) a person's test item responses are relatively stable over short time intervals; (b) items to which a person will show response changes on retest can be identified to a statistically significant degree; (c) the models based on response latencies constituted in both studies a significantly better predictor than the other models examined. The implications of these results for the threshold model were discussed as were the practical and theoretical applications of the response latency-item stability relationship at the level of an individual's test protocol.     

Sequential Computerized Mastery Tests—Three Simulation Studies

A simulation study of a sequential computerized mastery test is carried out with items modeled with the 3 parameter logistic item response theory model. The examinees' responses are either identically distributed, not identically distributed, or not identically distributed together with estimation errors in the item characteristics. The simulations indicated that the observed results from the operating characteristic function differ significantly from the theoretical results, which is probably due to the use of an approximation formula. The mean number of items in a test, the distribution of test length, and the variance depend highly on how well we know the true values of the item characteristics and whether they are identically distributed or not.     

Psychometric Stability of MMPI-A Item Modifications

The 478-item Minnesota Multiphasic Personality Inventory-Adolescent (MMPI-A) is a revision of the original test instrument for use in the assessment of adolescents. As part of the MMPI-A development, process, 70 items were modified from their appearance in the original test instrument to eliminate obsolete or sexist language, reduce awkward phrasing, increase item clarity, or improve item relevancy to adolescents' life experiences. If these modifications in the original item pool resulted in substantial differences in the frequency of respondents' endorsements of these lest items, such differences could pose a threat to the generalizability of research findings from the original form of the MMPI to the MMPI-A. This study examined the psychometric stability of modified items by comparing item endorsement frequency and item test-retest correlations in a group of 265 adolescents evaluated in repeated administrations design. Results of item analyses indicate that item modifications designed to improve the content or grammatical structure of these 70 items did not result in significant changes in response patterns.     

Items bundles

An item bundle is a small group of multiple choice items that share a common reading passage or graph, or a small group of matching items that share distractors. Item bundles are easily identified by paging through a copy of a test. Bundled items may violate the latent conditional independence assumption of unidimensional item response theory (IRT), but such a violation would not typically suggest the existence of a new fundamental human ability to read one specific reading passage or to interpret one specific graph. It is important, therefore, to have theoretical concepts and empirical checks that distinguish between, on the one hand, anticipated violations of latent conditional independence within item bundles, and, on the other hand, violations that cannot be attributed to idiosyncratic features of test format and instead suggest departures from unidimensionalty. To this end, two theorems on unidimensional IRT are extended to describe observable item response distributions when there is conditional independencebetween but not necessarilywithin item bundles.The author is grateful to Ivo Molenaar and the referees for many helpful suggestions, and to D. Thayer for assistance with computing.     

Validity of the Developmental Test of Visual-Motor Integration Supplemental Developmental Test of Visual Perception

Visual perceptual skills of school-age children are often assessed using the Supplemental Developmental Test of Visual Perception of the Developmental Test of Visual-Motor Integration. The study purpose was to consider the construct validity of this test by evaluating its scalability (interval level measurement), unidimensionality, differential item functioning, and hierarchical ordering of its items. Visual perceptual performance scores from a sample of 356 typically developing children (171 boys and 185 girls ages 5 to 11 years) were used to complete a Rasch analysis of the test. Seven items were discarded for poor fit, while none of the items exhibited differential item functioning by sex. The construct validity, scalability, hierarchical ordering, and lack of differential item functioning requirements were met by the final test version. Since 7 test items did not fit the Rasch analysis specifications, the clinical value of the test is questionable and limited.     

The role of familiarity in item recognition, associative recognition, and plurality recognition on self-paced and speeded tests

Four experiments compare the effect of familiarity on item, associative, and plurality recognition on self-paced and speeded tests. The familiarity of test items was enhanced by presenting a prime that matched the subsequent test item. On item and plurality recognition tests, participants were more likely to respond "old" to primed than to unprimed test items. In associative recognition, priming increased the proportion of old responses on a speeded test, but not on a self-paced test. This suggests that familiarity plays a larger role in item and plurality recognition than in associative recognition on self-paced tests. On speeded tests, priming has a similar effect on item, associative, and plurality recognition. Results suggest that item and associative recognition rely differentially on familiarity and recollection. They are also consistent with recent evidence suggesting that different processes underlie plurality and associative recognition.     

Why more can be less: An inference-based explanation for hyper-subadditivity in bundle valuation

We conceptualize, develop, and test a multiple-item bundle valuation model through which decision makers are able to make inferences about the value of uncertain items based on the value of certain items. Results of four experiments indicate that bundling a low-value certain item with a high-value uncertain item, which are not substitutes, results in a bundle valuation lower than the value of the uncertain item alone. We refer to this highly unexpected and previously unexplained phenomenon as “hyper-subadditivity.” In addition we find that bundling a high-value certain item with a low-value uncertain item leads to superadditivity, even though the items are not complements. Hence, we find that when two objects are bundled together, and one has a more certain value, decision makers use the value of the certain item to infer the value of the less certain item. They might infer that the other (less certain) object must be worth an amount similar to the item with which they are paired. We further demonstrate that reducing uncertainty eliminates these effects, and that direct value inferencing (not simple numeric priming, nor inferences about quality) is the most likely mechanism driving these effects.