Personality and problem-solving: The nature of individual differences in planning,scanning and verification

This study investigated the nature of individual differences in the time spent on different stages of a perceptual problem-solving task and the extent to which two task parameters (item complexity, goal setting) affected performance. A group of 24 Ss completed four sets of perceptual maze items as well as the EPI and AH5 tests. Perceptual mazes at two levels of item complexity were presented both with and without a computer-automated version. In addition to recording the overall solution time for each item, the program also records the time spent on the search, tracking and checking phases of maze-solving. The results indicated that the overall response speed is more closely related to intellectual ability (AH5 score) than to personality. In contrast, extraversion appeared to be more important in determining the relative amount of time spent on certain phases of maze-solving, particularly the initial period of scanning prior to making a response. The extent of these effects was found to depend on task parameters since the most striking differences were found on the most complex items presented without information as to the maximum solution obtainable.     

Order and serial position effects in response time with multiple-item probe recognition

We examined the order effect in item-recognition response time, that is, differences in response time for multiple-item probes containing items in the same or in the reverse order as those in the memory set. Experiment 1 used the response condition in which only one item must be positive for a positive response, Experiment 2 used homogeneous probes in which all the items are either positive or negative, and Experiment 3 used the condition in which all the items must be positive. Of particular interest were the serial position variations in order effects for probes containing items that were adjacent in the memory set. We previously found that such effects are an indication of subjective grouping of the memory set and the matching of the probe with these subgroups. The order effect in the one-positive condition was only weak in most cases, but it was strong with homogeneous probes when the memory set was objectively grouped or was ungrouped but with a constant set size. There were also strong order effects in the all-positive condition for probes with items that were nonadjacent in the memory set. Our results are interpreted in terms of a parallel match process based on a distribution over position of items in subjective or objective groups. We account for the origin of the distribution-over-position process in terms of multiple representations of the grouped memory sets. The model assumes that each subgroup is represented in memory several, and perhaps very many, times and that considerable error in item positioning can occur over the multiple representations of any group.     

Item banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS®): depression, anxiety, and anger

The authors report on the development and calibration of item banks for depression, anxiety, and anger as part of the Patient-Reported Outcomes Measurement Information System (PROMIS?). Comprehensive literature searches yielded an initial bank of 1,404 items from 305 instruments. After qualitative item analysis (including focus groups and cognitive interviewing), 168 items (56 for each construct) were written in a first person, past tense format with a 7-day time frame and five response options reflecting frequency. The calibration sample included nearly 15,000 respondents. Final banks of 28, 29, and 29 items were calibrated for depression, anxiety, and anger, respectively, using item response theory. Test information curves showed that the PROMIS item banks provided more information than conventional measures in a range of severity from approximately -1 to +3 standard deviations (with higher scores indicating greater distress). Short forms consisting of seven to eight items provided information comparable to legacy measures containing more items.     

Improving measurement precision of test batteries using multidimensional item response models

A conventional way to analyze item responses in multiple tests is to apply unidimensional item response models separately, one test at a time. This unidimensional approach, which ignores the correlations between latent traits, yields imprecise measures when tests are short. To resolve this problem, one can use multidimensional item response models that use correlations between latent traits to improve measurement precision of individual latent traits. The improvements are demonstrated using 2 empirical examples. It appears that the multidimensional approach improves measurement precision substantially, especially when tests are short and the number of tests is large. To achieve the same measurement precision, the multidimensional approach needs less than half of the comparable items required for the unidimensional approach.     

Exact person fit indexes for the rasch model for arbitrary alternatives

A new algorithm for obtaining exact person fit indexes for the Rasch model is introduced which realizes most powerful tests for a very general family of alternative hypotheses, including tests concerning DIF as well as model-deviating item correlations. The method is also used as a goodness-of-fit test for whole data sets where the item parameters are assumed to be known. For tests with 30 items at most, exact values are obtained, for longer tests a Monte Carlo-algorithm is proposed. Simulated examples and an empirical investigation demonstrate test power and applicability to item elimination.The author wishes to thank Elisabeth Ponocny-Seliger and the reviewers for many helpful comments. All exact goodness-of-fit tests proposed in this article are implemented in the menu-driven program T-Rasch 1.0 by Ponocny and Ponocny-Seliger (1999) which can be obtained from ProGAMMA (WWW: http://www.gamma.rug.nl) and also performs nonparametric tests.     

Tasks for Assessing Memory for Temporal Order Versus Memory for Items in Aging

Younger and older adults were compared on new tests for item and temporal memory suitable for use in future electrophysiological or neuroimaging studies. Following a series of 10 simple line drawings, 4-choice panels containing two previously seen targets and two novel distractors appeared. Participants identified which pictures had been seen previously (item), or which of the items was presented earliest (temporal order). Performance of younger adults was superior to that of older adults. Memory for item and temporal order were equivalent in the younger group. In contrast, the item memory of the older group was superior to memory for temporal order, confirming previous evidence of differential impairment of memory for temporal order with age.     

Qualitative variations in personality inventories: subjective understandings of items in a personality inventory

Most studies using personality inventories do not take individual, subjective understandings of the items into account. The present study is one of the few to have investigated the quality of individuals’ psychological processes when making the Likert-like responses often used in psychological inventories. Respondents were asked to elaborate verbally on their Likert item responses to the 10-item short version of the Big Five Inventory. A common assumption about personality inventories is that there is a relatively homogenous understanding of the items and, in particular, the rating scales across respondents. However, our results suggest that the same item responses to a given item can reflect a variety of qualities across individuals’ understandings. At the same time, similar understandings and ways of relating to an item can lead to different item responses. Such findings have substantial implications for quantitative personality studies as well as quantitative survey or questionnaire studies, in general.     

Selecting the most informative items in the IIP scales for personality disorders: an application of item response theory.

The first goal of the present analyses was to shorten the five scales (Pilkonis, P. A., Kim, Y., Proietti, J. M., & Barkham, M. [1996]. Journal of Personality Disorders, 10, 355-369) for personality disorders (PDs) developed from the Inventory of Interpersonal Problems (IIP), thereby increasing their attractiveness for screening purposes. The second goal was to illustrate, for more general purposes, the utility of item response theory (IRT) for such scale refinement. IRT analyses were performed using data collected from six different samples (N = 1149) at five sites and a two-parameter (2P) graded model designed for multiple response items like those on the IIP. The five most informative items from each scale were identified, based on the magnitude of item discrimination parameters and the range and elevation of individual item information functions. Preliminary analyses of the reliability and validity of the short forms of the scales (totaling 25 items) supported their value as alternatives to the longer forms (consisting of 47 items), although definitive tests of their psychometric properties await crossvalidation in independent samples. Analyses of the quality receiver operating characteristics (QROC) of the long and short forms showed that both versions can be useful in predicting the presence versus absence of any PD diagnosis arrived at by using either a "best estimate" clinical consensus method or a structured Axis II interview.     

Recognition and frequency judgments in young and elderly adults

Three experiments examined frequency judgments and recognition memory in young and elderly adults. Subjects were presented a long list of words at either a 5-s rate (Experiments 1 & 3) or a 1-s rate (Experiment 2), after which frequency-judgment and recognition memory tasks were administered. Either an absolute (Experiments 1 & 2) or a relative (Experiment 3) frequency-judgment task was used. The recognition test, which involved repeated tests of some items, involved either one incorrect item paired with each correct item (Experiments 1 & 2), or four incorrect items (Experiment 3). Age-related differences in frequency judgments, for the more frequently presented items, were found in all three experiments. For the recognition scores, the predicted interaction between age and successive tests was found only in Experiment 3. The results were interpreted within the framework of age-related differences in elaborative encoding and in distractibility to irrelevant stimuli.     

Initial codes for order in memory

The roles of positional and associative information in preserving the order of short sequences of items were investigated either by presenting an item in the same serial position on successive trials (position transfer), or by presenting the same pair of items in adjacent but shifted positions on successive trials (association transfer). While Experiment I found that positive transfer of position information occurred after just one repetition of an item in the same serial position, Experiment II found no evidence for association transfer after a single trial. If an item is presented in the same position, or if a pair of items are presented in adjacent positions on several successive trials, position transfer effects built up more rapidly than association transfer effects (Experiments III and IV). However, the rate at which association transfer effects are established depends on the type of item used, with lists of concrete words showing rapid establishment of associations (Experiment V) but lists of digits showing only slow development of associations (Experiment IV). It was concluded that serial order may be preserved both by positional and associative codes, but the establishment of an associative code is linked to the extraction of information about the items themselves.     

基于基尼指数的双目标CD-CAT选题策略

双目标CD-CAT的测验结果既可用于形成性评估也可用于终结性评估。基尼指数可度量随机变量的不确定性程度, 值越小则随机变量的不确定程度越低。本文用基尼指数度量被试知识状态类别以及能力估计置信区间后验概率的变化, 提出基于基尼指数的选题策略。Monte Carlo实验表明与已有的选题策略相比, 新策略的知识状态分类精度和能力估计精度都较高, 同时能有效兼顾题库利用均匀性, 并能快速实时响应, 且受认知诊断模型和被试知识状态分布的影响较小, 可用于实际测验中含多种认知诊断模型的混合题库。     

Two forms of minority-group test bias as psychometric artifacts with an animal model (Rattus norvegicus)

Controversy abounds over attributing group differences on tests to nature, nurture, or test bias. Limitations of correlational sampling from natural populations necessitate experimental methods to resolve underlying issues. In classical psychometrics test items are selected from a larger item pool through analysis of item responses in a sample of subjects. Rats of six inbred strains (n = 366) were tested in multiple mazes to provide a large item pool. Six populations were created, each with differing proportions of each strain. Items selected through independent item analyses within each population yielded six tests. An independent cross-validation sample (n = 146) provided scores on all six tests. This sample was also tested in another set of maze problems defined as the criterion to be predicted. Strain means and intrastrain predictive validities for the six tests varied with strain representation in the population used for item selection (p less than .001). Conventional item-selection procedures clearly produced two forms of minority test bias.     

Bayesian tests of measurement invariance

Random item effects models provide a natural framework for the exploration of violations of measurement invariance without the need for anchor items. Within the random item effects modelling framework, Bayesian tests (Bayes factor, deviance information criterion) are proposed which enable multiple marginal invariance hypotheses to be tested simultaneously. The performance of the tests is evaluated with a simulation study which shows that the tests have high power and low Type I error rate. Data from the European Social Survey are used to test for measurement invariance of attitude towards immigrant items and to show that background information can be used to explain cross‐national variation in item functioning.     

Explicit memory for rejected distractors during visual search

Although memory for the identities of examined items is not used to guide visual search, identity memory may be acquired during visual search. In all experiments reported here, search was occasionally terminated and a memory test was presented for the identity of a previously examined item. Participants demonstrated memory for the locations of the examined items by avoiding revisits to these items and memory performance for the items’ identities was above chance but lower than expected based on performance in intentional memory tests. Memory performance improved when the foil was not from the search set, suggesting that explicit identity memory is not bound to memory for location. Providing context information during test improved memory for the most recently examined item. Memory for the identities of previously examined items was best when the most recently examined item was tested, contextual information was provided, and location memory was not required.     

认知元反应理论--IRT直接应用于多值记分题

0-1记分测验的项目反应理论已经得到广泛的研究和应用.但是,许多测验都含有多值记分题,所以需要将IRT推广到此类情况.从认知理论的观点看,每个0-1记分题(项目)和多值记分题的每个测试点都可同样地看成一个由若干知识点构成的集合,称之为认知元;根据认知元之间存在的关系可以确定各受测者对各试题作出特定答案的概率,从而不需要引用任何其它假设就可将IRT的方法直接应用于含多值记分题的测验.本文应用这一理论分析了某些测验样本,结果表明是可行的.     

The recall of missing items

In two experiments, we studied the recall of missing items. Short lists of common words were presented once and were followed immediately by a random permutation of all but one of the presented items. The task of the subject was to recall the missing item--that is, the item present in the study set but missing from the probe set. Experiment 1 replicated the high accuracy with five-item lists originally reported by Yntema and Trask (1963) and showed that the latencies were quite short (about 750 msec). Experiment 2 varied list length unpredictably and showed that accuracy was a function of both list length (four, five, or six items) and serial position. Latency was again quite short but was essentially independent of list length and serial position. It was possible to simulate most of the effects with the power set model with no free parameters (i.e., parameters that varied with the experimental manipulations). The results seemed to be more consistent with a direct access model (the power set model of TODAM; Murdock, 1995) than with a simple search or serial-scanning model.     

Automated Item Generation with Recurrent Neural Networks

Utilizing technology for automated item generation is not a new idea. However, test items used in commercial testing programs or in research are still predominantly written by humans, in most cases by content experts or professional item writers. Human experts are a limited resource and testing agencies incur high costs in the process of continuous renewal of item banks to sustain testing programs. Using algorithms instead holds the promise of providing unlimited resources for this crucial part of assessment development. The approach presented here deviates in several ways from previous attempts to solve this problem. In the past, automatic item generation relied either on generating clones of narrowly defined item types such as those found in language free intelligence tests (e.g., Raven’s progressive matrices) or on an extensive analysis of task components and derivation of schemata to produce items with pre-specified variability that are hoped to have predictable levels of difficulty. It is somewhat unlikely that researchers utilizing these previous approaches would look at the proposed approach with favor; however, recent applications of machine learning show success in solving tasks that seemed impossible for machines not too long ago. The proposed approach uses deep learning to implement probabilistic language models, not unlike what Google brain and Amazon Alexa use for language processing and generation.     

The influence of cast shadows on visual search

We show that cast shadows can have a significant influence on the speed of visual search. In particular, we find that search based on the shape of a region is affected when the region is darker than the background and corresponds to a shadow formed by lighting from above. Results support the proposal that an early-level system rapidly identifies regions as shadows and then discounts them, making their shapes more difficult to access. Several constraints used by this system are mapped out, including constraints on the luminance and texture of the shadow region, and on the nature of the item casting the shadow. Among other things, this system is found to distinguish between line elements (items containing only edges) and surface elements (items containing visible surfaces), with only the latter deemed capable of casting a shadow.     

Item grouping and item randomization in personality measurement

This paper describes a study examining the impact of item order in personality measurement on reliability, measurement equivalence and scale-level correlations. A large sample of university students completed one of three forms of the International Personality Item Pool version of the Big Five personality inventory: items sorted at random, items sorted by factor, and items cycled through factors. Results showed that the underlying measurement model and the internal consistency of the IPIP-Big Five scale was unaffected by differences in item order. Also, most of the scale-level correlations among factors were not significantly different across forms. Implications for the administration of tests and interpretation of test scores are discussed, and future research directions are offered.