Item grouping and item randomization in personality measurement

This paper describes a study examining the impact of item order in personality measurement on reliability, measurement equivalence and scale-level correlations. A large sample of university students completed one of three forms of the International Personality Item Pool version of the Big Five personality inventory: items sorted at random, items sorted by factor, and items cycled through factors. Results showed that the underlying measurement model and the internal consistency of the IPIP-Big Five scale was unaffected by differences in item order. Also, most of the scale-level correlations among factors were not significantly different across forms. Implications for the administration of tests and interpretation of test scores are discussed, and future research directions are offered.     

Signal detection theory with item effects

Applications of signal detection theory (SDT) often involve presentations of different items on each trial, such as slides in a medical imaging study or words in a memory study. If factors particular to the items themselves, apart from being a signal or noise, affect observers’ responses, then ‘item effects’ are present. One way to model these effects is to use a latent continuous variable as an item ‘factor’, such as item ‘difficulty’. Details of SDT models with item effects are clarified via derivations of their implied conditional means, variances, and covariances. Intra-item correlations are defined and suggested as measures of the magnitude of item effects. The SDT-item models are simple random coefficient models and can be fit with standard software. More general models, such as item models with mixing and/or with random observer effects, are also considered.     

Development of an abbreviated Personal Globe Inventory using item response theory: The PGI-Short

The responses of 2813 individuals to the Personal Globe Inventory (Tracey, 2002) were examined with the goal of developing a shorter, yet valid version of the scale using item response theory to guide the process. A random sample of 1000 individuals was used to select the best items and then the remaining 1813 were used as a validation sample to examine psychometric properties. For items to be included in the shortened form, the option characteristic curves had to conform to theory and there could be no presence of differential item functioning across either gender or ethnicity. The best 80 items were retained forming the PGI-Short. This instrument demonstrated excellent reliability and adherence to a circular model, there was no differential item functioning across either gender or ethnicity. The PGI-Short was supported as an alternative to the fuller version of the PGI.     

项目功能差异在跨文化人格问卷分析中的应用

利用IRT的等级模型调查了中日两组被试关于SHIBA简易人格量表中“环境敏感性”的项目功能差异(DIF)的现状。研究发现：（1）量表中DIF的项目比例大(3/4);（2）DIF与项目内容、阈值有关而与区分度大小关系不大;（3）DIF项目间的日方特征曲线较之中方有较强的整合性。该研究利用DIF研究结果对跨文化的人格比较作了新尝试。最后提出了关于深化DIF研究的新课题     

Effect of individual thinking styles on item selection during study time allocation

The influence of individual differences on learners' study time allocation has been emphasised in recent studies; however, little is known about the role of individual thinking styles (analytical versus intuitive). In the present study, we explored the influence of individual thinking styles on learners' application of agenda‐based and habitual processes when selecting the first item during a study‐time allocation task. A 3‐item cognitive reflection test (CRT) was used to determine individuals' degree of cognitive reliance on intuitive versus analytical cognitive processing. Significant correlations between CRT scores and the choices of first item selection were observed in both Experiment 1a (study time was 5 seconds per triplet) and Experiment 1b (study time was 20 seconds per triplet). Furthermore, analytical decision makers constructed a value‐based agenda (prioritised high‐reward items), whereas intuitive decision makers relied more upon habitual responding (selected items from the leftmost of the array). The findings of Experiment 1a were replicated in Experiment 2 notwithstanding ruling out the possible effects from individual intelligence and working memory capacity. Overall, the individual thinking style plays an important role on learners' study time allocation and the predictive ability of CRT is reliable in learners' item selection strategy.     

Maintenance of item and order information in verbal working memory

Although verbal recall of item and order information is well-researched in short-term memory paradigms, there is relatively little research concerning item and order recall from working memory. The following study examined whether manipulating the opportunity for attentional refreshing and articulatory rehearsal in a complex span task differently affected the recall of item- and order-specific information of the memoranda. Five experiments varied the opportunity for articulatory rehearsal and attentional refreshing in a complex span task, but the type of recall was manipulated between experiments (item and order, order only, and item only recall). The results showed that impairing attentional refreshing and articulatory rehearsal similarly affected recall regardless of whether the scoring procedure (Experiments 1 and 4) or recall requirements (Experiments 2, 3, and 5) reflected item- or order-specific recall. This implies that both mechanisms sustain the maintenance of item and order information, and suggests that the common cumulative functioning of these two mechanisms to maintain items could be at the root of order maintenance.     

Detecting differential item functioning with confirmatory factor analysis and item response theory: toward a unified strategy

In this article, the authors developed a common strategy for identifying differential item functioning (DIF) items that can be implemented in both the mean and covariance structures method (MACS) and item response theory (IRT). They proposed examining the loadings (discrimination) and the intercept (location) parameters simultaneously using the likelihood ratio test with a free-baseline model and Bonferroni corrected critical p values. They compared the relative efficacy of this approach with alternative implementations for various types and amounts of DIF, sample sizes, numbers of response categories, and amounts of impact (latent mean differences). Results indicated that the proposed strategy was considerably more effective than an alternative approach involving a constrained-baseline model. Both MACS and IRT performed similarly well in the majority of experimental conditions. As expected, MACS performed slightly worse in dichotomous conditions but better than IRT in polytomous cases where sample sizes were small. Also, contrary to popular belief, MACS performed well in conditions where DIF was simulated on item thresholds (item means), and its accuracy was not affected by impact.     

Preference and reinforcer efficacy of high‐ and low‐tech items: A comparison of item type and duration of access

This study examined the interactions of stimulus type (high‐ vs. low‐tech) and magnitude (duration of access) on preference and reinforcer efficacy. Two preference assessments were conducted to identify highly preferred high‐tech and low‐tech items for each participant. A subsequent assessment examined preference for those items when provided at 30‐s and 600‐s durations. We then evaluated reinforcer efficacy for those same items when provided for a range of durations using progressive‐ratio schedules. Results suggested item type and access duration interacted to influence preference and reinforcer efficacy. Participants preferred high‐tech items at longer durations of access and engaged in more responding when the high‐tech item was provided for long durations, but these patterns were reversed for the low‐tech item. In addition, participants engaged in less responding when the high‐tech item was provided for short durations and when the low‐tech item was provided for long durations.     

The influence of item order on intentional response distortion in the assessment of high potentials: Assessing pilot applicants

An experiment was conducted to investigate the effects of item order and questionnaire content on faking good or intentional response distortion. It was hypothesized that intentional response distortion would either increase towards the end of a long questionnaire, as learning effects might make it easier to adjust responses to a faking good schema, or decrease because applicants' will to distort responses is reduced if the questionnaire lasts long enough. Furthermore, it was hypothesized that certain types of questionnaire content are especially vulnerable to response distortion. Eighty‐four pre‐selected pilot applicants filled out a questionnaire consisting of 516 items including items from the NEO five factor inventory (NEO FFI), NEO personality inventory revised (NEO PI‐R) and business‐focused inventory of personality (BIP). The positions of the items were varied within the applicant sample to test if responses are affected by item order, and applicants' response behaviour was additionally compared to that of volunteers. Applicants reported significantly higher mean scores than volunteers, and results provide some evidence of decreased faking tendencies towards the end of the questionnaire. Furthermore, it could be demonstrated that lower variances or standard deviations in combination with appropriate (often higher) mean scores can serve as an indicator for faking tendencies in group comparisons, even if effects are not significant.     

Post-encoding emotional arousal enhances consolidation of item memory,but not reality-monitoring source memory

The current study examined whether the effect of post-encoding emotional arousal on item memory extends to reality-monitoring source memory and, if so, whether the effect depends on emotionality of learning stimuli and testing format. In Experiment 1, participants encoded neutral words and imagined or viewed their corresponding object pictures. Then they watched a neutral, positive, or negative video. The 24-hour delayed test showed that emotional arousal had little effect on both item memory and reality-monitoring source memory. Experiment 2 was similar except that participants encoded neutral, positive, and negative words and imagined or viewed their corresponding object pictures. The results showed that positive and negative emotional arousal induced after encoding enhanced consolidation of item memory, but not reality-monitoring source memory, regardless of emotionality of learning stimuli. Experiment 3, identical to Experiment 2 except that participants were tested only on source memory for all the encoded items, still showed that post-encoding emotional arousal had little effect on consolidation of reality-monitoring source memory. Taken together, regardless of emotionality of learning stimuli and regardless of testing format of source memory (conjunction test vs. independent test), the facilitatory effect of post-encoding emotional arousal on item memory does not generalize to reality-monitoring source memory.     

Rasch analysis and item reduction of the hypomanic personality scale

The aim of the current study was to reduce the number of items in the 48-item hypomanic personality scale (HPS) and determine whether a unidimensional scale of the hypomanic trait could be derived. Previously collected HPS data from University students (n = 318) were applied to the Rasch model (one-parameter item response theory). Overall scale and individual item fit statistics were used to judge fit to the model and item maps employed to determine coverage of the trait. Cronbach’s Alpha and correlations with other questionnaires pre- and post-item reduction were evaluated. Rasch analysis indicated that the original HPS was not unidimensional, had significant redundancy and differential item functioning by age and gender. An iterative process of item reduction produced a 20-item HPS (HPS-20) that retained the concepts of the original HPS and had excellent fit to the Rasch model (χ² p = 0.27). Unidimensionality of the HPS-20 was confirmed. The traditional psychometric properties of the HPS-20 and coverage of the underlying hypomanic construct were similar to the original. It was possible to derive a unidimensional measure of the hypomanic trait. Further use of the HPS-20 is encouraged as it may increase understanding of the risk factors for affective disorders.     

Time-limited effects of emotional arousal on item and source memory

Two experiments investigated the time-limited effects of emotional arousal on consolidation of item and source memory. In Experiment 1, participants memorized words (items) and the corresponding speakers (sources) and then took an immediate free recall test. Then they watched a neutral, positive, or negative video 5, 35, or 50?min after learning, and 24 hours later they took surprise memory tests. Experiment 2 was similar to Experiment 1 except that (a) a reality monitoring task was used; (b) elicitation delays of 5, 30, and 45?min were used; and (c) delayed memory tests were given 60?min after learning. Both experiments showed that, regardless of elicitation delay, emotional arousal did not enhance item recall memory. Second, both experiments showed that negative arousal enhanced delayed item recognition memory only at the medium elicitation delay, but not in the shorter or longer delays. Positive arousal enhanced performance only in Experiment 1. Third, regardless of elicitation delay, emotional arousal had little effect on source memory. These findings have implications for theories of emotion and memory, suggesting that emotion effects are contingent upon the nature of the memory task and elicitation delay.     

Abortion attitudes, 1984-1987-1988: effects of item order and dimensionality.

The comparability of surveys is often hampered by differences in the item order of presentation. The major focus of the present study was to investigate whether a general item or a specific item at the beginning of the questionnaire would affect the endorsement as well as the scalability of a set of attitude items. By using a quasi-A-B-A experimental design for the six abortion items that appeared in the Edmonton Area Survey for the years 1984, 1987, and 1988, we found that the order of presentation of the items affected dramatically the endorsement of the abortion items. Approval of a general item was considerably higher when asked first than when asked after a specific item. In contrast, it was shown by means of a nonparametric item response theory model (the Mokken scale analysis) that the unidimensionality of the six abortion items was not affected by the manipulations of item order (i.e., the six abortion items measured the same concept in the three surveys). It was concluded that the six items are unidimensional and, therefore, create a single scale to measure the change in abortion attitudes across the three periods.     

Opposing effects of phonological similarity on item and order memory of words and nonwords in the serial recall task

This study shows that the classical phonological similarity effect (PSE) in immediate serial recall is critically affected by the lexicality of list items, the type of phonological similarity involved, and the scoring procedure. PSE was present in the serial recall score when phonologically distinct words were compared to words that share the middle vowel and end consonant (rhyming lists). PSE was absent in the serial recall score when phonologically distinct words were compared to words that share the initial and final consonants (consonant frame lists). There was a reversal of PSE in serial recall of nonwords when comparing distinct lists to both types of similar lists. Recall accuracy on the other hand was higher for distinct lists regardless of lexicality. Item errors dominated in relation to order errors in the case of nonwords, whereas order errors dominated in relation to item errors in the case of words. Furthermore, order errors were more common for phonologically similar lists, whereas item errors were more common for phonologically distinct lists. This may be the result of intra‐list and inter‐list interference, respectively. The dominance of the former error type may cause a classical PSE, whereas the dominance of the latter error type may cause a reversal of PSE. Finally, an item identification task yielded no evidence of an association between intra‐list interference and discriminability of items in a list.     

Scale validation via quantifying item validity using the Dm index

While most validity indices are based on total test scores, this paper describes a method for quantifying the construct validity of items. The approach is based on the item selection technique originally described by Piazza in 1980. Unfortunately, Piazza's P2 index suffers from some substantial limitations. The Dm coefficient provides an alternative which can be used for item selection and provides a validity index for a set of items. The index is similar to that of traditional criterion-related validity indices. Criterion-related validity is used to demonstrate the accuracy of hypothesized relations of the measure with outcome variables of interest in research and practice. This method may be useful when the sample of items or persons is small, rendering more traditional approaches such as factor analysis or item response theory inappropriate. An example of how to use the technique is provided.     

Use of item ratings to examine personality test item cognitive response processes

This research examines the processes respondents use to answer personality test items. A total of 158 true/false items from four scales of the Personality Research Form and the California Psychological Inventory were used as stimuli. University students (N = 120) responded to each item and indicated one of nine strategies used in deciding on a response. Obtained response strategy ratings for items were reliable and their frequencies corresponded closely to previous findings with other items. Subsequently, the associations between item response strategy frequencies and item-total correlations were computed. Congruent with previous research, better items avoided behaviours or experiences and evoked responding based on traits and on referring to the statements of others. The associations between item response strategies and other indices of item quality are discussed and implications regarding scale development are offered.     

Opposing effects of phonological similarity on item and order memory of words and nonwords in the serial recall task

This study shows that the classical phonological similarity effect (PSE) in immediate serial recall is critically affected by the lexicality of list items, the type of phonological similarity involved, and the scoring procedure. PSE was present in the serial recall score when phonologically distinct words were compared to words that share the middle vowel and end consonant (rhyming lists). PSE was absent in the serial recall score when phonologically distinct words were compared to words that share the initial and final consonants (consonant frame lists). There was a reversal of PSE in serial recall of nonwords when comparing distinct lists to both types of similar lists. Recall accuracy on the other hand was higher for distinct lists regardless of lexicality. Item errors dominated in relation to order errors in the case of nonwords, whereas order errors dominated in relation to item errors in the case of words. Furthermore, order errors were more common for phonologically similar lists, whereas item errors were more common for phonologically distinct lists. This may be the result of intra-list and inter-list interference, respectively. The dominance of the former error type may cause a classical PSE, whereas the dominance of the latter error type may cause a reversal of PSE. Finally, an item identification task yielded no evidence of an association between intra-list interference and discriminability of items in a list.     

高中英语阅读测验中题组模型的选择与应用

本研究通过高中英语阅读测验实测数据,对比分析双参数逻辑斯蒂克模型 (2PL-IRT)和加入不同数量题组的双参数逻辑斯蒂克模型 (2PL-TRT), 探究题组数量对参数估计及模型拟合的影响。结果表明：(1) 2PL-IRT模型对能力介于-1.50到0.50的被试,能力参数估计偏差较大;(2)将题组效应大于0.50的题组作为局部独立题目纳入模型,会导致部分题目区分度参数的低估和大部分题目难度参数的高估;(3)题组效应越大,将其当作局部独立题目纳入模型估计项目参数的偏差越大。     

Decomposition of a Rasch partial credit item into independent binary and indecomposable trinary items

For each Rasch (Masters) partial credit item, there exists a set of independent Rasch binary and indecomposable trinary items for which the sum of the scores and the partial credit score have identical probability density functions. If each indecomposable trinary item is further expressed as the sum of two binary items, then the binary items are positively dependent and cannot be both of the Rasch type. This paper was written while the author was working with Steve Ferrara and Hillary Michaels on some technical aspects of the Maryland School Performance Assessment Program. The author had been puzzled by the fact that most MSPAP assessment items have three or less score categories. With a psychometric justification now being apparent, this paper is dedicated to both of them.     

An item gains and losses analysis of false memories suggests critical items receive more item-specific processing than list items

In a repeated testing paradigm, list items receiving item-specific processing are more likely to be recovered across successive tests (item gains), whereas items receiving relational processing are likely to be forgotten progressively less on successive tests. Moreover, analysis of cumulative-recall curves has shown that item-specific processing produces a slower, but steadier rate of recall than relational processing. The authors relied on these findings to determine the type of processing that both list items and critical lures receive in the Deese-Roediger-McDermott false memory procedure. The first 2 experiments revealed that critical lures produced more item gains, but only the list items resulted in a decrease in item losses across successive tests. The critical lures also produced slower but steadier cumulative recall. In Experiments 3 and 4, the critical items were physically presented during study, which resulted in the lures producing progressively fewer losses across successive tests. The authors concluded that critical items receive more item-specific processing than list items but that unless they are presented in the list, they do not become part of participants' organized retrieval scheme.