The area between two item characteristic curves

Formulas for computing the exact signed and unsigned areas between two item characteristic curves (ICCs) are presented. It is further shown that when thec parameters are unequal, the area between two ICCs is infinite. The significance of the exact area measures for item bias research is discussed.The author expresses his appreciation to Jeffrey A. Slinde, Stephen Steinhaus, Audrey Qualls-Payne, Ivo Molenaar, and two anonymous reviewers for their very helpful and constructive comments.     

Source memory that encoding was self-referential: the influence of stimulus characteristics

Decades of research suggest that encoding information with respect to the self improves memory (self-reference effect, SRE) for items (item SRE). The current study focused on how processing information in reference to the self affects source memory for whether an item was self-referentially processed (a source SRE). Participants self-referentially or non-self-referentially encoded words (Experiment 1) or pictures (Experiment 2) that varied in valence (positive, negative, neutral). Relative to non-self-referential processing, self-referential processing enhanced item recognition for all stimulus types (an item SRE), but it only enhanced source memory for positive words (a source SRE). In fact, source memory for negative and neutral pictures was worse for items processed self-referentially than non-self-referentially. Together, the results suggest that item SRE and source SRE (e.g., remembering an item was encoded self-referentially) are not necessarily the same across stimulus types (e.g., words, pictures; positive, negative). While an item SRE may depend on the overall likelihood the item generates any association, the enhancing effects of self-referential processing on source memory for self-referential encoding may depend on how embedded a stimulus becomes in one’s self-schema, and that depends, in part, on the stimulus’ valence and format. Self-relevance ratings during encoding provide converging evidence for this interpretation.     

Whose picture is this? Children's memory for item and source information

Children aged 3½ to 6½ years viewed pictures of common objects presented either once or three times on one of two consecutive days. A different hand puppet was used to present the pictures on each day, providing both perceptual and temporal cues to source. At test, old (studied) and new (non‐studied) pictures were presented for item recognition and source identification. Results showed that both item and source accuracy were higher for older (M = 5; 9 years) than younger children (M = 4; 6 years). Significant interactions between Age and Day of study were found for both item and source accuracy. For younger children, accuracy was higher for pictures studied on Day 1 than Day 2 (significant for source identification but not item recognition), whereas older children showed the opposite pattern: Higher accuracy for Day 2 than Day 1 (significant for item recognition but not source identification). Results are interpreted with respect to proactive interference and response bias. The utility of signal detection theory measures in determining the basis of age differences in performance of source identification is discussed.     

Rasch analysis and item reduction of the hypomanic personality scale

The aim of the current study was to reduce the number of items in the 48-item hypomanic personality scale (HPS) and determine whether a unidimensional scale of the hypomanic trait could be derived. Previously collected HPS data from University students (n = 318) were applied to the Rasch model (one-parameter item response theory). Overall scale and individual item fit statistics were used to judge fit to the model and item maps employed to determine coverage of the trait. Cronbach’s Alpha and correlations with other questionnaires pre- and post-item reduction were evaluated. Rasch analysis indicated that the original HPS was not unidimensional, had significant redundancy and differential item functioning by age and gender. An iterative process of item reduction produced a 20-item HPS (HPS-20) that retained the concepts of the original HPS and had excellent fit to the Rasch model (χ² p = 0.27). Unidimensionality of the HPS-20 was confirmed. The traditional psychometric properties of the HPS-20 and coverage of the underlying hypomanic construct were similar to the original. It was possible to derive a unidimensional measure of the hypomanic trait. Further use of the HPS-20 is encouraged as it may increase understanding of the risk factors for affective disorders.     

Multilevel Reliability Measures of Latent Scores Within an Item Response Theory Framework

Abstract

This paper evaluated multilevel reliability measures in two-level nested designs (e.g., students nested within teachers) within an item response theory framework. A simulation study was implemented to investigate the behavior of the multilevel reliability measures and the uncertainty associated with the measures in various multilevel designs regarding the number of clusters, cluster sizes, and intraclass correlations (ICCs), and in different test lengths, for two parameterizations of multilevel item response models with separate item discriminations or the same item discrimination over levels. Marginal maximum likelihood estimation (MMLE)-multiple imputation and Bayesian analysis were employed to evaluate the accuracy of the multilevel reliability measures and the empirical coverage rates of Monte Carlo (MC) confidence or credible intervals. Considering the accuracy of the multilevel reliability measures and the empirical coverage rate of the intervals, the results lead us to generally recommend MMLE-multiple imputation. In the model with separate item discriminations over levels, marginally acceptable accuracy of the multilevel reliability measures and empirical coverage rate of the MC confidence intervals were found in a limited condition, 200 clusters, 30 cluster size, .2 ICC, and 40 items, in MMLE-multiple imputation. In the model with the same item discrimination over levels, the accuracy of the multilevel reliability measures and the empirical coverage rate of the MC confidence intervals were acceptable in all multilevel designs we considered with 40 items under MMLE-multiple imputation. We discuss these findings and provide guidelines for reporting multilevel reliability measures.     

Major Factors of Personality

Summary

French has recently resurveyed the literature regarding personality factors in order to (a) update his previous survey and (b) complement, in the noncognitive domain, the prestigious French, Ekstrom, and Price kit of reference tests for cognitive factors. This recent survey of personality factors is limited in at least two respects: (a) it relies on studies which are, in most cases, limited in size of item pools; (b) it has been greatly influenced by the two systems of Cattell and Guilford. Rather than rely on the pre-1971 literature (which belongs to an era when the factor analysis of a hundred variables was “a large factor analysis”), the writer has undertaken research aimed at accurate delineation of personality factors using itemfactoring coupled with item analysis. The article has two purposes: (a) to compare the factors in French's survey with those emerging in very recent work, and (b) to make the point that we are now likely to achieve greater comprehensiveness and accuracy in the identification of major personality factors.     

Development and Initial Validation of the Five-Factor Model Adolescent Personality Questionnaire (FFM–APQ)

This research reports on the 4-phase development of the 25-item Five-Factor Model Adolescent Personality Questionnaire (FFM–APQ). The purpose was to develop and determine initial evidence for validity of a brief adolescent personality inventory using a vocabulary that could be understood by adolescents up to 18 years old. Phase 1 (N = 48) consisted of item generation and expert (N = 5) review of items; Phase 2 (N = 179) involved item analyses; in Phase 3 (N = 496) exploratory factor analysis assessed the underlying structure; in Phase 4 (N = 405) confirmatory factor analyses resulted in a 25-item inventory with 5 subscales.     

Acting or listening: Adult age differences in source recall of enacted and nonenacted statements

This study explored age-related differences in item and source recall of enacted and nonenacted items in a sample of healthy adults between the ages of 35 and 80 years (N=1,000). Participants were screened on a variety of demographic, psychometric, and biological variables. They were presented with sentences (e.g., Lift the book) that they either read or enacted. They were then asked to recall the object of each stimulus sentence (item recall) and to identify its format of encoding (source recall). Age-related decrements were observed both in item and source memory, although age differences in source memory were more accentuated than in item memory. Further, the results indicated an overall impairment of source memory across age when individual differences in the demographic, psychometric, and biological variables were taken into consideration.     

An evaluation of empirical measures of source identification

Source identification refers to memory for the origin of information. A consistent nomenclature is introduced for empirical measures of source identification which are then mathematically analyzed and evaluated. The ability of the measures to assess source identification independently of identification of an item as old or new depends on assumptions made about how inconsistencies between the item and source components of a source-monitoring task may be resolved. In most circumstances, the empirical measure that is used most often when source identification is measured by collapsing across pairs of sources (sometimes called “the identification-of-origin score”) confounds item identification with source identification. Alternative empirical measures are identified that do not confound item and source identification in specified circumstances. None of the empirical measures examined provides a valid measure of source identification in all circumstances.     

Toward a complete decision model of item and source recognition

In a recognition memory test, subjects may be asked to decide whether a test item is old or new (item recognition) or to decide among alternative sources from which it might have been drawn for study (source recognition). Confidence-rating-based receiver operating characteristic (ROC) curves for these superficially similar tasks are quite different, leading to the inference of correspondingly different decision processes. A complete account of source and item recognition will require a single model that can be fit to the entire data set. We postulated a detection-theoretic decision space whose dimensions, in the style of Banks (2000), are item strength and the relative strengths of the two sources. A model that assumes decision boundaries of constant likelihood ratios, source guessing for unrecognized items, and nonoptimal allocation of attention can account for data from three canonical data sets without assuming any processes specifically devoted to recollection. Observed and predicted ROCs for one of these data sets are given in the article, and ROCs for the other two may be downloaded from the Psychonomic Society’s Archive of Norms, Stimuli, and Data, www.psychonomic.org/archive.     

The reactivation of associated information affects source monitoring

In this study, we examined a source-monitoring phenomenon that arises from reactivated related information from the study phase. Three experiments showed that source attributions for target events were influenced not only by the target item itself, but also by studied information about related items. In Experiment 1, source memory for target items that have a high forward association value to a single related study item (e.g., credit) were affected by the source of the associated information (e.g., card), so that memory performance was better when associated items were presented in the same source rather than a different source. A similar effect occurred with bidirectional associates (Exp. 2), as well as with synonymous pairs of words (Exp. 3). We argue that the source information of the reactivated material can be commingled with information about a candidate during a source judgment at retrieval and thereby can affect performance.     

Demographic Question Placement: Effect on Item Response Rates and Means of a Veterans Health Administration Survey

Purpose

This study examined whether demographic question placement affects demographic and non-demographic question completion rates, non-demographic item means, and blank questionnaire rates using a web-based survey of Veterans Health Administration employees.

Methodology

Data were taken from the 2010 Voice of the Veterans Administration Survey (VoVA), a voluntary, confidential, web-based survey offered to all VA employees. Participants were given two versions of the questionnaires. One version had demographic questions placed at the beginning and the other version had demographic questions placed at the end of the questionnaire.

Findings

Results indicated that placing demographic questions at the beginning of a questionnaire increased item response rate for demographic items without affecting the item response rate for non-demographic items or the average of item mean scores.

Implications

In addition to validity issues, a goal for surveyors is to maximize response rates and to minimize the number of missing responses. It is therefore important to determine which questionnaire characteristics affect these values. Results of this study suggest demographic placement is an important factor.

Originality/Value

There are various opinions about the most advantageous location of demographic questions in questionnaires; however, the issue has rarely been examined empirically. This study uses an experimental design and a large sample size to examine the effects of demographic placement on survey response characteristics.     

Asymmetric Item Characteristic Curves and Item Complexity: Insights from Simulation and Real Data Analyses

While item complexity is often considered as an item feature in test development, it is much less frequently attended to in the psychometric modeling of test items. Prior work suggests that item complexity may manifest through asymmetry in item characteristics curves (ICCs; Samejima in Psychometrika 65:319–335, 2000). In the current paper, we study the potential for asymmetric IRT models to inform empirically about underlying item complexity, and thus the potential value of asymmetric models as tools for item validation. Both simulation and real data studies are presented. Some psychometric consequences of ignoring asymmetry, as well as potential strategies for more effective estimation of asymmetry, are considered in discussion.     

A Bayesian Semiparametric Item Response Model with Dirichlet Process Priors

In Item Response Theory (IRT), item characteristic curves (ICCs) are illustrated through logistic models or normal ogive models, and the probability that examinees give the correct answer is usually a monotonically increasing function of their ability parameters. However, since only limited patterns of shapes can be obtained from logistic models or normal ogive models, there is a possibility that the model applied does not fit the data. As a result, the existing method can be rejected because it cannot deal with various item response patterns. To overcome these problems, we propose a new semiparametric IRT model using a Dirichlet process mixture logistic distribution. Our method does not rely on assumptions but only requires that the ICCs be a monotonically nondecreasing function; that is, our method can deal with more types of item response patterns than the existing methods, such as the one-parameter normal ogive models or the two- or three-parameter logistic models.     

Latent Trait Modeling with the Common beliefs Survey III: Using Item Response Theory to Evaluate an Irrational Beliefs Inventory

Assessment of irrational beliefs by such measures as the Common Beliefs Survey III (CBS) has traditionally relied upon classical test theory assumptions, in which the properties of specific test items are less important than the total test score as the aggregate of all item responses. An alternative approach using item response theory (IRT) methodology allows one to specify the parameters of difficulty and discrimination for each test item. Difficulty levels of CBS items range along a continuum of irrationality, the implied latent trait measured by responses to the questionnaire as a whole. We evaluated the CBS responses of 605 individuals from clinical and college settings, drawing from current and archival data. The original Likert scale ratings were recoded into dichotomous scores. Fourteen of the 54 items were highly or very highly discriminating in distinguishing respondents with high and low irrationality levels. However, discriminating items exhibited a very narrow range of difficulty; most functioned at a point a little above the halfway mark on the continuum of irrationality. Item characteristic curves and test information curves were very similar for female (n = 424) and male (n = 179) respondents. We derived a 4-item screening test for irrationality from our IRT analyses of the 54 CBS items. Further test development, focused on the selection and scaling of items with a much broader range of difficulty, would facilitate evaluation of the hierarchical structure of irrational beliefs. Portions of this paper were presented at the 39th Annual Convention of the Association for Behavioral and Cognitive Therapies, Washington, DC, November, 2005.     

First steps in developing the Dutch version of the Bayley III: Is the original Bayley III and its item sequence also adequate for Dutch children?

The Bayley Scales of Infant and Toddler Development [Bayley III, Bayley, N. (2006). The Bayley scales of infant and toddler development. San Antonio, TX: The Psychological Corporation] is currently developed and exclusively normed for American children. In this study, the appropriateness of the Bayley III item content and item sequence for Dutch children was evaluated. The translated version, the Bayley-III-NL was evaluated in two phases. In phase 1 (N = 100), analyses showed that overall the item content seemed to be appropriate for Dutch children. In addition, the item sequence was found to be suitable for Dutch children. After phase 1, small adaptations were made to the instructions of a few items based on the experiences of the examiners to improve standardization in performance. In phase 2 (N = 400), the findings of phase 1 were confirmed in a larger sample. It is concluded that Dutch norms can be based upon the current version of the Bayley-III-NL.     

Directed forgetting in the list method affects recognition memory for source

The effects of list-method directed forgetting on recognition memory were explored. In Experiment 1 (N = 40), observers were instructed to remember words and their type-cases; in Experiment 2 (N = 80), the instruction was to remember words and their colours. Two lists of 10 words were presented; after the first list, half of the observers (forget) were instructed to forget that list, and the other half (remember) were not given the forget instruction. Recognition of items (words) as well as source (encoding list + case/colour) was measured for forget and remember observers. The forget instruction affected case/colour memory more consistently than item and list memory; a multinomial analysis indicated that source information was affected by the forget instructions. The results indicated that recognition of source information may be a more sensitive indicator of forgetting than recognition of items.     

Comparing Job Applicants to Non‐applicants Using an Item‐level Bifactor Model on the HEXACO Personality Inventory

The present study evaluated the ability of item‐level bifactor models (a) to provide an alternative explanation to current theories of higher order factors of personality and (b) to explain socially desirable responding in both job applicant and non‐applicant contexts. Participants (46% male; mean age = 42 years, SD = 11) completed the 200‐item HEXACO Personality Inventory‐Revised either as part of a job application (n = 1613) or as part of low‐stakes research (n = 1613). A comprehensive set of invariance tests were performed. Applicants scored higher than non‐applicants on honesty‐humility (d = 0.86), extraversion (d = 0.73), agreeableness (d = 1.06), and conscientiousness (d = 0.77). The bifactor model provided improved model fit relative to a standard correlated factor model, and loadings on the evaluative factor of the bifactor model were highly correlated with other indicators of item social desirability. The bifactor model explained approximately two‐thirds of the differences between applicants and non‐applicants. Results suggest that rather than being a higher order construct, the general factor of personality may be caused by an item‐level evaluative process. Results highlight the importance of modelling data at the item‐level. Implications for conceptualizing social desirability, higher order structures in personality, test development, and job applicant faking are discussed. Copyright © 2017 European Association of Personality Psychology     

Negative congruence effects in letter and pseudo-letter recognition: the role of similarity and response conflict

Letters and pseudo-letters were presented in three experiments using a sequential same-different task. While first items were always presented in isolation, the second item was either presented in isolation or surrounded by geometrical non-target shapes that could be congruent or incongruent to the target. In two experiments, a physical sameness criterion was used. In Experiment 1, in one condition, different pairs were always distinct in shape, in another they were similar in shape. Negative congruence effects were obtained for different pairs that are similar. In Experiment 2, this effect is replicated within participants. In this experiment, similar and dissimilar stimuli were mixed. The results were explained in terms of the difficulty of responding different to stimuli that are similar in shape: when the second item is surrounded by a congruent shape, the similarity is emphasized, making this response even more difficult. In Experiment 3, the same stimuli were presented using a categorical sameness criterion. This reduces the role of physical similarity and thus eliminates the response conflict. As a result, negative congruence effects were no longer observed. Taken together, the three experiments identify another source of negative congruence effects besides the ones recently reported in the literature.     

The beneficial effect of contextual emotion on memory: the role of integration

This study investigates the effects of emotion on the integration mechanism which binds together the components of an event and the relations between these components and encodes them within a memory trace [Versace, R., Vallet, G. T., Riou, B., Lesourd, M., Labeye, É, & Brunel, L. (2014). Act-In: An integrated view of memory mechanisms. Journal of Cognitive Psychology, 26(3), 280–306. doi:10.1080/20445911.2014.892113]. Based on the literature, the authors argue that, in a memory task, contextual emotion could strengthen the integration mechanism and, more specifically, the relations between a target item and its contextual features. To test this hypothesis, the authors used two odorants (neutral and negative) to compare the effects of a negative context with those of a neutral one on three different types of recall: item recall (memory for pictures objects), source recall (spatial position of the pictures in a matrix) and recall of the association between an item and its location. The results showed that, in the negative odour context, association recall and source recall – but not item recall – were better than in the neutral odour context thus confirming the effect of emotion on integration. The results lead to the hypothesis that the effects of emotion on memory are linked to the way emotion is introduced into the experimental settings: via the items to be memorised or via the context.