Item grouping and item randomization in personality measurement

This paper describes a study examining the impact of item order in personality measurement on reliability, measurement equivalence and scale-level correlations. A large sample of university students completed one of three forms of the International Personality Item Pool version of the Big Five personality inventory: items sorted at random, items sorted by factor, and items cycled through factors. Results showed that the underlying measurement model and the internal consistency of the IPIP-Big Five scale was unaffected by differences in item order. Also, most of the scale-level correlations among factors were not significantly different across forms. Implications for the administration of tests and interpretation of test scores are discussed, and future research directions are offered.     

Comparative Studies on the Roles of Linguistic Knowledge and Sentence Processing Speed in L2 Listening and Reading Comprehension in an EFL Tertiary Setting

The present study investigated the relative contributions of vocabulary knowledge, grammar knowledge, and processing speed to second language listening and reading comprehension. Seventy-five Korean university students participated in the study. Results showed the three tested components had a significant portion of shared variance in explaining LC (33.9%) and RC (41.8%) without any unique significant contribution of each component. Two comprehension processes were similar in that vocabulary knowledge in its corresponding presentation mode and parsing best predicted a respective comprehension construct but different in that processing speed was part of the best combination of predictors only in RC.     

Methods for second order meta-analysis and illustrative applications

This paper presents methods for second order meta-analysis along with several illustrative applications. A second order meta-analysis is a meta-analysis of a number of statistically independent and methodologically comparable first order meta-analyses examining ostensibly the same relationship in different contexts. First order meta-analysis greatly reduces sampling error variance but does not eliminate it. The residual sampling error is called second order sampling error. The purpose of a second order meta-analysis is to estimate the proportion of the variance in mean meta-analytic effect sizes across multiple first order meta-analyses attributable to second order sampling error and to use this information to improve accuracy of estimation for each first order meta-analytic estimate. We present equations and methods based on the random effects model for second order meta-analysis for three situations and three empirical applications of second order meta-analysis to illustrate the potential value of these methods to the pursuit of cumulative knowledge.     

The Multitasking Preference Inventory: Toward an Improved Measure of Individual Differences in Polychronicity

This article describes the initial development of a new measure of individual polychronicity, a construct indicating a preference for multitasking as opposed to performing only one task at a time. Following a brief review of past definitional issues, a revised definition is offered and a new measure of polychronicity—the Multitasking Preference Inventory (MPI)—is created based on this definition. Three studies describe the creation, pilot testing, and psychometric refinement of an initial pool of items (Study 1); a convergent and discriminant validity analysis (Study 2); and initial findings with respect to the criterion-related validity of scores on the MPI (Study 3). Together, these findings provide preliminary evidence for the reliability of scores on the MPI and the validity of the MPI in predicting variables relevant to multitasking.     

The Team Personality–Team Performance Relationship Revisited: The Impact of Criterion Choice,Pattern of Workflow,and Method of Aggregation

Using meta-analytic evidence, this study tested trait- and task-based theoretical approaches to team personality management, using both team behaviors and team outcomes as criteria. Trait theories state that maximization of the team trait is harmful for Extroversion (complementary team fit) but beneficial for Conscientiousness, Agreeableness, and Emotional Stability (supplementary fit). Task-based theories state that tasks with few work exchanges are best reflected by mean trait scores, whereas tasks with frequent work exchanges are best represented by other types of scores (e.g., minimum score). Correlations between different aggregations of team personality and team performance were coded, as well as the study criterion choice and the pattern of workflow (as moderators). Partial support for both trait and task theories were found. Team Conscientiousness and Agreeableness provided supplementary fit primarily with team behaviors, but there was mixed evidence that Extroversion provided complementary fit. Furthermore, minimum and variance measures of the team trait related to team performance in tasks with frequent work exchanges, but not in tasks with few work exchanges. Results suggest several limitations with existing measurement methods, which are discussed.     

Normative Scoring of Multidimensional Pairwise Preference Personality Scales Using IRT: Empirical Comparisons With Other Formats

In this article, we offer some suggestions as to why tetrads and pentads have become the dominant formats for administering multidimensional forced choice (MFC) items but, in turn, raise questions regarding the underlying psychometric model and means of addressing item quality and scoring accuracy. We then focus our attention on multidimensional pairwise preference (MDPP) items and present an item response theory–based approach to constructing and modeling MDPP responses directly, assessing information at the item and scale levels, and a way of computing standard errors for trait scores and estimating scale reliability. To demonstrate the viability of this method for applied use, we show that the correspondence between MDPP scores derived from direct modeling with those obtained using single statement and unidimensional pairwise preference measures administered in a laboratory setting. Trait score correlations and criterion related validities are compared across testing formats and rating sources (i.e., self and other), and the usefulness of our model-based approach is further demonstrated by some illustrative results involving computerized adaptive tests (CAT).