应用项目反应理论创建图形推理测验题库

自编235个图形推理测验题目。采用铆测验等值设计,以72个联合型瑞文测验题目为铆题,对初中到大学各能力层次的1733名男性进行了测验。使用BILOG MG3.0（边际极大似然估计）对实测数据进行了分析,采用Logsitic 3参数模型。剔除数据与模型拟合不好的题目以及信息函数最大值小于0.3的题目,最终建立一个包含181道题目的题库。该题库可以用于淘汰智力较低的应征青年     

Conflict and metacognitive control: the mismatch-monitoring hypothesis of how others' knowledge states affect recall

Information about others' success in remembering is frequently available. For example, students taking an exam may assess its difficulty by monitoring when others turn in their exams. In two experiments, we investigated how rememberers use this information to guide recall. Participants studied paired associates, some semantically related (and thus easier to retrieve) and some unrelated (and thus harder). During a subsequent cued recall test, participants viewed fictive information about an opponent's accuracy on each item. In Experiment 1, participants responded to each cue once before seeing the opponent's performance and once afterwards. Participants reconsidered their responses least often when the opponent's accuracy matched the item difficulty (easy items the opponent recalled, hard items the opponent forgot) and most often when the opponent's accuracy and the item difficulty mismatched. When participants responded only after seeing the opponent's performance (Experiment 2), the same mismatch conditions that led to reconsideration even produced superior recall. These results suggest that rememberers monitor whether others' knowledge states accord or conflict with their own experience, and that this information shifts how they interrogate their memory and what they recall.     

Effect of individual thinking styles on item selection during study time allocation

The influence of individual differences on learners' study time allocation has been emphasised in recent studies; however, little is known about the role of individual thinking styles (analytical versus intuitive). In the present study, we explored the influence of individual thinking styles on learners' application of agenda‐based and habitual processes when selecting the first item during a study‐time allocation task. A 3‐item cognitive reflection test (CRT) was used to determine individuals' degree of cognitive reliance on intuitive versus analytical cognitive processing. Significant correlations between CRT scores and the choices of first item selection were observed in both Experiment 1a (study time was 5 seconds per triplet) and Experiment 1b (study time was 20 seconds per triplet). Furthermore, analytical decision makers constructed a value‐based agenda (prioritised high‐reward items), whereas intuitive decision makers relied more upon habitual responding (selected items from the leftmost of the array). The findings of Experiment 1a were replicated in Experiment 2 notwithstanding ruling out the possible effects from individual intelligence and working memory capacity. Overall, the individual thinking style plays an important role on learners' study time allocation and the predictive ability of CRT is reliable in learners' item selection strategy.     

The Development and Testing of a New Version of the Cognitive Reflection Test Applying Item Response Theory (IRT)

The cognitive reflection test (CRT) is a short measure of a person's ability to resist intuitive response tendencies and to produce a normatively correct response, which is based on effortful reasoning. Although the CRT is a very popular measure, its psychometric properties have not been extensively investigated. A major limitation of the CRT is the difficulty of the items, which can lead to floor effects in populations other than highly educated adults. The present study aimed at investigating the psychometric properties of the CRT applying item response theory analyses (a two‐parameter logistic model) and at developing a new version of the scale (the CRT‐long), which is appropriate for participants with both lower and higher levels of cognitive reflection. The results demonstrated the good psychometric properties of the original, as well as the new scale. The validity of the new scale was also assessed by measuring correlations with various indicators of intelligence, numeracy, reasoning and decision‐making skills, and thinking dispositions. Moreover, we present evidence for the suitability of the new scale to be used with developmental samples. Finally, by comparing the performance of adolescents and young adults on the CRT and CRT‐long, we report the first investigation into the development of cognitive reflection. Copyright © 2015 John Wiley & Sons, Ltd.     

Local context effects during emotional item directed forgetting in younger and older adults

This paper explored the differential sensitivity young and older adults exhibit to the local context of items entering memory. We examined trial-to-trial performance during an item directed forgetting task for positive, negative, and neutral (or baseline) words each cued as either to-be-remembered (TBR) or to-be-forgotten (TBF). This allowed us to focus on how variations in emotional valence (independent of arousal) and instruction (TBR vs. TBF) of the previous item (trial n-1) impacted memory for the current item (trial n) during encoding. Different from research showing impairing effects of emotional arousal, both age groups showed a memorial boost for stimuli when preceded by items high in positive or negative valence relative to those preceded by neutral items. This advantage was particularly prominent for neutral trial n items that followed emotional items suggesting that, regardless of age, neutral memories may be strengthened by a local context that is high in valence. A trending age difference also emerged with older adults showing greater sensitivity when encoding instructions changed between trial n-1 and n. Results are discussed in light of age-related theories of cognitive and emotional processing, highlighting the need to consider the dynamic, moment-to-moment fluctuations of these systems.     

CTTITEM: SAS macro and SPSS syntax for classical item analysis

This article describes the functions of a SAS macro and an SPSS syntax that produce common statistics for conventional item analysis including Cronbach’s alpha, item difficulty index (p-value or item mean), and item discrimination indices (D-index, point biserial and biserial correlations for dichotomous items and item-total correlation for polytomous items). These programs represent an improvement over the existing SAS and SPSS item analysis routines in terms of completeness and user-friendliness. To promote routine evaluations of item qualities in instrument development of any scale, the programs are available at no charge for interested users. The program codes along with a brief user’s manual that contains instructions and examples are downloadable from suen.ed.psu.edu/~pwlei/plei.htm.     

Where’s the difficulty in standardized reading tests: The passage or the question?

The purpose of this investigation was to examine the extent to which item and text characteristics predict item difficulty on the comprehension portion of the Gates-MacGinitie Reading Tests for the 7th–9th and 10th–12th grade levels. Detailed item-based analyses were performed on 192 comprehension questions on the basis of the cognitive processing model framework proposed by Embretson and colleagues (Embretson & Wetzel, 1987). Item difficulty was analyzed in terms of various passage features (e.g., word frequency and number of propositions) and individual-question characteristics (e.g., abstractness and degree of inferential processing), using hierarchical linear modeling. The results indicated that the difficulty of the items in the test for the 7th–9th grade level is primarily influenced by text features—in particular, vocabulary difficulty—whereas the difficulty of the items in the test for the 10th–12th grade level is less systematically influenced by text features.     

Concept learning set‐size functions for Clark's nutcrackers

Same/Different abstract‐concept learning by Clark's nutcrackers (Nucifraga columbiana) was tested with novel stimuli following learning of training set expansion (8, 16, 32, 64, 128, 256, 512, and 1024 picture items). The resulting set‐size function was compared to those from rhesus monkeys (Macaca mulatta), capuchin monkeys (Cebus apella), and pigeons (Columba livia). Nutcrackers showed partial concept learning following initial eight‐item set learning, unlike the other species (Magnotti, Katz, Wright, & Kelly, 2015). The mean function for the nutcrackers' novel‐stimulus transfer increased linearly as a function of the logarithm of training set size, which intersected its baseline function at the 128‐item set size. Thus, nutcrackers on average achieved full concept learning (i.e., transfer statistically equivalent to baseline performance) somewhere between set sizes of 64 to 128 items, similar to full concept learning by monkeys. Pigeons required a somewhat larger training set (256 items) for full concept learning, but results from other experiments (initial training and transfer with 32‐ and 64‐item set sizes) suggested carryover effects with smaller set sizes may have artificially prolonged the pigeon's full concept learning. We find it remarkable that these diverse species with very different neural architectures can fully learn this same/different abstract concept, and (at least under some conditions) do so with roughly similar sets sizes (64‐128 items) and numbers of training exemplars, despite initial concept learning advantages (nutcrackers), learning disadvantages (pigeons), or increasing baselines (monkeys).     

Combining CAT with cognitive diagnosis: A weighted item selection approach

Computerized adaptive testing (CAT) was originally proposed to measure θ, usually a latent trait, with greater precision by sequentially selecting items according to the student’s responses to previously administered items. Although the application of CAT is promising for many educational testing programs, most of the current CAT systems were not designed to provide diagnostic information. This article discusses item selection strategies specifically tailored for cognitive diagnostic tests. Our goal is to identify an effective item selection algorithm that not only estimates θ efficiently, but also classifies the student’s knowledge status α accurately. A single-stage item selection method with a dual purpose will be introduced. The main idea is to treat diagnostic criteria as constraints: Using the maximum priority index method to meet these constraints, the CAT system is able to generate cognitive diagnostic feedback in a fairly straightforward fashion. Different priority functions are proposed. Some of them are based on certain information measures, such as Kullback–Leibler information, and others utilize only the information provided by the Q-matrix. An extensive simulation study is conducted, and the results indicate that the information-based method not only yields higher classification rates for cognitive diagnosis, but also achieves more accurate θ estimation. Other constraint controls, such as item exposure rates, are also considered for all the competing methods.     

Source memory that encoding was self-referential: the influence of stimulus characteristics

Decades of research suggest that encoding information with respect to the self improves memory (self-reference effect, SRE) for items (item SRE). The current study focused on how processing information in reference to the self affects source memory for whether an item was self-referentially processed (a source SRE). Participants self-referentially or non-self-referentially encoded words (Experiment 1) or pictures (Experiment 2) that varied in valence (positive, negative, neutral). Relative to non-self-referential processing, self-referential processing enhanced item recognition for all stimulus types (an item SRE), but it only enhanced source memory for positive words (a source SRE). In fact, source memory for negative and neutral pictures was worse for items processed self-referentially than non-self-referentially. Together, the results suggest that item SRE and source SRE (e.g., remembering an item was encoded self-referentially) are not necessarily the same across stimulus types (e.g., words, pictures; positive, negative). While an item SRE may depend on the overall likelihood the item generates any association, the enhancing effects of self-referential processing on source memory for self-referential encoding may depend on how embedded a stimulus becomes in one’s self-schema, and that depends, in part, on the stimulus’ valence and format. Self-relevance ratings during encoding provide converging evidence for this interpretation.     

Using a Difficulty‐Anchored Rating Scale in Performing Angoff Ratings

While the Angoff (1971) is a commonly used cut score method, critics ( Berk, 1996; Impara & Plake, 1997 ) argue the Angoff places too‐high cognitive demands on raters. In response to criticisms of the Angoff, a number of modifications to the method have been proposed. Some suggested Angoff modifications include using an iterative rating process, presenting judges with normative data about item performance, revising the rating judgment into a Yes/No decision, assigning relative weights to dimensions within a test, and using item response theory in setting cut scores. In this study, subject matter expert raters were provided with a ‘difficulty anchored’ rating scale to use while making Angoff ratings; this scale can be viewed as a variation of the Angoff normative data modification. The rating scale presented test items having known p‐values as anchors, and served as a simple means of providing normative information to guide the Angoff rating process. Results are discussed regarding reliability of the mean Angoff rating (.73) and the correlation of mean Angoff ratings with item difficulty (observed r ranges from .65 to .73).     

Development and Validation of a Deontic Justice Scale

This paper describes the development and validation of a deontic justice scale (DJS). Study 1 (n = 124) was conducted to test the initial 36‐item version of the scale. It resulted in the reduction of the initial scale to 18 items, including 3 dimensions: moral obligation (8 items), moral accountability (6 items), and moral outrage (4 items). Study 2 (n = 101) was conducted to examine the construct validity and confirm the factor structure of the DJS. Findings from both studies showed evidence of the scale's construct validity. They also showed that deontic justice is a multidimensional construct encompassing moral obligation, moral accountability, and moral outrage. The scale's implications for use as an adequate research instrument are discussed.     

The curiosity and exploration inventory-II: Development, factor structure, and psychometrics

Given curiosity’s fundamental role in motivation, learning, and well-being, we sought to refine the measurement of trait curiosity with an improved version of the Curiosity and Exploration Inventory (CEI; [Kashdan, T. B., Rose, P., & Fincham, F. D. (2004). Curiosity and exploration: Facilitating positive subjective experiences and personal growth opportunities. Journal of Personality Assessment, 82, 291–305]). A preliminary pool of 36 items was administered to 311 undergraduate students, who also completed measures of emotion, emotion regulation, personality, and well-being. Factor analyses indicated a two-factor model—motivation to seek out knowledge and new experiences (Stretching; five items) and a willingness to embrace the novel, uncertain, and unpredictable nature of everyday life (Embracing; five items). In two additional samples (ns = 150 and 119), we cross-validated this factor structure and provided initial evidence for construct validity. This includes positive correlations with personal growth, openness to experience, autonomy, purpose in life, self-acceptance, psychological flexibility, positive affect, and positive social relations, among others. Applying item response theory (IRT) to these samples (n = 578), we showed that the items have good discrimination and a desirable breadth of difficulty. The item information functions and test information function were centered near zero, indicating that the scale assesses the mid-range of the latent curiosity trait most reliably. The findings thus far provide good evidence for the psychometric properties of the 10-item CEI-II.     

Simple mental addition in children with and without mild mental retardation.

The speeded performance on simple mental addition problems of 6- and 7-year-old children with and without mild mental retardation is modeled from a person perspective and an item perspective. On the person side, it was found that a single cognitive dimension spanned the performance differences between the two ability groups. However, a discontinuity, or "jump," was observed in the performance of the normal ability group on the easier items. On the item side, the addition problems were almost perfectly ordered in difficulty according to their problem size. Differences in difficulty were explained by factors related to the difficulty of executing nonretrieval strategies. All findings were interpreted within the framework of Siegler's (e.g., R. S. Siegler & C. Shipley, 1995) model of children's strategy choices in arithmetic. Models from item response theory were used to test the hypotheses.     

Motor assessment of preschool aged children: A preliminary investigation of the validity of the Bruininks–Oseretsky test of motor proficiency – Short form

The aim of this study was to examine the validity of the Bruininks–Oseretsky test of motor proficiency – short form (BOTMP-SF) [Bruininks, R. (1978). Bruininks–Oseretsky test of motor proficiency: Examiners manual. Circle Pines, MN] for the assessment of preschool aged children. Three-hundred and eighteen children 48–71 months old (M = 58.97 months, SD = 6.73) participated in the study. For the data analysis both an ANOVA and a MANOVA were applied with the total battery score and the 14 item scores being the dependent variables, respectively. Age was found to have a significant effect on both children’s total battery score (F(3, 314) = 110.65, p < .001, η² = .68) and 13 item scores (minimum F(3, 314) = 8.75, p < .001, minimum η² = .145). Although the aforementioned results represent an evidence for the validity of the BOTMP-SF, a closer study on the score of each item revealed a high percentage of zero scores on four items. Task difficulty has caused a floor effect, constituting a threat to the validity of the preschoolers’ motor assessment with the above battery. A modification of the battery items is suggested, so that the BOTMP-SF will give valid results for children 4–6 years old.     

粗糙集在项目认知属性标定中的应用

认知诊断是新一代测量理论的核心, 对形成性教学评估具有重要意义。项目认知属性标定是认知诊断中一项基础而重要的工作,现有的项目认知属性辅助标定方法的研究工作很少, 并且在应用上存在诸多局限。课堂评估是认知诊断应用的理想场所,但课堂评估中项目的选取具有随意性, 教师难以在短时间内准确标识项目认知属性。本研究首次提出采用粗糙集方法对项目认知属性进行标定, 该方法无需太多被试和项目, 亦无需已知项目参数, 且能当场诊断出结果, 适于采用纸笔测验的课堂评估。通过Monte Carlo模拟研究表明：采用粗糙集方法能迅速地对项目认知属性进行标定, 并具有较高的标定准确率; 而且, 项目认知属性越少、或被试估计判准率越高、或失误率越小则项目认知属性标定的准确率越高。粗糙集方法的引入, 对拓展认知诊断的应用范围, 真正实现其辅助性教学功能, 具有重要作用。     

Rasch analysis and item reduction of the hypomanic personality scale

The aim of the current study was to reduce the number of items in the 48-item hypomanic personality scale (HPS) and determine whether a unidimensional scale of the hypomanic trait could be derived. Previously collected HPS data from University students (n = 318) were applied to the Rasch model (one-parameter item response theory). Overall scale and individual item fit statistics were used to judge fit to the model and item maps employed to determine coverage of the trait. Cronbach’s Alpha and correlations with other questionnaires pre- and post-item reduction were evaluated. Rasch analysis indicated that the original HPS was not unidimensional, had significant redundancy and differential item functioning by age and gender. An iterative process of item reduction produced a 20-item HPS (HPS-20) that retained the concepts of the original HPS and had excellent fit to the Rasch model (χ² p = 0.27). Unidimensionality of the HPS-20 was confirmed. The traditional psychometric properties of the HPS-20 and coverage of the underlying hypomanic construct were similar to the original. It was possible to derive a unidimensional measure of the hypomanic trait. Further use of the HPS-20 is encouraged as it may increase understanding of the risk factors for affective disorders.     

Optimal Items for Assessing Parental Involvement across Different Groups During Middle Childhood

Various definitions and different approaches for assessing the complex construct of parental involvement (PI) have led to inconsistent findings regarding the impact of PI on child development. To date, limited information is available regarding the measurement invariance of PI measures across time and groups (e.g., children’s gender, ethnicity, and socio-economic status), leaving a concern that group differences in PI might reflect item bias instead of true differences in PI. The present study aimed to obtain a set of optimal items for measuring PI from kindergarten through the elementary school years and investigate whether they could be used for parents from different groups. A Rasch measurement model was implemented to investigate item difficulty, step calibrations, and measurement invariance (differential item functioning; DIF, here). The results from the Early Childhood Longitudinal Study, Kindergarten Class of 1998–1999 data set showed that 20 items can be used to measure three dimensions of PI—namely school/home involvement, family educational investment, and family routines—across four time points. Administrative time, children’s gender, ethnicity, and social economic status showed different levels of effect on item difficulty for half of these items. Practitioners and researchers should be cautious when using these items and are suggested to freely estimate the item parameters of DIF items as well as add more items to the PI scale to improve reliability.     

In Defense of a Disputed Study of Construct Validity from South Africa

This analysis of a critique finds that the original study accurately showed that the items found easy or difficult by Black South African undergraduates were those found easy or difficult by their White and South Asian counterparts (r's=.90). There was no evidence of any culture‐specific effect. Instead, African/non‐African differences were found to be most pronounced on g. This was shown by item‐total correlations (estimates of the item's g loading), which predicted the magnitude of African/non‐African differences on those same items, and by a confirmatory factor analysis. The tests were equally predictive for Blacks and non‐Blacks on external criteria such as course grades. The results indicate the remarkable cross‐cultural generalizability of item properties across sub‐Saharan Africans, South Asians, and Europeans and that these reflect g more than culturally specific ways of thinking.     

Latent Trait Modeling with the Common beliefs Survey III: Using Item Response Theory to Evaluate an Irrational Beliefs Inventory

Assessment of irrational beliefs by such measures as the Common Beliefs Survey III (CBS) has traditionally relied upon classical test theory assumptions, in which the properties of specific test items are less important than the total test score as the aggregate of all item responses. An alternative approach using item response theory (IRT) methodology allows one to specify the parameters of difficulty and discrimination for each test item. Difficulty levels of CBS items range along a continuum of irrationality, the implied latent trait measured by responses to the questionnaire as a whole. We evaluated the CBS responses of 605 individuals from clinical and college settings, drawing from current and archival data. The original Likert scale ratings were recoded into dichotomous scores. Fourteen of the 54 items were highly or very highly discriminating in distinguishing respondents with high and low irrationality levels. However, discriminating items exhibited a very narrow range of difficulty; most functioned at a point a little above the halfway mark on the continuum of irrationality. Item characteristic curves and test information curves were very similar for female (n = 424) and male (n = 179) respondents. We derived a 4-item screening test for irrationality from our IRT analyses of the 54 CBS items. Further test development, focused on the selection and scaling of items with a much broader range of difficulty, would facilitate evaluation of the hierarchical structure of irrational beliefs. Portions of this paper were presented at the 39th Annual Convention of the Association for Behavioral and Cognitive Therapies, Washington, DC, November, 2005.