The Usual and the Unusual: Solving Remote Associates Test Tasks Using Simple Statistical Natural Language Processing Based on Language Use

In this study we show how complex creative relations can arise from fairly frequent semantic relations observed in everyday language. By doing this, we reflect on some key cognitive aspects of linguistic and general creativity. In our experimentation, we automated the process of solving a battery of Remote Associates Test tasks. By applying Statistical Natural Language Processing techniques to a large web‐based corpus, we perform a frequency and collocation analysis of the test items. Results show that 37% of the 68 tasks were automatically solved, whereas human accuracy reached 43%. Our method outperformed humans in the tasks rated as difficult: 38% and 32%, respectively. Highly relevant is that the novel and adequate relations established in order to solve the RAT were not previously present in the corpus. The frequency based approach pervades all stages of our method: during the divergent stage, highly frequent collocations are listed, while the convergent stage starts by matching each task's triads output, shrinking that list, and finalizing it by choosing the least frequent, therefore more informative and often correct, result. Finally, we discuss the implications of our model in overcoming functional fixedness and understanding cognitive salience in the creative process.     

创造性研究的有效工具——远距离联想测验（RAT）

介绍了一个测量创造力的方法——远距离联想测验（remoteassociatestest,RAT）,并将它与其他创造性测验方法进行比较。文章还介绍了应用RAT进行的一些创造性科学研究的成果。作者认为RAT是适合创造性科学研究,尤其是神经科学研究的重要工具     

Validation of the Italian Remote Associate Test

The scientific approach to the study of creative problem-solving has shifted from using classic insight problems (e.g., the Nine-dots problem), toward sets of problems that have more robust psychometric properties, such as the Remote Associate Test (RAT). Because it is homogeneous, compact, quickly solvable, and easy to score, the RAT has been used more frequently in recent creativity studies. We applied the Item Response Theory (IRT) to develop an Italian version of this task. The final 51-item test was reliable (α = .89) and provided information over a wide range of ability levels, as revealed by the IRT analysis. The RAT correlated with five measures of creative performance: The Raven's Standard Progressive Matrices (SPM), three classic insight problems, a set of anagrams purposefully developed, the fluency and flexibility scores of the Alternative Uses Task (AUT), and the Creative Achievements Questionnaire (CAQ). The new measure provided is meant to encourage the study of creativity and problem-solving in the Italian language.     

Comparison of Scoring Criteria for Clock Drawing Test

This study compared clock drawings by 42 medically hospitalized patients with a mean age of 51.9 (SD = 10.1) years, using four sets of published scoring criteria to determine comparability of classification and to assess validity by comparison to other measures of cognitive functioning. We found impairment in 20 of 42 cases using the criteria of Mendez et al. (1992); 11 of 42 cases by Sunderland et al. (1989); 9 of 42 cases with Freedman et al. (1994); and 8 of 42 cases according to Rouleau et al. (1992). Kappa coefficients of impairment status between sets of scoring criteria ranged from .41 to .86. Pearson correlations of raw scores between schemes ranged from .72 to .94. All except Sunderland et al. were significantly correlated with the Standardized Mini-Mental State Examination. All correlated significantly with the Wechsler Adult Intelligence Scale–Revised Block Design; however, only Mendez et al. correlated significantly with the Neurobehavioral Cognitive Status Examination. On the basis of these results and our experience, we recommend using the Freedman scoring scheme.     

Cross-sectional and longitudinal confirmatory factor models for the german test anxiety inventory: A construct validation

Construct validity of the German Test Anxiety Inventory (TAI-G) was tested in two respects. Firstly, the purported four-dimensional structure of the TAI-G (comprising subscales Emotionality, Worry, Interference, and Lack of Confidence) as well as relations of the test anxiety dimensions to self-efficacy were tested. Secondly, the trait conception of the TAI-G was tested within the framework of Latent State-Trait theory. The TAI-G was given to a student sample (N=302) on three occasions with a time interval of 2 weeks along with a study-specific self-efficacy scale on occasion 1. Dimensionality assumptions as well as relations with self-efficacy were tested using cross-sectional second-order confirmatory factor analysis. The trait conception was tested separately for TAI-G subscales by specifying longitudinal confirmatory factor models (Latent State-Trait models) and by calculating variance proportions of manifest variables (Latent State-Trait coefficients) referring to different sources of systematic variance (person, situation, and method) based on parameter estimates of the models. Results were supportive of both the purported four-dimensional structure and hypothesized relationships to self-efficacy (i.e., acceptable model fit) as well as of the trait conception of test anxiety (i.e., acceptable model fit and high proportion of variance due to person component). Implications for further validation studies were discussed.     

Initial Review of the Psychometric Properties of a Computerised Career Preference Test for Career Guidance Assessment

Psychometric proprties of the Career Preference Computerised Adaptive Test (CPCAT) (De Beer & Marais, 2010; De Beer, Marais, Maree, & Skrzypczak, 2008) are reported. Participants were high school students (n=343; males=279, females=164)at Grade 9 and Grade 11 level from a South African school district. Reliability and construct validity indices suggest the CPCAT could be of utility in the career counseling of high school students.     

Investigation of Psychometric Properties of the Test for Creative Thinking—Drawing Production: Evidence from Study in Latvia

The Test for Creative Thinking—Drawing Production (TCT‐DP) is designed as an effective drawing‐based instrument for measuring creative potential. Many studies report adaptation efforts in other cultures pointing out good psychometric properties of the instrument nonetheless revealing also some trouble spots. The present study includes adaptation of TCT‐DP in Latvia and investigation of psychometric properties of the instrument such as measurement invariance between forms, sequence effect, gender differences, and factor structure of criteria employing methodology of structural equation modeling. Two samples were involved in the study—9th‐grade students (n = 300) and 15‐year‐old 9th‐grade students (n = 200). Results indicate that trained judges are able to achieve high reliability in evaluation of TCT‐DP total score and all criteria if some criteria are divided into subcategories. It was also found that TCT‐DP has measurement invariance between both forms but has small effect sizes regarding gender differences and method sequence. Observed differences of TCT‐DP total scores between the Latvian sample and relevant samples from Germany and Hong Kong could be considered as trivial. The study also revealed that, following original instructions, some test criteria had strong interdependence and therefore strategies in the evaluation process reducing interdependencies between criteria should be considered in future studies on the structure of TCT‐DP.     

The Machines Take Over: A Comparison of Various Supervised Learning Approaches for Automated Scoring of Divergent Thinking Tasks

Traditionally, researchers employ human raters for scoring responses to creative thinking tasks. Apart from the associated costs this approach entails two potential risks. First, human raters can be subjective in their scoring behavior (inter-rater-variance). Second, individual raters are prone to inconsistent scoring patterns (intra-rater-variance). In light of these issues, we present an approach for automated scoring of Divergent Thinking (DT) Tasks. We implemented a pipeline aiming to generate accurate rating predictions for DT responses using text mining and machine learning methods. Based on two existing data sets from two different laboratories, we constructed several prediction models incorporating features representing meta information of the response or features engineered from the response’s word embeddings that were obtained using pre-trained GloVe and Word2Vec word vector spaces. Out of these features, word embeddings and features derived from them proved to be particularly effective. Overall, longer responses tended to achieve higher ratings as well as responses that were semantically distant from the stimulus object. In our comparison of three state-of-the-art machine learning algorithms, Random Forest and XGBoost tended to slightly outperform the Support Vector Regression.     

The dark side of Facebook: Semantic representations of status updates predict the Dark Triad of personality

Using Latent Semantic Analysis, we quantified the semantic representations of Facebook status updates of 304 individuals in order to predict self-reported personality. We focused on, besides Neuroticism and Extraversion, the Dark Triad of personality: Psychopathy, Narcissism, and Machiavellianism. The semantic content of Facebook updates predicted Psychopathy and Narcissism. These updates had a more “odd” and negatively valanced content. Furthermore, Neuroticism, number of Facebook friends, and frequency of status updates were predictable from the status updates. Given that Facebook allows individuals to have major control in how they present themselves and draw benefits from these interactions, we conclude that the Dark Triad, involving socially malevolent behavior such as self-promotion, emotional coldness, duplicity, and aggressiveness, is manifested in Facebook status updates.     

The measurement of cognitive impulsiveness: Psychometric properties of two automated adult versions of the Matching Familiar Figures Test

An automated version of the Matching Familiar Figures Test (MFFT) was administered to undergraduates, along with a parallel from. The latency-errors correlation (–0.61) was higher than that reported for most studies in children and weakly supports the view that the correlation increases with age. Repeated exposure resulted in improved performance, which was faster, more accurate, and more efficient, but there was no effect on impulsiveness. Reliability and internal consistency of both forms were acceptably high and the forms were comparable. Use of the univariate measures (impulsiveness-reflectiveness and efficiency-inefficiency) is superior to other scoring methods.Supported in part by a grant from the Nuffield Foundation, England.     

Power Analysis for the Likelihood-Ratio Test in Latent Markov Models: Shortcutting the Bootstrap p-Value-Based Method

The latent Markov (LM) model is a popular method for identifying distinct unobserved states and transitions between these states over time in longitudinally observed responses. The bootstrap likelihood-ratio (BLR) test yields the most rigorous test for determining the number of latent states, yet little is known about power analysis for this test. Power could be computed as the proportion of the bootstrap p values (PBP) for which the null hypothesis is rejected. This requires performing the full bootstrap procedure for a large number of samples generated from the model under the alternative hypothesis, which is computationally infeasible in most situations. This article presents a computationally feasible shortcut method for power computation for the BLR test. The shortcut method involves the following simple steps: (1) obtaining the parameters of the model under the null hypothesis, (2) constructing the empirical distributions of the likelihood ratio under the null and alternative hypotheses via Monte Carlo simulations, and (3) using these empirical distributions to compute the power. We evaluate the performance of the shortcut method by comparing it to the PBP method and, moreover, show how the shortcut method can be used for sample-size determination.     

Factor Structure of the Torrance Tests of Creative Thinking Verbal Form B in a Spanish‐speaking Population

The objective of this study was to compare, through a Confirmatory Factor Analysis, two different theoretical models that explain the operationalized creativity construct with the Verbal Torrance Tests of Creative Thinking (TTCT), Form B. Model 1 is represented by six factors which correspond to each activity and its respective indicators while Model 2 is integrated by three factors which correspond to each TTCT ability (i.e., Fluency, Originality, and Flexibility) and the corresponding indicators for each variable. The study was carried out with a sample consisting of 432 Spanish‐speaking youngsters of both sexes aged 15–26. According to the research findings, the model which showed the most satisfactory fit identifies six correlated factors that correspond to each of the activities proposed (χ² = 414.48; df = 116; χ²/df = 3.57; GFI = .90; NFI = .95; CFI = .96 and RMSEA = .077). These results are discussed according to its psychometric implications for the construct assessment in different fields.     

The roles of the temporal lobe in creative insight: an integrated review

Recent studies have revealed that the temporal lobe, a cortical region thought to be in charge of episodic and semantic memory, is involved in creative insight. This work examines the contributions of discrete temporal regions to insight. Activity in the medial temporal regions is indicative of novelty recognition and detection, which is necessary for the formation of novel associations and the “Aha!” experience. The fusiform gyrus mainly affects the formation of gestalt-like representation and perspective taking. The anterior and posterior middle temporal gyri (MTG) are individually associated with extensive semantic processing and inhibiting salient or routine word associations, which are necessary to form non-salient, novel and remote associations. The anterior and posterior superior temporal gyri (STG) are individually responsible for integrating/binding and accessing various types of available conceptual representations. Based on the current knowledge, an integrated model of the temporal lobe's role in insight and some future directions are proposed.     

Assessing miserly information processing: An expansion of the Cognitive Reflection Test

The Cognitive Reflection Test (CRT; Frederick, 2005) is designed to measure the tendency to override a prepotent response alternative that is incorrect and to engage in further reflection that leads to the correct response. It is a prime measure of the miserly information processing posited by most dual process theories. The original three-item test may be becoming known to potential participants, however. We examined a four-item version that could serve as a substitute for the original. Our data show that it displays a .58 correlation with the original version and that it has very similar relationships with cognitive ability, various thinking dispositions, and with several other rational thinking tasks. Combining the two versions into a seven-item test resulted in a measure of miserly processing with substantial reliability (.72). The seven-item version was a strong independent predictor of performance on rational thinking tasks after the variance accounted for by cognitive ability and thinking dispositions had been partialled out.     

Further Psychometric Evaluation of the Japanese Version of an Academic Self-Concept Scale

In the present study, the author replicated earlier research (Paik & Michael, 1999) seeking additional information on the reliability and construct validity of a Japanese academic self-concept scale, a 70-item questionnaire comprising S subscales (Aspiration, Anxiety, Academic Interest and Satisfaction, Leadership and Initiative, and Identification vs. Alienation). A sample of 1% Japanese high school students completed the scale. Internal consistency reliability for the S subscales ranged from .75 to .87. Confirmatory factor analyses performed on several alternative models showed that the a priori 5-factor model fit the observed data best—a finding consistent with the previous study. Results of Z tests revealed statistically significant score differences between genders and between high and low academic achievers.     

Evidence for an illusion of causality when using the Implicit Association Test to measure learning

Our ability to detect causal relations and patterns of covariation is easily biased by a number of well-known factors. For example, people tend to overestimate the strength of the relation between a cue and an outcome if the outcome tends to occur very frequently. During the last years, several accounts have attempted to explain the outcome-density bias. On the one hand, dual-process performance accounts propose that biases are not due to the way associations are encoded, but to the higher-order cognitive processes involved in the retrieval and use of this information. In other words, the outcome-density bias is seen as a performance effect, not a learning effect. From this point of view, it is predicted that the outcome-density bias should be absent in any testing procedure that reduces the motivation or opportunity to engage in higher-order cognitive processes. Contrary to this prediction, but consistent with the most common single-process learning accounts, our results show that the outcome-density effect can be detected when the Implicit Association Test is used to measure the strength of cue–outcome associations.     

A Confirmatory Factor-Analytic Evaluation of the Purpose in Life Test: Preliminary Psychometric Support for a Replicable Two-Factor Model

This study’s purpose was to use confirmatory factor analysis to compare published factor-analytic models of the 20-item Purpose in Life test (PIL) to identify the one that provides the best fit to the data. To date many different models have been described, with limited evidence to support whether they are replicable. This study utilized data from undergraduates (N = 620) from a medium-sized university located in the southern United States. Ten different PIL models were tested, with support found for the two-factor model (exciting life, purposeful life) of Morgan and Farsides. Recommendations and implications for research are provided.     

The Influence of Rater Effects in Training Sets on the Psychometric Quality of Automated Scoring for Writing Assessments

Automated essay scoring engines (AESEs) are becoming increasingly popular as an efficient method for performance assessments in writing, including many language assessments that are used worldwide. Before they can be used operationally, AESEs must be “trained” using machine-learning techniques that incorporate human ratings. However, the quality of the human ratings used to train the AESEs is rarely examined. As a result, the impact of various rater effects (e.g., severity and centrality) on the quality of AESE-assigned scores is not known. In this study, we use data from a large-scale rater-mediated writing assessment to examine the impact of rater effects on the quality of AESE-assigned scores. Overall, the results suggest that if rater effects are present in the ratings used to train an AESE, the AESE scores may replicate these effects. Implications are discussed in terms of research and practice related to automated scoring.