Psychometric and Validity Issues in Machine Learning Approaches to Personality Assessment: A Focus on Social Media Text Mining

In the age of big data, substantial research is now moving toward using digital footprints like social media text data to assess personality. Nevertheless, there are concerns and questions regarding the psychometric and validity evidence of such approaches. We seek to address this issue by focusing on social media text data and (i) conducting a review of psychometric validation efforts in social media text mining (SMTM) for personality assessment and discussing additional work that needs to be done; (ii) considering additional validity issues from the standpoint of reference (i.e. ‘ground truth’) and causality (i.e. how personality determines variations in scores derived from SMTM); and (iii) discussing the unique issues of generalizability when validating SMTM for personality assessment across different social media platforms and populations. In doing so, we explicate the key validity and validation issues that need to be considered as a field to advance SMTM for personality assessment, and, more generally, machine learning personality assessment methods. © 2020 European Association of Personality Psychology     

To Challenge the Morning Lark and the Night Owl: Using Smartphone Sensing Data to Investigate Day–Night Behaviour Patterns

For decades, day–night patterns in behaviour have been investigated by asking people about their sleep–wake timing, their diurnal activity patterns, and their sleep duration. We demonstrate that the increasing digitalization of lifestyle offers new possibilities for research to investigate day–night patterns and related traits with the help of behavioural data. Using smartphone sensing, we collected in vivo data from 597 participants across several weeks and extracted behavioural day–night pattern indicators. Using this data, we explored three popular research topics. First, we focused on individual differences in day–night patterns by investigating whether ‘morning larks’ and ‘night owls’ manifest in smartphone-sensed behavioural indicators. Second, we examined whether personality traits are related to day–night patterns. Finally, exploring social jetlag, we investigated whether traits and work weekly day–night behaviours influence day–night patterns on weekends. Our findings highlight that behavioural data play an essential role in understanding daily routines and their relations to personality traits. We discuss how psychological research can integrate new behavioural approaches to study personality.     

Applying theory to the black box: A model for empirically scoring biodata

Despite being an effective predictor of job performance, empirically keyed biodata assessments have been criticized as black box empiricism unlikely to generalize to new contexts. This paper introduces a model that challenges this perspective, explicating how biodata content, job demands, and criterion variables collectively influence the construct validity, and generalizability of empirically scored biodata. Across two field studies, expected changes in scale correlations with external measures were found that coincided with changes in the contextual similarity between calibration and holdout contexts, the criteria used, and the content validity of biodata items. Collectively, this paper offers a framework that helps understand and optimize empirical biodata keying in practice, furthering confidence for their use in applied settings.     

How representations of number and numeracy predict decision paradoxes: A fuzzy-trace theory approach

Higher numeracy has been associated with decision biases in some numerical judgment-and-decision problems. According to fuzzy-trace theory, understanding such paradoxes involves broadening the concept of numeracy to include processing the gist of numbers—their categorical and ordinal relations—in addition to objective (verbatim) knowledge about numbers. We assess multiple representations of gist, as well as numeracy, and use them to better understand and predict systematic paradoxes in judgment and decision-making. In two samples (Ns = 978 and 957), we assessed categorical (some vs. none) and ordinal gist representations of numbers (higher vs. lower, as in relative magnitude judgment, estimation, approximation, and simple ratio comparison), objective numeracy, and a nonverbal, nonnumeric measure of fluid intelligence in predicting: (a) decision preferences exhibiting the Allais paradox and (b) attractiveness ratings of bets with and without a small loss in which the loss bet is rated higher than the objectively superior no-loss bet. Categorical and ordinal gist tasks predicted unique variance in paradoxical decisions and judgments, beyond objective numeracy and intelligence. Whereas objective numeracy predicted choosing or rating according to literal numerical superiority, appreciating the categorical and ordinal gist of numbers was pivotal in predicting paradoxes. These results bring important paradoxes under the same explanatory umbrella, which assumes three types of representations of numbers—categorical gist, ordinal gist, and objective (verbatim)—that vary in their strength across individuals.     

Maximizing data quality and shortening survey time: Three-form planned missing data survey design

Simulation studies have shown the three-form planned missing data design efficiently collects high quality data while reducing participant burden. This methodology is rarely used in sport and exercise psychology. Therefore, we conducted a re-sampling study with existing sport and exercise psychology survey data to test how three-form planned missing data survey design implemented with different item distribution approaches effect constructs’ internal measurement structure and validity. Results supported the efficacy of the three-form planned missing data survey design for cross-sectional data collection. Sample sizes of at least 300 (i.e., 100 per form) are recommended for having unbiased parameter estimates. It is also recommended items be distributed across survey forms to have representation of each facet of a construct on every form, and that a select few of these items be included across all survey forms. Further guidelines for three-form surveys based upon the results of this resampling study are provided.     

Limited information estimation and testing of discretized multivariate normal structural models

Discretized multivariate normal structural models are often estimated using multistage estimation procedures. The asymptotic properties of parameter estimates, standard errors, and tests of structural restrictions on thresholds and polychoric correlations are well known. It was not clear how to assess the overall discrepancy between the contingency table and the model for these estimators. It is shown that the overall discrepancy can be decomposed into a distributional discrepancy and a structural discrepancy. A test of the overall model specification is proposed, as well as a test of the distributional specification (i.e., discretized multivariate normality). Also, the small sample performance of overall, distributional, and structural tests, as well as of parameter estimates and standard errors is investigated under conditions of correct model specification and also under mild structural and/or distributional misspecification. It is found that relatively small samples are needed for parameter estimates, standard errors, and structural tests. Larger samples are needed for the distributional and overall tests. Furthermore, parameter estimates, standard errors, and structural tests are surprisingly robust to distributional misspecification. This research was supported by the Department of Universities, Research and Information Society (DURSI) of the Catalan Government, and by grants BSO2000-0661 and BSO2003-08507 of the Spanish Ministry of Science and Technology.     

Component Models for Fuzzy Data

The fuzzy perspective in statistical analysis is first illustrated with reference to the “Informational Paradigm” allowing us to deal with different types of uncertainties related to the various informational ingredients (data, model, assumptions). The fuzzy empirical data are then introduced, referring to J LR fuzzy variables as observed on I observation units. Each observation is characterized by its center and its left and right spreads (LR₁ fuzzy number) or by its left and right “centers” and its left and right spreads (LR₂ fuzzy number). Two types of component models for LR₁ and LR₂ fuzzy data are proposed. The estimation of the parameters of these models is based on a Least Squares approach, exploiting an appropriately introduced distance measure for fuzzy data. A simulation study is carried out in order to assess the efficacy of the suggested models as compared with traditional Principal Component Analysis on the centers and with existing methods for fuzzy and interval valued data. An application to real fuzzy data is finally performed. We would like to express our gratitude to the Editor, the Associate Editor, and the Referees whose comments and suggestions improved significantly the quality of the paper.     

Gender bias in estimating political expertise

The present study reveals that there is a gender bias in estimates of spouses' political expertise. Data were collected from married couples in one city in north-eastern Japan and the results showed that when all respondents were included, the estimation of spouses' political expertise was moderately accurate. However, while husbands' estimates of the political expertise of their wives were lowered and less accurate, wives' estimates of the political expertise of their husbands were more accurate. The implication of these findings for the political equality of males and females is discussed.     

Satisfaction et confort au travail. L’apport de la démarche implicative

A large body of research has investigated the psycho-organizational variables predicting satisfaction at work, but few studies have so far examined variables in the field of environmental psychology. Our study aims first to identify the impact of variables evaluating comfort at work (satisfaction with the work environment, attachment to the workplace) by comparing them with organizational variables known to be related to job satisfaction, and then to rank the relationships between these variables in a pseudo- implication sequence, based on statistical implicative analysis. The study involved 105 bank employees and confirms our theoretical model, showing that job satisfaction can be explained implicatively by the three other variables [(Comfort/Functionality => Control/privacy) => Workplace attachment] => job satisfaction.     

计划缺失设计——通过有意缺失让研究更高效

缺失值是社会科学研究中非常普遍的现象。全息极大似然估计和多重插补是目前处理缺失值最有效的方法。计划缺失设计利用特殊的实验设计有意产生缺失值, 再用现代的缺失值处理方法来完成统计分析, 获得无偏的统计结果。计划缺失设计可用于横断面调查减少(或增加)问卷长度和纵向调查减少测量次数, 也可用于提高测量有效性。常用的计划缺失设计有三式设计和两种方法测量。