Some Behaviorial Science Measurement Concerns and Proposals

Primarily from a measurement standpoint, we question some basic beliefs and procedures characterizing the scientific study of human behavior. The relations between observed and unobserved variables are key to an empirical approach to building explanatory theories and we are especially concerned about how the former are used as proxies for the latter. We believe that behavioral science can profitably reconsider the prevailing version of this arrangement because of its vulnerability to limiting idiosyncratic aspects of observed/unobserved variable relations. We describe a general measurement approach that takes into account idiosyncrasies that should be irrelevant to the measurement process but can intrude and may invalidate it in ways that distort and weaken relations among theoretically important variables. To clarify further our major concerns, we briefly describe one version of the measurement approach that fundamentally supports the individual as the primary unit of analysis orientation that we believe should be preeminent in the scientific study of human behavior.     

Empirical evaluation of a model of global psychophysical judgments: IV. Forms for the weighting function

Understanding the psychological interpretation of numerals is of both practical and theoretical interest. In classical magnitude estimation, respondents match numerals to sensations and in magnitude production they select sensations that stand in a prescribed numerical ratio to a given standard. The present work focusses on evaluating several possible, and related, forms for the function W formulating the distortion of numerals. The main form, of which a power function is a special case, is the Prelec exponential/power representation. Behavioral equivalents to power and to Prelec functions are formulated, tested, and rejected. It is argued that either the mathematical form or the assumption W(1)=1 is wrong. Whereas, the axiomatic literature has focussed exclusively on the former inference, we explore the alternate that W(1)≠1. Behavioral axioms are formulated in each case and experimentally tested. We conclude that most respondents satisfy a general power function and that those who do not, satisfy the general Prelec function.     

Invariance in Measurement and Prediction Revisited

Borsboom (Psychometrika, 71:425–440, 2006) noted that recent work on measurement invariance (MI) and predictive invariance (PI) has had little impact on the practice of measurement in psychology. To understand this contention, the definitions of MI and PI are reviewed, followed by results on the consistency between the two forms of invariance in the general case. The special parametric cases of factor analysis (strict factorial invariance) and linear regression analyses (strong regression invariance) are then described, along with findings on the inconsistency between the two forms of invariance in this context. Two numerical examples of inconsistency are reviewed in detail. The impact of violations of MI on accuracy of selection is illustrated. Finally, reasons for the slow dissemination of work on invariance are discussed, and the prospects for altering this situation are weighed. This paper is based on the Presidential Address given at the International Meeting of the Psychometric Society in Tokyo, Japan, on July 11, 2007. This research was supported by National Institute of Mental Health grants 1P30 MH 068685-01A1 and RO1 MH64707-01.     

Learning representations of wordforms with recurrent networks: comment on sibley, kello, plaut, & elman (2008)

Sibley et al. (2008) report a recurrent neural network model designed to learn wordform representations suitable for written and spoken word identification. The authors claim that their sequence encoder network overcomes a key limitation associated with models that code letters by position (e.g., CAT might be coded as C‐in‐position‐1, A‐in‐position‐2, T‐in‐position‐3). The problem with coding letters by position (slot‐coding) is that it is difficult to generalize knowledge across positions; for example, the overlap between CAT and TOMCAT is lost. Although we agree this is a critical problem with many slot‐coding schemes, we question whether the sequence encoder model addresses this limitation, and we highlight another deficiency of the model. We conclude that alternative theories are more promising.     

The Construct Validity of the Utrecht Work Engagement Scale: Multisample and Longitudinal Evidence

This study investigated the factor structure and factorial group and time invariance of the 17-item and 9-item versions of the Utrecht Work Engagement Scale (UWES; Schaufeli et al. (2002b) Journal of Happiness Studies 3:71–92). Furthermore, the study explored the rank-order stability of work engagement. The data were drawn from five different studies (N = 9,404), including a three-year longitudinal study (n = 2,555), utilizing five divergent occupational samples. Confirmatory factor analysis supported the hypothesized correlated three-factor structure—vigor, dedication, absorption—of both UWES scales. However, while the structure of the UWES-17 did not remain the same across the samples and time, the structure of the UWES-9 remained relatively unchanged. Thus, the UWES-9 has good construct validity and use of the 9-item version can be recommended in future research. Moreover, as hypothesized, Structural Equation Modeling showed high rank-order stabilities for the work engagement factors (between 0.82 and 0.86). Accordingly, work engagement seems to be a highly stable indicator of occupational well-being.     

网络测验和纸笔测验的测量不变性研究：以生活满意度量表为例

以生活满意度量表为例,运用实证性因素分析,考察在中国文化下网络测验和传统纸笔测验之间的测量不变性。结果显示,网络测验和纸笔测验之间存在弱不变性,即网络测验和纸笔测验有着相同的测量单位;但网络测验和纸笔测验只存在部分的强不变性和部分的严格不变性,测验实施环境对结果的影响不可忽视。该研究表明,恰当设计的网络测验是可靠的,同时还提示,当一个测验在不同情境下运用时,检验测量不变性十分必要     

What is the temperamental basis of humour like in China? A cross-national examination and validation of the standard version of the state–trait cheerfulness inventory

The State–Trait Cheerfulness Inventory–trait version (STCI-T60) consists of three dimensions of cheerfulness, seriousness, and bad mood integrated to measure the temperamental basis of the sense of humour. The present study replicated the three-dimensional factor structure of the STCI in China using 60 items consistent with other standard trait versions (e.g., English, Chilean-Spanish). Closer examination of associations between traits suggested bad mood showed curvilinear associations with both cheerfulness and seriousness, such that cheerfulness and bad mood were negatively associated for those low and average in trait bad mood but not for those with high trait bad mood. Seriousness was positively associated with bad mood at high levels of trait bad mood, but not at average or low levels of bad mood. Associations between the STCI traits and major personality dimensions, humour styles, and well-being were further examined. Cheerfulness and seriousness showed positive associations with satisfaction with life and emotional well-being (EWB) while bad mood showed a curvilinear association with EWB. Using multi-group confirmatory factor analyses, partial metric invariance was found between English and Chinese versions of the STCI-T60, but structural invariance was not observed. Implications based on the empirical literature in dialecticism and cross-cultural assessment were thoroughly discussed.     

Salutogenic constructs across Pakistan and Germany: A cross sectional study

Background/ObjectiveProtective factors are relevant for mental health in general, however, universality of the instruments has been rarely tested. Therefore, the current study aimed to examine psychometric properties and cross-cultural measurement invariance of salutogenic constructs.MethodData was collected from university students of Pakistan (n = 1,841) and Germany (n = 7,890). Single-group confirmatory analysis (CFA) and multiple-group CFA was tested to examine the proposed factor structure and measurement invariance of Positive Mental Health Scale, Resilience Scale, Perceived Social Support Questionnaire, and Life Satisfaction Scale across student samples from Pakistan and Germany respectively.ResultsWe found strong measurement invariance for the Positive Mental Health Scale, Life Satisfaction Scale, and partial strong measurement for the Resilience Scale, and Perceived Social Support Questionnaire.ConclusionsThe results indicate that these scales could be recommended for the meaningful comparison of latent means across cultures. Understanding these differences would further advance our knowledge about the mechanism underlying positive mental health.     

Validity evidence and measurement equivalence for the Dutch translation of the conditional reasoning test for aggression

The Conditional Reasoning Test for Aggression (CRT-A) indirectly measures the implicit motive to aggress by engaging respondents in inductive reasoning tasks. Most research involving the CRT-A has been based on the original English version of the test with most data being collected in the United States. The purpose of the current paper is to evaluate the psychometric properties of a Dutch translation of this test and to examine if it could be used to predict measures of integrity. In the first of two studies, we evaluated the psychometric properties and measurement equivalence of the CRT-A across US and Dutch samples. In the second study, we examined validity evidence for the Dutch version of the CRT-A. Results from Study 1 indicated that the test was mostly equivalent across cultures (i.e. limited differential item functioning was detected). Results from Study 2 demonstrated that the Dutch version of the CRT-A was correlated with measures of behavioral integrity and provided incremental prediction of integrity over and above traditional self-report measures of explicit personality traits. We discuss the implications for using CRTs across different cultures and languages.     

Approximately counting and sampling knowledge states

Approximately counting and sampling knowledge states from a knowledge space is a problem that is of interest for both applied and theoretical reasons. However, many knowledge spaces used in practice are far too large for standard statistical counting and estimation techniques to be useful. Thus, in this work we use an alternative technique for counting and sampling knowledge states from a knowledge space. This technique is based on a procedure variously known as subset simulation, the Holmes–Diaconis–Ross method, or multilevel splitting. We make extensive use of Markov chain Monte Carlo methods and, in particular, Gibbs sampling, and we analyse and test the accuracy of our results in numerical experiments.