The generality of the relation between familiarity and judged validity

Using factual information of uncertain truth value as the stimulus material, previous investigators have found that repeated statements are rated more valid than non-repeated statements. Experiments 1 and 1A were designed to determine if this effect would also occur for opinion statements and for statements initially rated either true or false. Subjects were exposed to a 108-statement list one week and a second list of the same length a week later. This second list was comprised of some of the statements seen earlier plus some statements seen for the first time. Results suggested that all types of repeated statements are rated as more valid than their non-repeated counterparts. Experiment 2 demonstrated that the validity-enhancing effect of repetition does not occur in subject domains about which a person claims not be knowledgeable. From the results of both studies we concluded that familiarity is a basis for the judged validity of statements. The relation between this phenomenon and the judged validity of decisions and predictions was also discussed.     

The Error-Oriented Motivation Scale: An examination of structural and convergent validity

The current paper describes an analysis of the factor structure and convergent validity of the Error-Oriented Motivation Scale (EOMS). The EOMS is an attempt to validate a context-general and brief measure of error orientations theoretically similar to the Error Orientation Questionnaire (EOQ) written by (Rybowiak, Garst, Frese, and Batinic (1999)). Motivation theory, particularly approach/avoidance motives and goal theory, was chosen as a foundational model for the new EOMS measure, arguing that errors take on properties of goals when they occur by creating a discrepancy, which generates arousal and direction toward a solution. In this article, the factor structure of the EOMS is examined, as well as expected correlations with subscales of the EOQ. The discussion focuses on the advantages of the EOMS measure and suggestions for its further validation and use in applied settings.     

Evidence of construct validity for work values

Despite the importance of work values in the process of career adjustment (Dawis, 2002), little empirical research has focused on articulating the domains represented within the construct of work values and the examination of evidence of validity for the construct has been limited. Furthermore, the larger number of work values measures has made it difficult to determine the key domains that constitute the construct. The current study sought to examine multiple measures of work values to understand domains represented within the construct of work values and to establish evidence of validity for these domains. Principal Components Analysis utilizing scores on the Minnesota Importance Questionnaire (Rounds, Henley, Dawis, Lofquist, & Weiss, 1981), the revised edition of Super's Work Values Inventory (SWVI-R; Zytowski, 2006), and Manhardt's Work Values Inventory (Manhardt, 1972) found that six components best explained the data. These components reflected the importance of: the working environment, having challenging work, opportunities for status and income, autonomy, organizational support, and relationships. Implications for research and practice are discussed.     

The concurrent validity of the N-back task as a working memory measure

The N-back task is used extensively in literature as a working memory (WM) paradigm and it is increasingly used as a measure of individual differences. However, not much is known about the psychometric properties of this task and the current study aims to shed more light on this issue. We first review the current literature on the psychometric properties of the N-back task. With three experiments using task variants with different stimuli and load levels, we then investigate the nature of the N-back task by investigating its relationship to WM, and its role as an inter-individual difference measure. Consistent with previous literature, our data suggest that the N-back task is not a useful measure of individual differences in WM, partly because of its insufficient reliability. Nevertheless, the task seems to be useful for experimental research in WM and also well predicts inter-individual differences in other higher cognitive functions, such as fluid intelligence, especially when used at higher levels of load.     

Test validity from a temporal perspective: Incorporating time in validation research

As psychological theories and methodologies are increasingly being permeated with time, the need for reviewing and updating test validation practices is becoming apparent. After a brief comparison of the differential (between-subjects) and temporal (within-subjects) approaches to psychological research, this article examines the role that time has played in traditional validation research, and identifies a number of conceptual and methodological shortcomings that complicate meta-analysis, hamper theory building, and limit practical applications. Starting from the assumption that human traits and behaviours unfold over time, and that stability is a special form of change, it argues that time should be an explicit part of any measurement, either test or criterion, and that validation should include an evaluation of dynamic aspects of the test–criterion relationship. It furthermore argues that when stability is assumed, its temporal boundaries should be marked and supporting evidence be presented, using instruments displaying measurement invariance over time. The article continues to present a framework based on a Variables × Subjects × Time data model, and distinguishes four conditions for validation, namely (1) stable predictor, stable criterion, (2) stable predictor, dynamic criterion, (3) dynamic predictor, stable criterion, and (4) dynamic predictor, dynamic criterion, which allow the definition of five new types of validity. It discusses implications for validation practice, meta-analysis, and theory development as well as for practical applications.     

The behavioral validity of dual-task driving performance in fixed and moving base driving simulators

Next generation automotive hardware and user interfaces are increasingly pre-tested in driving simulators. What are the potential limitations of such simulations? We determined the relative and absolute validity of five different driving simulators at the Daimler AG by evaluating five functions of an in-vehicle system based on the guideline of the Alliance of Automobile Manufacturers (2006). The simulations were compared to on-road driving. We hypothesized that not only simulator characteristics, but also user characteristics, such as simulator sickness, gender, or age, influence behavioral validity. Even though relating simulator characteristics and user characteristics to driving performance across different driving simulators and driving tasks is difficult, our results are surprisingly in line with the current body of research. We demonstrate the usefulness of all simulators on a relative and partially on an absolute level with moving-base simulators being preferable to fixed-base simulators. As hypothesized, we showed that simulator sickness was significantly associated with impaired performance. In the fixed-base simulators, we found a significant interaction between age and gender, which we could not find in moving-base simulators and in the on-road study. Explanations for our findings and practical implications are discussed.     

Initial judgment task and delay of the final validity-rating task moderate the truth effect

Repeatedly seen or heard statements are typically judged to be more valid than statements one has never encountered before. This phenomenon has been referred to as the truth effect. We conducted two experiments to assess the plasticity of the truth effect under different contextual conditions. Surprisingly, we did not find a truth effect in the typical judgment design when using a ten minutes interval between statement repetitions. However, we replicated the truth effect when changing the judgment task at initial statement exposure or when using an interval of one week rather than ten minutes. Because none of the current truth effect theories can fully account for these context effects, we conclude that the cognitive processes underlying truth judgments are more complex than has hitherto been assumed. To close the theoretical gap, we propose a revised fluency attribution hypothesis as a possible explanation of our findings.     

An examination of gender differences in the construct validity of the silencing the self scale

Jack’s (1991) theory of self-silencing was originally designed to explain higher rates of depression in women in comparison to men. However, research finding that men score equal or even higher than women on measures of self-silencing has lead theorists to speculate that self-silencing tendencies may be driven by different motivations and have different consequences for women versus men (Jack & Ali, 2010). Using a sample of 247 college students, we examined gender differences in the construct validity of the Silencing the Self Scale (STSS; Jack & Dill, 1992). We hypothesized that women would score higher on the Externalized Self-Perception subscale, but not the other three subscales. Gender differences in the relationship between the STSS subscales and theoretically-relevant constructs were also explored. The results indicated that women on average scored higher than men on the Externalized Self-Perception subscale, whereas men scored higher on the Care as Self-Sacrifice subscale. Further, there was a significant Gender × Care as Self-Sacrifice subscale interaction in the prediction of depression, such that this subscale was negatively correlated to depression in men, and uncorrelated in women. These results clarify how self-silencing might translate into different mental health outcomes for women and men.     

The reliability and validity of the Turkish version of the brief infant–toddler social emotional assessment (BITSEA)

In this study the reliability and validity of the Turkish version of the brief infant–toddler social emotional assessment (BITSEA) were investigated in a community sample. The sample consisted of 462 children (mean age: 24.60 ± 7.93 [12–42] months) who had applied to Turkish health centers for immunization. Both parents completed the BITSEA; mothers completed the child behavior checklist 2/3 (CBCL). Internal consistencies of the BITSEA–problem (P) and competence (C) scales were good to excellent (Cronbach's α = 0.82 and 0.72, respectively). Interrater reliability between parents and test–retest reliability were good. BITSEA/P scores were significantly correlated with CBCL internalizing, externalizing and total problem scores (p < 0.001). Maternal BITSEA/P cutpoint scores revealed that 30.6% of male toddlers and 28.6% of females were in the subclinical range and 13.1% of males and 17.6% of females were in clinical range. Results reveal that the Turkish version of BITSEA is a reliable, valid and simply applicable instrument for screening social, emotional and behavioral problems among toddlers. Clinical validation of the BITSEA/C and BITSEA/P is warranted.     

An exploratory investigation of the reliability and validity of the Independent-Interdependent Problem-Solving Style Scale

The Independent-Interdependent Problem-Solving Scale is based on Cross et al.'s conceptualisation of relational-interdependent self-construal. The IIPSS provides a relatively context-free measure of people's tendencies to solve problems independently or with the help of others. Because previous investigations have not provided extensive evidence for the reliability and validity of the IIPSS, the current research aimed to test the psychometric properties of this novel measure. Investigations of four student samples (combined N = 1157) and one sample comprised of academic researchers (N = 198) generally supported the reliability and validity of the IIPSS. Exploratory factor analysis of IIPSS items yielded a single factor structure. However, confirmatory factor analyses did not demonstrate good model fit for the one factor solution and instead yielded good model fit for two underlying factors. The IIPSS showed adequate test–retest reliability and predicted positive associations with social personality traits. It also showed no significant associations with measures of demand characteristics and social desirability. Future research needs to be undertaken to further assess the factor structure and address shortcomings of the present research such as utilising objective data in addition to self-reports to assess the scale's validity.     

Sad, thus true: Negativity bias in judgments of truth

An effect observable across many different domains is that negative instances tend to be more influential than comparably positive ones. This phenomenon has been termed the negativity bias. In the current work, it was investigated whether this effect pertains to judgments of truth. That is, it was hypothesized that information valence and perceived validity should be associated such that more negative information is deemed more true. This claim was derived from the findings that negative instances tend to demand more attentional resources and that more elaborate processing can render messages more persuasive. In three experiments, manipulating information valence through framing - and assessing judgments of truth - the hypothesized negativity bias was corroborated. Potential explanations and implications for further research are discussed.     

Towards multifaceted Machiavellianism: Content, factorial, and construct validity of a German Machiavellianism Scale

Machiavellianism comprises several distinct aspects (Rauthmann & Will, 2011), but it is unclear what Machiavellianism scales measure. The current study exemplarily investigates for a German Machiavellianism Scale (Henning & Six, 2008) (a) item content, (b) factorial structure, and (c) factor correlations with other scales (Big Five, narcissism, psychopathy, emotional manipulation, socio-emotional skills, self-esteem, self-monitoring) in a large sample (n = 438 women, n = 138 men). Findings yielded that (a) cynical/misanthropic views, agentic orientations, and deceitful behavior are (over-)represented in content; (b) different Machiavellianism factors emerge for men and women; and (c) there are differential associations of these factors with other scales. Findings are relevant for conceptualization and measurement of Machiavellianism.     

Factorial and construct validity of the revised short form integrative psychotherapy alliance scales for family, couple, and individual therapy

The Integrative Psychotherapy Alliance model brought an interpersonal and systemic perspective to bear on theory, research, and practice on the psychotherapeutic alliance. Questions have been raised about the independence of the theoretical factors in the model and their operationalization in the Individual, Couple, and Family Therapy Alliance Scales. This paper presents results of a confirmatory factor analysis of the scales that delineated at least three distinct interpersonal factors as well as shorter versions of the three scales to facilitate their use in research and practice. The paper also presents the results of a study testing each factor's association with client retention and progress over the first eight sessions in individual and couple therapy. At least two of the interpersonal factors were uniquely associated with progress in individual and couple functioning. Implications of the results for theory, research, practice, and training in individual, couple, and family therapy are elaborated.     

Reliability and validity of the Padua Inventory in an Iranian population

In order to study the reliability and validity of the Padua Inventory (PI), its items were translated into Persian and then back-translated into English as suggested by Brislin, Lonner, Thorndike, Cross-cultural research methods, Wiley, New York, 1973. Using the translated version of the PI, two studies were carried out. The first study included 219 healthy participants selected from Shiraz city in Iran to examine the reliability and factorial structure of the PI via test-retest, internal consistency and principal component analysis methods. The results of this study showed good reliability for the PI and confirmed its factorial structure reported by previous studies. The second study included four groups of participants as follows: patients with obsessive-compulsive disorder (OCD), patients with anxiety disorders, patients with depressive disorder and normal individuals. The PI scores were able to differentiate between OCD patients and normal individuals, but failed to differentiate between neurotic and OCD patients. Two subscales of PI-R version of the PI (precision and washing) differentiated depressive and OCD patients.     

The impact of wording effect on reliability and validity of the Core Self-Evaluation Scale (CSES): A bi-factor perspective

“Wording effect” refers to the systematic method variance caused by positive and negative item wordings on a self-report measure. Using the Core Self-Evaluation Scale (CSES) as an example, this study modeled the wording effect, and examined the impact of such wording effect on the reliability and validity of CSES. Five competing models were compared, and the findings suggested that the bi-factor model with a negative group factor best represented the CSES structure. The impact of wording effect on the reliability and validity of CSES was investigated, and the results showed that such impact existed and was non-negligible. Failure to model the wording effect would lead to biased estimates of the reliability and criterion-related validity of CSES. Limitations of the study are discussed, and considerations for future research directions are suggested.     

Comparing hierarchical models of personality pathology

Two dimensional, hierarchical classification models of personality pathology have emerged as alternatives to traditional categorical systems: multi-tiered models with increasing numbers of factors and models that distinguish between a general factor of severity and specific factors reflecting style. Using a large sample (N = 840) with a range of psychopathology, we conducted exploratory factor analyses of individual personality disorder criteria to evaluate the validity of these conceptual structures. We estimated an oblique, “unfolding” hierarchy and a bifactor model, then examined correlations between these and multi-method functioning measures to enrich interpretation. Four-factor solutions for each model, reflecting rotations of each other, fit well and equivalently. The resulting structures are consistent with previous empirical work and provide support for each theoretical model.     

Short-term predictive validity of cluster analytic and dimensional classification of child behavioral adjustment in school

A constructive debate over the classification of child psychopathology can be stimulated by investigating the validity of different classification approaches. We examined and compared the short-term predictive validity of cluster analytic and dimensional classifications of child behavioral adjustment in school using the Behavior Assessment System for Children (BASC; [Reynolds, C. R., and Kamphaus, R. W. (1992). Behavior Assessment System for Children (BASC). Circle Pines, MN: AGS.]) Teacher Rating Scales for Children (TRS-C). We cluster analyzed 14 dimensional scores of children's behavior in first grade and identified seven clusters. Then we examined the predictive power of the dimensions and the clusters using a variety of school outcomes in second grade, including academic achievement scores in reading and math, absenteeism, discipline reports, and participation rates in pre-referral intervention. Both methods significantly predicted all school outcomes with similar magnitudes of effect sizes, but the dimensional approach was more powerful in predicting the outcomes except pre-referral intervention. Moreover, coherent group differences in the clusters were identified with respect to the school outcomes. This latter finding may suggest differentiated prognoses of cluster membership. The implications for future validation studies of alternative classification systems of child behavioral adjustment are discussed.     

Evidence of validity for the Skill Scale scores of the Campbell Interest and Skill Survey

The purpose of the current study was to investigate the evidence of validity of the Skill Scale scores on the Campbell Interest and Skill Survey (CISS). In a sample of 221 college students, analyses between the CISS Skill Scale scores and the CISS Interest Scale, the Strong Interest Inventory, and selfreported MAEQ abilities scores were conducted. Results provided evidence of convergent and discriminate validity of the Skill Scale scores. Furthermore, Skill Scale scores were found to be predictive of declared college major above chance levels. Conclusions from the results support the use of the CISS Skill Scales in counseling settings. Further investigation of the content being measured by the Skill Scales and their use in career interventions is recommended.     

Reliability and validity of machine vision for the assessment of facial expressions

Automated assessment of facial expressions with machine vision software opens up new opportunities for the assessment of facial expression in a shrewd and economic way in psychological and applied research. We investigated the assessment quality of one machine vision algorithm (FACET) in a study using standardized databases of dynamic facial expressions in different conditions (angle, distance, lighting and resolution). We found high reliability in terms of ratings concordance across conditions for facial expressions (intraclass correlation, ICC = 0.96) and action units (ICC = 0.78). Signal detection analyses showed good classification for both facial expressions (area under the curve, AUC > 0.99) and action unit scores (AUC = 0.91). In a second study, we investigated the convergent validity of machine vision assessment and electromyography (EMG) with regard to reaction times measured during the production of smiles (action unit 12) and frowns (action unit 4). To this end, we simultaneously measured EMG and expression classification with machine vision software in a response priming task with validly and invalidly primed responses. Both, EMG and machine vision data revealed similar performance costs in reaction times of inhibiting the falsely prepared expression and reprogramming the correct one. These results support machine vision as a suitable tool for assessing experimental effects in facial reaction times.     

Phenomenological validity of an OCD-memory model and the remember/know distinction

In earlier experiments using interactive computer animation with healthy subjects, it was found that displaying compulsive-like repeated checking behavior affects memory. That is, checking does not alter actual memory accuracy, but it does affect 'meta-memory': as checking continues, recollections are experienced as less vivid and less detailed while confidence in memory is undermined. This procedure provides a model of OCD checking and suggests that checking is a counterproductive strategy to reduce memory distrust. The present experiment was carried out to specify the phenomenological quality of memory distrust after checking and to see if repeated checking produces a shift in the memory source that is used to decide about the outcome of checking: from 'remembering' to 'knowing' (Tulving, 1985). Using the same interactive computer-animation, the earlier findings on vividness, detail and confidence were replicated. In addition, it was found that checking made participants endorse quotations from OCD patients (Reed, 1985) expressing a specific ambivalence about memory: 'It is as though the memory is there, but is isn't definite enough', 'I remember doing it in a way, but it's all fuzzy....' And 'I can remember that I've done it. But the memory isn't clear somehow'. This finding adds to the validity of the experimental model. Furthermore, after checking subjects' beliefs about the outcome of checking became based on (general) knowing instead of (specific) remembering. It is suggested that OCD checkers feel a general and relatively strong need to be certain about the veracity of recollections and that they have high standards for memory performance. This may explain earlier findings that OCD checkers have a general tendency to distrust their episodic memory. A need for certainty and a critical attitude towards memory performance may not be problematic or abnormal. It is suggested that clinical problems arise when the patient tries to fight memory distrust by repeated checking. The latter does not reduce distrust but rather increases distrust and the patient may get trapped in a spiral of mutually reinforcing checking behavior and memory distrust.