Investigating the validity of the Perceptual Awareness Scale – The effect of task-related difficulty on subjective rating

The Perceptual Awareness Scale (PAS) is often used to probe conscious experience, but the assumptions behind the scale and its validity are rarely tested. Using a continuous colour estimation task to assess perceptual quality, we focus on how well PAS follows perceptual quality and how the presence of the estimation task affects PAS ratings. We varied the number and presentation time of stimuli in a simultaneous presentation and target position in a sequential presentation. In all experiments, PAS rating closely followed colour precision. However, it was affected by task-induced response bias, even when the possible task difficulty was judged, without performing the task itself. Still, this bias was only observed on the absolute rating level rather than the scale's ability to capture changes in perceptual quality. Reported studies shed light on factors influencing scales outside of the scale formulation and construction.     

Can competitive paradigms increase the validity of experiments on primate social cognition?

Experiments vary in their ability to distinguish between competing hypotheses. In tests on primate cognition the majority of this variation is due to an experimenter's ability to test primates in valid settings while providing the adequate amount of experimental control. While experimenters studying primate cognition can use methods of control perfected in captivity, it is still very unclear how to design and then objectively evaluate the external validity of new experimental paradigms. I recommend that more effort be allocated to specify how to create relevant test settings for primates. Primate social life is highly competitive. This means that all aspects of primates themselves, including their cognitive abilities, have likely been shaped by the need to out-compete conspecifics. Based on this hypothesis, sophisticated cognitive abilities of primates might best be demonstrated in competitive contexts. Thus, it is suggested that one possible measure of validity is whether investigators integrate a competitive component into their experimental designs. To evaluate this methodological prediction I review the literature on chimpanzee perspective-taking as a case study including several recent studies that include a competitive component in their experimental designs. Accepted after revision: 8 April 2001 Electronic Publication     

Do response time limitations counteract the effect of faking on personality inventory validity?

Different models of lying on personality scales make discrepant predictions on the association between faking and item response time. The current research investigated response time restriction as a method for reducing the influence of faking on personality scale validity. In 3 assessment simulations involving 540 university undergraduates responding to 2 common, psychometrically strong personality inventories, no evidence emerged to indicate that limiting respondents' answering time can attenuate the effects of faking on validity. Results were interpreted as failing to support a simple model of personality test item response dissimulation that predicts that lying takes time. Findings were consistent with models implying that lying involves primitive cognitive processing or that lying may be associated with complex processing that includes both primitive responding and cognitive overrides.     

How smart do you think you are? A meta-analysis on the validity of self-estimates of cognitive ability

Individuals' perceptions of their own level of cognitive ability are expressed through self-estimates. They play an important role in a person's self-concept because they facilitate an understanding of how one's own abilities relate to those of others. People evaluate their own and other persons' abilities all the time, but self-estimates are also used in formal settings, such as, for instance, career counseling. We examine the relationship between self-estimated and psychometrically measured cognitive ability by conducting a random-effects, multilevel meta-analysis including a total of 154 effect sizes reported in 41 published studies. Moderator variables are specified in a mixed-effects model both at the level of the individual effect size and at the study level. The overall relationship is estimated at r = .33. There is significant heterogeneity at both levels (i.e., the true effect sizes vary within and between studies), and the results of the moderator analysis show that the validity of self-estimates is especially enhanced when relative scales with clearly specified comparison groups are used and when numerical ability is assessed rather than general cognitive ability. The assessment of less frequently considered dimensions of cognitive ability (e.g., reasoning speed) significantly decreases the magnitude of the relationship. From a theoretical perspective, Festinger's (1954) theory of social comparison and Lecky's (1945) theory of self-consistency receive empirical support. For practitioners, the assessment of self-estimates appears to provide diagnostic information about a person's self-concept that goes beyond a simple "test-and-tell" approach. This information is potentially relevant for career counselors, personnel recruiters, and teachers.     

Are we puppets on a string? Comparing the impact of contingency and validity on implicit and explicit evaluations

Research has demonstrated that implicit and explicit evaluations of the same object can diverge. Explanations of such dissociations frequently appeal to dual-process theories, such that implicit evaluations are assumed to reflect object-valence contingencies independent of their perceived validity, whereas explicit evaluations reflect the perceived validity of object-valence contingencies. Although there is evidence supporting these assumptions, it remains unclear if dissociations can arise in situations in which object-valence contingencies are judged to be true or false during the learning of these contingencies. Challenging dual-process accounts that propose a simultaneous operation of two parallel learning mechanisms, results from three experiments showed that the perceived validity of evaluative information about social targets qualified both explicit and implicit evaluations when validity information was available immediately after the encoding of the valence information; however, delaying the presentation of validity information reduced its qualifying impact for implicit, but not explicit, evaluations.     

Do desirability ratings moderate the validity of probability ratings on the Smoking Consequences Questionnaire-Adult? A reanalysis using regression

Smokers' responses to the Smoking Consequences Questionnaire-Adult (A. L. Copeland, T. H. Brandon, & E. P. Quinn, 1995) were used to determine whether smoking outcome expectancies are moderated by subjective evaluations (desirability ratings) in their effect on smoking-related variables. Hierarchical regression in a reanalysis of data from A. L. Copeland et al. indicated that the product of probability and desirability ratings accounted for a significant amount of variance in smoking rate, nicotine dependence, saliva cotinine, and posttreatment smoking status above that accounted for by probability and desirability ratings alone. Results indicate that desirability ratings serve as moderators to probability ratings in explaining current smoking, nicotine dependence, and continued smoking. Effects were modest in magnitude but suggest that it may be important to address subjective evaluations of outcomes in smoking cessation and relapse prevention efforts.     

Does face recognition rely on encoding of 3-D surface? Examining the role of shape-from-shading and shape-from-stereo

It is now well known that processing of shading information in face recognition is susceptible to bottom lighting and contrast reversal, an effect that may be due to a disruption of 3-D shape processing. The question then is whether the disruption can be rectified by other sources of 3-D information, such as shape-from-stereo. We examined this issue by comparing identification performance either with or without stereo information using top-lit and bottom-lit face stimuli in both photographic positive and negative conditions. The results show that none of the shading effects was reduced by the presence of stereo information. This finding supports the notion that shape-from-shading overrides shape-from-stereo in face perception. Although shape-from-stereo did produce some signs of facilitation for face identification, this effect was negligible. Together, our results support the view that 3-D shape processing plays only a minor role in face recognition. Our data are best accounted for by a weighted function of 2-D processing of shading pattern and 3-D processing of shapes, with a much greater weight assigned to 2-D pattern processing.     

Discriminant validity of the Conners’ scales for ADHD subtypes

Differential diagnosis and treatment of children with Attention-Deficit/Hyperactivity Disorder (ADHD) remains difficult and controversial, particularly when children present with concomitant academic and behavior problems. Describing a sample of 184 chil-dren referred for evaluation in a university-affiliated outpatient pediatric clinic, this study subsequently examined the discriminant validity of the Conners’ parent and teacher rating scales for ADHD subtypes with and without comorbid conditions. Although discriminant analyses suggest several Conners’ subscales help differentiate subtypes, multimethod evaluations using a variety of data sources are necessary for accurate identification and classification of the disorder with and without comorbid conditions. These data were previously reported at the 107^th Annual Convention of the American Psychological Association, Boston, Massachusetts. The authors express gratitude to Ross D. Crosby, Ph.D., Director of Biomedical Statistics and Methodology at the Neuropsychiatrie Research Institute, for his pre-submission review of the article.     

Does recognition of single words predict recognition of two?

Participants studied a list of words, presented 1 at a time. When they were tested on pairs of words and asked to indicate whether both had occurred in the list, they gave more positive responses than would be expected on the basis of results from participants tested 1 word at a time. When participants were asked whether either member of a test pair had been shown on the study list, they gave positive responses less often than expected. When participants were asked to indicate how many words in a test pair came from the study list, they answered "2" more often and "1" less often than would be expected on the basis of results from single-item recognition. Accuracy was not affected by testing items in pairs.     

How do the internal details of the object contribute to recognition?

Object recognition becomes difficult when the main axis of the object is foreshortened. It has previously been reported that this so-called foreshortened disadvantage is larger when the silhouette of the object is presented than when the line drawing of the object is presented. The pronounced foreshortened disadvantage in silhouette recognition indicates that the internal details of the object, which are absent in the silhouette, provide useful information, particularly when the main axis is foreshortened. But the role of these internal details remains controversial. One account for the pronounced disadvantage is that the internal details contribute to the derivation of the main axis. The other account is that internal details provide the distinctive features that are directly matched to the object represented in memory. The aim in the present study was to determine which of these two explanations best accounts for the differential foreshortened disadvantage between line drawings and silhouettes. To reduce the uncertainty regarding the axis orientation, a 3-D arrow indicating the orientation of the main axis was presented as a cue before the object itself was presented. As a result, the difference in the foreshortened disadvantage between silhouettes and line drawings disappeared. This indicated that the pronounced foreshortened disadvantage for silhouettes was caused by a lack of axis information. In other words, the internal details provided the information necessary for axis derivation when the axis was foreshortened.     

Substance or style? An investigation of the NEO-PI-R validity scales

The Revised NEO Personality Inventory (NEO-PI-R; Costa & McCrae, 1992b) has been criticized for the absence of validity scales designed to detect response distortion. Recently, validity scales were developed from the items of the NEO-PI-R (Schinka, Kinder, & Kremer, 1997) and several studies have used a variety of methods to test their use. However, it is controversial whether these scales are measuring something that is substantive (such as psychopathology or its absence) or stylistic (which might be effortful distortion or less conscious processes such as lack of insight). In this study, we used a multimethod-multitrait approach to examine the validity of these scales in a clinical sample of 668 participants diagnosed with personality disorders or major depression. Using various indicators of both stylistic and substantive variance, confirmatory factor analyses (CFA) suggested that these validity scales measure something that may be conceptually distinct from, yet highly related to, substantive variance in responding.     

Movement contributes to infants’ recognition of the human form

Three experiments demonstrate that biological movement facilitates young infants’ recognition of the whole human form. A body discrimination task was used in which 6-, 9-, and 12-month-old infants were habituated to typical human bodies and then shown scrambled human bodies at the test. Recovery of interest to the scrambled bodies was observed in 9- and 12-month-old infants in Experiment 1, but only when the body images were animated to move in a biologically possible way. In Experiment 2, nonbiological movement was incorporated into the typical and scrambled body images, but this did not facilitate body recognition in 9- and 12-month-olds. A preferential looking paradigm was used in Experiment 3 to determine if infants had a spontaneous preference for the scrambled versus typical body stimuli when these were both animated. The results showed that 12-month-olds preferred the scrambled body stimuli, 9-month-olds preferred the typical body stimuli and the 6-month-olds showed no preference for either type of body stimuli. These findings suggest that human body recognition involves integrating form and movement, possibly in the superior temporal sulcus, from as early as 9 months of life.     

Get the gist? The effects of processing depth on false recognition in short-term and long-term memory

Gist-based processing has been proposed to account for robust false memories in the converging-associates task. The deep-encoding processes known to enhance verbatim memory also strengthen gist memory and increase distortions of long-term memory (LTM). Recent research has demonstrated that compelling false memory illusions are relatively delay-invariant, also occurring under canonical short-term memory (STM) conditions. To investigate the contributions of gist to false memory at short and long delays, processing depth was manipulated as participants encoded lists of four semantically related words and were probed immediately, following a filled 3- to 4-s retention interval, or approximately 20 min later, in a surprise recognition test. In two experiments, the encoding manipulation dissociated STM and LTM on the frequency, but not the phenomenology, of false memory. Deep encoding at STM increases false recognition rates at LTM, but confidence ratings and remember/know judgments are similar across delays and do not differ as a function of processing depth. These results suggest that some shared and some unique processes underlie false memory illusions at short and long delays.     

Construct validity of the Profile of Mood States — Adolescents for use with adults

Objectives: To extend the validation of the Profile of Mood States-Adolescents (POMS-A: J. Sports Sci. 17 (1999) 861–872) from adolescent to adult populations.Design: A strategy of assessing the invariance of the POMS-A factor structure among disparate samples and of testing relationships with concurrent measures was used.Methods: The POMS-A was administered to 2549 participants from four samples: adult athletes prior to competition (n=621), adult student athletes in a classroom (n=656), adolescent athletes prior to competition (n=676), and adolescent students in a classroom (n=596). A subset of 382 adult student athletes was used to test the criterion validity of the POMS-A.Results: Confirmatory factor analysis provided support for the factorial validity of a 24-item, six-factor model using both independent and multi-sample analyses. Relationships between POMS-A scores and previously validated measures, that were consistent with theoretical predictions, supported criterion validity.Conclusion: Evidence was found in support of the psychometric integrity of the POMS-A when extended from adolescent to adult populations.     

Ecological validity of the German Bruininks-Oseretsky Test of Motor Proficiency – 2nd Edition

The diagnosis of Developmental Coordination Disorder (DCD) is based on poor motor coordination in the absence of other neurological disorders. In order to identify the presence of movement difficulties, a standardised motor assessment is recommended to determine the extent of movement problems which may contribute to deficits in daily task performance. A German version of the Bruininks-Oseretsky Test of Motor Proficiency, Second Edition (German BOT-2) was recently published. This study aimed to determine the ecological validity of the German BOT-2 by considering the relationship between assessment of fundamental motor skills with the BOT-2 and performance of everyday motor activities as evaluated by parents. This study used data obtained from the German BOT-2 standardisation study (n = 1.177). Subtests were compared with theoretically corresponding tasks via parental ratings of overall fine and gross motor abilities and performance in six typical motor activities. Non-parametric Jonckheere Terpstra test was used to identify differences in ordered contrasts. Subtests reflecting ‘Strength’, ‘Running Speed and Agility’, ‘Upper-Limb Coordination’, ‘Balance’, and ‘Fine Motor Precision’ were associated with parental evaluation of gross motor skills (p < 0.001). The subtest ‘Fine Motor Integration’ significantly correlated with parental ratings of females’ fine motor skills. Parental ratings of males’ fine motor skills were associated with three further subtests. Regarding everyday motor activities, the first three fine motor BOT-2 subtests were associated with parent evaluations of drawing, writing and arts and crafts (p < 0.001). Gross motor subtests of ‘Bilateral Coordination’ and ‘Balance’ showed no relationship to bike riding or performance in sports. Subtests of ‘Upper-Limb Coordination’ and ‘Strength’ showed significant correlations with sports, ball games and cycling. The results of this study suggest that the closer the proximity in the nature of the motor skills assessed in the German BOT-2 to daily motor tasks, the stronger the relationship between the clinical test and parental report of everyday performance of their child. The body functions tested in the German BOT-2, and hypothesized to underpin certain skills, were not automatically relevant for specific activities undertaken by German children. Future research should investigate the relationships of the various BOT-2 constructs for diagnosis of DCD.     

“Passion” versus “patience”: the effects of valence and arousal on constructive word recognition

Accumulating evidence suggests that emotional information is often recognised faster than neutral information. Several studies examined the effects of valence and arousal on word recognition, but yielded partially diverging results. Here, we used two alternative versions of a constructive recognition paradigm in which a target word is hidden by a visual mask that gradually disappears, to investigate whether the emotional properties of words influence their speed of recognition. Participants were instructed either to classify the incrementally appearing word as emotional or non-emotional (semantic categorisation task) or to decide whether the appearing letter string is an existing word or not (lexical decision task). Results from both tasks revealed faster recognition times for high- compared to low-arousing words, and for positive compared to negative or neutral words. These findings indicate a recognition advantage for emotionally positive and highly arousing stimuli that persists even when visual word recognition is hampered and participants are encouraged to make more active, semantic inferences to generate the meaning of the emerging word.     

Comparing the validity of structured interviews for managerial‐level employees: Should we look to the past or focus on the future?

The current research investigated questions that persist regarding the criterion‐related and construct validity of situational (SI) versus past‐behaviour (PBI) structured interview formats in predicting the job performance of managers. Analyses of data collected from 157 applicants to managerial positions showed that the PBI format significantly predicted job performance ratings (r = .32, p <.01), whereas the SI format did not (r = .09, ns). Investigation of potential construct differences between the SI and PBI formats showed that the PBI was more highly related to manager‐relevant cognitive ability measures, assessment centre exercises and personality traits, as compared with the SI. Such differences help to explain the predictive validity differences between the SI and PBI observed in current and previous research.     

Is it typical? The ecological validity of the observation of mother-father-infant interactions in the Lausanne Trilogue Play

The Lausanne Trilogue Play (LTP) is a widely used laboratory situation designed for the systematic observation of mother-father-infant interactions for clinical and research purposes. Nevertheless, its ecological validity has never been tested. In this exploratory study, 49 primiparous parents with their 3-month-old infants were assessed by questionnaires to determine the extent to which their behaviour in the LTP was typical, that is, representative of everyday interactions. Results show that (a) most of the parents assessed their behaviour as typical and (b) the quality of triadic interactions was linked with parents’ assessment of the typicality of the infant’s behaviour.