Understanding and Improving the Validity of Self-Report of Parenting

Although researchers using parental self-report data have questioned its validity (Holden, 2001) and called for more work in this area (Krevans & Gibbs, 1996; Locke & Prinz, 2002), methodological concerns regarding self-report about parenting practices have not been addressed adequately. The susceptibility of parental report to systematic distortions suggests a need for research on ways to improve self-report. This paper attempts to synthesize research findings from other fields (e.g., survey methodology) about ways to improve the validity of self-report, and to discuss the implications for self-report of parenting behaviors. Methods for improving self-report are presented for each of the 5 major tasks in responding to a question: (1) understanding the question, (2) recalling relevant behavior, (3) inference and estimation, (4) mapping the answer onto the response format, and (5) “editing” the answer for reasons of social desirability. Self-administered interviewing, audio-computer-assisted interviewing, pretesting, conversational interviewing, and the decompositional item may be among the best candidates for use in parental self-report. Recommendations are offered with respect to strategies that might prove useful in improving parental assessment of parenting, and to research efforts to evaluate the utility and potential costs of these strategies.     

Methods of assessing craving to gamble: A narrative review

Although the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV) does not explicitly list craving as a diagnostic criterion for pathological gambling, theories of disordered gambling and research on relapse precipitants suggest that cravings-among other factors-provoke and maintain episodes of gambling. Assessment of craving to gamble is complicated by questions regarding (a) the emotional, cognitive, behavioral, and physiological components of such craving; (b) the degree to which craving is viewed as an acute and fluctuating experience or as a relatively stable preoccupation with or inclination to gamble; (c) the threshold separating mild desire from pathological craving to gamble; and (d) the degree to which disordered gamblers are aware of, and able to report on, their experience of craving. Our literature search revealed various self-report methods that could be used to assess craving to gamble, including single-item rating scales, multi-item questionnaires, and application of the think-aloud procedure. In addition, psychophysiological reactivity (e.g., heart rate, brain activation) to gambling-related stimuli and reaction time (RT) tasks (e.g., gambling Stroop, Lexical Salience Task) may serve as proxy measures of subjective craving to gamble. Although researchers have assessed elements of reliability and validity of many measures, most require additional evaluation to examine their predictive and construct validity and their utility across different modes of gambling. The field would also benefit from further research to develop and evaluate additional self-report and proxy measures. (PsycINFO Database Record (c) 2012 APA, all rights reserved).     

Honor Scale: evidence on construct validity

Honor is an important construct, mainly emphasized in collectivist cultures, such as Brazil. However, there is a lack of studies regarding honor in this cultural context. Therefore, the current research aims at proposing a short version of the Honor Scale (HS‐16) to the Brazilian context. Construct validity (i.e., factorial, convergent, and discriminant validity, as well as internal consistency and composite reliability) is reported in Studies 1 (n = 220) and 2 (n = 222), with participants from general population. In both studies, participants answered the Honor Scale and demographic questions. The authors find the HS‐16 to be a reliable and valid scale, thereby enabling its use in quantitative studies regarding the correlates of honor concerns.     

Screening for trauma in children and adolescents: the validity of the Traumatic Stress Disorder Scale of the screen for child anxiety related emotional disorders

Examined the validity of the Traumatic Stress Disorder scale of the Screen for Child Anxiety Related Emotional Disorders (SCARED), a recently developed self-report questionnaire measuring Diagnostic and Statistical Manual of Mental Disorders (4th ed. [DSM-IV]; American Psychiatric Association, 1994) defined anxiety disorders symptoms in children. A large sample of normal schoolchildren (N = 996) ages 7 to 19 years completed the SCARED. Children who scored high on the SCARED Traumatic Stress Disorder scale (i.e., trauma group; n = 43) and children who scored low on this scale (i.e., control group; n = 43) were then interviewed about their most aversive life event. In addition, children completed self-report questionnaires of traumatic experiences and posttraumatic stress disorder (PTSD) symptomatology. Results showed that children in the trauma group more frequently reported life events that independent judges considered to be 'potentially traumatic' than did control children. Furthermore, children in the trauma group reported having experienced more traumatic incidents and had higher scores on PTSD-related questionnaires compared with control children. Moreover, trauma group children more frequently fulfilled DSM-IV criteria for PTSD than did control children. The results of this study support the validity of the Traumatic Stress Disorder scale of the SCARED.     

Assessment of Panic Frequency: Reliability and Validity of a Time-Line Follow-Back Method

The purpose of this study was to investigate the reliability and validity of a Panic Attack Frequency Calendar (PAFC), modeled after a time-line follow-back (TLFB) procedure (e.g., Sobell & Sobell, 1979). The latter has been used for years to reliably and validly assess daily alcohol use through self-report over extended periods of time. Seventy-four adult individuals (ages 18–57) who had experienced a panic attack within the past 2 weeks completed a battery of self-report questionnaires, including retrospective frequency measures, and were administered an 8-week PAFC initially and a 10-week PAFC 2 weeks later. Half the participants self-monitored their panic attacks in a diary. The PAFC showed high stability over 2 weeks across several panic behavior variables. The PAFC was also shown to be a valid instrument in assessing panic frequency when compared with panic data obtained from the diary and the retrospective frequency measure.     

Development of a coping with stress scale for a non-western population of children and adolescents

Background and objectives: In the related literature numerous instruments have been developed to measure children and adolescents’ coping with stress. Considering the cultural differences in individuals’ choice for coping strategies, along with the limitations of the existing measures of coping for children and adolescents (e.g., being derived from coping measures developed for adults; unrepresentative samples with limited age range, etc.), the current study aimed to construct a self-report coping scale for a non-western population of children and adolescents.

Design and methods: The study design included both qualitative and quantitative methodology. Three consecutive studies were conducted for the development and validation of the Children and Adolescents’ Coping with Stress Scale (CACSS), a self-report measure assessing coping strategies of children and adolescents aged from 9 to 18 in response to self-identified stressors.

Results: Exploratory and confirmatory factor analyses resulted in a 61-item CACSS with 10 factors. The scale appears to have a clear factor structure; sufficient temporal stability; and good convergent, discriminant, and construct validity.

Conclusions: By addressing limitations of existing coping scales, CACSS is believed to contribute to the literature as a developmentally appropriate and multidimensional tool.     

Turning the process-dissociation procedure inside-out: A new technique for understanding the relation between conscious and unconscious influences

While there is now general agreement that memory gives rise to both conscious and unconscious influences, there remains disagreement concerning the process architecture underlying these distinct influences. Do they arise from independent underlying systems (e.g., Jacoby, 1991) or from systems that are interactive (e.g., Joordens & Merikle, 1993)? In the current paper we present a novel “inside-out” technique that can be used with the process-dissociation paradigm to arrive at more concrete conclusions concerning this central question and demonstrate this technique via a meta-analysis of currently published findings. Our results suggest that the data presented in these studies vary in ways most consistent with the assumption that conscious and unconscious influences behave independently.     

A Brief Report on the Assessment of Distress Tolerance: Are We Measuring the Same Construct?

Distress tolerance (DT) refers to an individual’s capacity to cope with aversive internal (e.g., physical, cognitive, emotional) states. A growing body of evidence suggests that there is a relationship between DT and the development and maintenance of problematic behavioral patterns. Despite emerging evidence for such associations, a number of issues remain unresolved. The results of recent studies suggest problems with the convergent validity of the primary measures used to assess DT, despite the fact that these measures are used interchangeably in the DT literature. In order to further examine the relationships among DT assessments, we evaluated intercorrelations among various self-report and behavioral measures of DT in an unselected undergraduate sample (n = 83). Results indicate that two self-report measures of DT were highly correlated with one another, but that neither measure was significantly correlated with the behavioral measures. The relationships among the behavioral measures and between self-report and behavioral measures were weak and non-significant. Correlations between self-report measures only were strong. The findings partially replicate prior research indicating weak correlations between certain, commonly used measures of DT, and raise questions about the current conceptualization of the construct.     

A closer look at self-reported suicide attempts: false positives and false negatives

The validity of self-reported suicide attempt information is undermined by false positives (e.g., incidences without intent to die), or by unreported suicide attempts, referred to as false negatives. In a sample of 1,385 Austrian adults, we explored the occurrence of false positives and false negatives with detailed, probing questions. Removing false positives decreased the rate of suicide attempters from 4.3% to 2.7%. Probing questions also revealed 0.8% false negatives. We recommend using probing questions with both those who report a suicide attempt and those who do not report a suicide attempt to increase the validity of self-reported suicide-related information.     

The Brief Aggression Questionnaire: Structure,Validity, Reliability,and Generalizability

In contexts that increasingly demand brief self-report measures (e.g., experience sampling, longitudinal and field studies), researchers seek succinct surveys that maintain reliability and validity. One such measure is the 12-item Brief Aggression Questionnaire (BAQ; Webster et al., 2014), which uses 4 3-item subscales: Physical Aggression, Verbal Aggression, Anger, and Hostility. Although prior work suggests the BAQ's scores are reliable and valid, we addressed some lingering concerns. Across 3 studies (N = 1,279), we found that the BAQ had a 4-factor structure, possessed long-term test–retest reliability across 12 weeks, predicted differences in behavioral aggression over time in a laboratory experiment, generalized to a diverse nonstudent sample, and showed convergent validity with a displaced aggression measure. In addition, the BAQ's 3-item Anger subscale showed convergent validity with a trait anger measure. We discuss the BAQ's potential reliability, validity, limitations, and uses as an efficient measure of aggressive traits.     

Factor analysis of the cognitive-somatic anxiety questionnaire

The Cognitive-Somatic Anxiety Questionnaire (CSAQ; Schwartz, Davidson, & Goleman, 1978) is a brief self-report rating scale designed to differentiate between cognitive and somatic symptoms of anxiety. The CSAQ gained relatively widespread acceptance as a behavioral assessment instrument before much was known about its psychometric properties. Psychometric studies of the CSAQ (e.g., DeGood & Tait, 1987) have raised questions about the instrument's item content and subscale structure. The primary purpose of the present study was to determine whether the Cognitive and Somatic subscales correspond to the instrument's factorial structure in a clinically anxious population. The sample consisted of 120 nonpsychotic psychiatric inpatients. Factor analysis with oblique rotation accounted for 60% of the total variance and yielded four factors that do not conform to the Cognitive-Somatic subscale structure. The results challenge the validity of the CSAQ and suggest the need for an improved instrument capable of differentiating the multidimensional features of clinical anxiety.     

The Italian Five Facet Mindfulness Questionnaire: A Contribution to its Validity and Reliability

Mindfulness refers to the ability to be aware of and attentive to internal and external events, without becoming overwhelmed or feeling the need to react automatically. Recent studies indicate that this ability is associated with the following factors: mental health, psychological functioning, and stress reduction. Although questionnaires have been developed to measure mindfulness, so far these have not been adapted for use within the Italian population. Thus, the aim of our studies was to investigate the reliability and validity of an Italian version of the Five Facet Mindfulness Questionnaire (FFMQ; Baer et al. Assessment, 13, 27–45, 2006). The FFMQ is a self-report measure that is based on a five-facet model (i.e., Observe, Describe, Act with Awareness, Nonjudge, and Nonreact). In the first study, internal consistency, factor structure, and concurrent validity analyses were carried out on a sample of 559 volunteers. The second study entailed test–retest analyses on a different sample of 43 students. The results showed that the Italian FFMQ: (a) has a similar factor structure to the original English version; (b) has good to excellent internal consistency as a whole (alpha?=?0.86) with sub-scale consistency ranging from 0.65 to 0.81, and test–retest stability for the total score being 0.71; (c) has good concurrent validity as demonstrated by significant correlations between the FFMQ scores and a number of self-report measures related to mindfulness. Our findings support both the adaptation of the FFMQ to the Italian context as well as the conceptualization of mindfulness in multi-faceted terms.     

Evaluation of the psychometric properties of a modified Positive and Negative Affect Schedule including a direction scale (PANAS-D) among French athletes

ObjectivesThe goal of these studies was to provide validity and reliability evidence of a modified Positive and Negative Affect Schedule (PANAS) including a direction scale (PANAS-D). Study 1 tested the validity and reliability of the PANAS-D to measure both intensity and direction of affects. Study 2 examined the relationships between direction of affects and selected variables (i.e., coping, attainment of achievement goals and sport satisfaction) by controlling for intensity of affects.MethodA total of 306 and 296 athletes (studies 1 and 2) completed the PANAS-D and other self-report questionnaires. Data were analysed with reliability, confirmatory factor analyses (study 1) and correlational analyses (studies 1 and 2).DesignCross-sectional with self-reported questionnaires.ResultsIn study 1, the 4-factor structure of the PANAS-D (intensity and direction of positive affect and negative affect) fitted the data adequately. Multiple-group CFAs showed that PANAS-D was partially invariant across the two measurement occasions (before and after competition). The patterns of relationships between PANAS-D, attainment of achievement goals and coping provided evidence for the criterion-related validity of the PANAS-D. In study 2, direction of positive affect and negative affect were associated with selected outcomes (i.e., coping, attainment of sport achievement goals, and/or sport satisfaction) after intensity of these affective states were held constant.ConclusionsThis study provided support for the reliability and validity of the PANAS-D (study 1) and the incremental validity of the direction of affective states (study 2), supporting the distinction between athletes' intensity and direction of affective states.     

A Modern Instrument for Evaluating Psychopathology--The Personality Assessment Inventory (Book)

The Splitting Index (SI), a self-report scale based on the writings of Kernberg (e.g., 1976) on self and object representations and the defense mechanism of splitting, was constructed. After development over the course of 6 pilot studies, the SI was validated through 2 further studies. FacTor analyses revealed a 24-item scale with three 8-item subscales, measuring the splitting of self, family, and others' images. The SI and its subscales were demonstrated to be internally consistent and stable over a 4-week period. Convergent validity was supported by significant correlations with measures of borderline and narcissistic personality disorders, self-image stability, self-esteem, depression, and negative affectivity. Discriminant validity was demonstrated by near-zero correlations with two measures of cognitive complexity. Contrary to predictions, the SI was significantly correlated with the Dogmatism Scale (Rokeach, 1960), a third measure of cognitive complexity. Research and clinical applications of the SI are discussed.     

Validity Concerns in the Measurement of Women’s and Men’s Report of Intimate Partner Violence

The measurement of intimate partner violence (IPV) has proven to be more complex than originally anticipated and content and construct validity need to be greatly improved for IPV assessment. For measurement of IPV in the United States, these validity issues range from providing the most accurate wording for the content domain to controversies as to when violent actions are counted (e.g., self-defense) or whether to include mild aggression (e.g., psychological conflict tactics) that may be more normative and not harmful. The three major forms of IPV (i.e., physical, sexual, and psychological abuse) have distinct validity issues and may require different modalities for assessment. Gender needs to be considered when establishing construct validity due to differences in the meaning of aggression, impacts of abuse, and even patterns of violence for women and men. External threats to validity include potential bias of self-report and motivations when reporting on a partner, discrepancies in couples’ reports, the influence of response styles, and design issues affecting reporting. Traditional methods used to establish validity for IPV scales are reviewed and critiqued. Recommendations for enhancing validity in IPV assessment are provided.     

THE EFFECTS OF THE QUESTION “WHAT IS THIS?” ON TACT-TRAINING OUTCOMES OF CHILDREN WITH AUTISM

Tact training is a common element of many habilitative programs for individuals with developmental disabilities. A commonly recommended practice is to include a supplemental question (e.g., “What is this?”) during training trials for tacts of objects. However, the supplemental question is not a defining feature of the tact relation, and prior research suggests that its inclusion might sometimes impede tact acquisition. The present study compared tact training with and without the supplemental question in terms of acquisition and maintenance. Two of 4 children with autism acquired tacts more efficiently in the object-only condition; the remaining 2 children acquired tacts more efficiently in the object + question condition. During maintenance tests in the absence of the supplemental question, all participants emitted tacts at end-of-training levels across conditions with no differential effect observed between training conditions.Key words: autism, language training, stimulus control, tacts, verbal behaviorSkinner (1957) defined the tact as a response “evoked by a particular object or event or property of an object or event” (p. 82) and considered it to be one of the most important verbal operants. Tacts are maintained by generalized social reinforcement and, thus, they are central to many social interactions. For example, the tact “That cloud looks like a horse” (under the control of a visual stimulus) could evoke a short verbal interaction about the sky or horses. The tact “My tummy hurts” (under the control of an interoceptive stimulus) could evoke soothing statements from a parent. A child who tacts “doggie” in the presence of a cat likely would evoke a correction statement from an adult, further refining two stimulus classes (i.e., dog and cat). These examples illustrate that, despite their topographical differences, the tact relations share antecedent control by a nonverbal discriminative stimulus (S^D) and are maintained by generalized social reinforcement.In habilitative programs for individuals with language impairments, autism, and intellectual disabilities, tacts often are taught for objects (e.g., ball), object features (e.g., color, size, shape), activities (e.g., jumping), prepositions (e.g., between), and emotions (e.g., sad) among others. Although conceptualized differently among therapeutic approaches, the tact relation occupies a central position in many early-intervention curricula. For example, Lovaas (2003) and Leaf and McEachin (1999) describe these relations as expressive labels and recommend that they be taught early in language training using three-dimensional objects accompanied by the supplemental questions “What is it?” or “What''s this?” Alternatively, Sundberg and Partington (1998) explicitly refer to the relation as a tact and recommend beginning instruction by including the question “What is it?” before eventually fading the question. In addition to these clinical manuals, the use of supplemental questions during tact training has appeared in some empirical studies on tact or expressive-label training (e.g., Braam & Sundberg, 1991; Coleman & Stedman, 1974), but not others (e.g., Williams & Greer, 1993). Regardless of whether tact training initially includes supplemental questions prior to response opportunities, tacts ultimately should be emitted readily under the sole control of the nonverbal S^D as well as when it happens to be accompanied by a question.Conceptually, at least four potential problems could arise from introducing supplemental questions early and consistently in tact training. First, the acquired responses might not be emitted unless the question is posed (i.e., prompt dependence). This problem would lead to few spontaneous tacts occurring outside the explicit stimulus control of the training environment. Williams and Greer (1993) compared comprehensive language training conducted under the stimulus control specified in Skinner''s (1957) taxonomy of verbal behavior to a more traditional psycholinguistic perspective with supplemental questions and instructions embedded within trials. For all three adolescents with developmental disabilities, the targets taught from the verbal behavior perspective were maintained better in natural contexts than those taught from the psycholinguistic perspective. However, because data were not reported for each individual verbal operant, it is unclear what specific impact their tact-training procedures had on the outcomes.The second potential problem is that the supplemental question might acquire intraverbal control over early responses and interfere with the acquisition of subsequent tact targets. For example, Partington, Sundberg, Newhouse, and Spengler (1994) showed that the tact repertoire of a child with autism had been hindered by prior instruction during which she was asked “What is this?” while being shown an object. The supplemental question subsequently evoked previously acquired responses and blocked the ability of new nonverbal S^Ds (i.e., objects) to evoke new responses. Partington et al. then showed that new tacts were acquired by eliminating the supplemental question from instructional trials.The third potential problem is that learners might imitate part of or the entire supplemental question prior to emitting the target response (e.g., “What is it” → “What is it … ball.”). For example, Coleman and Stedman (1974) demonstrated that a 10-year-old girl with autism imitated the question “What is this?” while being taught to label stimuli depicted in color photographs. Such an outcome results in a socially awkward tact repertoire and requires additional intervention to remedy the problem.Finally, including supplemental questions during tact training might impede skill acquisition, perhaps via a combination of the problems described earlier. Sundberg, Endicott, and Eigenheer (2000) taught sign tacts to two young children with autism who had prior difficulty acquiring tacts. In one condition, the experimenter held up an object and asked, “What is that?” In the comparison condition, the experimenter intraverbally prompted the participant to “sign [object name]” in the presence of the object. Sundberg et al. demonstrated substantially more efficient tact acquisition under the sign-prompt condition than when the question “What is that?” was included in trials; the latter condition sometimes failed to produce mastery-level responding.Teaching an entire tact repertoire while including supplemental questions (e.g., “What is it?”) during training trials could produce a learner who is able to talk about his or her environment only when asked to do so with similar questions. To the extent that this is not a therapist''s clinical goal, teaching the tact under its proper controlling variables may eliminate such problems. Of course, inclusion of supplemental questions during the early phases of language training could be faded over time such that the target tact relation is left intact prior to the end of training (Sundberg & Partington, 1998). However, the aforementioned studies have documented problems with using supplemental questions during tact training. Given the ubiquity of tact training in habilitation programs, the numerous problems that may arise when supplemental questions are included in training trials, and the limited research on the topic, further investigation is warranted. Thus, the purpose of the present study was to compare directly the rate of acquisition and subsequent maintenance of tacts taught using only a nonverbal S^D (i.e., object only) with tacts taught using a question (“What is this?”) in conjunction with the nonverbal S^D (i.e., object + question). The present study extends earlier research by examining both acquisition and maintenance and by including individuals with no prior history of formal tact training.     

Face Management,Question Wording,and Social Desirability1

Five experiments were conducted to examine the impact of question wording manipulations derived from face management theory (Brown & Levinson, 1987) on responses to survey questions. In general, it was expected that questions phrased so as to allow the respondent to maintain face while answering in a socially undesirable manner would result in lower rates of socially desirable responding than would control questions. The results strongly supported this hypothesis for questions regarding socially desirable knowledge (e.g., Are you familiar with NAFTA?), but not for questions about socially desirable behavior (e.g., Did you vote?). The results were partially supportive for questions about socially undesirable behaviors (e.g., Have you ever shoplifted?).     

Scale Imposition as Quantitative Alchemy: Studies on the Transitivity of Neuroticism Ratings

It is common practice in psychology to devise “measurement” procedures by imposing rating scales (e.g., Likert items) onto phenomena and treating the values they produce as quantities. The validity of these procedures goes untested. Validity checks are instead performed on sets of these measurement procedures (i.e., multi-item scales). We present results from three studies suggesting that people cannot be assumed to preserve transitivity when comparing themselves and others on NEO Neuroticism-domain trait items. As transitivity is one of the fundamental axioms of quantitative measurement, these studies challenge the validity of Neuroticism scales at the level of individual scale items.     

Depression and Contributors to Vocational Satisfaction in Roman Catholic Secular Clergy

A nationally selected, random sample of Roman Catholic secular (i.e., diocesan) priests was examined using the Center for Epidemiological Studies-Depression scale and an instrument developed for this study to assess contributors to priests' vocational satisfaction. In addition, a self-report inventory gathered information regarding participants' demographics as well as four categories of predictor variables (i.e., overall level of vocational satisfaction, social support, spiritual activities, physical environment). The study yielded a response rate of 45%. Secular clergy reported rates of depression approximately seven times greater than are found in the general population, and also indicated that the recent sexual abuse scandal in the Roman Catholic church had negatively affected their mood. Priests' engagement in sacramental activities contributed greatly to their vocational satisfaction, and low levels of vocational satisfaction were found to be most predictive of depression. Factors comprising priests' vocational satisfaction were External Manifestations (e.g., preaching, teaching), Internal Manifestations (e.g., prayer life, affirmation of God's call), and Social Manifestations (e.g., relationships with parishioners, appreciation from others).     

Constraints on children's color term acquisition

Kowalski and Zimiles (2006) and O'Hanlon and Roberson (2006) address an age-old question: Why do children find it difficult to learn color terms? Here these articles are reflected on, providing a focused examination of the issues central to this question. First, the criteria by which children are said to find color naming difficult are considered. Although the age of color term acquisition is decreasing, and color naming might not be more difficult than other abstract attributes, several stages of difficulty are identified. Second, it is argued that there are potentially multiple constraints (e.g., conceptual, attentional, and linguistic) for these multiple stages of difficulty with color term acquisition. Third, it is argued that the validity and reliability of techniques for identifying constraints need to be considered and that converging evidence for the constraints should be provided. Finally, a series of new questions that need to be asked to provide a well-rounded explanation of the difficulties children face when learning color terms is outlined.