QUAID: A questionnaire evaluation aid for survey methodologists

QUAID (question-understanding aid) is a software tool that assists survey methodologists, social scientists, and designers of questionnaires in improving the wording, syntax, and semantics of questions. The tool identifies potential problems that respondents might have in comprehending the meaning of questions on questionnaires. These problems can be scrutinized by researchers when they revise questions to improve question comprehension and, thereby, enhance the reliability and validity of answers. QUAID was designed to identify nine classes of problems, but only five of these problems are addressed in this article: unfamiliar technical term, vague or imprecise relative term, vague or ambiguous noun phrase, complex syntax, and working memory overload. We compared the output of QUAID with ratings of language experts who evaluated a corpus of questions on the five classes of problems. The corpus consisted of 505 questions on 11 surveys developed by the U.S. Census Bureau. Analyses of hit rates, false alarm rates,d′ scores, recall scores, and precision scores revealed that QUAID was able to identify these five problems with questions, although improvements in QUAID’s performance are anticipated in future research and development.     

Hiding valuables: A questionnaire study of mnemonically risky behavior

A survey investigated incidences where individuals intentionally hid objects but later forgot the item's location. Surveys from 111 younger and 99 older adults yielded the following prototypical scenario: the person hid the valuable 1 year ago to prevent theft, and waited 1 month before an initial 30-minute search, which included looking in dresser drawers plus 1.5 other locations. The person made three additional searches (total searches = 4) for the missing valuable over a 2 week period. Three-quarters of the hidden valuables were eventually found (usually by accident) about 1 month after hiding the object. About half of the time, the person had already searched the hiding area. Older adults typically hid jewellery from thieves, while younger adults hid money from friends and relatives. Older adults searched more often for longer periods of time for the hidden object, and took longer to eventually find the object, compared with younger adults.     

A multitrait-multimethod analysis of the preschool behavior questionnaire

The Preschool Behavior Questionnaire is a teacher rating instrument yielding a total score reflecting overall level of adjustment and three subscale scores labeled Hostile/Aggressive, Anxious/Fearful, and Hyperactive/Distractible. This study assessed the convergent and discriminant validity of this teacher rating measure against two alternative measures within a multitrait-multimethod analysis. The results supported the construct validity of the Hostile/Aggressive and Anxious/Fearful scores as well as the overall score. There was, however, no support for the validity of the Hyperactive/Distractible score.This research was supported, in part, by a grant from the Pickering Foundation. The authors would like to thank Maria Bhaneja for her help in the collection and coding of data. Thanks are also extended to the teachers and children who participated in the study.     

Patients' desire to participate in decision-making in psychiatry: a questionnaire survey in Japan

Japanese national sentiment has been described as paternalistic, which has potentially wide-ranging implications for the manner in which psychiatric patients should participate in medical decision-making. To examine the extent and possible determinants of the desire to participate in medical decision-making among Japanese people, we distributed a packet of questionnaires to 747 (nonmedical) university students and 114 of their parents. The questionnaires included an imaginary case vignette of psychotic depression. The participants were asked whether they would want various types of medical information, i.e., diagnosis, aetiology, treatment, outcomes, medical charts, etc., disclosed to them were they in such a psychiatric condition. Also included was the 1995 Scale for Independent and Interdependent Construal of the Self by Kiuchi. More than half of the participants wanted all the types of medical information disclosed to them. Those participants who wanted to have all types of information disclosed to them (n = 413) as compared to those who did not want to know at least one type of information (n = 445), tended to be male and to have an educational background in psychiatry (9.7% vs 5.4%) as well as an assertive attitude as indicated by a higher score on Independence on the Scale for Independence and Interdependent Construal of the Self. These results suggest that the Japanese in this sample are more likely to want to make an autonomous contribution to the psychiatric decision-making process and that less desire for information can be predicted by some demographic and personality factors.     

Assessment of abnormal personality in childhood: a Delphi survey of questionnaire data

As an investigation into the feasibility of recording personality status from questionnaire data in younger people we used a three phase Delphi survey to assess items from the Christchurch Health and Development Study, administered at ages 12 to 16 years. Twelve experts took part in Phase I, and 22 in Phases II and III, 16 of whom were experts in adult personality disorder (PD), and 6 were experts who work with children. In total, 189 questions (55% of the total (238) in the questionnaires) were identified as possibly being related to personality abnormality in one or more clusters with high consensus. Experts who work with children were less likely to label features as related to personality than experts in PD (p < 0.001), and the four personality factors (equivalent to Mulder and Joyce's antisocial, asocial, asthenic, and anankastic) chosen for assessment showed variable agreement. Confirmatory factor analysis showed the best fitting model of the data was a 3 factor solution involving asocial/asthenic, antisocial, and anankastic factors. This represents the first attempt to use existing recorded data to code personality status and the results of this Delphi survey give some grounds for optimism that this approach has potential in the early identification of personality features.     

Science and behavior therapy: A survey of research in the 1970s

A representative sample of a decade of contributors to the journals Behavior Therapy, Behaviour Research and Therapy, Journal of Applied Behavior Analysis and the Journal of Behavior Therapy and Experimental Psychiatry was asked to report on their motivations for conducting studies that had been published in that journal. Results indicated that the primary motivation of behavioral researchers was to build on prior research through gathering and analysis of data to illuminate applied-clinical issues and validate treatment procedures. Few researchers reported consultation arrangements or efforts to replicate as their motivation for research. Most respondents described the training model in which they had received their doctoral education as oriented toward a scientist-practitioner approach rather than one that focused primarily on research, therapy or assessment training. Most respondents noted that their philosophy of behavior therapy at the time of the study in question to be applied behavior analysis or social-learning theory with far less being affiliated with a neobehavioristic mediational S-R model or cognitive behavior modification. These results are discussed within the context of the relevance of clinical research in behavior therapy and professional psychology.     

Gender differences in math and english achievement: A longitudinal study

This study investigated gender-related developmental trends in math and English achievement as measured by classroom grades and standardized test scores. The sample consisted of 30 males and 30 females followed from 6th through 12th grade. Previous research suggests that developmental trends in male and female achievement may differ as a function of assessment context. Thus, it was hypothesized that female math and English classroom grades would remain stable over time but that achievement test scores in both subject areas would decline. For males, it was hypothesized that performance in both assessment contexts would remain stable. Results confirmed these hypotheses. Results also suggest that school-related responsibility is related to performance in both assessment contexts for females but not for males. The need to consider personal goals when assessing differences in academic ability and performance capabilities is discussed.     

A psychometric assessment of a two-factor solution for the preschool behavior questionnaire in mid-childhood

In this study, we examined Fowler and Park's (1979) two-factor solution (aggressive-hyperactive-distractible and anxious-fearful) for the Preschool Behavor Questionnaire (PBQ; Behar & Stringfield, 1974) in a sample of 179 (93 males, 86 females) grade 1 and 155 (73 males, 82 females) grade 2 children. The validity of this two-component solution was assessed by relating the PBQ factor to indices of peer popularity and observations and peer nominations of aggressive, sociable, and isolative/fearful behavior. Popularity was measured using a rating scale sociometric. Behavioral data were obtained by observing children at free play and recording the frequency of aggression, withdrawal, and sociability. Peer assessments of children's social behaviors were obtained using The Revised Class Play (Masten, Morison, & Pellegrini, 1985). A two-factor solution of the PBQ resulted in the identification of an externalizing (EXT) and an internalizing (INT) factor. These data replicated earlier findings employing younger children (e.g., Fowler & Park, 1979). Each child's EXT and INT score was then correlated (with age partialled out) with his/her scores on each of the observational and peer assessment variables. Teacher ratings of EXT were significantly related to indices of aggression and unpopularity. The teacher ratings of INT were related to measures of anxiety, withdrawal, and unpopularity. These results suggest that the two-factor solution employed herein is an economical and viable indicator of social maladjustment in samples of “normal” elementary school-aged children.     

Hyperactive and aggressive behaviors in childhood: Intertwined dimensions

The relationship between aggression and hyperactivity is unclear in childhood hyperkinesis. To examine the relationship between the two dimensions, a sample of 109 first-, second-, and third grade children who were rated as hyperactive were evaluated daily by their teachers for 12 consecutive school days on the Daily Behavior Checklist. Daily recordings were made on a total of 22 specific behaviors, 11 physically or verbally aggressive acts, and 11 acts of a restless or hyperactive nature. The Conners hyperactivity score correlated. 47 with daily aggression and .49 with daily hyperactivity. A subgroup of hyperactive children who exhibited both hyperactive and aggressive behaviors at rates greater than 98% of their classmates was selected. These children were rated significantly higher on the Conners scale than an alternate subgroup of hyperactive children who showed high rates of hyperactive but not aggressive behaviors. In view of heavy reliance on teacher ratings with the Conners scale in studies of hyperactivity, careful consideration of the potential confounding of hyperactivity and aggression is recommended. Implications for classification, longitudinal investigation, and evaluation of treatment of hyperactive children were discussed.     

Psychometric evaluation of the dating anxiety survey: A self-report questionnaire for the assessment of dating anxiety in males and females

The Dating Anxiety Survey (DAS) was constructed to assess dating anxiety in males and females. Factor analysis of the survey revealed three factors: passive contact, active intentions for dating, and dating interactions. The reliabilities of the three subscales, as determined by coefficient alpha, were .87, .91, and .93 for males and .90, .90, and .92 for females, respectively. Correlations with dating history and a measure of social anxiety were generally of a low but significant magnitude, providing some support for concurrent validity. The results of the factor analysis lend support to the construct validity of the DAS. These findings suggest that the DAS is a potentially useful instrument in the self-report of dating anxiety.Portions of this paper were presented at the meeting of the Southeastern Psychological Association, Atlanta, GA, March 1983.     

A self-reinforcement questionnaire

The hypothesis that the tendency to engage in self-reinforcement behavior is measurable by a 30-item true-false questionnaire is supported using self-monitoring as the validity criterion. The use of the self-reinforcement questionnaire has been supported previously by correlating questionnaire scores with frequency of self-reinforcement during simple experimental tasks. In the present study, 53 undergraduates monitored self-reinforcement behavior for 28 days and frequency of self-reinforcement correlated significantly with questionnaire scores as well as experimenter ratings of the tendency to engage in self-reinforcement. The results suggest that the questionnaire is a promising tool for the measurement of self-reinforcement in meaningful settings. The need for further validational evidence is noted and the potential usefulness of the questionnaire in testing hypotheses regarding the relation between self-reinforcement and depression is discussed.     

A survey os school psychologists'' use of behavior modification procedures

A national survey of behavior modification procedures used by school psychologists is reported. Usable questionnaires were received from 148 (49%) of 300 randomly selected members of the 1982 National Association of School Psychologists directory. Each respondent was asked to indicate the type of exposure, use, and inclusion of data collection procedures for each of 18 commonly used behavioral techniques. Conditional probabilities between type of exposure and use found strong relationships between supervision and subsequent use. The implications for the field of school psychology and future directions for training programs are discussed.