REPLICATION IN EXPERIMENTAL COMMUNICATION RESEARCH: AN ANALYSIS

This study proposes a taxonomy of replication designs and applies it to a representative sample of experimental communication research reports in Communication Monographs and Human Communication Research. Results of the analysis indicate that publishability and reporting practices hamper our ability to identify replication studies. The implications of these findings are discussed, and several suggestions for improving the situation are offered.     

POTENTIAL VERSUS ACTUAL FAKING OF A BIODATA FORM: AN ANALYSIS ALONG SEVERAL DIMENSIONS OF ITEM TYPE

Two studies examined faking of a 25-item biodata questionnaire. The first study investigated potential and actual faking of the form using three groups: a group told to make themselves look as good as possible, a group told to complete the form honestly, and a group completing the instrument in a real selection situation. Subjects were 58 current employees and 231 job applicants. Results indicated that subjects could fake the instrument when instructed to do so. Also, some faking appeared to be occurring in practice, although results depended upon the composition of the comparison group. Only eight items appeared to be fakable, and only three of these seemed to be faked in practice. In Study 2, 26 business majors rated the biodata items on eight dimensions of item type. Results showed that the three items faked in practice were less historical, objective, discrete, verifiable, and external than other items, and were more job relevant.     

AN APPOINTMENT-KEEPING IMPROVEMENT PACKAGE FOR OUTPATIENT PEDIATRICS: SYSTEMATIC REPLICATION AND COMPONENT ANALYSIS

Child health-care appointments that are not kept are an important pediatric problem. Previous research has shown that reducing effort (with a parking pass) and reminding patients (with mailed and telephone reminders) significantly improved appointment keeping for first-time and patient-scheduled appointments. This study, using a posttest-only group design, evaluated the effects of various combinations of that intervention applied to clinic-scheduled follow-up appointments. All combinations of the intervention significantly increased cancellations, but none increased appointments kept or decreased appointments not kept significantly. Log linear analyses showed that the lag time between scheduling and the appointment significantly influenced appointment keeping. The results suggest that if clinics want to increase cancellations, a mailed reminder and effort reduction are sufficient. To increase appointment keeping, other interventions, such as reduced lag time, may be necessary.     

EFFECT OF SITUATIONAL INTERVIEWS, CONVENTIONAL STRUCTURED INTERVIEWS, AND TRAINING ON INTERVIEW RATING AGREEMENT: AN EXPERIMENTAL ANALYSIS

The present study examines the degree to which agreement in interviewer ratings may be influenced by training, use of structured conventional interviews, or situational interviews. Results of an experimental study conducted among 42 managers who were experienced as interviewers revealed no training effect on rating agreement, but the impact of the situational format on consistency in assessments of applicant suitability was significant. Implications of these findings for usefulness of situational employment interviews and future research investigating the reliability of situational interview formats are discussed.     

AN EXPERIMENTAL REPLICATION OF THE DEPRESSED-ENTITLEMENT EFFECT AMONG WOMEN

Previous research has suggested that women pay themselves significantly less than men pay themselves for the same amount of work (Callahan-Levy & Messé, 1979; Major, McFarlin, & Gagnon, 1984). In an experimental study involving 132 participants (68 men and 64 women), this "depressed-entitlement effect" was replicated in a current sample of university students. Independent judges unaware of participant gender perceived no differences in quality between products of men and women, indicating that the two groups' efforts did not differ in objective terms. Results are interpreted in terms of a general system-justification framework (lost & Banaji, 1994), according to which members of disadvantaged groups internalize ideological justifications for their own disadvantage.     

A COMPARISON OF HOLISTIC VERSUS DECOMPOSED RATING OF POSITION ANALYSIS QUESTIONNAIRE WORK DIMENSIONS

Recent studies have attempted to reduce the cost and intrusiveness of the Position Analysis Questionnaire (PAQ) by limiting the amount of information provided to the analyst, with consistently negative results. We examined an alternative technique for improving the cost-effectiveness of the PAQ that avoids the need to rate the hundreds of items that constitute the instrument. Three groups of raters (professional job analysts, graduate students in industrial psychology who were familiar with the PAQ, and PAQ-unfamiliar undergraduates) made direct holistic ratings of the PAQ dimensions for four familiar jobs. The holistic ratings were compared with decomposed PAQ dimension profiles obtained from the item-level ratings of the professional analysts. Cronbach accuracy analyses indicated near-zero convergence between the holistic and decomposed dimension ratings, even for the professional PAQ job analysts. We conclude that holistic rating of dimensions is not an effective means of reducing the cost of a PAQ job analyses and that it is likely to be similarly ineffective with task- or ability-based instruments.     

THE INFLUENCE OF DIFFERENTIAL PRE-READING INSTRUCTIONS ON CONSTRUCTING DIFFERENT DIMENSIONS OF SITUATIONAL REPRESENTATIONS AND PROBLEM-SOLVING IN FL NARRATIVE TEXT COMPREHENSION

This article reports two experiments that address effects of differential pre-reading instructions: 1) IP (instruction to monitor procedural representations) and IN (no instruction); and 2) IS (instruction to monitor spatial representations) and IN, on subsequent situational representations construction and problem-solving in FL narrative text comprehension. While the structure of IP was consistent with the text, the structure of IS was not. The results from Experiment 1 suggest that IP helps readers with low FL proficiency to construct situational representations, but is of little help to solve an inference problem. In addition, readers with low proficiency are hindered from comprehending a text by IS. Experiment 2 suggests that for readers with higher (high-intermediate) FL proficiency, IP had little effect on constructing situational representations and had a negative effect on solving an inference problem. IS, on the other hand, enhances these readers' construction of situational representations and helps their problem-solving. Thus, language proficiency mediates the effects of prereading instructions on the subsequent text comprehension.     

ASSESSMENT AND TRAINING OF CLINICAL INTERVIEWING SKILLS: ANALOGUE ANALYSIS AND FIELD REPLICATION

Two studies were conducted to assess and train clinical interviewing skills. In Experiment 1, eight university practicum students (“therapists”) and either role played or volunteer “clients” were audiotaped during simulated interviews. Following the collection of baseline data on both therapist and client responses, training was provided by way of written materials, classroom instruction and practice, and quizzes. Results of a multiple baseline design across subjects showed improvements in therapists' interviewing skills and subsequent increases in client responding. Experiment 2 replicated and extended the research to a hospital outpatient clinic, in which therapists interviewed the parents of children with behavior problems. In addition, four months following the completion of Experiment 2, follow-up data collected during a maintenance condition showed continued high levels of therapist and client behavior. Finally, a panel of expert peers indicated that each response category was judged highly relevant to the behavioral assessment process.     

A PRACTITIONER-BASED ANALYSIS OF INTERVIEWER JOB EXPERTISE AND SCALE FORMAT AS CONTEXTUAL FACTORS IN SITUATIONAL INTERVIEWS

Since the first published Situational Interview (SI) study (Latham, Saari, Pursell, & Campion, 1980), research has shown practical and psychometric support for the usefulness of this behavioral interview method. However, such studies have often failed to distinguish the effects of "interview context" factors, such as the SI's behaviorally anchored scale and the use of job expert interviewers on SI ratings. To aid HR managers interested in adopting a behavioral interview system, this study examined the contributions of the SI's behaviorally anchored scale and the interviewer's job expertise to the interrater agreement and accuracy of ratings of situational questions. Two police samples (job content experts) and a student sample (naive raters) showed that ratings of videotaped interviews for police sergeant/lieutenant positions based on the SI scale were significantly superior to those gained using a more traditional rating format, and that job experts did not produce better ratings than naive raters.     

INCREASING INDICES OF HAPPINESS AMONG PEOPLE WITH PROFOUND MULTIPLE DISABILITIES: A PROGRAM REPLICATION AND COMPONENT ANALYSIS

We replicated a program to increase indices of happiness among people with profound multiple disabilities and conducted a component analysis of the program. The program involved presentation and contingent withdrawal of stimuli that had been identified as preferred based on preference assessments and staff opinion. The program was implemented with 3 adult students and was accompanied by increased happiness indices for each student. Subsequent implementation of the two types of stimuli indicated that preferred stimuli based on preference assessments were more consistently accompanied by increased happiness indices than were preferred stimuli based on staff opinion. Social validity measures supported the definition of happiness, in that raters' subjective opinions of the students' happiness coincided with the observed happiness indices. Results are discussed regarding the importance and practical implications of using preference assessments for determining stimuli for increasing happiness indices. Future research areas are suggested, focusing on increasing happiness and other quality-of-life indicators during the daily routines of people with profound disabilities.     

情境兴趣的研究进展

兴趣是一种重要的非智力因素,受到研究者的普遍关注。随着对兴趣的深入研究,研究者将兴趣分为个体兴趣和情境兴趣。其中,情境兴趣因其可被人为地控制和改变而成为国内外教育界的研究重点和热点。本文从情境兴趣的概念、分类、维度及影响因素等方面对情境兴趣的研究成果进行综述。     

FURTHER STUDIES OF SITUATIONAL TESTS

In the present paper, results are reported for two different situational judgment tests (SJT) used in validation studies with almost 4,000 employees in 7 different organizations. Across the 2 studies, it was shown that situational test scores were significantly related to performance (weighted average r = .19), cognitive ability (weighted average r = .45), and experience (weighted average r = .20). In one study, there was a slight tendency for experience and cognitive ability to interact in the prediction of situational judgment, such that cognitive ability became less predictive as experience increased. Situational judgment fully mediated the effects of cognitive ability in one study, but not in the other. Finally, SJT race effect sizes were consistent with past research and, while not trivial, smaller than those typically observed for cognitive ability tests. The studies yielded very similar results despite the use of different keying methods (empirical vs. subject matter expert opinion). The preponderance of the evidence indicates that situational judgment measures mediate avariety of job relevant skills. Limitations to these studies and directions for future research are reviewed.     

DIMENSIONS OF WORK OUTCOMES: A MULTIDIMENSIONAL SCALING APPROACH

There exists a need for basic research on the nature of work outcomes, to facilitate integration of theory and results and to aid in the sampling of outcomes for research and evaluation. Existing research, primarily using factor analysis, has led to a hierarchical model, with numerous sub-factors grouped under the major factors of "intrinsic" and "extrinsic." Recent research, however, has shown that industrial/organizational psychologists do not agree on the definition of intrinsic and extrinsic. It is argued that a more useful approach is a dimensional model, which allows each work outcome to be described by a set of values on multiple dimensions. A multidimensional scaling analysis of 21 outcomes yields a latent structure with three dimensions: value attached to the outcome by society, level of psychological need met, and extent to which the outcome is inherent in the work itself. Implications for the sampling of outcomes and integration of different literature are discussed. The results suggest that a multidimensional model of work outcomes is more useful than a categorical approach built on the intrinsic/extrinsic dichotomy.