REPLICATION IN EXPERIMENTAL COMMUNICATION RESEARCH: AN ANALYSIS

This study proposes a taxonomy of replication designs and applies it to a representative sample of experimental communication research reports in Communication Monographs and Human Communication Research. Results of the analysis indicate that publishability and reporting practices hamper our ability to identify replication studies. The implications of these findings are discussed, and several suggestions for improving the situation are offered.     

A Bibliometric Analysis of Communication Journals from 2002 to 2005

Journal impact ratings are often used by authors, promotion/hiring committees, and grant review teams as a proxy for scholarship quality. Journal citation data (2002–2005) from Social Sciences Citation Index were used to rank journals in the field of communication. A journal relatedness algorithm was applied to ascertain the 19 semantically related journals in communication. The mean journal impact index was 0.77 (SD= 0.28). Human Communication Research (HCR), Personal Relationships, Journal of Communication (JOC), and Communication Research (CR) were ranked the top four journals for the study years examined. Network analysis was conducted on in‐degree (i.e., citations to journals) and out‐degree (i.e., citations from journals) data for the 19 communication journals for 2003–2005. The purpose of the network analysis was to study the citation patterns among journals in the field of communication. Data using degree centrality indicate that Communication Monographs, CR, HCR, and JOC (in alphabetical order) are the four most central journals in the field.     

THE VALIDITY OF COMMUNICATION EXPERIMENTS USING HUMAN SUBJECTS: A REVIEW

Sixty-eight experiments published in 1973 and 1974 in the Journal of Communication, Speech Monographs, and Human Communication Research were analyzed. It was concluded that: reports of communication experimentation rarely provide sufficient information to allow critical evaluation of crucial aspects of validity: the validity of communication experimentation may be severely limited by the nature of subjects studied and by failures to eliminate, account for, or acknowledge experimental reactivity.     

A REASSESSMENT OF STATISTICAL POWER ANALYSIS IN HUMAN COMMUNICATION RESEARCH

This study conducted a statistical power analysis of 64 articles appearing in the first four volumes of Human Communication Research, 1974–1978. Each article was examined, using Cohen's revised handbook, assuming nondirectional null hypotheses and an alpha level of .05. Statistical power, the probability of rejecting a false null hypothesis, was calculated for small, medium, and large experimental effect sizes and averaged by article and volume. Results indicated that the average probability of beta errors appears to have decreased over time, providing a greater chance of rejecting false null hypotheses, but this also raised several power-related issues relevant to communication research in general.     

THE ALPHA PERCENTAGE AND EXPERIMENTWISE ERROR RATES IN COMMUNICATION RESEARCH

Experimentwise error rates of the type proposed by Ryan (1959) are discussed and contrasted with anew measure of the likelihood that the results of a series of significance tests are Type I errors. This new measure, the Alpha Percentage (a%), shares the advantages of experimentwise error rates over individual alpha levels in reducing Type I errors in communication research, but the Alpha Percentage has much greater power than currently used experimentwise error rates to detect significant effects. Four arguments against the use of experimentwise error procedures are discussed and EW, EP, and a% rates are reported for Communication Monographs and Human Communication Research.     

The Use of Exploratory Factor Analysis and Principal Components Analysis in Communication Research

Exploratory factor analysis is a popular statistical technique used in communication research. Although exploratory factor analysis (EFA) and principal components analysis (PCA) are different techniques, PCA is often employed incorrectly to reveal latent constructs (i.e., factors) of observed variables, which is the purpose of EFA. PCA is more appropriate for reducing measured variables into a smaller set of variables (i.e., components) by keeping as much variance as possible out of the total variance in the measured variables. Furthermore, the popular use of varimax rotation raises some concerns about the relationships among the factors that researchers claim to discover. This paper discusses the distinct purposes of PCA and EFA, using two data sets as examples to highlight the differences in results between these procedures, and also reviews the use of each technique in three major communication journals: Communication Monographs, Human Communication Research, and Communication Research.     

CONTINUOUS RECORDING AND INTEROBSERVER AGREEMENT ALGORITHMS REPORTED IN THE JOURNAL OF APPLIED BEHAVIOR ANALYSIS (1995–2005)

We reviewed all research articles in 10 recent volumes of the Journal of Applied Behavior Analysis (JABA): Vol. 28(3), 1995, through Vol. 38(2), 2005. Continuous recording was used in the majority (55%) of the 168 articles reporting data on free‐operant human behaviors. Three methods for reporting interobserver agreement (exact agreement, block‐by‐block agreement, and time‐window analysis) were employed in more than 10 of the articles that reported continuous recording. Having identified these currently popular agreement computation algorithms, we explain them to assist researchers, software writers, and other consumers of JABA articles.     

Book Reviews

Book reviewed in this article: Against Therapy: Emotional Tyranny and the Myth of Psychological Healing, by Jeffrey M. Masson. Schools for the 21st Century, by Phillip C. Schlechty. Adlerian Counseling: A Practical Approach for a New Decade (3rd ed.), by Thomas J. Sweeney. Muncie, Handbook of Racial/Ethnic Minority Counseling Research, by Joseph G. Ponterotto and J. Manuel Casas. Springfield, IL: Charles C Thomas, Human Change Processes, by Michael J. Mahoney.     

A REVIEW OF ACADEMIC PERMANENT-PRODUCT DATA COLLECTION AND RELIABILITY PROCEDURES IN APPLIED BEHAVIOR ANALYSIS RESEARCH1

Three publication sources were reviewed to determine the recent conventions for collecting, and assessing the reliability of, academic permanent-product data (handwriting, examination papers, etc in applied behavior analysis. The primary source was the Journal of Applied Behavior Analysis (1968–1974). Secondary sources included conference proceedings titled, A new direction in education: behavior analysis (E. Ramp and B. L. Hopkins, Eds., Lawrence, Kansas: Support and Development Center for Follow Through, Department of Human Development, University of Kansas, 1971), and Behavior Analysis and Education (G. Semb, Ed., Lawrence, Kansas: Support and Development Center for Follow Through, Department of Human Development, University of Kansas, 1972). Finally, as a test of the generality of the findings in the two applied behavior analysis sources, the current issue of each of 14 psychological and/or educational journals was reviewed. Thirty JABA studies reported academic permanent-product data, but only 14 reported reliability. Increasingly more product data through 1973 have been reported along with a greater proportion of authors reporting reliability. The review of the two conference proceeding publications revealed the same trend. In 1971, only three studies reported academic product data, none with reliability, while in 1972, 15 reported academic data, nine including reliability assessment. The review of 14 current education/psychology journal issues revealed four studies reporting academic data, none with reliability. Across all sources, about one-half of the studies reported reliability. Most of the studies reporting reliability described the frequency of reliability assessment, with approximately equal numbers of JABA studies reporting reliability for each paper or reliability for each session. The use of uninformed observers was reported in only three JABA studies and one conference study. Marks made on subjects' papers by either the teacher or the primary observer were reported masked for reliability purposes by only two JABA and two conference studies. Reliability was calculated on a session total basis in two JABA studies. Point-by-point agreement was given in nine JABA and three conference studies. Perfect reliability (mean agreement of 100%) was reported in only six JABA and three conference studies. Scores between 90 and 100% were reported in nine JABA and four conference studies. Scores below 80% were reported in three JABA studies. No other percentage agreement scores were reported, although one JABA study reported correlational reliability (Pearson r). In summary, recently more studies have dealt with academic data and, until 1974, a greater proportion of these studies reported reliability assessment, and relatively few studies reported either replicable methods, 100% agreement, or controls for maintaining rater independence.     

A reappraisal of the relationship between span memory and intelligence via “best evidence synthesis”

This paper examines the relationship between span memory [e.g., immediate memory, short-term memory (STM), simple span] and general ability (g) though a reanalysis of two data sets [Christal, R. E. (1959). Factor analytic study of visual memory. Psychological Monographs: General and Applied, 72 (13, Whole No. 466); Kelley, H.P. (1964). Memory abilities: A factor analysis. Psychometric Monographs, No. 11]. Because of their large sample sizes and the multiple measures used to identify each construct, the Christal and Kelley studies were examined within a “best evidence synthesis” framework. Modern structural equation modeling (SEM) techniques were used to examine the relationship between immediate memory and g. Results indicated that in both studies, the relationship between immediate memory and g was quite substantial (.71 and .83), and that this relationship was essentially reduced by half when the common content variance of the tests was accounted for (e.g., verbal, spatial, numerical). Results are discussed within the context of recent research examining the relationship between working memory (WM) and g.     

Book Reviews

Book reviewed in this article: The Handbook of Psychological Testing, by Paul Kline. International Dimensions of Human Resource Management, by Peter J. Dowling, Randall S. Schuler and Denice E. Welch. The Culture Factor: Corporate and International Perspectives, by the Institute of Personnel and Development (research and analysis by Angela Baron and Mike Walters). Leadership in Organizations, by Gary Yukl. Employee Relations in Europe, by Jeff Bridgford and John Stirling. Communication — Rewarding People; The Skill of Responding Positively, by D. Dickson, C. Saunders and M. Stringer. Management Skills: Making the Most of People, by Robin Evenden and Gordon Anderson. Human Resource Management – Issues and Strategies, by Rosemary Harrison.     

A Review of Spiritual and Religious Measures in Nursing Research Journals: 1995--1999

Background: A series of systematic reviews has revealed relatively high levels of interest in religion and spirituality in different nursing specialties, but not in general nursing research journals. Purpose: To identify the extent to which spirituality and religiousness were measured in all quantitative and qualitative research articles published in Research in Nursing and Health, Nursing Research, Advances in Nursing Science (ANS), and Image: The Journal of Nursing Scholarship from 1995 to 1999. Methods: A full-text search was conducted of ANS and Image using the Ovid search system. Nursing Research and Research in Nursing and Health were hand searched for spiritual/religious measures. Characteristics of selected studies, the measures taken, and their uses were coded for data analysis. Results: A total of 564 research studies were identified, of which 67 (11.9%) included at least one measure of spirituality or religiousness. A significant difference was found between the percentage of qualitative and quantitative studies that contained measures of these concepts. Of the 119 qualitative studies, 23 (19.3%) contained a measure of religion or spirituality, compared to 44 of the 445 (9.9%) quantitative studies. Nominal indicators of religious affiliation were the most commonly used measures in the quantitative studies and measures of religion and spirituality were rarely used in the analyses. Although only a few quantitative or qualitative studies intended to focus on religion or spirituality, these themes often emerged spontaneously in the qualitative research. Conclusions: Research in Nursing and Health, Advances in Nursing Science, Nursing Research, and Image: The Journal of Nursing Scholarship all published research measuring spirituality and religiousness during the time-period studied. The rate at which spirituality and religion appeared in these nursing research articles is substantially higher than that found in most fields outside of nursing. Even more frequent inclusion of spiritual and religious variables and richer measures of spirituality and religiousness would help to increase the available scientific information on the role of spirituality and religion in nursing care.     

Book Reviews

Books reviewed in this article: Likert , Rensis . The Human Organization: Its Management and Value. Mc Gregor , Douglas . (Edited by Warren G. Bennis and Caroline McGregor) The Professional Manager. Hofstede , G. H. The Game of Budget Control. Wickert , Frederic R. (Editor) Readings in African Psychology from French Language Sources. Haney , William V. Communication and Organizational Behavior: Text and Cases. Rulon , Phillip J., Tiedeman , David V., Tatsuoka , Maurice M., and Langmuir , Charles R. Multivariate Statistics for Personnel Classification. Strauss , George , and Sayles , Leonard R. Personnel: The Human Problems of Management. Huneryager , S. G., and Heckmann , I. L. Human Relations in Management. Mc Lennan , Kenneth . Managerial Skill and Knowledge. Barkin , Solomon (Editor). Technical Change and Manpower Planning: Co-ordination at the Enterprise Level. Hawk , Roger H. The Recruitment Function. Raudsepp , Eugene . What the Executive Should Know about Creating and Selling Ideas.     

Revisiting a meta-analysis of helpful aspects of therapy in a community counselling service

This small scale mixed methods study examines helpful events in a community counselling setting, categorising impacts of events according to Timulak’s [(2007). Identifying core categories of client-identified impact of helpful events in psychotherapy: A qualitative meta-analysis. Psychotherapy Research, 17, 305–314] meta-synthesis of significant events research. Comparisons were made between the types of impacts reported as helpful by clients receiving cognitive behavioural therapy (CBT) or relational integrative counselling. Seventy-five clients identified 291 significant events on 216 Helpful Aspects of Therapy (HAT) forms. The findings broadly reflect all nine impacts identified by Timulak (2007). The study also suggests that a potential new category, ‘voicing’ may exist. Differences between the types of impacts reported as helpful by clients in CBT and integrative therapy showed a higher percentage of CBT clients reporting ‘behavioural change/problem solution’, ‘reassurance’ and ‘client involvement’ as helpful. Differences in other categories were not statistically significant.     

RESEARCH SETTINGS IN INDUSTRIAL AND ORGANIZATIONAL PSYCHOLOGY: FACTS,FALLACIES, AND THE FUTURE1,2,3

Data were collected about research issues involving laboratory and field settings through a content analysis of the 1966, 1970, and 1974 volumes of the Journal of Applied Psychology, Organizational Behavior and Human Performance, and Personnel Psychology. Four general categories of data were collected: (a) affiliation of investigators, (b) topic areas investigated, (c) settings of the research, and (d) research strategies employed. Four common beliefs of either laboratory or field advocates were contradicted by the data. First, laboratory research was found fairly frequently in the industrial-organizational literature. Second, theory and hypothesis testing were not overemphasized. Third, laboratory research was as applied as field research. Lastly, studies published by researchers with nonacademic affiliations were as likely to have been conducted in laboratory settings as field settings. Recommendations were made to use the laboratory more frequently for theory and hypothesis based research and to use a variety of research strategies in both laboratory and field settings.     

Boehme's mysticism

David Walsh, The Mysticism of Innerworldly Fulfilment: A Stud.)' of Jacob Boehme. Gainesville, University of Florida Humanities Monographs, 53. x + 139 pp.     

Associations of non‐symbolic and symbolic numerical magnitude processing with mathematical competence: a meta‐analysis

Many studies have investigated the association between numerical magnitude processing skills, as assessed by the numerical magnitude comparison task, and broader mathematical competence, e.g. counting, arithmetic, or algebra. Most correlations were positive but varied considerably in their strengths. It remains unclear whether and to what extent the strength of these associations differs systematically between non‐symbolic and symbolic magnitude comparison tasks and whether age, magnitude comparison measures or mathematical competence measures are additional moderators. We investigated these questions by means of a meta‐analysis. The literature search yielded 45 articles reporting 284 effect sizes found with 17,201 participants. Effect sizes were combined by means of a two‐level random‐effects regression model. The effect size was significantly higher for the symbolic (r = .302, 95% CI [.243, .361]) than for the non‐symbolic (r = .241, 95% CI [.198, .284]) magnitude comparison task and decreased very slightly with age. The correlation was higher for solution rates and Weber fractions than for alternative measures of comparison proficiency. It was higher for mathematical competencies that rely more heavily on the processing of magnitudes (i.e. mental arithmetic and early mathematical abilities) than for others. The results support the view that magnitude processing is reliably associated with mathematical competence over the lifespan in a wide range of tasks, measures and mathematical subdomains. The association is stronger for symbolic than for non‐symbolic numerical magnitude processing. So symbolic magnitude processing might be a more eligible candidate to be targeted by diagnostic screening instruments and interventions for school‐aged children and for adults.     

Gender and gender-role identity influences upon self- and other-reports of communicative competence

This study examines the influence of gender-role identity on interpersonal perceptions of communicative competence in a natural setting over an extended period of time. After working in small task-oriented groups for 15 weeks, 146 students (74 females, 72 males) completed a questionnaire containing measures of gender-role identity (Bem Sex Role Inventory) and communicative competence [J. M. Wieman, Explication and test of a model of communicative competence. Human Communication Research, 1977, 3, 195–213]. A three-way ANOVA (masculinity, femininity, gender) was used for analysis. Predictions based on prior models were only partially supported. Findings suggest that gender-role identity functions in more complex ways than prior models suggest. For all dimensions of communicative competence, gender-role identity resulted in different patterns for males and females. Furthermore, comparisons of self- vs other-perceptions indicate the possibility that one model may hold for self-perceptions and another for other-perceptions.The authors gratefully acknowledge the helpful comments contributed by Kenneth R. Wilson and Kristina Troost.     

Is car following the real question – are equations the answer?

Within the constraints that they set themselves, Brackstone, M. and McDonald, M. (Transportation Research: Part F (2000)), do a good job in elucidating the current state-of-the-art concerning car following models. Further, their protestation concerning the importance of such models in the light of growing intelligent transportation systems (ITS) technology and potential application in collision-warning and collision-avoidance systems is also undoubtedly correct. Also, as a member of the human factors expert panel cited (Intelligent Transportation Society of America. (1997). In Proceedings of the Intelligent Vehicle Initiative Human Factors Workshop. Washington DC: ITS America) it can be confirmed that normative models of driving remain a crucial unsolved issue. Thus, for these reasons their present review is both helpful and timely and deserves wide circulation. The points of contention do not concern their specific observations but rather the intrinsic assumptions upon which they are based. In particular, the question asked is, is car following the real question and equations the answer?