Colloquy on Generalization Meta-Analytic Techniques When Generalizing to Message Populations Is Not Possible

Jackson and Jacobs (1983) have argued that many of the findings in communication research could be specific to the particular messages used in a given study. As a solution, Jackson and Jacobs propose generalizing to message populations by treating messages as a random variable. This article argues that the Jackson and Jacobs solution is inappropriate for complex messages and that meta-analytic techniques represent an alternative that permits the detection of results that are specific to particular messages, researchers, or paradigms.     

GENERALIZING ABOUT MESSAGES: SUGGESTIONS FOR DESIGN AND ANALYSIS OF EXPERIMENTS

Current research practices in communication create problems for both internal and external validity. One serious design flaw, which involves use of a single message to represent a category of messages, occurs in nearly all of the experimental research on communication effects. The problem with such a design is that an observed difference between categories may reflect only differences between individual, idiosyncratic cases. A related error, the “language-as-fixed-effect fallacy,” involves use of several replications of each category, but analysis of the cases as fixed effects. The consequence is that findings cannot be generalized beyond the sample used. Future research should use multiple cases within each message category studied and treat cases as nested random effects. Since the cases cannot be true random samples from message populations, care must be taken to avoid known sources of bias or other barriers to generalization.     

M > 1

Increasingly, communication experiments are incorporating replication/actors for the purpose of controlling confounds and increasing generalizability. If replications are considered to be samples of possible treatment implementations, treating the replication factor as random is more appropriate than treating it as fixed. Study 1 shows that treating sampled replications as a fixed effect leads to potentially serious alpha inflation in the test of the treatment effect while treating sampled replications as random controls alpha at its nominal level. Study 2 addresses a common objection to treating replications as random: the argument that to do so will lead to unacceptably low power in statistical testing. Although experiments with very few replications are likely to be deficient in power, the results of Study 2 establish that power can be improved to an unexpected degree by a relatively modest increase in the number of replications.     

The Search for Reliable Generalizations About Messages A Comparison of Research Strategies

Two ways of conducting the search for generalizations about messages are considered: Morley's (this volume) proposal that single-message research designs be used, with subsequent meta-analytic summaries, and Jackson and Jacobs's (1983) proposal that multiple-message designs be used, with messages treated as a random factor in the statistical analysis. Jackson and Jacobs's approach is shown to provide a more dependable, efficient, and practical means for gathering the requisite evidence for dependable generalizations. The charge that multiple-message designs suffer from irreparable problems of experimenter bias is refuted. The treatment of messages as a random factor is defended as statistically appropriate and as clearly preferable to the statistical alternatives.     

USE OF THE GATB “G” SCORE FOR PREDICTING ACHIEVEMENT ON THE GED

The present study investigates the relationship of the GATB “G” with the GED. Practical and theoretical implications are involved. Making use of samples from the Las Vegas, Nevada population, the study compares scores of individuals administered the GED and GATB. A substantial correlation for prediction purposes was devised to illustrate the use of the GATB “G” to predict, by interval, an individual's position on the GED. The expectancy table helps guide the Vocational Counselor with a constructive tool whereby he advises the next steps of action for an educationally deficient or proficient client. The study recommends further replications to determine if such studies could guide agency policies whereby vocational counseling practice is improved and agency testing costs reduced.     

A review of “The biology of mental Defect”

By Lionel S. Penrose, M.A., M.D. With a Preface by Professor J. B. S. Haldane, F.R.S. London: Sidgwick and Jackson, 1949-Pp. xiv + 285. With 7 plates. 21s.     

Taking “the boss” into the real world: Field interrater reliability of the Short‐Term Assessment of Risk and Treatability: Adolescent Version

There is emerging evidence that the performance of risk assessment instruments is weaker when used for clinical decision‐making than for research purposes. For instance, research has found lower agreement between evaluators when the risk assessments are conducted during routine practice. We examined the field interrater reliability of the Short‐Term Assessment of Risk and Treatability: Adolescent Version (START:AV). Clinicians in a Dutch secure youth care facility completed START:AV assessments as part of the treatment routine. Consistent with previous literature, interrater reliability of the items and total scores was lower than previously reported in non‐field studies. Nevertheless, moderate to good interrater reliability was found for final risk judgments on most adverse outcomes. Field studies provide insights into the actual performance of structured risk assessment in real‐world settings, exposing factors that affect reliability. This information is relevant for those who wish to implement structured risk assessment with a level of reliability that is defensible considering the high stakes.     

A Critical Appraisal of Amish Studies’ De Facto Paradigm, “Negotiating with Modernity”

The predominant paradigm for Amish research over the past three decades has been “negotiating with modernity” (NWM). NWM explains how the Amish as a distinctive American culture and religion have overcome the threats modernity poses to their existence. This article argues that NWM has been accepted and replicated uncritically and without empirical testing due to properties making it inherently untestable. We first synthesize the central NWM literature, as the paradigm has never been fully constructed in one place. We then call attention to two sets of impairments that prevent empirical tests. First, the perspective contains several logical fallacies and terminology problems. NWM reifies “the Amish” as its unit of analysis; creates circular variable relationships (the Amish are Amish because they are not something else); employs terms problematically, notably “modernity” and “modernization”; dichotomizes phenomena; and is grounded in no particular theoretical tradition. Second, we argue that the production of knowledge within core NWM literature has not been sufficiently substantiated. Data collection procedures and methodological strategies are unclear or undefined and scientific peer review is relatively low. We conclude by suggesting more promising theoretical directions for Amish studies.     

KNOWING WHEN TO “SECOND-GUESS”

Studies on social information gathering typically investigate how an interactant actively elicits needed information from another source. An alternative approach would focus on how a hearer gathers selected bits of information from the message(s) at hand. The process of second-guessing is proposed as one such interpretive approach to information gathering. Previous research confirmed that social actors do use this strategy and use it reasonably effectively. This article extends our knowledge of second-guessing by looking at the conditions or “cues” that provoke the process, and by showing that participants in an experimental study second-guess when confronted with those cues.     

Evangelism in “Christian” Societies

The globalization of the economy and the unthinkable development of mass media and transportation and communication have transformed the oldest Christian societies of the northern hemisphere and especially of Europe into multicultural societies. Re‐evangelization is the answer to the question of what must be done. Faith originates from hearing; the missionary preaching that is directed toward the people who are ignorant of the world‐saving work of Christ needs to be articulated, keeping in mind the scheme of communication and the special significance of the following factors: transmitter, receiver, topic, message, channel and code. The gospel is addressed to all ages. Youth and intellectuals exhibit more difficulties in acceptance, which calls for the adoption of a special code of communication. Of the utmost necessity is the flame of the transmitter. Electronic channels of communication offer unthinkable possibilities for the transmission of the message of the gospel to innumerable receivers by the use of a variety of codes.     

Begging the question as a pragmatic fallacy

The aim of this paper is to make it clear how and why begging the question should be seen as a pragmatic fallacy which can only be properly evaluated in a context of dialogue. Included in the paper is a review of the contemporary literature on begging the question that shows the gradual emergence over the past twenty years or so of the dialectical conception of this fallacy. A second aim of the paper is to investigate a number of general problems raised by the pragmatic framework.The work in this paper was supported by a Fellowship from the Netherlands Institute for Advanced Study in the Humanities and Social Sciences (NIAS) and a Research Grant from the Social Sciences and Humanities Research Council of Canada. Thanks are due to Erik Krabbe for discussions, and to the members of the NIAS Research Group on Fallacies as Violations of Rules of Argumentative Discourse: Frans van Eemeren, Rob Grootendorst, Sally Jackson, Scott Jacobs, Agnes Haft van Rees, Agnes Verbiest, Charles Willard, and John Woods.     

The Political Relevance of Emotions: “Reassessing” Revisited

Ladd and Lenz (2008 ) question a central claim of affective intelligence theory (AIT), namely that anxiety affects political judgments indirectly by reducing the role of predispositions and increasing the role of contemporary information. They claim that alternative hypotheses, especially the notion that emotions are simply rationalizations of political preferences, better explain the role of emotion in politics. Their ultimate conclusions, however, rest on an overly narrow view of both theory and evidence. Even if one accepts Ladd and Lenz's reanalysis of survey data, there is insufficient evidence to cast aside either AIT or the hypothesis that anxiety affects the mode of political judgment. AIT explains a broad range of political relationships that are unchallenged by the reassessment and which cannot be explained by the offered alternatives. Moreover, numerous experimental studies reject the alternative explanations in favor of an exogenous, often interactive role for anxiety. AIT will surely require amendment, if not abandonment, someday. Competing theories of emotion already exist and, unlike the alternative championed by Ladd and Lenz, these theories too suggest a meaningful role for emotion in political judgment and behavior. For now, given all of the research to date, AIT is a robust competitor, scarcely in need of being “salvaged.” Burgeoning research on the political relevance of emotions will benefit from further debates, replications, and extensions, as long as new claims and evidence are put in proper perspective.     

“COME TO THINK OF IT…”

In Phase 1 of this study, communicators responded truthfully or deceptively to positive or negative interrogative probes. In Phase 2, the interviews were shown, at four levels of probe exposure, to observers who rated the communicators' eracity. In Phase 3, verbal and nonverbal cues were correlated with actual and perceived deception. Although exposure level had no effect on detection accuracy, liars and trulhtellers exposed to negative interrogative probes were judged more truthful than those exposed to positive probes. Analysis of the verbal and nonverbal cues revealed that none of the nonverbal cues relied on as indicators of deception were related to actual deception, and only one of the verbal cues (verbal content) was related to actual message veracity. Implications of these findings and suggestions for future research are offered.     

“Sinking Into the Ground”: The Development and Consequences of Shame in Adolescence

Although in recent years the developmental antecedents and clinical consequences of shame have received increased theoretical and research attention, little is yet known about shame's development after early childhood or its role in developmental psychopathology. This article reviews the literature on shame and on adolescence and suggests that attention to the role of shame in adolescent development is warranted for several reasons. Social-cognitive, physical, and interpersonal changes associated with adolescence may each be associated with normative increases in shame during adolescence. In addition, several lines of evidence converge to suggest that shame may be implicated in some of the important gender-related shifts in self-esteem and developmental psychopathology that occur over the adolescent years.     

THE PROBLEM OF “GOD” IN PSYCHOLOGY OF RELIGION: LONERGAN'S “COMMON SENSE” (RELIGION) VERSUS “THEORY” (THEOLOGY)

The emphasis on God in American psychology of religion generates the problem of explaining divine‐versus‐natural causality in “spiritual experiences.” Especially “theistic psychology” champions divine involvement. However, its argument exposes a methodological error: to pit popular religious opinions against technical scientific conclusions. Countering such homogenizing “postmodern agnosticism,” Bernard Lonergan explained these two as different modes of thinking: “common sense” and “theory”—which resolves the problem: When theoretical science is matched with theoretical theology, “the God‐hypothesis” explains the existence of things whereas science explains their natures; and, barring miracles, God is irrelevant to natural science. A review of the field shows that the problem is pervasive; attention to “miracles”—popularly so‐named versus technically—focuses the claims of divine‐versus‐natural causality; and specifications of the meaning of spiritual, spirituality, science, worldview, and meaning itself (suffering that same ambiguity: personal import versus cognitive content) offer further clarity. The problem is not naturalism versus theism, but commonsensical versus theoretical thinking. This solution demands “hard” social science.     

The Distinct Psychologies of “Looking” and “Leaping”: Assessment and Locomotion as the Springs of Action

Actions require two essential functions: assessment and locomotion. Assessment determines one’s goals and selects the means. Locomotion translates these into concrete behavior. In past work, assessment and locomotion have been portrayed as co‐ordinated and interdependent, or associated with different action phases. In contrast, we review recent theorizing and research that depict assessment and locomotion as autonomous and complementary. Recent evidence supports this conception for the behavior of individuals, groups, organizations, and cultures in reference to actions at different levels of analysis.     

DIVINING “DIVINE ACTION” IN THEOLOGY‐AND‐SCIENCE: A REVIEW ESSAY

The topic of divine action has been central to the theology‐and‐science discussion over the last twenty years. Some tentative conclusions are currently being drawn in light of research initiatives that have been engaged on this topic. I review three recent books that have responded in some way to the ongoing discussion. These responses show that, notwithstanding the advances made in the conversation, much work remains to be done before a plausible theory of divine action emerges at the interface of theology and science.     

Understanding Self‐Effects in Social Media

The aim of this article is to improve understanding of self‐effects in social media, and to compare self‐effects with reception effects. Self‐effects are the effects of messages the cognitions, emotions, attitudes, and behaviors of the message creators/senders themselves. A total of 4 theories have tried to explain self‐effects in offline environments: self‐persuasion, self‐concept change, expressive writing, and political deliberation. The article reviews research into online self‐effects that evolved from each of these theories, and argues why self‐effects may be stronger online than offline. Based on this review, a model is introduced that helps explain how online self‐ and reception effects may coalesce and amplify each other. The article ends by presenting some suggestions for future research.     

EVALUATION OF THE “CONTROL OVER REINFORCEMENT” COMPONENT IN FUNCTIONAL COMMUNICATION TRAINING

The effectiveness of functional communication training (FCT) as a treatment for behavior disorders has been attributed to a number of variables, one of which is the individual's ability to exert control over the delivery of reinforcement. We evaluated this component of FCT by exposing individuals to conditions in which their behavior either did or did not affect the delivery of reinforcement. Three adults with mental retardation who engaged in self-injurious behavior (SIB) participated. Following a functional analysis of their SIB, the effects of FCT were compared to those of noncontingent reinforcement (NCR) in a multielement design. The amount of reinforcement during both conditions was equated by yoking the schedule of reinforcement during NCR sessions to that in effect during FCT sessions. Results indicated that FCT and NCR were equally effective in reducing the SIB of all participants and suggest that control over reinforcement delivery may not affect the degree to which FCT produces behavioral suppression. However, a different benefit of FCT was evident in the results: More consistent increases in the alternative response were observed during the FCT condition than during the NCR condition.