| Abstract: | Current research practices in communication create problems for both internal and external validity. One serious design flaw, which involves use of a single message to represent a category of messages, occurs in nearly all of the experimental research on communication effects. The problem with such a design is that an observed difference between categories may reflect only differences between individual, idiosyncratic cases. A related error, the “language-as-fixed-effect fallacy,” involves use of several replications of each category, but analysis of the cases as fixed effects. The consequence is that findings cannot be generalized beyond the sample used. Future research should use multiple cases within each message category studied and treat cases as nested random effects. Since the cases cannot be true random samples from message populations, care must be taken to avoid known sources of bias or other barriers to generalization. |
