MEASUREMENT ERROR IN RESEARCH ON HUMAN RESOURCES AND FIRM PERFORMANCE: ADDITIONAL DATA AND SUGGESTIONS FOR FUTURE RESEARCH

Gerhart and colleagues (2000) and Huselid and Becker (2000) recently debated the presence and implications of measurement error in measures of human resource practices. This paper presents data from 3 more studies, 1 of large organizations from different industries at the corporate level, 1 from commercial banks, and the other of autonomous business units at the level of the job. Results of all 3 studies provide additional evidence that single respondent measures of HR practices contain large amounts of measurement error. Implications for future research into the HR firm performance relationship are discussed.     

MEASUREMENT ERROR IN RESEARCH ON HUMAN RESOURCES and FIRM PERFORMANCE: HOW MUCH ERROR IS THERE AND HOW DOES IT INFLUENCE EFFECT SIZE ESTIMATES?

Studies of the relationship between human resource (HR) practices and firm performance typically use a single respondent to assess firm level HR practices or HR effectiveness. However, previous research in other substantive areas suggests that rater differences are a potentially important source of measurement error. We demonstrate analytically the potential consequences of both random and systematic measurement error in research on HR and firm performance. However, our main focus is on random error and we show how generalizability theory can be applied to obtain better estimates of reliability by simultaneously recognizing multiple sources (e.g., items, raters) of random measurement error. These more inclusive reliability estimates, in turn, offer the possibility of more precisely quantifying substantive relationships in the HR and firm performance literature. In our sample, reliabilities (as estimated by generalizability coefficients) for single-rater assessments of HR variables were generally below .50. This degree of measurement error, if present in substantive studies on HR and firm performance, could lead to considerable bias, given that an unstandardized regression coefficient is corrected for measurement error in the independent variable by dividing by its reliability coefficient (not its square root). We also found only limited convergent validity between HR and line managers ratings of a second type of HR measure, HR effectiveness. In general, our findings suggest that future researchers need to devote greater attention to measurement error and construct validity issues. Our study provides an example of how generalizability theory can be useful in this pursuit.     

THE RELATIONSHIP BETWEEN HR PRACTICES AND FIRM PERFORMANCE: EXAMINING CAUSAL ORDER

Significant research attention has been devoted to examining the relationship between HR practices and firm performance, and research support has assumed HR as the causal variable. Using data from 45 business units (with 62 data points), this study examines how measures of HR practices correlate with past, concurrent, and future operational performance measures. The results indicate that correlations with performance measures at all 3 times are both high and invariant, and that controlling for past or concurrent performance virtually eliminates the correlation of HR practices with future performance. Implications are discussed.     

Core Knowledge Employee Creativity and Firm Performance: The Moderating Role of Riskiness Orientation,Firm Size,and Realized Absorptive Capacity

In this study, we examine when creativity is positively or negatively related to firm performance. Building on the creation–implementation tension theorized in the literature and the attention capacity perspective, we argue that the relationship between creativity and firm performance is contingent on riskiness orientation, firm size, and realized absorptive capacity. Data were collected from 761 core knowledge employees, 148 CEOs, and 148 HR executives from 148 high‐technology firms. The results indicated that core knowledge employee creativity was negatively related to firm performance when riskiness orientation was high. The relationship was positive when realized absorptive capacity was high. Finally, the relationship was more positive in small firms than in large firms. We discuss the implications of our findings for creativity research and managerial practices.     

EMPLOYEE ATTRIBUTIONS OF THE “WHY” OF HR PRACTICES: THEIR EFFECTS ON EMPLOYEE ATTITUDES AND BEHAVIORS,AND CUSTOMER SATISFACTION

The construct of human resource (HR) attributions is introduced. We argue that the attributions that employees make about the reasons why management adopts the HR practices that it does have consequences for their attitudes and behaviors, and ultimately, unit performance. Drawing on the strategic HR literature, we propose a typology of 5 HR‐attribution dimensions. Utilizing data collected from a service firm, we show that employees make varying attributions for the same HR practices, and that these attributions are differentially associated with commitment and satisfaction. In turn, we show that these attitudes become shared within units and that they are related to unit‐level organizational citizenship behaviors and customer satisfaction. Findings and implications are discussed.     

Harmful transgressions qua moral transgressions: A deflationary view

One important issue in moral psychology concerns the proper characterisation of the folk understanding of the relationship between harmful transgressions and moral transgressions. Psychologist Elliot Turiel and associates have claimed with a broad range of supporting evidence that harmful transgressions are understood as transgressions that are authority independent and general in scope which, according to them, characterises these transgressions as moral transgressions. Recently many researchers questioned the position advocated by the Turiel tradition with some new evidence. We entered this debate proposing an original, deflationary view in which perceptions of basic-rights violation and injustice are fundamental for the folk understanding of harmful transgressions as moral transgressions in Turiel's sense. In this article we elaborate and refine our deflationary view, while reviewing the debate, addressing various criticisms raised against our perspective, showing how our perspective explains the existent evidence, and suggesting new lines of inquiry.     

End of 2016: Can We Save Research from Predators in 2017?

At the beginning of every year, we expect to see worthwhile improvements on the past. The end of 2016 showcased many important issues in the scientific world, ranging from criticisms of research misconduct and fraud to the introduction of new scientometrics. Despite the scientific community’s continuing efforts, predatory journals and publishers are still on the rise, and the Beall’s list calls attention to the need to take a firm action across the board. This short opinion piece highlights research conducted by the scholarly community on research publication predators during 2016, and offers suggestions as to how to bring about future improvements.     

CONDEMNATION OF A SCIENTIFIC ARTICLE: A CHRONOLOGY AND REFUTATION OF THE ATTACKS AND A DISCUSSION OF THREATS TO THE INTEGRITY OF SCIENCE

On July 12, 1999, our meta-analysis on child sexual abuse published in Psychological Bulletin, one of the American Psychological Association's (APA) premiere journals, was condemned by the U.S. Congress (H. Con. Res. 107). The condemnation followed months of attacks on the article, the APA, and us by various social conservatives and psychoanalytically-oriented clinicians. The American Association for the Advancement of Science (AAAS) was asked by the APA to independently review our article. After considering criticisms of it and the article itself, AAAS declined, but commented that it was the criticisms, not our methods or analyses, that troubled them because these criticisms misrepresented what we wrote. The current article chronicles this whole affair. First, we provide background, explaining why an article such as ours was needed. Then we accurately summarize the article, given that it has been so widely misrepresented. Next we present a chronology of the events leading up to and following the condemnation. We then present and refute all the major criticisms of the article, which have included both methodological and conceptual attacks. Next we discuss the threat to science that these events portend. We conclude by discussing the need to separate moral judgments from scientific research, the conflation of which formed the basis for the distortions and condemnation.     

A consideration of challenges to psychological assessment instruments used in forensic settings: Rorschach as exemplar

In this brief primer, we provide an outline of key issues that will help psychologists organize and prepare their expert testimony. These issues include the need to obtain essential sources of research, a review of the actual legal standards regarding admissibility of test data in expert testimony, the nature of the expert relative to the assessment instrument in expert testimony, the nature of legal versus scientific debate, and the examination of appropriate qualifications of expertise when offering legal testimony. In addition, we use a summary of information contained in several recent articles to address challenges directed against forensic psychological testing. We use the empirical literature on the Rorschach as an exemplar in discussing these issues, as the admissibility of the Rorschach in particular has been challenged, and the issues frequently focused on with the Rorschach are equally applicable to other psychological measures. In this article, we provide essential sources of Rorschach research regarding several empirical studies that summarize important information and directly address previous criticisms of the measure.     

The importance of frameworks for directing empirical questions: reply to Goodie and Fantino (2000)

A. S. Goodie and E. Fantino (2000) make two main criticisms of the predictions of M. C. Lovett and C. D. Schunn's (1999) RCCL model. (RCCL is pronounced "ReCyCLe"; it stands for Represent the task, Construct a set of action strategies, Choose from among those strategies according to success rate, Learn new success rates.) In both cases, the authors believe the criticisms reflect a failure to appreciate the difference between broad frameworks and specific mathematical/computational models. In this article, the value of a broad framework, such as RCCL, in directing new empirical analyses and guiding theoretical development is shown. In particular, RCCL expands on existing work to reveal how variability and change in mental representations influence base-rate sensitivity. The authors also address several other issues raised by A. S. Goodie and E. Fantino (2000) and show that qualitative shifts in individuals' choice behavior are present in their original data--a key prediction of RCCL that does not appear in previous accounts.     

THE PAST AND FUTURE OF EXPERIMENTAL PHILOSOPHY

Experimental philosophy is the name for a recent movement whose participants use the methods of experimental psychology to probe the way people think about philosophical issues and then examine how the results of such studies bear on traditional philosophical debates. Given both the breadth of the research being carried out by experimental philosophers and the controversial nature of some of their central methodological assumptions, it is of no surprise that their work has recently come under attack. In this paper we respond to some criticisms of experimental philosophy that have recently been put forward by Antti Kauppinen. Unlike the critics of experimental philosophy, we do not think the fledgling movement either will or should fall before it has even had a chance to rise up to explain what it is, what it seeks to do (and not to do), and exactly how it plans to do it. Filling in some of the salient details is the main goal of the present paper.     

Methodological Requirements for Studying Creativity in Organizations

This article begins by broadly defining creativity and then more narrowly focusing on the definition of this construct. The broad methodological issues associated with organizational creativity are then discussed. To a large extent, the methodological requirements for studying organizational creativity depend on theoretical models. The most relevant theoretical models are reviewed. Specific methodological requirements for studying creativity in organizations are then reviewed, with emphasis on issues of validity and reliability. Finally, the article considers how organizational creativity research might be studied in a manner consistent with the methodological issues raised.     

Evaluative conditioning is pavlovian conditioning: issues of definition, measurement, and the theoretical importance of contingency awareness

In her commentary of Field (1999), Hammerl (1999) has drawn attention to several interesting points concerning the issue of contingency awareness in evaluative conditioning. First, she comments on several contentious issues arising from Field's review of the evaluative conditioning literature, second she critiques the data from his pilot study and finally she argues the case that EC is a distinct form of conditioning that can occur in the absence of contingency awareness. With reference to these criticisms, this reply attempts to address Hammerl's comments by exploring the issues of how awareness is defined, how it is best measured, and whether it is reasonable to believe that EC uniformly occurs in the absence of contingency awareness. The article concludes that the available evidence supports Field's proposition that EC is, in fact, Pavlovian learning. Copyright 2000 Academic Press.     

The parasite-stress theory may be a general theory of culture and sociality

In the target article, we presented the hypothesis that parasite-stress variation was a causal factor in the variation of in-group assortative sociality, cross-nationally and across the United States, which we indexed with variables that measured different aspects of the strength of family ties and religiosity. We presented evidence supportive of our hypothesis in the form of analyses that controlled for variation in freedom, wealth resources, and wealth inequality across nations and the states of the USA. Here, we respond to criticisms from commentators and attempt to clarify and expand the parasite-stress theory of sociality used to fuel our research presented in the target article.     

Reliability and factorial validity of Adolescent Time Inventory–Time Attitude (ATI-TA) Scores in Scottish and Northern Irish adolescents

The contemporary literature investigating the construct broadly known as time perspective is replete with methodological and conceptual concerns. These concerns focus on the reliability and factorial validity of measurement tools, and the sample-specific modification of scales. These issues continue to hamper the development of this potentially useful psychological construct. An emerging body of evidence has supported the six-factor structure of scores on the Adolescent Time Inventory–Time Attitudes Scale, as well as their reliability. The present study utilized data from the first wave of a longitudinal study in the United Kingdom to examine the reliability, validity, and cross-cultural invariance of the scale. Results showed that the hypothesized six-factor model provided the best fit for the data; all alpha and omega estimates were > .70; scores on ATI-TA factors related meaningfully to self-efficacy scores; and the factor structure was invariant across both research sites. Results are discussed in the context of the extant temporal literature.     

Examining arguments against quantitative research: “Case studies” illustrating the challenge of finding a sound philosophical basis for a human sciences approach to psychology

This article offers a close examination of critiques of quantitative research by Michell, 2011, Marecek, 2011, and Morawski (2011). One goal is to show that these three critics actually share with most mainstream quantitative researchers commitments to the Cartesian framework, even though this is not obvious because Cartesianism can appear in different guises. As a result of these commitments, the three theorists advance criticisms of mainstream quantitative research that fail to identify its key failings, put forward flawed views about how we should conduct research, and offer misguided criticisms of an approach I advocate called explicitly interpretive quantitative research. Another goal is to use the examination of the three critiques as a vehicle for clarifying the participatory perspective, a philosophical viewpoint that departs from the Cartesian framework. With regard to research methodology, the participatory perspective provides the basis for explicitly interpretive quantitative research, leads to ideas about changes we should make in how we conduct qualitative research, and treats quantitative and qualitative research as fundamentally similar because both should be pursued as interpretive modes of inquiry. I suggest that my analyses of the three critiques of quantitative research – or “case studies,” as I call these analyses – also may prove useful to researchers and theorists who want to develop a human sciences approach to other issues besides research methodology by helping them (1) recognize when lines of thinking that seem to depart from the mainstream actually represent variants of Cartesianism, and (2) consider what the participatory perspective might have to offer if they were to use it as the philosophical basis for their efforts.     

Translating the Stuttering Severity Instrument (SSI-3) into Persian: Response to Karimi, Nilipour, Shafiei and Howell

Karimi, Nilipour, Shafiei and Howell have raised some issues in regards to a study of which we were authors (Bakhtiar, Seifpanahi, Hossein, Ghanadzade & Packman, 2010). In that study, we reported the translation of the Stuttering Severity Instrument-3 (SSI-3) into Persian and reported its reliability when used with 12 Persian speaking preschool children who stutter, in Iran. Karimi et al. criticize our methods and claim that we did not specify that the translated test be used with preschoolers. In this response, we maintain that their criticisms of our methods are largely unfounded, especially the suggestion that we did not follow the WHO guidelines for translating questionnaires in our translation procedure. In addition, we show that we did in fact stipulate in our paper that the Persian SSI-3 should be used with preschoolers and that Karimi et al. misquoted us in their claim that we did not.     

TRANSCENDING OUR BIOLOGY: A VIRTUE ETHICS INTERPRETATION OF THE APPEAL TO NATURE IN TECHNOLOGICAL AND ENVIRONMENTAL ETHICS

“Arguments from nature” are used, and have historically been used, in popular responses to advances in technology and to environmental issues—there is a widely shared body of ethical intuitions that nature, or perhaps human nature, sets some limits on the kinds of ends that we should seek, the kinds of things that we should do, or the kinds of lives that we should lead. Virtue ethics can provide the context for a defensible form of the argument from nature, and one that makes proper sense of its enduring role in debates concerning our relationship to technology and the environment. However, the notion of an ethics founded upon an account of the essential features of human nature is controversial. On the one hand, contemporary biological science no longer defines species by their essential characteristics, so from a biological point of view there just are no essential characteristics of human beings. On the other hand, it might be argued that humans have, in some sense, “transcended our biology,” so an understanding of humans as a biological species is extraneous to ethical questions. In this article, I examine and defend the argument from nature, as a way to ground an ethic of virtue, from some of the more common criticisms that are made against it. I argue that, properly interpreted as an appeal to an evaluative account of human nature, the argument from nature is defensible with the context of virtue ethics and, in this light, I show how arguments from nature made in popular responses to technological and environmental issues are best understood.     

How Should We Study Generational ‘Changes’—Or Should We? A Critical Examination of the Evidence for ‘Generation Me’

In this essay, we discuss some of the issues that are involved when making inferences about birth cohorts and we describe some of our work that has caused us to question the strength of the evidence for so-called 'Generation Me' effects. In particular, we describe our analyses from the Monitoring the Future project using self-esteem and respond to criticisms of our work. At this point, we believe that there is insufficient evidence to support the conclusion that the personalities and self-beliefs of today's young people are dramatically different from previous generations of American youth.