“Doing justice” in psychological methodology: From science and experiments to anecdotes

Methodology is intrinsically related to “object” and its quality is based on the degree to which a method is doing justice to the object, demonstrating the entanglement of ontic, epistemic, and ethical considerations. The intent of “doing justice” is at the core of methodology and is the de facto guiding principle for conducting research and for producing knowledge. Objects in psychology can range from subjectivity to science and conflicts emerge because of giving primacy to particular objects. Using this perspective, various meanings of doing justice, critics’ challenges, deviations from doing justice to an object, ethical-political dimensions and the dialectics of doing justice in relation to objects are discussed. If doing justice is at the core of methodology, then the issue becomes under what circumstances a particular method is doing justice in relation to a particular object. Contrasting the experiment with anecdotes, it is shown dialectically that the former has no privileged status in psychology, and that experiments that are not replicated only do justice as anecdotal evidence.     

A generalized least squares regression approach for computing effect sizes in single-case research: application examples

A new method for deriving effect sizes from single-case designs is proposed. The strategy is applicable to small-sample time-series data with autoregressive errors. The method uses Generalized Least Squares (GLS) to model the autocorrelation of the data and estimate regression parameters to produce an effect size that represents the magnitude of treatment effect from baseline to treatment phases in standard deviation units. In this paper, the method is applied to two published examples using common single case designs (i.e., withdrawal and multiple-baseline). The results from these studies are described, and the method is compared to ten desirable criteria for single-case effect sizes. Based on the results of this application, we conclude with observations about the use of GLS as a support to visual analysis, provide recommendations for future research, and describe implications for practice.     

Pet personality: A review

An increasing amount of research is being conducted on pet personality, with a variety of goals, from standardization of methodology to development. Most of this work is done on dogs (n = 95), but there is some on cats (n = 24) as well. Very little is done on rabbits or other pets, such as ferrets, or amphibians or reptiles. The work that is being done has improved, with many more researchers reporting reliability and validity for their measures. For dogs, 56.82% of studies reported reliability measures (as compared with 19% in a previous analysis), and 70.45% reported validity measures (as compared with 27% in a previous analysis). However, there is still much work to be done on the standardization of methods both within and across species. This type of research is important as it has the potential to affect the welfare and other health and life outcomes of these animals, as well as elucidating evolutionary relationships that may offer clues to domestication.     

The response dynamics of preferential choice

The ubiquity of psychological process models requires an increased degree of sophistication in the methods and metrics that we use to evaluate them. We contribute to this venture by capitalizing on recent work in cognitive science analyzing response dynamics, which shows that the bearing information processing dynamics have on intended action is also revealed in the motor system. This decidedly “embodied” view suggests that researchers are missing out on potential dependent variables with which to evaluate their models—those associated with the motor response that produces a choice. The current work develops a method for collecting and analyzing such data in the domain of decision making. We first validate this method using widely normed stimuli from the International Affective Picture System (Experiment 1), and demonstrate that curvature in response trajectories provides a metric of the competition between choice options. We next extend the method to risky decision making (Experiment 2) and develop predictions for three popular classes of process model. The data provided by response dynamics demonstrate that choices contrary to the maxim of risk seeking in losses and risk aversion in gains may be the product of at least one “online” preference reversal, and can thus begin to discriminate amongst the candidate models. Finally, we incorporate attentional data collected via eye-tracking (Experiment 3) to develop a formal computational model of joint information sampling and preference accumulation. In sum, we validate response dynamics for use in preferential choice tasks and demonstrate the unique conclusions afforded by response dynamics over and above traditional methods.     

School Effectiveness Findings 1979–2002

The London studies of school effectiveness in the 1970s provided evidence that the qualities of schools made a significant difference to pupil progress, after taking account of the characteristics and backgrounds of the pupils at the time of school entry. The purpose of this article is to review the further evidence that has accumulated since then in order to assess how far the findings have been confirmed or refuted, what new questions have arisen and what key challenges remain. The continuing areas of research and debate include school effects on behavioral outcomes, the effects of class size, the importance of parental involvement, the impact of intake mix, peer influences, and level of resources. There has been greater agreement on the many features fostering school effectiveness. However, questions remain on how to bring about desired changes. The paper concludes by drawing attention to 10 largely unanswered questions that matter for policy and practice.     

From crisis of evidence to a “crisis” of relevance? Incentive-based answers for social psychology’s perennial relevance worries

Current controversies in social psychology have sparked the promotion of new rules for evidence in the field. This “crisis of evidence” echoes prior concerns from the 1970s about a so-called “crisis of social psychology”, with such issues as replication and statistical significance once more under examination. I argue that parallel concerns about the relevance of our research, raised but not completely resolved in the 1970s crisis, also deserve a fresh look. In particular, the advances made in the current crisis of evidence came about because of changes in academic career incentives, particularly publishing. Today, many voices in psychology urge greater respect for relevance in topics, methods and communication, but the lack of clear and concrete incentives to do so has stood in the way of answers. I diagnose the current incentive structures, propose partial solutions that are within the reach of journal editors and professional societies, and conclude by discussing the links between relevance and evidence, as well as special challenges to the relevance of social psychology post-2016.     

Measuring consciousness: Is one measure better than the other?

What is the best way of assessing the extent to which people are aware of a stimulus? Here, using a masked visual identification task, we compared three measures of subjective awareness: The Perceptual Awareness Scale (PAS), through which participants are asked to rate the clarity of their visual experience; confidence ratings (CR), through which participants express their confidence in their identification decisions, and Post-decision wagering (PDW), in which participants place a monetary wager on their decisions. We conducted detailed explorations of the relationships between awareness and identification performance, looking to determine (1) which scale best correlates with performance, and (2) whether we can detect performance in the absence of awareness and how the scales differ from each other in terms of revealing such unconscious processing. Based on these findings we discuss whether perceptual awareness should be considered graded or dichotomous. Results showed that PAS showed a much stronger performance-awareness correlation than either CR or PDW, particularly for low stimulus intensities. In general, all scales indicated above-chance performance when participants claimed not to have seen anything. However, such above-chance performance only showed when we also observed a correlation between awareness and performance. Thus (1) PAS seems to be the most exhaustive measure of awareness, and (2) we find support for above-chance performance in the absence of subjective awareness, but such unconscious knowledge only contributes to performance when we observe conscious knowledge as well. Similarities and differences between scales are discussed in the light of consciousness theories and response strategies.     

Partial awareness distinguishes between measuring conscious perception and conscious content: Reply to Dienes and Seth

In their comment on Sandberg, Timmermans, Overgaard, and Cleeremans (2010), Dienes and Seth argue that increased sensitivity of the Perceptual Awareness Scale (PAS) is a consequence of the scale being less exclusive rather than more exhaustive. According to Dienes and Seth, this is because PAS may measure some conscious content, though not necessarily relevant conscious content, “If one saw a square but was only aware of seeing a flash of something, then one has not consciously seen a square.” In this reply, we claim that there is a difference between conscious visual experience, which may be partial, and the resulting conscious content, which is conceptual. Whereas PAS measures the first, confidence judgments and post-decision wagering measure the second.