Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods

Since its establishment, psychology has struggled to find valid methods for studying thoughts and subjective experiences. Thirty years ago, Ericsson and Simon (1980) proposed that participants can give concurrent verbal expression to their thoughts (think aloud) while completing tasks without changing objectively measurable performance (accuracy). In contrast, directed requests for concurrent verbal reports, such as explanations or directions to describe particular kinds of information, were predicted to change thought processes as a consequence of the need to generate this information, thus altering performance. By comparing performance of concurrent verbal reporting conditions with their matching silent control condition, Ericsson and Simon found several studies demonstrating that directed verbalization was associated with changes in performance. In contrast, the lack of effects of thinking aloud was merely suggested by a handful of experimental studies. In this article, Ericsson and Simon's model is tested by a meta-analysis of 94 studies comparing performance while giving concurrent verbalizations to a matching condition without verbalization. Findings based on nearly 3,500 participants show that the "think-aloud" effect size is indistinguishable from zero (r = -.03) and that this procedure remains nonreactive even after statistically controlling additional factors such as task type (primarily visual or nonvisual). In contrast, procedures that entail describing or explaining thoughts and actions are significantly reactive, leading to higher performance than silent control conditions. All verbal reporting procedures tend to increase times to complete tasks. These results suggest that think-aloud should be distinguished from other methods in future studies. Theoretical and practical implications are discussed.     

Cognitive mediation of putting: Use of a think-aloud measure and implications for studies of golf-putting in the laboratory

ObjectivesWhereas accounts of skilled performance based on automaticity (Beilock & Carr, 2001; Fitts & Posner, 1967) emphasize reduced cognitive involvement in advanced skill, other accounts propose that skilled performance relies on increased cognitive control (Ericsson & Kintsch, 1995). The objective of this study was to test predictions differentiating the automaticity and cognitive control accounts by assessing thinking during golf putting.DesignThe cognitive processes of less-skilled and more-skilled golfers were examined during putting using concurrent, think-aloud verbal reports. The design included putting conditions that differed in complexity and thus the need to adapt the putt to the particular conditions.MethodPutting complexity was manipulated via changes to putt length and perceived stress during putting. Putts were executed from two starting locations (i.e., the same starting location as the previous putt or a new starting location).ResultsThe analysis showed that, during putting: more thoughts were verbalized overall by more-skilled golfers than less-skilled golfers; both groups verbalized more thoughts overall during higher-complexity putts (i.e., longer distance putts, and putts under higher stress when executed from a new starting location) than lower-complexity putts; and the two groups did not differ significantly in the number of thoughts related to motor mechanics.ConclusionsThe results of this study provide support for a cognitive control account of skilled performance and suggest that the path to skilled performance involves the acquisition of more refined higher-level cognitive representations mediating planning and analysis.     

Developmental Changes In Switching Between Mental Task Sets: The Influence Of Verbal Labeling In Childhood

Age-related changes in the use of verbal processes for the efficient switching between tasks were investigated in 5-year-old children (N = 32, M age = 5.9 years) and 9-year-old children (N = 32, M age = 9.4 years). Task switching was assessed by means of a cued switching paradigm to examine two switching components: (a) to maintain and select and (b) to switch between task sets. The aim of this study is to determine (a) developmental changes in both switching components during childhood, (b) whether age-related changes in these components are influenced by verbalizations in terms of “thinking aloud” (Ericsson & Simon, 1993) during task performance, and (c) age changes in the content of the self-verbalizations. Results show an age-related impairment of the 5-year-old group on the level of task-set maintenance and selection. Children mainly verbalized the response labels. However, whereas 5-year-olds more often labeled their perceptions (Zelazo, 1999), 9-year-olds used language to maintain the currently relevant task set. Both age groups relied on the same verbal strategy facilitating task switching, supporting the view that language can be used to plan and guide actions (Luria, 1969).     

The validity of verbal protocols

The reactivity of a "think aloud" verbal protocol and the veridicality of different retrospective protocols were tested over four dissimilar tasks. Generating a concurrent protocol altered accuracy in two tasks, simple addition and a choice between two gambles, and generally prolonged response times. Such reactivity partially qualifies the dominant theory of protocol generation (Ericsson & Simon, 1984). Retrospective protocols yielded substantial forgetting or fabrication in all tasks, supporting the consensus on the nonveridicality of these methods. It is concluded that protocol validity should be based on an empirical check rather than on theory-based assurances.     

Enduring stress: A quantitative analysis on coping profiles and sport well-being in amateur endurance athletes

Endurance athletes experience physical and psychological stress during training and competition that can inhibit performance and promote negative health implications (i.e., lower well-being) without proper coping mechanisms (McCormick et al., 2018; Sakar & Fletcher, 2014). Additionally, these athletes that train at an amateur level have received limited attention regarding coping with stress and how it impacts well-being (McCormick et al., 2018). The purpose of this study was threefold: to (a) determine common coping profiles of trained amateur endurance athletes, (b) explore the relationship between sport well-being and these coping profiles, and (c) examine the potential roles that appraisals might play in the coping-well-being relationship. The results yielded five distinct coping profiles: Mixed Adaptive Copers (MAC), Mixed Maladaptive Copers (MMC), Engaged Copers (EC), Avoidant Copers (AC), and Social Copers (SC). Coping profiles differed across various variables including sport well-being, appraisals, and demographic factors. Overall, MAC and EC had higher levels of sport well-being. MAC viewed stressors as a challenge (i.e., opportunity) compared to EC and AC. These findings suggest the complex nature of coping in sport and that athletes should develop an assortment of coping strategies that provide different strategies for various stressful situations.     

Personal and Situational Variables That Describe Coping With Acute Stress in Competitive Sport

Abstract

The authors examined the degree to which competitive basketball players in Australia were consistent in their cognitive appraisals and coping strategies in response to 4 types of stressful situations that they had experienced during previous basketball games as functions of perceived stress intensity. The authors predicted that both approach and avoidance coping strategies would be dependent on the type of stressful event, in accord with the transactional model. The results supported that prediction: Approach strategies were more prevalent than avoidance strategies following 3 of the 4 events. Cognitive appraisals and perceived stress intensity also strongly influenced the participants' use of coping strategies, accounting for 34% of the variance.     

Appraisal and coping processes in acute time-limited stressful situations: A study of police officers

This study focuses on the appraisal and coping process in acute, time-limited stressful situations in a stress experienced group: 54 Swedish police officers. Each police officer retrospectively reported their thoughts, emotions, and actions during five recent stressful job events. The relations among appraisals of threat and of coping options, seven forms of problem- and emotion-focused coping strategies, and selfrated performance were examined. Compared with previous studies of ordinary people in stressful situations, the police officers felt less threatened, appraised the situations as more solvable, and used considerably more problem-focused forms of coping. Age had a significant effect on appraisal, while organizational position had a significant effect on coping. Using latent profile analysis, four appraisal styles were obtained. Consistency of cross-situational appraisal style amongst individuals was high. The functional relationships among variables, i.e. appraisal, coping, and performance, are discussed.     

Personal and situational variables that describe coping with acute stress in competitive sport

The authors examined the degree to which competitive basketball players in Australia were consistent in their cognitive appraisals and coping strategies in response to 4 types of stressful situations that they had experienced during previous basketball games as functions of perceived stress intensity. The authors predicted that both approach and avoidance coping strategies would be dependent on the type of stressful event, in accord with the transactional model. The results supported that prediction: Approach strategies were more prevalent than avoidance strategies following 3 of the 4 events. Cognitive appraisals and perceived stress intensity also strongly influenced the participants' use of coping strategies, accounting for 34% of the variance.     

Reactive effects of concurrent verbalization in person perception tasks

The reactive effects of concurrent verbalization (CV) on task performance in impression formation research have not been investigated, despite increasing use of this process tracing method. Since many person perception tasks involve multiple trials, assessment of reactivity should focus on carry-over and reactive-practice effects, as well as changes in task performance concurrent with thinking aloud. An experimental method for assessing these three forms of reactivity was demonstrated in an information-request impression formation task. Concurrent reactive effects were found for both continuous and intermittent CV. For the former, task processing was mildly slowed but altered in a manner suggestive of more efficient learning of, and memory for cues requested early in task performance. The substantial slowing of task processing during trial 1 intermittent CV largely disappeared by trial 2, indicating that subjects may require more practice for this method to be used effectively. The results for continuous CV are in line with recent findings which are inconsistent with certain predictions from the predominant theory of verbal protocol generation (Ericsson and Simon, 1984). Investigators using CV are urged to incorporate empirical checks for reactivity into their experimental designs.     

Application of a stress and coping model to antenatal depressive symptomatology

This study examined the utility of a stress/coping model of antenatal depressive symptomatology. The direct and moderating effects of appraisal, coping resources and coping strategies on depression were explored. A total of 242 primiparous women completed questionnaires during the third trimester of pregnancy. Predictors included life events, coping resources (social support, quality of women's earlier relationships with parents), appraisal (threat, self-efficacy) and coping strategies (wishful thinking, positive reappraisal, problem solving, emotional approach). Results of regression analyses indicated that higher depression was related to higher stressful life events, threat appraisal and wishful thinking coping, and lower positive reappraisal coping. The expected stress exacerbation effects of wishful thinking on depression were supported. There was no support for the expected stress buffering effects of coping resources and coping strategies on depression. Findings provide preliminary support for the use of a stress/coping model to guide future research into psychosocial predictors of antenatal depression.     

Thinking aloud performance: Individual consistencies over tasks

Previous research (Gilhooly, 1987) indicated considerable individual consistency in quantity and quality of thinking aloud performance over three 8-puzzle tasks. The present study investigated the consistency of individual thinking aloud performance over two types of tasks. One type was largely spatial (the 8-puzzle) and the other largely verbal (the Poisoned Foods task). Considerable individual consistency in thinking aloud performance was found within each task type and, more importantly, between types of task. Thus, subjects who produced lengthy and complete protocols on the 8-puzzle problems also did so in the Poisoned Foods task. These results are discussed in the context of Ericsson and Simon’s (1984) model of thinking aloud.     

Thinking aloud performance: Individual consistencies over tasks

Previous research (Gilhooly, 1987) indicated considerable individual consistency in quantity and quality of thinking aloud performance over three 8-puzzle tasks. The present study investigated the consistency of individual thinking aloud performance over two types of tasks. One type was largely spatial (the 8-puzzle) and the other largely verbal (the Poisoned Foods task). Considerable individual consistency in thinking aloud performance was found within each task type and, more importantly, between types of task. Thus, subjects who produced lengthy and complete protocols on the 8-puzzle problems also did so in the Poisoned Foods task. These results are discussed in the context of Ericsson and Simon’s (1984) model of thinking aloud.     

Do we need positive illusions to carry out plans? Illusion: and instrumental coping

This study tested the theory that positive illusions and instrumental (problem-focused) coping behaviours are related (Brown, J. D. (1993). Coping with stress: The beneficial role of positive illusions. In A. P. Turnbull, J. M. Patterson, S. K. Behr, D. L. Murphy, J. G. Marquis, & M. J. Blue-Banning (Eds.), Cognitive coping, families, and disability, Baltimore: Paul H. Brookes.). Positive illusion was assessed as a discrepancy between positive bias towards the self in trait ratings and positive bias towards average others, where ratings were self-favouring. Problem-focused coping was assessed through recall of coping with a recent stressful situation. The results showed that positive illusion scores were unrelated to problem-focused coping scores. It was suggested that although individuals need to be positive to cope well they do not need positive illusions. Tests for gender differences showed that males held more positive illusions than females. No gender differences were found for problem-focused coping but females used emotion-focused coping more than males to cope with a recent stressful situation.     

Coping strategies of Ethiopian immigrants in Israel: Association with PTSD and dissociation

Finklestein, M., Laufer, A. & Solomon, Z. (2012). Coping strategies of Ethiopian immigrants in Israel: Association with PTSD and dissociation. Scandinavian Journal of Psychology 53, 490–498. The aim of this study was to examine the relations between coping strategies, posttraumatic stress disorder (PTSD), and dissociation among Jewish Ethiopian refugees in Israel (following exposure to pre‐, peri‐ and post‐migration stressful events). Method: A random sample (N = 478) of three waves of refugees took part in the research (N = 165; N = 169; N = 144). Religiosity, coping strategies, stressful and traumatic events, pre‐ and peri‐ migration, post‐migration difficulties, posttraumatic symptoms, and dissociation were assessed. Results: A significant relationship was found between PTSD symptoms and avoidance coping over and above immigration wave and traumatic events. Dissociation was positively associated with passivity and antisocial coping and negatively associated with social joining and level of religiosity, over and above immigration wave and traumatic events. The findings are discussed in the light of the coping strategies employed by Ethiopian refugees.     

If at first you don't succeed: the neuroendocrine impact of using a range of strategies during social conflict

Using a variety of cognitive or behavioral strategies to manage stressful situations may be more adaptive than relying on a narrow selection. Although research has explored the psychological benefits of a range of coping responses, the physiological impact within and across stressful situations has not been examined. Moreover, research has primarily relied upon self-reports of what people believe they generally do across stressful situations, which may be subject to recall bias. This study observed and coded the range of behavioral response strategies that young adults (n=74, mean age 18.1) used to manage a laboratory-based, interpersonal conflict task and collected self-reports of the cognitive strategies used to manage similar stressors. Analyses examined the impact of response range on cortisol activity during the task. Greater range of observed response strategies predicted lower cortisol reactivity (t(133)=2.65; p=.009), whereas the range of self-reported strategies was unrelated to cortisol reactivity (t(133)=.53; p=.60). Results support observational assessment as an important supplement to self-reports of responses to stress and suggest that the range of strategies used to manage the momentary demands of a stressful situation may help explain individual differences in the impact of stress on physiological systems.     

Social support, coping, life events, and posttraumatic stress symptoms among former peacekeepers: a prospective study

This study examined both cross-sectionally and longitudinally the relationship between social support, coping strategies, additional stressful life events, and symptoms of posttraumatic stress disorder (PTSD) among Dutch former peacekeeping soldiers. Two groups of peacekeepers were investigated: 311 peacekeepers who participated in the peacekeeping operation in Lebanon between 1979 and 1985, and 499 peacekeepers who were deployed after 1990. These peacekeepers completed a questionnaire in 1996 and again in 1998. The results show that more negative social contacts and fewer positive social contacts were associated with more PTSD symptom severity. More use of the coping strategies ‘wishful thinking’ and ‘accepting responsibility’ was related to more PTSD symptoms. Conversely, more planful problem solving and seeking social support was related to less PTSD symptom severity. In addition, a bilateral relationship was found between additional stressful life events and PTSD symptom severity. The results indicate that social support and coping strategies may be valuable aspects of prevention and intervention programs.     

Recall or evaluation of chess positions revisited: the relationship between memory and evaluation in chess skill

We extend work by Holding and Reynolds (1982) on recall and problem solving with quasirandom chess positions. We tested 17 chess players on both quasirandom and structured chess positions. Consistent with the earlier study, initial recall of quasirandom chess positions is unrelated to chess skill level, and quality of the move selected in subsequent problem solving is related to skill level. However, recall following problem solving is related to chess skill level. These results support the view that pattern recognition processes underlie superior performance by skilled chess players, contrary to the conclusions of Holding and Reynolds (1982). Mechanisms such as long-term working memory retrieval structures (Ericsson & Kintsch, 1995) or templates (Gobet & Simon, 1996a) could explain the effective encoding of quasirandom positions during problem solving.     

Duty-Related Posttraumatic Stress Symptoms in 911 Telecommunicators: The Roles of Childhood Trauma Exposure and Emotion-Focused Coping

A history of childhood trauma exposure has been linked to the development of posttraumatic stress symptoms in adulthood following new exposure. Unhealthy coping behaviors that could develop or be utilized in response to early trauma could lend themselves to psychological issues in adulthood. Emotion-focused and problem-focused coping strategies in relation to stressful duty-related situations are examined as indirect pathways through which childhood trauma exposure could be associated with duty-related posttraumatic stress symptoms in 911 telecommunicators (N = 808). Multiple mediation models revealed that 3 of the 4 emotion-focused coping strategies, but not the problem-focused strategies, functioned as significant mediators in the association between childhood exposure and duty-related posttraumatic stress symptoms. Pairwise comparisons showed that self-controlling and escape-avoidance strategies were the strongest pathways of the indirect childhood trauma exposure–posttraumatic stress symptoms association. Implications of results regarding coping in response to new traumatic events in adulthood and potential research and intervention directions are discussed.