Experimental and Quasi-Experimental Designs in Visitor Studies: A Critical Reflection on Three Projects

Identifying causal relationships is an important aspect of research and evaluation in visitor studies, such as making claims about the learning outcomes of a program or exhibit. Experimental and quasi-experimental approaches are powerful tools for addressing these causal questions. However, these designs are arguably underused in visitor studies. In this article, we offer examples of the use of experimental and quasi-experimental designs in science museums to aide investigators interested in expanding their methods toolkit and increasing their ability to make strong causal claims about programmatic experiences or relationships among variables. Using three designs from recent research (fully randomized experiment, posttest only quasi-experimental design with comparison condition, and posttest with independent pretest design), we discuss challenges and tradeoffs related to feasibility, participant experience, alignment with research questions, and internal and external validity. We end the article with broader reflections on the role of experimental and quasi-experimental designs in visitor studies.     

Extracting parity and magnitude from Arabic numerals: developmental changes in number processing and mental representation.

Children from Grades 2, 3, 4, 6, and 8 (7.8, 9.2, 9.8, 11.7, and 13. 6 years old, respectively) made speeded, bimanual parity (odd/even) judgments of the Arabic numerals 0-9. Analysis of response times indicated that from fourth grade on, parity information is retrieved directly from memory rather than being extracted by means of a mental calculation strategy. As early as Grade 3, children exhibited the SNARC (Spatial-Numerical Association of Response Codes) effect, where small numbers were responded to faster with the left hand than with the right hand, while the converse held true for large numbers. This finding, previously demonstrated only in adults, confirms that (a) children represent magnitude information in the form of a left-to-right oriented mental number line, and (b) this information is accessed obligatorily even when irrelevant. Finally, although the SNARC effect remained strong at Grade 4, it was attenuated at Grades 6 and 8 by a linguistic effect based on associations between the unmarked adjectives "even" and "right" and between the marked adjectives "odd" and "left." Copyright 1999 Academic Press.     

The residual spatial abilities of hippocampally lesioned rats can be enhanced by peripheral sympathetic-adrenal interventions

In order to test the possible effectiveness of peripheral interventions with the adrenergic system for the alleviation of certain disorders that typically follow bilateral hippocampal lesions, rats with bilateral lesions of the hippocampus, the overlying neocortex, or sham operations were tested at two postoperative times in the Morris water maze, a frequently used "spatial task." Half of the animals in all groups were exposed to the adrenergic manipulations, i.e., a chronic, 7-day, systemic bretylium regime (5 mg/kg) and, in addition, a peripheral injection of norepinephrine (4 micrograms/kg) 30 min before the start of each training day. The other half received saline chronically and a single saline injection before each training day. Five days of training were given at each of the two training periods. The first began 7 days after surgery while the second began 33 days after surgery. As expected, the hippocampally lesioned animals were severely impaired in the task. The adrenergic treatment produced enhanced performances in the rats with hippocampal lesions at both training sessions, although the improvement was greatest at the later period. Although the animals receiving the pharmacologic treatment located the general area of the hidden platform better than the saline-treated animals with hippocampal lesions, the treated animals were still impaired, swimming directly to the hidden platform on fewer trials than did animals in the other groups.     

Validation of a reflective thinking instrument for third-year undergraduate nursing students participating in high-fidelity simulation

Background: nursing students are required to think reflectively in both real and simulated clinical practice. Although the Reflective Thinking instrument is reliable in its measurement of reflective thinking, its validity is unknown. Method: confirmatory factor analysis was undertaken in an iterative manner within a non-equivalent control-group study to measure nursing students’ reflective thinking and satisfaction with high-fidelity simulation. The validity and reliability of the Reflective Thinking instrument was tested. Results: the resulting instrument consisted of 15 items across four factors. The final model was plausible as it demonstrated ‘goodness of fit’; however, it had neither convergent or discriminatory validity, nor reliability. Internal consistency was fair to reasonable for the four subscales. Conclusion: the testing of the Reflective Thinking instrument using confirmatory factor analysis is a vital initial step in the modification of this instrument. Further development using exploratory factor analysis will enhance the instrument’s relevance within high-fidelity simulation.     

Toward Developing Laboratory-Based Parent–Adolescent Conflict Discussion Tasks that Consistently Elicit Adolescent Conflict-Related Stress Responses: Support from Physiology and Observed Behavior

Parent–adolescent conflict poses risk for youth maladjustment. One potential mechanism of this risk is that stress in the form of increased arousal during conflict interactions results in adolescents’ impaired decision-making. However, eliciting consistent adolescent stress responses within laboratory-based tasks of parent–adolescent conflict (i.e., conflict discussion tasks) is hindered by task design. This limitation may stem from how conflict topics are assessed and selected for discussion. Within a sample of 47 adolescents (ages 14–17) and parents, we investigated whether a modified version of a conflict discussion task could elicit physiological (i.e., arousal) and behavioral (i.e., hostility) displays of adolescents’ conflict-related stress responses. We assessed parent–adolescent conflict via structured interview to identify topics for dyads to discuss during the task. We randomly assigned dyads to complete a 5-min task to discuss either a putatively benign topic (i.e., control condition) or a conflict topic while undergoing direct assessments of continuous arousal. Trained raters coded dyad members’ hostile behavior during the task. Adolescents in the conflict condition exhibited significantly greater levels of arousal than adolescents in the control condition. We observed an interaction between discussion condition and baseline conflict. Specifically, higher baseline conflict predicted greater hostile behavior for adolescents in the conflict condition, yet we observed the inverse relation for adolescents in the control condition. Our modified laboratory discussion task successfully elicited both physiological and behavioral displays of adolescent conflict-related stress. These findings have important implications for leveraging experimental paradigms to understand causal links between parent–adolescent conflict and adolescent psychopathology, and their underlying mechanisms.     

Mindfulness Moderates the Association Between Perceived Burdensomeness and Suicide Ideation in Adults With Elevated Depressive Symptoms

A conceptual model in which the associations between perceived burdensomeness and suicide ideation, and between thwarted belongingness and suicide ideation, are moderated by mindfulness was examined. Participants were 218 undergraduates (mean age = 20.81) with moderate‐to‐severe depressive symptoms. Mindfulness significantly moderated the association between perceived burdensomeness and suicide ideation; participants with thoughts of burdensomeness experienced lower levels of suicide ideation if they were high, as opposed to low, in mindfulness. Mindfulness did not significantly moderate the association between thwarted belongingness and suicide ideation. Implications for intervention and elaboration of the interpersonal psychological theory of suicide are discussed.     

The Latent Change Score Model: A More Flexible Approach to Modeling Time in Self-Regulated Learning

Purpose

This research advances understanding of empirical time modeling techniques in self-regulated learning research. We intuitively explain several such methods by situating their use in the extant literature. Further, we note key statistical and inferential assumptions of each method while making clear the inferential consequences of inattention to such assumptions.

Design/Methodology/Approach

Using a population model derived from a recent large-scale review of the training and work learning literature, we employ a Monte Carlo simulation fitting six variations of linear mixed models, seven variations of latent common factor models, and a single latent change score model to 1500 simulated datasets.

Findings

The latent change score model outperformed all six of the linear mixed models and all seven of the latent common factor models with respect to (1) estimation precision of the average learner improvement, (2) correctly rejecting a false null hypothesis about such average improvement, and (3) correctly failing to reject true null hypothesis about between-learner differences (i.e., random slopes) in average improvement.

Implications

The latent change score model is a more flexible method of modeling time in self-regulated learning research, particularly for learner processes consistent with twenty-first-century workplaces. Consequently, defaulting to linear mixed or latent common factor modeling methods may have adverse inferential consequences for better understanding self-regulated learning in twenty-first-century work.

Originality/Value

Ours is the first study to critically, rigorously, and empirically evaluate self-regulated learning modeling methods and to provide a more flexible alternative consistent with modern self-regulated learning knowledge.

     

The effects of a public address system on the off-task behavior of elementary physical education students

The purpose of this study was to determine the effects of teacher feedback delivered via a public address system on the off-task behavior of elementary-school physical education students. A multiple baseline design across three classes was used in this investigation. Results indicated a consistent decline in off-task behavior when the public address feedback system was used.