Experimental and Quasi-Experimental Designs in Visitor Studies: A Critical Reflection on Three Projects

Identifying causal relationships is an important aspect of research and evaluation in visitor studies, such as making claims about the learning outcomes of a program or exhibit. Experimental and quasi-experimental approaches are powerful tools for addressing these causal questions. However, these designs are arguably underused in visitor studies. In this article, we offer examples of the use of experimental and quasi-experimental designs in science museums to aide investigators interested in expanding their methods toolkit and increasing their ability to make strong causal claims about programmatic experiences or relationships among variables. Using three designs from recent research (fully randomized experiment, posttest only quasi-experimental design with comparison condition, and posttest with independent pretest design), we discuss challenges and tradeoffs related to feasibility, participant experience, alignment with research questions, and internal and external validity. We end the article with broader reflections on the role of experimental and quasi-experimental designs in visitor studies.     

Patient-Informed Treatment Development of Behavioral Smoking Cessation for People With Schizophrenia

The objective of this study was to use qualitative methodology to tailor and refine an existing smoking cessation intervention for the population of people who use cigarettes and are diagnosed with schizophrenia, schizoaffective, or psychotic disorder. Successive cohort design methodology was used to iteratively modify the treatment in response to qualitative participant, therapist, and consultant feedback on the intervention. Qualitative methodology for participant feedback included analysis of semistructured interviews with participants, visualization of app utilization data, and stakeholder feedback from study therapists and consultants. Using the successive cohort design, a tailored multicomponent mobile health smoking cessation intervention was developed. The intervention included mobile contingency management (i.e., financial compensation for confirmed abstinence from smoking), pharmacotherapy for smoking cessation, cognitive-behavioral counseling sessions, and the Stay Quit app for relapse prevention. Two cohorts (N = 13) were completed in the study; after each cohort, the treatment protocol was revised. The intervention is described, as well as the qualitative findings from each cohort and subsequent changes made to the intervention based upon patient and provider feedback. Metrics of patient engagement included treatment adherence (40% in Cohort 1 and 63% in Cohort 2). Both participants and therapists reported that the intervention was helpful. Over one third of participants self-reported abstinence at posttreatment. Since qualitative methodology is often underutilized in mental health treatment development, this study demonstrates the utility of the successive cohort design for treatment development of behavior change interventions for at-risk, vulnerable populations.     

Improved low-cost,MR-compatible olfactometer to deliver tobacco smoke odor

We describe a low-cost, MRI-compatible olfactometer that delivers fresh cigarette smoke odor, a challenging odorant to present, as well as other odorants. This new olfactometer retains all of the advantages of an earlier design that was capable of only delivering volatile odors (Lowen & Lukas, Behavior Research Methods, 38, 307–313, 2006). The new system incorporates a novel switching mechanism that allows it to deliver fresh smoke generated from a burning cigarette during a stimulus presentation paradigm that might be employed in a cue-reactivity experiment. An evaluation study established that the olfactometer reliably delivered smoke to the participants and that tobacco smoke was discriminated from other odorants; there were no adverse reactions to the device.     

Identifying Creativity During Problem Solving Using Linguistic Features

Creativity is commonly assessed using divergent thinking tasks, which measure the fluency, flexibility, originality, and elaboration of participant output on a variety of different tasks. This study assesses the degree to which creativity can be identified based on linguistic features of participants’ language while completing collaborative divergent thinking tasks. To this end, 78 participants’ conversational dialogs (i.e., 39 dyads) within a chat environment were collected while completing three open-ended problem-solving tasks. Expert raters scored the dialogs in terms of fluency, flexibility, elaboration, and originality, as well as three types of creative language (metaphor and simile, humor, and word play). Factor analyses indicated that these scores captured two main constructs (creativity and elaboration). The linguistic features of the participants’ language (captured computationally using natural language processing tools) accounted for significant amounts of variation in both the creativity (R² = .640) and elaboration (R² = .550) scores within linear mixed effect (LME) models. These results highlight specific linguistic features that can be used to explain large amounts of variance in constructs related to creativity.     

Predictors of Treatment Completion and Recidivism Among Intimate Partner Violence Offenders

This study examined the variables that predicted completion of an AMEND-Emerge-based program designed to treat men (n = 202) who had engaged in intimate partner violence and the rates of recidivism among those who completed versus those who failed to complete the program. Data were collected from an archival database that housed historical and demographic data on each participant; recidivism was determined by accessing publicly available court records, and was defined very broadly as any charges filed within a year after participating in the program. Overall recidivism for this sample was low (22.28%). Demographic factors demonstrated to correctly classify post-treatment recidivism included age, race, and relationship status. Whereas those who completed the program had lower recidivism than non-completers, the differences between those groups in age, race, and relationship status suggest that features of the program may not address the needs of younger African Americans who are not married.     

How have people who have stopped or reduced their alcohol consumption incorporated this into their social rituals?

Objective: Limiting alcohol consumption is beneficial for health, but can be challenging given the role alcohol plays in the rituals of many social occasions. We examined how people who stopped or reduced their alcohol consumption incorporated this change within their social rituals.

Design: We conducted 16 semi-structured one-on-one interviews with adults aged 25–65 years, who lived in Australia and had stopped or significantly reduced their alcohol consumption in the previous year.

Results: Through thematic analysis, we identified four approaches to adapting drinking rituals: replacing alcohol with other drinks, replacing drinking with other social activities, changing the meaning of drinking rituals and replacing drinking occasions with activities that achieve different goals. These approaches varied in the extent to which they reflected a low or high change in the meanings and/or behaviours attached to the ritual. Approaches involving little change, such as using alternative drinks, were more readily accepted by participants’ social companions than approaches involving more substantial changes such as replacing drinking with activities achieving different goals.

Conclusions: Considering both the role and meaning alcohol carries in social interactions, and how else these might be achieved, may assist people to stop or reduce their drinking, without sacrificing their social lives.     

The fit and implementation of sexual harassment law to workplace evaluations

Three studies used videotaped harassment complaints to examine the impact of legal standards on the evaluation of social-sexual conduct at work. Study 1 demonstrated that without legal instructions, college students' judgment strategies were highly variable. Study 2 compared 2 current legal standards, the "severity or pervasiveness test" and a proposed utilitarian alternative (i.e., the rational woman approach). Undergraduate participants taking the perspective of the complainant were more sensitive to offensive conduct than were those adopting an objective perspective. Although the utilitarian altemative further increased sensitivity on some measures, it failed to produce a principled judgment strategy. Finally, Study 3 examined the kinds of errors that full-time workers make when applying the "severity or pervasiveness" test to examine more closely the sensitivity of the subjective approach.     

The convergent and discriminant validity of subjective fit perceptions

This study examined whether employees develop perceptions about 3 different types of fit: person-organization fit, needs-supplies fit, and demands-abilities fit. Confirmatory factor analyses of data from 2 different samples strongly suggested that employees differentiate between these 3 types of fit. Furthermore, results from a longitudinal design of 187 managers supported both the convergent and discriminant validity of the different types of fit perceptions. Specifically, person-organization fit perceptions were related to organization-focused outcomes (e.g., organizational identification, citizenship behaviors, turnover decisions), whereas needs-supplies fit perceptions were related to job- and career-focused outcomes (e.g., job satisfaction, career satisfaction, occupational commitment). Although demands-abilities fit perceptions emerged as a distinct construct, they were not related to hypothesized outcomes (e.g., job performance, raises).     

Automaticity and eyewitness accuracy: a 10- to 12-second rule for distinguishing accurate from inaccurate positive identifications

Eyewitness researchers have shown that witnesses accurately choosing the culprit out of a lineup reach their decisions more quickly than those erroneously choosing an innocent individual. However, this research is silent regarding how quickly or slowly witnesses must be, in absolute terms, to indicate that they are accurate or inaccurate. Across 4 studies, the authors discovered that a time boundary of roughly 10 to 12 s best differentiated accurate from inaccurate positive identifications. Witnesses making their identification faster than 10 to 12 s were nearly 90% accurate; those taking longer were roughly 50% accurate. This finding is consistent with previous research showing that accurate witnesses are more likely than inaccurate witnesses to reach their decisions automatically, that is, quickly, without conscious thought or effort.