Participatory Evaluation: A Case Study of Involving Stakeholders in the Evaluation Process

ABSTRACT

One way to ensure that an evaluation has “utility” is to use a participatory evaluation approach where the evaluator partners with primary users to carry out various phases of an evaluation. This article provides a case study of how participatory evaluation was used in an out-of-school youth development and employment program at the Science Museum of Minnesota's Kitty Andersen Youth Science Center. Youth staff participated in a series of evaluation workshops where they learned about evaluation, made their own meaning of evaluation data, generated recommendations to improve their museum work, and ultimately took more ownership and control of their program. As a result of the workshops, youth also carried out their own formative evaluation of four museum programs. The program manager and the youth staff reaped a number of benefits from this process. The factors leading to its success are discussed.     

Public Participation and Scientific Citizenship in the Science Museum in London: Visitors’ Perceptions of the Museum as a Broker

ABSTRACT

Science museums in Europe play an emerging and important role as brokers between the public and policy-making institutions and are becoming platforms that enable scientific citizenship. To do so, museums rely on the participation of their visitors. However, little is known about the relation between visitors’ interest to participate, their engagement with science, and their perceptions of the museum as a platform of scientific citizenship. This study analyzes visitors’ interest in 3 levels of participation: Sharing opinions and feedback (the museum as “forum”); co-developing programs and activities; and participating in the governance of the museum. Quantitative analysis of the data from a survey conducted among 364 adult visitors to the Science Museum in London reveals that interest in the forum function of the museum does not depend on visitors’ prior engagement with science, but rather on how the museum enables the scientific citizenship of its visitors. However, for interest in co-development the reverse was found—previous engagement and frequent visits are more important than scientific citizenship. The forum function of the museum and its perceived role in public policy further determine visitors’ interest in museum governance.     

A Natural History Museum in Transition: Reflections on Visitor Studies in Practice

ABSTRACT

This article documents work done at The Natural History Museum, London in applying exhibit evaluation methodology to the re-configuration of an entire museum. Over a span of 20 years, traditional, curatorial, collection-centered exhibitions were replaced with those reflecting the latest trends in ‘visitor-friendly’ exhibit design. The methodology of exhibit evaluation, as it was being developed and refined in the United States, was used as the basic framework for carrying out this program. Front-end, formative, and summative evaluation techniques became the foundation for the preparation of new exhibition galleries throughout the museum. Here I reflect on the successes achieved and obstacles faced in this process of changing the status quo. The lessons learned still have relevance today for anyone who has tried to incorporate empirical, visitor-based insights into the preparation of museum exhibits and programs.     

Comparing Visitors' Conceptions of Evolution: Examining Understanding Outside the United States

ABSTRACT

In contrast to the United States, other predominantly English-speaking countries generally have higher acceptance rates of evolutionary theory. This article explores whether natural history museum audiences in these regions also have fewer misconceptions about the underlying processes of evolution. Museum visitors in Great Britain, Canada, and Australia were asked to explain how evolution worked, as well as whether or not they believed that the theory was valid. Although findings suggest that these visitors were significantly less likely to reject evolution (compared with a recent investigation conducted at natural history museums across the United States), they were just as likely to harbor misconceptions as their American counterparts.     

What Matters to Stakeholders? Measuring Values at a University Museum

ABSTRACT

Perceptions of museum performance may be linked to individual values. This study asked members of different groups that hold a stake in a university museum (e.g., Alaska Native, Art Community, Business Community, Donors/Museum Members, Military Leaders, Science Community, K–12 Teachers, University Leaders) what they valued about their museum's performance. Results indicate that key differences exist with respect to performance indicators among some groups; however, stakeholders as a whole highly value indicators related to the human face of the museum and the collections. Museums can use these results to develop and pursue appropriate goals for specific audiences and to develop performance reports that matter to stakeholders.     

(De)Constructing Paula Radcliffe: Exploring media representations of elite running,pregnancy and motherhood through cultural sport psychology

ObjectivesLittle is known about elite athletes who are mothers within the context of sociocultural expectations concerning motherhood and sport. The aim of this study was to extend such understanding by examining how the media manages and constructs one elite athlete's (Paula Radcliffe) identities within the context of motherhood and sport.DesignA qualitative approach grounded in cultural sport psychology was used to explore motherhood and athletic identity as socio-cultural creations shaped by cultural narratives (i.e., media). The psychological and behavioural implications were of interest.MethodA textual analysis (see McKee, 2003) of two issues of Runner's World magazine (March 2008, October, 2010) surrounding elite British marathon runner Paula Radcliffe's pregnancies was conducted. Visual data analysis of 37 images (see Griffin, 2010) further contextualized textual meaning(s).ResultsRadcliffe's identities were constructed within a higher order narrative: pregnancy and motherhood as redemption. This narrative had fluid meanings depending on how it framed two contrasting identities: 1. athlete and mother as one and 2. primarily a mother; athlete as secondary. An athlete and mother as one identity reinforced an elite athlete identity and high performance narrative. A primarily mother, athlete as secondary identity was linked to athletic accomplishments being downplayed and/or sacrificed in favour of motherhood.ConclusionsThis study opens a new window of cultural understanding and possibility for research and application concerning motherhood and athletic identities. These findings add to the cultural sport psychology and qualitative literature exploring elite mothering athletes.     

Negotiating motherhood and athletic identity: A qualitative analysis of Olympic athlete mother representations in media narratives

ObjectivesAlthough a growing presence within sport, elite athlete mothers have minimal presence within sport psychology research, particularly within the context of sociocultural expectations concerning motherhood and sport. The purpose of this study was to extend this understanding by examining how news media constructed elite athlete identities of prominent athlete mothers during the 2012 Olympic year.DesignA qualitative approach grounded in critical cultural sport psychology (Schinke & McGannon, 2014) was used to explore motherhood and athletic identity as socio-cultural creations shaped by media and the potential psychological implications.MethodAn ethnographic content analysis (Altheide, 1996) of 80 stories from North American news media collected from January 27, 2011 to December 31, 2012surrounding 10 prominent US athlete mothers was conducted. Visual data analysis of 99 images and 4 videos accompanying were also analyzed to contextualized the textual meaning(s) (Altheide & Schneider, 2013).ResultsA central narrative within which identities were constructed was identified: motherhood and athletics as a transformative journey. The meaning(s) and implication(s) of this narrative emerged differently depending on how two athlete mother identities were constructed: 1. athlete and mother in conflict and 2. athlete mother as superwoman.ConclusionsThis study extends critical cultural sport psychology and qualitative literature exploring elite mother athletes and the implications for athletic career (dis)engagement.     

Evaluation of ICT Applications in the New Lleida Museum,Spain

ABSTRACT

This article analyzes the use of different multimedia displays (3D recreations, interactives, audiovisuals) in the Museum of Lleida (in northeast Spain) which opened its new venue in November 2007. This art and archaeological museum has actively used information and computer technology (ICT) applications in its permanent collection in order to improve communication with its public. However, after a couple of years it was realized that some applications were not properly used by visitors and most of them did not fulfill all the expectations. For this reason, the group Òliba from the Universitat Oberta de Catalunya conducted an evaluation in 2008 aiming to find the reasons for such behavior. A new methodological framework for evaluating ICT applications is proposed and its application at the Museum of Lleida is discussed. The framework focuses not only on visitor behavior, but also on technological aspects (i.e., usability, comfort, accessibility) and communication practices.     

An Experience for the Lifelong Journey: The Long-Term Effect of a Class Visit to a Science Center

ABSTRACT

The aim of this study was to explore the multiple outcomes of a class visit to a science center, and to investigate changes in these outcomes over time. The study is significant because relatively little research has been conducted on the long-term effects of school museum visits. The study was carried out in the National Museum of Science, Technology and Space in Israel, which is the largest science museum in the country. Participants were 8th-grade students who had a guided visit in the museum. Students from this class were interviewed immediately after and then again 16 months after the visit. The short- and the long-term interviews were analyzed according to 3 main categories that addressed meaningful learning outcomes: connecting knowledge, communicating knowledge, and fostering lifelong learning. After 16 months the students retained details of the experience; indicated the contribution of the visit to their knowledge; and emphasized peer interactions during the visit. This highlights the significant educational values that students take to their life's journey from the informal learning experience.     

Ethnic Identity Gratifications Selection and Avoidance by African Americans: A Group Vitality and Social Identity Gratifications Perspective

Abstract

This study explores the relatively new idea that individuals engage with media in an effort to meet their social identity needs. Specifically, the study broadens the social identity gratifications (SIG) approach to the domain of ethnicity by examining how African Americans’ ethnic identity gratifications selection and avoidance are related to their perceptions of ingroup vitality. Two mediation models involving level of ethnic identification are proposed. Although the model of television selection is not supported, the model of television avoidance is supported. Implications and future research are discussed.     

Liking the (Funny) Messenger: The Influence of News Parody Exposure,Mirth, and Predispositions on Media Trust

ABSTRACT

This article features a multi-study research effort (Study 1 N = 331; Study 2 N = 317) examining how predispositions toward a humor source and the perceived humor (i.e., mirth) of a related comedic message can, together, influence media trust-based expectations. Noting the revered status and cultural prominence of various news parody show hosts, this article proposes that feelings of favorability for a news parody humor source (for example, Jon Stewart or John Oliver) can translate into positive perceptions of the press upon exposure to news parody messages. Drawing from principles of the heuristic-systemic information processing model (HSM) and affective disposition theory, the study findings indicate that one’s affective disposition toward a news parody source can have an indirect effect on media trust, as mediated by a feeling of mirth. Upon conducting a test of moderated mediation, the effect is demonstrated to be conditioned by one’s news parody orientation. That is, affective disposition’s mediated effect is most pronounced among those who are least inclined to see news parody as a legitimate/appropriate source of news. Ultimately, this exploratory research contributes to a more nuanced understanding of how news parody programming may influence perceptions of the news media as an institutional entity.     

Mother runners in the blogosphere: A discursive psychological analysis of online recreational athlete identities

ObjectivesQualitative research on physically active mothers has shown that recreational sport may allow women to resist good mother ideals that often constrain exercise. The purpose of this study was to extend this understanding in a socio-cultural context by examining how recreational athlete mother identities were constructed within one form of new media - blogging.DesignA qualitative approach grounded in discursive psychology was used to theorize athlete mother identities as subject positions constructed within particular discourses circulated within a blog.MethodCritical discourse analysis of 30 stories and 177 reader comments from the North American online running community Another Mother Runner, was conducted. Visual data analysis of 102 images accompanying stories also contextualized the textual meanings of discourses and subject positions.ResultsTwo primary discourses were identified: discourse of transformation and empowerment and a discourse of disruption and resolution. Subject positions constructed within these two intersecting discourses were role mother/advocate and resilient mother runner.ConclusionsThis study extends discursive psychology literature in sport and exercise psychology as well as media work on athlete mother identities within sport psychology into the realm of the internet and identity construction.     

Visitor Usage of Digital and Physical Artifacts in Two Museum Programs

ABSTRACT

This article reports on a study designed to explore how digital and physical reproductions of original artifacts could be incorporated into museum programs that teach visitors about mechanical motion. Museum visitors were introduced to 3 different types of models through 2 different programs: an engineering design challenge and an exhibit interpretation exploring mechanical motion in every-day objects. Research, based on visitor observation and interviews (n = 79), explored how user interaction with the digital and physical models compared with interactions with the original artifacts, assessed what effect the integration of digital and physical models had on learning, and examined what the learning experience looked like for learners using the mechanisms in the museum activities. Analysis of the learning conversations in additional in-depth case studies (n = 8) for both programs provides a detailed look at how visitors integrated the 3 different model types into their group's experience. Results showed that visitors' usage of, attitudes toward, and learning from the three different types of mechanical models varied based on the visitor or educators' perceived value of the particular model, which was impacted by the context in which the objects were presented.     

“Dad look,she's sleeping”: Parent–Child Conversations about Human Remains

ABSTRACT

Research on parent–child learning in museums has shown that parents often use conversation to facilitate their children's exploration of exhibits. This observational study investigated whether the patterns of talk found in science and history exhibits are used during parent–child conversations about challenging content such as death and dying and forensics. This research was conducted at a traveling exhibit, Mysterious Bog People, which was on display at the Carnegie Museum of Natural History in Pittsburgh, Pennsylvania. Thirty-four families were observed as they interacted with and explored the exhibit. Parent comfort with the content of the exhibit was found to have an impact on the conversations they had with their children and their enjoyment of the exhibit overall. Implications for museum mediation are discussed.     

Implications of the Social Model of Disability for Visitor Research

ABSTRACT

Museum professionals have long held the view that their institutions can play a central role in supporting learning and inclusion but evidence seems to be in sort supply in the case of visitors with disabilities. Until recently, most of the work conducted in museums relating to learning and access for this particular visitor group has been about developing products and services rather than assessing their impact. The lack of research in this area reflects the slow development of disability studies as a distinct field and the fact that anti-discrimination and human rights legislation only extended to those with disabilities in the mid to late 1990s. The author introduces one approach to negotiating a balanced design of visitor research and the principles of the emancipatory research framework. She argues that this framework has the capacity to delve into the experiences of people with disabilities and provide a deeper analysis that would not otherwise be possible. The Heritage Education For All project will be used as an example to demonstrate how this approach can be developed in practice, and to show that informal learning programs designed using an emancipatory insight have the capacity to enable people with disabilities to gain new insights into their lives and identities.     

A Review of: “Learning in Places: The Informal Education Reader”

ABSTRACT

Fluctuations in visitor numbers over the past 50 years at the Norwegian University of Science and Technology (NTNU) Museum of Natural History and Archaeology in Trondheim, Norway, indicate an overall decline in visitation since the 1990s but with some positive figures in the last years. Separate data available from 1977 to 2006 show a strong correlation between the numbers of school students and the general public. The general public paid relatively few visits in December, but visitation peaked in summer and autumn. Sundays and midweek were the busiest days. This article discusses factors that may influence the short-term, monthly, and daily distribution of visitors as well as the more long-term variations. These include the development of museums in Norway and how the historical, cultural, and nationalistic background may influence the topics and information in exhibitions, and hence the public interest in what the museum has to offer.     

Evaluating Family Interactions to Inform Exhibit Design: Comparing Three Different Learning Behaviors in a Museum Setting

ABSTRACT

How do we know that families are learning during the museum visit itself? In order to address this question, this study focuses on three frequently evaluated indicators of during-visit learning: time spent, exhibit engagement, and interpretive talk. Each of these types of data makes certain assumptions about family learning and how it occurs. Four hundred ninety-three family interactions at 25 different children's museum exhibits were analyzed to examine what the three measures reveal about family learning behavior. When compared, the three measures were not highly correlated, suggesting that generalizing information about the family experience from a single indicator is insufficient. Looking at all three indicators simultaneously can help evaluators begin to explore the relationships between families’ learning behaviors, and think about how exhibit characteristics can influence potential learning opportunities in informal settings.     

Re-conceptualizing Museum Audiences: Power,Activity, Responsibility

ABSTRACT

This article offers a theoretical overview of how diverse disciplines, such as cultural studies, sociology, philosophy of art, education, and marketing, have contributed to the evolving conceptualization of museum audiences over the past 50 years. Audience research has moved through different paradigms. Every shift in the way audiences are viewed unavoidably influences the way museum professionals view themselves, their role, and the way they interact with their audiences. The most recent audience conceptualization envisions visitors as active interpreters who selectively construct meaning based on their personal experiences, associations, biases, and sense of identity, whereas the museum is envisioned as an open work that is only completed by the visitor. However, certain challenges loom over this new audience conceptualization. It is argued that such visualization underestimates power issues while romanticizing the power of audience activity, thereby ignoring issues of responsibility.     

Visitor Self-Report Behavior Mapping as a Tool for Recording Exhibition Circulation

ABSTRACT

The evaluation of visitor flow within a museum or exhibition has been a topic of interest for decades with several research approaches taken over the years. Direct observation or visitor tracking during museum occupancy is the most popular technique, but it generally requires substantial amounts of time and financial resources. An alternative approach to direct observation—visitor self-mapping—is presented using data obtained from 2 short-term, small-budget evaluations of a world-class collection museum. Results show that self-mapping provides usable data with more than 90% of maps having tracking data for the entire museum. Maps varied in the amount of detail, but more than 60% of visitors provided details beyond what was required. In Study 1, movement patterns, sweep rate indices, and timing data suggest that the mapping data accurately reflected the visitor experience. Study 2 directly paired the self-mapping method used in Study 1 with unobtrusive behavioral observations to address the reliability and validity of the new approach. A discussion compares the relative costs and benefits of the new approach with more conventional direct observation techniques and provides directions for future research.