A Qualitative Approach to Responsible Conduct of Research (RCR) Training Development: Identification of Metacognitive Strategies

Although Responsible Conduct of Research (RCR) training is common in the sciences, the effectiveness of RCR training is open to question. Three key factors appear to be particularly important in ensuring the effectiveness of ethics education programs: (1) educational efforts should be tied to day-to-day practices in the field, (2) educational efforts should provide strategies for working through the ethical problems people are likely to encounter in day-to-day practice, and (3) educational efforts should be embedded in a broader program of on-going career development efforts. This article discusses a complex qualitative approach to RCR training development, based on a sensemaking model, which strives to address the afore-mentioned training concerns. Ethnographic observations and prior RCR training served the purpose of collecting information specific to a multi-disciplinary and multi-university research center with the goal of identifying metacognitive reasoning strategies that would facilitate ethical decision-making. The extensive qualitative analyses resulted in the identification of nine metacognitive reasoning strategies on which future RCR training will be developed. The implications of the findings for RCR training in the sciences are discussed.     

Effectiveness of a responsible conduct of research course: a preliminary study

Training in the responsible conduct of research (RCR) is required for many research trainees nationwide, but little is known about its effectiveness. For a preliminary assessment of the effectiveness of a short-term course in RCR, medical students participating in an NIH-funded summer research program at the University of California, San Diego (UCSD) were surveyed using an instrument developed through focus group discussions. In the summer of 2003, surveys were administered before and after a short-term RCR course, as well as to alumni of the courses given in the summers of 2002 and 2001. Survey responses were analyzed in the areas of knowledge, ethical decision-making skills, attitudes about responsible conduct of research, and frequency of discussions about RCR outside of class. The only statistically significant improvement associated with the course was an increase in knowledge, while there was a non-significant tendency toward improvements in ethical decision-making skills and attitudes about the importance of RCR training. The nominal impact of a short-term training course should not be surprising, but it does raise the possibility that other options for delivering information only, such as an Internet-based tutorial, might be considered as comparable alternatives when longer courses are not possible.     

Student perceptions of the effectiveness of education in the responsible conduct of research

Responsible conduct of research (RCR) courses are widely taught, but little is known about the purposes or effectiveness of such courses. As one way to assess the purposes of these courses, students were surveyed about their perspectives after recent completion of one of eleven different research ethics courses at ten different institutions. Participants (undergraduate and graduate students, post-doctoral fellows and faculty, staff and researchers) enrolled in RCR courses in spring and fall of 2003 received a voluntary, anonymous survey from their instructors at the completion of the course. Responses were received from 268 participants. Seventy-seven percent of open-ended responses listed specific kinds of information learned; only a few respondents talked about changes in skills or attitudes. The perception that courses did more to provide information than to foster skills or attitudes was verified in quantitative responses (P<0.0001). Over 75% of the respondents specifically noted that courses were useful in preparing them to recognize, avoid, and respond to research misconduct. The two principal findings of this multi-institutional study are that respondents reported: (1) a wide variety of positive outcomes for research ethics courses, but that (2) the impact on knowledge was greater than that for changes in skills or attitudes.     

A Meta-Analysis of Ethics Instruction Effectiveness in the Sciences

Scholars have proposed a number of courses and programs intended to improve the ethical behavior of scientists in an attempt to maintain the integrity of the scientific enterprise. In the present study, we conducted a quantitative meta-analysis based on 26 previous ethics program evaluation efforts, and the results showed that the overall effectiveness of ethics instruction was modest. The effects of ethics instruction, however, were related to a number of instructional program factors, such as course content and delivery methods, in addition to factors of the evaluation study itself, such as the field of investigator and criterion measure utilized. An examination of the characteristics contributing to the relative effectiveness of instructional programs revealed that more successful programs were conducted as seminars separate from the standard curricula rather than being embedded in existing courses. Furthermore, more successful programs were case based and interactive, and they allowed participants to learn and practice the application of real-world ethical decision-making skills. The implications of these findings for future course development and evaluation are discussed.     

Developing a Framework for Assessing Responsible Conduct of Research Education Programs

Education in the responsible conduct of research (RCR) in the United States has evolved over the past decade from targeting trainees to including educational efforts aimed at faculty and staff. In addition RCR education has become more focused as federal agencies have moved to recommend specific content and to mandate education in certain areas. RCR education has therefore become a research-compliance issue necessitating the development of policies and the commitment of resources to develop or expand systems for educating faculty and staff and for assuring compliance. These changes implied the need to develop a program evaluation model that could be applied to institutional RCR education programs, which were expected to differ from traditional academic credit-bearing courses targeting trainees. Information gleaned from the examination of corporate compliance models was analyzed in order to create a program evaluation module that could be used to document and assess educational programs focused on teaching RCR. A programmed series of questions for each of the nine RCR content areas identified by the United States Office of Research Integrity was created based on a performance-monitoring evaluation model. The questions focus on educational goals, resources provided to support the educational efforts, educational content, content delivery, educational outcomes, compliance requirements and feedback. Answers collected in response to the questions could be used to both document and continually improve the quality of RCR educational programs through on-going formative assessment and feedback.     

Graduate Socialization in the Responsible Conduct of Research: A National Survey on the Research Ethics Training Experiences of Psychology Doctoral Students

Little is known about the mechanisms by which psychology graduate programs transmit responsible conduct of research (RCR) values. A national sample of 968 current students and recent graduates of mission-diverse doctoral psychology programs completed a Web-based survey on their research ethics challenges, perceptions of RCR mentoring and department climate, whether they were prepared to conduct research responsibly, and whether they believed psychology as a discipline promotes scientific integrity. Research experience, mentor RCR instruction and modeling, and department RCR policies predicted student RCR preparedness. Mentor RCR instruction, department RCR policies, and faculty modeling of RCR behaviors predicted confidence in the RCR integrity of the discipline. Implications for training are discussed.     

Using electronic discussion boards to teach responsible conduct of research

This study presents the results of a survey of student satisfaction with electronic discussion boards in a course on the responsible conduct of research (RCR). On a 1–5 scale, the respondents stated that the use of the electronic discussion board was an effective teaching tool (4.71), that it enabled them to get feedback from their peers (4.43), that it helped promote discussion and debate (4.36), that it helped them learn how to analyze ethical dilemmas in research (4.36), and that they would consider using an electronic discussion board, if they ever taught a course themselves (4.76). In their written comments, the respondents indicated that electronic discussion boards are a convenient way of promoting debate and in-depth discussion. These results suggest, but do not prove, that discussion boards can promote debate and discussion in courses on research ethics. Instructors who teach RCR should consider using electronic discussion boards in regular or online courses, and they should consider studying the effectiveness of electronic discussion boards in research ethics education. Although electronic discussion boards cannot replace the face-to-face interaction that occurs in a classroom setting, they may provide a useful medium for the exchange of ideas and opinions online.     

Teaching problem solving skills to teachers-in-training: An analogue experimental analysis of three methods

This analogue study sought to determine the relative effectiveness of different instructional methods for teaching problem solving skills to teachers-in-training. Three instructional conditions (didactic, didactic + modeling, and didactic + modeling with rehearsal/feedback) and a control condition were compared on measures of problem identification and problem analysis. Results indicated that the didactic + modeling and didactic + modeling with rehearsal/feedback conditions were superior to didactic training in teaching problem-solving skills. Discussed are possible explanations for differences between the instructional conditions, the implications of these findings, limitations of this study, and directions for further teacher-training research in problem solving.     

Measures of Mentoring,Department Climate,and Graduate Student Preparedness in the Responsible Conduct of Psychological Research

Drawing upon two independent national samples of 201 and 241 psychology graduate students, this article describes the development and psychometric evaluation of 4 Web-based student self-report scales tapping student socialization in the responsible conduct of research (RCR) with human participants. The Mentoring the Responsible Conduct of Research Scale (MRCR) is composed of 2 subscales assessing RCR instruction and modeling by research mentors. The 2 subscales of the RCR Department Climate Scale (RCR-DC) assess RCR department policies and faculty and student RCR practices. The RCR Preparedness scale (RCR-P) and the RCR Field Integrity scale (RCR-FI) measure respectively students' confidence in their ability to conduct research responsibly and their belief in the RCR integrity of psychology as a discipline. Factor analysis, coefficient alphas, correlations, and multiple regression analyses demonstrated each of the scales had good internal consistency and concurrent and construct validity.     

Rates of Specific Antecedent Instructional Practices and Differences Between Title I and non-Title I Schools

The use of effective instructional strategies is clearly emphasized in current educational reform, especially in the area of reading. The purposes of this study were to investigate the rates at which specific instructional practices (i.e., attention signals, prior knowledge supports, previews, instructor modeling, student modeling, organizational prompts) were utilized during literacy time in elementary schools, determine if there were relationships among the instructional variables, and explore if teachers in Title I schools and teachers in non-Title I schools differed in their use of specific practices. Participants included teachers and students from 35 classrooms who were each observed for 5 hours, resulting in a total of 175 observation hours. The Setting Factors Assessment Tool (SFAT) was used to measure the antecedent instructional variables. Main results included that teachers in non-Title I classrooms used significantly more prior knowledge references than teachers in Title I schools; the effect size for this finding was large. Several correlations among the instructional variables were significant. The article concludes with a discussion of the main findings, implications for future research and the limitations of this study.     

America COMPETES at 5 years: An Analysis of Research-Intensive Universities’ RCR Training Plans

This project evaluates the impact of the National Science Foundation's (NSF) policy to promote education in the responsible conduct of research (RCR). To determine whether this policy resulted in meaningful RCR educational experiences, our study examined the instructional plans developed by individual universities in response to the mandate. Using a sample of 108 U.S. institutions classified as Carnegie “very high research activity”, we analyzed all publicly available NSF RCR training plans in light of the consensus best practices in RCR education that were known at the time the policy was implemented. We found that fewer than half of universities developed plans that incorporated at least some of the best practices. More specifically, only 31% of universities had content and requirements that differed by career stage, only 1% of universities had content and requirements that differed by discipline; and only 18% of universities required some face-to-face engagement from all classes of trainees. Indeed, some schools simply provided hand-outs to their undergraduate students. Most universities (82%) had plans that could be satisfied with online programs such as the Collaborative Institutional Training Initiative's RCR modules. The NSF policy requires universities to develop RCR training plans, but provides no guidelines or requirements for the format, scope, content, duration, or frequency of the training, and does not hold universities accountable for their training plans. Our study shows that this vaguely worded policy, and lack of accountability, has not produced meaningful educational experiences for most of the undergraduate students, graduate students, and post-doctoral trainees funded by the NSF.     

THE COMPARATIVE EFFECTIVENESS OF WEB-BASED AND CLASSROOM INSTRUCTION: A META-ANALYSIS

Meta-analytic techniques were used to examine the effectiveness of Web-based instruction (WBI) relative to classroom instruction (CI) and to examine moderators of the comparative effectiveness of the 2 delivery media. The overall results indicated WBI was 6% more effective than CI for teaching declarative knowledge, the 2 delivery media were equally effective for teaching procedural knowledge, and trainees were equally satisfied with WBI and CI. However, WBI and CI were equally effective for teaching declarative knowledge when the same instructional methods were used to deliver both WBI and CI, suggesting media effects are spurious and supporting Clark's (1983, 1994) theory. Finally, WBI was 19% more effective than CI for teaching declarative knowledge when Web-based trainees were provided with control, in long courses, and when trainees practiced the training material and received feedback during training. Study limitations and directions for future research are discussed.     

A comparison of three interventions for increasing oral reading performance: Application of the instructional hierarchy

The instructional hierarchy is a behavior-analytic model that links level of academic skill development (i.e., acquisition, fluency, generalization, adaptation) with appropriate instructional techniques. The present study used the instructional hierarchy to compare the effects of three instructional interventions (listening passage preview, subject passage preview, and taped words) on subjects' oral reading performance on word lists and passages. Subjects were 4 male students with learning disabilities who ranged in age from 8 years 10 months to 11 years 11 months. A multielement design was used to compare the effects of the three interventions to each other and to baseline. Results indicated that the listening passage preview intervention (which contained modeling, drill, and generalization components) produced the largest performance gains. The implications of these results for selecting academic interventions based on the instructional hierarchy are discussed.     

Preference assessment training via self‐instruction: A replication and extension

We examined the effects of a self‐instructional and feedback package on participants’ implementation of a paired‐stimulus preference assessment. Specifically, in Experiment 1, we used a multiple baseline design across participants to replicate and extend the results of Graff and Karsten (2012) by evaluating the effectiveness of their self‐instructional manual. A majority of the participants (i.e., 5 of 7 undergraduate students and 4 of 5 in‐home behavior technicians) achieved mastery with the self‐instructional package. The remaining participants met the mastery criterion after brief modeling and feedback sessions. In Experiment 2, we identified the most effective component of the feedback condition from Experiment 1 when a self‐instructional package was not sufficient. Brief feedback sessions in which participants received a list of the targeted responses plus information regarding accuracy of emitted responses was sufficient for 5 of 6 participants to achieve mastery.     

The life and times of PSI

This paper describes the essential features of the personalized system of instruction (PSI). Results from outcome research examining the effectiveness of PSI-based courses relative to traditional methods provide unequivocal support for the superiority of PSI. Parametric studies, or component analyses, show that the mastery requirement, immediate performance feedback, and review units are the key features underlying high quality student performances in PSI courses. The use of student proctors as peer-tutors, optional lectures, and selfpacing do not, in and of themselves, appear to be vital to student success in PSI courses. Despite its superiority, PSI has not supplanted traditional methods as the dominant pedagogical system in higher education. Difficulties inherent in overcoming the inertia of the lecture within our established instructional system, the implications of PSI for that system, and the Zeitgeist that permeates educational reform are the major obstacles to widespread adoption of PSI.     

Teaching chained skills in a non-school setting using a divided half instructional format

This investigation evaluated the use of dyadic instructional arrangements in teaching chained skills to four students with moderate mental retardation. Each instructional dyad consisted of two students. Each of the three cooking skills was divided into two equal parts, and each student in a dyad received direct instruction on one part of the task analysis during each instructional session. During the following session, the two parts were reversed and a student received instruction on the second part. A constant time delay procedure was used to teach the targeted skills in an off-campus setting. A multiple probe design across skills and replicated across students evaluated the effectiveness of the instructional procedure and teaching arrangement. Results indicate that each of the four students learned the three cooking skills. Implications for future research are discussed.     

团队人格组成、团队过程对团队有效性的作用

由于团队对组织的重要性,团队有效性的研究倍受关注。通过对342个大学生短期团队完成决策任务的研究,旨在探讨团队人格组成与团队中间过程对团队有效性的作用机制。研究以自陈式问卷调查收集数据,采用层次回归与结构方程模型进行统计分析。研究结果表明：团队过程变量（协作、信念、凝聚力）在团队人格组成（外向性、宜人性、神经质）与团队有效性之间起中介作用,横断研究的I-P-O整体模型得到验证     

Classroom treatment of autistic children: II. individualized instruction in a group

This study was designed to assess the feasibility of providing individualized instruction in a classroom of autistic children. In particular, we investigated whether students' unsupervised responding was a variable which would influence the effectiveness of individualized instruction procedures. Initially, one teacher attempted to teach different academic behavior to four autistic children simultaneously. During these sessions the teacher rotated from child to child, providing individualized instructions and reinforcers to each. We found, however, that each child stopped responding whenever the teacher rotated to a different child and, further, no academic progress occurred for any of the children during these sessions. In a multiple baseline design, treatment procedures were then introduced (prompt fading, chaining, and programmed instructional materials) to teach each student to continue working for an extended period of time after each instruction from the teacher. After treatment, sessions of individualized instruction in a group were reintroduced for each child. The results showed that unsupervised responding was an important variable influencing the effectiveness of individualized instruction. Specifically, the data show that (1) unsupervised responding, acquired during treatment, generalized to the group setting; (2) after treatment, each child began to make progress on his assigned tasks; and (3) programming instructional materials in small steps appeared to be necessary in order to ensure the maintenance of unsupervised responding across changes in instructional materials. By the end of this study it was possible for one teacher to simultaneously teach four autistic children — with each child learning at his own rate, on his own task, with minimal supervision from the teacher. It is suggested that the generalizability of these results, to other settings where continuous supervision is difficult, may be an important concern of future research.     

Mentoring for Responsible Research: The Creation of a Curriculum for Faculty to Teach RCR in the Research Environment

Despite more than 25 years of a requirement for training in the responsible conduct of research (RCR), there is still little consensus about what such training should include, how it should be delivered, nor what constitutes “effectiveness” of such training. This lack of consensus on content, approaches and outcomes is evident in recent data showing high variability in the development and implementation of RCR instruction across universities and programs. If we accept that one of the primary aims of instruction in RCR/research ethics is “to foster a community of social responsibility” (Antes et al. 2009: 398), then it makes sense to consider the research environment itself—where learning one’s science happens where one also engages in social interaction around that science. In order to take the best advantage of that already existing/naturally occurring research environment, the authors, through a deliberative, collaborative, and integrative process, crafted a workshop curriculum meant to arm research faculty with concrete and specific tools to effectively introduce research ethics in the context of the research environment.     

Specificity of task constraints and effects of visual demonstrations and verbal instructions in directing learners' search during skill acquisition

In the present study, the efficacy of visual demonstrations and verbal instructions as instructional constraints on the acquisition of movement coordination was investigated. Fifteen participants performed an aiming task on 100 acquisition and 20 retention trials, under 1 of 3 conditions: a modeling group (MG), a verbally directed group (VDG), and a control group (CG). The MG observed a model intermittently throughout acquisition, whereas the VDG was verbally instructed to use the model's movement pattern. Participants in the CG received neither form of instruction. Kinematic analysis revealed that compared with verbal instructions or no instructions, visual demonstrations significantly improved participants' approximation of the model's coordination pattern. No differences were found in movement outcomes. Coordination data supported the visual perception perspective on observational learning, whereas outcome data suggested that the modeling effect is mainly a function of task constraints, that is, the novelty of a movement pattern.