THE EFFECTIVENESS OF CURRENTLY ENROLLED STUDENT PROCTORS IN AN UNDERGRADUATE SPECIAL EDUCATION COURSE1

Potential users of Keller's personalized college classroom method might find it impractical to provide student proctors from the ranks of those who had previously mastered the course material. An alternative is to let currently enrolled students proctor and tutor each other. Such a method imposes the burden of showing that its effectiveness is not a function of (1) student collaboration or (2) the effects of “practice” received by students who listen to the performance of others before their own performance. The present study compared students proctored by previously trained students (Control) with students who received proctoring from classmates (Experimental). Experimental group performance was superior to Control group performance. The effects of “practice” and collaboration were ruled out as determinants of the difference between groups. It is hypothesized that the superior performance of the Experimental students was related to their activities as proctors.     

Cheating,Reactions, and Performance in Remotely Proctored Testing: An Exploratory Experimental Study

Purpose

We sought to provide empirical insight and develop theory for a new organizational phenomenon: remote proctoring for Internet-based tests. We examined whether this technology is effective at decreasing cheating and whether it has unintended effects on test-taker reactions, performance, or selection procedures.

Design/methodology/approach

Participants (582) were randomly assigned to a webcam proctored or honor code condition and completed two (one searchable, one non-searchable) cognitive ability tests online. Complete data were collected from 295 participants. We indirectly determined levels of cheating by examining the pattern of test-score differences across the two conditions. We directly measured dropout rates, test performance, and participants’ perceived tension and invasion of privacy.

Findings

The use of remote proctoring was associated with more negative test-taker reactions and decreased cheating. Remote proctoring did not directly affect test performance or interact with individual differences to predict test performance or test-taker reactions.

Implications

Technological advances in selection should be accompanied by empirical evidence. Although remote proctoring may be effective at decreasing cheating, it may also have unintended effects on test-taker reactions. By outlining an initial classification of remote proctoring technology, we contribute to the theoretical understanding of technology-enhanced assessment, while providing timely insight into the practice of Internet-based testing.

Originality/value

We provide timely insight into the development and evaluation of remotely proctored tests. The current study utilizes a unique randomized experimental design in order to indirectly determine levels of cheating across two conditions. Following the results of the current study, we outline an integrative model for future research on remotely proctored tests.     

Social Interaction and Internet-Based Surveys: Examining the Effects of Virtual and In-Person Proctors on Careless Response

A lack of human interaction and environmental control in Internet-based data collection have been suggested as possible antecedents of careless responding, which occurs when participants respond to survey items without regard for item content. To address these possible antecedents, this study investigated whether survey proctoring deterred careless response in an undergraduate sample by reducing environmental distractions. The study randomly assigned respondents to one of three proctoring conditions: remote online un-proctored, remote online virtually proctored, and in-person classroom proctored. Data quality was examined via nine careless response indicators. Analyses indicated that proctor presence had effects on a small number of careless response indicators. Virtually proctored participants performed better than un-proctored participants on one of nine careless response indicators, and in-person proctored participants performed better on two careless response indicators compared to un-proctored participants. Environmental distraction fully mediated the relationship between in-person proctor presence and self-reported diligence. Implications for survey administration are discussed.     

The mediators of minority ethnic underperformance in final medical school examinations

Background. UK‐trained medical students and doctors from minority ethnic groups underperform academically. It is unclear why this problem exists, which makes it difficult to know how to address it. Aim. To investigate whether demographic and psychological factors mediate the relationship between ethnicity and final examination scores. Sample. Two consecutive cohorts of Year 5 (final year) UCL Medical School students (n= 703; 51% minority ethnic). A total of 587 (83%) had previously completed a questionnaire in Year 3. Methods. Participants were administered a questionnaire in 2005 and 2006 that included a short version of the NEO‐PI‐R, the Study Process Questionnaire, and the General Health Questionnaire (GHQ) as well as socio‐demographic measures. Participants were then followed up to final year (2007–2010). White and minority ethnic students’ questionnaire responses and final examination grades were compared using univariate tests. The effect of ethnicity on final year grades after taking into account the questionnaire variables was calculated using hierarchical multiple linear regression. Results. Univariate ethnic differences were found on age, personality, learning styles, living at home, first language, parental factors, and prior education. Minority ethnic students had lower final exam scores, were more likely to fail, and less likely to achieve a merit or distinction in finals. Multivariate analyses showed ethnicity predicted final exam scores even after taking into account questionnaire factors. Conclusions. Ethnic differences in the final year performance of two cohorts of UCL medical students were not due to differences in psychological or demographic factors, which suggests alternative explanations are responsible for the ethnic attainment gap in medicine.     

Do Online Exams Facilitate Cheating? An Experiment Designed to Separate Possible Cheating from the Effect of the Online Test Taking Environment

Despite recent growth in distance education, there has been relatively little research on whether online examinations facilitate student cheating. The present paper utilizes an experimental design to assess the difference in student performance between students taking a traditional, proctored exam and those taking an online, unproctored exam. This difference in performance is examined in a manner which considers both the effect of the different physical test environments and the possible effect of a difference in the opportunity for students to cheat. This study, utilizing regression models that also account for relevant control variables, examines 44 undergraduate statistics students, finds evidence that the difference in the testing environment creates a disadvantage to students taking the online exam which somewhat offsets the advantage that the unproctored students gain from greater opportunities to cheat.     

Equivalence of unproctored internet testing and proctored paper‐and‐pencil testing of the Big Five

The use of unproctored Internet‐based testing (IBT) for personality assessment is increasingly popular, especially in personnel selection. Previous studies on its equivalence to traditional proctored paper‐and‐pencil testing (PPT) have used between‐subjects designs, which makes it difficult to separate intergroup effects from format effects, and have shown mixed results. The aim of the present study was to assess the quantitative, qualitative, and auxiliary equivalence of unproctored IBT and proctored PPT of personality, using a within‐sample design. Undergraduate students (n = 407) completed both an Internet and a paper‐and‐pencil version of a measure of the Big Five, with a 1 to 3 week interval. The proctored paper‐and‐pencil assessment was completed in class and the unproctored Internet assessment was completed at the time and place chosen by the participants. Results showed effect sizes for mean differences to vary from null to small. Skewness and kurtosis indices, reliability coefficients, intercorrelations magnitudes, as well as factor solutions, were highly similar across formats. Respondents did not prefer IBT over PPT on a series of statement about their perceptions and reactions to IBT. In conclusion, results suggest that IBT and PPT of the Big Five personality traits are equivalent, and that the data obtained are comparable across formats.     

Memory modulation in the classroom: selective enhancement of college examination performance by arousal induced after lecture

Laboratory studies examining moderate physiological or emotional arousal induced after learning indicate that it enhances memory consolidation. Yet, no studies have yet examined this effect in an applied context. As such, arousal was induced after a college lecture and its selective effects were examined on later exam performance. Participants were divided into two groups who either watched a neutral video clip (n=66) or an arousing video clip (n=70) after lecture in a psychology course. The final examination occurred two weeks after the experimental manipulation. Only performance on the group of final exam items that covered material from the manipulated lecture were significantly different between groups. Other metrics, such as the midterm examination and the total final examination score, did not differ between groups. The results indicate that post-lecture arousal selectively increased the later retrieval of lecture material, despite the availability of the material for study before and after the manipulation. The results reinforce the role of post-learning arousal on memory consolidation processes, expanding the literature to include a real-world learning context.     

Examination performance in lecture-discussion and personalized instruction courses

Students enrolled in a Psychology of Learning course were assigned to either a lecture section, one of two similar personalized instruction sections, or a fourth section that rotated across all three teaching procedures. All students took identical midterms and a final examination. After correcting test performance for differences in the cumulative grade point average of students in the four sections, examination performance of students in the personalized sections was found to be superior to that of students in the lecture section. An analysis of class section examination performance by item type revealed that students in the lecture section scored lower on all item types, but the greatest differences occurred on items that required written responses (essay and fill-in items) rather than recognition responses (multiple choice items). A gross analysis of student performance in the class rotated across the instructional procedures suggests that personalized instruction had its greatest impact on students with "average" to "poor" academic records.     

COMMUNICATION APPREHENSION AND ACHIEVEMENT IN A PERFORMANCE-ORIENTED BASIC COMMUNICATION COURSE

The study of communication apprehension (CA) impact in the instructional environment is extended through examination of all three CA levels relative to various performance situations in a basic communication course. Results indicate significant differences in achievement indices between all three CA groups on final course grade and the first two of four performance assignments. Moderate and high CA groups were not significantly different from each other on the third and fourth performance tasks. Low and high CA groups were significantly different in all performance areas. No differences were indicated between CA groups on the final examination. Results are discussed relative to the naturalistic nature of the study.     

Combating Examination Stress among University Students: Action Research in an Institutional Context

167 students completed a questionnaire survey immediately after completing their final examinations. A significant minority of students reported finding their experiences moderately to highly stressful. Stress levels were related to general psychological states in the whole of the preceding year, particularly anxiety, depression and sleeplessness. Major sources of stress were seen to be work overload, revision difficulties, and uncertainty about examination content and the future. Stress was shown to have a detrimental effect on examination performance. Students employed a variety of coping strategies. Strategies that were widely used and felt to be effective included peer support, taking time off work and away from the institution, and exercise. There is some evidence that these strategies were actually effective in reducing stress, and that the early help of a counsellor increases the repertoire of coping strategies. The principal institutional response urged by respondents was the partial or total dismantling of the final examination system and its replacement with a variety of assessment forms distributed throughout the course. The implications of these findings for the roles of counsellors and other staff are discussed.     

EFFECTS OF CONTINGENCY CONTRACTING ON STUDY RATE AND TEST PERFORMANCE1

A contingency contracting program designed to increase study rate and subsequent test performance was implemented with a group of undergraduate psychology students. The function of the contingency contracting program in producing increased study rate was evaluated by individual experiments with each student in an experimental contracting group. The overall effect of the program on test performance was assessed by comparing the final scores for the course earned by the experimental group with those earned by two matched control groups. A reversal procedure established that contingency contracting did significantly increase the study rate of students of a wide range of ability. However, it was selectively effective in improving the test performance of below-average students only. Study rate gains in contracted courses did not generalize to noncontracted courses. Self-recording of study time in the absence of scheduled differential consequences did not improve test performance. Study rate under no-consequence conditions varied with test schedule. For both consequence and no-consequence groups, the correlation between study time and final score for the course was only moderate.     

Item-by-Item Versus End-of-Test Feedback in a Computer-Based PSI Course

This study examined the effects of three different types of computer feedback on the following variables in a Personalized System of Instruction (PSI) course: unit quiz and final exam performance, the amount of time tutors and other teaching staff spent answering student questions, and students' preference for each type of feedback. The feedback conditions were the following: (a) end-of-test, (b) item-by-item with the option to skip questions during the test, and (c) item-by-item without the option to skip questions during the test. Students who received item-by-item feedback with the skip option performed the same as students who received end-of-test feedback on the unit quizzes and final exam. However, the teaching staff spent significantly less time answering questions when students received item-by-item feedback with the skip option than when they received end-of-test feedback. Finally, 65% of the students preferred item-by-item feedback with the skip option. The authors concluded that this type of item-by-item feedback decreases the workload on teaching staff in a PSI course without sacrificing performance, and students like it more than the other types of feedback.     

Personality,cognition, and university students' examination performance

A prospective study explored the relationship between personality traits (as defined by the five factor model), type of motivation (as defined by self‐determination theory), and goal‐specific cognitions (including those specified by the theory of planned behaviour) as antecedents of degree performance amongst undergraduate students. A sample of 125 students completed a questionnaire two to three months before their final examinations. Structural equation modelling was used to explore relationships. Intention and perceived behavioural control explained 32% of the variance in final degree marks, with intention being the strongest predictor. Controlling for theory of planned behaviour variables, anticipated regret, good‐student identity, controlled extrinsic motivation, Conscientiousness, and Openness had direct significant effects on intention. In total, 65% of the variance in intention was explained. The resultant model illustrates how personality traits may affect examination performance by means of mediators such as intention, anticipated regret, student identity, and autonomous intrinsic motivation. Copyright © 2003 John Wiley & Sons, Ltd.     

THE EFFECTS OF PEER TUTORING WITH GROUP CONTINGENCIES ON THE ACADEMIC PERFORMANCE OF COLLEGE STUDENTS1

Improving the academic performance of college students who do not demonstrate mastery of course material is a major concern in traditional and nontraditional systems of instruction, where students may drop out, take incompletes, or continue to perform at low levels. The present study examined within-course peer tutoring as a potential solution. Twenty-one undergraduate students enrolled in a three-credit introductory course in Educational Psychology served as subjects. The class met one and a half hours each weekday for five weeks. Five students withdrew from the course and one student was placed on independent study before assignments to experimental conditions were made. The primary source materials were portions of Skinner's Technology of Teaching, plus two additional articles. The material was divided into nine equal units, each unit accompanied by study objectives. Nine one-hour essay exams were administered, one every other class day. Two review days were scheduled before a cumulative final was administered. Students could score a total of 20 points on each exam and the final. If a student scored 90% or better on an exam a score of 10 was earned. If a student scored 80% to 90%, a score of eight was awarded, and so on. A total score of 90 of 100 possible points at semester's end earned a student an “A”, 80 a “B”, and so on. The study consisted of three phases: Baseline I, Intervention, and Baseline II. Baseline I: after an initial introductory class, three lectures were presented—one for each unit. Each lecture day was followed by an exam day. Intervention: following the third exam, students were rank ordered and divided into high, medium, and low levels of performance on the basis of their raw scores on the previous three exams, and assigned to a paired or independent group. This assignment procedure resulted in three high-low pairs, three middle-middle pairs, two high-middle pairs, three low-independent students, and two middle-independent students. If, and only if, both students in a pair met a 90% mastery criterion on an exam did each receive five bonus points for the exam(s) reaching the criterion. The bonus points were used to offset points lost on the cumulative final. If both students in a pair met the 90% mastery criterion for units 4, 5, and 6, the pair received an automatic score of 10 on the cumulative final and had the two review days off. Other students who studied independently received identical payoffs if they met the same mastery criterion. The previous lecture time was used for inclass study. Baseline II: Baseline I procedures were reinstituted for the final three units. The test scores are the independent and paired students are shown in Figure 1. Compared to baseline, performance during peer tutoring improved for every student paired with a high partner, and not for those students who studied independently. Between-group comparisons suggest that the effective variables were related to the tutoring or its combination with the group contingency. However, the opportunity for intergroup discussion about treatment procedures and unequal assignment of subjects to the tutored and independent groups make conclusions about the between-group portion of the experiment tentative. Half to two-thirds of the students in each performance category viewed the peer-tutoring procedure favorably, and two-thirds or more reported that the procedure was effective in improving academic performance. Proportionately fewer students assigned to independent study found that procedure effective or viewed it favorably. It appears that pairing students with others who do better on tests and rewarding them for their combined performance results in considerable improvement in the performance of lower-level students.     

Proctored Versus Unproctored Internet Tests: Are unproctored noncognitive tests as predictive of job performance?

As the usage of unproctored Internet testing (UIT) increases in selection settings, concerns about the validity of such practices must be addressed. While recent examinations of the issue have focused primarily on the practicality, ethics, and potential legal ramifications of UIT, this paper provides an examination of the criterion‐related validity of unproctored assessments. Using a database of validity evidence, we examine the predictive validity of several noncognitive assessments (i.e., personality and biodata) with respect to a number of subjective and objective job performance metrics. Results generally indicate that assessments administered in proctored and unproctored settings have similar validities. Limitations of this research and implications for practice are discussed.     

Importance of attributions as a predictor of how people cope with failure

This study examined the extent to which causal attributions were predictive of depressed mood in college students who experienced a negative event. In a replication and extension of a study by Metalsky, Abramson, Seligman, Semmel, and Peterson (1982), we evaluated students' attributional style and their attributions for an examination performance in the college classroom. Additionally, an indirect probe was used to assess unsolicited attributions. Subjects were asked about their plans to prepare for the next examination in order to test for the motivational deficits predicted by the reformulated learned helplessness (RLH) model. Unlike Metalsky et al., attributional style did not predict depressed mood following a disappointing examination performance. Attributions for the particular examination performance were predictive of depressed mood for students who were disappointed in their examination performance. Few subjects, 31%, gave attributions in response to the indirect probe, and there was no support for the prediction that unexpected negative events would lead to subjects' making more attributions. Internal, stable, and global attributions for poor examination performance resulted in students making more plans to study for the next examination, a finding contrary to what is predicted by the RLH model.     

Following instructions on a final examination: an informal look

Of a total of 914 undergraduate college students over 5 years 600 (about 66%) were in full compliance with a simple instruction for how to enter their names on their final examination papers; approximately 33% were not. The compliance rate over the years declined from 73% to 59%.     

Improving the Performance of Failing Students by Overcoming Their Self-Serving Attributional Biases

The causal impact of attributions on academic performance was examined by changing low-scoring students' attributions regarding their poor performances. Initially, when students who were failing a college course identified the cause of the performance, they emphasized external, uncontrollable causes. Because these self-serving attributions could have perpetuated poor performance on subsequent examinations, students in the experimental condition were exposed to information that suggested that grades in college are caused by internal, controllable factors such as effort and motivation. As predicted, on subsequent tests and on the final examination, these students earned higher grades than control students who received no attributional information. These findings lend support to an attributional model of academic achievement and also suggest that educational interventions that shift attributions away from a self-serving pattern to a performance-facilitating pattern may improve academic outcomes.     

Touching as Teaching: The Effect of Touch on Students' Perceptions and Performance

The effect of instructor touch on students' ratings of the instructor and students' performance in the classroom was investigated. A total of 171 male and female college students participated in individual conferences with their instructors following the first examination in the course. Half were touched by their instructors during the conference; the other half were not. Students then evaluated the instructor, the instructor's teaching effectiveness, and the utility of the conference. Analysis of results revealed that students who were touched during the conference gave their instructors significantly higher ratings than those who were not touched. In addition, students in the touched condition showed superior performance on the next course examination, scoring .58 standard deviations higher on the examination compared to the untouched students. The authors conclude that touching, when conducted in a conference situation to help students improve class performance, can be a highly effective teaching tool.     

Agreement in interpersonal perception as a predictor of group performance

The present work deals with quantifying group characteristics. Specifically, dyadic measures of interpersonal perceptions were used to forecast group performance. Forty-six groups of students, 24 of four and 22 of five people, were studied in a real educational assignment context and marks were gathered as an indicator of group performance. Our results show that dyadic measures of interpersonal perceptions account for final marks. By means of linear regression analysis, 85% and 85.6% of group performance, respectively, was explained for group sizes equal to four and five. Results found in the scientific literature based on the individualistic approach are no larger than 18%. The results of the present study support the utility of dyadic approaches for predicting group performance in social contexts.