Cheating,Reactions, and Performance in Remotely Proctored Testing: An Exploratory Experimental Study

Purpose

We sought to provide empirical insight and develop theory for a new organizational phenomenon: remote proctoring for Internet-based tests. We examined whether this technology is effective at decreasing cheating and whether it has unintended effects on test-taker reactions, performance, or selection procedures.

Design/methodology/approach

Participants (582) were randomly assigned to a webcam proctored or honor code condition and completed two (one searchable, one non-searchable) cognitive ability tests online. Complete data were collected from 295 participants. We indirectly determined levels of cheating by examining the pattern of test-score differences across the two conditions. We directly measured dropout rates, test performance, and participants’ perceived tension and invasion of privacy.

Findings

The use of remote proctoring was associated with more negative test-taker reactions and decreased cheating. Remote proctoring did not directly affect test performance or interact with individual differences to predict test performance or test-taker reactions.

Implications

Technological advances in selection should be accompanied by empirical evidence. Although remote proctoring may be effective at decreasing cheating, it may also have unintended effects on test-taker reactions. By outlining an initial classification of remote proctoring technology, we contribute to the theoretical understanding of technology-enhanced assessment, while providing timely insight into the practice of Internet-based testing.

Originality/value

We provide timely insight into the development and evaluation of remotely proctored tests. The current study utilizes a unique randomized experimental design in order to indirectly determine levels of cheating across two conditions. Following the results of the current study, we outline an integrative model for future research on remotely proctored tests.     

Addressing Barriers to Change in the Treatment of Childhood Obsessive Compulsive Disorder

Although considerable evidence supports the use of cognitive behavior therapy (CBT) for the treatment of childhood obsessive compulsive disorder, large numbers of youth fail to respond and clinical remission is often elusive. Poor family functioning frequently is implicated as an obstacle for youth undergoing CBT, with features such as symptom accommodation, family conflict, and blame known to attenuate outcomes. These features are common in child and adolescent obsessive compulsive disorder (OCD) and they may pose particular challenges for exposure-based treatments. Nonetheless, interventions that focus specifically on family functioning have, to date, been limited. This paper reviews the literature on family features associated with childhood OCD and discusses their links to treatment outcome. It then describes the development of a brief family intervention tailored to address the needs of highly distressed families of youth with OCD in the service of improving individual child CBT outcomes. Preliminary pilot data are presented and clinical implications are discussed.     

Emotion-induced blindness reflects competition at early and late processing stages: An ERP study

Emotion-induced blindness (EIB) refers to impaired awareness of items appearing soon after an irrelevant, emotionally arousing stimulus. Superficially, EIB appears to be similar to the attentional blink (AB), a failure to report a target that closely follows another relevant target. Previous studies of AB using event-related potentials suggest that the AB results from interference with selection (N2 component) and consolidation (P3b component) of the second target into working memory. The present study applied a similar analysis to EIB and, similarly, found that an irrelevant emotional distractor suppressed the N2 and P3b components associated with the following target at short lags. Emotional distractors also elicited a positive deflection that appeared to be similar to the P_D component, which has been associated with attempts to suppress salient, irrelevant distractors (Kiss, Grubert, Petersen, & Eimer, 2012; Sawaki, Geng, & Luck, 2012; Sawaki & Luck, 2010). These results suggest that irrelevant emotional pictures gain access to working memory, even when observers are attempting to ignore them and, like the AB, prevent access of a closely following target.     

Personalized Genomic Results: Analysis of Informational Needs

Use of genomic information in healthcare is increasing; however data on the needs of consumers of genomic information is limited. The Coriell Personalized Medicine Collaborative (CPMC) is a longitudinal study investigating the utility of personalized medicine. Participants receive results reflecting risk of common complex conditions and drug—gene pairs deemed actionable by an external review board. To explore the needs of individuals receiving genomic information we reviewed all genetic counseling sessions with CPMC participants. A retrospective qualitative review of notes from 157 genetic counseling inquiries was conducted. Notes were coded for salient themes. Five primary themes; “understanding risk”, “basic genetics”, “complex disease genetics”, “what do I do now?” and “other” were identified. Further review revealed that participants had difficulty with basic genetic concepts, confused relative and absolute risks, and attributed too high a risk burden to individual single nucleotide polymorphisms (SNPs). Despite these hurdles, counseled participants recognized that behavior changes could potentially mitigate risk and there were few comments alluding to an overly deterministic or fatalistic interpretation of results. Participants appeared to recognize the multifactorial nature of the diseases for which results were provided; however education to understand the complexities of genomic risk information was often needed.     

Waiting When Both Certainty and Magnitude Are Increasing: Certainty Overshadows Magnitude

In everyday decision making, people often face decisions with outcomes that differ on multiple dimensions. The trade‐off in preferences between magnitude, temporal proximity, and probability of an outcome is a fundamental concern in the decision‐making literature. Yet, their joint effects on behavior in an experience‐based decision‐making task are understudied. Two experiments examined the relative influences of the magnitude and probability of an outcome when both were increasing over a 10‐second delay. A first‐person shooter video game was adapted for this purpose. Experiment 1 showed that participants waited longer to ensure a higher probability of the outcome than to ensure a greater magnitude when experienced separately and together. Experiment 2 provided a precise method of comparing their relative control on waiting by having each increase at different rates. Both experiments revealed a stronger influence of increasing probability than increasing magnitude. The results were more consistent with hyperbolic discounting of probability than with cumulative prospect theory's decision weight function. Copyright © 2014 John Wiley & Sons, Ltd.     

Aesthetics and the Dream of Objectivity: Notes from Set Theory

In this paper, we consider various ways in which aesthetic value bears on, if not serves as evidence for, the truth of independent statements in set theory.

... the aesthetic issue, which in practice will also for me be the decisive factor—John von Neumann, letter to Carnap, 1931     

A Psychometric Review of the Personality Inventory for DSM–5 (PID–5): Current Status and Future Directions

The paradigm of personality psychopathology is shifting from one that is purely categorical in nature to one grounded in dimensional individual differences. Section III (Emerging Measures and Models) of the Diagnostic and Statistical Manual of Mental Disorders (5th ed. [DSM–5]; American Psychiatric Association, 2013), for example, includes a hybrid categorical/dimensional model of personality disorder classification. To inform the hybrid model, the DSM–5 Personality and Personality Disorders Work Group developed a self-report instrument to assess pathological personality traits—the Personality Inventory for the DSM–5 (PID–5). Since its recent introduction, 30 papers (39 samples) have been published examining various aspects of its psychometric properties. In this article, we review the psychometric characteristics of the PID–5 using the Standards for Educational and Psychological Testing as our framework. The PID–5 demonstrates adequate psychometric properties, including a replicable factor structure, convergence with existing personality instruments, and expected associations with broadly conceptualized clinical constructs. More research is needed with specific consideration to clinical utility, additional forms of reliability and validity, relations with psychopathological personality traits using clinical samples, alternative methods of criterion validation, effective employment of cut scores, and the inclusion of validity scales to propel this movement forward.     

Effects of signaled and unsignaled alternative reinforcement on persistence and relapse in children and pigeons

Three experiments explored the impact of different reinforcer rates for alternative behavior (DRA) on the suppression and post‐DRA relapse of target behavior, and the persistence of alternative behavior. All experiments arranged baseline, intervention with extinction of target behavior concurrently with DRA, and post‐treatment tests of resurgence or reinstatement, in two‐ or three‐component multiple schedules. Experiment 1, with pigeons, arranged high or low baseline reinforcer rates; both rich and lean DRA schedules reduced target behavior to low levels. When DRA was discontinued, the magnitude of relapse depended on both baseline reinforcer rate and the rate of DRA. Experiment 2, with children exhibiting problem behaviors, arranged an intermediate baseline reinforcer rate and rich or lean signaled DRA. During treatment, both rich and lean DRA rapidly reduced problem behavior to low levels, but post‐treatment relapse was generally greater in the DRA‐rich than the DRA‐lean component. Experiment 3, with pigeons, repeated the low‐baseline condition of Experiment 1 with signaled DRA as in Experiment 2. Target behavior decreased to intermediate levels in both DRA‐rich and DRA‐lean components. Relapse, when it occurred, was directly related to DRA reinforcer rate as in Experiment 2. The post‐treatment persistence of alternative behavior was greater in the DRA‐rich component in Experiment 1, whereas it was the same or greater in the signaled‐DRA‐lean component in Experiments 2 and 3. Thus, infrequent signaled DRA may be optimal for effective clinical treatment.     

Are Perfectionism Dimensions Vulnerability Factors for Depressive Symptoms After Controlling for Neuroticism? A Meta‐analysis of 10 Longitudinal Studies

Extensive evidence suggests neuroticism is a higher‐order personality trait that overlaps substantially with perfectionism dimensions and depressive symptoms. Such evidence raises an important question: Which perfectionism dimensions are vulnerability factors for depressive symptoms after controlling for neuroticism? To address this, a meta‐analysis of research testing whether socially prescribed perfectionism, concern over mistakes, doubts about actions, personal standards, perfectionistic attitudes, self‐criticism and self‐oriented perfectionism predict change in depressive symptoms, after controlling for baseline depression and neuroticism, was conducted. A literature search yielded 10 relevant studies (N = 1,758). Meta‐analysis using random‐effects models revealed that all seven perfectionism dimensions had small positive relationships with follow‐up depressive symptoms beyond baseline depression and neuroticism. Perfectionism dimensions appear neither redundant with nor captured by neuroticism. Results lend credence and coherence to theoretical accounts and empirical studies suggesting perfectionism dimensions are part of the premorbid personality of people vulnerable to depressive symptoms. Copyright © 2016 European Association of Personality Psychology