Can you test me now? Equivalence of GMA tests on mobile and non‐mobile devices

As technology continues to evolve, organizations seek to use personal electronics like smartphones for selection and assessment. While this promises to increase access to a more diverse applicant pool, research is needed to examine whether commonly used assessments function similarly on these devices as on a conventional computer. Contrary to past research, we did not find meaningful differences in general mental ability (GMA) test scores between device groups. We also observed few differences in item functioning between devices. Screen size had a positive, but marginal effect on test scores. These results are optimistic for the use of mobile devices in GMA testing, but additional research is needed to examine the functioning of alternative GMA tests administered on mobile devices.     

The Use of Mobile Devices in High‐stakes Remotely Delivered Assessments and Testing

With Internet access no longer restricted to desktop and laptop computers, job applicants now have the opportunity to complete remotely delivered assessments on mobile, handheld small screen devices such as smartphones, and personal digital assistants. In this study, a large dataset is used to investigate demographic and score differences between job applicants who completed a remotely delivered high‐stakes assessment on a mobile device and those who completed it on a nonmobile device. Based on a sample of 3,575,207 job applicants who completed an unproctored Internet‐based assessment between January 2011 and April 2012, the percentage of applicants completing the assessment on a mobile device was small, 1.93%, but nevertheless represented more than 69,000 people. Overall, there were small test‐taker demographic differences in the use of mobile devices versus nonmobile devices in that mobile devices were slightly more likely to be used by women, African‐Americans and Hispanics, and younger applicants. Scores on a personality measure were similar for mobile and nonmobile devices but scores on a general mental ability test were substantially lower for mobile devices. Tests of measurement invariance also indicated equivalence across the mobile and nonmobile samples. Test taker and organizational implications for completing remotely delivered high‐stakes noncognitive and cognitive assessments on mobile versus nonmobile devices are discussed.     

The testing environment as an explanation for unproctored internet‐based testing device‐type effects

Device‐type effects on cognitive tests appear to covary with whether unproctored internet‐based test (UIT) scores were obtained operationally or nonoperationally. The present study examined whether the testing environment and distractions therein—one of three contextual factors identified as plausible explanations—accounts for this covariation. Four‐hundred and twenty‐five college students were randomly assigned to one of three conditions which differed by the testing environment and device used to complete a cognitive test. Test scores did not differ between conditions despite greater distractions reported by those who tested in the distracting environment. Results were replicated within‐study, and suggest that the self‐selection and testing‐stakes hypotheses warrant subsequent empirical examination as explanations for the differences in device‐type effects observed in operational and nonoperational settings.     

Managing Self-Uncertainty Through Group Identification

Hogg's uncertainty-identity theory (UIT) is briefly described to identify similarities and differences to Van den Bos's uncertainty management model (UMM). Against a background of significant overlap in scope, mission and concepts, four differences are identified: First, UMM is primarily a theory of motivation for ideological conviction; UIT is a theory of motivation for group identification. Second, UMM talks about personal uncertainty; UIT talks about self-uncertainty—the implications of this difference in terminology are discussed. Third, both theories focus on uncertainty about self; but UIT also focuses on an array of moderating variables that affect the experience of uncertainty and the way in which self-uncertainty is reduced. Finally, and most significantly, UMM does not detail the process of uncertainty reduction; UIT does—it specifies social cognitive processes that reduce self-uncertainty and contexts that direct these processes toward “normal” group phenomena or toward more extreme group phenomena.     

The Magnitude and Extent of Cheating and Response Distortion Effects on Unproctored Internet‐Based Tests of Cognitive Ability and Personality

The use of unproctored internet‐based testing (UIT) for employee selection is quite widespread. Although this mode of testing has advantages over onsite testing, researchers and practitioners continue to be concerned about potential malfeasance (e.g., cheating and response distortion) under high‐stakes conditions. Therefore, the primary objective of the present study was to investigate the magnitude and extent of high‐ and low‐stakes retest effects on the scores of a UIT speeded cognitive ability test and two UIT personality measures. These data permitted inferences about the magnitude and extent of malfeasant responding. The study objectives were accomplished by implementing two within‐subjects design studies ( Study 1 N=296; Study 2 N=318) in which test takers first completed the tests as job applicants (high‐stakes) or incumbents (low‐stakes) then as research participants (low‐stakes). For the speeded cognitive ability measure, the pattern of test score differences was more consonant with a psychometric practice effect than a malfeasance explanation. This result is likely due to the speeded nature of the test. And for the UIT personality measures, the pattern of higher high‐stakes scores compared with lower low‐stakes scores is similar to those reported for proctored tests in the extant literature. Thus, our results indicate that the use of a UIT administration does not uniquely threaten personality measures in terms of elevated scores under high‐stakes testing that are higher than those observed for proctored tests in the extant literature.     

Pre‐employment risk screening: Comparability of integrity assessment technology platforms

The prevalence of mobile devices (e.g., smartphones and tablets) has led to job applicants increasingly choosing to use these mobile devices to complete online pre‐employment assessments in the application process. This study identifies increasing mobile device usage trends for applicants completing an unproctored integrity assessment across 13 quarterly time periods. Further, the comparability of the integrity assessment results when completed on mobile devices compared to when it is completed on computers was examined. Results revealed that the integrity assessment had comparable mean scores, number of admissions, reliability estimates, and validity coefficients across mobile devices and computers. Pass rates for the assessment across the delivery modes showed small differences though some were statistically significant. Applicants using smartphones to complete the assessment took significantly more time than applicants using computers or tablets.     

Establishing the Measurement Equivalence of Online Selection Assessments Delivered on Mobile versus Nonmobile Devices

Recent usage data suggest job applicants are completing online selection assessments using mobile devices (e.g., smartphones) in greater numbers. To determine the appropriateness of this new technology, this study examined the measurement equivalence of selection assessments delivered on mobile and nonmobile devices (e.g., personal computers). Measurement invariance tests conducted with multigroup confirmatory factor analysis suggest mobile versions of a cognitive ability‐type assessment, two biodata assessments, a multimedia work simulation, and a text‐based situational judgment test appear to be equivalent to nonmobile versions. However, mobile device user latent means were half a standard deviation lower than their nonmobile counterparts for the situational judgment test. Implications for mobile device usage within selection and assessment are discussed.     

Associations between technoference,quality of parent-infant interactions,and infants’ vocabulary development

“Technoference” describes the distraction from interpersonal activities that can occur due to use of mobile screen devices. Focusing on parent-infant interactions, our study investigated the associations between potential sources of “technoference” and parental responsiveness, scaffolding and directiveness toward their infant, and coordinated joint attention (CJA). Previous research demonstrates that each of these dimensions is related to early language development. Potential sources of “technoference” employed in our study included the amount of time the parent spends on their mobile device per hour when with their infant; the number of audible notifications the parent receives per hour, the number of times per hour they check their device; and parents’ score on the Distraction In Social Relations and Use of Parent Technology (DISRUPT) scale. We investigated associations between our measures of parental “technoference” and infants’ language development, and whether parental responsiveness, scaffolding, directiveness or parent-infant CJA mediate associations between “technoference” and language. Frequency of audible notifications negatively predicted infant vocabulary, and this relationship was fully mediated by parental directiveness.     

The Influence of Rater Effects in Training Sets on the Psychometric Quality of Automated Scoring for Writing Assessments

Automated essay scoring engines (AESEs) are becoming increasingly popular as an efficient method for performance assessments in writing, including many language assessments that are used worldwide. Before they can be used operationally, AESEs must be “trained” using machine-learning techniques that incorporate human ratings. However, the quality of the human ratings used to train the AESEs is rarely examined. As a result, the impact of various rater effects (e.g., severity and centrality) on the quality of AESE-assigned scores is not known. In this study, we use data from a large-scale rater-mediated writing assessment to examine the impact of rater effects on the quality of AESE-assigned scores. Overall, the results suggest that if rater effects are present in the ratings used to train an AESE, the AESE scores may replicate these effects. Implications are discussed in terms of research and practice related to automated scoring.     

Optimizing Prediction of Attrition With the U.S. Army's Assessment of Individual Motivation (AIM)

The regression framework is often the method of choice used by psychologists for predicting organizationally relevant outcomes from test scores. However, alternatives to regression exist, and these techniques may provide better prediction of outcomes and a more effective means of classifying examinees for selection and placement. This research describes two of these alternatives—decision tree methodology and optimal appropriateness measurement (OAM)—and how they were used to optimize the prediction of attrition among a sample of first-term enlisted soldiers (N?=?22,537) using a temperament inventory called the Assessment of Individual Motivation (AIM). Results demonstrated that the OAM approach provided better differentiation between “stayers” and “leavers” after 12 months than either the traditional logistic regression or the decision tree methods.     

Mobile phone use while driving-literary review

This paper analysis published results of the research into the connection between mobile phone use while driving and traffic safety. Ever since the introduction of the first mobile phones, the authors have been examining the risks associated with its use in traffic, namely: prevalence or frequency of mobile phone use while driving, characteristics of drivers who make more frequent use of mobile phones, the connection between mobile phone use while driving and a car crash risk, a link between mobile phone use while driving and perception of risk, effects on driving performance of using various modes of mobile phones while driving (“hands-free” or “hand-held”), psychological factors influencing driver's decision to use a mobile phone while driving, etc.It is important to point out that the results of some studies indicate that using a hands-free mobile phone while driving does not provide greater safety as compared to the use of hand-held mobile phones while driving. Generally speaking, younger male drivers tend to use mobile phones more often while driving than women and older males.This paper analyses the results of studies which were published in 60 papers from 1994 to 2013.The analysis of the papers selected for research confirms detrimental effects of mobile phone use while driving. Also, four phases in the process of understanding the issue of mobile phone use impacts while driving have been established or identified. The first phase gives the analysis of the prevalence or frequency of mobile phone use while driving. The second phase identifies the characteristics of drivers who tend to use mobile phones more frequently while driving. The third phase concerns research into impacts of using different modes of mobile phones while driving on driving performance (“hands-free” or “hand-held”). Finally, the fourth phase deals with research into risks of mobile phone use while driving (“hands-free” or “hand-held”). The importance of this paper is reflected in that it can help traffic safety policy makers, on the basis of better understanding of the issue of mobile phone use impact while driving, to develop effective strategies aimed at reducing the extent of mobile phone use while driving.     

On the meaning of indexicals

The approach adopted in the paper is based on the theory known as Montague grammar. Accepting, in general, that theory — especially in its modified version, which is due to Thomason and Kaplan — the author points out certain inadequacy in its treatment of the meaning of some indexical expressions and suggests some modification of its theoretical framework in order to avoid that shortcoming. It is claimed that to do justice to the meaning of so-called indefinite indexicals (such as “we”, “you”, “now”) two kinds of dependence of their semantic values upon the context of use must be taken into account — a semantic (or lexical) and a pragmatic (or extralexical) one.     

The influence of selection test type on applicant reactions to employment testing

The impact of employment selection devices on the reactions of simulated applicants was investigated. In Study 1, 141 subjects were “failed” on either a biographical inventory, a cognitive ability test, a trainability test or a work sample test. Subjects rejected on the basis of biographical inventory scores perceived the test as less difficult, coped more adaptively, and had better moods than subjects tested with the other instruments. The biographical inventory was also seen as more fakable, but less amenable to improvement through feedback or training than the other devices. We hypothesized that differences among tests in cognitive demand is a primary cause of differences in applicant reaction. In Study 2 (N=151) the comparison of the effects of two biographical inventories to two cognitive ability tests supported our hypothesis, and replicated the findings of Study 1 across failure and success conditions.     

The Effects of Linguistic Devices on Consumer Information Processing and Persuasion: A Language Complexity × Processing Mode Framework

People—be they politicians, marketers, job candidates, product reviewers, or romantic interests—often use linguistic devices to persuade others, and there is a sizeable literature that has documented the effects of numerous linguistic devices. However, understanding the implications of these effects is difficult without an organizing framework. To this end, we introduce a Language Complexity × Processing Mode Framework for classifying linguistic devices based on two continuous dimensions: language complexity, ranging from simple to complex, and processing mode, ranging from automatic to controlled. We then use the framework as a basis for reviewing and synthesizing extant research on the effects of the linguistic devices on persuasion, determining the conditions under which the effectiveness of the linguistic devices can be maximized, and reconciling inconsistencies in prior research.     

Evaluating the technical adequacy of DBR-SIS in tri-annual behavioral screening: A multisite investigation

The implementation of multi-tiered systems in schools necessitates the use of screening assessments which produce valid and reliable data to identify students in need of tiered supports. Data derived from these screening assessments may be evaluated according to their classification accuracy, or the degree to which cut scores correctly identify individuals as “at-risk” or “not-at-risk.” The current study examined the performance of mean scores derived from over 1700 students in Grades 1, 2, 4, 5, 7, and 8 using Direct Behavior Rating—Single Item Scales. Students were rated across three time points (Fall, Winter, Spring) by their teachers in three areas: (a) academically engaged behavior, (b) disruptive behavior, and (c) respectful behavior. Classification accuracy indices and comparisons among behaviors were derived using Receiver Operating Characteristic (ROC) curve analyses, partial area under the curve (pAUC) tests, and bootstrapping methods to evaluate the degree to which mean behavior ratings accurately identified students who demonstrated elevated behavioral symptomology on the Behavioral and Emotional Screening System. Results indicated that optimal cut-scores for mean behavior ratings and a composite rating demonstrated high levels of specificity, sensitivity, and negative predictive value, with sensitivity point estimates for optimal cut-scores exceeding .70 for individual behaviors and .75 for composite scores across grade groups and time points.     

How teachers use the group I.Q. test scores

Ninety elementary school teachers, grades 2–6, in two school districts in Ohio were interviewed to obtain information on how they used the results of group IQ tests. Twenty-seven of 90 teachers (30%) reported making “little or no use” of the test scores. Most of these teachers presumably had more faith in their own tools and observations than in the IQ test scores. However, 76 of 90 teachers (84%, including the ones who made little use of the test scores) stated that they used the test scores for one or more purposes. The four most frequently mentioned uses were: (a) “for discussion in parent conferences” (51%); (b) “to know a student's potential and/or determine ability-motivation discrepancy” (42%); (c) “to plan instruction” (39.5%) and (d) “for referral purposes” (29%). The instructional strategies mentioned by the teachers varied and were categorized into 21 categories.     

Difference score correlations in relationship research: A conceptual primer

The practice of computing correlations between “difference” or “discrepancy” scores and an outcome variable is common in many areas of social science. Relationship researchers most commonly use difference scores to index the (dis)similarity of members of two-person relationships. Using an intuitive, graphical approach—and avoiding formulas and pointing fingers—we illustrate problems with using difference score correlations in relationship research, suggest ways to ensure that difference score correlations are maximally informative, and briefly review alternatives to difference score correlations in studying similarity, accuracy, and related constructs.     

Guilty knowledge versus innocent associations: Effects of trait anxiety and stimulus context on skin conductance

A simulated lie detection procedure was used to investigate skin conductance responsivity among self-reported skin responders. Subjects were grouped according to reported trait anxiety as measured by Lykken's Activity Preference Questionnaire and then engaged in either a mock crime or a neutral activity. The “crime” gave subjects “guilty knowledge,” and the neutral task provided “innocent associations” to relevant stimuli imbedded in each scenario. Subjects were then interrogated using Lykken's Guilty Knowledge technique. Anxiety classification, guilty vs. innocent treatment, and type of stimulus were manipulated in a 2 × 2 × (2) factorial design. Results for differential responsivity scores showed significant main and interaction effects. Low-anxiety subjects showed almost no effects, but highly anxious subjects responded strongly in the “guilty” rather than the “innocent” treatment. Contrary to previous findings, results showed a substantial—but qualified—relationship between self-reported anxiety and electrodermal responsivity. Also, the Guilty Knowledge technique with rank scoring yielded 97.5% correct classification.